Bit Rot and Search Engines

It's sad when you go to look for something you read once and discover that it's not there anymore. And not just not *there*, but unfindable. We've gotten spoiled by the idea that you can just go to Google and find anything, but it's not true anymore.

It was never really true. But there were grand ideas when the web was being created that URLs would persist and that soon all information would be digital and immediately at your fingertips. I remember an ad with the slogan "Everything new is digital and everything old is being digitized." Or this ad that said "all rooms have every movie ever made in any language anytime, day or night." Or Google's project to scan every book ever printed and make available a vast library of human knowledge.

Instead, much of the early internet has quietly withered away. Some of it can be tracked down at the Wayback Machine. If you have the time and patience. But the effort of carrying forward old internet content is non-zero and there's always pressure to focus on new content.

I remember a presentation at ContactCon where a guy was describing a pattern language for internet content. I found his talk somewhat incoherent at the time, but more easily recognize now what he was talking about. Rather than having a bunch of systems for styling and presenting information, we would do better focusing on tagging and relating content. It shouldn't matter as much whether you're posting an article or a comment, or posting it here or there. Let writers write and curators curate.

The most discouraging aspect is how Google now doesn't even bother to return results about vast swaths of the internet. Take this page for instance. OK. You can certainly argue that it's not a very important page. But this page used to appear in Google search results. And it doesn't anymore. Google does index some pages at "revo.bierfaristo.com' which includes links to that page. But Google has evidently decided it's not worth maintaining links to less interesting or important content. And that bodes ill.

It's not a "search engine" anymore if it's making decisions about what to include and what not to include. But that's where we're at. The Internet Genie has been stuffed back in the bottle and we're left with just a few giant corporations being the gatekeepers to all human knowledge.