I don’t quite agree with you that my information need about finding cafes in Prague that are located inside passages and at the end of tiny, old town meanering lanes is “bias”. That’s just my information need. Some people use a search engine to find Gucci stores in Singapore. Some people use a search engine to find used book stores in Newton, Iowa. I want to be able to use a search engine to find off-beaten-path cafes in Prague. That’s just my info need.

If the search engine is fundamentally incapable of finding information that meets my need, because it has built into it an algorithm that makes the cafes that I am looking for “unretrievable”, then the search engine itself has bias. But I do not.

And that’s also the answer to your question about “Why is it so important that all documents are (easily) retrievable?” Because if those documents are relevant to my information needs, I need to be able to find them. If the search engine prevents me from finding them, because of its bias, then the search engine is flawed.

Look at the experiments that were done in this paper: One was a collection of government documents from the web, the other was a collection of newswire articles. All easy jokes about the government information aside, I would say that all of that information is at least important to someone. So it needs to be retrievable. And same with the newswire articles. If it was newsworthy enough to be written about in the first place, then some future historian, trying to understand the early 21st century, is going to be able to need to retrieve it. If the systems we built forever condemn certain pages to non-retrievability, well.. we should at least be conscious of this fact.

In short, this is not just about duplicate content and spam. In fact, I’ll bet the collections that this paper above tested on has no spam, and very little duplicate content. And despite the “cleanness” of those collections, there were still large numbers of documents that were completely non-retrievable.

But not all documents are made equal. Why is it so important that all documents are (easily) retrievable? Maybe some really don’t have unique-enough content that doesn’t already exist in other, “better” documents. Or are you saying that there are no better or worse documents because there should be no bias?

And by non-trivial, I mean that you are not using the item itself (or a long exact quotation from the item itself) as the query — of course the item would come up top-ranked in that case, but that’s beside the point because why would you ever search for that item, if you already have it?

Because even if you rank all items, if a certain item never comes up in the top c to any reasonable query that a user might enter, then for all intents and purposes, that item is non-retrievable. For example, if that item never achieved a rank higher than 453 for any query, then no one is ever going to find it, even if it has been ranked.

This is one of the areas that we address ie. find items similar to the query items in “ranked” order.

Dinesh