I would like to point to a post worth reading, over at Blogoscoped, about personal, blind side-by-side comparisons of the various contending search engines. I have seen studies like this for years, both on the web and in published, academic papers (see my earlier post). And this current, informal study continues to confirm what all the other studies have shown: When you strip away branding information, there is no clear winner from among the top-contending search engines. Maybe years ago, Google was leaps and bounds better than all the others. Today, it does not appear to be the case.
The reason I point out this informal study is not only to continue to raise awareness of the essential parity among the engines, but to point out something interesting that the author of the post (Philipp Lenssen) says:
Some caveats: Even when the logo will not be visible, it’s entirely possible that using any particular search engine for a long time will subconsciously makes you adjust how you formulate queries – namely, you might train to phrase searches in such a way that your particular (imperfect) search engine of choice returns its most relevant results. So if you’ve used Google for years now, then it’s worth noting that your queries themselves might be more “Google-friendly”, thus give Google a slight edge over competition.
It is, of course, an obvious point. Still, it is a point that I do not hear getting made that often. And this is important, because there is a lot of discussion about how additional search engines appearing on the market can’t simply achieve parity to Google. They have to be markedly better. But if the user him- or herself is subconsciously stuck in a Google-biased query rut, then Google will always tautologically be better, no matter what other tools are available. So as information researchers, it seems to me that we really shouldn’t be worrying about whether one engine is 3% or 7% better or worse than another. The discussion that we as information retrieval researchers should be having is how to get the user out of existing ruts, and into consciously better ways of search engine interaction.
I did my own evaluations using my real searches over a day or two and found Bing and Google neck-and-neck, with some clear advantages one way or the other in verticals (Bing on travel, Google on software). What surprised me on the blind searches is that Yahoo! does so well.
I was wondering about bias the other way around, when I started using Google syntax on Bing. I realized Bing was pretty much forced into doing things Google’s way to support existing search habits.