This blew my mind, not only that an alphageek would prefer the (Google) interface on the left to the (Bing) interface on the right, but that the redditor alphageek community would so heavily upvote it.  The way I see it, this speaks directly to the issues of simplicity as storytelling vs. sparsity that I’ve talked about from time to time.  The interface on the left is anything but sparse.  In fact, it is extremely busy and filled with images,  a tool belt of various verticals (news, video images), query modification tools such as timelines and recency sorting, and query reformulation tools such as narrowly related searches (top middle) and broadly related searches (lower left).

In short, everything about it is “non-Googly”, i.e. non-sparse and non-clean.  Ironically, the Bing results for this particular query — which is held up as the example of what not to do — is the cleaner one.

So why is it that thousands of Google-loving redditors prefer the interface that is, well, more Bing-like?  Could it be that the user is finally starting to understand that simplicity is not the same thing as sparsity?  That what matters is the story?  The Google results in this case tell a really good story.  They give a concise overview of the latest matches and scores.  They link directly to highlights.  They give a concise overview of upcoming matches and the time at which each occurs.  And they acknowledge that when you search for “World Cup”, you’re not just trying to navigate to a single page.  Instead, you are “exploratorily” looking for as much information as you can about what is happening at the event as a whole, and perhaps even with football (soccer) as a whole. This is not just a “one box” answer. This is a whole “cluttered” set of rich information and interaction options.

That’s the story.  And if it takes a non-sparse (complex or cluttered) interface to tell that story, then so be it.  The story is more important than the strict adherence to sparsity.  Which is something that I’ve been hammering on about for at least the past half decade now.  It is just encouraging to see users finally start to acknowledge it.

Now, all we need to do is let the redditor community know that even though Google beat Bing on this one particular query, overall Bing has been pushing more of this story-appropriate, non-sparse, information rich (”cluttered”) interaction in their results.  What I wish users did more of is constantly rotate between the various engines, to know for themselves which queries work on which engines, and what each of the various engines are capable of.  Because the irony here is that the redditor that which “This is Why I Use Google, Not Bing” has chosen and interface that is much more Bing-like, and less traditionally “Googly”.

See also my related post, about two Googlers (Norvig and an anonymous employee) and their comments about Bing at the Semantic Technology conference in June 2009.

Update: In the couple of minutes between when I saw the reddit link and when I finished writing this post, the Google vs. Bing image went from 4th on the reddit home page (with ~500 upvotes) to 2nd (with ~750 upvotes).  Clearly this has touched a nerve.  It’s very interesting to see this reaction, especially because the preferred interface, again, is so traditionally non-Googly and cluttered.

This experiment goes unmentioned by Russkin-Gutman, a major omission since it relates so closely to his subject. Even more notable was how the advanced chess experiment continued. In 2005, the online chess-playing site Playchess.com hosted what it called a “freestyle” chess tournament in which anyone could compete in teams with other players or computers. Normally, “anti-cheating” algorithms are employed by online sites to prevent, or at least discourage, players from cheating with computer assistance. (I wonder if these detection algorithms, which employ diagnostic analysis of moves and calculate probabilities, are any less “intelligent” than the playing programs they detect.)

Lured by the substantial prize money, several groups of strong grandmasters working with several computers at the same time entered the competition. At first, the results seemed predictable. The teams of human plus machine dominated even the strongest computers. The chess machine Hydra, which is a chess-specific supercomputer like Deep Blue, was no match for a strong human player using a relatively weak laptop. Human strategic guidance combined with the tactical acuity of a computer was overwhelming.

The surprise came at the conclusion of the event. The winner was revealed to be not a grandmaster with a state-of-the-art PC but a pair of amateur American chess players using three computers at the same time. Their skill at manipulating and “coaching” their computers to look very deeply into positions effectively counteracted the superior chess understanding of their grandmaster opponents and the greater computational power of other participants. Weak human + machine + better process was superior to a strong computer alone and, more remarkably, superior to a strong human + machine + inferior process.

This result seems awfully similar to some of the other results I’ve reported on in the past.  For example, see this paper by Amatriain:

Data is always important, but what struck me in the writeup was his discovery that the biggest advances came not from accumulation of massive amount of data, log files, clicks, etc.  Rather, while dozens and dozens of researchers around the world were struggling to reach that coveted 10% improvement by eking out every last drop of value from large data-only methods, Amatriain comparatively easily blew past that ceiling and hit 14%.  How?  He simply asked users to denoise their existing data by rerating a few items.  In short, Amatriain resorted to HCIR:

See also Tessa Lau’s post about how good interaction design trumps smart algorithms:

I come to the field of HCI via a background in AI, having learned the hard way that good interaction design trumps smart algorithms in the quest to deploy software that has an impact on millions of users. Currently a researcher at IBM’s Almaden Research Center, I lead a team that is exploring new ways of capturing and sharing knowledge about how people interact with the web.  We conduct HCI research in designing and developing new interaction paradigms for end-user programming.

See also two of my previous posts, More and Faster versus Smarter and More Effective and A Bird in the Hand.

The theme that I see is that, while big data approaches do work well, what works even better is a small amount of user interaction.  With big data methods  (even ones that incorporate human interaction in the form of massive log data) all you can do is make inferences about what is good and what is not good.  The more historical user data you have, the more correct your inference about the current scenario is likely to be.  But none of it is as correct as receiving explicit feedback from the user, and turning a probability into a certainty.

And that’s where I see good interaction design coming into play.  By turning a probability into a certainty, your back end algorithms can stop wasting their CPU cycles doing all the inferential heavy lifting about what the user is actually trying to say or do, and can start using their CPU cycles to explore a wider range of consequences of that informational certainty.

HOW IS THE INTERNET CHANGING THE WAY YOU THINK?

As an Information Retrieval research scientist, I of course was quite interested in what search folks had to say.  I found this blurb from Marissa Mayer intriguing:

It’s not what you know, it’s what you can find out. The Internet has put at the forefront resourcefulness and critical-thinking and relegated memorization of rote facts to mental exercise or enjoyment. Because of the abundance of information and this new emphasis on resourcefulness, the Internet creates a sense that anything is knowable or findable — as long as you can construct the right search, find the right tool, or connect to the right people. The Internet empowers better decision-making and a more efficient use of time…

The Web has also enabled amazing dynamic visualizations, where an ideal presentation of information is constructed — a table of comparisons or a data-enhanced map, for example. These visualizations — be it news from around the world displayed on a globe or a sortable table of airfares — can greatly enhance our understanding of the world or our sense of opportunity. We can understand in an instant what would have taken months to create just a few short years ago. Yet, the Internet’s lack of structure means that it is not possible to construct these types of visualizations over any or all data. To achieve true automated, general understanding and visualization, we will need much better machine learning, entity extraction, and semantics capable of operating at vast scale.

It sounds like there is an increased awareness of (and respect for) Exploratory Search.  I’ve heard this via private channels, but this is the first time I’ve seen an acknowledgment of the need for more exploratory search from such an official channel.

I do want to point out, however, that in order to make this work at web scale, we won’t just need better automated methods.  I.e. we cannot rely solely on machine learning, entity extraction, or web-scale semantics.  Rather, what is also desperately needed is a way for the user him- or herself to inject personal semantics and structure into the search, visualization, and comparison process.  The search engine itself needs to be responsive to the structure that the user is giving to it, and rearrange itself around that information.

I am afraid that I am not being very clear in the vision that I’m attempting to lay out, so let me draw an analogy to parametric and non-parametric statistical modeling.  In parametric modeling, you assume that your data is distributed according to some function (say, Gaussian) and then you try and find those parameters that best fit the data.  On the other hand, with non-parametric modeling you make no such assumption.  You simply let the data describe itself through its own correlations and patterns.

By analogy: Assuming that the only way to visualize and compare information (do exploratory search) on the web is to rely on machine learning to do entity extraction and web-scale semantics is like assuming that one has to have a parametric model.  It helps, but it is not absolutely necessary.  My vision is for another approach, one analogous to non-parametric methods: Let the user give feedback on the relationship between items that he or she has examined during the search process and then use that comparison information to build personalized visualization or comparison tool for that user’s specific information need, from the ground up.  Don’t rely on the parametric form of semantic categories or named entities.  Use bottom-up patterns to facilitate organization and comparison, discovery and learning, decision making and exploration.  More importantly, use the feedback provided by the user (e.g. “these two items are similar”, and “these two items are not”) to drive your online, bottom-up exploration.

We have to get away from this attempt to solve the exploration problem ahead of time, off-line, before the user has ever issued a query.  That’s the parametric way of thinking, the way that presumes that categories and labels and entities are the best way of tackling organization and discovery.  Rather, we have to become better at involving the user, the person doing the exploration, in the feedback loop, and not rely solely on pre-computed, machine-learning-extracted entities.

Unlike navigational search, in which users are rarely willing to do any extra work themselves, users engaged in exploratory search by their very nature desire to interact more with the system and put more of their own sweat and tears into the search process.  They would not be exploring, if they weren’t.  So why not make use of this user willingness?

Computational resources are going to be a challenge.  But that’s where Google’s new commitment to openness (and Yahoo!’s initial, existing commitment) comes in handy.  There should be a willingness to offload some of the computation (and therefore also the search data itself) to the user’s own computer.  Instead of SETI@Home, we could have SEARCH@Home.  Let the user’s underutilized processing power be partially responsible for computing some of these bottom-up patterns in his or her own search data that will help make dynamic visualization a reality.  Make the user’s own computer partially responsible for the additional necessary processing.

Mayer is correct: “The Internet has put at the forefront resourcefulness and critical-thinking and relegated memorization of rote facts to mental exercise or enjoyment. Because of the abundance of information and this new emphasis on resourcefulness, the Internet creates a sense that anything is knowable or findable — as long as you can construct the right search, find the right tool, or connect to the right people.“  We should be developing systems that enable the users to construct the right search. The user should be able to rely on our her resourcefulness to mash up and explore the data herself, to shed light on patterns of information hitherto unknowable by single-line input box navigational search.  Users should be able to apply critical thinking to their search process in a way that makes sense to the user, not in a way that has been pre-computed through some semantic category and machine learning classifier.  And a good search engine should be a valuable partner in this process, by way of flexibility and openness, not by way of constraint and closedness.

Only then will we, the users of these systems, be able to find out what we previously could not find out.  At least, that is how the Internet is changing the way I think.

The article focuses on YouTube’s “plans to rely more heavily on personalization and ties between users to refine recommendations” and “suggesting videos that users may want to watch based on what they have watched before, or on what others with similar tastes have enjoyed.”  What is striking about this is how little this has to do with search. As described in the article, what YouTube needs to do is entertain people who are bored but do not entirely know what they want. YouTube wants to get from users spending “15 minutes a day on the site” closer to the “five hours in front of the television.” This is entertainment, not search. Passive discovery, playlists of content, deep classification hierarchies, well maintained catalogs, and recommendations of what to watch next will play a part; keyword search likely will play a lesser role.

My feeling is that the dichotomy that is being drawn does not exhaustively cover the space.  I would characterize the space using the following two orthogonal dimensions: (1) Information Need Clarity and (2) User Engagement.  The first dimension (clarity) is related to the degree with which the user understands his or her own information need, i.e. has something specific in mind that he is looking for and/or understands what he needs to do to find it.  That need may either be well understood, or (to borrow Nick Belkin’s terminology) “anomalous”: The user doesn’t know what he or she doesn’t know.  The second dimension is related to the level at which the user applies himself to the information seeking process.  That level may be active or passive.

Greg points out two modes: “Active Understood” (typical navigational web search) and “Passive Anomalous” (entertainment/discovery/recommendation).  But I believe that there are more than these two modes.  A large, interesting design space opens up when one realizes that information seeking can be “Active Anomalous” and “Passive Understood“.

Exploratory Search is a good example of Active Anomalous seeking.  One doesn’t yet fully know or understand what it is that one is looking for, but at the same time one is willing to engage with an information system in order to discover what it is that he or she does not yet know.  And the system itself is designed not necessarily toward trying to answer a well understood need, but toward helping the user map out and better comprehend a space.

Collaborative Information Seeking (see here and here and here) is a good example of where an need may be well understood, but a user does not necessarily have to actively express every last query detail in order to get more information on a topic.  Why not?  Because when User #1 is explicitly collaborating with User #2, an algorithmic mediation engine can push some of User #2’s activity on to User #1 without requiring User #1 to make additional effort.  Note that I am not implying that every aspect of collaborative information seeking is passive; quite the contrary, as it requires at least one co-collaborator to be active.  I am only pointing out that it is a domain in which it becomes possible for a user to passively obtain specific information on a well understood need.

There is a lot discussion in the Information Retrieval Community on the similarities and differences between Search and Recommendation.  A fruitful tension opens up as one travels back and forth along the diagonal from Active Understood to Passive Anomalous; the two approaches often end up complementing each other.  Where I see much less discussion is on the tension that opens up along the other diagonal, between Passive Understood and Active Anomalous.  When Exploratory Search meets Collaborative Information Seeking, it yields Collaborative Exploratory Search and a whole host of interesting possibilities.  Over the coming year I will be blogging more about the tension along this alternative diagonal (both here at on the FXPAL blog) and what it means for the Information Retrieval systems I and others are designing.  Happy 2010!

Lost amid this discussion, however, is that the links people tend to share on social networks – news, blog posts, videos – are in categories Google barely makes money on. (The same point also seems lost on Rupert Murdoch and news organizations who accuse Google of profiting off their misery).

Searches related to news, blog posts, funny videos, etc. are mostly a loss leaders for Google. Google’s real business is selling ads for plane tickets, dvd players, and malpractice lawyers. (I realize this might be depressing to some internet idealists, but it’s a reality). Online advertising revenue is directly correlated with finding users who have purchasing intent. Google’s true primary competitive threats are product-related sites, especially Amazon. As it gets harder to find a washing machine on Google, people will skip search and go directly to Amazon and other product-related sites.

I’ll repeat the salient bit: “Google’s real business is selling ads for plane tickets, dvd players, and malpractice lawyers.”  What struck me about this statement was not its veracity.  What struck me was its relationship to exploratory search.   It is when searching for a plane ticket, purchasing an expensive consumer good, or hiring a decent lawyer that my need for exploratory search is at its highest.

So my question is whether or not there is a tension here between getting the users off of the results page as quickly as possible — especially when the route off that page is typically via an advertisement on which the search engine makes money — versus enabling the user to remain on the results page in a process-oriented mode of sorting and filtering and playing around with the results in a myriad of different ways, so as to come up with a set of options that best satisfies the exploratory need.

Do these two goals conflict?  Why or why not?  It is an old question, but I am still searching for a satisfactory answer.

Update: Perhaps I should have been more clear as to what characterizes an exploratory search session.  There are dozens of papers out there that tell the story much better than I can, so I will quote one of them.  It’s by Michael Levi at the U.S. Bureau of Labor Statistics, published at the Information Seeking Support Systems (ISSS) workshop in June 2008.  Title of the paper is “Musings on Information Seeking Support Systems”.  (See http://ils.unc.edu/ISSS/ISSS_final_report.pdf) I quote:

Some characteristics of open-ended, discovery-oriented exploration emerge:

1) I may not know, at the beginning, whether a seemingly straightforward line of inquiry will expand beyond recognition. Sometimes it will, sometimes it won’t. A lot depends on my mood at any given moment.

2) I can’t predict when the exploration will end. It may be when I’m satisfied that I have learned enough (which also would vary from day to day and query to query.) It may be when I get tired or bored. It may be when I’ve run out of time. Or it may be when I get distracted by dinner or the allure of the swimming pool.

3) I can’t determine, objectively, whether the exploration has been a success. There is usually no “right answer” against which I can measure my progress.

4) My exploration is not a linear process. I could get interested in a tangent at any time from which I may not return. I am also likely to backtrack, possibly with some regularity, either because a tangent proved unfulfilling and I want to resume my original quest, or because I thought of a new question (or a new way of formulating a previous question) to direct at a resource I visited previously.

5) I am likely to want to combine, compare, or contrast information from multiple sources. One of those sources is my memory – which may or may not be reliable in any given circumstance.

Levi then makes a number of recommendations about what an information seeking support system should do, to enable this sort of exploratory search:

A useful information seeking support system, then, would require the following minimum functionality:

1) It should not interfere with my behavior as listed under Characteristics of Exploration above.

2) It should give me capabilities at least as good as those listed under Manual Tools above.

3) It should positively assist my explorations by making them easier or faster or more comprehensive or less error-prone or…

In addition, an ISSS might give me capabilities that I never employed before because they were not possible or because I didn’t think of them.

But, to be truly a leap forwards, an ISSS would need to exhibit at least elements of discernment, judgment, subject matter expertise, and research savvy.

Again the question: Is there a tension here between getting the users off of the results page as quickly as possible — especially when the route off that page is typically via an advertisement on which the search engine makes money — versus enabling the user to remain on the results page in a process-oriented mode of sorting and filtering and playing around with the results in a myriad of different ways, so as to come up with a set of options that best satisfies the exploratory need?

I’ve already heard certain search engines state that their goal is to get the user off the search page as quickly as possible.  That it and of itself tells me that they’re specifically designing the system so as to interfere with behaviors listed under Characteristics of Exploration above (Levi’s first recommendation).  Why does it interfere?  Because my goal is to stick around in the results and compare and contrast, whereas their goal is to get me off of the page as quickly as possible.  And so the whole system is designed to do the opposite of what I want it to do.

Additionally, I was also pointing out that the information domains on which I usually have the largest exploratory-type information needs are very similar to the information domains on which the search engines make most to all of their money.  I’m still trying to figure out what to make of that.

Thoughts?

I know what I want, but I don’t know how to describe it.

I don’t know what I want, but I hope to figure it out once I see what’s out there.

Perhaps this is a silly analogy, but framing the problem in this way reminded me abstractly of P vs. NP.  Some problems can be both computed and verified in polynomial (P) time.  Other problems can be verified in P time, but it is unknown whether a P-time solution to the problem exists.  These are the non-deterministic polynomial set of problems (NP).  In the worst case, it might take exponential time to get the answer.

Google and related web search engines are lookup, navigational, known item engines.  You can both obtain and verify your answer in polynomial time.  Linear even (hence the classic ranked list).

With an exploratory information need, the satisfaction of your information need can be verified in polynomial time.  It doesn’t take too long to examine the assembled set of summarized/contrasted/accumulated information and tell whether or not your information need has been satisfied.  Maybe it doesn’t take constant time, but it certainly can be accomplished in linear time.  But accumulating that information in the first place?  It is generally unknown how long that will take, as there is a bit of non-determinism in the information seeking pathways that you need to traverse.  Exploratory search is, dare I say, NP.

So the big question: Is P = NP.  That is to say, can one use a tool such as Yahoo!, Google, etc. which has been generally optimized for lookup, P-time problems and use it to satisfy one’s exploratory information seeking task?  Certainly one can try and use these tools in this manner.  Nothing stops a user from entering vast quantities of queries and accumulating the necessary set of information themselves.  But the tool has not been designed for that purpose.  So can it really be used to solve that problem?  Are multiple iterations of lookup search capable of satisfying an exploratory information need?  Does P = NP?

I don’t think that it does.

Question for the day: For certain classes of NP problems (e.g. the knapsack problem) there are often heuristic that yield good approximations (nearly-optimal) solutions in P-time.  What are the analogous classes of problems in the exploratory information seeking domain?  And how would we, in general, recognize them?

Matt Ghering, a product marketing manager at Google, has been one of the people talking to the big four music labels about the new service, we’ve heard from one of our sources. And he has supposedly sent these screenshots of the look and feel of Google Music search to various rights holders and potential partners. The first screenshot shows how a search result might look on Google for a search for “U2.” A picture of the band is to the left of four streaming options for various songs, and the user has the option of listening via either iLike or LaLa. Click on one of the results, and a player pops up from the services that streams the song, along with an option to purchase the song for download.

I suppose the ability to find/stream a particular, known song is nice.  But that is not what music search / music retrieval is about.  Music retrieval is a fundamentally exploratory domain.  When you are looking for music to accompany a photo slideshow, or music to create a playlist at a party, or music to DJ at a social dance event (e.g. salsa or waltz), or simply want to discover new and interesting bands, genres, etc. a known item search is not very helpful.  You have to already know exactly what song (or band) you want in order to ask for exactly that song (or band).

With exploratory search, on the other hand, you don’t know what you don’t know.  When you want to find that perfect song for your photo slideshow, you may have never heard the song before, much less even heard of the artist that wrote/performed it.  How are you going to navigate the space of all recorded music to find your song, if all you have is a single line text input box?

You can’t.

Simple user interfaces are nice, until they become so simple and focused that they become unusable for your information need.  The metaphor that has been so successful for years in Web search does not apply to music search.  We will have to wait until next week to see if the leaked screenshots are indeed what the service will look like.  But if those turn out to be accurate, I have to seriously question the decision-making process that led to this conflation of Web Search user experience and Music Search user experience.  The goals are often so fundamentally different that I have a hard time understanding why the former got applied to the latter.

See also some of my previous posts:

• Music and Exploratory Search
• The Tyranny of Simplicity

UPDATE: Don’t forget that ISMIR 2009, held in Kobe, Japan, starts next week!  http://ismir2009.ismir.net/

UPDATE 2: Looks like the new Google Music service is being released today 28 October 2009 (see here and here).  Right in the middle of the ISMIR conference. Classy.  Despite the fact that the name of the launch event is “Google Discover Music”, the screenshots make it abundantly clear that there is nothing being offered beyond basic, known-item song or artist lookup.  There do not appear to be any real discovery tools, any exploratory interfaces.  In fact, the thing that strikes me most about the reports that I am seeing is this line:

If you search for an artist or album name, the OneBox will include a set of four songs that are chosen algorithmically by the partner music site, not by Google. Each song will be linked to an audio clip that will play in a Flash-based pop-up window provided by the partner site. In some cases, the partner may provide one full play of the song before defaulting to a 30-second preview.

Not only is this lookup, rather than discovery, but by popping up a flash-based audio window, rather than taking the user to the partner music site, Google is going against its long-held creed of getting the user off of its properties as quickly as possible.  Instead, this interface encourages users to linger on Google, rather than begin exploring music elsewhere.  That does not seem very Googly to me.

UPDATE 3: If you read the liveblogs of the event (links above), you’ll see that Google is saying that right now they are not planning to commercialize this service.  However, should that situation change, they are free to use my suggestion from April 2006 on how they might inject advertising into music: http://blogs.sun.com/plamere/entry/guaranteed_to_raise_a_smile

Via Daniel Tunkelang’s recent post, I think that Government information might be a perfect domain in which to ask (and answer) these sorts of questions.  The U.S. Open Government Initiative has as its goal the release of loads of raw government data for use by any individual or organization.  How are people going to use this data?  What types of questions will they ask?  What types of questions could they ask, if given the proper tools (i.e. what might they not know that they want to ask, until it becomes possible?)

Two types of information retrieval might be perfect for this domain: Exploratory Search and (Explicitly) Collaborative Search.  In exploratory search, the goal of your information seeking is to learn, discover, compare, contrast, etc. In explicitly collaborative search, your goal is to do something similar, but with another set of like-minded partners working with you on the same task/topic.  Each partner may have different expertise; one may be an expert in energy policy, another might understand trade and commerce, and another might have experience with the inner workings of Congress and understand how it works on a practical level.  If you put all these people together right now, the only way they can work together on a shared task is to search separately and then email each other their results.  What if, however, you could design a system that not only mediated between them on an interface level (immediate notification of marked documents and passages, shared highlighting of seen documents, etc.) but mediated between them on an algorithmic level as well?  Algorithmic mediation of the collaborative process would mean that the retrieval system itself has a hand in both combining and partitioning the inputs and actions of the search team members, as necessary.  They might then be able to find important, valuable information that none of the searchers, had they been working alone, could have.

It seems like an interesting domain, and one with real, potentially quite important consequences and societal implications.  It will be interesting to watch as this develops.

The problem with the Web, as I see it, is that it imposes, with its imperialistic iron fist, the “ecstatic surfing” behavior on everything and to the exclusion of other modes of experience (not just for how we listen to music, but for how we interact with all media once they’ve been digitized). In the pre-Web world, we not only enjoyed the thrill of the overnight sensation – the 45 that became the center of your waking hours for a week only to be replaced by the new song – but also the deeper thrill of the favorite band in whose work we deeply immersed ourselves, often following its progression over many records and many years. It wasn’t that long ago that buying an album represented, particularly for your average teenager, a significant investment. You thought a lot about that album before you bought it, and once you bought it you took it seriously – you listened to it. Repeatedly. Today, we’re quick to dismiss those ancient days of “scarcity” and to celebrate our current “abundance,” but scarcity had something going for it: it encouraged a deep engagement in listening to a particular piece of music, across the expanse of an album, and it also encouraged, in the artist, an interest in rewarding that engagement. I would like to get back the money I spent on records in my youth, but I would not give up the experience that money bought me.

Perhaps relevance feedback hasn’t been implemented on the web, not because it isn’t useful (it is), not because it doesn’t work (it does), not because it’s too complicated (it isn’t), or not even because it’s too inefficient (depends on the implementation, I suppose).  No, perhaps relevance feedback hasn’t been implemented on the web because most people are busy “ecstatically surfing” the web, favoring quick, easy, surface answers over deeper engagement and knowledge.  A breadth of popular answers may be more valuable to society than being able to go deeper on any one answer.  Carr continues:

It’s the deep, attentive engagement that the Web is draining away, as we fill our iTunes library with tens of thousands of “tracks” at little or no cost. What the Web tells us, over and over again, is that breadth destroys depth. Just hit Shuffle.

I do wonder if there are similarities, and if so, what the social implications are of a society built on breadth over depth.  Not that we aren’t mostly there, already.  But if we’re building information retrieval systems that purposely accelerate skimming and breadth at the expense of depth, if we increase that feedback loop, what does that portend?  It is a question I occasionally ponder as a think about the social impact of my research.

The panel covered a large number of topics.  But it was inevitable that the moderator would turn to the Google panelist (Peter Norvig) and ask him what he thought about Bing.  There has been too much buzz lately for that question not to be asked.  I was pleasantly surprised by his answer.  I’m not going to risk quoting him, only paraphrasing. And if I misrepresent anything, any mistakes are mine, and not intentional.

[Paraphrase] Norvig’s first answer to the Bing question was to say that he likes the idea of innovation in the user interface.  He thinks that there is a lot of room for more such innovation, and for a lot of different reasons.  Historically, there has been too much emphasis on getting the ranking right, at the expense of all else.  Of course (he added) a quality ranking is something that you absolutely must have.  But for too long it has been the only thing that has been worked on, and that needs to change.  He thinks Bing has made some good steps, and that there are a lot more that can be made as well.

Wow!  This is not the Google that I’ve known for a decade, the Google that has actively shunned most forms of interactivity, feedback, and exploration other than spelling correction.  Now, here is someone from the company echoing some of the frustrations that I’ve had, actually saying that too much emphasis has been placed on the ranking alone.  I never thought I’d hear that.  I am finally beginning to see a different side of the company, an attitudinal shift.  This is a very good thing.  Norvig continued:

[Paraphrase] He rhetorically asked whether “conversation” (search as a dialogue, interactive search, etc.) is the ultimate user interface.  His feeling was that the more you know what you are doing, the more keyword-like your queries are, and the less you need conversation.  It is when you don’t know what you are talking about that conversation comes into play.

I agree.  At the risk of overgeneralizing, the more you know what you are doing, the more likely it is that you’re doing a known-item query.  The more you don’t know, the more you need exploratory search.  This is a point most recently reiterated in a JCDL 2009 poster entitled “Designing Exploratory Search Tasks for User Studies of Information Seeking Support Systems“.  Many of us here have been clamoring about these issues for years, some folks even for decades.  But for the past 10-15 years of web search, there has been little to no acknowledgement of the need for exploratory search, and even less actually done about it.  It is encouraging to see the attitude starting to change.

I’ve ranted in the past about how functionality and interactivity is often sacrificed in the name of simplicity.  Perhaps increased competition is finally pushing Google out of the simplicity local maximum, and toward bigger, more effective information seeking peaks. One of the more widely re-tweeted tidbits on the #semtech2009 channel was the following:

@philsophygeek A Google employee just told me that Bing was the best thing that ever happened to interface design at Google #semtech2009

Indeed.