There is an interesting blogpost on the Official Google blog today, about seeing stars:
We’ve long believed that personalization makes search more relevant and fun. For nearly five years, we’ve been tailoring results with personalized search. Today we’re announcing a new feature in search that makes it easier for you to mark and rediscover your favorite web content — stars. With stars, you can simply click the star marker on any search result or map and the next time you perform a search, that item will appear in a special list right at the top of your results when relevant. That means if you star the official websites for your favorite football teams, you might see those results right at the top of your next search for [nfl].
So it sounds to me like this is a sort of bookmarking. What it not as obviously, however, is what this sentence means:“the next time you perform a search, that item will appear in a special list right at the top of your results when relevant”. Does that mean the next time you perform the same search (e.g. [nfl]) that starred item will appear at the top? Or is it more dynamic than that? I.e., if I happen to perform the search [new england patriots], and that same link that I’d previously starred after executing the [nfl] query happens to be ranked in the top k, will it again appear at the top of my list? (And if so, what is the cutoff/threshold for k?) Similarly, if Google’s ranking of my original [nfl] query changes, due to shifting PageRank calculations, changes in freshness, or any of the hundreds++ of other signals that go into the ranking algorithm, and my particular starred web page no longer appears in the top k because it is no longer relevant to the [nfl] query using the signal vector from the current state of index, will the starred item not appear? After all, Google says that the starred item will only appear if it is relevant, and if it is no longer relevant to the [nfl] query, as determined by Google’s relevance algorithm, then it won’t appear? Even though I had previously starred it with respect to that exact query?
The post continues:
In our testing, we learned that people really liked the idea of marking a website for future reference, but they didn’t like changing the order of Google’s organic search results. With stars, we’ve created a lightweight and flexible way for people to mark and rediscover web content.
Now I am thoroughly confused. People didn’t like changing the order of Google’s organic search results, but at the same time, they claim earlier in the post that “For nearly five years, we’ve been tailoring results with personalized search.” What does it mean to personalize search results, if not to change the order of Google’s organic search results? (Quoting the earlier post:
With the launch of Personalized Search, you can use that search history you’ve been building to get better results. You probably won’t notice much difference at first, but as your search history grows, your personalized results will gradually improve.
So if users didn’t like changing the order of the organic search results, does this mean that Google has turned off (or will be turning off) personalization completely for all signed-in users? Or does personalization co-exist with explicit starring/bookmarks? If so, how exactly does that work? Will Google change the order (personalize) your organic results using only the signals of query history and implicit relevance (i.e. clickthrough), but not the signal of explicit starring? That’s even more confusing…the amount of mental jazz involved is a bit overwhelming. Sure, the interface jazz is kept to a minimum, but at the expense of making the user’s mental model of what the search engine is actually doing for him or her even more muddled.
Perhaps the best way to sort out this confusion is to dive in headfirst and start playing around with the system, seeing what it actually does and when. But I personally have a difficult time generating the gumption to use a feature for which I have an unclear mental model, an unclear understanding of what it is trying to do for me, how it might change, when it might or might not magically appear. Especially when some of my actions affect the state of the system and others do not.
One thing I do like about this feature, however, is that it uses out-of-band displays to show different types of information. Rather than trying to mix global/non-personalized results, implicit personalized results, and starred results, it lets you know via a separate channel whether there is any information that you have previously starred. This is an IR design principle that I would like to see more of — separate goals in separate channels. Examples of different IR goals include navigation, re-finding, discovery, exploration, etc. Rather than trying to mix results from all of these goals into a single channel (a single ranked list) it is quite useful to separate each goal from the other. This new Google interface does that. What exactly the goal attached to that separate channel is, again, unclear. But the existence of a separate channel is an interesting and exciting approach, one that I hope to see more of.