On Twitter today, Josh Young made an interesting observation to which I would like to call attention:
Ya, @jerepick, with “fauxpen” attached, google’s “nav. search as the top of the stack” is a fragile local maximum for the web.
This observation is a followup to the web-wide discussion that Google kicked off about the meaning of open. Essentially, Rosenberg says that all of Google’s products at that are not at search layers of the stack should work toward being open, but that the search layer itself should be closed. To protect it from spammers, you understand {cough}.
Earlier in the same post Rosenberg makes a distinction between open source and open data, calling for increased openness in both. However, when it comes to defending closed-search, this distinction gets lost. But this distinction between open source vs. open data is important. Here is how it translates to the search domain:
- Open Source = Open search algorithm is about letting the world know what features are used to rank pages and how those features interrelate (are weighted)
- Open Data = Open search results is about letting users refactor, remix, reuse, mashup, store and re-search locally any and all query results that the user issues. And about letting the user use any software that they want to accomplish this — not just Google software
The excuse given about why Google cannot open up is that of spammers would be able to game the engine. But if we look closely, we’ll see that it is an excuse that is primarily, if not exclusively, related to the “open source” aspect of openness. Black hat SEO algorithmic gaming is not an issue when it comes to user results re-use and remixing.
And so the point (I think) Josh is making is that by closing not only the algorithm, but also the results of that algorithm, Google has effectively declared a moratorium on Internet application stack progress along that vertical. Google is essentially saying to the Internet: “You shall not pass. If anyone wants to develop a application that makes use of search results as a “lower” stack layer, that person will have to write an entire search engine, themself. We are in favor of any layer underneath us — or parallel to us such as gmail — growing the internet pie, but we will not directly participate in growing the pie ourselves by opening up our results so as to allow search itself to become a middling layer in someone else’s stack.”
This does not sit well with me. A search engine by nature is built on the stack layer of web page content, which is built on the stack layer of internet and transmission control protocols, and so on. To say that users have no right to use whatever software they choose to build further on this search engine layer denies users the same basic open rights that people like Lawrence Lessig so passionately fight for. In fact, some have argued that these rights are not even Google’s to grant, that fair use lets us re-use the search results that we, by our querying effort, had a hand in creating. So in the spirit of openness Google should fully open up its results to programmatic, API access. And allow users to remix and reuse, to metasearch and to share (e.g. social search and collaborative search). That grows the pie, does it not?
I do understand why Google will not do this. It’s because by so doing, Google would effectively allow itself to be disintermediated, the same way they are currently disintermediating the newspaper industry. By decoupling results from ads (where money is made), it makes it much more difficult for Google to monetize its traffic — a problem that all disintermediated layers of the stack face. Naturally Google doesn’t want to put itself in this position. But that is what makes their current stance on “open” all the more perplexing — they expect others (e.g. newspapers) to open up their revenue-stream stack layers, but refuse to do so themselves. Why take such a strong position on openness and then give an unrelated (spammer) excuse about why you cannot be?
So why am I writing about this? Again, let’s go back to something Josh just retweeted:
RT @jonathanglick The Open debate matters because, right now, for the first time in a decade, the forces of Closed are on the march.
This, combined with Google’s open call for earnest web-wide discussion and debate, has increased my desire to add to the conversation. And my point here is that spammers are not the issue when it come to making Google search “open”. You can open the data without opening the algorithm. If anyone has pointers to refutations to this line of reasoning, I welcome them.