The Elephant in the Room: Google Monoculture

I was browsing the sessions at an upcoming Search Conference, which describes itself thusly:

The way to online success is through being easily found in search engines such as Google, Yahoo!, and Microsoft Live Search. While developers have historically thought of search as a marketing activity, technical architecture has now become critical for search success.

Anyone else see the elephant in the room, there? No?

Just two weeks after we launched Stack Overflow, I mentioned that search engines already made up 50% of our traffic. Well, not so much search engines as search engine:

I try to be politically correct in discussing web search, avoiding the g-word whenever possible, desperately attempting to preserve the illusion that web search is actually a competitive market. But it's becoming a transparent and cruel joke at this point. When we say "web search" we mean one thing, and one thing only: Google. Rich Skrenta explains:

I'm not a professional analyst, and my approach here is pretty back-of-the-napkin. Still, it confirms what those of us in the search industry have known for a long time.

The New York Times, for instance, gets nearly 6 times as much traffic from Google as it does from Yahoo. Tripadvisor gets 8 times as much traffic from Google vs. Yahoo.

Even Yahoo's own sites are no different. While it receives a greater fraction of Yahoo search traffic than average, Yahoo's own flickr service gets 2.4 times as much traffic from Google as it does from Yahoo.

My favorite example: According to Hitwise, [ex] Yahoo blogger Jeremy Zawodny gets 92% of his inbound search traffic from Google, and only 2.7% from Yahoo.

That was written almost two years ago. Guess which way those numbers have gone since then?

Now that Stack Overflow has been chugging right along for almost six months, allow me to share the last month of our own data. Currently, 83% of our total traffic is from search engines, or rather, one particular search engine:

Search Engine	Visits
Google	3,417,919
Yahoo	9,779
Live	5,638
Search	2,961
AOL	1,274
Ask	1,186
MSN	1,177
Altavista	202
Yandex	191
Seznam	103

Those 6x and 8x numbers that Rich quoted two years ago seem awfully quaint now. Google delivers 350x the traffic to Stack Overflow that the next best so-called "search engine" does. Three hundred and fifty times!

Now, I don't claim that Stack Overflow is representative of every site on the internet -- obviously it isn't. It's a site for programmers. And let me be absolutely crystal clear that I have no problem at all with Google. That said, I find it profoundly disturbing that if every other search engine in the world shut down tomorrow, our website's traffic would be effectively unchanged. That's downright scary.

Yes, I like Google. Yes, Google works great and has been my homepage for about eight years now. Google nailed search, and they deserve the leadership position they've earned. But where's the healthy competition? Where's the incentive for Google to improve? All I see is a large and growing monoculture that acts as the start page for the internet.

I'm a little surprised all the people who were so up in arms about the Microsoft "monopoly" ten years ago aren't out in the streets today lighting torches and sharpening their pitchforks to go after Google. Does the fact that Google's products are mostly free and ad-supported somehow exempt it from the same scrutiny? Isn't anyone else concerned that Google, even with the best of "don't be evil" intentions, has become more master than servant?

Calling the current state of search engine competition a horse race is an insult to horse races. No, what we have here is a one horse race where all the other horses were shipped off to glue factories years ago. Forget "search conference", you should be throwing a "Google conference", because there's no difference.

I don't know. Maybe that's OK. But it does mean that if Google, for whatever reason, decided to remove you from its search results, your website no longer exists. At least not as a viable business, anyway.

[advertisement] Tired of restoring deleted files? Get PA File Sight and track down the culprit. PA File Sight ? file auditing made easy. Download the Free Trial!

Read More...