Google wins — because it doesn’t suck

LeeAnn Prescott from Hitwise has a much-discussed report about Google’s blog search getting a greater “market share of web visits” (Hitwise terminology for a combination of page views and visitors) than Technorati, the original blog search engine. This has led Om Malik, among others to write Technorati’s eulogy. I’m not…

LeeAnn Prescott from Hitwise has a much-discussed report about Google’s blog search getting a greater “market share of web visits” (Hitwise terminology for a combination of page views and visitors) than Technorati, the original blog search engine. This has led Om Malik, among others to write Technorati’s eulogy.

I’m not sure whether the ascendence of Google’s blog search spells the end of Technorati and/or Sphere — another blog search tool, which has done deals with media outlets such as Time magazine to put a “Sphere It” button on their stories — but I am sure of one thing. Google’s blog search is better for one very simple reason: It doesn’t suck.

I should qualify that. Technorati can be useful for searching specific terms, and using the “authority” ranking is not a bad tool. But when it comes right down to it, I agree with Erick searching for posts on a topic through Technorati is just not very useful — or not as useful as Google’s blog search. As Zoli Erdos and others have pointed out many times (here’s his latest roundup), Technorati also has numerous technical problems that continue to crop up.

Searching related posts through Sphere, meanwhile, is quite honestly pathetic. Whenever I see a “Sphere It” button or link, I click it just to see what happens, and 99 times out of 100 it is a boatload of crap. I’ve seen links to things that literally don’t make any sense at all. Beside a blog post I recently saw a link that said there were over 1,000 related posts at Sphere, and I just knew that the vast majority were going to be functionally useless.

For the record, Mark Cuban’s IceRocket blog search isn’t much better. When I want to write about a particular issue, either on the blog or for a story at the Globe and Mail, I will almost always search to see if a blogger somewhere has linked to something that might present an alternative point of view or an interesting perspective — and I routinely wind up back at Google’s blog search.

Is it that Google’s algorithm is better? I’m not enough of a geek to know. You have to admit that Google knows search. And one thing I know for sure about their blog search: It is just better.

Update:

Mark Cuban, who is not only IceRocket’s founder but also clearly its chief evangelist, has posted several comments here defending his company’s blog search that are well worth reading. After giving it some thought, I would like to revise my original comment that IceRocket “isn’t much better” than Sphere. For some searches, it clearly is better — and arguably as good as Google. And no, I didn’t change my mind just because Mark beat up on me :-)

Update 2:

I spent some time on the phone with Tony and Martin of Sphere (who responded in the comments on my initial post), and I think that — much as I like the imagery — “boatload of crap” might have been a little harsh when describing Sphere’s results. As Martin points out, some of the searches, including ones that use blog posts of mine as the source, bring fairly targeted and relevant results.

I think the problem, at least from my perspective, is twofold: One, Sphere draws relevance from the entire post as well as from the rest of the blog — therefore, if a post is short and/or the blog writes about a lot of different subjects, then a Sphere search isn’t going to come up with results that are all that relevant (Tony says it didn’t work particularly well on Scoble’s blog for that reason).

What happens in those cases is that Sphere comes up with a lot of related posts, which is why on some blogs I see a Sphere widget that says “56,975 related posts” and the first thing I think is “bullshit — there can’t possibly be that many related posts.” (Tony agreed with me on that one).

The bottom line is that Sphere is still trying to find the best method, just as Google and IceRocket are. I appreciate Tony and Martin taking the time to talk to me about it, especially after I dumped on them.

I’m glad to see someone else write what I’ve been thinking about Sphere. I keep trying to see how it’s so great, but to me it’s just…well, it serves up crap, like you said. It seems like it should be a simple enough thing to aggregate tags and/or categories if nothing else, but 9 times out of ten it just seems to pull up random, spammy sites that don’t really have any relationship to the one with the Sphere It! link.

I’m not a fan of any of the blog search engines, but I must admit that I gave up on google blog search a long time ago — perhaps it’s gotten better, at least enough to reach the “doesn’t suck” category as you say, Mathew?

I think the popularity of memetrackers such as memeorandum is partly the result of blog search engines generally sucking.

I agree, Eric — the popularity of Techmeme is probably a result of blog search generally not being great. I tend to use it after I spot something on Techmeme and want to find what other bloggers that Techmeme missed might be saying.

Look Matthew, I know you like to rip anything Im involved with, but come on.

Im happy to match icerocket.com results against Technorati or google and I dont find sphere even usable.

Google is littered with splogs. Techno does a better job, but is splog heavy and slow as can be, incredibly cluttered and often is missing posts.

Based on the comscore pageview numbers for both, our traffic is right up there with either. However, in blogsearch, pageviews dont mean much. MOST repeat users setup RSS feeds for their searchs. That is what Icerocket is optimal for, repetitive searchs. Its clean, its easy, its fast.

And if you didnt know, we also power A9 blog search along with others.

Now, Mark, that’s not true at all — I don’t rip everything you’re involved with. In fact, I’m a big fan of yours. Okay, I gave you a hard time over that whole Google-YouTube “moron” thing, but that was a one-time deal, I assure you :-)

I will accept your argument that IceRocket’s results stack up against Google search, and that your traffic is “right up there” according to comScore. All I can say is that when I do a search looking for blogs that are related to a topic or a specific post, I find more hits that are of value to me with Google than I do with Sphere or IceRocket.

I don’t want anyone to think I was ripping IceRocket or even Sphere just to be critical — I was just saying what works for me. As the Slashdot crowd likes to say, your mileage may vary. In any case, thanks for the comment.

I manage icerocket on a daily basis. The only area Google has an advantage on us with is with Blogger.com. They get those automatically and sooner. Plus we scrub our results harder to get fewer splogs, so we are usually an hour behind google.

Tell me when you find fewer results on icerocket so i can find out why.

Usually when i review with someone, it comes down to they just trust google more and our results are usually, not always, but usually better.

Try our RSS feeds as well. I think you find they work as well , if not better

As of a year ago or so (the last time I was really engaged with blog search) I have to say that I liked Mr. Cuban’s IceRocket better than Google, which I personally ranked behind both IceRocket and Technorati (the latter of which was a clunky mess).

That said, I just searched for “Mathew Ingram” on both Google blog search and Ice Rocket… and I have to say that I found more Google *a little* more relevant. But I really do like Ice Rocket’s feature that lets you select between post by Mathew and posts about Mathew. As someone who taught basic library and Internet research sessions all over the desert of southern California for more than a year, I *really* appreciate this distinction.

Mark, I wrote my comment at nearly the same time you wrote yours (which is kind of a strange coincidence, really). I didn’t use the advanced search feature to search by author using IceRocket, I must note!

I searched Google, Ask, IceRocket, Technorati and Sphere for links to the Susan Crawford blog post that is at the top of Techmeme right now (about the AT&T deal and network neutrality), which I know that many bloggers have written about.

To prove my point about Sphere, it said it had 6,954 related posts — but only the first three or four had anything to do with the issue. Technorati had 6 posts, all of which were relevant. Ask had 10 (although it said it had 48), all of which were relevant. Google blog search had links to 5 relevant posts, and IceRocket had a single result — a link from Tailrank.

you got us on these kinds of searchs for 2 reasons.
1. because its blogware.com, we are going to take longer to scrub it to make sure its not a splog. Like I said before, we are slower, but we try to be more efficient.
2. I dont think looking for “who links” is the most relevant way to search, and our icerocket its rarely how people search. I think a lot of bloggers search as you do, but not business people. In this case, I think they would do a search on AT&T BellSouth

compare IceRocket, we include all RSS feeds and have more news than Google. Google on the other hand has more splogs. Content thats repeated from site to site.

I like the way we do it of coures, but Google’s has definitely gotten better, while Technorati has easily gotten far worse

I think Google and Icerocket have good delmiters, but Icerocket makes them more obvious. As a previous poster mentioned, you can quickly see if any tags match your search. If any authors match your search, etc

Where technorati has excelled IMHO, is being a vanity search outlet.

Icerocket will have lots of new features coming. From video to social networking features. We arent high profile, but we have a growing base of users and RSS subs

Thanks, Mark. I think you are probably right about “who links” being something that bloggers tend to do but regular people don’t. And on a simple search for links about a topic, IceRocket looks pretty good — and Technorati definitely looks bad, with only four links and two of those clearly splogs (although one of your top six is too).

I must say I like the “focus or exclude” option in IceRocket, and I also like the ability to narrow down by a variety of time periods, which Google doesn’t do unless you go to the Google News archive.

At the same time though, I wonder why IceRocket includes so many links to regular news sites like MercuryNews or aggregators like Google News (three links out of 10) in what is supposed to be a blog search.

In any case, I appreciate you explaining things as much as you have. That’s more than I’ve gotten from the folks at either Technorati or Sphere.

what a love fest :)
To answer your question on News. I want EVERY RSS feed except comments (because of so much spam) included. Although we call ourselves a blog search engine, thats a misnomer. Its just easier for people to understand.

We want to be the best place to search for fresh information. rather than just blog posts. So often we include rss feeds of popular search topics.

I also think in the future that sitemaps will allow us to know when and what has changed on traditional websites, which we will include as well. So if ibm.com adds a new segment on support for a certain product, I want that to be a result in a search for that product.

If a new video is added to a video hosting site about a topic, we want to see that included.

When you search for ibm on google, you will always get the same answer, or they will make you search all the different tabs, image, video, news, web, etc. We want to be the destination for those who want to know the latest information on the web about IBM.

Notice that you can get an RSS /Live Bookmark even from our web search..

icerocket is profitable and growing. We like being under the radar and making changes based on user feedback. Its worked for us

How could I not jump in here with a counter-example? Here are the SphereIt results for this very post: Sphere This.

Looks very on-topic to me. I’ll give you that there are some pages out there where SphereIt doesn’t perform as well, but in those cases the conventional, relational technologies for finding related content (e.g. Technorati’s link-based approach) is prone to veering off-target and showing spam, among other non-related content. SphereIt is a fresh approach to related content that doesn’t rely on the “echo chamber” approach taken by Technorati and others, who show only linked or tagged content, which in many cases is sparse and spammy.

Mark C – No, you’re right, I don’t think mainstream web users are going to copy and paste a URL. I also don’t think they’re going to go to a blog search engine to search for blogs, at least not yet. I do believe a large number of web users will discover high quality blog posts by using tools like sphere it (which is essentially a URL search). Look at TIME.com or ZDNet as examples of how well this tool works on a URL search. More and more publishers are looking for ways to engage with the blogosphere and many of them, with broad reach to mainstream web users, are adopting sphere it. Link based strategies like Technorati don’t work: too prone to spam, shallow result sets and latency. We believe URL search is an effective way to surface blog posts that are contextually relevant to the article they’re reading. Connecting readers to the larger conversation happening around topics is an interesting problem to solve and that’s what we’re focused on doing.

Thanks for the comment, Tony (Note: Tony is the founder and CEO of Sphere). I agree that URL search is a good way to find blogs that are writing about a story or an article, which is why it makes sense to have that type of search at Time.com or TechCrunch — but as I mentioned, every time I use a Sphere link, I get so many worthless results that it’s a substantial disincentive to try it again.

thats where disagree. As I mentioned to Matthew. Icerocket being called a blog search engine is a misnomer. Its a freshness engine. What is the freshest information available about anything.

For some reason you guys, techno and some others think its about helping people explore . As you call it, contextual relevance. Thats fine, and there is a market for that. I just dont think its the big market.

I like what you guys do with the pop up windows. Smart marketing, but also very confusing to people who arent looking for tourguides, but are looking for information. Some people dont want a conversation. They want information. My wife likes conversation, I like the details so I can get on with making a decision.

Icerocket is more of a business and productivity tool.

We arent in any rush to become the biggest and best as of yesterday. We are profitable, growing and enhancing our product.

Thanks for that comment above, Martin — I didn’t see it until now. Too busy trying to keep up with Mark and Tony :-) And you are right that many of the results from the Sphere search on this post were arguably relevant.

But how relevant is it to bring up links that simply mention the word Google or the word search? Maybe Sphere’s definition of relevance and mine are a little different.

Mark brings up an interesting point: Should we even be thinking of blog search as a separate category? Maybe blogs just be part of what comes up when you do a regular “news” search.

Mathew – I think we’re all trying to get to a better (more informative) user experiene from different angles. Whether or not blog results are presented separately from news results is simply a question of presentation. In our SphereIt Contextual Widget as used on TIME.com for example, blog results are presented alongside MSM results — in Sphere.com, we also show news results on blog search results pages. That aside, the more interesting aspect is that SphereIt can provide context even for posts that aren’t linked (where Technorati shows zero results) and for linked articles it provides more comprehensive context (context that is comprehensive enough and precise enough to form “detail” as Mark C. defines it). The SphereIt results for this post, btw, are much more precise than simply keyword matches on google and search. The SphereIt technology is providing context and detail specifically on thehitwise/google/blogsearch discussion. I tried it on a couple of your other posts, and it works well there, too. For example SphereIt:Sure, I’d love a free Ferrari, but …

I tried to compare the different blog search engines a little while back by testing out some phrases and I found that icerocket and google were the best (icerocket may have been slightly ahead of google at the time but google has probably pulled ahead now). I actually found this post because I was wondering if sphere had made deals with bloggers to put their bookmarklet exclusively on their blogs. Gigaom posts only use sphere now- they used to list digg and delicious before- and Paul Kedrosky exclusively uses sphere on an individual blog post’s webpage (while keeping the other two on the front page). I wonder if you can dig into this and see what kind of deals sphere has made. If they’re making these type of deals, I think they’re wasting their money hooking up with blogs when they should be spending it improving their mediocre product.

I actually never referenced your use of sphere in my comment, but now that you mention it, I do notice that you lump all the other services together into one button but sphere alone gets displayed by itself. As I said before, I notice similar behavior on other blogs. It makes me wonder if sphere is paying bloggers to do this: it’s clearly not because sphere is a great service or because they’re the market leader. Why is it that sphere gets such exclusive treatment? I can only assume they’re paying the bloggers for it.

All I can say is that Sphere isn’t paying me — I agreed to host the widget because I wanted to see how it worked, and because they asked me to.

As for lumping the other services together, I’m not sure what you mean — del.icio.us and digg and the rest are for saving or sharing a link. Sphere is for finding related links. They’re two completely different functions.

OK, I looked up sphere in more detail and I can see that I was confused about what sphere offered. Since they call their tool a bookmarklet, I assumed that they also offered saving/sharing similar to those other sites. Since I never signed up for sphere’s service or read about the “bookmarklet” in more detail, I did not know that it only ran a search on their engine, I assumed they did both bookmarking and relevance searching. Nevertheless, I never referenced your use of sphere in my original comment and I’m not sure why you brought it up. I’m still curious why Om and Paul give sphere such good treatment and as a journalist I was hoping it might make an interesting topic for you to clarify or follow up on. Just a suggestion, that’s all.

Ajay – we had a chance to catch up with Matthew following his original post. He had some really good insights that have driven some changes we’ve made/ are making to our service.

A few weeks ago, we asked Matthew to try out our sphere it plug-in. He agreed to do so and it is working well with his content. It’s not perfect but it gets better every week as we make adjustments.

Regarding your question about paying bloggers to deploy our plug-in, we have NEVER paid anyone to deploy the plug-in. It’s a service. It connects readers to more content. It drives page-views for ou partner sites. And, when deployed on major publisher sites like TIME, ZDNet, Market Watch and a bunch more going live in next few weeks, it’s an opportunity to connect a larger audience to blog content. Our icon is already deployed live on 2 billion mobthly article pages. That number will double by end of March. That’s a big opportunity to connect mainstream internet users with blog content.

If you have specific thought on hopw to make our service or the plug-in better, please email me (conrad@sphere.com) or call me at 650-319-2151. I’ve got thick skin so feel free to be direct.

Ajay, just to be clear — I know you didn’t bring up the Sphere link in reference to me. I only brought my involvement up because it seemed relevant to your comment about whether they paid bloggers or not, that’s all — which I think Tony has since clarified. I appreciate your comments.

Tony, thank you for the clarification. Wow, this is what makes blogs so great, that this question could be answered this way. Obviously you’re doing something right if all these people highlight your service without being paid to. I think these relevance services are going to be huge so you’ve certainly picked the right market.

Matthew, I never meant to refer to your use of sphere and I understand that you have no financial involvement with them. It’s great that you have a forum like this where discussions, like the one taking place in the comments here, can happen and that you don’t censor them (I’ve seen other bloggers do this, which is funny considering their stances on others doing similar stuff).

[…] So is Technorati the leader? The charts on Dave’s post look pretty good, although neither Hitwise nor Quantcast are infallible when it comes to measuring such things. But as Zoli Erdos points out, Technorati’s lead over Google may not be as large as it seems, and there is clearly still room for improvement — for example, Google’s blog search indexes comments as well as posts. And as I have written about previously, Sphere and Icerocket have their strengths (and weaknesses) as well. […]

[…] since Tony and his team seem like a great group. Before I added the Sphere plugin to my blog, I tore a strip off the company’s blog search for being irrelevant — and Tony not only took it in stride, […]

Where to find me

You can find me on Twitter at @mathewi, on Facebook and on LinkedIn, and I've been known to show up on Google+ from time to time. You can also contact me at mathew at mathewingram dot com and I will do my best to respond

Search

Search for:

Archives

Archives

about me

I'm a Toronto-based writer, and this is where I write about online media, technology and other interesting things I come across on the Web. I am a former senior writer at Gigaom, a former blogger, columnist, reporter and social-media editor at the Globe and Mail and a former writer with the Financial Times of Canada.