Importance of MetaSearch, a Study by Dogpile

6 comments
Source Title:
Dogpile via BusinessWire
Story Text:

Dogpile have today released a study which 'surprisingly' announced that, on average, there is a mere 3% overlap among the results returned on the first page by Google, Yahoo and Ask Jeeves.

"The results of the study show that search engines are not alike; in fact, the results among them can be quite different," said Dr. Jim Jansen, Assistant Professor at the School of Information Sciences and Technology at The Pennsylvania State University. "For anyone who utilizes search engines to find information, these findings demonstrate that a broader range of top-ranked results can be found by utilizing a metasearch engine, like Dogpile.com, which searches multiple engines simultaneously and brings together the highest ranked results in one place."

The study was conducted in collaboration with 2 US universities (The University of Pittsburgh and The Pennsylvania State University). The article also highlights Dogpile's new nifty flash tool Missing Pieces which allows users to enter any keyword and see a graphical display showing the level of overlap among Google, Yahoo and Ask Jeeves.

Comments

This study has further implications

The Dogpile study looked at the first page of results only. Lest some armchair critics chirp in and claim that it all comes together by the second or third page of results, let me tell you that it doesn't.

On Yahoo-Watch for at least six months I ran Yahoo and Google searches simultaneously, to 100 deep on each engine for each search. The average uniqueness (the percentage of Google links in a top 100 search that do not appear in the same top 100 search on Yahoo) was consistently at around 80 percent. This was for 1000 searches a day for six months. In other words, even when going 100 deep we had only a 20 percent overlap.

This degree of disparity between these two engines shows that the term "relevance ranking" is misleading. There is no agreement or consistency among engineers when it comes to defining this term. Whatever engine you use, you have to accept that most of the links that you consider most relevant will appear deeper than page two. In other words, most searchers are not finding the best results, because the engineers themselves are unable to rank them, and searchers don't have the time to wade many pages deep to examine all the links.

The algorithms will probably not get much better than they are already. This is a good argument for randomizing the top 100 or so results for the user. And any engine that claims that things like PageRank, or any other algorithm, has anything to do with relevance, should be required to prove it to the Federal Trade Commission. There are laws against false advertising.

Nice find

- actually both the Dogpile and the Yahoo-Watch findings. And the tool's great too, i've put it on my tools page.

This is a great argument for using Dogpile more, and the new layout is quite nifty as well - i like those columns you can show and hide, like A9.

 

Why not compare MSN as well in this study?

Also did the study include CPC results? If so it is skewed.

MSN are going to be added to Dogpile

close to the end of the year. A two year deal.

 

>Also did the study include CPC results?

I don't think they did. But I also think they forgot to mention how many CPC ads THEIR results include ;)

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.