The Jaded Surfer
[Part 2 in a series about relevancy.]
Search engines love to tell us that they are the most relevant, and I don’t blame them. A glance through any engine’s press releases will include claims like “most relevant updateâ€, “a dramatic increase in relevancyâ€, and so forth. These conflicting claims are like political rhetoric. Ultimately they have the opposite effect of what was intended, because we are all becoming jaded searchers. Here are some examples, and be sure to note how many claim to be the most relevant:
- With the largest index of websites available on the World Wide Web and the industry’s most advanced search technology, Google Inc. delivers the fastest and easiest way to find relevant information on the Internet.
http://www.google.com/press/pressrel/aol.html - The [MSN] index, much of which is updated weekly or even daily, provides the most relevant, timely and accurate data as quickly as possible, while minimizing frustrating dead links. http://www.microsoft.com/presspass/press/2004/nov04/11-11SearchBetaLaunchPR.asp
- [Accoona] empowers users to find the most relevant information through its Artificial Intelligence powered search engine.
http://biz.yahoo.com/bw/041206/66005_1.html - A pioneer in Web search technology, Inktomi provides millions of users worldwide with the freshest and most relevant search experience, and ensures that thousands of online retailers and other sites have their content constantly represented.
http://docs.yahoo.com/docs/pr/release1110.html - [Mamma.com]…makes it easier and faster for people to find information by gathering the most relevant results from the best search engines on the Internet. http://www.mammamediasolutions.com/corporate/pr/2004/08-11-04.html
- Yahoo! Search offers one of the most extensive search services available to ensure consumers find the most relevant results to their search.
http://docs.yahoo.com/docs/pr/release983.html
Search companies must be allowed to say their product is relevant. Otherwise, how can they market themselves? I am not disputing any of these company’s claims or passing judgment on their right to proclaim their search product as being relevant. The issue I am emphasizing is that these claims are subjective and need to be understood as such because they can not all be the most relevant, easiest, most extensive, and fastest.
Relevancy is like Pornography
So, with all that being said, what exactly is this elusive specter called relevancy, and how is it identified? It’s like the classic question: what is pornography? We don’t know how to define it, but we know it when we see it. Similarly, search engine relevancy is subjective and means something different to everyone. And not only that, it can mean different things to the same person at different times. A relevant result can be the site that provides the exact answer to a question; it can be the authority in its topic that provides a broad selection of information; or it might be a new site in a topic that is already known very well. That is why relevancy evaluations must be comparison-based. I know that the results on one engine are bad because another engine has better results. If the other engine did not exist, then my level of expectation would be lowered and it is possible that the first engine’s results would seem relevant to me. Underlying this notion is the notion that both engines pass a minimum threshold of relevancy. It is conceivable that the results could all be not relevant, in which case the comparison does not even come in to play.
Relevancy evaluation changes based on the types of information being sought. Each and every query for information needs to be reevaluated every single time. Users must never think that the results on their favorite engine are always the most relevant. If searchers do not find what they want, they can do a few things: they can come up with a new search strategy and approach the problem from a different angle; they can stick with the same strategy, but refine the query by making it narrower or broader, or by using advanced options and syntax; and, lastly, if the engine is still not finding what they are looking for, they can go elsewhere and try a different search tool.
Next Installment - Part 3: A Call to Arms
