Kartoo

Type of engine: Visual search.

SUMMARY
Relevancy of results:Needs improvement.
Freshness of results:Needs improvement.
Features and functionality:Good.
Quality of help and “about us” pages:Average.
Business model:They sell a variety of search packages that can be reviewed on their solutions page.

INTRO
Remember a few years ago when you were sitting at your desk one slow afternoon and you got an email about a cool new search tool called Kartoo? The sender wrote something like, “You’ve got to check this out. It’s really cool.” And you checked it out and you thought, “Yeah, that is cool.” Remember that day? I certainly do. And then remember what happened next? Every time someone asked you about a cool search tool, you said, “Have you seen Kartoo?”
For years, “cool search engine” has been synonymous with Kartoo. In fact, check out the Yahoo results for cool search engine. What is that in position #1? It’s none other than Kartoo.

But here is the big question: besides calling Kartoo cool, how often do you use it? If you’re like me, the answer is seldom. So I decided to take a closer look at Kartoo and investigate its usefulness beyond the cool factor.

Kartoo launched on April 25, 2002 and since then has gained a reputation as the visual web search engine. On Google, MSN and Yahoo, the search visual search engine returns Kartoo in the first position. Kartoo offers a unique experience because although they are another meta-search engine, their search is based on their “visual display interface”, as they call it. The visual interface uses Macromedia Flash, though, they do offer an HTML version as well.

UI AND FEATURES
Kartoo has a legend for their visual display so that you can identify things such as Sponsored Sites, recently updated pages, clusters, certain file types, and domain types (.org, .net, .com). These are all useful designations but it takes some practice to get used to them. There is a lot to look at on a Kartoo results page, or map as it’s called, and so it’s not easy to distinguish some of the subtle visual clues that are available.

Kartoo offers Boolean and other advanced search syntax. Take a look at their Key Tips page for more about this. They state that by adding a question mark at the end of a query, Kartoo interprets the query as a natural language query. But I tried what is the population of Scotland? and then what year did Kartoo launch? and did not find the results useful. The Kartoo interface is not built for natural language searches because you have to mouse-over each result looking to see if it has your answer. And even doing that I could not find the answers I was looking for.

There is a FAQs page that gives more explanations about their technology. One particular question I found interesting is this one:
Is KartOO technology more pertinent than other search engines?
“It often is but not always… In fact, KartOO technology analyses the words you are asking for and then decides to question the most accurate search engines….As to the notion of relevance: when you ask for the word “ray” for example, you may mean the sea animal or the light device. The results you obtain may therefore be accurate or totally irrelevant to what you are looking for.
What is significant about KartOO in such a situation is that this technology provides a map that summarizes all the various and possible topics so that retrieved sites are in fact grouped into a form of topical “family”. A list, i.e., a linear classification of search results, could not represent all the applications connected to a word like “nuclear” for example, and above all, a list could not display the links existing between the applications.”

In other words, Kartoo’s visual interface acts as a clustering engine because it lets users look horizontally across a broad selection of sites quicker than going through a linear list. As their example states, this can be particularly effective for ambiguous queries where the searcher is trying to understand various meanings of the search term. Though it is my opinion that if someone is searching for info about something like the planet Saturn they will type in Saturn the planet rather than the ambiguous Saturn. But in any case I am a big believer in clustering even though I also think engines can expect searchers to help them out a bit.

The search results are a bit slow, but they distract you by showing a neat looking genie who is deep in thought.

One thing that bothered me is that after I clicked on something in the map (that is to say, the search results page), I could not go back one step. I had to reload the original query. It’s really frustrating that I can’t take one step backwards. You can do so when you click on the Next Map link down in the lower right hand corner, there is an option to go back to Previous Map, but not if you click on a topic. There is a drop-down list of my recent searches so I can get back that way. However, the interface let me down because my queries were too long so I can’t tell the difference between “raymond chandler” “dashiell hammett” and “raymond chandler” “dashiell hammett” mystery because the words get cut off.

On the search results page if you mouse over the paper looking icons you’ll see the text summary for the site appear to the left. I should mention a small thing, but something I like. When I click on a site, Kartoo counts the number of times I click on it. That’s helpful with a visual interface so that I can quickly see the paths I have already traveled.

RELEVANCY EVALUATION
I’ve been reading, and really enjoying, some classic hard-boiled mysteries by the masters Raymond Chandler and Dashiell Hammett. I wanted to know what influence Hammett had on Chandler. What did Chandler think of Hammett’s stories?

I searched very generally, just using the men’s names, “raymond chandler” “dashiell hammett”. The first thing I did was to click on the two links that figure most prominently in the middle of the map. The first one is a Wal-Mart page selling a book called “Hard Boiled Mystery Writers: Raymond Chandler, Dashiell Hammett, Ross Macdonald.” OK, that is relevant to my search terms, but I was not looking for a shopping site selling a book about the writers. I was looking for information on the web about the writers. It turns out the second link is also a retail site selling the same book, only it’s from AddALL.com and it doesn’t have the three paragraph summary that the Wal-Mart site has.

Two of the results on the map were classified as articles, (in other words, there is a yellow line that connects the listings with the word “articles” implying that both listings are related to that topic) which sounded promising. One was in French and the other was from High Beam, but to view it I needed to sign up with them.

To summarize the remainder of my experience for this first search, each site I clicked on was either a shopping site selling the book I mentioned earlier or a page in French, with one exception: there was a detailed bibliography of Raymond Chandler that includes this nice quote, “Dashiell Hammett may have shown how mean those streets could be, but Raymond Chandler imagined a man who could go down those streets who was not himself mean.” Not exactly a detailed comparison, but a good quote nonetheless.

Over on the left side of the page is a list of twenty related topics. Normally I would acknowledge that my query isn’t very good and would refine or adjust it accordingly. But in this case I will use Kartoo’s related topics as my refinement.

Here are some of them:
Hardboiled mystery writers, Ross MacDonald, Auteur, Roman, Article, Amazon, Fiction, Hard, Library, Mystery, Writers, Book, Matthew

I clicked on hardboiled mystery writers. Doing so creates a new set of topics, some of which are good, such as detective fiction. Some of which are not so good, like isbn and featured.
I noticed that there are different Amazon country sites showing up, such as .UK. and .CA.. There are also other shopping sites like Overstock. Along with these shopping sites there are a couple of decent sites such as a Dashiell Hammett bibliography from a fan site that lists four books about the two writers.

I also noticed that even though I limited my search to English pages only, there were still French results.

Not having much luck with my Chandler and Hammett query, it is time to try a whole different user mission. Lately there has been a lot of news here in California about the resignation of the Secretary of State. So I decided it would be a good idea to learn more about just what exactly that position entails.
I searched for California secretary of state job functions. A cursory glance through the results shows a variety of suggested topics that are not quite relevant. There is logistique supply chain, California whitewater, vacation, features, and so forth. None of the related topics offered me anything useful.

Turning to the site results, there is a site about long distance phone rates, another about whitewater rafting, and a travel site. There is also a report written by the former Secretary of State in 2000. As with the related topics, none of these results are helpful to me.

I clicked on the next map and was shown some job listings, a hotel site, a vacation home rental site and so forth. Again, nothing to help me understand the job functions of this position.

I refined my search and entered in California secretary of state responsibilities. Again there are vacation rental listings, a cat breeder site, a computer store, insurance company, long distance phone service, and so forth. Again, nothing close to my user mission.

CONCLUSION
So what is going on here? I am seeing some obvious issues. One is that shopping sites are being boosted high in results. Although I can see why some shopping sites would be returned for my Chandler/Hammett search, these should have been a relatively small percentage of the result set. And for my Secretary of State search there should have been few if any commercial sites. It seems like the word California created a slew of false positives which would explain the vacation rental and whitewater rafting sites.

Another area I see for improvement is the related topics. By way of comparison, I tried California Secretary of State responsibilities on Clusty and got some relevant clusters, such as Kevin Shelleyand Office of the California Secretary of State. And in the list of results is this helpful site called State Executive Branch Overview that has a section called What are the Duties of the Secretary of State?

And lastly, the results are not fresh. There were too many results from several years ago appearing in my maps. I realize that Kartoo is a meta-search engine so they are relying on external indexes, but they should still be able to improve upon relevancy, topics and freshness of results from the engines they are pulling from.

So yes, there is no doubt that Kartoo is cool looking, but it really needs to create better topics and to return more relevant sites in order for it to be useful. Right now I consider Kartoo a novelty with great potential more than a really useful search tool. It may be that the Kartoo.com web search is just their way of getting attention to the search solutions that they are selling, but if that is the case I think it is all the more reason to improve upon the web search part of their business.