Netnose
Type of Engine: User ratings.
Overall: Needs Improvement.
If this engine were a drink it would be…a wine cooler. I like wine and I like juice, but the two together aren’t so great.

Intro
Netnose offers the ability for everyone to rate the relevancy of search terms to specific web sites. If I understand it correctly, sites only appear in Netnose results if they’ve been rated as relevant by a user. I guess that explains why so many queries I tried performed very poorly; the content simply had not been added to their database. I like the idea of user ratings, but a little-known engine like Netnose needs to do more to supplement their immediate relevance, otherwise users won’t come back. Right now it seems like they’re waiting for the community to do enough to make it a usable search tool, but they’re far from that goal.
I almost hate to give them a Needs Improvement rating, but my perspective is that of the searcher and relevance is my goal. So although the technology and goals of letting users add ratings may be admirable, I won’t be totally convinced until I see the relevancy improve.

UI and Features
One thing that bothered me was how do I rate specific sites rather than random ones? Let me explain. If you click on rating you’ll go to a page that lets you rate now. A Javascript pop-up window then opens along with a random web site. On the pop-up window there’s a selection of suggested query terms for the site. There’s also a ratings scale of bad, fair, good, better or best. You can also add commercial and adult tags. So you rate each term and then go to the next random site. Generally the suggested terms were decent. OK, so far so good, I suppose. But what about when I do a search? I want to be able to rate the search results right then and there and I don’t see a way to do that. You can assign a category at that point, but not a rating other than dead link/totally irrelevant. It seems like mapping the search results to the rating process would facilitate the process and garner more ratings.

The site submission page actually lets you enter metadata for newly addes sites, including 5 search terms and a category. Although I love the category idea, the categories don’t make a whole lot of sense to me: Just for Kids, Research, How To, Entertainment, Real Time Stats, Shopping, Business Related. Go ahead and try assigning one of these categories, it’s not always easy. There are also questions about whether the site sells products, has adult content, uses pop-ups, or has paid content. Letting the public add metadata is a noble idea and I’m far from giving up on it as viable, but having worked for two different search engines I know first-hand how devious spammers can be. Though I should add that sites don’t go live in search results until other user(s) rate them. All very interesting ideas similar to what some other community sustained sites have done, such as Zeal and ODP. Though those two directories are more browsing-based and Netnose is purely search-driven.

Query Examples
The relevancy is not so good. In fact, some times it’s very bad. For example, I searched for Barry Zito and the 1^st and 4^th results were about Barry Manilow. The 2^nd was about Dave Barry and the 3^rd Barry Choral Society. So then I went to the Advanced Search page and required both terms Barry and Zito and then I got no results. That’s an outcome of no one having mapped Barry Zito to a web page about him. It would be better to default to fall-through results from another engine than to show me sites about Barry Manilow that only match the first term of a two-term phrase query. Bad.

Try doing a search and then refreshing the browser. The ordering of results slightly changes each time. Odd…

Conclusion
I like the idea of Netnose, but for something like this to work there really has to be a critical mass of users working diligently to make a difference. Right now I don’t think that’s the case. Letting users rate sites is an idea that’s been discussed for years. I know Google played around with a toolbar rating system. I don’t think that ever caught on, but does anyone else know differently? It’s a chicken and egg thing here, because you won’t get enough users interested unless the search is good, but the search won’t get good until enough users rate sites.