Monthly Archives: March 2005

You are browsing the site archives by month.

Brainboost Officially Launches

Brainboost, the natural language engine, has implemented some changes and re-launched. As regular readers know, I’m a fan of their search technology. I recently interviewed Assaf Rozenblatt, and a while back I also did a review of Brainboost.

Besides the new logo and different color scheme, there are two major relevancy differences that I immediately noticed. The first is that on the home page there is now a whole slew of sample questions to ask. In fact, it looks just like a directory in that it is arranged hierarchically by subject. The subjects themselves are not links, so clicking on History is not possible. But beneath History is a selection of three sample questions. I suppose these sample questions help first-time users know how best to search Brainboost. And they also provide positive examples of the technology for those evaluating it for enterprise search. For me, as someone who already uses Brainboost on a regular basis, the categorized questions didn’t help in any way.

The second big difference that stands out is that on the results pages the first results are Brainboost’s natural language results; OK same as before. But beneath those are regular search results from a selection of other engines. Brainboost gets its results to natural language queries by reformulating queries and sending them against other engines; smart meta-search. It is not exactly clear to me the relation between the two sets of search results, and to be honest I probably will not use the regular search results section much.

A less noticeable thing is that Brainboost now has tips. I did a search and then got this: Brainboost is not a chatbot. It was designed to answer questions which are factual in nature. In case you are wondering what query triggered the tip, it was how do you jumpstart a motorcycle?, since my motorcycle isn’t starting again after being out in the rain and I can’t remember the positive/negative hook ups. The one result for this search is not relevant at all.

For other queries, like what is the population of Scotland? the results were just as good as when I reviewed Brainboost. I will keep playing around with the new Brainboost to see if I can find other differences. If you haven’t used this engine, I recommend giving it a shot. It can definitely come in handy when you are looking for the answer to a question.

Soople

Soople is a simple, but powerful idea. It is a search interface that overlays Google so that searchers don’t have to know Google’s syntax in order to be able to take advantage of of their advanced search features.

One of the things that frustrates me is that general web search engines, with few exceptions such as MSN’s search builder, have not done much of anything beyond tabs to help users formulate queries, and MSN’s search builder isn’t great at this point. Tabs work to target queries to source types rather than helping to formulate searches. Soople changes all that by laying a very easy to use interface over Google’s bare search box.

On Soople, rather than typing in syntax like define: ontology, there is a search box called Definitions, so that all you have to do is enter the word ontology into that box and it formulates the query and sends it against Google. Now, you may be thinking that define: is not too difficult a thing to remember, but there are about fifteen other things that Soople helps with such as searching within a site, filtering by file type, mathematical equations, related sites, conversions, etc. To see all they offer, check out their overview of all functions.

Something I really like is that Soople has pop-up JavaScript windows that succinctly explain each type of search. That way there’s no confusion for me as a user about what’s happening. They also offer a personalized My Soople interface and a free account takes about 3 seconds to set up. Then you can personalize the My Soople section. I personalized it by including every type of search they offer. Is that really personalization? If a tree falls….oh, never mind, it works for me.

Soople also offers some proprietary search help not available on Google. Such as searching for a topic that provides advanced features for focusing searches by subjects, such as books, fashion, sports, etc.

The search set functionality lets each user target searches to a selection of sites. In other words, you can create a search set for something like sports by adding sites like ESPN, MLB, Yahoo Sports, etc. Then save that set of sites and run searches against just that group. As Soople’s instructions aptly put it, “This way you can create your own miniwebs to search in.”

Recently I sent Soople over to a friend of mine whose opinion about search I respect more than anyone else I know. His response was: I gave up years ago and don’t even bother with the advanced operators. I use the single box and simply alter my query words. Works well enough most of the time and I think the masses fall into this camp. I’ll call it Camp Lazy Users. So, if I’ve given up on advanced operators then I really have no need for a Soople type page — it tries to give me stuff I do not use to begin with. Not saying it doesn’t have value, just doesn’t work for me.

My response to him was as follows: In my opinion, presenting a single search box and a few tabs to users and expecting that to satisfy all queries is a lot to ask. It’s difficult enough for me to keep up with the syntax each engine uses, let alone for most users, so I think an interface like this is great.

I believe that “helping” users with queries is one of the things we’ll see web search engines working on in the next year. Right now everyone is convinced that the simple Google type of UI is the be all, end all. But I disagree. Remember Dialog? Searching on Dialog offered some of the most powerful searching I’ve run across and subsequently its syntax was the most difficult to learn. Because of that I hated using Dialog and I never go back to use them because I’ve forgotten how to. And that’s a shame. But we’re going to see the same thing happening to web search engines if they’re not careful. They’re going to throw in all these new search options that hardly anyone will use because they’re so stuck on the clean UI. But it’s not too late. They can still build interfaces, or improve upon the advanced search pages, to help guide us through the search process.

Soople may not change the world of search, but I for one hope it does. For web search engines, who purport to be the interface to the world’s information, why is it that they leave us hanging with a naked search box augmented with tabs? That’s not enough. Although there are advanced search pages to use, and they’re a step in the right direction, they seem to be focused on parameters like language, adult content filtering, and Boolean terminology. All of that is important, no doubt. But where is the easy to use syntactical interfaces (is that the right way to say that?) for things like unit conversions and equations and flight information and time zones and so forth and so on? I hope the Soople idea catches on. Helping users create better queries will result in better search relevancy.

When Giant Directories Roamed the Earth

Written by guest contributor Dave Jansik

Back in the day, directories were the definitive sources for finding quality content on the Web. Following the success of Yahoo, several companies and volunteer-driven efforts sprung up to get a piece of the action. The concept: we do all the searching ahead of time for our users, we establish proprietary traffic, and then the inevitable follow up: we charge businesses for inclusion in the directory. Brilliant. Simple.

Over the last six years, I have been involved in building one of the largest editorially and volunteer driven directory of Web sites. LookSmart’s founders recognized the need to “sort through the crap” that search engines were spewing out in the late 90s. At the time, their intentions were pure and good. The bubble was growing when Microsoft contracted LookSmart to create comprehensive directories of all the best sites out there. With all that money flowing in, we could afford to hire over 100 editors and ontologists worldwide. With the acquisition of Zeal.com we enabled a Web-based platform to harness the skills of hundreds of volunteers to add non-commercial sites and manage categories. Some were in it to get traffic for their own sites while others truly wanted to help build a great search experience. They searched for sites that were determined to be the authority on a topic. Descriptions and titles were hand-written one at a time. The platform enabled users to add numerous types of metadata, link sites and categories, perform mass edit functions, and easily find transgressions such as empty categories. Soon after, we began a paid inclusion service that played off the huge traffic numbers LookSmart received through its distribution on MSN. Thousands of companies were given the right to have as many product pages as they wanted placed in the directory. Suffice it to say that from a quality standpoint, directory search results were heavy on commercial and light on the quality we first intended. I was on the Ontology team whose task was to create and manage an ontology that accommodated what ended up being over 2 million commercial and non-commercial sites neatly organized into a highly intuitive 250,000 category ontology.

When Google entered the fray, directories came to be considered limited in scope. Many specific queries – the quick answers – couldn’t be answered by a directory. Google got you what you wanted in that crucial first five site returns. LookSmart responded by acquiring WiseNut and Grub.org to position it in the search arena. MSN recognized the need to compete with Google, dropped our contract in favor of building their own search engine from scratch, and we were sent scrambling to find a way to replace seventy percent of our revenue because MSN accounted for the bulk of our user traffic. Without Microsoft, resources to maintain our search products were pared. Editorial took a big hit, and with such a large directory on our hands the remaining staff and dwindling number of Zeal volunteers couldn’t keep up with the exponential growth of the Web.

It’s 2005 and most of the editors are gone. I have spent much time clearing out most of the commercial sites that once challenged our credibility. A handful of passionate and dedicated volunteers continue to shape and develop the directory – some oversee its vast landscape while others continue to focus on the topics they hold dear. It is their hobby and their community of friends. It’s their baby. Still, fly-by-night contributors and savvy SEOs endure the Zeal quiz solely to submit their sites knowing that being in LookSmart’s directory will boost Google PageRank.

So, what is the future of the directory? The web is a vast, vast place. Will search engines be able to deliver specific content out of the billions of pages they are able to index? Those of us on the directory bandwagon feel that contextual search and browse tools will always be an essential tool for researching the Web. Consider how useful it is for a searcher looking for everything on the paper and packaging industry to have neatly organized categories of guides, associations, major companies, professional journals, and equipment suppliers already found for him. Kids can search for information on mummies and then climb back up or across the directory to find facts on King Tut, the pyramids, and Cleopatra. People who aren’t sure where to go on vacation can look up official travel guides to hundreds of cities worldwide and see what kind of hotels, entertainment, activities, and restaurants are there before booking.

Here are a few other uses and features that can keep directories relevant, interesting, and even profitable:
Search engine seeding. From a technical standpoint, it is widely acknowledged that search engines can use directories “seeded” with relevant URLs to improve relevancy.
Marketing to a specific segment of Web users by creating verticals. The best of the Web has already been categorized – it’s a great place to start zeroing in on an audience and advertisers. Freely distribute the category structure to other verticals to build traffic.
Making them more consumer-friendly for users and contributors. Add simple site rating and ranking tools that bring the best sites to the top of category site lists, incorporate message boards to share ideas and suggest other sites to fellow users, make it easy to submit a new site, offer basic search engine features such as preview panes and highlighted keywords.
Address staleness issue. By including topical RSS feeds, news feeds, or links to community bookmarking services such as Furl. Once a directory is built it is fairly easy for a small staff to maintain and add new content.

It will be interesting for me to see if directories last as a useful research tool. Yahoo’s directory, once their crown jewel, now occupies a smidge of real estate towards the bottom of their homepage. There are other players popping up solely for the submission money, though just about all of them are so thin on content that one visit is plenty. Bookmarking services like del.icio.us and Furl are helping us dynamically learn about what people find interesting on the Web, yet there are issues with the free-wheeling and personalized metadata, or folksonomies, that users of these services employ. No one has invented the perfect, omniscient search tool yet. And ya know, I wouldn’t be surprised if the person who creates one gets their idea while in a directory.

***
Dave Jansik is a Senior Ontologist and Content Producer at LookSmart.com. He has worked on Web directories for 6 years.
[Editor’s note: I’d like to thank my friend Dave for taking the time to put this commentary together. He has been a quiet advocate for helping people find information on the Web for years and it’s great to have him sharing his thoughts with us.]