Monthly Archives: March 2006

You are browsing the site archives by month.

Keotag

Keotag is a tag search engine, but it only searches one engine at a time. You enter a query and then see the results that that were tagged for the term on Technorati, Furl, Yahoo, and more than a dozen other sites, but not all at once.

The trick with meta-tag searching is to integrate and rank the results rather than segregate them by source. Even a tabbed approach by content type would be OK. As in returning tagged results for:
web | video | blogs | images, etc.

Providing a UI like Keotag is kind of helpful, but it’s not as helpful as I’d like. To build a real meta-tag search engine requires some tricky indexing and a nifty ranking algorithm. As well as related search prompts, stemming, wildcarding, etc. You know, a meta-search engine.

Guest Obituary for Zeal, by Alice Swanberg

I’m excited to have a guest writer for the Lounge, Alice Swanberg. In this article she writes about her experiences with LookSmart and Zeal. Alice and I worked closely together for 5 years and I have tremendous respect for her work.

The Ancient Past

I started working at LookSmart in 1999, as a junior contract ontologist. I basically created categories in the directory and helped out with directory growth planning and maintenance. It was a lot of fun, like many early dotcom jobs, and I soon got hired on permanently and promoted above my abilities like everyone in the Internet industry at the time. After almost 18 months, a long 18 months, the senior management announced the acquisition of a new company called Zeal. At this point we’d already tried and had limited commercial success with another community-style project, LookSmart Live. The LookSmart management was trying to figure out a way to scale the directory to make it competitive and as relevant as a web search engine. No matter how many paid editors you throw at the Web, you’re never going to be able to index it all. Therefore, why not see if other people are willing to help?

I’ve been in online communities of one kind or another since 1992. That probably dates me quite a bit. Even with all my old school BBS-roomaide and other community experience, I wasn’t really prepared when I was told that MY directory at LookSmart was going to be opened up to a bunch of average joes. I was pretty tweaked out. I didn’t want just anyone messing around with the categories I had painstakingly built. It sounded crazy. No way would random internet users be able to grasp the style guidelines that took LookSmart editors two weeks of intensive grammar and style bootcamp to master.

Every site description in the directory at that time was copy edited within an inch of its life. You could get fired for bad grammar and spelling. The pressure to produce more and more site reviews was intense for the editorial staff. Because the team was split into subject specialties, people became pretty territorial over their areas of the directory. Around this time I was promoted to Ontology Manager, and as far as I was concerned, I owned the top node of the United States directory and everything underneath it. Talk about territorial.

It was pretty shocking to suggest to such a quality-focused staff, who had been trying to carefully grow a balanced directory of sites, that we should allow just anyone into the system to edit. We were, however, given a lot of say on how the systems were integrated. The prep time for merging Zeal with LookSmart was intense. We had to move from our directory management tool, which I loved even though I griped, to this web-based platform whose strengths were not immediately apparent to us. For one thing, the old tool was drag and drop. It gave us the whole picture of the directory hierarchy. We could link categories to other locations with a flick of the wrist, and create hundreds in just an hour. As a manager, I could just see our productivity stats circling the drain. I was certain this was going to be a nightmare.

The Salad Days

When D-Day came and all the Zealots were allowed back in, it was chaotic for a couple of weeks. The editors on staff had been trained and relatively well-prepared, but we didn’t really internalize the fact that these were volunteers. We tried to control the volunteers and we marked them harshly and inconsistently. Pretty quickly we caught on that most people knew what to do already and that the only thing we needed to do was answer questions and help people regain their points from the old system, which they did incrementally. The members were annoyed to have to start over again, and were loud about it. A lot of the editors had never been exposed to Internet hyperbole, where a user feels the only way to get attention is to wildly exaggerate his or her feelings or position. While I giggled at being called a “soulless automaton” by a frustrated community member, some really took the community’s harsh criticism of them to heart. It took a while to grow a thick skin and start to read the messages from the community for their content and not for their emotion. I was proud that as one of the community moderators I never deleted a message that was critical of the company or staff. I think I probably deleted only a few messages a week, and 99% of those were spam.

Once Zeal started running smoothly, I loved it. Watching the community members build their own system and monitor each other was great. Thinking up new and clever ways to track down the porn spammers and non-subtle SEOs was like a game to me. I really felt like I “knew” some of the zealots well.

To my surprise, the directory’s quality didn’t suffer. We’d streamlined our style guide quite a bit (although I think it was still pretty hard for new members to get used to), which helped everyone. Having a lot of people from different walks of life participating in the project meant that the directory focus shifted from “A general directory which focuses on things young, single people in San Francisco are interested in” to “Things a lot of people who use the Internet are interested in.” We got resources for the elderly, home schooling from an inside perspective, gardening for people who actually have the room for it, sites about goats, and teenage diaries. I soon became a firm (and only occasionally rabid) advocate for the community within the company. I’d always take a few hours at the end of the day to make sure all the community activity and requests were being handled “properly.”

Along with these great sites came plenty of commercial submissions of one type or another. At the time LookSmart was growing its paid inclusion program where commercial sites paid for inclusion in the directory, and so the editorial team was stuck as the unlikely guardian of LookSmart’s main revenue stream. Because of this delicate balance I felt like a post-modern critic a lot of the time – I didn’t want to judge intent, they had to fall inside the lines of acceptable content that we ourselves had drawn. We had long battles with some submitters which helped us refine our guidelines.

Monday Morning Quarterback

LookSmart itself was still trying to find its way after six years – having lots and lots of sites in a directory still didn’t make them a search engine, and eventually MSN decided to move on from using the LookSmart directory as their means of making money from search through paid inclusion.

Here was our chance to try for real search relevance, and we didn’t take it. Chris writes in this very blog about non-search search companies. This was the real turning point, where we didn’t have the significant advantages and disadvantages of the 800-pound gorilla that was Microsoft. We didn’t have to answer to their needs first any more, and could finally become a real search company.

I think that LookSmart should have thrown everything at Wisenut (which was and maybe still is a nice search engine) and possibly the directory. I would have liked to have stripped all of the granular subsites and categories out of the directory. A directory should never try to be a search engine, and that was where we screwed up. It shouldn’t contain all the web, only the best of it. In the early days of Web directories, it was possible to index a lot of the content out there, but as it grew, that became a pipe dream. It didn’t really sink into the skulls of Yahoo!, ODP, and LookSmart for quite a while, as we were having “who has more sites?” cage matches almost up until the point I left the company in June of 2004.

Zeal had a site-rating tool when it was acquired, and this tool was never really integrated. If we’d turned it on, maybe we could have started sifting the “good” sites to the top of categories. With the innovative community piece, there might have been a way to capture people’s thirst for “good” information, the sort of thirst that Wikipedia is answering. If you want to know a single fact, you go to Google or Yahoo! Search. If you want the big picture with reasonably authoritative sites, go to a directory. The ODP, Zeal, and best of all (in my opinion) the Librarians’ Internet Index can give you this.

Do I really think LookSmart should continue to support Zeal? Not if it doesn’t fit their strategy, no. It’s not a horrible thing to drop an underperforming product which you can no longer maintain or don’t wish to maintain. I’m quite surprised that they are essentially taking their toy and going home, though. I’m sure they could continue to make a few bucks off of the (PageRank 8 ) domain and content. Heck, I’d try to buy it if I was in any position to do so. I do think LookSmart has never had a great feel for the Web community at large and how to build up trust. They dabble in purchasing or creating popular community sites, and then shut them down, suggesting to the community that it move onto their next venture. Compared to companies like eBay, who still keeps half.com open based on their community’s wishes in spite of their own often-expressed desire to totally integrate it with eBay Stores, LookSmart looks pretty callous. On the other hand, it’s clear this has been in the offing for months. I noticed when they took the directory off of looksmart.com several months ago – at that point, why not warn your loyal community?

LookSmart’s going to go the way it goes, and it seems comfortable as a third-tier search company – I hope they stay happy there and find a way to make pots of money. I still know former co-workers who bought stock in the IPO and would like to cash it in someday. . . and findarticles.com and furl do continue to rule.

Zeal.com, R.I.P.

Something I knew about for a while has finally hit the airwaves, Zeal.com is closing. I was part of LookSmart’s Editorial Management team in 2000 when Zeal was acquired. We thought it was a great idea because it allowed the directory to scale. Whereas ODP was a bit of a free for all, the differentiation of Zeal was the editorial oversight. We thought the two ideas, openness and oversight, blended well together. But as time wore on and the Zeal executives who came with the acquisition moved on to bigger roles, LS lost interest in the directory and peeled away resources to work on other projects. Zeal was left to die on its own.

When I first heard this was going to happen I was a bit bummed out. A lot of people, community members and editors, had put a lot of work into Zeal. The LS directory in general got a bad rap because it was part of LS and because people who write about search love to tell everyone how directories aren’t used by anyone. But, IMO, that’s because it wasn’t properly used. It’s a perfect example of a non-search search company not supporting a product that could have had niche value. It could have been truly integrated into blended search (mixing directory and algorithmic results) or in today’s environment it could have been an interesting foundation for a tagging system or social networking site. It already had morphed, in essence, into a folksonomy in that it was built by many people. But instead of trying to make it into a valuable resource LS pursued other goals and left behind what had been the foundation of the company.

To read more:

On the Search Lounge Guest Obituary for Zeal, by Alice Swanberg

Ontologista They are shutting down Zeal

Search Engine Journal Looksmart Closes Zeal, Concentrates on Furl.net

Search Engine Watch LookSmart’s Zeal Directory To Close

The Information Literacy Land of Confusion The End of the Zeal Directory

Non-search Search Companies

The media and the public think search engine relevancy started with Google. It didn’t. What Google did, compared to other search companies, was that they built a whole business around being good at Search, and for the first several years they didn’t get distracted. All the other early search companies, such as Excite, Lycos, Snap, LookSmart, Hotbot, Go, Infoseek, AltaVista, Inktomi, Ask Jeeves, Yahoo, MSN, etc., lived a dual life. They said they were search companies, but in reality most of these companies were ruled by business development and marketing. They were non-search search companies.

At each of those companies there were hidden people – engineers, product managers, and editors – who were trying to build relevant search. But the business side of these companies had the ears of the executives, and they were whispering that the product only needed to be decent to bring users. After all, in the mid and late 90s users were happy enough just to know one place to go when they got online, and if they knew how to type in *insert company name from above*.com that was fine for the business arms.

Some of the big players, like Yahoo and MSN, who did believe in providing a good user experience, outsourced much of their search technology. And the good engines, AV and Inktomi, didn’t know what to do with a good thing. Had they focused on public-facing algorithmic search, instead of portalization and enterprise search without a consumer facing interface, either of them could be king.

Google came along and said look, all you guys are doing all these other things, but we’re going to do Search. In the beginning they didn’t know how to make money out of it, so they outsourced it. They also had google.com, so while their competitors distributed their search, users realized they could go right to the source. In those early days was Google more relevant than AV, Inktomi, and IBM’s Clever? There’s no reason any of those other products couldn’t have kept up in the race if they’d had the same business-side support from their companies.

Those of us who were the hidden people in non-search search companies, we talked about how search was neglected. But for the execs, search was not something to make money off of (in business speak: monetization), search was something you had to have or else people wouldn’t come to your portal and look at your banner ads (business speak: impressions) or read your articles (business speak: content). And so while all the non-search search companies slept, Google built Search.

What’s the point of all this? The point is that the people who advocated for search and relevancy existed in the companies mentioned above, but they were under-nourished in their corporate diaspora. Now search makes busines sense, and the non-search search companies (those still in business along with a host of new ones), are all working on Search. Here at Yahoo, many of the hidden people have gathered from their respective non-search search companies and the result is a group whose sole and only mission is improving Search.

Tagging and Meta-Search

Tagging systems are segmented by media. You can tag photos on Flickr, web pages on Furl, del.icio.us, Yahoo!’s My Web., videos on YouTube, your book collection on LibraryThing, and so forth and so on. But why should my video tags be different from my photo tags? There might be one-off situations, but generally my tags should follow me rather than me having to follow my tags around from search box to search box.

I want one search box or tag cloud to give me access to all of it. At times I may want to limit my search or browse by media (as in tabbed searching for local vs. news vs. images, etc), but often I’ll want all relevant results regardless of media type.

Tag Central does this, but it lists results by source rather than integrating them together with a ranking algorithm. But it’s fun to play around with.

Has anyone built a meta-search engine or a widget of some sort that actually integrates and ranks the results across tagging systems?

Post Navigation