MSN

MSN (beta-search)
Type of Engine: Full-scale Internet search engine.
Overall: Average.
If this engine were a drink it would be…a Cosmo, but hold the Triple Sec. It’s got most of the basic ingredients, but something’s missing. And without that sweet ingredient, in this case relevance, it just doesn’t taste right.

Intro
I’ve shied away from reviewing the big kids because I didn't feel I could add anything to the abundant amount of literature already out there, but I couldn’t ignore MSN’s beta search. Its release is simply too important to the industry. It floors me that anyone expects MSN’s newly built search to go live in Beta and be better, or even as good, as Google. But the mainstream media, as well as industry watchers, all write about how it compares to Google. Fine, that’s a legitimate question to ask, but I’d like to step back from that comparison and simply judge MSN on its own. It could be argued that MSN might have done better waiting until it has an excellent product, but that isn’t Microsoft’s way of doing things. Building a large-scale search engine with crawling, indexing, and an algorithm is a huge –large, big, mammoth- task, but I guess since they’re Microsoft, it’s expected that they can do it. I’m going to write this review as if MSN search is a new company and not the search arm of the largest software company in the world. Even though MSN has offered search for years, it’s always been farmed out to other companies, until now.

UI & Features
I really like the Search Builder because it’s surfaced right there on the search page. It’s right under the main search box. The advanced fields they currently have are useful, but I bet MSN will be really juicing these up in the future. Be sure to play around with the sliders, one of the industry's latest fads, under Results Ranking. Yeah yeah, I know, only 2 or 3% of users use advanced logic. But maybe that’s because engines don’t let you build the logic the way MSN does now. In my opinion this is the single best front-end feature of MSN’s search, and I hope it jumpstarts the industry to be more creative in helping regular users build complex queries through natural language prompts.

There’s a settings page that lets you do the basic stuff like porn filtering and language preferences. There’s also a place to set default location for local search.

Following in the wake of Google and Yahoo, MSN search now has a blog. They also have a good About Us section, especially the Web Search Help page. I like this kind of transparency, it’s very helpful.

Query Examples
I was at Internet Librarian for a few days, and when I got home my motorcycle wouldn’t start. So what better query to test than instructions for jumpstarting a motorcycle? The very first result was a PDF file. Unfortunately I didn’t bother to look closely at the file extension so when I clicked the link it opened a 76 page PDF file. Not cool. I take full responsibility for clicking on this link, but how about helping me out by putting a PDF icon near the display title instead of in faded letters after the URL. I did a Control F to find motorcycle in the PDF document and the only reference was to inflating a motorcycle’s tires. Not helpful at all. The next result was a blog by a guy from Texas. There are some motorcycle references and the word jumpstart appears, but the ideas are not connected at all. Not helpful. Result # 3 was equally bad, another PDF file, this one having to do with campus security at San Jacinto college. I clicked on every one of the ten links on the first page of results and they were all bad. Not a single one, NOT ONE, was relevant to my query. So what the heck is going on here? I’d say there’s something funky with their proximity matching. Several of the sites were quite large and had a lot of text on them, so it seems like MSN is indexing all the words on the page but paying no attention to how close the terms are to each other. I don’t know if that’s a problem with their index, or if their algorithm isn’t taking proximity into account; if I had to guess, I’d say the latter.

OK, so maybe my query needs to better; computers don’t make mistakes, humans do. I tried jumpstarting motorcycle battery. The first difference I noticed is that this query produced four sponsored results whereas my previous one had none. Do I need to say that all of the sponsored sites are irrelevant? Can you guess what they’re about? They want to sell me a new motorcycle battery. The second thing I noticed is that my Texas blogger is back and so is my 76 page PDF file. But wait, there’s good news. The first result took me to a page with a link to a somewhat relevant subpage. To be specific it’s a message board from the Philippines with various people discussing the topic. I wish MSN had taken me right to the page, but at least it got me close. The second result is pretty good, it’s about batteries for one particular motorcycle model. It doesn’t give actual instructions, but it’s close. The rest of the results are not so great, but they’re better than my previous query.

Time for another query refinement, this time including the name of my bike: Suzuki gs500e jumpstart. Not many results for this one, only one sponsored result from eBay and three not relevant web results. Same problem again, the terms appear on the sites, but nowhere near each other.

A new approach was called for: Search Builder. Going back to the search homepage I entered the following:
Search Terms: jumpstarting a motorcycle (I required the exact phrase).
Site/Domain: *blank*
Links to: *blank*
Country/Region: *blank*
Language: English
Results Ranking: I slid the first bar far towards Exact Match. The second bar, for popularity, I left in the middle. And updated recently I also left in the middle.
…NO RESULTS…

At this point I’m a sad customer and I leave. Or instead of doing it myself I just give up and call a tow truck to jumpstart my bike for me.

I also wanted to do a local search: florist (with location set to Loudonville, NY). Before I get to relevance, one bad thing is that I can’t change my location on the fly. I have to click into Settings and do it there. So even though I’m in California, I wanted to buy some flowers for my sister-in-law so I had to change my default location for her region. Ignoring the sponsored sites, the first few results are good enough that I’m happy. Some of the following results are slightly off, like a weather page that also has a link to shopping categories like florists. The only complaint I have is that from the display data it doesn’t look like any of these florists are actually in Loudonville, but they are in the Albany area so I assume they deliver to Loudonville.

They’re also providing some direct answers through Encarta and other sources. You can ask questions like who is the prime minister of the united kingdom? And MSN will show you this:
Answer: United Kingdom: Prime Minister: Tony Blair
You can also do mathematical calculations (e.g. 4 *4), dictionary definitions (e.g. define laconic) and measurement conversions (e.g. how many centimeters in a mile). I tried other questions like who was jefferson’s vice-president, but it didn't work.

MSN has tabs for News and Image searches. I won’t go into a full review, but I played around with both of these and they were both good. The news was recent and included authoritative sources, and the photo results were relevant.

Conclusion:
Within the context of being a brand-new search tool, it’s not bad. There’s definitely a lot of documents in their index but their relevancy needs a lot of fine tuning. Right now it’s not entering my regular repertoire, but when it takes the next step past this Beta version I will reevaluate it then. From the outside looking in, it seems like they’ve built up their crawling technology, but I’m not convinced they’ve done enough with their relevancy algorithm yet. Nonetheless it's always exciting to have a new large scale engine appear with its own crawling and indexing technology, and this is no exception.