The big lie in search

The biggest lie in search is also its most touted benefit -- that it is intuitive and easy to use.

Perhaps, as much as a blank piece of paper is intuitive, so too can be a single, empty text box partnered with a single submit button.

But the search results page? Easy and intuitive, it never has been, and it is getting worse and worse as search companies keep adding more options, suggestions, clusters, facets and widgets to try and help you make sense of the endless pages of results. Even the Google with its once pristine layout recently felt compelled to add more clutter to the interface, an explicit acknowledgement that the results they serve up simply are not good enough.

So how could the search experience be improved?

One way is to make the results better by asking just a little more of the searcher up front. If you are able to ask the right questions, you can infer much. For example, if, instead of a single submit button, suppose there were three; "animal," "vegetable," and "mineral." By forcing the user to select, huge amounts of ambiguous results could be eliminated. Of course, this approach impacts the simple, intuitive search box and button combo myth, but if the experience and results are better, does it really matter?

Another approach is to build applications and interfaces that integrate rules to constrain search queries to a specific business purpose or process. To quote from a patent currently pending (um, mine):

"This invention relates generally to software-based search technology, and particularly to rules-based (rules being any standardized set of principles or methods for organizing and structuring information (e.g. The AP Style Manual; IDEF, a business process language; the Marquess of Queensberry rules for boxing) searching via a constrained or guided query structure."

 You can read the full patent here should you be interested.

Bottom line -- attributing the property of "intuitive" to a text box forces a free-for-all environment that must cast too far and too wide, and demands far too much processing on the backend in the attempt to guess at user intent.  Constraining the queries, either by empowering the user or by integrating them into the user experience, can more easily deliver better, more relevant information.

Fun and games with disambiguation

One of the coolest things I have learned in the last few years is the notion of disambiguation, a linguist's term for determining the right meaning of a word within a specific context. My current favorite example to illustrate this is:

This is important, as not knowing which meaning of “manhattan" is being referenced, whether in a search query or in piece of content, can really impact how well algorithms can leverage words. While my interests are currently focused on automated query building, certainly there are many, many interesting things that can be done when proper word definitions are known. Here's an example from our experimental search engine, howdoya.com on how to mix a manhattan. Note all the useful words on the left you can use to refine your query -- disambiguation! Now, if you want to see something really interesting, click on the x-ray glasses next to each search result -- again, disambiguation! A world of possibilities.

Would you pay for great search results?

In our experiments at Emantix with automating search queries, we discovered something really curious -- the longer and richer the query, more often than not the top result was very, very good. But the following results were not -- at all.  In fact, in our experimental search engine, howdoya.com, we had to tune down the richness of the query so that, at a glance, the entire results page looked good, giving the user an overall perception of quality.  And of course, when we tested with organically occurring ads, the same was true. Really great queries -- fewer, if any, ads, mediocre queries -- more ads.

So the question is, do more relevant search results diminish the quantity and quality of ads displayed? And if so, what are the implications of and opportunities in removing the obligations inherent in a paid advertising revenue model?

Why not an ad-free, fee-based search engine?  A low annual fee that is painless to an individual, but collectively, on a web-scale, could be a highly profitable enterprise?

Would you pay a nominal fee for better results and less visual clutter?

I would like to propose a word, a verb, to describe this opportunity  -- to "craigslist" -- or to disrupt an entire industry by eliminating a critical source of revenue.