There is a paradox about search engine technology. Almost everyone uses Google or something similar to search the web, and most of those praise its power and reach. But ask a search engine specialist, and they will say that most Internet and corporate search facilities in the future will scarcely use Google-like interfaces at all.

Indeed, some argue that Google, whose easy interface and simple architecture has helped to fire-up investment in search, is already behind the times. “Web search is not the state of the art for what is possible today,” says Bjorn Olstad, chief technology officer for FAST, the enterprise search technology company. “It is fixed on the search pattern of seven to eight years ago. You get a search box, you key in your query, you get a list of answers. If you don’t like it, you redo the search yourself,” he says.

There are at least two problems with this approach. First, millions of relevant documents could be missed, because they are stuck down the ‘long tail’. That calls for a better way of reaching into the content of the documents, to extract meaning and content (see main article).

But second, the search makes no attempt to understand the context of the search. If the user doesn’t provide it, the search will be usually too imprecise to be immediately valuable. Further searches, or the use of advanced searches, will be required. Research has shown that many Internet users are too impatient or unskilled with these.

The way forward is contextual search, or implicit search. “In a business, I want the search tool to provide context,” says Ray Lane of the venture capital firm Kleiner Perkins Caufield & Byers, who says that his PC may know more about him than his own mother, and should therefore be able to deliver more relevant searches. “I want the search to find me. If I’m going to visit GE, it [should] tell me what I might need to know,” he says.

This is exactly the point made by John Battelle, the author of the highly regarded 2005 book, The Search. In his keynote presentation to February’s FASTForward conference in San Diego, Battelle illustrated the shortcomings of today’s search facilities. The search engine of one major US newspaper, for example, fails to pick up on the context of his searches and subsequent selections, in spite of the fact it has enough information – implicitly – to not only dynamically generate more accurate results, but to better understand him and his needs. “Consumers want a site that ‘gets’ them, that organises itself around their intent, both declared as well as implied,” said Battelle.

If the site can pick up the visitor’s intent, he says, the business can use search technology to really have a dialogue with the consumer.

At FASTforward, Olstad showed examples of how search can be used in this way: although there may well be a search button on the web page, most of the searches were buried within contextual boxes. If a user clicks on a certain area, or a certain document, it would use that behaviour to generate background searches and display further relevant information and perhaps product offers. Olstad even demonstrated an as-yet unreleased music player that was able to adapt to the tastes expressed by several separate listeners, by implicitly searching out and ordering relevant tunes in real time – without a search button in sight. “Implicit search has many forms and will appear in many applications”.

Further reading in Information Age

Search as a platform