For the past few years, some pioneers from the search industry have been saying that their tools could form the basis of a powerful, fast and user friendly form of business intelligence (BI).
On the whole, their claims have met with scepticism. BI is based on structured relational or multi-dimensional databases. It is not always fast, nor easy, but it is precise and reliable. Search, on the other hand, uses flat indexes and pulls out strings of text, which may be ambiguous. Search is fast but as most users of Google will testify, it is rarely precise. The worlds seem far apart.
The FASTForward 2007 conference in San Diego in February may go down as the point when this perception changed. Not only did the search engine company FAST showcase its new Adaptive Information Warehouse, which overturns assumptions about how search relates to BI, but a string of powerful voices joined the debate.
One of these is Bill Inmon, a man steeped in the structured database world and widely credited with developing the data warehouse. “The people in the world of structured technology are in for a surprise,” he said. “There’s a whole new vista out there that they never considered. BI built on search will be a substantial part of our futures.”
Matt Brown, a collaboration and search analyst with Forrester Research, even suggested that, “in 10 years time, when companies consider dumping their data into a database, they’ll think twice.”
And that thinking is already evident. Zach Friedland, ING’s vice president for enterprise data solutions, told the conference: “We have no data warehouse. We built a warehouse off the search engine, the reverse of how it is usually done.”
A marriage of opposites
The marriage of search and BI technologies is not unique to FAST. Most of the major BI vendors are, through partnerships or acquisition, attempting to bring the two together by placing a search engine on top of the structured databases.
But FAST’s Adaptive Information Warehouse takes a more radical approach, putting the BI tools alongside the unstructured data source. The product essentially integrates three underlying products: the FAST ESP search engine; a structured relational analytical processing database; and an extraction, transformation and loading tool.
This allows users to search both structured and unstructured data. “SQL is completely limiting. It doesn’t give you the intelligence you need”, says John Marcus Lervik, the CEO of FAST. He says that FAST wants to open up the data so people can ask questions not predetermined by the data schema.
The central issue, however, is whether it can pull out precise, non-ambiguous information and present it in useful ways. Davor Sutija, vice president for strategic market development for FAST explains that with fuzzy matching, entity extraction and linguistic analysis, FAST can extract information from within documents and then present it alongside the results from the structured database. Effectively, FAST is moving the search down from the document level to the information entities below.
Inmon is impressed by the possibilities: “The ability to access and analyse unstructured data is very exciting. But even more exciting is the ability to take the unstructured data and merge it with the structured data and get a blended analytical result.”
Further reading in Information Age
