NHS Scotland’s education branch decided early this year to implement enterprise search technology to improve and consolidate access to its extensive archive of medical resources.

It was a challenge, says Dr Ann Wales, NHS Scotland’s program director for knowledge management, “with no precedent in either the NHS or the public sector.” The NHS team decided to use Microsoft’s FAST technology as the search layer for a dataset taking in 50 databases, 500 e-books and 5,000 journals.

“The most difficult challenge was indexing the content, either by crawling, using existing or imported connectors, or in some cases building them ourselves,” Wales explains. “The project involved a lot of specialised technical work, especially in creating the connectors.”

The process was further complicated by a ‘significant’ amount of negotiation with commercial providers of several medical databases in order to store their metadata in the NHS’s single searchable index.

The final content came in at 11 million records, accessible across seven pilot portals with differing rules and emphasis depending on the user’s professional focus: social services, healthcare management, midwives, rehabilitation and other areas.

Usability was a key challenge from the outset, says Wales, with, inevitably, “users asking for ‘something a bit more like Google’. I tried to explain that this was a very difficult thing to do”.

The short-term costs of the FAST solution proved higher than those for the existing real-time federated search, in part because of all the work on the bespoke connectors, but ongoing costs also proved daunting. Storing the 11 million records, scalable to 18 million, involves supporting and maintaining 22 servers – a technical infrastructure Wales says “is not for the faint-hearted.”

Further reading