In January 2011, marketing and credit information provider Callcredit acquired a small but data-rich competitor called The Trading Floor.

The Trading Floor’s innovation had been to partner with insurance and financial services providers that collected reams of data about customers and prospects but, as competition in their markets intensified, were seeing decreasing returns on that data. 

"Our model was to look for alternative ways for those companies to monetise their data assets," explains Zoe Palethorpe, who joined Callcredit as data assets director as part of the acquisition. "We don’t buy any data from our partners, we take their data and offer it to our customers, and for every pound that we make, 50p goes to the original company."

In October last year, it was decided that Callcredit and the Trading Floor’s respective marketing databases would be combined. The resulting system, launched last month, is called Define and is, the company says, the "largest, most accurate and most verified consumer database of its kind". 

The database contains 43 million postal contacts, 23 million email address, 20 million landline numbers and 15 million mobile numbers. For individuals listed in the database, there can be up to 300 variables, ranging from their preferred method of marketing contact to the make of their mobile phone. 

The backbone of the data is the Edited Electoral Register, the subset of the UK’s electorate that excludes people who have asked to be left off the version that is accessible to marketers.

Supplementary data is collected from around 80 different partners, including publishers and retailers. When a customer fills out an application form for a loan or subscription, for example, and ticks the box saying "Do you want to receive product, offers and services that are relevant to you from a third party?", this may well be where their data ends up.

Combining the systems

Both companies used Microsoft SQL Server to support their databases, so the new system was built on that platform too. Combining the data sources has lead to improvements in the accuracy of the combined dataset, says Palethorpe. "If one database says an individual is unemployed, but the other one says they drive a Porsche, we’re able to see that discrepancy and make a decision on which information we should present."

Because Callcredit offers marketing distribution services to some clients, it has been able to build a degree of intelligence into the combined system that determines the most appropriate means of contact for an individual. "We might have seven or eight different email addresses for the same person" explains Palethorpe, "so we have set business rules that define which email is the most appropriate to use. If an email goes out and we recieve a bounce-back message from that address, the system will reroute it and present another email address."

With such a large dataset to analyse, Callcredit can make informed guesses about missing pieces of information. "For example, we have a model for household composition that allows us to put a score against each individual and say, this person is likely to have three children based on the information that we have."

The whole system was built by a team of just four people, over six months. “It was a massive undertaking for just four people, but having a small team meant everyone knew what they were supposed to be doing,” says Palethorpe.

Despite covering nearly 70% of the UK population in some depth, at 1TB the database itself is relatively slight. It is currently hosted on a single server in Callcredit’s Leeds data centre, although the company is looking to move it onto a new system with room to grow to 3TB, as it is plans to integrate a huge amount of data about online marketing preferences in future.

“We have a massive expansion plan that will merge online and offline data, including social media profile information,” says Palethorpe. “When you start to think about that kind of information, there’s always going to be more to do.”