Employees have long had the ability to process and analyse small data sets themselves, thanks mainly to Microsoft Excel.
But when it came to anything that exceeded Excel’s maximum row limit (65,000 in Excel 2007), they were dependent on the reports and dashboards that the IT department had built using the enterprise standard business intelligence tools.
Recent years, however, have seen not only a staggering uptick in the volume of data produced and collected by businesses, but also a steady increase in the awareness of the power of data analytics.
The combined effect is that users are increasingly dissatisfied with the prescriptive reports and dashboards that are handed down to them from IT. In particular, the time it takes for new reports to be issued cannot keep pace with employees’ need to answer questions as they arise.
It is little wonder, then, that there is high demand for ‘self-service BI’.
In a 2011 white paper, analysts Claudia Imhoff and Colin White defined self-service BI as “the facilities within the BI environment that enable BI users to become more self-reliant and less dependent on the IT organisation”.
And according to the Wisdom of Crowds report, an annual survey of business intelligence users by BI guru Howard Dresner, self-service BI has been among the top technology priorities for two years running.
At first glance, it seems like a straightforward idea. But on reflection it quickly emerges that there are many different ways to approach self-service BI – in particular, how much autonomy is extended to users.
When it comes to enterprise-wide self-service BI programmes, IT organisations are still finding their way. But the growth of providers selling easy-to-use and easy-to-deploy BI software reveals that users are helping themselves, with or without the IT department’s aid.
Here is yet another field of technology where the IT department must find a way to offer users the tools they want while simultaneously applying the data governance and security controls that the organisation needs.
Self-service BI in practice
One organisation working towards self-service business intelligence is sportswear-maker Nike. According to Jimmy Lee, an expert architect at the company, it is doing so in response to the constantly changing needs of business users.
“Every time the IT department builds a new report, our users say, 'We have more questions that we need answered,'” says Lee.
The problem is that the IT department’s workload is such that it can no longer keep up with those changing demands. “We've tried things like Agile but no matter what, if a user comes to us with a new requirement, we're at least going to take a day to provide it, and probably six weeks.”
Meanwhile, the complexity of the analysis that the business needs is growing too. “We're having to do more complicated work – more data types, deeper questions – in a shorter amount of time.”
Allowing business users to build their own reports seems like a happy compromise. In fact, Lee says, Nike's business users already have self-service BI in a sense – they have access to the data warehouse, and they are happy to slice and dice the data in Microsoft Excel if possible.
But this does not support the information and data governance processes that the IT department demands. “We want them to do it with more governance.”
There are a number of reasons why Lee believes that now is the time for Nike to implement self-service business intelligence.
Firstly, its core BI infrastructure is up to the job. “All the data from SAP is in our Teradata data warehouse and it's enhanced with other data feeds; we've got data quality tools in place; and the terms in our semantic layers are all aligned,” Lee explains. “We're in a pretty good place.”
Secondly, there is clear evidence of demand. “Business teams are starting to hire their own data scientists and statisticians, and are buying their own BI tools,” Lee says. “The demand signals are there.”
And thirdly, technology is now available that allows non-expert users to conduct complex analyses themselves, such as Tableau and QlikView.
“Now is the time to act,” says Lee. “If our technology organisation doesn't get in front of this, we'll have business groups doing it on their own and end up with lots of point solutions across the organisation.”
To gauge what kind of self-service BI functions it should be building, Nike's IT department has interviewed various stakeholders on how they use analytics today, and how they plan to in the future. This process has identified three end-user personas within the organisation.
There are what Lee calls ‘front-line users’. “They don't want to make their reports, they just want pre-built reports with the parameters that they need,” he says.
Then there are conventional analysts, who want to perform OLAP reports on defined data sets. And then there are the ‘data scientists’, who want access to as much data as possible to perform ‘exploratory’ analysis.
“We need to support all three groups at an appropriate level,” says Lee.
Another interesting finding from the interviews was that when asked, “Do we have enough BI tools?”, a number of business users replied, “We have too many!”
“That was surprising because we only have [IBM's BI platform] Cognos,” Lee says. “But to fill the gaps, they'd bought their own tools, and now they have too many.”
Software selection is just one of the challenges facing self-service BI initiatives. As Lee found, there is also the thorny issue of democratising access to information.
“We ran a proof of concept using Tableau, and the project got stalled for a week because the team couldn't get access to the data they needed,” Lee explains. It turned out that the project team was using the wrong process to apply for access to the data, and was being blocked on security grounds.
“I met with our security guys and said, if a financial analyst wants this data, shouldn't he be able to access it? They replied, that would be really complex for us to do. My response was, yes, it is complex, but we're currently pushing the complexity onto the users.”
The lesson? “We need global support for this strategy or it won't work,” Lee says.
As for paying for the self-service BI initiative, Lee's plan is to tie investments to existing initiatives in each of the departments. “If the supply chain team has some initiative under way to improve their performance, and I can tie the self-service BI strategy to that initiative, they will support it, and I won't need to work out some complicated return-on-investment case that probably won't deliver anyway.”
The aim is roll out self-service business intelligence in 2014, Lee says. The implementation will start by identifying potential early adopters. “We'll focus on the areas with more interest first,” he explains.
The first project will be a reference implantation – a “stake in the ground”, Lee says –that will be used as the template for all future projects.
Another company putting business intelligence in the hands of its employees is Barclays.
Lee Mooney is visual analytics manager at the bank's innovation and customer experience team. The unit is tasked with finding ways to improve customer engagement through technology, Mooney says, and it uses business intelligence to provide insight into what Barclays customers want.
In the past, there was not one easily accessible repository for customer data, however. “Barclays is really old and really big, and its data landscape is the same,” says Mooney.
The team therefore constructed its own BI infrastructure. It built a data warehouse using Teradata technology, and chose Tableau's visual BI tool as the user interface. “I think of Teradata as the engine and Tableau as the vehicle.”
The system has had some remarkable benefits, Mooney says. It has revealed some fundamental truths about Barclays' customers that were not previously known, for example.
“It showed us that 2 million of our customers only have a savings account,” says Mooney. “No-one knew that before.”
It has also dramatically accelerated the speed of analysis at the bank. “By using this capability, our team of three data scientists can outperform hundreds of staff,” Mooney claims. “One executive was pretty jumping for joy after seeing what we could do.”
Now, the Teradata/Tableau platform is available for anyone in the organisation to use. “If you want it, you can have it,” Mooney says.
That means that front-line customer service staff are now building dashboards to track complaints, for example.
“Even our internal auditors are using Tableau now,” Mooney explains. “We showed it to one auditor, who had no background in business intelligence, and he had built his own dashboard in ten minutes.”
Industry impact
Looking at the recent financial performances of some of the BI software vendors, it certainly seems as though a significant change is afoot.
Tableau Software, for example, saw sales double in 2012 to $128 million. The company raised in the region of $250 million in its recent initial public offering (IPO).
CEO Christian Chabot recently told Information Age that Tableau owes it success to the fact that people of any skill level can use the software. This breaks the hegemony of BI experts over analysis.
“I think we'll look back at the analytics industry like we currently look back at typing pools,” he said. “We live in that age now when it comes to data – anyone who has a question is reliant on a high priesthood of specialists in the back room.”
Another company riding high in the BI software market is Sweden’s QlikTech, which in April reported 22% revenue growth year-on-year for its first quarter of the financial year, up to $96.5 million.
QlikTech CEO Lars Bjork says that, like Tableau, usability by non-BI experts has been the key to its success.
“We are driving consumerisation of enterprise software in our space,” he says. “We always focused on the use. You want the user to be able to make changes to the application, add an extra data source, build a different chart on their own.
“Because if it doesn't have that element of self-service, the tool is going to be abandoned.”
More established BI vendors have not fared as well. For example, MicroStrategy, the largest remaining independent BI software supplier, reported a 6% decline in sales to $130 million in its most recent quarter. Behind that figure lay a 24% fall in licence sales.
The company rejects the suggestion that this is due to the rise of Tableau and QlikView. According to Nick Barth, the company’s UK sales engineering manager, the company recently launched a number of new, non-BI applications, and its sales force has yet to adapt.
“Whenever you add a bunch of new products to your kit bag, sometimes you're going to see your sales team stumble,” he says.
But the company has introduced self-service functionality to its BI platform. This includes the ability for users to upload their own datasets – i.e. not from the enterprise data warehouse – and an in-memory visual analytics tool.
Barth is candid about the inspiration for this last feature. “We have customers who are coming to us saying, “We’ve got these 20 users who all want to use Tableau.’”
That is proof enough that there has been a fundamental shift in the way that BI systems are being procured and used. The challenge for IT departments is to enable that shift while keeping data under control.
