In economics, the myth that human beings are rational creatures is beginning to crumble.

The growing field of behavioural economics takes what we now know about the psychology of human decision-making, with all its flaws and biases, and uses it to develop new models of economic activity.

Business management, alas, has yet to follow suit. It is still heavily influenced by Frederick Winslow Taylor's 1911 text "The Scientific Principles of Management", which sought to apply empirical analysis to the field of work but in so doing has enshrined a misplaced belief that managers are rational actors.

Anyone who has spent any time in business knows how bogus that assumption is. Decisions are made on hunches and prejudices, and are subject to all manner of cognitive biases.

See also: Simulated manufacturing at Ford Motor Company

That is not to say managers do not base their decisions on ‘evidence’. There is plenty of information sloshing around businesses today – too much, perhaps.

But rarely is that information analysed methodically, dispassionately and in a way that provides genuine guidance for business strategy.

This makes management decision-making unreliable and ineffective, which is why, according to Nigel Rayner, research VP at Gartner, it is ripe for automation.

"Algorithms are generally better at predicting the future [than humans]," Rayner said in a recent webinar. "Machine-based simulations are better predictors because they can rationally weigh all of the circumstances."

"They will predict based on all the evidence rather than bias."

But, almost paradoxically, as decision-making becomes increasingly automated and analytical, it must also become more collaborative and participatory, Rayner believes, because the assumptions on which the analysis is based must be constantly questioned.

Or to put it another way, as business management becomes more empirical, the need for peer review becomes paramount.

Prescriptive analytics

One of Gartner's newest ideas concerns the evolution of business intelligence and analytics. In the early days of BI, it was simply 'descriptive' – i.e. certain performance metrics were summarised so that managers could see roughly what was happening.

The current era, Gartner says, is the age of ‘diagnostic’ BI. This means that BI platforms give users the ability to interrogate the data to find the causes for certain trends. In-memory BI and visual analytics represent the state of the art in ‘diagnostic’ business intelligence, Rayner says.

But even diagnostic BI has proven underwhelming, he believes. "Most people are pretty disillusioned with BI.

"Diagnostic BI lets people find out what's going on in their data, but it still doesn't tell them what to do about it," Rayner explains. "That requires more sophisticated modelling and simulation."

See also: Why nearly half of all jobs are at high risk of computerisation

The next step in the evolution of BI is ‘predictive’ analytics. This is already in widespread use, but it typically requires trained statisticians using specialist software.

The next wave of innovation in BI, Rayner believes, will be the development of analytical applications that build sophisticated statistical modelling into systems that ordinary business uses can operate.

To begin with, this will rely heavily on the contribution of system integrators, who can combine a variety of BI and analytics tools with statistical know-how and industry knowledge in order to build bespoke systems.

Rayner points to a recent example from CGI (formerly Logica). A public sector organisation had a balanced scorecard, a common BI tool for monitoring performance, but found that it provided little guidance on how it should actually improve its performance.

Using a variety of tools, CGI built a statistical model of the organisation’s operations and integrated it with the balanced scorecard. Employees can now use the interface to evaluate ‘what if’ scenarios, see how they would impact key business outcomes, and plan their activities accordingly.

In time, though, specialist software providers will begin to sell predictive analytical applications as packaged solutions, probably tailored to industry verticals. "Over the next five years, the battleground is going to be 'Who has got the best statistical model?'"

The next step on from ‘predictive’ analytics is ‘prescriptive’. This means that BI systems will not just allow employees to make better decisions; they will begin to make the decisions themselves.

Instead of suggesting an ideal course of action to a human operator, ‘prescriptive’ systems will trigger the next step in a business process, whether that means conducting some automated action or sending an instruction to a human employee.

Collaborative decision-making

Clearly, putting a ‘prescriptive’ analytics system in charge of material decisions will require a great deal of trust in the underlying analysis.

After all, algorithms may themselves be rational, but the way human beings choose to apply them is as vulnerable to bias as anything else.

This is why Rayner believes that decision-making needs to become more collaborative even as it becomes more automated.

"If you've got a black box that does everything, the person who designed the black box is in a position of power," he says. "They can bring their biases to bear, and set up the system so that it always recommends what they want it to."

Instead, the assumptions that are embedded in the statistical models that underpin predictive and prescriptive analytics must be open for discussion, Rayner says. "Businesses will need to use the tools collaboratively in order to compare different scenarios, look at different options, assess different outcomes."

See also: How White Castle used predictive modelling on a chicken sandwich

This may be a matter of technology – some business intelligence platforms incorporate social media-like functionality to allow discussion around the analysis – or it may simply be a matter of organising meetings or email discussions that include the right stakeholders.

This combination of algorithmic analysis and the collective wisdom of the organisation is a recipe that Rayner believes will overcome today's flawed method of business decision-making.

"I've worked with a number of leadership teams, and the way that most decisions are made is by strong leaders having a preferred action and everybody else supporting that action," he explains. 

"Now, you can replace that with a machine that comes up with an answer," he says. "Clearly, though, you have to be able to say, 'I don't agree with what you put into the machine.'

"Because while the chance of having bias in the actual analytics is much smaller when it is automated, how the analytics is used is still subject to behavioural bias."