Let the computer make the forecast – not the boss 

You know how it works. Just lie on your back on a warm summer’s day and watch the clouds as they float by. Chances are that one of them will look like a dragon, an ice cream cone or maybe your old cocker spaniel.

“Humans are incredibly good at seeing patterns in randomness. We are programmed to look for patterns and trends,” says Professor Paul Goodwin from the Management School at the University of Bath in England.

To illustrate his point, he shows a graph based on completely random numbers that he has plotted into his computer.

“Look how perfectly this correlates with a six-month cycle,” he says, laying a second graph on top of the first. “And what happens when we base forecasts on such a cycle?” he asked, letting everyone in the audience answer the question for themselves.

Goodwin made his points at the A2009 conference in his keynote presentation, “Knowing when to intervene: How to make effective judgmental adjustments to your statistical forecasts”.

Smarter than expensive software
Goodwin and his colleagues have observed a number of companies to analyze how they attempt to predict the future at various points in their supply chains. Among other things, they have found that companies adjust up to 93 percent of forecasts generated by analytical software.

“These organizations use extremely sophisticated software to run analyses all night, then sit down first thing in the morning and adjust the figures. We followed a pharmaceutical company which held 17 meetings in the course of a month to review their forecasts. This cost a total of 80 management hours,” says Goodwin.

According to Goodwin, there are situations in which people are smarter than their analytical software. “Software cannot know about a major strike or a new government initiative. You might also be able to add historical data that the software doesn’t have. And there are also cases in which management directly affects outcomes, for example by reducing prices for a given product.”

Still alive and kicking
But Goodwin’s research shows that the vast majority of changes made to software forecasts are quite small and that small adjustments are not usually a good idea.

“Small adjustments of just a few percent waste time and make forecasts less accurate. It often seems that small adjustments are made primarily to let people higher up in the organization know that you’re still alive and kicking. We’ve also heard a lot of people say that the sooner a forecast is discussed in a meeting, the greater the probability of it being changed. By the end of the meeting people just want to go home or back to work, and having adjusted the first forecasts justifies the meeting,” he says.

On the other hand, large adjustments are often more accurate because specific knowledge is required to make radical changes to a forecast. People rarely purchase five times as many of a given product on a hunch.

A piece of paper 212,206 km thick
Goodwin’s main point is that people should leave demanding calculations to the analytical software on the computer. He presented nine common errors that people often make when they use their judgment to second-guess computer calculations.

One of them is called “anchoring”. When we make an estimate, we usually base it on a given value – the anchor – and adjust forecasts according to it.

“We base next month’s numbers on this month’s numbers, next year’s on this year’s, and so on. But this is like gravity on Jupiter – you can’t really get away from it. If the point of reference is too low, then most people will make estimates that are too low. This is particularly bad in situations of exponential growth,” explains Goodwin.

To illustrate his point, he takes an ordinary piece of paper merely 0.193 mm thick and asks the audience to use their judgment to estimate how thick the paper would be if it were folded across the middle 40 times. As many people know, it is difficult to fold a piece of paper more than seven times. But many jaws in the audience dropped more than a few millimeters when Goodwin revealed that a piece of paper folded 40 times would be 212,206 km thick – more than half way to the moon.

“The point is this: Don’t use your judgment to predict trends; use analysis,” he says.

Rats outperform managers
Among the other common errors are succumbing to peer pressure, being prejudiced by optimism or political pressure, or basing forecasts on too short a period.

“Companies come to us and ask for statistics based on three weeks of data. Our best advice to them is to put the data into a spreadsheet and take an average,” says Goodwin.

However, if both the historical data and the analytical software are available, then it is simply a question of adding as much data as you can.

“Many companies don’t want to go back further than three years. Trends were different back then, they say. But the software can handle massive amounts of data, and it needs it in order to make reliable forecasts. Use as much data as you can, because unlike us, the software will definitely not see patterns that aren’t there,” he says.

And it is precisely this human tendency to look for patterns that creates perhaps the biggest problem for forecasters. He explained this with an example from research conducted at Yale University, in which rats were sent into a labyrinth which ended in a T. The rats could either turn left towards Point A, or right towards Point B. Food was placed at Point A 60 percent of the time, and at Point B 40 percent of the time. The rats quickly spotted the pattern and turned left consistently, so they got food 60 percent of the time.

Students were also included in the experiment – without having to enter the labyrinth physically.

“The students were convinced that they could spot a pattern: something like go in one direction twice, then go in the other direction. We come up with more and more complex explanations to prove our observations, even though the truth is that the sequence is random. The students got it right just 52 percent of the time, so in fact rats provide better forecasts than either we or our bosses can,” explained Paul Goodwin.

His explanation for our wanting to explain the numbers is straightforward. “Statistics are boring, stories are fun. A dull statistical forecast won’t hold a candle to an exciting story,” he says.

Paul Goodwin, The Management School University of Bath, United Kingdom