Budgeting no longer helps organisations to perform better. On the contrary. It prevents organisations from performing to their full potential, because given the level of volatility, uncertainty and ambiguity in the world only thing we know for sure is that we don’t know. “The future ain’t what it used to be”, as the American baseball player Yogi Berra once put it.
In this article, Steve Morlidge, author of "Future Ready: How to Master Business Forecasting" and The Little Book of Beyond Budgeting", argues that the quality of business forecasting – used to steer an organisation – is unacceptably poor. He goes on to present six simple principles that will help executives significantly improve the performance of their forecast processes. More reliable forecasts speed up decision making and so help make businesses more agile.
The recent economic crash has been badly damaged reputations as well as fortunes – no more so than those of economic ‘experts’ who have been roundly criticised for failing to forecast the catastrophe. However, failures in forecasting are not confined to large-scale economic systems. The forecasts used by business executives to steer their businesses have also proved highly fallible.
‘The financial crisis has obliterated corporate forecasts’ reports the CFO Magazine (Ryan, 2009); 70% of respondents to their recent survey said that they were unable to see more than one quarter ahead. However, the problem is not restricted to times of economic turmoil. Over the last four years, the 1300 companies quoted on the London Stock Exchange issued, on average, 400 profit warnings every year. On average, each one resulted in a loss of 10 to 20% of market capitalization (Bloom et al., 2009); some $200 million.
It is, therefore, no surprise that a survey of 540 senior executives recently conducted for KPMG (EIU, 2007) found that improving forecasting came at the top of the priority list for the next three years. ‘Ability to forecast results’ also comes at the top of the list of ‘Internal Concerns’ for CFO’s right across the globe (Karaian, 2009).
CFO’s are right to be concerned; business forecasting is riddled with bad practices. For example, most businesses for much of the year forecast no further than the financial year-end. As a result, there is little visibility of ‘the road ahead’. Forecasts are often too detailed and too late for managers to take action. Boardrooms resonate with acrimonious debate about what is the ‘right number’, yet many organisations have multiple forecast processes, each presenting competing views of the future, which are never reconciled.
Obsession with accuracy may coexist with a culture where professionally prepared forecasts are arbitrarily adjusted on a routine basis. Leaders often give contradictory messages, such as ‘give me your best estimate of what you think will happen’ and ‘your forecast must come back to target’, leaving managers confused and disorientated. Manipulation of forecasts as part of a corporate political game is rife; numbers are frequently ‘sandbagged’ or ‘spun’ to create a favourable impression.
At the heart of the problems experienced with forecasting is a fundamental misconception: that forecasting is the same as prediction. The role of forecasting is to provide us with information about what might happen so that we can take action to avoid the forecast outcome if it is not what we want. If we do so, we invalidate the forecast. Forecasts are for helping you to steer to your destination; they do not prophesy your fate.
The failure to grasp the fundamental nature of forecasting is compounded by a second misconception. In my experience, managers either believe that forecasting is straightforward – ‘just common sense’ – or that it is extremely complicated - requiring the use of complex mathematics – and so best left to experts. The reality is that it is neither; it is a matter of properly understanding the nature of forecasting as an aid to decision making, and working in an organised and disciplined way to produce ‘good enough’ forecasts. Good tools and techniques may be necessary, but they are not sufficient. You need to know how to use them properly and create the kind of culture that encourages people to tell the truth.
Forecasting in business is a complex mess, but it need not be. I believe that there are six simple principles, which once mastered, will significantly improve the quality of forecasting in almost any organisation.
Principle 1: Mastering Purpose
Business forecasting is like sailing at sea. It makes sense to plan before you start the journey, but the original plan is often soon out of date because of changes in the weather or tides. At this point, you need to forecast where you are headed, so that you can work out what corrective action is needed to get you to your destination.
The first thing that is clear from this example is that it is important to make a sharp distinction between a forecast (where you think you will be) and a target (where you would like to be). The second thing it helps us to understand is the role of forecasting: to support decision making. In order to do this well, a forecast needs the following qualities. It should be:
- Timely. If you are heading towards trouble, a rough and ready forecast delivered quickly is much more valuable than a perfect one that is too late to take avoiding action
- Actionable. We need to be able to use a forecast to take decisions. Do we need to make lean on the tiller or hoist different sails? In business, this means that you may need much more information about ‘projects’ than a traditional budgeting system provides; much less detailed information for ‘business as usual’
- Reliable. A forecast does not need to be precise to be reliable for the purposes of decision making ; it merely needs to be ‘accurate enough’. In practice, this means it should be free from bias and with acceptable variation
- Aligned. No one would like to be in a boat where every member of the crew had a different view of where the ship was heading, and what course to steer to avoid the rocks. Finally, providing all the other criteria are met, a forecast process should be:
- Least cost.
Principle 2: Mastering Time
How far ahead do you need to forecast? The answer depends on how long it takes to enact a decision.
The captain of a super tanker needs to consistently forecast 3 miles ahead because that is how long it takes it to stop. A speedboat, on the other hand, may require very little forward visibility. In practice, this means that businesses need a rolling forecast horizon, based on the lead times associated with ‘steering actions’. A traditional year-end forecast is like overtaking on a blind bend – you have no idea of the possible outcome of your decision.
How frequently should you forecast? That depends on how quickly things change. More frequent forecasts are needed to safely navigate through the busy Singapore Straights than in the wide open seas of the South Pacific.
Principle 3: Mastering Models
Any form of forecast requires a model; a set of assumptions about the way the world works. The model used in forecasting could be a statistical model; one that extrapolates into the future from the past. This approach can be effective, but often the future is not like the past. You might, therefore, choose to use a mathematical, or driver based, model; for example one that helps you to forecast the impact of volume on the cost base of the business. However, often the world is too complex, or the business too fast changing to make this approach workable.
That is why forecasting in business often relies heavily on judgment; where the model is in the head of an expert or a larger number of people who ‘know what is going on’. This approach is not without problems. Human judgment can be flawed, and managers can feel under pressure to adjust forecasts to ‘avoid giving nasty surprises’ or ’sounding defeatist’. As a result, judgmental forecasts are particularly prone to bias.
The trick is to understand the range of methodologies available, choose the appropriate one, and take steps to mitigate its weaknesses. So, for example, a statistical or mathematical technique might be used to produce a baseline or ‘business as usual’ forecast and judgement to estimate the impact of the decision made to change the course of affairs.
Principle 4: Mastering Measurement
The only guarantee that you can rely on a forecast to make decisions that affect the future is that those previous ones have proved to be reliable in the past. Yet, few businesses take the simple steps required to monitor their processes for evidence of bias, so that they can take action to eliminate it if detected. Most businesses fail to measure forecast quality at all.
Those businesses that do attempt to monitor forecast quality often measure the wrong things at the wrong time. A common mistake is to measure forecast errors at a point in time that is likely to be after the forecast has been acted upon. This is like blaming the navigator for having forecast a calamity that her forecast has helped avert. At its simplest, a series of four short-term errors with the same sign (positive or negative) is evidence of bias; any fewer is likely to be the result of chance.
Principle 5: Mastering Risk
The only thing that we know with absolute confidence about the future is that any forecast we make is likely to be wrong! Where there is the debate about the forecast, it should not focus on whether you have the right ‘single point forecast’, but how it might be wrong, why, and what to do about it. In particular, it is important to distinguish between ‘risk’ - random variation around a realistic single point forecast - and ‘uncertainty’ – resulting from a shift in the behaviour of a system that completely invalidates the forecast. Banks’ over-reliance on risk models that failed to take account of uncertainty was a major contributor to the recent economic collapse. Whatever form your ignorance of the future takes, it is important to develop the capability to spot and diagnose deviations from forecast quickly, and to create a ‘play book’ of potential actions to enable a swift and effective response.
Principle 6: Mastering Process
Mastering forecasting is not an art, but neither is it complex science. It is mainly a matter of applying a modest amount of knowledge in a disciplined and organised fashion; as a process. A good process – like a good golf swing - will produce good results.
Building a good process involves doing the right things in the right order (cultivating a good technique), in the same way over and over again (grooving the swing). Those things that are responsible for bias (hooks and slices) should be designed out of the process (remodelling the swing), and the results of the process continuously monitored (the score) and minor flaws corrected as they become evident Again, like golf, temperament is as important as technique. Blaming people when the process is at fault is a sure way to encourage dishonest forecasting
Done well, forecasting will help a business respond swiftly and effectively to emerging reality and so gain a competitive edge. Done badly, management may be misled into making the wrong decisions. However, businesses have little option but to forecast, because without any kind of ability to anticipate, organizations can only react to those things that have already happened, which, by definition, they have no ability to influence. The tools and techniques that business need are already available, managers simply need to learn how to use them effectively.
It is difficult to think of another business process that is as universally detested as annual budgeting.
The list of complaints will be familiar to anyone who has run a budget process or has been subjected to one…and that probably means everyone reading this article.
- It consumes an enormous amount of time and effort…often the time of those that have the least to spare. Indeed, the Hackett Group reckons that up to 10% of management time is spent on the annual budget, which would make it the single most expensive business process.
- It takes so long that by the time it is complete it is out of date and of limited value. A survey by Business Finance magazine found that two-thirds of budgets were out of date between 4 and 6 months into the year…so most companies are flying blind most of the time.
- Budgets are inflexible by design. Once people realise they are out of date it is difficult to change them. This is partly because of the level of detail at which they are produced and mainly because they have become the anchor point for so many management processes – from incentives to performance measures, to decision making rights – all of which are inter related meaning that a change to one aspect will impact many other peoples ‘rights’ and obligations.
- Budgets drive dysfunctional ‘gaming’ behaviour. It is always easier to negotiate a lower target than it is to improve performance so unsurprisingly people devote a lot of effort attempting to manage the budget process. As a result revenue and profit are sub optimised, costs artificially inflated and political behaviour is rewarded.
Budgeting has been around for nearly a century but it is still with us, despite its well-known failings.
My guess is the reason is that most people are not aware that there is an alternative to traditional budgeting. Most of the ones that have been touted over the last few decades – Zero Based Budgeting and ‘Better Budgeting’ – amount to no more than doing the same things in a slightly different way. In my view, they do the wrong things slightly righter.
The best alternative – Beyond Budgeting – has also been around for some time but it has failed to have the impact that it should have because it has not been properly understood, or it has been actively misrepresented.
The name ‘Beyond Budgeting’ is an accurate description of what ‘it’ is but unfortunately in some people’s minds it conjures up a nihilistic vision of chaos. I believe these fears are misplaced. Control (in the non-pejorative sense of the word) is key to Beyond Budgeting – it is simply exercised in a different way, using different tools. The ends are is the same, but the means differ.
This is summed up in the following table:
The Beyond Budgeting ideas have matured over the last decade or so and there are now many examples of companies that have applied these ideas and seen their business performance improve dramatically. As a result, I think it is time to correct false impressions of what BB is, and why it is successful. To help relaunch these ideas I have written a short book based on my experience of working with the concepts since they first burst onto the scene in 1998.
The concept of ‘Beyond Budgeting’ has been around for nearly twenty years now. Although it has helped transform many businesses and has become part of mainstream management thinking in some parts of the world, I talk to many business people who have still not heard of Beyond Budgeting. And many of those that are aware of it find the concepts difficult to grasp.
I believe that these ideas are too important to be overlooked or ignored – so I wrote a short book to fill these gaps in awareness and understanding.
The name of the organisation itself gives some clue as to why these gaps exist. ‘Beyond Budgeting’ (BB) describes what it wants to get rid of but not what should take its place. And the alternative that it advocates is not a simple blueprint that can be copied and rolled out across an organisation. Rather BB is a set of principles that have to be interpreted in the context of the unique challenges and opportunities faced by any particular organisation
For finance professionals, Beyond Budgeting changes how they do things not what they do.
So, instead of setting fixed targets for the end of the financial year, they would set relative targets that do not expire at period ends. And forecasting across a rolling horizon would replace detailed annual plans.
Resource allocation is a continuous process, made in response to emerging threats and opportunities, rather than an annual set piece event. And performance is measured by tracking trends rather than analyzing variances to budget.
Finally, the elimination of fixed annual budgets also has implications for other, related, business processes, such as how incentives are set and business activities co-ordinated.
If BB sounds like a small idea, of interest only to the Finance community, think again. But in reality, it is a large and subtle set of concepts that have important implications for the way that work is done and how organisations are structured and governed. To use an analogy from computing, BB looks like an organisational ‘app’ but it is more like an operating system - largely hidden from view but critically important to the functioning of the entire system and how it performs for its users: its employees, customers, suppliers and shareholders.
Beyond Budgeting creates an opportunity for finance professionals to contribute to business decisions making in a positive way rather than being the guardian of a process that everybody hates and adds so little to the business. I see BB as being liberating for the business and for the people working in it - finance people included.
As far as I know, we are not legally required to forecast.
So why do we do it?
My sense is that forecasting practitioners rarely stop to ask themselves this question. This might be because they are so focussed on techniques and processes. In practice, unfortunately, often forecasting is such a heavily politicised process, with blame for ‘failure’ being liberally spread around, that forecasters become defensive and focus on avoiding ‘being wrong’ rather than thinking about how they can maximise their contribution to the business.
This is a pity, because asking fundamental question ‘how does what I do add value to the business’ could help forecasters escape the confines of geek ghetto and the dynamics of the blame game and reposition the profession as important business partners.
So why do we forecast? Let’s answer this question by considering the alternative.
If we didn’t forecast what would we do? The obvious answer is that we would react. So, if we had just sold x units we would ask our supplier for x units to replenish our stock, which in order to provide good customer service we would maintain at a level sufficient to deal with the volatility of demand until the replenishment order turns up.
Pretty simple stuff, eh? Simple, but not stupid, though, because this is exactly the philosophy that underpins Kanban, and Toyota seem to have done pretty well over the last couple of decades.
So why do we forecast rather than simply replenishing our stock based on prior demand?
The answer is that by forecasting demand we anticipate demand volatility and so do not have to hold as much buffer stock to provide the same level of customer service. In technical terms, safety stock levels are based on the volatility (as measured by the standard deviation) of forecast error which is lower than the volatility (standard deviation) of demand…at least that is the aim.
So the reason we forecast is that it allows the business to hold lower inventory levels…and this is how forecasters add value. And wouldn’t it be great to transform the forecast process from sterile discussions about MAPE levels and inquisitions to discover who has ‘caused’ the errors to one where we could talk about how to add even more value?
So how do we do this?
The key is to compare actual forecast error to the simple naïve forecast error. A simple naïve forecast is where we use the last set of actuals to produce the forecast for the subsequent period, which if you think about it mimics a simple replenishment strategy. So if our actual forecast error is lower than the naïve forecast error then we have added value. Easy isn’t it? And it gets better.
Because, as a general rule, more volatile data series are more difficult to forecast than stable data series this approach automatically allows for forecastability – the difficult or ease of forecasting. So all of those pointless arguments about whether comparisons are fair or not disappear. In fact, our research has shown that it is possible to use this approach to calculate the limit of forecastability, so we can, for the first time objectively measure the quality of forecasting. So instead of wasting our emotional energy on attributing blame we can celebrate our successes and work out how to get even better.
This is not completely new. For a number of years Mike Gilliland of SAS has been advocating that forecasters measure forecast error at different steps in the forecast process (for example before and after market intelligence is added to statistical forecasts) to determine where value is added and where it is destroyed. This approach supercharges Mike’s value added methodology, allowing us to analyse the value added by the entire process, not just process steps, make objective judgements about the quality of forecasting and facilitate benchmarking in a way not possible before.
Sounds great, doesn’t it, but a few words of caution are in order before you dive headlong into this pool.
First, to be meaningful, value added has to be measured at a very granular level – at the level at which replenishment orders are generated; perhaps at warehouse/SKU level. Also, we need to calculate averages as we cannot look at individual periods in isolation, on the absolute amount of value added in order to focus attention on the items that matter. In addition we would ideally be able to make a distinction between systematic error (bias) and unsystematic error (variation) as they often have different causes and so require different corrective action.
All of this likely to take the using value-added measures beyond what can be accomplished on a spreadsheet, particularly as we will want to drill up and down product hierarchies to track down the source of problems.
Finally, be prepared for a shock. Our research shows that it is not as easy to beat a simple naïve forecast on average as you might think. And at the most granular level typically up to 50% of forecasts destroy value. On the positive side, this demonstrates how easy to should be to increase the value of the forecast process.
So there will perhaps be some cost and perhaps some false pride to be swallowed but I would argue that this is a small price to pay for repositioning forecasting from back office whipping boys to respected professionals making a measurable contribution to the value of the business.
Steve Morlidge is an accountant by background and has 25 years of practical experience in senior operational roles in Unilever, designing, and running performance management systems. He also spent 3 years leading a global change project in Unilever.
He is a former Chairman of the European Beyond Budgeting Round Table and now works as an independent consultant for a range of major companies, specialising in helping companies break out of traditional, top-down ‘command and control’ management practice.
He has recently published ‘Future Ready: How to Master Business Forecasting’ (John Wiley 2010), and has a PhD in Organisational Cybernetics at Hull Business School. He also cofounder of Catchbull, a supplier of forecasting performance management software.
As you read this, hundreds of thousands of managers in most organisations, from junior to the most senior will be in the middle of an annual ritual that will dominate their lives for the rest of the year and perhaps beyond, for little tangible benefit.
The ritual is traditional budgeting.
It is difficult to find anyone who enjoys the process; at best it is regarded as a necessary evil. But in this year of Brexit chaos in the UK the exercise has a particular futility since it depends on maintaining the pretence that it is possible to predict the future with sufficient confidence to enable managers to rationally allocate resources and set meaningful performance targets.
But there was a time when it did work. When a young Chicago University professor called James O McKinsey published ‘Budgetary Control’ in 1922 he provided the first wave of professional managers with a tool to maintain financial control of large divisionalised organisations without the communications and computing technology that we now take for granted.
Today it is different. Budgeting no longer helps organisations to perform better. On the contrary. It prevents organisations from performing to their full potential, because given the level of volatility, uncertainty and ambiguity in the world only thing we know for sure is that we don’t know. “The future ain’t what it used to be”, as the American baseball player Yogi Berra once put it.
Why then, nearly 100 years later when we have the ability to sense and respond to what is going on in our organisations in close to real time do we continue to perform this archaic set of practices?
Part of the reason for continuing to perform what the management theorist Russell Ackoff called ‘The Corporate Raindance’ is the fear of something bad happening if we stop. Like all rituals, it requires sacrifices for its potency to be believed and thrives where the link between the act and the outcome is opaque.
Another reason is ignorance of the alternatives. Well established, sober companies like the Swedish bank Handelsbanken who have not used traditional budgets for nearly half a century while consistently delivering above average returns testifies that robust alternatives do exist. And starting in 1997 the Beyond Budgeting Institute has codified the practices that enable pioneering companies like Handelsbanken to react and adapt to a rapidly changing world without the bureaucracy and gaming behaviour associated with budgets.
Managers need to look up from their spreadsheets reflect on what they see going on around them, explore the options and have the courage to do something different.
But perhaps a more fundamental reason for the lack of progress is that the traditional model of management, of which budgeting is one manifestation, is built on a deeply ingrained assumption that is rarely surfaced. The assumption is that people cannot be trusted to do the right thing, either because they are not competent or because they do not have the best interests of the organisation at heart. If this is what you believe, their ability to act needs to be constrained and they need to be incentivised to ‘perform’.
Ironically, this often has the opposite effect to that intended. People are either incentivised to game the system by negotiating tame targets that they are careful never to beat or to hit their targets whatever the consequences as almost every one of the corporate calamities and scandals that we read about almost daily testify.
We don’t tame anti social behaviour with budgets. We create it.
We live in a bizarre world where we trust strangers that we buy things from over the web more than the people with whom we share offices.
It is time to call a halt to the annual budgeting charade.
Our uncertain world won’t allow it. The new generation of workers won’t tolerate it. And our organisations can’t afford it.
The game is up.
In this article, Steve Morlidge argues that the quality of business forecasting is unacceptably poor. He goes on to present six simple principles that will help executives significantly improve the performance of their forecast processes.
Many millions of people are stuck with the habit of smoking. They know its bad for them and it will eventually kill them, yet they continue.
In a previous blog post, I mentioned that people who are highly sensitive to the lack of flexibility of traditional budgeting often see rolling forecasts as the answer. So, you might think, forecasts are like budgets but done more frequently – right?