Have you ever been at an annual strategy off-site, where you and the team developed some great plans, walked away…. and a year later very few of those ideas or plans were implemented?
So often I hear of organisations, or rather their management teams, who have decided that once a year, they need an off-site to think about their strategy for next year. (I have certainly run many of these over the years).
When reading about decision-making in the excellent book, “Thinking Fast and Slow” (Daniel Kahneman) I was struck how it related to the thinking that goes on in these short, annual offsites. How much it related to how teams judge the challenge of problems when they make decisions, choose their strategies and develop their plans.
How we think, whether it be fast or slow, has implications for strategies decided at annual off-sites.
The example comes from Chapter 23 of Thinking Fast and Slow, by Nobel prize winner Daniel Kahneman. In this example, a team of academics with which he is involved, are estimating how long it will take them to develop their material into a brand new course, design a curriculum and write the course book.
How long will we take? The inside view
The first question, was “How long might this take?”. They decided to get everyone to estimate for themselves how long they thought the project might take, before sharing their thoughts with one another.
They had already done some experience and had done some product development for this course, so they understood the complexity. They even had some examples developed already. They surely knew what they were doing and facing.
In general the team estimated around 2 years, give or take 6 months. To all of them, this seemed a reasonable estimate.
This estimate by the team seemed reasonable. They had some experience and had started their work together.
We’ll call this “the inside view”.
Asking an expert: the external view
They then asked the curriculum expert amongst their team. They asked him whether he knew of similar teams who had set out on such a venture, how long other teams had taken to develop such a similar product, from where they were now. The expert did indeed know of such teams, and projects, but the answer was a shock.
In general, not all these other teams completed the task at all. Around 40% failed to complete the task! This was a shock to the team: they had never even considered that they would fail.
In creating the plan, the team never even considered that they might fail!
OK, what about those that did complete the task – how long did they take? The answer was again a shock. The expert could not think of a team that took less that seven years! On the good new side, (if there was any) no team took longer than ten years (apart from those that failed altogether to complete the task).
They had underestimated the time and effort by a factor of between 3 and 4!
Grasping for hope, the next question was about capability. Perhaps we are better than these other teams? How do we compare against these other groups? The answer was not one that was expected. “Well you are below average, but not by much”.
We’ll call this the external view.
We should have quit that day!
What is clear here is that the optimistic view of that team in that room at that time, bore absolutely no semblance to the reality of the task, to benchmarks about how long the task would actually take, not even their likelihood of success.
Daniel Kahneman described this episode as one of the most instructive episodes of his life. What happened next? Well, after “a few minutes of desultory debate” they carried on.
The expert’s view was that 40% of such projects fail to deliver anything. The typical project time was 3-4 times their estimate. Yet, despite this new information, they just carried on.
The course, and its book, were eventually completed – eight years later. By then, Daniel was no longer part of the team. By the time the text and course were finally delivered, the customer had lost interest and the material was never used.
By the time it was eventually delivered the customer had lost interest: it was never used.
Lessons for the snnual strategy off-site and strategic thinking in general
As I said before, I have run many an annual strategy off-site session. You put a bunch of executives in a room and ask them about their strategy: they will start by telling you that it will succeed and that it is achievable in a reasonable timescale.
Teams need time to research, reflect and do some homework.
Nowadays, I always encourage my clients to run strategy workshops as a series of days separated by breaks of up to two to three weeks. This means the teams have time to reflect on what they are thinking, validate their ideas, check things externally and basically “Do their homework”.
Most importantly, the organisation is not committed to some adventurous over-ambitious “strategic project” that has little real chance of succeeding because the decision-making was so poor.
The organisation must not commit to some adventurous over-ambitious “strategic project”, with little chance of success, because of a poor decision-making process.
Appreciating the complexity: Challenging the optimism
Another aspect is about the complexity of the task and challenge and the optimism with which it was put together.
At the end of a day, I quite often look at the set of initiatives, or the changes that are planned, or the objectives and performance gaps to be closed: ones that a team may have spent some time putting together.
I then turn to them and say, “How realistic is this given all your time and resource constraints?” It is not meant as a provocative challenge (even though it is). It is meant as a constructive, observation about where they are in their decision-making process. They are not committing to these projects or objectives: or at least they should not be. They are placing them down as a set to be thought through, validated and made feasible and achievable. They should not be told, “Great – you now have a plan – go and execute it!”.
At this point, never say, “You now have a plan – go and execute it”. Instead ask, “How can we test and validate our estimates against the real world?”
On the contrary, they should be asked, “Given your time, resources, distractions, ambitions, culture, external events, and energy…. Is this realistic?” “How can you validate this, not to prove yourself right but to ensure it is realistic?”
They should be asked, “If you took an external experienced, perspective on this from someone who had done it before – what would they say?”
Better still, you should ask, “Who do we know that has done this, who can provide us with a benchmark to see if we are being realistic?”
What is going wrong in these situations?
Daniel Kahneman has some observations about the reasons for these over optimistic and ambitious estimates and plans. All of which apply to annual strategy off-site meetings.
It is vital you understand the reason these optimistic, overambitious, unachievable plans are created.
- The inside view extrapolated from their specific circumstances. They searched for evidence from their own experience. There was no external validation. Even the cautious only added a few months to their estimates.
- The plan was sketchy. No one had spent time working through the details, analysing the challenges that the approach and landscape presented.
- The outline of the task was simplistic – they knew what chapters they had to write, but only in outline. The two easiest had been completed. No basis for further extrapolation. Kahneman calls this WYSIATI (What you see is all there is).
- There was no way for us to foresee, or even imagine, the challenges, (divorces, illnesses, deaths, changes of job) that would really happen.
- They had no ‘baseline prediction’ from which to work. their estimates were from their head, with no comparable situation. As a result the inside view was not even close.
I also wonder whether there was a degree of team and group-think occurring. “We are planning this and we want it to succeed”. Kahneman says they produced their estimates silently, without consultation, and only when all were revealed were they discussed. This is a far better way than letting the estimates come out and be discussed as people progress.
Only one person in the room had the outside view. Even he ignored it when first addressing the question and producing his estimate.
Even when they were all exposed to the outside view, they collectively ignored it.
How we avoid such thinking in our annual strategy off-sites
First we must be aware of, what Daniel Kahneman calls, “The planning fallacy”. The planning fallacy is a manifestation of “pervasive optimistic bias”. that is we look on the world more favourably than it truly is.
The planning fallacy is a manifestation of “pervasive optimistic bias”.
The Planning Fallacy describes plans and forecasts that are:
- Unrealistically close to best case scenarios, without any thought of the many things that might or could go wrong.
- Uninformed by the information that is available by consulting the statistics and track records of similar cases?
In other words, They are optimistic and uninformed. Where was the learning that fed into these estimates?
This planning fallacy affects all the parties involved. It affects the planners who under-estimate how long it will take. It affects the implementers who get blamed for delays against the over-optimistic plan. It affects the customers who naively believe (and want to believe) these planner’s optimistic estimates. As Kahneman puts it, “they have an inability to imagine how much their wishes will escalate over time”. They have an inability to conceive that a ‘perfectly written’ specification or ITT can produce a plan that will not be implemented.
Planners desire to have their plans approved. Buyers want to believe it can be delivered quickly and at low cost. There is a lot of self-deception.
Kahneman talks about the desire planners have for their plans to be approved. That desire creates a blindness and over optimism, that goes unchecked (in both senses). In public sector procurement terms, the customers have an inability to appreciate how much the planners are being optimistic, and peddling a fallacy, whilst they try to cut costs that are impossible to deliver for the customer. Yet the customers believe that if they take these plans, at these lowest (over optimistic costs) they will actually be deliverable.
There is a lot of self-deception.
So how do we increase forecasting accuracy at a strategy away day?
Kahnemann refers to a summary by Oxford academic, and Danish planning expert, Bent Flyvbjerg. He says, use reference examples to avoid optimism and to make sure you are not planning for the optimistic best case.
Use reference examples to avoid optimism and to make sure you are not planning for the optimistic best case.
“The prevalent tendency to underweight or ignore distributional information (external information from similar ventures) is perhaps the major source of error in forecasting.”
He continues, “Planners should therefore make every effort to frame the forecasting problem so as to facilitate utilizing all the distributional information that is available”.
Using such an approach is appropriately called, “Reference class forecasting”. It involves
- Identifying and finding appropriate reference classes and cases
- Getting the statistics from these other reference cases, and
- Adjusting their base cases match the circumstances you are dealing with, making sure to avoid biases and optimism.
Implications for strategy away days
For me, and I hope for you, there are clear implications for strategy away days.
- Yes, do planning and forecasting, BUT be sure to validate them afterwards.
- Look to the expertise in the room, BUT ask them to check “How do they know?” and what is their reference case?
- Take a pessimistic (realistic) view of the ambition, hopes, objectives, pace of change and plans. (This is good scenario planning).
- Never go with what is produced on the day – it will be wrong. You just don’t know how wrong and where yet.
- Take time to find reference cases validate the view. Give people time to reflect and consider the whole picture. Time in that room, that day, is not enough.
As I said at the start, we make space in, and between these events for thought, research, homework and reality. We challenge our clients to view their thoughts and work critically. To give it a reality test. To ask, if this were someone else’s work, what would you think? What caution would you provide?
We challenge our clients to view their thoughts and work critically. To give it a reality test.
We want our clients to walk out of the room all telling the same story of their strategy, and with the confidence that they can deliver that strategy, as a team.
We see this as part of developing the quality of conversation, thinking and decision making that goes into strategy choice, execution and learning.
To have any credibility, that strategy, plan and confidence in that strategy and plan, takes more than a single annual strategy off-site away-day (or two) to develop.
To find out more how we work, talk to us…. or simply drop us an email using the form below
