As someone who predicts the future for a living (or leads those who do), it is always fascinating to see how misunderstood the concept of forecasting can be. The good old crystal ball isn’t particularly helpful. Neither is a random number generator. Both may seem like they are being used sometimes when the results vary wildly from reality, but neither are typically part of the process. (Note: I didn’t say never, just typically!) Like most anything, there is a rough process to follow that can aid both your results and the perception of your results – which are both equally important in business.
The process is iterative and has a few steps (ok 7 steps), but none of the steps are cost, time, or effort prohibitive and all have plenty of room for variation to your specific situation. As a matter of fact, going through all the steps will likely save time and energy in the long run. So with that, let’s get into the process!
Observe: Observation will take more than one form. The most obvious: quantified data. You want to physically observe to gather data or use automated tools that have gathered data for you. Either way, you want to observe the current state of things and see as much into the past as possible. These observations can be invaluable as you understand the relationship between the past, present, and future. Observations like:
- “how often does this happen?”
- “what has happened in the past?”
- “how long does this take?”
- “what are the most common reasons it happens?”
- “what are the most common outcomes?”
and on and on will help you shape your ideas about the future of whatever you are trying to predict. Observing both the specific topic you are predicting and the context around it can very much shape your ability to predict accurately.
Listen: Beyond gathering data and observing the thing you hope to predict, take ample to time to listen to those that are familiar with the topic, both directly and indirectly. Have conversations with folks who are directly involved with your prediction topic – they will provide insight that will help explain the data and help explain how the future could vary from the past. They’ll have information that you’d not be able to infer from data alone that can improve your ability to understand (and re-observe) data that simply won’t exist otherwise. Additionally, spending time with folks who are indirectly involved (familiar but not hands-on) will likely give you some color as to what is working well and what could be improved with a less biased perspective (since they aren’t the ones that are actually involved). This is second order information, so do gauge it with a grain of salt, but look for insights that could work in tandem with information gleaned in other ways to speed up your prediction capabilities.
Model: Once you have some reliable data and some insight to that data, it’s time to start building models for prediction. The range of possibilities isn’t infinite but it is broad. Depending on the type of prediction you are making, there likely will be a multitude of mathematical or logical options to create models of the future based on what you’ve learned about the past, the present, and the future trends you’ve observed or heard. One note of caution: Id tend to approach this with Occam’s razor in mind – don’t use overcomplicated modeling when something simple will do. Ive seen math for math’s sake in forecasting more often than Id like to admit and it rarely adds value (although it does add complexity, time, effort, energy, and inefficiency). The same holds for building logical prediction models – a simple model that is easy to understand and update that produces 80% accuracy may end up being far wiser than a highly complex model (with much more intricacy and room for breakage) that produces 85%. Simple to make, simple to understand, and simple to adjust will likely make you and the folks that need to understand the forecast far more confident in the results, so long as they are reasonable.
Range of outcomes: As we noted above, there will be a multitude of potential forecasting tools/options. You can use one or more than one. Also within each one you use, you will have the ability to adjust assumptions both historically and into the future. With all these combinations of tools, assumptions, and outlooks you are creating a range of possible outcomes. If you think in sales terms, you can make an outlook with the most conservative of assumptions (remove big historical sales and assume none will repeat, assume your lowest lead to sale conversion rates from any period in the history for the entire future, etc.) and then with the same model do the opposite. Or use different models. This gives you several different projections outlining the impact of various assumptions – from most conservative to insanely aggressive if you’d like!
Predict: A good forecaster makes their proverbial name in this phase. Ultimately, you have to take all the knowledge you’ve acquired in the prior steps and ask yourself “what outcome is most likely from this range of outcomes?” This could be done by selecting the most representative model, create an amalgamated model from you most likely few, or the simply by dramatically tightening your range of outcomes from say A to Z to M to P, explaining the other outcomes are possible but far less likely. While a good logician or mathematician can do the other steps flawlessly, a good forecaster has the ability to make the most consistent predictions based on all that information and not get distracted by the extremes.
Align: Once you’ve landed on your own prediction based on your information, then you need to be able to readily and willingly bring your stakeholders on board an align around your prediction. This is the first big test of your prediction – do the people who are relying on the data feel like it is reasonable? Do they have any strong feedback that makes you reconsider you decisions? Do they know any up until now undiscovered information that would change the modeling? Creating alignment and agreement on the outlook is critical to make sure everyone is pointing at the same proverbial north star. Conversely, if you can’t get alignment, diving into why that is the case is essential as it’s a great learning opportunity for you and your stakeholders.
Learn: Finally, with a prediction in place it is time to observe how it performs. Every data point is valuable…but don’t react to single data points (unless your prediction is of one single event of course!). With each data point, see if there is any obvious insight. If there isn’t, talk to those closer to the subject to see what insight they can offer (quantitatively or qualitatively – don’t underestimate the power of anecdotal beliefs/information). As you watch data come in, start to understand where your model could be improved – Assumptions? Process? Prediction? Don’t get hung up on the failure…learn from the variance in the hopes of reducing it next time.
Ultimately, you can think of forecasting somewhat like filling out a crossword puzzle. In this example, we’ll say 52 across is a 15 letter word. Maybe the clue for that one gives it away! (Forecast done!) That’s not super likely though. But once you start getting other pieces of information, like 52 down, 53 down, and so on. Next thing you know, you’ll have both a clue and 40% of the letters filled in and it gets a lot easier to figure out the word. The analogy does break down in one key way: no matter how many clues you get in the real world, the odds are very unlikely that you’ll be predicting a correct answer most of the time! If it’s a binary decision like a coin toss, sure…but if it is more akin to the “what are my sales next year” point, then you are just trying to get as many relevant assumptions/variables involved to get as close as you can to what is ultimately an unpredictable outcome.
All this being said, if you follow the process outline above and focus on aligning your predictions with the wider team, you will be on your way to improving both your outcome and the broader perception of your predictions!
View from the Middle 