Should you use a Model or an Expert for your next forecasting? Cast differently, if you read between the lines: data or intuition?
It is actually not a simple question: industry experts often have decades of experience and what is too quickly trivialized as “intuition” is really “deep professional know-how developed through training, practice, and resolution of countless challenges during the past several decades, that ended up building an unprecedented understanding of the industry”. A tall order for whoever tries to substitute this. However, it does not mean that the experts are perfect — as human beings we cannot escape Seneca diagnosis: “errare humanum est”, to err is human — and therefore, it means that the experts can be (should be!) assisted. The typical reasons listed to justify using AI to assist experts are: (i) explosion in the amount of data, (ii) explosion in the variety of data available, (iii) increasing speed of information (recognized the classical three V’s of big data already?), and (iv) increasing connectivity between industries. In addition to those, we want to point at two other reasons why companies should develop forecasting models to assist their experts. We have found these reasons to be incredibly ubiquitous across industries, deeply connected with the reality on the ground. To every Executive who has tried to make a case for change in a company, these reasons will appear more tangible and probably more actionable than the theoretical 3Vs:
Models don’t walk out the door:What would happen if your best forecaster suddenly decided to leave your company? And worse, decided to go work for your competition?
That is the main fear of every company, small and large. If you rely entirely on your experts to produce forecasts, you are running a company without backup. It is like living your life without insurance: everything runs smoothly while it lasts. But of course, things happen and we must preemptively prepare for them. Forecasting models give you this insurance. Recent techniques, from neural networks to more interpretable decision trees, have become extremely good at quickly predicting outcomes with expert accuracy levels. Of course, these techniques need to be operated properly, which is why they are assisting the experts rather than replacing them. But the day your expert walks out the door and is replaced by a junior person, the body of knowledge you lose is counterbalanced by the forecasting capabilities that remain. No question that it is still a blow to your company, but at least not a deadly one, and companies usually recover very fast. In other words, your company has become agile, reactive, and competitive!
Models are more transparent and accountable: How many times did you end up in endless discussions simply because participants were pushing their intuitions by self-selecting arguments?
The origin of this discord is confirmation bias, by which we look for facts that suit our theories. The use of models, instead, bypasses this problem entirely: results are not the output of an opinion anymore but are based on a transparent process for everyone to see and test. If participants end up disagreeing, it is usually on the assumptions rather than on the process. The use of models is therefore a powerful way of refocusing discussions in meetings around what is most fundamentals: assumptions, rather than interpretation. As a participant in a meeting, you cannot just say that you disagree with a result and propose an alternative (which sparks more meetings, more debates, and invites additional opinions – spiraling into these endless discussions), but you have to precisely point at what you disagree with in the model. Unless you are a data analyst, this refocuses your attention back onto the fundamental information that fed the model, which is often directly related with the operation of your company. If there is a disagreement on this, it is certainly worth spending the time to clarify with your team!
While working with several companies, we have found these two reasons to be standing out as most clearly highlighting the value add of data analytics. Of course, we all agree that models are also faster and more powerful than our brains, but that is a given and not a basis for a competitive advantage anymore. However, knowing that you can become more agile, more competitive, and better focused as a company should help you sleep at night – if you already made the move to embrace data analytics of course.