Data Science & Human Knowledge: A Powerful Formula for Forecasting Demand

At the dawn of retail forecasting, merchandisers were reliant on “gut feel” to make decisions - how they felt a product would sell likely based on what they would see in the market. In today’s retail setting, if your company relies on merchandisers or planners having to use their “gut feel” when creating a product’s forecast then you are no more scientific than a guess.

Retailers became more sophisticated and started recording products’ sales history which enabled historical performance to be a guide for future performance. Product hierarchies and attributes such as category, size, color, etc. were then included to build a more detailed understanding of how products of certain characteristics perform relative to one another.

Leading forecasting practices now involve data science and machine learning to automatically calculate forecasts based on numerous quantitative variables (price, number of selling locations and competitors' price are key examples) and numerous qualitative variables (such as the weather, marketing support, and channel) – being able to account for such a big data set containing multiple variables increases forecast accuracy by up to 20%pts. The complexity and time intensity of forecasting demand means the accuracy benefit can only be unlocked using data science.

Human knowledge is not without purpose.

Despite the huge capabilities data science has, it still requires enhancements made by human knowledge. The validation of the data science must be performed by a human – formulaic calculations can only be as good as the data that feeds it and relying on machine learning to mark its own work is risky.

Data science is most successful when forecasting repeated behavior; a weakness of data science is an inability to plan for the unknown. Human knowledge is required to amend the data science outputs to account for one-off events such as the Queen’s diamond jubilee which may cause a halo effect for certain products. New trends created by social media or influencers are another example where human intervention is required to provide insight that data science cannot capture on its own.

At TPC, we believe a combination of data science and human expertise creates the highest forecast accuracy. We work with retailers to ensure they are maximizing the capability of their forecast software configuration and supporting business processes. Our latest client saw a 16%pt forecast accuracy improvement which drove over £20million of revenue growth. To find out more about our services, contact us via our website or LinkedIn.