Current Issues

Regular Industry Development Updates, Opinions and Talking Points relating to Manufacturing, the Supply Chain and Logistics.

Moving away from category-based models for accurate promotion forecasting

06-Mar-2018
Mikko Kärkkäinen (pictured)
Group CEO, RELEX Solutions
Moving away from category-based models for accurate promotion forecasting
The importance of sales promotions as the drivers of consumer behaviour has grown during recent decades and a significant proportion of all retail sales are made as a result. The increased usage of digital marketing with personalised, time-dependent offers has only served to amplify this phenomenon. It is therefore inevitable that the management and forecasting of promotional activities must keep up apace with this trend. The challenge lies in that retailers’ traditional category-based models for forecasting and planning promotions are no longer sophisticated enough to cope.

Traditionally, forecasting for promotions has been based on categorising products into distinct groups based on their characteristics: ie what price reduction the promotion had, what kind of in-store display was used and what kind of marketing activities were executed during the promotion. The forecast for the upcoming promotion would then be based on the average sales performance of the group of promotions with similar characteristics.

While this method is sophisticated enough in many ways, it also has significant drawbacks which are mainly based on assumed data correlation, in itself necessary for the model to work.

Category-based modelling works on identifying groupings and responses around product categories or location: bread can have a different response to promotional activities than confectionery, and small stores can have different responses than large stores. Existing best in class promotion forecasting solutions also work on intelligently searching for an appropriate grouping when there is no exactly matching data available. For example, a lack of historical data for similar promotions with the same characteristics for a certain SKU-store combination, or a statistically insignificant number of matching promotions.

However, the shortcomings with this forecasting model lie in the way categorisation has to be used to statistically generalise what are actually highly specific and non-linear correlations in the data.
  • The limitations of categorising: Let’s say the price reductions of promotions are categorised into categories of “0-20%”, “20-40%” and “over 40%”. This leads to a situation where the forecasted effects of price reductions of 21% and 39% are equal, even though they probably should be closer to the effects of categories “0-20%” and “over 40%” respectively. However, defining more fine-grained categories reduces the reliability and accuracy of the promotion forecasts, as the amount of data behind each distinct category is reduced. On the other hand, adding manual rules unnecessarily increases the complexity of a retailer’s promotion forecasting setup and makes it a lot harder to manage. 
  • Utilisation of the data: Using only exactly matching reference promotions (e.g. same price reduction category, same in-store display and same marketing activities) as input in the promotion forecasting model neglects a significant amount of data that partially matches the upcoming promotion (e.g. different price reduction category, but same in-store display and same marketing activities) and that would be valuable information for promotion forecasting. 
  • Big data isn’t always meaningful data: Not too seldom when going through the promotion master data there are promotions that belong to, for example, more than 10 categories which all are ‘required’ to be utilised in forecasting: we live in the big data era after all. However, big data doesn’t always mean meaningful data: It can be that only three or four of those categories contribute to sales performance, or that the 10 categories could be combined into smaller numbers of categories for forecasting purposes. With the category-based approach it’s not instantly clear which categories are relevant – and which irrelevant - in terms of forecasting. 
  • Simultaneous promotions with different categorisations: Promotion data can include promotions that overlap with each other which makes it difficult to say which part of the promotion performance is achieved by which promotion and category combination. 

So how can retailers achieve more sophisticated and accurate models for forecasting for promotions?

Causal modeling is a methodology that can overcome some of these drawbacks. These methods are designed to extract the cause-effect relationships between certain group of variables, ie ‘explaining’ the data as well as possible with given parameters. So, when forecasting promotional sales using causal methods, instead of asking “How have similar promotions performed in the past?”, one could ask “What’s the effect of a 24% price discount on demand; what’s the effect of a special display; or both of those combined?”.

In more detail, causal methods tackle the identified drawbacks by:
  • Utilising continuous variables instead of category variables: Establishing a continuous relationship between e.g. the price decrease and demand allows a retailer to estimate more precisely what’s the discount’s effect on demand. 
  • Separating the effects from one another: Causal methods can shed light on the estimated effects of individual variables and combinations of those variables, and thus drawing insights from the whole data set, not only from exactly similar promotions. 
  • Separating meaningful data from the noise: Examining the results of causal methods, and automating the analysis of results allows a retailer to determine which variables have a significant impact on the promotional sales, which don’t, and use that information in forecasting. 

Increased accuracy
Utilising these advanced models in promotion forecasting and benchmarking them against the previous Historical Averages model has revealed significant benefits: forecast error decreases up to 15 percentage points on average, and forecasts are substantially stabilised, meaning that the number of high error peaks decreases significantly.

Better planning
The added value isn’t only about forecasting more accurately – the promotion planning process can also be improved by using causal methods: knowing the effect of individual promotion characteristics to the demand of each individual SKU-store combination allows planning to be done in a more refined way by concentrating the promotional activities to where it matters the most. For example, there’s no point in adding heavy discounts or marketing activities to SKUs that respond poorly to price decreases or advertising. Turning that setting upside down, it’s possible to optimise the price decrease and promotional activities to meet certain sales targets – e.g. in case of markdown sales, where the target is to sell out the existing inventory. These activities help to increase the efficiency of the promotion planning and managing process, and ultimately to increase profit.

Best practices for forecasting promotions

When implementing a promotion forecasting process the golden rule of ‘garbage in, garbage out’ applies. When utilising more sophisticated promotion forecasting models, it is even more true.

A good first step is therefore to ensure that the promotion master data is in good shape: promotion prices are correct and promotion characteristics are thoroughly and systematically set up. This isn’t as self-evident as it sounds: Manual typos are quite typical in promotion master data, e.g. a promotion price of 49.9 instead of 4.99 and tv marketing instead of tv-marketing. Cleaning the master data really pays off when automating the promotion forecasting process.

The second step is to establish a process where causal promotion forecasting models are estimated on multiple levels and the most appropriate of them is chosen. The acceptance criteria should consider at least the reliability of the estimates as well as the granularity of the data: usually the lower the level of the estimation, the better, meaning for example that in the typical retail setting, the SKU-store level yields the most accurate results. The promotion forecasting process should also be automated; nobody wants to spend countless hours going through the different models on different levels for possibly millions of SKU-store combinations.

Once the promotion forecasting process has been satisfyingly implemented, the results can also be extended beyond forecasting – start planning the promotional activities operations based on the results of the automated analysis.

Conclusion
By breaking down the promotional sales into price component, user-defined promotional activity components and non-promotional sales components, and enabling the analysis and forecasting of the impact of each of the components on SKU-store level, causal modeling has proven to be significantly more accurate than traditional category-based promotion forecasting, increasing a retailer’s margins and profitability levels.

For promotions forecasting to move in line with today’s rapid pace of promotional activity, it’s time to move away from historical, aggregated data models and utitlise the power of big data analytics to its full advantage.

The Advantages of an Automated Artwork Management Solution
23 hours ago
Half of English council spending goes to a handful of suppliers, risking another Carillion-sized failure
5 days ago
The Time to Modernize Your Artwork Management is Now
5 days ago
It’s time to bin receipts: ParcelHero calls for an end to wasteful paper receipts
6 days ago
Ocado sales jump thanks to 'automation done right'
1 week ago
Preparing your employees for Industry 4.0
1 week ago
How Do Product and Shipping Labels Affect Brand Visibility?
1 week ago
How will Brexit affect the events industry?
1 week ago
100 days to Christmas - ParcelCompare says it’s never too early to start planning those presents and orders
1 week ago
It’s time to blow up your business model and switch to DTC
1 week ago
How retailers can survive the death of the high street
1 week ago
ClickSoftware Announces Recent Customer Wins, Product Updates and Industry Accolades
1 week ago
Deloitte increases its focus on AI & Marketing Technology through new acquisition
2 weeks ago
How Centralized Labeling Helps QA Departments
2 weeks ago
How IoT is Changing the Waste Water Industry
2 weeks ago
M&A can put quality compliance at risk
2 weeks ago
Chinese traders must pay fair postal charges or more British retailers will collapse
2 weeks ago
Enhance real-time visibility across your operation with a proof of delivery system
3 weeks ago
Snapfulfil publishes top five tips for winning at warehousing
3 weeks ago
Government’s no-deal Brexit advice papers ‘are the equivalent of “duck and cover” nuclear warnings’
1 month ago
Brexit and Stockpiling: What’s the answer?
1 month ago
Peak? Which peak?
1 month ago
Empowering Your Supply Chain Workforce: 3 Tips for Manufacturing Directors
1 month ago
64 per cent of manufacturers rate their shop-floor IT capabilities as just average, weak or non-existent
1 month ago
The supply chain lessons of the modern grocer
1 month ago
Time to mitigate the risk of a ‘no-deal’ Brexit?
1 month ago
App Insights: How can artificial intelligence innovate supply chain management?
1 month ago
Managing Technological Change Without any Supply Chain Disruption
1 month ago
Moody’s Blues for House of Fraser; but Website Failings are Just as Damaging
1 month ago
Artificial Intelligence: What’s Now And Next In IoT-driven Supply Chain Innovation
1 month ago
Indiana Jones and the supply chain bullwhip effect
1 month ago
How Enterprise Labeling Helps You Conquer Regulatory Challenges
1 month ago
3 Tips for Empowering Millennial Warehouse Workers
1 month ago
More robots means easier delivery - but what about the infrastructure?
2 months ago
Safety in the warehouse: Five key risk areas for 360-degree protection
2 months ago
Automotive resilience, not just airbags and crash tests, but IT too
2 months ago
New High Street panel is facing a race against a terminal diagnosis
2 months ago
Delivering the supply chain revolution
2 months ago
A market of one: the potential of the personalized supply chain
2 months ago
5 EIS rules you are probably not aware of
2 months ago
TMS: an enterprise priority
2 months ago
Labeling Solutions for Global Expansion
2 months ago
Prime Day will be a $1.6bn loss-leader creating millions more Prime members, says e-commerce expert
2 months ago
Say what? The new possibilities of voice and machine driven retail
2 months ago
‘Peak faith’ in the cloud only 12-18 months away with slump to follow, warns 6point6 Cloud Gateway
2 months ago
As you were for UK manufacturing sector, but new business growth at three month high
2 months ago
Back to IoT basics: Automating outcomes to improve customer experience
2 months ago
Investigation of the "Fast Fashion" impact
3 months ago
Redefining Rental for Materials Handling Equipment
3 months ago