Insights-Powered Retail: Predicting and Generating Demand

North Highland Insights

Throughout this report are results from a North Highland-sponsored survey conducted in January 2018. More than 200 business leaders in the U.S. and UK were asked about their experience, priorities, and organizational cultures related to data and analytics (D&A).


Key Takeaways

The problem: Several challenges face the retail industry today, including shifting customer expectations, rising transportation costs, and the adverse effects of tariffs. Up against these obstacles, retail leaders must simultaneously drive top and bottom-line growth.

The analysis: As a core capability, advanced analytics can help C-suite leaders move the needle on top and bottom-line growth by driving results in predicting demand and generating demand.

The solution: In establishing your advanced analytics capability to move from assets to insights, North Highland recommends the following approach:

  1. Engage with organizational visionaries who understand data and know how it can be applied to answer complex business questions.
  2. Design and deploy the platforms and tools that empower employees to use data to solve business challenges.
  3. Focus on data governance to foster enterprise-wide data management best practices and encourage trusted insights-driven decision-making.
  4. Apply Agile methodologies to enable the design of analytics solutions that are more responsive to user input.

The retail landscape of today looks nothing like that of yesterday. Retailers are facing rapidly evolving customer expectations, higher transportation costs, rising employee wages, and the potential adverse effects of tariffs, which together put a damper on industry optimism.

In this environment, regardless of their area of expertise, members of the C-Suite are simultaneously focused on increasing the topline and the bottom-line. Efforts to do so are centered on increasing revenue, lowering cost to operate, and/or reducing risk (i.e., risk to stock outs). One of the most effective pathways to realizing these goals? Advanced analytics.

Analytics is nothing new in retail. The industry is rich in data, and organizations have been doing predictive work for years to manage inventory, better understand customers, and improve the supply chain. But applications of analytics that were cutting-edge just a few years ago are now little more than table stakes.

The field of analytics has advanced exponentially over the past several years, as artificial intelligence (AI) and machine learning have enabled dynamic models that apply automatic, real-time adjustments to fast-track retailers’ response rates. These systems have increased speed-to-insight, enabling organizations to rapidly act on changes in the market and experience faster return on their investments. As a result, machine learning has become a top priority for 45 percent of retail companies undergoing digital transformation.2 Leveraging these advances in analytics allows companies to more quickly differentiate themselves from their competitors. Like never before, advanced analytics and models can help retailers achieve C-Suite goals by driving results in two key areas: (1) predicting demand and (2) generating demand.

“The implications of AI and the rise of the algorithm have never been more relevant, more important, or more complex. What is irrefutable: tech’s newest wave is upon us, and there’s no going back.” 1

Accomplishing More With Analytics

Today’s analytics programs eclipse the capabilities that some retailers are familiar with. They’re no longer just producing better business outcomes – artificial intelligence and machine learning are enabling companies to do more for less. They’re improving the entire analytics process, uncovering insights faster, and reducing program costs. Demand planning costs 20 times less than it used to, Forbes estimates,3 and companies that adopt data-driven marketing are six times more likely to be profitable year-over-year.4 Retailers no longer need an army of engineers to setup and maintain hardware, and software, and the tech platforms, hardware and maintenance are less expensive, with costs expected to continue to drop by eight percent over the next four years.5 Those cost savings are compounded by an influx of data – consumer and operational – almost all of which has the potential to measure and forecast outcomes for nearly every retail business process.6


In the following piece, we provide the considerations and associated actions for retailers ready to adapt and advance their analytics capabilities for market leadership.

1. Accelerate Growth Through Demand Prediction

In the past, retailers often predicted demand by taking last year’s results and increasing targets by several percentage points that business leaders felt were reasonable to achieve. Analytics may have been used to find likely outcomes for “what-if” scenarios across a number of market factors, but advanced analytics has largely been kept on the sidelines when it comes to forecasting and demand prediction.

With the advances in technology and an ever-growing cache of quality public and private data, retailers now have a multitude of building blocks at their disposal, which can be creatively assembled to predict customer behavior and better align inventories to match. Instead of simply answering questions about the impact of seasonality on sales, for example, retailers can now simultaneously factor in seasonality, promotions, in-store signage, weather, demographics, psychographics, and even industry-specific data such as birth rates, housing starts, and new car purchases (where applicable). Creative business thinkers that work with advanced analytics talent can develop and refine predictive models that surpass their rivals and gain a true competitive edge. And not just the static, short-lifespan models of yesteryear, but dynamic models that automatically adjust to conditions on the ground, resulting in more accurate and faster outputs.

For example, a major automotive parts company uses public vehicle registration data to predict how demand for specific parts is likely to change at individual stores. Armed with this data and advanced analytics, the company can stock its stores with the correct parts for its unique customer base.


Applying Advanced Analytics to Weather Demand Spikes

To help a national mass retailer better predict demand generated by weather events, North Highland developed a predictive model that drew from realtime and historical NOAA weather data, landscape characteristics from the National Land Cover Database, and census information to provide highly accurate predictions. Pre and post-event demand spikes could be identified for individual stores, products, and lengths of time, and 90 percent of the model’s post-event predictions are within $150 daily average error. The end result: the retailer can make the products that customers need available during storms, reduce supply chain disruption during weather events, and increase revenue. The retailer can also increase customer loyalty by supporting their needs during trying events.


2. Unveiling New Opportunities to Generate Demand

One of the best ways to generate demand and drive revenue is to meet customers’ expectations at the moment they’re ready to buy. When evaluating purchasing decisions, shoppers want to understand the product and assess its fit for their purposes. Understanding these expectations can reduce buying anxiety and lower the number of customers who put the product back on the shelf or abandon the shopping cart altogether.
But how do we know what those customer expectations are? They’re different for nearly every product. With the richness of data available and creative people attacking the problem, the combination of information that people are looking for in a particular product category begins to unravel. Retailers can answer questions such as, “Did a customer buy or not because of cost?” “Was the right information available on packaging and/or online?” “Did the customer understand how to use or apply the product?” and “What combination of these factors together are relevant?”

Online shopping can be one of the worst places for buyer anxiety, yet it can be one of the best places to improve sales due to the multitude of content retailers can provide. It’s rare to find content like how-to videos, for example, at brick-and-mortar locations, but they’re prevalent online. If customer reviews, close-up photos, or instructional manuals are the key to selling a product, it’s easy to include as many as customers need to convert. Leveraging analytics to dig deep into factors that are most important to customer conversion can dramatically add to the bottom-line.


Automated Digital Assets

A major U.S. retailer needed to identify the right mix of assets to feature on its product information pages (PIPs) to more efficiently optimize the quantity and quality of images, videos, copy, and rich content to display on each PIP. North Highland worked with the client’s analytics team to define an actionable, appropriate, and repeatable methodology for its analysts to discover the effects of variations in digital assets and define KPIs. We built processes for collecting and mining data, conducted analysis, and built models for predicting and reporting on PIP performance after implementing changes driven by the analytic insights. Together, we helped the client realize more than 100 times the incremental revenue from improved purchase conversion, and drove a 15-20 percent lift in conversion on pages touched.


From Assets to Insights: Start Here to Advance Your Analytics

Retailers, particularly in the business-to-consumer space, are in an excellent position to make use of advanced analytics. Few industries have such granular information about their sales—down to the individual SKU, store, and minute of a purchase, as well as the combination of products purchased together in the basket. In online transactions, customer identity is known in every interaction, and browsing behavior before, during, and after purchase provides rich insight into who is purchasing. Combining these two data sources and supplementing them with an endless variety of publicly available data sets the stage for powerful outcomes.

But how do organizations mature their analytics capabilities? What people, process, and technology changes are needed to create a high-performing team capable of advanced analytics? This is a topic all its own, as we cover in our white paper on Taking the Lead with Analytics, but there are four main components that every organization needs in place to first take advantage of data:

1. Engage with business-focused visionaries that understand data and know how to apply it to answer complex questions.

Charge the visionary with leading the organization through a measured process to gain acceptance for advanced analytics and adapt accordingly as your business changes. And they have to be willing to start small. Advanced analytics is the tip of the triangle for your analytics function, and organizations must have a mature data system in order to fully optimize it.

2. Implement platforms and tools that enable your employees to better capture and leverage data.

Capture data from internal customers and throughout development to enable insights that are relevant to the decisions they make and challenges they face. This will allow internal customers to more rapidly gain the information they need, while reducing the time and expense spent on ineffective iterations of your analytics program. In turn, it lowers your total cost of ownership. Focus on tools that are simple in design and effective in problem solving to reorient internal culture around data. Assign decision-makers with data support roles and responsibilities.

3. Enact data governance programs that enable clean, high-quality data for use by the business and double down on master data management.

For analytics to enable data-driven decision making, it’s important to start with clean, trusted data. The first step to realizing this in your organization is to establish a single source of truth that all stakeholders can trust. According to our January 2018 survey, 96 percent of analytics leaders have strong data quality processes in place, and 86 percent have an enterprise-wide platform for managing data.

4. Use agile methodologies to design analytics systems. 

In our survey, we also identified the characteristics that separate analytics leaders and laggards. Leaders overwhelmingly applied Agile ways of working to design and manage their analytics systems. Specifically, 93 percent of leaders said that user input informed the design of solutions and 79 percent of leaders noted that their culture encouraged distributed decision-making.

The Retail Imperative: Predict Demand and Drive Revenue Automatically and Instantaneously

The time to act is now. Analytics business-as-usual aren’t enough anymore to capture market leadership while leaders such as Target, Walmart, and more leverage advanced tools to redefine the playing field. Opportunities await retailers that systematically and holistically evaluate their analytics abilities across people, process, and technology to tackle advanced analytics in ways that predict demand and drive revenue. By following these steps, retailers can lay the groundwork for data-informed decision making that enables long-term success.


1 “5 lessons of the AI imperative, from Netflix to Spotify,” Fast Company, Sept. 11, 2018

2 “Retail and CPG: Human Amplification in the Enterprise,” Infosys, 2017

3 “The Future of Retail is Cheaper Than It’s Present,” Forbes, May 14, 2018

4 “The State of Data-Driven Marketing 2018,” V12 Data, March 5, 2018

5 “Juniper Research: Retailer Spending on AI to Grow Nearly Fourfold, Reaching $7.3 Billion by 2022,” Juniper Research, Jan. 31, 2018

6 “Predictive Analytics and Machine Learning AI in the Retail Supply Chain,” Forbes, Sept. 12, 2017