Shelf image recognition technology has become an essential tool for retailers and suppliers, with space planners, category managers, and other key functions utilizing the technology’s power to drive operational improvement. The effects of the technology can be impactful downstream, too, for buyers and supply-chain professionals.

In this feature, The Space Planning Community sets out to not only highlight the present state of shelf image recognition technology usage, but also to share insights on how our industry can potentially leverage this powerful tech to even greater effect.

We will begin with a look at today’s landscape, then dive right into what’s ahead – and how space planners and others in our industry could greatly benefit from evolving along with the tools.

How is the technology currently being used?

On a most basic level, shelf image recognition technology involves the real-time electronic scanning of shelves through cameras, whether via smartphones, robots, computers, or fixed cameras within stores. The data is then fed into systems powered by artificial intelligence and employing machine-learning techniques to identify business opportunities and gaps on the shelf.

Areas in which shelf image recognition technology can help include:

On-shelf availability / out-of-stocks
Planogram compliance
Pricing compliance
Validation of displays

Out-of-stocks loom as a particularly vexing issue for retailers and CPGs.

According to SymphonyAI, which cited data from IHL Group, out-of-stocks cost North American retailers more than $140 billion, with the opportunity in the Asia/Pacific region exceeding $500 billion.

According to experts interviewed by The Space Planning Community, shelf image recognition technology’s biggest initial lift is solving targeted problems, such as in-store execution.

“I would say right now, [the use of the technology] is usually focused in a singular business area,” said Becky Gilman, product marketing director at SymphonyAI, where she works with retail and CPG clients. “We’ve seen usually it’s focused more in-store operations. And that’s where they’re going to see the immediate results.”

Mark Abernathy, chief commercial officer at Pensa Systems, also believes that shelf image technology is used primarily in a focused manner, to address a stated business problem.

“The way I would think about current use of shelf image technology is, people are trying to address specific business issues, and they’re looking for a tool that they can plug in to solve these specific things. One example is in-store execution. There are others on the space planning side of the equation, whether it’s monitoring trade fund spend or looking at space to sales,”

Abernathy said. “Historically it’s been looked at as individual pieces because until more recently, there’s not been a broad understanding or adoption of image recognition technology. That’s changing.” Abernathy gave two recent examples of how CVAI technology (computer vision and AI) quickly solved problems for CPG manufacturers.

“We did a study with one of our CPG clients looking at the field teams that they send out to stores and all of the questions that they ask them to answer while they’re in store. ‘Is this item full? Does it have three facings? Is this new product innovation there?’ ” Abernathy said. “We took all of the questions that they served up and answered 70% of those automatically through AI. So, you don’t have to manually do that anymore. Off you go. We’re going to do it with AI. And not only are we going to automate the effort with AI, we’re going to deliver more accurate data through AI.

“Another CPG client of ours did a very in-depth across the ecosystem study. And they said, if we look at the accuracy of results that we get through AI versus the accuracy of results we get from a person in the store doing things manually, what’s the difference in accuracy due to human error, time constraints and conflicting priorities? They found between 50-90% more accurate results by leveraging AI.”

For CPGs like Kraft Heinz, shelf image recognition technology allows for a view into what’s actually occurring on shelf in ways that other data simply cannot.

“The simplest way I could describe the value is that we have an opportunity to increase our understanding of where most of our business is being done at the shelf.,” said Andy McKeon, director of omnichannel Business Development at Kraft Heinz. “We want to see on an aggregated level, what is our consumer seeing, when they’re at that pivotal moment of truth or moment of purchase. That’s the simple thesis.”

How should the technology be leveraged?

It’s clear shelf image recognition technology can be used to great effect to identify and address the shelf issues that have vexed retailers and suppliers for generations.

The question is, how can the technology be utilized to drive even greater results? Below are some suggestions from our panelists.

Suggestion No. 1: Shelf image recognition technology should be aligned with a company’s overall strategy, with AI as an enabler.

Pensa’s Abernathy believes any technology discussion starts with a retailer/CPG understanding how it can best use AI overall.

“Where we see real success is where you actually have an AI strategy as a CPG or retailer,” Abernathy said. “It’s a very, very different thing than solving for an individual use case. And there’s value there for people all through the supply chain, retailers and CPGs, as they look at category management and in-store execution.

“When you think about implementing CVAI, one of the traditional barriers to adoption has been that in space planning, you have a lot of imputed data that’s out there. We know it’s not accurate, but it’s existing ‘data’ and you can use it. And typically there’s a limited budget available for doing something different to improve business results. So, how do you improve but not add to your budget as a space planner or category manager? But when you think about an actual AI shelf intelligence strategy as a CPG, now that path to improvement is much clearer with a few well-defined steps that are fairly straightforward and deliver the ROI necessary to fund the change.”

Added Abernathy: "If you’re a leader in the CPG space, if you don’t have a broader AI shelf intelligence strategy that runs out three in five years, you’re way behind at this point.”

Suggestion No. 2: Getting space planners even more involved in the technology usage is essential, as well as having space planners at the table in a consultative way on key shelf decisions.

Symphony’s Gilman sees space as the next frontier – no pun intended – in unleashing the full power of shelf image recognition technology.

“I think (shelf image technology) is underused by space planners, because where we see most of the use is on the store-operation side of things on the execution side,” Gilman said.” And I think that’s phenomenal, and I think retailers and suppliers get amazing results, real-time results, just from filling real-time out-of-stocks or adjusting pricing or adjusting facings that have shifted and things of that nature. But I think where there’s a huge opportunity in the market is to take those insights and pull them back upstream to space planning, but also likewise to supply chain planning as well.”

We’ll circle back to Gilman’s point on supply chain planning later in our feature.

Suggestion No. 3: Shelf image recognition technology can provide meaningful insights leveraged throughout a supplier or retailer organization.

Here, Kraft Heinz’s McKeon shares a powerful example.

“I stumbled upon a category at a retailer for a brand we knew was in distribution and was not being recognized on a couple of retailer shelves,” McKeon said. “Obviously, this was an error. We knew that we have multiple SKUs in distribution.

“But when you broke up the data, the weeks that the shelf pictures (didn’t have the brand) were corresponded to heavy promotional events that we were running. And you could see a very clear pattern.

“I immediately brought that to our Sales team, and they showed the pictures and data to the customer, saying, ‘People are coming to buy the category, and there’s not enough holding capacity on the shelf.’

“What’s amazing, too, is that you literally just need to show one picture of it. Hey, here’s the data. And here’s what I’m talking about, where there’s no product on the shelf.”

The Next Level

So what does the future hold for shelf image recognition technology? In speaking to experts, two areas jumped out.

The first is in the non-UPC part of the store: Fresh, which is a such an important driver for food retailers.

“One thing that I do see that is interesting is [potential for] image recognition for the Fresh area,” Symphony’s Gilman said. “Fresh is kind of this zone that are really hard for those traditional technologies to break through. We’re working really hard on that.

“That’s a whole other set of business cases too, with Fresh. Like expiration: what does the food look like? How can I tell the difference between pineapple spears and mango spears? They look exactly alike. How can we take this technology that’s really exciting and that we’re really successful with for the rest of the store and get it into the Fresh side as well?”

Should shelf image recognition technology begin to have a major impact in Fresh, the benefits could be tremendous for retailers around inventory management as well prescribing activities to maximize sales.

Longer-term, the broader opportunity for shelf image recognition technology — the most exciting one, perhaps — is quicker decision making for retailers and suppliers due to greater linkage throughout the supply chain.

Here, Pensa’s Abernathy gave a hypothetical example of what this could mean.

“In the world that we live in today, because of where we’re at on the journey, we’re building a shelf insights dataset as we adopt,” Abernathy said. “But in a future state, there’s readily available broad and dense shelf data everywhere."

“When you think about things like supply chain — when you know instantly at any store, in any category, by any SKU, what’s happening on the real shelf, then you change how you plan and how you demand forecast and how you manufacture upstream. Historically, shelf data would have a week, two weeks, three-week lag time depending on point-of-sale data. Now you know that specific store on the East Coast was out of a product three hours ago, and you’re able to dynamically change your production.

“As you think about the longer-term implications, broad and dense shelf data to drive the manufacturing process is 100% in the mix. And it’s a big value unlock both for the CPGs from a manufacturing standpoint, for the retailer from a sales standpoint and of course for the consumer from a fulfillment standpoint. Everybody wins.”

Symphony’s Gilman had a similar view on the potential power of shelf image recognition technology.

“Where we see it going as a company is that the insights that we could get from shelf recognition will be pervasive throughout the business, that it’s not only going to be store operations, that these insights will also provide recommendations through supply chain, through space planning, merchandising, assortment, promotions, and that everything, almost like a central pillar of real-time analytics that can help retailers and their employees just make much better decisions faster than ever,” Gilman said. “All of the pieces are there. The technology is there. The AI is there. Now we just have to get people to get on board with this idea that all the connectedness of everything is only what makes things better at the end of the day.”