Computer Vision in Shopping Malls: 3 Emerging Use Cases
In 2012, Aileen Lee, then a partner at Kleiner Perkins, gave some advice that perfectly encapsulated the coming decade in retail: “We are just at the beginning of a revolution of e-commerce, and existing retailers are going to have to get better at personalizing the experience for consumers.”
Since then, ecommerce has jumped from accounting for about 7% of all retail sales (excluding restaurant, auto, and gas sales) to 17.2% in 2021, with some projections indicating that it will hit 23.4% in 2023.
Without a doubt, some of this growth has been driven by the surgical precision with which modern tracking algorithms can serve up personalized ads. Nowadays, advertisers can target exactly the right people at just the right time.
While the online shopping world has undergone a personalization and data revolution, brick and mortar retail has remained largely untouched by these forces—capturing useful, granular data has simply been too difficult in the physical world. Personalization and optimization have yet to go beyond shopkeepers knowing their best customers and making intuitive decisions about their store layout.
But that’s all beginning to change. Computer vision (CV), a technology that uses artificial intelligence to interpret visual data, is beginning to offer retailers and shopping mall operators the ability to collect data that can be used to optimize operations and create personalized experiences for their customers.
Ready to update your retail operations? Learn more about how curbFlow can help with people counting and heatmapping.
What Is Computer Vision?
Computer vision is an emerging technology that uses artificial intelligence to interpret visual data. It’s essentially a way to give a computer eyes so that it can interact with the world in ways it couldn’t before.
If you’ve ever smiled at a robot rolling around a supermarket looking for spills or driven through a toll that snapped a photo of your license plate and mailed you a bill, you’ve already seen CV in action. Indeed, computer vision technology is already making an impact on many facets of daily life, like parking, retail, and restaurants.
Although the technical workings of CV are complex, the overall concept is fairly straightforward. First, a camera is used to collect visual data in the form of videos or pictures—usually in real time. Then, that data is fed into a machine learning or deep learning model, which parses the objects in the data in a process called segmentation. From there, a user can employ additional algorithms to instruct the model to look for important relationships between the objects it’s parsed, such as where the highest crowd density is or whether a parking spot is empty or taken.
Computer Vision in Malls: 3 Use Cases
Computer vision is still a new technology, and we have likely only begun to scratch the surface of possible use cases. That said, here are three ways you can use computer vision in malls.
Personalized Recommendations and Shopping Experiences
Personalization has been a driver of growth for online retail, but its implementation has lagged behind in the brick-and-mortar world. While small, local businesses can offer arguably the best personalization of all—a friendly face that knows you and helps you find what you need—those types of experiences are not feasible at scale.
Instead, retailers and mall operators need to extract elements of that archetypal personalized shopping experience and repurpose them for use by advanced technologies. For example, even if it’s not realistic to have mall staff greet everyone at the entrances and remember every face they see, it is feasible for a smart information display panel to recognize returning guests, thank them for coming back, and give them personalized recommendations when they approach the panel.
In fact, this technology is already in use. I-MALL, a computer vision system developed by researchers at the University of Verona, University of Salerno, University of Palermo, and University of Florence, uses CV to track customer behavior and the customer journey through malls so that it can make personalized recommendations when customers approach information displays.
This could be a gamechanger for in-person retail experiences. Just like a friendly store manager can recognize you when you walk in and tell you about a new version of your favorite product that just came in, a CV system can recognize you when you approach an information panel and recommend stores based on your interests and behaviors. These types of experiences can create a pleasant, welcoming feeling, while also more efficiently directing customers to stores where they’re likely to make a purchase.
The opening of the first Amazon Go store in 2018 represented a paradigm shift in retail. Thanks to developments in facial recognition and geofencing technologies, customers could enjoy the convenience of Amazon’s new “just walk out” payment method, wherein customers simply walk out of the store while the tech deciphers what they purchased and charges them for it.
If this payment method follows the trajectory of self-checkout, then we can expect cashierless stores to become ubiquitous within the next few decades. Not only will it allow retailers to have employees focus on engaging and serving customers, but it will be even quicker than self-checkout, practically eliminating queues.
Although a mall-wide implementation is still something of a pipe dream, it’s not hard to imagine this same technology being used throughout an entire mall: guests are scanned when they enter the mall, which automatically opens tabs that will include all purchases they make during their visits. When they leave, a bill could be sent to them automatically.
On a smaller scale, computer vision can be used to streamline the parking experience. The same technology that is used for automated tolls is being used to facilitate more convenient payment options when leaving gated parking lots: cameras recognize license plates and connect them to user accounts so drivers can pay from their phones instead of looking around for a payment terminal and a cashier is no longer necessary at the lot exit.
Operations and Layout Optimization
While the previous two use cases have direct and obvious effects on the customer experience, computer vision is equally powerful when used behind the scenes for optimization of mall operations and layouts.
Commercial computer vision systems, like curbFlow, offer a variety of data collection capabilities, including:
- Staff engagement: Measures staff-customer engagement to identify when more training or new policies are needed.
- Staff detection: Measures the amount of staff in a given area to determine whether more staff are needed. Adequate staffing is necessary to reduce wait times and provide a smooth customer experience.
- Pedestrian counts/footfall: Measures how many pedestrians pass by the outside of the mall. Can be used to calculate entrance rates and determine whether more at-entrance advertising is needed.
- Vehicle counts: Counts how many vehicles enter an area during different time intervals, such as average vehicles per hour or total vehicles on a given day. Can be used to make better-informed parking management decisions.
- Guest/door counts: Measures the number of people that enter the mall. Data can be provided for individual entrances and averaged for days of the week. When combined with pedestrian and vehicle counts, guest counts can show a more accurate picture of the sales funnel, from pedestrian or vehicle to purchase, with guests in the middle of the funnel. This metric can also be used to assess marketing and operations initiatives.
- Guest journey tracing: Tracks the average trajectory that guests take through the mall. This can be used to make better decisions about store placement based on guest traffic.
- Dwell time: Measures how much time guests spend in a specific location. This can help identify bottlenecks or areas that guests enjoy spending time in, which can be used to inform marketing initiatives.
- Real-time occupancy tracking: Measures how many people are in the mall at any given time. Can be used to benchmark average time of visit per guest, adjust staff resources based on occupancy trends, and monitor occupancy for capacity constraints.
- Heatmapping asset utilization: Visually displays which mall assets guests are engaging with the most or spending the most time visiting through a color-coded heatmap. Can be used to inform mall layout decision making.
- Parking management: Tracks parking lot occupancy for parking operations adjustments.
Key Takeaways: Computer Vision for Shopping Malls
New AI technologies are giving retail businesses new ways to create personalized customer experiences, provide more convenient payment options, and optimize layouts and operations.
Want to get a deeper look at your mall operations? Let curbFlow help you get started with door counting.