Computer Vision: Coming to a Store Near You
•Télécharger en tant que PPTX, PDF•
1 j'aime•954 vues
Background: Some early applications of Computer Vision in Retail arose from e-commerce use cases - but increasingly, it is being used in physical stores in a variety of new and exciting ways, such as: ● Optimizing merchandising execution, in-stocks and sell-thru ● Enhancing operational efficiencies, enable real-time customer engagement ● Enhancing loss prevention capabilities, response time ● Creating frictionless experiences for shoppers Abstract: This talk will cover the use of Computer Vision in Retail, the implications to the broader Consumer Goods industry and share business drivers, use cases and benefits that are unfolding as an integral component in the remaking of an age-old industry. We will also take a ‘peek under the hood’ of Computer Vision and Deep Learning, sharing technology design principles and skill set profiles to consider before starting your CV journey. Deep learning has matured considerably in the past few years to produce human or superhuman abilities in a variety of computer vision paradigms. We will discuss ways to recognize these paradigms in retail settings, collect and organize data to create actionable outcomes with the new insights and applications that deep learning enables. We will cover the basics of object detection, then move into the advanced processing of images describing the possible ways that a retail store of the near future could operate. Identifying various storefront situations by having a deep learning system attached to a camera stream. Such things as; identifying item stocks on shelves, a shelf in need of organization, or perhaps a wandering customer in need of assistance. We will also cover how to use a computer vision system to automatically track customer purchases to enable a streamlined checkout process, and how deep learning can power plausible wardrobe suggestions based on what a customer is currently wearing or purchasing. Finally, we will cover the various technologies that are powering these applications today. Deep learning tools for research and development. Production tools to distribute that intelligence to an entire inventory of all the cameras situation around a retail location. Tools for exploring and understanding the new data streams produced by the computer vision systems. By the end of this talk, attendees should understand the impact Computer Vision and Deep Learning are having in Consumer Goods, key use cases, techniques and key considerations leaders are exploring and implementing today. Speaker Brent Biddulph, GM, Retail & CG Solutions Hortonworks
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Computer Vision: Coming to a Store Near You
Similaire à Computer Vision: Coming to a Store Near You (20)
Plus de DataWorks Summit
Plus de DataWorks Summit (20)
Dernier
Dernier (20)
Computer Vision: Coming to a Store Near You
- 1. © Cloudera, Inc. All rights reserved. COMPUTER VISION: COMING TO A STORE NEAR YOU Brent Biddulph, Managing Director, Retail & Consumer Goods, Cloudera Florian Muellerklein, Data Scientist, Miner & Kasch
- 2. © Cloudera, Inc. All rights reserved. 2 AGENDA Industry Trends Business Drivers Exemplar Use Cases How it Works Bringing It All Together
- 3. © Cloudera, Inc. All rights reserved. 3 …OF RETAIL SALES WILL STILL OCCUR IN-STORE IN 2021. (STATISTA, 2018)
- 4. © Cloudera, Inc. All rights reserved. 4 Mobile Apps ESLs, Robots Autonomous Vehicles Magic Mirrors+ Shelf, Bin, & Rack Sensors Surveillance CamerasCompetitive PricingSatellite Images Logistics & Asset TelematicsIn Home Devices Frictionless Checkout RFID Streams SINGLE VIEW OF CUSTOMER FRICTIONLESS COMMERCE UBIQUITOUS FULFILLMENT RELEVANT INTERACTIONS OPERATIONAL EFFICIENCY Streaming Capabilities Enable New Insights, Right-Time Response New Data Sources Extend Retail Intelligence = video, image data
- 5. © Cloudera, Inc. All rights reserved. 5 SOLVING FOR KEY RETAIL BUSINESS IMPERATIVES • Enabling Frictionless Commerce • Improving Operational Efficiencies • Improving Customer Experiences • Reducing Fraud & Shrink -5%in annual lost revenues due to EACH store out-of-stocks and fraud for a typical retailer (HBR,, 2017) $800b In AI / IoT economic value-add in retail by 2025, with some of the most valuable use cases will be driven by streaming video analytics. (McKinsey, 2017) 24%of Amazon revenue can be attributed to brick & mortar store out-of-stocks. (IHL, 2018) 50%of digital shoppers are inspired to purchase via an image (Google, Consumer Survey, 2019) COMPUTER VISION
- 6. © Cloudera, Inc. All rights reserved. 6 ENABLING FRICTIONLESS COMMERCE Amazon Go Store • Computer Vision (100s of ceiling mounted cameras) • Sensor Fusion (in-shelf cameras, scales) • Deep Learning (advanced pattern recognition, 100s of algorithm’s)
- 7. © Cloudera, Inc. All rights reserved. 7 ENABLING FRICTIONLESS COMMERCE Alibaba’s HEMA store • Robots delivery hot prepared foods tableside, ESLs, and overhead conveyor belts for fulfillment delivery • Using facial recognition customers can pay using their face - no QR code nor credit card swipe required
- 8. © Cloudera, Inc. All rights reserved. 8 IMPROVING OPERATIONAL EFFICIENCIES On-Shelf & Display Execution Insights: • Out-of-Stock Notifications • Price, Promotion, Schematic Compliance • CPG Information Sharing Photo Source: Retail Technology Corp.
- 9. © Cloudera, Inc. All rights reserved. 9 IMPROVING OPERATIONAL EFFICIENCIES On-Shelf & Display Execution Insights: • Out-of-Stock Notifications • Price, Promotion, Schematic Compliance • CPG Information Sharing
- 10. © Cloudera, Inc. All rights reserved. 10 IMPROVING CUSTOMER EXPERIENCES • Identify VIPs, ensure high-touch CX (‘clienteling’) • Understand, respond (anonymously) with proactive customer service opportunities
- 11. © Cloudera, Inc. All rights reserved. 11 • Simplifying the dressing room experience • Assist to complete the ‘look’ - cross-sell of matching shoes, accessories • Reduce likelihood of returns IMPROVING CUSTOMER EXPERIENCES
- 12. © Cloudera, Inc. All rights reserved. 12 The use cases for CV in retail are growing, and with promising business impact Frictionless Checkout Loss Prevention Neighborhood Insights Personalization Customer Insights Customer Engagement Dynamic Merchandising Autonomous DeliverySource: Allure Source: Ahold/Delhaize Source: Caper Source: Orbital Insight Source: Aura visionSource: Trigo-vision
- 13. © Cloudera, Inc. All rights reserved. 13 HOW IT WORKS…
- 14. © Cloudera, Inc. All rights reserved. 14 HOW DOES IMAGE RECOGNITION WORK? Detect & Score Convolutional neural network analyzes images and produces actionable representations Compare Representation manifolds can be mapped to find matches, similar, or complementary images Match & Alert Users alerted and shown matches
- 15. © Cloudera, Inc. All rights reserved. 15 DEEP LEARNING • Deep learning has facilitated the effective use of ML models in vision and text applications that were previously extremely difficult • Deep learning models allow multi-stage representations of data to be learned, promoting better accuracy on a variety to tasks • Model components can be treated as building blocks for accelerated creation of new models • Advances in hardware have made the large calculations used in deep learning practical on desktop computers with graphics cards
- 16. © Cloudera, Inc. All rights reserved. 16 DEEP LEARNING DOES REPRESENTATION LEARNING • Neural networks learn useful ways of representing data as it flows through the layers • Specific network architectures allow us to make useful assumptions about the data and generate the appropriate outputs for a given task • Learn useful latent vector representations • Exploit spatial structure • Exploit temporal structure
- 17. © Cloudera, Inc. All rights reserved. 17 REPRESENTATION LEARNING FOR RETAIL • Having effective ways of extracting latent representations of our perceptual data enables to create new tools for the retail space • Infer customer demographics from just images • Track customer movement through store • Automated shelf tracking based on a camera feed • Missing items from shelves • Tracking which items are in customers baskets • Semantic reverse image search for your products • Locate similar products to query image • Find complimentary products within inventory
- 18. © Cloudera, Inc. All rights reserved. 18 COMPUTER VISION IN RETAIL STORES • Tracking customers throughout stores • Create heatmaps at most trafficked areas • Which displays do individuals linger at • Which areas of the shop gets the most attention • Make inferences about customer demographics • Cater products to the most frequent visitors • Add products to draw more visitors • Automated checkout • Track who products move with individuals • Automatically purchase items if it leaves with individual • Inventory tracking • Shelves needing attention Aura Vision, https://auravision.ai
- 19. © Cloudera, Inc. All rights reserved. 19© Cloudera, Inc. All rights reserved. FASHION PRODUCT RECOMMENDATION DEMO
- 20. © Cloudera, Inc. All rights reserved. 20
- 21. © Cloudera, Inc. All rights reserved. 21
- 22. © Cloudera, Inc. All rights reserved. 22 COMPUTER VISION FOR PRODUCT RECOMMENDATION • It's often difficult to describe in words a visual style unless you happened to know the technical terminology for that style • Deep learning techniques can help to sort various products or inventory by visual attributes generated directly from the pixel data • Given that deep learning models can learn rich semantic representations, we can leverage these latent factors to make recommendations for similar or complimentary products • If a user 'likes', views, or add certain items to their carts recommendations can be made not only on hard coded attributes but also perceptual semantic information • Think standard recommender systems augmented with features learned by deep learning models
- 23. © Cloudera, Inc. All rights reserved. 23 FASHION PRODUCT RECOMMENDATION EXAMPLE • Creating a semantic fashion recommender system can be broken down into a few components • First, we need to have the ability to extract the type and clothing that we want to match on • Then, we have to create the ability to create semantic representations from those extracted images • Finally, compare those representations to those of our catalog and return the best possible match
- 24. © Cloudera, Inc. All rights reserved. 24 Extract the type and clothing that we want to match on
- 25. © Cloudera, Inc. All rights reserved. 25 Extract the type and clothing that we want to match on
- 26. © Cloudera, Inc. All rights reserved. 26 Create semantic representations from those extracted Numeric vector representing visual patterns and attributes
- 27. © Cloudera, Inc. All rights reserved. 27 Compare those representations to those of our catalog A B Best Match
- 28. © Cloudera, Inc. All rights reserved. 28 Compare those representations to those of our catalog
- 29. © Cloudera, Inc. All rights reserved. 29 BRINGING IT ALL TOGETHER
- 30. © Cloudera, Inc. All rights reserved.30 WHAT INDUSTRIALIZED “EDGE TO AI” LOOKS LIKE Streaming Ingest Batch Ingest Machine Learning Tools BI Tools and SQL Editors Data Products DATA, METADATA, SECURITY, GOVERNANCE, WORKLOAD MANAGEMENT MACHINE LEARNING DATA ENGINEERING DATA WAREHOUSE OPERATIONAL DATABASE
- 31. © Cloudera, Inc. All rights reserved. THANK YOU Let’s Keep the Conversation Going… Brent Biddulph Managing Director, Retail & CG Cloudera bbiddulph@cloudera.com +1 425.273.6851 www.linkedin.com/in/brentbiddulph/ @brentbiddulph Florian Muellerklein Data Scientist Miner & Kasch fmuellerklein@minerkasch.com +1 410.564.1720 www.linkedin.com/in/florian-muellerklein/ @mllrkln
Notes de l'éditeur
- Hello, and thanks for joining this session titled: Computer Vision: Coming to a Store Near You. My name is Brent Biddulph, GM, Retail & CG, Cloudera Joining me is Florian Mullerklein, Data Scientist, Miner & Kasch
- AND INDEED, it likely CV may ALREADY be in a store near you. Today’s talk we will share how innovation using computer vision is already impacting brick & mortar… …illustrative use cases retailers are testing and deploying today. And finally, sharing foundational concepts of how it works, and what it looks like.
- So why are we talking about “in-store” when all the press lately has been about traditional retailers improving their online / omni-channel capabilities? For many - focus and attention is shifting back to the core business – brick and mortar - where more than 86% of top line revenues still occur. New technologies enable new possibilities, a few of which we will share today.
- IoT is not just reserved for industrial based industries. Retailers are also leveraging new technologies to gather “more observations” and generate “real-time responses” to improve customer experiences and operational efficiencies. Video streams are an emergent area showing much promise in retail (we will look at a few examples shortly)
- Applying Computer Vision techniques can be applied to tackle Key Business Imperatives such as: Enabling Frictionless Commerce – customers are coming to EXPECT more and more as part of overall CX Improving Operational Efficiencies – providing insights into operational challenges such as OOS (-5% revenues) – 24% goes to Amazon Improving Customer Experiences – providing image based search capabilities (50% of buyer search inspiration) Reduce Fraud & Shrink – on sales floor, checkout, receiving dock, fresh foods departments (-5% of revenues)
- Perhaps the most widely known example in NA, was the public launch of the Amazon Go store summer of 2018. A convenience store at 1,800 sq. ft. This ‘test’ originally ran 2 years in incubation, then rolled out to employees for a full year before opened to the public. Today, Amazon has since opened additional locations in Chicago, San Francisco with plans to add more, with larger assortments in NA as part of their brick and mortar strategy.
- Alibaba has now opened 100 HEMA retail stores in the past year, in an effort to merge online and offline retail. - Customers use an app to scan products, get information (country of origin, recipes, etc.) and pay for their groceries. - The stores double as distribution centers, where employees fill bags with online orders, then place them on a conveyor belt to the delivery center. - Customers pay via Alipay. Customers can even pay by scanning their faces at kiosks.
- Perhaps one of the most promising high value uses cases for CV ‘in-store’ is tackling the age-old challenge of merchandising execution and in-stocks. Leading grocers like Walmart, Kroger, Ahold have been testing for over a year… …and Ahold-Delhaize being first to announce a chain wide rollout across 500 Giant Store locations US East Coast. These robots can scan a 40’ aisle in 60 seconds, scan an entire store 3x day, improving accuracy over time.
- ESLs now available with built-in cameras enable ‘cross aisle’ views, similar to the mobile robots
- Some retailers are using in-store cameras pointed at the entry to identify ‘guest engagement opportunities for Customer Service associates.
- In fashion and beauty, “magic mirrors’ can improve the CX at stores like Nieman Marcus and Sephora > whether in-store or at home on your own mobile device. Allowing for speedy assessment of style, color and ‘looks’. Disrupters like Stitch Fix and Untuck It embedded this as a core capability.
- The possibilities for CV to impact retail seem limitless. Extending online analytic capabilities (and more) to the physical brick & mortar rhealm are gaining traction. Smart Carts - image recognition, payment + scales – sensor fusion (caper) Ceiling mounted cameras - In-store customer paths, dwell time, even distinguishing staff from customers (improve engagement, CX, hot spots, choke points, conversion, etc.) Robots & ESLs – with built in cameras (facilitate proximity marketing, dynamic pricing and pick-pack pathing at the item, store level) – AND provide merchandising execution insights much like the mobile robots. Satellite images capture retail and mall parking lot traffic (orbital insights). Competitive insights for retailers, new store opening, traffic counts, construction impact, etc. Autonomous vehicles – e.g. Stop & Shop (Ahold Delhaize USA) testing delivery AND mobile stores (Kroger, Walmart also continuing to test in pilot markets)
- Now the legacy Clouderans especially may think of ML on Cloudera as simply a CDSW play.. but here’s what industrialized ML actually looks like on the platform: First, data streaming or batch ingested into the pipeline Data engineering to clean and prep Building and deploying models Scoring and feeding results through to BI dashboards or real-time data views The bottom line is that Enterprise ML requires the big picture.