When I worked for Kinect for Windows (the commercial version of Kinect), I learned about an Argentinean company called Kimetric. I recently stumbled on their site, and was excited to see they’re doing well, and some of their customers include car company Renault, beverage and brewing company Anheuser-Busch InBev, and snack manufacturer Mondelez International, whose brain soup includes Cadbury, Chips Ahoy, Dentyne, Gevalia, Halls, Honey Maid, and Nabisco, Nutter Butter, Ritz, Oreo, Sour Patch Kids, Tang, Tassimo, Trident, Triscuit, and Wheat Thins.
Kimetric has a very unique application, and depending on your point-of-view, could be perceived as very exciting or extremely “Big Brother.” Using Kinect and other types of sensors, they capture information about shoppers as they walk in the door – sensing their age, gender, size, and mood — and what they do as they shop. The sensors capture what products consumer pick up and examine, and whether these products are placed back on the shelf or end up in their shopping carts.
Along with having five age categories for each gender, the Kimetric application can detect shoppers’ mood – happy, angry, sad, and surprised – and their head and eye movement. It can also differentiate between specific products, by deciphering brand logos, type of product, and SKU.
Capturing What You Do
Imagine, you walk into your neighborhood grocery store. The Kinect unit (1) over the door instantly determines your age and gender, and that you appear sad. Initially, you walk down an aisle with shelves of beverages (2). The Kimetric applications tracks the number and brand of bottles of soda, flavored waters, and teas you examine, before putting several types of VitaminWater in your cart.
Your mood changes as you walk down the aisles towards a refrigerator case (4). A display on the case presents a humorous clip about Wonka Peel-a-Pop frozen treats. You’re intrigued and can’t believe you can really peel them like a banana. You open the refrigerator case, and toss in a box. You also grab a pint of Ben & Jerry’s Chubby Hubby.
As you make your way down the aisles, you see interactive displays (3), which enable you to search for recipes, see a list of sale items, and watch short clips about featured products. Because these displays are linked to Kinect sensors, they are able to present personalized experiences to increase purchases.
With a cart full of goods, you’re ready to check-out (5). The Kinect sensor captures what you purchased. This data, along with the data from other sensors and other stores are aggregated in the Kimetric Analytics Engine, including consumer profiles, clothing, product selection, and paths taken through a stores.
The date can then be analyzed and measured to glean insights, which enable the implementation of additional user experiences to further increase purchases and customer satisfaction.
Kimetric in Action
To promote the FIFA World Cup 2014, Kimetric was hired by Anheuser-Busch InBev to created interactive shelves, each with a LCD screen and Kinect sensor. As consumers walked by the shelves, the sensor detected whether they were wearing an Argentinean soccer jersey. If so, they were given a beer coupon.
Using machine learning, the displays were able to discern 50,000 different t-shirts. During the course of the promotion, 500,000 shoppers were tracked with 400,000 profiles being created. With thousands of beer coupons having been given out, beer sales significantly increased.
Next time you walk by a display, which seems to be tracking you, it might be capturing your movement, age, gender, and socio-economic class, based on your clothing.