Most people have at some point stood at the self-scanning station, turning a carton of milk in search of the barcode while the queue behind them grows. That moment may now be on its way out.
With new AI technology, products are identified in milliseconds. Here, externally employed doctoral student Tobias Pettersson demonstrates how it works.
With the new AI technology, it may be enough to place the items on a surface. The system identifies them instantly, without traditional scanning.
Self-checkout works well for most customers, but is not entirely frictionless for the retail trade. Incorrect registrations and shrinkage are difficult to detect in real time.
Tobias Pettersson is an externally employed doctoral student at ITAB and affiliated with the Smart Industry Sweden graduate school and the University of Skövde. He is leading the development of a technology that can make self-checkout both smoother and more secure.
"Instead of just looking at the barcode, the technology analyses the entire product, everything from product descriptions and layout to images, to understand what the product is," says Tobias Pettersson.
Cameras both see and read
Instead of scanning each item, the system combines cameras with advanced image and text recognition. When items are placed on a surface, the system takes a picture, reads the text on the packaging and simultaneously analyses colour, shape and logos.
In just a few milliseconds, an AI model determines what products they are and what information belongs to them.
"You won’t even have time to blink before the system knows what has been placed."
An extra eye in the self-checkout
The technology is not only intended to make things easier for customers, but also to help stores. Shrinkage, goods disappearing without being paid for, is a growing problem in self-checkouts.
"Today’s systems don’t catch everything. Here, the technology functions as an extra eye."
By analysing patterns, the system can detect situations where something is amiss – for example, when an item ends up in a bag without being scanned.
"For customers where the system detects irregularities, a check can be made. For the majority, checkout can be completely frictionless," he says, emphasising that the aim is not surveillance but to reduce errors and shrinkage, which leads to a better customer experience.
Monitoring shelves and flows
In-store, the technology can also be used to monitor shelves. Cameras and algorithms can detect when products are running low, misplaced, or arranged in a disorderly manner.
"Empty or messy shelves directly affect sales. Here, preventive work can be done instead of just putting out fires."
The technology is ready
It is still unclear when the technology will be put into practice, but Tobias Pettersson estimates that it could happen relatively soon. The challenge does not lie in the AI model, but in integration. The technology is no longer the limiting factor. He describes the solution of the future as flexible and complemented by new solutions.
"Those who want to shop traditionally should be able to do so. Those who want to place items on a surface and leave quickly should also have that option. At the same time, companies can gain better control of their flows."
For both consumers and businesses, this could mean that a familiar moment, the hunt for the barcode, may soon become obsolete.
More facts about Tobias’s research
- On 23 January, Tobias Pettersson will defend his doctoral thesis: Product Recognition with OCR Text: Advancing Grocery Product Recognition through Robust Approaches, Fine-Grained Recognition, and Domain Adaptation for Real-Time Performance at the University of Skövde, ASSAR Industrial Innovation Arena.
- The time and location, as well as a summary of the thesis, can be found in the University’s calendar.
