US-20260127875-A1 - SYSTEM AND METHOD FOR AGGREGATING METADATA FOR ITEM IDENTIFICATION USING DIGITAL IMAGE PROCESSING

US20260127875A1US 20260127875 A1US20260127875 A1US 20260127875A1US-20260127875-A1

Abstract

A system for identifying items based on aggregated metadata obtains images of an item. The system extracts a set of features from images of the item. The system identifies a first value of a first feature associated with a first image of the item. The system identifies a second value of the first feature associated with a second image of the item. The system aggregates the first value and the second value. The system associates the item to the aggregated first value and the second value, where the aggregated first value and the second value represent the first feature of the item. The system adds a new entry for each image of the item to a training dataset associated with an item identification model.

Inventors

Sailesh Bharathwaaj Krishnamurthy
Tejas Pradip Rode
Crystal Maung
Shahmeer Ali Mirza

Assignees

7-ELEVEN, INC.

Dates

Publication Date: 20260507
Application Date: 20251231

Claims (20)

1 . A system for identifying items based on aggregated metadata, comprising: a memory operable to store a plurality of images of an item; a processor, operably coupled with the memory, and configured to: receive the plurality of images of the item; extract features from at least two of the plurality of images; for a first feature: identify a first value of the first feature associated with a first image of the item; identify a second value of the first feature associated with a second image; aggregate the first value with the second value; and associate the item with the aggregated first value and second value, wherein the aggregated first value and second value represent the first feature of the item; and identify the item based at least in part upon the aggregated first value and second value.
2 . The system of claim 1 , wherein the processor is further configured to, for a second feature: identify a third value of the second feature associated with the first image of the item; identify a fourth value of the second feature associated with the second image of the item; aggregate the third value with the fourth value; and identify the item based at least in part upon the aggregated third value and fourth value.
3 . The system of claim 1 , wherein: the first feature comprises one or more dominant colors of the item; the processor is further configured to: identify one or more first dominant colors of the item from the first image of the item, wherein each dominant color from among the one or more first dominant colors is determined based at least in part upon determining that a number of pixels that have the dominant color is more than a threshold number; determine a first percentage of each dominant color from among the one or more first dominant colors, wherein the first percentage of a first dominant color in the first image is determined by determining a ratio of a number of pixels that has the first dominant color in relation to the total number of pixels illustrating the item in the first image; identify one or more second dominant colors of the item from the second image of the item, wherein each dominant color from among the or more second dominant colors is determined based at least in part upon determining that a number of pixels that have the dominant color is more than the threshold number; determine a second percentage of each dominant color from among the one or more second dominant colors, wherein the second percentage of a second dominant color in the second image is determined by determining a ratio of a number of pixels that has the second dominant color in relation to the total number of pixels illustrating the item in the second image; determine the one or more dominant colors of the item by determining which dominant colors from among the one or more first dominant colors and the one or more second dominant colors have percentages more than a threshold percentage; and associate the one or more dominant colors to the item.
4 . The system of claim 1 , further comprising a weight sensor configured to measure weights for items on a platform, wherein: the first feature comprises a weight of the item; the processor is further configured to: receive a plurality of weights of multiple instances of the item; determine a mean of the plurality of weights of the item; and associate the mean of the plurality of weights of the item to the item.
5 . The system of claim 1 , wherein: the first feature comprises a dimension of the item; the processor is further configured to: identify a first dimension of the item from the first image, wherein the first dimension is represented by a first width, a first length, and a first height for the item detected on the first image; identify a second dimension of the item from the second image, wherein the second dimension is represented by a second width, a second length, and a second height of the item; determine the dimension of the item by determining a mean of the first dimension and the second dimension; and associate the mean of the first dimension and the second dimension to the item.
6 . The system of claim 1 , wherein: the first feature comprises a mask of the item; the processor is further configured to: identify a first mask that defines a first contour around the item in the first image; identify a second mask that defines a second contour around the item in the second image; determine differences between the first mask and the second mask; determine at least a portion of a three-dimensional mask around the item based at least in part upon the determined differences between the first mask and the second mask; and associate the three-dimensional mask around the item to the item.
7 . The system of claim 6 , wherein determining the three-dimensional mask around the item is in response to determining that the item is not identified based on the mask of the item.
8 . The system of claim 2 , wherein the processor is further configured to identify the item based at least in part upon the first feature and the second feature.
9 . A method for identifying items based on aggregated metadata, comprising: obtaining a plurality of images of an item; extracting features from at least two of the plurality of images; for a first feature: identifying a first value of the first feature associated with a first image of the item; identifying a second value of the first feature associated with a second image; aggregating the first value with the second value; and associating the item with the aggregated first value and second value, wherein the aggregated first value and second value represent the first feature of the item; and identifying the item based at least in part upon the aggregated first value and second value.
10 . The method of claim 9 , further comprising, for a second feature: identifying a third value of the second feature associated with the first image of the item; identifying a fourth value of the second feature associated with the second image of the item; aggregating the third value with the fourth value; associating the item with the aggregated third value and fourth value, wherein the aggregated third value and fourth value represent the second feature of the item; and identifying the item based at least in part upon the aggregated third value and fourth value.
11 . The method of claim 9 , further comprising: identifying one or more first dominant colors of the item from the first image of the item, wherein each dominant color from among the one or more first dominant colors is determined based at least in part upon determining that a number of pixels that have the dominant color is more than a threshold number; determining a first percentage of each dominant color from among the one or more first dominant colors, wherein the first percentage of a first dominant color in the first image is determined by determining a ratio of a number of pixels that has the first dominant color in relation to the total number of pixels illustrating the item in the first image; identifying one or more second dominant colors of the item from the second image of the item, wherein each dominant color from among the or more second dominant colors is determined based at least in part upon determining that a number of pixels that have the dominant color is more than the threshold number; determining a second percentage of each dominant color from among the one or more second dominant colors, wherein the second percentage of a second dominant color in the second image is determined by determining a ratio of a number of pixels that has the second dominant color in relation to the total number of pixels illustrating the item in the second image; determining one or more dominant colors of the item by determining which dominant colors from among the one or more first dominant colors and the one or more second dominant colors have percentages more than a threshold percentage; and associating the one or more dominant colors to the item.
12 . The method of claim 9 , further comprising: receiving a plurality of weights of multiple instances of the item; determining a mean of the plurality of weights of the item; and associating the mean of the plurality of weights of the item to the item.
13 . The method of claim 9 , further comprising: identifying a first dimension of the item from the first image, wherein the first dimension is represented by a first width, a first length, and a first height for the item detected on the first image; identifying a second dimension of the item from the second image, wherein the second dimension is represented by a second width, a second length, and a second height of the item; determining a dimension of the item by determining a mean of the first dimension and the second dimension; and associating the mean of the first dimension and the second dimension to the item.
14 . The method of claim 9 , further comprising: identifying a first mask that defines a first contour around the item in the first image; identifying a second mask that defines a second contour around the item in the second image; determining differences between the first mask and the second mask; determining at least a portion of a three-dimensional mask around the item based at least in part upon the determined differences between the first mask and the second mask; and associating the three-dimensional mask around the item to the item.
15 . The method of claim 14 , wherein determining the three-dimensional mask around the item is in response to determining that the item is not identified based on the mask of the item.
16 . A non-transitory computer-readable medium storing instructions that when executed by a processor cause the processor to: obtain a plurality of images of an item; extract features from at least two of the plurality of images; for a first feature: identify a first value of the first feature associated with a first image of the item; identify a second value of the first feature associated with a second image; aggregate the first value with the second value; and associate the item with the aggregated first value and second value, wherein the aggregated first value and second value represent the first feature of the item; and identify the item based at least in part upon the aggregated first value and second value.
17 . The non-transitory computer-readable medium of claim 16 , wherein the instructions further cause the processor to, for a second feature: identify a third value of the second feature associated with the first image of the item; identify a fourth value of the second feature associated with the second image of the item; aggregate the third value with the fourth value; associate the item with the aggregated third value and fourth value, wherein the aggregated third value and fourth value represent the second feature of the item; and identify the item based at least in part upon the aggregated third value and fourth value.
18 . The non-transitory computer-readable medium of claim 16 , wherein the instructions further cause the processor to: identify one or more first dominant colors of the item from the first image of the item, wherein each dominant color from among the one or more first dominant colors is determined based at least in part upon determining that a number of pixels that have the dominant color is more than a threshold number; determine a first percentage of each dominant color from among the one or more first dominant colors, wherein the first percentage of a first dominant color in the first image is determined by determining a ratio of a number of pixels that has the first dominant color in relation to the total number of pixels illustrating the item in the first image; identify one or more second dominant colors of the item from the second image of the item, wherein each dominant color from among the or more second dominant colors is determined based at least in part upon determining that a number of pixels that have the dominant color is more than the threshold number; determine a second percentage of each dominant color from among the one or more second dominant colors, wherein the second percentage of a second dominant color in the second image is determined by determining a ratio of a number of pixels that has the second dominant color in relation to the total number of pixels illustrating the item in the second image; determine one or more dominant colors of the item by determining which dominant colors from among the one or more first dominant colors and the one or more second dominant colors have percentages more than a threshold percentage; and associate the one or more dominant colors to the item.
19 . The non-transitory computer-readable medium of claim 16 , wherein the instructions further cause the processor to: receive a plurality of weights of multiple instances of the item; determine a mean of the plurality of weights of the item; and associate the mean of the plurality of weights of the item to the item.
20 . The non-transitory computer-readable medium of claim 16 , wherein the instructions further cause the processor to: identify a first dimension of the item from the first image, wherein the first dimension is represented by a first width, a first length, and a first height for the item detected on the first image; identify a second dimension of the item from the second image, wherein the second dimension is represented by a second width, a second length, and a second height of the item; determine a dimension of the item by determining a mean of the first dimension and the second dimension; and associate the mean of the first dimension and the second dimension to the item.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. patent application Ser. No. 18/474,414 filed Sep. 26, 2023, entitled “SYSTEM AND METHOD FOR AGGREGATING METADATA FOR ITEM IDENTIFICATION USING DIGITAL IMAGE PROCESSING,” which is a continuation of U.S. patent application Ser. No. 17/455,895 filed Nov. 19, 2021, entitled “SYSTEM AND METHOD FOR AGGREGATING METADATA FOR ITEM IDENTIFICATION USING DIGITAL IMAGE PROCESSING,” now U.S. Pat. No. 11,823,444 issued Nov. 21, 2023, which is a continuation-in-part of U.S. patent application Ser. No. 17/362,261 filed Jun. 29, 2021, and entitled “ITEM IDENTIFICATION USING DIGITAL IMAGE PROCESSING,” now U.S. Pat. No. 11,887,332 issued Jan. 30, 2024, which are incorporated herein by reference. TECHNICAL FIELD The present disclosure relates generally to digital image processing, and more specifically to a system and method for aggregating metadata for item identification using digital image processing. BACKGROUND Identifying and tracking objects within a space using computer vision poses several technical challenges. Conventional systems are unable to identify an item from among multiple items in an image. SUMMARY Particular embodiments of systems disclosed in the present disclosure are particularly integrated into a practical application of using computer vision and artificial intelligence to identify items, and features about items, depicted in computer images. Accordingly, the present disclosure improves item identification technology, which can be helpful in a large number of computer vision applications, such as facilitating contactless interactions at a grocery or convenience store. Thus, particular embodiments of the disclosed systems improve digital image processing technologies and various aspects of item identification technologies. Existing technology typically requires a user to scan or manually identify items to complete an interaction at, for example, a grocery store or convenience store. This creates a bottleneck in the system's ability to quickly identify items and complete item interactions. In contrast, the disclosed systems can identify one or more particular items from among multiple items depicted in a computer image. This provides an additional practical application of identifying multiple items at a time, which reduces the bottleneck and amount of resources that need to be dedicated to the item interaction process. For example, a user can place multiple items on a platform of an imaging device such as, for example, at a grocery store or convenience store checkout. The imaging device may capture one or more images from each of the multiple items. The disclosed system may process the captured one or more images and identify each of the multiple items. These practical applications are described in greater detail below. Although the present disclosure is described with reference to item interactions at a grocery store or convenience store as an example, it should be understood that the technologies described herein have wider application in a variety of other contexts and environments, such as item interaction at different types of warehouses, shipping facilities, transportation hubs (e.g., airports, bus stations, train stations), and the like. Updating a Training Dataset of an Item Identification Model The present disclosure contemplates systems and methods for updating a training dataset of an item identification model. The item identification model may be configured to identify items based on their images. In an example scenario, assume that the item identification model is trained and tested to identify a particular set of items. In some cases, a new item may be added to a list of items that are desired to be identified by the item identification model. One technical challenge currently faced is that to configure the item identification model to be able to identify new items (that the item identification model has not been trained to identify), the item identification technology may go through a retraining process where weight and bias values of perceptrons of neural network layers of the item identification model are changed. However, this process can be time-consuming and requires a lot of processing and memory resources. In addition, it will be challenging to retrain the item identification model for each new item, especially if new items are added to the list of items to be identified by the item identification model frequently. The disclosed system provides technical solutions for the technical problems mentioned above by configuring the item identification model to be able to identify new items without retraining the item identification model to be able to identify new items, as described below. Typically, the item identification model of the present disclosure is configured to output an identifier of an item. For example, the item identification model may comprise a set of neural network layers where the output lay