US-12619950-B2 - Automated food selection using hyperspectral sensing

US12619950B2US 12619950 B2US12619950 B2US 12619950B2US-12619950-B2

Abstract

A solution for automated food selection includes: an order processing component operable to receive an order identifying an item; a selection component comprising: a first hyperspectral sensor operable to sense a reflection from the item and produce a sensor output based at least on the reflection; and a picking mechanism; a control component operable to: based at least on identification of the item, select a hyperspectral profile from a set of hyperspectral profiles; compare the sensor output with the selected hyperspectral profile; and based at least on the comparison, determine whether to select the item for fulfillment of the order, wherein the picking mechanism is operable to divert the item to a selection output based at least on a determination to select the item for fulfillment of the order; and a transport component operable to transport the item to an order storage zone.

Inventors

James Benjamin Edwards
Brian C. Roth

Assignees

WALMART APOLLO, LLC

Dates

Publication Date: 20260505
Application Date: 20231116

Claims (20)

1 . A system comprising: a control processor operable to: receive an order identifying a perishable item from an order processor; receive, from a sensor, sensor output associated with an instance of the perishable item, wherein the sensor is configured to sense reflected light and the sensor output is based on light reflected from the perishable item and sensed by the sensor; select a hyperspectral profile associated with the perishable item from a set of hyperspectral profiles stored at a data store, the hyperspectral profile including a lower threshold and a higher threshold; determine a disposition for the instance of the perishable item by comparing the sensor output to the lower threshold and the higher threshold of the hyperspectral profile; and control a picking mechanism to divert the instance of the perishable item to a target output of a plurality of outputs based on the determined disposition.
2 . The system of claim 1 , wherein the sensor output is received from a selection component comprising a light source positioned to produce a reflection from the instance of the perishable item.
3 . The system of claim 1 , wherein the sensor output is received from a selection component comprising: a first sensor positioned a first viewing angle; a second sensor positioned at a second different viewing angle than the first sensor relative to the perishable item; and an optical camera, wherein the determination to divert the instance of the perishable item to the target output for fulfillment of the order is based on the respective sensor outputs of both the first sensor and the second sensor being between the lower threshold and the higher threshold within the profile.
4 . The system of claim 1 , wherein the determined disposition is to withhold selection of the instance of the perishable item for fulfillment of the order due to the sensor output being outside the lower threshold and the higher threshold within the profile.
5 . The system of claim 1 , wherein the target output is associated with a customer order tote.
6 . The system of claim 5 , wherein the customer order tote is transported, by a transport vehicle, from the target output to an order storage zone.
7 . The system of claim 1 , further comprising: a disposal zone, wherein the instance of the perishable item is transported from the target output to the disposal zone; wherein a transport vehicle is further operable to transport the instance of the perishable item to the disposal zone.
8 . The system of claim 1 , wherein the control processor is further operable to: identify a ripeness preference using the order received from the order processor; and select the profile based at least on the ripeness preference.
9 . The system of claim 1 , further comprising: a profile update component operable to update the profile based at least on received feedback or environmental conditions.
10 . The system of claim 1 , wherein the sensor output is received from a selection component comprising a hyperspectral sensor.
11 . A method comprising: receiving, by a computing node, an order identifying a perishable item; receiving, by the computing node from a sensor, sensor output associated with an instance of the perishable item, wherein the sensor is configured to sense reflected light and the sensor output is based on light reflected from the perishable item and sensed by the sensor; selecting, by the computing node, a hyperspectral profile associated with the perishable item from a set of hyperspectral profiles stored at a data store, the hyperspectral profile including a lower threshold and a higher threshold; determining, by the computing node, a disposition for the instance of the perishable item by comparing the sensor output to the lower threshold and the higher threshold of the hyperspectral profile; and controlling, by the computing node, a picking mechanism to divert the instance of the perishable item to a target output of a plurality of outputs based on the determined disposition.
12 . The method of claim 11 , wherein the determined disposition is to withhold selection of the instance of the perishable item for fulfillment of the order due to the sensor output being outside the lower threshold and the higher threshold within the profile.
13 . The method of claim 11 , wherein the sensor output is received from a selection component comprising a light source positioned to produce a reflection from the instance of the perishable item.
14 . The method of claim 11 , wherein the sensor output is received from a selection component comprising a first sensor positioned a first viewing angle, a second sensor positioned at a second different viewing angle than the first sensor relative to the instance of the perishable item, and an optical camera, and wherein the determination to divert the instance of the perishable item to the target output for fulfillment of the order is based on the respective sensor outputs of both the first sensor and the second sensor being between the lower threshold and the higher threshold within the profile.
15 . The method of claim 11 , wherein the determined disposition is to divert the instance of the perishable item to the target output for fulfillment of the order, the method further comprising: controlling the picking mechanism to divert the instance of the perishable item to a selection output; and controlling transport of the instance of the perishable item from a selection area to an order storage zone.
16 . The method of claim 15 , wherein the order storage zone comprises an automated storage and retrieval system (ASRS).
17 . The method of claim 11 , further comprising: identifying a ripeness preference using the order received; and selecting the profile based at least on the ripeness preference.
18 . The method of claim 11 , further comprising: updating the profile based on at least one of received feedback or environmental conditions.
19 . One or more computer storage devices having computer-executable instructions stored thereon, which, on execution by a control processor, cause the control processor to perform operations comprising: receiving, by the control processor, an order identifying a perishable item; receiving, by the control processor from a sensor, sensor output associated with an instance of the perishable item, wherein the sensor is configured to sense reflected light and the sensor output is based on the light reflected from the perishable item and sensed by the sensor; selecting, by the control processor, a hyperspectral profile associated with the perishable item from a set of hyperspectral profiles stored at a data store, the hyperspectral profile including a lower threshold and a higher threshold; determining, by the control processor, a disposition for the instance of the perishable item by comparing the sensor output to the lower threshold and the higher threshold of the hyperspectral profile; and controlling, by the control processor, a picking mechanism to divert the instance of the perishable item to a selection output of a plurality of outputs based on the determined disposition.
20 . The one or more computer storage devices of claim 19 , wherein the determined disposition is to withhold selection of the perishable item for fulfillment of the order due to the sensor output being outside the lower threshold and the higher threshold within the profile.

Description

This application is a continuation of U.S. application Ser. No. 18/064,344 filed Dec. 12, 2022, which is a continuation of U.S. application Ser. No. 17/363,404 filed Jun. 30, 2021, now U.S. Pat. No. 11,551,185 issued Jan. 10, 2023, which claims the benefit of U.S. Provisional Application No. 63/067,724 filed Aug. 19, 2020, the entirety of which are hereby incorporated by reference. BACKGROUND Selecting perishable food items, such as produce, for customer order fulfillment typically requires human labor. This is because, even if the perishable food items had passed inspection upon arrival at the order-fulfilling facility (such as a retail facility that delivers or provides curbside pick-up), the condition of the perishable food items may change between arrival and ultimate delivery to a customer. Although the perishable food items may have been fresh or even in a pre-ripe state, multiple factors, including time and environmental conditions (temperature and humidity), may impact the rate of spoilage while stored in inventory locations or a staging location for customer pick-up. Humans that are trained to properly ascertain the state of perishable food items are thus used to pick specific ones of the perishable food items for order fulfillment. Any limitations on the human workforce, such as training, illness, and other scheduling issues may then negatively impact order fulfillment capacity. SUMMARY The disclosed examples are described in detail below with reference to the accompanying drawing figures listed below. The following summary is provided to illustrate some examples disclosed herein. It is not meant, however, to limit all examples to any particular configuration or sequence of operations. A solution for automated food selection includes: an order processing component operable to receive an order identifying an item; a selection component comprising: a first hyperspectral sensor operable to sense a reflection from the item and produce a sensor output based at least on the reflection; and a picking mechanism; a control component operable to: receive the order from the order processing component; receive the sensor output from the selection component; based at least on identification of the item, select a hyperspectral profile from a set of hyperspectral profiles; compare the sensor output with the selected hyperspectral profile; and based at least on the comparison, determine whether to select the item for fulfillment of the order, wherein the picking mechanism is operable to divert the item to a selection output based at least on a determination to select the item for fulfillment of the order; and a transport component operable to transport the item to an order storage zone. BRIEF DESCRIPTION OF THE DRAWINGS The disclosed examples are described in detail below with reference to the accompanying drawing figures listed below: FIG. 1 illustrates an exemplary arrangement that advantageously uses hyperspectral sensing in automated food selection (e.g., selection of perishable food items for automated order fulfillment); FIGS. 2A, 2B, and 2C show further detail for components of the arrangement of FIG. 1; FIG. 3 illustrates notional hyperspectral selection criteria for use in the arrangement of FIG. 1; FIGS. 4A and 4B illustrate exemplary options for implementing aspects of the arrangement of FIG. 1; FIG. 5 illustrates an exemplary automated storage and retrieval system (ASRS) that may be used in the arrangement of FIG. 1; FIG. 6 shows a flow chart of exemplary operations associated with the arrangement of FIG. 1; FIG. 7 shows another flow chart of exemplary operations associated with the arrangement of FIG. 1; and FIG. 8 is a block diagram of an example computing node for implementing aspects disclosed herein. Corresponding reference characters indicate corresponding parts throughout the drawings. Elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions and/or relative positioning of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding. Also, common but well-understood elements that are useful or necessary in a commercially feasible embodiment may not be depicted, in order to facilitate a less obstructed view. DETAILED DESCRIPTION A more detailed understanding may be obtained from the following description, presented by way of example, in conjunction with the accompanying drawings. The entities, connections, arrangements, and the like that are depicted in, and in connection with the various figures, are presented by way of example and not by way of limitation. As such, any and all statements or other indications as to what a particular figure depicts, what a particular element or entity in a particular figure is or has, and any and all similar statements, that may in isolation and out of context be read as absolute and therefore limiting, may only properly be rea