US-12616184-B2 - Real-time monitoring and early detection system for insect activity in grains during storage

Abstract

A system for real-time monitoring of insects includes a smart trap and an image processor. The smart trap includes a chamber with perforations sized to admit insects into an interior of the smart trap; a collection chamber located within the interior of the smart trap; and an imaging system for capturing images that include the collection chamber. The image processor is configured to receive images captured by the smart trap and to determine a count of insects within the collection chamber based on image analysis of the received image. Image analysis includes identifying a region within the received image corresponding with a boundary of the collection chamber, cropping the received image to the identified region to generate a cropped image, modifying at least one characteristic of the cropped image to generate a modified, cropped image, and determining a count of insects based on the modified, cropped image.

Inventors

• Ragab KHIR
• Zhongli Pan

Assignees

• THE REGENTS OF THE UNIVERSITY OF CALIFORNIA

Dates

Publication Date

20260505

Application Date

20210224

Claims (19)

1. 1 . A system for real-time monitoring of insects comprising: a smart trap comprising: a chamber with perforations sized to admit insects into an interior of the smart trap; a collection chamber located within the interior of the smart trap; a cap covering a top opening of the chamber, the cap housing an electronic system comprising an imaging system for capturing images that include the collection chamber of the smart trap; and an image processor configured to receive images captured by the smart trap and to determine a count of insects within the collection chamber based on image analysis of the received image, wherein image analysis includes identifying a region within the received image corresponding with a boundary of the collection chamber, cropping the received image to the identified region to generate a cropped image, modifying at least one characteristic of the cropped image to generate a modified, cropped image, and determining a count of insects based on the modified, cropped image; wherein determining the count of insects based on the modified, cropped image comprises applying a particle detection algorithm, wherein the particle detection algorithm comprises: identifying one or more bounding boxes, each bounding box identifying a region of interest in the modified, cropped image; placing the bounding box into a set of bounding boxes; filtering the set bounding boxes to a subset of bounding boxes likely to contain an insect; and counting insects in the subset of bounding boxes to determine the count of insects.
2. 2 . The system of claim 1 , wherein the image processor is included as part of the smart trap.
3. 3 . The system of claim 1 , further including a server located remotely from the smart trap, wherein the server is communicatively coupled to the smart trap to receive data from the smart trap, wherein the image processor is located on the server and wherein data received from the smart trap includes images captured by the smart trap.
4. 4 . The system of claim 1 , wherein identifying a region within the received image corresponding with a boundary of the collection chamber includes applying a Hough Circle transform to the captured image to identify the region in the received image corresponding with the boundary of the collection chamber.
5. 5 . The system of claim 1 , wherein modifying at least one characteristic of the cropped image includes one or more of converting the cropped image to greyscale, adjusting brightness/contrast of the cropped image, binarizing dark regions of the cropped image, reducing image/noise of the cropped image, and binarizing the cropped image.
6. 6 . The system of claim 1 , wherein filtering the set of bounding boxes to the subset of bounding boxes likely to contain an insect includes restricting bounding boxes in the set to bounding boxes based on one or more of location of the bounding box within a certain area band, presence of black pixels within the bounding box, presence of object having a specified eccentricity; and/or presence of an object having an oval-shaped object.
7. 7 . A smart trap for detecting insects, comprising: a perforated chamber with openings sized to admit insects into an interior of the smart trap; a collection chamber located within the interior of the smart trap for collecting admitted insects; and a cap configured to cover the perforated chamber; an electronic system housed in the cap, the electronic system comprising: a main board including a microcontroller and a communication module; a shield board including a lighting module configured to illuminate the collection chamber during image capture; a camera board including an imaging system to capture an image of the collection chamber; wherein the main board, the shield board, and the camera board are stacked with the shield board positioned between the main board and the camera board, and the microcontroller is configured to provide instructions to control the lighting module to illuminate the collection chamber and to control the camera board to capture the image.
8. 8 . The smart trap of claim 7 , the electronic system further including a sensor module for collecting data on one or more ambient conditions.
9. 9 . The trap of claim 7 , wherein; a shield board further comprising a communication module configured to communicate with the lighting module and the main board.
10. 10 . A method of counting insects in a captured image, comprising: cropping and masking the captured image to produce a first processed image containing only a region in the captured image that correlates to a collection chamber; modifying at least one characteristic of the first processed image to produce a second processed image; and determining a count of insects in the captured image by executing a particle detection algorithm on the second processed image, wherein the particle detection algorithm includes: identifying one or more bounding boxes, each bounding box identifying a region of interest in the second processed image; placing the bounding box into a set of bounding boxes; filtering the set bounding boxes to a subset of bounding boxes likely to contain an insect; and counting insects in the subset of bounding boxes to determine a count of insects in the captured image.
11. 11 . The method of claim 10 , wherein cropping and masking the captured image includes applying a Hough Circle transform to define the region.
12. 12 . The method of claim 10 , wherein modifying at least one characteristic of the first processed image makes any insects in the first processed image more pronounced for easier identification by the particle detection algorithm.
13. 13 . The method of claim 10 , wherein modifying at least one characteristic of the first processed image includes one or more of converting the first processed image to greyscale, adjusting brightness/contrast of the first processed image, binarizing dark regions of the first processed image, reducing image/noise of the first processed image, and binarizing the first processed image.
14. 14 . The method of claim 10 , wherein filtering the set of bounding boxes to a subset of bounding boxes likely to contain an insect includes restricting bounding boxes in the set to bounding boxes based on one or more of location of the bounding box within a certain area band, presence of black pixels within the bounding box, presence of object having a specified eccentricity; and/or presence of an oval-shaped object.
15. 15 . The trap of claim 7 , wherein the collection chamber comprises a white base.
16. 16 . The trap of claim 9 , wherein the shield board further comprises a sensor module comprising at least one sensor configured to collect data about an ambient condition, wherein the microcontroller is further configured to provide instructions to the shield board to collect sensor data.
17. 17 . The system of claim 1 , the electronic system further comprising a microcontroller, wherein the microcontroller is configured to provide instructions to the imaging system to capture an image of the interior of the smart trap.
18. 18 . The system of claim 17 , wherein: a camera board comprising the imaging system, the imaging system comprising: a camera; and a camera interface configured to receive instructions from a main board the electronic system further comprising: the main board comprising: a power module comprising a battery socket for receiving power from at least one battery; and a communication module; a shield board comprising: a lighting module and a communication module configured to communicate with the lighting module and the main board; wherein the main board, the shield board, and the camera board are stacked horizontally with the shield board positioned between the main board and the camera board.
19. 19 . The system of claim 18 , the shield board further comprising: a sensor module comprising at least one sensor for monitoring and/or recording an ambient condition; wherein: the communication module of the shield board is configured to provide confirmation of lights being turn on and/or sensor data being collected by the sensor module; the lighting module is located on a first side of the shield board, the first side directed towards the collection chamber; and the sensor and communication modules are located on a second side of the shield board.

Description

BACKGROUND Insect infestation of stored grains negatively affects the grade of the stored grains, increases the grain temperature, and promotes the growth of microorganisms that cause spoilage and thereby further reduce grain quality. Consequently, an infestation can lead to significant financial losses for the grain growers and processors. The early detection of insect infestation is, therefore, an important need in the grain industry. SUMMARY According to one aspect, a system for real-time monitoring of insects includes a trap and an image processor. The trap includes a chamber with perforations sized to admit insects into an interior of the smart trap, a collection chamber located within the interior of the smart trap, and an imaging system for capturing images that include the collection chamber of the smart trap. The image processor is configured to receive images captured by the smart trap and to determine a count of insects within the collection chamber based on image analysis of the received image, wherein image analysis includes identifying a region within the received image corresponding with a boundary of the collection chamber, cropping the received image to the identified region to generate a cropped image, modifying at least one characteristic of the cropped image to generate a modified, cropped image, and determining a count of insects based on the modified, cropped image. According to another aspect, a trap for detecting insects includes a perforated chamber with openings sized to admit insects into an interior of the trap, a collection chamber located within the interior of the smart trap for collecting admitted insects, and a cap configured to cover the perforated chamber, the cap housing an electronic system including an imaging system to capture an image of the collection chamber. According to a further aspect, a method of counting insects in a captured image includes cropping and masking the captured image to produce a first processed image containing only a region in the captured image that correlates to a collection chamber, modifying at least one characteristic of the first processed image to produce a second processed image, and determining a count of insects in the captured image by executing a particle detection algorithm on the second processed image. BRIEF DESCRIPTION OF DRAWINGS This written disclosure describes illustrative embodiments that are non-limiting and non-exhaustive. Reference is made to illustrative embodiments that are depicted in the figures, in which: FIG. 1 shows an embodiment of an insect detection system. FIGS. 2A-B are views of a smart trap, according to some embodiments of this disclosure. FIG. 2A is a schematic diagram of a smart trap. FIG. 2B is a photo of a trap, according to some embodiments of this disclosure. FIG. 3 is a block diagram of the systems of a smart trap, according to some embodiments of this disclosure. FIG. 4 is a block diagram of modules of a main board of a smart trap, according to some embodiments of this disclosure. FIG. 5 is a block diagram of modules of the shield board, according to some embodiments of this disclosure. FIG. 6 is a block diagram of a camera board, according to some embodiments of this disclosure. FIG. 7 shows an exemplary embodiment of an arrangement for the systems for a smart trap. FIGS. 8A-B shows an exemplary embodiment of the main board, showing (A) the first side of the main board and (B) the second side of the main board. FIG. 9A-C shows an exemplary embodiment of the shield board, showing (A) the first side of the shield board; (B) the second side of the shield board; and (C) an exemplary connection schematic for the modules of the shield board. FIGS. 10A-B shows an exemplary embodiment of the camera board, showing (A) the first side of the camera board and (B) the second side of the camera board. FIG. 11 is a flowchart of an algorithm to count the number of insects in an image, according to some embodiments of this disclosure. FIG. 12 is a flowchart of an algorithm to count the number of insects in an image according to some embodiments of this disclosure. FIG. 13 is a flowchart of an exemplary subroutine for determining the region in a captured image that corresponds to the collection chamber. FIG. 14 is a flowchart of an exemplary subroutine for modifying one or more characteristics of the first processed image. FIG. 15 is a flowchart of an exemplary subroutine particle detection algorithm, according to some embodiments of this disclosure. Arrows indicate that a step may be repeated at least one time before proceeding to the next step of the method. FIG. 16 is a graph showing the time to detect the emergence of the first insect during lab and commercial tests of an insect system according to some embodiments of this disclosure. DETAILED DESCRIPTION Early insect-detection is considered an effective technique to determine the optimal pest management practice to eliminate the infestation risk and maintain sto