CN-122023941-A - Intelligent garbage recycling bin internal real-time fault identification method based on machine vision

CN122023941ACN 122023941 ACN122023941 ACN 122023941ACN-122023941-A

Abstract

The invention belongs to the technical field of image processing, and discloses a machine vision-based intelligent garbage collection box interior real-time fault identification method; the intelligent garbage collection box fault detection method comprises the steps of constructing a fault data set by acquiring fault image data, preprocessing the fault data set to obtain a standard data set, constructing an improved YOLOv intelligent garbage collection box fault detection model, training the model through the standard data set to obtain an optimal intelligent garbage collection box fault detection model, acquiring real-time image data, carrying out real-time fault detection according to the optimal intelligent garbage collection box fault detection model, and carrying out early warning monitoring on the intelligent garbage collection box according to the real-time image data acquired under a time sequence. The feature extraction module C3k2_ MambaOut module based on Mamba framework and CNN network combines a gating mechanism and convolution operation, can more effectively extract image features, and can selectively enhance or inhibit certain features through the gating mechanism, so that the feature extraction efficiency is improved, and the feature extraction capability is enhanced.

Inventors

GUO KEJIAN
YAO LIYIN
Mao Xinman
FANG MENG
Chen Shuipeng

Assignees

宁波搭把手生态数字科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260325

Claims (9)

1. The intelligent garbage collection box internal real-time fault identification method based on machine vision is characterized by comprising the following steps of: Shooting the inside of an intelligent garbage collection box with faults through a camera to obtain fault image data, and constructing a fault data set of the intelligent garbage collection box based on the fault image data; Preprocessing a fault data set, sequentially completing fault position marking, data expansion enhancement and data set division, and converting processed data into YOLOv network model identification format to obtain a standard data set, wherein the data set is divided into a verification set, a training set and a testing set; Step three, an improved YOLOv intelligent garbage collection box fault detection model is constructed, a P2 detection head is additionally arranged on a detection layer of the improved YOLOv intelligent garbage collection box fault detection model, a C3k2_ MambaOut characteristic extraction module based on the combination of a Mamba framework and a CNN network is arranged on a main network, a re-parameterized heterogeneous multi-scale RepHMS module is introduced into a neck, and model training is carried out on the improved YOLOv intelligent garbage collection box fault detection model by utilizing a standard dataset; Step four, model training is completed based on a training set, a testing set and a verification set in a standard data set, and an optimal intelligent garbage collection box fault detection model is obtained; Installing a camera in the intelligent garbage collection box, acquiring real-time image data in the intelligent garbage collection box, and inputting the real-time image data into an optimal intelligent garbage collection box fault detection model to realize real-time fault detection of the intelligent garbage collection box; and step six, performing early warning monitoring on the intelligent garbage collection box according to the real-time image data acquired in the time sequence.
2. The machine vision-based intelligent garbage collection bin interior real-time fault identification method according to claim 1, wherein the working method of the c3k2_ MambaOut feature extraction module is as follows: The method comprises the steps of processing fault image data in a standard data set through a main network preamble convolution layer to obtain a characteristic vector XA, dividing the characteristic vector XA into two parallel processed main branches and a second branch through convolution splitting operation, processing and extracting characteristics of the main branches through a gating mechanism through a GatedCNNBlock _ BCHW module to generate a characteristic vector XA1, performing basic convolution operation on the characteristic vector XA by the second branch to generate a characteristic vector XA2, and splicing the characteristic vector XA1 and the characteristic vector XA2 along channel dimensions to obtain a characteristic vector YA.
3. The machine vision-based intelligent garbage collection box real-time fault identification method according to claim 2, wherein the working method of the GatedCNNBlock _ BCHW module is as follows: The method comprises the steps of taking input features of a characteristic vector XA entering a main branch after convolution splitting as input tensors AM, normalizing the input tensors AM to obtain feature tensors AM1 with the shape of (N, C, H and W), carrying out channel expansion on the feature tensors AM1 through 1X 1 convolution to expand the channel number of the feature tensors AM1 from C to 2H, dividing the expanded feature tensors AM1 into gating signals g, direct branches i and deep convolution branches C, and carrying out depth separable convolution on the deep convolution branches C to obtain deep convolution branches Direct branch i and deep convolution branch The method comprises the steps of performing stitching along channel dimensions to obtain a characteristic tensor AM2 with the shape of (N, H, H, W), multiplying a gating signal g by the characteristic tensor AM2 element by element after applying an activating function, recovering the channel number from H to C through 1X 1 convolution, performing random discarding treatment through DropPath, performing residual connection with an input tensor AM, outputting the characteristic tensor AM3 with the shape of (N, C, H, W), and marking the characteristic tensor AM3 as a characteristic vector XA1, wherein N is expressed as the number of images to be processed, C is expressed as the channel number, H is expressed as the height of a characteristic graph, and W is expressed as the width of the characteristic graph.
4. The machine vision-based intelligent garbage collection box interior real-time fault identification method according to claim 2, wherein the working method of RepHMS modules is as follows: The method comprises the steps of marking a feature vector YA as an input feature map K, carrying out channel expansion on the input feature map K through 1X 1 convolution, dividing the input feature map K into a plurality of parallel branches, carrying out feature enhancement on each parallel branch through a stacked DepthBottleneckv module, carrying out residual connection on non-first parallel branches and previous parallel branches to keep bottom layer information, obtaining output feature tensors KA of each parallel branch, splicing the output feature tensors KA of all the parallel branches along channel dimensions, compressing the output feature tensors KA to a target output channel number through 1X 1 convolution, obtaining feature tensors KA1, and carrying out residual connection fusion on the feature tensors KA1 and the input feature map K, thus obtaining a target feature map KN.
5. The machine vision-based intelligent garbage collection box real-time fault identification method according to claim 4, wherein the working method of DepthBottleneckv modules is as follows: The input feature of each parallel branch divided by the input feature graph K is recorded as an input tensor KB, normalization processing is carried out on the input tensor KB, and the channel number of the input tensor KB is expanded from C to C through 1X 1 convolution The extended feature tensor FG is sequentially subjected to depth separable convolution and point-by-point convolution operation to obtain an enhanced feature tensor Will enhance the feature tensor Performing depth separable convolution and point-by-point convolution operations again to obtain an enhanced feature tensor The feature tensor is enhanced by 1X 1 convolution Compression of channel number to target output dimension Obtaining final characteristic enhancement tensor Enhancing the characteristic by tensor And carrying out residual connection fusion with the input tensor KB to obtain the output characteristic tensor KA of each parallel branch.
6. The machine vision-based intelligent garbage can interior real-time fault identification method of claim 1, wherein the method for obtaining the optimal intelligent garbage can fault detection model is as follows: The method comprises the steps of inputting a training set and a verification set into an improved YOLOv intelligent garbage collection box fault detection model to perform model optimization, setting training times, judging that the improved YOLOv intelligent garbage collection box fault detection model is trained to an optimal state when a loss function curve of the improved YOLOv intelligent garbage collection box fault detection model gradually converges and tends to be stable, storing a weight file in the optimal state, inputting fault image data in a test set into the improved YOLOv intelligent garbage collection box fault detection model in the optimal state, and outputting an intelligent garbage collection box fault detection image marked with a fault type and a fault position.
7. The machine vision-based intelligent garbage collection box interior real-time fault identification method according to claim 1, wherein the method for realizing the real-time fault detection of the intelligent garbage collection box is as follows: When the optimal intelligent garbage collection box fault detection model detects real-time faults, continuously acquiring the fault existence time, and when the fault existence time exceeds a preset time threshold, judging that the intelligent garbage collection box has faults, and at the moment, outputting a detection result containing a fault type label and fault position coordinates by the optimal intelligent garbage collection box fault detection model, sending the detection result to a control unit, and executing a preset processing strategy by the control unit according to the fault type label.
8. The machine vision-based intelligent garbage collection bin interior real-time fault identification method according to claim 1, wherein the working method of the P2 detection head is as follows: the P2 detection head is matched with the original detection head provided with the YOLOv network model and used for extracting characteristic diagrams with 4 sizes in the standard data set, and the sizes are 120 multiplied by 120, 64 multiplied by 64, 32 multiplied by 32 and 16 multiplied by 16 respectively corresponding to the fault targets with the small, medium and large sizes in the intelligent garbage collection box.
9. The machine vision-based intelligent garbage collection box interior real-time fault identification method according to claim 1, wherein the method for carrying out early warning and monitoring on the intelligent garbage collection box is as follows: when the real-time faults are not detected, acquiring real-time image data of all components in the intelligent garbage collection box, carrying out feature extraction on the real-time image data to obtain real-time feature expression vectors of all the components, acquiring image data of all the components when leaving the factory in the intelligent garbage collection box, extracting corresponding sequence features of all the components through a C3k2_ MambaOut module, establishing standard feature expression vectors based on the sequence features, calculating cosine similarity between the real-time feature expression vectors and the standard feature expression vectors, carrying out cosine similarity normalization processing to obtain real-time state quantities of all the components, acquiring real-time state quantities under a time sequence T, constructing a real-time state quantity function, calculating an integral value of the real-time state quantity function under the time sequence T, calculating a standard deviation of the real-time state quantity based on the real-time state quantity under the time sequence T, carrying out normalization processing on the integral value and the standard deviation of the standard deviation, dividing the normalized standard deviation by the integral value, obtaining fault risk coefficients of all the components, comparing the fault risk coefficients with preset fault risk coefficient thresholds, and if the fault risk coefficients are larger than the fault risk coefficient thresholds, judging that the intelligent garbage collection box has potential fault risks.

Description

Intelligent garbage recycling bin internal real-time fault identification method based on machine vision Technical Field The invention relates to the technical field of image processing, in particular to a machine vision-based intelligent garbage collection box interior real-time fault identification method. Background With the acceleration of the urban process and the improvement of environmental awareness, garbage classification has become an important component of urban management. Traditional garbage classification schemes typically rely on timed and fixed-point garbage bin placement and management by human supervising staff. The mode has the problems of fixed throwing time, singleness, high labor cost, non-traceability of garbage and the like. The intelligent garbage collection box is gradually popularized as a novel garbage throwing mode which is convenient, efficient and 24-hour to operate. The intelligent garbage recycling bin helps users to better classify through setting up multiple kinds of bin positions, and front-end garbage subdivision is achieved. However, in the long-term operation process of the intelligent garbage recycling bin, the recognition and monitoring of faults in the bin still have obvious defects, and the usability of equipment and user throwing experience are directly affected. The existing garbage bin fault detection multi-purpose maintenance personnel carry out inspection to find fault problems, manual inspection consumes a large amount of manpower and material resources, equipment faults can not be found for a long time during inspection intervals, usability of the equipment is reduced, breakthrough progress of deep learning technology in recent years brings revolutionary transformation to the field of real-time fault recognition detection in the intelligent garbage collection bin, a feature extraction module of an existing target detection model mostly adopts a traditional convolution network structure, fault features cannot be selectively enhanced according to image feature distribution in the intelligent garbage collection bin, irrelevant background features such as garbage accumulation are restrained, feature extraction efficiency is low, characterization capability of effective fault features is insufficient, and meanwhile, feature capturing capability of the traditional convolution module on visual tasks such as garbage adhesion, conveying belt transmission, bag turning mechanism action and the like in the intelligent garbage collection bin is weak, normal garbage accumulation and fault clamping state are difficult to effectively distinguish, and accordingly fault feature extraction effects related to time and space sequences are poor. In addition, the problem that the transverse receptive field is smaller, the vertical receptive field is not subjected to targeted expansion exists in the characteristic fusion module of the existing model, contextual information of faults in the box cannot be captured across a space hierarchy structure, characteristic extraction is carried out only by adopting a single-size convolution kernel in the same convolution layer, detection requirements of large-scale structural faults such as blockage of a large-scale convolution check delivery port and deformation of a mechanism are difficult to meet, and retention requirements of key details of micro fault targets such as slight deformation of a small-scale convolution check baffle, falling of a small fastener and local garbage clamping stagnation are difficult to meet, omission of the micro fault targets is easily caused, accurate detection requirements of multi-scale faults in the intelligent garbage recycling box cannot be met, in addition, the intelligent garbage recycling box is arranged outdoors, illumination conditions in the box change in real time along with the external environment, the box is various in garbage types, accumulation forms are complex, fault points are easy to be shielded by garbage, the existing target detection model which is not subjected to scene improvement is insufficient in characteristic suitability and robustness, and the accuracy of fault detection and the fault recycling rate cannot be further reduced, and the intelligent garbage recycling box can not be reliably supported in real time. In view of the above, the present invention proposes a machine vision-based method for identifying real-time faults in an intelligent garbage collection box to solve the above problems. Disclosure of Invention In order to overcome the defects in the prior art and achieve the purposes, the invention provides the following technical scheme: the intelligent garbage collection box internal real-time fault identification method based on machine vision comprises the following steps: Shooting the inside of an intelligent garbage collection box with faults through a camera to obtain fault image data, and constructing a fault data set of the intelligent garbage collection box based o