CN-121982534-A - Method, device, processor and computer readable storage medium for realizing detection of foreign matter on apron based on machine vision

CN121982534ACN 121982534 ACN121982534 ACN 121982534ACN-121982534-A

Abstract

The invention relates to a method for realizing detection of foreign matters on an apron based on machine vision, which comprises the following steps of carrying out image acquisition and pretreatment, continuously acquiring real-time video streams of a periphery and surrounding apron areas, decomposing images acquired below an corridor bridge into illumination components and reflection components, carrying out pixel-level accurate identification on a corridor bridge supporting wheel by adopting an improved semantic segmentation algorithm to generate a high-precision segmentation mask, and carrying out connected domain analysis on pixel areas with obvious differences in a residual image by introducing an unsupervised pretraining and a small amount of abnormal supervision fine tuning strategies, and screening and marking potential abnormal areas. The method, the device, the processor and the computer readable storage medium for realizing the detection of the foreign matters on the parking apron based on the machine vision are adopted, and the full flow from image preprocessing to anomaly identification is particularly optimized by utilizing an unsupervised learning strategy, so that the dependence on manual inspection is reduced, and the automation level and the response efficiency of airport security management are improved.

Inventors

WANG LILI
LIU YONGHENG
LIN ZHEN
FEI ZIXIANG
DU CHAOHUI
FEI MINRUI

Assignees

海南壹联科技有限公司
上海大学

Dates

Publication Date: 20260505
Application Date: 20260126

Claims (10)

1. The method for realizing the detection of the foreign matter on the apron based on the machine vision is characterized by comprising the following steps: (1) Image acquisition and preprocessing are carried out, real-time video streams of the wheel circumference and the surrounding parking apron area are continuously acquired through an industrial camera fixed below the corridor bridge, and an original image is scaled to 1280 multiplied by 720 to be uniform in resolution by adopting a bicubic interpolation algorithm; (2) Decomposing an image collected below a gallery bridge into an illumination component and a reflection component based on an improved Retinex theory, introducing an adaptive operator to enhance the illumination component, and synchronously inhibiting image noise; (3) Carrying out pixel-level accurate identification on the gallery bridge supporting wheel by adopting an improved semantic segmentation algorithm, introducing a focus loss function, and generating a high-precision segmentation mask; performing morphological closing operation on the segmentation result to eliminate tiny holes and smooth contour edges in a mask, and extracting background areas except a peri-rotation area from an original image through Boolean inverse operation to be used as a core attention range for subsequent abnormality detection; (4) Constructing a self-encoder model of an encoder-decoder structure, introducing a strategy of unsupervised pre-training and a small amount of abnormal supervision fine tuning, and training a self-encoder neural network by using a large amount of normal apron background images; (5) The method comprises the steps of carrying out connected domain analysis on pixel areas with significant differences in a residual map, screening and marking potential abnormal areas, synchronously counting core parameters of the abnormal areas to finally realize unsupervised foreign matter detection, and triggering an alarm mechanism when the number of the abnormal areas is detected to be greater than zero to realize foreign matter detection and early warning closed loop.
2. The method for detecting the foreign matter on the tarmac based on the machine vision according to claim 1, wherein the step (1) specifically comprises the following steps: (1.1) continuously acquiring real-time RGB images of a bottom scene of the gallery by using an industrial camera installed in the gallery area, and normalizing the image resolution to 1280 multiplied by 720 by adopting a bicubic interpolation method; (1.2) calculating gray values of 16 points around a point to be interpolated of the gallery image, and carrying out weighted calculation on the gray values; (1.3) pair Linear interpolation of the entries in the direction is performed once after the pixels of four horizontal points are obtained Interpolation calculation in the direction to find the target pixel Pixel values of (2); (1.4) according to the neighboring points Calculating the pixel value of the point; (1.5) obtaining the pixel value of each point, and then weighting the pixel values of the 16 points to perform interpolation calculation.
3. The method for detecting the foreign matter on the tarmac based on the machine vision according to claim 1, wherein the step (2) specifically comprises the following steps: (2.1) imaging under the gallery bridge Decomposition into illumination components And a reflection component Converting the multiplicative relationship into an additive relationship by logarithmic transformation, the illumination component Convolution operation is carried out on an image of the image under the gallery bridge by adopting a plurality of Gaussian kernel functions with different scales, the image is divided into non-overlapping local blocks, and the standard deviation of pixel values in each block is calculated High statistics The proportion P of the area to the whole graph; (2.2) for the separated reflection component Performing contrast stretching; (2.3) the enhanced reflection component Re-combining the obtained image with the adjusted illumination component, converting the fusion result back to a linear domain through exponential transformation, and carrying out pixel value standardization processing on the converted image; and fusing the enhanced brightness channel with the chromaticity channel of the original image to obtain the image below the gallery bridge with enhanced brightness.
4. The method for detecting the foreign matter on the tarmac based on the machine vision according to claim 1, wherein the step (3) specifically comprises the following steps: (3.1) taking ResNet-50 as a main feature extraction network, extracting a multi-scale feature map of an apron image through convolution and pooling operation, introducing a cavity space pyramid pooling module containing four different cavity rates into the network, capturing multi-scale context information from local details of an apron hub to an overall tire contour on the premise of keeping the resolution of the feature map, and outputting a binarization segmentation mask of an airplane wheel; (3.2) performing morphological closing operation on the airplane wheel binarization mask, and processing the airplane wheel mask through Boolean logic inverse operation to generate an inverse region; (3.3) to obtain a reverse region As an initial background constraint, with aircraft wheels And taking the wheel mask as an initial prospect constraint, applying an iterative GrabCut algorithm to finely divide the fuzzy boundary of the contact area between the ground of the corridor bridge and the wheel, and outputting a high-precision corridor bridge bottom ground area division result.
5. The method for detecting foreign matter on an apron based on machine vision according to claim 1, wherein the step (4) specifically comprises the following steps: (4.1) training an image reconstruction model by adopting a normal tarmac image sample, wherein the image reconstruction model is a self-encoder, and double differences between a reconstructed image and an original normal tarmac image in a pixel space and a feature space are minimized; (4.2) embedding a multi-scale feature fusion module and an attention mechanism in the encoder-decoder structure of the self-encoder, namely carrying out fusion of two types of features by encoding and extracting shallow detail features and deep semantic features; (4.3) introducing a small amount of marked abnormal samples in a training stage to enable the model to learn a typical reconstruction deviation mode corresponding to an abnormal scene; and (4.4) accurately extracting an abnormal region of the apron image from the residual map by a threshold segmentation method.
6. The method for detecting the foreign matter on the tarmac based on the machine vision according to claim 1, wherein the step (5) specifically comprises the following steps: (5.1) adopting a two-pass scanning method to carry out connected domain analysis; (5.2) calculating three types of core parameters for each extracted abnormal connected domain; And (5.3) displaying the number of the current abnormal areas in real time on a monitoring interface, outputting detailed parameters of each abnormality, and automatically triggering an audible and visual alarm if the number of the abnormalities is greater than 0.
7. The method for implementing tarmac foreign matter detection based on machine vision according to claim 6, wherein the step (5.1) specifically comprises the steps of: Traversing the binarization residual map according to the raster scanning sequence in the first pass, distributing temporary labels for pixels according to the 8-neighborhood communication relation, recording the equivalence relation among the labels, analyzing the equivalence label set in the second pass, merging the pixels belonging to the same communication domain to finish the extraction of the initial region, and merging and optimizing the region with higher space overlapping degree.
8. The utility model provides a device based on machine vision realizes apron foreign matter detection which characterized in that, the device include: A processor configured to execute computer-executable instructions; A memory storing one or more computer-executable instructions which, when executed by the processor, perform the steps of the machine vision-based method of performing tarmac foreign object detection of any one of claims 1 to 7.
9. A processor for implementing tarmac foreign object detection based on machine vision, characterized in that the processor is configured to execute computer-executable instructions which, when executed by the processor, implement the steps of the method for implementing tarmac foreign object detection based on machine vision according to any one of claims 1 to 7.
10. A computer readable storage medium having stored thereon a computer program executable by a processor to perform the steps of the machine vision-based method of performing tarmac foreign object detection of any one of claims 1 to 7.

Description

Method, device, processor and computer readable storage medium for realizing detection of foreign matter on apron based on machine vision Technical Field The invention relates to the field of machine vision, in particular to the technical field of image processing, and specifically relates to a method, a device, a processor and a computer readable storage medium for realizing detection of foreign matters on an apron based on machine vision. Background The apron bridge, also called boarding bridge (AirportBoardingBridge), is a movable closed aisle equipment for connecting the boarding gate of terminal building and the cabin door of airplane. The main function of the system is to provide safe and comfortable boarding and disembarking environment for passengers, so that the passengers can directly enter and exit the cabin without going through an open air apron, the weather effects such as sun, rain, strong wind and the like are effectively avoided, and the walking distance is obviously shortened. In the process of docking a corridor bridge with an aircraft, a blind area may exist in the line of sight of an operator due to the influence of the corridor bridge structure itself and the complex environment of the apron. If personnel or foreign matters (FOD, foreignObjectDebris) exist in the blind areas, and an operator cannot find the blind areas in time, collision risks can be caused when the corridor bridge is moved to dock, and safety of equipment or personnel is threatened. Therefore, how to effectively detect potential obstacles in the active area of the corridor bridge and send an alarm to operators in time in the key operation link becomes one of important technical requirements for guaranteeing the operation safety and improving the operation reliability of airport ground. The research and development of the deep learning algorithm at home and abroad are rapid, the related knowledge is very wide, various detection networks are endless, but many are established on the recognition of foreign matters in known categories, and the foreign matters needing to be recognized are provided in the training stage. However, FOD targets tend to be of no fixed size and unevenly distributed in the racetrack, so that their available for detection differs significantly from conventional target features. Meanwhile, in the FOD target detection process, with the proposal of a self-encoder, a generation countermeasure network and other generation type models, a reconstruction method based on unsupervised learning is widely applied in the field of anomaly detection. The techniques open up new research directions for tasks such as image recognition, classification and the like, and promote innovation and development of anomaly detection techniques in multiple fields. Disclosure of Invention The invention aims to overcome the defects of the prior art and provides a method, a device, a processor and a computer readable storage medium thereof for realizing detection of foreign matters on an apron based on machine vision, which are comprehensive in detection, strong in adaptability and wide in application range. In order to achieve the above object, the method, the device, the processor and the computer readable storage medium thereof for realizing the detection of the foreign matter on the tarmac based on the machine vision of the invention are as follows: the method for detecting the foreign matter on the parking apron based on the machine vision is mainly characterized by comprising the following steps of: (1) Image acquisition and preprocessing are carried out, real-time video streams of the wheel circumference and the surrounding parking apron area are continuously acquired through an industrial camera fixed below the corridor bridge, and an original image is scaled to 1280 multiplied by 720 to be uniform in resolution by adopting a bicubic interpolation algorithm; (2) Decomposing an image collected below a gallery bridge into an illumination component and a reflection component based on an improved Retinex theory, introducing an adaptive operator to enhance the illumination component, and synchronously inhibiting image noise; (3) Carrying out pixel-level accurate identification on the gallery bridge supporting wheel by adopting an improved semantic segmentation algorithm, introducing a focus loss function, and generating a high-precision segmentation mask; performing morphological closing operation on the segmentation result to eliminate tiny holes and smooth contour edges in a mask, and extracting background areas except a peri-rotation area from an original image through Boolean inverse operation to be used as a core attention range for subsequent abnormality detection; (4) Constructing a self-encoder model of an encoder-decoder structure, introducing a strategy of unsupervised pre-training and a small amount of abnormal supervision fine tuning, and training a self-encoder neural network by using a large amount of normal apro