CN-121989958-A - Image recognition-based wheeled coal shuttle car motion control method and system

CN121989958ACN 121989958 ACN121989958 ACN 121989958ACN-121989958-A

Abstract

The application relates to a wheel type coal conveying shuttle car motion control method and system based on image recognition, which relate to the technical field of vehicle control and comprise the steps of acquiring continuous image frames of a front roadway environment acquired by image acquisition equipment arranged on a shuttle car; the method comprises the steps of carrying out stabilization correction on continuous image frames, identifying image edges, image textures, image colors and image high-brightness areas of each frame of image in the corrected continuous image frames, determining potential roadway boundary features, extracting a time sequence change feature set and a space vision feature set of the potential roadway boundary features in continuous multi-frame images, judging real roadway boundaries from the potential roadway boundary features according to the time sequence change feature set and the space vision feature set, and generating a movement control instruction of a shuttle car according to the real roadway boundaries. The method can identify the real roadway boundary, effectively inhibit misjudgment caused by dynamic interference factors such as wet rock wall reflection and the like, and improve the navigation accuracy of the shuttle car.

Inventors

LIU TIAN
WANG XIAONAN
LIANG LIBO
ZHANG CHONG
HOU MINGMING

Assignees

海纳美腾智能制造(山东)有限公司

Dates

Publication Date: 20260508
Application Date: 20260213

Claims (10)

1. The wheel type coal shuttle car motion control method based on image recognition is characterized by comprising the following steps of: Acquiring continuous image frames of a front roadway environment acquired by image acquisition equipment mounted on a shuttle car; performing a stabilization correction on the continuous image frames; Identifying an image edge, an image texture, an image color and an image high-brightness area of each frame image in the corrected continuous image frames, and determining potential roadway boundary characteristics; Extracting a time sequence change feature set and a space vision feature set of the potential roadway boundary features in continuous multi-frame images; judging a real roadway boundary from the potential roadway boundary characteristics according to the time sequence change characteristic set and the space vision characteristic set; and generating a motion control instruction of the shuttle car according to the real roadway boundary.
2. The method for controlling movement of a wheel type shuttle car based on image recognition according to claim 1, wherein the step of performing stabilization correction on the continuous image frames comprises: acquiring vehicle motion data acquired by an inertial measurement unit mounted on a shuttle car; Calculating global pixel displacement of the successive image frames based on the vehicle motion data; and performing inverse geometric transformation on the continuous image frames according to the global pixel displacement to compensate image shake caused by shuttle car vibration, so as to obtain corrected continuous image frames.
3. The method of claim 2, wherein the step of calculating global pixel displacement of the successive image frames based on the vehicle motion data comprises: carrying out Kalman filtering pretreatment on the vehicle motion data to obtain the angular speed and the linear acceleration of the shuttle car; Acquiring a coordinate transformation relation between a camera internal reference matrix calibrated in advance in the image acquisition equipment and an inertial measurement unit; converting the angular velocity and the linear acceleration into a motion transformation matrix of a camera coordinate system based on the coordinate transformation relation; And calculating global pixel displacement of the continuous image frames according to the camera internal parameter matrix and the motion transformation matrix.
4. The method for controlling movement of a wheeled shuttle based on image recognition according to claim 1, wherein the step of identifying the image edge, the image texture, the image color and the image highlighting area of each image of the corrected continuous image frames, and determining the potential roadway boundary features comprises: Identifying the image edge of each frame of the corrected continuous image frames by adopting a Canny edge detection algorithm to obtain edge characteristics with single pixel width; Identifying the image texture of each frame of image by adopting a local binary pattern algorithm to obtain a texture distribution map; Converting each frame of image from RGB color space to HSV color space, and identifying image color by analyzing hue component and saturation component distribution of each frame of image to obtain a chromaticity distribution map; Marking the area of which the pixel value of each frame of image exceeds a preset brightness threshold value as an image high-brightness area; identifying a pixel communication area of each frame of image meeting preset conditions as a potential roadway boundary feature based on the edge feature, the texture distribution map, the chromaticity distribution map and the image high-brightness area; The preset conditions include that the gradient strength of the edge characteristic is higher than a preset edge strength threshold, the texture response value of the texture distribution diagram is in a preset rock wall texture range, the hue and saturation components of the chromaticity distribution diagram accord with preset roadway rock wall color characteristics, and the overlapping area ratio of the hue and saturation components of the chromaticity distribution diagram and the image high-brightness area is lower than a preset overlapping area threshold.
5. The method for controlling movement of a wheel type shuttle car based on image recognition according to claim 1, wherein the step of extracting a time sequence change feature set and a space vision feature set of the potential roadway boundary features in continuous multi-frame images comprises the following steps: Obtaining a time sequence change feature set by extracting the motion trail, brightness change data and color fluctuation degree of the potential roadway boundary features in the continuous multi-frame images; And extracting the local edge gradient strength and the local texture map of the potential roadway boundary features to obtain a space vision feature set.
6. The method for controlling movement of a wheel type shuttle car based on image recognition according to claim 5, wherein the step of obtaining the time sequence variation feature set by extracting the movement track, brightness variation data and color fluctuation degree of the potential roadway boundary feature in the continuous multi-frame images comprises the following steps: Tracking pixel displacement of the potential roadway boundary features in continuous multi-frame images through an optical flow method to obtain a motion trail of the potential roadway boundary features, and calculating the trail smoothness of the motion trail; Calculating the average brightness value of the local area where the potential roadway boundary feature is located in the continuous multi-frame image to obtain brightness change data of the potential roadway boundary feature, and calculating the brightness stability of the brightness change data; Obtaining the color fluctuation degree of the potential roadway boundary feature by calculating the standard deviation of the hue and saturation component average value of the local area where the potential roadway boundary feature is located in a continuous multi-frame; And generating a time sequence change characteristic set based on the track smoothness degree, the brightness stability degree and the color fluctuation degree.
7. The method for controlling movement of a wheel type shuttle car based on image recognition according to claim 5, wherein the step of obtaining the set of spatial vision features by extracting the local edge gradient strength and the local texture map of the boundary feature of the potential roadway comprises: Calculating the image gradient amplitude of the area where the potential roadway boundary features are located by adopting a Canny edge detection operator to obtain local edge gradient strength, and calculating the edge significance of the local edge gradient strength; Calculating texture feature descriptors of the areas where the potential roadway boundary features are located by adopting a local binary pattern algorithm to obtain a local texture map, and calculating the texture consistency degree of the local texture map; and generating a spatial vision feature set based on the edge saliency degree and the texture consistency degree.
8. The method for controlling movement of a wheel type shuttle car based on image recognition according to claim 5, wherein the step of determining a real roadway boundary from the potential roadway boundary features according to the time sequence variation feature set and the space vision feature set comprises: based on the time sequence change characteristic set and the space vision characteristic set, potential roadway boundary characteristics meeting preset quantitative judgment conditions are judged to be real roadway boundaries, wherein the quantitative judgment conditions comprise: The standard deviation of the displacement amount of the motion trail is lower than a preset motion fluctuation threshold value; the standard deviation of the brightness change data is lower than a preset brightness fluctuation threshold value; the color fluctuation degree is lower than a preset color fluctuation threshold value; the local edge gradient strength is in a preset strength judgment range; The Hamming distance variation of the local texture map is lower than a preset texture volatility threshold.
9. The method for controlling movement of a wheel type coal transporting shuttle car based on image recognition according to claim 1, wherein the step of generating the movement control command of the shuttle car according to the real roadway boundary comprises: fitting a central line of the current roadway according to the identified real roadway boundary; Acquiring current position information acquired by a positioning unit arranged on a shuttle car, and calculating pose deviation of the current position information relative to the central line; and generating a control instruction for adjusting the speed and the steering of the shuttle car based on the pose deviation.
10. The utility model provides a wheeled fortune coal shuttle motion control system based on image recognition which characterized in that includes: The image acquisition module is used for acquiring continuous image frames of the front roadway environment acquired by the image acquisition equipment arranged on the shuttle car; an image correction module for performing stabilization correction on the continuous image frames; the feature recognition module is used for recognizing the image edge, the image texture, the image color and the image high-brightness area of each frame of image in the corrected continuous image frames and determining potential roadway boundary features; the feature extraction module is used for extracting a time sequence change feature set and a space vision feature set of the potential roadway boundary features in the continuous multi-frame images; the boundary judging module is used for judging a real roadway boundary from the potential roadway boundary characteristics according to the time sequence change characteristic set and the space vision characteristic set; and the instruction generation module is used for generating a movement control instruction of the shuttle car according to the real roadway boundary.

Description

Image recognition-based wheeled coal shuttle car motion control method and system Technical Field The application relates to the technical field of vehicle control, in particular to a wheel type coal conveying shuttle car motion control method and system based on image recognition. Background The wheel type coal conveying shuttle car mainly captures roadway images through a vehicle-mounted camera in an underground mine, and utilizes an image analysis technology to identify roadway boundaries so as to realize autonomous navigation. In an ideal roadway environment with uniform drying and illumination, the recognition algorithm based on texture, color and contour features can work stably. However, the actual environment of the mine is complex and changeable, and a large-area wet area is often present on the roadway rock wall due to underground water infiltration. When the head lamp of the shuttle car irradiates on the wet surfaces, strong specular reflection is generated, and dynamic light spots with extremely high local brightness and rapid change of shape and position along with the movement of the vehicle are formed. This poses a serious challenge for visual navigation systems, on the one hand, highlighting spots of light is very likely to cause local overexposure of the camera, causing complete loss of key rock wall texture and contour details, and on the other hand, spot edges are easily misidentified as false roadway boundaries or obstacles. To solve this problem, an automatic exposure technique has been adopted, but the effect is poor. Because it is difficult to simultaneously consider the region with the great brightness difference in the tunnel, the overexposure region is enlarged by increasing the exposure, and the dark detail is lost by decreasing the exposure. More importantly, the vibration of the shuttle car in running and the change of the irradiation angle of the car lamp lead to the light spot to present rapid and irregular dynamic characteristics, so that the traditional method based on single-frame image analysis is difficult to stably and accurately extract the real roadway boundary, thereby causing navigation deviation, car scraping and even collision accidents. Disclosure of Invention The application provides a wheel type coal shuttle car motion control method and system based on image recognition, and aims to solve the technical problems that a roadway boundary is difficult to accurately recognize by a traditional image recognition method under a complex illumination environment of a mine, particularly when a moist rock wall generates a highlight reflection light spot, and vehicle navigation deviation and even safety accidents are caused. In a first aspect, the application provides a wheel type coal shuttle motion control method based on image recognition, which comprises the following steps: Acquiring continuous image frames of a front roadway environment acquired by image acquisition equipment mounted on a shuttle car; performing a stabilization correction on the continuous image frames; Identifying an image edge, an image texture, an image color and an image high-brightness area of each frame image in the corrected continuous image frames, and determining potential roadway boundary characteristics; Extracting a time sequence change feature set and a space vision feature set of the potential roadway boundary features in continuous multi-frame images; judging a real roadway boundary from the potential roadway boundary characteristics according to the time sequence change characteristic set and the space vision characteristic set; and generating a motion control instruction of the shuttle car according to the real roadway boundary. According to some embodiments of the application, the step of stabilizing the successive image frames comprises: acquiring vehicle motion data acquired by an inertial measurement unit mounted on a shuttle car; Calculating global pixel displacement of the successive image frames based on the vehicle motion data; and performing inverse geometric transformation on the continuous image frames according to the global pixel displacement to compensate image shake caused by shuttle car vibration, so as to obtain corrected continuous image frames. According to some embodiments of the application, the step of calculating global pixel displacement of the successive image frames based on the vehicle motion data comprises: carrying out Kalman filtering pretreatment on the vehicle motion data to obtain the angular speed and the linear acceleration of the shuttle car; Acquiring a coordinate transformation relation between a camera internal reference matrix calibrated in advance in the image acquisition equipment and an inertial measurement unit; converting the angular velocity and the linear acceleration into a motion transformation matrix of a camera coordinate system based on the coordinate transformation relation; And calculating global pixel displacement of the continuou