CN-122023217-A - Visual sense sensing information preprocessing method, system, equipment and medium applied to nuclear emergency robot

CN122023217ACN 122023217 ACN122023217 ACN 122023217ACN-122023217-A

Abstract

The application discloses a vision sensing information preprocessing method, a system, equipment and a medium applied to a nuclear emergency robot, relates to the technical field of image processing, solves the technical problem of poor image quality of the nuclear emergency robot, the technical scheme is characterized in that the cooperative optimization of low-illumination enhancement and a super-resolution algorithm effectively improves the image detail reduction capability in extreme environments, enables the structural similarity of dark areas to reach more than 90%, and improves the irradiation resistance of the vision module to a certain extent at the algorithm level. The framework disclosed by the application can be applied to a nuclear emergency robot vision system, can be popularized to other emergency low-illumination radiation field scenes, and solves the problem of low return imaging quality of a traditional vision system.

Inventors

HE JIAPENG
JIA LINSHENG
WANG NING
LIANG BONING
YANG YAPENG
DU QINGBO
LIU LIYE
Dong Zhuben
LV QI
ZHAO XIAOYU
TANG YUYAO
YU HAO
LIU ZHE
FENG ZONGYANG

Assignees

中国辐射防护研究院

Dates

Publication Date: 20260512
Application Date: 20251229

Claims (10)

1. The vision sensing information preprocessing method applied to the nuclear emergency robot is characterized by comprising the following steps of: Step S1, acquiring an original image under a radiation condition by a nuclear emergency robot; Step S2, carrying out non-overlapping sub-block division on an original image, and dynamically dividing a normal region, a dark region and an overexposure region according to the pixel mean value and the pixel variance of each sub-block to obtain a normal region sub-block, a dark region sub-block and an overexposure region sub-block; Step S3, respectively carrying out low-illumination enhancement on the sub-blocks in the dark area and the sub-blocks in the overexposed area, and then carrying out multi-scale feature fusion on the sub-blocks subjected to low-illumination enhancement and the adjacent sub-blocks and optimizing in a dynamic range to obtain a first enhanced image; s4, performing anti-flicker super-resolution reconstruction on the first enhanced image to obtain a final enhanced image; The method comprises the steps of carrying out low-illumination enhancement on a dark region sub-block by a dynamic gamma correction method and an iterative Retinex algorithm, and carrying out low-illumination enhancement on an overexposed region sub-block by a bilateral filtering method and a dynamic range compression method; The step S4 includes: Step S41 image sequence for the first enhanced image Carrying out single-frame preprocessing, and carrying out pixel normalization, tensor conversion and boundary filling on each frame of image to obtain a preprocessed image frame; step S42, inter-frame alignment and motion supplementation of the current image frame and the image frame of the previous frame are carried out, so that a low-resolution aligned frame is obtained; S43, performing feature extraction and upsampling on the low-resolution alignment frame through a lightweight Real-ESRGAN model to obtain a high-resolution superframe; And S44, carrying out inter-frame weighted average on continuous frames in the high-resolution super-resolution frames to realize fluctuation smoothing, and obtaining a final enhanced image.
2. The method for preprocessing visual sensing information according to claim 1, wherein the low-illumination enhancement of the dark-area sub-block by a dynamic gamma correction method and an iterative Retinex algorithm comprises: Step S311, correcting the gamma value of each sub-block in the dark area to make the gamma value of each sub-block within a preset range, wherein the gamma value is expressed as: ; Wherein, the Representing coordinates in a sub-block of a dark area A corrected gamma value at which the gamma value is calculated, Represents the maximum value of the gamma value within a preset range, Represents the minimum value of the gamma value within a preset range, A pixel reference value representing a dark area, Representing coordinates in a sub-block of a dark area The mean value of the pixels at the point, A row index is represented and a column index is represented, Representing a column index; Step S312, normalizing the gamma value corrected sub-block to obtain a normalized sub-block ; Step S313, initializing the reflection component of the normalized sub-block, expressed as: , wherein, Representing the initialized reflection component; step S314, according to the iterative Retinex algorithm and based on the initialized reflection component For normalized subblocks The illumination component and the reflection component are iteratively updated to update the illumination component and remove the illumination component, and finally an enhanced reflection component is obtained, wherein the iterative Retinex algorithm is expressed as follows: ; Wherein, the Representing the enhanced sub-block; representing the reflected component of the light, Representing the illumination component and when the illumination component is removed, ; Step 315, normalizing the enhanced reflection component; And S316, carrying out inverse normalization on the normalized enhanced reflection component to obtain an enhanced dark region sub-block.
3. The method for preprocessing visual sensing information according to claim 2, wherein in step S314, the method is based on the initialized reflection component according to an iterative Retinex algorithm For normalized subblocks The iterative updating of the illumination component and the reflection component of (a) includes: ; ; Wherein, the Represent the first The illumination component obtained after a number of iterations, The gaussian kernel is represented by the number of gaussian kernels, Represent the first The reflected component at the time of the iteration, ; Represent the first The reflection component obtained after the iteration is carried out for a plurality of times; wherein the illumination constraint is expressed as: 。
4. The method for preprocessing visual sensing information according to claim 3, wherein in step S315, said normalizing the enhanced reflection component is expressed as: ; Wherein, the Representing the normalized enhanced reflection component; Representing the minimum value of the reflected component in the iterative process, Representing the maximum value of the reflected component in the iterative process.
5. The method for preprocessing visual sensing information according to claim 4, wherein said performing low-illumination enhancement on the overexposed region sub blocks by a bilateral filtering method and a dynamic range compression method comprises: step S321, bilateral filtering is carried out on the sub-blocks of the overexposed region, and the obtained filtered sub-blocks are expressed as: ; Wherein, the Representing coordinates in sub-blocks of overexposed areas The output pixel value at which, Representing coordinates in an original image Pixel value at location, coordinates Representing coordinates The coordinates of the pixels in the neighborhood, Representing the neighborhood row index, Representing a neighborhood column index; Weight coefficients representing bilateral filtering, and: ; Wherein, the The spatial kernels are represented and the spatial kernels are represented, Representing a value range kernel; And S322, normalizing the filtered sub-blocks, performing power compression on the dynamic range of the normalized filtered sub-blocks to obtain compressed sub-blocks, and performing inverse normalization on the compressed sub-blocks to obtain the enhanced overexposed region sub-blocks.
6. The method for preprocessing visual sensing information according to claim 5, wherein said step S42 comprises: Step S421, performing gray level conversion on the current image frame and the image frame of the previous frame, and aligning the coordinates of the image frame of the previous frame to the coordinates of the current image frame according to pyramid LK optical flow estimation to obtain a low resolution aligned frame; Step S422, dividing a static area and a dynamic area of the low-resolution aligned frame according to the amplitude of the optical flow field, respectively performing motion compensation on the static area and the dynamic area to obtain a compensated static area and a compensated dynamic area, and finally merging the compensated static area and the compensated dynamic area according to a static mask and a dynamic mask to obtain a low-resolution compensated frame; Wherein the optical flow field amplitude is expressed as: ; Representing the magnitude of the optical flow field, When the amplitude of the optical flow field is larger than a preset threshold value, judging the area as a dynamic area, otherwise, judging the area as a static area; the compensated static area is expressed as: ; The compensated dynamic region is expressed as: ; Wherein, the Representing the static area after compensation, Representing the compensated dynamic region; A current image frame is represented and, Representing the previous image frame.
7. The method for preprocessing visual sensing information as set forth in claim 6, wherein said step S43 comprises: step S431, extracting features of the low-resolution aligned frames to obtain basic texture features; step S431, the basic texture features sequentially pass through a residual block and a channel attention module to obtain enhanced features; step S431, up-sampling the enhanced features to obtain a super-resolution feature map; And step S431, the super-resolution feature map is mapped back to RGB dimension through output layer convolution, and then the super-resolution frame is obtained through inverse normalization.
8. A vision-sensing information preprocessing system applied to a nuclear emergency robot, which is used for the vision-sensing information preprocessing method applied to a nuclear emergency robot as set forth in any one of claims 1 to 7, characterized in that the system comprises: The acquisition module is used for acquiring an original image under the radiation condition through the nuclear emergency robot; the first preprocessing module is used for carrying out non-overlapping sub-block division on the original image, and carrying out dynamic division on a normal region, a dark region and an overexposure region according to the pixel mean value and the pixel variance of each sub-block to obtain a normal region sub-block, a dark region sub-block and an overexposure region sub-block; the low-illumination enhancement module is used for respectively carrying out low-illumination enhancement on the dark area sub-block and the overexposed area sub-block, then carrying out multi-scale feature fusion on the sub-block subjected to low-illumination enhancement and the adjacent sub-block, and optimizing in a dynamic range to obtain a first enhanced image; The anti-flicker super-resolution reconstruction module is used for performing super-resolution reconstruction on the first enhanced image to obtain a final enhanced image; The anti-flicker super-resolution reconstruction module comprises: a second preprocessing module for preprocessing the image sequence of the first enhanced image Carrying out single-frame preprocessing, and carrying out pixel normalization, tensor conversion and boundary filling on each frame of image to obtain a preprocessed image frame; the space-time alignment module aligns and supplements the motion between the current image frame and the image frame of the previous frame, thereby obtaining a low-resolution aligned frame; The lightweight super-resolution network performs feature extraction and upsampling on low-resolution alignment through a lightweight Real-ESRGAN model to obtain a high-resolution super-resolution frame; And the anti-flicker processing module is used for carrying out inter-frame weighted average on continuous frames in the high-resolution super-resolution frames so as to realize fluctuation smoothing and obtain a final enhanced image.
9. Computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the visual sensory information preprocessing method applied to a nuclear emergency robot according to any one of claims 1 to 7 when the computer program is executed.
10. A computer storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the vision-sensing information preprocessing method applied to a nuclear emergency robot according to any one of claims 1 to 7.

Description

Visual sense sensing information preprocessing method, system, equipment and medium applied to nuclear emergency robot Technical Field The application relates to the technical field of image processing, in particular to a visual sense sensing information preprocessing method, a system, equipment and a medium applied to a nuclear emergency robot. Background The image processing of the existing nuclear emergency robot vision system is based on a traditional ISP (IMAGE SIGNAL Processor, image signal processing) processing pipeline, and comprises basic modules such as automatic exposure control (AEC, automatic Exposure Control), automatic White Balance (AWB), gamma correction, spatial domain denoising (such as BM3D algorithm) and the like, and the processing flow is fixed and is not optimized for a radiation special environment. However, the conventional ISP's fixed processing flow cannot effectively eliminate special ionizing radiation noise (such as random punctiform noise points and periodic scanning line interference) in a nuclear environment, so that the image signal-to-noise ratio is reduced below 15dB, while the general denoising algorithm (such as BM 3D) can excessively smooth details when filtering radiation noise, so that key characteristics such as pipeline cracks and meter readings are lost, low-illumination enhancement depends on simple histogram equalization, and the dynamic range is insufficient (DR <80 dB), so that details in a dark area are invisible or a high-light area is overexposed. In addition, the existing vision system architecture design adopts an original image return and background server processing mode, namely, a front-end camera directly outputs an image in a RAW or JPEG format, and the image is transmitted to a control center through wireless transmission, and then enhancement and recognition processing are completed by a server. However, the original image transmission of the traditional method needs to occupy more than 12Mbps bandwidth (1080p@30fps), data packet loss or delay jitter is easy to occur under the electromagnetic interference of a nuclear environment, and the background processing link delay (acquisition, transmission, processing and back display) exceeds 500ms, so that the response delay of remote operation is caused. Finally, the anti-interference means of the existing vision system mainly reduces the damage of radiation to camera hardware through physical protection (such as lead layer shielding), but does not solve the problem of image degradation caused by radiation from the algorithm level. However, physical protection tends to cause distortion of the lens and degradation of light transmittance, exacerbating image quality degradation. The image thermal noise variance is higher than the normal value because a thermal noise suppression algorithm is not adopted under the high-temperature working condition, and the working efficiency is low under the dark condition because no mature coping strategy is adopted under the low-light working condition. How to realize the low-illumination enhancement and super-resolution reconstruction of the visual sensing information of the nuclear emergency robot is to be solved. Disclosure of Invention The application provides a visual sense information preprocessing method, a system, equipment and a medium applied to a nuclear emergency robot, and the technical purpose is to enable the visual sense information of the nuclear emergency robot to realize the integrated processing of low-illumination enhancement and super-resolution reconstruction. The technical aim of the application is realized by the following technical scheme: A vision sensing information preprocessing method applied to a nuclear emergency robot comprises the following steps: Step S1, acquiring an original image under a radiation condition by a nuclear emergency robot; Step S2, carrying out non-overlapping sub-block division on an original image, and dynamically dividing a normal region, a dark region and an overexposure region according to the pixel mean value and the pixel variance of each sub-block to obtain a normal region sub-block, a dark region sub-block and an overexposure region sub-block; Step S3, respectively carrying out low-illumination enhancement on the sub-blocks in the dark area and the sub-blocks in the overexposed area, and then carrying out multi-scale feature fusion on the sub-blocks subjected to low-illumination enhancement and the adjacent sub-blocks and optimizing in a dynamic range to obtain a first enhanced image; s4, performing anti-flicker super-resolution reconstruction on the first enhanced image to obtain a final enhanced image; The method comprises the steps of carrying out low-illumination enhancement on a dark region sub-block by a dynamic gamma correction method and an iterative Retinex algorithm, and carrying out low-illumination enhancement on an overexposed region sub-block by a bilateral filtering method and a dynamic range com