Search

US-12620053-B2 - Image processing method and apparatus, medium, device and driving system

US12620053B2US 12620053 B2US12620053 B2US 12620053B2US-12620053-B2

Abstract

The present disclosure provides an image processing method and apparatus, a system, a medium, a device and a driving system, which belong to the field of data processing technologies. The image processing method mainly includes: according to perception target features of each perception target feature class in a plurality of predetermined perception target feature classes, and a pixel requirement of a neural network image perception model corresponding to each perception target feature class in a plurality of predetermined neural network image perception models, performing corresponding adjustment to a current environmental image to obtain a plurality of current environmental feature images; by using the corresponding neural network image perception model, perceiving a perception target in each current environmental feature image to obtain a plurality of perception results; and fusing the plurality of perception results to obtain perception result data of the current environmental image. The present disclosure can improve perception accuracy.

Inventors

  • Jun Gong

Assignees

  • Momenta (suzhou) Technology Co., Ltd.

Dates

Publication Date
20260505
Application Date
20230907
Priority Date
20220228

Claims (11)

  1. 1 . An image processing method, comprising: according to perception target features of each perception target feature class in a plurality of predetermined perception target feature classes, and a pixel requirement of a neural network image perception model corresponding to each perception target feature class in a plurality of predetermined neural network image perception models, performing corresponding adjustment to a current environmental image to obtain a plurality of current environmental feature images; by using the corresponding neural network image perception model, perceiving a perception target in each current environmental feature image to obtain a plurality of perception results, and, fusing the plurality of perception results to obtain perception result data of the current environmental image; wherein the plurality of perception target feature classes are in one-to-one correspondence with the plurality of neural network image perception models; wherein according to the perception target features of each perception target feature class in a plurality of predetermined perception target feature classes, and the pixel requirement of a neural network image perception model corresponding to each perception target feature class in a plurality of predetermined neural network image perception models, performing corresponding adjustment to the current environmental image to obtain a plurality of current environmental feature images comprises: according to an orientation of each perception target feature class in the current environmental image and a sharpness of each perception target feature class, determining a feature class portion of the corresponding current environmental feature image; according to a perception pixel upper limit of the neural network image perception model corresponding to each perception target feature class and a pixel of the current environmental image, determining a feature class pixel of the corresponding current environmental feature image; and, according to the feature class portion and the feature class pixel, adjusting the current environmental image to obtain the current environmental feature images.
  2. 2 . The image processing method of claim 1 , wherein according to the feature class portion and the feature class pixel, adjusting the current environmental image to obtain the current environmental feature images comprises: performing cropping and/or scaling for the current environmental image with a pixel greater than the perception pixel upper limit of the neural network image perception model corresponding to each perception target feature class to obtain the current environmental feature images of the current environmental image.
  3. 3 . The image processing method of claim 2 , wherein performing cropping and/or scaling for the current environmental image with the pixel greater than the perception pixel upper limit of the neural network image perception model corresponding to each perception target feature class to obtain the current environmental feature images of the current environmental image comprises: performing cropping for the current environmental image to retain the sharpness of the perception target feature class, and then determining whether the pixel of the cropped current environmental image is greater than the perception pixel upper limit of the corresponding neural network image perception model; if yes, performing scaling for the cropped current environmental image, and if no, determining the cropped current environmental image as the current environmental feature image; or, firstly, performing scaling for the current environmental image to retain an integrity of the perception target feature class, and then determining whether the pixel of the scaled current environmental image is greater than the perception pixel upper limit of the corresponding neural network image perception model; if yes, performing cropping for the scaled current environmental image, and if no, determining the scaled current environmental image as the current environmental feature image.
  4. 4 . The image processing method of claim 1 , wherein fusing the plurality of perception results to obtain the perception result data of the current environmental image comprises: associating perception target information in the plurality of perception results with corresponding pixel positions of the current environmental image to obtain the perception result data of the current environmental image.
  5. 5 . The image processing method of claim 1 , wherein, the plurality of perception target feature classes comprise a first perception target feature class and a second perception target feature class, a perception target in the first perception target feature class comprises an obstacle, and the second perception target feature class comprises a lane line; the plurality of neural network image perception models comprise an obstacle perception model corresponding to the first perception target feature class and a lane line perception model corresponding to the second perception target feature class.
  6. 6 . An image processing apparatus, comprising: one or more processors, and a non-transitory storage medium in communication with the one or more processors, the non-transitory storage medium configured to store program instructions, wherein, when executed by the one or more processors, the instructions cause the apparatus to perform steps of: performing corresponding adjustment to a current environmental image to obtain a plurality of current environmental feature images, according to perception target features of each perception target feature class in a plurality of predetermined perception target feature classes, and a pixel requirement of a neural network image perception model corresponding to each perception target feature class in a plurality of predetermined neural network image perception models; perceiving a perception target in each current environmental feature image to obtain a plurality of perception results, by using the corresponding neural network image perception model, and, fusing the plurality of perception results to obtain perception result data of the current environmental image; wherein the plurality of perception target feature classes are in one-to-one correspondence with the plurality of neural network image perception models; wherein according to the perception target features of each perception target feature class in a plurality of predetermined perception target feature classes, and the pixel requirement of a neural network image perception model corresponding to each perception target feature class in a plurality of predetermined neural network image perception models, performing corresponding adjustment to the current environmental image to obtain a plurality of current environmental feature images comprises: according to an orientation of each perception target feature class in the current environmental image and a sharpness of each perception target feature class, determining a feature class portion of the corresponding current environmental feature image; according to a perception pixel upper limit of the neural network image perception model corresponding to each perception target feature class and a pixel of the current environmental image, determining a feature class pixel of the corresponding current environmental feature image; and, according to the feature class portion and the feature class pixel, adjusting the current environmental image to obtain the current environmental feature images.
  7. 7 . The image processing apparatus of claim 6 , wherein according to the feature class portion and the feature class pixel, adjusting the current environmental image to obtain the current environmental feature images comprises: performing cropping and/or scaling for the current environmental image with a pixel greater than the perception pixel upper limit of the neural network image perception model corresponding to each perception target feature class to obtain the current environmental feature images of the current environmental image.
  8. 8 . The image processing apparatus of claim 7 , wherein performing cropping and/or scaling for the current environmental image with the pixel greater than the perception pixel upper limit of the neural network image perception model corresponding to each perception target feature class to obtain the current environmental feature images of the current environmental image comprises: performing cropping for the current environmental image to retain the sharpness of the perception target feature class, and then determining whether the pixel of the cropped current environmental image is greater than the perception pixel upper limit of the corresponding neural network image perception model; if yes, performing scaling for the cropped current environmental image, and if no, determining the cropped current environmental image as the current environmental feature image; or, firstly, performing scaling for the current environmental image to retain an integrity of the perception target feature class, and then determining whether the pixel of the scaled current environmental image is greater than the perception pixel upper limit of the corresponding neural network image perception model; if yes, performing cropping for the scaled current environmental image, and if no, determining the scaled current environmental image as the current environmental feature image.
  9. 9 . The image processing apparatus of claim 6 , wherein fusing the plurality of perception results to obtain the perception result data of the current environmental image comprises: associating perception target information in the plurality of perception results with corresponding pixel positions of the current environmental image to obtain the perception result data of the current environmental image.
  10. 10 . The image processing apparatus of claim 6 , wherein, the plurality of perception target feature classes comprise a first perception target feature class and a second perception target feature class, a perception target in the first perception target feature class comprises an obstacle, and the second perception target feature class comprises a lane line; the plurality of neural network image perception models comprise an obstacle perception model corresponding to the first perception target feature class and a lane line perception model corresponding to the second perception target feature class.
  11. 11 . A non-transitory computer readable storage medium, storing a computer instruction wherein the computer instruction is executed to enable a computer to perform the image processing method, which comprises: according to perception target features of each perception target feature class in a plurality of predetermined perception target feature classes, and a pixel requirement of a neural network image perception model corresponding to each perception target feature class in a plurality of predetermined neural network image perception models, performing corresponding adjustment to a current environmental image to obtain a plurality of current environmental feature images; by using the corresponding neural network image perception model, perceiving a perception target in each current environmental feature image to obtain a plurality of perception results, and, fusing the plurality of perception results to obtain perception result data of the current environmental image; wherein the plurality of perception target feature classes are in one-to-one correspondence with the plurality of neural network image perception models; wherein according to the perception target features of each perception target feature class in a plurality of predetermined perception target feature classes, and the pixel requirement of a neural network image perception model corresponding to each perception target feature class in a plurality of predetermined neural network image perception models, performing corresponding adjustment to the current environmental image to obtain a plurality of current environmental feature images comprises: according to an orientation of each perception target feature class in the current environmental image and a sharpness of each perception target feature class, determining a feature class portion of the corresponding current environmental feature image; according to a perception pixel upper limit of the neural network image perception model corresponding to each perception target feature class and a pixel of the current environmental image, determining a feature class pixel of the corresponding current environmental feature image; and, according to the feature class portion and the feature class pixel, adjusting the current environmental image to obtain the current environmental feature images.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of International Application No. PCT/CN2022/100666, filed on Jun. 23, 2022, which claims priority to Chinese Patent Application No. 202210185274.7, filed on Feb. 28, 2022. The disclosures of the aforementioned applications are hereby incorporated by reference in their entireties. TECHNICAL FIELD The present disclosure relates to the field of data processing technologies, and in particular to an image processing method and apparatus, a medium, a device and a driving system. BACKGROUND At present, image analysis and processing are applied more and more widely, for example, applied to positioning, recognition and tracking etc. of a target in a practical scenario. During image analysis and processing, it is required to obtain perception information by perceiving a target and then use the perception information to perform corresponding analysis and processing, where an accuracy of perceiving a target in an image directly affects an accuracy of a result of the image analysis and processing. Therefore, one set of accurate image target perception solution is needed. SUMMARY In order to address the problems in the prior arts, the present disclosure provides an image processing method and apparatus, a system, a medium, a device and a driving system, where segmentation is performed for a raw environmental image based on characteristics of different perception target classes and perception is performed using a corresponding perception model, and then perception results are fused to obtain a perception result of the raw environmental image so as to improve perception efficiency and accuracy. According to a first aspect of embodiments of the present disclosure, there is provided an image processing method, including: according to perception target features of each perception target feature class in a plurality of predetermined perception target feature classes, and a pixel requirement of a neural network image perception model corresponding to each perception target feature class in a plurality of predetermined neural network image perception models, performing corresponding adjustment to a current environmental image to obtain a plurality of current environmental feature images; by using the corresponding neural network image perception model, perceiving a perception target in each current environmental feature image to obtain a plurality of perception results, and, fusing the plurality of perception results to obtain perception result data of the current environmental image; where the plurality of perception target feature classes are in one-to-one correspondence with the plurality of neural network image perception models. According to a second aspect of embodiments of the present disclosure, there is provided an image processing apparatus, including: an image adjusting module, configured to, according to perception target features of each perception target feature class in a plurality of predetermined perception target feature classes, and a pixel requirement of a neural network image perception model corresponding to each perception target feature class in a plurality of predetermined neural network image perception models, perform corresponding adjustment to a current environmental image to obtain a plurality of current environmental feature images; a perceiving module, configured to, by using the corresponding neural network image perception model, perceive a perception target in each current environmental feature image to obtain a plurality of perception results, and, a fusing module, configured to fuse the plurality of perception results to obtain perception result data of the current environmental image; where the plurality of perception target feature classes are in one-to-one correspondence with the plurality of neural network image perception models. According to a third aspect of embodiments of the present disclosure, there is provided a driving system, which includes the image processing apparatus in the above solution. According to a fourth aspect of embodiments of the present disclosure, there is provided a computer readable storage medium, storing a computer instruction, where the computer instruction is executed to perform the image processing method in the above solution. According to a fifth aspect of embodiments of the present disclosure, there is provided a computer device, including a processor and a memory, where the memory stores a computer instruction and the computer instruction is executed to perform the image processing method in the above solution. The technical solution of the present disclosure can achieve the following beneficial effects: in the image processing method and apparatus, the system, the medium, the device and the driving system, segmentation is performed for a raw environmental image based on characteristics of different perception target classes and perception is performed using a corresponding perce