Search

CN-122023964-A - Image data mining method and system for corner cases in automatic driving field

CN122023964ACN 122023964 ACN122023964 ACN 122023964ACN-122023964-A

Abstract

The invention discloses an image data mining method and system for corner cases in the automatic driving field, which belong to the technical field of data processing, wherein natural language is used for determining data mining requirements corresponding to an image data set, the data mining requirements are disassembled through a large language model to obtain the number requirements and a plurality of requirement elements required by image content, mining tools of each requirement element are determined based on historical mining data, the mining priority corresponding to the mining tools of each requirement element is determined by acquiring the proportion of the number of pictures with the characteristics of each requirement element in the total number of pictures of the image data set, the smaller the proportion is, the higher the mining priority is, and data mining is performed according to the mining priority to obtain a data mining result. According to the method, the requirements are determined through natural language and the requirements are disassembled through a large language model, so that the requirement elements related to the extreme scenes can be accurately positioned, the mining process is more targeted, blind acquisition is avoided, and the acquisition difficulty is reduced.

Inventors

  • CAI BAO
  • GU HONGLIANG
  • SUN ZHANGCHI

Assignees

  • 上海第二工业大学

Dates

Publication Date
20260512
Application Date
20260127

Claims (10)

  1. 1. The image data mining method for the corner cases in the automatic driving field is characterized by comprising the following steps of: Acquiring an image dataset of a corner case to be mined, determining a data mining requirement corresponding to the image dataset by using natural language based on potential requirements of corresponding image processing tasks in the automatic driving field; Determining the mining priority corresponding to the mining tool of each demand element by acquiring the proportion of the number of pictures with the characteristics of each demand element in the image dataset to the total number of the pictures, wherein the mining priority is higher as the proportion is smaller; And performing data mining according to the mining priority corresponding to the mining tool of each requirement element to obtain a data mining result, and forming a mining data set according to the quantity requirement and the data mining result.
  2. 2. The method for mining image data of corner cases in the automatic driving field according to claim 1, wherein the method for mining image data is characterized in that the data mining requirement is disassembled through a large language model to obtain a plurality of requirement elements, wherein the requirement elements are instructions executed by a mining tool, and specifically include: Training a large language model serving as an inference engine based on a few-sample learning paradigm in prompt word engineering; The data mining requirement is disassembled into a plurality of requirements and requirement elements of a plurality of image content requirements through a trained large language model; Disassembling the data mining requirement into the number of images, the environmental condition, the target object and the density degree through a large language model; the number of images is taken as a quantity requirement, and the environmental condition, the target object and the density are taken as requirement factors of the image content requirement.
  3. 3. The method for mining image data of corner cases in the autopilot area according to claim 1, wherein the determining the mining tool for each demand element based on the historical mining data specifically comprises: based on historical mining data, guiding a large language model to understand the intention of input data mining requirements by providing a small amount of examples, and obtaining a plurality of requirement elements by disassembling comprises generating corresponding text search keywords, picture search keywords, model verification instructions and counter configuration; The mining tools corresponding to the text search keywords, the picture search keywords, the model checking instructions and the counter configuration comprise a text search model, a picture search model, a checking model and a counter.
  4. 4. The method for mining image data of corner cases in the autopilot area according to claim 1, wherein the determining the mining priority corresponding to the mining tool of each demand element by obtaining the proportion of the number of images with characteristics of each demand element in the image dataset to the total number of images, wherein the smaller the proportion is, the higher the mining priority is, specifically includes: The method comprises the steps that based on the determined excavation tools of all the requirement elements, the functions and the characteristics of different excavation tools are combined, and the number of pictures with all the requirement element characteristics in an image data set is obtained through excavation; taking the proportion of the number of pictures of each demand element characteristic in the image data set to the total number of the pictures as the effective mining efficiency; And determining the excavating priority corresponding to the excavating tool of each requirement element according to the size of the excavating effective rate, wherein the smaller the proportion is, the smaller the excavating effective rate is, which indicates that the higher the excavating priority is.
  5. 5. The method for mining image data of corner cases in the automatic driving field according to claim 4, wherein the mining data is obtained by performing data mining according to mining priorities corresponding to mining tools of each requirement element, and the mining data set is formed according to the number requirement and the data mining result, specifically comprising: When the demand factors include the number of images, the environmental condition, the target object and the degree of density, and the mining priority of the corresponding mining tool is ranked as the highest priority of the text search model for performing semantic search on the environmental condition, the highest priority of the picture search model and the counter for identifying the target object and counting the number of images, and the lowest priority of the verification model for verifying the degree of density: The text search model is utilized to screen the semantic relevance of the environmental condition with high priority, and a picture level vector of each image in the image dataset is generated; The method comprises the steps of using a picture search model to identify target objects with medium priority and count the number of images to generate object level vectors corresponding to images screened with high priority, obtaining boundary frames and category information of each target object in the images screened with high priority through unsupervised or semi-supervised learning detection and identification according to the object level vectors, and simultaneously counting the number of detected target objects in each initially screened image through a counter, and obtaining the images with medium priority when the number of target objects reaches a preset threshold value; Checking the density degree of the acquired low-priority images by adopting a checking model, and determining the relative position of each target object; when the relative positions reach a preset threshold value of the density degree, outputting images meeting the quantity requirement as a mining data set.
  6. 6. The method for mining image data of corner cases in the automatic driving field according to claim 5, wherein the image search model is used to count the number of images and the object recognition of the middle priority, and the object level vector corresponding to the image for generating the high priority screening is specifically generated by the CLIP model, and the text embedding vector related to the environmental condition semantics is generated.
  7. 7. The method for mining image data of corner cases in the automatic driving field according to claim 5, wherein the text search model is used to screen semantic relevance of environmental conditions with high priority, and the image level vector of each image in the image dataset is generated, specifically, the image embedding vector corresponding to object recognition is generated through DINO model.
  8. 8. The method for mining image data of corner cases in the autopilot area of claim 5 wherein said employing a verification model to verify the intensity of the acquired low priority images further comprises: Obtaining the similarity between the text embedded vector and all the image embedded vectors; And carrying out logic consistency and knowledge verification on the images meeting the similarity threshold requirements based on the verification model.
  9. 9. The method for mining image data of corner cases in the autopilot area of claim 5 wherein said outputting images meeting said quantity requirement further comprises: and carrying out external feedback on the output images meeting the quantity requirement, wherein: when the output images meeting the number requirements meet the preset image number threshold requirements, auditing the images with the corresponding number of the mining priorities; And when the output images meeting the number requirements do not meet the preset image number threshold requirements, re-disassembling the requirement elements, planning the excavating tool and excavating data.
  10. 10. An image data mining system for corner cases in the field of autopilot, comprising: The system comprises a demand element disassembly module, a data mining requirement and a data processing module, wherein the demand element disassembly module is used for acquiring an image data set of a corner case to be mined, determining the data mining requirement corresponding to the image data set by using natural language based on the potential requirement of the corresponding image processing task in the automatic driving field; The system comprises a mining priority determining module, a processing module and a processing module, wherein the mining priority determining module is used for determining the mining tool of each demand element based on historical mining data; the data mining module is used for performing data mining according to the mining priority corresponding to the mining tool of each requirement element to obtain a data mining result, and forming a mining data set according to the quantity requirement and the data mining result.

Description

Image data mining method and system for corner cases in automatic driving field Technical Field The invention relates to the technical field of data processing, in particular to an image data mining method and system for corner cases in the automatic driving field. Background In development and application of the autopilot technology, the key of realizing the system from "available" to "reliable" is to improve the robustness and generalization capability of the algorithm. The corner cases are used as scenes with low probability but high risk in an automatic driving system, and are key factors affecting the safety and reliability of the system. Data mining of corner cases (EDGE CASES) refers to the accurate identification, extraction and analysis of rare, complex or extreme scenarios from massive driving data by systematic methods. These scenarios are often beyond the scope of conventional testing, but are critical to the performance optimization of the autopilot system. The key links of perception, decision and control of an automatic driving system are that a large number of driving scene pictures meeting specific conditions are required to be used as data support, and especially various corner cases deviating from the conventional driving scene are required. The capability of the algorithm for coping with complex situations can be effectively improved by collecting, excavating and marking the corner case data, so that the safety of an automatic driving system in actual operation is ensured. However, the traditional data mining method mainly relies on manual collection, and the process is time-consuming and is affected by personal understanding difference, so that the demand grasping is inaccurate. Especially in the technical field of automatic driving, corner cases need to cover various extreme scenes, such as rare weather, special illumination, abnormal traffic conditions and the like, and the difficulty of manual acquisition is greatly increased due to the rarity and unpredictability of the extreme scenes and the complexity of scene simulation. Disclosure of Invention Aiming at the problems in the field, the invention provides the image data mining method and the system for the corner cases in the automatic driving field, which are characterized in that the data mining requirements are disassembled, and the planning and the data mining of the mining tool are sequentially carried out according to the requirement elements obtained by the disassembly, so that the mining process is more targeted, the blind acquisition is avoided, and the acquisition difficulty is reduced. In order to solve the technical problems, the invention discloses an image data mining method for corner cases in the automatic driving field, which comprises the following steps: Acquiring an image dataset of a corner case to be mined, determining a data mining requirement corresponding to the image dataset by using natural language based on potential requirements of corresponding image processing tasks in the automatic driving field; Determining the mining priority corresponding to the mining tool of each demand element by acquiring the proportion of the number of pictures with the characteristics of each demand element in the image dataset to the total number of the pictures, wherein the mining priority is higher as the proportion is smaller; And performing data mining according to the mining priority corresponding to the mining tool of each requirement element to obtain a data mining result, and forming a mining data set according to the quantity requirement and the data mining result. Preferably, the data mining requirement is disassembled through a large language model to obtain a plurality of requirement elements, wherein the requirement elements are instructions executed by a mining tool, and the instructions specifically include: Training a large language model serving as an inference engine based on a few-sample learning paradigm in prompt word engineering; The data mining requirement is disassembled into a plurality of requirements and requirement elements of a plurality of image content requirements through a trained large language model; Disassembling the data mining requirement into the number of images, the environmental condition, the target object and the density degree through a large language model; the number of images is taken as a quantity requirement, and the environmental condition, the target object and the density are taken as requirement factors of the image content requirement. Preferably, the determining the mining tool of each requirement element based on the historical mining data specifically includes: based on historical mining data, guiding a large language model to understand the intention of input data mining requirements by providing a small amount of examples, and obtaining a plurality of requirement elements by disassembling comprises generating corresponding text search keywords, picture search keywo