Search

CN-116861984-B - Multitasking detection method, electronic device, vehicle and medium

CN116861984BCN 116861984 BCN116861984 BCN 116861984BCN-116861984-B

Abstract

The invention provides a multi-task detection method, electronic equipment, a vehicle and a medium, wherein the multi-task detection method comprises the steps of obtaining multi-task perception data; the method comprises the steps of inputting the multi-task perception data into a trained light multi-task model to output detection information corresponding to each task, wherein the light multi-task model is obtained by optimizing a baseline multi-task model based on each unit sensitivity level, and the baseline multi-task model is obtained by training based on the multi-task perception training data and multi-task target detection information. Based on each unit sensitivity level, the baseline multitasking model is optimized, different units in the model adopt different optimization modes, so that the multitasking model is more extremely light, the deployment time consumption, namely the calculation occupied resources, are reduced, and the multitasking detection efficiency and the detection result accuracy are improved.

Inventors

  • LU QIANG

Assignees

  • 嬴彻星创智能科技(上海)有限公司

Dates

Publication Date
20260508
Application Date
20230530

Claims (8)

  1. 1. A method for the detection of a plurality of tasks, characterized by comprising the following steps: Acquiring multi-task perception data, wherein the multi-task perception data comprises perception data of a camera, a laser radar and a millimeter wave radar; inputting the multitask sensing data into a trained light multitask model to output detection information corresponding to each task; The light-weight multi-task model is obtained by optimizing a baseline multi-task model based on each unit sensitivity level; the method for calculating the sensitivity level of each unit comprises the following steps: calculating the sensitivity of each layer by taking the layer as a unit; averaging the sensitivity of each layer contained in each block, and determining a sensitivity level of the corresponding block level according to the layer sensitivity average value; Calculating the duty ratio of blocks with sensitivity levels lower than a preset level threshold value in each unit, and determining the sensitivity level of each unit according to the calculated duty ratio of the blocks; Or alternatively, the first and second heat exchangers may be, Calculating the sensitivity of each block in units of blocks; averaging the sensitivity of each block contained in each unit, and determining the sensitivity level of each unit according to the sensitivity average value of the block; The optimizing the baseline multitasking model based on each unit sensitivity level comprises: If the block duty ratio of a sensitivity level which is a first level in a certain unit exceeds a duty ratio threshold value, model pruning is carried out on the unit; If the block duty ratio of the sensitivity level of the first level in a certain unit does not exceed the duty ratio threshold, and the block duty ratio of the sensitivity level of the first level and the second level in the certain unit exceeds the duty ratio threshold, performing model search optimization on the unit; If the block duty ratio of the sensitivity level of a certain unit, which is a first level and a second level, does not exceed the duty ratio threshold value, the unit is not optimized; the baseline multitasking model is obtained based on multitasking perception training data and multitasking target detection information.
  2. 2. The method of claim 1, wherein the baseline multitasking model includes a backbone network element, wherein a block duty cycle of a sensitivity level of a first level in the backbone network element does not exceed a duty cycle threshold, and wherein block duty cycles of a sensitivity level of the first level and a second level exceed the duty cycle threshold, and wherein performing model search optimization on the backbone network element comprises: Converting a backbone network element in the baseline multitasking model into a super-network structure according to a plurality of search dimensions; training the super-network structure; searching an optimal subnet in the trained super-network structure, and taking the optimal subnet as a new backbone network unit of the baseline multitasking model; training a baseline multitasking model including new backbone network elements to obtain a multitasking model with lightweight backbone network elements.
  3. 3. The method of multi-tasking detection according to claim 2, wherein said searching for the best subnet in the trained super-network structure comprises: randomly generating all subnets meeting preset limiting conditions; And calculating the reasoning result of each subnet and the manual calibration result, carrying out reasoning loss calculation, obtaining the reasoning loss value corresponding to each subnet, and taking the subnet with the minimum reasoning loss value as the optimal subnet.
  4. 4. The method of claim 2, wherein converting backbone network elements in the baseline multitasking model to a super-network structure comprises: And converting the corresponding layer of the backbone network unit into a subnet in the super-network structure according to the sensitivity of each layer in the backbone network unit.
  5. 5. The method of claim 1, wherein the baseline multitasking model includes a plurality of detection heads, wherein a block duty cycle of a first level of sensitivity levels of the plurality of detection heads exceeds a duty cycle threshold, and wherein model pruning the plurality of detection heads comprises: setting the pruning rate of a layer to be pruned of each detection head; pruning is carried out on the convolution kernel and the filter of the layer to be pruned according to the pruning rate; calculating the pruning rate of each layer of convolution kernel and the pruning rate of the filter; pruning all convolution kernels of the current layer if the pruning rate of the convolution kernels of the current layer exceeds a first preset threshold value; If the pruning rate of the filters of the current layer exceeds a second preset threshold, pruning all the filters of the current layer to obtain the multi-task model with the light detection head.
  6. 6. The multitasking detection method of claim 2, further comprising: obtaining a light-weight multi-task model according to the multi-task model with the light-weight main network unit and the multi-task model with the light-weight detection head; and performing fine tuning training on the light-weight multi-task model to obtain a trained light-weight multi-task model.
  7. 7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the multi-tasking detection method according to any of claims 1 to 6 when executing the program.
  8. 8. A vehicle comprising the electronic device of claim 7.

Description

Multitasking detection method, electronic device, vehicle and medium Technical Field The present invention relates to the field of automatic driving technologies, and in particular, to a multitasking detection method, an electronic device, a vehicle, and a medium. Background In the whole technical scheme of automatic driving, sensing is used as a ring of the bottommost layer, multiple information is often required to be output for downstream (planning control), and after sensing data (such as sensing data of a camera, a laser radar and a millimeter wave radar) are received by a sensing model, multiple information is required to be output simultaneously. For example, by detecting or identifying various elements in a road, such as lane lines, traffic lights, signs, road surface marks, and drivable areas, each of the elements can be used as a sensing task, and each task can share a backbone network through a multi-task model, so that feature maps obtained from the backbone network are shared, and the respective processing procedures of different tasks are performed based on the feature maps, thereby improving the overall detection efficiency. Because the multitasking model is complex, deployment is time consuming and computational resources are more involved. In the related art, the model weight reduction process is performed to reduce the calculation amount or the model size of the multitasking model. In the method for lightening the multi-task model, the whole model is usually pruned or NAS (Neural Architecture Search, neural network structure search) is performed on the whole model, and because the sensitivity level of each module in the model structure is different, the same processing mode is adopted for each module, for example, pruning is performed on each module in the model structure or NAS is performed on the whole model, which results in insufficient or excessive lightening of the multi-task model and influences on detection efficiency and detection result. Disclosure of Invention The invention provides a multitasking detection method, electronic equipment, a vehicle and a medium, which are used for solving the defect that the traditional multitasking model lightening method influences the output effect and the output efficiency of a model by adopting the same lightening mode for each unit or module in a model structure. The invention provides a multi-task detection method, which comprises the following steps: Acquiring multi-task perception data; inputting the multitask sensing data into a trained light multitask model to output detection information corresponding to each task; The light-weight multi-task model is obtained by optimizing a baseline multi-task model based on each unit sensitivity level; the baseline multitasking model is obtained based on multitasking perception training data and multitasking target detection information. According to the multi-task detection method provided by the invention, the sensitivity level calculation method of each unit comprises the following steps: calculating the sensitivity of each layer by taking the layer as a unit; averaging the sensitivity of each layer contained in each block, and determining a sensitivity level of the corresponding block level according to the layer sensitivity average value; Calculating the duty ratio of blocks with sensitivity levels lower than a preset level threshold value in each unit, and determining the sensitivity level of each unit according to the calculated duty ratio of the blocks; Or alternatively, the first and second heat exchangers may be, Calculating the sensitivity of each block in units of blocks; The sensitivity of each block contained in each unit is averaged, and the sensitivity level of each unit is determined according to the block sensitivity average. According to the multi-task detection method provided by the invention, the base line multi-task model is optimized based on each unit sensitivity level, and the method comprises the following steps: If the block duty ratio of a sensitivity level which is a first level in a certain unit exceeds a duty ratio threshold value, model pruning is carried out on the unit; If the block duty ratio of the sensitivity level of the first level in a certain unit does not exceed the duty ratio threshold, and the block duty ratio of the sensitivity level of the first level and the second level in the certain unit exceeds the duty ratio threshold, performing model search optimization on the unit; If the block duty ratios of the sensitivity level of a certain unit, which is the first level and the second level, do not exceed the duty ratio threshold value, the unit is not optimized. According to the method for detecting the multiple tasks provided by the invention, the baseline multiple task model comprises a main network unit, the block duty ratio of the sensitivity level of the main network unit with the first level does not exceed a duty ratio threshold, the block duty rati