CN-116912317-B - Target detection and distance estimation method and device

CN116912317BCN 116912317 BCN116912317 BCN 116912317BCN-116912317-B

Abstract

The invention discloses a target detection and distance estimation method and device, wherein the method comprises the steps of collecting image information of a target by a camera; the method comprises the steps of obtaining distance information of a target and a camera, marking the target in image information to obtain position information of the target, marking the target in the image information to obtain distance information of the target according to the distance information of the target and the camera, optimizing a target detection and distance estimation network model to obtain an optimized target detection and distance estimation network model, training the optimized target detection and distance estimation network model to obtain a training target detection and distance estimation network model by utilizing the position information of the target and the distance information of the target, obtaining image information to be processed, and processing the image information to be processed by utilizing the training target detection and distance estimation network model to obtain target detection and distance estimation results. The invention only needs to perform one-time operation, and calculates the distance of the target while detecting the target.

Inventors

LONG ZHIZHOU
MA TIAN
CHEN JUNXIAN
LI WEIPING
TANG RONGFU

Assignees

中国人民解放军军事科学院系统工程研究院
威海众合机电科技有限公司

Dates

Publication Date: 20260512
Application Date: 20230714

Claims (9)

1. A method of target detection and distance estimation, the method comprising: S1, acquiring image information of a target by using a camera; S2, obtaining distance information of the target and the camera; s3, labeling the target in the image information to obtain the position information of the target; S4, marking the target in the image information according to the distance information of the target and the camera to obtain the distance information of the target; S5, optimizing the target detection and distance estimation network model to obtain an optimized target detection and distance estimation network model, wherein the method comprises the following steps: s51, modifying the dimension of the output feature map of the target detection and distance estimation network model to obtain an improved target detection and distance estimation network model; S52, introducing a distance information loss function; The distance information loss function is: In the formula, As a function of the loss of distance information, For the true distance to be the true distance, Predicting a distance for the network; S53, processing the distance information loss function to obtain a network model total loss function; s54, training the improved target detection and distance estimation network model by utilizing the network model total loss function to obtain an optimized target detection and distance estimation network model; s6, training the optimized target detection and distance estimation network model by utilizing the position information of the target and the distance information of the target to obtain a training target detection and distance estimation network model; S7, obtaining image information to be processed, and processing the image information to be processed by utilizing the training target detection and distance estimation network model to obtain target detection and distance estimation results.
2. The method of object detection and distance estimation according to claim 1, wherein the acquiring distance information of the object and a camera includes: storing image information of a target acquired by a camera, and storing a time stamp when the image is acquired; setting a high-frequency differential GPS to be synchronous with the clock of the camera; and acquiring distance information between the target and the camera by using the high-frequency differential GPS.
3. The method for detecting and estimating a distance according to claim 1, wherein labeling the target in the image information to obtain the position information of the target comprises: marking the target in the image information by using a marking tool to obtain the position information of the target; The location information includes coordinates of an upper left corner of the target, the width, and category information of the target.
4. The method for detecting and estimating a distance between a target and a camera according to claim 1, wherein labeling the target in the image information according to the distance information between the target and the camera to obtain the distance information of the target comprises: s41, acquiring a high-frequency differential GPS time stamp of the target and a time stamp of the target in the image information; S42, processing the high-frequency differential GPS time stamp of the target and the time stamp of the target in the image information to obtain time difference information; S43, presetting a time difference threshold, comparing the time difference information with the preset time difference threshold, and if the time difference information is smaller than the preset time difference threshold, processing the position information of the target to obtain the distance information of the target.
5. The target detection and distance estimation method of claim 1, wherein the network model total loss function is: Wherein: as a function of the loss of distance information, For the true distance to be the true distance, Predicting a distance for the network; As a function of the loss of interest, In order to output the size of the resolution, 、 Respectively representing the situation of the presence or absence of the real target, In order to predict the number of targets, Representing the probability that the network outputs the predicted target, Indicating the presence or absence of an actual target, Is a super parameter; Wherein, the Center coordinates representing the true position of the target and height and width, Representing the predicted target center position and height and width of the network output respectively, Is a position loss function; A loss function is predicted for the target location, The probability of predicting the target class c is output for the network, Representing the true probability that the position is the target c, The number of categories is classes.
6. The target detection and distance estimation method of claim 1, wherein the target detection and distance estimation network model comprises an input layer, a first processing layer, a second processing layer, a third processing layer, a fourth processing layer, a fifth processing layer, and an output layer; the dimension of the input layer is 512×512×3; The output of the input layer is the input of a first processing layer, and the dimension of the first processing layer is 256 multiplied by 16; the output of the first processing layer is the input of the second processing layer, and the dimension of the second processing layer is 128×128×32; The output of the second processing layer is the input of the third processing layer, and the dimension of the third processing layer is 64 multiplied by 64; The output of the third processing layer is the input of the fourth processing layer, the fourth treatment layer dimension of 32 x 32; the output of the fourth processing layer is the input of the fifth processing layer, the fifth treatment layer dimension 64 x 64; The output of the fifth processing layer is the input of the output layer, the dimension of the output layer is 32 x [3 x (6+n) ], and n is the number of target categories.
7. An object detection and distance estimation device, the device comprising: The image acquisition module is used for acquiring image information of a target by using a camera; the distance information acquisition module is used for acquiring the distance information of the target and the camera; The position information acquisition module is used for marking the target in the image information to obtain the position information of the target; The distance information acquisition module is used for marking the target in the image information according to the distance information of the target and the camera to obtain the distance information of the target; the network model optimizing module is used for optimizing the target detection and distance estimation network model to obtain an optimized target detection and distance estimation network model, and comprises the following steps: s51, modifying the dimension of the output feature map of the target detection and distance estimation network model to obtain an improved target detection and distance estimation network model; S52, introducing a distance information loss function; The distance information loss function is: In the formula, As a function of the loss of distance information, For the true distance to be the true distance, Predicting a distance for the network; S53, processing the distance information loss function to obtain a network model total loss function; s54, training the improved target detection and distance estimation network model by utilizing the network model total loss function to obtain an optimized target detection and distance estimation network model; The network model training module is used for training the optimized target detection and distance estimation network model by utilizing the position information of the target and the distance information of the target to obtain a training target detection and distance estimation network model; the target detection and distance estimation module is used for acquiring image information to be processed, and processing the image information to be processed by utilizing the training target detection and distance estimation network model to obtain target detection and distance estimation results.
8. An object detection and distance estimation device, the device comprising: a memory storing executable program code; a processor coupled to the memory; The processor invokes the executable program code stored in the memory to perform the object detection and distance estimation method according to any one of claims 1-6.
9. A computer-readable storage medium storing computer instructions that, when invoked, are operable to perform the object detection and distance estimation method of any one of claims 1-6.

Description

Target detection and distance estimation method and device Technical Field The invention relates to the technical fields of computer vision, mixed reality, automatic driving and artificial intelligence, in particular to a target detection and distance estimation method and device. Background Mixed reality technology (including augmented reality and virtual reality) is a novel man-machine interaction technology which has been rapidly developed and is in great attention in recent years. The mixed reality technology fuses the virtual world and the real world to create a visual environment where virtual and real objects coexist and are interactable in real time. The mixed reality essentially provides a brand new computer interaction interface and means, can be widely applied to military (such as military training, combat, maintenance and the like), industry (such as remote expert support, inspection, information visualization and the like), entertainment, education and the like, and is more likely to become a next generation computing platform following a mobile phone in the future. In the mixed reality technology, the background computer needs to superimpose a virtual picture on a real scene picture acquired by a camera, and since the content output to both eyes is stereoscopic, it is necessary to obtain distance information of a real world position where we want to superimpose so as to ensure that the virtual picture has parallax and presents a stereoscopic impression when output to both eyes. In this process, the target is detected from the image captured by the camera, and the distance from the target to the camera is estimated, and in the previous scheme, two different algorithms are often used to detect the target and estimate the distance respectively, so that a large amount of computing resources are consumed and a large time delay is caused. The algorithm only needs to perform one operation, and calculates the distance of the target while detecting the target. Disclosure of Invention The invention aims to solve the technical problem of providing a target detection and distance estimation method and device, and solving the problem that image target information and distance information need to be simultaneously given. In order to solve the above technical problems, a first aspect of an embodiment of the present invention discloses a target detection and distance estimation method, which includes: S1, acquiring image information of a target by using a camera; S2, obtaining distance information of the target and the camera; s3, labeling the target in the image information to obtain the position information of the target; S4, marking the target in the image information according to the distance information of the target and the camera to obtain the distance information of the target; S5, optimizing the target detection and distance estimation network model to obtain an optimized target detection and distance estimation network model; s6, training the optimized target detection and distance estimation network model by utilizing the position information of the target and the distance information of the target to obtain a training target detection and distance estimation network model; S7, obtaining image information to be processed, and processing the image information to be processed by utilizing the training target detection and distance estimation network model to obtain target detection and distance estimation results. In a first aspect of the embodiment of the present invention, the obtaining the distance information between the target and the camera includes: storing image information of a target acquired by a camera, and storing a time stamp when the image is acquired; setting a high-frequency differential GPS to be synchronous with the clock of the camera; and acquiring distance information between the target and the camera by using the high-frequency differential GPS. In a first aspect of the embodiment of the present invention, the marking the target in the image information to obtain the position information of the target includes: marking the target in the image information by using a marking tool to obtain the position information of the target; The location information includes coordinates of an upper left corner of the target, the width, and category information of the target. In a first aspect of the embodiment of the present invention, labeling the target in the image information according to the distance information between the target and the camera to obtain the distance information of the target includes: s41, acquiring a high-frequency differential GPS time stamp of the target and a time stamp of the target in the image information; S42, processing the high-frequency differential GPS time stamp of the target and the time stamp of the target in the image information to obtain time difference information; S43, presetting a time difference threshold, comparing the time difference information