CN-116340736-B - Heterogeneous sensor information fusion method and device

CN116340736BCN 116340736 BCN116340736 BCN 116340736BCN-116340736-B

Abstract

The invention discloses a heterogeneous sensor information fusion method and device, which comprises the steps of S01 obtaining data detected by two heterogeneous sensors, performing time and space registration to obtain registered data, S02 obtaining measurement information of the two heterogeneous sensors at the same moment from the registered data according to joint distribution states of the measurement information of the two heterogeneous sensors, performing association pairing to obtain an association observation pair, S03 determining a suspected measurement pair from the association observation pair according to joint distribution relations between the measurement information of the two heterogeneous sensors and noise variances, fusing the measurement information of the suspected measurement pair to obtain a fused suspected measurement pair, and S04 performing decision-level fusion recognition according to the fused suspected measurement pair to obtain a final recognition result. The method can realize heterogeneous sensor expression fusion, and has the advantages of simple realization method, high fusion efficiency, good adaptability to complex scenes, strong anti-interference performance and the like.

Inventors

LUO PENG
HAN NAIJUN
HE XIN

Assignees

湖南华诺星空电子技术有限公司

Dates

Publication Date: 20260505
Application Date: 20221226

Claims (10)

1. The heterogeneous sensor information fusion method is characterized by comprising the following steps of: S01, acquiring data detected by two heterogeneous sensors, and performing time and space registration to obtain registered data; S02, according to the joint distribution state of the measurement information of the two heterogeneous sensors, acquiring the measurement information of the two heterogeneous sensors at the same moment from the registered data to perform association pairing to obtain an association observation pair; S03, determining a suspected measurement pair from the associated observation pair according to a joint distribution relation between measurement information of two heterogeneous sensors and noise variance, and fusing measurement information of the suspected measurement pair to obtain a fused suspected measurement pair, wherein the suspected measurement pair is two heterogeneous sensor measurement values meeting the joint distribution in the associated observation pair; And S04, carrying out decision-stage identification according to the suspected measurement pair after fusion to obtain a final identification result.
2. The heterogeneous sensor information fusion method according to claim 1, wherein the measurement information includes azimuth angle and pitch angle, and the step S02 is to construct a first statistical decision T obeying chi-square distribution with a degree of freedom of 2 by using the azimuth angle and pitch angle obtained by the two heterogeneous sensors, when the first statistical decision T is greater than a preset first threshold When judging that the measured values of the two heterogeneous sensors to be judged are the associated observation pairs from the same position, otherwise judging that the measured values are not the associated observation pairs from the same position, wherein the first threshold is preset And the method is obtained according to chi-square distribution setting with the degree of freedom of 2.
3. The heterogeneous sensor information fusion method according to claim 2, wherein the calculation expression of the first statistical decision T is: , Wherein, the And Azimuth angle measurement values obtained by the first sensor and the second sensor respectively, And Pitch angle measurement values obtained by the first sensor and the second sensor respectively, The variances of the azimuth and pitch angle measurements obtained by the first sensor, The variance of the azimuth angle measurement value and the pitch angle measurement value obtained by the second sensor are respectively, and the first sensor and the second sensor are heterogeneous sensors.
4. The method according to claim 1, wherein in the step S03, determining suspected measurement pairs from the correlation observation pairs according to the joint distribution relation between the measurement information of the heterogeneous sensors and the noise variance comprises constructing chi-square distribution with a degree of freedom of 2 using azimuth angles and pitch angles obtained by the two heterogeneous sensors and azimuth angle measurement noise variances of the two heterogeneous sensors and pitch angle measurement noise variances of the two heterogeneous sensors Is used for the second statistical decision quantity of the (a), In order to misjudge the two different target observations as the probability of the same target observation, when the second statistical judgment quantity is larger than a preset second threshold Determining that the measurement values of the two heterogeneous sensors to be discriminated are suspected measurement pairs, wherein the preset second threshold Chi-square distribution according to degree of freedom 2 Setting to obtain the product.
5. The heterogeneous sensor information fusion method according to claim 4, wherein the calculation expression of the second statistical decision quantity is: Wherein, the Representation of Time of day (time) A second statistical decision between the first sensor measurement information and the j-th second sensor measurement information, Respectively is First sensor of moment Azimuth observation, second sensor's first Azimuth observation, first sensor's first Pitch angle observation and second sensor The individual pitch angles are observed and, The method comprises the steps of measuring noise variance of azimuth angles of a first sensor, measuring noise variance of pitch angles of the first sensor, measuring noise variance of azimuth angles of a second sensor and measuring noise variance of pitch angles of the second sensor, wherein the first sensor and the second sensor are heterogeneous sensors.
6. The heterogeneous sensor information fusion method according to any one of claims 1 to 5, wherein in the step S03, the azimuth angles of the suspected measurement pair are weighted by using the azimuth angle measurement noise variance of the two heterogeneous sensors to obtain a fused azimuth angle, and the pitch angles of the suspected measurement pair are weighted by using the pitch angle measurement noise variance of the two heterogeneous sensors to obtain a fused pitch angle.
7. The heterogeneous sensor information fusion method according to claim 6, wherein the post-fusion azimuth angle and the post-fusion pitch angle are respectively calculated according to the following formulas: , , Wherein, the And Respectively is A fused rear differential angle and a fused rear pitch angle at the moment, Respectively is First sensor of moment Azimuth observation, second sensor's first Azimuth observation, first sensor's first Pitch angle observation and second sensor The individual pitch angles are observed and, The method comprises the steps of measuring noise variance of azimuth angles of a first sensor, measuring noise variance of pitch angles of the first sensor, measuring noise variance of azimuth angles of a second sensor and measuring noise variance of pitch angles of the second sensor, wherein the first sensor and the second sensor are heterogeneous sensors.
8. The heterogeneous sensor information fusion method according to any one of claims 1 to 5, wherein in the step S04, decision-level recognition is performed by using an evidence theory method, wherein the mutual support degree and the collision strength between two heterogeneous sensor information are calculated by taking the target generic confidence degree in the recognition results obtained by two sensors as an evidence body, so as to measure the contribution degree of different sensor information to the final fusion information, and then the two heterogeneous sensor information is weighted according to the mutual support degree and the collision strength, so as to obtain the fusion result.
9. The heterogeneous sensor information fusion method of claim 8, wherein the step of decision-level fusion recognition using evidence theory methods comprises: S401, initializing parameters, namely taking the confidence coefficient of a target class in the identification result obtained by the two heterogeneous sensors as an evidence body, setting a basic probability distribution function for any evidence, and calculating a conflict strength value and a mutual support value among all evidence bodies; s402, conflict detection, namely judging whether the mutual support value is larger than a preset threshold value, if so, fusing the data to obtain the current fusion confidence coefficient, and then turning to step S404, otherwise turning to step S403; S403, respectively calculating the integral distance between the two heterogeneous sensors and the fused evidence at the previous moment, and selecting a sensor with a small distance from the fused evidence at the previous moment as the current period evidence; s404, calculating the total support degree of all evidence bodies on each evidence body by integrating the evidence obtained in the current period and the previous historical periods; S405, calculating weight values according to the total support degree of all evidence bodies to all evidence bodies so as to weight and correct the evidence ; S406, calculating the weighted correction evidence And fusing for multiple times to obtain fused confidence coefficient of each category, and judging the category corresponding to the highest confidence coefficient as the final target category.
10. A heterogeneous sensor information fusion device, the device comprising: The space-time registration module is used for acquiring data detected by the two heterogeneous sensors and carrying out time and space registration to obtain registered data; the fusion detection module is used for acquiring measurement information of the two heterogeneous sensors at the same moment from the registered data according to the joint distribution state of the measurement information of the two heterogeneous sensors, and carrying out association pairing to obtain an association observation pair; The fusion tracking module is used for determining a suspected measurement pair from the association observation pair according to the distribution relation between the measurement information of the two heterogeneous sensors and the noise variance, and fusing the measurement information of the suspected measurement pair to obtain a fused suspected measurement pair, wherein the suspected measurement pair is the measurement value of the two heterogeneous sensors in the association observation pair, and the measurement value of the two heterogeneous sensors meets the joint distribution; the fusion identification module is used for carrying out decision-level identification according to the suspected measurement pair after fusion to obtain a final identification result; or the apparatus comprises a processor and a memory, the memory being for storing a computer program, the processor being for executing the computer program to perform the method according to any one of claims 1 to 9.

Description

Heterogeneous sensor information fusion method and device Technical Field The invention relates to the technical field of sensor information fusion, in particular to a heterogeneous sensor information fusion method and device. Background The sensor has different advantages, such as all-weather operation, small volume, speed and distance measurement, and the infrared sensor has the advantages of little influence of illumination condition, accurate angle measurement, and the like. However, the single sensor is limited by the performance and the function of the sensor, the working mode and the acquired target information are single, for example, the point cloud data of the millimeter wave radar are sparse, the false alarm is more and is easy to interfere, the visible light camera is greatly influenced by weather, the illumination condition is greatly influenced, the monitoring range of the infrared sensor is small, the temperature is greatly influenced, and the like, so that the single sensor is difficult to adapt to the current increasingly complex use requirements. Therefore, on the basis of detection of a single sensor, information of different sensors is fused, and the recognition effect and the adaptability to complex environments are improved. The heterogeneous sensor detects different types of information, such as angle, distance, speed and the like of millimeter wave radar, and the information obtained by the infrared imaging sensor is usually position, angle and the like in an image pixel, and the information is detected by sensors with different dimensions, so that information fusion cannot be directly carried out. Therefore, in the prior art, the multi-sensor information fusion method is usually performed on the data association level, for example, the common method for information fusion of the millimeter wave radar and the infrared camera is to perform spatial coordinate conversion on the millimeter wave radar and the infrared camera, map the radar point onto the infrared image pixel coordinate system, and finally simply match the radar point with the infrared image target. However, the method is difficult to express and fuse heterogeneous sensor information, and is difficult to obtain better effects in complex environments, for example, millimeter wave radars have the problem of more false alarms, and textures and detailed information of infrared images are relatively simple, so that radar false alarm points or real radar target points without infrared targets can be matched. The Chinese patent application CN114994655A discloses a radar point trace and infrared point trace compound tracking processing method based on AdaBoost, which uses an assumed track as a machine learning training sample to train an AdaBoost classifier to carry out true and false classification on the assumed track and carry out filtering update on the compound track. However, the method needs to rely on an AdaBoost classifier to classify true and false points/tracks and mark samples offline, so that a large amount of prior information is needed, the implementation is complex, and the problem of poor adaptability to complex scenes still exists. Disclosure of Invention Aiming at the technical problems existing in the prior art, the invention provides the heterogeneous sensor information fusion method and device which can realize heterogeneous sensor expression fusion, and have the advantages of simple realization method, high fusion efficiency, good complex scene adaptability and strong anti-interference performance. In order to solve the technical problems, the technical scheme provided by the invention is as follows: a heterogeneous sensor information fusion method comprises the following steps: S01, acquiring data detected by two heterogeneous sensors, and performing time and space registration to obtain registered data; S02, according to the joint distribution state of the measurement information of the two heterogeneous sensors, acquiring the measurement information of the two heterogeneous sensors at the same moment from the registered data to perform association pairing to obtain an association observation pair; S03, determining a suspected measurement pair from the associated observation pair according to the joint distribution relation between the measurement information of the two heterogeneous sensors and the noise variance, and fusing the measurement information of the suspected measurement pair to obtain a fused suspected measurement pair; And S04, carrying out decision-stage identification according to the suspected measurement pair after fusion to obtain a final identification result. Further, the measurement information includes an azimuth angle and a pitch angle, in the step S02, a first statistical decision T obeying chi-square distribution with a degree of freedom of 2 is constructed by using the azimuth angle and the pitch angle obtained by the two heterogeneous sensors, when the first statistical decision