CN-122020071-A - Multi-mode perception optimization method and system based on domain knowledge enhancement

CN122020071ACN 122020071 ACN122020071 ACN 122020071ACN-122020071-A

Abstract

The invention discloses a multi-mode perception optimization method and system based on domain knowledge enhancement. The method comprises the steps of collecting multi-mode battlefield sensing data, carrying out space-time synchronization and military feature extraction, constructing a standardized battlefield state vector, constructing a battlefield structure geometric knowledge model, a battlefield tactical rule knowledge model and a three-layer military domain knowledge constraint model of a weapon equipment physical motion consistency knowledge model, wherein the model takes the battlefield state vector as input, outputs a battlefield structure geometric constraint item, a battlefield rule constraint item and a weapon equipment physical motion consistency constraint item, and respectively evaluates consistency of geometric structure features and a battlefield priori structure in the battlefield state vector, target state transition and battlefield tactical logic in the battlefield state vector and the like. According to the invention, through knowledge constraint and knowledge triggered compensation mechanism in three layers of fields, the perception error is effectively restrained, the optimization failure caused by data deletion is avoided, and the perception stability and reliability are remarkably improved.

Inventors

Hong Wanfu
ZHAO BOWEI
ZENG LI
HUANG ZAIBIN

Assignees

厦门渊亭信息科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260413

Claims (10)

1. A domain knowledge enhancement-based multi-modal awareness optimization method, the method comprising: Acquiring multi-mode battlefield sensing data, performing space-time synchronization and military feature extraction, and constructing a standardized battlefield state vector; constructing a three-layer military domain knowledge constraint model of a battlefield structure geometric knowledge model, a battlefield tactical rule knowledge model and a weapon equipment physical motion consistency knowledge model, wherein the model takes a battlefield state vector as input, outputs a battlefield structure geometric constraint item, a battlefield rule constraint item and a weapon equipment physical motion consistency constraint item, and respectively evaluates the consistency of geometric structure features and a battlefield priori structure in the battlefield state vector, the consistency of target state transition and battlefield logic in the battlefield state vector and the consistency of motion parameters and equipment physical motion limits in the battlefield state vector; Carrying out weighted fusion on observation error items of multi-mode battlefield sensing data and three types of constraint items output by a three-layer military domain knowledge constraint model, and constructing a combined optimization objective function based on a battlefield state vector; According to the current battlefield situation, the confidence coefficient of each knowledge constraint in the current battlefield environment is quantized, an exponential function is used as a regulator, and the weight of each constraint in the joint optimization objective function is dynamically distributed, so that the optimization model is adapted to the real-time battlefield environment; Calculating the missing proportion of battlefield sensing data, triggering a knowledge compensation mechanism when the missing proportion exceeds a preset threshold value, carrying out structural reconstruction and historical state prediction completion on the battlefield state vector, and inputting the compensated battlefield state vector into a joint optimization objective function again for optimization; And (3) iteratively solving the combined optimization objective function until the convergence condition is met, and outputting the optimized battlefield state vector as a final high-precision battlefield sensing result.
2. A domain knowledge-based enhanced multi-modal sense optimization method as claimed in claim 1, wherein, The method for acquiring the multi-mode battlefield sensing data, carrying out space-time synchronization and military feature extraction, and constructing a standardized battlefield state vector specifically comprises the following steps: Deploying a laser radar, a visible light camera, an infrared thermal imaging camera, a millimeter wave radar and an inertial measurement unit, and collecting multi-mode battlefield sensing data; The multi-mode sensing data is subjected to a composite calibration method combining military-grade timestamp hard matching and anti-interference interpolation soft alignment, and all sensing data are uniformly anchored to the same combat time node, so that space-time synchronization is realized; Extracting terrain gradient, traffic capacity, building and tunnel outline and equipment outline curvature from the synchronized laser radar point cloud as battlefield geographic structure characteristics, extracting individual combat attitude, equipment thermal imaging characteristics and military target exclusive texture characteristics from visible light or infrared images as combat target characteristics, extracting target distance, moving speed and azimuth from millimeter wave radar as target movement situation characteristics, and extracting attitude change rate, acceleration fluctuation and maneuvering track from an inertia measurement unit as combat platform maneuvering characteristics; all the extracted military features are integrated and constructed into the battlefield state vector.
3. A domain knowledge-based enhanced multi-modal sense optimization method as claimed in claim 1, wherein, The battlefield structural geometric knowledge model is as follows: Wherein, the As a geometric constraint term of the battlefield structure, For the real-time attitude angle of the combat platform, To the desired attitude angle based on the battlefield prior structure, Is a symmetric deviation value of a battlefield structure, 、 Is a dynamic weight coefficient.
4. A domain knowledge-based enhanced multi-modal sense optimization method as claimed in claim 1, wherein, The combat tactical rule knowledge model is verified by establishing a combat state transition logic matrix, when target state transition in a combat state vector accords with preset combat state transition logic, the value of an output constraint item is 0, and when state transition does not accord with preset combat state transition logic, the value of the output constraint item is a preset punishment coefficient.
5. A domain knowledge-based enhanced multi-modal sense optimization method as claimed in claim 1, wherein, The weapon equipment physical movement consistency knowledge model is as follows: Wherein, the For the physical motion consistency constraint term of the weapon equipment, 、 The motion speeds of the combat platform at the current moment and the previous moment are respectively, For the acceleration at the present moment, 、 The angular velocities at the present moment and the previous moment respectively, As the angular acceleration at the present moment, Weight coefficients are constrained for gesture motion.
6. A domain knowledge-based enhanced multi-modal sense optimization method as claimed in claim 1, wherein, Dynamically assigning weights for each constraint in the joint optimization objective function, comprising: Calculate the first Confidence of class knowledge constraints , wherein, Is the first Variance of class constraint term errors; Updating weight coefficients based on confidence level , wherein, As a result of the initial reference weight, For the adjustment factor constructed as an exponential function, the weight is positively correlated with the confidence.
7. A domain knowledge-based enhanced multi-modal sense optimization method as claimed in claim 1, wherein, Triggering a knowledge compensation mechanism comprising: Calculating battlefield data missing proportion Wherein In order to be able to delete the data amount, The total data quantity is the corresponding mode; When (when) When the geometric sense data exceeds a preset threshold value, a standardized structural template is called through a battlefield structural geometric knowledge model to reconstruct the missing geometric sense data, and a motion prediction formula is used for solving the problem that the geometric sense data is lost For the battlefield state vector at the current moment Performing predictive complement in which For the adjacent sensing time interval, For the state vector at the previous time instant, Is the speed of movement.
8. A domain knowledge-based enhanced multi-modal awareness optimization system, characterized in that it comprises, based on the method of any of claims 1-7: The acquisition module is used for acquiring multi-mode battlefield sensing data, carrying out space-time synchronization and military feature extraction, and constructing a standardized battlefield state vector; The constraint module is used for constructing a three-layer military domain knowledge constraint model of a battlefield structure geometric knowledge model, a battlefield tactical rule knowledge model and a weapon equipment physical motion consistency knowledge model, wherein the model takes a battlefield state vector as input, outputs a battlefield structure geometric constraint item, a battlefield rule constraint item and a weapon equipment physical motion consistency constraint item, and respectively evaluates the consistency of geometric structure features and a battlefield priori structure in the battlefield state vector, the consistency of target state transition and battlefield tactical logic in the battlefield state vector and the consistency of motion parameters and equipment physical motion limits in the battlefield state vector; The construction module is used for carrying out weighted fusion on the observation error items of the multi-mode battlefield sensing data and three types of constraint items output by the three-layer military domain knowledge constraint model, and constructing a combined optimization objective function based on a battlefield state vector; The weight module is used for dynamically distributing the weight of each constraint in the joint optimization objective function by quantifying the confidence coefficient of each knowledge constraint in the current battlefield environment according to the current battlefield situation and using the exponential function as a regulator so as to enable the optimization model to adapt to the real-time battlefield environment; The compensation module is used for calculating the missing proportion of the battlefield sensing data, triggering a knowledge compensation mechanism when the missing proportion exceeds a preset threshold value, carrying out structural reconstruction and historical state prediction completion on the battlefield state vector, and inputting the compensated battlefield state vector into the joint optimization objective function again for optimization; and the iteration module is used for iteratively solving the joint optimization objective function until the convergence condition is met, and outputting the optimized battlefield state vector as a final high-precision battlefield sensing result.
9. A domain knowledge enhancement-based multi-modal awareness optimization device, comprising: at least one processor and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.
10. A computer readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the method of any one of claims 1 to 7.

Description

Multi-mode perception optimization method and system based on domain knowledge enhancement Technical Field The invention relates to the technical field of data processing in modern complex battlefield countermeasure environments, in particular to a multimode perception optimization method and system based on field knowledge enhancement. Background Modern war has entered an informatization and intelligence era, and battlefield situation awareness becomes a key factor for determining the success or failure of battle. Along with the wide application of unmanned combat platforms, individual soldier digital equipment and combined combat command systems, the multi-mode perception technology has become a core support for battlefield reconnaissance, target identification and situation research and judgment. The cooperative use of the laser radar, the visible light camera, the infrared thermal imager, the millimeter wave radar, the inertia measurement unit and other sensors provides unprecedented battlefield information acquisition capability for fighters. However, the strong challenge of modern complex battlefield environments presents a serious challenge to multi-modal perception techniques. In typical military scenarios such as field mobile assault, urban lane fight, underground gallery fight, border universe reconnaissance, multiple army combined fight, the perception system faces the following prominent problems: 1. strong environmental interference causes sudden drop in perceived accuracy The imaging quality of the optical sensor is seriously interfered by shields such as battlefield smoke, nitrate smoke, sand dust and the like, so that visible light/infrared images are blurred, target characteristics are lost, multipath reflection interference in dense areas of complex terrains and buildings is caused, a large number of false echoes are generated by millimeter wave radars and laser radars, and an fight platform (armored vehicle and unmanned aerial vehicle) is severely vibrated, so that IMU posture drift and point cloud distortion are caused. The superposition of the interference factors greatly reduces the precision of the existing perception system in the actual combat environment, and the requirements of accurate striking and collaborative combat are difficult to meet. 2. Perceived data burst loss causes system instability In extreme cases of strong electromagnetic interference suppression, communication link interruption, local damage of sensors and the like, sudden and large-scale loss of perceived data can occur. In the prior art, continuous and complete data flow is mostly relied on for optimization, and once the data missing proportion exceeds a threshold value, the optimization algorithm is often caused to diverge, the perception result is often caused to fail, and even the whole situation perception system is caused to be interrupted. This "data-dependent" technology route presents significant vulnerability in a battlefield strong opposing environment. 3. The prior art lacks deep integration of military domain knowledge The current mainstream multi-mode perception optimization method is mainly based on pure data driving thought, such as methods of multi-sensor fusion filtering (Kalman filtering and particle filtering), deep learning feature extraction and fusion, graph optimization and the like. Although the method is good in performance under certain conventional scenes, the method cannot utilize the inherent geographic structure of a battlefield (tunnel section standardized characteristics, urban building symmetry, defence work boundaries and the like) to geometrically correct the perceived result, so that structural recognition distortion is serious under the conditions of shielding and smog, the perceived result possibly violates basic tactical logic, such as direct locking without recognition, striking without unlocking, state jump violating the operational flow and the like, the output situation information is not available in military although being reasonable in mathematics, the perceived motion state cannot be verified by utilizing the physical limits (maximum speed, maximum acceleration, steering overload limit) of weapon equipment, and false perceived results, such as ultra-high-speed movement of an individual soldier, unpowered sudden rise and fall of an unmanned aerial vehicle and the like, which violate the physical rules, are easy to appear. In summary, the existing battlefield multi-modal sensing technology has an optimization space for sensing accuracy, robustness and dynamic adaptation capability under a strong countermeasure environment, so as to meet the high-accuracy sensing requirements of modern war on all weather, all terrain and anti-disturbance. Therefore, a multi-mode perception optimization method capable of deeply fusing military domain knowledge, adapting to change of battlefield environment and having strong anti-interference capability is needed, so as to solve the pro