CN-121999516-A - Scene-oriented multispectral and infrared bimodal camouflage character decision-making system

CN121999516ACN 121999516 ACN121999516 ACN 121999516ACN-121999516-A

Abstract

The invention relates to a scene-oriented multispectral and infrared bimodal camouflage character decision-making system, and belongs to the technical field of multisource information perception and intelligent decision-making. The system comprises an unmanned aerial vehicle carrying platform, a bimodal imaging unit, a data processing and communication unit and a ground computing and decision unit. The dual-mode imaging unit synchronously acquires images of the target area through the infrared camera and the spectrum camera, the data processing unit preprocesses and transmits the images, and the ground calculation and decision unit comprises a data set construction module, a dual-mode target detection module, an image registration module, a self-adaptive decision module and a decision fusion module. The system calculates a homography matrix based on the flying height to realize image registration, adaptively selects a decision path according to the light intensity parameter, and outputs a detection result through cross-ratio matching and weighted fusion. The method and the system improve the accuracy and the robustness of camouflage character detection in a complex environment through a system-level architecture and a formulated decision mechanism, and are suitable for unmanned aerial vehicle platform deployment.

Inventors

WANG SHU
QIN TAO
HUANG FENG
CHEN JING
Lin Xuancheng

Assignees

福州大学

Dates

Publication Date: 20260508
Application Date: 20260211

Claims (10)

1. A scene-oriented multispectral, infrared bimodal camouflage character decision making system, comprising: The unmanned aerial vehicle carrying platform is used for carrying and deploying system hardware; The dual-mode imaging unit is arranged on the unmanned aerial vehicle carrying platform and is used for synchronously acquiring a spectrum image and an infrared image of a target area; the data processing and communication unit is used for preprocessing and transmitting the spectrum image and the infrared image acquired by the bimodal imaging unit; the ground computing and decision-making unit is in communication connection with the data processing and communication unit and is used for carrying out camouflage character detection, image registration, self-adaptive decision-making and decision-making fusion on the received bimodal image and outputting a final camouflage character detection result; The ground computing and decision unit comprises a data set construction module, a bimodal target detection module, an image registration module, an adaptive decision module and a decision fusion module.
2. The scene-oriented multispectral, infrared bimodal camouflage character decision making system of claim 1, wherein the bimodal imaging unit comprises an infrared camera and a spectral camera, the infrared camera and the spectral camera are arranged side by side, the optical axes of the infrared camera and the spectral camera are parallel, the visual field of the infrared camera is partially overlapped, the infrared camera is used for collecting the heat radiation characteristic of the camouflage character, and the spectral camera is used for collecting the spectral reflection characteristic, the edge information and the texture information of the camouflage character.
3. The scene-oriented multispectral, infrared bimodal camouflage character decision system of claim 1, wherein the data processing and communication unit comprises an on-board processing module and a communication module, wherein the on-board processing module is configured to buffer, compress, or format convert the bimodal image, and the communication module is configured to transmit the processed bimodal image to the ground computing and decision unit.
4. The scene oriented multispectral, infrared bimodal camouflage character decision system of claim 1, wherein the data set construction module constructs the bimodal images in a time ordered manner into a data set and divides the data set into a training data set, a validation data set and a test data set in a ratio of 6:2:2.
5. The scene-oriented multispectral, infrared bimodal camouflage character decision system of claim 1, wherein the bimodal target detection module constructs a camouflage character detection model based on the infrared image and the spectral image, outputs detection frame position and confidence information, and performs model training by adopting a freeze training and learning rate attenuation strategy.
6. The scene-oriented multispectral, infrared bimodal camouflage character decision system of claim 1, wherein the image registration module calculates a homography matrix based on unmanned aerial vehicle fly height and maps infrared modal detection results to a spectral coordinate system using the homography matrix.
7. The scene-oriented multispectral, infrared bimodal camouflage character decision system of claim 1, wherein the adaptive decision module is configured to calculate an adaptive decision parameter based on light intensity information of the spectral image, and select a spectral dominant decision path or an infrared dominant decision path based on the adaptive decision parameter.
8. The scene-oriented multispectral, infrared bimodal camouflage character decision system recited in claim 7, wherein the adaptive decision parameters comprise spectral adaptive decision parameters Infrared adaptive decision parameters Wherein 1, , 、 Respectively representing the light intensity factor and the illumination intensity.
9. The scene-oriented multispectral infrared bimodal camouflage character decision system according to claim 1, wherein the decision fusion module is used for constructing IoU matrixes based on the cross-over ratio between bimodal detection frames to match, carrying out weighted fusion on the successfully matched detection frames according to the confidence level, and outputting a final camouflage character detection result.
10. The scene-oriented multispectral, infrared bimodal camouflage character decision system of claim 6, wherein the image registration module selects 5 sets of corresponding feature points and implements conversion between the reference image and the input image by a coordinate mapping formula when calculating the homography matrix.

Description

Scene-oriented multispectral and infrared bimodal camouflage character decision-making system Technical Field The invention belongs to the technical field of multisource information perception and intelligent decision making, and particularly relates to a scene-oriented multispectral and infrared bimodal camouflage character decision making system. Background In complex natural environments and field scenes, personnel are often highly fused with the surrounding environment in camouflage mode such as camouflage clothes and Jili clothes, so that the probability of being found by an optical reconnaissance system is reduced. With the development of unmanned aerial vehicle reconnaissance, intelligent perception and target detection technology, a target detection method based on single visible light imaging has a certain effect under ideal illumination conditions, but the detection performance of the target detection method is obviously reduced under weak illumination, night or complex background conditions, and the actual application requirements are difficult to meet. To improve the detection capabilities of camouflage characters, researchers are gradually introducing infrared imaging and multispectral imaging technologies. The infrared imaging mainly depends on the heat radiation difference between the target and the background, is not influenced by visible light conditions, is suitable for night and weak light environments, and can provide more abundant information in the aspects of material distinction and camouflage identification by acquiring the reflection or radiation characteristics of the target in a plurality of spectrum bands. However, there are inherent limitations to either modality alone, such as infrared imaging being susceptible to ambient temperature variations, while multispectral imaging is highly sensitive to lighting conditions. In the prior art, research on multi-mode information fusion is mainly focused on feature level fusion methods, which generally require that images of different modes are highly aligned in space and time, and complex preprocessing operation is required, so that real-time deployment on an unmanned plane platform is not facilitated. In addition, the partial decision-level fusion method adopts a fixed fusion strategy, and the dominant positions of different modes in the decision process can not be dynamically adjusted according to the environmental condition change, so that the detection stability and the robustness are insufficient in a complex scene. Therefore, a camouflage character detection system which can cooperate with the advantages of multispectral and infrared imaging at the system level and can adaptively adjust a decision path according to scene conditions is urgently needed, so that the problems of insufficient detection precision, poor environmental adaptability, limited system instantaneity and the like in the prior art are solved. Disclosure of Invention The invention aims to provide a scene-oriented multispectral and infrared bimodal camouflage character decision system aiming at the problems that in the prior art, single-modality camouflage character detection has low detection precision under a complex scene, poor environmental adaptability, fixed multimodal fusion strategy and difficulty in dynamic adjustment according to scene change. The invention aims to effectively cooperate multispectral imaging and infrared imaging in a decision level through system-level architecture design, and introduce a scene self-adaptive decision mechanism to dynamically adjust the dominant positions of different modes in a decision process according to ambient illumination conditions, so that the accuracy and the robustness of camouflage character detection in a complex environment are improved while the real-time performance of a system is ensured. In order to achieve the purpose of the invention, the invention provides a scene-oriented multispectral and infrared bimodal camouflage character decision system from multiple layers of system overall architecture design, function module division cooperation, multimodal data processing and decision mechanism and the like. The system does not simply splice the existing method steps, but converts the original method flow which depends on manual or off-line analysis into a decision mechanism which can be automatically completed in the system operation process through functional reconstruction and modularized division at the system level. Specifically, the invention relates to a scene-oriented multispectral infrared bimodal camouflage character decision system, which comprises: The unmanned aerial vehicle carrying platform is used for carrying and deploying system hardware; The dual-mode imaging unit is arranged on the unmanned aerial vehicle carrying platform and is used for synchronously acquiring a spectrum image and an infrared image of a target area; the data processing and communication unit is used for preprocessing and