CN-121978031-A - Light and small sensing and calculating integrated hyperspectral imaging and target positioning system

CN121978031ACN 121978031 ACN121978031 ACN 121978031ACN-121978031-A

Abstract

The application provides a light and small sensing and calculation integrated hyperspectral imaging and target positioning system which comprises a multispectral imaging and hyperspectral point detection optical front end, a laser radar, an aerospace-level FPGA and an AI processor, wherein the multispectral imaging and hyperspectral point detection optical front end comprises a multispectral camera and a point spectrometer, the aerospace-level FPGA and the AI processor is used for processing data output by the multispectral imaging and hyperspectral point detection optical front end and the laser radar, outputting target morphology and mineral types, selecting potential sampling targets for path planning, and the two-dimensional turntable is used for bearing the multispectral imaging and hyperspectral point detection optical front end, the laser radar and the aerospace-level FPGA and the AI processor, supporting rotation in the horizontal direction and feeding back rotation angles. The method has the advantages that the identification, screening and positioning of the detection target can be automatically completed on the track, the approaching detection can be carried out without depending on ground control instructions, and the intelligent level of the detection process point is remarkably improved.

Inventors

XU HAITAO
LI JIAOJIAO
XUE CHANGBIN
ZHENG TIE
SONG RUI
LI YUNSONG

Assignees

中国科学院国家空间科学中心
西安电子科技大学

Dates

Publication Date: 20260505
Application Date: 20260312

Claims (10)

1.A lightweight, compact, integrated hyperspectral imaging and target positioning system, the system comprising: the multispectral imaging+hyperspectral point detection optical front end comprises a multispectral camera and a point spectrometer; a laser radar; an aerospace-level FPGA+AI processor for processing the data output by the multispectral imaging+hyperspectral point detection optical front end and the laser radar, outputting the shape and the mineral type of the target, selecting a potential sampling target, carrying out path planning, and The two-dimensional turntable is used for bearing the multispectral imaging and hyperspectral point detection optical front end, the laser radar and the aerospace-level FPGA and AI processor, supporting rotation in the horizontal direction and feeding back the rotation angle.
2. The light-weight and small-size integrated sensing and computing hyperspectral imaging and target positioning system according to claim 1, wherein the aerospace-level FPGA+AI processor is provided with a built-in spectrum preprocessing module, a point cloud preprocessing module, a space-time accurate registration module, a three-dimensional spectrum data fusion module, a target detection module and a path planning module, The spectrum preprocessing module is used for calculating and compensating correction of data generated by the multispectral imaging and hyperspectral point detection optical front end, and comprises non-uniformity correction, bad pixel replacement, phase error compensation and fast Fourier transformation; the point cloud preprocessing module is used for processing data generated by the laser radar, and comprises distance calculation, intensity calculation, point cloud generation, noise suppression and motion compensation; The space-time accurate registration module is used for performing space-time registration on the data processed by the spectrum preprocessing module and the point cloud preprocessing module; The three-dimensional spectrum data fusion module is used for processing the data generated by the space-time accurate registration module, and carrying out three-dimensional space coordinate calculation, reconstruction spectrum and surface normal calculation on each point; the target detection module is used for carrying out target detection, substance identification, abnormal marking and priority sequencing on the data generated by the three-dimensional spectrum data fusion module, outputting target morphology, mineral types and selecting potential sampling targets; and the path planning module is used for planning the travelling path according to the sampling target space position information output by the target detection module.
3. The integrated lightweight and compact hyperspectral imaging and target localization system as claimed in claim 2, wherein the target detection module comprises a mapping layer, a fourier transform layer, a feature extraction network layer, an inverse fourier transform layer, a linear layer, a cross-attention fusion layer, and an identification and output priority layer, wherein, The system comprises a mapping layer, a hyperspectral imaging layer, a hyperspectral point detection optical front end, a laser radar mapping layer, a laser radar imaging layer and a laser radar imaging layer, wherein the hyperspectral imaging layer and the hyperspectral point detection optical front end are connected in parallel; The system comprises a hyperspectral mapping layer, a Fourier transform layer, a first laser radar branch, a second laser radar branch and a second laser radar branch, wherein the hyperspectral mapping layer comprises a first hyperspectral branch and a first laser radar branch which are parallel; The device comprises a characteristic extraction network layer, a characteristic extraction system and a characteristic extraction system, wherein the characteristic extraction network layer comprises a second hyperspectral branch and a second laser radar branch which are parallel, wherein the two branches respectively comprise N stacked residual units and parallel residual connection, and each residual unit comprises KANLINEAR subunits and a depth separable convolution subunit; The Fourier inverse transformation layer comprises a third hyperspectral branch and a third laser radar branch which are parallel, wherein the third hyperspectral branch performs one-dimensional Fourier inverse transformation on the output of the second hyperspectral branch; The linear layer comprises a fourth hyperspectral branch and a fourth laser radar branch which are parallel, wherein the two branches respectively carry out 1X 1 convolution processing on the output of the third hyperspectral branch and the output of the third laser radar branch, and unify the channel number; The cross attention fusion layer is used for fusing the outputs of the fourth hyperspectral branch and the fourth laser radar branch, and then carrying out characteristic flattening, attention calculation and characteristic reconstruction; The system comprises an output priority layer, an element-by-element weighted fusion layer, a fusion feature vector and a priority result, wherein the output priority layer is used for carrying out 1X 1 convolution processing on the output of the cross attention fusion layer, mapping the channel number into the classification category number, carrying out global average pooling, outputting classification probability by using an activation function to obtain the identification result, accessing the identification result and external expert knowledge into one sub-linear layer together, carrying out element-by-element weighted fusion on the mapped features, outputting the fusion feature vector, accessing the fusion feature vector into the other sub-linear layer, mapping the fusion feature vector into the priority category vector, and outputting the priority result.
4. The lightweight, compact, and computationally integrated hyperspectral imaging and target positioning system as claimed in claim 3, wherein the hyperspectral mapping layer performs a1 x 1 convolution on a hyperspectral image with an input dimension of 9 x 64, maps the number of channels from 64 to 100, and an output dimension of 9 x 100; the laser radar mapping layer inputs a laser radar intensity image with the dimension of 9 multiplied by 1, carries out 1 multiplied by 1 convolution, maps the channel number from 1 to 100, and outputs the laser radar intensity image with the dimension of 9 multiplied by 100.
5. The lightweight, compact, integrated hyperspectral imaging and targeting system as claimed in claim 4 wherein the first hyperspectral branch performs a one-dimensional fourier transform on a 9 x 100 input along a spectral channel dimension, downsampling the spectral dimension from 100 to 50, the spatial dimension remains unchanged, and the output dimension is 9 x 50; The second lidar branch performs a two-dimensional fourier transform on the 9×9×100 input along a spatial dimension, downsamples the spatial dimension from 9×9 to 5×5, maintains the number of channels unchanged, and outputs the dimension as 5×5×100.
6. The lightweight, compact, integrated hyperspectral imaging and targeting system as claimed in claim 5, wherein the KANLINEAR subunit includes SiLU activation function and Linear layer connected in sequence, and parallel Spline mapping layer; the depth separable convolution subunit includes a depth convolution layer and a point-by-point convolution layer having a convolution kernel size of 3×3.
7. The lightweight, compact, integrated hyperspectral imaging and targeting system as claimed in claim 6 wherein the third hyperspectral branch, the inverse transform on 9 x 50 input along the channel dimension, the channel dimension reverts from 50 to 100, the output dimension is 9 x 100; And the third laser radar branch performs inverse transformation on the input of 5 multiplied by 100 along the space dimension, the space dimension is restored from 5 multiplied by 5 to 9 multiplied by 9, the channel number is kept to be 100, and the output dimension is 9 multiplied by 100.
8. The lightweight, compact, integrated hyperspectral imaging and targeting system as claimed in claim 7 wherein the fourth hyperspectral branch performs a1 x 1 convolution on a 9 x 100 input, the number of channels maps from 100 to 96, the spatial dimension remains 9 x 9, the output dimension is 9 x 96; the fourth lidar branch performs 1×1 convolution on a 9×9×100 input, maps the number of channels from 100 to 96, maintains the spatial dimension to 9×9, and outputs the dimension to 9×9×96.
9. The light and small sensing integrated hyperspectral imaging and target positioning system according to claim 8, wherein the feature flattening is to convert 9 x 9 spatial dimension into a one-dimensional sequence with length of 81, form a sequence feature with dimension of 81 x 96, and complete conversion from spatial dimension to sequence dimension; the attention calculation is carried out, the Query and Key of the sequence feature are linearly projected to be Q, K matrix of 81 multiplied by 12, the attention score is calculated and then multiplied by Value matrix of 81 multiplied by 12, and the attention weighting feature of 81 multiplied by 96 is obtained; The feature reconstruction restores the 81 x 96 attention weighted features to 9 x 96 spatial features, adds the spatial features to the original hyperspectral features element by element and normalizes the features by LayerNorm to output 9 x 96 fusion features.
10. The lightweight, compact, integrated hyperspectral imaging and target positioning system of claim 1, further comprising: The thermal control module comprises a plurality of dispersed heating plates and a temperature measuring circuit, and is used for monitoring and heating the temperature of the system under the control of the aerospace-level FPGA+AI processor.

Description

Light and small sensing and calculating integrated hyperspectral imaging and target positioning system Technical Field The application belongs to the field of planetary surface scientific research, and particularly relates to a light and small sensing and calculation integrated hyperspectral imaging and target positioning system which can be used for selecting and positioning extraterrestrial planetary surface detection targets such as moon and Mars. Background In the tasks of deep space exploration, moon scientific research station construction, mars inspection exploration and the like, scientific detection targets are often required to be efficiently and accurately selected and autonomous positioning is realized. The prior art relies on ground to manually interpret remote sensing data, has the problems of large response delay, low efficiency, high cost and the like, and is difficult to meet the real-time and autonomous requirements of deep space exploration tasks. Although some intelligent recognition algorithms are applied to planetary surface feature extraction, the problems of high model complexity, poor adaptability, difficulty in deployment on a satellite-borne lightweight platform and the like generally exist. The existing detection task main operation mode also depends on the whole participation of a ground scientist system, the efficiency of the space-earth closed-loop operation mode is low, the moving detection is slow to run, the detection efficiency is low, and a light and small sensing and calculation integrated hyperspectral imaging and target positioning system capable of carrying out on-orbit rapid identification, rapid positioning and guiding autonomous planning of a moving path on a detection target is not available. Therefore, a light and small intelligent system with high integration level, low power consumption and autonomous operation is needed to improve the accuracy and real-time of the identification and positioning of the detection targets of the surface of the extraterrestrial celestial body and support the subsequent scientific detection and resource utilization tasks. Disclosure of Invention The application aims to overcome the defects of large response delay, low efficiency and high cost of the prior art which rely on ground to manually interpret remote sensing data. In order to achieve the above object, the present application proposes a light-weight and small-size integrated hyperspectral imaging and target positioning system, the system comprising: the multispectral imaging+hyperspectral point detection optical front end comprises a multispectral camera and a point spectrometer; a laser radar; an aerospace-level FPGA+AI processor for processing the data output by the multispectral imaging+hyperspectral point detection optical front end and the laser radar, outputting the shape and the mineral type of the target, selecting a potential sampling target, carrying out path planning, and The two-dimensional turntable is used for bearing the multispectral imaging and hyperspectral point detection optical front end, the laser radar and the aerospace-level FPGA and AI processor, supporting rotation in the horizontal direction and feeding back the rotation angle. As an improvement of the system, the aerospace-level FPGA+AI processor is internally provided with a spectrum preprocessing module, a point cloud preprocessing module, a space-time accurate registration module, a three-dimensional spectrum data fusion module, a target detection module and a path planning module, The spectrum preprocessing module is used for calculating and compensating correction of data generated by the multispectral imaging and hyperspectral point detection optical front end, and comprises non-uniformity correction, bad pixel replacement, phase error compensation and fast Fourier transformation; the point cloud preprocessing module is used for processing data generated by the laser radar, and comprises distance calculation, intensity calculation, point cloud generation, noise suppression and motion compensation; The space-time accurate registration module is used for performing space-time registration on the data processed by the spectrum preprocessing module and the point cloud preprocessing module; The three-dimensional spectrum data fusion module is used for processing the data generated by the space-time accurate registration module, and carrying out three-dimensional space coordinate calculation, reconstruction spectrum and surface normal calculation on each point; the target detection module is used for carrying out target detection, substance identification, abnormal marking and priority sequencing on the data generated by the three-dimensional spectrum data fusion module, outputting target morphology, mineral types and selecting potential sampling targets; and the path planning module is used for planning the travelling path according to the sampling target space position information output by the