CN-121527631-B - Pollen typhae content prediction method and system based on hyperspectral double-flow multi-scale CNN

CN121527631BCN 121527631 BCN121527631 BCN 121527631BCN-121527631-B

Abstract

The invention provides a pollen typhae content prediction method and system based on hyperspectral double-flow multi-scale CNN, and belongs to the field of quality detection of traditional Chinese medicinal materials. The method comprises the steps of collecting hyperspectral image data of a pollen typhae sample, preprocessing the collected hyperspectral image data, extracting spectral data of an interested region of the pollen typhae sample, inputting the extracted spectral data of the interested region into a trained double-flow multi-scale residual fusion CNN model, carrying out multi-scale feature extraction and fusion by utilizing a VNIR multi-scale branch and an NIR multi-scale branch, outputting multi-scale fusion features, inputting the multi-scale fusion features into an attention fusion module, outputting final fusion features, and outputting content predicted values of pollen typhae effective components through a content predicted layer based on the final fusion features. The method solves the problems of low efficiency, sample damage and insufficient precision caused by neglecting the band characteristic difference of the traditional spectrum model, and realizes the rapid, nondestructive and high-precision prediction of the pollen typhae content.

Inventors

LIN YONGQIANG
SUN PEILIN
ZHOU QIANQIAN
LUAN YONGFU
WANG BING
XIE YINGYING
DONG YUNING
YU FENGRUI
XUE FEI
CUI WEILIANG
ZHOU GUANGTAO
LIU HONGCHAO

Assignees

山东省食品药品检验研究院
中国食品药品检定研究院

Dates

Publication Date: 20260505
Application Date: 20260115

Claims (8)

1. The pollen typhae content prediction method based on hyperspectral double-flow multi-scale CNN is characterized by comprising the following steps: collecting hyperspectral image data of a pollen typhae sample, wherein the hyperspectral image data comprises visible-near infrared band data and short wave infrared band data; preprocessing the collected hyperspectral image data, and extracting spectral data of an interested region of a pollen typhae sample by adopting an interested region extraction algorithm based on a centroid; The method comprises the steps of inputting the extracted spectral data of the region of interest into a trained double-flow multi-scale residual fusion CNN model, and outputting final fusion characteristics, wherein the trained double-flow multi-scale residual fusion CNN model utilizes VNIR multi-scale branches and NIR multi-scale branches to extract and fuse the multi-scale characteristics of the spectral data of the region of interest in corresponding wave bands, and outputs the multi-scale fusion characteristics, inputting the multi-scale fusion characteristics into a cross-band attention fusion module for interaction and dynamic weighted fusion, and outputting the final fusion characteristics, and the cross-band attention fusion module comprises: A position coding unit for adding a learnable position code for the input multi-scale fusion feature; the multi-head self-attention unit comprises a plurality of parallel attention heads and is used for extracting cross-band associated features of the features added with the position codes and outputting attention features; The dynamic weight fusion unit is used for calculating the dynamic weight between the attention features output by the VNIR multi-scale branch and the NIR multi-scale branch, and carrying out weighted summation on the attention features of the two branches based on the dynamic weight to obtain a final fusion feature; outputting a content prediction value of the pollen typhae effective component through a content prediction layer based on the final fusion characteristic, wherein the content prediction value comprises the following steps: performing a first full-connection layer operation on the input final fusion features, performing feature space mapping through a linear transformation function, then sequentially performing batch normalization operation to reduce feature distribution differences, and introducing nonlinear feature expression by using a ReLU activation function operation to obtain features after first refining; Taking the features after the first refining as input, executing a second full-connection layer operation, further mapping to a low-dimensional feature space through a linear transformation function, repeating batch normalization and ReLU activation function processing, eliminating feature redundancy and strengthening effective feature association, and obtaining features after the second refining; and performing final linear transformation on the features after the second refining, and mapping the dimensional features into one-dimensional numerical values through a linear transformation function to obtain a content predicted value of the final output pollen typhae effective components.
2. A hyperspectral double-flow multi-scale CNN-based pollen typhae content prediction method as claimed in claim 1 wherein preprocessing the collected hyperspectral image data comprises: and carrying out black-and-white correction on hyperspectral image data of the collected pollen typhae sample to eliminate noise interference, wherein the correction formula is as follows: Wherein, the In order to correct for the reflectivity after the correction, In order to be the original reflectance of the light, In the case of a black reference image, Is a white reference image.
3. A hyperspectral double-flow multi-scale CNN-based pollen typhae content prediction method as claimed in claim 1 wherein extracting the region of interest spectral data of the pollen typhae sample using a centroid-based region of interest extraction algorithm comprises: extracting three-dimensional data of the preprocessed hyperspectral image, and calculating barycenter coordinates; Drawing a circle according to the centroid coordinates, judging whether related pixel points are in the circle according to the Euclidean distance, taking the pixel points in the circle as ROI candidate pixels, extracting the spectrum data of all the ROI candidate pixels, and arranging preferentially according to the wavelength; the wavelength points of each batch of samples are combined together in turn to form the spectral data of the region of interest of the pollen Typhae sample.
4. A hyperspectral dual-flow multiscale CNN-based pollen typhae content prediction method as claimed in claim 1 wherein the VNIR multiscale branch and NIR multiscale branch each comprise a multiscale feature extraction path; the multi-scale feature extraction path includes: The first path is used for extracting local detail characteristics and comprises a1 multiplied by 1 convolution layer, a batch normalization layer, a ReLU activation function layer and a maximum pooling layer which are connected in sequence; The second path is used for extracting mesoscale associated features and comprises a 1 multiplied by 1 convolution layer, a one-dimensional cavity convolution layer with a first expansion rate, a batch normalization layer, a ReLU activation function layer and a maximum pooling layer which are sequentially connected; And the third path is used for extracting global trend characteristics and comprises a1 multiplied by 1 convolution layer, a one-dimensional cavity convolution layer with a second expansion rate, a batch normalization layer, a ReLU activation function layer and a maximum pooling layer which are sequentially connected.
5. The hyperspectral double-flow multi-scale CNN-based pollen typhae content prediction method as claimed in claim 4, wherein the multi-scale feature extraction path further comprises a multi-scale feature fusion unit for performing channel splicing on the output features of the first path, the second path and the third path, and performing dynamic weight fusion through a1×1 convolution layer to obtain a multi-scale fusion feature.
6. Pollen typhae content prediction system based on hyperspectral double-flow multi-scale CNN is characterized by comprising: the hyperspectral data acquisition module is configured to acquire hyperspectral image data of a pollen typhae sample, wherein the hyperspectral image data comprises visible-near infrared band data and short wave infrared band data; The data processing module is configured to preprocess the collected hyperspectral image data and extract the spectral data of the region of interest of the pollen typhae sample by adopting a region of interest extraction algorithm based on mass centers; The final fusion feature output module is configured to input the extracted region of interest spectral data into a trained double-flow multi-scale residual fusion CNN model to output final fusion features, wherein the trained double-flow multi-scale residual fusion CNN model utilizes VNIR multi-scale branches and NIR multi-scale branches to extract and fuse the region of interest spectral data of corresponding wave bands to output multi-scale fusion features, the multi-scale fusion features are input into a cross-band attention fusion module to perform interaction and dynamic weighted fusion, and the cross-band attention fusion module comprises: A position coding unit for adding a learnable position code for the input multi-scale fusion feature; the multi-head self-attention unit comprises a plurality of parallel attention heads and is used for extracting cross-band associated features of the features added with the position codes and outputting attention features; The dynamic weight fusion unit is used for calculating the dynamic weight between the attention features output by the VNIR multi-scale branch and the NIR multi-scale branch, and carrying out weighted summation on the attention features of the two branches based on the dynamic weight to obtain a final fusion feature; The content prediction output module is configured to output a content prediction value of the pollen typhae effective component through the content prediction layer based on the final fusion characteristic, and comprises the following steps: performing a first full-connection layer operation on the input final fusion features, performing feature space mapping through a linear transformation function, then sequentially performing batch normalization operation to reduce feature distribution differences, and introducing nonlinear feature expression by using a ReLU activation function operation to obtain features after first refining; Taking the features after the first refining as input, executing a second full-connection layer operation, further mapping to a low-dimensional feature space through a linear transformation function, repeating batch normalization and ReLU activation function processing, eliminating feature redundancy and strengthening effective feature association, and obtaining features after the second refining; and performing final linear transformation on the features after the second refining, and mapping the dimensional features into one-dimensional numerical values through a linear transformation function to obtain a content predicted value of the final output pollen typhae effective components.
7. A computer readable storage medium having stored thereon a program which when executed by a processor performs the steps of the hyperspectral double flow multi-scale CNN based pollen Type content prediction method as claimed in any one of claims 1 to 5.
8. An electronic device comprising a memory, a processor and a program stored on the memory and executable on the processor, wherein the processor performs the steps in the hyperspectral dual stream multi-scale CNN based pollen Type content prediction method as claimed in any one of claims 1 to 5 when the program is executed.

Description

Pollen typhae content prediction method and system based on hyperspectral double-flow multi-scale CNN Technical Field The invention belongs to the technical field of quality detection of traditional Chinese medicinal materials, and particularly relates to a pollen typhae content prediction method and system based on hyperspectral double-flow multi-scale CNN. Background The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art. Pollen Typhae is dry pollen of Typhaceae plant such as herba Saussureae Involueratae, oriental typha or the same genus plant, and has hemostatic, blood stasis dispelling, and stranguria treating effects. The pharmacological activity is closely related to the content of effective components (such as isorhamnetin-3-O-neohesperidin and typhonine). According to the relevant regulations, the total amount of the above components in pollen Typhae should not be less than 0.50%. Therefore, an accurate and efficient content determination method is established, and is important for guaranteeing the quality of the cattail pollen medicinal material and ensuring the clinical curative effect. At present, the content determination of the active ingredients of the cattail pollen mainly depends on traditional chemical analysis methods such as high performance liquid chromatography and the like. The method has higher measurement precision, but has the defects that the operation flow is complicated, complex pretreatment steps such as sample extraction and purification are needed, the detection period is long, the analysis of a single sample usually takes several hours, a large amount of organic solvents are needed, the method does not accord with the concept of green detection, the rapid and on-line analysis of a large amount of samples is difficult to realize, and the urgent requirements of the modern traditional Chinese medicine industry on real-time and in-situ quality monitoring of raw materials and finished products in the production process cannot be met. The hyperspectral imaging technology is used as an emerging nondestructive testing technology, can synchronously acquire the spatial image information and continuous spectrum information of a tested object, forms a three-dimensional data cube with a combined pattern, and provides a new technical path for rapid and nondestructive analysis of traditional Chinese medicinal materials. However, hyperspectral data brings rich information and also faces challenges such as high data dimension, more redundant information, complex correlation among bands and the like. Particularly in the field of traditional Chinese medicines, the interaction mechanisms of different spectral ranges and different chemical groups in medicinal materials are different, and the loaded chemical information is different in emphasis. In the prior art, when the hyperspectral technology is combined with the traditional machine learning or uniflow deep convolutional neural network to predict the content, all wave band data are often simply mixed, and the difference of physical characteristics and information values between different wave band intervals is ignored, so that the extracted characteristics of a model are insufficient, the discrimination is not strong, the prediction precision and the generalization capability of the model are limited, and the high requirement of pharmacopoeia standards on quantitative analysis is difficult to be achieved. Disclosure of Invention In order to overcome the defects of the prior art, the invention provides a pollen typhae content prediction method and system based on hyperspectral double-flow multi-scale CNN, which are used for solving the problems of low detection efficiency, insufficient precision and incapability of adapting to multiband spectral characteristic difference in the prior art. To achieve the above object, one or more embodiments of the present invention provide the following technical solutions: The first aspect of the invention provides a pollen typhae content prediction method based on hyperspectral double-flow multi-scale CNN; A pollen typhae content prediction method based on hyperspectral double-flow multi-scale CNN comprises the following steps: collecting hyperspectral image data of a pollen typhae sample, wherein the hyperspectral image data comprises visible-near infrared band data and short wave infrared band data; preprocessing the collected hyperspectral image data, and extracting spectral data of an interested region of a pollen typhae sample by adopting an interested region extraction algorithm based on a centroid; Inputting the extracted spectral data of the region of interest into a trained double-flow multi-scale residual fusion CNN model to output final fusion characteristics, wherein the trained double-flow multi-scale residual fusion CNN model utilizes VNIR multi-scale branches and NIR multi-scale branches to extract and