CN-122019777-A - Text classification method and system based on quantum self-attention hybrid neural network

CN122019777ACN 122019777 ACN122019777 ACN 122019777ACN-122019777-A

Abstract

The invention discloses a text classification method and system based on a quantum self-attention hybrid neural network, and belongs to the technical field of quantum computing and artificial intelligence. The invention designs a shallow parameterized quantum circuit at a quantum end to respectively generate query, key and value vectors, realizes high-dimensional feature mapping by quantum superposition and entanglement, calculates self-attention weight by using cosine similarity and Softmax function at a classical end, reduces dependence on quantum measurement and simplifies calculation flow. In order to adapt to different task types, an extensible input coding strategy is introduced, one-Hot or TF-IDF coding is adopted in a short text task, and Word2Vec and Doc2Vec coding is adopted in a long text task, so that self-adaptive matching of a coding layer is realized. The invention applies the quantum neural network to natural language processing, and obviously reduces the parameter quantity and improves the training efficiency and the model stability while maintaining the quantum characteristic expression capability.

Inventors

LU YANJING
DING FEI
ZHU PENGCHENG
CHENG WEI
FENG YUNLIANG
CHENG HONGYING
CHENG XUEYUN
GUAN ZHIJIN

Assignees

南通大学

Dates

Publication Date: 20260512
Application Date: 20251223

Claims (10)

1. The text classification method based on the quantum self-attention hybrid neural network is characterized by comprising the following steps of: S1, acquiring text data to be classified, and performing feature coding on the input data by adopting a double-channel coding strategy according to the type of the text data; s2, converting the coded feature vector into a quantum state through amplitude coding; S3, inputting quantum states to a shallow parameterized quantum circuit PQC to extract features, performing Pauli-Z measurement after training to obtain a dimension reduction vector, and fusing the two-channel features to form a first fused vector; S4, sending the first fusion vector into a quantum self-attention module to form three groups of quantum representations of query Q, key K and value V, obtaining self-attention weight through cosine similarity and Softmax normalization calculation, weighting the value vector, introducing identity residual connection, and obtaining a second fusion vector; S5, inputting the second fusion vector into a classical full-connection layer to obtain a prediction classification label corresponding to the text data, and taking the prediction classification label as an output result.
2. The text classification method based on quantum self-attention hybrid neural network according to claim 1, wherein the two-channel coding strategy in step S1 comprises: For structured or numerical data, one-Hot coding and TF-IDF coding are adopted as double channels; For semantic class text data, doc2Vec coding and Word2Vec coding are adopted as dual channels.
3. The text classification method based on the quantum self-attention hybrid neural network according to claim 1, wherein in step S2, the encoded feature vector is converted into a quantum state through amplitude encoding, specifically by the following formula: ; wherein the classical vector , Representing the quantum state of the input data, Representing the length of the input vector and, Is the ground state of the quantum state.
4. A method of classifying text based on a quantum self-attention hybrid neural network according to claim 3, wherein step S3 specifically comprises: s3-1, inputting the coded quantum state into a shallow parameterized quantum circuit PQC for feature extraction, wherein the parameterized quantum circuit PQC consists of a CNOT gate ring and a single-bit rotation gate column, and adopts an Adam optimizer for parameter updating and takes a cross entropy loss function as an optimization target: ; Wherein, the In order for the cross-entropy loss to occur, Representing data samples Is a real tag of the (c) in the (c), Representation of samples Is used for predicting the probability of (1); s3-2 for each qubit Performing Pauli-Z measurements, performing Sub-measuring and respectively counting quantum states as And Times of (a) And (3) with Calculate the corresponding frequency And (3) with ; S3-3, performing binarization mapping according to the measurement result, if the measured quantum state is The number of times of the quantum state is dominant, the corresponding output is 1, if the measured quantum state is The number of times of the corresponding output is-1, and the quantum bit Classical value of (2) The mapping formula of (2) can be expressed as: ; S3-4, for all The measurement frequency results of the quantum bits are arranged in sequence, and finally, a low-dimensional vector corresponding to each channel is obtained : ; Wherein j represents the number of channels; S3-5, splicing the two into a fusion vector As the first fusion vector.
5. The text classification method based on quantum self-attention hybrid neural network according to claim 4, wherein step S4 specifically comprises: S4-1, inputting the first fusion vector into a quantum self-attention module, wherein the module consists of three parameterized quantum circuits with the same structure and independent parameters, and the parameterized quantum circuits are respectively used as query circuits Key line Sum line ; S4-2, pauli-Z measurement is carried out on the inquiry and key line to obtain vector 、 Pauli measurements performed on value lines Obtaining t-dimension classical vector ; S4-3, calculating quantum attention weight through cosine similarity: ; With the attention weight pairs Weighting to obtain output vector At the same time as input to the next layer: ; quantum feature fusion is achieved through identical residual connection, and a second fusion vector is obtained : 。
6. The text classification method based on quantum self-attention hybrid neural network according to claim 5, wherein the parameter training of the fully connected layer adopts a cross entropy loss and Adam optimizer.
7. A text classification system based on a quantum self-attention hybrid neural network, comprising: The dual-channel coding module is configured to code the input data by adopting a dual-channel coding strategy according to the type of the input data; The quantum state embedding module is configured to execute the following processes of converting the coded feature vector into a quantum state through amplitude coding; the quantum-classical feature fusion module is configured to input quantum states to a shallow parameterized quantum circuit PQC to extract features, perform Pauli-Z measurement after training to obtain a dimension reduction vector, and fuse the two-channel features to form a first fusion vector; The quantum self-attention module is configured to execute the following processes that the first fusion vector is sent to the quantum self-attention module to form three groups of quantum representations of query Q, key K and value V, self-attention weight is obtained through cosine similarity and Softmax normalization calculation, value vectors are weighted, and identity residual error connection is introduced to obtain a second fusion vector; And the classification prediction module is configured to input the second fusion vector into a classical full-connection layer, perform final training classification prediction and output a prediction label.
8. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any one of claims 1 to 6.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the computer program is executed to implement the steps of the method according to any of claims 1 to 6.
10. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the method of any of claims 1 to 6.

Description

Text classification method and system based on quantum self-attention hybrid neural network Technical Field The invention relates to the technical field of quantum computing and artificial intelligence, in particular to a text classification method and system based on a quantum self-attention hybrid neural network. Background In recent years, the continuous development of quantum computing technology brings new opportunities to the field of artificial intelligence, wherein QNNs (Quantum Neural Networks, quantum neural network) is used as a model structure for fusing quantum parallelism characteristics and neural network learning ability, and becomes a research hotspot. The traditional neural network depends on huge parameter quantity and complex hierarchical structure in the characteristic expression and reasoning process, and the QNN can realize mapping and compression of high-dimensional data in a limited sub-bit space through quantum superposition and entanglement, so that the neural network has potential advantages in a small sample learning task. However, since existing Quantum hardware is still in the NISQ (Noisy Intermediate-Scale Quantum) stage, the number of available Quantum bits is limited and the noise level is high, so how to achieve efficient and stable QNNs under limited Quantum resources becomes a key issue. Text classification is the fundamental task of natural language processing. Although classical approaches based on deep learning (such as transfomer) have made significant progress, the computational complexity of their self-attention mechanisms grows in square order with text length, and large-scale model parameters are numerous, resulting in high training and reasoning costs. This provides an incentive to explore more efficient computational paradigms (e.g., quantum computing). However, the current text classification method based on the quantum neural network generally adopts a simple classical coding mode (such as One-Hot), cannot fully adapt to deep semantic features of the text, is limited by noise and resource constraint of NISQ equipment, and is also challenged by unstable training and difficulty in expansion of a complex quantum circuit. To improve the expression capability and global modeling performance of QNN, some studies have attempted to introduce self-attention mechanisms. The quantum self-attention neural network provided by the prior art respectively generates query (Q), key (K) and value (V) vectors by designing three groups of independent quantum circuits, performs feature extraction on a quantum layer, and calculates attention weight by adopting a Gaussian kernel function at a classical end, thereby realizing global dependency modeling of a quantum feature space. The architecture effectively merges quantum characterization with classical self-attention computation, and a typical structure thereof is shown in fig. 1. However, subsequent review studies indicate that most QSAM (Quantum Self-Attention Mechanism) models still suffer from unstable training and limited expansibility during measurement and parameter updating. Hybrid quantum-classical neural networks are considered as a compromise of the NISQ stages. The traditional QCQ-CNN model combines quantum convolution feature extraction, classical convolution middle layer learning and quantum classifier, can realize feature compression and classification under a shallower quantum circuit, and the HQNN model combines a classical embedding layer and a quantum classification module, so that higher robustness is shown in a small sample task. However, the performance of these structures still depends on the line depth and the number of qubits, and as the line scale increases, PQC (Parameterized Quantum Circuit, parameterized quantum lines) is prone to gradient extinction, and training efficiency and model accuracy are significantly affected. In terms of text feature coding, the existing QNLP (Quantum Natural Language Processing ) model mostly adopts fixed One-Hot or TF-IDF coding to vector texts, and the methods are simple and visual, but cannot fully capture semantic relations among vocabularies. Classical Transformer structures have a significant effect in modeling long-range dependencies, but their complexity of self-attention computation grows squarely with input length, resulting in extremely costly training in resource-constrained environments. In addition, quantum hardware noise and measurement error are also important challenges facing current QNNs. The existing quantum error probability estimation and zero noise extrapolation methods can reduce the influence of readout errors to a certain extent, but require additional multiple measurement and noise calibration, and further increase the calculation cost. In summary, the existing QSAM hybrid model still has significant shortcomings in terms of structural complexity, trainability, measurement cost, and task adaptability. On one hand, the fixed text coding mode limits