Search

CN-121982407-A - Multi-view image fusion characterization learning method, device and equipment based on double-layer optimization

CN121982407ACN 121982407 ACN121982407 ACN 121982407ACN-121982407-A

Abstract

Multi-view image fusion characterization learning method, device and equipment based on double-layer optimization relate to the technical field of machine learning. The method comprises splicing feature matrices of multi-view dataset to obtain multi-view data matrix . Calculating an initial affinity matrix of each view, and initializing a fusion affinity matrix View weight set Latent feature representation Shared mapping matrix . Building a two-layer optimization model to learn potential feature representations . The model comprises an objective function of an upper layer optimization task for fusing an affinity matrix at a fixed position Under the constraint of (a), updating potential feature representations Sharing a mapping matrix And the objective function of the underlying optimization task is used to determine the potential feature representation at a fixed point Under the constraint of (a), updating the fusion affinity matrix And view weight set . Performing alternate optimization iteration, namely firstly performing upper optimization task update And Then executing the update of the lower optimizing task And Until termination. Outputting a final latent feature representation For performing tasks.

Inventors

  • LIU JIANWU
  • SONG NA

Assignees

  • 莆田学院

Dates

Publication Date
20260505
Application Date
20260127

Claims (10)

  1. 1. A multi-view image fusion characterization learning method based on double-layer optimization is characterized by comprising the following steps: s1, acquiring a multi-view data set, and splicing feature matrixes of all views to obtain a multi-view data matrix ; S2, calculating an initial affinity matrix of each view by adopting a kernel function according to the multi-view data set, and initializing a fusion affinity matrix based on the initial affinity matrix View weight set Simultaneous representation of potential features for multiple views Shared mapping matrix Carrying out random initialization; s3, constructing a double-layer optimization model to obtain a data matrix And fusion affinity matrix Learning latent feature representations Wherein the double-layer optimization model comprises an upper-layer optimization task and a lower-layer optimization task, and an objective function of the upper-layer optimization task is used for fusing an affinity matrix in a fixed manner Under the constraint of (a), updating potential feature representations Sharing a mapping matrix The objective function of the underlying optimization task is used to determine the potential feature representation Under the constraint of (a), updating the fusion affinity matrix And view weight set ; S4, executing alternate optimization iteration, namely, in each iteration, firstly executing upper optimization task update And (3) with Based on updated Performing lower-level optimization task updates And (3) with Until a preset iteration termination condition is met; s5, outputting final potential characteristic representation Affinity matrix with fusion Wherein the potential features represent For performing clustering or classification tasks.
  2. 2. The multi-view image fusion characterization learning method based on double-layer optimization according to claim 1, wherein the double-layer optimization model is constructed as follows; ; In the middle of Representing a minimum value; representing the real number domain; is the number of samples; Is a dimension of a low-dimensional space; Is an upper layer optimization objective function; representing constraint conditions; a parameter indicating that the objective function takes a minimum value; Is a lower layer optimization objective function; To express the first An affinity matrix for each view; Representing the number of views.
  3. 3. The multi-view image fusion characterization learning method based on double-layer optimization according to claim 1, wherein the lower-layer optimization task is simplified into the following constraint optimization target; ; The constraint conditions are as follows: ; In the middle of Represent the first Weights for the individual views; is a pre-determined super parameter; Is that And Similarity of (2); Representing a minimum value; Is a lower layer optimization objective function; representing the number of views; To express the first An affinity matrix for each view; Representing the Frobenius norm; Weight coefficient for consistency loss; is the number of samples; And Index of the samples respectively; And Respectively is Is the first of (2) Line and th The number of row vectors, Represents the encoding of one sample; Representing the L2 norm.
  4. 4. The multi-view image fusion characterization learning method based on double-layer optimization according to claim 3, wherein the execution lower-layer optimization task of S4 is updated And (3) with Comprises fixing fusion affinity matrix Updating view weight sets by closed-form solution Wherein the weight is The solution model of (2) is as follows; When (when) When (1): ; When (when) When (1): ; In the middle of Is that Is the optimal solution of (a); representing the parameters that cause the objective function to take a minimum value.
  5. 5. The multi-view image fusion characterization learning method based on double-layer optimization according to claim 3, wherein the execution lower-layer optimization task of S4 is updated And (3) with Also include a fixed view weight set Updating fusion affinity matrix by adopting projection gradient descent method ; ; ; ; ; ; In the middle of For after updating ; A projection operation for the second stage; is a projection operation of the first stage; Is the first Fusion affinity matrix in the time of iteration; Is the first Learning rate at the time of iteration; is an upper layer optimization objective function At the point Gradient at; To represent matrix based on current characteristics The calculated square matrix of the distance between the samples; Is that Is the first of (2) A row vector; representing a large value; Representing the transpose.
  6. 6. The multi-view image fusion characterization learning method based on double-layer optimization as claimed in claim 1, wherein the objective function of the upper-layer optimization task is as follows Specifically defined as a simplified penalty applicable to unsupervised and semi-supervised learning; ; ; In the middle of An affinity matrix normalized for symmetry; Is that Middle sample And sample Similarity values after the symmetry; Degree matrix Is the first of (2) Diagonal elements; Degree matrix of Diagonal elements; Is a weighting parameter; Is the first Each of the marked samples with a single thermal encoding; Is that Is the number of (3); Representing a minimum value; And Index of the samples respectively; And Respectively is Is the first of (2) Line and th The number of row vectors, Represents the encoding of one sample; Represents an L2 norm; Indicating the Frobenius norm.
  7. 7. The multi-view image fusion characterization learning method based on double-layer optimization as set forth in claim 6, wherein the upper-layer optimization task update is performed in S4 And (3) with Comprising the following steps: Fixed shared mapping matrix Obtained by solving the Sylvester equation The Sylvester equation is in the form as follows; ; when all data labels are not available, the Sylvester equation is in the form as follows; ; In the middle of Is a unit matrix; Is a transposition; To represent the true tag matrix of the marked samples; fixing latent feature representations Updating a shared mapping matrix ; 。
  8. 8. The multi-view fusion characterization learning method based on the double-layer optimization according to any one of claims 1 to 7, wherein in step S2, the kernel function adopts a gaussian kernel, a polynomial kernel or a linear kernel; fusion affinity matrix Initializing to be the average value of all view initial affinity matrices; view weight set Initializing to be uniformly distributed, namely, the weight of each view is equal, and the number of the views is one-half; the random initialization is specifically to randomly generate a matrix of a preset shape from a standard normal distribution.
  9. 9. A multi-view fusion characterization learning device based on double-layer optimization, comprising: The data module is used for acquiring a multi-view data set, and splicing the feature matrixes of all views to obtain a multi-view data matrix ; An initialization module for calculating an initial affinity matrix of each view by using a kernel function according to the multi-view dataset, and initializing a fusion affinity matrix based on the initial affinity matrix View weight set Simultaneous representation of potential features for multiple views Shared mapping matrix Carrying out random initialization; A model module for constructing a double-layer optimization model according to the data matrix And fusion affinity matrix Learning latent feature representations Wherein the double-layer optimization model comprises an upper-layer optimization task and a lower-layer optimization task, and an objective function of the upper-layer optimization task is used for fusing an affinity matrix in a fixed manner Under the constraint of (a), updating potential feature representations Sharing a mapping matrix The objective function of the underlying optimization task is used to determine the potential feature representation Under the constraint of (a), updating the fusion affinity matrix And view weight set ; The iteration module is used for executing alternate optimization iteration, namely, in each iteration, the upper layer optimization task update is firstly executed And (3) with Based on updated Performing lower-level optimization task updates And (3) with Until a preset iteration termination condition is met; An output module for outputting the final potential feature representation Affinity matrix with fusion Wherein the potential features represent For performing clustering or classification tasks.
  10. 10. A multi-view image fusion characterization learning device based on double-layer optimization, comprising a processor, a memory, and a computer program stored in the memory, wherein the computer program is executable by the processor to implement a multi-view image fusion characterization learning method based on double-layer optimization as claimed in any one of claims 1 to 8.

Description

Multi-view image fusion characterization learning method, device and equipment based on double-layer optimization Technical Field The invention relates to the technical field of machine learning, in particular to a multi-view image fusion characterization learning method, device and equipment based on double-layer optimization. Background Multiview data refers to a collection of observations describing the same underlying object or phenomenon from different perspectives, such as capturing the same event through modalities such as text, video, and audio, with a low-level representation that is heterogeneous but shares consistent high-level semantics. The data format can surpass the limitation of single-view data by integrating complementary information from multiple dimensions and modes, and provides a strong foundation for revealing the underlying structure and the inherent attribute of the data, so that the requirement for learning uniform and robust potential representation is promoted, and the performance of downstream tasks such as clustering, classification and the like is improved. To enable multi-view token learning, existing approaches mainly include typical correlation analysis and its variants, such as sparse CCA and nuclear CCA, which aim to project different views to a shared low-dimensional subspace and maximize correlation. Deep learning based methods use deep neural networks instead of linear projections to learn more expressive nonlinear potential representations. Graph-based methods explicitly model pairwise similarities between views, e.g., graph convolutional networks discover consensus graphs by jointly integrating structure and feature information, and mainstream methods encompass predefined graph fusion, learnable graph optimization, and subspace-based learning paradigms. Despite the advances made in the prior art, they suffer from significant drawbacks. The prior method generally ignores the coupling learning of graph topology consistency and feature space invariance and fails to fully meet the requirement of consistent alignment of topology and feature distribution on multiple granularities. The mainstream method adopts simple strategies such as weighted average or a final layer attention mechanism, can not jointly model topological consistency and characteristic invariance across multiple layers, which limits the integrity and discriminant of the obtained cross-view representation and promotes urgent demands for new learning paradigms. Disclosure of Invention The invention provides a multi-view image fusion characterization learning method, device and equipment based on double-layer optimization, which are used for improving at least one of the technical problems. In a first aspect, the invention provides a multi-view image fusion characterization learning method based on double-layer optimization, which comprises steps S1 to S5. S1, acquiring a multi-view data set, and splicing feature matrixes of all views to obtain a multi-view data matrix。 S2, calculating an initial affinity matrix of each view by adopting a kernel function according to the multi-view data set, and initializing a fusion affinity matrix based on the initial affinity matrixView weight set. Simultaneous representation of potential features for multiple viewsShared mapping matrixRandom initialization is performed. S3, constructing a double-layer optimization model to obtain a data matrixAnd fusion affinity matrixLearning latent feature representations. The double-layer optimization model comprises an upper-layer optimization task and a lower-layer optimization task. The objective function of the upper layer optimization task is used for fusing the affinity matrix at fixedUnder the constraint of (a), updating potential feature representationsSharing a mapping matrix. The objective function of the underlying optimization task is used to determine the potential feature representationUnder the constraint of (a), updating the fusion affinity matrixAnd view weight set。 S4, executing alternate optimization iteration, namely, in each iteration, firstly executing upper optimization task updateAnd (3) withBased on updatedPerforming lower-level optimization task updatesAnd (3) withUntil a preset iteration termination condition is satisfied. S5, outputting final potential characteristic representationAffinity matrix with fusion. Wherein the potential features representFor performing clustering or classification tasks. As a further aspect of the present invention, a two-layer optimization model is constructed as follows. 。 In the middle ofRepresenting a minimum value.Representing the real number domain.Is the number of samples.Is the dimension of the low-dimensional space.Is an upper layer optimization objective function.Representing constraints.Representing the parameters that cause the objective function to take a minimum value.Is the lower layer optimization objective function.To express the firstAffinity matrix for individua