CN-118297995-B - Image alignment method, device, electronic equipment, chip and medium

CN118297995BCN 118297995 BCN118297995 BCN 118297995BCN-118297995-B

Abstract

The method comprises the steps of obtaining a reference image and a plurality of intermediate alignment images corresponding to a source image, carrying out alignment processing on the source image by combining the reference image and a plurality of image alignment strategies to obtain the intermediate alignment images, determining a prediction conversion matrix for converting the reference image according to the reference image and the plurality of intermediate alignment images, carrying out conversion processing on the reference image according to the prediction conversion matrix to obtain a target alignment image corresponding to the source image, wherein different image alignment strategies are suitable for processing images in different fields, so that the scheme can be suitable for images in a plurality of fields, has strong generalization, and improves image alignment efficiency.

Inventors

SUN FEIRAN
Yin xuanwu

Assignees

上海玄戒技术有限公司

Dates

Publication Date: 20260508
Application Date: 20240329

Claims (11)

1. An image alignment method, the method comprising: Acquiring a reference image and a plurality of intermediate alignment images corresponding to a source image, wherein the plurality of intermediate alignment images comprise images obtained by combining the reference image with a plurality of image alignment strategies to perform alignment processing on the source image; determining a prediction conversion matrix for converting the reference image according to the reference image and the plurality of intermediate alignment images; performing conversion processing on the reference image according to the prediction conversion matrix to obtain a target alignment image corresponding to the source image; The method for determining the prediction conversion matrix for converting the reference image according to the reference image and the plurality of intermediate aligned images comprises the steps of inputting the reference image and the plurality of intermediate aligned images into a conversion matrix prediction model, and obtaining the prediction conversion matrix output by the conversion matrix prediction model; The transformation matrix prediction model is obtained through training according to a sample reference image and a plurality of sample alignment images, and the sample reference image and the plurality of sample alignment images are obtained through image processing of the same sample original image.
2. The method of claim 1, wherein the acquiring the reference image and the plurality of intermediate alignment images corresponding to the source image comprises: acquiring the source image and a reference image corresponding to the source image; performing alignment processing on the source image by combining the reference image and the plurality of image alignment strategies to obtain a plurality of candidate alignment images; and selecting the plurality of intermediate alignment images from the plurality of candidate alignment images according to the reference image.
3. The method of claim 2, wherein the selecting the plurality of intermediate alignment images from the plurality of candidate alignment images based on the reference image comprises: determining alignment scores of the plurality of candidate alignment images from the reference image; according to the alignment score, performing descending order sorting treatment on the plurality of candidate alignment images to obtain a sorting result; And determining a plurality of candidate alignment images which are positioned in front in the sequencing result as the intermediate alignment image.
4. A method according to claim 3, wherein said determining alignment scores for the plurality of candidate alignment images from the reference image comprises: determining a reference hash sequence corresponding to the reference image and candidate hash sequences corresponding to the candidate alignment images according to a difference hash algorithm; Determining, for each candidate alignment image, a distance between a candidate hash sequence of the candidate alignment image and the reference hash sequence; and determining alignment scores of the candidate alignment images according to the distances corresponding to the candidate alignment images.
5. A method of training a transition matrix predictive model, the method comprising: The method comprises the steps of acquiring training data, wherein the training data comprises a sample reference image and a plurality of sample alignment images corresponding to the sample reference image, and the sample reference image and the plurality of sample alignment images are obtained by processing the same sample original image; acquiring an initial conversion matrix prediction model; determining a predicted image according to the sample reference image, a plurality of sample alignment images corresponding to the sample reference image and the conversion matrix prediction model; according to the predicted image, a sample original image corresponding to the sample reference image and a loss function of the conversion matrix predicted model, carrying out parameter adjustment processing on the conversion matrix predicted model to realize training; The method comprises the steps of determining a predicted image according to the sample reference image, a plurality of sample alignment images corresponding to the sample reference image and the conversion matrix prediction model, inputting the sample reference image and the plurality of sample alignment images corresponding to the sample reference image into the conversion matrix prediction model, obtaining a prediction conversion matrix which is output by the conversion matrix prediction model and is used for converting the sample reference image, and converting the sample reference image according to the prediction conversion matrix to obtain the predicted image.
6. The method of claim 5, wherein the acquiring training data comprises: acquiring the original image of the sample; performing image enhancement processing on the sample original image to obtain the sample reference image; and carrying out local area transformation processing on the original sample image for multiple times to obtain a plurality of sample alignment images corresponding to the sample reference image.
7. An image alignment apparatus, the apparatus comprising: The system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring a reference image corresponding to a source image and a plurality of intermediate alignment images, and the plurality of intermediate alignment images comprise images obtained by combining the reference image with a plurality of image alignment strategies to perform alignment processing on the source image; a determining module, configured to determine a prediction conversion matrix for performing conversion processing on the reference image according to the reference image and the plurality of intermediate aligned images; the conversion processing module is used for carrying out conversion processing on the reference image according to the prediction conversion matrix to obtain a target alignment image corresponding to the source image; The determination module is specifically configured to input the reference image and the plurality of intermediate aligned images into a conversion matrix prediction model, obtain the prediction conversion matrix output by the conversion matrix prediction model, train the conversion matrix prediction model according to a sample reference image and a plurality of sample aligned images, and perform image processing on the sample reference image and the plurality of sample aligned images from the same sample original image.
8. A training device for a transformation matrix prediction model, the device comprising: the system comprises a first acquisition module, a second acquisition module and a first acquisition module, wherein the first acquisition module is used for acquiring training data, the training data comprises a sample reference image and a plurality of sample alignment images corresponding to the sample reference image, and the sample reference image and the plurality of sample alignment images are processed by the same sample original image; the second acquisition module is used for acquiring an initial conversion matrix prediction model; the determining module is used for determining a predicted image according to the sample reference image, a plurality of sample alignment images corresponding to the sample reference image and the conversion matrix prediction model; The training module is used for carrying out parameter adjustment processing on the conversion matrix prediction model according to the prediction image, the sample original image corresponding to the sample reference image and the loss function of the conversion matrix prediction model so as to realize training; The determination module is specifically configured to input the sample reference image and a plurality of sample alignment images corresponding to the sample reference image into the transformation matrix prediction model, obtain a prediction transformation matrix output by the transformation matrix prediction model and used for transforming the sample reference image, and transform the sample reference image according to the prediction transformation matrix to obtain the prediction image.
9. An electronic device, comprising: A processor; a memory for storing the processor-executable instructions; Wherein the processor is configured to: A step of implementing the image alignment method according to any one of claims 1 to 4, or a training method of implementing the transformation matrix prediction model according to any one of claims 5 to 6.
10. A non-transitory computer readable storage medium, which when executed by a processor, causes the processor to perform the image alignment method of any one of claims 1 to 4, or to perform the training method of the transformation matrix prediction model of any one of claims 5 to 6.
11. A chip comprising one or more interface circuits and one or more processors, the interface circuits being configured to receive signals from a memory of an electronic device and to send the signals to the processor, the signals comprising computer instructions stored in the memory, which when executed by the processor, cause the electronic device to perform the image alignment method of any one of claims 1 to 4, or to perform the training method of the transition matrix prediction model of any one of claims 5 to 6.

Description

Image alignment method, device, electronic equipment, chip and medium Technical Field The disclosure relates to the technical field of image processing, and in particular relates to an image alignment method, an image alignment device, electronic equipment, a chip and a medium. Background Currently, the image alignment technique can align the source image with the reference image through spatial rotation, translation, pixel mapping, and other transformations to obtain an aligned image. The matrix during conversion is determined according to the source image and the reference image. Among them, an image alignment algorithm involved in the image alignment technique, for example, an AI image alignment algorithm, or the like. The AI image alignment algorithm requires training of image data in a specific field, and has large processing amount and poor generalization, so that the image alignment efficiency is poor. Disclosure of Invention The disclosure provides an image alignment method, an image alignment device, electronic equipment, a chip and a medium. According to a first aspect of an embodiment of the present disclosure, an image alignment method is provided, which includes obtaining a reference image and a plurality of intermediate alignment images corresponding to a source image, wherein the plurality of intermediate alignment images are obtained by performing alignment processing on the source image in combination with the reference image and a plurality of image alignment strategies, determining a prediction conversion matrix for performing conversion processing on the reference image according to the reference image and the plurality of intermediate alignment images, and performing conversion processing on the reference image according to the prediction conversion matrix to obtain a target alignment image corresponding to the source image. In one embodiment of the disclosure, the determining a prediction transformation matrix for transforming the reference image according to the reference image and the plurality of intermediate aligned images includes inputting the reference image and the plurality of intermediate aligned images into a transformation matrix prediction model, and obtaining the prediction transformation matrix output by the transformation matrix prediction model. In one embodiment of the disclosure, the transformation matrix prediction model is trained according to a sample reference image and a plurality of sample alignment images, wherein the sample reference image and the plurality of sample alignment images are obtained by performing image processing on the same sample original image. In one embodiment of the disclosure, the acquiring a reference image and a plurality of intermediate alignment images corresponding to a source image includes acquiring the source image and the reference image corresponding to the source image, performing alignment processing on the source image in combination with the reference image and the plurality of image alignment strategies to obtain a plurality of candidate alignment images, and selecting the plurality of intermediate alignment images from the plurality of candidate alignment images according to the reference image. In one embodiment of the disclosure, the selecting the plurality of intermediate aligned images from the plurality of candidate aligned images according to the reference image includes determining alignment scores of the plurality of candidate aligned images according to the reference image, performing descending order sorting processing on the plurality of candidate aligned images according to the alignment scores to obtain a sorting result, and determining a plurality of candidate aligned images which are earlier in the sorting result as the intermediate aligned images. In one embodiment of the disclosure, determining the alignment scores of the plurality of candidate alignment images according to the reference image comprises determining a reference hash sequence corresponding to the reference image and a candidate hash sequence corresponding to the plurality of candidate alignment images according to a difference hash algorithm, determining a distance between the candidate hash sequence of the candidate alignment image and the reference hash sequence for each candidate alignment image, and determining the alignment scores of the plurality of candidate alignment images according to the distances corresponding to the plurality of candidate alignment images. According to a second aspect of the disclosed embodiments, a training method for a transformation matrix prediction model is provided, which includes obtaining training data, wherein the training data includes a sample reference image and a plurality of sample alignment images corresponding to the sample reference image, the sample reference image and the plurality of sample alignment images are processed by the same sample original image, obtaining an initial transformation m