CN-117197621-B - Image quality partial order processing method and device, electronic equipment and storage medium

CN117197621BCN 117197621 BCN117197621 BCN 117197621BCN-117197621-B

Abstract

The embodiment of the disclosure discloses an image quality partial sequence processing method, an image quality partial sequence processing device, electronic equipment and a storage medium. The method comprises the steps of determining visual information and non-visual information of an image to be processed, wherein the visual information comprises image characteristics which can be obtained through visual elements in the image to be processed, the non-visual information comprises image characteristics which can be obtained through analysis of non-visual elements associated with the image to be processed, and performing image quality judgment operation by fusing the visual information and the non-visual information to obtain predicted image quality of the image to be processed, further performing image out-of-order operation on the image to be processed according to the predicted image quality of the image to be processed, considering information which is easily and intuitively identified such as image picture content and the like in image quality judgment, and simultaneously considering characteristics which cannot be intuitively identified, so that the image quality judgment effectiveness can be realized from the whole image characteristics, the out-of-order processing efficiency between images is greatly improved, and further, high-quality images are automatically screened.

Inventors

WANG RENKUI
ZHU HANGCHENG
MA GUOJUN

Assignees

北京字跳网络技术有限公司

Dates

Publication Date: 20260508
Application Date: 20220531

Claims (12)

1. A method for partial order processing of image quality, the method comprising: The method comprises the steps of determining visual information and non-visual information of an image to be processed, wherein the visual information comprises image features which can be obtained through visual elements in the image to be processed, and the non-visual information comprises image features which are obtained through analysis of non-visual elements associated with the image to be processed; performing image quality judgment operation by fusing the visual information and the non-visual information to obtain the predicted image quality of the image to be processed; Performing image partial sequence operation on an image to be processed according to the predicted image quality of the image to be processed; the image quality judging operation is carried out by fusing the visual information and the non-visual information to obtain the predicted image quality of the image to be processed, and the method comprises the following steps: inputting visual information of the image to be processed into a first image quality judging model, and obtaining visual characteristics of the image to be processed by executing visual characteristic extraction operation; Fusing the non-visual features acquired from the non-visual information with the visual features and inputting the fused non-visual features into a second image quality judgment model to obtain the predicted image quality of the image to be processed; The first image quality judging model is obtained by training the model through visual information of training sample images in a training set, the second image quality judging model is obtained by training the model through visual information and non-visual information of the training sample images, the image clicking times of the training sample images are larger than a preset threshold, and the training sample images are evenly distributed on different image clicking ratios in a preset range after being ordered according to the image derived clicking ratio.
2. The method of claim 1, wherein the visual information is used to measure image picture content attributes described by at least one of image color, image texture, image lines, and image content richness, and/or The non-visual information is used to measure image-appended attributes other than image frame content, the image-appended attributes being described by at least one of image format, image color space type, image size, and image authoring information.
3. The method of claim 1, wherein the first image quality determination model and the second image quality determination model are each model trained with an image derived click ratio as an image quality determination metric.
4. The method of claim 1, wherein fusing the non-visual features in the non-visual information with the visual features for input to a second image quality decision model comprises: coding and mapping the non-visual features in the non-visual information to obtain discretized non-visual features; fusing the visual features of the image to be processed with the discretized non-visual features to obtain multi-modal features; The multi-modal feature is input to the second image quality determination model to perform an image quality determination operation to output a predicted image quality of the image to be processed.
5. The method according to any one of claims 1, 3, 4, wherein the first image quality decision model is obtained by model iterative training based at least on a visual feature extraction network.
6. The method of any of claims 1, 3, 4, wherein the second image quality decision model is based on at least a gradient lifting framework network of weak classifiers for model iterative training.
7. The method according to claim 1, wherein the method further comprises: determining a test sample image from a test set and a reference image of the test sample image; performing image quality judgment on the test sample image and a reference image of the test sample image through the first image quality judgment model and the second image quality judgment model to obtain the image quality of the test sample image and the predicted image quality of the reference image; and counting the partial order of the predicted image quality and the partial order of the pre-marked image quality between the test sample image and the reference image to determine whether to start updating the first image quality judging model and the second image quality judging model.
8. The method of claim 7, wherein the image derived click ratio difference between the test sample image and the reference image is greater than a test standard deviation of a preset multiple, the test standard deviation being determined based on the image derived click ratio standard deviation of each test sample image in the test set.
9. The method of claim 7, wherein counting the predicted image quality partial order and the pre-labeled image quality partial order between the test sample image and the reference image comprises: If the predicted image quality partial order and the pre-marked image quality partial order between the test sample image and the reference image are consistent, judging the image quality of the test sample image and the reference image as a test positive example; If the predicted image quality partial order between the test sample image and the reference image is inconsistent with the pre-marked image quality partial order, judging the image quality of the test sample image and the reference image as a test negative example; and counting the number of the test positive cases and the number of the test negative cases, and calculating the number ratio between the test positive cases and the test negative cases as a positive-negative sequence ratio.
10. An image quality partial sequence processing apparatus, characterized in that the apparatus comprises: The information determining module is used for determining visual information and non-visual information of an image to be processed, wherein the visual information comprises image features which can be obtained through visual elements in the image to be processed, and the non-visual information comprises image features which are obtained through analysis of non-visual elements associated with the image to be processed; the image quality judging module is used for fusing the visual information with the non-visual information to execute image quality judging operation to obtain the predicted image quality of the image to be processed; The image partial sequence processing module is used for executing image partial sequence operation on the image to be processed according to the predicted image quality of the image to be processed; Wherein the image quality determination module comprises: inputting visual information of the image to be processed into a first image quality judging model, and obtaining visual characteristics of the image to be processed by executing visual characteristic extraction operation; fusing the non-visual features in the non-visual information with the visual features and inputting the fused non-visual features into a second image quality judgment model to obtain the predicted image quality of the image to be processed; The first image quality judging model is obtained by training the model through visual information of training sample images in a training set, the second image quality judging model is obtained by training the model through visual information and non-visual information of the training sample images, the image clicking times of the training sample images are larger than a preset threshold, and the training sample images are evenly distributed on different image clicking ratios in a preset range after being sequenced according to the image derived clicking ratio, but are not distributed in a part of the image clicking ratio range in an aggregation mode.
11. An electronic device, the electronic device comprising: At least one processor, and A memory communicatively coupled to the at least one processor, wherein, The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the image quality partial order processing method of any of claims 1-9.
12. A computer readable medium, characterized in that it stores computer instructions for causing a processor to implement the image quality partial order processing method of any of claims 1-9 when executed.

Description

Image quality partial order processing method and device, electronic equipment and storage medium Technical Field The embodiment of the disclosure relates to the technical field of image processing, in particular to a method and a device for processing partial order of image quality, electronic equipment and a storage medium. Background In many consumption scenes facing to wide users, such as classification pages, the image contents in the classification pages can be configured and then displayed in sequence for the users to derive. At present, the image quality is mainly characterized by giving the corresponding evidence meaning score to the image through manual scoring, but with the increasing of business systems, the image categories and the image quantity are continuously increased, and the regular updating is difficult to achieve through manual configuration. In addition, many images are not high in quality and the actual deriving use times are not high, but the images are displayed in front of the sequence and the clicking rate is high, so that many high-quality images cannot be better displayed and are missed to be derived for use. Therefore, it becomes important to analyze images in similar classified pages in order to optimally present quality images. Disclosure of Invention The embodiment of the disclosure provides a method, a device, electronic equipment and a storage medium for processing partial image quality order, so as to optimize an image quality evaluation process, automatically select high-quality images as much as possible for optimization display, and greatly improve image ordering efficiency and accuracy. In a first aspect, an embodiment of the present disclosure provides a method for processing partial order of image quality, where the method includes: The method comprises the steps of determining visual information and non-visual information of an image to be processed, wherein the visual information comprises image features which can be obtained through visual elements in the image to be processed, and the non-visual information comprises image features which are obtained through analysis of non-visual elements associated with the image to be processed; performing image quality judgment operation by fusing the visual information and the non-visual information to obtain the predicted image quality of the image to be processed; and executing image partial sequence operation on the image to be processed according to the predicted image quality of the image to be processed. In a second aspect, in an embodiment of the present disclosure, there is further provided an apparatus for processing partial image quality sequences, including: The information determining module is used for determining visual information and non-visual information of an image to be processed, wherein the visual information comprises image features which can be obtained through visual elements in the image to be processed, and the non-visual information comprises image features which are obtained through analysis of non-visual elements associated with the image to be processed; the image quality judging module is used for fusing the visual information with the non-visual information to execute image quality judging operation to obtain the predicted image quality of the image to be processed; And the image partial sequence processing module is used for executing image partial sequence operation on the image to be processed according to the predicted image quality of the image to be processed. In a third aspect, an embodiment of the present disclosure further provides an electronic device, including: At least one processor, and A memory communicatively coupled to the at least one processor, wherein, The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the image quality partial order processing method of any of the above embodiments. In a fourth aspect, there is also provided in an embodiment of the disclosure a computer readable medium storing computer instructions for causing a processor to execute the method for partial order processing of image quality according to any one of the above embodiments. According to the technical scheme, when the partial order processing is carried out on the image, visual information and non-visual information of the image to be processed are determined, the visual information comprises image features which can be obtained through visual elements in the image to be processed, the non-visual information comprises image features which are obtained through analysis of non-visual elements related to the image to be processed, the visual information and the non-visual information are fused to execute image quality judging operation, the predicted image quality of the image to be processed is obtained, further, the partial order operation is carried out on the image to be processed according to the predicted image quali