CN-122023570-A - Multi-mode-based image processing system

Abstract

The application provides an image processing system based on multiple modes, which comprises the steps of obtaining image information to be processed, and processing target information in the image information to be processed to obtain processed image information.

Inventors

• Mu Zhangqian
• San Xinpei

Assignees

• 深圳市TCL高新技术开发有限公司

Dates

Publication Date

20260512

Application Date

20241105

Claims (20)

1. 1. A method, the method comprising: acquiring image information to be processed; And processing the image information to be processed according to the target information in the image information to be processed to obtain processed image information.
2. 2. The method of claim 1, wherein the processing of the image information to be processed according to target information in the image information to be processed is based on a target information processing model, wherein the target information processing model includes a text feature extraction module, an image feature extraction module, and a target information processing module.
3. 3. The method of claim 2, wherein an input of the text feature extraction module and an input of the image feature extraction module are configured to receive the image information to be processed; The output end of the text feature extraction module and the output end of the image feature extraction module are connected with the input end of the target information processing module; and the output end of the target information processing module outputs the processed image information.
4. 4. The method of claim 3, wherein the text feature extraction module is configured to perform text feature extraction on text information in the image information to be processed to obtain target word information; the image feature extraction module is configured to extract image features of the image information in the image information to be processed so as to obtain target image information; The target information processing module is configured to process the image information to be processed according to the target word information and the target image information to obtain the processed image information.
5. 5. The method according to claim 4, wherein the processing the image information to be processed according to the target word information and the target image information to obtain the processed image information includes: When the target word information is target processing word information, carrying out feature processing on the target word information in the image information to be processed to obtain processed image information, and/or, And when the target image information is target processing image information, performing feature processing on the target image information in the image information to be processed to obtain processed image information.
6. 6. The method of claim 5, wherein before the target word information is target processing word information, the method further comprises: determining similarity information of the target word information and word information to be compared; and determining whether the target word information is target processing word information according to the similarity information.
7. 7. The method of claim 6, wherein said determining similarity information of the target word information and word information to be compared comprises: performing information mapping processing on the target word information and the word information to be compared to obtain a first comparison factor of the target word information and a second comparison factor of the word information to be compared; And determining similarity information of the target word information and the word information to be compared according to the first comparison factor and the second comparison factor.
8. 8. The method of claim 2, wherein the text feature extraction module comprises a text information extraction module, a word segmentation module, and a target text extraction module.
9. 9. The method of claim 8, wherein an input of the text information extraction module is configured to receive the image information to be processed; The output end of the text information extraction module is connected with the input end of the word segmentation module; the output end of the word segmentation module is connected with the input end of the target text extraction module; and the output end of the target text extraction module is used for outputting target word information.
10. 10. The method according to claim 8, wherein the text information extraction module is configured to extract text information from text information in the image information to be processed to obtain initial text information; the word segmentation module is configured to perform word segmentation processing on the initial text information to obtain a plurality of segmented word information; The target text extraction module is configured to determine each of the divided word information, and determine whether each of the divided word information is target word information.
11. 11. The method of claim 10, wherein said determining each of said segmented word information to determine whether each of said segmented word information is a target word information comprises: each piece of segmentation word information is used as word information to be processed, and word sequence information of the word information to be processed in the initial text information is determined; determining probability distribution information of the word information to be processed according to the first order of the word order information and the second order of the word order information; and determining whether the word information to be processed is the target word information according to the probability distribution information.
12. 12. The method of claim 11, wherein the target text extraction module comprises a first feature determination module, a second feature determination module, a probability determination module, and a target word information output module; The input end of the first characteristic determining module and the input end of the second characteristic determining module are respectively connected with the output end of the word segmentation module; The output end of the first characteristic determining module and the output end of the second characteristic determining module are respectively connected with the input end of the probability determining module; And the output end of the probability determining module is connected with the input end of the target word information output module.
13. 13. The method of claim 12, wherein the determining the probability distribution information of the word information to be processed according to the first order of the word order information and the second order of the word order information comprises: according to word sequence information of each word information to be processed, inputting each word information to be processed into the first feature determining module according to the first sequence to obtain first feature information; according to word sequence information of each word information to be processed, inputting each word information to be processed into the second feature determining module according to the second sequence to obtain second feature information; and inputting the first characteristic information and the second characteristic information into the probability determining module to obtain the probability distribution information.
14. 14. The method of claim 2, wherein the image feature extraction module comprises an initial image feature extraction module, a feature enhancement module, a multi-scale reconstruction module, an information fusion module, and a target image information output module.
15. 15. The method of claim 14, wherein an input of the initial image feature extraction module is configured to receive the image information to be processed; The output end of the initial image feature extraction module is respectively connected with the input end of the feature enhancement module and the input end of the multi-scale reconstruction module; The output end of the characteristic enhancement module and the output end of the multi-scale reconstruction module are connected with the input end of the information fusion module; The output end of the information fusion module is connected with the input end of the target image information output module; And the output end of the target image information output module is used for outputting target image information.
16. 16. The method of claim 14, wherein the initial image feature extraction module is configured to perform initial image feature extraction on image information in the image information to be processed to obtain initial image feature information; the characteristic enhancement module is configured to enhance the initial image characteristic information to obtain enhanced image characteristic information; The multi-scale reconstruction module is configured to perform nesting processing on the initial image characteristic information to obtain reconstructed image characteristic information; The information fusion module is configured to fuse the enhanced image characteristic information and the reconstructed image characteristic information to obtain fused image characteristic information; The target image information output module is configured to recognize the fused image characteristic information to obtain target image information.
17. 17. The method of claim 16, wherein the enhancing the initial image feature information to obtain enhanced image feature information comprises: extracting attention information from the initial image characteristic information to obtain target attention information; Performing initial tag identification on the initial image characteristic information according to the target attention information to obtain initial tag information, wherein the initial tag information is used for representing initial classification result information of the initial image characteristic information; carrying out semantic information identification on the initial tag information to obtain target semantic information; and obtaining the enhanced image characteristic information according to the target semantic information.
18. 18. The method of claim 16, wherein the nesting the initial image feature information to obtain reconstructed image feature information comprises: extracting first features of the initial image feature information to obtain first image feature information; Extracting second features of the first image feature information to obtain second image feature information; Fusing the first image characteristic information and the second image characteristic information to obtain third image characteristic information; extracting third characteristics from the third image characteristic information to obtain fourth image characteristic information; and fusing the fourth image characteristic information with the initial image characteristic information to obtain reconstructed image characteristic information.
19. 19. A system, the system comprising: The acquisition module is used for acquiring the image information to be processed; the processing module is used for processing the image information to be processed according to the target information in the image information to be processed to obtain processed image information; Further, the processing module comprises a target information processing model, wherein the target information processing model comprises a text feature extraction module, an image feature extraction module and a target information processing module; Further, the target information processing model connecting structure in the processing module comprises an input end of the text feature extraction module and an input end of the image feature extraction module, wherein the input end of the image feature extraction module is used for receiving the image information to be processed; The output end of the text feature extraction module and the output end of the image feature extraction module are connected with the input end of the target information processing module; The output end of the target information processing module outputs the processed image information; further, the text feature extraction module in the processing module is configured to extract text features of text information in the image information to be processed so as to obtain target word information; the image feature extraction module is configured to extract image features of the image information in the image information to be processed so as to obtain target image information; The target information processing module is configured to process the image information to be processed according to the target word information and the target image information to obtain processed image information; further, the processing module processes the image information to be processed according to the target word information and the target image information to obtain the processed image information, and includes: When the target word information is target processing word information, carrying out feature processing on the target word information in the image information to be processed to obtain processed image information, and/or, When the target image information is target processing image information, performing feature processing on the target image information in the image information to be processed to obtain processed image information; further, the processing module determining the before the target word information is the target processing word information further includes: determining similarity information of the target word information and word information to be compared; Determining whether the target word information is target processing word information according to the similarity information; Further, the processing module determines similarity information of the target word information and word information to be compared, including: performing information mapping processing on the target word information and the word information to be compared to obtain a first comparison factor of the target word information and a second comparison factor of the word information to be compared; Determining similarity information of the target word information and word information to be compared according to the first comparison factor and the second comparison factor; Further, the text feature extraction module in the processing module comprises a text information extraction module, a word segmentation module and a target text extraction module; further, the architecture of the text feature extraction module in the processing module comprises that the input end of the text information extraction module is used for receiving the image information to be processed; The output end of the text information extraction module is connected with the input end of the word segmentation module; the output end of the word segmentation module is connected with the input end of the target text extraction module; the output end of the target text extraction module is used for outputting target word information; Further, the text information extraction module in the processing module is configured to extract text information from the text information in the image information to be processed to obtain initial text information; the word segmentation module is configured to perform word segmentation processing on the initial text information to obtain a plurality of segmented word information; The target text extraction module is configured to determine each piece of divided word information, and determine whether each piece of divided word information is target word information; Further, the processing module determines each of the segmented word information, and determines whether each of the segmented word information is target word information, including: each piece of segmentation word information is used as word information to be processed, and word sequence information of the word information to be processed in the initial text information is determined; determining probability distribution information of the word information to be processed according to the first order of the word order information and the second order of the word order information; determining whether the word information to be processed is the target word information according to the probability distribution information; Further, the target text extraction module comprises a first feature determination module, a second feature determination module, a probability determination module and a target word information output module; The input end of the first characteristic determining module and the input end of the second characteristic determining module are respectively connected with the output end of the word segmentation module; The output end of the first characteristic determining module and the output end of the second characteristic determining module are respectively connected with the input end of the probability determining module; The output end of the probability determining module is connected with the input end of the target word information output module; Further, the processing module determines probability distribution information of the word information to be processed according to the first order of the word order information and the second order of the word order information, including: according to word sequence information of each word information to be processed, inputting each word information to be processed into the first feature determining module according to a first sequence to obtain first feature information; according to word sequence information of each word information to be processed, inputting each word information to be processed into the second feature determining module according to a second sequence to obtain second feature information; Inputting the first characteristic information and the second characteristic information into the probability determining module to obtain the probability distribution information; Further, the image feature extraction module in the processing module comprises an initial image feature extraction module, a feature enhancement module, a multi-scale reconstruction module, an information fusion module and a target image information output module; Further, the input end of the initial image feature extraction module in the processing module is used for receiving the image information to be processed; The output end of the initial image feature extraction module is respectively connected with the input end of the feature enhancement module and the input end of the multi-scale reconstruction module; The output end of the characteristic enhancement module and the output end of the multi-scale reconstruction module are connected with the input end of the information fusion module; The output end of the information fusion module is connected with the input end of the target image information output module; The output end of the target image information output module is used for outputting target image information; further, the initial image feature extraction module in the processing module is configured to perform initial image feature extraction on the image information in the image information to be processed so as to obtain initial image feature information; the characteristic enhancement module is configured to enhance the initial image characteristic information to obtain enhanced image characteristic information; The multi-scale reconstruction module is configured to perform nesting processing on the initial image characteristic information to obtain reconstructed image characteristic information; The information fusion module is configured to fuse the enhanced image characteristic information and the reconstructed image characteristic information to obtain fused image characteristic information; the target image information output module is configured to recognize the fused image characteristic information to obtain target image information; further, the processing module performs enhancement processing on the initial image feature information to obtain enhanced image feature information, including: extracting attention information from the initial image characteristic information to obtain target attention information; Performing initial tag identification on the initial image characteristic information according to the target attention information to obtain initial tag information, wherein the initial tag information is used for representing initial classification result information of the initial image characteristic information; carrying out semantic information identification on the initial tag information to obtain target semantic information; obtaining enhanced image feature information according to the target semantic information; Further, the processing module performs nesting processing on the initial image feature information to obtain reconstructed image feature information, and the method includes: extracting first features of the initial image feature information to obtain first image feature information; Extracting second features of the first image feature information to obtain second image feature information; Fusing the first image characteristic information and the second image characteristic information to obtain third image characteristic information; extracting third characteristics from the third image characteristic information to obtain fourth image characteristic information; and fusing the fourth image characteristic information with the initial image characteristic information to obtain reconstructed image characteristic information.
20. 20. An apparatus comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to perform the steps in the method of any one of claims 1 to 18.

Description

Multi-mode-based image processing system Technical Field The application relates to the technical field of computers, in particular to an image processing system based on multiple modes. Background Along with the development of computer technology, images become an important form of information transmission, and the analysis and effective processing of target information in image information are of great significance to the effectiveness and safety of information transmission. In the prior art, target information in image information cannot be effectively analyzed and processed. Disclosure of Invention The application provides an image processing system based on multiple modes. In a first aspect, the present application provides a method comprising: acquiring image information to be processed; And processing the image information to be processed according to the target information in the image information to be processed to obtain the processed image information. In a second aspect, the present application also provides a system comprising: The acquisition module is used for acquiring the image information to be processed; The processing module is used for processing the image information to be processed according to the target information in the image information to be processed to obtain the processed image information. In a third aspect, the application also provides an apparatus comprising a processor, a memory and a computer program stored in the memory and executable on the processor, the processor executing the computer program to perform the steps of any of the methods. In a fourth aspect, the application also provides a computer readable storage medium having a computer program stored thereon, the computer program being executable by a processor to perform the steps of any of the methods. Drawings In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. FIG. 1 is a schematic view of a scene of an image processing system provided in an embodiment of the application; FIG. 2 is a flow chart of an embodiment of an image processing method according to an embodiment of the present application; FIG. 3 is a schematic diagram of a model architecture of an embodiment of an image processing method according to an embodiment of the present application; FIG. 4 is a schematic diagram of a model architecture of an embodiment of an image processing method according to an embodiment of the present application; FIG. 5 is a schematic diagram of a model architecture of an embodiment of an image processing method according to an embodiment of the present application; FIG. 6 is a schematic diagram of a model architecture of an embodiment of an image processing method according to an embodiment of the present application; FIG. 7 is a schematic diagram of a model architecture of an embodiment of an image processing method according to an embodiment of the present application; FIG. 8 is a functional block diagram of an image processing system according to an embodiment of the present application; fig. 9 is a schematic structural diagram of an electronic device in an embodiment of the present application. Detailed Description The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application. In the description of the present application, it should be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present application, the meaning of "a plurality" is two or more, unless explicitly defined otherwise. In the present application, the term "exemplary" is used to mean "serving as an example, instance, or illustration. Any embodiment described as "exemplary" in this disclosure is not necessarily to be construed as preferred or advantageous over other embodiments. Meanwhile, it can be understood that, in the specific embodiment of the present application, related data such as user information and user data are related, when the a