CN-122023559-A - Image oil painting stylization method and system based on deep learning

Abstract

The application relates to an image oil painting stylization method and system based on deep learning, which comprises the steps of outputting conversion intention information when receiving an image to be converted and corresponding requirement information input by a user side, screening out style vectors matched with the art concept information to generate an oil painting sketch of the image to be converted, identifying and extracting the art concept information, associated modification area information and modification type information from the modification requirement information when receiving the modification requirement information input by the user side, judging whether the type of the style vectors needs to be modified, screening out to-be-replaced style vectors matched with the art concept information in the modification requirement information if required, and adjusting style vector parameters of corresponding areas in the oil painting sketch based on the modification type information and the modification area information if not required. The application has the effect of realizing the man-machine collaborative creation function in the process of carrying out the stylization treatment of the oil painting on the image.

Inventors

• LI XIANGZHEN
• Chen Zutong

Assignees

• 广东开放大学（广东理工职业学院）

Dates

Publication Date

20260512

Application Date

20260205

Claims (10)

1. 1. The image oil painting stylization method based on deep learning is characterized by comprising the following steps of: When receiving an image to be converted and corresponding requirement information input by a user terminal, carrying out semantic recognition on the requirement information, and outputting conversion intention information, wherein the conversion intention information comprises artistic conception information; Selecting style vectors matched with artistic conception information from a preset knowledge base, and fusing the matched style vectors to an image to be converted to generate an oil painting sketch of the image to be converted; when receiving modification requirement information input by a user side, identifying and extracting artistic conception information, associated modification area information and modification type information from the modification requirement information; judging whether the type of the style vector needs to be changed or not based on the modified type information; If so, screening a to-be-replaced style vector matched with the artistic conception information in the modification requirement information from a preset knowledge base, and based on the modification region information, fusing and replacing the to-be-replaced style vector to a corresponding region in the oil painting sketch; If not, based on the modification type information and the modification area information, adjusting style vector parameters of the corresponding area in the oil painting sketch.
2. 2. The method for stylizing an image oil painting based on deep learning according to claim 1, wherein the requirement information includes a requirement text, the step of semantically recognizing the requirement information when receiving an image to be converted and corresponding requirement information input by a user terminal, outputting conversion intention information, and the conversion intention information includes artistic conception information, includes: carrying out semantic analysis on vocabulary information in the demand text, and selecting single or two or more keywords in the demand text; identifying the artistic concept type of the keyword; and screening all related art concepts of the affiliated art concept type from a preset concept database to obtain art concept information, wherein the related art concepts comprise genre information, artist information, stroke information, color information or composition information.
3. 3. The method for stylizing an image oil painting based on deep learning according to claim 2, wherein the step of identifying the type of artistic conception to which the keyword belongs comprises the steps of: identifying the association degree of the keywords and each associated artistic concept, and binding the current keywords with the associated artistic concepts exceeding the association threshold when the associated artistic concepts exceeding the association threshold exist; when a plurality of keywords exist, identifying the artistic conception types to which each bound related artistic concept belongs, and obtaining single or a plurality of artistic conception types to be selected; when the obtained artistic concept types are multiple, judging the weight value of each artistic concept type, and taking the artistic concept type with the highest weight value as the artistic concept type to which the keyword belongs.
4. 4. The method for image oil painting stylization based on deep learning of claim 1, wherein the step of identifying and extracting artistic conception information and associated modification area information and modification type information from modification requirement information when receiving modification requirement information inputted by a user terminal comprises: extracting keywords from the modification requirement information based on semantic recognition, and acquiring artistic concept information corresponding to the keywords; Determining modification type information based on artistic conception information in the modification demand information, the change condition of the artistic conception information in the demand information and text in the modification demand information; extracting an image area selected by a user side from the modification requirement information and associated area feature words; and modifying the image area based on the selected image area and the area feature words, generating modified area information and sending the modified area information to a user side, wherein the modified area information comprises a modified area range.
5. 5. The method for stylizing an image oil painting based on deep learning according to claim 4, wherein the step of modifying the image area based on the frame selected image area and the area feature word to generate modified area information, the modified area information including a modified area range, comprises: Acquiring an image to be converted, and carrying out feature recognition on the image to be converted based on the regional feature words to obtain a first regional feature vector which can be accurately matched with the regional feature words in the image to be converted; screening out a second region feature vector with the similarity to the first region feature vector being larger than a first approximation threshold value and adding the second region feature vector to the frame-selected image region; mapping the modified area range to an oil painting sketch and sending the oil painting sketch to a user side for range confirmation; When a confirmation instruction sent by a user side is received, determining the current modification area range; When an active adjustment instruction sent by the user side is received, a modification result when the user side sends a confirmation instruction is used as a modification area range.
6. 6. The method for stylizing an image oil painting based on deep learning according to claim 5, wherein the step of obtaining the image to be converted, performing feature recognition on the image to be converted based on the regional feature words, and obtaining a first regional feature vector capable of accurately matching the regional feature words in the image to be converted comprises the steps of: Inputting the image to be converted into a preset image feature recognition model to recognize element features in the image, and matching the element features with corresponding element name information; acquiring coordinate information of various types of element name information in an image to be converted, and screening out coordinate information of all element features related to the element name information and the regional feature words; And packing the element characteristics and the coordinate information to obtain a first region characteristic vector.
7. 7. The method for image oil painting stylization based on deep learning of claim 5, wherein when the active adjustment instruction sent by the user side is received, the modification result when the user side sends the confirmation instruction is used as the modification area range, and the active adjustment instruction includes scaling, translating and adjusting the size of the modification area range, and deleting and recovering the newly added and deleted areas.
8. 8. An image oil painting stylization system based on degree of depth study, its characterized in that: The intention conversion module is used for carrying out semantic recognition on the requirement information when receiving the to-be-converted image input by the user side and the corresponding requirement information, and outputting conversion intention information, wherein the conversion intention information comprises artistic conception information; the sketch generation module is used for screening out style vectors matched with the artistic conception information from a preset knowledge base, fusing the matched style vectors to the image to be converted, and generating an oil painting sketch of the image to be converted; The modification requirement module is used for identifying and extracting artistic conception information, associated modification area information and modification type information from the modification requirement information when the modification requirement information input by the user terminal is received; The system comprises a modification module, a modification module and a modification module, wherein the modification module is used for judging whether the type of the style vector needs to be modified based on modification type information, screening a style vector to be replaced, which is matched with artistic conception information in modification requirement information, from a preset knowledge base if the type of the style vector needs to be modified, fusing and replacing the style vector to be replaced to a corresponding area in the oil painting sketch based on modification area information, and adjusting style vector parameters of the corresponding area in the oil painting sketch based on modification type information and modification area information if the style vector is not required.
9. 9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of a deep learning based image oil painting stylization method according to any of claims 1 to 7 when the computer program is executed.
10. 10. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the steps of a deep learning based image oil painting stylization method according to any one of claims 1 to 7.

Description

Image oil painting stylization method and system based on deep learning Technical Field The application relates to the technical field of image generation, in particular to an image oil painting stylization method and system based on deep learning. Background Currently, AI software for image stylization migration exists in the market, and a user can complete image style conversion by inputting an image and selecting a style to be converted, including multiple styles such as oil painting, sketch, watercolor and the like. However, the characteristics of model learning in the current migration technology comprise various art styles, a user can only input a content image and select the art style which is required to be converted and generated, the image generation process is uncontrollable, and fine adjustment is difficult to be carried out on specific areas or specific art effects in detail. The images generated many times are random, and it is difficult to obtain the expected result. Therefore, an image conversion method special for the oil painting style is designed to realize man-machine cooperation in the image generation process. Disclosure of Invention In order to realize a man-machine collaborative creation function in the process of carrying out oil painting stylization processing on an image so as to generate a stylized oil painting meeting the requirements of a user, the application provides an image oil painting stylization method and an image oil painting stylization system based on deep learning. The first object of the present application is achieved by the following technical solutions: an image oil painting stylization method based on deep learning comprises the following steps: When receiving an image to be converted and corresponding requirement information input by a user terminal, carrying out semantic recognition on the requirement information, and outputting conversion intention information, wherein the conversion intention information comprises artistic conception information; Selecting style vectors matched with artistic conception information from a preset knowledge base, and fusing the matched style vectors to an image to be converted to generate an oil painting sketch of the image to be converted; when receiving modification requirement information input by a user side, identifying and extracting artistic conception information, associated modification area information and modification type information from the modification requirement information; judging whether the type of the style vector needs to be changed or not based on the modified type information; If so, screening a to-be-replaced style vector matched with the artistic conception information in the modification requirement information from a preset knowledge base, and based on the modification region information, fusing and replacing the to-be-replaced style vector to a corresponding region in the oil painting sketch; If not, based on the modification type information and the modification area information, adjusting style vector parameters of the corresponding area in the oil painting sketch. Through the technical scheme, when the user performs oil painting style conversion of the image, the user inputs the requirement information through the user side, the requirement information comprises text or image requirement display, semantic recognition is further performed on the requirement information, the conversion intention of the user, namely artistic conception information, such as concepts including artists, works, strokes, color composition and the like, such as strokes of the 'star night' works, strokes of the Mongolian, impression assignment, strong light and dark contrast, is obtained through semantic recognition of the text or feature analysis of the image, and the corresponding feature strokes and colors are fused into the input image based on the initially output artistic conception information, so that a preliminary oil painting sketch is obtained. After the oil painting sketch is generated, a user can also input modification requirement information through a user side, the modification requirement information also comprises a text type and an image type requirement expression, in the modification requirement information, the user can select all or part of modification areas, the artistic conception information, the modification area information and the modification type information in the modification requirement of the user are obtained through semantic identification or image feature identification, firstly, whether the artistic conception information needs to be replaced in the current modification requirement or not is identified through the modification type information, namely, the type of a style vector is changed, if so, the style vector matched with the artistic conception information is screened out again, the style vector is replaced to the selected area to be modified in the image, if not, only parame