CN-121986494-A - Encoding/decoding method, encoder, decoder, and storage medium

Abstract

The application discloses a coding and decoding method, a coder, a decoder and a storage medium, wherein at a coding and decoding end, the use or permission of a loop filtering technology based on a neural network for a current block is determined, a first reconstruction image of the current block is obtained, the first reconstruction image comprises reconstruction samples of the current block, a first edge image of the current block is obtained, the first edge image comprises edge information of the reconstruction samples, and the first reconstruction image and the first edge image are input into a loop filtering model based on the neural network to obtain a filtered image of the current block. In this way, the edge image is used as newly added side information to be input into the loop filtering model, the edge information of the sample level can be provided for the network, the learning of the filtering intensity is adjusted at the sample level, the filtering effect is improved, and therefore the quality of the reconstructed image and the encoding and decoding performance are improved.

Inventors

• XIE ZHIHUANG

Assignees

• OPPO广东移动通信有限公司

Dates

Publication Date

20260505

Application Date

20230927

Claims (20)

1. A decoding method applied to a decoder, the method comprising: decoding the code stream and determining first indication information; determining that the current block uses a loop filtering technology based on a neural network according to the first indication information; acquiring a first reconstructed image of a current block, wherein the first reconstructed image comprises reconstructed samples of the current block; acquiring a first edge image of a current block, wherein the first edge image comprises edge information of the reconstructed sample; And inputting the first reconstructed image and the first edge image into a loop filtering model based on a neural network to obtain a filtered image of the current block.
2. The method of claim 1, wherein the edge information comprises edge strength.
3. The method according to claim 1 or 2, wherein the method further comprises: acquiring a second reconstructed image of a frame where the current block is located; Performing edge detection on the second reconstructed image based on a first edge detection operator to determine a second edge image; The obtaining the first edge image of the current block includes: And acquiring the first edge image of the current block from the second edge image.
4. A method according to claim 3, wherein said obtaining a second reconstructed image of a frame in which the current block is located comprises: acquiring an initial reconstructed image of a frame where a current block is located; The initial reconstructed image is used as the second reconstructed image And performing deblocking filtering and/or sample self-adaptive compensation on the initial reconstructed image to obtain the second reconstructed image.
5. A method according to claim 3, wherein the method further comprises: decoding a code stream, and determining an index value of the first edge detection operator; and determining the first edge detection operator from an edge detection operator list according to the index value of the first edge detection operator.
6. A method according to claim 3, wherein the first edge detection operator is a sobel operator; the edge detection is performed on the second reconstructed image based on a first edge detection operator, and the determining of the second edge image comprises the following steps: Detecting gradient information of the second reconstructed image, and determining transverse gradient information and longitudinal gradient information; and determining the second edge image according to the transverse gradient information and the longitudinal gradient information.
7. The method of claim 6, wherein the determining the second edge image from the lateral gradient information and the longitudinal gradient information comprises: Determining the absolute value of the gradient of the reconstructed sample according to the transverse gradient of the reconstructed sample in the transverse gradient information and the longitudinal gradient of the reconstructed sample in the longitudinal gradient information; and taking the absolute value of the gradient as edge information of the reconstructed sample.
8. A method according to claim 3, wherein after said determining the second edge image, the method further comprises: and denoising the second edge image to obtain a denoised second edge image.
9. The method of claim 8, wherein the denoising the second edge image to obtain a denoised second edge image comprises: If the edge intensity of the first reconstructed sample is greater than or equal to a first threshold value, reserving the edge intensity of the first reconstructed sample; And if the edge intensity of the second reconstructed sample is smaller than the first threshold value, setting the edge intensity of the second reconstructed sample to zero.
10. The method of claim 9, wherein the method further comprises: Determining an average value of edge intensities of all reconstructed samples in the second edge image; And determining the first threshold according to the average value.
11. A method according to claim 3, wherein the first and second edge images are chrominance component edge images, the method further comprising: Determining a luminance component second edge image and a chrominance component second edge image; And determining a final second edge image of the chrominance component according to the second edge image of the luminance component and the second edge image of the chrominance component.
12. The method of claim 1, wherein the acquiring a first reconstructed image of the current block comprises: acquiring an initial reconstructed image of a frame where a current block is located; The first reconstructed image of the current block is acquired from the initial reconstructed image.
13. The method of claim 1, wherein the method further comprises: Acquiring at least one of a predicted image, boundary strength information, slice type information and quantization information of a current block; And simultaneously inputting at least one of the predicted image, the boundary strength information, the slice type information and the quantization information into the loop filtering model based on the neural network to obtain a filtered image of the current block.
14. The method of claim 1, wherein the first indication information comprises a first syntax element identification and a second syntax element identification; The first syntax element identifier is used for indicating whether the image sequence of the current block allows the loop filtering technology based on the neural network to be used or not, and the second syntax element identifier is used for indicating whether the current block uses the loop filtering technology based on the neural network or not.
15. The method according to any one of claims 1 to 14, wherein the neural network based loop filter model comprises an input unit, a feature extraction unit, and an output unit; The step of inputting the first reconstructed image and the first edge image into a loop filtering model based on a neural network to obtain a filtered image of the current block, including: Inputting the first reconstructed image to the input unit, and then, through the feature extraction unit, inputting the feature information of the output first reconstructed image to the output unit by the feature extraction unit; And the first edge image is input to the output unit, the output unit processes the characteristic images of the first edge image and the first reconstructed image, and the filtered image is output.
16. The method of any one of claims 1 to 15, wherein the method further comprises: In the model training stage, performing edge detection on the training image based on a plurality of edge detection operators to obtain a plurality of edge images; Constructing a training sample set according to the training image, the plurality of edge images and the truth image; And training the loop filtering model based on the neural network by using the training sample set to obtain a trained loop filtering model.
17. An encoding method applied to an encoder, the method comprising: determining that the current block allows use of a neural network based loop filtering technique; acquiring a first reconstructed image of a current block, wherein the first reconstructed image comprises reconstructed samples of the current block; acquiring a first edge image of a current block, wherein the first edge image comprises edge information of the reconstructed sample; Inputting the first reconstructed image and the first edge image into a loop filtering model based on a neural network to obtain a filtered image of the current block; According to the original image of the current block and the filtered image, carrying out cost calculation to determine a first generation value; Determining whether the current block uses a loop filtering technology based on a neural network according to the first generation value, and setting first indication information of the current block; And encoding the first indication information, and writing the obtained encoded bits into a code stream.
18. The method of claim 17, wherein the edge information comprises edge strength.
19. The method according to claim 17 or 18, wherein the method further comprises: acquiring a second reconstructed image of a frame where the current block is located; Performing edge detection on the second reconstructed image based on a first edge detection operator to determine a second edge image; The obtaining the first edge image of the current block includes: And acquiring the first edge image of the current block from the second edge image.
20. The method of claim 19, wherein the acquiring the second reconstructed image of the frame in which the current block is located comprises: acquiring an initial reconstructed image of a frame where a current block is located; The initial reconstructed image is used as the second reconstructed image And performing deblocking filtering and/or sample self-adaptive compensation on the initial reconstructed image to obtain the second reconstructed image.

Description

Encoding/decoding method, encoder, decoder, and storage medium Technical Field The embodiment of the application relates to the technical field of video encoding and decoding, in particular to an encoding and decoding method, an encoder, a decoder and a storage medium. Background With the improvement of the requirements of people on the video display quality, new video application forms such as high-definition and ultra-high-definition videos and the like are generated. The international organization for standardization ISO/IEC and ITU-T joint video research group (Joint Video Exploration Team, JVET) established the next generation video coding standard h.266/multi-function video coding (VERSATILE VIDEO CODING, VVC). At present, a neural network is introduced in the video coding and decoding field, and coding and decoding tools based on the neural network have very high coding and decoding efficiency by means of strong learning capability of the neural network. For example, an intra prediction method based on a neural network, an inter prediction method based on a neural network, and a loop filtering method based on a neural network, wherein the encoding performance of the loop filtering method based on a neural network is most prominent. However, the current loop filtering method based on the neural network does not fully exert the advantages of the neural network model, and in some coding and decoding scenes, the loop filtering method based on the neural network does not greatly improve the filtering effect, and even worsens the filtering efficiency. Therefore, the neural network-based loop filtering method is to be optimized. Disclosure of Invention The embodiment of the application provides a coding and decoding method, an encoder, a decoder and a storage medium, which are used for inputting an edge image as newly added side information into a loop filtering model and improving the filtering effect. The technical scheme of the embodiment of the application can be realized as follows: in a first aspect, an embodiment of the present application provides a decoding method, applied to a decoder, including: decoding the code stream and determining first indication information; determining that the current block uses a loop filtering technology based on a neural network according to the first indication information; acquiring a first reconstructed image of a current block, wherein the first reconstructed image comprises reconstructed samples of the current block; acquiring a first edge image of a current block, wherein the first edge image comprises edge information of the reconstructed sample; And inputting the first reconstructed image and the first edge image into a loop filtering model based on a neural network to obtain a filtered image of the current block. In a second aspect, an embodiment of the present application provides an encoding method, applied to an encoder, including: determining that the current block allows use of a neural network based loop filtering technique; acquiring a first reconstructed image of a current block, wherein the first reconstructed image comprises reconstructed samples of the current block; acquiring a first edge image of a current block, wherein the first edge image comprises edge information of the reconstructed sample; Inputting the first reconstructed image and the first edge image into a loop filtering model based on a neural network to obtain a filtered image of the current block; According to the original image of the current block and the filtered image, carrying out cost calculation to determine a first generation value; Determining whether the current block uses a loop filtering technology based on a neural network according to the first generation value, and setting first indication information of the current block; And encoding the first indication information, and writing the obtained encoded bits into a code stream. In a third aspect, an embodiment of the present application provides an encoder, where the encoder includes a first determining unit, a first filtering unit, a decision unit, and an encoding unit, where: the first determining unit is configured to determine that the current block allows using a loop filtering technology based on a neural network; the first determining unit is further configured to obtain a first reconstructed image of the current block, wherein the first reconstructed image comprises reconstructed samples of the current block; The first determining unit is further configured to acquire a first edge image of the current block, wherein the first edge image comprises edge information of the reconstructed sample; The first filtering unit is configured to input the first reconstructed image and the first edge image into a loop filtering model based on a neural network to obtain a filtered image of the current block; The decision unit is configured to perform cost calculation according to the original image of the current block and the