US-20260129242-A1 - ITERATIVELY UPDATING A FILTERING MODEL

US20260129242A1US 20260129242 A1US20260129242 A1US 20260129242A1US-20260129242-A1

Abstract

In a data processing method, a first iteration of a filter model update operation is performed. The first iteration includes obtaining a current video coding application that includes a current filtering model, generating to-be-filtered reconstructed video data based on encoding sample video data via the current video coding application, obtaining an updated filtering model based on training the current filtering model with a current training data set derived from the sample video data and the to-be-filtered reconstructed video data, and obtaining an updated video coding application that updates the current filtering model with the updated filtering model. In the method, when the updated filtering model is determined to meet a quality requirement condition, a target video stream is generated based on encoding, by the processing circuitry via the updated video coding application, target video data.

Inventors

Liqiang Wang

Assignees

TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED

Dates

Publication Date: 20260507
Application Date: 20251229
Priority Date: 20211221

Claims (20)

1 . A data processing method, comprising: performing a first iteration of a filter model update operation, the first iteration including: obtaining a current video coding application that includes a current filtering model; generating to-be-filtered reconstructed video data based on encoding, by processing circuitry via the current video coding application, sample video data; obtaining an updated filtering model based on training the current filtering model with a current training data set derived from the sample video data and the to-be-filtered reconstructed video data; and obtaining an updated video coding application that updates the current filtering model with the updated filtering model; and when the updated filtering model is determined to meet a quality requirement condition, generating a target video stream based on encoding, by the processing circuitry via the updated video coding application, target video data.
2 . The method according to claim 1 , further comprising: when the updated filtering model is determined not to meet the quality requirement condition, performing one or more successive iterations of the filter model update operation until the updated filtering model is determined to meet the quality requirement condition.
3 . The method according to claim 2 , wherein the one or more successive iterations of the filter model update operation is K iterations, denoted as a second iteration to a (K+1)-th iteration, K being a positive integer, a k-th iteration of the filter model update operation is based on: a (k−1)-th updated video coding application including a (k−1)-th filtering model from the (k−1)-th iteration as a k-th video coding application; and a k-th training data set derived from the sample video data and a k-th to-be-filtered reconstructed video data that is obtained based on encoding, by the processing circuitry via the (k−1)-th updated video coding application, the sample video data, and k is a positive integer ranging from 2 to K+1.
4 . The method according to claim 3 , wherein the k-th training data set includes one or more training data pairs, each of the one or more training data pairs of the k-th training data set includes a sample frame from the sample video data and a corresponding to-be-filtered reconstructed frame from the k-th to-be-filtered reconstructed video data.
5 . The method according to claim 4 , wherein the k-th iteration comprises: obtaining a current filtered reconstructed frame based on processing a current to-be-filtered reconstructed frame of one of the one or more training data pairs via the (k−1)-th filtering model; obtaining an error value between the current filtered reconstructed frame and a current sample frame of the one of the one or more training data pairs; and adjusting one or more parameters of the (k−1)-th filtering model based on the error value.
6 . The method according to claim 5 , wherein the k-th iteration comprises: when the adjusted (k−1)-th filtering model with the one or more adjusted parameter is determined to meet a model convergence condition, determining the adjusted (k−1)-th filtering model as the k-th filtering model of the k-th iteration.
7 . The method according to claim 5 , wherein the obtaining the error value during the k-th iteration comprises: obtaining a loss function corresponding to the k-th video coding application; obtaining original image quality corresponding to the current sample frame; obtaining filtered image quality corresponding to the current filtered reconstructed frame; and determining the error value between the current filtered reconstructed frame and the current sample frame based on the loss function, the original image quality, and the filtered image quality.
8 . The method according to claim 1 , further comprising: obtaining a loss function corresponding to the updated video coding application; generating filtered reconstructed video data based on processing, by the processing circuitry via the updated filtering model, the to-be-filtered reconstructed video data; obtaining original image quality corresponding to the sample video data; obtaining filtered image quality corresponding to the filtered reconstructed video data; determining an error value between the filtered reconstructed video data and the sample video data based on the loss function, the original image quality, and the filtered image quality; determining that the updated filtering model meets the quality requirement condition based on the error value being less than a reference value; and determining that the updated filtering model does not meet the quality requirement condition based on the error value being not less than the reference value.
9 . The method according to claim 8 , wherein the loss function comprises an absolute value loss function or a square error loss function.
10 . The method according to claim 1 , wherein the current filtering model comprises: an intra filtering model for filtering a to-be-filtered reconstructed frame belonging to an intra prediction type; or an inter filtering model for filtering a to-be-filtered reconstructed frame belonging to an inter prediction type.
11 . A data processing apparatus, comprising: processing circuitry configured to: perform a first iteration of a filter model update operation, the first iteration including: obtaining of a current video coding application that includes a current filtering model; generation of to-be-filtered reconstructed video data based on encoding, via the current video coding application, sample video data; obtaining of an updated filtering model based on training the current filtering model with a current training data set derived from the sample video data and the to-be-filtered reconstructed video data; and obtaining of an updated video coding application that updates the current filtering model with the updated filtering model; and when the updated filtering model is determined to meet a quality requirement condition, generate a target video stream based on encoding, via the updated video coding application, target video data.
12 . The data processing apparatus according to claim 11 , wherein the processing circuitry is configured to: when the updated filtering model is determined not to meet the quality requirement condition, perform one or more successive iterations of the filter model update operation until the updated filtering model is determined to meet the quality requirement condition.
13 . The data processing apparatus according to claim 12 , wherein the one or more successive iterations of the filter model update operation is K iterations, denoted as a second iteration to a (K+1)-th iteration, K being a positive integer, a k-th iteration of the filter model update operation is based on: a (k−1)-th updated video coding application including a (k−1)-th filtering model from the (k−1)-th iteration as a k-th video coding application; and a k-th training data set derived from the sample video data and a k-th to-be-filtered reconstructed video data that is obtained based on encoding, via the (k−1)-th updated video coding application, the sample video data, and k is a positive integer ranging from 2 to K+1.
14 . The data processing apparatus according to claim 13 , wherein the k-th training data set includes one or more training data pairs, each of the one or more training data pairs of the k-th training data set includes a sample frame from the sample video data and a corresponding to-be-filtered reconstructed frame from the k-th to-be-filtered reconstructed video data.
15 . The data processing apparatus according to claim 14 , wherein, to perform the k-th iteration, the processing circuitry is configured to: obtain a current filtered reconstructed frame based on processing a current to-be-filtered reconstructed frame of one of the one or more training data pairs via the (k−1)-th filtering model; obtain an error value between the current filtered reconstructed frame and a current sample frame of the one of the one or more training data pairs; and adjust one or more parameters of the (k−1)-th filtering model based on the error value.
16 . The data processing apparatus according to claim 15 , wherein, to perform the k-th iteration, the processing circuitry is configured to: when the adjusted (k−1)-th filtering model with the one or more adjusted parameter is determined to meet a model convergence condition, determine the adjusted (k−1)-th filtering model as the k-th filtering model of the k-th iteration.
17 . The data processing apparatus according to claim 15 , wherein, to obtain the error value during the k-th iteration, the processing circuitry is configured to: obtain a loss function corresponding to the k-th video coding application; obtain original image quality corresponding to the current sample frame; obtain filtered image quality corresponding to the current filtered reconstructed frame; and determine the error value between the current filtered reconstructed frame and the current sample frame based on the loss function, the original image quality, and the filtered image quality.
18 . The data processing apparatus according to claim 11 , wherein the processing circuitry is configured to: obtain a loss function corresponding to the updated video coding application; generate filtered reconstructed video data based on processing, via the updated filtering model, the to-be-filtered reconstructed video data; obtain original image quality corresponding to the sample video data; obtain filtered image quality corresponding to the filtered reconstructed video data; determine an error value between the filtered reconstructed video data and the sample video data based on the loss function, the original image quality, and the filtered image quality; determining that the updated filtering model meets the quality requirement condition based on the error value being less than a reference value; and determining that the updated filtering model does not meet the quality requirement condition based on the error value being not less than the reference value.
19 . A non-transitory computer-readable storage medium storing computer-readable instructions thereon, which, when executed by processing circuitry, cause the processing circuitry to perform a data processing method comprising: performing a first iteration of a filter model update operation, the first iteration including: obtaining a current video coding application that includes a current filtering model; generating to-be-filtered reconstructed video data based on encoding, via the current video coding application, sample video data; obtaining an updated filtering model based on training the current filtering model with a current training data set derived from the sample video data and the to-be-filtered reconstructed video data; and obtaining an updated video coding application that updates the current filtering model with the updated filtering model; and when the updated filtering model is determined to meet a quality requirement condition, generating a target video stream based on encoding, via the updated video coding application, target video data.
20 . The non-transitory computer-readable storage medium according to claim 19 , wherein the method further comprises: when the updated filtering model is determined not to meet the quality requirement condition, performing one or more successive iterations of the filter model update operation until the updated filtering model is determined to meet the quality requirement condition.

Description

RELATED APPLICATIONS This application is a continuation of U.S. patent application Ser. No. 18/389,483, filed on Nov. 14, 2023, which is a continuation of International Patent Application No. PCT/CN2022/126828, filed on Oct. 21, 2022, which claims priority to Chinese Patent Application No. 202111576383.3, filed on Dec. 21, 2021. The disclosures of the prior applications are hereby incorporated by reference in their entirety. FIELD OF THE TECHNOLOGY This application relates to the technical field of computers, including a data processing method and apparatus, a computer device, a computer-readable storage medium, and a computer program product. BACKGROUND OF THE DISCLOSURE At present, in the video coding and compression process, video coding and compression algorithms all adopt lossy compression, that is, a certain distortion is present between an image of a coded and compressed video and an image of an original video; furthermore, in the case of high compression rate, image warping and distortion are more serious. A loop filter has been introduced in an existing video coding standard to filter the image of the compressed video, so that the distortion degree is reduced, and the image quality of the compressed video can be infinitely close to the image quality of the original video. However, since a filtering coefficient of a related loop filter is mainly designed manually, which relies too much on manual experience, so the accuracy is not high and the distortion degree cannot be reduced well. SUMMARY Embodiments of this disclosure provide a data processing method and apparatus, a computer device, a computer-readable storage medium, and a computer program product, which can improve the filtering performance, reducing the image distortion degree of a coded video and enhancing the image quality of the coded video. In an embodiment, a data processing method includes generating first training data through a kth updated video coding application including a first filtering model, the first training data including a sample original video frame as a training label, and a first sample to-be-filtered reconstructed frame output from the kth updated video coding application and corresponding to the sample original video frame. K is a positive integer. The method further includes, based on the sample original video frame and the first sample to-be-filtered reconstructed frame, training a second filtering model in the kth updated video coding application to obtain the second filtering model in a training convergence state, and integrating the second filtering model in the training convergence state into the kth updated video coding application to obtain a (k+1)th updated video coding application. The method further includes, in response to a determination that the (k+1)th updated video coding application meets a filtering quality requirement condition, determining the (k+1)th updated video coding application as a target video coding application for performing video coding processing. In an embodiment, a data processing apparatus includes processing circuitry configured to generate first training data through a kth updated video coding application including a first filtering model, the first training data including a sample original video frame as a training label, and a first sample to-be-filtered reconstructed frame output from the kth updated video coding application and corresponding to the sample original video frame. K is a positive integer. The processing circuitry is further configured to, based on the sample original video frame and the first sample to-be-filtered reconstructed frame, train a second filtering model in the kth updated video coding application to obtain the second filtering model in a training convergence state. The processing circuitry is further configured to integrate the second filtering model in the training convergence state into the kth updated video coding application to obtain a (k+1)th updated video coding application, and, in response to a determination that the (k+1)th updated video coding application meets a filtering quality requirement condition, determine the (k+1)th updated video coding application as a target video coding application for performing video coding processing. In an embodiment, a non-transitory computer-readable storage medium stores computer-readable instructions thereon, which, when executed by processing circuitry, cause the processing circuitry to perform a data processing method that includes generating first training data through a kth updated video coding application including a first filtering model, the first training data including a sample original video frame as a training label, and a first sample to-be-filtered reconstructed frame output from the kth updated video coding application and corresponding to the sample original video frame. K is a positive integer. The method further includes, based on the sample original video frame and the first sample to-be-