Search

EP-4465172-B1 - METHOD FOR PERFORMING AUTOMATIC ACTIVATION CONTROL REGARDING VARIABLE RATE SHADING, AND ASSOCIATED APPARATUS

EP4465172B1EP 4465172 B1EP4465172 B1EP 4465172B1EP-4465172-B1

Inventors

  • HUANG, PO-YU
  • LIN, SHIH-CHIN
  • TSAI, CHING-YI
  • TSAO, YOU-MING

Dates

Publication Date
20260506
Application Date
20240226

Claims (15)

  1. A method for performing automatic activation control regarding variable rate shading, in the following also referred to as VRS, the method being applied to a processing circuit within an electronic device, the method comprising: utilizing a rendering classifier within the processing circuit to intercept at least one set of original graphic commands on a command path to obtain at least one rendering property from the at least one set of original graphic commands, for classifying rendering corresponding to the at least one set of original graphic commands (S11); and utilizing the rendering classifier to classify the rendering corresponding to the at least one set of original graphic commands into at least one predetermined rendering type among multiple predetermined rendering types according to the at least one rendering property, in order to determine at least one shading rate corresponding to the at least one predetermined rendering type for the rendering corresponding to the at least one set of original graphic commands (S12); utilizing a shading rate controller within the processing circuit to control the processing circuit to selectively activate a VRS function of the processing circuit, for rendering at the at least one shading rate corresponding to the at least one predetermined rendering type (S13), characterized in that the utilizing the shading rate controller within the processing circuit to control the processing circuit to selectively activate the VRS function of the processing circuit (S13) comprises: utilizing the rendering classifier to dynamically decide whether to activate the VRS function on the at least one predetermined rendering type according to graphic knowledge, an experience rule and/or a hint.
  2. The method of claim 1, wherein the graphic knowledge comprises a strategy or policy which is used for performing selection or filtering on a rendering property to identify a VRS targeting rendering to which the VRS function should be applied; and/or the experience rule comprises avoiding using the VRS function when one or more rendering property conditions are met; and/or the hint comprises controlling information or parameter(s) on the VRS activation, wherein the hint is preferably provided by an interface between an application, a system and/or any other framework.
  3. The method of claim 1 or 2, wherein utilizing the shading rate controller within the processing circuit to control the processing circuit to selectively activate the VRS function of the processing circuit for rendering at the at least one shading rate corresponding to the at least one predetermined rendering type (S13) further comprises: utilizing the shading rate controller to perform at least one operation among a command insertion operation and a command modifying operation on the at least one set of original graphic commands, in order to convert the at least one set of original graphic commands into at least one set of new graphic commands, for controlling the processing circuit to perform rendering corresponding to the at least one set of new graphic commands at the at least one shading rate, rather than performing the rendering corresponding to the at least one set of original graphic commands.
  4. The method of claim 3, wherein the at least one set of original graphic commands comprise a first set of original graphic commands and a second set of original graphic commands; the rendering classifier is arranged to intercept the first set of original graphic commands on the command path to obtain a first rendering property from the first set of original graphic commands, for classifying rendering corresponding to the first set of original graphic commands, and to intercept the second set of original graphic commands on the command path to obtain a second rendering property from the second set of original graphic commands, for classifying rendering corresponding to the second set of original graphic commands; the rendering classifier is arranged to classify the rendering corresponding to the first set of original graphic commands into a first predetermined rendering type among the multiple predetermined rendering types according to the first rendering property, in order to determine a first shading rate corresponding to the first predetermined rendering type for the rendering corresponding to the first set of original graphic commands; and the rendering classifier is arranged to classify the rendering corresponding to the second set of original graphic commands into a second predetermined rendering type among the multiple predetermined rendering types according to the second rendering property, in order to determine a second shading rate corresponding to the second predetermined rendering type for the rendering corresponding to the second set of original graphic commands.
  5. The method of claim 4, wherein the shading rate controller is arranged to perform at least one first operation among the command insertion operation and the command modifying operation on the first set of original graphic commands, in order to convert the first set of original graphic commands into a first set of new graphic commands among the at least one set of new graphic commands, for controlling the processing circuit to perform rendering corresponding to the first set of new graphic commands at the first shading rate, rather than performing the rendering corresponding to the first set of original graphic commands; and the shading rate controller is arranged to perform at least one second operation among the command insertion operation and the command modifying operation on the second set of original graphic commands, in order to convert the second set of original graphic commands into a second set of new graphic commands among the at least one set of new graphic commands, for controlling the processing circuit to perform rendering corresponding to the second set of new graphic commands at the second shading rate, rather than performing the rendering corresponding to the second set of original graphic commands.
  6. The method of any one of claims 1 to 3, wherein the at least one set of original graphic commands comprise a first set of original graphic commands; the rendering classifier is arranged to intercept the first set of original graphic commands on the command path to obtain a first rendering property from the first set of original graphic commands, for classifying rendering corresponding to the first set of original graphic commands; and the rendering classifier is arranged to classify the rendering corresponding to the first set of original graphic commands into a first predetermined rendering type among the multiple predetermined rendering types according to the first rendering property, in order to determine a first shading rate corresponding to the first predetermined rendering type for the rendering corresponding to the first set of original graphic commands.
  7. The method of claim 6, wherein the shading rate controller is arranged to control the processing circuit to activate the VRS function of the processing circuit, for rendering at the first shading rate corresponding to the first predetermined rendering type.
  8. The method of claim 6 or 7, wherein the at least one set of original graphic commands further comprise a second set of original graphic commands; the rendering classifier is arranged to intercept the second set of original graphic commands on the command path to obtain a second rendering property from the second set of original graphic commands, for classifying rendering corresponding to the second set of original graphic commands; and the rendering classifier is arranged to classify the rendering corresponding to the second set of original graphic commands into a second predetermined rendering type among the multiple predetermined rendering types according to the second rendering property, in order to determine a second shading rate corresponding to the second predetermined rendering type for the rendering corresponding to the second set of original graphic commands.
  9. The method of claim 8, wherein the shading rate controller is arranged to control the processing circuit to activate the VRS function of the processing circuit, for rendering at the second shading rate corresponding to the second predetermined rendering type.
  10. The method of any one of claims 6 to 9, wherein the at least one set of original graphic commands further comprise another set of original graphic commands; the rendering classifier is arranged to intercept the other set of original graphic commands on the command path to obtain another rendering property from the other set of original graphic commands, for classifying rendering corresponding to the other set of original graphic commands; and the rendering classifier is arranged to classify the rendering corresponding to the other set of original graphic commands into another predetermined rendering type among the multiple predetermined rendering types according to the other rendering property, in order to determine another shading rate corresponding to the other predetermined rendering type for the rendering corresponding to the other set of original graphic commands, wherein the other shading rate is equal to a default shading rate of the processing circuit; wherein preferably the shading rate controller is arranged to control the processing circuit to deactivate the VRS function of the processing circuit, for rendering at the other shading rate corresponding to the other predetermined rendering type.
  11. The method of any one of claims 1 to 10, wherein the command path starts from an upper layer among multiple layers of a graphic control architecture within the processing circuit and reaches a lower layer among the multiple layers, wherein preferably the at least one set of original graphic commands are sent from the upper layer among the multiple layers toward the lower layer among the multiple layers, and are intercepted by the rendering classifier, for being converted into at least one set of new graphic commands by the shading rate controller; and/or wherein preferably the multiple layers of the graphic control architecture comprise at least one software layer, at least one firmware layer and at least one hardware layer; and the rendering classifier and the shading rate controller are implemented in either a same layer or different layers among the multiple layers.
  12. The method of any one of claims 1 to 11, wherein any rendering property among the at least one rendering property comprises one or a combination of at least one pipeline state, at least one buffer setting, at least one texture setting, at least one shader code and at least one enabled extension.
  13. The method of any one of claims 1 to 12, wherein the multiple predetermined rendering types comprise a first predetermined rendering type regarding rendering at least one normal three-dimensional, in the following also referred to as 3D, object, a second predetermined rendering type regarding rendering at least one vegetation object, and at least one other predetermined rendering type regarding at least one predetermined image processing operation; wherein preferably the at least one predetermined image processing operation comprises any operation among operations of post-processing, rendering visual effects, in the following also referred to as VFX, rendering at least one user interface, in the following also referred to as UI, rendering based on a depth image and rendering at least one protected object.
  14. The method of any one of claims 1 to 13, wherein the rendering corresponding to the at least one set of original graphic commands comprises any rendering operation among multiple rendering operations respectively corresponding to multiple rendering control levels; wherein preferably the multiple rendering control levels comprise a drawcall level, a renderpass level and a thread-process level.
  15. An apparatus that is adapted to operate according to the method of any one of claims 1 to 14, wherein the apparatus comprises at least the processing circuit (103) within the electronic device (100).

Description

Cross Reference to Related Applications This application claims the benefit of U.S. Provisional Application No. 63/503,175, filed on May 19th, 2023. Field The present invention is related to rendering processing, and more particularly, to a method for performing automatic activation control regarding variable rate shading (VRS), and an associated apparatus. Background According to the related art, a mobile device such as a multifunctional mobile phone may be arranged to visit an application (App) store for obtaining a variety of games therefrom, and the games may require a large amount of mobile system resources, and therefore it may become challenging to fulfill better user experience. For example, high resolution and high frame-rate games may increase the requirement of computing powers beyond the capabilities of the mobile platform and consume more battery power. One or more suggestions may be proposed to try solving the problems, but further problems such as some side effects may be introduced. More particularly, it is suggested that the mobile device such as the multifunctional mobile phone may support VRS hardware in order to provide a VRS function. As activating the VRS function during rendering can gain benefit on power reduction and bandwidth saving, it is one choice to increase efficiency of graphics processing unit (GPU) computing power. However, using the VRS function may require code change for adding corresponding graphic application programming interface (API) calling on an application by a developer. In some type of rendering, using the VRS function may even cause rendering to go wrong, for example, because the pixel result of any pixel is different. As a result, the application developer may have to be familiar with various VRS use cases to balance between the benefit, the correctness and the quality, or spending lots of time on trial and error may be needed, causing the associated costs such as labor costs to be increased. Thus, a novel method and associated architecture are needed for solving the problems without introducing any side effect or in a way that is less likely to introduce a side effect. CN 113 837 920 A and CN 114 972 001 A disclose image rendering methods and corresponding devices, which use variable rate shading (VRS). GB 2 600 203 A discloses graphic processing using VRS. Summary It is an objective of the present invention to provide a method for performing automatic activation control regarding VRS, and an associated apparatus, in order to solve the above-mentioned problems. A method and an apparatus according to the invention are defined in the independent claims. The dependent claims define preferred embodiments thereof. At least one embodiment of the present invention provides a method according to claim 1. At least one embodiment of the present invention provides an apparatus according to claim 15. It is an advantage of the present invention that, the present invention method, as well as the associated apparatus such as the processing circuit and the electronic device, can automatically activate VRS without introducing any related art problem. More particularly, the method of the present invention, such as the low overhead auto-activated (or autoactivated) VRS method, can reduce the code change effort of any application developer using the VRS, and can also help original equipment manufacturer (OEM) or original design manufacturer (ODM) parties (e.g., OEM/ODM customers of an integrated circuit (IC) design company) landing the VRS function faster. In addition, the present invention method and apparatus can solve the related art problems without introducing any side effect or in a way that is less likely to introduce a side effect. These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings. Brief Description of the Drawings FIG. 1 is a diagram illustrating an automatic VRS control architecture within an electronic device according to an embodiment of the present invention.FIG. 2 illustrates a low overhead autoactivated VRS control scheme of a method for performing automatic activation control regarding VRS according to an embodiment of the present invention.FIG. 3 illustrates various ways regarding implementation of the low overhead autoactivated VRS control scheme shown in FIG. 2 according to some embodiments of the present invention.FIG. 4 illustrates a classification and shading rate decision control scheme of the method according to an embodiment of the present invention.FIG. 5 illustrates a rendering control scheme of the method according to an embodiment of the present invention.FIG. 6 illustrates respective examples of multiple images involved with the rendering control scheme shown in FIG. 5 according to an embodiment of the present invention.FIG. 7 illustrates a shading r