US-12626451-B2 - Multi-view image generation apparatus and method, and graphics processing unit

Abstract

The present disclosure relates to a multi-view image generation apparatus and method, and a graphics processing unit. Waves are constructed according to primitive information and vertex information of geometric primitives to generate a plurality of waves for parallel processing, and the waves are scheduled by a wave management unit for execution, which can improve execution efficiency. Since the primitive information of the geometric primitives is stored in a first cache unit, data sharing between waves can be implemented. During the parallel processing of the waves, vertex information in a second cache unit is updated in real time according to vertex address information outputted by the waves, so that a raster processing unit can read the latest vertex information from the second cache unit. Finally, through an image generation unit, a multi-view image is generated according to image colors obtained by executions of the waves and views corresponding to pixel points.

Inventors

• Huaisheng ZHANG

Assignees

• GLENFLY TECH CO., LTD.

Dates

Publication Date

20260512

Application Date

20240530

Priority Date

20240204

Claims (19)

1. 1 . A multi-view image generation apparatus, comprising: a processing unit, a wave construction unit, a wave management unit, an execution unit (EU), a first cache unit, a second cache unit, a raster processing unit, and an image generation unit; wherein the processing unit is configured to assemble drawing data streams to obtain geometric primitives, the geometric primitives corresponding to at least one view; the first cache unit is configured to store primitive information of the geometric primitives, the primitive information comprising the number of primitives, the number of vertices, the number of primitive bunches, and the number of waves; the second cache unit is configured to store vertex information of the geometric primitives, the vertex information comprising a vertex position, a vertex attribute, and a vertex normal vector; the wave construction unit is configured to construct waves according to the primitive information and the vertex information; the EU comprises at least one computing unit; the wave management unit is configured to schedule the waves into the computing unit for execution to perform image processing on the geometric primitives, and update the vertex information in the second cache unit according to vertex address information obtained by the image processing; the raster processing unit is configured to rasterize the updated vertex information to obtain a corresponding two-dimensional image, and determine a view corresponding to each pixel point in the two-dimensional image; and the image generation unit is configured to generate a multi-view image according to image colors obtained by the executions of the waves and the view corresponding to each pixel point.
2. 2 . The apparatus according to claim 1 , wherein data is stored in the first cache unit through a two-dimensional table, rows in the two-dimensional table in the first cache unit are configured to represent draw batches of geometric primitives, a first column is configured to represent whether valid data exists in corresponding rows, second column to the fifth column are sequentially configured to represent the number of primitives, the number of vertices, the number of primitive bunches, and the number of waves included in the geometric primitives corresponding to a current draw batch, the sixth column to the last column are sequentially configured to represent output positions of the vertices, and the number of waves is not greater than a product of the number of primitive bunches and the number of views.
3. 3 . The apparatus according to claim 2 , wherein initial positions of all vertices corresponding to each draw batch are identical, the second cache unit is configured to store data through a two-dimensional table, rows in the two-dimensional table in the second cache unit are sequentially configured to represent occupied spaces of vertices in primitive bunches corresponding to views according to the initial positions, and columns are configured to represent vertex information of vertices, the occupied spaces are calculated according to the vertex information of the corresponding vertices and the number of views of the draw batches corresponding to the vertices.
4. 4 . The apparatus according to claim 2 , wherein the wave management unit is further configured to request to add a new blank row to the two-dimensional table in the first cache unit, to allow the first cache unit to store the primitive information of the geometric primitives into the blank row.
5. 5 . The apparatus according to claim 1 , wherein the wave construction unit is configured to determine views corresponding to the vertices of the geometric primitives, split the vertices to obtain primitive bunches corresponding to the views, and construct waves according to the primitive bunches, wherein each wave is configured to perform the image processing on one primitive bunch.
6. 6 . The apparatus according to claim 5 , wherein the wave construction unit is configured to: expand the primitive bunches to correspond to the views when the number of primitive bunches reaches a preset number or the splitting of the vertices of the geometric primitives is completed, construct a wave for performing the image processing on each primitive bunch, mark the wave with the same view as the corresponding primitive bunch, and mark all waves corresponding to primitive bunches corresponding to each view with the same view name.
7. 7 . The apparatus according to claim 1 , wherein the waves comprise a front-end wave and a back-end wave, the front-end wave is configured to perform front-end shading (FS) on the geometric primitives, the back-end wave is configured to perform back-end shading (BS) on the geometric primitives, the FS comprises vertex shading, hull shading, and domain shading, and the BS comprises pixel shading.
8. 8 . The apparatus according to claim 7 , wherein the image generation unit is configured to generate the multi-view image according to image colors of pixel points obtained by executions of the front-end wave and the back-end wave and views corresponding to the pixel points.
9. 9 . The apparatus according to claim 1 , wherein the wave management unit is configured to: schedule the waves to the computing unit for execution; perform the image processing on the vertices of the geometric primitives; after all vertices are processed, for each vertex corresponding to each view, calculate an initial position of the vertex in a current view; and generate vertex address information according to initial positions and vertex information of all vertices.
10. 10 . A graphics processing unit (GPU), comprising the multi-view image generation apparatus according to claim 1 .
11. 11 . A multi-view image generation method, comprising: assembling drawing data streams to obtain geometric primitives, the geometric primitives corresponding to at least one view, primitive information of the geometric primitives comprising the number of primitives, the number of vertices, the number of primitive bunches, and the number of waves, vertex information of the geometric primitives comprising vertex positions, vertex attributes, and vertex normal; constructing waves according to the primitive information and the vertex information; executing the waves to perform image processing on the geometric primitives, and updating the vertex information according to vertex address information obtained by the image processing; rasterizing the updated vertex information to obtain a corresponding two-dimensional image, and determining views corresponding to pixel points in the two-dimensional image; and generating a multi-view image according to image colors obtained by executions of the waves and the views corresponding to the pixel points.
12. 12 . The method according to claim 11 , wherein the primitive information of the geometric primitives is stored in a two-dimensional table, rows in the two-dimensional table are configured to represent draw batches of the geometric primitives, the first column in the two-dimensional table is configured to represent whether valid data exists in the corresponding rows, the second column to the fifth column in the two-dimensional table are sequentially configured to represent the number of primitives, the number of vertices, the number of primitive bunches, and the number of waves included in geometric primitives corresponding to a current draw batch, the sixth column to the last column in the two-dimensional table are sequentially configured to represent output positions of the vertices, and the number of waves is not greater than a product of the number of primitive bunches and the number of views.
13. 13 . The method according to claim 12 , wherein initial positions of all vertices corresponding to each draw batch are identical, the vertex information of the geometric primitives is stored in the two-dimensional table, rows in the two-dimensional table are sequentially configured to represent occupied spaces of vertices in primitive bunches corresponding to the views according to the initial positions, and columns in the two-dimensional table are configured to represent vertex information of the vertices, the occupied spaces are calculated according to the vertex information of the corresponding vertices and the number of views in the draw batches corresponding to the vertices.
14. 14 . The method according to claim 12 , further comprising: requesting to add a new blank row to the two-dimensional table, to store the primitive information of the geometric primitives into the blank row.
15. 15 . The method according to claim 11 , wherein the constructing the waves according to the primitive information and the vertex information comprises: determining views corresponding to the vertices of the geometric primitives, splitting the vertices to obtain primitive bunches corresponding to the views, and constructing the waves according to the primitive bunches, wherein each wave is configured to perform the image processing on one primitive bunch.
16. 16 . The method according to claim 15 , wherein the constructing waves according to the primitive bunches comprises: expanding the primitive bunches to correspond to the views when the number of primitive bunches reaches a preset number or the splitting of the vertices of the geometric primitives is completed, constructing a wave for the image processing on each primitive bunch, marking the wave with the same view as the corresponding primitive bunch, and marking all waves corresponding to the primitive bunches corresponding to each view with the same view name.
17. 17 . The method according claim 11 , wherein the waves comprise a front-end wave and a back-end wave, the front-end wave is configured to perform front-end shading (FS) on the geometric primitives, the back-end wave is configured to perform back-end shading (BS) on the geometric primitives, the FS comprises vertex shading, hull shading, and domain shading, and the BS comprises pixel shading.
18. 18 . The method according claim 17 , wherein the generating the multi-view image according to the image colors obtained by the executions of the waves and the views corresponding to the pixel points comprises: generating the multi-view image according to the image colors of the pixel points obtained by executions of the front-end wave and the back-end wave and the views corresponding to the pixel points.
19. 19 . The method according claim 11 , wherein the executing the waves to perform the image processing on the geometric primitives further comprises: executing the waves to perform the image processing on the vertices of the geometric primitives, after all vertices are processed, for each vertex corresponding to each view, calculating an initial position of the vertex in a current view, and generating vertex address information according to initial positions and vertex information of all vertices.

Description

CROSS REFERENCES TO RELATED APPLICATION The present application claims priority to Chinese patent application No. 202410161229.7, titled “Multi-View Image Generation Apparatus and Graphics Processing Unit”, and filed on Feb. 4, 2024, the content of which is hereby incorporated by reference in its entirety. TECHNICAL FIELD The present disclosure relates to the field of image drawing technologies, and particularly to a multi-view image generation apparatus and method, and a graphics processing unit (GPU). BACKGROUND In the virtual reality technology, there is generally a need to generate images from left and right viewpoints. For a certain scene, drawing tasks (including a front end (FE) and a back end (BE)) for generating left and right images are the same, except that respective view transformation matrices are different. In the related technologies, since the drawing and computation are performed for the left and right viewpoints respectively, outputted to two images, and then displayed. Accordingly, the calculation process is complicated, which may result in lower drawing efficiency. SUMMARY In view of this, in order to address the above technical problems, it is necessary to provide a multi-view image generation apparatus and method, and a GPU capable of improving the image drawing efficiency. In the first aspect of the present disclosure, a multi-view image generation apparatus is provided, which includes: a processing unit, a wave construction unit, a wave management unit, an execution unit (EU), a first cache unit, a second cache unit, a raster processing unit, and an image generation unit; the processing unit is configured to assemble drawing data streams to obtain geometric primitives, the geometric primitives corresponding to at least one view; the first cache unit is configured to store primitive information of the geometric primitives, the primitive information including the number of primitives, the number of vertices, the number of primitive bunches, and the number of waves; the second cache unit is configured to store vertex information of the geometric primitives, the vertex information includes a vertex position, a vertex attribute, and a vertex normal; the wave construction unit is configured to construct waves according to the primitive information and the vertex information; the EU includes at least one computing unit; the wave management unit is configured to schedule the waves into the computing unit for execution to perform image processing on the geometric primitives, and update the vertex information in the second cache unit according to vertex address information obtained by the image processing; the raster processing unit is configured to rasterize the updated vertex information to obtain a corresponding two-dimensional image, and determine a view corresponding to each pixel point in the two-dimensional image; and the image generation unit is configured to generate a multi-view image according to image colors obtained by the executions of the waves and the view corresponding to each pixel point. In an embodiment, data is stored in the first cache unit through a two-dimensional table, rows in the two-dimensional table in the first cache unit are configured to represent draw batches of geometric primitives, a first column is configured to represent whether valid data exists in corresponding rows, second column to the fifth column are sequentially configured to represent the number of primitives, the number of vertices, the number of primitive bunches, and the number of waves included in the geometric primitives corresponding to a current draw batch, the sixth column to the last column are sequentially configured to represent output positions of the vertices, and the number of waves is not greater than a product of the number of primitive bunches and the number of views. In an embodiment, initial positions of all vertices corresponding to each draw batch are identical, the second cache unit is configured to store data through a two-dimensional table, rows in the two-dimensional table in the second cache unit are sequentially configured to represent occupied spaces of vertices in primitive bunches corresponding to views according to the initial positions, and columns are configured to represent vertex information of vertices, the occupied spaces are calculated according to the vertex information of the corresponding vertices and the number of views of the draw batches corresponding to the vertices. In an embodiment, the wave construction unit is configured to determine views corresponding to the vertices of the geometric primitives, split the vertices to obtain primitive bunches corresponding to the views, and construct waves according to the primitive bunches, each wave is configured to perform the image processing on one primitive bunch. In an embodiment, the wave construction unit is configured to: expand the primitive bunches to correspond to the views when the number of primitive