Search

JP-2026075405-A - Image processing system, method for controlling the image processing system, and program

JP2026075405AJP 2026075405 AJP2026075405 AJP 2026075405AJP-2026075405-A

Abstract

[Problem] To provide a technology that reduces power consumption associated with image processing in an image processing system. [Solution] The image processing system includes an image generation means that generates virtual image data using alpha values and RGB values relating to a virtual object, a conversion means that converts the generated virtual image data into converted image data, and an image synthesis means that synthesizes the converted image data and real image data to generate composite image data. The conversion means applies a first RGB value to areas where the alpha value is smaller than a predetermined alpha value, and applies a second RGB value to areas where the RGB value is smaller than a predetermined RGB value among areas where the alpha value is greater than or equal to the predetermined alpha value, thereby converting to the converted image data. The image synthesis means generates the composite image data without performing color conversion processing on the areas to which the first RGB value is applied and the areas to which the second RGB value is applied. [Selection Diagram] Figure 1A

Inventors

  • 森 健作

Assignees

  • キヤノン株式会社

Dates

Publication Date
20260508
Application Date
20241022

Claims (8)

  1. Image generation means that generates virtual image data using an alpha value indicating transparency information about a virtual object and an RGB value related to the virtual object, A conversion means that converts the generated virtual image data into converted image data using the alpha value, Image synthesis means for generating composite image data by combining the converted image data and real image data, It has, The conversion means applies a first RGB value to the region in the virtual image data where the alpha value is smaller than a predetermined alpha value, and applies a second RGB value to the region in the virtual image data where the alpha value is greater than or equal to the predetermined alpha value, and where the RGB value of the virtual image data is smaller than a predetermined RGB value, thereby converting the virtual image data into the converted image data. The image synthesis means is characterized in that it generates the synthesized image data without performing a predetermined color conversion process on the region to which the first RGB value of the converted image data is applied and the region to which the second RGB value of the converted image data is applied.
  2. The image processing system according to claim 1, characterized in that the first RGB value is a value determined in steps according to the magnitude of the alpha value.
  3. The image processing system according to claim 1, characterized in that the conversion means also reflects the distance information of the virtual image data in the first RGB value.
  4. The image processing system according to claim 1, characterized in that the conversion means uses a value represented by the least significant bit of the converted image data as at least one of the first RGB value and the second RGB value.
  5. The system further comprises an image transmission means for transmitting the converted image data and an image receiving means for receiving the transmitted converted image data. The image processing system according to claim 1, characterized in that the converted image data is transmitted from the image transmission means to the image receiving means by wired communication.
  6. The system further comprises an image transmission means for transmitting the converted image data and an image receiving means for receiving the transmitted converted image data. The image processing system according to claim 1, characterized in that the converted image data is transmitted from the image transmission means to the image receiving means by wireless communication.
  7. A method for controlling an image processing system, Image generation step: Generates virtual image data using an alpha value indicating transparency information for a virtual object and an RGB value for the virtual object. A conversion step of converting the generated virtual image data into converted image data using the alpha value, Image synthesis step: Combining the converted image data and real image data to generate composite image data, It has, The conversion step involves applying a first RGB value to a region in the virtual image data where the alpha value is smaller than a predetermined alpha value, and applying a second RGB value to a region in the virtual image data where the alpha value is greater than or equal to the predetermined alpha value, and where the RGB value of the virtual image data is smaller than a predetermined RGB value, thereby converting the virtual image data into the converted image data. The control method for an image processing system is characterized in that the image synthesis step generates the synthesized image data without performing a predetermined color conversion process on the region to which the first RGB value of the converted image data is applied and the region to which the second RGB value of the converted image data is applied.
  8. A program for causing a computer to function as one of the means of the image processing system described in any one of claims 1 to 6.

Description

This invention relates to an image processing system, a control method for an image processing system, and a program. In recent years, Mixed Reality (MR) technology has become known as a technology that seamlessly merges real and virtual spaces in real time. One method used to realize MR technology is to combine a head-mounted display (HMD) with a personal computer (PC). In this method, the position and orientation of the HMD are estimated, and a computer graphics (CG) image—a virtual image of the virtual space as seen from the estimated position and orientation—is generated by rendering on a PC. The generated CG image data is transmitted to the HMD frame by frame. The HMD combines the received CG image data with real-world image data captured by a camera mounted on the HMD to generate composite image data. The HMD then displays this composite image data on a display panel to provide the user with a mixed reality (MR) image. Furthermore, to achieve miniaturization and cost advantages, it is desirable to implement HMDs using existing technologies and components as much as possible. Specifically, it is desirable that the output interface of the PC generating CG image data uses components that implement the general-purpose DP (DisplayPort) standard. The DP standard generally transmits and receives image data in RGB format, and outputs image data in RGB format from the PC. In the RGB format, one pixel is represented in the red, green, and blue color space (8 bits of bit depth per color, 24 bits total). Therefore, since the bit depth of each pixel in the RGB format output by the PC is fixed at 8 bits x 3 = 24 bits, there is a concern that the PC may not be able to transmit alpha value information to the HMD. Japanese Patent Publication No. 2005-107780 Block diagram showing the configuration of the image processing system according to the first embodiment.Block diagram showing the configuration of the alpha value setting unit according to the first embodiment.Flowchart of image processing performed by the image processing system according to the first embodimentA schematic diagram showing an example of image processing according to the first embodiment.Flowchart of image processing performed by the image processing system according to the first embodimentA schematic diagram showing an example of image processing according to the second embodiment. The embodiments for carrying out the present invention will be described in detail below with reference to the accompanying drawings. The embodiments described below are merely examples of means for realizing the present invention, and may be modified or changed as appropriate depending on the configuration of the apparatus to which the present invention is applied and various conditions. Furthermore, it is possible to combine the embodiments as appropriate. (First Embodiment) A CG image is a virtual image that includes virtual CG objects and a background area. Since the background area of a virtual image does not contain information about the CG objects, it is preferable that the background area of the virtual image is not drawn when generating an MR image by combining a virtual image with a real image. By excluding the background area of the virtual image from the combination target when combining a real image acquired by a camera with a virtual CG object, it becomes possible to generate a more realistic MR image. In the image processing system according to this embodiment, when generating CG image data, information called an alpha value is added to the virtual image data in order to distinguish between the virtual CG object and the background area. Different alpha values are set for the virtual CG object and the background area, and an alpha value is added to every pixel in the virtual image. The alpha value is a value that indicates transparency information related to the virtual object. By using the alpha value, it is possible to distinguish between the background area and the virtual CG object, and to generate MR image data with the background area of the virtual image data removed. The image processing system according to the first embodiment of the present invention will be described below with reference to the figures. As shown in Figure 1A, the image processing system 1 according to this embodiment includes an information processing device (such as a PC) 100 and an HMD 110. The information processing device 100 and the HMD 110 are connected to each other. The HMD 110 may be configured to have some or all of the functions of the information processing device 100 shown in Figure 1A. The information processing device 100 includes a position and orientation calculation unit 101, a CG image generation unit 102, an alpha value setting unit 103, and an image transmission unit 104. The HMD 110 includes a sensor unit 111, an image processing unit 112, an imaging unit 113, a synthesis unit 114, a display unit 115, an alpha value restoration unit 116, and an image receiv