JP-2024160556-A5 -

Dates

Publication Date

20260511

Application Date

20230501

Description

In response to this, the image correction unit 413 performs image correction processing on the CG image 602c so that it temporally matches the captured image 601c in accordance with the movement of the HMD 101, as described above. That is, as shown in Figure 6(c), the image correction processing by the image correction unit 413 corrects the CG image 602c to match the changes in the captured image 601c caused by the head movements of the HMD user, resulting in a corrected CG image 603c. As a result, even if the captured image 601c and the CG image 602c are temporally mismatched, the composite image 604c , when the corrected CG image 603c is combined, will be an image that is less unnatural to the HMD user. When the process proceeds to S305, the plane correction unit 222 uses the homography matrix calculated in S303 to perform image correction (image transformation) on the color channels of the CG image. At this time, the color channels of the CG image need to be planarly aligned with the captured image that follows the movement of the HMD 101. For this reason, the plane correction unit 222 calculates coordinate information with decimal precision from the homography matrix as transformed coordinates, and uses these transformed coordinates as reference pixel positions to perform image correction by pixel interpolation such as bilinear interpolation. In other words, the image correction by pixel interpolation in the plane correction unit 222 refers to the color information at the reference pixel position, and uses the smoothed value of the color information corresponding to that reference pixel position as the correction value for pixel interpolation. In the data 723c shown in Figure 7(c) above, interpolation processing by smoothing changes the value A of pixel Q701 to 50% and the value Z to 3. In particular, the change in value Z from its original value reverses the front-to-back relationship between the captured image and the CG image at that pixel. In contrast, in the configuration of Figure 2 , the color channel and the composite information channel (alpha channel, depth channel) are separated, and interpolation processing is not performed on the composite information channel. Therefore, the values A and Z of pixels R800 and R801 in data 823c of Figure 8(c), which correspond to pixels Q700 and Q701 in data 723c of Figure 7(c), are retained at their original values. In particular, for pixel R801, image correction is performed that retains information with value A at 100% and value Z at 6. As shown in Figure 9, the image processing apparatus 903 according to the second embodiment includes, in addition to the functional units of the image processing apparatus 103 shown in Figure 2, a separation unit 911 and a correction unit 920 as image correction units. The separation unit 911 separates the image after development processing by the imaging processing unit 211 into color channels and depth channels. The color channels separated by the separation unit 911 are input to the planar correction unit 912 of the correction unit 920, and the depth channels are input to the spatial correction unit 913 of the correction unit 920. Next, in S1004, the correction unit 920 branches the processing based on whether it is a color channel or a depth channel. Specifically, if it is a color channel, the correction unit 920 proceeds to S1005, and if it is a depth channel , it proceeds to S1006. This embodiment includes the following configurations, methods, and programs. (Composition 1) An acquisition means that acquires an image captured by an imaging device that captures the real world, An operation acquisition means for acquiring motion information of the imaging device within the space of the real world, A CG generation means generates a computer graphics image that includes composite information related to the synthesis with the captured image, based on the motion information of the imaging device. A CG correction means that separates a computer graphics image containing the composite information into image channels and composite information channels, performs correction on the image channels by pixel interpolation according to the motion information of the imaging device, and performs correction on the composite information channels by pixel replacement according to the motion information of the imaging device, A synthesis means for combining the captured image and the computer graphics image corrected by the CG correction means, An image processing apparatus characterized by having (Structure 2) The acquisition means also acquires depth information of the real world corresponding to the captured image, The image processing apparatus according to configuration 1, characterized in that the composite information includes at least one of depth information and transparency information. (Composition 3) The image processing apparatus according to configuration 1 or 2, characterized in that the CG correction means calculates a homo