JP-7857416-B2 - Image processing apparatus, method, and program

JP7857416B2JP 7857416 B2JP7857416 B2JP 7857416B2JP-7857416-B2

Inventors

田中康一
塚越一真
沖山和也
大鶴基格

Assignees

富士フイルム株式会社

Dates

Publication Date: 20260512
Application Date: 20230915
Priority Date: 20220930

Claims (20)

Equipped with a processor, The aforementioned processor, Determine the virtual environment to be provided. The virtual space is constructed by reproducing the real space in the determined environment. By acquiring positional information within a virtual space that replicates the aforementioned real space, Based on the first image captured in the real space and associated with the location information, an object image representing the first image is displayed in the virtual space. The virtual space is changed to correspond to the image information of the first image. The system is configured to display a second image of the virtual space observed from a viewpoint corresponding to the aforementioned positional information on the display unit. Image processing device.
The processor determines the environment of the virtual space to be provided based on the current date and time information. The image processing apparatus according to claim 1.
The processor accepts the selection of the virtual space environment to be provided and determines the virtual space environment to be provided. The image processing apparatus according to claim 1.
The processor accepts a selection of at least one of season, weather, and time of day to determine the environment of the virtual space to be provided. The image processing apparatus according to claim 3.
The first image is an image taken within a range of a first distance from the position in the real space corresponding to the position information in the virtual space. The image processing apparatus according to claim 1.
The first image is further an image taken within the range in the real space that corresponds to the range of the second image in the virtual space. The image processing apparatus according to claim 5 .
The processor is configured to display the object image at a position in the virtual space corresponding to the position in the real space where the first image was captured. The image processing apparatus according to claim 1.
The processor is configured to change the virtual space corresponding to the image information of the first image when the position indicated by the position information moves to a position within a second distance range from the display position of the object image. The image processing apparatus according to any one of claims 1 to 7 .
The processor is configured to display the object image in the orientation from which the first image was captured. The image processing apparatus according to any one of claims 1 to 7 .
The processor is configured to display the object image with a size corresponding to the field of view of the first image. The image processing apparatus according to any one of claims 1 to 7 .
The aforementioned processor, If the field of view of the first image exceeds a threshold, the object image is displayed with a size corresponding to the field of view. If the field of view of the first image is less than or equal to the threshold, the system is configured to display the object image at a predetermined size. The image processing apparatus according to claim 10 .
The aforementioned processor, Based on the image information of the first image, the shooting environment of the first image is determined. It is configured to change to the virtual space corresponding to the determined shooting environment. The image processing apparatus according to any one of claims 1 to 7 .
The processor is configured to determine at least one of the season, weather, and time of day as the shooting environment. The image processing apparatus according to claim 12 .
The processor is configured to change the virtual space data to construct a different virtual space. The image processing apparatus according to any one of claims 1 to 7 .
The processor is configured to apply effects to modify the virtual space. The image processing apparatus according to any one of claims 1 to 7 .
The aforementioned processor, If multiple first images exist, the object images are displayed multiple times within the virtual space. The selection of the aforementioned object image is accepted. The selected object image is configured to change the virtual space to one that corresponds to the image information of the first image represented by the selected object image. The image processing apparatus according to any one of claims 1 to 7 .
The processor is configured to accept the selection of an object image, with the object image located in the line of sight being the selected object image. The image processing apparatus according to claim 16 .
The first image is an image taken within a range of a first distance from the position in the real space corresponding to the position information in the virtual space. The image processing apparatus according to claim 16 .
The first image is further an image taken within the range in the real space that corresponds to the range of the second image in the virtual space. The image processing apparatus according to claim 18 .
The processor is configured to display the selected object image and other object images in different display modes. The image processing apparatus according to claim 16 .

Description

The present invention relates to an image processing apparatus, method, and program, and more particularly to an image processing apparatus, method, and program that provides a virtual space. Patent documents 1 to 4 describe a technology for viewing images taken in real space within a virtual space. Japanese Patent Publication No. 2009-296248Japanese Patent Publication No. 2011-186565Japanese Patent Publication No. 2012-252716Japanese Patent Publication No. 2022-68642 One embodiment of the technology described herein provides an image processing apparatus, method, and program capable of providing a good image viewing space. Diagram showing an overview of the image viewing system.A diagram showing an example of the system configuration of an image viewing system.Block diagram showing an example of the display terminal configuration.Block diagram of the main functions of the control unit of the display terminal.A diagram showing an example of the hardware configuration of an image processing device.Block diagram of the main functions of an image processing device.Conceptual diagram of image searchA diagram showing an example of virtual space data stored in a virtual space database.This diagram shows an example of an image of a virtual space displayed on the display terminal.A flowchart illustrating the procedure for providing a virtual space using an image processing device.A flowchart illustrating the procedure for providing a virtual space using an image processing device.Conceptual diagram of image displayA flowchart showing the steps for processing and displaying captured images.Conceptual diagram of image displayA flowchart showing the steps for processing and displaying captured images.This diagram shows another example of how captured images are displayed.This diagram shows another example of how captured images are displayed.Block diagram of the main functions of the image processing device regarding changes to the virtual space.Conceptual diagram for determining the quality of images taken during viewing.Flowchart of the procedure for processing changes to the virtual spaceA flowchart showing the procedure for determining the shooting environment.Conceptual diagram for determining the quality of images taken during viewing.A diagram illustrating an example of a user's image selection operation.This diagram shows another example of displaying captured images in a virtual space.This diagram shows another example of how captured images are displayed in a virtual space. Preferred embodiments of the present invention will be described in detail below with reference to the attached drawings. [First Embodiment] In recent years, with the widespread adoption of social networking services (SNS), images taken at tourist destinations are being shared in near real-time. Therefore, even without actually visiting a tourist spot, it's now possible to indirectly get a sense of what the area is like by viewing these images. On the other hand, in recent years, advancements in technologies such as VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality) have made digital twins—recreating real space (actual space, existing space) in a virtual space—a reality that is becoming increasingly common. This disclosure provides a new image viewing system that utilizes digital twins. [overview] Figure 1 is a diagram illustrating the overview of the image viewing system disclosed herein. For example, a virtual space that replicates a real-world location, such as a tourist spot, is provided to the user via a head-mounted display (HMD). The user can move freely within the virtual space, just as they would in the real world. If an image taken in the real world exists, when the user approaches the virtual space location corresponding to the real-world location where the image was taken, that image will be displayed within the virtual space. [Image Viewing System] Figure 2 shows an example of the system configuration of an image viewing system. As shown in the figure, the image viewing system 1 of this embodiment includes a display terminal 10 that displays images in a virtual space, an image processing device 100 that provides images in the virtual space to the display terminal 10, and an image transmission terminal 200 that transmits images taken in real space to the image processing device 100. The display terminal 10 and the image processing device 100 are connected to each other via a network 2 so as to be able to communicate. The image transmission terminal 200 and the image processing device 100 are also connected to each other via a network 2 so as to be able to communicate. [Image transmission terminal] The image transmission terminal 200 consists of, for example, a computer with communication capabilities (e.g., a personal computer), a mobile terminal (e.g., a smartphone, mobile phone, tablet device), or a photographic device with communication capabilities (e.g., a digital camera). Since these configurations are