JP-2022532238-A5 -

Dates

Publication Date

20230517

Application Date

20200515

Description

Additional and other purposes, features, and benefits of this disclosure are described in the detailed description, figures, and claims. The present invention provides, for example, the following: (Item 1) A device configured to be worn on the head by a user, A screen configured to display graphics for the user, A camera system configured to visually observe the environment in which the user is located, A processing unit coupled to the camera system, wherein the processing unit comprises The method involves obtaining the location of features related to image data associated with the aforementioned environment, wherein the location of the features is identified by a neural network. The process involves determining a region of interest relating to one of the features in the aforementioned image, wherein the region of interest has a size less than the size of the aforementioned image. An angle detection algorithm is used to perform angle detection and identify the angles within the area of interest. A processing unit configured to perform the following: A device equipped with the following features. (Item 2) The apparatus according to item 1, wherein the processing unit is configured to determine a region of interest having a position based on at least one of the locations identified by the neural network, the position being relative to the image. (Item 3) The apparatus according to item 1, wherein the image data is associated with at least one image, and the at least one image is generated by the camera system and transmitted to the neural network. (Item 4) The neural network is the device described in item 1, located within the module of the device. (Item 5) The apparatus according to item 1, wherein the neural network is implemented in one or more computing devices located remotely from the apparatus. (Item 6) The aforementioned neural network is the device described in item 1, which has machine learning capabilities. (Item 7) The apparatus according to item 1, wherein the processing unit is configured to obtain the location of the feature by acquiring a heatmap generated by the neural network, and the heatmap indicates the location of the feature. (Item 8) The apparatus according to item 1, wherein the region of interest comprises N × N patches, and the processing unit is configured to perform the angle detection on the N × N patches, where N is an integer greater than 1. (Item 9) The apparatus according to item 1, wherein the region of interest comprises a patch having 144 pixels or fewer, and the processing unit is configured to perform the angle detection on the patch. (Item 10) The apparatus according to item 1, wherein the region of interest comprises 8 x 8 patches, and the processing unit is configured to perform the angle detection on the 8 x 8 patches. (Item 11) The apparatus according to item 1, wherein the image data includes at least one low-resolution image obtained by reducing the resolution of at least one high-resolution image generated by the camera system. (Item 12) The apparatus according to item 11, wherein the processing unit is also configured to convert an image having the first resolution into another image having the second resolution. (Item 13) The apparatus according to item 11, wherein the first resolution is VGA resolution. (Item 14) The apparatus according to item 11, wherein the second resolution is a QVGA resolution. (Item 15) The apparatus described in item 1, further comprising the aforementioned neural network. (Item 16) The apparatus described in item 15, wherein the neural network is trained using a reference dataset. (Item 17) The aforementioned neural network is the apparatus described in item 15, comprising a convolutional neural network. (Item 18) The apparatus according to item 15, wherein the neural network is configured to calculate the location of the point of interest and its descriptor. (Item 19) The apparatus according to item 15, wherein the neural network comprises an encoder configured to spatially downsample an input image. (Item 20) The aforementioned neural network also, A focus decoder, wherein the focus decoder is configured to act on the encoder output from the encoder and produce a score for each pixel in the input image, A descriptor decoder, wherein the descriptor decoder is configured to act on the encoder output, upsample the encoder output to a higher resolution, and produce a vector for each pixel in the input image. The apparatus described in item 19, comprising: (Item 21) The apparatus according to item 15, wherein the neural network is configured to use homography fitting to improve the geometric consistency of the point of interest detector. (Item 22) The apparatus according to item 21, wherein the neural network comprises a convolutional neural network configured to train the focus point detector. (Item 23) The apparatus according to item 21, wherein the neural network is configured to perform image warping and create one or more warped im