EP-4742652-A1 - ELECTRONIC DEVICE AND DISPLAY METHOD THEREFOR, PROCESSING DEVICE, AND COMPUTER-READABLE STORAGE MEDIUM

EP4742652A1EP 4742652 A1EP4742652 A1EP 4742652A1EP-4742652-A1

Abstract

An electronic device and a display method therefor, a processing device, and a computer-readable storage medium. The electronic device comprises an image collection apparatus. The method comprises: detecting whether an electronic device enables a see-through function; and when the electronic device enables the see-through function, controlling the collection frame rate of an image collection apparatus to be a second frequency, wherein the second frequency is an integer multiple of a first frequency, and the first frequency is the light-emission frequency of ambient light.

Inventors

YAO, Chaoquan
CHU, Xiao
SU, JING
CHEN, Shaobei
ZHAO, JUNJIE
ZHANG, HONGLEI
LV, Yaoyu
Li, Zhifu
SHI, Juanjuan
FAN, Qingwen
QIN, RUIFENG
MIAO, JINGHUA

Assignees

Boe Technology Group Co., Ltd.

Dates

Publication Date: 20260513
Application Date: 20241224

Claims (18)

A display method of an electronic device, wherein the electronic device comprises an image acquisition apparatus, and the method comprises: detecting whether the electronic device enables a see-through function; and controlling an acquisition frame rate of the image acquisition apparatus to be a second frequency when the electronic device enables the see-through function, wherein the second frequency is an integer multiple of a first frequency, and the first frequency is a light emitting frequency of ambient light.
The display method according to claim 1, wherein before the controlling the acquisition frame rate of the image acquisition apparatus to be the second frequency, the method further comprises: detecting a light emitting frequency of ambient light; and triggering the act of controlling the acquisition frame rate of the image acquisition apparatus to be the second frequency when the light emitting frequency of the ambient light is the first frequency.
The display method according to claim 1 or 2, wherein the detecting whether the electronic device enables the see-through function comprises any one or more of the following: detecting whether an instruction for enabling the see-through function is received, detecting whether a user reaches a safety boundary, and detecting whether a distance between an external object and the user is less than or equal to a preset distance threshold.
The display method according to claim 1 or 2, further comprising: acquiring a screen display frame rate of the electronic device; detecting whether a ratio of the screen display frame rate to the acquisition frame rate of the image acquisition apparatus is within a preset ratio range; and when the ratio of the screen display frame rate to the acquisition frame rate of the image acquisition apparatus is not within the preset ratio range, adjusting the screen display frame rate, so that the ratio of the screen display frame rate to the acquisition frame rate of the image acquisition apparatus is within the preset ratio range.
The display method according to claim 1 or 2, further comprising: detecting whether the electronic device disables the see-through function; detecting whether an instruction for enabling a gesture recognition function is received when the electronic device disables the see-through function; and controlling the acquisition frame rate of the image acquisition apparatus to be a third frequency when the instruction for enabling the gesture recognition function is received, wherein the third frequency is greater than the second frequency.
The display method according to claim 1 or 2, further comprising: detecting whether the light emitting frequency of the ambient light changes; when the light emitting frequency of the ambient light changes, controlling the acquisition frame rate of the image acquisition apparatus to be a fourth frequency, wherein the fourth frequency is greater than the second frequency, and detecting whether a ratio of a screen display frame rate of the electronic device to the acquisition frame rate of the image acquisition apparatus is within a preset ratio range; and when the ratio of the screen display frame rate to the acquisition frame rate of the image acquisition apparatus is not within the preset ratio range, adjusting the screen display frame rate, so that the ratio of the screen display frame rate to the acquisition frame rate of the image acquisition apparatus is within the preset ratio range.
The display method according to claim 1 or 2, further comprising: when the electronic device enables the see-through function and the light emitting frequency of the ambient light is not the first frequency, controlling the acquisition frame rate of the image acquisition apparatus to be a fourth frequency, wherein the fourth frequency is greater than the second frequency.
The display method according to claim 1 or 2, further comprising: when the electronic device has enabled the see-through function, detecting whether a head movement speed is greater than or equal to a preset movement threshold; when the head movement speed is greater than or equal to the preset movement threshold, controlling the acquisition frame rate of the image acquisition apparatus to be a fifth frequency, wherein the fifth frequency is greater than the second frequency; and when the head movement speed is less than the preset movement threshold, triggering the act of detecting the light emitting frequency of the ambient light.
The display method according to claim 1 or 2, further comprising: when the electronic device does not enable the see-through function, controlling the acquisition frame rate of the image acquisition apparatus to be a third frequency, wherein the third frequency is greater than the second frequency.
The display method according to claim 9, further comprising: when the electronic device does not enable the see-through function, acquiring a frame rate recommended by an application; when the frame rate recommended by the application is less than or equal to the third frequency, controlling the acquisition frame rate of the image acquisition apparatus to be the frame rate recommended by the application; and when the frame rate recommended by the application is greater than the third frequency, controlling the acquisition frame rate of the image acquisition apparatus to be a sixth frequency, wherein the sixth frequency is less than the frame rate recommended by the application.
The display method according to claim 10, further comprising: acquiring a maximum screen display frame rate of the electronic device, and detecting a magnitude of the maximum screen display frame rate and a magnitude of the frame rate recommended by the application; when the maximum screen display frame rate is greater than the frame rate recommended by the application, adjusting the acquisition frame rate of the image acquisition apparatus to be consistent with the frame rate recommended by the application; and when the maximum screen display frame rate is less than the frame rate recommended by the application, performing data statistics according to data amount acquired by the image acquisition apparatus and data amount to be displayed on a display screen of the electronic device, and determining the acquisition frame rate of the image acquisition apparatus and the screen display frame rate according to a statistical result.
The display method according to claim 9, wherein the second frequency is selected and generated by a user according to a first selectable frame rate of the electronic device, or the second frequency is generated by the electronic device based on a power supply power frequency in a region where the electronic device is located, wherein the first selectable frame rate comprises at least one second frequency recommended frame rate, and the second frequency recommended frame rate is consistent with the power supply power frequency in the region where the electronic device is located; and the third frequency is selected and generated by the user according to a second selectable frame rate of the electronic device, or the third frequency is generated by the electronic device based on a load condition of the electronic device.
The display method according to claim 1 or 2, wherein when the electronic device enables the see-through function and simultaneously displays an environment image and a virtual view, the electronic device starts a camera thread, a rendering thread, and an asynchronous time warp thread, and the method further comprises: at a first starting time point, acquiring, by the camera thread, image data acquired by the image acquisition apparatus, and sending the image data to the asynchronous time warp thread; at the first starting time point, rendering, by the rendering thread, first screen scene data based on first predicted user pose data; at a first intermediate time point, sending the rendered first screen scene data to the asynchronous time warp thread, and rendering second screen scene data based on second predicted user pose data; and at a first display time point, sending the rendered second screen scene data to the asynchronous time warp thread; and at the first intermediate time point, performing, by the asynchronous time warp thread, a fusion operation on the first screen scene data and the image data and performing correction based on third predicted user pose data to obtain first screen output data; at the first display time point, outputting the first screen output data to a display screen for display, performing a fusion operation on the second screen scene data and the image data and performing correction based on the third predicted user pose data to obtain second screen output data; and at a second display time point, outputting the second screen output data to the display screen for display, wherein the first intermediate time point is located between the first starting time point and the first display time point, and the first display time point is located between the first intermediate time point and the second display time point.
The display method according to claim 1 or 2, wherein when the electronic device enables the see-through function and only displays an environment image, the electronic device starts a rendering thread and an asynchronous time warp thread, and the method further comprises: at a second starting time point, acquiring, by the rendering thread, image data acquired by the image acquisition apparatus and sending the image data to the asynchronous time warp thread; and at a second intermediate time point, correcting, by the asynchronous time warp thread, the image data based on fourth predicted user pose data; at a third display time point, outputting the corrected data to a display screen for display, wherein the second intermediate time point is located between the second starting time point and the third display time point.
The display method according to claim 1 or 2, wherein when the electronic device disables the see-through function and only displays a virtual view, the electronic device starts a rendering thread and an asynchronous time warp thread, and the method further comprises: at a third starting time point, rendering, by the rendering thread, first screen scene data based on fifth predicted user pose data; at a third intermediate time point, sending the rendered first screen scene data to the asynchronous time warp thread, and rendering second screen scene data based on sixth predicted user pose data; and at a fourth display time point, sending the rendered second screen scene data to the asynchronous time warp thread; and at the third intermediate time point, correcting, by the asynchronous time warp thread, the rendered first screen scene data based on seventh predicted user pose data to obtain first screen output data; at the fourth display time point, outputting the first screen output data to a display screen for display, and correcting the rendered second screen scene data based on the seventh predicted user pose data to obtain second screen output data; at a fifth display time point, outputting the second screen output data to the display screen for display, wherein the third intermediate time point is located between the third starting time point and the fourth display time point, and the fourth display time is located between the third intermediate time point and the fifth display time point.
A processing device, comprising: a processor and a memory storing a computer program that is runnable on the processor, wherein when executing the computer program, the processor performs the acts of the display method according to any one of claims 1 to 15.
An electronic device, comprising: an image acquisition apparatus and the processing device according to claim 16.
A computer readable storage medium storing computer executable instructions, wherein the computer executable instructions are used for performing the display method according to any one of claims 1 to 15.

Description

The present application claims priority to Chinese Patent Application No. 202311861641.1 filed to the CNIPA on December 29, 2023, and entitled "Electronic Device and Display Method Therefor, Processing Device, and Computer-Readable Storage Medium", contents of which should be understood as being incorporated herein by reference. Technical Field The present disclosure relates to intelligent display technology, in particular to an electronic device and a display method thereof, a processing device, and a computer-readable storage medium. Background Currently, the Virtual Reality (VR)/Mixed Reality (MR) industry is developing rapidly. A VR application is a computer-generated 3D environment, which may enable users to be completely immersed in a virtual world presented by a head mounted device instead of seeing a real environment. The MR application mixes a real world and a virtual world to create a new visual and interactive environment, which contains both a physical entity and virtual information. In such environment, characters and objects in the real world and virtual world may cross a boundary of reality, thus creating a more complex and exciting experience. In some VR/MR products, a camera for see through is reused by a camera for gesture tracking. When a condition for starting see through is met, the camera acquires an external image. Because a light emitting frequency of ambient light interferes with an acquisition frame rate of the camera, especially when indoor light emitting diodes (LEDs) are on, a light emitting frequency of light of the indoor LEDs will seriously interfere with the acquisition frame rate of the camera, resulting in obvious flicker phenomenon (Flicker), which will cause uses' discomfort and affect adversely the use experience of the users. Summary The following is a summary of subject matters described herein in detail. This summary is not intended to limit the protection scope of claims. The present disclosure provides a display method of an electronic device, wherein the electronic device includes an image acquisition apparatus, and the display method includes: detecting whether the electronic device enables a see-through function; and controlling an acquisition frame rate of the image acquisition apparatus to be a second frequency when the electronic device enables the see-through function,, wherein the second frequency is an integer multiple of a first frequency, and the first frequency is a light emitting frequency of ambient light. An embodiment of the present disclosure also provides a processing device, including: a processor and a memory storing a computer program that is runnable on the processor, wherein when executing the program, the processor performs acts of the display method as described above. An embodiment of the present disclosure also provides an electronic device, including an image acquisition apparatus and the processing device according to any embodiment of the present disclosure. An embodiment of the present disclosure also provides a computer readable storage medium for storing executable instructions, wherein the display method as described in any one of the above embodiments may be implemented when the executable instructions are executed by a processor. In the electronic device and the display method thereof, the processing device and the computer-readable storage medium according to embodiments of the present disclosure, by controlling an acquisition frame rate of an image acquisition apparatus to be a second frequency when the electronic device enables a see-through function, wherein the second frequency is an integer multiple of a first frequency, and the first frequency is a light emitting frequency of ambient light, the problem of flicker caused by interference between the light emitting frequency of ambient light and the acquisition frame rate of the camera is solved, and the use experience of the users is improved. Other features and advantages of the present disclosure will be set forth in the following specification, and moreover, partially become apparent from the specification, or are understood by implementing the present disclosure. Other advantages of the present disclosure may be achieved and obtained through solutions described in the specification and drawings. Brief Description of Drawings Accompanying drawings are used to provide an understanding of technical solutions of the present disclosure, and form a part of the specification. The accompanying drawings and embodiments of the present disclosure are adopted to explain the technical solutions of the present disclosure, and do not form limitations on the technical solutions of the present disclosure. FIG. 1 is a schematic flowchart of a display method according to an exemplary embodiment of the present disclosure.FIG. 2 is a schematic diagram of a method of using a head mounted display according to an exemplary embodiment of the present disclosure.FIG. 3 to FIG. 8 are schematic flow