US-20260129146-A1 - VIRTUAL DESKTOP CREATION AND APPLICATION NAVIGATION TO INCREASE ACCESS TO SHARED INFORMATION

Abstract

A method, according to one approach, includes: analyzing a screen sharing video stream in response to receiving the screen sharing video stream from a presenter's computer. The method also includes identifying application information and navigation actions included in the screen sharing video stream. Navigation metadata is generated in real-time, where the navigation metadata includes application static information and dynamic navigation actions. The navigation metadata is also sent to at least one participant of the screen sharing video. Furthermore, the method includes causing the at least one participant to reorganize keyframes and build up a virtual desktop using the navigation metadata sent.

Inventors

• Zhe Yan
• Li Li Guan
• Rong Zhao
• Li Bo Zhang
• Hao Xiang Wu

Assignees

• INTERNATIONAL BUSINESS MACHINES CORPORATION

Dates

Publication Date

20260507

Application Date

20241104

Claims (20)

1. 1 . A method comprising: in response to receiving a screen sharing video stream from a presenter's computer, analyzing the screen sharing video stream; identifying application information and navigation actions included in the screen sharing video stream; generating navigation metadata in real-time, the navigation metadata including application static information and dynamic navigation actions; sending the navigation metadata to at least one participant of the screen sharing video; and causing the at least one participant to reorganize keyframes and build up a virtual desktop using the navigation metadata.
2. 2 . The method of claim 1 , wherein the identifying application information and navigation actions included in the screen sharing video stream includes: identifying application static information from keyframes using object detection and image processing techniques, capturing navigation actions which cause screen change; identifying duplicate keyframes; and capturing application switching, link source, and target application.
3. 3 . The method of claim 2 , wherein the capturing navigation actions which cause screen change includes: identifying element click events; identifying scrollbar change; and leveraging vocal input received from the presenter.
4. 4 . The method of claim 2 , wherein the application static information identified from the keyframes is selected from the group consisting of: application type, title, fixed area, and content area.
5. 5 . The method of claim 1 , wherein the screen sharing video stream is part of a video call connecting the presenter with the participant and one or more other participants, wherein audio and visual information is exchanged between the presenter and the participants.
6. 6 . A computer program product comprising: one or more computer-readable storage media; and program instructions stored on the one or more storage media to perform operations comprising: in response to receiving a screen sharing video stream from a presenter's computer, analyzing the screen sharing video stream; identifying application information and navigation actions included in the screen sharing video stream; generating navigation metadata in real-time, the navigation metadata including application static information and dynamic navigation actions; sending the navigation metadata to at least one participant of the screen sharing video; and causing the at least one participant to reorganize keyframes and build up a virtual desktop using the navigation metadata.
7. 7 . The computer program product of claim 6 , wherein the identifying application information and navigation actions included in the screen sharing video stream includes: identifying application static information from the keyframes using object detection and image processing techniques, capturing navigation actions which cause screen change; identifying duplicate keyframes; and capturing application switching, link source, and target application.
8. 8 . The computer program product of claim 7 , wherein the capturing navigation actions which cause screen change includes: identifying element click events; identifying scrollbar change; and leveraging vocal input received from the presenter.
9. 9 . The computer program product of claim 7 , wherein the application static information identified from the keyframes is selected from the group consisting of: application type, title, fixed area, and content area.
10. 10 . The computer program product of claim 6 , wherein the screen sharing video stream is part of a video call connecting the presenter with the participant and one or more other participants, wherein audio and visual information is exchanged between the presenter and the participants.
11. 11 . A computer system comprising: a processor set; one or more computer-readable storage media; and program instructions stored on the one or more storage media to cause the processor set to perform operations comprising: in response to receiving a screen sharing video stream from a presenter's computer, analyzing the screen sharing video stream; identifying application information and navigation actions included in the screen sharing video stream; generating navigation metadata in real-time, the navigation metadata including application static information and dynamic navigation actions; sending the navigation metadata to at least one participant of the screen sharing video; and causing the at least one participant to reorganize keyframes and build up a virtual desktop using the navigation metadata.
12. 12 . The computer system of claim 11 , wherein the screen sharing video stream is part of a video call connecting the presenter with the participant and one or more other participants, wherein audio and visual information is exchanged between the presenter and the participants.
13. 13 . The computer system of claim 11 , wherein the identifying application information and navigation actions included in the screen sharing video stream includes: identifying application static information from the keyframes using object detection and image processing techniques, capturing navigation actions which cause screen change; identifying duplicate keyframes; and capturing application switching, link source, and target application.
14. 14 . The computer system of claim 13 , wherein the capturing navigation actions which cause screen change includes: identifying element click events; identifying scrollbar change; and leveraging vocal input received from the presenter.
15. 15 . The computer system of claim 13 , wherein the application static information identified from the keyframes is selected from the group consisting of: application type, title, fixed area, and content area.
16. 16 . A method comprising: receiving navigation metadata from a central server; using the navigation metadata to reorganize keyframes and build up a virtual desktop; loading a current application in the virtual desktop; using the current application to display a screen sharing video stream received from a presenter's computer; and in response to receiving one or more navigation inputs from a participant, updating the virtual desktop to reflect the one or more navigation inputs, wherein the one or more navigation inputs are received from the participant in response to interacting with a user interface (UI) that corresponds to the virtual desktop, wherein the one or more navigation inputs include switching between displayed applications and/or adjusting a view in a current application.
17. 17 . The method of claim 16 , wherein the one or more navigation inputs are received from the participant in response to interacting with a user interface (UI) that corresponds to the virtual desktop.
18. 18 . The method of claim 17 , wherein the one or more navigation inputs include switching between displayed applications and/or adjusting a view in a current application.
19. 19 . The method of claim 16 , wherein the navigation metadata includes application static information and dynamic navigation actions, wherein the using the navigation metadata to reorganize keyframes and build up the virtual desktop includes: grouping keyframes into applications; displaying keyframes in an application view; and rendering hotspots on the keyframes based at least in part on the dynamic navigation actions.
20. 20 . The method of claim 16 , wherein the screen sharing video stream is part of a video call connecting the presenter with the participant and one or more other participants, wherein audio and visual information is exchanged between the presenter and the participants.

Description

BACKGROUND The present invention relates to distributed communication systems, and more specifically, this invention relates to increasing accessibility during video calls. Web conferencing is an umbrella term which includes various types of online audio and/or video collaborative services, including webinars, video calls, group calls using voice over Internet protocol, etc. Applications for web conferencing include meetings, training events, lectures, presentations shared between web-connected computers, etc. In general, web conferencing is made possible by Internet technologies which allow for communication to exist between different locations. Web conferencing thereby offers data streams of text-based messages, audio signals, video and/or still images, etc., to be shared simultaneously, across geographically dispersed locations. Web conferencing has become a frequently used tool to facilitate virtual work meetings and other group environments, like online teaching. In these online meetings, a presenter may share a view of what is currently displayed on their personal computer screen in order to direct participants (e.g., viewers) of the online meeting to specific content, including slides, spread sheets, videos, demo applications, etc. SUMMARY A method, according to one approach, includes: analyzing a screen sharing video stream in response to receiving the screen sharing video stream from a presenter's computer. The method also includes identifying application information and navigation actions included in the screen sharing video stream. Navigation metadata is generated in real-time, where the navigation metadata includes application static information and dynamic navigation actions. The navigation metadata is also sent to at least one participant of the screen sharing video. Furthermore, the method includes causing the at least one participant to reorganize keyframes and build up a virtual desktop using the navigation metadata sent. A computer program product, according to another approach, includes: one or more computer-readable storage media. The computer program product also includes program instructions that are stored on the one or more storage media to perform any combination(s) of the foregoing methodologies. A computer system, according to another approach, includes: a processor set, and one or more computer-readable storage media. The computer system also includes program instructions that are stored on the one or more storage media to cause the processor set to perform any combination(s) of the foregoing methodologies. A method, according to still another approach, includes: receiving navigation metadata from a central server. The navigation metadata is used to reorganize keyframes and build up a virtual desktop. The method also includes loading a current application in the virtual desktop. The current application is further used to display a screen sharing video stream received from a presenter's computer. Furthermore, in response to receiving one or more navigation inputs from a participant, the virtual desktop is updated to reflect the one or more navigation inputs. The one or more navigation inputs are received from the participant in response to interacting with a user interface (UI) that corresponds to the virtual desktop. The one or more navigation inputs may include switching between displayed applications and/or adjusting a view in a current application. A computer program product according to yet another approach, includes: one or more computer-readable storage media. The computer program product also includes program instructions that are stored on the one or more storage media to perform any combination(s) of the foregoing methodologies. Other aspects and implementations of the present invention will become apparent from the following detailed description, which, when taken in conjunction with the drawings, illustrate by way of example the principles of the invention. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram of a computing environment, in accordance with one approach. FIG. 2A is a representational view of a distributed system, in accordance with one approach. FIG. 2B is a representational view of components in a portion of the distributed system of FIG. 2A, in accordance with one approach. FIG. 3A is a flowchart of a method, in accordance with one approach. FIG. 3B is a flowchart of sub-processes for one of the operations in the method of FIG. 3A, in accordance with one approach. FIG. 3C is a flowchart of sub-processes for one of the operations in the method of FIG. 3A, in accordance with one approach. FIG. 4A is a representational view of a GUI at a group call participant location, in accordance with an in-use example. FIG. 4B is another representational view of a GUI at a group call participant location, in accordance with an in-use example. FIG. 4C is another representational view of a GUI at a group call participant location, in accordance with an in-use example.