EP-4742083-A1 - DYNAMICALLY POPULATING A WEBPAGE IN REAL TIME WITH CONTENT FROM A CAPTURED IMAGE

EP4742083A1EP 4742083 A1EP4742083 A1EP 4742083A1EP-4742083-A1

Abstract

In one example, a system can establish a communication channel with a first client device displaying a webpage. The system can also receive an image and a unique identifier of the communication channel from a second client device. The system can provide the image as input to a model, which can extract and output content from the image. Based on the unique identifier, the system can select the communication channel as a destination for the content extracted from the image. The system can then transmit the content to the first client device via the communication channel to update the webpage to include the content in real time.

Inventors

Kobin, Henry Roy
Robertson, Zachary Clement
ABDOLLAHNEJADBAROUGH, HOSSEIN
Dolan, Brian David

Assignees

Fortrea Inc.

Dates

Publication Date: 20260513
Application Date: 20251111

Claims (15)

A method comprising: establishing, by a server system, a communication channel with a first client device of a user for updating a webpage displayed on the first client device; receiving, by the server system, an image over a network from a second client device; receiving, by the server system, a unique identifier of the communication channel over the network from the second client device; based on receiving the image from the second client device, providing, by the server system, the image as input to a model, the model being configured to extract and output content of the image; based on receiving the unique identifier from the second client device, selecting, by the server system, the communication channel as a destination for the content extracted from the image; and based on selecting the communication channel as the destination, transmitting, by the server system, the content to the first client device via the communication channel to update the webpage to include the content in real time.
The method of claim 1, wherein the unique identifier is embedded in a barcode displayed on the webpage, and wherein the second client device is configured to: extract the unique identifier from the barcode by scanning the barcode on the webpage; after extracting the unique identifier from the barcode on the webpage, transmit the unique identifier over the network to the server system; capture the image using an imaging device; and after capturing the image, transmit the image over the network to the server system.
The method of claim 2, wherein the barcode is a Quick Response (QR) code.
The method of any preceding claim, wherein the image depicts a data table, and the content includes textual or numerical content in the data table.
The method of any preceding claim, wherein the model is a trained large language model (LLM) that is configured to: analyze the image to identify columns and rows forming cells of the data table; analyze the image to extract the content from the cells of the data table; and generate the output in a declarative format that correlates the content to one or more specific portions of the webpage.
The method of claim 5, wherein the declarative format is a JavaScript Object Notation (JSON) format or an extensible Markup Language (XML) format.
The method of any preceding claim, wherein the webpage includes a form, and further comprising automatically populating the form with the content.
The method of any preceding claim, further comprising, after transmitting the content to the first client device via the communication channel: receiving an identifier of missing data from the first client device via the communication channel, the missing data being absent from the content; transmitting a request for the missing data to the user; receiving a response that includes the missing data from the user; and transmitting the missing data to the first client device via the communication channel to update the webpage to include the missing data in real time.
The method of any preceding claim, wherein the model generates metadata specifying one or more specific locations in the webpage into which to insert the content, and further comprising: transmitting, by the server system, the metadata to the first client device via the communication channel for causing the content to be inserted into the specific locations of the webpage while the webpage is displayed on the first client device.
The method of any preceding claim, wherein the first client device is the same as the second client device.
The method of any preceding claim, wherein the first client device is different than the second client device.
The method of any preceding claim, wherein the communication channel is a Websocket channel.
The method of any preceding claim, wherein the communication channel is selected from among a plurality of communication channels corresponding to a plurality of webpages.
A server system comprising: one or more processors; and one or more memories storing instructions that are executable by the one or more processors for causing the one or more processors to perform the method of any preceding claim.
A non-transitory computer-readable medium comprising program code that is executable by one or more processors for causing the one or more processors to perform the method of any of claims 1 to 13.

Description

Technical Field The present disclosure relates generally to webpages. More specifically, but not by way of limitation, this disclosure relates to dynamically populating a webpage in real time with content from a captured image. Background It has become increasingly commonplace for webpages to be dynamically updated in real-time during user interactions to display new content. By using technologies such as AJAX (Asynchronous JavaScript and XML), WebSockets, and server-sent events, modern web applications can seamlessly fetch and render updated data without requiring a full page reload. This dynamic updating capability enhances user experience by providing instantaneous feedback and more interactive, responsive interfaces. As users perform actions such as filling out forms, selecting menu options, or navigating through content, the webpage can automatically refresh specific elements to reflect changes, display relevant information, or suggest further actions. This not only improves the efficiency and fluidity of user interactions, but also reduces server load and bandwidth usage by minimizing redundant data transfers and page reloads. Summary One example of the present disclosure includes a server system comprising one or more processors and one or more memories. The one or more memories store instructions that are executable by the one or more processors for causing the one or more processors to perform operations. The operations include establishing a communication channel with a first client device of a user for updating a webpage displayed on the first client device. The operations include receiving an image over a network from a second client device. The operations include receiving a unique identifier of the communication channel over the network from the second client device. The operations include, based on receiving the image from the second client device, providing the image as input to a model, the model being configured to extract and output content of the image. The operations include, based on receiving the unique identifier from the second client device, selecting the communication channel as a destination for the content extracted from the image. The operations include, based on selecting the communication channel as the destination, transmitting the content to the first client device via the communication channel to update the webpage to include the content in real time. Another example of the present disclosure includes a method of operations. The operations include establishing a communication channel with a first client device of a user for updating a webpage displayed on the first client device. The operations include receiving an image over a network from a second client device. The operations include receiving a unique identifier of the communication channel over the network from the second client device. The operations include, based on receiving the image from the second client device, providing image as input to a model, the model being configured to extract and output content of the image. The operations include, based on receiving the unique identifier from the second client device, selecting the communication channel as a destination for the content extracted from the image. The operations include, based on selecting the communication channel as the destination, transmitting the content to the first client device via the communication channel to update the webpage to include the content in real time. Some or all of the operations can be implemented by a server system. Yet another example of the present disclosure includes a non-transitory computer-readable medium comprising program code that is executable by one or more processors for causing the one or more processors to perform operations. The operations include establishing a communication channel with a first client device of a user for updating a webpage displayed on the first client device. The operations include receiving an image over a network from a second client device. The operations include receiving a unique identifier of the communication channel over the network from the second client device. The operations include, based on receiving the image from the second client device, providing image as input to a model, the model being configured to extract and output content of the image. The operations include, based on receiving the unique identifier from the second client device, selecting the communication channel as a destination for the content extracted from the image. The operations include, based on selecting the communication channel as the destination, transmitting the content to the first client device via the communication channel to update the webpage to include the content in real time. The first client device may be the same as the second client device or different than the second client device. In some implementations the unique identifier may be embedded in a barcode displayed on the webpage. The second c