CN-121971850-A - Cross-terminal unified operation transmission system and method under cloud game scene

CN121971850ACN 121971850 ACN121971850 ACN 121971850ACN-121971850-A

Abstract

The invention discloses a cross-terminal unified operation transmission system and a cross-terminal unified operation transmission method under a cloud game scene, wherein the system comprises a client and a cloud game server; the cloud game service end comprises a binary decoding module, an event state management module, a coordinate restoration module and an event injection module. The method has the advantages that the operation delay is remarkably reduced, the data volume of a single operation message can be reduced by more than 75% by adopting a customized binary protocol to replace JSON, the network transmission time is reduced, the end-to-end operation delay is remarkably reduced, and the response speed and the following hand feeling of the operation are greatly improved. The real cross-platform unification is realized, namely through the operation abstraction and the coordinate proportionality of the client, the server does not need to sense what equipment or input mode the front end is, and all inputs are normalized, so that the logic complexity and the code coupling degree of the server are greatly reduced.

Inventors

Lin Xuexun
Chen Wangke
XIE WEIJUN

Assignees

四三九九网络股份有限公司

Dates

Publication Date: 20260505
Application Date: 20260203

Claims (7)

1. The cross-terminal unified operation transmission system under the cloud game scene is characterized by comprising a client and a cloud game server; the client comprises a unified input capturing module, an operation abstracting module, a coordinate proportioning module and a binary coding module; the unified input capturing module is used for monitoring and capturing original physical input events generated by interaction between a user and various input devices, wherein the original physical input events comprise touch screen input events, mouse input events and keyboard input events; the operation abstraction module is used for converting the captured various types of original physical input events into predefined standardized operation events, wherein the standardized operation events are represented as { action types, contact IDs, absolute coordinates }; The coordinate scaling module is used for converting absolute coordinates in the standardized operation event into scaled coordinates relative to an effective game display area, so as to obtain the standardized operation event { action type, contact ID, scaled coordinates }, which are subjected to coordinate scaling; The binary coding module is used for binary coding the standardized operation event { action type, contact ID, scaled coordinates } according to a predefined compact binary format to generate a compact binary data stream, wherein the binary data stream is sent to the cloud game server through a network interface of the client; The cloud game service end comprises a binary decoding module, an event state management module, a coordinate restoration module and an event injection module; The binary decoding module is used for receiving the binary data stream from the client, analyzing the binary data stream according to a predefined binary format and extracting an action type, a contact ID and a scaled coordinate; the event state management module is used for tracking the state of each touch point in the memory according to the contact ID, and further reconstructing a series of discrete single-point action messages into a complete action gesture conforming to the operating system specification; The coordinate reduction module is used for reducing the extracted proportioned coordinates into absolute coordinates in a cloud virtual environment screen; The event injection module is used for calling a bottom layer system interface of the cloud virtual environment, simulating the operation comprising the restored absolute coordinates and the complete action gesture into one or a series of original touch events, and injecting the touch events into the target game application.
2. The cross-terminal unified operation transmission system under the cloud game scene as claimed in claim 1, wherein the touch screen input event is a touch event of a touch screen, the mouse input event is a click and movement event of a mouse, and the keyboard input event is a key event of a keyboard.
3. The system for transmitting unified operations across terminals in a cloud game scenario according to claim 1, wherein the operation abstraction module is specifically configured to: the standardized operation event is a standardized touch operation event; if the captured original physical input event is a touch screen input event, directly obtaining the standardized touch operation event; and if the captured original physical input event is a keyboard input event or a mouse input event, executing logic judgment and data conversion of keyboard coordinate or mouse finger, and converting the logic judgment and data conversion into the standardized touch operation event.
4. The system of claim 3, wherein the keyboard is coordinated to map discrete keyboard key events to continuous touch events of a preset or user-defined coordinate point on a screen, wherein the keyboard key events comprise a keyboard key press event and a keyboard key lift event; The mouse is characterized in that the track of the mouse and the state of the mouse key are simulated to be single-finger touch, the left button of the mouse is pressed DOWN to correspond to the DOWN action, the corresponding continuous MOVE action is pressed and moved, and the left button is released to correspond to the UP action.
5. The cross-terminal unified operation transmission system in a cloud game scenario according to claim 1, wherein the coordinate scaling module converts absolute coordinates (X, Y) in the standardized operation event into scaled coordinates (axialX, axialY) relative to the effective game display area, wherein axialX represents an X coordinate scale and axialY represents a Y coordinate scale, and the specific conversion method is as follows: Setting the screen resolution as wide L1 x high H1, the size of the effective game display area as wide L2 x high H2, the center of the effective game display area coinciding with the center of the screen, the absolute coordinates (x, y) of the contact ID being the screen coordinates; a. axialX =x/L2, axialy=y/H2 if the game effective display area size and screen resolution are the same; b. If the resolution of the game picture and the resolution of the screen are not completely matched, the game picture is high and the screen is full, namely, the game picture is long and the screen is not full, namely, the height H1 is the same as the height H2, and the width L2 is smaller than the width L1, then: (1) If x does not enter the effective game display area and is at the left side of the effective game display area, resetting x=0 and the y value is unchanged, wherein the values are axialX =0/L2=0 and axialY =y/H2; (2) If x enters the effective game display area, the real abscissa in the effective game display area needs x minus the length outside the game picture area, namely realx =x- (L1-L2)/2, namely axialX = realx/L2 and axialY =y/H2; (3) If x does not enter the effective game display area and is on the right side of the effective game display area, resetting the value of x=wide L2 and the value of y is unchanged, wherein the values are axialX =L2/L2=1 and axialY =y/H2; c. If the resolution of the game picture and the resolution of the screen are not completely matched, the game picture is long and the screen is full, namely, the height H2 is smaller than the height H1, and the width L2 is equal to the width L1, the game picture is long and the screen is not full, namely: (1) If y does not enter the effective game display area and is at the upper side of the effective game display area, resetting y=0 and the value of x is unchanged, wherein the values of the values are axialX =x/L2, axialY =0/H2=0; (2) If y enters the effective game display area, the true ordinate in the effective game display area needs y minus the length outside the game picture area, namely realy =y- (H1-H2)/2, axialy = realy/H2, axialX =x/L2; (3) If y does not enter the effective game display area and is below the effective game display area, resetting y=high H2 and the value of x is unchanged, wherein axialX =x/L2 and axialy =H2/H2=1.
6. The cross-terminal unified operation transmission system under the cloud game scene as claimed in claim 1, wherein the coordinate restoring module is configured to restore the extracted scaled coordinates to absolute coordinates in a cloud virtual environment screen, specifically: the coordinate restoring module executes inverse mathematical operation on the scaled coordinates according to the screen resolution of the cloud virtual environment, restores the scaled coordinates to absolute coordinates (absoluteX, absoluteY) in the cloud virtual environment screen, and has the following formula: absoluteX = axialX virtual screen width; absoluteY = axialY virtual screen height; Wherein the virtual screen width and the virtual screen height are known fixed parameters of the cloud game service end.
7. A method of unifying operation of a transmission system across terminals in a cloud gaming scenario as set forth in any one of claims 1-6, comprising: step S1, a client side data processing and converting process: Step S11, a unified input capturing module of a client monitors and captures original physical input events generated by interaction of a user and various input devices, wherein the original physical input events comprise touch screen input events, mouse input events and keyboard input events; Step S12, an operation abstraction module of the client converts the captured various types of original physical input events into predefined standardized operation events, wherein the standardized operation events are represented as { action types, contact IDs, absolute coordinates }, and the action types comprise a DOWN type, a UP type and a mobile MOVE type; Step S13, the coordinate scaling module of the client converts absolute coordinates in the standardized operation event into scaled coordinates relative to an effective game display area, so as to obtain the standardized operation event { action type, contact ID, scaled coordinates }; Step S14, the binary coding module of the client performs binary coding on the standardized operation event { action type, contact ID, proportioned coordinates } according to a predefined compact binary format to generate a compact binary data stream, wherein the binary data stream is sent to a cloud game server through a network interface of the client; step S2, receiving and injecting cloud game server side data: Step S21, a binary decoding module of a cloud game server receives the binary data stream from the client, analyzes the binary data stream according to a predefined binary format, and extracts an action type, a contact ID and a scaled coordinate; Step S22, an event state management module of the cloud game service end tracks the state of each touch point in a memory according to the contact ID, and further reconstructs a series of discrete single-point action messages into a complete action gesture conforming to the operating system specification; Step S23, a coordinate reduction module of the cloud game server restores the extracted proportioned coordinates to absolute coordinates in a cloud virtual environment screen; Step S24, an event injection module at the cloud game service end calls a bottom layer system interface of the cloud virtual environment, simulates operation comprising restored absolute coordinates and complete action gestures into one or a series of original touch events, and injects the touch events into a target game application.

Description

Cross-terminal unified operation transmission system and method under cloud game scene Technical Field The invention relates to the technical fields of computer network communication and cloud computing, in particular to a cross-terminal unified operation transmission system and method in a cloud game scene. Background The cloud game is an interactive application that performs operation and rendering of a game program on a cloud server, and transmits rendered game pictures to a user terminal in a video stream form. In this mode, the operation instruction of the user at the user terminal needs to be quickly and accurately transmitted to the cloud server to drive the game progress. The delay in operation transmission is a central indicator in determining the cloud gaming experience, particularly "heel-hand". A commonly employed operational data transmission scheme in the prior art is the use of JSON (JavaScript Object Notation) format. For example, a single touch down operation by the user at the user terminal may be encoded as { "action": "Down", "pointerId": 1, "x": 100, "y": 100}. The above solution has the following serious drawbacks: 1. Data redundancy and high latency JSON as a text format contains a large number of structural characters, such as characters { }, characters "", characters:, and repeated key names, such as action, pointerId. In a multi-touch or high frequency operation scenario, such as a quick screen swipe in a first person shooter game, a large amount of such operation data can be instantaneously generated, resulting in an excessively large volume of operation data packets. This significantly increases the burden and time of operating the data network transmission in environments where network bandwidth is limited, especially mobile networks, and the user experiences significant operating delays and jams, i.e. "operation is not left behind". 2. The cross-platform suitability is poor, the maintenance cost is high, and the service objects of the cloud game are expanded from a single Android platform to iOS, H5 webpages (running in Windows/macOS) and the like. The user input sources of different platforms have the fundamental difference that the mobile terminal is mainly multi-point touch control, and the PC webpage terminal is mainly a keyboard and a mouse. The keyboard operation involves the pressing and lifting of a large number of keys, and the mouse involves clicking, moving and dragging. In addition, screen sizes, resolutions, and aspect ratios of different terminal devices are different. If the game screen is scaled unevenly to fit the terminal screen, such as transversely or longitudinally, the absolute coordinates of the terminal are directly transmitted, which results in the fact that the position cannot be mapped to the correct in-game position at the cloud. The cloud game server can realize a set of event analysis, coordinate conversion and event injection logic on the cloud game server for each newly accessed platform, which results in extremely complex code logic of the cloud game server, high coupling degree and high maintenance and iteration cost. Therefore, there is a strong need in the art for a new solution that can greatly reduce the operation delay and uniformly process multi-platform, multi-device inputs in a low cost manner. Disclosure of Invention Aiming at the defects existing in the prior art, the invention provides a cross-terminal unified operation transmission system and a cross-terminal unified operation transmission method under a cloud game scene, which can effectively solve the problems. The technical scheme adopted by the invention is as follows: the invention provides a cross-terminal unified operation transmission system under a cloud game scene, which comprises a client and a cloud game server; the client comprises a unified input capturing module, an operation abstracting module, a coordinate proportioning module and a binary coding module; the unified input capturing module is used for monitoring and capturing original physical input events generated by interaction between a user and various input devices, wherein the original physical input events comprise touch screen input events, mouse input events and keyboard input events; the operation abstraction module is used for converting the captured various types of original physical input events into predefined standardized operation events, wherein the standardized operation events are represented as { action types, contact IDs, absolute coordinates }; The coordinate scaling module is used for converting absolute coordinates in the standardized operation event into scaled coordinates relative to an effective game display area, so as to obtain the standardized operation event { action type, contact ID, scaled coordinates }, which are subjected to coordinate scaling; The binary coding module is used for binary coding the standardized operation event { action type, contact ID, scaled coordinates } accordi