Search

CN-118557959-B - Game automation system and method based on OCR (optical character recognition)

CN118557959BCN 118557959 BCN118557959 BCN 118557959BCN-118557959-B

Abstract

The invention provides a game automation system and a game automation method based on OCR (optical character recognition), wherein the game automation system comprises a service module, an automation module, an OCR module and a scheduling module, the service module is used for game automation configuration information management, task management and execution equipment management, the automation module is used for matching an automation script, controlling a task queue, distributing scheduling tasks and standardizing an automation execution result, the OCR module is used for receiving a screenshot sample in an automation execution process, judging the current automation execution state, extracting and recalling key parameters in an automation scene, and the scheduling module is used for distributing equipment nodes for task execution and controlling the task execution queue.

Inventors

  • Bi Shinan

Assignees

  • 金华市瀚宁科技有限公司

Dates

Publication Date
20260508
Application Date
20240702

Claims (6)

  1. 1. The game automation system based on OCR recognition is characterized by comprising a service module, an automation module, an OCR recognition module and a scheduling module; The business module is used for automatically configuring information management, task management and executing equipment management; the automatic module is used for matching an automatic script, controlling a task queue, distributing scheduling tasks, normalizing automatic execution results and generating a statistical report comprising execution time and operation success rate; The OCR recognition module is used for receiving screenshot samples in the automatic execution process, judging the current game state of automatic execution, extracting and recalling key parameters in an automatic scene, receiving automatic screen images intercepted by an automatic script at key points of the game by the OCR recognition module, preprocessing the screenshot by utilizing an image processing technology to improve the accuracy of OCR recognition, The key points of the game comprise a menu interface, a combat scene and a task prompt of the game; Then using the image recognition technology of feature point matching to locate the region containing key information in the screenshot, extracting the image key points based on ORB algorithm, then using OCR technology to recognize the text in the locating region, converting the recognized text into structured data, if the recognized text is blood volume, converting it into numerical value, if it is task description, extracting key word or task ID, then according to the extracted information, judging the state of current game by automatic script, then recalling the recognized and extracted key parameters into automatic script, so that script makes decision according to these parameters, if the blood volume is lower than a certain threshold, judging that blood-replenishing operation needs to be performed, triggering blood-replenishing mechanism, if the specific task key word needs to be switched to the corresponding task execution flow, finally, after executing operation, carrying out screenshot and recognition again by automatic script to monitor the effect of operation and adjust strategy according to the need; the scheduling module is used for distributing equipment nodes for executing tasks, controlling queues for executing the tasks and monitoring equipment states under the system.
  2. 2. The game automation system based on OCR recognition of claim 1, wherein the business module configures parsing parameters in the automation task, downloading key information of the code scene, modifying the device alarm threshold, and associating the task with the execution device cluster through the interface.
  3. 3. The game automation system based on OCR recognition of claim 1, wherein the automation module receives valid request parameters which pass verification, matches equipment node clusters for executing corresponding game automation tasks according to the parameters, sends the automation tasks to the matched available equipment nodes by inquiring the available equipment nodes, downloads game automation codes to the equipment nodes from a cloud warehouse according to engineering addresses configured by the data module, executes the game automation codes and monitors the execution process, judges the current game state by automatically capturing images and requesting services of the OCR recognition module through a request library, determines the next automation operation strategy through the return result of the OCR recognition module, and achieves the aim of accurately recognizing game elements and states.
  4. 4. The game automation system based on OCR recognition of claim 1, wherein the image processing technique comprises graying, binarizing, denoising, and Sobe operator edge detection.
  5. 5. The game automation system based on OCR recognition of claim 1, wherein the scheduling module allocates equipment nodes for executing tasks, controls queues for executing tasks, monitors connection states of the equipment node clusters, alarms specific project clusters when equipment is disconnected, and receives requests of the automation module, and executes instruction strategies for disabling, restarting and reconnecting the equipment nodes when game update and abnormal network disconnection abnormality are encountered in the process of automation execution.
  6. 6. A game automation method based on the OCR-based game automation system of any one of claims 1 to 5, the method comprising the steps of: s1, designing a game automation script, wherein a game interface identification link returns to formulate an automation execution strategy by means of identification of an OCR module; S2, after the local automation script is debugged, uploading codes to GitLab, adding game automation scene configuration through a configuration management module, and setting parameters required to be received by the script and machine nodes to be executed through a task configuration module; s3, initiating a game automation task through an interface provided by an automation module, inquiring a task state according to an automation task unique id returned by the interface, analyzing and checking parameters by the automation module after receiving the request, informing a dispatching layer to inquire an executable machine and distributing the task to an idle executable machine if the verification is passed, and executing an alarm strategy if the executable machine is insufficient; s4, starting an automation task on an execution machine, enabling a scheduling layer to clear a working space, downloading and starting the automation script in the S1 on the execution machine, enabling the game script to drive a corresponding game program, uploading a screenshot in the execution process, requesting an OCR interface, judging the current game state according to information returned by the OCR interface, executing a strategy of the next step in the script, and repeating the steps in the whole game automation process; S5, after the game automation is normally finished or abnormal occurs, the automation script can call back the executed result and state to the automation module, the automation module formats the parameters and stores the parameters in a warehouse, and if abnormal occurs, alarm pushing is performed according to alarm configuration preset in configuration management, so that a developer is guided to position the problem; s6, the developer can query the task log and the task state through the automation module according to the task id, the task type and the task name field, and pull the data statistical report according to the game scene and the time.

Description

Game automation system and method based on OCR (optical character recognition) Technical Field The invention relates to the field of game automation, in particular to a game automation system and method based on OCR (optical character recognition). Background Game automation is a complex process and involves technical challenges in multiple fields, the current game automation faces the following difficulties that 1, game environments have diversity, different games have different rules, interfaces and interaction modes, an automation system needs to be capable of adapting to the differences, 2, the automation system needs to be capable of identifying and analyzing visual information in the games, including characters, objects, environments and the like, which is particularly difficult on games with complex graphics or pixelated, 3, game automation needs to respond to changes of the game environments in real time, which puts high demands on the efficiency and response speed of algorithms, and 4, game developers may update game rules or repair vulnerabilities, which requires the automation system to be capable of adapting to the changes. The existing game automation system is taken into account, has strong specialization, is limited by element positioning and image matching, and does not have a technical solution or system for solving the combination of a plurality of difficulties including game interface diversity, graphic complexity and the like. For example PyAutoGame (Python-based game automation framework), whose core is screen recognition and event simulation, which uses OpenCV (cross platform computer vision library) for image processing, locating elements in the game by template matching or color recognition of the screen shots, and then using PyAutoGUI module to simulate mouse and keyboard operations, accurately controlling behavior in the game. However, with the progress of technology, the complexity of a real business scene exceeds that of the past, and elements in a game cannot be stably positioned simply through template matching and color recognition. For another example, chinese patent with publication No. CN115858358a provides a multi-terminal automatic game testing system, in which an automatic testing module identifies and clicks a screen area in a client currently in an active state, and the system divides a game into several segments to test simultaneously, so as to improve testing efficiency and reduce time cost of conventional testing. Disclosure of Invention In view of the defects of the existing automation system, the technical problem to be solved by the invention is to provide an OCR (optical character recognition) -based game automation system and method, which can solve the problems of difficult recognition of game elements and difficult positioning of game scenes caused by complex game environments and real-time page changes. In order to achieve the above object, according to one aspect of the present invention, the present invention is achieved by a game automation system based on OCR recognition, including a service module, an automation module, an OCR recognition module, a scheduling module; The business module is used for automatically configuring information management, task management and executing equipment management; The automatic module is used for matching an automatic script, controlling a task queue, distributing scheduling tasks and normalizing an automatic execution result; The OCR module is used for receiving screenshot samples in the automatic execution process, judging the current game state of automatic execution, and extracting and recalling key parameters in an automatic scene; the scheduling module is used for distributing equipment nodes for executing tasks, controlling queues for executing the tasks and monitoring equipment states under the system. Further, the service module configures key information of scenes such as analysis parameters, downloaded codes and the like in the automation task through an interface, modifies an equipment alarm threshold value, and associates the task with an execution equipment cluster. Further, the automation module receives the effective request parameters which pass verification, matches the equipment node clusters for executing the corresponding game automation tasks according to the parameters, transmits the automation tasks to the matched available equipment nodes by inquiring the available equipment nodes, downloads game automation codes to the equipment nodes from the cloud warehouse according to the engineering addresses configured by the data module, executes the game automation codes and monitors the execution process, judges the current game state by automatically capturing the images and requesting the services of the OCR recognition module through the request library, and determines the next automation operation strategy by the return result of the OCR recognition module, thereby achieving the aim of accurately