Search

CN-121996754-A - Mobile terminal agent processing method, system and equipment based on natural language processing

CN121996754ACN 121996754 ACN121996754 ACN 121996754ACN-121996754-A

Abstract

The invention discloses a mobile terminal agent processing method, a system and equipment based on natural language processing, wherein the method specifically comprises the steps of analyzing natural language commands, extracting operation intention and operation entity, inquiring a pre-constructed operation knowledge graph based on the operation intention and the operation entity, determining similar historical operation scripts recommended by the multiplexing operation knowledge graph or generating new operation scripts by comparing an inquiring result with a preset similarity threshold value, dynamically adjusting parameters of the operation scripts to be executed according to equipment state and environment information, collecting feedback data of an executing process and an executing result, inputting the feedback data into a reinforcement learning model, optimizing the parameters of the operation scripts, and updating the optimized parameters to the operation knowledge graph. The invention realizes the intelligent processing of the mobile terminal operation based on natural language processing, and improves the automation and intelligent level of the mobile terminal operation from instruction analysis to script execution and optimization to form a complete closed loop.

Inventors

  • LIU LIANG

Assignees

  • 广州三七极耀网络科技有限公司

Dates

Publication Date
20260508
Application Date
20251229

Claims (10)

  1. 1. A mobile terminal agent processing method based on natural language processing is characterized by comprising the following steps: Receiving a natural language command input by a user, analyzing the natural language command by using a natural language processing model, and extracting an operation intention and an operation entity; Inquiring a pre-constructed operation knowledge graph based on the operation intention and the operation entity, and determining a similar historical operation script recommended by the multiplexing operation knowledge graph or generating a new operation script by comparing an inquiry result with a preset similarity threshold; Acquiring equipment state and environment information in real time through a context awareness engine, and dynamically adjusting parameters of an operation script to be executed according to the equipment state and the environment information; executing the operation script with the parameters adjusted, collecting feedback data of the execution process and the execution result, inputting the feedback data into the reinforcement learning model, optimizing the parameters of the operation script, and updating the optimized parameters to the operation knowledge graph; And aggregating model update data from a plurality of mobile terminals by utilizing a federal learning framework, and cooperatively optimizing global performances of a natural language processing model and a reinforcement learning model.
  2. 2. The method according to claim 1, wherein the parsing the natural language command using the natural language processing model extracts the operation intention and the operation entity, specifically comprising: performing domain term standardization processing on the natural language command to obtain a processed command text; inputting the command text into a lightweight natural language understanding model subjected to field self-adaptive training, and extracting the operation intention and the operation entity jointly from the command text; In the joint extraction process, a state buffer memory for storing the latest interaction context is maintained, and when the fact that the current command text contains a preset reference word or the fact that the components are omitted is detected, the most relevant entity information is retrieved from the state buffer memory for completing the semantic representation of the current command text; And performing post-processing based on confidence filtering on the intention classification result and the entity recognition result output by the lightweight natural language understanding model, and combining a preset semantic role framework to assemble a structured operation instruction object.
  3. 3. The method of claim 2, wherein the training step of the lightweight natural language understanding model comprises: constructing and labeling a corpus containing the field of mobile equipment operation instructions, wherein each instruction of the corpus is labeled with an intention label and an entity sequence label; selecting a basic transducer encoder model, and adding parallel task heads for intention classification and entity identification at the top of the basic transducer encoder model to form a composite model to be finely tuned; Performing two-stage progressive fine tuning on the composite model, using domain corpus to perform self-supervision task fine tuning on an encoder part in a first stage, and enabling a parallel task head to perform end-to-end joint fine tuning based on labeling corpus in a multi-task learning mode in a second stage; And (3) training by using a knowledge distillation technology and utilizing a pre-trained teacher model to guide the fine-tuned composite model to obtain a lightweight natural language understanding model.
  4. 4. The method according to claim 1, wherein the querying the pre-constructed operation knowledge graph based on the operation intention and the operation entity determines similar historical operation scripts recommended by the operation knowledge graph or generates new operation scripts by comparing the query result with a preset similarity threshold, specifically comprising: inquiring a pre-constructed operation knowledge graph based on the operation intention and the operation entity, wherein nodes of the operation knowledge graph are used for representing historical operation tasks and storing historical scripts and execution contexts in an associated mode; for candidate historical task nodes in the operation knowledge graph, calculating semantic similarity of the current instruction and the candidate historical task nodes on operation intention, set similarity on an operation entity and context matching degree between a real-time context and a historical context in parallel; Weighting and fusing semantic similarity, aggregate similarity and context matching degree to obtain comprehensive recommendation scores of candidate historical task nodes; And comparing the highest comprehensive recommendation score with a preset multiplexing threshold, if the highest comprehensive recommendation score exceeds the preset multiplexing threshold, using a historical operation script associated with the corresponding candidate historical task node, and if the highest comprehensive recommendation score does not exceed the preset multiplexing threshold, generating a new operation script.
  5. 5. The method according to claim 1, wherein the acquiring, by the context awareness engine, the device state and the environment information in real time, and dynamically adjusting the parameters of the operation script to be executed according to the device state and the environment information, specifically includes: the method comprises the steps that the context awareness engine acquires the device endogenous state, the network state and the space-time environment information in parallel to form original context information; carrying out normalization processing and feature extraction on the original situation information to generate a standardized situation feature vector; Inputting the situation feature vector into a strategy decision layer, matching based on a preset situation-strategy rule base, and outputting an adjustment strategy identifier; inquiring a parameter adjustment mapping table according to the adjustment strategy identification to obtain specific adjustment instructions aiming at various parameters in the operation script; analyzing an operation script to be executed, and positioning adjustable parameter nodes in the operation script; Dynamically modifying the values of the adjustable parameter nodes according to the specific adjustment instructions to generate target executable scripts adapting to the real-time situation.
  6. 6. The method according to claim 1, wherein the executing the operation script after parameter adjustment, collecting feedback data of the execution process and the execution result, inputting the feedback data into the reinforcement learning model, optimizing parameters of the operation script, and updating the optimized parameters to the operation knowledge graph, specifically includes: Executing the operation script after parameter adjustment, and collecting feedback data comprising an execution result, time consumption of the process and interaction precision through a monitor; Carrying out structuring treatment on the feedback data to construct a reinforcement learning input vector containing the current script parameter state and the comprehensive performance rewards; inputting the reinforcement learning input vector into a locally deployed reinforcement learning model, driving the reinforcement learning model to learn based on a historical state, actions and rewarding sequences, and outputting an optimization adjustment action for script parameters; Generating a group of optimized script parameter proposal values according to the optimized adjustment actions output by the reinforcement learning model; And storing the optimized script parameter proposal value and the context information for generating the optimization in an optimization parameter set of a corresponding script node in the operation knowledge graph in a correlated manner.
  7. 7. The method of claim 6, wherein the inputting reinforcement learning input vectors into a locally deployed reinforcement learning model, driving the reinforcement learning model to learn based on historical states, actions and rewards sequences, and outputting optimized adjustment actions for script parameters, specifically comprises: constructing the reinforcement learning input vector and the new state caused by the reinforcement learning input vector together as an experience unit, and storing the reinforcement learning input vector and the new state into a local experience playback buffer area; when a preset learning triggering condition is met, randomly sampling a batch of experience units from an experience playback buffer area; Calculating an estimated value of a dominance function of each state action pair by using the sampled experience units; based on the dominance function estimated value, updating parameters of the local strategy network by adopting a near-end strategy optimization algorithm, so that the action distribution output by the local strategy network tends to generate high-rewarding actions; inputting the script parameter state to be optimized into the updated local strategy network, and selecting specific parameter optimization adjustment actions by combining the explored and utilized strategies according to the action probability distribution output by the local strategy network.
  8. 8. The method according to claim 1, wherein aggregating model update data from a plurality of mobile terminals using a federal learning framework, co-optimizing global performance of a natural language processing model and a reinforcement learning model, specifically comprises: initializing a global reference model of a natural language processing model and a reinforcement learning model through a central server, and distributing the global reference model to a plurality of mobile terminals; training two global reference models by using private data locally by each mobile terminal respectively to generate respective local model update data; Each mobile terminal performs privacy protection and compression processing on the local model update data to obtain safe update data; Uploading the safety updating data to a central server by each mobile terminal, decrypting and aggregating the safety updating data based on the safety updating data from a plurality of mobile terminals in the central server, and respectively calculating global aggregation updating data of a natural language processing model and a reinforcement learning model; And applying global aggregation update data through the central server to optimize the global reference model maintained by the central server, and distributing the optimized global reference model to each mobile terminal as a new round of global reference model.
  9. 9. A mobile terminal agent processing system based on natural language processing is characterized in that the system specifically comprises: The command analysis module is used for receiving a natural language command input by a user, analyzing the natural language command by utilizing a natural language processing model, and extracting an operation intention and an operation entity; the map query module is used for querying a pre-constructed operation knowledge map based on the operation intention and the operation entity, and determining a similar historical operation script recommended by the multiplexing operation knowledge map or generating a new operation script by comparing a query result with a preset similarity threshold; The context awareness module is used for acquiring the equipment state and the environment information in real time through the context awareness engine, and dynamically adjusting parameters of the operation script to be executed according to the equipment state and the environment information; The feedback optimization module is used for executing the operation script after parameter adjustment, collecting feedback data of an execution process and an execution result, inputting the feedback data into the reinforcement learning model, optimizing the parameters of the operation script, and updating the optimized parameters to the operation knowledge graph; And the federal learning module is used for aggregating model update data from a plurality of mobile terminals by utilizing a federal learning framework and cooperatively optimizing the global performance of the natural language processing model and the reinforcement learning model.
  10. 10. A computer device comprising a memory and a processor and a computer program stored on the memory, which when executed on the processor, implements the mobile-side agent processing method based on natural language processing according to any one of claims 1 to 8.

Description

Mobile terminal agent processing method, system and equipment based on natural language processing Technical Field The invention relates to the technical field of artificial intelligence, in particular to a mobile terminal agent processing method, a system and equipment based on natural language processing. Background With the rapid development of the mobile internet, the number of mobile devices and applications has presented an explosive growth situation. This trend has led to an increasing and ever-increasing need for testing, data capture, and automation of mobile devices and applications. However, in the current field of mobile terminal operation and control, the conventional method mainly relies on manual operation and script writing. This approach exposes a number of significant drawbacks in practical applications: 1. The operation is complex, the manual operation and script writing process are very complicated, and the manual operation and script writing process relate to numerous details and steps and require operators to have higher technical level. For example, when an automation test script is written, an operator needs to know the architecture, interfaces and various operation logics of an application program in depth, and grasp a specific script language, which is difficult for a common user, so that the popularization range of automation operation and control is greatly limited. 2. The efficiency is low that the conventional method requires a lot of time to perform repeated manual operations or script writing in the face of repeated operation tasks. For example, when testing a large-scale application program, if a manual mode is adopted for repeated test operation under different versions and different scenes, the efficiency is extremely low, the test result is easy to be inaccurate due to human negligence, even if script writing is used, a lot of time is required for re-writing or modifying the script according to new test requirements each time, and the quick iterative application development rhythm is difficult to meet. 3. The intelligent degree is low, the traditional method is difficult to operate and control through natural language, and a user cannot drive the mobile device to execute corresponding operation through visual and convenient natural language instructions. For example, a user expects to take a picture by simply turning on a camera and adjusting to a night view mode, and the conventional method cannot directly understand and execute the instruction, so that the user is required to perform a series of complicated manual operations or write complicated scripts, which greatly reduces the user experience, and makes the operation of the mobile device not convenient and intelligent enough. 4. The flexibility is insufficient, the traditional method is difficult to flexibly set timing tasks, and automatic operation and control in the true sense cannot be realized. For example, users want to automatically perform operations such as data backup and system update at specific time, and conventional methods have difficulty meeting the requirement, require users to manually perform operations at specific time, rely on complex timing script settings, have poor universality and portability of scripts, and may need to be rewritten under different devices or environments, thereby further increasing operation difficulty and cost. Disclosure of Invention The invention aims to provide a mobile terminal intelligent agent processing method, a system and equipment based on natural language processing, which realize the intelligent processing of mobile terminal operation based on natural language processing, and the complete closed loop is formed from instruction analysis to script execution and optimization, thereby improving the automation and the intelligent level of mobile terminal operation and solving at least one of the problems in the prior art. In a first aspect, the present invention provides a mobile terminal agent processing method based on natural language processing, where the method specifically includes: Receiving a natural language command input by a user, analyzing the natural language command by using a natural language processing model, and extracting an operation intention and an operation entity; Inquiring a pre-constructed operation knowledge graph based on the operation intention and the operation entity, and determining a similar historical operation script recommended by the multiplexing operation knowledge graph or generating a new operation script by comparing an inquiry result with a preset similarity threshold; Acquiring equipment state and environment information in real time through a context awareness engine, and dynamically adjusting parameters of an operation script to be executed according to the equipment state and the environment information; executing the operation script with the parameters adjusted, collecting feedback data of the execution process and the ex