CN-122027465-A - Auxiliary operation and maintenance method, program product, electronic equipment and storage medium

CN122027465ACN 122027465 ACN122027465 ACN 122027465ACN-122027465-A

Abstract

The application provides an operation and maintenance assisting method, a program product, electronic equipment and a storage medium, wherein the method comprises the steps of obtaining an operation and maintenance interface accessed by a user, carrying out multi-mode identification on the operation and maintenance interface, and generating multi-mode fusion characteristics; the multi-mode identification comprises visual identification and element structure identification, prompt content corresponding to an operation and maintenance interface is generated based on multi-mode fusion characteristics, user context information and an operation and maintenance knowledge base, the prompt content is displayed to a user, the user context information comprises user capability images determined based on user historical operation data, an assistance request of the user is obtained, the assistance request comprises an assistance mode, guide information corresponding to the assistance mode is generated according to the multi-mode fusion characteristics and the user context information, and the guide information is used for assisting the user in operation and maintenance operation. The users with different experiences can obtain more proper support, the provided information has more pertinence, and the user experience is improved.

Inventors

ZHAO ZHONGWEI
JIANG WEI
Yu hongda

Assignees

北京天融信网络安全技术有限公司
北京天融信科技有限公司
北京天融信软件有限公司

Dates

Publication Date: 20260512
Application Date: 20260202

Claims (12)

1. A method of assisting in operation and maintenance, comprising: Acquiring an operation and maintenance interface accessed by a user, and carrying out multi-modal identification on the operation and maintenance interface to generate multi-modal fusion characteristics, wherein the multi-modal identification comprises visual identification and element structure identification; Generating prompt content corresponding to the operation and maintenance interface based on the multi-mode fusion characteristics, user context information and an operation and maintenance knowledge base, and displaying the prompt content to the user; Acquiring an assistance request of a user, wherein the assistance request comprises an assistance mode; and generating guide information corresponding to the assistance mode according to the multi-mode fusion characteristics and the user context information, wherein the guide information is used for assisting a user in carrying out operation and maintenance operation.
2. The method of claim 1, wherein performing multi-modal identification on the operation and maintenance interface to generate multi-modal fusion features comprises: Identifying visual elements in the operation and maintenance interface by using a trained convolutional neural network, and extracting visual feature vectors; Analyzing the document object model tree of the operation and maintenance interface through a DOM analysis channel, and extracting structural feature vectors of interface elements; The method comprises the steps of acquiring a context feature vector of the operation and maintenance interface, wherein the context feature vector is used for reflecting the operation environment of the operation and maintenance interface; and carrying out fusion processing on the visual feature vector, the structural feature vector and the context feature vector to obtain the multi-mode fusion feature.
3. The method of claim 2, wherein obtaining the contextual feature vector of the operation and maintenance interface comprises: acquiring context information reflecting the operation environment of the operation and maintenance interface, wherein the context information comprises at least one of a user role, a current operation sequence and an interface state; the contextual feature vector is generated based on the contextual information.
4. The method of claim 1, wherein prior to generating the reminder content corresponding to the operation and maintenance interface based on the multimodal fusion feature, user context information, and operation and maintenance knowledge base, the method further comprises: the user history operation data comprises an operation sequence, an operation result and operation time consumption; And determining the user capacity image according to the user history operation data and the preset weight.
5. The method of claim 1, wherein generating the reminder content corresponding to the operation and maintenance interface based on the multimodal fusion feature, the user context information, and the operation and maintenance knowledge base comprises: the multi-mode fusion features are used as semantic indexes, the operation and maintenance knowledge base is queried, and planning information corresponding to the operation and maintenance interface is retrieved, wherein the planning information comprises concept interpretation, parameter range, constraint conditions and risk prompt information; Determining the personalized style of the prompt content according to the user context information; and sorting the planning information according to the personalized style to generate the prompt content.
6. The method of claim 1, wherein the assistance modes include at least a first assistance mode and a second assistance mode; generating the guiding information corresponding to the assistance mode comprises the following steps: If the first assistance mode is the first assistance mode, searching matched dynamic demonstration contents according to the multi-mode fusion characteristics and the user context information, wherein the dynamic demonstration contents are interactive micro videos which are generated according to real-time interface states and reflect real configuration flows; if the operation and maintenance interface is in the second assistance mode, analyzing a configuration form of the operation and maintenance interface, identifying an input parameter list and a deduction parameter list, generating a parameter collection interface to guide the user to input parameters based on the input parameter list, and automatically generating parameters needed in the deduction parameter list based on the parameters input by the user, the multi-mode fusion characteristics and the operation and maintenance knowledge base.
7. The method of claim 6, wherein after automatically generating the parameters needed in the list of derived parameters, the method further comprises: Verifying the filled parameters through a verification engine, wherein the verification engine is integrated with the operation and maintenance knowledge base; and under the condition that the verification result characterization is not passed, generating warning information and performing rollback operation.
8. The method of claim 6, wherein after generating a parameter collection interface to direct the user to input parameters based on the list of input parameters, the method further comprises: Invoking a prerecorded abstract flow description, wherein the abstract flow description contains semantic operation intention; and instantiating the abstract flow description into an operation sequence according to the parameters input by the user and the parameter mapping map so as to generate the parameters required in the deduction parameter list.
9. The method according to claim 1, wherein the method further comprises: Acquiring interactive behavior data in the operation process of a user based on the guide information; And optimizing a fusion model used for generating the multi-mode fusion feature by using successful operation in the interaction behavior data as a positive feedback sample.
10. A computer program product comprising computer program instructions which, when executed by a processor, perform the method of any of claims 1 to 9.
11. An electronic device comprising a processor and a memory, the memory storing computer program instructions that, when executed by the processor, perform the method of any one of claims 1 to 9.
12. A computer readable storage medium, characterized in that it has stored thereon computer program instructions which, when executed by a processor, perform the method according to any of claims 1 to 9.

Description

Auxiliary operation and maintenance method, program product, electronic equipment and storage medium Technical Field The present application relates to the field of computer technologies, and in particular, to a method for assisting operation and maintenance, a program product, an electronic device, and a storage medium. Background In modern network operation and maintenance environments, a Web management interface has become a core platform for operation and maintenance personnel to perform device configuration, service management and application release. The operation and maintenance interface generally comprises a large number of configuration options and multi-level operation menus, and in the case that the operation and maintenance operation of a user is problematic, the prior art can only provide fixed information, and can not provide more targeted assistance information for the user. Disclosure of Invention An embodiment of the application aims to provide an operation assisting method, a program product, electronic equipment and a storage medium, which are used for improving the problems. The embodiment of the application provides an operation and maintenance assisting method, which comprises the steps of obtaining an operation and maintenance interface accessed by a user, carrying out multi-mode identification on the operation and maintenance interface to generate multi-mode fusion characteristics, wherein the multi-mode identification comprises visual identification and element structure identification, generating prompt contents corresponding to the operation and maintenance interface based on the multi-mode fusion characteristics, user context information and an operation and maintenance knowledge base, displaying the prompt contents to the user, wherein the user context information comprises user capability images determined based on user historical operation data, obtaining an assistance request of the user, the assistance request comprises an assistance mode, generating guide information corresponding to the assistance mode according to the multi-mode fusion characteristics and the user context information, and the guide information is used for assisting the user in operation and maintenance operation. In the implementation process, through the combination of vision and multi-mode identification of the code structure, operation and maintenance interfaces of various styles can be accurately understood, and a spoken prompt which is fit with the context is generated, so that the efficiency and the accuracy of operation and maintenance operation are improved. According to the multi-mode fusion characteristics and the user context information, the guiding information corresponding to the assisting mode is generated, the prompting and assisting depth is dynamically adjusted according to the historical operation level of the user, so that users with different experiences can obtain more proper support, the provided information is more targeted, and the user experience is improved. Optionally, in the embodiment of the application, the multi-modal identification is performed on the operation and maintenance interface to generate the multi-modal fusion feature, which comprises the steps of utilizing a trained convolutional neural network to identify visual elements in the operation and maintenance interface, extracting visual feature vectors, analyzing a document object model tree of the operation and maintenance interface through a DOM analysis channel, extracting structural feature vectors of the interface elements, acquiring context feature vectors of the operation and maintenance interface, wherein the context feature vectors are used for reflecting the operation environment of the operation and maintenance interface, and carrying out fusion processing on the visual feature vectors, the structural feature vectors and the context feature vectors to obtain the multi-modal fusion feature. In the implementation process, the visual recognition and DOM structure analysis are combined, so that the system can not only 'see' the patterns of the interface elements, but also understand the roles of the interface elements in the page function, thereby overcoming the limitation of a single recognition mode and having good adaptability to various types of operation and maintenance interfaces. The context features are fused, so that the understanding of the system is not isolated, the user identity and the operation scene are combined, and the generated multi-mode fusion features are used as uniform semantic representation, so that the subsequent prompt content generation and the assistance operation guidance are more accurate and reliable. Optionally, in the embodiment of the application, the method for obtaining the context feature vector of the operation and maintenance interface comprises the steps of obtaining context information reflecting the operation environment of the operation and maintenance interface,