CN-122022827-A - Intelligent customer service interaction method, device, equipment and medium based on image recognition

CN122022827ACN 122022827 ACN122022827 ACN 122022827ACN-122022827-A

Abstract

The application relates to the technical field of artificial intelligence, and provides an intelligent customer service interaction method, device, equipment and medium based on image recognition, wherein the method comprises the steps of obtaining multi-mode data uploaded by a user in the interaction process with after-sales customer service; the multi-mode data comprises image data and text data associated with the image data, the multi-mode data is input into a target recognition model to obtain a solution output by the target recognition model, the target recognition model is configured with a knowledge database in the field of household appliances, and the target recognition model is used for extracting characteristic information from the multi-mode data and matching corresponding solutions from the knowledge database based on the characteristic information. The method and the system can process multi-mode data, a user does not need to convert image information into characters any more, the target recognition model is configured with a knowledge database in the field of household appliances, and corresponding solutions can be matched from the database based on the extracted characteristic information, so that the accuracy rate and the user experience of after-sale diagnosis of the household appliances are improved.

Inventors

LI FUQING
JIANG JIAHAO
CHENG CHENG
LIN YANTING
ZHANG SONG
LI JUNDA
HU ZIJIAN
HU WENJU
Deng Mengshi
SUN DAN

Assignees

宁波美美家园电器服务有限公司

Dates

Publication Date: 20260512
Application Date: 20251229

Claims (10)

1. An intelligent customer service interaction method based on image recognition is characterized by comprising the following steps: The method comprises the steps of obtaining multi-modal data uploaded by a user in the interaction process with after-sales customer service, wherein the multi-modal data comprises image data and text data associated with the image data; inputting the multi-mode data into a target recognition model to obtain a solution output by the target recognition model; the target recognition model is used for extracting characteristic information from the multi-mode data and matching corresponding solutions from the knowledge database based on the characteristic information.
2. The intelligent customer service interaction method based on image recognition according to claim 1, wherein, The image data comprises at least one of a product physical picture, a fault phenomenon picture and a product nameplate picture which are shot by a user; And/or the characteristic information comprises at least one of category information, model information and fault code information in the field of household appliances.
3. The intelligent customer service interaction method based on image recognition according to claim 2, wherein when the feature information is fault code information, the solution for obtaining the target recognition model output comprises: and matching self-help obstacle removing solutions aiming at the fault code information from the knowledge database.
4. The intelligent customer service interaction method based on image recognition according to claim 2, wherein when the characteristic information is category information, the solution for obtaining the target recognition model output comprises: Matching a list of recommended questions for the category information from the knowledge database.
5. The intelligent customer service interaction method based on image recognition according to claim 2, wherein when the feature information is model information, the solution for obtaining the target recognition model output comprises: Product parameter information for the model information is matched from the knowledge database.
6. The intelligent customer service interaction method based on image recognition according to any one of claims 1 to 5, wherein after the multi-modal data is input into the target recognition model, further comprising: Preprocessing the image data; Acquiring historical dialogue context based on the image data between a user and the after-sales customer service in the text data; the feature information is determined based on the preprocessed image data and the historical dialog context.
7. The intelligent customer service interaction method based on image recognition according to any one of claims 1 to 5, wherein the knowledge database comprises a home appliance knowledge graph, wherein the home appliance knowledge graph comprises a plurality of nodes and relationship edges, wherein the nodes at least comprise product entity nodes, fault entity nodes and solution entity nodes; Said matching corresponding solutions from said knowledge database based on said characteristic information comprises: Mapping the characteristic information into a node in the home appliance knowledge graph; determining a solution entity node by performing path derivation in the home appliance knowledge graph; outputting a solution corresponding to the solution entity node.
8. An intelligent customer service interaction device based on image recognition, which is characterized by comprising: The system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring multi-mode data uploaded by a user in the interaction process with after-sales customer service, and the multi-mode data comprises image data and text data associated with the image data; The processing module is used for inputting the multi-mode data into a target recognition model to obtain a solution output by the target recognition model; the target recognition model is used for extracting characteristic information from the multi-mode data and matching corresponding solutions from the knowledge database based on the characteristic information.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and running on the processor, wherein the processor implements the intelligent customer service interaction method based on image recognition as claimed in any one of claims 1 to 7 when executing the computer program.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the intelligent customer service interaction method based on image recognition as claimed in any one of claims 1 to 7.

Description

Intelligent customer service interaction method, device, equipment and medium based on image recognition Technical Field The application relates to the technical field of artificial intelligence, in particular to an intelligent customer service interaction method, device, equipment and medium based on image recognition. Background In modern business environments, after-market services are a key element in improving customer satisfaction. Especially in the field of household appliances, the technology of household appliances is accelerated in iteration, and the problems encountered by users in the use process also show a tendency of diversification and specialization. Currently, the after-market customer service mode is mainly composed of an online manual customer service and an intelligent customer service robot. The intelligent customer service robot mainly relies on text processing technology, only can process text information input by a user, and lacks effective recognition and understanding capability for images uploaded by the user, such as appearance diagrams of household appliances, fault diagrams of household appliances and the like, so that the user still needs to expend a great deal of energy to convert visual information into text description, and the single interaction mode reduces user experience. When the intelligent customer service robot cannot meet the requirements, users seek on-line manual customer service assistance instead, new problems are faced, namely, due to the limited number of customer service positions, the users often need to wait in a queue for a long time in peak hours, and the user experience is seriously affected. Moreover, the professional level, experience and responsibility of the artificial customer service are uneven, different artificial customer service may give different solutions and treatment schemes for the same problem, and standardization and consistency of service quality cannot be ensured. Disclosure of Invention The present application is directed to solving at least one of the technical problems existing in the related art. Therefore, the intelligent customer service interaction method based on image recognition can efficiently and accurately process after-sales problems of household appliances containing complex visual information, and improves diagnosis accuracy and after-sales experience of users. The application further provides an intelligent customer service interaction device based on image recognition, electronic equipment and a storage medium. The intelligent customer service interaction method based on image recognition comprises the steps of obtaining multi-mode data uploaded by a user in the interaction process with after-sales customer service, wherein the multi-mode data comprise image data and text data associated with the image data, inputting the multi-mode data into a target recognition model to obtain a solution output by the target recognition model, wherein the target recognition model is configured with a knowledge database in the field of household appliances, and is used for extracting feature information from the multi-mode data and matching corresponding solutions from the knowledge database based on the feature information. According to the intelligent customer service interaction method based on image recognition, the multi-mode data uploaded by the user can be obtained, the image data are included, the image can be directly recognized and processed, the user does not need to convert image information into characters, the information input efficiency is greatly improved, the target recognition model is provided with a knowledge database in the field of household appliances, corresponding solutions can be matched from the knowledge database based on extracted characteristic information, the expertise and the accuracy of the solutions can be guaranteed, the problem solving quality is improved, and the accuracy and the user experience of after-sale diagnosis of the household appliances are further improved. According to one embodiment of the application, the image data comprises at least one of a product physical picture, a fault phenomenon picture and a product nameplate picture taken by a user, and/or the characteristic information comprises at least one of category information, model information and fault code information of the household appliance field. According to one embodiment of the application, when the characteristic information is fault code information, the solution output by the target recognition model is obtained, and the self-help obstacle removing solution for the fault code information is matched from the knowledge database. According to one embodiment of the application, when the characteristic information is category information, a solution of outputting the target recognition model is obtained, which comprises matching a recommendation problem list aiming at the category information from a knowledge database.