Search

CN-122018702-A - Accompanying robot interaction control method and device, electronic equipment and storage medium

CN122018702ACN 122018702 ACN122018702 ACN 122018702ACN-122018702-A

Abstract

The application relates to the technical field of intelligent accompanying robot interaction control, in particular to an accompanying robot interaction control method, an accompanying robot interaction control device, electronic equipment and a storage medium, which comprise the steps of synchronously acquiring a touch signal, a voice signal and a visual signal of a user to an accompanying robot to obtain a tri-modal signal; preprocessing a tri-modal signal, extracting tri-modal characteristics, fusing the tri-modal characteristics through a pre-constructed dynamic weight distribution model to generate a cooperative characteristic vector, generating an action and voice control instruction according to the cooperative characteristic vector, and controlling the accompanying robot based on the action control instruction and the voice control instruction so as to enable the accompanying robot to interact with a user. Therefore, the problems of feedback machinery, emotion rupture and poor scene adaptation stress of the accompanying robot caused by isolated processing of touch, voice and visual signals are solved, personification and high-adaptation interactive control are realized, various accompanying robot terminals are adapted, and real-time performance and stability are ensured.

Inventors

  • JING WEN
  • LI JIANGYONG
  • YU BO

Assignees

  • 深圳市明日实业有限责任公司

Dates

Publication Date
20260512
Application Date
20260410

Claims (10)

  1. 1. The accompanying robot interactive control method is characterized by comprising the following steps of: Synchronously acquiring a touch signal, a voice signal and a visual signal of a user to the accompanying robot to obtain a tri-modal signal; Preprocessing the tri-modal signals, extracting tri-modal characteristics, and fusing the tri-modal characteristics through a pre-constructed dynamic weight distribution model to generate a collaborative characteristic vector; Generating an action control instruction and a voice control instruction for the accompanying robot according to the cooperative feature vector; and controlling the accompanying robot based on the action control instruction and the voice control instruction so as to enable the accompanying robot to interact with the user.
  2. 2. The method of claim 1, wherein generating motion control instructions and voice control instructions for the companion robot based on the collaborative feature vector comprises: Matching the collaborative feature vector with a pre-constructed three-mode collaborative mapping rule base to obtain an action control instruction and a voice control instruction corresponding to the current combination relation, wherein the three-mode collaborative mapping rule base is a dynamic iterative rule base, and the multi-dimensional combination relation of touch features, voice features and visual features and the action control instruction and the voice control instruction corresponding to each combination relation are prestored in the rule base; Based on the three-mode characteristics and the distance parameters between the accompanying robot and the user, judging whether to trigger the active approaching action of the accompanying robot; if the trigger is triggered, the action parameters of the active approaching action are added in the action control instruction.
  3. 3. The method according to claim 2, further comprising, after the generating of the motion control instruction and the voice control instruction for the companion robot according to the cooperative feature vector: acquiring current visual scene characteristics; optimizing action parameters in the action control instruction according to the visual scene characteristics, and optimizing voice parameters in the voice control instruction.
  4. 4. The method of claim 2, wherein preprocessing the trimodal signals and extracting trimodal features, and fusing the trimodal features through a pre-constructed dynamic weight distribution model, generating collaborative feature vectors, comprises: Extracting the touch position, the touch strength and the touch time sequence of the touch signal to obtain touch characteristics; Extracting semantic intention and voice emotion of the voice signal to obtain voice characteristics; extracting the user gesture, the user facial emotion and the scene type of the visual signal to obtain visual characteristics; Based on an optimized D-S evidence theory and the signal credibility of the tri-modal signals, constructing the dynamic weight distribution model; And carrying out dynamic fusion processing on the touch feature, the voice feature and the visual feature based on the dynamic weight distribution model to generate the cooperative feature vector.
  5. 5. The method of claim 4, wherein after the controlling the companion robot based on the motion control instruction and the voice control instruction to interact the companion robot with the user, further comprising: Acquiring interactive feedback data of the accompanying robot and the user; according to the interactive feedback data, carrying out iterative optimization processing on the three-mode collaborative mapping rule base; The interactive feedback data comprise positive feedback data, negative feedback data and neutral feedback data of a user.
  6. 6. The method for controlling interaction of a companion robot according to claim 5, wherein the weight value range of the touch feature in the dynamic weight distribution model is 0.3-0.5, the weight value range of the voice feature is 0.2-0.4, and the weight value range of the visual feature is 0.2-0.4; when the credibility of the visual features is more than or equal to 0.8, the weight of the visual features is adjusted to 0.4, and the weights of the touch features and the voice features are both 0.3; When the touch signal is continuously and stably acquired for more than or equal to 3 times, the touch characteristic weight is adjusted to 0.5, and the weights of the voice characteristic and the visual characteristic are both 0.25.
  7. 7. An interactive control device for a companion robot, comprising: the acquisition module is used for synchronously acquiring a touch signal, a voice signal and a visual signal of a user to the accompanying robot to obtain a tri-modal signal; The fusion module is used for preprocessing the tri-modal signals, extracting tri-modal characteristics, and fusing the tri-modal characteristics through a pre-constructed dynamic weight distribution model to generate a collaborative characteristic vector; The generation module is used for generating an action control instruction and a voice control instruction for the accompanying robot according to the cooperative feature vector; And the control module is used for controlling the accompanying robot based on the action control instruction and the voice control instruction so as to enable the accompanying robot to interact with the user.
  8. 8. The apparatus of claim 7, wherein the generating module is specifically configured to: Matching the collaborative feature vector with a pre-constructed three-mode collaborative mapping rule base to obtain an action control instruction and a voice control instruction corresponding to the current combination relation, wherein the three-mode collaborative mapping rule base is a dynamic iterative rule base, and the multi-dimensional combination relation of touch features, voice features and visual features and the action control instruction and the voice control instruction corresponding to each combination relation are prestored in the rule base; Based on the three-mode characteristics and the distance parameters between the accompanying robot and the user, judging whether to trigger the active approaching action of the accompanying robot; if the trigger is triggered, the action parameters of the active approaching action are added in the action control instruction.
  9. 9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method according to any one of claims 1 to 6 when the computer program is executed.
  10. 10. A computer readable storage medium for storing a computer program, the computer program being loaded by a processor to perform the steps of the method of any one of claims 1 to 6.

Description

Accompanying robot interaction control method and device, electronic equipment and storage medium Technical Field The application relates to the technical field of intelligent accompanying robot interaction control, in particular to an accompanying robot interaction control method, an accompanying robot interaction control device, electronic equipment and a storage medium. Background Along with the iterative upgrade of emotion accompanying technology, the accompanying robot is gradually upgraded from a single entertainment carrier to an interaction partner with emotion perception and feedback capability, the core requirement of a user is changed from passive response single instruction to active adaptation, multi-sense resonance and scene interaction, and the short board of the prior art in the multi-mode collaborative decision level becomes a core bottleneck for restricting the interactive experience upgrade. The conventional accompanying robot interaction control technology generally has the essential defect of lack of cooperative capability, and cannot construct a multi-sense linkage mechanism conforming to the real pet interaction logic. Disclosure of Invention The application provides a companion robot interaction control method, a companion robot interaction control device, electronic equipment and a storage medium, which can solve at least one technical problem in the background technology to a certain extent. In order to achieve the above purpose, the application adopts the following technical scheme: in a first aspect, there is provided a companion robot interaction control method, the method comprising: Synchronously acquiring a touch signal, a voice signal and a visual signal of a user to the accompanying robot to obtain a tri-modal signal; Preprocessing the tri-modal signals, extracting tri-modal characteristics, and fusing the tri-modal characteristics through a pre-constructed dynamic weight distribution model to generate a collaborative characteristic vector; Generating an action control instruction and a voice control instruction for the accompanying robot according to the cooperative feature vector; and controlling the accompanying robot based on the action control instruction and the voice control instruction so as to enable the accompanying robot to interact with the user. In a second aspect, there is provided an interactive control device for a companion robot, including: the acquisition module is used for synchronously acquiring a touch signal, a voice signal and a visual signal of a user to the accompanying robot to obtain a tri-modal signal; The fusion module is used for preprocessing the tri-modal signals, extracting tri-modal characteristics, and fusing the tri-modal characteristics through a pre-constructed dynamic weight distribution model to generate a collaborative characteristic vector; The generation module is used for generating an action control instruction and a voice control instruction for the accompanying robot according to the cooperative feature vector; And the control module is used for controlling the accompanying robot based on the action control instruction and the voice control instruction so as to enable the accompanying robot to interact with the user. In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the method for controlling interaction of a companion robot according to any one of the first aspects when the processor executes the computer program. In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for controlling interaction of a companion robot according to any one of the first aspects. In a fifth aspect, an embodiment of the present application provides a computer program product, which when run on an electronic device, causes the electronic device to execute the companion robot interaction control method according to any one of the first aspects. It will be appreciated that the advantages of the second to fifth aspects may be found in the relevant description of the first aspect, and are not described here again. In the embodiment of the application, firstly, a touch signal, a voice signal and a visual signal of a partner robot by a user are synchronously acquired to obtain a tri-modal signal, then the tri-modal signal is preprocessed and a tri-modal feature is extracted, the tri-modal feature is fused through a pre-constructed dynamic weight distribution model to generate a cooperative feature vector, then a motion control instruction and a voice control instruction of the partner robot are generated according to the cooperative feature vector, and then the partner robot is controlled based on the motion control instruction and the voice co