Search

CN-122024737-A - Low-delay data communication method, system, equipment and medium for emotion robot

CN122024737ACN 122024737 ACN122024737 ACN 122024737ACN-122024737-A

Abstract

The invention provides a low-delay data communication method, a system, equipment and a medium of an emotion robot, wherein the method comprises the steps of creating a non-circular ring-shaped queue and a shared message object pool, collecting multi-mode data and the like to process to obtain a visual/emotion recognition event, a voice front-end event, a touch/environment interaction event and a system running state event, storing event content into the shared message object pool and storing an event ID into the non-circular ring-shaped queue, detecting the event ID to be processed in the non-circular ring-shaped queue in real time by adopting a priority scheduler, reading the corresponding event content from the shared message object pool through the event ID, processing according to the event priority in sequence, judging whether the event belongs to a reflection path when processing the event, if so, executing corresponding decision and feedback optimization, otherwise, transmitting the event content to a subscription layer to process according to a dialogue path, and executing corresponding strategy and feedback optimization. The invention can effectively solve the problems of delay, jitter and the like in the transmission process of the existing emotion robot.

Inventors

  • WANG XUEZHEN
  • TU CHAO

Assignees

  • 福建星网智慧科技有限公司

Dates

Publication Date
20260512
Application Date
20251210

Claims (10)

  1. 1. A low-delay data communication method of an emotion robot is characterized by comprising the following steps: S1, creating a lock-free ring-shaped queue and a shared message object pool; S2, acquiring multi-mode data and system operation data and processing the multi-mode data and the system operation data to obtain a visual/emotion recognition event, a voice front-end event, a touch/environment interaction event and a system operation state event; step S3, storing the event content of the event obtained by processing into a shared message object pool, and storing the corresponding event ID into an unclocked ring-shaped queue; s4, detecting an event ID to be processed in the non-lock ring queue in real time by adopting a priority scheduler, reading corresponding event content from a shared message object pool through the event ID, and processing in sequence according to the priority of the event content; And S5, judging whether the event belongs to a reflection path when the event is processed, if so, executing corresponding decision and feedback optimization, otherwise, transmitting the event to a subscription layer for processing according to a dialogue path, and executing corresponding strategy and feedback optimization.
  2. 2. The method of claim 1, wherein the multi-modal data is transmitted via the message bus with the event as a minimal transmission unit.
  3. 3. The method for communicating data with a low latency emotion robot according to claim 1, wherein said step S2 comprises: collecting multi-modal data and system operation data, wherein the multi-modal data comprises voice signals, image signals, touch signals and environment signals; preprocessing the acquired multi-mode data; The method comprises the steps of carrying out abstract processing on an event, taking a VAD paragraph as a semantic natural boundary, sampling visual, tactile and environmental modes according to a fixed frequency, and aggregating all modes in a preset time window to form a layer-by-layer abstract mechanism from signal characteristics to time events and then to semantic events, wherein the semantic events comprise visual/emotion recognition events, voice front-end events and touch/environmental interaction events; carrying out labeling processing on the event, and setting a standardized event label for the semantic event, wherein the standardized event label comprises an event type, a refinement label, a confidence level, duration, an event initiator, an event object, intention and time; The system operation data is system output data and is directly packaged into system operation state events.
  4. 4. The method for communicating data with a low latency emotion robot as recited in claim 1, wherein step S4 further comprises: if an event is not completed within a preset deadline, the priority scheduler records a timeout event and performs retransmission or performs other priority events according to preset rules.
  5. 5. The method for low-latency data communications of emotion robot according to claim 1, wherein in step S5, the reflection path is a rule-based response path for processing system running state events, and the dialogue path is a semantically and contextually relevant response path for processing semantic events identified by a large model.
  6. 6. A low-delay data communication system of an emotion robot is characterized in that the system comprises: the storage construction module is used for creating a lock-free ring-shaped queue and a shared message object pool; The event construction module is used for acquiring and processing the multi-mode data and the system operation data to obtain a visual/emotion recognition event, a voice front-end event, a touch/environment interaction event and a system operation state event; the data storage module is used for storing the event content of the event obtained by processing into a shared message object pool and storing the corresponding event ID into a non-lock ring queue; The data calling module is used for detecting the event ID to be processed in the non-lock ring queue in real time by adopting the priority scheduler, reading the corresponding event content from the shared message object pool through the event ID, and processing the event content in sequence according to the priority of the event content; and the event processing and transmitting module is used for judging whether the event belongs to a reflection path when the event is processed, if so, executing corresponding decision and feedback optimization, otherwise, transmitting the event to a subscription layer for processing according to a dialogue path, and executing corresponding strategy and feedback optimization.
  7. 7. The low-latency data communications system of an emotion robot of claim 6, wherein said multimodal data is transmitted over a message bus with events as a minimal transmission unit.
  8. 8. The emotion robot low-delay data communication system of claim 6, wherein said event construction module comprises: collecting multi-modal data and system operation data, wherein the multi-modal data comprises voice signals, image signals, touch signals and environment signals; preprocessing the acquired multi-mode data; The method comprises the steps of carrying out abstract processing on an event, taking a VAD paragraph as a semantic natural boundary, sampling visual, tactile and environmental modes according to a fixed frequency, and aggregating all modes in a preset time window to form a layer-by-layer abstract mechanism from signal characteristics to time events and then to semantic events, wherein the semantic events comprise visual/emotion recognition events, voice front-end events and touch/environmental interaction events; carrying out labeling processing on the event, and setting a standardized event label for the semantic event, wherein the standardized event label comprises an event type, a refinement label, a confidence level, duration, an event initiator, an event object, intention and time; The system operation data is system output data and is directly packaged into system operation state events.
  9. 9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 4 when the program is executed by the processor.
  10. 10. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any one of claims 1 to 4.

Description

Low-delay data communication method, system, equipment and medium for emotion robot Technical Field The invention relates to the technical field of artificial intelligence and robots, in particular to a low-delay data communication method, a system, equipment and a medium for an emotion robot. Background The desktop emotion accompanying robot needs to construct an end-in full-closed-loop interaction of 'multi-mode perception (voice/vision/touch/environment) →voice recognition/intention recognition→emotion/behavior/voice/action feedback' under the scene of limited computational power and power consumption. The interaction naturalness is highly related to the end-to-end delay, namely, the daily dialogue needs to realize response feedback within 1-3 seconds, and non-language instant feedback (such as expression response) needs to be controlled within 400-500 ms, so that the interaction consistency and immersion sense are ensured. From finishing the data acquisition and input of the sensor, to the data processing of the robot and finishing the intention recognition, the data are converted into corresponding actions or interactive output, and the time delay design and budget of each link are required. The data processing and intention recognition part relates to the triggering of a plurality of events and the flow of data transmission, and the traditional scheme of multithreading parallel, shared memory and mutual exclusion lock has low delay but high coupling, and can cause the problems of priority reversal, deadlock, unexpected jitter and the like after programming. Aiming at the problems, the invention provides an emotion robot communication mechanism capable of realizing low-coupling low-delay data transmission, and the robot interaction experience of a user is improved Disclosure of Invention The invention aims to solve the technical problems of delay, jitter and the like in the conventional emotion robot data transmission process. In a first aspect, the present invention provides a low-latency data communication method for an emotion robot, comprising the steps of: S1, creating a lock-free ring-shaped queue and a shared message object pool; S2, acquiring multi-mode data and system operation data and processing the multi-mode data and the system operation data to obtain a visual/emotion recognition event, a voice front-end event, a touch/environment interaction event and a system operation state event; step S3, storing the event content of the event obtained by processing into a shared message object pool, and storing the corresponding event ID into an unclocked ring-shaped queue; s4, detecting an event ID to be processed in the non-lock ring queue in real time by adopting a priority scheduler, reading corresponding event content from a shared message object pool through the event ID, and processing in sequence according to the priority of the event content; And S5, judging whether the event belongs to a reflection path when the event is processed, if so, executing corresponding decision and feedback optimization, otherwise, transmitting the event to a subscription layer for processing according to a dialogue path, and executing corresponding strategy and feedback optimization. Further, the multi-mode data uses the event as the minimum transmission unit and is transmitted through the message bus. Further, the step S2 specifically includes: collecting multi-modal data and system operation data, wherein the multi-modal data comprises voice signals, image signals, touch signals and environment signals; preprocessing the acquired multi-mode data; The method comprises the steps of carrying out abstract processing on an event, taking a VAD paragraph as a semantic natural boundary, sampling visual, tactile and environmental modes according to a fixed frequency, and aggregating all modes in a preset time window to form a layer-by-layer abstract mechanism from signal characteristics to time events and then to semantic events, wherein the semantic events comprise visual/emotion recognition events, voice front-end events and touch/environmental interaction events; carrying out labeling processing on the event, and setting a standardized event label for the semantic event, wherein the standardized event label comprises an event type, a refinement label, a confidence level, duration, an event initiator, an event object, intention and time; The system operation data is system output data and is directly packaged into system operation state events. Further, the step S4 further includes: if an event is not completed within a preset deadline, the priority scheduler records a timeout event and performs retransmission or performs other priority events according to preset rules. Further, in the step S5, the reflection path is a rule-based response path for processing a system running state event, and the dialogue path is a semantical and context-related response path for processing a semantic event identified by a large model.