CN-122024721-A - Ship controller offline voice command control method and system

CN122024721ACN 122024721 ACN122024721 ACN 122024721ACN-122024721-A

Abstract

The embodiment of the specification discloses a ship controller offline voice command control method and system. The method comprises the steps of obtaining a ship environment voice signal, converting the ship environment voice signal into a first digital audio stream, carrying out audio front-end enhancement processing on the first digital audio stream, carrying out wake-up word local off-line detection on a first audio feature based on a local lightweight neural network model, obtaining a second audio feature, carrying out local off-line recognition on the second audio feature through a local command word recognition model, analyzing a natural language instruction into a structural control instruction based on a predefined ship control instruction mapping table, and sending the structural control instruction to a ship controller through a wired serial port communication or Bluetooth low-power wireless communication mode based on a system integration mode of multi-mode instruction forwarding. According to the embodiment of the specification, the background noise and echo interference of the ship can be effectively restrained, and the high recognition accuracy can be maintained.

Inventors

ZHOU JUNWU
HU YULIANG
WANG MINGSHENG

Assignees

宁波恒力达科技有限公司
杭州海兴电力科技股份有限公司

Dates

Publication Date: 20260512
Application Date: 20260204

Claims (10)

1. An off-line voice command control method for a ship controller is characterized by comprising the following steps: Acquiring a ship environment voice signal and converting the ship environment voice signal into a first digital audio stream; performing audio front-end enhancement processing on the first digital audio stream to inhibit ship environmental noise and extract first audio features after enhancement processing; based on a local lightweight neural network model, performing wake-up word local offline detection on the first audio feature to obtain a wake-up word detection result corresponding to the first audio feature; When the wake-up word detection result meets the wake-up word detection condition, switching to a command word recognition mode, wherein the command word recognition mode is used for acquiring a second audio feature, and carrying out local offline recognition on the second audio feature through a local command word recognition model to acquire a natural language instruction corresponding to the second audio feature; analyzing the natural language instruction into a structural control instruction based on a predefined ship control instruction mapping table; based on a system integration mode of multi-mode instruction forwarding, the structured control instruction is sent to a ship controller through a wired serial port communication or Bluetooth low-power consumption wireless communication mode, and the ship controller receives and executes the structured control instruction.
2. The method of claim 1, wherein the audio front-end enhancement processing includes at least echo cancellation and noise suppression processing, wherein performing audio front-end enhancement processing on the first digital audio stream to suppress marine environmental noise and extracting the enhanced first audio feature includes: Acquiring a second digital audio source signal generated by the system and used for driving a loudspeaker to play, wherein the second digital audio source signal is a reference signal used for echo cancellation; The echo cancellation algorithm module determines echo components mixed into the first digital audio stream according to the second digital audio stream through an adaptive filter, and subtracts the echo components from the first digital audio stream to obtain a primary audio signal after echo cancellation; And performing noise suppression processing on the frequency domain representation of the primary audio signal based on a preset noise spectrum model to obtain a noise reduction audio signal, wherein the noise reduction audio signal is a first audio feature after enhancement processing.
3. The method of claim 2, wherein the performing wake-up word local offline detection on the first audio feature based on the local lightweight neural network model, before obtaining the wake-up word detection result corresponding to the first audio feature, includes: acquiring signal characteristics corresponding to the noise reduction audio signal, and inputting the signal characteristics into a pre-trained voice activity detection classifier, wherein the voice activity detection classifier outputs a confidence score representing that the noise reduction audio signal contains human voice; And outputting a voice activity mark and triggering the local offline detection flow of wake-up words for the first audio feature when the noise reduction audio signal is judged to contain voice of a person based on the confidence score.
4. The method of claim 1, wherein the local lightweight neural network model comprises at least WakeNet model and the local command word recognition model comprises at least a MultiNet model.
5. The method according to claim 1, wherein the system integration mode based on the multimode instruction forwarding sends the structured control instruction to the ship controller through a wired serial port communication or a bluetooth low energy wireless communication mode, and the method comprises: when a wired serial port communication mode is selected, the structural control instruction is sent to a corresponding serial port of the ship controller through a universal asynchronous receiving and transmitting transmitter interface; When the Bluetooth low-power wireless communication mode is adopted, wireless connection is established with the ship controller, and the structured control instruction is packaged into an application layer data packet conforming to the Bluetooth protocol and then transmitted to the ship controller.
6. The off-line voice command control system of the ship controller is characterized by comprising a voice control module and the ship controller, wherein the voice control module at least comprises a main control unit and a communication interface unit; The main control unit is used for loading and running the off-line voice command control method of the ship controller according to any one of claims 1 to 5; the communication interface unit at least comprises a wired serial interface and a Bluetooth low-power consumption wireless interface and is used for being in communication connection with the ship controller.
7. The system of claim 6, wherein the master control unit comprises: the audio conversion module is used for acquiring an analog voice signal in a ship environment and converting the analog voice signal into a first digital audio stream; The audio processing module is used for carrying out audio front-end enhancement processing on the first digital audio stream so as to inhibit ship environment noise and extract first audio characteristics after the enhancement processing; The wake-up module is used for carrying out wake-up word local off-line detection on the first audio feature based on a local lightweight neural network model to obtain a wake-up word detection result corresponding to the first audio feature; the recognition module is used for switching to a command word recognition mode when the wake-up word detection result meets the wake-up word detection condition, wherein the command word recognition mode is used for acquiring a second audio feature, and carrying out local offline recognition on the second audio feature through a local command word recognition model to obtain a natural language instruction corresponding to the second audio feature; The instruction analysis module is used for analyzing the natural language instruction into a structural control instruction based on a predefined ship control instruction mapping table, wherein the structural control instruction at least comprises an instruction type and a parameter value; The communication module is used for transmitting the structured control instruction to the ship controller through a wired serial port communication or a Bluetooth low-power consumption wireless communication mode based on a system integration mode of multi-mode instruction forwarding, and the ship controller receives and executes the structured control instruction.
8. The system of claim 6, wherein the hardware of the voice control module is of modular design, the voice control module being a sub-module embedded in the PCB of the marine controller.
9. The system of claim 6, wherein the voice control module is a stand-alone add-on module that interfaces with the vessel controller through a connector.
10. The system of claim 7, wherein the master control unit employs an ESP32-S3 series module, and the audio conversion module comprises a codec employing an ES8311 chip.

Description

Ship controller offline voice command control method and system Technical Field One or more embodiments of the present disclosure relate to the technical field of ship electronic devices, and in particular, to a method and a system for controlling offline voice commands of a ship controller. Background The voice interaction technology is widely applied to the fields of intelligent home and the like due to the convenience. In industrial scenes such as ship control, real-time performance, reliability and offline working capacity of operation are strictly required, so that an offline voice recognition technology becomes a key for realizing safe and efficient man-machine interaction. In the process of ship driving and controlling, an operator needs to continuously observe a complex instrument panel and an external environment while densely operating physical buttons and knobs, and the risk of distraction exists. The voice control is introduced to remarkably improve the operation efficiency and the navigation safety. However, the ship working environment is special, the continuous strong noise such as engine booming, sea wave slapping, wind and rain, running of equipment in the cabin and the like is commonly existed, and the ocean voyage always faces the condition that the network signal is unstable or completely lacks. Therefore, a voice control scheme suitable for a ship must have extremely high performance in terms of offline capability, anti-noise interference capability, recognition reliability, low delay response, and the like. Currently, to implement offline voice control of embedded devices, mainstream technologies include a voice module scheme that generates firmware based on a dedicated voice recognition chip or on-line configuration, and the like. The scheme based on the special voice recognition chip is used for connecting the special chip with the main control MCU through an SPI and other interfaces, and the special chip is recognized by depending on a fixed algorithm solidified in the chip. In addition, the hardware interface, communication protocol and mechanical structure of the voice module scheme for generating the firmware by on-line configuration often do not meet the industrial standard of the ship, and the flexibility of deep integration with complex ship control logic is lacking, and the existing modularized scheme generally lacks a non-invasive integration method without changing the original hardware circuit of the controlled equipment, which forms a main obstacle for intelligent upgrading of old ship control equipment which is huge in quantity and has been operated for many years. There is a need for an offline speech control solution that combines high performance, low development threshold, high flexibility, and support for non-intrusive integration to facilitate intelligent upgrades of marine control systems. Disclosure of Invention The embodiment of the specification provides a ship controller offline voice command control method and system, and the technical scheme is as follows: According to the embodiment of the specification, the method comprises the steps of obtaining a ship environment voice signal, converting the ship environment voice signal into a first digital audio stream, performing audio front-end enhancement processing on the first digital audio stream to inhibit ship environment noise, extracting first audio features after enhancement processing, performing wake-up word local off-line detection on the first audio features based on a local lightweight neural network model to obtain wake-up word detection results corresponding to the first audio features, switching to a command word recognition mode when the wake-up word detection results meet wake-up word detection conditions, enabling the command word recognition mode to be used for obtaining second audio features, performing local off-line recognition on the second audio features through the local command word recognition model to obtain natural language instructions corresponding to the second audio features, analyzing the natural language instructions into structural control instructions based on a predefined ship control instruction mapping table, transmitting the structural control instructions to a ship controller through wired serial port communication or Bluetooth low-power wireless communication mode based on a system integration mode forwarded by the multi-mode, and receiving and executing the structural control instructions by the ship controller. In a second aspect, an embodiment of the present disclosure provides an offline voice command control system for a ship controller, where the voice control module includes at least a main control unit and a communication interface unit, the main control unit is configured to load and operate the offline voice command control method for the ship controller provided in the first aspect, and the communication interface unit includes at least a wired serial interface