CN-121987466-A - Intelligent obstacle avoidance crutch system based on multi-sensor fusion and lightweight AI

CN121987466ACN 121987466 ACN121987466 ACN 121987466ACN-121987466-A

Abstract

The disclosure relates to an intelligent obstacle avoidance crutch system based on multi-sensor fusion and lightweight AI, which is used for solving the problems that the existing intelligent crutch is single in environment sensing dimension, difficult to balance with the accuracy of fall detection and real-time performance, and incapable of seamlessly switching indoor and outdoor navigation scenes. The intelligent obstacle avoidance crutch system with high robustness, low power consumption and self-adaptive environment is constructed through multi-sensor depth fusion and lightweight algorithm design, and can run on an embedded processor of the crutch, so that the travel safety of vision obstacle and old users is facilitated.

Inventors

JIA DONGYAO
WANG SHUO
DU DEQI
CHEN JIEAN

Assignees

北京交通大学

Dates

Publication Date: 20260508
Application Date: 20251223

Claims (10)

1. An intelligent obstacle avoidance crutch system based on multi-sensor fusion and lightweight AI, characterized in that the system operates on an embedded processor of the crutch, comprising: the positioning navigation module is configured to judge whether an outdoor navigation mode or an indoor navigation mode is adopted through the environment and conduct path preliminary planning; The obstacle detection and recognition module is configured to collect ground images by using a camera positioned at the middle upper part of the crutch during the running of a user, recognize the road surface state based on segmentation and semantics of the ground images, judge blind area obstacles of the camera by using ultrasonic waves deployed at the front and at the side of the bottom of the crutch, detect gradient by using an IMU sensor in the crutch, perform comprehensive risk assessment on a running path of the user based on the road surface state, the blind area obstacles and the gradient, and adjust a planned path or perform voice prompt on the user based on the comprehensive risk assessment result.
2. The intelligent obstacle avoidance crutch system of claim 1 wherein the environmental determination comprises detecting the number of visible satellites and the effective point cloud coverage of the camera depth map once per second using the GPS module, noting the number of visible satellites as N_sat, noting the effective point cloud coverage as D_ cov, determining as the outdoor mode if N_sat is greater than or equal to the preset number of satellites, and determining as the indoor mode if N_sat is less than the preset number of satellites and D_ cov > is the preset coverage.
3. The intelligent obstacle avoidance crutch system of claim 1 wherein the outdoor navigation mode adopts GPS and IMU positioning, and comprises the steps of fusing low-frequency but absolute accurate position data of the GPS with high-frequency but accumulated error gesture data of the IMU by a Kalman filter, and outputting a smooth and more accurate real-time position, wherein the low-frequency range is 1 Hz-10 Hz, and the high-frequency range is 50 Hz-400 Hz.
4. The intelligent obstacle avoidance crutch system of claim 1 wherein the indoor navigation mode is to acquire real-time position and construct an environment map by SLAM algorithm based on the image acquired by the camera and depth space distance information.
5. The intelligent obstacle avoidance crutch system of claim 1 wherein the segmentation of the ground image and semantic recognition of the walkable region is performed using a trained Fast-SCNN based on the ground image.
6. The intelligent obstacle avoidance crutch system of claim 1 wherein performing preliminary screening for suspected fall events based on IMU monitoring data comprises calculating a tri-axial combined acceleration of the IMU, calculating an included angle between the Z-axis of the IMU and a gravity acceleration vector, and determining that a suspected fall event occurred if the tri-axial combined acceleration exceeds a first preset threshold or the included angle exceeds a second preset threshold.
7. The intelligent obstacle avoidance crutch system of claim 1 wherein the system further comprises an alarm module, a voice module, and a communication module; the voice module is configured to carry out voice broadcasting on the comprehensive risk assessment result through a loudspeaker, or carry out voice inquiry after confirming falling; The alarm module is configured to send a distress message containing GPS positioning through the communication module when a cancel instruction is not received during voice inquiry; the communication module is configured to send information or make a call, and the information comprises positioning information and an environment image of a user.
8. The intelligent obstacle avoidance crutch system of claim 1 further comprising a fall detection module configured to perform preliminary screening of suspected fall events using IMU monitoring data while the user is traveling, and to input six-axis data of the IMU to a trained one-dimensional CNN model to perform fall event determination when a suspected fall event is determined to occur.
9. A computer-readable storage medium, characterized in that a computer program capable of being loaded and executed by an embedded multi-core processor is stored, the system according to any one of claims 1 to 8.
10. An intelligent obstacle avoidance method is characterized by comprising the following steps: judging whether an outdoor navigation mode or an indoor navigation mode is adopted through the environment, and planning an initial path; When a user travels on an initial planning path, acquiring a ground image by using a camera positioned at the middle upper part of the crutch, identifying the road surface state based on segmentation and semantics of the ground image, judging blind area obstacles of the camera by using ultrasonic waves deployed at the front and the side of the bottom of the crutch, detecting the gradient by using an IMU sensor in the crutch, comprehensively evaluating the risk of the user travelling path based on the road surface state, the blind area obstacles and the gradient, and adjusting the planning path based on the comprehensive risk evaluation result.

Description

Intelligent obstacle avoidance crutch system based on multi-sensor fusion and lightweight AI Technical Field The disclosure relates to computer vision, in particular to an intelligent obstacle avoidance crutch system based on multi-sensor fusion and lightweight AI. Background In the prior art, the intelligent crutch and the auxiliary travel system for the aged with inconvenient actions and the crowd with visual impairment have the defects that (1) the environment perception dimension is single, the existing products mostly only rely on ultrasonic waves to carry out simple distance measurement, and the problems that detection blind areas exist, object attributes cannot be identified and the like. The conventional visual navigation scheme is greatly influenced by illumination conditions, has high calculation amount, and is difficult to realize real-time identification of ground critical dangerous details on low-power-consumption equipment. (2) The fall detection accuracy and the real-time performance are difficult to balance, the traditional threshold method which depends on an inertial sensor (IMU) is extremely easy to misjudge severe daily actions such as rapid bending, sitting and the like as falling, and the introduction of a complex deep learning model can improve the accuracy, but the reasoning delay is too high on the portable embedded equipment with limited resources, so that the real-time requirement of early warning under sudden accidents can not be met. (3) Indoor and outdoor navigation scenes cannot be switched seamlessly, an existing navigation scheme usually uses outdoor GPS positioning or indoor SLAM mapping in isolation, and an adaptive mechanism capable of automatically judging the scenes according to multi-dimensional environment characteristics is lacking, so that navigation service is interrupted when a user enters and exits a building. Disclosure of Invention Aiming at the problems that the environment perception dimension is single, the fall detection accuracy and the real-time are difficult to balance and the indoor and outdoor navigation scenes cannot be switched seamlessly in the prior art, the invention aims to construct an intelligent obstacle avoidance crutch system with high robustness, low power consumption and self-adaption to the environment through the design of a multi-sensor depth fusion and lightweight algorithm so as to improve the travel safety and experience of vision obstacle and old users. The intelligent obstacle avoidance crutch system based on the multi-sensor fusion and the light AI is operated on an embedded processor of the crutch, and comprises a positioning navigation module, an obstacle detection and identification module, a road surface state recognition module, an image recognition module and a comprehensive risk evaluation module, wherein the positioning navigation module is configured to judge whether an outdoor navigation mode or an indoor navigation mode is adopted through the environment to conduct preliminary planning of a path, the obstacle detection and identification module is configured to acquire a ground image by a camera positioned at the middle upper part of the crutch during traveling of a user, judge blind area obstacles of the camera by utilizing ultrasonic waves deployed at the front and the side of the bottom of the crutch based on segmentation and semantic recognition of the ground image, detect gradients by utilizing an IMU sensor in the crutch, conduct comprehensive risk evaluation on a traveling path of the user based on the road surface state, the blind area obstacles and the gradients, and adjust the planned path or conduct voice prompt on the user based on the comprehensive risk evaluation result. In one implementation manner of the above technical solution, the environment judgment includes detecting the number of visible satellites and the coverage rate of the effective point cloud of the camera depth map once every second by using the GPS module, recording the number of visible satellites as N_sat, recording the coverage rate of the effective point cloud as D_ cov, judging as an outdoor mode if N_sat is greater than or equal to the preset number of satellites, and judging as an indoor mode if N_sat is less than the preset number of satellites and D_ cov is greater than the preset coverage rate. In one embodiment of the above technical scheme, the outdoor navigation mode adopts GPS and IMU positioning, and comprises the steps of fusing low-frequency but absolute accurate position data of the GPS with high-frequency but accumulated error posture data of the IMU by using a Kalman filter, and outputting a smooth and more accurate real-time position, wherein the low-frequency range is 1 Hz-10 Hz, and the high-frequency range is 50 Hz-400 Hz. In an implementation manner of the above technical solution, the indoor navigation mode is to acquire a real-time position and construct an environment map by using a SLAM algorithm based on an im