CN-121999619-A - Traffic participant road condition early warning method and system based on smart phone

CN121999619ACN 121999619 ACN121999619 ACN 121999619ACN-121999619-A

Abstract

The invention provides a traffic participant road condition early warning method and system based on a smart phone, and relates to the technical field of active safety. The method comprises the steps of collecting video streams through a camera of a smart phone, synchronously obtaining gyroscope, accelerometer, ambient light sensor and GPS data, identifying a user motion state and an ambient scene based on the sensor data, carrying out target detection, semantic segmentation and distance estimation on video frames by adopting a lightweight neural network, carrying out risk assessment according to target types, distances and user motion speeds and generating an early warning instruction, sending a multi-mode warning to a user through a voice, vibration, vision or interface switching mode, and dynamically adjusting analysis frequency and power consumption modes according to environment complexity. The intelligent mobile phone and the method fully utilize the existing hardware of the intelligent mobile phone, do not need additional equipment, realize low-cost and high-popularity real-time environment sensing and active early warning, and remarkably improve the trip safety of traffic participants such as walking, riding and the like.

Inventors

WU QIONG
CHEN YUFEI
LV SHUAISHUAI
LU YIXUE
WANG PENGFEI
NI HONGJUN
WANG XINGXING
ZHU YU
YAO JIANNAN
ZHANG FUBAO

Assignees

南通大学

Dates

Publication Date: 20260508
Application Date: 20251224

Claims (10)

1. A traffic participant road condition early warning method based on a smart phone is characterized by comprising the following steps: s1, environmental data acquisition, namely acquiring video streams through a camera of a smart phone and synchronously acquiring sensor data for preprocessing; S2, a motion state and environment context identification step, namely identifying the motion state of a user and judging an environment scene based on sensor data; s3, performing visual analysis and distance estimation, namely performing target detection, semantic segmentation and distance estimation on the video frame by adopting a lightweight neural network model; s4, risk assessment and decision step, calculating risk level based on the target type, distance and user movement speed, and generating an early warning instruction; S5, a multi-mode early warning prompt step is carried out, and a voice, vibration, vision or interface switching mode is called according to an early warning instruction to give a warning to a user; s6, a low-power consumption and self-adaptive adjustment step, wherein the analysis frequency and the working mode are dynamically adjusted according to the environment complexity.
2. The traffic participant road condition early warning method based on the smart phone according to claim 1 is characterized in that the step S1 comprises the steps of calling a main camera arranged behind the smart phone to collect video streams, performing digital anti-shake, automatic exposure, automatic focusing and lens distortion correction processing, synchronously reading data of a gyroscope, an accelerometer, a magnetometer, an ambient light sensor and a GPS module, fusing the data of the gyroscope, the accelerometer and the magnetometer by using a Kalman filtering algorithm, solving the three-dimensional gesture of the smart phone, and dynamically adjusting an image analysis area.
3. The traffic participant road condition early warning method based on the smart phone according to claim 1, wherein the step S2 comprises judging the movement state of the user based on accelerometer data, including walking, running or static state, judging the day and night mode by combining ambient light sensor data, identifying the scene type by combining GPS data, and calling an electronic map API to assist in judging the road type and the environment attribute.
4. The traffic participant road condition early warning method based on the smart phone according to claim 3, wherein the step S3 comprises the steps of detecting obstacles, road condition features and traffic signals in real time by using a target detection model optimized by a mobile terminal, performing pixel-level segmentation on an image by adopting a semantic segmentation model to identify a passable area and a pavement boundary, and calculating the relative distance between a user and a target by using a monocular vision depth estimation model and combining object size priori knowledge.
5. The traffic participant road condition early warning method based on the smart phone according to claim 4, wherein the step S4 comprises the steps of calculating risk scores according to the target type, the distance and the relative speed, recording logs only at low risk, triggering vibration and screen flickering at medium risk, triggering voice broadcasting enhancement system switching interface at high risk.
6. The traffic participant road condition early warning method based on the smart phone according to claim 5 is characterized in that the step S5 comprises the steps of generating natural language warning voice through a TTS engine, adaptively adjusting sound volume along with environmental noise, vibrating the prompt, designing a plurality of vibration modes to correspond to different risk levels, drawing a semitransparent warning layer on the top layer of a screen through visual prompts, flashing icons with different colors, and forcibly switching to a warning interface through a system notification or a suspension window under emergency risk.
7. The traffic participant road condition early warning method based on the smart phone according to claim 6 is characterized in that the step S6 comprises dynamically adjusting video analysis frequency and model accuracy according to scene complexity, enabling a low-light enhancement algorithm to improve image quality in a night mode, and automatically switching an analysis model based on GPS and historical data.
8. A traffic participant road condition early warning system based on a smart phone is characterized by comprising: the environment sensing and information acquisition module is used for acquiring and preprocessing environment data; The analysis module is based on a neural network and is used for visual analysis and distance estimation; the risk assessment and decision module is used for risk scoring and early warning instruction generation; the multi-mode early warning prompt module is used for sending warning to a user in various modes; and the low-power consumption and scene self-adaptive module is used for dynamically adjusting the system power consumption and the processing strategy.
9. The traffic participant road condition early warning system based on the smart phone according to claim 8, wherein the environment sensing and information acquisition module comprises a camera sub-module for acquiring video streams and preprocessing images, a sensor fusion sub-module for fusing gyroscope, accelerometer and magnetometer data to calculate gestures, an environment sensing sub-module for judging day and night modes and scene types, and a motion state identification sub-module for identifying walking, running or static states of a user.
10. The traffic participant road condition early warning system based on the smart phone according to claim 8, wherein the analysis module based on the neural network comprises a target detection sub-module for detecting obstacles, road condition features and traffic signals, a distance estimation sub-module for calculating the relative distance between a user and a target, and a semantic segmentation sub-module for identifying a passable area and a road boundary.

Description

Traffic participant road condition early warning method and system based on smart phone Technical Field The invention relates to the technical field of active safety, in particular to a traffic participant road condition early warning method and system based on a smart phone. Background With the popularization of smart phones, many people develop the habit of looking at the mobile phone while walking or riding a bicycle. The behavior greatly disperses the attention of the user, so that the perception capability of the surrounding environment is reduced, and especially, safety accidents such as collision, falling and mistaken entering a motor vehicle lane are very easy to happen at night or under complex road conditions. Currently, solutions to this problem are mainly: and the safety propaganda and education reminds people to walk without watching the mobile phone through public welfare advertisements, warning marks and the like. The disadvantage is that the real-time physical protection cannot be provided depending on the consciousness of the individual, and the effect is limited. The user sets the shielding part notification according to the requirement. The method has the defects that information interference can be reduced only, and actual dangers (such as obstacles, pits, vehicles and the like) in the external physical environment can not be perceived and pre-warned. And the special wearable equipment is intelligent glasses with an environment sensing function. The disadvantage is that the user is required to purchase and carry additional hardware equipment, the cost is high, and the popularity is poor. Therefore, a solution that is low-cost, highly popular, and capable of actively sensing the environment and early warning in real time is highly desirable. Disclosure of Invention Therefore, the invention provides a traffic participant road condition early warning method and system based on a smart phone to solve the problems, and the system for providing real-time road condition early warning for a user in walking or riding without additional hardware is realized by utilizing the existing hardware resources of the smart phone through the method and system innovation. The method has the core ideas that the hardware such as a mobile phone camera and a sensor is used as a third eye together, the road condition of the advancing direction of a user is continuously analyzed, the potential risk is identified through an intelligent algorithm, and finally, different degrees of warning is sent to the user through modules such as vibration, audio, screen flickering or forced switching of the mobile phone. The invention provides a traffic participant road condition early warning method and system based on a smart phone, which mainly comprises the following steps: s1, environmental data acquisition, namely acquiring video streams through a camera of a smart phone and synchronously acquiring sensor data for preprocessing; S2, a motion state and environment context identification step, namely identifying the motion state of a user and judging an environment scene based on sensor data; s3, performing visual analysis and distance estimation, namely performing target detection, semantic segmentation and distance estimation on the video frame by adopting a lightweight neural network model; s4, risk assessment and decision step, calculating risk level based on the target type, distance and user movement speed, and generating an early warning instruction; S5, a multi-mode early warning prompt step is carried out, and a voice, vibration, vision or interface switching mode is called according to an early warning instruction to give a warning to a user; s6, a low-power consumption and self-adaptive adjustment step, wherein the analysis frequency and the working mode are dynamically adjusted according to the environment complexity. The step S1 further comprises the steps of calling a main camera arranged behind the mobile phone to collect video streams, carrying out digital anti-shake, automatic exposure, automatic focusing and lens distortion correction processing, synchronously reading data of a gyroscope, an accelerometer, a magnetometer, an ambient light sensor and a GPS module, fusing the data of the gyroscope, the accelerometer and the magnetometer by using a Kalman filtering algorithm, solving the three-dimensional gesture of the mobile phone, and dynamically adjusting an image analysis area. Further, the step S2 comprises the steps of judging the motion state of the user, including walking, running or static state, based on accelerometer data, judging the day and night mode by combining ambient light sensor data, identifying the scene type by combining GPS data, and calling an electronic map API to assist in judging the road type and the environment attribute, such as a night street. Further, the step S3 comprises the steps of detecting obstacles, road condition features and traffic signals in real time by using a targe