KR-20260064968-A - Device for monitoring vehicle occupant

KR20260064968AKR 20260064968 AKR20260064968 AKR 20260064968AKR-20260064968-A

Abstract

The present invention relates to a vehicle occupant monitoring device and is characterized by comprising: a sensing unit that measures occupant status information through a camera and a pressure sensor; a collecting unit that collects the measured occupant status information; a preprocessing unit that removes noise and normalizes the collected occupant status information using a digital image filter; an inference unit that analyzes the preprocessed occupant status information using a CNN-LSTM model; an output unit that displays the final occupant status information as voice or warning notification on a display; and a communication unit that transmits the final occupant status information to a server and an administrator's mobile terminal.

Inventors

최원혁
정우진
최태웅

Assignees

한서대학교 산학협력단

Dates

Publication Date: 20260508
Application Date: 20241030

Claims (3)

A sensing unit that measures passenger status information through a camera and a pressure sensor; A collection unit that collects the above-mentioned passenger status information; A preprocessing unit that removes noise and normalizes the above-mentioned occupant status information using a digital image filter; An inference unit that analyzes the above-mentioned preprocessed passenger state information using a CNN-LSTM model; An output unit that displays the above-mentioned final passenger status information on a display as a voice or warning notification; and A vehicle occupant monitoring device characterized by including a communication unit that transmits the above-mentioned final occupant status information to a server and an administrator's mobile terminal.
In paragraph 1, A vehicle occupant monitoring device characterized by further including a control unit that executes the overall control function of the vehicle occupant monitoring device.
In paragraph 1, A vehicle occupant monitoring device characterized by an always-on CNN-LSTM model that analyzes collected occupant images through a CNN and generates a caption explaining the situation through an LSTM.

Description

Vehicle Occupant Monitoring Device The present invention relates to a vehicle occupant monitoring device, and more specifically, to a vehicle occupant monitoring device that ensures the safety and convenience of occupants in real time. Autonomous vehicles are vehicles that recognize and assess their driving environment on their own without intervention from drivers or passengers, automatically performing driving tasks such as driving, braking, and steering. This reduces the risk of traffic accidents and significantly enhances passenger convenience. However, since autonomous vehicles have no driver or a limited role, there is a growing need for devices that can monitor the passenger's condition in real time to prevent accidents and respond to emergencies. Existing vehicle monitoring devices primarily focus on the external environment and lack the ability to detect changes in the occupant's condition or the environment in real time. Therefore, in order to provide passenger safety and convenience, there is a need for technology capable of monitoring the passenger's condition in real time, predicting their state, and providing services appropriate to the predicted results. FIG. 1 is a block diagram showing a vehicle occupant monitoring device according to the present invention. Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Now, a preferred embodiment of the vehicle occupant monitoring device according to the present invention will be described in detail. FIG. 1 is a block diagram showing a vehicle occupant monitoring device according to the present invention. Referring to FIG. 1, the vehicle occupant monitoring device (100) according to the present invention can be installed on a device at a location corresponding to a seat inside the vehicle. The vehicle occupant monitoring device (100) may include a detection unit (110), a collection unit (120), a preprocessing unit (130), an inference unit (140), a control unit (150), an output unit (160), and a transmission unit (170). The constant detection unit (110) can measure the condition of the occupant through a camera and a pressure sensor. Here, the camera can detect the occupant's face and body movements. For example, the camera can detect drowsiness, fatigue, abnormal movements, etc. The pressure sensor is attached to the seat and seat belt to detect the occupant's sitting posture and movements. The constant collection unit (120) can collect passenger status information (passenger image data) through the detection unit (110). The above preprocessing unit (130) can remove noise and normalize the collected image data using a preset filter. That is, the above preprocessing unit (130) can remove noise using a digital image filter and normalize the input image through white balancing, color temperature adjustment, angle correction, distortion correction, size correction, blur correction, etc. The inference unit (140) can analyze the passenger's state information detected in real time using a CNN-LSTM model. That is, the processing unit (300) can learn the passenger's state information, such as drowsiness, rest, conversation, and forward gaze, using a CNN-LSTM model, and automatically determine the situation and explain the scenario for a new image input. As an example, the inference unit (140) analyzes an image captured by a camera inside the vehicle through a CNN and infers the situation. Depending on the inferred situation, the LSTM generates a corresponding text caption. For example, in a situation where a passenger is dozing off, a caption such as "The driver is dozing off. Caution is required." may be generated. The above CNN module receives images collected in real-time from the vehicle's interior camera, processes them to extract the occupant's situation as features, and the CNN extracts key visual information from the images through a multi-layered structure and converts it into encoded feature vectors. The above LSTM module receives feature vectors extracted from CNN as input and generates text for scenario prediction and description in chronological order. LSTM demonstrates excellent performance in processing time-series data and naturally generates a continuous description of the behavior of vehicle occupants or changing situations. The above control unit (150) can perform overall control functions of the deep learning-based vehicle and pedestrian detection image analysis device (100). The above control unit (150) can execute overall control functions of the vehicle occupant monitoring device (100) using programs and data stored in the storage unit. The output unit (160) can display passenger final state information as voice or warning notification on the display through the control unit (150). Here, the passen