KR-20260067039-A - Three-dimensional behavior estimation system and method using multimodal contactless sensor technology

KR20260067039AKR 20260067039 AKR20260067039 AKR 20260067039AKR-20260067039-A

Abstract

A three-dimensional posture and behavior estimation system using a multimodal non-contact sensor according to one embodiment of the present invention includes: at least one multimodal sensor module that is placed within a pre-set space and collects user body information via mmWave; and an information processing server that analyzes the mmWave obtained from the at least one multimodal sensor module to estimate the user's posture and behavior.

Inventors

최대영
양우석
박주희
정재호

Assignees

주식회사 파미티

Dates

Publication Date: 20260512
Application Date: 20241105

Claims (10)

At least one multimodal sensor module that is positioned within a pre-configured space and collects body information and point clouds of a target object via mmWave; and A data processing server comprising an information processing server that analyzes mmWave acquired from at least one multimodal sensor module to estimate the user's posture and behavior. 3D posture and behavior estimation system using multimodal non-contact sensors.
In paragraph 1, The above information processing server is Characterized by processing mmWave acquired from at least one multimodal sensor module into time-series data, and then preprocessing the data through labeling and annotation processing. 3D posture and behavior estimation system using multimodal non-contact sensors.
In paragraph 2, The above information processing server is Characterized by converting preprocessed mmWave data into a 3D point cloud, and if the number of the 3D point clouds exceeds a preset value, dividing the 3D point clouds into voxels and downsampling using the average of the points within each voxel or downsampling by adjusting the size of the voxels. 3D posture and behavior estimation system using multimodal non-contact sensors.
In paragraph 3, The above information processing server is Characterized by estimating the subject's body parts based on the clustering shape of the above 3D point cloud, and estimating the subject's posture according to the location of the estimated body parts. 3D posture and behavior estimation system using multimodal non-contact sensors.
In paragraph 4, The above information processing server is Characterized by estimating the behavior of a subject by tracking the location of the labeling data of the above 3D point cloud, 3D posture and behavior estimation system using multimodal non-contact sensors.
A step of collecting mmWave sequence data from a multimodal sensor module in a data collection unit in at least one multimodal sensor module; and A step comprising analyzing mmWave acquired from at least one multimodal sensor module at an information processing server to estimate the user's posture and behavior, 3D posture and behavior estimation method using multimodal non-contact sensors.
In paragraph 6, The step of estimating the posture and behavior of the user mentioned above is, Characterized by including the step of processing mmWave acquired from at least one multimodal sensor module into time-series data, and then preprocessing the data through labeling and annotation processing. 3D posture and behavior estimation method using multimodal non-contact sensors.
In Paragraph 7, The step of estimating the posture and behavior of the user mentioned above is, Characterized by including a step of converting preprocessed mmWave data into a 3D point cloud, and if the number of 3D point clouds exceeds a preset value, dividing the 3D point cloud into voxels and downsampling using the average of the points within each voxel or downsampling by adjusting the size of the voxels. 3D posture and behavior estimation method using multimodal non-contact sensors.
In paragraph 8, The step of estimating the posture and behavior of the user mentioned above is, Characterized by including the step of estimating the subject's body parts based on the clustering shape of the above 3D point cloud, and estimating the subject's posture according to the location of the estimated body parts. 3D posture and behavior estimation method using multimodal non-contact sensors.
In Paragraph 9, The step of estimating the posture and behavior of the user mentioned above is, Characterized by including the step of estimating the behavior of a subject by tracking the location of the labeling data of the above 3D point cloud, 3D posture and behavior estimation method using multimodal non-contact sensors.

Description

Three-dimensional behavior estimation system and method using multimodal contactless sensor technology The present invention relates to a three-dimensional posture and behavior estimation system and method using a multimodal non-contact sensor. Telemedicine is a crucial component of modern medical services, significantly contributing to improved access to healthcare, cost reduction, increased efficiency, and the protection of patient privacy, with 32 million consultations recorded in 2023. The necessity of telemedicine has become even clearer, particularly in light of the recent global pandemic and the collapse of the healthcare system caused by severe labor shortages due to medical strikes. By providing remote consultations without the risk of infection, the safety of both patients and medical staff can be guaranteed. This method of treatment is of great benefit to patients residing in geographically restricted areas or regions with limited medical resources, and it facilitates the establishment of more efficient monitoring systems within hospitals. Furthermore, it reduces healthcare service costs and enables medical staff to effectively manage a larger number of patients without being constrained by time or location. Telemedicine requires the support of contactless technology, and among these, mmWave technology is driving groundbreaking advancements in the medical field. Contactless monitoring systems utilizing this technology can accurately measure vital signs, such as heart rate, respiratory rate, and movement, without direct physical contact with the patient. This enables precise patient monitoring while reducing the risk of infection, offering significant advantages, particularly during infectious disease outbreaks. By providing real-time data, medical professionals can continuously monitor the patient's condition and take necessary medical measures immediately. Furthermore, mmWave technology contributes to enhancing the accuracy and efficiency of telemedicine by enabling the acquisition of biometric information while protecting patient privacy. By continuously monitoring patients' biometric data and behavior through this technology even during medical consultations, the quality of home care and in-hospital medical services can be significantly improved. Additionally, mmWave technology plays a crucial role in protecting patient privacy. Since it collects data without using physical images or videos of the patient, it contributes significantly to the protection of patients' personal information. The patient monitoring system is established according to each zone within the hospital. Patient conditions are checked via monitors installed at every bedside, and the status of patients within each department is comprehensively monitored through central monitoring systems installed in areas such as the emergency room, intensive care unit, operating room, and wards. However, the current central monitoring system is not connected between different departments. Therefore, if an alarm sounds due to a patient's abnormality, medical staff must go directly to the relevant department to examine the patient. Additionally, if a patient is transferred to a different department, the monitoring system must be switched to and reconnected to that department, which may temporarily interrupt the collection of the patient's clinical information. A surgical resident stated, “Because each medical staff member has to care for a large number of patients, emergencies occur in multiple locations, so we are constantly moving back and forth between the intensive care unit, operating room, and emergency room dozens of times a day.” In fact, as of 2019, the number of consultations per doctor in Korea was 6,989, which is three times higher than the OECD average of 2,130. Currently, methods for monitoring patient health in medical facilities primarily rely on contact sensors or direct observation by medical staff. These methods cause patient discomfort, the risk of infection through medical personnel—whose interest has increased due to the dangers of COVID-19—and high dependence on manpower; therefore, there is a need for a more efficient system that cares for patients through real-time monitoring without the need for direct observation by doctors. Meanwhile, the use of CCTV in hospitals aims to ensure patient safety and security, but at the same time presents a dual problem of potentially infringing upon personal privacy. As widely reported in the news, controversy has arisen regarding the protection of patients' privacy as CCTVs in hospital rooms record patients' private moments. In this situation, the need for hospitals to ensure safety while protecting patients' privacy has been highlighted. Non-contact mmWave technology was devised as a solution to this. mmWave technology does not record images or video, but uses high-frequency signals to detect patient movements and estimate only behavioral patterns. By using this technology, it is