US-12618991-B2 - Person identification and imposter detection using footfall generated seismic signals

US12618991B2US 12618991 B2US12618991 B2US 12618991B2US-12618991-B2

Abstract

A smart device, biometric authentication system and a corresponding method thereof for person identification and imposter detection has been disclosed. The method comprises detection and extraction of seismic signals generated from corresponding footfalls, by means of unsupervised learning based detection and extraction module (USLEEM) and detection and identification of imposter and/or registered users respectively by means of an identification module.

Inventors

Subrat KAR
Bodhibrata MUKHOPADHYAY
Sahil ANCHAL

Assignees

INDIAN INSTITUTE OF TECHNOLOGY DELHI

Dates

Publication Date: 20260505
Application Date: 20201204
Priority Date: 20191205

Claims (9)

1 . A method for person identification and imposter detection, the method, executed by a smart device, comprising: during a training phase; detecting, by at least one sensing module of the smart device, a plurality of seismic signals generated from behavioral properties of a person, wherein the behavioral properties of the person comprise at least a footfall signature of the person, the footfall signature associated to corresponding footfalls of the person, by means of unsupervised learning based detection and extraction module (USLEEM) to generate a seismic event; converting, by an analog-to-digital converter module of the smart device, the detected plurality of seismic signals, the plurality of seismic signals being analog signals, into digital signals; splitting, by an event extraction module of the smart device, each digital seismic signal of plurality of digital seismic signals into N equal segments; extracting, for the each digital signal of the plurality of digital signals, by the event extraction module, first feature vectors (FE-I) corresponding to time domain and frequency domain features from each segment of the N equal segments; clustering, by the event extraction module, each of the first feature vectors (FE-I) between clustered events without using predefined thresholds, using a Gaussian Mixture Model (GMM), wherein one cluster represents footfall events, and another cluster represents noise events, to form at least one trained GMM based model capable of distinguishing the footfall events from the noise events based on one or more statistical properties; multiplying the each segment of the N equal segments classified as the footfall events with a Gaussian window centered at segment's peak amplitude to obtain smooth and spectrally consistent footfall signals; extracting, from the each resulting smooth and spectrally consistent footfall signal, second feature vectors (FE-II), corresponding to refined time domain and frequency domain features, to further train: a plurality of one-class learning models, each trained with the second feature vectors (FE-II) of an individual registered user to enable imposter detection by modelling only user specific class; and a multi-class learning model using the second feature vectors (FE-II) to associate footfall signatures with corresponding registered users; and storing the trained GMM based model; during a live phase; detecting, by the sensing module, a plurality of new seismic signals generated from one or more footfalls during a user activity, the plurality of new seismic signals being analog signals; converting, by the analog-to-digital converter module, each of the detected plurality of new analog seismic signals into new digital signals; splitting, by the event extraction module of the smart device, each new digital seismic signal of plurality of new digital seismic signals into N equal segments; extracting, for the each new digital signal of the plurality of new digital signals, by the event extraction module, first feature vectors (FE-I) corresponding to time domain and frequency domain features from each segment of the N equal segments; clustering, using the trained GMM based model, each segment of the N equal segments as a footfall event or a noise event based on the each of the first feature vectors (FE-I); multiplying the each segment of the N equal segments classified as the footfall events with a Gaussian window centered at segment's peak amplitude to obtain smooth and spectrally consistent footfall signals; extracting, from the each resulting smooth and spectrally consistent footfall signal, second featured vectors (FE-II), corresponding to refined time domain and frequency domain features; feeding each of the second featured vectors (FE-II) to the trained plurality of one-class models; detecting, by the trained plurality of one-class learning models, an imposter, when the each of the second feature vectors (FE-II) are identified as outliers by the all trained plurality of one-class learning models, to return a binary output of −1; and identifying, by the multi-class learning model, a registered user, from when the each of the second feature vectors (FE-II) are classified into predefined user class corresponding to the registered user.
2 . The method as claimed in claim 1 , wherein the trained GMM based model clustering comprises: classifying the cluster with a higher variance to the footfall events and classifying the cluster with a lower variance to the noise events facilitates clustering each of the first feature vectors (FE I) into the clustered event.
3 . The method as claimed in claim 2 , wherein each of the footfall events cluster and the noise events cluster is parameterized by the one or more statistical properties comprising at least a set of φ, μ, and Σ, wherein to classify and label the cluster with the higher variance and the lower variance to the footfall events or the noise events respectively comprises utilizing the equation: Class = { C 1 → Event , C 2 → Noise ⁢ : ⁢ ⁢  Σ C 1  >  Σ C 2  C 1 → Noise , C 2 → Event ⁢ : ⁢ ⁢  Σ C 2  >  Σ C 1  where |Σ C k | is determinant of co-variance matrix of kth clusters.
4 . The method as claimed in claim 1 , wherein during the live phase, the each of the N equal segments of the footfall events generated seismic signals are classified by: Class = { C 1 ⁢ : ⁢ ⁢ p ⁡ ( C 1 ❘ f test w i ) > p ⁡ ( C 2 ❘ f test w i ) C 2 ⁢ : ⁢ ⁢ p ⁡ ( C 2 ❘ f test w i ) > p ⁡ ( C 1 ❘ f test w i ) where f test w i is the feature vector of the i th segment of a test signal Signal test , p ⁡ ( C k ⁢ ❘ "\[LeftBracketingBar]" f test w i ) ⁢ ( = φ ⁢ C k ⁢ N ⁡ ( f test w i ⁢ ❘ "\[LeftBracketingBar]" μ C k , Σ C k ) ⁢ for ⁢ k = 1.2 ) is a probability that f test w i belongs to class C k , test signal Signal test is the seismic signal generated by the imposter or the registered user during the live phase and the test signal Signal test is segmented into equal parts (w i test ).
5 . A smart device for person identification and imposter detection, wherein the smart device comprises: at least one sensing module to detect a plurality of seismic signals generated from behavioral properties of a person, wherein the behavioral properties of the person comprise at least a footfall signature of the person, the footfall signature associated to corresponding footfalls of the person to generate a seismic event; an analog-to-digital converter module to convert the detected plurality of seismic signals, the plurality of seismic signals being analog signals, into digital signals: an event extraction module to: split each digital seismic signal, of plurality of digital seismic signals, into N equal segments; extract, for the each digital signal of the plurality of digital signals, first feature vectors (FE-I) corresponding to time domain and frequency domain features from each segment of the N equal segments; cluster, each of the first feature vectors (FE-I) between clustered events without using predefined thresholds, using a Gaussian Mixture Model (GMM), wherein one cluster represents footfall events, and another cluster represents noise events, to form at least one trained GMM based model capable of distinguishing the footfall events from the noise events based on one or more statistical properties; multiply the each segment of the N equal segments classified as the footfall events with a Gaussian window centered at segment's peak amplitude to obtain smooth and spectrally consistent footfall signals; extract, from the each resulting smooth and spectrally consistent footfall signal, second feature vectors (FE-II), corresponding to refined time domain and frequency domain features, to further train: a plurality of one-class learning models, each trained with the second feature vectors (FE-II) of an individual registered user to enable imposter detection by modelling only user specific class; and a multi-class learning model using the second feature vectors (FE-II) to associate footfall signatures with corresponding registered users; and store the trained GMM based model; and an identification module to detect an imposter and/or to identify a registered user during a live phase comprising user activity, wherein the imposter is detected if each of second feature vectors (FE-II) obtained during the user activity are identified as outliers by the all trained plurality of one-class learning models, and wherein if the each of the second feature vectors (FE-II) obtained during the user activity are not identified as outliers by any of the all trained plurality of one-class learning models, the registered user is identified by the multi-class learning model, wherein the each of the second feature vectors (FE-II) are classified into predefined user class corresponding to the registered user.
6 . A biometric authentication system for person identification and imposter detection, the biometric authentication system comprising: an array of smart devices distributed over a pre-determined zone; and a central controller communicably coupled to the array of smart devices, wherein each smart device of the array of smart devices comprises: at least one sensing module to detect a plurality of seismic signals generated from behavioral properties of a person, wherein the behavioral properties of the person comprise at least a footfall signature of the person, the footfall signature associated to corresponding footfalls of the person to generate a seismic event; an analog-to-digital converter module to convert the detected plurality of seismic signals, the plurality of seismic signals being analog signals, into digital signals; an event extraction module to: split each digital seismic signal, of plurality of digital seismic signals, into N equal segments; extract, for the each digital signal of the plurality of digital signals, first feature vectors (FE-I) corresponding to time domain and frequency domain features from each segment of the N equal segments; cluster, each of the first feature vectors (FE-I) between clustered events without using predefined thresholds, using a Gaussian Mixture Model (GMM), wherein one cluster represents footfall events, and another cluster represents noise events, to form at least one trained GMM based model capable of distinguishing the footfall events from the noise events based on one or more statistical properties; multiply the each segment of the N equal segments classified as the footfall events with a Gaussian window centered at segment's peak amplitude to obtain smooth and spectrally consistent footfall signals; extract, from the each resulting smooth and spectrally consistent footfall signal, second feature vectors (FE-II), corresponding to refined time domain and frequency domain features, to further train: a plurality of one-class learning models, each trained with the second feature vectors (FE-II) of an individual registered user to enable imposter detection by modelling only user specific class; and a multi-class learning model using the second feature vectors (FE-II) to associate footfall signatures with corresponding registered users; and store the trained GMM based model; and an identification module to detect an imposter and/or to identify a registered user during a live phase comprising user activity, wherein the imposter is detected if each of second feature vectors (FE-II) obtained during the user activity are identified as outliers by the all trained plurality of one-class learning models, and wherein if the each of the second feature vectors (FE-II) obtained during the user activity are not identified as outliers by any of the all trained plurality of one-class learning models, the registered user is identified by the multi-class learning model, wherein the each of the second feature vectors (FE-II) are classified into predefined user class corresponding to the registered user.
7 . The smart device as claimed in claim 6 , wherein the event extraction module facilitates the trained GMM based model clustering, wherein the clustering comprises: classifying the cluster with a higher variance to the footfall events and classifying the cluster with a lower variance to the noise events.
8 . The smart device as claimed in claim 7 , wherein the event extraction module facilitates the footfall events clustering and the noise events clustering by parameterizing each of the footfall events cluster and the noise events cluster based on one or more statistical properties comprising at least a set of φ, μ, and Σ, wherein to classify and label the cluster with the higher variance and the lower variance to the footfall events or the noise events respectively comprises utilizing the equation: Class = { C 1 → Event , C 2 → Noise ⁢ : ⁢ ⁢  Σ C 1  >  Σ C 2  C 1 → Noise , C 2 → Event ⁢ : ⁢ ⁢  Σ C 2  >  Σ C 1  where |Σ C k | is determinant of co-variance matrix of kth clusters.
9 . The smart device as claimed in claim 6 , wherein during the live phase, the each of the N equal segments of the footfall events generated seismic signals are classified by: Class = { C 1 ⁢ : ⁢ ⁢ p ⁡ ( C 1 ❘ f test w i ) > p ⁡ ( C 2 ❘ f test w i ) C 2 ⁢ : ⁢ ⁢ p ⁡ ( C 2 ❘ f test w i ) > p ⁡ ( C 1 ❘ f test w i ) where f test w i is the feature vector of the i th segment of a test signal Signal test , p ⁡ ( C k ⁢ ❘ "\[LeftBracketingBar]" f test w i ) ⁢ ( = φ ⁢ C k ⁢ N ⁡ ( f test w i ⁢ ❘ "\[LeftBracketingBar]" μ C k , Σ C k ) ⁢ for ⁢ k = 1.2 ) is a probability that f test w i belongs to class C k , test signal Signal test is the seismic signal generated by the imposter or the registered user during the live phase and the test signal Signal test is segmented into equal parts (w i test ).

Description

FIELD OF THE INVENTION Present invention in general relates to automatic human identification and imposter detection technique, more particularly automatic human identification and imposter detection technique using footfall generated seismic signal. BACKGROUND OF THE INVENTION Surveillance is an integral part of an institution or organization, and due to an increased level of various kinds of threats, a lot of research and experimentation is being carried out to ensure a full-proof security system. Covert observation of people using smart camouflageable sensor is gaining popularity. It is important for an organization, especially high-security establishment to identify its own people i.e., their registered users and also to detect imposters i.e., non-registered users with high accuracy. Increased security breaches coupled with the misuse of the present all-pervasive power of technology and its subsequent deleterious effects on mankind have made it all the more important than ever to device new preventive measures which can nip any malicious intent right in the bud. Predominantly human identification is carried manually by access cards (RFID cards). Advancement of signal processing and recent developments in modern sensors gave opportunity to biometric based identification or verification systems. These systems use a physical or behavioural property as biometrics for person identification. Camera, fingerprint scanner, interferometric reflectance imaging sensor (IRIS), microphone sensors are used to identify humans by exploiting physical biometrics like facial images, fingerprints, voice etc. On the other hand, biometric using behavioural features like gait, walking patterns, Infrared radiation from body surface, footfall signature etc. are carried by sensors like video camera, accelerometer, pressure sensor, Passive Infrared (PIR) sensor, ultra-wide band sensor, acoustic, and seismic sensor. Other sensors used for identification are cameras that demand a certain amount of ambient light and a clear facial image, microphones that require a low level of background noise, and fingerprint sensors that require an individual to place his/her finger on the scanner. Direct line of sight (LOS) is necessary for radio frequency related sensors, and wearable sensors like accelerometer need to be attached to the body of the subject. A crucial part of any behavioural based biometric system is its event detection and extraction technique. Researchers have used techniques like amplitude thresholding (AMP-Th), STA-LTA, kurtosis, UREDT, and noise modelling for event detection in seismic signal. Another vital aspect of any surveillance system is its ability to detect intruders (imposters). Most of the automated biometric systems (especially the ones that use behavioural properties) work on the principle of matching signatures of a fresh data to a pre stored data. However, little work has been done in detecting imposters (individuals whose data are not present in the system database) using behavioural properties of individuals. The existing systems are able to predict only those individuals whose data are already present in the database. In the absence of an individual's data in the database, the existing systems will predict the class (or individual) with which the test data have maximum similarity. Reference is made to non-patent literature documents “Indoor person identification through footstep induced structural vibration” and “Footprint id: Indoor pedestrian identification through ambient structural vibration sensing” by S. Pan. The documents teach utilizing footstep induced structural vibrations to identify humans. In “Indoor person identification through footstep induced structural vibration”, time and frequency related features from the footfall signal have been extracted and classified by using SVM (support vector machine). Their dataset consisted of ˜ 1500 footsteps of 5 individuals. The performance of the system was calculated for both step level and trace level accuracies. In the step level scenario only a single footfall is considered, and in the trace level scenario 5 footfalls of the highest SNR (signal to noise ration) are taken as a single sample for identification. The system achieved an accuracy of 63% in step level and 83% in trace level scenarios. A confidence level thresholding (CLT) of the signals have also been performed, by 50% of the traces were discarded and eliminated potentially incorrect classification cases. Using this CLT they observed an increased accuracy of 96.5%. In “Footprint id: Indoor pedestrian identification through ambient structural vibration sensing”, an event detection technique of another non-patent literature document “Boes: building occupancy estimation system using sparse ambient vibration monitoring”, has been implemented modelling noise as a Gaussian distribution. An event is detected if the energy of the signal inside the current window is beyond three standard-deviation abov