US-12618707-B2 - Method for determining abnormal acoustic source and AI acoustic image camera

US12618707B2US 12618707 B2US12618707 B2US 12618707B2US-12618707-B2

Abstract

Disclosed is an AI acoustic camera including an acoustic source localizing means unit of generating position-specific acoustic level data by determining a position of an acoustic source, an AI acoustic analysis unit of recognizing a type of acoustic source estimated as an abnormal acoustic source by extracting a regeneration time domain acoustic signal for the acoustic source with the determined position and AI-learning and recognizing an acoustic feature image of the extracted time domain acoustic signal, an object recognition unit of recognizing a type of object positioned in the acoustic source through image analysis of an area recognized as that the acoustic source is positioned, and a determination unit of determining the acoustic source as a true acoustic source when the type of acoustic source and the type of object have commonality.

Inventors

Young-Key Kim
In-Keon KIM
Wook-Jin JEONG
Jung-Seop KIM

Assignees

SM INSTRUMENTS CO LTD

Dates

Publication Date: 20260505
Application Date: 20211108
Priority Date: 20210526

Claims (18)

1 . A method for determining an abnormal acoustic source using artificial intelligence, the method comprising: an acoustic data acquisition step of acquiring acoustic data through an acoustic sensor array; an abnormal acoustic source candidate local area selecting step of selecting at least one abnormal acoustic source candidate local area, in which at least one abnormal acoustic source candidate position forms a group, based on the acquired acoustic data; an acoustic feature image generating step of generating an acoustic feature image from a signal extracted from the abnormal acoustic source candidate local area, wherein the signal includes a time-domain acoustic signal; an acoustic scene classifying step of classifying the acoustic feature image as one of at least one pre-learned acoustic scene; an object type recognizing step of recognizing a type of object located in the abnormal acoustic source candidate local area based on an object image of the abnormal acoustic source candidate local area or an object image of an area adjacent to the abnormal acoustic source candidate position; and a determining step of determining whether the abnormal acoustic source candidate local area or abnormal acoustic source candidate position is an abnormal acoustic source based on the classified acoustic scene and the recognized object type, wherein the acoustic feature image generating step further comprises: a step of extracting a time-domain acoustic signal of a representative position belonging to the abnormal acoustic source candidate local area; and a step of generating the acoustic feature image from the time-domain acoustic signal of the representative position, wherein the representative position is a local maximum position of the abnormal acoustic source candidate local area.
2 . The method of claim 1 , wherein the abnormal acoustic source candidate local area selecting step further comprises a step of selecting at least one abnormal acoustic source candidate local area, in which abnormal acoustic source candidate positions having sound levels exceeding a predefined value form a group.
3 . The method of claim 1 , wherein the abnormal acoustic source candidate local area selecting step uses whether an acoustic level gradually increases toward a central portion of the abnormal acoustic source candidate local area as a parameter for selecting the abnormal acoustic source candidate local area.
4 . The method of claim 1 , wherein the acoustic feature image generating step further comprises generating the acoustic feature image by imaging at least one feature parameter selected from a discrete wavelet transform (DWT), multi-resolution short-time Fourier transform, mel filterbank, log mel filterbank energy, mel-frequency filterbank conversion, and multi-resolution log-mel spectrogram.
5 . The method of claim 1 , wherein the acoustic scene classifying step is performed using an artificial intelligence model learned to classify the acoustic scene from the acoustic feature image.
6 . The method of claim 1 , wherein the object type recognizing step is performed using an artificial intelligence model learned to classify the object type from the object image.
7 . The method of claim 1 , the method further comprises an alarm signal generating step of generating an alarm signal when the abnormal acoustic source candidate local area or the abnormal acoustic source candidate position is determined to be an abnormal acoustic source.
8 . The method of claim 1 , the method further comprises a step of generating an optical-acoustic image by overlapping an optical video image with an acoustic field visualizing image.
9 . The method of claim 1 , wherein the determining step further comprises a step of determining the abnormal acoustic source if the classification of the acoustic scene, the type of the recognized object, and a predefined monitoring target range are all matched.
10 . An AI (Artificial Intelligence) acoustic image camera comprising: an acoustic data acquisition unit configured to acquire acoustic data through an acoustic sensor array; an abnormal acoustic source candidate local area selecting unit configured to select at least one abnormal acoustic source candidate local area, in which at least one abnormal acoustic source candidate position forms a group, based on the acquired acoustic data; an acoustic feature image generation unit configured to generate an acoustic feature image from a signal extracted from the abnormal acoustic source candidate local area, wherein the signal includes a time-domain acoustic signal; an acoustic analysis unit configured to classify the acoustic feature image as one of at least one pre-learned acoustic scene; an object recognition unit configured to recognize a type of object located in the abnormal acoustic source candidate local area based on an object image of the abnormal acoustic source candidate local area or an object image of an area adjacent to the abnormal acoustic source candidate position; and a determination unit configured to determine whether the abnormal acoustic source candidate local area or the abnormal acoustic source candidate position is an abnormal acoustic source based on the classified acoustic scene and the recognized object type, wherein the acoustic feature image generation unit is configured to extract a time-domain acoustic signal of a representative position belonging to the abnormal acoustic source candidate local area, and generate the acoustic feature image from the time-domain acoustic signal of the representative position, wherein the representative position is a local maximum position of the abnormal acoustic source candidate local area.
11 . The AI acoustic image camera of claim 10 , wherein the abnormal acoustic source candidate local area selecting unit is further configured to group positions having acoustic levels exceeding a predefined level to form the abnormal acoustic source candidate local area.
12 . The AI acoustic image camera of claim 10 , wherein the abnormal acoustic source candidate local area selecting unit is further configured to use whether an acoustic level gradually increases toward a central portion of the abnormal acoustic source candidate local area as a parameter for selecting the abnormal acoustic source candidate local area.
13 . The AI acoustic image camera of claim 10 , wherein the acoustic feature image generation unit is further configured to generate the acoustic feature image by imaging at least one feature parameter selected from a discrete wavelet transform (DWT), multi-resolution short-time Fourier transform, mel filterbank, log mel filterbank energy, mel-frequency filterbank conversion, and multi-resolution log-mel spectrogram.
14 . The AI acoustic image camera of claim 10 , wherein the acoustic analysis unit is further configured to classify the acoustic scene using an artificial intelligence model learned to classify the acoustic scene from the acoustic feature image.
15 . The AI acoustic image camera of claim 10 , wherein the object recognition unit is further configured to recognizing the object using an artificial intelligence model learned to classify the object type from the object image.
16 . The AI acoustic image camera of claim 10 , the AI acoustic image camera further comprising: an alarm unit configured to generate an alarm signal when the abnormal acoustic source candidate local area or the abnormal acoustic source candidate position is determined to be an abnormal; and a transmission unit configured to transmit an optical-acoustic image, in which an optical video image and an acoustic field visualizing image are overlapped.
17 . A method for determining an abnormal acoustic source using artificial intelligence, the method comprising: an acoustic data acquisition step of acquiring acoustic data through an acoustic sensor array; an acoustic source localizing step of estimating a position of the acoustic source by calculating an acoustic level of the acoustic source at each location based on the acquired acoustic data; an acoustic classifying step of classifying the acoustic source as one of at least one pre-learned acoustic scene; an object type determination step of determining a type of an object through image analysis of an area within a critical distance from the acoustic source; a determining step of determining whether the acoustic source is an abnormal acoustic source based on the classified acoustic scene and the recognized object type, wherein in the acoustic feature image generating step comprises: a step of extracting a time-domain acoustic signal of a representative position belonging to the abnormal acoustic source candidate local area; and a step of generating the acoustic feature image from the time-domain acoustic signal of the representative position, wherein the representative position is a local maximum position of the abnormal acoustic source candidate local area.
18 . The method of claim 17 , wherein the determining step further comprises a step of determining the abnormal acoustic source if the classification of the acoustic scene, the type of the recognized object, and a predefined monitoring target range are all matched.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the priority of Korean Patent Application No. 10-2021-0067629, filed on 26 May 2021, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference. FIELD The present disclosure relates to a method for determining an abnormal acoustic source and an AI acoustic image camera. BACKGROUND In Korean Patent Registration No. 10-1213539, there is disclosed an acoustic camera using an MEMS microphone array which includes an acoustic sensing device that is configured to mount a plurality of MEMS microphones on a print circuit board and transmits a signal of a sensed sound; a data collection unit which is connected with the acoustic sensing device and samples an analogue signal for the sound transmitted from the acoustic sensing device to convert the analogue signal to a digital signal for the sound and transmits the converted digital signal to a central processing unit; a central processing unit which is connected with the data collection unit and calculates a noise level based on the digital signal for the sound transmitted from the data collection unit; and a display unit which is connected with the central processing unit and displays the noise level calculated from the central processing unit in colors, wherein the MEMS microphone has 2 to 10 blade portions extended in a radial direction. The above-described technical configuration is the background art for helping in the understanding of the present disclosure, and does not mean a conventional technology widely known in the art to which the present disclosure pertains. SUMMARY An object of the present disclosure is to provide an AI acoustic image camera which determines a position of an acoustic source through an acoustic field visualizing means, extracts and AI-learns time data on the acoustic source with the determined position to recognize a type of acoustic source to be estimated as the acoustic source, recognizes a type of object positioned in the acoustic source through image analysis of an area recognized as that the acoustic source is positioned, and then determines the acoustic source as a true acoustic source when the type of acoustic source and the type of object have commonality. <Method for Determining Abnormal Acoustic Source> According to an aspect of the present disclosure, there is provided a method for determining an abnormal acoustic source including: an acoustic source localizing step of calculating a level (size) of an acoustic source for each position based on acoustic data acquired by a plurality of acoustic sensor array; a candidate acoustic source time domain acoustic source signal extraction step of extracting a regeneration time domain acoustic signal of a position estimated as that the acoustic source is present based on the level of the acoustic source for each position; an acoustic feature image generation step of generating a color feature image by extracting a feature of the time domain acoustic source signal of the candidate acoustic source; an AI acoustic classification step of recognizing the acoustic feature image and performing the acoustic classification for the candidate acoustic source by using a pre-learned AI acoustic classification means; and an abnormal acoustic source determination step of determining the acoustic source as the abnormal acoustic source when the acoustic classification for the candidate acoustic source belongs to a predefined monitoring target range. The method for determining the abnormal acoustic source may further include an object image classification step of determining a type of object located at the candidate acoustic source by video analysis of a candidate acoustic source coordinate or adjacent position, wherein in the abnormal acoustic source determination step, when the acoustic classification and the type of object are included in a predetermined monitoring target range, the acoustic source may be determined as an abnormal acoustic source and an alarm signal may be generated. According to another aspect of the present disclosure, there is provided a method for determining an abnormal acoustic source including: an acoustic data acquisition step (S10) of acquiring, by an acoustic data acquisition unit, acoustic data through an acoustic sensor array configured by a plurality of acoustic sensors; a position-specific acoustic level calculation step (S20) of calculating, by an acoustic calculation unit of an acoustic processing unit, a position-specific acoustic level in a direction of the acoustic sensor array; an abnormal acoustic source candidate selection step (S30) of selecting, by an abnormal acoustic source candidate selection unit, one position as a local area representative position (e.g., the representative position is a local maximum position) in at least one local area (abnormal acoustic source candidate local area) of grouping positions having acoustic levels exceeding a predetermi