KR-102960654-B1 - PRIVACY PROCESSOR ENABLE PERSONAL PRIVACY PROTECTION IN CAMERA DEVICE AND OPERATION METHOD OF THE SAME

KR102960654B1KR 102960654 B1KR102960654 B1KR 102960654B1KR-102960654-B1

Abstract

The present disclosure provides a privacy processor and a method of operation thereof that enables the protection of an individual's privacy in a camera device. The camera device includes a camera module having an image sensor configured to collect raw pixel data, a privacy processor configured to process raw pixel data from the image sensor for privacy protection, and an image signal processor configured to generate image information using raw pixel data from the privacy processor. Specifically, the privacy processor is configured to detect basic background data from first raw pixel data collected through the image sensor, detect a privacy exposure area in second raw pixel data collected through the image sensor using the basic background data, mask the privacy exposure area to process the second raw pixel data, and transmit the processed second raw pixel data to the image signal processor.

Inventors

최두호
한미란
이상만
조금환
최찬호
정경미
김호영

Assignees

고려대학교 세종산학협력단

Dates

Publication Date: 20260507
Application Date: 20240603
Priority Date: 20240503

Claims (15)

In a method for protecting the privacy of a camera device, before image data is generated, A step of detecting basic background data from first raw pixel data collected through an image sensor; A step of detecting a privacy exposure area in the second raw pixel data collected through the image sensor using the above basic background data; A step of processing the second raw pixel data by masking the privacy exposure area; and Step of transmitting the processed second raw pixel data to an image signal processor Includes, The step of detecting the privacy exposure area above is, A step of calculating a pixel-specific correlation coefficient by comparing the above basic background data and the above second raw pixel data; and A step of detecting at least one of the pixels of the second raw pixel data as the privacy exposure area based on the correlation coefficient. including, method.
delete
In a method for protecting the privacy of a camera device, before image data is generated, A step of detecting basic background data from first raw pixel data collected through an image sensor; A step of detecting a privacy exposure area in the second raw pixel data collected through the image sensor using the above basic background data; A step of processing the second raw pixel data by masking the privacy exposure area; and Step of transmitting the processed second raw pixel data to an image signal processor Includes, While detecting the basic background data from the first raw pixel data, the image sensor stores metadata generated when collecting the first raw pixel data, and statistical data calculated for the first raw pixel data by the image signal processor. including, method.
In Paragraph 3, The step of processing the second raw pixel data above is, A step of masking the privacy exposure area using the above metadata and the above statistical data including, method.
In Article 1, The step of calculating the above correlation coefficient is, A step of calculating the correlation coefficient by partially or entirely comparing the basic background data and the second raw pixel data according to a predetermined window size. including, method.
In Article 1, The step of detecting the above privacy exposure area is, For each of the pixels of the second raw pixel data, if the correlation coefficient is below a predetermined threshold, detecting it as the privacy exposure area including, method.
In a method for protecting the privacy of a camera device, before image data is generated, A step of detecting basic background data from first raw pixel data collected through an image sensor; A step of detecting a privacy exposure area in the second raw pixel data collected through the image sensor using the above basic background data; A step of processing the second raw pixel data by masking the privacy exposure area; and Step of transmitting the processed second raw pixel data to an image signal processor Includes, The step of transmitting the processed second raw pixel data is, A step of performing a bit operation on the processed second raw pixel data to restore the size of the processed second raw pixel data to its size before processing; and Step of transmitting the processed second raw pixel data to the image signal processor including, method.
In a privacy processor that enables privacy protection in a camera device, An interface module configured for interfacing with a camera module having an image sensor; and An image controller connected to the above interface module and configured to process raw pixel data collected through the image sensor. Includes, The above image controller is, Detecting basic background data from the first raw pixel data collected through the image sensor, and Using the above basic background data, a privacy exposure area is detected in the second raw pixel data collected through the image sensor, and By masking the privacy exposure area, the second raw pixel data is processed, and It is configured to transmit the processed second raw pixel data to an image signal processor, and The above image controller is, Calculate pixel-specific correlation coefficients by comparing the above basic background data and the above second raw pixel data, and Configured to detect at least one of the pixels of the second raw pixel data as the privacy exposure area based on the above correlation coefficient, Privacy processor.
delete
In a privacy processor that enables privacy protection in a camera device, An interface module configured for interfacing with a camera module having an image sensor; and An image controller connected to the above interface module and configured to process raw pixel data collected through the image sensor. Includes, The above image controller is, Detecting basic background data from the first raw pixel data collected through the image sensor, and Using the above basic background data, a privacy exposure area is detected in the second raw pixel data collected through the image sensor, and By masking the privacy exposure area, the second raw pixel data is processed, and It is configured to transmit the processed second raw pixel data to an image signal processor, and The above image controller is, While detecting the basic background data from the first raw pixel data, the image sensor is configured to store metadata generated when collecting the first raw pixel data, and statistical data calculated for the first raw pixel data by the image signal processor. Privacy processor.
In Article 10, The above image controller is, Configured to mask the privacy exposure area using the above metadata and the above statistical data, Privacy processor.
In Article 8, The above image controller is, Configured to calculate the correlation coefficient by partially or entirely comparing the basic background data and the second raw pixel data according to a predetermined window size, Privacy processor.
In Article 8, The above image controller is, For each of the pixels of the second raw pixel data, if the correlation coefficient is below a predetermined threshold, the area is configured to be detected as the privacy exposure area. Privacy processor.
In a privacy processor that enables privacy protection in a camera device, An interface module configured for interfacing with a camera module having an image sensor; and An image controller connected to the above interface module and configured to process raw pixel data collected through the image sensor. Includes, The above image controller is, Detecting basic background data from the first raw pixel data collected through the image sensor, and Using the above basic background data, a privacy exposure area is detected in the second raw pixel data collected through the image sensor, and By masking the privacy exposure area, the second raw pixel data is processed, and It is configured to transmit the processed second raw pixel data to an image signal processor, and The above image controller is, A bit operation is performed on the processed second raw pixel data to restore the size of the processed second raw pixel data to its size before processing, and Configured to transmit the processed second raw pixel data to the image signal processor, Privacy processor.
In a camera device for privacy protection, A camera module having an image sensor configured to collect raw pixel data; A privacy processor configured to process the raw pixel data from the image sensor for privacy protection; and An image signal processor configured to generate image information using the raw pixel data from the privacy processor. Includes, The above privacy processor is, Detecting basic background data from the first raw pixel data collected through the image sensor, and Using the above basic background data, a privacy exposure area is detected in the second raw pixel data collected through the image sensor, and By masking the privacy exposure area, the second raw pixel data is processed, and It is configured to transmit the processed second raw pixel data to the image signal processor, and The above privacy processor is, Calculate pixel-specific correlation coefficients by comparing the above basic background data and the above second raw pixel data, and Configured to detect at least one of the pixels of the second raw pixel data as the privacy exposure area based on the above correlation coefficient, Camera device.

Description

Privacy processor enabling personal privacy protection in a camera device and method of operation thereof The present disclosure relates to a privacy processor and a method of operation thereof that enables the protection of an individual's privacy in a camera device. Although there are various definitions of privacy, it is defined as "a state of being free from being the subject of interest or interference in one's life by others." The concept of privacy, which originated in the 18th century, began as a legal framework in the 20th century with the enactment of the Privacy Act by the New York State Legislature in 1903, and it was rapidly strengthened throughout the 20th century. In Korea, under Article 25 of the Personal Information Protection Act, the installation and operation of video information processing devices are prohibited in spaces where there is a concern about privacy infringement, but crimes or disaster situations such as fires can still occur in such spaces. On the other hand, with the advancement of digital technology, low-cost, high-performance camera technology using CMOS image sensors has become commonplace, making it impossible for individuals to voluntarily prevent being filmed by CCTV or personal mobile devices anywhere inside or outside a building. Furthermore, public and private organizations are continuously increasing the installation of CCTVs to protect individuals from the many accidents occurring in society. As described above, considering the perspective of personal privacy that prohibits filming in certain areas, and the case where personal privacy is infringed by many devices without the individual's awareness, a privacy-guaranteeing filming device can solve both problems. Since data already stored in video devices is susceptible to manipulation, image processing performed after storage to protect personal privacy cannot fundamentally safeguard such privacy. Personal privacy can only be protected by applying privacy protection processing at the time of data creation to restore the data to its original state. Figure 1 is a diagram illustrating the schematic configuration of a typical camera device. FIG. 2 is a diagram illustrating the schematic configuration of a camera device according to the present disclosure. Figure 3 is a diagram illustrating the schematic configuration of the image sensor of Figure 2. FIG. 4 is a diagram illustrating the schematic configuration of a privacy processor according to the present disclosure. Figure 5 is a drawing illustrating an exemplary configuration of the privacy processor of Figure 4. FIG. 6 is a drawing illustrating an exemplary configuration of the image controller of FIG. 5. Figure 7 is a diagram illustrating the schematic configuration of the image processing module of the image controller of Figure 4. FIG. 8 is a diagram showing the structure of raw pixel data received by a privacy processor according to the present disclosure. FIG. 9 is a diagram illustrating raw pixel recombination by a privacy processor according to the present disclosure. Figure 10 is a diagram illustrating a window determined to calculate the correlation coefficient in the image processing module of Figure 7. FIG. 11 is a drawing illustrating a method of operation of a privacy processor according to the present disclosure. Hereinafter, various embodiments of the present disclosure are described with reference to the accompanying drawings. Figure 1 is a diagram illustrating the schematic configuration of a general camera device (10). Referring to FIG. 1, a camera device (10) may include a lens (12), an image sensor (13), and an image signal processor (ISP) (15). The lens (12) can form an image on the image sensor (13). The image sensor (13) can collect raw pixel data from the image formed through the lens (12). The image signal processor (15) can store the raw pixel data in memory. The raw pixel data can then be re-stored as a photograph or video through various image processing steps. FIG. 2 is a diagram illustrating the schematic configuration of a camera device (100) according to the present disclosure. Referring to FIG. 2, a camera device (100) may include a camera module (110), a privacy processor (140), and an image signal processor (150). The camera module (110) may include a lens (120) and an image sensor (130). The lens (120) may form an image on the image sensor (130). The image sensor (130) may collect raw pixel data from the image formed through the lens (120). The privacy processor (140) may distinguish between the background and the person from the raw pixel data from the image sensor (130) and mask the area corresponding to the person. The image signal processor (150) may store the raw pixel data in memory. The raw pixel data may be re-stored as a photo or video through various image processing. Specifically, the image signal processor (150) can perform functions such as black level correction, lens correction, bad pixel correction, noise removal,