EP-3929808-B1 - FINGERPRINT AUTHENTICATION DEVICE, DISPLAY DEVICE INCLUDING THE SAME, AND METHOD OF AUTHENTICATING FINGERPRINT

EP3929808B1EP 3929808 B1EP3929808 B1EP 3929808B1EP-3929808-B1

Inventors

LEE, SOON GYU
KANG, JIN A
KIM, MI YOUNG
KIM, JIN WOO
BEEN, KYOUNG HUN
JEON, BYEONG KYU

Dates

Publication Date: 20260506
Application Date: 20210616

Claims (11)

A fingerprint authentication device comprising: a sensor unit configured to output a sensing signal by sensing a fingerprint; an image processing unit configured to generate a first fingerprint image based on the sensing signal; a storage unit configured to store a template including an enrolled image; and a learning unit configured to generate a first pseudo image and add the first pseudo image to the template, wherein the learning unit is configured to convert the enrolled image into the first pseudo image by using a first image conversion model, wherein a signal-to-noise ratio of the first pseudo image is smaller than that of the enrolled image, and wherein the learning unit is configured to set or update the first image conversion model through learning from fingerprint images for learning, which are provided from outside of the fingerprint authentication device, wherein the fingerprint images for learning include high-quality fingerprint images of which signal-to-noise ratios are within a reference quality range and low-quality fingerprint images of which signal-to-noise ratios are out of the reference quality range with respect to a plurality of objects, wherein the plurality of objects comprises a plurality of fingerprints of users, wherein the image processing unit is configured to generate the first fingerprint image in a fingerprint enrollment period, and is configured to generate a second fingerprint image in a fingerprint authentication period, wherein the enrolled image includes at least a portion of the first fingerprint image, and wherein the first pseudo image is different from the second fingerprint image, the fingerprint authentication device further comprising an authentication unit configured to compare a fingerprint image generated by the image processing unit with the template, and configured to output a comparison result, wherein the image processing unit is configured to generate a third fingerprint image corresponding to the second fingerprint image while changing a driving condition of the sensor unit in the fingerprint authentication period, the second fingerprint image and the third fingerprint image obtained from a substantially same fingerprint, wherein a signal-to noise ratio of the third fingerprint image is smaller than that of the second fingerprint image, wherein the storage unit is configured to store the second fingerprint image and the third fingerprint image based on the comparison result of the authentication unit indicating a successful authentication, and wherein the learning unit is configured to: (i) set a second image conversion model by learning from the second fingerprint image and the third fingerprint image or update the first image conversion model by learning from at least one of the first and second fingerprint images and the third fingerprint image, (ii) to convert the first fingerprint image into a second pseudo image by using the second image conversion model or the updated first image conversion model, and (iii) to add the second pseudo image to the template.
The fingerprint authentication device of claim 1, wherein the sensor unit includes a photo sensor configured to sense light reflected by the fingerprint, wherein the image processing unit is configured to generate the second fingerprint image by receiving the reflected light for a first exposure time through the sensor unit, and wherein the image processing unit is configured to generate the third fingerprint image by receiving the reflected light for a second exposure time through the sensor unit.
The fingerprint authentication device of claim 2, wherein the second exposure time is shorter than the first exposure time.
The fingerprint authentication device of any preceding claim, wherein the sensor unit includes an ultrasonic sensor configured to sense an ultrasonic wave reflected by the fingerprint, wherein the image processing unit is configured to generate the second fingerprint image by emitting an ultrasonic wave having a first frequency through the sensor unit, and wherein the image processing unit is configured to generate the third fingerprint image by emitting an ultrasonic wave having a second frequency through the sensor unit.
The fingerprint authentication device of claim 4, wherein the second frequency is lower than the first frequency.
The fingerprint authentication device of any preceding claim, wherein the learning unit is configured to update the first image conversion model by additionally learning from the second fingerprint image and the third fingerprint image, is configured to convert the first fingerprint image into a third pseudo image by using the updated first image conversion model, and is configured to add the third pseudo image to the template.
A display device comprising: a display panel including pixels emitting light; and a fingerprint authentication device disposed on a bottom of the display panel, the fingerprint authentication device configured to sense a fingerprint based on a signal reflected by the fingerprint, wherein the fingerprint authentication device is the fingerprint authentication device according to any preceding claim.
A method of authenticating a fingerprint, the method comprising the steps of: generating a first fingerprint image of a first object through a sensor unit in a fingerprint enrollment period; adding, to a template, enrolled images including at least a portion of the first fingerprint image; generating a first pseudo image based on the enrolled image using a first image conversion model, wherein a signal-to-noise ratio of the first pseudo image is smaller than that of the enrolled image, and wherein the step of generating of the first pseudo image includes the steps of: setting the first image conversion model through learning from a fingerprint; and converting the enrolled image into the first pseudo image by using the first image conversion model, and wherein the enrolled images include high-quality fingerprint images of which signal-to-noise ratios are within a reference quality range and low-quality fingerprint images of which signal-to-noise ratios are out of the reference quality range, with respect to a plurality of objects, wherein the plurality of objects comprises a plurality of fingerprints of users, generating a second fingerprint image through the sensor unit in a fingerprint authentication period; adding the first pseudo image to the template, wherein the first pseudo image is different from the second fingerprint image; and comparing the second fingerprint image with the template and outputting a comparison result, wherein the step of generating of the second fingerprint image includes the step of generating a third fingerprint image corresponding to the second fingerprint image while changing a driving condition of the sensor unit, the second fingerprint image and the third fingerprint image obtained from a substantially same object, and wherein a signal-to-noise ratio of the third fingerprint image is lower than that of the second fingerprint image; storing the second fingerprint image and the third fingerprint image based on the comparison result; setting a second image conversion model by learning from the second fingerprint image and the third fingerprint image; converting the enrolled image into a second pseudo image by using the second image conversion model; and adding the second pseudo image to the template.
The method of any of claim8, wherein the sensor unit includes a photo sensor configured to sense light reflected by a fingerprint, and wherein the step of generating the third fingerprint image includes the steps of: generating the second fingerprint image by receiving the reflected light for a first exposure time through the sensor unit; and generating the third fingerprint image by receiving the reflected light for a second exposure time through the sensor unit.
The method of claim 9, wherein the sensor unit includes an ultrasonic sensor configured to sense an ultrasonic wave reflected by a fingerprint, and wherein the step of generating of the third fingerprint image includes the steps of: generating the second fingerprint image by emitting an ultrasonic wave having a first frequency through the sensor unit; and generating the third fingerprint image by emitting an ultrasonic wave having a second frequency through the sensor unit.
The method of any of claims 8 to 10, further comprising the steps of: setting a first image conversion model through learning from fingerprint images; generating fingerprint images having different signal-to-noise ratios with respect to a first object while changing a driving condition of the sensor unit; and updating the first image conversion model by learning from the fingerprint images.

Description

BACKGROUND FIELD Implementations of the invention relate generally to a fingerprint authentication device, a display device including the same, and a method of authenticating a fingerprint and more specifically, to a fingerprint authentication device using a learning technique, a display device including the same, and a method of authenticating a fingerprint. DISCUSSION OF THE BACKGROUND Recently, as display devices such as smartphones and tablet PCs are used in many fields, a biometric authentication method using a fingerprint of a user, and the like has been widely used. In order to provide a fingerprint sensing function, a fingerprint sensor may be built in a display device or be attached to the top and/or the bottom of the display device. The display device integrated with the fingerprint sensor is referred to as a Fingerprint on Display (FoD). The FoD may include, for example, a photosensitive type fingerprint sensor. The photosensitive type FoD may use, as a light source, a light emitting device included in a pixel, and include a photo sensor array as the fingerprint sensor. The photo sensor array may be implemented as, for example, a CMOS Image Sensor (CIS). The FoD may acquire a fingerprint image through the fingerprint sensor, and compare the fingerprint image with a pre-enrolled fingerprint image (e.g., a template including the same), thereby authenticating a fingerprint (i.e., authenticating that a sensed fingerprint is an enrolled fingerprint of a user). According to a state of skin (e.g., when the skin is dry or when the skin is contracted) or an authentication environment (e.g., when surroundings are very bright or dark), the accuracy of fingerprint recognition may be lowered, and a fingerprint authentication result of the same fingerprint may be changed. The above information disclosed in this Background section is only for understanding of the background of the inventive concepts, and, therefore, it may contain information that does not constitute prior art. US 2019/122024 and US 2017/076132 both provide disclosures related to fingerprint authentication. SUMMARY According to a first aspect, there is provided a fingerprint authentication device according to claim 1. According to a second aspect, there is provided a method of fingerprint authentication according to claim 8. Details of embodiments are provided in the dependent claims. Applicant discovered that when a state of skin of a user or an authentication environment is not normal, the accuracy of fingerprint recognition of a fingerprint authentication device of a display device is degraded. Fingerprint authentication devices of display devices constructed according to the principles and implementations of the invention are capable of improving the accuracy of fingerprint authentication by using learning techniques of fingerprint images. Methods of authenticating a fingerprint according to the principles and of the invention are capable of improving the accuracy of the fingerprint authentication by using learning techniques of fingerprint images. Additional features of the inventive concepts will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the inventive concepts. According to one or more embodiments of the invention, a fingerprint authentication device includes: a sensor unit configured to output a sensing signal by sensing a fingerprint; an image processing unit configured to generate a fingerprint image based on the sensing signal; a storage unit configured to store a template including an enrolled image; and a learning unit configured to generate a first pseudo image and add the first pseudo image to the template. A signal-to-noise ratio of the first pseudo image may be smaller than that of the enrolled image. The learning unit may be configured to convert the enrolled image into the first pseudo image by using a first image conversion model. The image processing unit may be configured to generate a first fingerprint image in a fingerprint enrollment period, and may be configured to generate a second fingerprint image in a fingerprint authentication period, wherein the enrolled image may include at least a portion of the first fingerprint image, and wherein the first pseudo image may be different from the second fingerprint image. The learning unit may be configured to set the first image conversion model through learning of fingerprint images for learning, which are provided from outside of the fingerprint authentication device, and wherein the fingerprint images for learning may include high-quality fingerprint images of which signal-to-noise ratios are within a reference quality range and low-quality fingerprint images of which signal-to-noise ratios are out of the reference quality range with respect to a plurality of objects. The fingerprint authentication device may further include an authentication unit configured to compare the fingerprint im