KR-102962042-B1 - AI-BASED REAL-TIME VEHICLE LICENSE PLATE MONITORING AND CONTROL SYSTEM

KR102962042B1KR 102962042 B1KR102962042 B1KR 102962042B1KR-102962042-B1

Abstract

An artificial neural network learning method performed by an electronic device is disclosed. The artificial neural network learning method includes the steps of acquiring input data and ground truth data corresponding to the input data, acquiring output data which is a predicted vehicle number from the input data using an artificial neural network module, calculating a loss based on the output data and ground truth data, and training the artificial neural network module based on the loss. The input data is a vehicle image including a plurality of vehicle license plates, and the ground truth data is a vehicle number corresponding to each of the plurality of vehicle license plates.

Inventors

이승오
임태민
임현철
박영호
박장원

Assignees

주식회사 트러스테이

Dates

Publication Date: 20260507
Application Date: 20250915

Claims (13)

As an artificial neural network learning method performed by an electronic device, A step of obtaining input data and correct answer data corresponding to the input data; A step of obtaining output data, which is a predicted vehicle number, from the input data using an artificial neural network module; A step of calculating loss based on the above output data and the above correct answer data; and The method includes the step of training the artificial neural network module based on the above loss; The above input data is, It is a vehicle image containing multiple license plates, and The above correct answer data is, It is a vehicle number corresponding to each of the above plurality of vehicle license plates, and The step of calculating the loss based on the above output data and the above correct answer data is, A step of calculating a numerical value for the degree of frame shaking for each of a plurality of consecutive frames of the above vehicle image; A step of applying a first weight to frames in which the above numerical value is greater than or equal to a preset threshold; A step of applying a second weight to frames where the above numerical value is less than a preset threshold; A step of obtaining the output data based on the frame to which the first weight is applied and the frame to which the second weight is applied; and The method includes the step of calculating the loss by comparing the correct answer data and the output data; The above first weight is, An artificial neural network learning method having a value greater than the second weight mentioned above.
In paragraph 1, The above input data is, A vehicle image including multiple license plates from a captured image that has undergone low-resolution processing, and The step of obtaining the above output data is, A step of dividing the above vehicle image into a plurality of grid units; A step of identifying the probability of license plate area existence for each of the plurality of grid units using the artificial neural network module; A step of identifying a license plate area from the vehicle image based on the identification result; and An artificial neural network learning method comprising the step of identifying a string included in the above license plate area to obtain the above output data.
In paragraph 2, The step of identifying the string included in the above license plate area is, A step of identifying each string included in the above-mentioned license plate area as an individual object; A step of identifying an object at a pre-set location among the individual objects above as a first class; and The method includes the step of identifying a string included in the number plate area based on the first class above, and The above-mentioned first class is, It is a string class corresponding to the region name among the strings included in the above license plate, and The above-mentioned pre-set location is, An artificial neural network learning method comprising a location including at least one of the upper or left side of the above number plate.
In paragraph 3, The step of identifying the string included in the above license plate area is, A step of identifying strings as valid strings, excluding strings corresponding to the first class among the strings, based on a pre-set list of valid strings; A step of identifying numeric strings among the above-identified valid strings as a second class; A step of identifying a character string among the above-identified valid strings as a third class; and An artificial neural network learning method comprising the step of identifying a string included in the license plate area based on the second and third classes.
delete
As a method for recognizing vehicle numbers performed by an electronic device, Step of acquiring a vehicle image; and The method includes the step of obtaining a vehicle number from the vehicle image using an artificial neural network module. The above artificial neural network module is, It is trained using a training dataset in which the input data is a vehicle image containing multiple license plates, the output data is a predicted number, and the correct data is a vehicle number, and For each of a plurality of consecutive frames of the vehicle image, a numerical value for the degree of frame shaking is calculated, a first weight is applied to frames where the numerical value is greater than or equal to a preset threshold, and a second weight is applied to frames where the numerical value is less than the preset threshold, and output data is obtained based on the frames to which the first weight is applied and the frames to which the second weight is applied, and learning is performed using a loss calculated by comparing the correct data and the output data. The above first weight is, A method for recognizing a vehicle number that is greater than the second weight mentioned above.
In paragraph 6, The step of acquiring the above vehicle image is, The method includes the step of obtaining the vehicle image in which low-resolution processing is performed from a captured image including a vehicle license plate; The step of obtaining a vehicle number from the vehicle image using the artificial neural network module above is: A step of identifying a license plate area from the vehicle image above; and A method for recognizing a vehicle number, comprising the step of obtaining a vehicle number from the above-mentioned number plate area.
In Paragraph 7, The vehicle image above is, Includes a first number plate and a second number plate, The step of obtaining a vehicle number from the vehicle image using the artificial neural network module above is: A step of identifying a license plate area corresponding to a first license plate and a second license plate for each of the plurality of consecutive frames from the vehicle image including the plurality of consecutive frames; A step of tracking the same license plate for the consecutive frames based on a tracking correction algorithm for the identified license plate area; and The method includes the step of obtaining a vehicle number for each of the first number plate and the second number plate based on the tracking result. The above tracking correction algorithm is, A method for recognizing a vehicle number, which is an algorithm for tracking and correcting the number plate area for each of the first number plate and the second number plate in a continuous frame to the same area.
In paragraph 8, The step of tracking the same license plate for the consecutive frames based on the tracking correction algorithm for the identified license plate area is: A step of calculating license plate tracking reliability for a first frame among consecutive frames based on the result of identifying license plate areas between consecutive frames; A step of determining the final vehicle number for the license plate area when the above license plate tracking reliability is greater than or equal to a preset threshold reliability; and If the above license plate tracking reliability is less than a preset threshold reliability, the method includes the step of performing license plate recognition based on a second frame following the first frame among the consecutive frames; The above license plate tracking reliability is, A method for recognizing a vehicle number, wherein the number is a numeric value calculated based on at least one of positional similarity or string similarity regarding whether the number plate identification result identified in the first frame is the same as the number plate identification result identified in the frame prior to the first frame.
In paragraph 8, The step of obtaining the vehicle number for each of the first and second number plates based on the above tracking results is: A step of identifying a string corresponding to the license plate area in each of the plurality of consecutive frames; A step of calculating string similarity between the consecutive frames by accumulating identification results; and A method for recognizing a vehicle number, comprising the step of determining a string whose similarity is greater than or equal to a preset threshold as the final vehicle number based on the tracking correction algorithm.
In Paragraph 10, The above tracking correction algorithm is, A method for recognizing license plates, comprising an algorithm including a Kalman filter algorithm and a SORT (Simple Online and Realtime Tracking) algorithm.
In paragraph 6, A step of querying the vehicle number in a database storing registered vehicle information to determine whether the vehicle number is a registered number or an unregistered number; A step of indicating an area of the vehicle number using a first color in the vehicle image when the vehicle number is the registration number; A step of indicating an area of the vehicle number using a second color different from the first color in the vehicle image when the vehicle number is the unregistered number; When receiving user input for the area of the above vehicle number, if the above vehicle number is the registration number, a step of outputting a vehicle type, occupancy information, and entry time corresponding to the above vehicle number; and A method for recognizing a vehicle number, further comprising the step of, when receiving user input for the area of the vehicle number, if the vehicle number is the unregistered number, counting and outputting the number of violations corresponding to the vehicle number.
It includes a processor and memory connected to the processor, The above memory is configured to store a program, and The above processor is configured to execute the above program, and An electronic device in which, when the above program is executed, the method of any one of claims 1 to 4 and claims 6 to 12 is implemented.

Description

AI-Based Real-Time Vehicle License Plate Monitoring and Control System (RCVA) The present disclosure relates to an AI-based real-time vehicle license plate monitoring and control system (RCVA), and in particular to an artificial neural network learning method and a vehicle license plate recognition method and apparatus utilizing an artificial neural network. License plate recognition technology is utilized in various fields, including traffic management, security surveillance, parking systems, and smart city infrastructure. Early license plate recognition systems were based on cameras installed in fixed positions, and could reliably recognize license plates only when the vehicle was captured from a specific angle or location. While these early systems were effective for stationary vehicles or limited environments, their limitations in handling diverse shooting conditions have been pointed out. Subsequently, technologies for vehicle license plate detection and character recognition based on deep learning were developed. However, existing technologies were mostly structures designed for recognition under single vehicle or limited resolution conditions, and had the problem of failing to consider computational efficiency in real-time application environments. Furthermore, existing license plate recognition technologies suffered from a sharp decline in recognition rates due to shaking and blur in moving footage, confusion between license plates when multiple vehicles were included in a single frame, and difficulties in stable recognition caused by image quality degradation in low-light environments or weather conditions such as rain and snow. These factors acted as obstacles that significantly limited the practical application of license plate recognition technology in real-world settings. FIG. 1 is a drawing for explaining a vehicle number recognition method according to one or more embodiments. FIG. 2 is a diagram illustrating a learning method of an artificial neural network module according to one or more embodiments. FIG. 3 is a drawing for explaining a license plate area identification process according to one or more embodiments. FIG. 4 is a diagram illustrating a string identification process according to one or more embodiments. FIG. 5 is a diagram illustrating a string identification process excluding a first class according to one or more embodiments. FIG. 6 is a diagram illustrating a loss calculation method based on output data and correct answer data according to one or more embodiments. FIG. 7 is a drawing for explaining a vehicle number recognition method according to one or more embodiments. FIG. 8 is a drawing for explaining the process of obtaining a vehicle number according to one or more embodiments. FIG. 9 is a drawing for explaining the license plate recognition early termination process according to one or more embodiments. FIG. 10 is a drawing for explaining the process of determining the final vehicle number according to one or more embodiments. FIG. 11 is a diagram illustrating the process of obtaining a vehicle number according to one or more operating system types according to one or more embodiments. FIGS. 12 and FIGS. 13 are drawings for explaining a vehicle number lookup process according to one or more embodiments. FIG. 14 is a drawing for illustrating a parking management UI according to one or more embodiments. FIG. 15 is a block diagram illustrating the configuration of an electronic device according to one or more embodiments. The various embodiments described in this specification are illustrative for the purpose of clearly explaining the technical concept of this disclosure and are not intended to limit it to specific embodiments. The technical concept of this disclosure includes various modifications, equivalents, alternatives, and embodiments optionally combined from all or part of each embodiment described in this specification. Furthermore, the scope of the technical concept of this disclosure is not limited to the various embodiments presented below or the specific descriptions thereof. Terms used in this specification, including technical or scientific terms, may have the meaning generally understood by those skilled in the art to which this disclosure pertains, unless otherwise defined. Expressions used herein such as “comprising,” “may compose,” “possessing,” “possessing,” “having,” and “possessing” imply the existence of the subject feature (e.g., function, operation, or component, etc.) and do not exclude the existence of other additional features. That is, such expressions should be understood as open-ended terms implying the possibility of including a second embodiment. In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described a