CN-121980553-A - Password point coordinate recognition method and device of actual grid dot matrix based on gesture password

CN121980553ACN 121980553 ACN121980553 ACN 121980553ACN-121980553-A

Abstract

The invention discloses a password point coordinate recognition method and device of an actual grid lattice based on gesture passwords, comprising the steps of obtaining an interface image of the actual grid lattice used for inputting the gesture passwords, matching a password point graph in a template image with the interface image to obtain a matching result, determining unique candidate points corresponding to each password point in the interface image based on the matching result, constructing a theoretical grid lattice according to preset intervals, matching the theoretical password point with all the unique candidate points, determining whether unique candidate points serving as supporting points exist or not, counting the number of the supporting points and corresponding matching quality scores, determining a theoretical grid lattice matched with the actual grid lattice based on the number of the supporting points existing in each theoretical grid lattice and the corresponding matching quality scores, and calculating the coordinates of the theoretical password points of the theoretical grid lattice as the coordinates of the corresponding password points in the actual grid lattice. The gesture password interface automatic recognition and analysis method can realize automatic recognition and analysis of the gesture password interface.

Inventors

LIN KAIXING
SUN YI
ZHANG RONGPING
ZHANG HUIJI

Assignees

厦门市美亚柏科信息安全研究所有限公司

Dates

Publication Date: 20260505
Application Date: 20251208

Claims (10)

1. The password point coordinate recognition method of the actual grid lattice based on the gesture password is characterized by comprising the following steps of: Acquiring an interface image of an actual grid lattice which is displayed on a mobile terminal and is used for inputting a gesture password and is formed by N multiplied by M password points, creating a template image containing password point patterns, matching the password point patterns in the template image with the interface image to obtain a matching result, extracting original candidate points based on the matching result, and determining unique candidate points corresponding to each password point in the interface image; Traversing each unique candidate point, and constructing a plurality of theoretical grid lattices containing N multiplied by M theoretical code points by taking the unique candidate points as original points according to preset intervals; Traversing each theoretical code point in each theoretical grid lattice, matching the theoretical code points with all unique candidate points, determining whether the theoretical code points have unique candidate points serving as corresponding support points, and counting the number of the support points existing in each theoretical grid lattice and the corresponding matching quality fraction; And determining a theoretical grid lattice matched with the actual grid lattice based on the number of the supporting points existing in each theoretical grid lattice and the corresponding matching quality score, and calculating the coordinates of each theoretical code point as the coordinates of the corresponding code point in the actual grid lattice.
2. The method for recognizing the code point coordinates of the actual grid lattice based on the gesture code according to claim 1, wherein the matching of the code point pattern in the template image and the interface image is performed to obtain a matching result, the original candidate points are extracted based on the matching result, and n×m unique candidate points are determined, specifically comprising: Respectively graying the interface image and the template image to obtain a template gray image and an interface gray image, wherein the template image is a single code point graph; Matching a matching region similar to the password point pattern in the template gray level image in the interface gray level image by using MATCHTEMPLATE functions of an OpenCV library, and obtaining a corresponding matching value to be used as a matching result corresponding to each password point in the interface image; Taking the center point of a matching area with the matching value exceeding a preset threshold value in the matching result corresponding to each password point in the interface image as an original candidate point corresponding to each password point in the interface image; And performing non-maximum value inhibition processing on all original candidate points corresponding to each password point in the interface image to obtain a unique candidate point corresponding to each password point in the interface image.
3. The method for recognizing the code point coordinates of the actual grid lattice based on the gesture code according to claim 1, wherein a plurality of theoretical grid lattices containing n×m theoretical code points are constructed with the unique candidate points as origins and according to a preset interval, specifically comprising: Taking the unique candidate point serving as an origin as a theoretical code point of the upper left corner in the theoretical grid lattice; and generating the rest theoretical code points according to a preset interval from the theoretical code points at the upper left corner in the theoretical grid lattice.
4. The method for recognizing the code point coordinates of the actual grid lattice based on the gesture code according to claim 1, wherein the method for recognizing the code point coordinates of the actual grid lattice based on the gesture code is characterized by matching the theoretical code point with all unique candidate points, determining whether the theoretical code point has unique candidate points serving as corresponding support points, and counting the number of the support points existing in each theoretical grid lattice and the matching quality score of each theoretical code point and the corresponding support points, and specifically comprises the following steps: Calculating the distance between the center point of each unique candidate point and the center point of the theoretical code point, and taking the unique candidate point, of which the distance between the center point of each unique candidate point and the center point of the theoretical code point is within a preset tolerance radius, as a support point corresponding to the theoretical code point; Repeating the steps for each theoretical code point in each theoretical grid lattice, finding out supporting points corresponding to all theoretical code points in each theoretical grid lattice, and summing the numbers of the supporting points to obtain the number of the supporting points existing in each theoretical grid lattice; And calculating the sum of squares of the distances between the center point of each theoretical code point in each theoretical grid lattice and the center point of each supporting point corresponding to the center point of each theoretical code point, obtaining the sum of squares of the total distances between each theoretical code point in each theoretical grid lattice and the corresponding supporting point, and taking the sum of squares of the total distances as the matching quality score corresponding to each theoretical grid lattice.
5. The method for recognizing the code point coordinates of the actual grid lattice based on the gesture code according to claim 4, wherein the method for recognizing the code point coordinates of the actual grid lattice based on the gesture code is characterized by determining the theoretical grid lattice matched with the actual grid lattice based on the number of the support points existing in each theoretical grid lattice and the corresponding matching quality score, and calculating the coordinates of each theoretical code point as the coordinates of the corresponding code point in the actual grid lattice, and specifically comprises: in response to determining that only one theoretical grid lattice with the largest number of support points exists, taking the theoretical grid lattice with the largest number of support points as a theoretical grid lattice matched with the actual grid lattice; In response to determining that the number of the theoretical grid lattices with the largest number of the support points exceeds one, selecting the theoretical grid lattice with the smallest sum of squares of total distances as the theoretical grid lattice matched with the actual grid lattice; acquiring the coordinates of the origin of a theoretical grid lattice matched with an actual grid lattice by adopting a target identification mode; And calculating the coordinates of other theoretical code points according to the coordinates of the origin of the theoretical grid lattice matched with the actual grid lattice and the corresponding preset intervals, and taking the coordinates as the coordinates of the corresponding code points in the actual grid lattice.
6. The method for identifying the code point coordinates of the actual grid lattice based on the gesture code according to claim 1, wherein the interface image is obtained by shooting with a camera or is obtained by screen capturing.
7. The utility model provides a password point coordinate recognition device of actual grid dot matrix based on gesture password which characterized in that includes: The template matching module is configured to acquire an interface image of an actual grid lattice which is displayed on the mobile terminal and is used for inputting a gesture password and is formed by N multiplied by M password points, create a template image containing password point graphics, match the password point graphics in the template image with the interface image to obtain a matching result, extract original candidate points based on the matching result, and determine unique candidate points corresponding to each password point in the interface image; The theoretical grid lattice construction module is configured to traverse each unique candidate point, and construct a plurality of theoretical grid lattices containing N multiplied by M theoretical code points according to preset intervals by taking the unique candidate points as original points; The theoretical password point matching module is configured to traverse each theoretical password point in each theoretical grid lattice, match the theoretical password points with all unique candidate points, determine whether the theoretical password points have unique candidate points serving as corresponding support points, and count the number of the support points existing in each theoretical grid lattice and the corresponding matching quality score; And the screening module is configured to determine a theoretical grid lattice matched with the actual grid lattice based on the number of the supporting points existing in each theoretical grid lattice and the corresponding matching quality score, and calculate the coordinates of each theoretical code point as the coordinates of the corresponding code point in the actual grid lattice.
8. An electronic device, comprising: One or more processors; storage means for storing one or more programs, When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-6.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-6.
10. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-6.

Description

Password point coordinate recognition method and device of actual grid dot matrix based on gesture password Technical Field The invention relates to the field of image recognition, in particular to a password point coordinate recognition method and device of an actual grid lattice based on gesture passwords. Background The mobile phone is used as an essential intelligent mobile terminal in daily life, in the process of evidence obtaining of the mobile phone, the password is often required to be input for unlocking and opening USB debugging, the gesture password is a common mobile phone password, and nine points in a screen are required to be connected according to a specific sequence for inputting the gesture password. The current mainstream method is to manually input a gesture password, if the gesture password is automatically input, password point coordinates of a grid lattice used for inputting the gesture password need to be acquired first, the conventional grid lattice adopts a nine-grid pattern, but the password point coordinates of the grid lattice used for inputting the gesture password in interfaces of mobile terminals of different brands or different systems are not the same. If the target identification mode is adopted, only the code points can be identified, but the specific code points in the nine-square lattice corresponding to the code points cannot be confirmed, so that the manual operation requirement can be reduced by the method capable of automatically identifying the code point coordinates, and the evidence obtaining process is more convenient. Disclosure of Invention The application aims to provide a password point coordinate recognition method and device of an actual grid lattice based on gesture passwords aiming at the technical problems. In a first aspect, the present invention provides a method for identifying password point coordinates of an actual grid lattice based on gesture password, including the following steps: acquiring an interface image of an actual grid lattice which is displayed on a mobile terminal and is used for inputting a gesture password and is formed by N multiplied by M password points, creating a template image containing a password point graph, matching the password point graph in the template image with the interface image to obtain a matching result, extracting original candidate points based on the matching result, and determining unique candidate points corresponding to each password point in the interface image; traversing each unique candidate point, and constructing a plurality of theoretical grid lattices containing N multiplied by M theoretical code points by taking the unique candidate points as original points according to preset intervals; Traversing each theoretical code point in each theoretical grid lattice, matching the theoretical code points with all unique candidate points, determining whether the theoretical code points have unique candidate points serving as corresponding support points, and counting the number of the support points existing in each theoretical grid lattice and the corresponding matching quality score; and determining a theoretical grid lattice matched with the actual grid lattice based on the number of the support points existing in each theoretical grid lattice and the corresponding matching quality score, and calculating the coordinates of each theoretical code point as the coordinates of the corresponding code point in the actual grid lattice. Preferably, the matching of the code point graph in the template image and the interface image is performed to obtain a matching result, the original candidate points are extracted based on the matching result, and n×m unique candidate points are determined, specifically including: Graying the interface image and the template image respectively to obtain a template gray image and an interface gray image, wherein the template image is a single code point graph; matching a matching region similar to the password point pattern in the template gray level image in the interface gray level image by using MATCHTEMPLATE functions of the OpenCV library, and obtaining a corresponding matching value to be used as a matching result corresponding to each password point in the interface image; Taking the center point of a matching area with the matching value exceeding a preset threshold value in the matching result corresponding to each password point in the interface image as an original candidate point corresponding to each password point in the interface image; and performing non-maximum value inhibition processing on all original candidate points corresponding to each password point in the interface image to obtain a unique candidate point corresponding to each password point in the interface image. Preferably, a plurality of theoretical grid lattices including n×m theoretical code points are constructed with the unique candidate points as origins and according to a preset pitch, specifi