US-12620147-B2 - Systems and methods for recognizing hand-drawn shapes

US12620147B2US 12620147 B2US12620147 B2US 12620147B2US-12620147-B2

Abstract

Described herein is a computer implemented method. The method includes receiving, via an input device, first user input drawing an input shape and generating, based on the first user input, original drawing data that includes an ordered set of points that define the input shape. The original drawing is processed to generate an input vector which also includes an ordered set of points. The input shape is then classified as a first template shape by processing the input vector using a machine learning model. A new shape is then generated based on the first template shape and the original drawing data.

Inventors

Kevin Andrew WU WON
Kerry Jayne Halupka
Rowan James KATEKAR

Assignees

CANVA PTY LTD

Dates

Publication Date: 20260505
Application Date: 20240711
Priority Date: 20230313

Claims (20)

1 . A computer implemented method for generating a new shape based on a template shape and an input shape, wherein: the template shape is defined by template shape data that includes an ordered set of template shape points; the ordered set of template shape points has a defined number of template shape points; the input shape is defined by input shape data that includes an ordered set of input shape points; the ordered set of input shape points has the defined number of input shape points; and the method includes: receiving original shape data that includes an ordered set of original shape points, wherein the ordered set of original shape points has a first number of original shape points, the first number being different to the defined number; processing, by a processing unit, the original shape data to generate the input shape data that has the defined number of input shape points; determining a plurality of test rotation values; for each test rotation value in the plurality of test rotation values, performing a test that includes: generating a test shape based on the template shape and the test rotation value; and calculating a dissimilarity score for the test shape, the dissimilarity score being a measure of dissimilarity between the input shape and the test shape; determining, based on the dissimilarity scores, a particular test shape that most closely matches the input shape; and generating the new shape based on the particular test shape.
2 . The computer implemented method of claim 1 , wherein generating the new shape includes: performing a first test corresponding to a first test rotation value of the plurality of test rotation values, the first test including: generating a first test shape by rotating the template shape according to the first test rotation value; and calculating a first dissimilarity score for the first test shape.
3 . The computer implemented method of claim 2 , wherein generating the first test shape further includes translating the template shape.
4 . The computer implemented method of claim 2 , wherein generating the first test shape further includes translating the template shape by: calculating an input shape centroid; calculating a template shape centroid; and translating the template shape so that the template shape centroid equals the input shape centroid.
5 . The computer implemented method of claim 2 , wherein generating the first test shape further includes stretching the template shape.
6 . The computer implemented method of claim 2 , wherein generating the first test shape further includes stretching the template shape in two orthogonal directions.
7 . The computer implemented method of claim 6 , wherein stretching the template shape in two orthogonal directions includes stretching the template shape so an x-coordinate variance of the stretched template shape matches an x-coordinate variance of the input shape and a y-coordinate variance of the stretched template shape matches a y-coordinate variance of the input shape.
8 . The computer implemented method of claim 2 , wherein performing the first test further includes: generating a plurality of test shape variants, each test shape variant based on the first test shape; calculating a dissimilarity sub-score for each of the test shape variants, the dissimilarity sub-score for a selected test shape variant being a measure of dissimilarity between the input shape and the selected test shape variant; and wherein generating the new shape based on the particular test shape includes: determining, based on the dissimilarity sub-scores, a particular test shape variant that most closely matches the input shape; and generating the new shape based on the particular test shape variant.
9 . The computer implemented method of claim 8 , wherein generating the plurality of test shape variants based on the first test shape includes generating a first variant by point-order rotating the first test shape.
10 . The computer implemented method of claim 1 , wherein the first number is greater than the defined number.
11 . The computer implemented method of claim 1 , further including processing a vector graphic to generate the template shape data.
12 . The computer implemented method of claim 11 , wherein processing the vector graphic to generate the template shape data includes generating a linear approximation of the vector graphic.
13 . A computer processing system including: a processing unit; and a non-transitory computer-readable storage medium storing instructions, which when executed by the processing unit, cause the processing unit to perform a method for generating a new shape based on a template shape and an input shape, wherein: the template shape is defined by template shape data that includes an ordered set of template shape points; the ordered set of template shape points has a defined number of template shape points; the input shape is defined by input shape data that includes an ordered set of input shape points; the ordered set of input shape points has the defined number of input shape points; and the method includes: receiving original shape data that includes an ordered set of original shape points, wherein the ordered set of original shape points has a first number of original shape points, the first number being different to the defined number; processing the original shape data to generate the input shape data that has the defined number of input shape points; determining a plurality of test rotation values; for each test rotation value in the plurality of test rotation values, performing a test that includes: generating a test shape based on the template shape and the test rotation value; and calculating a dissimilarity score for the test shape, the dissimilarity score being a measure of dissimilarity between the input shape and the test shape; determining, based on the dissimilarity scores, a particular test shape that most closely matches the input shape; and generating the new shape based on the particular test shape.
14 . The computer processing system of claim 13 , wherein generating the new shape includes: performing a first test corresponding to a first test rotation value of the plurality of test rotation values, the first test including: generating a first test shape by rotating the template shape according to the first test rotation value; and calculating a first dissimilarity score for the first test shape.
15 . The computer processing system of claim 14 , wherein generating the first test shape further includes translating the template shape.
16 . The computer processing system of claim 13 , wherein the method further includes processing a vector graphic to generate the template shape data.
17 . A non-transitory storage medium storing instructions executable by processing unit to cause the processing unit to perform a method for generating a new shape based on a template shape and an input shape, wherein: the template shape is defined by template shape data that includes an ordered set of template shape points; the ordered set of template shape points has a defined number of template shape points; the input shape is defined by input shape data that includes an ordered set of input shape points; the ordered set of input shape points has the defined number of input shape points; and the method includes: receiving original shape data that includes an ordered set of original shape points, wherein the ordered set of original shape points has a first number of original shape points, the first number being different to the defined number; processing the original shape data to generate the input shape data that has the defined number of input shape points; determining a plurality of test rotation values; for each test rotation value in the plurality of test rotation values, performing a test that includes: generating a test shape based on the template shape and the test rotation value; and calculating a dissimilarity score for the test shape, the dissimilarity score being a measure of dissimilarity between the input shape and the test shape; determining, based on the dissimilarity scores, a particular test shape that most closely matches the input shape; and generating the new shape based on the particular test shape.
18 . The non-transitory storage medium of claim 17 , wherein generating the new shape includes: performing a first test corresponding to a first test rotation value of the plurality of test rotation values, the first test including: generating a first test shape by rotating the template shape according to the first test rotation value; and calculating a first dissimilarity score for the first the test shape.
19 . The non-transitory storage medium of claim 18 , wherein generating the first test shape further includes translating the template shape.
20 . The non-transitory storage medium of claim 17 , wherein the method further includes processing a vector graphic to generate the template shape data.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a Continuation application of U.S. Non-Provisional application Ser. No. 18/599,906, filed on Mar. 8, 2024, that claims priority to Australian Patent Application No. 2023201533, filed on Mar. 13, 2023, that are each hereby incorporated by reference in their entirety. TECHNICAL FIELD The present disclosure is directed to systems and methods for recognising hand-drawn shapes. BACKGROUND Various computer applications for creating and publishing digital designs exist. Generally speaking, such applications allow users to create a design by, for example, creating a page and adding design elements to that page. In order to add design elements to a page a design application may provide various tools. For example, a design application may provide a draw tool which allows a user to create a design element by drawing it on the page (using, for example, a cursor control or touch input device). In certain instances, a user may use such a draw tool to try and draw geometric shapes e.g. circles, rectangles, arrows, stars, and/or other shapes. Where a draw tool is used to draw such a shape, however, the shape as drawn will almost always be imperfect due to the imprecision of the drawing input. Various imperfections in a shape drawn using a drawing tool may arise, for example lines that are intended to be horizontal or vertical (or parallel to other lines in the drawn shape) will not be quite horizontal/vertical/parallel, curves that are intended to be of a constant radius will not be, angles between shape segments that are supposed to be 90 degrees (or complementary to other angles in the drawn shape) will not be. SUMMARY Described herein is a computer implemented method including: receiving, via an input device, first user input drawing an input shape; generating, based on the first user input, original drawing data, the original drawing data including an ordered set of points that define the input shape; processing the original drawing data to generate an input vector, the input vector including an ordered set of points; and classifying the input shape into a first class that corresponds to a first template shape by processing the input vector using a machine learning model; generating a new shape based on the first template shape and the original drawing data; and outputting the new shape. BRIEF DESCRIPTION OF THE DRAWINGS In the drawings: FIG. 1 depicts example hand drawn shapes and corresponding template shapes. FIG. 2 is a diagram depicting a computing environment in which various features of the present disclosure may be implemented. FIG. 3 is a block diagram of a computer processing system configurable to perform various features of the present disclosure. FIG. 4 depicts an example user interface. FIG. 5 is a flowchart depicting operations performed in a method for recognising an input shape. FIG. 6 is a flowchart depicting operations performed in an example method for standardising drawing data to a defined number of points. FIG. 7 is a flowchart depicting operations performed in an alternative example method for standardising drawing data to a defined number of points. FIG. 8 is a flowchart depicting operations performed in an example method for generating a new shape based on a template shape and an input shape. FIG. 9 is a flowchart depicting operations performed in an example method for generating a set of training examples. FIG. 10 depicts an example user interface. While the description is amenable to various modifications and alternative forms, specific embodiments are shown by way of example in the drawings and are described in detail. It should be understood, however, that the drawings and detailed description are not intended to limit the invention to the particular form disclosed. The intention is to cover all modifications, equivalents, and alternatives falling within the scope of the present invention as defined by the appended claims. DETAILED DESCRIPTION In the following description numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, that the present invention may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessary obscuring. As discussed above, design applications for use in creating designs are known. As also discussed above, one type of tool such applications may provide is a draw tool. In the context of the present specification, a draw tool is a tool that allows a user to draw a path using an input device—e.g. to draw a path using a mouse or other cursor control device or to draw a path on a touch screen. Based on the input, the application renders the path drawn on a display and generates an ordered set of points (e.g. coordinate pairs) that describe the path. A draw tool such as this may be known as a pen tool, a freehand drawing tool, a fre