CN-121999506-A - Handwriting track analysis and correction method and system based on intelligent segmentation and Hungary algorithm

CN121999506ACN 121999506 ACN121999506 ACN 121999506ACN-121999506-A

Abstract

The invention provides a handwriting track analysis and correction method and a handwriting track analysis and correction system based on intelligent segmentation and a Hungary algorithm, the method comprises a handwriting connection pseudo-sample generation step, an example segmentation model adjustment step and a rectangular frame matching step. The system comprises a handwriting connection pseudo-sample generation module, an example segmentation model adjustment module and a rectangular frame matching module. The invention aims to solve the problem of inaccurate end point matching under the condition of connecting line intersection and overlapping in the traditional method, and can realize efficient and accurate automatic correction of the handwriting connecting line problem.

Inventors

WANG TIAN
PENG ZIHAO
LI GUO
ZENG JIANDIAN
ZHANG GUANGXUE

Assignees

北京师范大学珠海校区

Dates

Publication Date: 20260508
Application Date: 20260115

Claims (10)

1. The handwriting track analysis and correction method based on the intelligent segmentation and Hungary algorithm is characterized by comprising the following steps of: Generating a handwriting connection pseudo-sample, namely randomly generating a synthetic connection track by introducing a three-time Bezier curve, simulating handwriting pen pressure change by combining a pen width periodic fluctuation and a random jitter model, and generating a handwriting track pseudo-sample data set; an example segmentation model adjusting step, namely carrying out mixed training on a handwriting track pseudo-sample data set and a real student handwriting track sample based on a universal pre-trained U-Net neural network architecture to construct an example segmentation model; And a rectangular frame matching step, namely extracting a connected component by utilizing a binary mask image output by the example segmentation model after parameter adjustment, calculating the furthest endpoint pair coordinate of each component, constructing an Euclidean distance cost matrix from an endpoint to the center of a rectangular frame of a predefined question area, and solving a minimum total cost matching scheme of the matrix by applying a Hungary algorithm to realize the spatial correspondence of the handwriting track endpoint and the question rectangular frame.
2. The method of claim 1, wherein randomly generating the composite wire trace by introducing a cubic bezier curve comprises: Each composite wire trace is described by a cubic bezier curve, expressed as the following formula: Wherein, the For the four control points generated randomly, uniformly sampling is performed in the image area respectively.
3. The method according to claim 1, wherein the step of simulating the change in pen pressure in handwriting to periodically or randomly fluctuate the width of the pen strokes in the curved direction is performed by: Presetting an average pen width according to the actual handwriting characteristics and the image resolution Setting amplitude A for controlling the amplitude of the fluctuation of the width of the pen touch, setting vibration frequency f to determine the speed of the variation of the width of the pen touch, and setting phase For adjusting the initial position of the fluctuation, and at the same time, defining a random jitter function (T) setting the jitter amplitude according to the fineness of the image and the actual demand; calculating a corresponding stroke width value at each sampling point on the curve, expressed as the following formula: Wherein by a sine function Realizing the periodic variation simulation of the width of the pen touch by using a random dithering function Introducing randomness to make the change of the stroke width more close to the fluctuation generated by factors such as unstable writing force in real handwriting, and based on the stroke width value at each sampling point obtained by calculation And drawing a curve part at the corresponding sampling point position according to the width, so as to form a widening curve with the periodical or random fluctuation effect of the stroke width along the curve direction, and the generated synthetic connecting line track is more similar to the real handwriting in the aspect of the stroke width change.
4. A method according to claim 3, characterized in that: the pixels on which the widening curve is drawn are processed by adopting a Gaussian blur algorithm to simulate noise during scanning or photographing, and the noise is expressed as the following formula: Wherein, the In order to blur the kernel size, As the standard deviation of noise, gaussian noise is accurately added to curve pixels, and noise interference generated during scanning or photographing is simulated; Preparing a plurality of background materials, including at least blank answer sheets, backgrounds with different paper textures and gray uneven backgrounds common in scanning; The connecting line added with noise and various backgrounds are synthesized, and the connecting line is naturally fused into different backgrounds through an image fusion technology to form a final training image; generating corresponding binary mask labels for final training images in a row and column alignment mode Where H is the number of pixels in the image height direction and W is the number of pixels in the image width direction.
5. The method according to claim 1, characterized in that: In the example segmentation model adjustment step, a plurality of handwriting links are cut out from a real test paper A sub-graph, wherein H is the number of pixels in the height direction of the sub-graph, W is the number of pixels in the width direction of the sub-graph, and each cut sub-graph is provided with a manually marked binary mask label The label is used for marking the position information of the handwriting connection line in the subgraph; maintaining the original encoder-decoder symmetrical design structure of the U-Net neural network architecture, repeatedly executing convolution-batch normalization-ReLU operation at the encoder side, wherein the operation formula is as follows: Wherein W is a convolution kernel weight matrix, b is a bias term, and characteristic information of the handwriting link image is extracted through the operation; In the convolution operation process of the encoder, the maximum pooling P is used for carrying out downsampling operation, the resolution of the feature map is gradually reduced, higher-level semantic features are extracted, and the convolution is transposed at the decoder side Up-sampling is performed, the resolution of the feature map is gradually restored, and jump connection is performed with the corresponding encoder layer feature map.
6. The method according to claim 5, wherein: The total loss of the example segmentation model includes classification loss, bounding box regression loss, and mask segmentation loss, where: the classification loss is used to solve the problem of extreme imbalance of positive and negative samples, giving higher weight to difficult samples, expressed as the following formula: Wherein, the Is the first The target class probability of the individual anchor point prediction, And Is a balance factor which is a function of the balance, Is the positive number of samples; the frame loss is used for measuring the overlapping degree of the prediction boundary frame and the real frame, and the calculation formula is as follows: Wherein, the Is the minimum closure area containing the predicted and real frames, Representing area, by calculating GIoU loss between predicted box B pred and real box B gt ; The mask segmentation penalty is used to measure the overlap of the prediction mask and the truth mask, and the calculation formula is: Wherein, the Is the predicted pixel value after Sigmoid, Is a label pixel that is associated with the label, For avoiding zero denominator; And carrying out weighted summation on the classification loss, the frame regression loss and the mask segmentation loss to obtain the total network loss: Where λ box and λ mask are corresponding weight coefficients for balancing the impact of different loss terms on the total loss.
7. The method according to any one of claims 1 to 6, wherein: In the rectangular frame matching step, after the example segmentation model outputs a binary mask, the optimal rectangular frame matching is realized according to the following specific steps: first using a threshold value Probability map output by segmentation model Performing binarization processing, converting the probability map into binary images only containing 0 and 1, and extracting connected component set Each connected component represents a possible handwriting connection area; For each communicating assembly Searching a contour point set through a related function in an OpenCV library By calculating a set of connected component profile points The far two endpoints are determined by Euclidean distance between each pair of points, and the calculation formula is as follows: By traversing all the point pairs, two points with the largest distance are found to be used as two endpoints of the handwriting connection line represented by the communication component, and accurate endpoint information is provided for matching the rectangular frames of the predefined question areas.
8. The method according to claim 7, wherein: setting a binary mask output from the segmentation model to extract a total Each endpoint, noted as Meanwhile, the setting of the link target includes A plurality of rectangular frames, the center of each rectangular frame is marked as ; Constructing an initial cost matrix For each element in the matrix The calculation formula is as follows: Wherein each element Represent the first End point and the first The Euclidean distance between the centers of the rectangular frames is calculated by two pairs of distances between all the endpoints and the centers of the rectangular frames and is filled into the corresponding positions of the matrix to form a complete initial cost matrix, and the matrix reflects the preliminary matching cost between the endpoints and the rectangular frames; Performing replication operation on each column of the initial cost matrix C to obtain a new square matrix 。
9. The method according to claim 8, wherein: Square matrix constructed by Hungary algorithm pair Solving the represented minimum cost perfect matching problem until the number of covered lines is equal to the order of the square matrix; In the operation process of the Hungary algorithm, the matching scheme minimizing the total distance is gradually searched by utilizing the matching cost information between the endpoints represented by the elements in the square matrix and the rectangular frame, and finally a matching function is output The function defines the matching relation between the endpoints corresponding to each element in the square matrix and the rectangular frame, and is expressed as follows: By the matching function Mapping the matching result back to the original A column, namely an original rectangular frame sequence, so that two ends of each connecting line are respectively corresponding to the question rectangular frames; And comparing the matching result with a preset standard answer one by one, giving a corresponding score to a connecting line of which the matching result is consistent with the standard answer, namely the correct matching, and marking the connecting line with the matching error or the missing matching condition as zero score by the system.
10. The handwriting track analysis and correction system based on the intelligent segmentation and Hungary algorithm is characterized by comprising the following components: The handwriting connection pseudo-sample generation module randomly generates a synthetic connection track by introducing a three-time Bezier curve, simulates handwriting pen pressure change by combining a pen width periodic fluctuation and a random jitter model, and simultaneously generates a handwriting track pseudo-sample data set; The example segmentation model adjustment module is used for constructing an example segmentation model by utilizing the mixed training of the formed handwriting track pseudo sample data set and the real handwriting track sample of the student based on the universal pre-trained U-Net neural network architecture, and carrying out parameter adjustment on the example segmentation model through a combined optimization strategy of a weighted cross entropy loss function and a DiceLoss loss function so as to improve the segmentation precision of the handwriting track; and the rectangular frame matching module is used for extracting the connected components and calculating the furthest endpoint pair coordinates of each component by utilizing the binary mask image output by the example segmentation model after parameter adjustment, constructing an Euclidean distance cost matrix from the endpoint to the center of the rectangular frame of the predefined question area, and solving the minimum total cost matching scheme of the matrix by applying a Hungary algorithm to realize the spatial correspondence of the handwriting track endpoint and the question rectangular frame.

Description

Handwriting track analysis and correction method and system based on intelligent segmentation and Hungary algorithm Technical Field The invention relates to the technical fields of computer vision, image processing and machine learning, in particular to a handwriting track analysis and correction method based on intelligent segmentation and Hungary algorithm, which can be widely applied to the scenes of handwriting wiring problem operation correction, examination paper evaluation and the like in the education field, and effectively improves correction efficiency and accuracy. Background In the process of correcting the connection questions, the handwriting connection of students is usually in a curve shape, and may have complex situations such as crossing or overlapping. Traditional correction methods of connection problems rely on straight line detection and connected domain analysis technologies, but when the methods face bent, crossed or overlapped connection lines, the end points of each connection line cannot be accurately identified, so that correction errors are caused. Currently, many correction systems use simple image processing methods based on edge detection or connected domain analysis. These methods cannot effectively cope with changes in handwriting of students, and particularly when the lines are bent or crossed, the end points of each line cannot be accurately distinguished, which results in inaccurate correction. In addition, since it too relies on rough geometric feature extraction, detail variations such as stroke weight or curvature variations are ignored, thereby affecting accuracy and robustness of recognition. In recent years, large models of deep learning-based vision have made significant progress, however, these models still have significant limitations in handling fine-grained visual tasks, especially in handling fine, complex handwritten patterns. In particular, attention loss is susceptible to the problem that large visual models typically focus on global information, but for small local changes in the connection problem, attention loss may occur and the endpoint position of the connection cannot be accurately captured. This can lead to a "phantom" problem, i.e. the model erroneously identifies regions in the image, making predictions that do not match the reality. The capability of processing complex links is insufficient, the links of the handwritten link problem are usually in a curve form, and multiple links may be intersected or overlapped. Traditional deep learning models, particularly models for object detection, are mainly suitable for regular geometric shapes or simple scenes, and for complex wiring forms in wiring questions, it is often difficult to make correct decisions. Therefore, how to efficiently and accurately identify the end points of the connection in the handwriting connection questions, solve the complex situations of intersection and overlapping, etc. is still a difficulty in the current technology. Disclosure of Invention The invention provides a handwriting track analysis and correction method and system based on intelligent segmentation and a Hungary algorithm, which aim to solve the problem of inaccurate end point matching under the condition of line crossing and overlapping in the traditional method and realize efficient and accurate automatic correction of handwriting line problems. The invention realizes the above purpose through the following technical scheme: a handwriting track analysis and correction method based on intelligent segmentation and Hungary algorithm comprises the following steps: Generating a handwriting connection pseudo-sample, namely randomly generating a synthetic connection track by introducing a three-time Bezier curve, simulating handwriting pen pressure change by combining a pen width periodic fluctuation and a random jitter model, and generating a handwriting track pseudo-sample data set; an example segmentation model adjusting step, namely carrying out mixed training on a handwriting track pseudo-sample data set and a real student handwriting track sample based on a universal pre-trained U-Net neural network architecture to construct an example segmentation model; And a rectangular frame matching step, namely extracting a connected component by utilizing a binary mask image output by the example segmentation model after parameter adjustment, calculating the furthest endpoint pair coordinate of each component, constructing an Euclidean distance cost matrix from an endpoint to the center of a rectangular frame of a predefined question area, and solving a minimum total cost matching scheme of the matrix by applying a Hungary algorithm to realize the spatial correspondence of the handwriting track endpoint and the question rectangular frame. According to the handwriting track analysis and correction method based on intelligent segmentation and Hungary algorithm, the synthetic connecting line track is randomly generated by int