Search

CN-121330688-B - Handwriting normalization monitoring method and system based on image enhancement

CN121330688BCN 121330688 BCN121330688 BCN 121330688BCN-121330688-B

Abstract

The application discloses a handwriting normalization monitoring method and a handwriting normalization monitoring system based on image enhancement, which relate to the technical field of image processing and intelligent evaluation, wherein the method comprises the following steps: the method comprises the steps of obtaining a handwritten character image through multi-light source adaptive light supplementing, extracting topological structure features of a character skeleton by using a convolutional neural network to perform structural normalization analysis after image preprocessing such as geometric correction and adaptive binarization, extracting nib motion tracks based on an optical flow method and performing sequential normalization analysis, merging the structural and sequential bimodal features, inputting a grading model constructed based on a style migration network to calculate normalization grading, and automatically generating a layered error correction report according to grading grades and deviation types. The method solves the technical problems of incomplete stroke extraction, insufficient structural analysis precision, lack of dynamic writing feature modeling and lack of layering of error correction feedback under complex illumination and shielding conditions in the prior art.

Inventors

  • WU JIANNING
  • DUAN FENG
  • Yang Xishuai
  • LIU JUNFEI
  • ZHANG YADONG

Assignees

  • 大连厚仁科技有限公司

Dates

Publication Date
20260508
Application Date
20251028

Claims (8)

  1. 1. The handwriting normalization monitoring method based on image enhancement is characterized by comprising the following steps of: acquiring an original handwritten character image through an image acquisition device with a multi-light source self-adaptive light supplementing function, and sequentially performing geometric distortion correction, self-adaptive binarization processing based on local image block brightness statistics and morphological opening operation on the original handwritten character image to obtain a continuous binary image of strokes; Inputting the binarized image into a convolutional neural network, extracting skeleton topological structure characteristics of characters, extracting stroke width uniformity parameters and substructure unit interval consistency parameters from the skeleton topological structure characteristics, comparing the skeleton topological structure characteristics with a standard character template in a structural characteristic mode, and generating a structural normalization index and a structural deviation analysis result, wherein the structural deviation analysis result comprises a structural deviation type and position information of the structural deviation type in a handwritten character image; Extracting a pen point motion track in a writing process by an optical flow method, analyzing motion direction change and instantaneous speed change between adjacent track points based on a track point sequence marked by continuous time stamps, giving an evaluation weight higher than a straight track section to the high curvature track section by a dynamic time sequence feature analysis step based on geometrical features of writing motion, analyzing writing sequence normalization and pen-carrying rhythm consistency by time sequence matching, and obtaining a time sequence normalization index and time sequence deviation analysis result, wherein the method comprises the following steps: based on the nib motion track continuously extracted by the optical flow method, acquiring a track point sequence marked by a continuous timestamp; Continuously calculating the ratio of the coordinate displacement between adjacent track points to the time interval to obtain the variation of the motion direction and the instantaneous speed, and updating the direction variation sequence and the speed variation sequence in real time; Based on the statistical characteristics of the writing direction change quantity, detecting and marking an input direction change sequence in a high curvature area, wherein the high curvature area is defined as a continuous track section of which the direction change quantity exceeds a preset direction threshold value; Dynamically allocating evaluation weight coefficients to track segments according to the detected state of the high-curvature region, wherein the weight coefficients comprise a first evaluation weight coefficient endowed by the track segments of the high-curvature region and a second evaluation weight coefficient endowed by the track segments of the non-high-curvature region, and the first evaluation weight coefficient is larger than the second evaluation weight coefficient; Weighting the track point sequence based on the first evaluation weight coefficient and the second evaluation weight coefficient to obtain a weighted track point sequence; Calculating the minimum accumulated path distance between the nib motion track and the standard writing sequence time sequence information in the standard character template through a dynamic time warping algorithm based on the weighted track point sequence, and mapping the minimum accumulated path distance into track sequence deviation degree; Comparing the speed of the weighted track point sequence with the distribution difference of the standard speed curve in the standard writing sequence time sequence information, and calculating to obtain a speed consistency index; Generating a time sequence normalization index and a time sequence deviation analysis result based on the track sequence deviation degree, the speed consistency index, the position information of an abnormal track segment corresponding to the track sequence deviation degree in the pen point movement track and the position information of an abnormal speed segment corresponding to the speed consistency index in the pen point movement track; the time sequence deviation analysis result comprises the time sequence deviation type and the position information of the time sequence deviation type in a writing sequence; the structural normalization index and the time sequence normalization index are subjected to weighted fusion according to preset weights to obtain a comprehensive normalization value, the stroke width uniformity parameter, the sub-structural unit interval uniformity parameter, the structural deviation analysis result and the time sequence deviation analysis result are input into a scoring model based on machine learning training, and normalization scores are obtained through calculation; And automatically determining the error correction strength and the instruction content according to the level of the normalization score, the structural deviation analysis result and the time sequence deviation analysis result, and generating and outputting a layered error correction report, wherein the layered error correction report comprises a targeted copying template and a writing sequence demonstration.
  2. 2. The method for monitoring handwriting normalization based on image enhancement according to claim 1, wherein the morphological opening operation adopts 5×5 rectangular structural elements.
  3. 3. The handwriting normalization monitoring method based on image enhancement according to claim 1, wherein the image acquisition device comprises: The image sensor is provided with 24 analog-to-digital conversion digits and is used for improving the color depth and the brightness resolution of an original image of the handwritten character; the light source module with the self-adaptive light supplementing function is used for automatically adjusting the light supplementing brightness according to the ambient light intensity so as to inhibit reflection and shadow shielding of paper.
  4. 4. The method for monitoring handwriting normalization based on image enhancement according to claim 1, wherein the standard character template comprises standard skeleton topology information and standard writing sequence time sequence information of each character, and the standard writing sequence time sequence information comprises writing sequence, direction and standard speed curves of all strokes.
  5. 5. The method for monitoring handwriting normalization based on image enhancement according to claim 1, wherein the inputting the binarized image into a convolutional neural network, extracting skeleton topological structure characteristics of characters, extracting stroke width uniformity parameters and substructure unit interval uniformity parameters from the skeleton topological structure characteristics, comparing the skeleton topological structure characteristics with standard character templates to generate structure normalization indexes and structure deviation analysis results, comprises: Returning the space coordinates of key points of character strokes in the binary image through a convolutional neural network based on ResNet-50 architecture, and forming the topological structure characteristics of the skeleton by the space coordinates of the key points, wherein the key points comprise stroke end points, crossing points and high curvature points; based on the space coordinates of the key points, connecting adjacent key points through a linear interpolation algorithm to generate continuous stroke center lines; based on character structure specifications, clustering key points with adjacent space positions and topological association into sub-structure units to obtain sub-structure unit segmentation information; calculating the variance of the width of the pixel area communicated with the two normal sides of the stroke center line based on the stroke center line, and obtaining a stroke width uniformity parameter; Based on the sub-structural unit segmentation information, calculating Euclidean distance between minimum circumscribed rectangular center points of the sub-structural units, comparing the Euclidean distance with standard spacing of corresponding sub-structural units in the standard character template, and calculating a relative deviation value to obtain a sub-structural unit spacing consistency parameter; Calculating Hausdorff distance between the skeleton topological structure characteristic and the corresponding key point set in the standard character template, and generating a structure deviation matrix; Carrying out weighted summation on the stroke width uniformity parameter, the substructure unit interval uniformity parameter and the structure deviation matrix to generate a structure normalization index; and generating a structural deviation analysis result based on the deviation value and the position information of each key point in the structural deviation matrix.
  6. 6. The method for monitoring handwriting normalization based on image enhancement according to claim 1, wherein the scoring model based on machine learning training is calculated to obtain a normalization score, and the method comprises the following steps: normalizing the comprehensive normalization value, the stroke width uniformity parameter and the substructure unit interval uniformity parameter to generate a feature vector with unified dimension; encoding the structural deviation analysis result and the position information in the time sequence deviation analysis result into a space weight matrix, wherein the space weight matrix corresponds to the space positions of strokes and sub-structural units in the handwritten character image one by one; The scoring model trained by machine learning is a normalization scoring model based on a style migration network, forward propagation calculation is carried out on the feature vector through the normalization scoring model, and a global stroke weight uniformity score and a global substructure unit interval coordination score are output; Based on the space weight matrix, decomposing and mapping the global stroke weight uniformity score to corresponding positions of all strokes, and carrying out weighted calculation by combining the space weights corresponding to the positions to obtain a local uniformity score of each stroke; based on the space weight matrix, decomposing and mapping the global sub-structure unit interval coordination score to corresponding positions of all sub-structure units, and carrying out weighted calculation by combining the space weights corresponding to the positions to obtain a local coordination score of each sub-structure unit; And carrying out weighted summation on the local uniformity scores of all strokes and the local coordination scores of all the sub-structural units, mapping the summation result to a scale from 1 level to 10 levels, and generating a final normalization score.
  7. 7. The method for monitoring handwriting normalization based on image enhancement according to claim 1, wherein the automatically determining the error correction strength and the instruction content according to the level of the normalization score, the structural deviation analysis result and the time sequence deviation analysis result, and generating and outputting a layered error correction report comprises: The normalization score, the structure deviation type in the structure deviation analysis result and the time sequence deviation type in the time sequence deviation analysis result are input into a predefined mapping rule together, The mapping rule defines the error correction intensity level corresponding to the combination of the normalization scoring level and the deviation type based on the scoring interval division so as to output the error correction intensity level with high intensity, medium intensity or low intensity; Extracting standard images of corresponding strokes or sub-structural units from the standard character templates according to deviation position coordinates in the structural deviation analysis result to generate a first copying template; According to the position information of the abnormal track segment in the time sequence deviation analysis result, standard writing sequence time sequence information of corresponding strokes is called from the standard character template, and a first writing sequence demonstration animation is generated; performing differentiation processing on the first copying template and the first writing sequence demonstration animation according to the error correction intensity level to obtain an enhanced copying template and a writing sequence demonstration animation; The differentiation processing comprises the steps of superposing visual guides of the pen conveying direction and the force channel on the first copying template and reducing the playing speed of the first writing sequence demonstration animation when the error correction intensity level is high intensity, superposing visual guides of the pen conveying direction on the first copying template and moderately reducing the playing speed of the first writing sequence demonstration animation when the error correction intensity level is medium intensity, and directly outputting the first copying template and the first writing sequence demonstration animation as basic contents when the error correction intensity level is low intensity; And combining the reinforced copying template with the writing sequence demonstration animation, the normalization score, the structural deviation analysis result and the time sequence deviation analysis result to generate the layered error correction report.
  8. 8. An image enhancement based handwriting normalization monitoring system for implementing the image enhancement based handwriting normalization monitoring method according to any of claims 1 to 7, comprising: The image acquisition module is used for acquiring an original handwritten character image through an image acquisition device with a multi-light source self-adaptive light supplementing function, and sequentially executing geometric distortion correction, self-adaptive binarization processing based on local image block brightness statistics and morphological opening operation on the original handwritten character image to obtain a continuous binary image of strokes; The image enhancement processing module is used for inputting the binary image into a convolutional neural network, extracting skeleton topological structure characteristics of characters, extracting stroke width uniformity parameters and substructure unit interval uniformity parameters from the skeleton topological structure characteristics, comparing the skeleton topological structure characteristics with standard character templates in structural characteristics, and generating structural normalization indexes and structural deviation analysis results, wherein the structural deviation analysis results comprise structural deviation types and position information of the structural deviation types in handwritten character images; The characteristic extraction and analysis module is used for extracting a nib motion track in a writing process through an optical flow method, analyzing motion direction change and instantaneous speed change between adjacent track points based on a track point sequence marked by continuous time stamps, giving evaluation weight higher than a straight track section to a high-curvature track section through a dynamic time sequence characteristic analysis step based on geometric characteristics of writing motions, and analyzing writing sequence normalization and pen-carrying rhythm consistency through time sequence matching to obtain a time sequence normalization index and a time sequence deviation analysis result, wherein the time sequence deviation analysis result comprises a time sequence deviation type and position information of the time sequence deviation type in the writing sequence; The comprehensive scoring module is used for carrying out weighted fusion on the structural normalization index and the time sequence normalization index according to preset weights to obtain a comprehensive normalization value, inputting the comprehensive normalization value, the stroke width uniformity parameter, the sub-structural unit interval uniformity parameter, the structural deviation analysis result and the time sequence deviation analysis result into a scoring model based on machine learning training, and calculating to obtain normalization scores; The error correction report generation module is used for automatically determining error correction strength and guidance content according to the level of the normalization score, the structural deviation analysis result and the time sequence deviation analysis result, and generating and outputting a layered error correction report which comprises a targeted copying template and a writing sequence demonstration.

Description

Handwriting normalization monitoring method and system based on image enhancement Technical Field The application relates to the technical field of image processing and intelligent evaluation, in particular to a handwriting normalization monitoring method and system based on image enhancement. Background Handwriting normalization monitoring is one of core technologies in the field of intelligent education, and has wide application requirements in character enlightenment teaching, handwriting exercise and special education. However, the existing handwriting normalization monitoring technology still has a plurality of limitations, and accurate and comprehensive automatic evaluation and guidance are difficult to realize. Firstly, in the aspect of image acquisition and preprocessing, the prior art is difficult to cope with complex actual writing scenes. Factors such as uneven illumination, paper reflection, shadow shielding and the like of the writing environment can seriously deteriorate the quality of the acquired image, so that the follow-up stroke extraction is incomplete, broken or stuck, and the accuracy of feature analysis is directly affected. The conventional image binarization and denoising method has poor adaptability and lacks a robust solution to complex optical interference. Secondly, in the dimension of normative analysis, the existing method is often single. Most techniques only focus on the static structure normalization of writing, such as evaluating stroke shape and inter-frame structure by template matching or simple contour comparison, and completely ignore the dynamic time sequence normalization of writing, i.e. the correctness of stroke order and rhythmic feeling of strokes. However, the order and the delivery of strokes are key to the formation of writing habits, and if errors are not corrected in time, the writing speed and the attractive character appearance are affected. In addition, the traditional structural analysis method is mostly based on the characteristics of manual design, has insufficient perceptibility of fine structural deviations such as stroke width uniformity, substructure unit interval consistency and the like, and lacks deep utilization of skeleton topological structure characteristics. Again, in terms of evaluation models, existing scoring mechanisms typically rely on simple rules or shallow machine learning models, which are difficult to quantify "normative" or "harmony" contained in written results that are difficult to speak, resulting in a gap between the scoring results and professional judgment of human teachers, lacking in convincing strength. Finally, in the feedback and guiding links, most of the existing systems only can provide simple correct/error judgment or demonstration of a single mode, and cannot generate a layered and targeted error correction scheme according to the error type and severity of the user. The learner cannot obtain personalized reinforced training contents such as a targeted copying template, dynamic writing sequence demonstration and the like, so that the teaching effect is greatly reduced. Therefore, there is an urgent need in the art for a comprehensive solution capable of overcoming the above drawbacks, implementing high-quality image acquisition in complex scenes, fusing dual normative analysis of structure and time sequence, performing accurate and humanized intelligent scoring, and finally providing targeted hierarchical error correction guidance. Disclosure of Invention The application aims to provide a handwriting normalization monitoring method and system based on image enhancement. The method is used for solving the technical problems of incomplete stroke extraction, insufficient structural analysis precision, lack of dynamic writing feature modeling and lack of layering of error correction feedback under complex illumination and shielding conditions in the prior art. In view of the technical problems, the application provides a handwriting normalization monitoring method and system based on image enhancement. In a first aspect of an embodiment of the present application, there is provided a handwriting normalization monitoring method based on image enhancement, the method including: acquiring an original handwritten character image through an image acquisition device with a multi-light source self-adaptive light supplementing function, and sequentially performing geometric distortion correction, self-adaptive binarization processing based on local image block brightness statistics and morphological opening operation on the original handwritten character image to obtain a continuous binary image of strokes; Inputting the binarized image into a convolutional neural network, extracting skeleton topological structure characteristics of characters, extracting stroke width uniformity parameters and substructure unit interval consistency parameters from the skeleton topological structure characteristics, comparing the skeleton top