CN-121768005-B - Handwriting formula image recognition method and related device based on rule injection

CN121768005BCN 121768005 BCN121768005 BCN 121768005BCN-121768005-B

Abstract

The invention provides a handwriting formula image recognition method and a related device based on rule injection, which belong to the technical field of artificial intelligence and computer vision, and comprise the following steps of carrying out formula region positioning and segmentation on an acquired handwriting formula image to obtain a region image; the method comprises the steps of obtaining a LaTeX format formula by identifying formulas in an obtained area image, carrying out semantic detection and correction on the LaTeX format formula obtained by identification by utilizing a preset rule base to obtain a corrected LaTeX format formula, and comparing the corrected LaTeX format formula with a preset answer pool to obtain a handwriting answer condition.

Inventors

CHEN JIANRUI
WANG YUHAO
WANG YONGZHENG
HAO FEI
SHAO ZHONGSHI
WANG YUXUAN
LEI MING

Assignees

陕西师范大学

Dates

Publication Date: 20260512
Application Date: 20260304

Claims (8)

1. The handwriting formula image recognition method based on rule injection is characterized by comprising the following steps of: Carrying out formula region positioning and segmentation on the obtained handwriting formula image to obtain a region image; identifying a formula in the obtained region image to obtain a LaTeX format formula; Carrying out semantic detection and correction on the LaTeX format formula obtained by recognition by using a preset rule base to obtain a corrected LaTeX format formula; Comparing the corrected LaTeX format formula with a preset answer pool to obtain a handwriting answer condition, wherein: The preset rule base comprises grammar correction functions and formula expression structure rules, wherein the grammar correction functions are corresponding relations between the recognition error description and corrected correct target results; The method for acquiring the grammar correction function comprises the following steps: Identifying the obtained standard answer print body image to obtain a predicted LaTeX format formula; manually correcting the obtained predicted LaTeX format formula to obtain a corrected LaTeX format formula and a grammar correction function; The method for acquiring the structural rule of the formula expression is as follows: Based on the corrected LaTeX format formula, extracting a calculation step, a symbol rule and a fixed solving method in a standard answer to form a formula expression structure rule; the specific construction method of the preset answer pool comprises the following steps: Identifying the obtained standard answer print body image to obtain a predicted LaTeX format formula; manually correcting the obtained predicted LaTeX format formula and marking knowledge point types to obtain a corrected LaTeX format formula; taking the corrected LaTeX format formula as a preset answer pool; semantic detection and correction are carried out on the LaTeX format formula obtained through recognition by using a preset rule base, and a corrected LaTeX format formula is obtained, and the specific method is as follows: and constructing a rule injection agent based on a preset rule base and a preset answer pool, and carrying out semantic detection and correction on the recognized LaTeX format formula by using the rule injection agent to obtain a corrected LaTeX format formula.
2. The method of claim 1, wherein the recognition error description includes symbol ambiguity, bracket missing writing, recognition error line, laTeX symbol misspellings, continuous operator and variable naming non-norms.
3. The method for recognizing handwritten formula image based on rule injection according to claim 1, wherein the recognizing formulas in the obtained area image to obtain a LaTeX format formula comprises: And identifying the formula in the obtained region image by utilizing Unimernet network to obtain the LaTeX format formula.
4. A handwriting formula image recognition system based on rule injection, characterized in that it comprises, based on the recognition method of claim 1: the image segmentation unit is used for carrying out formula area positioning and segmentation on the acquired handwriting formula image to obtain an area image; the formula identification unit is used for identifying the formula in the obtained regional image to obtain a LaTeX format formula; the formula correction unit is used for carrying out semantic detection and correction on the LaTeX format formula obtained by recognition by using a preset rule base to obtain a corrected LaTeX format formula, wherein: The preset rule base comprises grammar correction functions and formula expression structure rules, wherein the grammar correction functions are corresponding relations between the recognition error description and corrected correct target results; and the formula comparison unit is used for comparing the corrected LaTeX format formula with a preset answer pool to obtain a handwriting answer condition.
5. An electronic device comprising a processor and a memory having stored thereon computer instructions that, when executed by the processor, cause the electronic device to perform the method of any of claims 1 to 3.
6. A cluster of computing devices, comprising at least one computing device, each computing device comprising a processor and a memory; A processor of the at least one computing device is configured to execute instructions stored in a memory of the at least one computing device to cause the cluster of computing devices to perform the method of any one of claims 1 to 3.
7. A computer program product comprising computer-executable instructions embodied on the computer program product, the computer executable instructions, when executed, implement the method of any one of claims 1 to 3.
8. A computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, the computer executable instructions, when executed by a processor, implement the method of any one of claims 1 to 3.

Description

Handwriting formula image recognition method and related device based on rule injection Technical Field The invention belongs to the technical field of artificial intelligence and computer vision, and particularly relates to a handwriting formula image recognition method based on rule injection and a related device. Background In the intelligent education and automatic correction scene, handwriting formula recognition and analysis become the core links for realizing 'man-machine collaborative correction' and 'formula semantic understanding'. However, most of the existing formula recognition models are based on a direct mapping mode of optical character recognition (Optical Character Recognition, OCR) and image convolutional neural network (Convolutional Neural Network, CNN), and can only complete recognition at a character level, so that it is difficult to capture the structural hierarchy of formulas, the symbol grammar relationship and the mathematical logic consistency. The simple visual recognition method is very easy to cause the problems of structural mismatch (Structure Mismatch) and semantic ambiguity (Semantic Ambiguity) when facing linear algebraic special structures such as symbol overlapping, matrix indentation, fractional nesting and the like, so that deviation exists between recognized LaTeX output and a real calculation expression. Conventional deep learning methods generally rely on large scale annotation data for training, but in a teaching scenario, standard answers and student work tend to be unbalanced in number and significantly different. Due to the lack of a sufficiently high quality labeling sample, models suffer from insufficient generalization ability in the face of complex handwritten symbols. Meanwhile, the existing OCR system lacks constraint on grammar consistency and symbol dependency structure of mathematical expression, logic verification and error correction cannot be carried out in the recognition stage, and the final output precision and the final output interpretability are seriously affected. Furthermore, although recognition methods based on a transducer and a large language model (Large Language Model, LLM) have progressed in recent years in terms of visual semantic understanding, such models lack Rule-level prior Injection mechanisms (Rule Injection), and it is difficult to achieve efficient self-correction at the symbolic logic and mathematical semantic levels. The pure supervision training is difficult to cover the differences of different teacher writing styles and student homework, and has insufficient adaptability to complex formulas. Disclosure of Invention The invention aims to provide a handwriting formula image recognition method based on rule injection and a related device, which solve the defects in the prior art. In order to achieve the above purpose, the invention adopts the following technical scheme: In a first aspect, the invention provides a handwriting formula image recognition method based on rule injection, which comprises the following steps: Carrying out formula region positioning and segmentation on the obtained handwriting formula image to obtain a region image; identifying a formula in the obtained region image to obtain a LaTeX format formula; Carrying out semantic detection and correction on the LaTeX format formula obtained by recognition by using a preset rule base to obtain a corrected LaTeX format formula; Comparing the corrected LaTeX format formula with a preset answer pool to obtain a handwriting answer condition, wherein: The preset rule base comprises grammar correction functions and formula expression structure rules, wherein the grammar correction functions are corresponding relations between the recognition error description and corrected correct target results, and the formula expression structure rules are formed by calculation steps, symbol rules and a fixed solving method in standard answers. Preferably, the recognition error description includes symbol ambiguity, bracket leak writing, recognition error line, laTeX symbol misspellings, occurrence of continuous operators and variable naming unnormalization. Preferably, the method for obtaining the grammar correction function is as follows: carrying out formula identification on the obtained standard answer print body image to obtain a predicted LaTeX format formula; manually correcting the obtained predicted LaTeX format formula to obtain a corrected LaTeX format formula and a grammar correction function; The method for acquiring the structural rule of the formula expression is as follows: Based on the corrected LaTeX format formula, the calculation step, the symbol rule and the fixed solving method in the standard answer are extracted to form the formula expression structure rule. Preferably, the specific construction method of the preset answer pool is as follows: Identifying the obtained standard answer print body image to obtain a predicted LaTeX format formula; manually c