CN110751137A - Method and system for automatically solving mathematical problem - Google Patents
Method and system for automatically solving mathematical problem Download PDFInfo
- Publication number
- CN110751137A CN110751137A CN201910831074.2A CN201910831074A CN110751137A CN 110751137 A CN110751137 A CN 110751137A CN 201910831074 A CN201910831074 A CN 201910831074A CN 110751137 A CN110751137 A CN 110751137A
- Authority
- CN
- China
- Prior art keywords
- mathematical
- calculation
- tree
- analysis
- questions
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 81
- 238000004364 calculation method Methods 0.000 claims abstract description 119
- 238000004458 analytical method Methods 0.000 claims abstract description 57
- 239000011159 matrix material Substances 0.000 claims abstract description 18
- 230000008569 process Effects 0.000 claims abstract description 14
- 230000011218 segmentation Effects 0.000 claims abstract description 6
- 238000005516 engineering process Methods 0.000 claims description 9
- 238000012916 structural analysis Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 7
- 238000013135 deep learning Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 4
- 239000004816 latex Substances 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 3
- 230000000877 morphologic effect Effects 0.000 claims description 3
- 238000012549 training Methods 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 2
- 238000013519 translation Methods 0.000 abstract description 3
- 230000014509 gene expression Effects 0.000 description 12
- 230000010354 integration Effects 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 238000012545 processing Methods 0.000 description 2
- 206010019233 Headaches Diseases 0.000 description 1
- 206010047571 Visual impairment Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 231100000869 headache Toxicity 0.000 description 1
- 238000007620 mathematical function Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012015 optical character recognition Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 208000029257 vision disease Diseases 0.000 description 1
- 230000004393 visual impairment Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Business, Economics & Management (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Tourism & Hospitality (AREA)
- Mathematical Physics (AREA)
- Educational Technology (AREA)
- Primary Health Care (AREA)
- Databases & Information Systems (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Educational Administration (AREA)
- Mathematical Analysis (AREA)
- Computational Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Software Systems (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Algebra (AREA)
- Character Discrimination (AREA)
Abstract
The invention discloses a method and a system for automatically solving a mathematical problem, wherein the method comprises the following steps: acquiring mathematical problem information to be solved, and converting the mathematical problem information into corresponding image matrix signals; positioning, character segmentation and recognition of a mathematical problem are carried out on the image matrix signal to obtain a character sequence; and carrying out spatial structure analysis and semantic analysis on the recognized character sequence so as to realize automatic solving of the mathematical calculation problem. The invention uses positioning, character cutting and character thinning method in the character recognizing process, and uses the character recognizing classifier in character recognition. And generating a parse tree during spatial structure analysis and semantic analysis, and finally performing grammar guidance translation on the parse tree so as to realize automatic solving of the calculation problem. The invention avoids the disadvantage that only mathematical calculation questions can be input through a keyboard, and the questions of the type can be automatically solved as long as the types and symbols of the mathematical questions are defined, so that more types of questions can be processed, and the method has better practicability and higher accuracy.
Description
Technical Field
The invention relates to the field of artificial intelligence research, in particular to a method and a system for automatically solving a mathematical problem.
Background
The problem of mathematical calculation makes students have headache. In order to guide students to do mathematical calculations, teachers and parents spend a lot of time reviewing mathematical knowledge and repeating trivial calculations. If a program can automatically solve the mathematical calculation questions, students can independently complete the mathematical calculation questions and compare the reference answers.
To meet this need, researchers have proposed building a system that can identify and automatically solve mathematical problems. In the system, a user inputs a mathematical calculation question in a camera shooting or handwriting mode, the system outputs a mathematical calculation question recognition result, and a solving step and an answer are given. The system has a plurality of use scenes, such as digitizing paper mathematical literature and permanently storing the paper mathematical literature on an electronic storage device. The system can also recognize the mathematical formula on the book, and the audio frequency of the mathematical formula is changed for people with visual impairment to learn mathematical knowledge. In the education field, the system can help students to give ideas and calculation processes for solving the mathematical calculation questions, and can also help teachers to automatically change the mathematical homework questions.
Although current optical character recognition technology is well developed, document recognition software on the market can only process one-dimensional text. The mathematical calculation formula has complex two-dimensional spatial relationship and nested relationship. This makes it very difficult to identify the mathematical formula. Most of the work of predecessors is focused on positioning, cutting, identifying and structural analysis of mathematical formulas, and few study people deeply on semantic analysis of mathematical formulas. Briefly, it is proposed to recognize a single character using statistical and structural features, then decompose the recognized expression into a series of sub-expressions for structural analysis, and finally output the recognized mathematical formula in the LaTeX format. The method of using the Chensong to classify the characters roughly and then finely to match the templates and finally combining the positions among the characters and partial recognition results is used for recognizing the mathematical symbols. In the structural analysis, a box merging method is adopted to merge a complete expression from bottom to top, namely the identified expression. The objects processed by the above two articles are machine-printed fonts and do not relate to the recognition of handwritten mathematical formulas. Furthermore, they only stay in the structural analysis phase and do not perform semantic analysis and automatic evaluation on mathematical formulas.
There are many software that can realize the function of searching for the question automatically. The method comprises the steps of preprocessing and segmenting pictures uploaded by users through software such as ape search questions and Baidu search questions, identifying characters through a deep learning technology, converting questions into characters through the steps, searching and sorting the similarity of the questions in a question bank accumulated for a long time, and finally returning answers to the questions and analyzing the questions. The techniques of image processing, character recognition, question bank matching and the like are used in the ape search questions and the Baidu search questions, but semantic analysis and automatic solving are not carried out on mathematical formulas, but solving is carried out through the question bank input in front, so that new problem programs of the same type cannot work well. And an APP in the mobile phone application store is called 'love homework', and the function of the APP can be used for photographing and correcting oral problems. The APP can accurately identify a plurality of oral calculation questions in the picture, and also carries out structural analysis and semantic analysis on the mathematical calculation formula and provides a correct correction result. However, only arithmetic calculation questions, fractional expressions and the like of primary schools can be processed, and variables and mathematical function symbols cannot be recognized.
In order to solve the defects of the technology and the method, the method and the system which can automatically solve the mathematical problem have great research significance and practical value.
Disclosure of Invention
The invention aims to overcome the defects of the conventional shooting question searching method and provide a method and a system for automatically solving a mathematical question based on an artificial intelligence technology, which have strong practicability, do not cause the problem not to conform to an answer and have multiple applicable question types.
The purpose of the invention is realized by the following technical scheme: a method for automatically solving a mathematical problem, comprising the steps of:
(1) acquiring mathematical calculation problem information to be solved, and converting the mathematical calculation problem information into corresponding image matrix signals;
(2) positioning, character segmentation and recognition of a mathematical calculation problem are carried out on the image matrix signal to obtain a character sequence;
(3) combining the category, size and space coordinate information of the character sequence, carrying out structural analysis on the mathematical computation questions, analyzing the mathematical computation questions by using a method based on a syntactic rule, continuously combining the character sequence, and finally obtaining the representation of the tree structure of the mathematical computation questions, namely an analysis tree;
(4) defining attributes and a calculation method of the nodes of the analytical tree according to mathematical semantic rules of the symbols; traversing the analysis tree in a subsequent order, transmitting the attribute values of the nodes of the analysis tree from bottom to top, and finally obtaining the attribute value of the root node; and (4) taking the attribute value of the root node as an answer of the mathematical calculation problem to finish the automatic solving of the mathematical calculation problem.
As a preferred embodiment of the present invention, in the step (1), the mathematical computation problem information to be solved is in any one of the following forms: the system comprises an image of a mathematical calculation question, a handwritten text track of the mathematical calculation question and a scanning piece of a document of the mathematical calculation question. The mathematical calculation questions include elementary mathematics, advanced mathematics, linear algebra, probability theory, etc.
As a preferred embodiment of the present invention, in the step (1), the image matrix signal I is pre-processed, and the pre-processing method is as follows: thresholding is firstly carried out, then Gaussian smoothing is carried out, and then the image is denoised by using the morphological method of the image to obtain a preprocessed image I1. The influence of irrelevant factors is avoided.
As a preferred embodiment of the present invention, in the step (2), the problem of mathematical computation is located on the image matrix signal by: different text regions of the image are divided through connected body analysis or a projection method, the text regions are classified by extracting features through a deep learning technology, the classified categories comprise a text region and a formula region, and the formula region is a position region of a mathematical calculation question.
As a preferable mode of the present invention, in the step (2),character segmentation and recognition are carried out on the positioned mathematical calculation questions, and the method comprises the following steps: constructing a mathematical symbol library, collecting a certain number of handwritten samples of each symbol as a data set according to the definition of the mathematical symbol library, and training by using a deep learning target detection model to obtain a classifier for identifying the mathematical symbols; identifying the symbols in the image matrix signal to obtain an identified character sequence { c1,c2,c2,...,cn}; each element of the sequence contains the character's category, size, spatial coordinate information.
As a preferred embodiment of the present invention, in the step (3), the structure analysis is performed on the mathematical computation questions by a method comprising:
(3-1) designing an automaton to identify elements of the mathematical formula according to the category, the size and the space coordinate information of the characters to form a minimum element sequence { t ] of the mathematical formula1,t2,...,tnH, each element t of the sequenceiThe mathematical sign is at least composed of one mathematical symbol, and the elements form the minimum unit of a mathematical calculation question;
(3-2) defining a syntax rule of the mathematical computation problem, and identifying the structure of the mathematical computation problem through syntax analysis; in the process of analyzing the structure of the calculation problem, mathematical elements are continuously merged into subtrees according to the syntactic rules until a parse tree T of the semantics of the mathematical calculation problem is finally obtained.
Further, in step (3-2), a parser is constructed to identify the structure of the mathematical computation problem, the parser receiving the sequence of elements as input and analyzing the structure of the mathematical computation problem using operator-first or recursive descent.
As a preferred embodiment of the present invention, in step (4), semantic analysis is performed according to the parse tree to realize automatic solution of the mathematical computation problem, and the method is as follows:
(4-1) writing attributes of tree nodes and corresponding calculation methods for the analytic tree T according to the mathematical meaning of the mathematical symbol;
(4-2) performing bottom-to-top syntactic analysis by semantic analysis, and realizing automatic solution of the mathematical computation formula by adopting a mode of traversing a parse tree in subsequent order, wherein the method specifically comprises the following steps: traversing attribute values of the analytical tree in a subsequent order, calling a self calculation method by each node according to the type of the node, and then returning a calculation result to a father node; finally, analyzing the root node of the tree to obtain the answer of the mathematical calculation question;
and (4-3) obtaining LaTeX format output and calculation steps of the mathematical calculation problem and a solving result, and returning the results to a user, thereby realizing the function of automatically solving the mathematical calculation problem.
Further, in the step (4-1), the attributes of the tree node at least include a state attribute and a value attribute, the state attribute indicates whether the node has been calculated, whether the node is a constant or a variable, and the like; the value attribute is a result calculated according to mathematical semantics.
A system for automatically solving a mathematical problem, comprising:
the information acquisition module is used for acquiring mathematical calculation problem information to be solved and converting the mathematical calculation problem information into a corresponding image matrix signal;
the character extraction module is used for positioning, segmenting and identifying the mathematical calculation questions of the image matrix signals to obtain a character sequence;
the structure analysis module is used for carrying out structure analysis on the mathematical computation questions by combining the category, size and space coordinate information of the character sequence, and the method is to analyze the mathematical computation questions by using a method based on a syntactic rule, constantly merge the character sequence and finally obtain the representation of the tree structure of the mathematical computation questions, namely an analysis tree;
the semantic analysis module is used for defining the attribute and the calculation method of the analysis tree node according to the mathematical semantic rule of the symbol; traversing the analysis tree in a subsequent order, transmitting the attribute values of the nodes of the analysis tree from bottom to top, and finally obtaining the attribute value of the root node; and (4) taking the attribute value of the root node as an answer of the mathematical calculation problem to finish the automatic solving of the mathematical calculation problem.
Compared with the prior art, the invention has the following advantages and beneficial effects:
1) the invention can input the mathematical calculation questions through the pictures shot by the camera, the handwriting equipment or the scanner, avoids the defect that the mathematical calculation questions can only be input through a keyboard, forms a whole set of complete algorithm for identifying the handwritten mathematical calculation questions and automatically solving the problems, and has better practicability.
2) The invention realizes the automatic solving of the mathematical calculation problem by identifying the structure and the semantics of the mathematical calculation problem, and can automatically solve the problem of the type as long as the type and the symbol of the mathematical calculation problem are defined. Therefore, the method can theoretically process various problems, such as various mathematical problems including arithmetic operation, fractional operation, definite integral, equation solution, derivative calculation and limitation calculation.
3) The method for automatically solving the mathematical calculation problems by identifying the structure and the semantics of the mathematical calculation problems is essentially different from the current method for identifying the text and searching the problem base. The method for searching the question bank is easy to output answers which are not in accordance with the questions when the questions which do not exist in the question bank exist, but the method of the invention returns correct question analysis under the condition of well-defined mathematical question types and symbols.
Drawings
FIG. 1 is a flow chart of a method of the present invention;
FIG. 2 is an example of a mathematical symbol library in an embodiment of the present invention;
FIG. 3 is a mathematical computation question parsing tree constructed according to an embodiment of the present invention;
FIG. 4 is a flow chart of automatically solving a mathematical computational problem according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating an automatic solution problem according to the present invention.
Detailed Description
The present invention will be described in further detail with reference to examples and drawings, but the present invention is not limited thereto.
Example 1
As shown in fig. 1, this embodiment provides a method for automatically solving a mathematical problem, in which a user may input a mathematical calculation problem by taking a picture or handwriting based on an artificial intelligence technique, and then the system automatically locates the position of the mathematical calculation problem, identifies it and automatically solves it, and returns the identification result and a reference answer to the user. The method uses a calculation problem positioning method, a character cutting method and a character thinning method in picture preprocessing, and uses a character recognition classifier in character recognition. The classifier uses a model similar to a support vector machine or a convolutional neural network. In the stage of structural analysis and semantic analysis of the mathematical calculation questions, the system generates a parse tree by using a method of combining structural analysis and grammar analysis, and finally performs grammar-guided translation on the parse tree so as to realize automatic solution of the calculation questions.
The algorithm flow of the implementation process is shown in fig. 1, and the implementation steps are as follows:
s1: the user inputs the information of the mathematical computation questions through an intelligent terminal, wherein the intelligent terminal refers to all devices capable of inputting the information of the mathematical computation questions into the system, and the intelligent terminal comprises but is not limited to a camera of a mobile device, an electronic device supporting handwriting and a device with a document scanning function. The mathematical calculation question information specifically refers to a scanning piece including an image of a mathematical calculation question, a handwritten text track of the mathematical calculation question and a document of the mathematical calculation question. The mathematical calculation questions include elementary mathematics, advanced mathematics, linear algebra, probability theory, etc.
In this embodiment, a picture including a mathematical calculation problem is obtained by photographing by a user (fig. 2(a)), and first, the image is preprocessed, specifically, the image includes graying of a color picture (fig. 2(b)), gaussian filtering (fig. 2(c)), morphological denoising, binarization processing, and the like, so as to obtain a binary image. As shown in fig. 2, image I after preprocessing1After the position of the mathematical computation question is located by the connected body analysis, contour detection (fig. 2(d)), character segmentation (fig. 2(e)), and character skeleton extraction (fig. 2(f)) are performed. And storing the extracted character skeleton for subsequent character recognition.
S2: a library of mathematical symbols was constructed as shown in the following table:
table 1 constructed mathematical symbol library
The symbol library has 32 mathematical characters, and can form most mathematical calculation question types. A number of handwriting samples are collected for each symbol as a data set according to the definition of the symbol library. The present embodiment employs recognition of data sets accumulated by online handwritten mathematical formula Competition (CROHME). And then defining a classification network-VGG model of a deep learning technology, and training by using a data set to finally obtain a classifier capable of recognizing mathematical symbols. Identifying the symbol sequence cut in the step (1) to obtain an identified symbol sequence { c }1,c2,c2,...,cnEach element of the sequence contains information of size, position, category, etc. of the mathematical symbol.
S3: after the symbol sequence is identified, the structure of the mathematical computation problem needs to be analyzed, and finally, the parse tree is obtained. The method specifically comprises the following steps:
(3-1) for the number and decimal number with the length larger than 1, the number and decimal number are composed of a plurality of mathematical symbols, but the number and decimal number are also independent units of the mathematical calculation problem, so that an automaton is designed to identify the elements of the mathematical formula according to the spatial relation, the category and the size information of the symbols, and finally a minimum element sequence { t & lt/EN & gt & lt/EN & gt of the mathematical calculation problem is formed1,t2,...,tn}；
(3-2) subsequently, for each type of the mathematical computation questions, defining a composition rule; for example, the composition rules of arithmetic expressions and integral expressions can be written in the form:
< arithmetic expression > → < sub-expression > [ + | - | × | ÷ ] < sub-expression > | (< sub-expression >)
< integral expression > → < integral character > [ < integral character lower limit > < integral character upper limit > ] < sub expression > < integral variable > ]
In this way, the composition rule of different expressions in each mathematical calculation problem can be defined.
(3-3) constructing a parser to parse the structure of the mathematical computation problem, the parser receiving the sequence of elements as input, parsing the structure of the mathematical computation problem using operator precedence or recursive descent, and finally obtaining a parse tree T of the mathematical computation problem.
FIG. 3 is a parse tree for a mathematical computational problem; the mathematical calculation problem corresponding to the analytic tree is
The tree nodes can be mathematical operators, numbers or variables.
(3-4) automatically solving the answer of the mathematical calculation question; and traversing the attribute values of the analysis tree in the subsequent order, calling a self calculation method by each node according to the self node type, and then returning the calculation result to the father node. And finally, analyzing the root node of the tree to obtain the answer of the mathematical calculation question.
S4: the parse tree recognized in step S3 includes all semantic information of the mathematical computation question, and from this information, the answer to the mathematical computation question can be obtained. In the semantic analysis process, the method expands the technology of attribute grammar and grammar guidance translation and applies the technology to the semantic analysis of the parse tree. The method comprises the following specific steps:
(4-1) for the mathematical computation question analysis tree T obtained in the step (3), firstly, according to the mathematical meaning of the mathematical symbol, writing the attribute of the tree node and the corresponding computation method. The attributes of the parse tree node include at least a state and a value. The state attribute indicates whether the node has been computed, whether the node is a constant or a variable, etc.; the value attributes are the result of the mathematical semantic calculations. The value attribute of the root node is the answer of the mathematical calculation question; meanwhile, different node definition calculation methods are required, for example, the square node is the calculation of the square: the value of the square node is equal to the value of the left sub-treeValue of the right sub-treeThe integration nodes need to firstly solve the primitive function F of the integration expression F, wherein they satisfy F', F, and then substitute the integration upper limit s and the integration lower limit t into F to obtain the integration result F(s) -F (t).
And (4-2) performing bottom-to-top syntactic analysis by semantic analysis, and realizing automatic solution of the mathematical computation by adopting a mode of traversing the parse tree in subsequent order. The system defines a set of semantic rules for calculating values and returning expressions in LaTeX format based on each node type. In the backward traversal process, the state and the value of the child node are calculated firstly, then the state and the value are transmitted to the parent node in sequence, and finally the value of the root node is obtained.
FIG. 4 is a process for automatically solving computational problems. Where the circle nodes and solid lines are the structure of the parse tree. The dotted line is the step of the child node passing the attribute information to the parent node. In the semantic analysis stage, the system first determines the values and states of the child nodes. Such as leaf node 3 having a value of 3 and a state that is evaluable, and then passes the attribute information to the parent node. In steps 1 and 2 of FIG. 4, leaf nodes 3 and 2 pass the attributes to the squaring node, which then calls its own computation function to yield a square of 3 of 9 and a state of evaluable. The leaf node x has a coefficient of 2, a state of first power, and a value of 'x'. In step 3, the x node transmits the attribute value to the integration node, and then the integration node calculates the definite integral to obtain a value of 1, and the state is evaluable. And the final square node and the integral node transmit 9 and 1 to the plus node, and the values of the plus node are added to obtain a final answer 10.
S5: after the four steps, the identification and automatic solving process of the mathematical calculation questions is completed, and finally the identified mathematical calculation questions, the calculation steps and the answers are printed to the user, and fig. 5 is an example of an output result of the automatic solving calculation questions provided by the invention.
In general, it can be seen from the results that the method for automatically solving the mathematical computation problem adopted by the present invention is effective, and can give the correct answer to the solving step in most cases. The invention mainly contributes to the research of an automatic question answering system standing at the scientific and technological frontier and provides a set of complete and feasible algorithm framework aiming at the recognition of handwritten mathematical calculation questions and the automatic problem solving. Compared with other similar methods, the method has the characteristics that the method can identify a plurality of symbols, can process most of different types of mathematical calculation questions, has strong practicability in research content, can be used for not only the education field, but also the electronization of mathematical documents and the terminal input of mathematical formulas, and has high application value.
Example 2
The present embodiment provides a system for automatically solving a mathematical problem, including:
the information acquisition module is used for acquiring mathematical calculation problem information to be solved and converting the mathematical calculation problem information into a corresponding image matrix signal;
the character extraction module is used for positioning, segmenting and identifying the mathematical calculation questions of the image matrix signals to obtain a character sequence;
the structure analysis module is used for carrying out structure analysis on the mathematical computation questions by combining the category, size and space coordinate information of the character sequence, and the method is to analyze the mathematical computation questions by using a method based on a syntactic rule, constantly merge the character sequence and finally obtain the representation of the tree structure of the mathematical computation questions, namely an analysis tree;
the semantic analysis module is used for defining the attribute and the calculation method of the analysis tree node according to the mathematical semantic rule of the symbol; traversing the analysis tree in a subsequent order, transmitting the attribute values of the nodes of the analysis tree from bottom to top, and finally obtaining the attribute value of the root node; and (4) taking the attribute value of the root node as an answer of the mathematical calculation problem to finish the automatic solving of the mathematical calculation problem.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the modules described above may refer to the corresponding process in the foregoing method embodiment, and is not described herein again.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A method for automatically solving a mathematical problem, comprising the steps of:
(1) acquiring mathematical calculation problem information to be solved, and converting the mathematical calculation problem information into corresponding image matrix signals;
(2) positioning, character segmentation and recognition of a mathematical calculation problem are carried out on the image matrix signal to obtain a character sequence;
(3) combining the category, size and space coordinate information of the character sequence, carrying out structural analysis on the mathematical computation questions, analyzing the mathematical computation questions by using a method based on a syntactic rule, continuously combining the character sequence, and finally obtaining the representation of the tree structure of the mathematical computation questions, namely an analysis tree;
(4) defining attributes and a calculation method of the nodes of the analytical tree according to mathematical semantic rules of the symbols; traversing the analysis tree in a subsequent order, transmitting the attribute values of the nodes of the analysis tree from bottom to top, and finally obtaining the attribute value of the root node; and (4) taking the attribute value of the root node as an answer of the mathematical calculation problem to finish the automatic solving of the mathematical calculation problem.
2. The method for automatically solving the mathematical problems as claimed in claim 1, wherein in the step (1), the information of the mathematical calculation problems to be solved is in any form of: the system comprises an image of a mathematical calculation question, a handwritten text track of the mathematical calculation question and a scanning piece of a document of the mathematical calculation question.
3. The method for automatically solving the mathematical problem according to claim 1, wherein in the step (1), the image matrix signal is pre-processed, and the pre-processing method comprises: thresholding is carried out firstly, then Gaussian smoothing is carried out, and then the image is denoised by using the morphological method of the image to obtain the preprocessed image.
4. The method for automatically solving the mathematical problem according to claim 1, wherein in the step (2), the mathematical calculation problem is located on the image matrix signal by: different text regions of the image are divided through connected body analysis or a projection method, the text regions are classified by extracting features through a deep learning technology, the classified categories comprise a text region and a formula region, and the formula region is a position region of a mathematical calculation question.
5. The method for automatically solving the mathematical problems as claimed in claim 1, wherein in the step (2), the character segmentation and recognition are performed on the positioned mathematical calculation problems by: constructing a mathematical symbol library, collecting a certain number of handwritten samples of each symbol as a data set according to the definition of the mathematical symbol library, and training by using a deep learning target detection model to obtain a classifier for identifying the mathematical symbols; identifying the symbols in the image matrix signal to obtain an identified character sequence { c1,c2,c2,...,cn}; each element of the sequence contains the character's category, size, spatial coordinate information.
6. The method for automatically solving mathematical problems as claimed in claim 1, wherein in the step (3), the mathematical problems are structurally analyzed by:
(3-1) designing an automaton to identify elements of the mathematical formula according to the category, the size and the space coordinate information of the characters to form a minimum element sequence { t ] of the mathematical formula1,t2,...,tnH, each element t of the sequenceiThe mathematical sign is at least composed of one mathematical symbol, and the elements form the minimum unit of a mathematical calculation question;
(3-2) defining a syntax rule of the mathematical computation problem, and identifying the structure of the mathematical computation problem through syntax analysis; in the process of analyzing the structure of the calculation problem, mathematical elements are continuously merged into subtrees according to the syntactic rules until a parse tree T of the semantics of the mathematical calculation problem is finally obtained.
7. The method of automatically solving mathematical problems of claim 6 wherein in step (3-2) a parser is constructed to identify the structure of the mathematical computational problem, the parser receiving the sequence of elements as input and using operator precedence or recursive descent to parse the structure of the mathematical computational problem.
8. The method for automatically solving the mathematical problems as claimed in claim 1, wherein in the step (4), the semantic analysis is performed according to the parse tree to realize the automatic solving of the mathematical computation problems, and the method comprises the following steps:
(4-1) writing attributes of tree nodes and corresponding calculation methods for the analytic tree T according to the mathematical meaning of the mathematical symbol;
(4-2) performing bottom-to-top syntactic analysis by semantic analysis, and realizing automatic solution of the mathematical computation formula by adopting a mode of traversing a parse tree in subsequent order, wherein the method specifically comprises the following steps: traversing attribute values of the analytical tree in a subsequent order, calling a self calculation method by each node according to the type of the node, and then returning a calculation result to a father node; finally, analyzing the root node of the tree to obtain the answer of the mathematical calculation question;
and (4-3) obtaining LaTeX format output and calculation steps of the mathematical calculation problem and a solving result, and returning the results to a user, thereby realizing the function of automatically solving the mathematical calculation problem.
9. The method for automatically solving a mathematical problem according to claim 8, wherein in the step (4-1), the attributes of the tree node include at least a state attribute and a value attribute, the state attribute indicates whether the node has been calculated, whether the node is a constant or a variable; the value attribute is a result calculated according to mathematical semantics.
10. A system for automatically solving a mathematical problem, comprising:
the information acquisition module is used for acquiring mathematical calculation problem information to be solved and converting the mathematical calculation problem information into a corresponding image matrix signal;
the character extraction module is used for positioning, segmenting and identifying the mathematical calculation questions of the image matrix signals to obtain a character sequence;
the structure analysis module is used for carrying out structure analysis on the mathematical computation questions by combining the category, size and space coordinate information of the character sequence, and the method is to analyze the mathematical computation questions by using a method based on a syntactic rule, constantly merge the character sequence and finally obtain the representation of the tree structure of the mathematical computation questions, namely an analysis tree;
the semantic analysis module is used for defining the attribute and the calculation method of the analysis tree node according to the mathematical semantic rule of the symbol; traversing the analysis tree in a subsequent order, transmitting the attribute values of the nodes of the analysis tree from bottom to top, and finally obtaining the attribute value of the root node; and (4) taking the attribute value of the root node as an answer of the mathematical calculation problem to finish the automatic solving of the mathematical calculation problem.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910831074.2A CN110751137A (en) | 2019-09-04 | 2019-09-04 | Method and system for automatically solving mathematical problem |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910831074.2A CN110751137A (en) | 2019-09-04 | 2019-09-04 | Method and system for automatically solving mathematical problem |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110751137A true CN110751137A (en) | 2020-02-04 |
Family
ID=69276051
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910831074.2A Pending CN110751137A (en) | 2019-09-04 | 2019-09-04 | Method and system for automatically solving mathematical problem |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110751137A (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111259632A (en) * | 2020-02-10 | 2020-06-09 | 暗物智能科技(广州)有限公司 | Semantic alignment-based tree structure mathematical application problem solving method and system |
| CN111340661A (en) * | 2020-02-21 | 2020-06-26 | 电子科技大学 | Automatic application problem solving method based on graph neural network |
| CN111340662A (en) * | 2020-02-25 | 2020-06-26 | 上海乂学教育科技有限公司 | Geometric certification question making system |
| CN111339312A (en) * | 2020-02-18 | 2020-06-26 | 清华大学 | Knowledge graph for solving high school mathematics evaluation problem and construction and solving method thereof |
| CN111783737A (en) * | 2020-07-29 | 2020-10-16 | 郑州航空工业管理学院 | Mathematical formula identification method and device |
| CN111950701A (en) * | 2020-07-13 | 2020-11-17 | 中国科学技术大学 | Mathematical problem solving method and system fusing formula information |
| CN112101359A (en) * | 2020-11-11 | 2020-12-18 | 广州华多网络科技有限公司 | Text formula positioning method, model training method and related device |
| CN112232681A (en) * | 2020-10-19 | 2021-01-15 | 山东山大鸥玛软件股份有限公司 | Intelligent paper marking method for computational analysis type non-selection questions |
| CN112580653A (en) * | 2020-12-24 | 2021-03-30 | 广州久邦世纪科技有限公司 | Automatic problem solving method and device |
| CN112860856A (en) * | 2021-02-10 | 2021-05-28 | 福州大学 | Intelligent problem solving method and system for arithmetic application problem |
| CN112906559A (en) * | 2021-02-10 | 2021-06-04 | 网易有道信息技术（北京）有限公司 | Machine-implemented method for correcting formulas and related product |
| CN113033152A (en) * | 2021-04-01 | 2021-06-25 | 北京有竹居网络技术有限公司 | LaTeX formula display method and device |
| CN113034109A (en) * | 2021-03-26 | 2021-06-25 | 北京乐学帮网络技术有限公司 | Method and system for verifying answer result and readable storage medium |
| CN113095314A (en) * | 2021-04-07 | 2021-07-09 | 科大讯飞股份有限公司 | Formula identification method and device, storage medium and equipment |
| CN113257063A (en) * | 2021-06-08 | 2021-08-13 | 北京字节跳动网络技术有限公司 | Interaction method and terminal equipment |
| CN113689418A (en) * | 2021-09-01 | 2021-11-23 | 北京云蝶智学科技有限公司 | Formula calculation method based on image recognition |
| CN113837157A (en) * | 2021-11-26 | 2021-12-24 | 北京世纪好未来教育科技有限公司 | Topic type identification method, system and storage medium |
| CN117033847A (en) * | 2023-07-20 | 2023-11-10 | 华中师范大学 | Mathematical application problem solving method and system based on hierarchical recursive tree decoding model |
| CN113689418B (en) * | 2021-09-01 | 2024-06-07 | 北京碧云数创科技有限公司 | Formula calculation method based on image recognition |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5544262A (en) * | 1992-04-07 | 1996-08-06 | Apple Computer, Inc. | Method and apparatus for processing graphically input equations |
| CN101329731A (en) * | 2008-06-06 | 2008-12-24 | 南开大学 | Automatic recognition method pf mathematical formula in image |
| CN107301163A (en) * | 2016-04-14 | 2017-10-27 | 科大讯飞股份有限公司 | Text semantic analysis method and device comprising formula |
| CN107357757A (en) * | 2017-06-29 | 2017-11-17 | 成都考拉悠然科技有限公司 | A kind of algebra word problems automatic calculation device based on depth enhancing study |
| CN107423286A (en) * | 2017-07-05 | 2017-12-01 | 华中师范大学 | The method and system that elementary mathematics algebraically type topic is answered automatically |
| CN107729865A (en) * | 2017-10-31 | 2018-02-23 | 中国科学技术大学 | A kind of handwritten form mathematical formulae identified off-line method and system |
| CN109977958A (en) * | 2019-03-25 | 2019-07-05 | 中国科学技术大学 | A kind of offline handwritten form mathematical formulae identification reconstructing method |
-
2019
- 2019-09-04 CN CN201910831074.2A patent/CN110751137A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5544262A (en) * | 1992-04-07 | 1996-08-06 | Apple Computer, Inc. | Method and apparatus for processing graphically input equations |
| CN101329731A (en) * | 2008-06-06 | 2008-12-24 | 南开大学 | Automatic recognition method pf mathematical formula in image |
| CN107301163A (en) * | 2016-04-14 | 2017-10-27 | 科大讯飞股份有限公司 | Text semantic analysis method and device comprising formula |
| CN107357757A (en) * | 2017-06-29 | 2017-11-17 | 成都考拉悠然科技有限公司 | A kind of algebra word problems automatic calculation device based on depth enhancing study |
| CN107423286A (en) * | 2017-07-05 | 2017-12-01 | 华中师范大学 | The method and system that elementary mathematics algebraically type topic is answered automatically |
| CN107729865A (en) * | 2017-10-31 | 2018-02-23 | 中国科学技术大学 | A kind of handwritten form mathematical formulae identified off-line method and system |
| CN109977958A (en) * | 2019-03-25 | 2019-07-05 | 中国科学技术大学 | A kind of offline handwritten form mathematical formulae identification reconstructing method |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111259632A (en) * | 2020-02-10 | 2020-06-09 | 暗物智能科技(广州)有限公司 | Semantic alignment-based tree structure mathematical application problem solving method and system |
| CN111259632B (en) * | 2020-02-10 | 2023-04-11 | 暗物智能科技(广州)有限公司 | Semantic alignment-based tree structure mathematical application problem solving method and system |
| CN111339312A (en) * | 2020-02-18 | 2020-06-26 | 清华大学 | Knowledge graph for solving high school mathematics evaluation problem and construction and solving method thereof |
| CN111340661A (en) * | 2020-02-21 | 2020-06-26 | 电子科技大学 | Automatic application problem solving method based on graph neural network |
| CN111340661B (en) * | 2020-02-21 | 2020-12-01 | 电子科技大学 | Automatic application problem solving method based on graph neural network |
| CN111340662A (en) * | 2020-02-25 | 2020-06-26 | 上海乂学教育科技有限公司 | Geometric certification question making system |
| CN111950701B (en) * | 2020-07-13 | 2022-09-02 | 中国科学技术大学 | Mathematical problem solving method and system fusing formula information |
| CN111950701A (en) * | 2020-07-13 | 2020-11-17 | 中国科学技术大学 | Mathematical problem solving method and system fusing formula information |
| CN111783737A (en) * | 2020-07-29 | 2020-10-16 | 郑州航空工业管理学院 | Mathematical formula identification method and device |
| CN111783737B (en) * | 2020-07-29 | 2024-02-02 | 郑州航空工业管理学院 | Mathematical formula identification method and device |
| CN112232681A (en) * | 2020-10-19 | 2021-01-15 | 山东山大鸥玛软件股份有限公司 | Intelligent paper marking method for computational analysis type non-selection questions |
| CN112101359A (en) * | 2020-11-11 | 2020-12-18 | 广州华多网络科技有限公司 | Text formula positioning method, model training method and related device |
| CN112101359B (en) * | 2020-11-11 | 2021-02-12 | 广州华多网络科技有限公司 | Text formula positioning method, model training method and related device |
| CN112580653A (en) * | 2020-12-24 | 2021-03-30 | 广州久邦世纪科技有限公司 | Automatic problem solving method and device |
| CN112860856B (en) * | 2021-02-10 | 2022-06-14 | 福州大学 | Intelligent problem solving method and system for arithmetic application problem |
| CN112906559B (en) * | 2021-02-10 | 2022-03-18 | 网易有道信息技术（北京）有限公司 | Machine-implemented method for correcting formulas and related product |
| CN112860856A (en) * | 2021-02-10 | 2021-05-28 | 福州大学 | Intelligent problem solving method and system for arithmetic application problem |
| CN112906559A (en) * | 2021-02-10 | 2021-06-04 | 网易有道信息技术（北京）有限公司 | Machine-implemented method for correcting formulas and related product |
| CN113034109A (en) * | 2021-03-26 | 2021-06-25 | 北京乐学帮网络技术有限公司 | Method and system for verifying answer result and readable storage medium |
| CN113033152A (en) * | 2021-04-01 | 2021-06-25 | 北京有竹居网络技术有限公司 | LaTeX formula display method and device |
| CN113033152B (en) * | 2021-04-01 | 2024-05-28 | 北京有竹居网络技术有限公司 | LaTeX formula display method and device |
| CN113095314A (en) * | 2021-04-07 | 2021-07-09 | 科大讯飞股份有限公司 | Formula identification method and device, storage medium and equipment |
| CN113257063A (en) * | 2021-06-08 | 2021-08-13 | 北京字节跳动网络技术有限公司 | Interaction method and terminal equipment |
| CN113689418A (en) * | 2021-09-01 | 2021-11-23 | 北京云蝶智学科技有限公司 | Formula calculation method based on image recognition |
| CN113689418B (en) * | 2021-09-01 | 2024-06-07 | 北京碧云数创科技有限公司 | Formula calculation method based on image recognition |
| CN113837157A (en) * | 2021-11-26 | 2021-12-24 | 北京世纪好未来教育科技有限公司 | Topic type identification method, system and storage medium |
| CN117033847A (en) * | 2023-07-20 | 2023-11-10 | 华中师范大学 | Mathematical application problem solving method and system based on hierarchical recursive tree decoding model |
| CN117033847B (en) * | 2023-07-20 | 2024-04-19 | 华中师范大学 | Mathematical application problem solving method and system based on hierarchical recursive tree decoding model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110751137A (en) | Method and system for automatically solving mathematical problem | |
| Mahdavi et al. | ICDAR 2019 CROHME+ TFD: Competition on recognition of handwritten mathematical expressions and typeset formula detection | |
| US10013636B2 (en) | Image object category recognition method and device | |
| CN111597908A (en) | Test paper correcting method and test paper correcting device | |
| CN111259724A (en) | Method and system for extracting relevant information from image and computer program product | |
| CN110363194A (en) | Intelligently reading method, apparatus, equipment and storage medium based on NLP | |
| CN111401353A (en) | Method, device and equipment for identifying mathematical formula | |
| CN104123550A (en) | Cloud computing-based text scanning identification method | |
| Matan et al. | Reading handwritten digits: A zip code recognition system | |
| CN113298151A (en) | Remote sensing image semantic description method based on multi-level feature fusion | |
| CN111581367A (en) | Method and system for inputting questions | |
| CN112069900A (en) | Bill character recognition method and system based on convolutional neural network | |
| CN109213853A (en) | A kind of Chinese community's question and answer cross-module state search method based on CCA algorithm | |
| Pacha et al. | Towards self-learning optical music recognition | |
| CN114092938B (en) | Image recognition processing method and device, electronic equipment and storage medium | |
| CN112347997A (en) | Test question detection and identification method and device, electronic equipment and medium | |
| Nikitha et al. | Handwritten text recognition using deep learning | |
| Akanksh et al. | Automated invoice data extraction using image processing | |
| CN113159014A (en) | Objective question reading method, device, equipment and storage medium based on handwritten question numbers | |
| Song et al. | A handwritten character extraction algorithm for multi-language document image | |
| CN117095414A (en) | Handwriting recognition system and recognition method based on dot matrix paper pen | |
| Nath et al. | Improving various offline techniques used for handwritten character recognition: a review | |
| Satav et al. | Data extraction from invoices using computer vision | |
| CN115359486A (en) | Method and system for determining custom information in document image | |
| Chen et al. | Scene text recognition based on deep learning: a brief survey |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200204 |
|
| RJ01 | Rejection of invention patent application after publication |