CN111027557B - Subject identification method based on subject image and electronic equipment - Google Patents
Subject identification method based on subject image and electronic equipment Download PDFInfo
- Publication number
- CN111027557B CN111027557B CN201910183503.XA CN201910183503A CN111027557B CN 111027557 B CN111027557 B CN 111027557B CN 201910183503 A CN201910183503 A CN 201910183503A CN 111027557 B CN111027557 B CN 111027557B
- Authority
- CN
- China
- Prior art keywords
- image
- subject
- topic
- information corresponding
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000001514 detection method Methods 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 7
- 238000012216 screening Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 description 9
- 238000013527 convolutional neural network Methods 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 125000004122 cyclic group Chemical group 0.000 description 5
- 239000000284 extract Substances 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000003708 edge detection Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention relates to the technical field of electronic equipment, and discloses a subject identification method based on a subject image and the electronic equipment, wherein the method comprises the following steps: firstly, identifying the subject information corresponding to the subject image by using the image identification model, then detecting whether the image identification model successfully identifies the subject information corresponding to the subject image, if the image identification model cannot successfully identify the subject information, training the subject image by using the image training model, and setting the training result of the subject image output by the image training model as the subject information corresponding to the subject image. Therefore, the invention uses the image training model to train the topic image which can not be identified by the image identification model to obtain the topic information, and obtains the topic information corresponding to the topic image output by the image training model, thereby improving the topic identification accuracy rate of the topic image and having better user experience.
Description
Technical Field
The invention relates to the technical field of electronic equipment, in particular to a subject identification method based on a subject image and the electronic equipment.
Background
At present, part of home education machines have the function of searching questions, and can search and obtain contents related to the questions according to the question images. In practical use, the conventional method for identifying the subjects by adopting the image identification model is low in identification rate due to the diversity of the subjects and uneven image quality of the subject images, and the subjects corresponding to the subjects cannot be identified under partial conditions, so that the user cannot obtain required contents, and the use experience is poor. Therefore, the existing subject identification method based on the subject image is low in identification rate and poor in user experience.
Disclosure of Invention
Aiming at the defects, the embodiment of the invention discloses a subject identification method based on a subject image and electronic equipment, which are used for solving the problems of low identification accuracy and poor user experience in the prior art.
The first aspect of the embodiment of the invention discloses a subject identification method based on a subject image, which comprises the following steps:
identifying subject information corresponding to the subject image by using an image identification model;
detecting whether the image recognition model successfully recognizes and obtains subject information corresponding to the subject image;
if not, training the topic image by using an image training model, and outputting the topic information corresponding to the topic image.
In a first aspect of the present invention, the identifying subject information corresponding to the subject image using the image identification model includes:
using the image recognition model to recognize and obtain a connected domain of the topic image;
analyzing the connected domain of the topic image to obtain character information included in the topic image;
and analyzing according to the character information to obtain the subject information corresponding to the subject image.
As an optional implementation manner, in the first aspect of the embodiment of the present invention, the method further includes:
extracting image features of the historical topic image;
acquiring effective image features included in the image features of the historical topic image;
and generating the image training model according to the effective image characteristics of the historical subject image and subject information corresponding to the historical subject image.
In a first aspect of the embodiment of the present invention, after the training the topic image using the image training model and outputting the topic information corresponding to the topic image, the method further includes:
pushing the subject information corresponding to the subject image to a user, and inquiring whether the subject information corresponding to the subject image of the user is correct or not;
and if receiving the information which is input by the user and indicates that the subject information corresponding to the subject image is incorrect, executing the step of training the subject image by using an image training model and outputting the subject information corresponding to the subject image.
In an optional implementation manner, in the first aspect of the embodiment of the present invention, after receiving the information input by the user and indicating that the subject information corresponding to the subject image is correct, the method further includes:
searching according to the topic image and the topic information corresponding to the topic image to obtain learning information corresponding to the topic image;
and pushing learning information corresponding to the topic image to the user.
A second aspect of an embodiment of the present invention discloses an electronic device, including:
a subject identification unit for identifying subject information corresponding to the subject image using the image identification model;
the identification detection unit is used for detecting whether the image identification model successfully identifies the subject information corresponding to the subject image;
and the image training unit is used for training the topic image by using the image training model and outputting the topic information corresponding to the topic image when the recognition detection unit detects that the image recognition model does not successfully recognize and obtain the topic information corresponding to the topic image.
As an optional implementation manner, in the second aspect of the embodiment of the present invention, the subject identifying unit includes:
the first subunit is used for identifying and obtaining the connected domain of the topic image by using the image identification model;
the second subunit is used for analyzing the connected domain of the topic image to obtain character information included in the topic image;
and the third subunit is used for analyzing and obtaining the subject information corresponding to the subject image according to the character information.
As an optional implementation manner, in the second aspect of the embodiment of the present invention, the electronic device further includes:
the feature extraction unit is used for extracting image features of the historical topic images;
the feature screening unit is used for acquiring effective image features included in the image features of the historical subject images;
and the model generating unit is used for generating the image training model according to the effective image characteristics of the historical subject image and subject information corresponding to the historical subject image.
As an optional implementation manner, in the second aspect of the embodiment of the present invention, the electronic device further includes:
the inquiring unit is used for pushing the subject information corresponding to the subject image to the user after the image training unit trains the subject image by using the image training model and outputs the subject information corresponding to the subject image, and inquiring whether the subject information corresponding to the subject image of the user is correct or not;
the image training unit is further configured to execute the step of training the topic image using an image training model and outputting the topic information corresponding to the topic image when receiving the information input by the user and indicating that the topic information corresponding to the topic image is incorrect.
As an optional implementation manner, in the second aspect of the embodiment of the present invention, the electronic device further includes:
and the pushing unit is used for searching and obtaining learning information corresponding to the topic image according to the topic image and the topic information corresponding to the topic image after receiving the information which is input by the user and indicates that the topic information corresponding to the topic image is correct, and pushing the learning information corresponding to the topic image to the user.
A third aspect of an embodiment of the present invention discloses an electronic device, including:
a memory storing executable program code;
a processor coupled to the memory;
the processor invokes the executable program code stored in the memory to execute the subject identification method based on the subject image disclosed in the first aspect of the embodiment of the invention.
A fourth aspect of the embodiments of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute a subject identification method based on a subject image disclosed in the first aspect of the embodiments of the present invention.
A fifth aspect of the embodiments of the present invention discloses a computer program product which, when run on a computer, causes the computer to perform part or all of the steps of any one of the methods of the first aspect.
A sixth aspect of the embodiments of the present invention discloses an application publishing platform for publishing a computer program product, wherein the computer program product, when run on a computer, causes the computer to perform part or all of the steps of any one of the methods of the first aspect.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
in the embodiment of the invention, firstly, the subject information corresponding to the subject image is identified by using the image identification model, then whether the image identification model successfully identifies the subject information corresponding to the subject image is detected, if the subject information cannot be successfully identified, the subject image is trained by using the image training model, and the training result of the subject image output by the image training model is set as the subject information corresponding to the subject image. Therefore, the invention uses the image training model to train the topic image which can not be identified by the image identification model to obtain the topic information, and obtains the topic information corresponding to the topic image output by the image training model, thereby improving the topic identification accuracy rate of the topic image and having better user experience.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a subject identification method based on a subject image according to an embodiment of the present invention;
FIG. 2 is a flowchart of another subject identification method based on a subject image according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of another electronic device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of another electronic device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that the terms "comprises" and "comprising," along with any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The embodiment of the invention discloses a subject identification method based on a subject image and electronic equipment, which can improve the accuracy of subject identification of the subject image and improve the use experience of a user. The following detailed description is made in connection with the accompanying drawings from the perspective of an electronic device.
Example 1
Referring to fig. 1, fig. 1 is a flow chart of a subject identification method based on a subject image according to an embodiment of the invention. The method for identifying the title described in fig. 1 is suitable for electronic devices such as a home teaching machine, a smart phone, a tablet computer, a personal computer and the like. The embodiment of the invention describes the method for identifying the title by taking the electronic equipment as an example, and the method is not limited. As shown in fig. 1, the method of identifying a title may include the following steps.
101. And identifying the subject information corresponding to the subject image by using the image identification model.
In the embodiment of the invention, the topic image can be a topic image on a textbook shot by a user through a shooting module arranged on the electronic equipment, or can be a topic image on a display screen of the electronic equipment intercepted when the user exercises on the electronic equipment.
In the embodiment of the invention, the image recognition model is used for primarily recognizing the topic image, and can rapidly recognize the topic information of the topic image under the condition that the image characteristics of the topic image are obvious.
As an optional implementation manner, the connected domain of the topic image is identified by using the image identification model, and character information included in the topic image is obtained by analyzing the connected domain of the topic image, so that the corresponding topic information of the topic image is obtained according to the character information analysis. Specifically, the question characters which are orderly arranged on the question image form a connected domain of the question image, and the image recognition model recognizes and obtains the connected domain formed by the question characters according to the edge detection algorithm, so that the recognition of the area of the question image except for the connected domain is not needed, and the recognition efficiency is improved; after identifying the connected domain, further identifying character information included in the connected domain, analyzing the character information, for example, identifying the connected domain of the subject image to obtain the following character information (solving the following primary equation of 2 x=2), and analyzing the corresponding keyword which is the 'primary equation' in the character information and is mathematics according to the comparison table stored with the subject information and the corresponding keyword of the subject information, so that the subject information of the subject image can be obtained as mathematics. Therefore, the topic image with obvious image characteristics can be quickly identified by using the image identification model.
102. And detecting whether the image recognition model successfully recognizes and obtains subject information corresponding to the subject image.
In the embodiment of the invention, because the image recognition model cannot recognize the subject information of the subject image due to the condition that the image features of part of the subject images are not obvious, and the like, whether the image recognition model successfully recognizes the subject information corresponding to the subject image is needed to be detected.
As an optional implementation manner, a preset recognition duration can be set according to the historical recognition duration of the image recognition model for recognizing the historical subject image, and if the recognition duration of the image recognition model for recognizing the subject image is detected to exceed the preset recognition duration, the image recognition model can be considered to be unsuccessfully recognized to obtain subject information corresponding to the subject image, so that the problem image is prevented from being invalid recognized by the image recognition model for a long time.
103. And training the topic image by using the image training model, and outputting the topic information corresponding to the topic image.
In the embodiment of the invention, for the topic image which cannot be identified by the image identification model and is detected in the step 102, training is performed on the topic image based on the image training model, so that a training result is obtained according to the image characteristic training of the topic image, and further the topic information of the topic image is analyzed and obtained. Before that, training is needed to obtain an image training model according to the historical subject image and subject information corresponding to the historical subject image.
As an optional implementation manner, the image features of the historical topic image are extracted, and the effective image features included in the image features of the historical topic image are obtained so as to generate an image training model according to the effective image features of the historical topic image and the topic information corresponding to the historical topic image. Specifically, a convolutional neural network can be adopted to train a historical topic image, the historical topic image is processed by randomly generating a convolutional check, the historical topic image is divided into a plurality of areas, pixel point information of each area is extracted to be used as image characteristics of each area, an average value of the pixel points of each area is used for representing the area, areas with the difference value of the average values of adjacent pixel points smaller than a preset threshold value are combined to obtain combined effective image characteristics, an image training model corresponding to the effective image characteristics of the historical topic image and the topic information of the historical topic image is obtained through training, and the input topic image can be trained and the topic information corresponding to the topic image is obtained.
As another optional implementation manner, the image training model can be obtained by training in a mode of combining a convolutional neural network and a cyclic neural network, the convolutional neural network extracts image features of the historical topic image, the cyclic neural network combines the image features for a plurality of times and selects effective image features in the image features to generate the image training model, and therefore the image training model can more accurately identify the topic information of the topic image.
It can be understood that in the embodiment of the invention, besides the convolutional neural network or the cyclic neural network can be adopted to train the historical topic image, algorithms such as the deep neural network and the like can be adopted to train the historical topic image, so that the topic image with poor definition or complicated content can be flexibly identified by the image training model.
In the embodiment of the invention, after the training is performed to obtain the image training model, the image training model can be used to train the topic image which is detected in the step 102 and can not be identified by the image identification model to obtain the topic information.
As an optional implementation manner, the effective image features of the topic image can be identified and obtained by inputting the topic image into the image training model, and the topic image is classified according to the effective image features, so that the topic information corresponding to the topic image can be accurately obtained.
Therefore, in the embodiment of the invention, the subject information corresponding to the subject image is identified by using the image identification model, whether the subject information corresponding to the subject image is successfully identified by the image identification model is detected, if the subject information cannot be successfully identified, the subject image is trained by using the image training model, and the training result of the subject image output by the image training model is set as the subject information corresponding to the subject image. The topic images are pre-identified by using the image identification model, and then the topic images which cannot be identified by using the image training model are trained, so that the accuracy of topic identification of various topic images is improved, the situation of identification failure is greatly reduced, and the user experience is good.
Example two
Referring to fig. 2, fig. 2 is a flowchart illustrating a method for identifying a topic according to another embodiment of the invention. As shown in fig. 2, the method of identifying a title may include the following steps.
201. And identifying the subject information corresponding to the subject image by using the image identification model.
202. And detecting whether the image recognition model successfully recognizes and obtains subject information corresponding to the subject image.
203. And training the topic image by using the image training model, and outputting the topic information corresponding to the topic image.
In the embodiment of the present invention, after the image training model identifies the subject information, the process goes to step 204.
204. And pushing the subject information corresponding to the subject image to the user, and inquiring whether the subject information corresponding to the subject image of the user is correct or not.
In the embodiment of the invention, after the subject information of the subject image is obtained by the recognition of the image training model, the situation that the recognition accuracy of the subject information is higher but the recognition error exists is considered in the image training model, so that the subject recognition result of the image training model can be identified by a user.
As an alternative implementation mode, after training the topic image by using the image training model and outputting the topic information corresponding to the topic image, pushing the topic information corresponding to the topic image to the user and inquiring whether the topic information corresponding to the topic image of the user is correct or not. Specifically, after the subject information corresponding to the subject image is identified, the subject image identified at this time and the corresponding subject information are displayed on a display screen of the electronic device, and an option button is arranged on the display screen, so that after the subject information is identified, a user clicks the corresponding option button, and the electronic device can be informed of whether the subject identification at this time is correct or not, thereby being convenient for prompting the accuracy of the subject identification according to the feedback of the user.
Further optionally, if receiving information input by the user indicating that the subject information corresponding to the subject image is incorrect, executing the step of training the subject image by using the image training model, and outputting the subject information corresponding to the subject image. Specifically, after the user determines that the subject information obtained by recognition according to the subject image is incorrect, the subject image is trained again by using the image training model, and the accuracy of the image training model can be improved by recognizing the subject image with wrong subject information for multiple times because the image training model has the characteristics of larger training data volume and higher accuracy. In addition, when the subject information of the subject image is recognized incorrectly, the user can input correct subject information for the subject image, so that the image training model can efficiently train according to the correct subject information of the subject image.
Further, optionally, for the subject information with larger recognition quantity and higher subject recognition accuracy, the user can be queried whether the user still needs to pop up a query box to query whether the subject information is correct when the user recognizes the subject information. For example, the electronic device identifies a large number of topic images corresponding to mathematical topics, the accuracy rate of identification of the mathematical topics is 99%, the electronic device inquires the user whether the topic information of the topic images is still required to be inquired when the topic information obtained by later identification is mathematical, the user decides whether to close the inquiry function of the mathematical topics according to actual use conditions, so that the business process is simplified, and the user experience is optimized.
205. And pushing learning information corresponding to the topic image to the user.
In the embodiment of the invention, after the user confirms that the electronic equipment has identified the correct subject information of the subject image, the electronic equipment can push the related learning content to the user according to the subject information.
As an optional implementation manner, after receiving the information indicating that the subject information corresponding to the subject image is correct, which is input by the user, searching to obtain learning information corresponding to the subject image according to the subject image and the subject information corresponding to the subject image, and pushing the learning information corresponding to the subject image to the user. Specifically, the electronic device identifies character information on the topic image, searches the database according to the character information and the topic information to obtain learning contents related to the topic image, such as answer and analysis of the topic, teaching materials corresponding to the topic or learning contents including exercise rolls of the topic, and the like, so that a user can conveniently select required learning contents according to requirements for learning.
As another alternative implementation mode, the electronic equipment can record the subject information of the subject image which is recently identified by the user and search related problems according to the character information of the subject image to conduct intelligent scrolling, so that the user can exercise the recently searched content and strengthen the learning effect.
Therefore, in the embodiment of the invention, the user can identify the subject information identified by the electronic equipment and timely inform the electronic equipment of the wrong subject image. For the topic image successfully identified to obtain the topic information, the electronic equipment pushes the learning content related to the topic image to the user for the user to learn, so that the user can learn in a targeted manner.
Example III
Referring to fig. 3, fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the invention. As shown in fig. 3, the electronic device may include:
a feature extraction unit 301 for extracting image features of the history topic image;
a feature screening unit 302, configured to obtain valid image features included in image features of the historical topic image;
the model generating unit 303 is configured to generate an image training model according to the effective image features of the historical topic image and the topic information corresponding to the historical topic image.
A subject identifying unit 304 for identifying subject information corresponding to the subject image using the image identifying model;
an identification detection unit 305, configured to detect whether the image identification model successfully identifies subject information corresponding to the subject image;
the image training unit 306 is configured to, when the recognition detection unit detects that the image recognition model does not successfully recognize and obtain the subject information corresponding to the subject image, train the subject image using the image training model, and output the subject information corresponding to the subject image;
the subject identifying unit 304 specifically includes:
a first subunit 3041, configured to identify a connected domain of the topic image by using an image identification model;
a second subunit 3042, configured to analyze the connected domain of the topic image to obtain character information included in the topic image;
and a third subunit 3043, configured to obtain subject information corresponding to the subject image according to the character information.
In the embodiment of the present invention, the feature extraction unit 301 extracts the image features of the historical topic image, and the feature screening unit 302 selects the valid image features therein, so that the model generation unit 303 generates an image training model, when the topic image is identified, the topic image is identified by the topic identification unit 304 using the image training model, if the identification detection unit 305 detects that the topic identification unit 304 cannot identify the topic information, the image training unit 306 trains the topic image using the image training model, and outputs the topic information corresponding to the topic image.
As an alternative embodiment, the feature extraction unit 301 extracts image features of the history topic image, the feature screening unit 302 acquires effective image features included in the image features of the history topic image, and the model generation unit 303 generates an image training model according to the effective image features of the history topic image and the topic information corresponding to the history topic image. Specifically, the feature extraction unit 301 may train the historical topic image by using a convolutional neural network, process the historical topic image by randomly generating a convolutional check, divide the historical topic image into a plurality of regions, extract pixel point information of each region as an image feature of each region, the feature screening unit 302 further represents the region with an average value of pixel points of each region, and combine regions with a difference value of average values of adjacent pixel points smaller than a preset threshold value to obtain a combined effective image feature, so that the model generating unit 303 generates an image training model corresponding to the effective image feature of the historical topic image and the topic information of the historical topic image, and train the input topic image and learn the topic information corresponding to the topic image.
As another alternative implementation manner, the image training model may be obtained by training the convolutional neural network and the cyclic neural network, the feature extraction unit 301 extracts the image features of the historical topic image by using the convolutional neural network, and the feature screening unit 302 combines the image features multiple times by using the cyclic neural network and selects the effective image features therein for the model generating unit 303 to generate the image training model, so that the image training model can more accurately identify the topic information of the topic image.
As an alternative embodiment, the first subunit 3041 identifies the connected domain of the topic image by using the image identification model, the second subunit 3042 analyzes the connected domain of the topic image to obtain character information included in the topic image, and the third subunit 3043 analyzes the character information to obtain the topic information corresponding to the topic image. Specifically, the connected domains of the topic images are formed by the topic characters which are orderly arranged on the topic images, and the first subunit 3041 identifies the connected domains formed by the topic characters according to the edge detection algorithm, so that the identification of the regions of the topic images except the connected domains is not needed, and the identification efficiency is improved; after identifying the connected domain, the second subunit 3042 further identifies character information included in the connected domain, and the third subunit 3043 analyzes the character information, for example, the subject identifying unit 304 identifies the connected domain of the subject image to obtain the following character information (solving the following primary equation: 2x=2), and according to the comparison table in which the subject information and the corresponding keyword of the subject information are stored, can analyze to obtain the corresponding keyword in which "primary equation of the subject information" is mathematical, and can learn that the subject information of the subject image is mathematical. Therefore, the topic image with obvious image characteristics can be quickly identified by using the image identification model.
As an alternative implementation manner, the recognition detection unit 305 may set a preset recognition duration according to the history recognition duration of the image recognition model for recognizing the history topic image, and if the recognition detection unit 305 detects that the recognition duration of the topic image recognized by the topic recognition unit 304 exceeds the preset recognition duration, it may be considered that the topic recognition unit 304 does not successfully recognize and obtain the topic information corresponding to the topic image, so that the image recognition model is prevented from performing ineffective recognition on the topic image for a long time.
As an alternative implementation manner, the image training unit 306 inputs the topic image into the image training model, and the image training model can identify and obtain the effective image features of the topic image, and classify the topic image according to the effective image features, so as to accurately obtain the topic information corresponding to the topic image.
It can be seen that, in the embodiment of the present invention, the subject recognition unit 304 first uses the image recognition model to recognize the subject information corresponding to the subject image, and then the recognition detection unit 305 detects whether the image recognition model successfully recognizes the subject information corresponding to the subject image, if the subject recognition unit 304 cannot successfully recognize the subject information, the image training unit 306 uses the image training model to train the subject image, and sets the training result of the subject image output by the image training model as the subject information corresponding to the subject image. The topic images are pre-identified by using the image identification model, and then the topic images which cannot be identified by using the image training model are trained, so that the accuracy of topic identification of various topic images is improved, the situation of identification failure is greatly reduced, and the user experience is good.
Example IV
Referring to fig. 4, fig. 4 is a schematic structural diagram of an electronic device according to another embodiment of the invention; the electronic device shown in fig. 4 is optimized based on the electronic device shown in fig. 3, and the electronic device shown in fig. 4 may further include:
an inquiry unit 307, configured to push, to a user, subject information corresponding to the subject image after the image training unit trains the subject image using the image training model and outputs the subject information corresponding to the subject image, and inquire whether the subject information corresponding to the subject image of the user is correct;
the pushing unit 308 is configured to search for learning information corresponding to the topic image according to the topic image and the topic information corresponding to the topic image after receiving information indicating that the topic information corresponding to the topic image is correct, which is input by the user, and push the learning information corresponding to the topic image to the user;
in addition, the image training unit 306 is further configured to perform, when receiving information input by the user indicating that the subject information corresponding to the subject image is incorrect, a step of training the subject image using the image training model, and outputting the subject information corresponding to the subject image.
In the embodiment of the present invention, the query unit 307 pushes the image training unit 306 to the user to identify the subject information, and after the user confirms that the subject information is correct, the pushing unit 308 pushes the learning information corresponding to the subject image to the user.
As an alternative embodiment, after the image training unit 306 trains the topic image using the image training model and outputs the topic information corresponding to the topic image, the query unit 307 pushes the topic information corresponding to the topic image to the user and queries the user whether the topic information corresponding to the topic image is correct. Specifically, the query unit 307 may display the topic image and the corresponding topic information that are identified this time on the display screen of the electronic device after the image training unit 306 identifies the topic information that corresponds to the topic image, and set an option button on the display screen, and after identifying the topic information, the user clicks the corresponding option button, so as to inform the electronic device whether the topic identification is correct, thereby facilitating prompting the accuracy of the topic identification according to the feedback of the user.
Further alternatively, if receiving the information indicating that the subject information corresponding to the subject image is incorrect, the image training unit 306 performs the step of training the subject image using the image training model, and outputting the subject information corresponding to the subject image. Specifically, after the user determines that the subject information identified according to the subject image is incorrect, the image training unit 306 retrains the subject image by using the image training model, and the accuracy of the image training model can be improved by identifying the subject image with incorrect subject information multiple times because the image training model has the characteristics of larger training data volume and higher accuracy. In addition, when the subject information of the subject image is recognized incorrectly, the user can input correct subject information for the subject image, so that the image training model can efficiently train according to the correct subject information of the subject image.
Further alternatively, for subject information having a large number of identifications and a high accuracy of subject identification, the query unit 307 may query the user whether the user still needs to pop up a query box to query the user as to whether the subject information is correct when the user identifies the subject information. For example, the image training unit 306 has identified a large number of topic images corresponding to mathematical topics, and the accuracy of identifying the mathematical topics is 99%, the query unit 307 will query the user whether the user still needs to query whether the topic information of the topic image is identified correctly when the topic information obtained by the subsequent identification is a mathematical topic, and the user decides whether to close the query function of the mathematical topic according to the actual use situation, thereby simplifying the business process and optimizing the user experience.
As an alternative embodiment, the pushing unit 308 searches for learning information corresponding to the topic image according to the topic image and the topic information corresponding to the topic image after receiving the information indicating that the topic information corresponding to the topic image is correct, which is input by the user, and pushes the learning information corresponding to the topic image to the user. Specifically, the pushing unit 308 identifies character information on the topic image, and searches the database according to the character information and the topic information to obtain learning contents related to the topic image, such as answer and analysis of the topic, teaching materials corresponding to the topic, training rolls containing the topic, and the like, so that a user can conveniently select required learning contents according to requirements for learning.
As another alternative embodiment, the pushing unit 308 may record the subject information of the subject image recently identified by the user, and search for related problems according to the character information of the subject image to perform intelligent grouping, so that the user may exercise the recently searched content and strengthen the learning effect.
Therefore, in the embodiment of the invention, the user can identify the subject information identified by the electronic device, and timely inform the electronic device of the wrong subject image identified by the query unit 307. For the topic image successfully identified to obtain the topic information, the pushing unit 308 pushes learning content related to the topic image to the user for the user to learn, so that the user can learn in a targeted manner.
Example five
Referring to fig. 5, fig. 5 is a schematic structural diagram of another electronic device according to another embodiment of the present invention. As shown in fig. 5, the electronically controllable device may include:
a memory 401 storing executable program codes;
a processor 402 coupled with the memory 401;
the processor 402 invokes executable program codes stored in the memory 401 to execute any of the subject image-based subject recognition methods of fig. 1 and 2.
The embodiment of the invention discloses a computer readable storage medium storing a computer program, wherein the computer program enables a computer to execute any one of the topic image-based topic identification methods of fig. 1 and 2.
The embodiments of the present invention also disclose a computer program product, wherein the computer program product, when run on a computer, causes the computer to perform some or all of the steps of the method as in the method embodiments above.
Those of ordinary skill in the art will appreciate that all or part of the steps of the various methods of the above embodiments may be implemented by a program that instructs associated hardware, the program may be stored in a computer readable storage medium including Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), programmable Read-Only Memory (Programmable Read-Only Memory, PROM), erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), one-time programmable Read-Only Memory (OTPROM), electrically erasable programmable Read-Only Memory (EEPROM), compact disc Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM) or other optical disk Memory, magnetic disk Memory, tape Memory, or any other medium that can be used for carrying or storing data that is readable by a computer.
The above describes in detail a subject identification method based on a subject image and an electronic device, and specific examples are applied to illustrate the principles and embodiments of the present invention, where the above description of the embodiments is only for helping to understand the method and core ideas of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Claims (2)
1. A subject identification method based on a subject image, comprising:
extracting image features of the historical topic image; acquiring effective image features included in the image features of the historical topic image; generating an image training model according to the effective image characteristics of the historical subject image and subject information corresponding to the historical subject image;
identifying a connected domain of the topic image by using an image identification model; analyzing the connected domain of the topic image to obtain character information included in the topic image; analyzing according to the character information to obtain subject information corresponding to the subject image;
detecting whether the identification time length of the image identification model for identifying the topic image exceeds a preset identification time length;
if yes, determining that the image recognition model does not successfully recognize and obtain the subject information corresponding to the subject image, training the subject image by using the image training model, and outputting the subject information corresponding to the subject image;
pushing the subject information corresponding to the subject image to a user, and inquiring whether the subject information corresponding to the subject image of the user is correct or not;
if receiving the information which is input by the user and indicates that the subject information corresponding to the subject image is incorrect, executing the step of training the subject image by using an image training model and outputting the subject information corresponding to the subject image;
after receiving the information which is input by the user and indicates that the subject information corresponding to the subject image is correct, searching and obtaining learning information corresponding to the subject image according to the subject image and the subject information corresponding to the subject image, and pushing the learning information corresponding to the subject image to the user.
2. An electronic device, comprising:
the feature extraction unit is used for extracting image features of the historical topic images;
the feature screening unit is used for acquiring effective image features included in the image features of the historical subject images;
the model generation unit is used for generating an image training model according to the effective image characteristics of the historical subject image and subject information corresponding to the historical subject image;
a subject identification unit for identifying subject information corresponding to the subject image using the image identification model;
the recognition detection unit is used for detecting whether the recognition duration of the image recognition model for recognizing the question image exceeds the preset recognition duration;
the image training unit is used for determining that the image recognition model does not successfully recognize and obtain the subject information corresponding to the subject image when the recognition duration of the image recognition model for recognizing the subject image detected by the recognition detection unit exceeds the preset recognition duration, training the subject image by using the image training model, and outputting the subject information corresponding to the subject image;
the inquiring unit is used for pushing the subject information corresponding to the subject image to the user and inquiring whether the subject information corresponding to the subject image of the user is correct or not;
the image training unit is further configured to execute the step of training the topic image using an image training model and outputting the topic information corresponding to the topic image when receiving the information input by the user and indicating that the topic information corresponding to the topic image is incorrect;
the pushing unit is used for searching and obtaining learning information corresponding to the topic image according to the topic image and the topic information corresponding to the topic image after receiving the information which is input by the user and indicates that the topic information corresponding to the topic image is correct, and pushing the learning information corresponding to the topic image to the user;
the subject identification unit includes:
the first subunit is used for identifying and obtaining the connected domain of the topic image by using the image identification model;
the second subunit is used for analyzing the connected domain of the topic image to obtain character information included in the topic image;
and the third subunit is used for analyzing and obtaining the subject information corresponding to the subject image according to the character information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910183503.XA CN111027557B (en) | 2019-03-11 | 2019-03-11 | Subject identification method based on subject image and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910183503.XA CN111027557B (en) | 2019-03-11 | 2019-03-11 | Subject identification method based on subject image and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111027557A CN111027557A (en) | 2020-04-17 |
| CN111027557B true CN111027557B (en) | 2024-03-19 |
Family
ID=70203600
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910183503.XA Active CN111027557B (en) | 2019-03-11 | 2019-03-11 | Subject identification method based on subject image and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111027557B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114973815A (en) * | 2022-04-27 | 2022-08-30 | 龙诚教育科技(深圳)有限公司 | System and method for testing students in online teaching |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6621941B1 (en) * | 1998-12-18 | 2003-09-16 | Xerox Corporation | System of indexing a two dimensional pattern in a document drawing |
| CN101620680A (en) * | 2008-07-03 | 2010-01-06 | 三星电子株式会社 | Recognition and translation method of character image and device |
| CN102169542A (en) * | 2010-02-25 | 2011-08-31 | 汉王科技股份有限公司 | Method and device for touching character segmentation in character recognition |
| CN103886343A (en) * | 2014-04-09 | 2014-06-25 | 合肥科飞视觉科技有限公司 | Method and system for improving blurred picture recognition capability of visual system |
| CN105184232A (en) * | 2015-08-12 | 2015-12-23 | 祁亨年 | O2O Internet homework book, O2O Internet homework book system and realization method |
| CN107491752A (en) * | 2017-08-14 | 2017-12-19 | 中国石油大学(华东) | Ship board character recognition method, device in a kind of natural scene based on deep learning |
| CN108121984A (en) * | 2016-11-30 | 2018-06-05 | 杭州海康威视数字技术股份有限公司 | A kind of character identifying method and device |
| CN108416003A (en) * | 2018-02-27 | 2018-08-17 | 百度在线网络技术（北京）有限公司 | A kind of picture classification method and device, terminal, storage medium |
| CN108764226A (en) * | 2018-04-13 | 2018-11-06 | 顺丰科技有限公司 | Image text recognition methods, device, equipment and its storage medium |
| CN108875486A (en) * | 2017-09-28 | 2018-11-23 | 北京旷视科技有限公司 | Recongnition of objects method, apparatus, system and computer-readable medium |
-
2019
- 2019-03-11 CN CN201910183503.XA patent/CN111027557B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6621941B1 (en) * | 1998-12-18 | 2003-09-16 | Xerox Corporation | System of indexing a two dimensional pattern in a document drawing |
| CN101620680A (en) * | 2008-07-03 | 2010-01-06 | 三星电子株式会社 | Recognition and translation method of character image and device |
| CN102169542A (en) * | 2010-02-25 | 2011-08-31 | 汉王科技股份有限公司 | Method and device for touching character segmentation in character recognition |
| CN103886343A (en) * | 2014-04-09 | 2014-06-25 | 合肥科飞视觉科技有限公司 | Method and system for improving blurred picture recognition capability of visual system |
| CN105184232A (en) * | 2015-08-12 | 2015-12-23 | 祁亨年 | O2O Internet homework book, O2O Internet homework book system and realization method |
| CN108121984A (en) * | 2016-11-30 | 2018-06-05 | 杭州海康威视数字技术股份有限公司 | A kind of character identifying method and device |
| CN107491752A (en) * | 2017-08-14 | 2017-12-19 | 中国石油大学(华东) | Ship board character recognition method, device in a kind of natural scene based on deep learning |
| CN108875486A (en) * | 2017-09-28 | 2018-11-23 | 北京旷视科技有限公司 | Recongnition of objects method, apparatus, system and computer-readable medium |
| CN108416003A (en) * | 2018-02-27 | 2018-08-17 | 百度在线网络技术（北京）有限公司 | A kind of picture classification method and device, terminal, storage medium |
| CN108764226A (en) * | 2018-04-13 | 2018-11-06 | 顺丰科技有限公司 | Image text recognition methods, device, equipment and its storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 王熙.好未来教育服务平台的设计与实现.《中国优秀硕士学位论文全文数据库》.2018,(第12期),第1-81页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111027557A (en) | 2020-04-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109271401B (en) | Topic searching and correcting method and device, electronic equipment and storage medium | |
| US11410407B2 (en) | Method and device for generating collection of incorrectly-answered questions | |
| CN109446315B (en) | Question solving auxiliary method and question solving auxiliary client | |
| CN107343223B (en) | The recognition methods of video clip and device | |
| KR20180122926A (en) | Method for providing learning service and apparatus thereof | |
| CN108768654B (en) | Identity verification method based on voiceprint recognition, server and storage medium | |
| CN109656465B (en) | Content acquisition method applied to family education equipment and family education equipment | |
| CN109408821B (en) | Corpus generation method and device, computing equipment and storage medium | |
| CN109284355B (en) | Method and device for correcting oral arithmetic questions in test paper | |
| CN108197300B (en) | Question searching method and system | |
| CN106484132B (en) | Input error correction method and input method device | |
| CN106649849A (en) | Text information base building method and device and searching method, device and system | |
| CN113450147B (en) | Product matching method, device, equipment and storage medium based on decision tree | |
| CN111026949A (en) | Question searching method and system based on electronic equipment | |
| CN111078856A (en) | Group chat conversation processing method and device and electronic equipment | |
| CN111241400B (en) | Information searching method and device | |
| CN111027557B (en) | Subject identification method based on subject image and electronic equipment | |
| CN108093032B (en) | Answer lock control method and client | |
| CN111639630B (en) | Operation modifying method and device | |
| CN113205046A (en) | Method, system, device and medium for identifying question book | |
| CN109993053B (en) | Electronic device, interview assisting method based on microexpressive recognition, and storage medium | |
| Boillet et al. | Confidence estimation for object detection in document images | |
| CN111079777B (en) | Page positioning-based click-to-read method and electronic equipment | |
| CN110750712A (en) | Software security requirement recommendation method based on data driving | |
| KR20210092552A (en) | Method and device for detecting nutrition data of product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |