WO2019075819A1 - Paper grading method and paper grading system - Google Patents

Paper grading method and paper grading system Download PDF

Info

Publication number
WO2019075819A1
WO2019075819A1 PCT/CN2017/111809 CN2017111809W WO2019075819A1 WO 2019075819 A1 WO2019075819 A1 WO 2019075819A1 CN 2017111809 W CN2017111809 W CN 2017111809W WO 2019075819 A1 WO2019075819 A1 WO 2019075819A1
Authority
WO
WIPO (PCT)
Prior art keywords
answer
cloud
word
phrase
vocabulary
Prior art date
Application number
PCT/CN2017/111809
Other languages
French (fr)
Chinese (zh)
Inventor
卢启伟
杨宁
刘佳
Original Assignee
深圳市鹰硕技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市鹰硕技术有限公司 filed Critical 深圳市鹰硕技术有限公司
Publication of WO2019075819A1 publication Critical patent/WO2019075819A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/237—Lexical tools
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/103—Formatting, i.e. changing of presentation of documents
    • G06F40/106—Display of layout of documents; Previewing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/20—Education

Definitions

  • the invention relates to the field of multimedia teaching technology, in particular to a method for judging a volume and a judgment system for multimedia teaching based on an internet teaching platform.
  • test questions can be divided into subjective questions and objective questions.
  • objective questions are mostly based on multiple-choice questions. Since the answers to objective questions are fixed, it is very easy to use computer to score objective questions, thus avoiding the use of old manual scoring methods, which can shorten the scoring time and save the labor cost of scoring. Improve the efficiency of marking.
  • students usually use the way of discussion to answer questions in the questions.
  • a smart scoring method and apparatus, computer equipment and computer readable medium are now proposed in the art.
  • the method includes: obtaining a user answer corresponding to the test question; generating, according to the user answer, a user word segment corresponding to the user answer; and according to the user participle library, a standard word segment corresponding to the pre-generated standard answer And the weight of each standard word segment in the standard word segment, and the user answer is scored.
  • the program can intelligently score the user answers of the questions of the type of subjective questions, and can overcome the defects of using the manual marking method in the prior art, resulting in a long scoring time and high labor cost, thereby effectively shortening the pair.
  • the scoring time of the questions can also be effective The labor cost is saved, and the scoring efficiency of the user answers to the type of subjective questions is greatly improved.
  • the prior art also proposes a subjective title automatic scoring system and method based on semantic similarity interval.
  • the subjective title automatic scoring method based on semantic similarity interval includes the following steps: initializing the score of the subject to be scored Stotal; defining the length of the block L; the reference answer is divided into a number of blocks of length not greater than L, forming a reference answer block set R; the answer to be evaluated is divided into a number of blocks of length not greater than L, forming a set of T to be evaluated; Compare the set R and T, calculate the semantic similarity SRT of the two; map the SRT to the similarity interval, record the score as Sfinal, and end the score; the subjective title automatic scoring system based on the semantic similarity interval includes: answer terminal, subjective question Marking module, test score generation module.
  • the scheme can classify the subjective answers according to the semantic similarity threshold by automatically scoring and scoring the subjective questions, and then give the final score through the score constraint and the scores of each level similarity.
  • the score will be automatically reviewed; for questions such as fill in the blanks, corrections, etc., the questions may not be unique.
  • the questions are automatically reviewed through statement comparison, code substitution program operation, etc. For one answer, only one review is performed, and other answers are automatically completed. Review of the same answer; for programming questions, according to different inputs, whether the running result is consistent with the expected answer (automatic review), whether it can compile and run (manual intervention review), source code logic analysis (manual intervention review) give different points value.
  • This prior art introduces intelligent evaluation of subjective questions to a certain extent, but this prior art can only be judged by subjective questions in the computer field, and subjective questions in other fields cannot be verified by means of calculation, and there is no About the content of the end user maintenance score database.
  • an annotation method including: scanning a paper document to acquire a first image; and acquiring text content of a plurality of titles and a plurality of second images of the plurality of answers according to the first image Obtaining a plurality of search words according to the plurality of second images; retrieving the annotation database according to the plurality of search terms; adding the annotation content to the unretrieved search terms, and storing the newly added annotation content in the annotation database; a search term, modify the annotation content, and store the modified annotation content into the annotation database; for all the search terms, an annotation list is created, the annotation list includes the search term and the corresponding annotation content; according to the annotation list, the generation includes multiple The text content of the title and corresponding annotation content and automatically send an email.
  • the method improves the efficiency of the annotation by recording and summarizing the annotation information, establishing an annotation database, and retrieving annotations through search terms.
  • This method effectively improves the efficiency of teachers.
  • it is only a simple patchwork of machine scores and manual scores. There is no reference between teachers of each score.
  • the text of the annotation is not limited by the length.
  • the advantage is that there is a lot of flexibility in annotating the content of the text.
  • different teachers can give their own annotations independently. Teachers can demonstrate the characteristics of their teaching work through the review of homework, and convey their understanding and experience of education and teaching to students. This clearly gives the revelation of the independent comments made by the teachers, and believes that personalized annotations are conducive to teaching. This is equivalent to negating the necessity of the consistency of the review standards, and it is difficult for the skilled person in the field to obtain the enlightenment of the online batch.
  • a method for determining a volume comprising the steps of:
  • S4 is specifically:
  • S402 determining whether the highlighted word or phrase is selected; if yes, executing S403; if not, executing S5;
  • S403 Calling related information of the word or phrase from the cloud rating vocabulary and displaying it on the screen.
  • step S6 the word or phrase in the answer is selected by double-clicking or more clicking on the position of the word or phrase on the display screen of the rating terminal, and automatically selecting before the location is automatically generated.
  • the character and word between the front and back selectors will be selected, and the choice of the word or phrase specific content can be determined by adjusting the position of the front and back selectors, while pre-scheduled on the screen.
  • the area or the area near the selector pops up a menu that includes at least the command options uploaded to the cloud.
  • a pre-defined gesture or a separate option is used to trigger the generation of the front and back selectors.
  • the application also proposes a method of judging, comprising the following steps:
  • S3' select a part of the text area in the electronic answer, and call the cloud rating lexicon information
  • S4' highlighting a word or a phrase appearing in the cloud scoring vocabulary in the electronic answer, and using the highlighted word or phrase of the partial text area as a term to display related information of the term;
  • S5' determining whether the term is added to the cloud rating vocabulary, if yes, executing S6'; if not, executing S7';
  • step S4' is specifically:
  • the word or phrase in the answer is selected by double clicking or more clicking on the position of the word or phrase on the display screen of the rating terminal, and automatically generating the pre-selection at the position.
  • the character and word between the front and back selectors will be selected, and the choice of the word or phrase specific content can be determined by adjusting the position of the front and back selectors, while on the screen A pop-up menu is popped up in the predetermined area or near the selected area, and the menu includes at least command options uploaded to the cloud.
  • the rating terminal limits the number of keywords and the number of keyword categories included in the entry, and the number of keywords appearing in the text area drawn by the reviewer exceeds a predetermined value, S3 The middle part of the text area will be considered an invalid text area.
  • step S4' if the number of keywords and the category included in the selected answer text area does not change, only the total number of words in the answer text area changes, and the judgment terminal may consider that the entry does not change.
  • the application also proposes a judgment system, which has a judgment cloud and a plurality of judgment terminals connected in the cloud, and the judgment cloud includes a cloud rating vocabulary, and the reviewer uses the judgment terminal to perform subjective questions, and its characteristics are characterized. It is that the judgment terminal uses the aforementioned method of judgment.
  • the method of judging the application is a method of judging between the manual scoring and the complete computer judging in the prior art, and the participation of the reviewer makes the judgment more objective, and in the method of judging, the computer is used.
  • the electronic answer obtained by the judgment terminal and the cloud score vocabulary The stored judgment standard is relatively high. If a word or phrase is pre-existing in the cloud-based vocabulary, the word or phrase will be highlighted. It is very easy for the reviewer to find this score keyword in the electronic answer, which greatly improves the efficiency of the work.
  • Keyword highlighting also has an important technical effect. If certain words are encountered, they are basically correct, but they are not recorded in the standard answer, or it is difficult to give a score based on the standard answer.
  • the invention proposes a technical idea for the reviewer to maintain the standard answer. If the reviewer easily finds that the word is not highlighted on the screen, the word can be selected and added to the cloud rating vocabulary, thereby making the content of the standard answer more comprehensive. This clearly distinguishes the technical solution of the present application from the prior art.
  • the present invention further proposes that the rating terminal limits the number and category of keywords included in the entry.
  • the rating terminal limits the number and category of keywords included in the entry.
  • the judging terminal further determines whether the number of words included in the selected answer text area is greater than a preset first threshold, and whether the number of included keywords is greater than a preset second threshold, and if not, the entry can be performed or Read.
  • FIG. 1 is a flow chart of a first method of determining a volume according to the present invention
  • FIG. 3 is a flow chart of a second method of determining the invention.
  • the judgment system has a judgment cloud and a plurality of terminals connected to the cloud, and the reviewer uses the terminal to perform a review of the subjective question.
  • a method for determining a volume includes the following steps:
  • S2 Get the subjective answer and get an electronic answer.
  • the user answer corresponding to the exam question is obtained.
  • the user answer of the question is in electronic form
  • the user answer of the question is directly read.
  • the text recognition technology can be used to identify the text in the user's answer on the paper, and the user's answer in electronic form can be obtained.
  • the method of obtaining the answer to the subjective question is not limited to the above two methods, and only needs to be able to obtain an electronic answer.
  • the cloud scoring vocabulary is set in the cloud of the judgment system.
  • the scoring standard of subjective questions is preset, which is the same as the traditional manual scoring.
  • the scoring standard for subjective questions is mainly to see if there is a key in the answer.
  • Words or phrases, which are commonly referred to as score points are scored based on subjective questions on the coverage of score points and other considerations.
  • the keywords of the answers to the subjective questions are also preset, and together with the keywords, the pairs are also included. The interpretation of the keyword, the synonym of the keyword, etc. may appear related expression.
  • S4 Highlight the words or phrases that appear in the cloud rating vocabulary in the electronic answer.
  • the electronic answer obtained by the judgment terminal is compared with the pre-stored scoring standard in the cloud scoring vocabulary. If a word or phrase belongs to a pre-stored keyword or its related expression in the cloud scoring vocabulary, the word or phrase will be highlighted. show. It is very easy for the reviewer to find these scoring keywords in the electronic answer, which greatly improves the efficiency of the work.
  • step S4 If the reviewer finds in the electronic answer displayed in step S4, one of the words can also be used as a score item, but is not recorded in the cloud scoring vocabulary.
  • the most intuitive performance is that the word is not highlighted on the screen, you can select the word or phrase and add it to the cloud rating vocabulary, so that the standard answer content is more comprehensive.
  • S6 Get the selected word or phrase, add it to the cloud rating vocabulary, and return to S3.
  • the terminal will re-compare the electronic answer with the cloud rating vocabulary. After comparison, the word will also be highlighted. The important thing is that if the word appears in other electronic answers, the word will also be highlighted on the other terminal.
  • S7 Get the score entered by the reviewer.
  • the reviewer gives the final score based on the full presentation of the keywords appearing in the electronic answer and other scoring factors and enters the terminal.
  • step S4 is specifically:
  • S401 Highlight the words or phrases appearing in the cloud scoring vocabulary in the electronic answer according to different categories; this will be more conducive to the scoring person to intuitively understand the scoring situation in the electronic answer. For example, different keywords can be distinguished by different colors.
  • S402 determining whether the highlighted word or phrase is selected; if yes, executing S403; if not, executing S5;
  • S403 Calling related information of the word or phrase from the cloud rating vocabulary and displaying it on the screen, After executing S5.
  • Steps S402 and S403 are mainly for facilitating the reader to better understand the information related to the keyword, or the relevant scoring standard.
  • the answer may be selected in multiple ways, for example, by double clicking or more clicking on a display screen of the rating terminal, where the word or phrase is located, and the pre-selector is automatically generated at the position.
  • the post selector the character or word between the front and back selectors will be selected, and the choice of the word or phrase specific content can be determined by adjusting the position of the front and rear selectors, while at the predetermined area of the screen.
  • a pop-up menu near the area of the selector the menu includes at least the command options uploaded to the cloud.
  • the menu item After the menu item is triggered, it also includes at least adding an explanation or annotation to the word or phrase to explain why the word is added to the cloud rating vocabulary.
  • the method of selecting a word or phrase can also be just the location of the word or phrase on which the cursor stays on the screen.
  • pre- and post-selection can be triggered using predefined gestures or separate options. For example, multiple clicks on a certain area of the screen, long press on the screen, these modes of operation are conventional means in the art, and will not be enumerated.
  • Embodiment 1 the difference from Embodiment 1 is:
  • a method of determining a volume includes the following steps:
  • S3' select a part of the text area in the electronic answer, and call the cloud rating lexicon information
  • S4' highlighting a word or a phrase appearing in the cloud scoring vocabulary in the electronic answer, and using the highlighted word or phrase of the partial text area as a term to display related information of the term;
  • S5' determining whether the term is added to the cloud rating vocabulary, if yes, executing S6'; if not, executing S7';
  • step S4' the judgment terminal first detects the answer text area selected by the reviewer, and judges how many different types of keywords appear in the area according to the highlight of the keyword.
  • the reviewer can divide the selected area according to the actual situation of the answer, such as a complete expression paragraph, such as a response to a specific question in the title.
  • These answer areas with a more complete form of meaning will be subjectively divided by the reviewer. come out.
  • the keyword information of the answer text area will then be extracted, for example including the first keyword and the second keyword, which will be identified as the first term.
  • the information corresponding to the term is called from the cloud rating vocabulary and displayed on the screen, and then S5' is executed.
  • the scorer will give a specific evaluation of this particular keyword combination, including but not limited to a series of information such as the correctness, accuracy, relevance, and recommended score of the entry, and the first word
  • the bars form a mapping relationship.
  • the reviewer first delineates a valid text area, that is, multiple key categories of keywords appear in a valid text area, which avoids the candidate's desire to disperse the expected keyword scores; For controversial and difficult answers, they can be promptly presented and quickly analyzed and resolved. These controversial and difficult answers are generally unpredictable at the beginning of the answer.
  • S5' determine whether to add the current entry to the cloud rating vocabulary.
  • the first entry has been indexed, but if it is not sent by the reviewer, it will not be recorded by the cloud scoring vocabulary.
  • the reviewer decides whether the first entry should be based on his own subjective judgment. Was added to the standard answer, whether it should be the standard answer for this review.
  • the rating terminal limits the number and category of keywords included in the entry.
  • the text area drawn by the reviewer is too large, or the keyword appears too much, it is considered to be an invalid text area. This is because if the text area is too large or contains too many keywords, it cannot be distinguished from the candidate's behavior of listing keywords, and for a regular expression, a statement with independent expression will not contain too much. Keywords. Therefore, the judging terminal further determines whether the number of words included in the selected answer text area is greater than a preset first threshold, and whether the number of included keywords is greater than a preset second threshold, and if not, the entry can be performed or Read.
  • the words or phrases or terms uploaded by the reviewer to the cloud scoring vocabulary are included in the cloud scoring vocabulary after being confirmed by the advanced user.
  • the standard answer in the cloud rating vocabulary is first preset: the first type of keywords: love; the second type of keywords: sadness, pain; the third type of keywords: the motherland, the country; the fourth category of keywords: Tall, great, admired; fifth type of keyword: Little Francis; sixth type of keyword: Han Maier.
  • the candidate's electronic answer is displayed at the judgment terminal.
  • the judgment terminal will be scored from the cloud.
  • the auxiliary information of the entry is retrieved. If the entry is not created, the reviewer can actively create the entry and send it to the cloud rating vocabulary to add a rating suggestion for the entry or other scoring criteria.
  • the text area keyword drawn by the reviewer appears too much or the category of the keyword is too large, it will be considered as invalid.
  • the text area because if the text area contains too many keywords, it can't be distinguished from the behavior of the candidate's keyword listing. For example, the reviewer will use the "Little Francis” and “Han Maier” in the second answer. The four keywords of "Motherland” and “Love” are selected. If this entry is created, it may not be conducive to the progress of the score, because the combination of the four words is too likely, and contains a large number of correct and wrong. The combination.
  • the method of judging the application can first improve the work efficiency of the reviewer by keyword highlighting, and further, it can provide related information of the corresponding entry according to the terms of the keyword. Whether it is a keyword or an entry, the reviewer can modify and enrich the data in the cloud during the marking process, so that the entire scoring process becomes more and more unified and more scientific.
  • a decision terminal which may be a computer device, such as in the form of a general purpose computing device.
  • Components of a computer device include, but are not limited to, one or more processors, system memory, buses that connect different system components, including system memory and processors.
  • bus generally represents one or more of several types of bus structures in the art, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or any bus structure using a plurality of bus structures.
  • Local bus These architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA Bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.
  • ISA Industry Standard Architecture
  • MAC Micro Channel Architecture
  • VESA Video Electronics Standards Association
  • PCI Peripheral Component Interconnect
  • the computer device can also be in communication with one or more external devices (eg, a keyboard, pointing device, display, etc.), and can also communicate with one or more devices that enable a user to interact with the computer device, and/or with the computer device Anything that can communicate with one or more other computing devices Devices (such as network cards, modems, etc.) communicate. This communication can be done via an input/output (I/O) interface.
  • the computer device can communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through a network adapter.
  • the network adapter communicates with other modules of the computer device over the bus.
  • Computer devices typically include a variety of computer system readable media. These media can be any available media that can be accessed by a computer device, including both volatile and nonvolatile media, removable and non-removable media.
  • the system memory can include computer system readable media in the form of volatile memory, such as random access memory (RAM) and/or cache memory.
  • the computer device may further include other removable/non-removable, volatile/non-volatile computer system storage media.
  • the system memory can include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the above-described assertions of the present invention.
  • a program/utility having a set of (at least one) program modules, which may be stored, for example, in system memory, such program modules including an operating system, one or more applications, other program modules, and program data, in these examples Implementations of the network environment are included in each or some combination.
  • Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including an object oriented programming language such as Java, Smalltalk, C++, and conventional A procedural programming language - such as the "C" language or a similar programming language.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer, partly on the remote computer, or entirely on the remote computer or server.
  • the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (eg, using an Internet service provider) Internet connection).
  • LAN local area network
  • WAN wide area network
  • Internet service provider Internet service provider
  • the invention allows the reviewer to maintain the standard answer. If the reviewer easily finds that the word is not highlighted on the screen, the word can be selected and added to the cloud scoring vocabulary for reference by other reviewers, so that the standard answer can be continued. Update.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Tourism & Hospitality (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Economics (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Primary Health Care (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Electrically Operated Instructional Devices (AREA)

Abstract

A paper grading system, which has a paper grading cloud end and multiple paper grading terminals. The paper grading cloud end comprises a cloud-end scoring word library; a marker uses the paper grading terminals to mark a subjective question; during marking, an electronic answer is compared with a standard answer pre-stored in the cloud-end scoring word library, and if a certain word or phrase is pre-stored in the cloud-end scoring word library, then this word or phrase is highlighted. The present invention facilitates the finding of these scoring keywords in the electronic answer by the marker, thus greatly improving the efficiency of grading papers. If some words are also correct, but are not recorded in the standard answer, or are difficult to score using the standard answer as a basis, the marker is allowed to maintain the standard answer. If the marker easily finds that the word is not highlighted on a screen, then the word can be selected and added to the cloud-end scoring word library for other markers' reference, thus, the standard answer can be continuously updated.

Description

一种判卷方法及判卷***Method of judging and judging system 技术领域Technical field
本发明涉及多媒体教学技术领域，具体而言，涉及一种用于基于互联网教学平台的多媒体教学的判卷方法及判卷***。The invention relates to the field of multimedia teaching technology, in particular to a method for judging a volume and a judgment system for multimedia teaching based on an internet teaching platform.
背景技术Background technique
在培训、考试、教育等***中，为了对学员的学习情况进行考核，通常需要对学员进行考试，并针对考试结果评分，以了解学员对学习的知识的掌握程度。通常情况下，为了对学员进行有效地测评，测试考试的考题可以分为主观题和客观题。其中客观题多采用选择题的方式，由于客观题的答案是固定的，非常便于采用计算机对客观题进行评分，从而避免采用老旧的人工阅卷的方式，能够缩短阅卷时间，节省阅卷的人力成本，提高阅卷效率。而对于主观题，学员通常采用论述的方式解答考题中的问题，主观题的解答过程中，通常需要学员自我发挥，按照自己的思维方式去解答考题中的问题。因此主观题的答案仅仅是参考，不是绝对的标准，因此，现在技术中，主观题一般是由人工来阅卷。人工阅卷有很多缺陷和不足，主要有两点，一是人工阅卷效率比较低，二是人工阅卷的阅卷标准难以统一，因为阅卷的人员人数较多，而且考生在答题过程中表达是多样的，而主观题的答案及判分标准相对单一，因此，主观题的阅卷人判分尺度难以统一，自由裁量权过大。In the training, examination, education and other systems, in order to assess the learning situation of the students, it is usually necessary to take the examinations for the students and score the examination results to understand the mastery of the students' knowledge of the learning. Usually, in order to effectively evaluate the students, the test questions can be divided into subjective questions and objective questions. Among them, objective questions are mostly based on multiple-choice questions. Since the answers to objective questions are fixed, it is very easy to use computer to score objective questions, thus avoiding the use of old manual scoring methods, which can shorten the scoring time and save the labor cost of scoring. Improve the efficiency of marking. For subjective questions, students usually use the way of discussion to answer questions in the questions. In the process of answering subjective questions, students usually need to play their own way and solve the questions in the questions according to their own way of thinking. Therefore, the answer to the subjective question is only a reference, not an absolute standard. Therefore, in the current technology, the subjective question is generally read by hand. There are many defects and shortcomings in manual marking. There are two main points. First, the efficiency of manual marking is relatively low. Second, the marking standards for manual marking are difficult to be unified. Because the number of people who read the paper is large, and the candidates are diverse in the process of answering questions. The answer to the subjective question and the criteria for judging the score are relatively simple. Therefore, the scoring scale of the subjective question is difficult to unify and the discretion is too large.
随着计算机应用的发展，人工阅卷也更多的使用计算机技术。一种是将主观题直接扫描后以图片的形式收录到***中，阅卷人在电脑前看主观题的图片，这和直接阅卷没什么区别，仅是便于试卷的收录和统计；另一种是更多的技术人员希望用计算机阅卷来替代人工阅卷。With the development of computer applications, manual scoring is also more using computer technology. One is to scan the subjective questions directly into the system in the form of pictures. The reviewers look at the pictures of subjective questions in front of the computer. This is no different from direct reading. It is only for the collection and statistics of the test papers; the other is more Many technicians want to use computer scoring instead of manual scoring.
现在技术中提出过一种智能评分方法及装置、计算机设备及计算机可读介质。其中所述方法包括：获取考题对应的用户答案；根据所述用户答案，生成所述用户答案对应的用户分词库；根据所述用户分词库、预先生成的标准答案对应的标准分词库以及所述标准分词库中的各标准分词的权重，为所述用户答案进行评分。该方案可以对类型为主观题的考题的用户答案进行智能化地评分，能够克服现有技术中采用人工阅卷的方式，导致评分时间较长，人力成本较高的缺陷，从而能够有效地缩短对考题的评分时间，还能够有效 地节约人力成本，进而大大地提高对类型为主观题的用户答案的评分效率。A smart scoring method and apparatus, computer equipment and computer readable medium are now proposed in the art. The method includes: obtaining a user answer corresponding to the test question; generating, according to the user answer, a user word segment corresponding to the user answer; and according to the user participle library, a standard word segment corresponding to the pre-generated standard answer And the weight of each standard word segment in the standard word segment, and the user answer is scored. The program can intelligently score the user answers of the questions of the type of subjective questions, and can overcome the defects of using the manual marking method in the prior art, resulting in a long scoring time and high labor cost, thereby effectively shortening the pair. The scoring time of the questions can also be effective The labor cost is saved, and the scoring efficiency of the user answers to the type of subjective questions is greatly improved.
现有技术还提出了一种基于语义相似度区间的主观题自动评分***及方法，该基于语义相似度区间的主观题自动评分方法包括以下步骤：初始化待评分题目分值Stotal；定义字块长度L；将参***切分成若干个长度不大于L的字块，形成参***字块集R；将待评答案切分成若干个长度不大于L的字块，形成待评答案字块集T；比较集合R、T，计算二者语义的相似度SRT；将SRT映射到相似度区间，记录评分为Sfinal，评分结束；该基于语义相似度区间的主观题自动评分***包括：答题终端、主观题阅卷模块、考试成绩生成模块。该方案对主观题的自动阅卷、评分环节，可实现按语义相似度阈值对主观题答案进行等级划分，进而通过分值约束以及各等级相似度分值化给出最终评分。The prior art also proposes a subjective title automatic scoring system and method based on semantic similarity interval. The subjective title automatic scoring method based on semantic similarity interval includes the following steps: initializing the score of the subject to be scored Stotal; defining the length of the block L; the reference answer is divided into a number of blocks of length not greater than L, forming a reference answer block set R; the answer to be evaluated is divided into a number of blocks of length not greater than L, forming a set of T to be evaluated; Compare the set R and T, calculate the semantic similarity SRT of the two; map the SRT to the similarity interval, record the score as Sfinal, and end the score; the subjective title automatic scoring system based on the semantic similarity interval includes: answer terminal, subjective question Marking module, test score generation module. The scheme can classify the subjective answers according to the semantic similarity threshold by automatically scoring and scoring the subjective questions, and then give the final score through the score constraint and the scores of each level similarity.
上述两种阅卷方式基本不需要人参与，但这种阅卷方式或方法仅适用于普通的测试和练习，不能适用于大型的重要考试。另外，在设置预先生成的标准答案时，设置尺度难以把握，在之后的阅卷中也无法调整标准答案。The above two methods of marking do not require human participation, but this method or method of marking is only applicable to ordinary tests and exercises, and cannot be applied to large-scale important examinations. In addition, when setting a pre-generated standard answer, setting the scale is difficult to grasp, and the standard answer cannot be adjusted in subsequent scoring.
还有现有技术认为，对于几乎所有高校都开设的C程序设计课程，既不可能只考选择题和判断题，也不宜将学生编写的程序因结果与参***不完全一致而直接判为零分。鉴于同一问题用程序实现可能有多种算法或书写方式，使得完全用计算机实现自动阅卷非常困难，因此需要将自动阅卷与人工阅卷相结合，在提高阅卷效率的同时兼顾合理性。因此在判主观题时，若输入的答案与参***相同，返回评分；若输入的答案与参***不相同，对输入的答案进行测试，这里的测试为对于阅读程序题可根据考生提交的答案种类设定分值后自动评阅；对于程序填空、改错等答案可能不唯一的题型通过语句比对、代码代入程序运行等方式进行自动评阅，对于一种答案只进行一次评阅，自动完成其他相同答案的评阅；对于编程题则按对于不同的输入，运行结果与预期答案是否一致(自动评阅)、能否编译运行(人工干预评阅)、源代码逻辑分析(人工干预评阅)给予不同的分值。这种现有技术一定程度的引入了对主观题的智能评价，但这个现有技术只能用在计算机领域的主观题判断，其他领域的主观题无法通过代入计算的方式进行验证，另外也没有关于终端用户维护评分数据库的内容。There is also the prior art that it is impossible to test only the multiple-choice and judgment questions for almost all colleges and universities. It is also not possible to directly judge the procedures written by students because the results are not completely consistent with the reference answers. Minute. In view of the same problem, there may be multiple algorithms or writing methods implemented by the program, which makes it very difficult to implement automatic marking by computer. Therefore, it is necessary to combine automatic marking with manual marking, and improve the efficiency of marking while taking into consideration the rationality. Therefore, when the subject is judged, if the input answer is the same as the reference answer, the score is returned; if the input answer is not the same as the reference answer, the input answer is tested, and the test here is the answer that can be submitted according to the candidate for the reader problem. After the category is set, the score will be automatically reviewed; for questions such as fill in the blanks, corrections, etc., the questions may not be unique. The questions are automatically reviewed through statement comparison, code substitution program operation, etc. For one answer, only one review is performed, and other answers are automatically completed. Review of the same answer; for programming questions, according to different inputs, whether the running result is consistent with the expected answer (automatic review), whether it can compile and run (manual intervention review), source code logic analysis (manual intervention review) give different points value. This prior art introduces intelligent evaluation of subjective questions to a certain extent, but this prior art can only be judged by subjective questions in the computer field, and subjective questions in other fields cannot be verified by means of calculation, and there is no About the content of the end user maintenance score database.
现有技术中还有一种对主观题的判分方法，对于非制图类题型区域的图 像，也就是主观题区域的图像，进行图像模式识别，提取答案关键字，将关键字与关键字数据进行匹配，根据匹配结果进行非制图类题型初步给分；不同科目的老师根据课程代码接入数据库的对应文件夹进行评阅，进行非制图类题型补充给分；将非制图类题型初步给分和非制图类题型补充给分进行合并，得到非制图类题型分数。这个现有技术提出机器评分和人工评分终合的评分，其中人工的补充评分很主观题的评分更加客观。但这篇文献也没有公开终端用户维护评分数据库的内容。各门课的教师在补充给分时，和现有的主观题评分方式没什么区别，只是机器评分和人工评分的简单拼凑，各评分教师之间没有任何的借鉴。In the prior art, there is also a method for judging subjective questions, and a map for non-graphical type regions. Image, that is, the image of the subjective question area, image pattern recognition, extract the answer keyword, match the keyword with the keyword data, and perform preliminary scores for the non-graphic type questions according to the matching result; the teachers of different subjects according to the course code The corresponding folder of the access database is reviewed, and the non-graphical type of questions is added to the points; the non-graphical type of initial and non-graphical types are added to the points for merging, and the non-graphical type questions are obtained. This prior art proposes a score for the final score of the machine score and the manual score, wherein the artificial supplement score is more objective than the score of the subjective question. However, this document does not disclose the content of the end user maintenance scoring database. The teachers in each course are no different from the existing subjective questions in the way of supplementing the points. It is just a simple patchwork of machine scores and manual scores. There is no reference between the score teachers.
还有一种现有技术，其使用了批注方法，包括：扫描纸质文件，以获取第一图像；根据所述第一图像，获取多个题目的文本内容和多个答案的多个第二图像；根据多个第二图像获取多个检索词；根据多个检索词检索批注数据库；针对未检索到的检索词，新增批注内容，并将新增批注内容存储至批注数据库中；针对检索到的检索词，修改批注内容，并将修改的批注内容存储至批注数据库中；针对全部检索词，建立批注列表，所述批注列表包括检索词和相应的批注内容；根据批注列表，生成包括多个题目和相应批注内容的文本内容并自动发送电子邮件。该方法通过记录和汇总批注信息，建立批注数据库，通过检索词检索批注，提高了批注使用效率。这种方法有效的提高了教师的工作效率。但其还只是机器评分和人工评分的简单拼凑，各评分教师之间没有任何的借鉴。特别是这种技术中明确的认为在该批注方法中，批注内容文字不受篇幅限制。其优势在于：批注内容文字内容具有很大的灵活性。针对相同的问题，不同的教师可以独立给出自己的批注内容。教师能够通过作业的批阅，展现其教学工作特色，并且将自己对教育教学工作的认识、体会转达给学生。这明确的给出教师之间独立做出的批注的启示，认为个性化的批注有利于教学。这相当于否定了批阅标准一致的必要性，从中本领域技术人员难以获得联机批卷的启示。There is also a prior art that uses an annotation method, including: scanning a paper document to acquire a first image; and acquiring text content of a plurality of titles and a plurality of second images of the plurality of answers according to the first image Obtaining a plurality of search words according to the plurality of second images; retrieving the annotation database according to the plurality of search terms; adding the annotation content to the unretrieved search terms, and storing the newly added annotation content in the annotation database; a search term, modify the annotation content, and store the modified annotation content into the annotation database; for all the search terms, an annotation list is created, the annotation list includes the search term and the corresponding annotation content; according to the annotation list, the generation includes multiple The text content of the title and corresponding annotation content and automatically send an email. The method improves the efficiency of the annotation by recording and summarizing the annotation information, establishing an annotation database, and retrieving annotations through search terms. This method effectively improves the efficiency of teachers. However, it is only a simple patchwork of machine scores and manual scores. There is no reference between teachers of each score. In particular, it is clear in this technique that in the annotation method, the text of the annotation is not limited by the length. The advantage is that there is a lot of flexibility in annotating the content of the text. For the same problem, different teachers can give their own annotations independently. Teachers can demonstrate the characteristics of their teaching work through the review of homework, and convey their understanding and experience of education and teaching to students. This clearly gives the revelation of the independent comments made by the teachers, and believes that personalized annotations are conducive to teaching. This is equivalent to negating the necessity of the consistency of the review standards, and it is difficult for the skilled person in the field to obtain the enlightenment of the online batch.
如何利用计算机既能高效的完成人工阅卷，又能标准一致的保证阅卷的质量是本领域长期希望解决的技术问题，有鉴于此，提出本发明。 How to use the computer to efficiently perform manual scoring and to ensure the quality of scoring in a consistent manner is a technical problem that has long been desired in the art, and in view of this, the present invention has been proposed.
发明内容Summary of the invention
本发明的目的在于提供一种判卷方法及***，以解决上述问题。It is an object of the present invention to provide a method and system for determining a problem to solve the above problems.
为了实现所述发明目的，本发明采用如下技术方案：In order to achieve the object of the invention, the present invention adopts the following technical solutions:
一种判卷方法，其特征在于，包括以下步骤：A method for determining a volume, comprising the steps of:
S1：开始；S1: Start;
S2：获取主观题答案，并得到电子答案；S2: Get the subjective answer and get an electronic answer;
S3：调用云端评分词库信息；S3: calling the cloud rating vocabulary information;
S4：将电子答案中出现在云端评分词库中的词或词组高亮；S4: highlighting a word or phrase appearing in the cloud rating vocabulary in the electronic answer;
S5：判断是否从当前电子答案取词或词组并加入到云端评分词库，如果是，执行S6；如果否，执行S7；S5: determining whether to take a word or phrase from the current electronic answer and join the cloud rating vocabulary, if yes, execute S6; if not, execute S7;
S6：获取被选中的词或词组，并将其加入云端评分词库，并返回S3；S6: obtaining the selected word or phrase, adding it to the cloud rating vocabulary, and returning to S3;
S7：获取阅卷人录入的分值；S7: obtaining the score entered by the reviewer;
S8：结束。S8: End.
优选为，S4具体为：Preferably, S4 is specifically:
S401：将电子答案中出现在云端评分词库中的词或词组按不同类别高亮显示；S401: Highlight words or phrases appearing in the cloud rating vocabulary in the electronic answer according to different categories;
S402：判断高亮的词或词组是否被选中；如果是，执行S403；如果否，执行S5；S402: determining whether the highlighted word or phrase is selected; if yes, executing S403; if not, executing S5;
S403：从云端评分词库调用该词或词组的相关信息并在屏幕上显示。S403: Calling related information of the word or phrase from the cloud rating vocabulary and displaying it on the screen.
优选为，在步骤S6中，选中答案中的词或词组的方式为，在评分终端的显示屏幕上双击或更多次的点击某个词或词组所在的位置，在该位置自动生成前选定符和后选定符，前、后选定符之间的字符或字将被选中，词或词组具体内容的选择可以通过对前、后选定符的位置调整来确定，同时在屏幕的预定区域或靠近选定符的区域弹出菜单，菜单至少包括向云端上传的命令选项。Preferably, in step S6, the word or phrase in the answer is selected by double-clicking or more clicking on the position of the word or phrase on the display screen of the rating terminal, and automatically selecting before the location is automatically generated. The character and word between the front and back selectors will be selected, and the choice of the word or phrase specific content can be determined by adjusting the position of the front and back selectors, while pre-scheduled on the screen. The area or the area near the selector pops up a menu that includes at least the command options uploaded to the cloud.
优选为，如果是触屏式的操作，使用预定义的手势或单独的选项来触发生成前、后选定符。Preferably, if it is a touch screen operation, a pre-defined gesture or a separate option is used to trigger the generation of the front and back selectors.
本申请还提出一种判卷方法，包括以下步骤：The application also proposes a method of judging, comprising the following steps:
S1′：开始；S1': start;
S2′：获取主观题答案，并得到电子答案； S2': Get the answer to the subjective question and get an electronic answer;
S3′：选中电子答案中的部分文字区域，并调用云端评分词库信息；S3': select a part of the text area in the electronic answer, and call the cloud rating lexicon information;
S4′：将电子答案中出现在云端评分词库中的词或词组高亮，并将所述部分文字区域的高亮词或词组作为词条，显示该词条的相关信息；S4': highlighting a word or a phrase appearing in the cloud scoring vocabulary in the electronic answer, and using the highlighted word or phrase of the partial text area as a term to display related information of the term;
S5′：判断是否将该词条加入到云端评分词库，如果是，执行S6′；如果否，执行S7′；S5': determining whether the term is added to the cloud rating vocabulary, if yes, executing S6'; if not, executing S7';
S6′：获取被选中的词条，并将其加入云端评分词库，并返回S3′；S6': obtaining the selected entry, adding it to the cloud rating vocabulary, and returning to S3';
S7′：获取阅卷人录入的分值；S7': obtaining the score entered by the reviewer;
S8′：结束。S8': End.
优选为，步骤S4′具体为：Preferably, step S4' is specifically:
S4′：将电子答案中出现在云端评分词库中的词或词组按不同类别高亮显示；将在划定的答案文字区域存在有多个高亮的词或词组作为词条；S4': highlighting the words or phrases appearing in the cloud scoring vocabulary in the electronic answer in different categories; there will be multiple highlighted words or phrases in the delimited answer text area as terms;
优选为，在步骤S6′中，选中答案中的词或词组的方式为，在评分终端的显示屏幕上双击或更多次的点击某个词或词组所在的位置，在该位置自动生成前选定符和后选定符，前、后选定符之间的字符或字将被选中，词或词组具体内容的选择可以通过对前、后选定符的位置调整来确定，同时在屏幕的预定区域或靠近选定符的区域弹出菜单，菜单至少包括向云端上传的命令选项。Preferably, in step S6', the word or phrase in the answer is selected by double clicking or more clicking on the position of the word or phrase on the display screen of the rating terminal, and automatically generating the pre-selection at the position. The character and word between the front and back selectors will be selected, and the choice of the word or phrase specific content can be determined by adjusting the position of the front and back selectors, while on the screen A pop-up menu is popped up in the predetermined area or near the selected area, and the menu includes at least command options uploaded to the cloud.
优选为，S4′中，评分终端对词条内包含的关键词的数量及关键词类别数量进行限制，当阅卷人划取的文字区域中关键词出现的过多或类别数量超过预定值，S3′中部分文字区域将被认为是一个无效的文字区域。Preferably, in S4', the rating terminal limits the number of keywords and the number of keyword categories included in the entry, and the number of keywords appearing in the text area drawn by the reviewer exceeds a predetermined value, S3 The middle part of the text area will be considered an invalid text area.
优选为，在步骤S4′中，如果选中的答案文字区域包含的关键词数和类别没有发生变化，仅是答案文字区域的字数总量变化，这时判卷终端会认为词条并没有变化。Preferably, in step S4', if the number of keywords and the category included in the selected answer text area does not change, only the total number of words in the answer text area changes, and the judgment terminal may consider that the entry does not change.
本申请还提出一种判卷***，其具有一个判卷云端和连接在云端的多个判卷终端，判卷云端包括云端评分词库，阅卷人利用判卷终端对主观题进行阅卷，其特征在于：判卷终端使用前述的判卷方法。The application also proposes a judgment system, which has a judgment cloud and a plurality of judgment terminals connected in the cloud, and the judgment cloud includes a cloud rating vocabulary, and the reviewer uses the judgment terminal to perform subjective questions, and its characteristics are characterized. It is that the judgment terminal uses the aforementioned method of judgment.
有益效果：Beneficial effects:
1.本申请的判卷方法是介于现有技术中的人工阅卷和完全计算机判卷之间的判卷方法，有阅卷人的参与使判卷更加客观，而在判卷方法中，使用计算机来辅助判卷，具体为，判卷终端将得到的电子答案与云端评分词库内预 存的判卷标准相对比，如果某词或词组被预存在云端评分词库中，那么这个词或词组将被高亮显示出来。阅卷人非常容易地在电子答案中找到这此得分关键词，极大的提高了工作的效率。1. The method of judging the application is a method of judging between the manual scoring and the complete computer judging in the prior art, and the participation of the reviewer makes the judgment more objective, and in the method of judging, the computer is used. To assist the judgment, specifically, the electronic answer obtained by the judgment terminal and the cloud score vocabulary The stored judgment standard is relatively high. If a word or phrase is pre-existing in the cloud-based vocabulary, the word or phrase will be highlighted. It is very easy for the reviewer to find this score keyword in the electronic answer, which greatly improves the efficiency of the work.
2.关键词高亮还有一个重要的技术效果，如果遇到某些词也是基本正确的，但没有被记录在标准答案中，或者难以用标准答案作为依据给出分数。本发明提出了阅卷人维护标准答案的技术构思，阅卷人容易发现这个词在屏幕上没有被高亮，则可以将这个词选中并加入到云端评分词库中，从而使标准答案的内容更全面，这一点明确的将本申请的技术方案与现有技术区分开。2. Keyword highlighting also has an important technical effect. If certain words are encountered, they are basically correct, but they are not recorded in the standard answer, or it is difficult to give a score based on the standard answer. The invention proposes a technical idea for the reviewer to maintain the standard answer. If the reviewer easily finds that the word is not highlighted on the screen, the word can be selected and added to the cloud rating vocabulary, thereby making the content of the standard answer more comprehensive. This clearly distinguishes the technical solution of the present application from the prior art.
3.在主观题的判断过程中，往往不仅需要对单个词或词组进行评价，更多的时候是将多个关键词作为一个整体来考虑其答案的有效性和正确性。因此，词条的提出就解决了这个问题。词条的使用极大的提高了评分标准的一致性。首先，阅卷人首先划定一个有效的文字区域，也就是说，多个不同类别的关键词集中的出现在一个有效的文字区域，这避免了考生希望分散的罗列关键词期望得分的情况；其次，对于有争议的、疑难的答案可以及时的提出，并得到快速的分析和解决，而这些有争议的、疑难的答案在定立答案之初一般无法预测到。3. In the judgment process of subjective questions, it is often necessary to evaluate not only individual words or phrases, but also more than one keyword to consider the validity and correctness of the answers. Therefore, the introduction of the term solves this problem. The use of terms greatly improves the consistency of the scoring standards. First, the reviewer first delineates a valid text area, that is, multiple key categories of keywords appear in a valid text area, which avoids the candidate's desire to disperse the expected keyword scores; For controversial and difficult answers, they can be promptly presented and quickly analyzed and resolved. These controversial and difficult answers are generally unpredictable at the beginning of the answer.
4.本发明进一步提出，评分终端会对词条内包含的关键词的数量及类别进行限制，当阅卷人划取的文字区域过大，或关键词出现的过多将被认为是一个无效的文字区域，这是因为如果文字区域过大或者包含的关键词过多，无法与考生罗列关键词的行为区分开，而且对于一个常规的表达形式来说，一个具有独立表义功能的语句不会包含过多的关键词。所以，判卷终端进一步判断选中的答案文字区域包含的字数是否大于预先设置的第一阈值，包含的关键词数是否大于预先设置的第二阈值，如果均不超过，才能进行词条的创建或读取。当阅卷人移动光标过程中，如果答案文字区域包含的关键词数和类别没有发生变化，仅是答案文字区域的字数总量变化，这时判卷终端会认为词条并没有变化。这样会大大减少词条的总量，防止词条创建过多，反而影响阅卷人的判断。当阅卷人移动光标过程中，关键词的数量或种类发生的变化，判卷终端所认别的词条也会发生变化，使得用户在移动光标过程中时，可以快速看到不同的词条的解释，快速了解对于不同词条的评分标准，实质上也是快速了解不同的关键词组合后的评分标准。 4. The present invention further proposes that the rating terminal limits the number and category of keywords included in the entry. When the text area drawn by the reviewer is too large, or the keyword appears too much, it will be considered as invalid. Text area, because if the text area is too large or contains too many keywords, it cannot be distinguished from the candidate's behavior of listing keywords, and for a regular expression, a statement with independent expression will not Contains too many keywords. Therefore, the judging terminal further determines whether the number of words included in the selected answer text area is greater than a preset first threshold, and whether the number of included keywords is greater than a preset second threshold, and if not, the entry can be performed or Read. When the reviewer moves the cursor, if the number of keywords and the category contained in the answer text area does not change, only the total number of words in the answer text area changes, and the judgment terminal will consider that the entry has not changed. This will greatly reduce the total number of entries and prevent the creation of too many entries, which will affect the judgment of the reviewers. When the reviewer moves the cursor, the number or type of keywords changes, and the term recognized by the judgment terminal also changes, so that the user can quickly see different terms while moving the cursor. Explain that quickly understanding the scoring criteria for different terms is essentially a quick understanding of the scoring criteria for different keyword combinations.
附图说明DRAWINGS
为了更清楚地说明本发明实施例中的技术方案，下面将对本发明实施例描述中所需要使用的附图作简单的介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据本发明实施例的内容和这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other drawings may be obtained according to the contents of the embodiments of the present invention and the drawings without any creative work.
图1为本发明的第一种判卷方法的流程图；1 is a flow chart of a first method of determining a volume according to the present invention;
图2为本发明的电子答案高亮显示方法的流程图；和2 is a flow chart of an electronic answer highlighting method of the present invention; and
图3为本发明的第二种判卷方法的流程图。3 is a flow chart of a second method of determining the invention.
具体实施方式Detailed ways
下面将结合附图对本发明的具体实施方式进行详细说明。应当理解，此处所描述的实施例仅仅是用于解释本发明，并不是用于限制本发明。有关领域的普通技术人员在不背离本发明精神的情况下所做的各种变化和变形，都在本发明所附的独立权利要求和从属权利要求的范围内。Specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It is understood that the embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. Various changes and modifications may be made by those skilled in the art without departing from the spirit and scope of the invention.
根据本发明的实施例，判卷***具有一个判卷云端和连接在云端的多个终端，阅卷人利用终端对主观题进行阅卷。According to an embodiment of the present invention, the judgment system has a judgment cloud and a plurality of terminals connected to the cloud, and the reviewer uses the terminal to perform a review of the subjective question.
实施例1：Example 1:
如图1所示，一种判卷方法，包括以下步骤：As shown in FIG. 1, a method for determining a volume includes the following steps:
S1：开始；S1: Start;
S2：获取主观题答案，并得到电子答案。在对某考题的用户答案进行评分时，获取该考题对应的用户答案。例如，当考题的用户答案为电子形式时，直接读取该考题的用户答案。当该考题的用户答案为纸件形式时，可以采用文字识别技术对纸件上的用户答案中的文字进行识别，得到电子形式的用户答案。获取主观题答案的方法并不局限于上述两种方法，只需要能够得到电子答案即可。S2: Get the subjective answer and get an electronic answer. When scoring the user answer of an exam question, the user answer corresponding to the exam question is obtained. For example, when the user answer of the question is in electronic form, the user answer of the question is directly read. When the user answer of the question is in the form of paper, the text recognition technology can be used to identify the text in the user's answer on the paper, and the user's answer in electronic form can be obtained. The method of obtaining the answer to the subjective question is not limited to the above two methods, and only needs to be able to obtain an electronic answer.
S3：调用云端评分词库信息。云端评分词库设在判卷***的云端，在云端评分词库中预设了主观题的评分标准，与传统的人工阅卷相同，对于主观题的评分标准主要是看答案中是否出现了关键的词或词组，也就是通常所谓的得分点，根据主观题对得分点的覆盖情况及其他考量因素给出分数。在云端评分词库也预设了主观题的答案的关键词，并与关键词一起，还收录了对 关键词的解释、关键词的同义词等可能出现的相关表达。S3: Call the cloud rating lexicon information. The cloud scoring vocabulary is set in the cloud of the judgment system. In the cloud scoring vocabulary, the scoring standard of subjective questions is preset, which is the same as the traditional manual scoring. The scoring standard for subjective questions is mainly to see if there is a key in the answer. Words or phrases, which are commonly referred to as score points, are scored based on subjective questions on the coverage of score points and other considerations. In the cloud scoring thesaurus, the keywords of the answers to the subjective questions are also preset, and together with the keywords, the pairs are also included. The interpretation of the keyword, the synonym of the keyword, etc. may appear related expression.
S4：将电子答案中出现在云端评分词库中的词或词组高亮。判卷终端将得到的电子答案与云端评分词库内预存的评分标准相对比，如果某词或词组属于云端评分词库中预存的关键词或其相关表达，那么这个词或词组将被高亮显示出来。阅卷人非常容易地在电子答案中找到这些得分关键词，极大的提高了工作的效率。S4: Highlight the words or phrases that appear in the cloud rating vocabulary in the electronic answer. The electronic answer obtained by the judgment terminal is compared with the pre-stored scoring standard in the cloud scoring vocabulary. If a word or phrase belongs to a pre-stored keyword or its related expression in the cloud scoring vocabulary, the word or phrase will be highlighted. show. It is very easy for the reviewer to find these scoring keywords in the electronic answer, which greatly improves the efficiency of the work.
S5：判断是否需要从当前电子答案中获取相关词或词组加入到云端评分词库，如果是，执行S6；如果否，执行S7。在主观题的阅卷过程中，经常会遇到某些答案也是基本正确的，但没有被记录在标准答案中，或者难以用标准答案作为依据给出分数。一般来说，阅卷人将根据自己的经验直接给出分数，那么这个评分就带有十分强的主观判断，而且可能其他阅卷人也会遇到同样的问题，大家的尺度难以统一。本发明提出了阅卷人维护标准答案的技术构思，如果阅卷人在步骤S4中显示的电子答案中，发现了其中某个词也可以作为得分项，但没有被记录在云端评分词库中，其最直观的表现就是这个词在屏幕上没有被高亮，则可以将这个词或词组选中并加入到云端评分词库中，从而使标准答案的内容更全面。S5: Determine whether it is necessary to obtain related words or phrases from the current electronic answer to join the cloud rating vocabulary, if yes, execute S6; if not, execute S7. In the scoring process of subjective questions, it is often true that some answers are basically correct, but they are not recorded in the standard answer, or it is difficult to give a score based on the standard answer. Generally speaking, the reviewer will directly give the score according to his own experience, then the score has a very strong subjective judgment, and other reviewers may encounter the same problem, and the scale of everyone is difficult to unify. The present invention proposes a technical idea for a reviewer to maintain a standard answer. If the reviewer finds in the electronic answer displayed in step S4, one of the words can also be used as a score item, but is not recorded in the cloud scoring vocabulary. The most intuitive performance is that the word is not highlighted on the screen, you can select the word or phrase and add it to the cloud rating vocabulary, so that the standard answer content is more comprehensive.
S6：获取被选中的词或词组，并将其加入云端评分词库，并返回S3。当云端评分词库增加了某个词或词组后，终端会重新的将电子答案与云端评分词库进行对比，经过对比，这个词也将会被高亮。重要的是，如果其他的电子答案中如果也出现这个词时，这个词也将在其他的终端上被高亮。S6: Get the selected word or phrase, add it to the cloud rating vocabulary, and return to S3. When the cloud rating vocabulary adds a word or phrase, the terminal will re-compare the electronic answer with the cloud rating vocabulary. After comparison, the word will also be highlighted. The important thing is that if the word appears in other electronic answers, the word will also be highlighted on the other terminal.
S7：获取阅卷人录入的分值。阅卷人根据电子答案中出现关键词的全面呈度及其他评分因素给出最终的得分并录入到终端。S7: Get the score entered by the reviewer. The reviewer gives the final score based on the full presentation of the keywords appearing in the electronic answer and other scoring factors and enters the terminal.
S8：结束。S8: End.
作为一种优选，步骤S4具体为：As a preferred step, step S4 is specifically:
S401：将电子答案中出现在云端评分词库中的词或词组按不同类别高亮；这将更有利于阅卷人直观的了解电子答案中的得分情况。比如按不同的颜色来区别不同类别的关键词。S401: Highlight the words or phrases appearing in the cloud scoring vocabulary in the electronic answer according to different categories; this will be more conducive to the scoring person to intuitively understand the scoring situation in the electronic answer. For example, different keywords can be distinguished by different colors.
S402：判断高亮的词或词组是否被选中；如果是，执行S403；如果否，执行S5；S402: determining whether the highlighted word or phrase is selected; if yes, executing S403; if not, executing S5;
S403：从云端评分词库调用该词或词组的相关信息并在屏幕上显示，然 后执行S5。S403: Calling related information of the word or phrase from the cloud rating vocabulary and displaying it on the screen, After executing S5.
步骤S402及S403主要是方便阅卷人更好的了解关键词相关的信息，或者相关的评分标准。在S402中。在步骤S6或S402中，选中答案的方式可以为多种，比如，在评分终端的显示屏幕上双击或更多次的点击某个词或词组所在的位置，在该位置自动生成前选定符和后选定符，前、后选定符之间的字符或字将被选中，词或词组具体内容的选择可以通过对前、后选定符的位置调整来确定，同时在屏幕的预定区域或靠近选定符的区域弹出菜单，菜单至少包括向云端上传的命令选项。在菜单项被触发之后，还至少包括对词或词组添加解释内容或标注，用来说明为什么要将该词加入云端评分词库。选中词或词组的方法也可以仅是光标停留在屏幕上的词或词组所在的位置。Steps S402 and S403 are mainly for facilitating the reader to better understand the information related to the keyword, or the relevant scoring standard. In S402. In step S6 or S402, the answer may be selected in multiple ways, for example, by double clicking or more clicking on a display screen of the rating terminal, where the word or phrase is located, and the pre-selector is automatically generated at the position. And the post selector, the character or word between the front and back selectors will be selected, and the choice of the word or phrase specific content can be determined by adjusting the position of the front and rear selectors, while at the predetermined area of the screen. Or a pop-up menu near the area of the selector, the menu includes at least the command options uploaded to the cloud. After the menu item is triggered, it also includes at least adding an explanation or annotation to the word or phrase to explain why the word is added to the cloud rating vocabulary. The method of selecting a word or phrase can also be just the location of the word or phrase on which the cursor stays on the screen.
如果是触屏式的操作，可以使用预定义的手势或单独的选项来触发生成前、后选定符。例如是多次点击屏幕某一区域、长按屏幕，这些操作方式是本领域的常规手段，不再列举。In the case of touch-screen operations, pre- and post-selection can be triggered using predefined gestures or separate options. For example, multiple clicks on a certain area of the screen, long press on the screen, these modes of operation are conventional means in the art, and will not be enumerated.
实施例2：Example 2:
本实施例中，与实施例1的区别在于：In this embodiment, the difference from Embodiment 1 is:
一种判卷方法，包括以下步骤：A method of determining a volume includes the following steps:
S1′：开始；S1': start;
S2′：获取主观题答案，并得到电子答案；S2': Get the answer to the subjective question and get an electronic answer;
S3′：选中电子答案中的部分文字区域，并调用云端评分词库信息；S3': select a part of the text area in the electronic answer, and call the cloud rating lexicon information;
S4′：将电子答案中出现在云端评分词库中的词或词组高亮，并将所述部分文字区域的高亮词或词组作为词条，显示该词条的相关信息；S4': highlighting a word or a phrase appearing in the cloud scoring vocabulary in the electronic answer, and using the highlighted word or phrase of the partial text area as a term to display related information of the term;
S5′：判断是否将该词条加入到云端评分词库，如果是，执行S6′；如果否，执行S7′；S5': determining whether the term is added to the cloud rating vocabulary, if yes, executing S6'; if not, executing S7';
S6′：获取被选中的词条，并将其加入云端评分词库，并返回S3′；S6': obtaining the selected entry, adding it to the cloud rating vocabulary, and returning to S3';
S7′：获取阅卷人录入的分值；S7': obtaining the score entered by the reviewer;
S8′：结束。S8': End.
这里提出词条的概念，在主观题的判断过程中，往往不仅需要对单个词或词组进行评价，更多的时候是将多个关键词作为一个整体来考虑其答案的有效性和正确性。往往多个关键词是正确的或相关的，但最终的答案却是不准确的或不相关的。通常有一种情况是，答题人会提出超出必要限度篇幅的 答案，以期望阅卷人能够在这些答案里找到得分点，并给出较高的分数。一方面，这种将对答案的提取工作交给阅卷人是不恰当的，另一面，可能许多关键词分散在答案中，其并没有组合起来的特定含义，仅仅是因为篇幅过长而被包含在答案中。提出词条的概念还有一个原因是，标准答案无法穷举出所有的不同种类的关键词组合在一起的评分标准，有些组合是正确的，被列在标准答案中，有些则是不正确的，但还有一些是不够准确的答案，这些分数应在标准答案的基础上酌减。因此，词条的出现就解决了这个问题。The concept of the term is proposed here. In the process of judging subjective questions, it is often necessary to evaluate not only individual words or phrases, but also more than one keyword to consider the validity and correctness of the answers. Often multiple keywords are correct or relevant, but the final answer is inaccurate or irrelevant. There is usually a situation where the respondent will ask for more than the necessary limit. The answer is that the prospective reviewer can find score points in these answers and give higher scores. On the one hand, this will not be appropriate for the extraction of the answer to the reviewer. On the other hand, many keywords may be scattered in the answer, and the specific meanings that are not combined are simply included because the length is too long. In the answer. Another reason for the concept of the entry is that the standard answer cannot exaggerate the scoring criteria of all the different types of keywords. Some combinations are correct, listed in the standard answer, and some are incorrect. , but there are some that are not accurate enough. These scores should be reduced based on the standard answer. Therefore, the emergence of the term solves this problem.
在步骤S4′中，判卷终端首先检测阅卷人选中的答案文字区域，并根据关键词的高亮情况来判断这个区域里出现了多少个、多少种不同的关键词。阅卷人这时可以根据答案的实际情况来划分选中的区域，比如一个完整的表达段落，比如一个针对题目中具体问题的答复，这些具有比较完整表义形式的答案区域将被阅卷人主观的划分出来。然后答案文字区域的关键词信息将被提取出来，比如包括第一关键词和第二关键词，这两个关键词将作为第一词条被识别。从云端评分词库调用该词条相对应的信息并在屏幕上显示，然后执行S5′。即如果在云端评分词库收录了该第一词条，那么关于第一词条具体评价将被显示。比如评分人将对这种特定关键词组合给出具体的评价，包括但不限于词条的正确性、准确性、相关性、建议的评分等等一系列的信息，这些信息与该第一词条形成映射关系。In step S4', the judgment terminal first detects the answer text area selected by the reviewer, and judges how many different types of keywords appear in the area according to the highlight of the keyword. At this time, the reviewer can divide the selected area according to the actual situation of the answer, such as a complete expression paragraph, such as a response to a specific question in the title. These answer areas with a more complete form of meaning will be subjectively divided by the reviewer. come out. The keyword information of the answer text area will then be extracted, for example including the first keyword and the second keyword, which will be identified as the first term. The information corresponding to the term is called from the cloud rating vocabulary and displayed on the screen, and then S5' is executed. That is, if the first term is included in the cloud rating vocabulary, then a specific evaluation about the first term will be displayed. For example, the scorer will give a specific evaluation of this particular keyword combination, including but not limited to a series of information such as the correctness, accuracy, relevance, and recommended score of the entry, and the first word The bars form a mapping relationship.
传统的阅卷方式中，一般认为只要考生将关键词写在答案中就会给分，但正如前面分析所述，关键词在答案中没有逻辑的罗列并不意味着可以得分，关键词有逻辑的排列才能体现答案的正确性。与现有技术相比，词条的使用极大的提高了评分标准的一致性。首先，阅卷人首先划定一个有效的文字区域，也就是说，多个不同类别的关键词集中的出现在一个有效的文字区域，这避免了考生希望分散的罗列关键词期望得分的情况；其次，对于有争议的、疑难的答案可以及时的提出，并得到快速的分析和解决，而这些有争议的、疑难的答案在定立答案之初一般无法预测到。In the traditional method of scoring, it is generally considered that as long as the candidate writes the keyword in the answer, it will give points, but as the previous analysis, the fact that the keyword has no logical list in the answer does not mean that the score can be scored. The keyword has logic. Arrangement can reflect the correctness of the answer. Compared to the prior art, the use of terms greatly improves the consistency of the scoring standards. First, the reviewer first delineates a valid text area, that is, multiple key categories of keywords appear in a valid text area, which avoids the candidate's desire to disperse the expected keyword scores; For controversial and difficult answers, they can be promptly presented and quickly analyzed and resolved. These controversial and difficult answers are generally unpredictable at the beginning of the answer.
但如果云端评分词库没有收录该第一词条，那么屏幕上并不会显示相关的信息。这时，就需要执行S5′，S5′：判断是否将当前的词条加入到云端评分词库。此时第一词条已经被标引，但如果不被阅卷人发送云端的话将不被云端评分词库所记录，阅卷人按自己的主观判断来决定该第一词条是否应 被加入到标准答案中，其是否应成为该次阅卷普适的标准答案。However, if the first term is not included in the cloud rating vocabulary, the relevant information will not be displayed on the screen. At this time, it is necessary to execute S5', S5': determine whether to add the current entry to the cloud rating vocabulary. At this point, the first entry has been indexed, but if it is not sent by the reviewer, it will not be recorded by the cloud scoring vocabulary. The reviewer decides whether the first entry should be based on his own subjective judgment. Was added to the standard answer, whether it should be the standard answer for this review.
优选的是，评分终端会对词条内包含的关键词的数量及类别进行限制，当阅卷人划取的文字区域过大，或关键词出现的过多将被认为是一个无效的文字区域，这是因为如果文字区域过大或者包含的关键词过多，无法与考生罗列关键词的行为区分开，而且对于一个常规的表达形式来说，一个具有独立表义功能的语句不会包含过多的关键词。所以，判卷终端进一步判断选中的答案文字区域包含的字数是否大于预先设置的第一阈值，包含的关键词数是否大于预先设置的第二阈值，如果均不超过，才能进行词条的创建或读取。Preferably, the rating terminal limits the number and category of keywords included in the entry. When the text area drawn by the reviewer is too large, or the keyword appears too much, it is considered to be an invalid text area. This is because if the text area is too large or contains too many keywords, it cannot be distinguished from the candidate's behavior of listing keywords, and for a regular expression, a statement with independent expression will not contain too much. Keywords. Therefore, the judging terminal further determines whether the number of words included in the selected answer text area is greater than a preset first threshold, and whether the number of included keywords is greater than a preset second threshold, and if not, the entry can be performed or Read.
在阅卷人移动光标过程中，如果答案文字区域包含的关键词数和类别没有发生变化，仅是答案文字区域的字数总量变化，这时判卷终端会认为词条并没有变化。这样会大大减少词条的总量，防止词条创建过多，反而影响阅卷人的判断。在阅卷人移动光标过程中，关键词的数量或种类发生的变化，判卷终端所识别的词条也会发生变化，使得用户在移动光标过程中时，可以快速看到不同的词条的解释，快速了解对于不同词条的评分标准，实质上也是快速了解不同的关键词组合后的评分标准。In the process of moving the cursor by the reviewer, if the number of keywords and the category contained in the answer text area does not change, only the total number of words in the answer text area changes, and the judgment terminal will consider that the entry has not changed. This will greatly reduce the total number of entries and prevent the creation of too many entries, which will affect the judgment of the reviewers. During the process of moving the cursor by the reviewer, the number or type of keywords changes, and the term recognized by the judgment terminal also changes, so that the user can quickly see the interpretation of different terms while moving the cursor. To quickly understand the scoring criteria for different terms, it is also a quick understanding of the different scoring criteria for different keyword combinations.
在一个较佳的实施例中，阅卷人上传到云端评分词库的词或词组或词条在经过高级用户的确认后才被收录到云端评分词库中。In a preferred embodiment, the words or phrases or terms uploaded by the reviewer to the cloud scoring vocabulary are included in the cloud scoring vocabulary after being confirmed by the advanced user.
下面通过一个例题来更好的理解本发明的内容。The contents of the present invention will be better understood by way of an example.
例题：小弗郎士为什么感到韩麦尔先生“从来没有这么高大”？Example: Why does Little Francis feel that Mr. Hanmer has never been so tall?
标准答案：小弗郎士从韩麦尔先生惨白的脸色感到他对祖国的热爱和失去祖国的悲愤、痛苦的心情。韩麦尔先生的爱国精神，使小弗郎士觉得他不仅是一位法语老师，而且是一位爱国志士，所以觉得他的形象高大。Standard answer: Little Francis felt his love for the motherland and the grief and pain of losing the motherland from the pale face of Mr. Hanmel. Mr. Han Maier’s patriotism made him feel that he was not only a French teacher but also a patriot, so he felt that his image was tall.
评分标准：考生只需回答出韩麦尔先生对祖国的热爱、对失去祖国的悲愤及小弗郎士对韩麦尔先生敬仰即可得满分。Scoring Criteria: Candidates only need to answer Mr. Han Maier’s love for the motherland, his grief for losing his motherland, and the small Francis’s respect for Mr. Hanmel.
基于该评分标准，在云端评分词库首先预设标准答案：第一类关键词：热爱；第二类关键词：悲伤、痛苦；第三类关键词：祖国、国家；第四类关键词：高大、伟大、敬仰；第五类关键词：小弗郎士；第六类关键词：韩麦尔。Based on the scoring standard, the standard answer in the cloud rating vocabulary is first preset: the first type of keywords: love; the second type of keywords: sadness, pain; the third type of keywords: the motherland, the country; the fourth category of keywords: Tall, great, admired; fifth type of keyword: Little Francis; sixth type of keyword: Han Maier.
在判卷终端显示考生的电子答案。The candidate's electronic answer is displayed at the judgment terminal.
答案一：小弗郎士感受到韩麦尔先生失去祖国的悲愤、难过的心情。小 弗郎士觉得他很高大。Answer 1: Little Francis felt the sadness and sadness of Mr. Hanmel’s loss of his motherland. small Francis thinks he is very tall.
答案一相对于标准答案来说有较大区别，但得分点抓得很好，写上很多得分点关键词。首先，在判卷终端的屏幕上，“祖国”、“悲愤”、“小弗郎士”、“韩麦尔”和“高大”这几个词将被高亮，阅卷人可迅速发现考生没有把第一类关键词的“热爱”这个得分点写入答案，因此将酌情减分。同时，阅卷人发现“难过”这个词也可以表达出第二类关键词的悲伤、痛苦的含义，这时，阅卷人用光标将这个词选中，发送到云端评分词库中。此时，第二类关键词包括了悲伤、痛苦和难过。当第二个阅卷人再次遇到答案一时，在其终端屏幕上“难过”这个词将被高亮。这样就丰富了云端评分词库答案内容，便于统一评分标准。The answer one is quite different from the standard answer, but the score points are very good, and many score points are written. First of all, on the screen of the judgment terminal, the words "the motherland", "sorrowful anger", "small franchise", "Han Maier" and "high" will be highlighted, and the reviewers can quickly find that the candidates are not Write the score point of the "love" of the first type of keyword into the answer, so the score will be reduced as appropriate. At the same time, the reviewer found that the word "sad" can also express the sadness and painful meaning of the second type of keyword. At this time, the reviewer selects the word with the cursor and sends it to the cloud scoring vocabulary. At this time, the second type of keywords includes sadness, pain, and sadness. When the second reviewer encounters the answer again, the word "sad" on its terminal screen will be highlighted. This enriches the answer content of the cloud rating lexicon, which is convenient for unified scoring standards.
答案二：小弗郎士从韩麦尔先生惨白的脸色感到他很悲痛。这体现了小弗郎士对祖国的热爱，他认为韩麦尔先生的形象高大。Answer 2: Little Francis feels sad from Mr. Hanmel's pale face. This reflects Xiao Erlang's love for the motherland. He believes that Mr. Han Maier's image is tall.
这个答案中，虽然也出现了大部分的关键词，但从题目所给的材料中无法直接得出“小弗郎士对祖国的热爱”这个信息，对祖国热爱的主语应该是韩麦尔，因此，尽管答案中出现了“小弗郎士”、“祖国”和“热爱”这几个关键词，但答案仍然是不正确的。可见仅用关键词来判断还不够客观，这种情况就要使用词条的判断。In this answer, although most of the keywords have appeared, the information given by the title cannot directly lead to the message “Little Fransch’s love for the motherland”. The subject of love for the motherland should be Hanmer. Therefore, although the keywords "Little Francis", "Homeland" and "Love" appear in the answer, the answer is still incorrect. It can be seen that it is not objective enough to judge only by keywords. In this case, the judgment of the terms is used.
首先，只有当几个关键词同时在答案的某一选中的区域内同时出现时才将作为一个词条，判卷终端将选中区域的“小弗郎士”、“祖国”、“热爱”三个关键词作为一个词条，如果同时这个词条也曾被收录到云端评分词库时，判卷终端才会显示词条辅助信息，在答案二中，由于“小弗郎士”与“祖国”和“热爱”搭配的这个词条是错误的答案，并没有记载在云端评分词库，在判卷终端，这个答案也不会显任何与该词条相关的信息。因此，词条的使用有助于对试卷的客观判断。但“韩麦尔”、“祖国”、“热爱”三个关键词作为一个词条是正确的答案，如果阅卷人选定的文字的区域有这个词条，判卷终端将从云端评分词库调取该词条的相关辅助信息，如果这个词条没有被创建，阅卷人可以主动创建这个词条并发送到云端评分词库，从而加入对这个词条的评分建议或是其他的评分标准。First of all, only when several keywords appear simultaneously in a selected area of the answer, it will be used as a term, and the judgment terminal will select the area of "Little Francis", "Homeland", "Love" The keyword is used as a term. If the term is also included in the cloud scoring vocabulary, the judgment terminal will display the auxiliary information. In the second answer, due to the "Little Francis" and the "Homeland" The term "matching" with "love" is the wrong answer. It is not recorded in the cloud-based vocabulary. At the judgment terminal, this answer does not show any information related to the entry. Therefore, the use of the term helps to objectively judge the test paper. However, the three keywords “Han Maier”, “Motherland” and “Love” are the correct answers. If the area selected by the reviewer has this entry, the judgment terminal will be scored from the cloud. The auxiliary information of the entry is retrieved. If the entry is not created, the reviewer can actively create the entry and send it to the cloud rating vocabulary to add a rating suggestion for the entry or other scoring criteria.
在对答案二评分时，当阅卷人移动光标过程中，如果答案文字区域包含的“小弗郎士”、“祖国”、“热爱”三个关键的词数和类别没有发生变化，仅 是选中的文字区域的字数总量变化，这时判卷终端会认为词条并没有变化。When scoring the answer two, when the reviewer moves the cursor, if the answer text area contains three key words and categories of "Little Francis", "Homeland", "Love", there is no change, only It is the total number of words in the selected text area. At this time, the judgment terminal will think that the entry has not changed.
比如“这体现了小弗郎士对祖国的热爱，他认为韩麦尔先生的形”与“这体现了小弗郎士对祖国的热爱，他认为韩麦尔先生的”这两种取词的方式中包括的关键词都没有变化，所以答案文字区域的词条就没有变化，这对判卷***是一个重大的改进。将词条的判断设计为与选中区域的文字总量无关，这样会大大减少词条的总量，防止词条创建过多，反而影响阅卷人的判断。正如前文所述，评分终端应对词条内包含的关键词的数量及类别进行限制，当阅卷人划取的文字区关键词出现的过多或关键词的类别太多将被认为是一个无效的文字区域，这是因为如果文字区域包含的关键词过多，无法将其与考生罗列关键词的行为区分开，比如阅卷人将答案二中的“小弗郎士”、“韩麦尔”、“祖国”、“热爱”四个关键词都选中，这种词条如果被创建可能会不利于评分的进行，因为四个词的组合的可能性太多，同时包含了大量的正确的和错误的组合。For example, "This reflects the love of the young Francis to the motherland, he believes that the shape of Mr. Hanmel" and "this reflects the love of the young Francis to the motherland, he believes that Mr. Hanmel's two words" The keywords included in the way have not changed, so the entry in the answer text area has not changed, which is a major improvement on the judgment system. The judgment of the term is designed to be independent of the total amount of text in the selected area. This will greatly reduce the total amount of the entry and prevent the creation of too many entries, which will affect the judgment of the reviewer. As mentioned above, the scoring terminal should limit the number and type of keywords contained in the entry. When the text area keyword drawn by the reviewer appears too much or the category of the keyword is too large, it will be considered as invalid. The text area, because if the text area contains too many keywords, it can't be distinguished from the behavior of the candidate's keyword listing. For example, the reviewer will use the "Little Francis" and "Han Maier" in the second answer. The four keywords of "Motherland" and "Love" are selected. If this entry is created, it may not be conducive to the progress of the score, because the combination of the four words is too likely, and contains a large number of correct and wrong. The combination.
对于上述案例的分析可知，本申请的判卷方法首先能够通过关键词高亮显示来提高阅卷人的工作效率，进一步，其可以根据关键词组成的词条的情况来提供相应词条的相关信息，不论是关键词还是词条，阅卷人都能够在评卷过程中对云端的数据进行修改和丰富，从而使整个的评分过程越来越统一，越来越科学。As can be seen from the analysis of the above cases, the method of judging the application can first improve the work efficiency of the reviewer by keyword highlighting, and further, it can provide related information of the corresponding entry according to the terms of the keyword. Whether it is a keyword or an entry, the reviewer can modify and enrich the data in the cloud during the marking process, so that the entire scoring process becomes more and more unified and more scientific.
下面作为可选的实施例来给出实现本发明的硬件，作为判卷终端，其可以为计算机设备，比如以通用计算设备的形式表现。计算机设备的组件包括但不限于一个或者多个处理器，***存储器，连接不同***组件(包括***存储器和处理器)的总线。The hardware embodying the present invention is presented below as an alternative embodiment as a decision terminal, which may be a computer device, such as in the form of a general purpose computing device. Components of a computer device include, but are not limited to, one or more processors, system memory, buses that connect different system components, including system memory and processors.
上述的总线在本领域中一般表示几类总线结构中的一种或多种，包括存储器总线或者存储器控制器，***总线，图形加速端口，处理器或者使用多种总线结构中的任意总线结构的局域总线。这些体系结构包括但不限于工业标准体系结构(ISA)总线，微通道体系结构(MAC)总线，增强型ISA总线、视频电子标准协会(VESA)局域总线以及***组件互连(PCI)总线。The above-described bus generally represents one or more of several types of bus structures in the art, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or any bus structure using a plurality of bus structures. Local bus. These architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA Bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.
计算机设备也可以与一个或多个外部设备(例如键盘、指向设备、显示器等)通信，还可与一个或者多个使得用户能与该计算机设备交互的设备通信，和/或与使得该计算机设备能与一个或多个其它计算设备进行通信的任何 设备(例如网卡，调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口进行。并且，计算机设备还可以通过网络适配器与一个或者多个网络(例如局域网(LAN)，广域网(WAN)和/或公共网络，例如因特网)通信。网络适配器通过总线与计算机设备的其它模块通信。The computer device can also be in communication with one or more external devices (eg, a keyboard, pointing device, display, etc.), and can also communicate with one or more devices that enable a user to interact with the computer device, and/or with the computer device Anything that can communicate with one or more other computing devices Devices (such as network cards, modems, etc.) communicate. This communication can be done via an input/output (I/O) interface. Also, the computer device can communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through a network adapter. The network adapter communicates with other modules of the computer device over the bus.
计算机设备典型地包括多种计算机***可读介质。这些介质可以是任何能够被计算机设备访问的可用介质，包括易失性和非易失性介质，可移动的和不可移动的介质。Computer devices typically include a variety of computer system readable media. These media can be any available media that can be accessed by a computer device, including both volatile and nonvolatile media, removable and non-removable media.
***存储器可以包括易失性存储器形式的计算机***可读介质，例如随机存取存储器(RAM)和/或高速缓存存储器。计算机设备可以进一步包括其它可移动/不可移动的、易失性/非易失性计算机***存储介质。***存储器可以包括至少一个程序产品，该程序产品具有一组(例如至少一个)程序模块，这些程序模块被配置以执行本发明上述判卷的功能。还具有一组(至少一个)程序模块的程序/实用工具，可以存储在例如***存储器中，这样的程序模块包括操作***、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中包括网络环境的实现。The system memory can include computer system readable media in the form of volatile memory, such as random access memory (RAM) and/or cache memory. The computer device may further include other removable/non-removable, volatile/non-volatile computer system storage media. The system memory can include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the above-described assertions of the present invention. A program/utility having a set of (at least one) program modules, which may be stored, for example, in system memory, such program modules including an operating system, one or more applications, other program modules, and program data, in these examples Implementations of the network environment are included in each or some combination.
可以以一种或多种程序设计语言或其组合来编写用于执行本发明操作的计算机程序代码，所述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如”C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including an object oriented programming language such as Java, Smalltalk, C++, and conventional A procedural programming language - such as the "C" language or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer, partly on the remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (eg, using an Internet service provider) Internet connection).
对于判卷云端，其主要的功能为数据存储功能，现有技术已经有很多可以使用，在这里不再赘述。For the judgment cloud, its main function is the data storage function. There are many existing technologies that can be used, and will not be described here.
以上介绍了本发明的较佳实施方式，旨在使得本发明的精神更加清楚和便于理解，并不是为了限制本发明，凡在本发明的精神和原则之内，所做的更新、替换、改进，均应包含在本发明所附的权利要求概况的保护范围之内。 The preferred embodiments of the present invention have been described above, and are intended to provide a further understanding of the embodiments of the present invention. It is intended to be included within the scope of the appended claims.
工业实用性Industrial applicability
本发明允许阅卷人维护标准答案，阅卷人容易发现这个词在屏幕上没有高亮显示，则可以将这个词选中并加入到云端评分词库中，供其他阅卷人参考，从而使标准答案得以持续更新。 The invention allows the reviewer to maintain the standard answer. If the reviewer easily finds that the word is not highlighted on the screen, the word can be selected and added to the cloud scoring vocabulary for reference by other reviewers, so that the standard answer can be continued. Update.

Claims (12)

  1. 一种判卷方法，其特征在于，包括以下步骤：A method for determining a volume, comprising the steps of:
    S1：开始；S1: Start;
    S2：获取主观题答案，并得到电子答案；S2: Get the subjective answer and get an electronic answer;
    S3：调用云端评分词库信息；S3: calling the cloud rating vocabulary information;
    S4：将电子答案中出现在云端评分词库中的词或词组高亮；S4: highlighting a word or phrase appearing in the cloud rating vocabulary in the electronic answer;
    S5：判断是否从当前电子答案取词或词组并加入到云端评分词库，如果是，执行S6；如果否，执行S7；S5: determining whether to take a word or phrase from the current electronic answer and join the cloud rating vocabulary, if yes, execute S6; if not, execute S7;
    S6：获取被选中的词或词组，并将其加入云端评分词库，并返回S3；S6: obtaining the selected word or phrase, adding it to the cloud rating vocabulary, and returning to S3;
    S7：获取阅卷人录入的分值；S7: obtaining the score entered by the reviewer;
    S8：结束。S8: End.
  2. 根据权利要求1所述的判卷方法，其特征在于，The method of claiming according to claim 1, wherein
    优选的，步骤S4进一步包括：Preferably, step S4 further comprises:
    S401：将电子答案中出现在云端评分词库中的词或词组按不同类别高亮显示；S401: Highlight words or phrases appearing in the cloud rating vocabulary in the electronic answer according to different categories;
    S402：判断高亮的词或词组是否被选中；如果是，执行S403；如果否，执行S5；S402: determining whether the highlighted word or phrase is selected; if yes, executing S403; if not, executing S5;
    S403：从云端评分词库调用该词或词组的相关信息并在屏幕上显示。S403: Calling related information of the word or phrase from the cloud rating vocabulary and displaying it on the screen.
  3. 根据权利要求1所述的判卷方法，其特征在于，The method of claiming according to claim 1, wherein
    步骤S6中，选中答案中的词或词组的方式为，在评分终端的显示屏幕上双击或更多次的点击某个词或词组所在的位置，在该位置自动生成前选定符和后选定符，前、后选定符之间的字符或字将被选中，词或词组具体内容的选择可以通过对前、后选定符的位置调整来确定，同时在屏幕的预定区域或靠近选定符的区域弹出菜单，菜单至少包括向云端上传的命令选项。In step S6, the word or phrase in the answer is selected by double clicking or more clicking on the position of the word or phrase on the display screen of the rating terminal, and automatically generating the pre-selector and the post-selection at the position. The character or word between the preceding and following selectors will be selected, and the choice of the specific content of the word or phrase can be determined by adjusting the position of the front and rear selectors, and at the same time in the predetermined area of the screen or close to the selection. A defined area pop-up menu that includes at least the command options uploaded to the cloud.
  4. 根据权利要求3所述的判卷方法，其特征在于，The method of claiming according to claim 3, characterized in that
    如果是触屏式的操作，使用预定义的手势或单独的选项来触发生成前、后选定符。In the case of touch-screen operations, use pre-defined gestures or separate options to trigger pre- and post-selection.
  5. 一种判卷方法，其特征在于，包括以下步骤：A method for determining a volume, comprising the steps of:
    S1′：开始；S1': start;
    S2′：获取主观题答案，并得到电子答案； S2': Get the answer to the subjective question and get an electronic answer;
    S3′：选中电子答案中的部分文字区域，并调用云端评分词库信息；S3': select a part of the text area in the electronic answer, and call the cloud rating lexicon information;
    S4′：将电子答案中出现在云端评分词库中的词或词组高亮，并将所述部分文字区域的高亮词或词组作为词条，显示该词条的相关信息；S4': highlighting a word or a phrase appearing in the cloud scoring vocabulary in the electronic answer, and using the highlighted word or phrase of the partial text area as a term to display related information of the term;
    S5′：判断是否将该词条加入到云端评分词库，如果是，执行S6′；如果否，执行S7′；S5': determining whether the term is added to the cloud rating vocabulary, if yes, executing S6'; if not, executing S7';
    S6′：获取被选中的词条，并将其加入云端评分词库，并返回S3′；S6': obtaining the selected entry, adding it to the cloud rating vocabulary, and returning to S3';
    S7′：获取阅卷人录入的分值；S7': obtaining the score entered by the reviewer;
    S8′：结束。S8': End.
  6. 根据权利要求5所述的判卷方法，其特征在于，The method of claiming according to claim 5, characterized in that
    步骤S4′具体为：Step S4' is specifically:
    将电子答案中出现在云端评分词库中的词或词组按不同类别高亮显示；Highlight words or phrases that appear in the cloud-based vocabulary in the electronic answer in different categories;
    将在划定的答案文字区域存在的多个高亮词或词组作为词条。A plurality of highlighted words or phrases that exist in the defined answer text area are taken as terms.
  7. 根据权利要求5所述的判卷方法，其特征在于，The method of claiming according to claim 5, characterized in that
    在步骤S6′中，选中答案中的词或词组的方式为，在评分终端的显示屏幕上双击或更多次的点击某个词或词组所在的位置，在该位置自动生成前选定符和后选定符，前、后选定符之间的字符或字将被选中，词或词组具体内容的选择可以通过对前、后选定符的位置调整来确定，同时在屏幕的预定区域或靠近选定符的区域弹出菜单，菜单至少包括向云端上传的命令选项。In step S6', the word or phrase in the answer is selected by double clicking or more clicking on the position of the word or phrase on the display screen of the rating terminal, and automatically generating the previous selector and the position. After the selector, the character or word between the front and back selectors will be selected, and the choice of the word or phrase specific content can be determined by adjusting the position of the front and rear selectors, while at the predetermined area of the screen or A pop-up menu close to the area of the selector, the menu includes at least the command options uploaded to the cloud.
  8. 根据权利要求5所述的判卷方法，其特征在于，The method of claiming according to claim 5, characterized in that
    在步骤S4′中，评分终端对词条内包含的关键词的数量及关键词类别数量进行限制，In step S4', the rating terminal limits the number of keywords and the number of keyword categories included in the entry.
    当阅卷人划取的文字区域中关键词出现的过多或类别数量超过预定值，步骤S3′中部分文字区域将被认为是一个无效的文字区域。When the keyword appears too much in the text area drawn by the reviewer or the number of categories exceeds a predetermined value, the partial text area in step S3' will be regarded as an invalid text area.
  9. 根据权利要求5所述的判卷方法，其特征在于，The method of claiming according to claim 5, characterized in that
    在步骤S4′中，如果选中的答案文字区域包含的关键词数和类别没有发生变化，仅是答案文字区域的字数总量变化，这时判卷终端判断词条并没有变化。In step S4', if the number of keywords and the category included in the selected answer text area does not change, only the total number of words in the answer text area changes, and the judgment terminal determines that the entry does not change.
  10. 一种判卷***，其具有一个判卷云端和连接在云端的多个判卷终端，判卷云端包括云端评分词库，阅卷人利用判卷终端对主观题进行阅卷，其特征在于： A judging system has a judging cloud and a plurality of judging terminals connected in the cloud, and the judging cloud includes a cloud scoring vocabulary, and the scoring person uses the judging terminal to perform subjective questions, which is characterized by:
    判卷终端执行如权利要求1-9任一项所述的判卷方法。The judgment terminal performs the method of judging according to any one of claims 1-9.
  11. 一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其特征在于，所述处理器执行所述程序时可以实现如权利要求1-9中任一项的方法步骤。A computer device comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor can implement the program as claimed in any one of claims 1-9 The method steps of the item.
  12. 一种计算机存储介质，其存储了可以被计算机执行的程序，执行所述程序时可以实现如权利要求1-9中任一项的方法步骤。 A computer storage medium storing a program executable by a computer, the method steps of any of claims 1-9 being implemented when the program is executed.
PCT/CN2017/111809 2017-10-20 2017-11-20 Paper grading method and paper grading system WO2019075819A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710981922.9A CN109697274B (en) 2017-10-20 2017-10-20 Examination paper judging method and examination paper judging system
CN201710981922.9 2017-10-20

Publications (1)

Publication Number Publication Date
WO2019075819A1 true WO2019075819A1 (en) 2019-04-25

Family

ID=66172993

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/111809 WO2019075819A1 (en) 2017-10-20 2017-11-20 Paper grading method and paper grading system

Country Status (2)

Country Link
CN (1) CN109697274B (en)
WO (1) WO2019075819A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112270295A (en) * 2020-11-13 2021-01-26 广东小天才科技有限公司 Question framing method and device under student homework scene, terminal equipment and storage medium

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110263651B (en) * 2019-05-23 2023-06-16 五邑大学 Method, device and storage medium for assisting in on-line reading test questions
CN110609953A (en) * 2019-08-28 2019-12-24 苏州承儒信息科技有限公司 Reading and amending method and system for internet education
CN110750694A (en) * 2019-09-29 2020-02-04 支付宝(杭州)信息技术有限公司 Data annotation implementation method and device, electronic equipment and storage medium
CN110705905B (en) * 2019-10-15 2022-02-08 李晚华 High-accuracy intelligent online paper marking method
CN114580375B (en) * 2022-05-09 2022-08-12 南京赛宁信息技术有限公司 Distributed online match subjective question marking and scoring method and system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101593233A (en) * 2008-05-26 2009-12-02 北京智慧东方信息技术有限公司 A kind of appraisal system of Word operation questions
CN103942994A (en) * 2014-04-22 2014-07-23 济南大学 Computer assessment method for subjective questions
CN104463101A (en) * 2014-11-06 2015-03-25 科大讯飞股份有限公司 Answer recognition method and system for textual test question
CN104504023A (en) * 2014-12-12 2015-04-08 广西师范大学 High-accuracy computer automatic marking method for subjective items based on domain ontology
CN106844296A (en) * 2016-12-01 2017-06-13 网易（杭州）网络有限公司 A kind of communication means and device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150348433A1 (en) * 2014-05-29 2015-12-03 Carnegie Mellon University Systems, Methods, and Software for Enabling Automated, Interactive Assessment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101593233A (en) * 2008-05-26 2009-12-02 北京智慧东方信息技术有限公司 A kind of appraisal system of Word operation questions
CN103942994A (en) * 2014-04-22 2014-07-23 济南大学 Computer assessment method for subjective questions
CN104463101A (en) * 2014-11-06 2015-03-25 科大讯飞股份有限公司 Answer recognition method and system for textual test question
CN104504023A (en) * 2014-12-12 2015-04-08 广西师范大学 High-accuracy computer automatic marking method for subjective items based on domain ontology
CN106844296A (en) * 2016-12-01 2017-06-13 网易（杭州）网络有限公司 A kind of communication means and device

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112270295A (en) * 2020-11-13 2021-01-26 广东小天才科技有限公司 Question framing method and device under student homework scene, terminal equipment and storage medium

Also Published As

Publication number Publication date
CN109697274A (en) 2019-04-30
CN109697274B (en) 2020-10-02

Similar Documents

Publication Publication Date Title
WO2019075819A1 (en) Paper grading method and paper grading system
Faez et al. Connecting language proficiency to teaching ability: A meta-analysis
US8250071B1 (en) Disambiguation of term meaning
US10733197B2 (en) Method and apparatus for providing information based on artificial intelligence
Churches Bloom's digital taxonomy
Wyatt-Smith et al. Multimodal reading and comprehension in online environments
US20120141959A1 (en) Crowd-sourcing the performance of tasks through online education
Gough The patterns of interaction between professional translators and online resources
Heift et al. Language learning through technology
Kalpokas et al. Bridging the gap between methodology and qualitative data analysis software: A practical guide for educators and qualitative researchers
Chen The process of note-taking<? br?> in consecutive interpreting: A digital pen recording approach
Alahmadi et al. Vocabulary learning strategies and vocabulary size: Insights from educational level and learner styles
BR122017002793A2 (en) systems and methods for language learning
JP7147185B2 (en) Information processing device, information processing method and information processing program
WO2020199600A1 (en) Sentiment polarity analysis method and related device
JP6306786B1 (en) Intellectual property support apparatus, intellectual property support method, and intellectual property support program
Cohen et al. Fine-tuning word meanings through mobile app and online resources: A case study of strategy use by a hyperpolyglot
WO2020238498A1 (en) Question and answer information processing method and system, computer device and storage medium
Lee An editable learner model for text recommendation for language learning
US9104880B2 (en) Apparatus for E-learning and method therefor
WO2020211397A1 (en) Courseware page display method and apparatus, page set construction method and apparatus, device, and medium
Oyama et al. Visual clarity analysis and improvement support for presentation slides
TWI530921B (en) Cloud-based vocabulary learning system and method
CN115511672A (en) Method for evaluating mental calculation ability of children
Tsai et al. An automatic text annotation system to improve reading comprehension of Chinese ancient texts

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17929143

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17929143

Country of ref document: EP

Kind code of ref document: A1