US20200134260A1 - System and method for implementing a proficiency-driven feedback and improvement platform - Google Patents
System and method for implementing a proficiency-driven feedback and improvement platform Download PDFInfo
- Publication number
- US20200134260A1 US20200134260A1 US16/731,904 US201916731904A US2020134260A1 US 20200134260 A1 US20200134260 A1 US 20200134260A1 US 201916731904 A US201916731904 A US 201916731904A US 2020134260 A1 US2020134260 A1 US 2020134260A1
- Authority
- US
- United States
- Prior art keywords
- learner
- proficiency
- learners
- feedback
- questions
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 230000006872 improvement Effects 0.000 title claims abstract description 64
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000011156 evaluation Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 description 25
- 230000008569 process Effects 0.000 description 20
- 238000004891 communication Methods 0.000 description 19
- 230000001413 cellular effect Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 7
- 230000009471 action Effects 0.000 description 5
- 230000015654 memory Effects 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 3
- 238000011982 device technology Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 230000003213 activating effect Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/253—Grammatical analysis; Style critique
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/232—Orthographic correction, e.g. spell checking or vowelisation
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B21/00—Teaching, or communicating with, the blind, deaf or mute
- G09B21/001—Teaching or communicating with blind persons
- G09B21/003—Teaching or communicating with blind persons using tactile presentation of the information, e.g. Braille displays
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B7/00—Electrically-operated teaching apparatus or devices working with questions and answers
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B7/00—Electrically-operated teaching apparatus or devices working with questions and answers
- G09B7/02—Electrically-operated teaching apparatus or devices working with questions and answers of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/18—Speech classification or search using natural language modelling
- G10L15/183—Speech classification or search using natural language modelling using context dependencies, e.g. language models
- G10L15/187—Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
Definitions
- This patent application relates to computer-implemented software systems, computer-network enabled systems, and user interfaces, according to one embodiment, and more specifically to a system and method for implementing a proficiency-driven feedback and improvement platform.
- the example embodiments disclosed herein provide a proficiency-assessment platform that generates a distribution of questions to evaluate a learner's level of proficiency on a set of juxtaposing sub-skills (referred to herein as “codes” and “counter-codes”), such that a learner will continue to answer questions on each code and counter-code until the learner has demonstrated proficiency on all of them, in order to most accurately determine the learner's level of proficiency on a topic composed of that set of sub-skills.
- codes and counter-codes
- the example embodiments disclosed herein also provide a proficiency-driven feedback and improvement platform to allow learners to submit work and receive feedback from other learners who have first demonstrated proficiency through the platform on one or all of a set of criteria related to that work.
- FIG. 1 illustrates the traditional teaching and learning model
- FIG. 2 illustrates the structural relationship between learners and an instructor/qualified expert in the proficiency-driven feedback and improvement system of an example embodiment
- FIG. 3 illustrates an example embodiment of a system in which various embodiments may operate
- FIG. 4 illustrates a subset of inputs related to each topic of instruction
- FIGS. 5 and 6 illustrate examples of the processing performed by the proficiency-assessment module of an example embodiment
- FIG. 7 illustrates the basic steps of an example embodiment of the proficiency-driven feedback and improvement platform
- FIG. 8 illustrates the basic steps of another example embodiment of the proficiency-driven feedback and improvement platform
- FIG. 9 illustrates the relationship between “codes and counter-codes,” “topics,” and “criteria” of an example embodiment of the proficiency-assessment module, whereby a learner who demonstrates proficiency on topics is enabled to provide feedback to other learners on the criteria associated with those topics;
- FIGS. 10 through 13 illustrate various example user interface screen snapshots that show some basic elements of the user interface for displaying data and receiving user inputs associated with the proficiency-driven feedback and improvement system in an example embodiment
- FIG. 14 illustrates another example embodiment of a networked system in which various embodiments may operate
- FIG. 15 illustrates a processing flow diagram that shows an example embodiment of a method as described herein.
- FIG. 16 shows a diagrammatic representation of a machine in the example form of a computer system within which a set of instructions when executed may cause the machine to perform any one or more of the methodologies described herein.
- a system and method for implementing a proficiency-driven feedback and improvement platform is disclosed.
- a computer-implemented tool is described to provide such a proficiency-driven feedback and improvement platform.
- the example embodiments disclosed herein allow learners to submit work into a digital platform, practice concepts designed to help them strengthen a particular set of skills, receive feedback from other learners who have demonstrated proficiency on some or all of those skills and/or feedback from an instructor, give feedback themselves on criteria on which they have demonstrated proficiency, and leverage the feedback they have received to revise and improve their own work.
- FIG. 1 the existing teaching and learning model in schools, as shown, relies heavily on the instructor/qualified expert to impart feedback that will result in performance gains for a plurality of learners.
- This model creates an excessive burden on the instructor and creates a significant lag in the time between submission of work and receipt of feedback, making feedback less helpful and actionable.
- Traditional solutions have often proven inadequate at actually driving improvement.
- the example embodiments disclosed herein provide a proficiency-driven feedback and improvement system 101 that allows learners to submit work into a digital platform.
- the digital platform 101 of an example embodiment enables proficiency-driven feedback between learners, facilitated by the online platform 101 .
- the example embodiment also enables an instructor/qualified expert to monitor the work of the learners, view the feedback that learners have left for each other, leave learners feedback, assign revisions, and/or grade or assess learners' submission, all of which is facilitated by the platform.
- learners can be both sources and receivers of feedback using the proficiency-driven feedback and improvement system 101 .
- the example embodiments provide a system and method for improving a learner's work that is unique.
- the example embodiments disclosed herein improve performance by providing a proficiency-driven feedback and improvement system that:
- the disclosed proficiency-driven feedback and improvement platform ensures that learner feedback is accurate and of high quality.
- either or both of two main concepts are involved in this process of proficiency-driven feedback and improvement: 1) a proficiency-assessment process, and 2) a proficiency-driven feedback and improvement process.
- a system and method for implementing a proficiency-driven feedback and improvement platform are disclosed.
- a computer-implemented tool or software application implements the proficiency-driven feedback and improvement platform 101 .
- a computer or computing system on which the described embodiments can be implemented can include personal computers (PCs), portable computing devices, laptops, tablet computers (e.g., iPadTM), personal digital assistants (PDAs), personal communication devices (e.g., cellular telephones, smartphones, or other wireless devices), network computers, set-top boxes, servers, mainframe computers, wearable computing devices, Internet-of-Things (IoT) devices, or any other type of computing, data processing, communication, networking, or electronic system.
- the network can include any local-area network or wide-area network, such as the Internet.
- FIG. 3 in an example embodiment, illustrates a system for implementing a proficiency-driven feedback and improvement platform 101 .
- an application or service typically provided by or operating on a host site (e.g., a website) 110 , is provided to simplify and facilitate the downloading or hosted use of the proficiency-driven feedback and improvement system 200 of an example embodiment.
- the proficiency-driven feedback and improvement system 200 can be downloaded from the host site 110 by a user at a user platform 140 .
- the proficiency-driven feedback and improvement system 200 can be hosted by the host site 110 for a networked user at a user platform 140 .
- the details of the proficiency-driven feedback and improvement system 200 for an example embodiment are provided below.
- the proficiency-driven feedback and improvement system 200 can be in data communication with a plurality of learner devices 120 , a plurality of instructor/qualified expert user devices 130 , and/or a plurality of online instructional resources 135 .
- the learners can include self-directed individuals, individuals receiving individual instruction, or individuals attending classes, seminars, online forums, or another academic or educational event, whether attending physically or virtually.
- the proficiency-driven feedback and improvement system 200 can be configured to obtain the information provided by or available from the learner devices 120 in a digital or computer-readable form via the network 115 .
- the proficiency-driven feedback and improvement system 200 can also be in data communication with a plurality of instructor/qualified expert user devices 130 .
- the instructors/qualified experts can include individuals providing instruction or hosting classes, seminars, or other academic or educational events, whether attending physically or virtually.
- the proficiency-driven feedback and improvement system 200 can be configured to obtain information provided by or available from the instructor/qualified expert devices 130 in a digital or computer-readable form via the network 115 .
- the proficiency-driven feedback and improvement system 200 can also be in network data communication with a plurality of online instructional resources 135 .
- the plurality of online instructional resources 135 can include websites, academic resource or content sites, library or information content sites, or other network resources at which information related to the topics of the academic or educational event is available.
- the proficiency-driven feedback and improvement system 200 can be configured to obtain the information content available from the plurality of online instructional resources 135 in a digital or computer-readable form via the network 115 .
- One or more of the plurality of learner devices 120 , the plurality of instructor/qualified expert user devices 130 , and/or the plurality of online instructional resources 135 can be grouped together and provided by one or more third party providers operating at various locations in a digital ecosystem. It will be apparent to those of ordinary skill in the art that the plurality of learner devices 120 , the plurality of instructor/qualified expert devices 130 , and the plurality of online instructional resources 135 can be enabled by any of a variety of networked third party information providers, websites, network-accessible information nodes, or other network resources as described in more detail below.
- a resource list maintained at the host site 110 can be used as a registry or list of all learner devices 120 , instructor/qualified expert devices 130 , and online instructional resources 135 , which users or the host site 110 may visit/access and from which users or the host site 110 can obtain or submit information content.
- the host site 110 , learner devices 120 , instructor/qualified expert devices 130 , or online instructional resources 135 , and user platforms 140 may communicate and transfer data and information in the data network ecosystem shown in FIG. 3 via a wide-area data network (e.g., the Internet) 115 .
- Various components of the host site 110 can also communicate internally via a conventional intranet or local-area network (LAN) 114 .
- LAN local-area network
- Networks 115 and 114 are configured to couple one computing device with another computing device.
- Networks 115 and 114 may be enabled to employ any form of computer readable media for communicating information from one electronic device to another.
- Network 115 can include the Internet in addition to LAN 114 , wide-area networks (WANs), direct connections, such as through a universal serial bus (USB) port, other forms of computer-readable media, or any combination thereof.
- WANs wide-area networks
- USB universal serial bus
- a router and/or gateway device acts as a link between LANs, enabling messages to be sent between computing devices.
- communication links within LANs typically include twisted wire pair or coaxial cable
- communication links between networks may utilize analog telephone lines, full or fractional dedicated digital lines including T1, T2, T3, and T4, Integrated Services Digital Networks (ISDNs), Digital Subscriber Lines (DSLs), wireless links including satellite links, or other communication links known to those of ordinary skill in the art.
- ISDNs Integrated Services Digital Networks
- DSLs Digital Subscriber Lines
- wireless links including satellite links, or other communication links known to those of ordinary skill in the art.
- remote computers and other related electronic devices can be remotely connected to either LANs or WANs via a wireless link, WiFi, BluetoothTM, satellite, or modem and temporary telephone link.
- Networks 115 and 114 may further include any of a variety of wireless sub-networks that may further overlay stand-alone ad-hoc networks, and the like, to provide an infrastructure-oriented connection. Such sub-networks may include mesh networks, Wireless LAN (WLAN) networks, cellular networks, and the like. Networks 115 and 114 may also include an autonomous system of terminals, gateways, routers, and the like connected by wireless radio links or wireless transceivers. These connectors may be configured to move freely and randomly and organize themselves arbitrarily, such that the topology of networks 115 and 114 may change rapidly and arbitrarily.
- WLAN Wireless LAN
- Networks 115 and 114 may further employ a plurality of access technologies including 2nd (2G), 2.5, 3rd (3G), 4th (4G) generation radio access for cellular systems, WLAN, Wireless Router (WR) mesh, and the like.
- Access technologies such as 2G, 3G, 4G, and future access networks may enable wide-area coverage for mobile devices, such as one or more of client devices 141 , with various degrees of mobility.
- networks 115 and 114 may enable a radio connection through a radio network access such as Global System for Mobile communication (GSM), General Packet Radio Services (GPRS), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), CDMA2000, and the like.
- GSM Global System for Mobile communication
- GPRS General Packet Radio Services
- EDGE Enhanced Data GSM Environment
- WCDMA Wideband Code Division Multiple Access
- CDMA2000 Code Division Multiple Access 2000
- Networks 115 and 114 may also be constructed for use with various other wired and wireless communication protocols, including TCP/IP, UDP, SIP, SMS, RTP, WAP, CDMA, TDMA, EDGE, UMTS, GPRS, GSM, UWB, WiFi, WiMax, IEEE 802.11x, and the like.
- networks 115 and 114 may include virtually any wired and/or wireless communication mechanisms by which information may travel between one computing device and another computing device, network, and the like.
- network 114 may represent a LAN that is configured behind a firewall (not shown), within a business or educational facility data center, for example.
- the learner devices 120 , instructor/qualified expert devices 130 , and/or the online instructional resources 135 may include any of a variety of providers or consumers of digital data.
- the digital data can be transported in any of a family of file formats and associated mechanisms usable to enable a host site 110 and/or a user platform 140 to receive information from or transfer information to learner devices 120 , instructor/qualified expert devices 130 , and/or online instructional resources 135 over the network 115 .
- the file format can be a HyperText Markup Language (HTML) format, a WordPressTM format, a MicrosoftTM Word text format, a MicrosoftTM Excel spreadsheet format, a CSV (Comma Separated Values) format, or the like; however, the various embodiments are not so limited, and other file formats and transport protocols may be used.
- HTML HyperText Markup Language
- WordPressTM format a WordPressTM format
- MicrosoftTM Word text format a MicrosoftTM Excel spreadsheet format
- CSV Common Separated Values
- Any electronic file format such as MicrosoftTM Access Database Format (MDB), Portable Document Format (PDF), audio (e.g., Motion Picture Experts Group Audio Layer 3-MP3, and the like), video (e.g., MP4, and the like), and any proprietary interchange format defined by specific sites can be supported by the various embodiments described herein.
- MDB MicrosoftTM Access Database Format
- PDF Portable Document Format
- audio e.g., Motion Picture Experts Group Audio Layer 3-MP3, and the like
- video e.g., MP4, and the like
- a user platform 140 with one or more client devices enables a user to access data provided by the proficiency-driven feedback and improvement system 200 via the host 110 and network 115 .
- Any of the learner devices 120 or the instructor/qualified expert devices 130 can be implemented as user platforms 140 .
- Client devices of user platform 140 may include virtually any computing device that is configured to send and receive information over a network, such as network 115 .
- Such client devices may include portable devices 144 , such as, cellular telephones, smart phones, display pagers, radio frequency (RF) devices, infrared (IR) devices, global positioning devices (GPS), Personal Digital Assistants (PDAs), handheld computers, IoT devices, wearable computers, tablet computers, integrated devices combining one or more of the preceding devices, and the like.
- portable devices 144 such as, cellular telephones, smart phones, display pagers, radio frequency (RF) devices, infrared (IR) devices, global positioning devices (GPS), Personal Digital Assistants (PDAs), handheld computers, IoT devices, wearable computers, tablet computers, integrated devices combining one or more of the preceding devices, and the like.
- the client devices may also include other computing devices, such as personal computers 142 , multiprocessor systems, microprocessor-based or programmable consumer electronics, network PC's, and the like.
- the client devices may also include other processing devices, such as servers or mainframe computers 146 and/or mobile computing devices 148 ,
- the client devices of user platform 140 may range widely in terms of capabilities and features.
- the web-enabled client device may include a browser application enabled to receive and to send network transportable data content and executable or interpretable instructions, wireless application protocol messages (WAP), and/or wired application messages, and the like.
- the browser application is enabled to employ HyperText Markup Language (HTML), Dynamic HTML, Handheld Device Markup Language (HDML), Wireless Markup Language (WML), WMLScript, JavaScriptTM, EXtensible HTML (xHTML), Compact HTML (CHTML), and the like, to display and/or send digital information.
- mobile devices can be configured with applications (apps) with which the functionality described herein can be implemented.
- the client devices of user platform 140 may also include at least one client application that is configured to receive information content and/or control data from another computing device via a wired or wireless network transmission.
- the client application may include a capability to provide and receive textual data, graphical data, video data, audio data, and the like.
- client devices of user platform 140 may be further configured to communicate and/or receive a message, such as through a Short Message Service (SMS), direct messaging (e.g., TwitterTM), email, Multimedia Message Service (MMS), instant messaging (IM), internet relay chat (IRC), mIRC, Jabber, Enhanced Messaging Service (EMS), text messaging, Smart Messaging, Over the Air (OTA) messaging, or the like, between another computing device, and the like.
- SMS Short Message Service
- MMS Multimedia Message Service
- IM instant messaging
- IRC internet relay chat
- mIRC Jabber
- EMS Enhanced Messaging Service
- text messaging Smart Messaging, Over the Air (OTA) messaging, or the like,
- the proficiency-driven feedback and improvement system 200 for an example embodiment is shown to include a database 112 .
- the database 112 can be used to retain a variety of information data sets including, but not limited to, network resource information, user information, user instructional content, information view configurations, information view templates, user document configuration information, user-generated content, and the like. It will be apparent to those of ordinary skill in the art that the database 112 can be locally resident at the host site 110 or remotely located at other server locations or stored in network cloud storage.
- host site 110 of an example embodiment is shown to include the proficiency-driven feedback and improvement system 200 .
- proficiency-driven feedback and improvement system 200 may include a proficiency-assessment module 210 and/or a proficiency-driven feedback module 230 .
- These modules can be implemented as software components executing within an executable environment of the proficiency-driven feedback and improvement system 200 operating on host site 110 or user platform 140 .
- Each of these modules of an example embodiment is described in more detail below in connection with the figures provided herein.
- the proficiency-driven feedback and improvement system 200 may include a proficiency-assessment module 210 .
- the proficiency-assessment module 210 can be configured to perform the processing as described herein. Initially, the proficiency-assessment module 210 can be configured to establish a data connection with at least one of the plurality of learner devices 120 , at least one of the plurality of instructor/qualified expert devices 130 , and optionally one or more of the plurality of online instructional resources 135 .
- the proficiency-assessment module 210 can be configured to generate and display on a user interface of a user platform 140 information content and at least one user-selectable input object or icon associated with available user-selectable option elements of a pre-defined set of user-selectable option elements as described in more detail below.
- the proficiency-assessment module 210 can be further configured to gather and process information pertaining to each learner.
- the learner information is processed to assess each learner's level of understanding.
- a proficiency-assessment process uses adaptive technology to gauge each learner's understanding of concepts as he or she works toward proficiency. This process is unique in its ability to accurately assess learners' proficiency of intertwined and related skills.
- example embodiments may include a system by which a learner or learners evaluate another learner according to one or more criteria; a system by which a learner's ability to evaluate other learners on a single criterion may be determined by his or her demonstrated proficiency on one or more concepts or “topics”; and/or a system by which a learner's proficiency on a topic may be determined by his or her demonstrated proficiency on a specific skill or “code” and juxtaposed but related skills, or “counter-codes.”
- an example proficiency-assessment module 210 for assessing the skills and level of proficiency of each learner on one or more topics of instruction. For each topic of instruction, the proficiency-assessment module 210 tracks a collection of information related to the skills for each learner to ensure complete and accurate learner comprehension. This process prevents a learner from getting questions correct when tested in isolation, only to later be unable to apply those concepts in authentic situations because his or her understanding of the material was surface-level and un-nuanced.
- the proficiency-assessment module 210 considers among four inputs when determining which questions to display to a learner on a given “topic” of instruction:
- the proficiency-assessment module 210 may use the inputs described above to generate a customized set of questions for each learner on a topic.
- the questions can be presented to each learner by the proficiency-assessment module 210 via the user interface of a user platform 140 as described above.
- the customized set of questions may be generated and presented for each learner such that each learner receives a mix of questions from codes and their counter-codes.
- the questions presented may also vary in difficulty depending on learner performance, such that learners see easier questions after making errors and harder questions after answering correctly. Learners may ultimately interact with progressively more difficult questions on each code or counter-code until they have demonstrated a desired level of proficiency on each one. The learner may continue to work on the topic until he or she has demonstrated proficiency on all codes and counter-codes within the topic.
- the proficiency-assessment module 210 ensures that:
- the proficiency-assessment module 210 uses each topic's codes and counter-codes to generate customized sets of questions presented to learners via the user interface for assessing the skills and level of proficiency of each learner on one or more topics of instruction.
- FIGS. 5 and 6 illustrate two examples of this process for question generation.
- codes 1 , 2 , 3 , and 4 on a topic are all counter-codes of each other, and a learner has demonstrated proficiency of codes 1 , 2 , and 3 by correctly answering questions from each code. The learner has not yet, however, demonstrated proficiency on code 4 .
- the proficiency-assessment module 210 generates questions from code 4 x% of the time and questions from codes 1 , 2 , and 3 , the other ( 100 - x )% of the time while ensuring that questions do not alternate in a predictable pattern.
- the proficiency-assessment module 210 can ensure that when the learner answers a question correctly from code 4 , the next question he or she encounters from that code will be more difficult than the previous one from that code. The same can be said for the learner encountering a less difficult question from code 4 subsequent to getting a code 4 question incorrect.
- the proficiency-assessment module 210 can continue to oscillate questions among all codes in the manner described above until the learner has demonstrated proficiency on all codes in the topic.
- FIG. 6 another example illustrates the processing performed by the proficiency-assessment module 210 .
- the example shows that a particular learner has demonstrated proficiency on topic codes 1 , 2 , and 3 by correctly answering questions related to these three codes.
- codes 1 and 2 are counter-codes of each other and codes 3 and 4 are counter-codes of each other.
- the proficiency-assessment module 210 can generate questions from code 4 x% of the time and questions from code 3 the other ( 100 - x )% of the time while ensuring that questions do not alternate in a predictable pattern.
- the proficiency-assessment module 210 can ensure that when the learner answers a question correctly from code 4 , the next question he or she encounters from that code will be more difficult than the previous one from that code. The same can be said for the learner encountering a less difficult question from code 4 subsequent to getting a code 4 question incorrect.
- the proficiency-assessment module 210 can continue to oscillate questions among all codes in the manner described above until the learner has demonstrated proficiency on all codes in the topic.
- the proficiency-driven feedback and improvement system 200 can be configured to perform the processing as described herein.
- the proficiency-driven feedback and improvement system 200 can be configured to establish, by use of a data processor and the data network 115 , a network data connection with at least one of the plurality of learner devices 120 , at least one of the plurality of instructor/qualified expert devices 130 , and optionally one or more of the plurality of online instructional resources 135 .
- the proficiency-driven feedback and improvement system 200 can be configured to generate and display on a user interface of a user platform 140 information content and at least one user-selectable input object r icon associated with available user-selectable option elements of a pre-defined set of user-selectable option elements as described in more detail below.
- learners are guided through the process of submitting and/or editing work, practicing concepts that help them strengthen a particular skill or set of skills, evaluating the work of other learners (e.g., submitting input on a binary, ternary, or Likert scale, annotating or highlighting work, and submitting free-form comments), receiving feedback from other learners, and/or revising their own work as necessary.
- evaluating the work of other learners e.g., submitting input on a binary, ternary, or Likert scale, annotating or highlighting work, and submitting free-form comments
- instructors/qualified experts are able to monitor learners' work and progress on skill attainment, offer feedback, view feedback provided by other learners, and assign revisions and/or grade submissions.
- instructors/qualified experts can create and configure customized assignments, enter or select prompts, and/or select criteria for assessing learners and/or specific skills to assess using the proficiency-driven feedback and improvement process, using the proficiency-driven feedback and improvement system 200 , via the user interface of a user platform 140 .
- the proficiency-driven feedback and improvement system 200 may include a proficiency-driven feedback module 230 .
- the proficiency-driven feedback module 230 may be configured to perform the processing as described herein. Initially, the proficiency-driven feedback module 230 may be configured to establish a data connection with at least one of the plurality of learner user devices 120 , at least one of the plurality of instructor/qualified expert devices 130 , and optionally one or more of the plurality of online instructional resources 135 .
- the proficiency-driven feedback module 230 may be configured to generate and display on a user interface of a user platform 140 information content and at least one user-selectable input object or icon associated with available user-selectable option elements of a pre-defined set of user-selectable option elements as described in more detail below.
- a proficiency-driven feedback and improvement process may also be used as part of the proficiency-driven feedback and improvement system 200 .
- the proficiency-driven feedback module 230 provided for this purpose, may be configured to determine and enforce a policy wherein a learner can only evaluate or provide feedback on those criteria on which he or she has demonstrated proficiency, helping to ensure that learners receive accurate and high-quality feedback. Proficiency can be evaluated using a variety of different techniques and processes as described herein.
- a learner in order to demonstrate proficiency on a criterion, a learner must demonstrate proficiency on each topic associated with that criterion and all codes and counter-codes associated with each topic. If a learner has not demonstrated proficiency on all topics associated with a criterion, that criterion may be removed from the learner's peer evaluation experience. In FIG. 9 , for example, a learner who has demonstrated proficiency on topics 1 - 4 , 6 , and 7 , but not topic 5 , may be enabled to provide feedback to other learners on criterion 1 and 2 , but not on criterion 3 .
- proficiency-driven feedback module 230 learners evaluate their own work using the same criteria that they would use to evaluate the work of other learners. These embodiments may be configured such that learners must evaluate their own work before the proficiency-driven feedback module 230 reveals feedback on their work from other learners. Some example embodiments of the proficiency-driven feedback and improvement system 200 maintain a visually consistent experience and/or utilize similar language across the practice and evaluation phases, maximizing the transference of skills from practice to evaluation of other learners to self-evaluation.
- various example embodiments may automatically: 1) present and enable selection and configuration of a plurality of criteria; 2) present and enable configuration and selection of a plurality of prompt options or other guiding questions; 3) manage a variety of other operations related to the generation, configuration, and execution of a system and method for implementing a computer proficiency-driven feedback and improvement platform.
- the computer-generated user interface can be illustrated in several screen snapshots as illustrated herein and described below.
- FIGS. 10 through 13 illustrate various example user interface screen snapshots, implemented as a web application, that show the basic elements of the user interface for displaying data and receiving user inputs associated with the proficiency-driven feedback and improvement system 200 in an example embodiment.
- FIGS. 10 and 11 a sample user interface of an example embodiment is shown. Under the direction of the proficiency-driven feedback and improvement system 200 of an example embodiment, a user/learner can be prompted with any of a customized set of questions and may respond with answers by interacting with the input objects as shown in FIGS. 10 and 11 .
- the examples shown in FIGS. 10 and 11 are described in more detail below.
- FIGS. 12 and 13 another sample user interface of an example embodiment is shown. Under the direction of the proficiency-driven feedback and improvement system 200 of an example embodiment, a user/learner can be prompted with any of a customized set of questions and may respond with answers by interacting with the input objects as shown in FIGS. 12 and 13 .
- the examples shown in FIGS. 13 and 14 are described in more detail below.
- the web app implementation of the example user interfaces shown in FIGS. 10 through 13 can be similarly implemented as a mobile application (app) interface or an interface for other interactive devices.
- the web app and the mobile app of various example embodiments can have a similar software architecture, which is described for various example embodiments herein.
- the host site 110 is shown to include the proficiency-driven feedback and improvement system 200 .
- the proficiency-driven feedback and improvement system 200 is shown to include the functional components 210 and 230 , as described above.
- the host site 110 may also include a web server 904 , having a web interface with which users may interact with the host site 110 via a user interface or web interface.
- the host site 110 may also include an application programming interface (API) 902 with which the host site 110 may interact with other network entities or resources on a programmatic or automated data transfer level.
- the API 902 and web interface 904 may be configured to interact with the proficiency-driven feedback and improvement system 200 either directly or via an interface 906 .
- the proficiency-driven feedback and improvement system 200 may be configured to access a database 112 either directly or via the interface 906 .
- the method 2000 of an example embodiment includes: establishing, by use of a data processor, a data connection with a learner (processing block 2010 ); generating a distribution of questions to determine the learner's level of proficiency on a topic, the topic being composed of a set of codes and counter-codes (processing block 2012 ); determining the learner proficient on the topic after the learner has demonstrated proficiency on each code and counter-code within the topic (processing block 2014 ); and distributing questions to the learner on codes and counter-codes within the topic until the learner has demonstrated proficiency on each code and counter-code within the topic (processing block 2016 ).
- the method 2000 of an example embodiment further includes: establishing, by use of a data processor and a data network, a network data connection with a learner (processing block 2011 ); providing the learner feedback on work according to a set of criteria (processing block 2013 ); and ensuring that only users who have first demonstrated proficiency on one or more criteria in the set of criteria can give feedback on those criteria (processing block 2015 ).
- FIG. 16 shows a diagrammatic representation of a machine in the example form of a mobile computing and/or communication system 700 within which a set of instructions when executed and/or processing logic when activated may cause the machine to perform any one or more of the methodologies described and/or claimed herein.
- the machine operates as a standalone device or may be connected (e.g., networked) to other machines.
- the machine may operate in the capacity of a server or a client machine in server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.
- the machine may be a personal computer (PC), a laptop computer, a tablet computing system, a Personal Digital Assistant (PDA), a cellular telephone, a smartphone, a web appliance, a set-top box (STB), a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) or activating processing logic that specify actions to be taken by that machine.
- PC personal computer
- PDA Personal Digital Assistant
- STB set-top box
- STB set-top box
- network router switch or bridge
- the example mobile computing and/or communication system 700 includes a data processor 702 (e.g., a System-on-a-Chip (SoC), general processing core, graphics core, and optionally other processing logic) and a memory 704 , which can communicate with each other via a bus or other data transfer system 706 .
- the mobile computing and/or communication system 700 may further include various input/output (I/O) devices and/or interfaces 710 , such as a touchscreen display, an audio jack, and optionally a network interface 712 .
- I/O input/output
- the network interface 712 can include one or more radio transceivers configured for compatibility with any one or more standard wireless and/or cellular protocols or access technologies (e.g., 2nd (2G), 2.5, 3rd (3G), 4th (4G) generation, and future generation radio access for cellular systems, Global System for Mobile communication (GSM), General Packet Radio Services (GPRS), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), LTE, CDMA2000, WLAN, Wireless Router (WR) mesh, and the like).
- GSM Global System for Mobile communication
- GPRS General Packet Radio Services
- EDGE Enhanced Data GSM Environment
- WCDMA Wideband Code Division Multiple Access
- LTE Long Term Evolution
- CDMA2000 Code Division Multiple Access 2000
- WLAN Wireless Router
- Network interface 712 may also be configured for use with various other wired and/or wireless communication protocols, including TCP/IP, UDP, SIP, SMS, RTP, WAP, CDMA, TDMA, UMTS, UWB, WiFi, WiMax, BluetoothTM, IEEE 802.11x, and the like.
- network interface 712 may include or support virtually any wired and/or wireless communication mechanisms by which information may travel between the mobile computing and/or communication system 700 and another computing or communication system via network 714 .
- the memory 704 can represent a machine-readable medium on which is stored one or more sets of instructions, software, firmware, or other processing logic (e.g., logic 708 ) embodying any one or more of the methodologies or functions described and/or claimed herein.
- the logic 708 may also reside, completely or at least partially within the processor 702 during execution thereof by the mobile computing and/or communication system 700 .
- the memory 704 and the processor 702 may also constitute machine-readable media.
- the logic 708 , or a portion thereof may also be configured as processing logic or logic, at least a portion of which is partially implemented in hardware.
- the logic 708 , or a portion thereof may further be transmitted or received over a network 714 via the network interface 712 .
- machine-readable medium of an example embodiment can be a single medium
- the term “machine-readable medium” should be taken to include a single non-transitory medium or multiple non-transitory media (e.g., a centralized or distributed database, and/or associated caches and computing systems) that stores the one or more sets of instructions.
- the term “machine-readable medium” can also be taken to include any non-transitory medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the various embodiments, or that is capable of storing, encoding or carrying data structures utilized by or associated with such a set of instructions.
- the term “machine-readable medium” can accordingly be taken to include, but not be limited to, solid-state memories, optical media, and magnetic media.
- a system and method for implementing a proficiency-driven feedback and improvement platform are disclosed.
- a software application program is used to enable the development and presentation of information categories, information sub-categories, and information components in customizable views on the display screen of a computing or communication system, including mobile devices.
- the proficiency-driven feedback and improvement system 200 of an example embodiment can be configured to automatically obtain a variety of information from one or more 3rd party sites or network resources via a data network to facilitate the user experience of searching for desired information datasets, configuring a proficiency-driven feedback and improvement platform, and sharing information, all from the convenience of a computing device or a portable electronic device, such as a smartphone.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Business, Economics & Management (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Electrically Operated Instructional Devices (AREA)
Abstract
A system and method for implementing a proficiency-driven feedback and improvement platform is disclosed. A particular embodiment includes: establishing a data connection with a learner; generating a distribution of questions to determine the learner's level of proficiency on a topic, the topic being composed of a set of codes and counter-codes; determining the learner proficient on the topic after the learner has demonstrated proficiency on each code and counter-code within the topic; and distributing questions to the learner on codes and counter-codes within the topic until the learner has demonstrated proficiency on each code and counter-code within the topic. A particular embodiment further includes: providing the learner feedback on work according to a set of criteria; and ensuring that only users who have first demonstrated proficiency on one or more criteria in the set of criteria can give feedback on those criteria.
Description
- This patent application is a continuation patent application drawing priority from U.S. non-provisional patent application Ser. No. 15/706,701; filed Sep. 16, 2017. This present non-provisional patent application draws priority from the referenced patent application. The entire disclosure of the referenced patent application is considered part of the disclosure of the present application and is hereby incorporated by reference herein in its entirety.
- A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright rights whatsoever. The following notice applies to the software and data as described below and in the drawings that form a part of this document: Copyright 2016-2019 NoRedlnk Corp., All Rights Reserved.
- This patent application relates to computer-implemented software systems, computer-network enabled systems, and user interfaces, according to one embodiment, and more specifically to a system and method for implementing a proficiency-driven feedback and improvement platform.
- Helping learners make demonstrated gains in a given subject area has long been constrained by the time, effort, and expertise required to effectively assess learners' work and help them improve. These factors are among the most significant deterrents to learning.
- The example embodiments disclosed herein provide a proficiency-assessment platform that generates a distribution of questions to evaluate a learner's level of proficiency on a set of juxtaposing sub-skills (referred to herein as “codes” and “counter-codes”), such that a learner will continue to answer questions on each code and counter-code until the learner has demonstrated proficiency on all of them, in order to most accurately determine the learner's level of proficiency on a topic composed of that set of sub-skills.
- The example embodiments disclosed herein also provide a proficiency-driven feedback and improvement platform to allow learners to submit work and receive feedback from other learners who have first demonstrated proficiency through the platform on one or all of a set of criteria related to that work.
- The various embodiments are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which:
-
FIG. 1 illustrates the traditional teaching and learning model; -
FIG. 2 illustrates the structural relationship between learners and an instructor/qualified expert in the proficiency-driven feedback and improvement system of an example embodiment; -
FIG. 3 illustrates an example embodiment of a system in which various embodiments may operate; -
FIG. 4 illustrates a subset of inputs related to each topic of instruction; -
FIGS. 5 and 6 illustrate examples of the processing performed by the proficiency-assessment module of an example embodiment; -
FIG. 7 illustrates the basic steps of an example embodiment of the proficiency-driven feedback and improvement platform; -
FIG. 8 illustrates the basic steps of another example embodiment of the proficiency-driven feedback and improvement platform; -
FIG. 9 illustrates the relationship between “codes and counter-codes,” “topics,” and “criteria” of an example embodiment of the proficiency-assessment module, whereby a learner who demonstrates proficiency on topics is enabled to provide feedback to other learners on the criteria associated with those topics; -
FIGS. 10 through 13 illustrate various example user interface screen snapshots that show some basic elements of the user interface for displaying data and receiving user inputs associated with the proficiency-driven feedback and improvement system in an example embodiment; -
FIG. 14 illustrates another example embodiment of a networked system in which various embodiments may operate; -
FIG. 15 illustrates a processing flow diagram that shows an example embodiment of a method as described herein; and -
FIG. 16 shows a diagrammatic representation of a machine in the example form of a computer system within which a set of instructions when executed may cause the machine to perform any one or more of the methodologies described herein. - In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments. It will be evident, however, to one of ordinary skill in the art that the various embodiments may be practiced without these specific details.
- In various example embodiments, a system and method for implementing a proficiency-driven feedback and improvement platform is disclosed. In various example embodiments, a computer-implemented tool is described to provide such a proficiency-driven feedback and improvement platform. The example embodiments disclosed herein allow learners to submit work into a digital platform, practice concepts designed to help them strengthen a particular set of skills, receive feedback from other learners who have demonstrated proficiency on some or all of those skills and/or feedback from an instructor, give feedback themselves on criteria on which they have demonstrated proficiency, and leverage the feedback they have received to revise and improve their own work.
- Referring now to
FIG. 1 , the existing teaching and learning model in schools, as shown, relies heavily on the instructor/qualified expert to impart feedback that will result in performance gains for a plurality of learners. This model creates an excessive burden on the instructor and creates a significant lag in the time between submission of work and receipt of feedback, making feedback less helpful and actionable. Traditional solutions have often proven inadequate at actually driving improvement. - Referring now to
FIG. 2 , the example embodiments disclosed herein provide a proficiency-driven feedback andimprovement system 101 that allows learners to submit work into a digital platform. Thedigital platform 101 of an example embodiment enables proficiency-driven feedback between learners, facilitated by theonline platform 101. The example embodiment also enables an instructor/qualified expert to monitor the work of the learners, view the feedback that learners have left for each other, leave learners feedback, assign revisions, and/or grade or assess learners' submission, all of which is facilitated by the platform. Unlike inFIG. 1 , learners can be both sources and receivers of feedback using the proficiency-driven feedback andimprovement system 101. - While the various example embodiments described herein may refer to learners and instructors/qualified experts, it should be understood that the present invention is not limited to any particular learning structure or hierarchy, nor is it limited to any one particular skill. The present invention can be applied with or without any instructor, and it may be applied in any situation where feedback is useful, including without limitation various forms of writing, reading, speaking, mathematics, science, language acquisition or improvement, critical thinking, and performing and creative arts. It may be effectively applied in any context where visual, written, audio, or other forms of feedback can help one improve.
- The example embodiments provide a system and method for improving a learner's work that is unique. In contrast to other solutions, the example embodiments disclosed herein improve performance by providing a proficiency-driven feedback and improvement system that:
-
- removes the instructor/qualified expert as the sole source of feedback. In many learning environments, learners only receive one source of feedback from an instructor who may simultaneously have to provide feedback to many other learners, causing the feedback to be rushed and of low quality, the learners to wait for an extended time period receive it, and the resulting performance gains from the feedback to be inadequate
- teaches learners how to analyze work according to a guided set of criteria. The disclosed example embodiments improve learners' skills as they first practice concepts designed to strengthen their understanding and then further enhance those skills through the act of giving feedback on the criteria on which they have demonstrated proficiency.
- provides learners with a higher level of qualitative feedback. Unlike fully auto-graded solutions, which are more adept at pointing out surface-level errors on learners' original work, the disclosed embodiments foster in-depth, more valuable feedback to learners.
- The disclosed proficiency-driven feedback and improvement platform ensures that learner feedback is accurate and of high quality. In various example embodiments, either or both of two main concepts are involved in this process of proficiency-driven feedback and improvement: 1) a proficiency-assessment process, and 2) a proficiency-driven feedback and improvement process. These processes and their related systems and technical implementation are described in more detail below. It should be understood that these two processes can be used independently or together as appropriate.
- In various example embodiments, a system and method for implementing a proficiency-driven feedback and improvement platform are disclosed. In the various example embodiments described herein, a computer-implemented tool or software application (app) implements the proficiency-driven feedback and
improvement platform 101. As described in more detail below, a computer or computing system on which the described embodiments can be implemented can include personal computers (PCs), portable computing devices, laptops, tablet computers (e.g., iPad™), personal digital assistants (PDAs), personal communication devices (e.g., cellular telephones, smartphones, or other wireless devices), network computers, set-top boxes, servers, mainframe computers, wearable computing devices, Internet-of-Things (IoT) devices, or any other type of computing, data processing, communication, networking, or electronic system. The network can include any local-area network or wide-area network, such as the Internet. -
FIG. 3 , in an example embodiment, illustrates a system for implementing a proficiency-driven feedback andimprovement platform 101. In various example embodiments, an application or service, typically provided by or operating on a host site (e.g., a website) 110, is provided to simplify and facilitate the downloading or hosted use of the proficiency-driven feedback andimprovement system 200 of an example embodiment. In a particular embodiment, the proficiency-driven feedback andimprovement system 200 can be downloaded from thehost site 110 by a user at auser platform 140. Alternatively, the proficiency-driven feedback andimprovement system 200 can be hosted by thehost site 110 for a networked user at auser platform 140. The details of the proficiency-driven feedback andimprovement system 200 for an example embodiment are provided below. - Referring again to
FIG. 3 , the proficiency-driven feedback andimprovement system 200 can be in data communication with a plurality oflearner devices 120, a plurality of instructor/qualifiedexpert user devices 130, and/or a plurality of onlineinstructional resources 135. The learners can include self-directed individuals, individuals receiving individual instruction, or individuals attending classes, seminars, online forums, or another academic or educational event, whether attending physically or virtually. The proficiency-driven feedback andimprovement system 200 can be configured to obtain the information provided by or available from thelearner devices 120 in a digital or computer-readable form via thenetwork 115. The proficiency-driven feedback andimprovement system 200 can also be in data communication with a plurality of instructor/qualifiedexpert user devices 130. The instructors/qualified experts can include individuals providing instruction or hosting classes, seminars, or other academic or educational events, whether attending physically or virtually. The proficiency-driven feedback andimprovement system 200 can be configured to obtain information provided by or available from the instructor/qualified expert devices 130 in a digital or computer-readable form via thenetwork 115. The proficiency-driven feedback andimprovement system 200 can also be in network data communication with a plurality of onlineinstructional resources 135. The plurality of onlineinstructional resources 135 can include websites, academic resource or content sites, library or information content sites, or other network resources at which information related to the topics of the academic or educational event is available. The proficiency-driven feedback andimprovement system 200 can be configured to obtain the information content available from the plurality of onlineinstructional resources 135 in a digital or computer-readable form via thenetwork 115. - One or more of the plurality of
learner devices 120, the plurality of instructor/qualifiedexpert user devices 130, and/or the plurality of onlineinstructional resources 135 can be grouped together and provided by one or more third party providers operating at various locations in a digital ecosystem. It will be apparent to those of ordinary skill in the art that the plurality oflearner devices 120, the plurality of instructor/qualified expert devices 130, and the plurality of onlineinstructional resources 135 can be enabled by any of a variety of networked third party information providers, websites, network-accessible information nodes, or other network resources as described in more detail below. In a particular embodiment, a resource list maintained at thehost site 110 can be used as a registry or list of alllearner devices 120, instructor/qualified expert devices 130, and onlineinstructional resources 135, which users or thehost site 110 may visit/access and from which users or thehost site 110 can obtain or submit information content. Thehost site 110,learner devices 120, instructor/qualified expert devices 130, or onlineinstructional resources 135, anduser platforms 140 may communicate and transfer data and information in the data network ecosystem shown inFIG. 3 via a wide-area data network (e.g., the Internet) 115. Various components of thehost site 110 can also communicate internally via a conventional intranet or local-area network (LAN) 114. -
Networks Networks Network 115 can include the Internet in addition toLAN 114, wide-area networks (WANs), direct connections, such as through a universal serial bus (USB) port, other forms of computer-readable media, or any combination thereof. On an interconnected set of LANs, including those based on differing architectures and protocols, a router and/or gateway device acts as a link between LANs, enabling messages to be sent between computing devices. Also, communication links within LANs typically include twisted wire pair or coaxial cable, while communication links between networks may utilize analog telephone lines, full or fractional dedicated digital lines including T1, T2, T3, and T4, Integrated Services Digital Networks (ISDNs), Digital Subscriber Lines (DSLs), wireless links including satellite links, or other communication links known to those of ordinary skill in the art. Furthermore, remote computers and other related electronic devices can be remotely connected to either LANs or WANs via a wireless link, WiFi, Bluetooth™, satellite, or modem and temporary telephone link. -
Networks Networks networks -
Networks networks Networks networks network 114 may represent a LAN that is configured behind a firewall (not shown), within a business or educational facility data center, for example. - The
learner devices 120, instructor/qualified expert devices 130, and/or the onlineinstructional resources 135 may include any of a variety of providers or consumers of digital data. The digital data can be transported in any of a family of file formats and associated mechanisms usable to enable ahost site 110 and/or auser platform 140 to receive information from or transfer information tolearner devices 120, instructor/qualified expert devices 130, and/or onlineinstructional resources 135 over thenetwork 115. In various embodiments, the file format can be a HyperText Markup Language (HTML) format, a WordPress™ format, a Microsoft™ Word text format, a Microsoft™ Excel spreadsheet format, a CSV (Comma Separated Values) format, or the like; however, the various embodiments are not so limited, and other file formats and transport protocols may be used. For example, data formats other than Excel or CSV or formats other than open/standard formats can be supported by various embodiments. Any electronic file format, such as Microsoft™ Access Database Format (MDB), Portable Document Format (PDF), audio (e.g., Motion Picture Experts Group Audio Layer 3-MP3, and the like), video (e.g., MP4, and the like), and any proprietary interchange format defined by specific sites can be supported by the various embodiments described herein. Moreover,learner devices 120, instructor/qualified expert devices 130, and/or the onlineinstructional resources 135 may provide or consume a variety of different data sets. - In a particular embodiment, a
user platform 140 with one or more client devices enables a user to access data provided by the proficiency-driven feedback andimprovement system 200 via thehost 110 andnetwork 115. Any of thelearner devices 120 or the instructor/qualified expert devices 130 can be implemented asuser platforms 140. Client devices ofuser platform 140 may include virtually any computing device that is configured to send and receive information over a network, such asnetwork 115. Such client devices may includeportable devices 144, such as, cellular telephones, smart phones, display pagers, radio frequency (RF) devices, infrared (IR) devices, global positioning devices (GPS), Personal Digital Assistants (PDAs), handheld computers, IoT devices, wearable computers, tablet computers, integrated devices combining one or more of the preceding devices, and the like. The client devices may also include other computing devices, such aspersonal computers 142, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PC's, and the like. The client devices may also include other processing devices, such as servers ormainframe computers 146 and/ormobile computing devices 148, which are known to those of ordinary skill in the art. As such, the client devices ofuser platform 140 may range widely in terms of capabilities and features. Moreover, the web-enabled client device may include a browser application enabled to receive and to send network transportable data content and executable or interpretable instructions, wireless application protocol messages (WAP), and/or wired application messages, and the like. In one embodiment, the browser application is enabled to employ HyperText Markup Language (HTML), Dynamic HTML, Handheld Device Markup Language (HDML), Wireless Markup Language (WML), WMLScript, JavaScript™, EXtensible HTML (xHTML), Compact HTML (CHTML), and the like, to display and/or send digital information. In other embodiments, mobile devices can be configured with applications (apps) with which the functionality described herein can be implemented. - The client devices of
user platform 140 may also include at least one client application that is configured to receive information content and/or control data from another computing device via a wired or wireless network transmission. The client application may include a capability to provide and receive textual data, graphical data, video data, audio data, and the like. Moreover, client devices ofuser platform 140 may be further configured to communicate and/or receive a message, such as through a Short Message Service (SMS), direct messaging (e.g., Twitter™), email, Multimedia Message Service (MMS), instant messaging (IM), internet relay chat (IRC), mIRC, Jabber, Enhanced Messaging Service (EMS), text messaging, Smart Messaging, Over the Air (OTA) messaging, or the like, between another computing device, and the like. - Referring again to
FIG. 3 , the proficiency-driven feedback andimprovement system 200 for an example embodiment is shown to include adatabase 112. Thedatabase 112 can be used to retain a variety of information data sets including, but not limited to, network resource information, user information, user instructional content, information view configurations, information view templates, user document configuration information, user-generated content, and the like. It will be apparent to those of ordinary skill in the art that thedatabase 112 can be locally resident at thehost site 110 or remotely located at other server locations or stored in network cloud storage. - Referring again to
FIG. 3 ,host site 110 of an example embodiment is shown to include the proficiency-driven feedback andimprovement system 200. In an example embodiment, proficiency-driven feedback andimprovement system 200 may include a proficiency-assessment module 210 and/or a proficiency-drivenfeedback module 230. These modules can be implemented as software components executing within an executable environment of the proficiency-driven feedback andimprovement system 200 operating onhost site 110 oruser platform 140. Each of these modules of an example embodiment is described in more detail below in connection with the figures provided herein. - The proficiency-driven feedback and
improvement system 200 may include a proficiency-assessment module 210. The proficiency-assessment module 210 can be configured to perform the processing as described herein. Initially, the proficiency-assessment module 210 can be configured to establish a data connection with at least one of the plurality oflearner devices 120, at least one of the plurality of instructor/qualified expert devices 130, and optionally one or more of the plurality of onlineinstructional resources 135. The proficiency-assessment module 210 can be configured to generate and display on a user interface of auser platform 140 information content and at least one user-selectable input object or icon associated with available user-selectable option elements of a pre-defined set of user-selectable option elements as described in more detail below. - The proficiency-
assessment module 210 can be further configured to gather and process information pertaining to each learner. The learner information is processed to assess each learner's level of understanding. In various example embodiments, a proficiency-assessment process uses adaptive technology to gauge each learner's understanding of concepts as he or she works toward proficiency. This process is unique in its ability to accurately assess learners' proficiency of intertwined and related skills. - Referring now to
FIG. 9 , example embodiments may include a system by which a learner or learners evaluate another learner according to one or more criteria; a system by which a learner's ability to evaluate other learners on a single criterion may be determined by his or her demonstrated proficiency on one or more concepts or “topics”; and/or a system by which a learner's proficiency on a topic may be determined by his or her demonstrated proficiency on a specific skill or “code” and juxtaposed but related skills, or “counter-codes.” - Referring now to
FIGS. 4 through 6 , examples are illustrated of the processing that may be performed by an example proficiency-assessment module 210 for assessing the skills and level of proficiency of each learner on one or more topics of instruction. For each topic of instruction, the proficiency-assessment module 210 tracks a collection of information related to the skills for each learner to ensure complete and accurate learner comprehension. This process prevents a learner from getting questions correct when tested in isolation, only to later be unable to apply those concepts in authentic situations because his or her understanding of the material was surface-level and un-nuanced. - Referring now to
FIG. 4 , in one example embodiment, the proficiency-assessment module 210 considers among four inputs when determining which questions to display to a learner on a given “topic” of instruction: -
- the
relationship 410 between juxtaposing skills, referred to here as “codes” and their “counter-codes,” within the topic. These sets of codes andcounter-codes 410 may be assessed together to ensure that learners do not falsely extrapolate patterns. - the
necessary threshold 412 for demonstrating proficiency of each skill. The threshold for demonstratingproficiency 412 may vary based on the difficulty and complexity of each skill. - the
relative frequency 414 to assess each skill. Thisrelative frequency ratio 414 is also intended to mirror the desired emphasis in the topic. - the difficulty of all questions available on the site. The proficiency-
assessment module 210 can calculate the difficulty of questions based on a collection (e.g., billions) of prior attempts from learners on answering the questions.
- the
- In various example embodiments, the proficiency-
assessment module 210 may use the inputs described above to generate a customized set of questions for each learner on a topic. The questions can be presented to each learner by the proficiency-assessment module 210 via the user interface of auser platform 140 as described above. The customized set of questions may be generated and presented for each learner such that each learner receives a mix of questions from codes and their counter-codes. The questions presented may also vary in difficulty depending on learner performance, such that learners see easier questions after making errors and harder questions after answering correctly. Learners may ultimately interact with progressively more difficult questions on each code or counter-code until they have demonstrated a desired level of proficiency on each one. The learner may continue to work on the topic until he or she has demonstrated proficiency on all codes and counter-codes within the topic. - In various example embodiments, the proficiency-
assessment module 210 ensures that: -
- learners are not “finished” until they have shown understanding of all necessary codes and their counter-codes.
- learners see gradually more difficult questions as they answer correctly, and are guided toward slightly easier questions when they are struggling.
- learners see questions from all codes in the topic in randomized fashion so that learners cannot “hack” their way to completion (e.g., by answering all questions on one code, where the answer always follows a particular pattern, and then answering all questions on another code, where the answer always follows the opposite pattern). In various example embodiments, the proficiency-
assessment module 210 can oscillate between codes, emphasizing codes on which the learner has not demonstrated proficiency (while also assessing any counter-codes, even if the learner has demonstrated proficiency on those), and still ensuring that there is no predictable pattern to the questions being displayed. - as learners gain proficiency on individual codes in a topic, they still see questions from a given code until they have demonstrated proficiency on that code and all its counter-codes. In a given topic, if there are two sets of codes and counter-codes, one called set A and one called set B (see
FIG. 6 for reference), learners will continue to see questions from all codes and counter-codes in Set A until learners demonstrate proficiency on all codes and counter-codes in Set A.
- As such, the proficiency-
assessment module 210 uses each topic's codes and counter-codes to generate customized sets of questions presented to learners via the user interface for assessing the skills and level of proficiency of each learner on one or more topics of instruction.FIGS. 5 and 6 illustrate two examples of this process for question generation. - Referring now to
FIG. 5 , an example illustrates the type of processing performed by the proficiency-assessment module 210. In this example,codes codes code 4. In this sample case, the proficiency-assessment module 210 generates questions from code 4 x% of the time and questions fromcodes assessment module 210 can ensure that when the learner answers a question correctly fromcode 4, the next question he or she encounters from that code will be more difficult than the previous one from that code. The same can be said for the learner encountering a less difficult question fromcode 4 subsequent to getting acode 4 question incorrect. The proficiency-assessment module 210 can continue to oscillate questions among all codes in the manner described above until the learner has demonstrated proficiency on all codes in the topic. - Referring now to
FIG. 6 , another example illustrates the processing performed by the proficiency-assessment module 210. As shown inFIG. 6 , the example shows that a particular learner has demonstrated proficiency ontopic codes codes codes codes assessment module 210 can generate questions from code 4 x% of the time and questions fromcode 3 the other (100-x)% of the time while ensuring that questions do not alternate in a predictable pattern. The proficiency-assessment module 210 can ensure that when the learner answers a question correctly fromcode 4, the next question he or she encounters from that code will be more difficult than the previous one from that code. The same can be said for the learner encountering a less difficult question fromcode 4 subsequent to getting acode 4 question incorrect. The proficiency-assessment module 210 can continue to oscillate questions among all codes in the manner described above until the learner has demonstrated proficiency on all codes in the topic. - Referring again to
FIG. 3 , the proficiency-driven feedback andimprovement system 200, with a proficiency-assessment module 210 and/or a proficiency-drivenfeedback module 230 therein, can be configured to perform the processing as described herein. Initially, the proficiency-driven feedback andimprovement system 200 can be configured to establish, by use of a data processor and thedata network 115, a network data connection with at least one of the plurality oflearner devices 120, at least one of the plurality of instructor/qualified expert devices 130, and optionally one or more of the plurality of onlineinstructional resources 135. The proficiency-driven feedback andimprovement system 200 can be configured to generate and display on a user interface of auser platform 140 information content and at least one user-selectable input object r icon associated with available user-selectable option elements of a pre-defined set of user-selectable option elements as described in more detail below. - In various example embodiments, through the proficiency-driven feedback and
improvement system 200, learners are guided through the process of submitting and/or editing work, practicing concepts that help them strengthen a particular skill or set of skills, evaluating the work of other learners (e.g., submitting input on a binary, ternary, or Likert scale, annotating or highlighting work, and submitting free-form comments), receiving feedback from other learners, and/or revising their own work as necessary. As shown inFIG. 7 , an example embodiment of the basic steps in this process: -
- enables learners to submit work or update and submit their work (process block 750).
- enables learners to practice skills through a series of exercises on relevant concepts. The exercises may be specifically configured for the skill level and/or prior performance of each learner. While learners practice, the platform assesses each learner's level of proficiency (process block 752).
- enables learners to evaluate the submissions of other learners using a guided set of criteria. The platform may only allow learners to evaluate work on the concepts on which they have proven proficient (process block 754).
- enables learners to review feedback from other learners and to submit revisions to their prior work (process block 756).
- As shown in
FIG. 8 , in various example embodiments of the proficiency-driven feedback andimprovement system 200, instructors/qualified experts are able to monitor learners' work and progress on skill attainment, offer feedback, view feedback provided by other learners, and assign revisions and/or grade submissions. In various example embodiments, instructors/qualified experts can create and configure customized assignments, enter or select prompts, and/or select criteria for assessing learners and/or specific skills to assess using the proficiency-driven feedback and improvement process, using the proficiency-driven feedback andimprovement system 200, via the user interface of auser platform 140. - Referring again to
FIG. 3 , the proficiency-driven feedback andimprovement system 200 may include a proficiency-drivenfeedback module 230. The proficiency-drivenfeedback module 230 may be configured to perform the processing as described herein. Initially, the proficiency-drivenfeedback module 230 may be configured to establish a data connection with at least one of the plurality oflearner user devices 120, at least one of the plurality of instructor/qualified expert devices 130, and optionally one or more of the plurality of onlineinstructional resources 135. The proficiency-drivenfeedback module 230 may be configured to generate and display on a user interface of auser platform 140 information content and at least one user-selectable input object or icon associated with available user-selectable option elements of a pre-defined set of user-selectable option elements as described in more detail below. - In many example embodiments, a proficiency-driven feedback and improvement process may also be used as part of the proficiency-driven feedback and
improvement system 200. The proficiency-drivenfeedback module 230, provided for this purpose, may be configured to determine and enforce a policy wherein a learner can only evaluate or provide feedback on those criteria on which he or she has demonstrated proficiency, helping to ensure that learners receive accurate and high-quality feedback. Proficiency can be evaluated using a variety of different techniques and processes as described herein. - In
FIG. 9 and other example embodiments of the proficiency-driven feedback andimprovement system 200, in order to demonstrate proficiency on a criterion, a learner must demonstrate proficiency on each topic associated with that criterion and all codes and counter-codes associated with each topic. If a learner has not demonstrated proficiency on all topics associated with a criterion, that criterion may be removed from the learner's peer evaluation experience. InFIG. 9 , for example, a learner who has demonstrated proficiency on topics 1-4, 6, and 7, but nottopic 5, may be enabled to provide feedback to other learners oncriterion criterion 3. - In some embodiments of the process implemented by proficiency-driven
feedback module 230, learners evaluate their own work using the same criteria that they would use to evaluate the work of other learners. These embodiments may be configured such that learners must evaluate their own work before the proficiency-drivenfeedback module 230 reveals feedback on their work from other learners. Some example embodiments of the proficiency-driven feedback andimprovement system 200 maintain a visually consistent experience and/or utilize similar language across the practice and evaluation phases, maximizing the transference of skills from practice to evaluation of other learners to self-evaluation. - As configured and directed by the user via a computer-generated user interface as described herein, various example embodiments may automatically: 1) present and enable selection and configuration of a plurality of criteria; 2) present and enable configuration and selection of a plurality of prompt options or other guiding questions; 3) manage a variety of other operations related to the generation, configuration, and execution of a system and method for implementing a computer proficiency-driven feedback and improvement platform. In various example embodiments, the computer-generated user interface can be illustrated in several screen snapshots as illustrated herein and described below.
-
FIGS. 10 through 13 illustrate various example user interface screen snapshots, implemented as a web application, that show the basic elements of the user interface for displaying data and receiving user inputs associated with the proficiency-driven feedback andimprovement system 200 in an example embodiment. Referring toFIGS. 10 and 11 , a sample user interface of an example embodiment is shown. Under the direction of the proficiency-driven feedback andimprovement system 200 of an example embodiment, a user/learner can be prompted with any of a customized set of questions and may respond with answers by interacting with the input objects as shown inFIGS. 10 and 11 . The examples shown inFIGS. 10 and 11 are described in more detail below. -
Learner Practice Experience Learner Feedback Experience Example 1 See FIG. 11 see FIG. 10 Instructions: Is the thesis Instructions: Is the thesis a fact or an opinion? a fact or an opinion? Exercise: Thesis statement Exercise: Thesis statement can be generated by the can be generated by a peer proficiency-driven feedback in the learner's class and and improvement system 200electronically routed to other learners for consideration. Learner action: Learner action: Select between two options: Select between two options: “Fact” and “Opinion” “Fact” and “Opinion” - Referring to
FIGS. 12 and 13 , another sample user interface of an example embodiment is shown. Under the direction of the proficiency-driven feedback andimprovement system 200 of an example embodiment, a user/learner can be prompted with any of a customized set of questions and may respond with answers by interacting with the input objects as shown inFIGS. 12 and 13 . The examples shown inFIGS. 13 and 14 are described in more detail below. -
Learner Practice Experience Learner Feedback Experience Example 2 See FIG. 13 see FIG. 12 Instructions: Highlight the Instructions: Highlight the vague words or phrases in vague sections of the thesis the thesis statement below statement above Exercise: Thesis statement Exercise: Thesis statement can be generated by the can be generated by another proficiency-assessment peer in the learner's class module 210 Learner Action: Learner Action: Click to highlight vague Click to highlight vague words in the thesis statement words in the thesis statement - It will be apparent to those of ordinary skill in the art in view of the disclosure herein that the web app implementation of the example user interfaces shown in
FIGS. 10 through 13 can be similarly implemented as a mobile application (app) interface or an interface for other interactive devices. The web app and the mobile app of various example embodiments can have a similar software architecture, which is described for various example embodiments herein. - Referring now to
FIG. 14 , anotherexample embodiment 103 of a system in which various embodiments may operate is illustrated. In the embodiment illustrated, thehost site 110 is shown to include the proficiency-driven feedback andimprovement system 200. The proficiency-driven feedback andimprovement system 200 is shown to include thefunctional components host site 110 may also include aweb server 904, having a web interface with which users may interact with thehost site 110 via a user interface or web interface. Thehost site 110 may also include an application programming interface (API) 902 with which thehost site 110 may interact with other network entities or resources on a programmatic or automated data transfer level. TheAPI 902 andweb interface 904 may be configured to interact with the proficiency-driven feedback andimprovement system 200 either directly or via aninterface 906. The proficiency-driven feedback andimprovement system 200 may be configured to access adatabase 112 either directly or via theinterface 906. - Referring now to
FIG. 15 , a processing flow diagram illustrates an example embodiment of a method implemented by the proficiency-driven feedback andimprovement system 200 as described herein. Themethod 2000 of an example embodiment includes: establishing, by use of a data processor, a data connection with a learner (processing block 2010); generating a distribution of questions to determine the learner's level of proficiency on a topic, the topic being composed of a set of codes and counter-codes (processing block 2012); determining the learner proficient on the topic after the learner has demonstrated proficiency on each code and counter-code within the topic (processing block 2014); and distributing questions to the learner on codes and counter-codes within the topic until the learner has demonstrated proficiency on each code and counter-code within the topic (processing block 2016). Themethod 2000 of an example embodiment further includes: establishing, by use of a data processor and a data network, a network data connection with a learner (processing block 2011); providing the learner feedback on work according to a set of criteria (processing block 2013); and ensuring that only users who have first demonstrated proficiency on one or more criteria in the set of criteria can give feedback on those criteria (processing block 2015). -
FIG. 16 shows a diagrammatic representation of a machine in the example form of a mobile computing and/orcommunication system 700 within which a set of instructions when executed and/or processing logic when activated may cause the machine to perform any one or more of the methodologies described and/or claimed herein. In alternative embodiments, the machine operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server or a client machine in server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine may be a personal computer (PC), a laptop computer, a tablet computing system, a Personal Digital Assistant (PDA), a cellular telephone, a smartphone, a web appliance, a set-top box (STB), a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) or activating processing logic that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” can also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions or processing logic to perform any one or more of the methodologies described and/or claimed herein. - The example mobile computing and/or
communication system 700 includes a data processor 702 (e.g., a System-on-a-Chip (SoC), general processing core, graphics core, and optionally other processing logic) and amemory 704, which can communicate with each other via a bus or otherdata transfer system 706. The mobile computing and/orcommunication system 700 may further include various input/output (I/O) devices and/orinterfaces 710, such as a touchscreen display, an audio jack, and optionally anetwork interface 712. In an example embodiment, thenetwork interface 712 can include one or more radio transceivers configured for compatibility with any one or more standard wireless and/or cellular protocols or access technologies (e.g., 2nd (2G), 2.5, 3rd (3G), 4th (4G) generation, and future generation radio access for cellular systems, Global System for Mobile communication (GSM), General Packet Radio Services (GPRS), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), LTE, CDMA2000, WLAN, Wireless Router (WR) mesh, and the like).Network interface 712 may also be configured for use with various other wired and/or wireless communication protocols, including TCP/IP, UDP, SIP, SMS, RTP, WAP, CDMA, TDMA, UMTS, UWB, WiFi, WiMax, Bluetooth™, IEEE 802.11x, and the like. In essence,network interface 712 may include or support virtually any wired and/or wireless communication mechanisms by which information may travel between the mobile computing and/orcommunication system 700 and another computing or communication system vianetwork 714. - The
memory 704 can represent a machine-readable medium on which is stored one or more sets of instructions, software, firmware, or other processing logic (e.g., logic 708) embodying any one or more of the methodologies or functions described and/or claimed herein. Thelogic 708, or a portion thereof, may also reside, completely or at least partially within theprocessor 702 during execution thereof by the mobile computing and/orcommunication system 700. As such, thememory 704 and theprocessor 702 may also constitute machine-readable media. Thelogic 708, or a portion thereof, may also be configured as processing logic or logic, at least a portion of which is partially implemented in hardware. Thelogic 708, or a portion thereof, may further be transmitted or received over anetwork 714 via thenetwork interface 712. While the machine-readable medium of an example embodiment can be a single medium, the term “machine-readable medium” should be taken to include a single non-transitory medium or multiple non-transitory media (e.g., a centralized or distributed database, and/or associated caches and computing systems) that stores the one or more sets of instructions. The term “machine-readable medium” can also be taken to include any non-transitory medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the various embodiments, or that is capable of storing, encoding or carrying data structures utilized by or associated with such a set of instructions. The term “machine-readable medium” can accordingly be taken to include, but not be limited to, solid-state memories, optical media, and magnetic media. - As described herein for various example embodiments, a system and method for implementing a proficiency-driven feedback and improvement platform are disclosed. In various example embodiments, a software application program is used to enable the development and presentation of information categories, information sub-categories, and information components in customizable views on the display screen of a computing or communication system, including mobile devices. As described above, in a variety of contexts, the proficiency-driven feedback and
improvement system 200 of an example embodiment can be configured to automatically obtain a variety of information from one or more 3rd party sites or network resources via a data network to facilitate the user experience of searching for desired information datasets, configuring a proficiency-driven feedback and improvement platform, and sharing information, all from the convenience of a computing device or a portable electronic device, such as a smartphone. This collection of particular user-selected information datasets and customized views has traditionally been possible only via multiple, personal interactions with a plurality of different parties at different locations. The embodiments as presently disclosed and claimed enable these disparate transactions to be integrated into a single set of electronic interactions with a mobile device or other computing device. As such, the various embodiments as described herein are necessarily rooted in computer and network technology and serve to improve these technologies when applied in the manner as presently claimed. In particular, the various embodiments described herein improve the use of computing device technology and mobile device technology in combination with data network technology in the context of a proficiency-driven feedback and improvement platform via electronic means. - The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment.
Claims (16)
1.-2. (canceled)
3. A method for network-enabled proficiency-driven feedback and improvement for learners, the method comprising:
establishing, by use of a data processor and a data network, a data connection with a learner;
generating a distribution of questions to determine the learner's level of proficiency on a criterion associated with one or more topics;
determining the learner proficient on the criterion after the learner has demonstrated proficiency by correctly answering a predetermined threshold of questions presented on the criterion, the learner determined proficient on the criterion being designated a proficient user; and
enabling the proficient user to provide, via the data network, feedback to other learners on the criterion for which the proficient user has demonstrated proficiency.
4. The method of claim 3 including enabling a qualified expert to provide feedback for the learner by presenting to the qualified expert a view of the work completed by the learner, a view of the proficiencies demonstrated by the learner, and a view of feedback provided by other proficient users.
5. The method of claim 3 including enabling a qualified expert to provide feedback for the learner by enabling the qualified expert to assign revisions to the learner, grade submissions of the learner, or create an assignment for the learner.
6. The method of claim 3 including enabling learners to self-evaluate their own work.
7. The method of claim 3 including creating an assignment containing the distribution of questions to determine the learner's proficiency on the one or more topics.
8. The method of claim 3 including enabling the learner to create an assignment containing the distribution of questions to enable the learner to determine their own proficiency on the one or more topics.
9. The method of claim 3 wherein the distribution of questions correspond to the learner's prior proficiency on codes and counter-codes corresponding to each topic of the one or more topics.
10. The method of claim 3 wherein the distribution of questions correspond to a relative frequency for assessing the learner's proficiency on codes and counter-codes corresponding to each topic of the one or more topics.
11. The method of claim 3 wherein the distribution of questions correspond to a difficulty of questions available on each code and counter-code corresponding to each topic of the one or more topics.
12. The method of claim 3 wherein the distribution of questions correspond to a grouping of codes and their counter-codes within each topic of the one or more topics.
13. The method of claim 3 including using a randomizing function to determine to which code or counter-code a next question of the distribution of questions corresponds.
14. The method of claim 3 including enabling the proficient user to provide feedback to other learners by submitting input on a binary, ternary, or Likert scale, annotating or highlighting work, or submitting free-form comments.
15. The method of claim 3 including enabling the proficient user to provide feedback to other learners by responding to prompts presented to the proficient user, the prompts requesting specific evaluations of the learner's proficiency on the one or more topics.
16. The method of claim 3 including enabling the proficient user to provide feedback to other learners by presenting to the proficient user a view of the work completed by one or more of the other learners, a view of the proficiencies demonstrated by the one or more of the other learners, and a view of feedback provided by other proficient users.
17. The method of claim 3 including enabling the proficient user to provide feedback to other learners by enabling the proficient user to assign revisions to one or more of the other learners, grade submissions of the one or more of the other learners, or create an assignment for the one or more of the other learners.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/731,904 US20200134260A1 (en) | 2017-09-16 | 2019-12-31 | System and method for implementing a proficiency-driven feedback and improvement platform |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/706,701 US10565304B2 (en) | 2017-09-16 | 2017-09-16 | System and method for implementing a proficiency-driven feedback and improvement platform |
US16/731,904 US20200134260A1 (en) | 2017-09-16 | 2019-12-31 | System and method for implementing a proficiency-driven feedback and improvement platform |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/706,701 Continuation US10565304B2 (en) | 2017-09-16 | 2017-09-16 | System and method for implementing a proficiency-driven feedback and improvement platform |
Publications (1)
Publication Number | Publication Date |
---|---|
US20200134260A1 true US20200134260A1 (en) | 2020-04-30 |
Family
ID=65720313
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/706,701 Active US10565304B2 (en) | 2017-09-16 | 2017-09-16 | System and method for implementing a proficiency-driven feedback and improvement platform |
US16/731,904 Abandoned US20200134260A1 (en) | 2017-09-16 | 2019-12-31 | System and method for implementing a proficiency-driven feedback and improvement platform |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/706,701 Active US10565304B2 (en) | 2017-09-16 | 2017-09-16 | System and method for implementing a proficiency-driven feedback and improvement platform |
Country Status (1)
Country | Link |
---|---|
US (2) | US10565304B2 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11430435B1 (en) * | 2018-12-13 | 2022-08-30 | Amazon Technologies, Inc. | Prompts for user feedback |
US11625388B2 (en) | 2019-08-29 | 2023-04-11 | Hartford Fire Insurance Company | System with task analysis framework display to facilitate update of electronic record information |
KR102320723B1 (en) * | 2019-12-20 | 2021-11-02 | 라인플러스 주식회사 | Method and system for verifying users |
US11417337B1 (en) * | 2021-08-12 | 2022-08-16 | Cresta Intelligence Inc. | Initiating conversation monitoring system action based on conversational content |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6424983B1 (en) | 1998-05-26 | 2002-07-23 | Global Information Research And Technologies, Llc | Spelling and grammar checking system |
US7628614B2 (en) * | 2004-08-23 | 2009-12-08 | Educational Testing Service | Method for estimating examinee attribute parameters in cognitive diagnosis models |
US8417581B2 (en) * | 2009-11-05 | 2013-04-09 | Lensoo Inc. | Method, system, and medium for enabling a user to access learning content via a single-window learner interface |
JP2011232871A (en) * | 2010-04-26 | 2011-11-17 | Sony Corp | Information processor, text selection method and program |
US9002700B2 (en) | 2010-05-13 | 2015-04-07 | Grammarly, Inc. | Systems and methods for advanced grammar checking |
US20130040275A1 (en) * | 2011-08-12 | 2013-02-14 | Pavan R. Gowda | Method for Educational Instruction |
US9460085B2 (en) * | 2013-12-09 | 2016-10-04 | International Business Machines Corporation | Testing and training a question-answering system |
US20180165653A1 (en) * | 2016-12-13 | 2018-06-14 | Coursera, Inc. | Online education platform including facilitated learning |
-
2017
- 2017-09-16 US US15/706,701 patent/US10565304B2/en active Active
-
2019
- 2019-12-31 US US16/731,904 patent/US20200134260A1/en not_active Abandoned
Also Published As
Publication number | Publication date |
---|---|
US10565304B2 (en) | 2020-02-18 |
US20190087404A1 (en) | 2019-03-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20200134260A1 (en) | System and method for implementing a proficiency-driven feedback and improvement platform | |
US11934770B2 (en) | System and methods for context specific annotation of electronic files | |
CN108255730A (en) | Software interface test approach, test equipment, storage medium and device | |
US20150121231A1 (en) | Systems and Methods for Interactively Presenting a Presentation to Viewers | |
US20130224716A1 (en) | Cloud-based multimedia teaching system, development method and interaction method thereof | |
US20160111018A1 (en) | Method and system for facilitating learning of a programming language | |
KR20210038971A (en) | Curriculum package development method and device, system, device and storage medium | |
CN111223015B (en) | Course recommendation method and device and terminal equipment | |
US10785325B1 (en) | Audience binning system and method for webcasting and on-line presentations | |
US20160062621A1 (en) | User interface for funnel analysis | |
Calvo et al. | Evaluation of accessibility barriers and learning features in m-learning chat applications for users with disabilities | |
AU2010203294A1 (en) | System for providing roadmaps for buidling proficiencies in skill areas | |
KR101380692B1 (en) | Apparatus for on-line study and method for the same | |
García et al. | Rethinking remote laboratories: Widgets and smart devices | |
Marković et al. | INSOS—educational system for teaching intelligent systems | |
JP2016156883A (en) | Display control program, display control method and display controller | |
US20150221229A1 (en) | Asynchronous online learning | |
KR101533760B1 (en) | System and method for generating quiz | |
Gil Parga et al. | Pedagogical design in education using augmented reality: a systematic review | |
US20200202739A1 (en) | Customized resources for correcting misconceptions | |
Pollanen et al. | MC2: mathematics classroom collaborator | |
JP2016156886A (en) | Display control program, display control method and display controller | |
CN111556156A (en) | Interaction control method, system, electronic device and computer-readable storage medium | |
CN111210673A (en) | Course data processing method and device, terminal equipment and computer storage medium | |
Che et al. | Collaborative Learning and Information Dissemination Based on Social and Mobile Computing |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
AS | Assignment |
Owner name: NOREDINK CORP., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SCHEUR, JEFF;KARLAN, JOCELYN;LEVEN, JOSHUA;SIGNING DATES FROM 20171004 TO 20190913;REEL/FRAME:054931/0487 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |