GB2392543A

GB2392543A - Teaching assessment based on comparing students work with database of exemplars.

Info

Publication number: GB2392543A
Application number: GB0220230A
Authority: GB
Inventors: Lyndon Arthur Fitzgerald; James Michael Owers; Myfanwy Powell
Original assignee: Research Machines PLC
Current assignee: Research Machines PLC
Priority date: 2002-08-30
Filing date: 2002-08-30
Publication date: 2004-03-03
Also published as: GB0220230D0

Abstract

A system for facilitating the input and storage of assessment data indicating whether one or more teaching objectives has been obtained by a student comprises an exemplar processor 18 arranged to retrieve 16 one or more exemplars of work 14 and a selector 24 by which a teacher may select an exemplar which best represents a student's work. Each exemplar is related to a set of teaching objectives and the exemplar processor is arranged to store 22, based on the exemplar selected, a record that the respective set of teaching objectives have been attained by the student.

Description

- 1 - TEACHING ASSESSMENT SYSTEM

FIELD OF THE INVENTION

The present invention relates to a teaching assessment system of the type operable on a computer or computer network, particularly to assist teachers in a classroom environment. BACKGROUND OF THE INVENTION

An integral part of teaching, particularly but not exclusively in a school environment, is assessing students. Most educational delivery follows a prescribed curriculum which is designed to deliver learning outcomes/objectives. It is usual after delivery to "test" if a student has achieved/mastered those objectives. This process is known as assessment. A learning outcome/ objective is the concept which is being taught by explanation using different expressions of that concept.

There are two types of assessment: Formative Assessment - this is done continuously as delivery happens and is used to fine-tune what a student sees next. i.e. It is used to inform the planning of the next piece of delivery.

Summative Assessment - this is done periodically e.g. at the end of a term/year and is an overall statement of what

level a student is working at.

- 2 For some subjects like Maths, simple normalised multiple choice tests can be used to provide good data automatically for both formative and summative assessment.

However, for some subjects it is impossible to create a deterministic test of this type and assessment must be based on a qualitative analysis such as: Examining a piece of work produced by a student and marking it against a marking guide.

Observing a student while he does something and monitor how he makes a certain decision and/or what techniques/though processes he uses.

It is this type of "qualitative" assessment for which the present invention is particularly useful.

We have appreciated the need to assist teachers in the input, storage and manipulation of data relating to assessment of student learning achievement. We have particularly appreciated the need for a simple interface by which teachers may enter data relating to students attainment of multiple objectives.

SUMMARY OF THE INVENTION

The invention is defined in the claims to which reference is now directed. Preferred features are set out in the dependent claims.

A system embodying the invention provides advantages of speed and simplicity of data entry thereby increasing the speed of processing assessment data.

- 3 BRIEF DESCRIPTION OF THE FIGURES

An embodiment of the invention will now be described, by way of example only, and with reference to the figures, in which: Figure 1: shows a textual example of how learning outcomes are linked to evidence of achievement of that outcome; Figure 2: shows a textual example of how learning outcomes are logically linked to teaching units; Figure 3: shows a textual example of how learning outcomes are logically linked to exemplars; Figure 4: shows further detail of one exemplar; and Figure 5: shows the main functional components of a system embodying the invention.

DESCRIPTION OF A PREFERRED EMBODIMENT

The system embodying the invention may be a dedicated arrangement of terminals and storage, or a general purpose computer network or standalone computer running software.

The details of implementation are not important, but the substance of the input, manipulation and extraction of data is, as will be described first with reference to Figure 5.

The system embodying the invention shown is a general purpose computer network comprising a central server and a plurality of client terminals. For simplicity, only one

4 - display and processor are shown, though it will be appreciated that a variety of logical arrangements are possible. The system as a whole provides an efficient interface for teachers to instruct assessment data to be stored in an assessment store 22 as a database showing, for each student, the objectives that they have attained as a result of a lesson or series of lessons. As previously mentioned, an objective is the abstract concept that is taught and can be expressed by one or more examples. Such concepts include skills, techniques, knowledge etc. that can be taught and tested. The term teaching or learning "objective" is well understood in the field of teaching and is used interchangeably herein with

the term leaching or learning "outcome". The "attainment" of an objective/outcome by a student means that they have understood and reached a sufficient level of competence in relation to that objective.

A teacher first uses the system to teach a subject. This is achieved by retrieving teaching material from a teaching unit store 12 to demonstrate objectives held in an objective repository 10 and showing these on a display.

A course is a years worth of a subject curriculum. The curriculum consists of a series of progressive learning objectives arranged into several strands/themes. The delivery of the learning objectives is in delivery units, with each unit comprising a plurality of lessons. Note, therefore, that the objectives in the objective repository 10 and the units of teaching material and unit store 12 are logically linked such that presentation of units teaches the underlying objectives.

Each unit has been designed to cover a series of learning objectives, starting with simpler skills (such as "can cut and paste text") and culminating with the application of those skills within an "open ended" integrated task that addresses higher order learning objectives (e.g. can format a document to match its intended audience). An example of how learning objectives/outcomes relate to units of activity is shown in Figure 2.

At the end of each unit a student will be assigned an integrated tasks' which has been designed to cover a different set of objectives which may also be covering different levels of achievement. Three levels of task have been designed, low, medium and high and each student is assigned one only. The student now does this task and the teacher may also observe etc. The teacher now needs to assess how well the student got on with his task.

The input of assessment data is achieved using data in an exemplar store 14 extracted by an extractor 16 and processed by an exemplar processor 18 for display on the display 20. A teacher first selects one of the activities that they have taught (shown logically in Figure 2). The extractor 16 then interrogates the exemplar store 14 and retrieves ''exemplars"' that match the integrated activities that the teacher has taught. Each exemplar comprises one or more "evidence of achievement" data items, as shown in Figure 1, each of which match each learning objective.

These data items contain some text, possibly a fragment of some work (e.g. a screen shot) and are designed to put the learning objective into a context that the teacher can recognize. They are designed in such a way that we can

- 6 say to the teacher.... "If your student produced something like this/did something like this then they have probably attained this objective".

The exemplar data items, shown in Figure 3, match each of the integrated activities (low, medium, high). Exemplars can include: One or more screen shots of actual work Text based explanations of what to look for.

Exemplars include data from each of the evidence of achievement data items that are covered by this activity.

The exemplar data items are thus example representations of the type of work that a student would produce if they have attained the objectives which each of the evidence of achievement data items represent. The teacher now browses these exemplars on the display 20 and double checks that the student achieved the things that the activity was testing. The teacher can choose the appropriate exemplar which best matches the work of the student being assessed and selects this using selector 24. Once selected, the student gets marked as '"assessed for this unit" and each of the learning objectives represented by the exemplar is automatically marked as achieved in the assessment store 22. Note, the database thus constructed holds a "set" of objectives for each student for each course arranged both by strand and by unit of delivery. These data entries get marked as achieved/unmarked as appropriate.

The teacher can now optionally switch to a "detailed view" and work at individual objective level to fine tune the individual "ticks" showing whether each objective is achieved thereby amending the data in the assessment store denoting whether each objective has been attained.

The advantage provided is that the teacher starts with the exemplar that is most like/closest to the level of the student's actual work. This then does some broad assessment after which the teacher can for certain students drill in and fine-tune by indicating at individual learning objective level whether each objective has been achieved.

One of the prime drivers for this design was speed of use for a busy teacher, i.e. with a few clicks the whole class can be assessed at the end of a unit (with suitable fine tuning where needed) and all this can be done by a non expert low confidence teacher/classroom assistant.

The teacher can also add text to each indication that an objective has been attained in relation to one student or a group of students to provide an alert. When the objective is next selected, by teaching a unit containing that objective for example, the alert is provided to the teacher. As ticks are built up along each strand, we automatically calculate "levels". Each strand has some ''marker posts" where, for example, you achieve level-1, level-2. An algorithm is used to decide what and how many ticks constitute attainment of that level (for example, if there

- 8 are seven objectives between the level-1 and level-2 posts, we may say that level-2 is achieved if you get the two key-objectives and at least three of the remaining five). Algorithms will be tuned differently for different strands. When we have a level calculated for each strand, we now create an overall level as an "average" of the level across the individual strands. The teacher can optionally override this automatically calculated overall level.

Note the "overall level" viewing/setting screen has been designed to visually remind the teacher what has been delivered as well as what has been achieved, using a double "thermometertt/'field gauge" type design metaphor.

In the example embodiment, the selector may be any appropriate input device, e.g. mouse, keyboard or touchscreen, and appropriate software drivers to instruct the exemplar processor.

Claims

9 - CLAIMS

1. A system for facilitating the input and storage of assessment data indicating whether a set of one or more teaching objectives have been attained by students, comprising: - an exemplar processor arranged to retrieve one or more exemplars of work each relating to a respective set of teaching objectives for presentation on a display; - a selector by which a teacher may select an exemplar shown on the display which best represents examples of student work; and - wherein the exemplar processor is arranged to store, for the exemplar selected, an indication that the respective set of teaching objectives has been attained by a given student or students.

2. A system according to claim 1, further comprising: - a store of data items, each data item being logically linked to one or more related teaching objectives; - an extractor arranged to retrieve the data items logically linked to the respective set of teaching objectives for the one or more exemplars; - wherein the exemplar processor is configured to receive the retrieved data items and to format the data items for presentation with the one or more respective exemplars.

- 10 3. A system according to claim 1, wherein the exemplar processor is further arranged to retrieve the indications that the set of objectives have been obtained for a student or students for presentation on a display, and the selector is further arranged such that a teacher may select individual objectives to modify the indication that each individual objective within the set has been obtained for each student or students.

4. A system according to any preceding claim, wherein the exemplar processor is further arranged to store text data input by a teacher associated with a given objective in relation to a student or students, and to provide an alert for display when that given objective is next selected. 5. A method of operating a computer interface for facilitating the input and storage of assessment data indicating whether a set of one or more teaching objectives have been attained by students, comprising: retrieving one or more exemplars of work each relating to a respective set of teaching objectives for presentation on a display; - displaying the one or more exemplars on a display; - receiving a selection input denoting one of the one or more exemplars in response to a user input selecting an exemplar shown on the display which best represents examples of student work; and - storing, for the exemplar selected, an indication that the respective set of teaching

- 11 objectives has been attained by a given student or students. 6. A method according to claim 5, wherein the retrieving step further comprises retrieving one or more data items logically linked to the respective set of teaching objectives for the one or more exemplars; and the displaying step comprises displaying the data items for that are logically linked to the respective set of teaching objectives with the respective exemplar.

7. A method according to claim 5 or 6, further comprising: - retrieving the stored indications that the set of objectives have been obtained for a student or students for presentation on a display, - presenting the stored indications on a display; and - modifying the stored indications of individual objectives to thereby modify the indication that each individual objective within the set has been obtained for each student or students in response to a further user input. 8. A method according to any of claims 5, 6 or 7, further comprising storing text data associated with a given objective input by a teacher in relation to a student or students, and providing an alert for display when that given objective is next selected.

9. A computer program which when executed on a computer undertakes the method of any of claims 5 to 8.