CN116975558A - Calculation thinking evaluation method based on multi-dimensional project reaction theory - Google Patents
Calculation thinking evaluation method based on multi-dimensional project reaction theory Download PDFInfo
- Publication number
- CN116975558A CN116975558A CN202310780976.4A CN202310780976A CN116975558A CN 116975558 A CN116975558 A CN 116975558A CN 202310780976 A CN202310780976 A CN 202310780976A CN 116975558 A CN116975558 A CN 116975558A
- Authority
- CN
- China
- Prior art keywords
- questions
- thinking
- dimensional
- index
- parameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 38
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 28
- 238000004364 calculation method Methods 0.000 title description 12
- 238000000034 method Methods 0.000 claims abstract description 39
- 239000013598 vector Substances 0.000 claims abstract description 17
- 238000012360 testing method Methods 0.000 claims description 46
- 238000012549 training Methods 0.000 claims description 30
- 238000004422 calculation algorithm Methods 0.000 claims description 24
- 238000013461 design Methods 0.000 claims description 14
- 230000004069 differentiation Effects 0.000 claims description 7
- 238000007689 inspection Methods 0.000 claims description 6
- 238000007476 Maximum Likelihood Methods 0.000 claims description 5
- 230000004044 response Effects 0.000 claims description 5
- 238000000354 decomposition reaction Methods 0.000 claims description 4
- 238000000546 chi-square test Methods 0.000 claims description 3
- 238000013508 migration Methods 0.000 claims description 3
- 230000005012 migration Effects 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000004590 computer program Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 10
- 238000004458 analytical method Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000013210 evaluation model Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012731 temporal analysis Methods 0.000 description 1
- 238000000700 time series analysis Methods 0.000 description 1
- 238000011269 treatment regimen Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Analysis (AREA)
- Evolutionary Biology (AREA)
- Mathematical Physics (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Mathematics (AREA)
- Mathematical Optimization (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Operations Research (AREA)
- Probability & Statistics with Applications (AREA)
- Artificial Intelligence (AREA)
- Algebra (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
According to the method for evaluating the computing thinking based on the multi-dimensional project reaction theory, the answer data are analyzed and processed based on the multi-dimensional project reaction theory, the computing thinking level of the tested person is calculated, and the evaluation results of the computing thinking of the tested person under the condition of different m-dimensional capability vectors are displayed according to a preset form. The method can evaluate the capability level of the individual in terms of computing thinking, help the individual find out the advantages and the disadvantages of the individual in terms of computing thinking, and further formulate a proper strategy for learning and improving the computing thinking capability.
Description
Technical Field
The application relates to the technical field of computer data intelligent processing, in particular to a computing thinking evaluation method based on a multi-dimensional project reaction theory.
Background
Computational Thinking (CT) is a problem-solving process that people create solutions and treatment strategies based on thinking habits and thinking patterns formed by computer science, including many features, such as logic ordering and analyzing data, using a series of ordered steps (or algorithms), such as confidently handling complexity and open problems. CT is critical to the development of computer applications, but it can also be used to support problem resolution in all disciplines, including mathematics, science, and humanity. The study of the cultivation of the calculation thinking is not separated from the study of the evaluation of the calculation thinking. Since the calculation thinking is hidden and cannot be directly measured by related tools like height, weight, temperature and the like, how to evaluate the calculation thinking and how to judge that the student has related calculation thinking and the degree to which the student reaches can be measured from which aspects and indexes, and the premise and the urgent need of researching the calculation thinking culture are formed.
Evaluation modes and ways of computing thinking are various, such as: in various scenarios, student's specific performance is observed, analyzed and evaluated by student works, student's computational thinking is examined by questionnaires or test questions, and the like.
The current construction process of the computing thinking evaluation index system generally comprises the steps of firstly constructing an index system, generally selecting a part of indexes as first-level indexes according to the opinion of an expert, then further decomposing each first-level index into second-level indexes, and then determining the weights of the indexes from the second-level indexes to the third-level indexes.
The evaluation scheme should not only be high representative, but also easy to understand and use and have strong operability. However, in the computing thinking evaluation index system, the total number of indexes at each level may be large, and the difference between the indexes may be large, so that the final selection of the indexes is difficult.
Therefore, there is a need to develop an efficient and accurate computing thinking evaluation method.
Disclosure of Invention
The application mainly aims to provide a computing thinking evaluation method based on a multi-dimensional project reaction theory so as to solve the technical problems.
In order to achieve the above object, the application provides a computing thinking assessment method based on a multi-dimensional project reaction theory, comprising the steps of,
s1, evaluating task design, specifically comprising the following steps:
s11, distributing evaluation tasks comprising W questions to N known training testers for testing and obtaining answer data of the known training testers; the evaluation task comprises a plurality of preset evaluation task question types, and each question type corresponds to a score;
s12, constructing a 3PL model of the following formula according to the preset dimension of the computing thinking ability and the answer data of the known training testers,
wherein P (X) ij ＝1|θ i ) Representing the probability that the ith said known training tester correctly answers the jth question; x is X ij Indicating the response of the ith said known training tester to the jth question, if the answer is correct, x ij =1, otherwise x ij =0; the answer data of the known training testers is X= (X) ij ) N×W ；θ i ＝(θ i1 ,θ i2 ,……θ im ) ' represents the m-dimensional capability vector of the i-th known training tester; a, a j ＝(a j1 ,a j2 ,……a jm ) ' an m-dimensional discrimination vector representing the j-th topic; d, d j Is a question difficulty parameter representing a j-th question; c j A lower asymptotic parameter for a question representing a j-th question; d is a model adjustment parameter;
s13, performing parameter estimation on the 3PL model by using a quasi-Monte Carlo expectation maximization algorithm to obtain a project parameter delta corresponding to each topic j ＝(a j ,d j ,c j )；
S14, according to the a j And d j Combining the W questions to obtain a plurality of sets of test papers with the differentiation degree and the difficulty meeting preset conditions;
s2, acquiring test paper test results of the testee, and according to the project parameters delta obtained in the step S13 j ＝(a j ,d j ,c j ) And a 3PL model in step S12, obtaining an m-dimensional capacity vector of the tested person.
Preferably, the types of the questions comprise single choice questions, 1-division questions, blank questions, 1-division questions and 1-division questions, and the questions are judged and 1-division questions are 1-division questions; the difficulty of the test paper comprises four types of entrance level, step level, high hand level and intelligent level, and the problem that the problem difficulty parameter is larger than the preset value accounts for 10%,20%,30% and 40% of the problem proportion in the four types of test paper; the number of questions of each test paper is between 20 and 25 questions.
Preferably, the step S13 includes:
s131, according to the answer data X= (X) of the known training testers ij ) N×W Estimating the project parameter delta of the 3PL model j ＝(a j ,d j ,c j ) Is set to an initial value of (1);
s132 according to formula P (θ=μ k |Λ)＝P(μ k |λ k )＝λ k And the initial value, the m-dimensional capacity vectors of all the known training testers are recorded as theta= (theta) 1 ,……,θ N ) Approximating the m-dimensional capability vector θ of the known training tester in the 3PL model as a discrete hidden variable, and representing the continuous hidden variable θ as k known discrete values μ 1 ，……，μ k And the corresponding unknown probability is denoted as lambda k With Λ= (λ) 1 ，λ 2 ，λ 3 ，……，λ k ) A distribution parameter representing a hidden variable; then estimating the hidden variable value of the missing data, and re-matching the project parameter delta according to the hidden variable value and the initial value j ＝(a j ,d j ,c j ) Estimating to obtain an iteration value;
s133, taking the iteration value as a new initial value and returning to the step S131 for the next iteration;
s134, judging whether iteration reaches a preset condition, if so, ending; if not, continuing iteration.
Preferably, the step S132 includes:
e, step E: the conditional probability expectation of the joint distribution is calculated as follows:
indicating all student competence values as mu k Sum of conditional probabilities of->Indicating all student competence values as mu k And the sum of the conditional probabilities of the questions by the answers, t is the iteration number;
m step: updating the parameter values according to the expected potential capability values by using a maximum likelihood estimation method, and finding the parameter values maximizing the likelihood function by using a numerical optimization algorithm, wherein the formula is as follows:
preferably, the method comprises, among other things,
preferably, the preset conditions in step S134 include:
whether the iteration number reaches the preset iteration number or not;
and/or the number of the groups of groups,
the difference of log likelihood function values of two adjacent iterations is smaller than a set threshold.
Preferably, the step S134 further includes a step S135 of model fitting test: and carrying out fitting inspection on the evaluation model, and inspecting whether the degree of fitting data of the model is reasonable or not.
8. The method of claim 7, wherein the model fitting test comprises χ2 test, RMSEA, CFI, or SRMR.
Preferably, said d j The formula for calculating the multidimensional difficulty coefficient is satisfied:wherein,,MDISC j is a multi-dimensional differentiation index, MDIFF j Multidimensional difficulty coefficient for j-th question, MDIFF j The transformed multidimensional difficulty coefficient is represented, f is a scoring standard, and f=1.
Preferably, the preset computing thinking multi-capability dimension comprises a plurality of primary dimension indexes, wherein the primary dimension indexes comprise concept knowledge indexes, problem exploration indexes and algorithm thinking indexes; the conceptual knowledge index comprises two secondary dimension indexes of a definition knowledge index and an operability knowledge index; the algorithm thinking index comprises two secondary indexes of an algorithm understanding index and an algorithm design index; the problems are explored into three secondary dimension indexes of an index abstract index, a decomposition index and a migration index
According to the method for evaluating the computing thinking based on the multi-dimensional project reaction theory, the answer data are analyzed and processed based on the multi-dimensional project reaction theory, the computing thinking level of a tested person is calculated, the questions of each set of test paper are participated by a large number of testers after the design is finished, the answer data of the participators are subjected to a 3PL model of the multi-dimensional project reaction theory, the parameter information of the test paper is obtained through a quasi Monte Carlo expectation maximization algorithm (QMC), and the most suitable questions are selected as the questions of the test paper with different grades through data index analysis and priority; the answer data of the testers are brought into a 3-parameter Logistic model of the multidimensional project reaction theory, and the capability parameters of the students can be obtained by few iterations because the test question parameters are estimated, namely the calculation thinking level of the students is obtained. The method can evaluate the capability level of the individual in terms of computing thinking, help the individual find out the advantages and the disadvantages of the individual in terms of computing thinking, and further formulate a proper strategy for learning and improving the computing thinking capability.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application. It is evident that the drawings in the following description are only examples, from which other drawings can be obtained by a person skilled in the art without the inventive effort. In the drawings:
fig. 1 is a flowchart illustrating steps of an evaluation task design of a computing thinking evaluation method based on a multi-dimensional project response theory according to an embodiment of the application.
Fig. 2 is a schematic flow chart of step S13 of the evaluation task design of the computing thinking evaluation method based on the multi-dimensional project reaction theory in an embodiment of the application.
Fig. 3 is a schematic diagram of a hardware structure for running a computing thinking assessment method based on a multi-dimensional project response theory according to an embodiment of the application.
The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
The technical problems solved by the embodiments of the present application, the technical schemes adopted and the technical effects achieved are clearly and completely described below with reference to the accompanying drawings and the specific embodiments. It will be apparent that the described embodiments are merely some, but not all embodiments of the application. All other equivalent or obvious modifications of embodiments can be made by those skilled in the art based on the embodiments of this application without departing from the scope of the application. The embodiments of the application may be embodied in a number of different ways, which are defined and covered in the claims.
In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding. It may be evident, however, that the present application may be practiced without these specific details.
It should be noted that, without explicit limitation or conflict, the embodiments of the present application and the technical features thereof may be combined with each other to form a technical solution.
Referring to fig. 1-2 together, the application provides a computing thinking evaluation method based on a multi-dimensional project reaction theory, which comprises the steps of,
s1, evaluating task design;
s2, acquiring test paper test results of the testee, and obtaining an m-dimensional capacity vector of the testee.
Referring to fig. 1 together, step S1 specifically includes S11 to S14.
S11, distributing evaluation tasks comprising W questions to N known training testers for testing and obtaining answer data of the known training testers; the evaluation task comprises a plurality of preset evaluation task question types, and each question type corresponds to a score.
Preferably, the types of the questions comprise single choice questions, 1-division questions, blank questions, 1-division questions and 1-division questions, and the questions are judged and 1-division questions are 1-division questions; the difficulty of the test paper comprises four types of entrance level, step level, high hand level and intelligent level, and the problem that the problem difficulty parameter is larger than the preset value accounts for 10%,20%,30% and 40% of the problem proportion in the four types of test paper; the number of questions of each test paper is between 20 and 25 questions. The differentiation parameters ensure that each set of test paper has the full capability of examining and calculating thinking (namely, secondary indexes). For example, a topic is considered too difficult when the difficulty factor is greater than 1.8, and too easy when less than-1.8. The problems are 10%,20%,30% and 40% of the questions of four test papers of entry level, step level, high-hand level and intelligent level.
S12, constructing a 3PL model of the following formula according to the preset dimension of the computing thinking ability and the answer data of the known training testers,
wherein P (X) ij ＝1|θ i ) Representing the probability that the ith said known training tester correctly answers the jth question; x is X ij Indicating the response of the ith said known training tester to the jth question, if the answer is correct, x ij =1, otherwise x ij =0; the answer data of the known training testers is X= (X) ij ) N×W ；θ i ＝(θ i1 ,θ i2 ,……θ im ) ' represents the m-dimensional capability vector of the i-th known training tester; a, a j ＝(a j1 ,a j2 ,……a jm ) ' an m-dimensional discrimination vector representing the j-th topic; d, d j Is a question difficulty parameter representing a j-th question; c j A lower asymptotic parameter for a question representing a j-th question; d is a model tuning parameter (generally 1.7).
Preferably, said d j The formula for calculating the multidimensional difficulty coefficient is satisfied:wherein,,MDISC j is a multi-dimensional differentiation index, MDIFF j Multi-dimensional difficulty coefficient for j-th topic, MDIFF j The transformed multidimensional difficulty coefficient is represented, f is a scoring standard, and f=1.
The purpose of evaluation of the computing thinking ability is to evaluate the ability level of an individual in terms of computing thinking, help the individual find out the advantages and the disadvantages of the individual in terms of computing thinking, and further formulate a proper strategy for learning and improving the computing thinking ability.
The composition of the calculated thinking elements is generally analyzed by using a word frequency statistical method, firstly, text data related to the calculated thinking is collected, then the text is cleaned, useless symbols, stop words and the like are removed, and secondly, the text is divided into words. And counting the occurrence times of each word in the text to obtain a word frequency table. Finally, ordering word frequency tables, determining the component elements of the computing thinking according to the occurrence frequency, classifying the computing thinking elements with similar investigation types into one class, and replacing the computing thinking elements with more accurate words as shown in the following table 1:
table 1 calculation of thought evaluation index System
According to the characteristics of the first-level index in the evaluation index system for calculating the thinking capability, the evaluation data for calculating the thinking capability is generally collected through three ways, so that the data can be multi-sourced. Selecting the question type and the difficulty of the evaluation task: the questions of the evaluation task design of the computing thinking are classified into the selected questions, the blank-filling questions, the judgment questions, the difficulty and the like very easily, generally, very difficultly, and the corresponding question number proportion can be 2:3:3:1:1.
(1) The concept knowledge has basic knowledge characteristics, so that a test question evaluation method is adopted to evaluate the mastery level of students on definition knowledge and operability knowledge, which is one of sources of evaluation data. Developing and developing a question library to generate a set of random test questions comprising questions related to definition knowledge and questions related to operability knowledge, for collecting scores of learners on definition knowledge and operability knowledge;
(2) The problem exploration and algorithm thinking have practical properties, so that the method of programming questions is adopted to collect evaluation data, and the capability level of learner problem exploration and algorithm thinking on secondary indexes is obtained through analysis of a program, and is the second source of the evaluation data. Specifically, a mapping relation between programming and computing thinking is established, programming characteristics of a learner are mined based on the mapping relation, and scores of the learner in terms of abstraction and decomposition, module definition, module design, logic, control and construction are obtained.
Preferably, the computing thinking ability index includes a plurality of primary dimension indexes, and the primary dimension indexes include a concept knowledge index, a problem exploration index and an algorithm thinking index.
Preferably, the concept knowledge index comprises two secondary dimension indexes of a definition knowledge index and an operability knowledge index; the algorithm thinking index comprises two secondary indexes of an algorithm understanding index and an algorithm design index; the problems explore three secondary dimension indexes, namely an index abstract index, a decomposition index and a migration index.
S13, performing parameter estimation on the 3PL model by using a quasi-Monte Carlo expectation maximization algorithm to obtain a project parameter delta corresponding to each topic j ＝(a j ,d j ,c j )。
Preferably, referring to fig. 2, the step S13 may include S131 to S135.
S131, according to the answer data X= (X) of the known training testers ij ) N×W Estimating the project parameter delta of the 3PL model j ＝(a j ,d j ,c j ) Is set to be a constant value.
Specifically, the method can comprise initializing a difficulty parameter d of the test question j Discrimination parameter a j And guess degree parameter c j Is a function of the estimated value of (2); initializing an estimated value of a capacity distribution parameter gamma of a student; let iteration number t=0.
S132 according to formula P (θ=μ k |Λ)＝P(μ k |λ k )＝λ k And the initial value, the m-dimensional capacity vectors of all the known training testers are recorded as theta= (theta) 1 ,……,θ N ) Approximating the m-dimensional capability vector θ of the known training tester in the 3PL model as a discrete hidden variable, and representing the continuous hidden variable θ as k known discrete values μ 1 ，……，μ k And the corresponding unknown probability is denoted as lambda k With Λ= (λ) 1 ，λ 2 ，λ 3 ，……，λ k ) A distribution parameter representing a hidden variable; then estimating the hidden variable value of the missing data, and re-matching the project parameter delta according to the hidden variable value and the initial value j ＝(a j ,d j ,c j ) And estimating to obtain an iteration value.
Preferably, the step S132 includes:
e, step E: the conditional probability expectation of the joint distribution is calculated as follows:
indicating all student competence values as mu k Sum of conditional probabilities of->Indicating all student competence values as mu k And the sum of the conditional probabilities of the questions by the answers, t is the iteration number;
m step: updating the parameter values according to the expected potential capability values by using a maximum likelihood estimation method, and finding the parameter values maximizing the likelihood function by using a numerical optimization algorithm, wherein the formula is as follows:
wherein the formula in the M step is calculated as follows:
wherein t is the iteration number.
Because the project parameters and the student capacity distribution parameters are mutually independent and do not influence each other in the process of carrying out maximum likelihood estimation, the equation can be split into two parts, and the maximum likelihood estimation is carried out respectively.
δ (t+1) ＝argmaxφ(δ)+φ(Λ)
Calculate phi (delta):
calculate phi (Λ):
δ(t+1) δ (t+1) =argmax Φ (δ) +Φ (Λ) is then converted into:
preferably, the method comprises, among other things,
s133, taking the iteration value as a new initial value and returning to the step S131 for the next iteration;
s134, judging whether iteration reaches a preset condition, if so, ending; if not, continuing iteration. Preferably, the preset conditions in step S134 include: whether the iteration number reaches the preset iteration number or not; and/or the difference of log likelihood function values of two adjacent iterations is smaller than a set threshold.
Preferably, the step S134 further includes a step S135 of model fitting test: and carrying out fitting inspection on the evaluation model, and inspecting whether the degree of fitting data of the model is reasonable or not. Wherein the model fitting test comprises a χ2 test, RMSEA, CFI, or SRMR.
Fitting inspection is carried out on the estimated model, whether the degree of fitting data of the model is reasonable or not is inspected, and common inspection indexes comprise X2 inspection, RMSEA, CFI, SRMR and the like. χ2 verifies model fitness by comparing the differences between the observed data and the model estimates. The smaller the χ2 value, the better, but generally it is not possible to deviate entirely from the observed data; the RMSEA is an index for measuring the fitting degree of the model according to a fitting degree measurement function (FitFunction), the number of free parameters in the model is considered, and for the condition that the fitting degree of the model is good, the RMSEA value is close to 0, and the fitting degree of the model is good when the RMSEA value is smaller than 0.05; CFI is an abbreviation for relative fitting index (compartivefitndex), which compares the fitting degree of a hypothetical model to that of an independent model, the closer it is to 1, indicating that the better the fitting degree of the model, the better the fitting degree when the CFI value is greater than 0.95 is generally considered; SRMR is the root mean square of the normalized mean residual, which is used to measure the fitness of the model, with smaller SRMR indicating better fitness, and with SRMR values less than 0.08, it is generally considered that fitness is good.
S14, according to the a j And d j And combining the W questions to obtain a plurality of sets of test papers with the differentiation degree and the difficulty meeting preset conditions.
S2, acquiring test paper test results of the testee, and according to the project parameters delta obtained in the step S13 j ＝(a j ,d j ,c j ) And a 3PL model in step S12, obtaining an m-dimensional capacity vector of the tested person. Wherein, the test paper is one of the sets of test paper obtained in the step S14.
In the step S2, since the test item design module has basically estimated the parameter size of the item, we can put the answer data of the tested person into the model to further estimate the capability value θ of the tested person. We can expect a posterior estimate with bayesian, without requiring iteration. The formula is as follows:
discretization (numerical score calculation)
Thus, an estimated capacity of the subject can be obtained.
In the step S2, when outputting the result, the result format may output the evaluation result in the form of text, graphics, etc., so as to facilitate the user to view and understand. For example, for capability parameters in a multi-dimensional project reaction theoretical model, the capability parameters can be output in a table or chart form to show the scoring condition of different capabilities. In addition to the results themselves, the results interpretation needs to provide the corresponding interpretation and explanation for the users, so that the users can better understand the meaning of the results. For example, for a score of a certain capability, a corresponding explanation may be given telling the user the meaning of the capability and the corresponding assessment task, etc. In the result analysis, the result can be further analyzed by a data analysis method, and rules and trends in the result analysis can be mined. For example, for a trend of a certain ability score, a time series analysis method may be used to analyze, and corresponding conclusions and suggestions are made. The user interaction can consider that the result output module interacts with other modules, and better user experience is provided. For example, a user may obtain more detailed result information through interaction with the result output module, or perform specific data processing and analysis, etc.
According to the method for evaluating the computing thinking based on the multi-dimensional project reaction theory, the answer data are analyzed and processed based on the multi-dimensional project reaction theory, the computing thinking level of a tested person is calculated, the questions of each set of test paper are participated by a large number of testers after the design is finished, the answer data of the participators are subjected to a 3PL model of the multi-dimensional project reaction theory, the parameter information of the test paper is obtained through a quasi Monte Carlo expectation maximization algorithm (QMC), and the most suitable questions are selected as the questions of the test paper with different grades through data index analysis and priority; the answer data of the testers are brought into a 3-parameter Logistic model of the multidimensional project reaction theory, and the capability parameters of the students can be obtained by few iterations because the test question parameters are estimated, namely the calculation thinking level of the students is obtained. The method can evaluate the capability level of the individual in terms of computing thinking, help the individual find out the advantages and the disadvantages of the individual in terms of computing thinking, and further formulate a proper strategy for learning and improving the computing thinking capability.
Fig. 3 is a schematic diagram of a hardware structure for running a computing thinking assessment method based on a multi-dimensional project reaction theory according to an embodiment of the present application. As shown in fig. 3, this embodiment/computer 6 includes: a processor 60, a memory 61 and a computer program 62 stored in said memory 61 and executable on said processor 60, for example a program running a computational thinking assessment method based on a BP neural network fused with multisource data. The processor 60, when executing the computer program 62, implements the steps described above in the embodiments of the method for computing thinking evaluation based on the fusion of BP neural network and multi-source data. Alternatively, the processor 60, when executing the computer program 62, performs the functions of the modules/units of the apparatus embodiments described above.
Illustratively, the computer program 62 may be partitioned into one or more modules/units that are stored in the memory 61 and executed by the processor 60 to complete the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing the specified functions, which instruction segments are used to describe the execution of the computer program 62 in the computer 6.
The computer 6 may be a desktop computer, a notebook computer, a palm computer, a cloud server, or other computing devices. The computer 6 device may include, but is not limited to, a processor 60, a memory 61. It will be appreciated by those skilled in the art that fig. 3 is merely an example of the computer 6 and is not limiting of the computer 6, and may include more or fewer components than shown, or may combine certain components, or different components, e.g., the computer 6 may also include input and output devices, network access devices, buses, etc.
The processor 60 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf Programmable gate arrays (FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 61 may be an internal storage unit of the computer 6, such as a hard disk or a memory of the computer 6. The memory 61 may be an external storage device of the computer 6, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided in the terminal device. Further, the memory 61 may also include both an internal storage unit and an external storage device of the computer 6. The memory 61 is used for storing the computer program and other programs and data required by the terminal device. The memory 61 may also be used for temporarily storing data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other manners. For example, the apparatus/terminal device embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated modules/units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. . Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, randomAccess Memory), an electrical carrier signal, a telecommunication signal, a software distribution medium, and so forth. It should be noted that the computer readable medium contains content that can be appropriately scaled according to the requirements of jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is subject to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.
The above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.
Claims (10)
1. A computing thinking evaluation method based on a multi-dimensional project reaction theory is characterized by comprising the steps of,
s1, evaluating task design, specifically comprising the following steps:
s11, distributing evaluation tasks comprising W questions to N known training testers for testing and obtaining answer data of the known training testers; the evaluation task comprises a plurality of preset evaluation task question types, and each question type corresponds to a score;
s12, constructing a 3PL model of the following formula according to the preset dimension of the computing thinking ability and the answer data of the known training testers,
wherein P (X) ij ＝1|θ i ) Representing the probability of the ith said known training tester correctly answering the jth question；X ij Indicating the response of the ith said known training tester to the jth question, if the answer is correct, x ij =1, otherwise x ij =0; the answer data of the known training testers is X= (X) ij ) N×W ；θ i ＝(θ i1 ,θ i2 ,……θ im ) ' represents the m-dimensional capability vector of the i-th known training tester; a, a j ＝(a j1 ,a j2 ,……a jm ) ' an m-dimensional discrimination vector representing the j-th topic; d, d j Is a question difficulty parameter representing a j-th question; c j A lower asymptotic parameter for a question representing a j-th question; d is a model adjustment parameter;
s13, performing parameter estimation on the 3PL model by using a quasi-Monte Carlo expectation maximization algorithm to obtain a project parameter delta corresponding to each topic j ＝(a j ,d j ,c j )；
S14, according to the a j And d j Combining the W questions to obtain a plurality of sets of test papers with the differentiation degree and the difficulty meeting preset conditions;
s2, acquiring test paper test results of the testee, and according to the project parameters delta obtained in the step S13 j ＝(a j ,d j ,c j ) And a 3PL model in step S12, obtaining an m-dimensional capacity vector of the tested person.
2. The method for evaluating the computing thinking based on the multi-dimensional project reaction theory according to claim 1, wherein the types of the questions comprise single choice questions, 1-minute questions, blank questions, 1-minute questions, judgment questions and 1-minute questions; the difficulty of the test paper comprises four types of entrance level, step level, high hand level and intelligent level, and the problem that the problem difficulty parameter is larger than the preset value accounts for 10%,20%,30% and 40% of the problem proportion in the four types of test paper; the number of questions of each test paper is between 20 and 25 questions.
3. The method for evaluating the computing thinking based on the theory of multi-dimensional project reaction according to claim 1, wherein said step S13 comprises:
s131, according to the answer data X= (X) of the known training testers ij ) N×W Estimating the project parameter delta of the 3PL model j ＝(a j ,d j ,c j ) Is set to an initial value of (1);
s132 according to formula P (θ=μ k |Λ)＝P(μ k |λ k )＝λ k And the initial value, the m-dimensional capacity vectors of all the known training testers are recorded as theta= (theta) 1 ,……,θ N ) Approximating the m-dimensional capability vector θ of the known training tester in the 3PL model as a discrete hidden variable, and representing the continuous hidden variable θ as k known discrete values μ 1 ，……，μ k And the corresponding unknown probability is denoted as lambda k With Λ= (λ) 1 ，λ 2 ，λ 3 ，……，λ k ) A distribution parameter representing a hidden variable; then estimating the hidden variable value of the missing data, and re-matching the project parameter delta according to the hidden variable value and the initial value j ＝(a j ,d j ,c j ) Estimating to obtain an iteration value;
s133, taking the iteration value as a new initial value and returning to the step S131 for the next iteration;
s134, judging whether iteration reaches a preset condition, if so, ending; if not, continuing iteration.
4. The method for computing thinking assessment based on multi-dimensional project reaction theory according to claim 3, wherein the step S132 comprises:
e, step E: the conditional probability expectation of the joint distribution is calculated as follows:
indicating all student competence values as mu k Sum of conditional probabilities of->Indicating all student competence values as mu k And the sum of the conditional probabilities of the questions by the answers, t is the iteration number;
m step: updating the parameter values according to the expected potential capability values by using a maximum likelihood estimation method, and finding the parameter values maximizing the likelihood function by using a numerical optimization algorithm, wherein the formula is as follows:
5. the method for evaluating computational thinking based on multi-dimensional project reaction theory according to claim 4, wherein,
6. the method for evaluating computational thinking based on multi-dimensional project reaction theory according to claim 3, wherein the preset conditions in step S134 include:
whether the iteration number reaches the preset iteration number or not;
and/or the number of the groups of groups,
the difference of log likelihood function values of two adjacent iterations is smaller than a set threshold.
7. The method for evaluating the computing thinking based on the multi-dimensional project reaction theory according to claim 3, wherein the step S134 further comprises a step S135 of model fitting test: and carrying out fitting inspection on the 3PL model, and inspecting whether the degree of fitting data of the model is reasonable or not.
8. The method of claim 7, wherein the model fitting test comprises χ2 test, RMSEA, CFI, or SRMR.
9. The method for computing thinking assessment based on multi-dimensional project reaction theory according to claim 1, wherein d j The formula for calculating the multidimensional difficulty coefficient is satisfied:wherein (1)>MDISC j Is a multi-dimensional differentiation index, MDIFF j Multidimensional difficulty coefficient for j-th question, MDIFF j The transformed multidimensional difficulty coefficient is represented, f is a scoring standard, and f=1.
10. The method for evaluating the computing thinking based on the multi-dimensional project reaction theory according to claim 1, wherein the preset computing thinking multi-capability dimension comprises a plurality of primary dimension indexes, and the primary dimension indexes comprise concept knowledge indexes, problem exploration indexes and algorithm thinking indexes; the conceptual knowledge index comprises two secondary dimension indexes of a definition knowledge index and an operability knowledge index; the algorithm thinking index comprises two secondary indexes of an algorithm understanding index and an algorithm design index; the problems explore three secondary dimension indexes, namely an index abstract index, a decomposition index and a migration index.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310780976.4A CN116975558A (en) | 2023-06-29 | 2023-06-29 | Calculation thinking evaluation method based on multi-dimensional project reaction theory |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310780976.4A CN116975558A (en) | 2023-06-29 | 2023-06-29 | Calculation thinking evaluation method based on multi-dimensional project reaction theory |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116975558A true CN116975558A (en) | 2023-10-31 |
Family
ID=88484097
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310780976.4A Pending CN116975558A (en) | 2023-06-29 | 2023-06-29 | Calculation thinking evaluation method based on multi-dimensional project reaction theory |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116975558A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117635381A (en) * | 2023-11-07 | 2024-03-01 | 华南师范大学 | Method and system for evaluating computing thinking quality based on man-machine conversation |
-
2023
- 2023-06-29 CN CN202310780976.4A patent/CN116975558A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117635381A (en) * | 2023-11-07 | 2024-03-01 | 华南师范大学 | Method and system for evaluating computing thinking quality based on man-machine conversation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Fox et al. | Random item effects modeling for cross-national survey data | |
| Özdil et al. | Investigation of the mediator variable effect using BK, sobel and bootstrap methods (mathematical literacy case). | |
| Bradshaw | Diagnostic classification models | |
| Eleje et al. | Comparative study of classical test theory and item response theory using diagnostic quantitative economics skill test item analysis results | |
| Oshima et al. | Raju's differential functioning of items and tests (DFIT) | |
| Azevedo et al. | Bayesian longitudinal item response modeling with restricted covariance pattern structures | |
| Göker et al. | The estimation of students' academic success by data mining methods | |
| Ma et al. | Category-level model selection for the sequential G-DINA model | |
| CN116975558A (en) | Calculation thinking evaluation method based on multi-dimensional project reaction theory | |
| Ainley | The 1999 postgraduate research experience questionnaire | |
| Komleva et al. | ME Methodology of Information Monitoring and Diagnostics of Objects Represented by Quantitative Estimates Based on Cluster Analysis | |
| Jiménez et al. | Dimensionality assessment in bifactor structures with multiple general factors: A network psychometrics approach. | |
| CN113239669B (en) | Test Question Difficulty Prediction Method | |
| Veenman et al. | Bayesian hierarchical modeling: An introduction and reassessment | |
| Setiawati et al. | Study item parameters of classical and modern theory of differential aptitude test: is it comparable? | |
| Hox | Accommodating measurement errors | |
| Thompson | Bayesian psychometrics for diagnostic assessments: A proof of concept | |
| Şengül Avşar | Aberrant individuals’ effects on fit indices both of confirmatory factor analysis and polytomous IRT models | |
| CN116861358A (en) | BP neural network and multi-source data fusion-based computing thinking evaluation method | |
| Kartal et al. | Examining the dimensionality and monotonicity of an attitude dataset based on the item response theory models | |
| Brown et al. | Modification indices for diagnostic classification models | |
| Wu et al. | Bayesian modeling of factorial time-course data with applications to a bone aging gene expression study | |
| Prasasti et al. | Developing an instrument of national examination of equivalency education Package C of mathematics subject | |
| De la Torre et al. | Cognitive Diagnosis Models | |
| Li | Estimation of Q-matrix for DINA Model Using the Constrained Generalized DINA Framework |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |