Every question part has a marking algorithm, which is is responsible for:
- Rejecting the student’s answer if it’s invalid. If the answer is rejected, no credit or feedback will be given and the student must change their answer before resubmitting.
- If the student’s answer is valid, assigning credit and giving feedback messages.
The credit for a part is the proportion of the marks available which should be awarded to the student. The total marks available are set by the question author, and might be reduced if the student reveals steps, or if this part is a gap in a gap-fill part.
The feedback messages shown to the student are strings of text shown after the part has been marked. These might only become visible to the student after they have finished the exam, so don’t rely on feedback messages to convey information that students might need in subsequent parts.
Two marking notes are required:
marknote should award credit and provide feedback based on the student’s answer. If the student’s answer is invalid,
interpreted_answernote should produce a value representing the student’s answer to this part, which can be used by other parts with adaptive marking.
Each note evaluates to a value, and also produces a list of feedback items, which modify the amount of credit awarded or give a message to the student. When a feedback item modifies the amount of credit awarded, a message describing the number of marks awarded or taken away from the previous total is displayed to the student.
If a note fails, either because it applies the
fail() function or an error is thrown while it is evaluated, it will produce no value and no feedback items.
Any notes referring to a failed note also fail.
interpreted_answer notes fail, the student’s answer is rejected and the student must change their answer before resubmitting.
Like question variables, marking notes can refer to each other.
When another note is referred to in another note’s definition, its value is substituted in.
To apply another note’s feedback items, use
Variables available to marking algorithms¶
The following variables are available for use by marking algorithms:
The path to this part, in the form
pN(gN|sN). The first part (part a) has path
p0. As an example, the second gap in part c would have path
The student’s answer to the part. The data type of this value depends on the part of the type. See the list of standard part type values, and custom part type answer input methods for details on the data types produced by different part types.
dictof the part’s settings.
dictionaryof the variables, functions and rulesets defined by the question. You can use this with
unset()to evaluate expressions without access to the question’s variables, for example if you want the student to use a variable called
xbut there’s also a question variable
Marking-specific JME functions¶
All the built-in JME functions are available in marking notes, as well as the following functions specifically to do with marking:
Set the credit to 1 and give the feedback message
messageis omitted, the default “Your answer is correct” message for the current locale is used.
Set the credit to 0 and give the feedback message
messageis omitted, the default “Your answer is incorrect” message for the current locale is used.
true, set the credit to 1 and give the default feedback message. Otherwise, set the credit to 0 and give the default feedback message.
Set the credit to
credit, and give the feedback message
message. The message should explain why the credit was awarded.
creditto the current total, to a maximum of 1, and give the feedback message
message. The message should explain why the credit was awarded.
creditis negative, credit is taken away, to a minimum of 0.
creditfrom the current total and give the feedback message
message. The message should explain why the credit was taken away.
Multiply the current credit by
proportionand give the feedback message
message. The message should explain why the credit was modified.
This operation is displayed to the student as an absolute change in marks awarded, not a multiplication. For example, if the student already had 2 marks and multiply_credit(0.5,message) was applied, the message displayed would be along the lines of “1 mark was taken away”.
End the marking here. Any feedback items produced after this one are not applied.
This is most useful as a way of stopping marking once you’ve decided the student’s answer is incorrect partway through a multi-step marking process.
Reject the student’s answer as invalid, set the credit to 0 and give the feedback message
message. The message should explain why the student’s answer was rejected.
Since the student might not see the feedback message until the exam is over, you should also use
warn()to add a warning message next to the input field describing why the student’s answer was rejected.
Give the feedback message
message, without modifying the credit awarded.
x ; y
Add feedback items generated by
xto those generated by
y, and return
This is a way of chaining multiple feedback items together.
incorrect() ; end()- mark the student’s answer as incorrect, then end marking.
apply(note1) ; apply(note2)- apply feedback generated by
note1, then feedback generated by
feedbackis the name of a marking note, apply its feedback items to this note.
feedbackis a list of feedback items generated by a function such as
submit_part(), apply them to this note.
apply(validNumber)- add the feedback from the note
validNumberto this note.
apply([submit_part(gaps["path"]), submit_part(gaps["path"])])- mark the first two gaps and add their feedback to this note.
apply_marking_script(name, studentanswer, settings, marks)¶
Apply the marking script with the given name, with the given values of the variables
Any feedback items generated by the marking script are applied to this note.
The built-in marking scripts are stored in the marking_scripts folder of the Numbas source repository. Use the name of the script without the
.jmeextension as the
nameparameter of this function.
apply_marking_script("numberentry",studentAnswer,settings+["minvalue":4,"maxvalue":5],1)- mark this part using the number entry part’s marking script, but with the minimum and maximum accepted values set to 4 and 5.
Submit the part with the given path. If
answeris given, the answer stored for that part is overwritten with the given value. Returns a dictionary of the following form:
[ "answered": has the student given a valid answer to the part?, "credit": credit awarded for the part, "marks": number of marks awarded, "feedback": feedback items generated by the part's marking algorithm ]
Custom part types can’t depend on other parts being available. However, you might want to allow the question author to provide the path of another part, or do something with this part’s gaps or steps, whose paths are listed in
Mark the part with the given path, using the given value for
Returns a dictionary of the following form:
[ "valid": is the given answer a valid answer to the part?, "credit": credit awarded for the part, "marks": number of marks awarded, "feedback": feedback items generated by the part's marking algorithm, "states": a dictionary mapping the name of each marking note to a list of feedback items, "state_valid": a dictionary mapping the name of each marking note to a boolean representing whether that note failed, "values": a dictionary mapping the name of each marking note to its value ]
This function is most useful in a custom marking algorithm for a gap-fill part, when you want to reassign the student’s answers to each of the gaps. For example, in a part with two number entry gaps, you could ensure that the lowest answer is marked by the first gap, and the highest answer is marked by the second. This would allow the student to enter their answers in any order, and the question author to set the expected answer for the first and second gaps to the lowest and highest correct answers, respectively.
Apply the given list of feedback items (generated by
mark_part()) to this note, scaling the credit awarded by
Mark gap 0, and award credit proportional to the number of marks available:
let(result,mark_part(gaps["path"],studentanswer), concat_feedback(result["feedback"], result["marks"]) )