What actually happens when software marks a paper.

And, more usefully, the places where it should not be believed. If you are deciding whether to let any of this near your students’ marks, the second half matters more than the first.

The four steps

Every system doing this, ours included, is really doing four separate jobs. They fail in different ways, and lumping them together as “AI checking” is how people end up surprised.

1. Reading the handwriting

A photograph of a page becomes text. This is the part that has improved most, and it is now genuinely good on ordinary handwriting at an angle, in bad light, on ruled paper. It is still the step most likely to go wrong, and it goes wrong quietly — a misread digit is not obviously a misread.

2. Working out what the question wanted

Marks per question, and what a full answer has to contain. Some systems need you to type a marking scheme in. Ours builds one from the question paper itself, which is the difference between fifteen minutes of setup and an afternoon of it — and the reason a centre actually uses the thing on the second test rather than only the first. A whole paper we marked shows the output of all of this on six real pages. What a scheme actually has to contain is narrower than most people picture, which is why this works at all.

3. Comparing the answer to the scheme

Step marks for a derivation. Error carried forward where the method holds and the arithmetic slips. Credit for a labelled diagram. This is the part that most resembles what a teacher does, and the part people most expect to be wrong. On structured subjects it is more consistent than a tired human at 11pm — not because it is cleverer, but because it does not get tired.

4. Deciding what to do when it is unsure

This is the step that separates a useful tool from a dangerous one, and it is the one nobody asks about.

Where it must not be trusted

Concretely, these are the cases where no marking software should be believed:

• A page that did not scan properly. Blurred, half-cut, upside down, or simply missing. The right behaviour is to say which page and stop, not to mark what it can see and present a total.
• A free-hand diagram with no labels. There is nothing to read. Judging it is guessing, and we hand it back rather than guess.
• An answer that argues something unusual but correct. A student who takes an unexpected route to the right answer is exactly the student you least want mis-marked.
• Anything where the marks matter formally. A board result, a scholarship cut-off, a promotion decision. Internal tests are the right place for this; a decision a family will act on needs a human who has looked.

Four questions worth asking any vendor

Including us. If a tool cannot answer these plainly, that is the answer.

What we do about all this

• Uncertain answers are flagged and handed back, with the reason said first. A typical paper comes back with a handful of questions to look at rather than all thirty-three.
• Every mark quotes the text that earned it, so checking a paper is reading, not re-marking.
• The teacher approves before anything is released. There is no setting that turns this off.
• Answers are transcribed, so you read typed text at a legible size rather than pinch-zooming a photograph of handwriting on a phone.

None of that makes it infallible. It makes it checkable, which is the property that actually matters when a parent is on the phone asking why their child lost a mark.