Weeks after students in England, Wales and Northern Ireland sweated through A-level and GCSE exams, their fate is being decided in a windowless basement near the centre of Cambridge – decisions that mean joy for some and despair for others.
There’s a hush of concentration in the room, the only sound the turning of pages and scratching of pens, as the group sit poring over the anonymous exam papers in front of them – but no exams are actually being marked.
Instead, the small group of experts for the OCR examination board is working its way through piles of scripts arranged by score, weighing up where the boundary should fall between an A or B for this particular humanities subject.
This is the most highly regulated and delicate part of the examination system, which the public never sees. OCR, one of the five main examination boards, granted the Guardian exclusive access to its inner workings – revealing a process characterised by caution, rigour and sophisticated technology.
“Everything has changed in the last three to four years. The days when examiners would get delivered 200 scripts, and then we’d sit and wait for them to come back, have gone,” said Sarah McPhee, OCR’s chair of examiners for history, in the middle of what she calls “the white heat of the examination season”.
The stakes and numbers involved are huge. July at OCR is like Christmas for retailers, as the exam board receives, processes, marks and grades 2.7m GCSE and 1.4m A-level scripts.
While pupils still sit in halls and write answers in booklets, almost everything that happens from there on has been revolutionised by technology.
Once the invigilators call for pupils to put down their pens, the process begins: a six- to eight-week period in which OCR and the other exam boards have to mark every paper and deliver up the results to the Department for Education, the examinations regulator, Ofqual, and finally the schools, parents and students themselves.
During the first step, the scripts are transported to OCR’s headquarters in Cambridge, where they are scanned into a total of 52m images to be distributed electronically for marking. There’s little room for error, as OCR found out last year when its computer network went down for a week, compounded by a shortage of markers. Only help from another exam board and a herculean internal effort saw it meet the DfE’s deadlines.
Mark Dawe, OCR’s chief executive, says things have gone smoothly this year, thanks to network improvements and the successful hiring of more than 14,000 examiners and assessors, nearly all of them current or former teachers.
The new technology makes it easy for OCR to keep a closer eye on quality: would-be examiners are given 10 model scripts to mark. If their assessments fail to match the marking of OCR’s experts, they do not get the job. That hurdle alone tripped nearly 200 applicants this year.
The software also tracks individual examiners; unusual patterns trigger alarm bells. Model papers are also inserted at random and the results compared again with expert marks. In both cases, an examiner’s performance will be judged, and by mid-July OCR had removed a further 200 from its books.
Christopher Green, a veteran examiner and director of English and drama at the private Perse school in Cambridge, demonstrates on his laptop how the software allows examiners to annotate scripts with symbols and comments.
Does it turn examiners into robots? “I don’t think a machine could mark,” Green said. “We’re not looking for any type of set response, we’re looking to reward candidates for their ideas.”
Despite claims that exams have been dumbed down, Green says he has seen standards rise in his 20 years as an examiner. “The best answers are written by people with amazing minds, producing incredible material. Candidates today are more involved in their work, their answers are less routine and more creative,” he said.
By the second half of July, about 90% of papers have been marked, but that is only half the process. Back in the windowless basement, the group of expert examiners are engaged in the other half: converting marks into grades.
As one of the senior examiners says: “The judgment call here is: at the lowest permissible A boundary, is there a reasonable amount of sustained, critical evaluation that you might expect of an 18-year-old?”
The consensus is that the A-grade boundary will be somewhere between papers gaining 65 marks and those gaining 69. That’s when the hard work starts, as the senior examiners carefully sift through stacks of high-scoring scripts to determine the “zone of uncertainty”, as one calls it.
Then a debate begins and spreadsheets are consulted: if 66 is the bottom mark then 29% of this year’s candidates would get As, while at 68 fewer than 23% would.
“I think at 66 the weight of evidence is that we do not think it is worth the [A] grade,” says one, after further discussion. Finally, 67 is set as the lowest mark for an A. That means a fractionally higher percentage of candidates will get As in this particular unit than the previous year. But it’s bad news for the 3.25% of candidates who scored 66: they miss out on the top grade by the narrowest of margins.
That decision must still go through multiple checks before OCR signs off its results. Ofqual – kept informed at every step – can intervene and demand tweaks or changes after talking to all the exam boards.
Sylke Scheiner, OCR’s director of assessment standards, explains that exam boards get little leeway in awarding grades, no matter how teachers and pupils gossip about certain boards being easier than others.
“We get quite stringent guidelines from Ofqual, there are only certain tolerances we are allowed,” Scheiner says. For the most popular exams, a board can vary the proportion of grades it awards by no more than 1% of Ofqual’s guidelines.
So no matter if an exam was hard or easy, the proportion of grades awarded will not change; which will come as a relief to GCSE maths pupils this year who were stumped by a question involving “Hannah’s sweets”, and took to social media to complain loudly.
Come 13 August, the decisions of those meetings in windowless rooms will come out into the open, as sixth formers across the country finally find out if they made the grade.
