It’s been one hell of a week for everyone involved with exams – especially for the thousands of students receiving A levels. Earlier this week I posted my thoughts about the system: Adjusting CAGs: no politics, no disgrace, no injustice. It’s just technical. Well – that headline didn’t turn out so well did it! The blog was my attempt to lay the ground for what ought to have been a sound technical process, adjusting moderating centre-assessed grades. I have a bias to assume technical wisdom and neutrality and was reacting to a few problems that I have seen in the discourse around assessment:

• The tendency to over-characterise technical decisions as political ones designed by ‘people who are against us’ – even when this is far from the reality. (It’s nearly always cock-up, not conspiracy)
• The tendency to see technical concepts such as normal distributions and grade boundaries and algorithms as inherently unfair. (Lots of commentators really don’t understand the concept of standards and how grades are arrived at and maintained in normal circumstances.)
• A poor understanding of the intrinsic need for moderation of different people’s judgements of standards – which has nothing whatsoever to do with trusting people. (Amazing to me that some people’s response to all this is to suggest that we need even more teacher assessment – please no!)
• A lack of concern about grade inflation – people can be quick to wave this away but, in my view, there are risks to the value of everyone’s qualifications if the level of inflation is too high. (eg If schools hadn’t been urged to keep results ‘realistic’ – A*/A % could easily have been well over 40%. Is that ok? Not if you want some sense of long-term value to any of the grades)
• The tendency to offer critique without alternatives that can be shown to be better. (Even the alternatives for the moderation approach that have since been shared with me were not, to my knowledge, discussed and circulated widely enough. They’re quite technical. Jon Coles from United Learning tweeted an excellent long thread explaining his alternative. https://twitter.com/JonColes01/status/1293233928384520192?s=20 His principle that CAGs should be used unless there was a reason not to was interesting – I wish we’d all been discussing it in April. Generating detailed solutions and getting them known about is not a strength in our community. Easier to critique and make general claims that ‘there are lots of other solutions’ without offering any specifics. )

However, I now realise I was making some assumptions about the model that didn’t play out. Mainly I assumed, just from common sense in terms of easing the pain of disappointment in a crisis recovery model, that there would be a built-in inflation much higher than we’ve seen. I’d have gone for something closer to 10%, not the 2% we’ve had at A level. I thought that introducing a controlled but generous level of inflation was obvious. Secondly, I imagined the adjustment process would entail shifting and smoothing CAGs distributions to fit a wider national pattern via some factors that had some origin in the cohort baseline and progress data as well as past school results- not forcing rank orders to shoe horn into quirky small-cohort distributions in the crude way we’ve seen. People are shocked by how crude the algorithm appears to be. For example, nothing I had imagined would allow rank order neighbours with the same CAG to be then separated by two or more grades following the adjustment based on rank order. It also did’t occur to me that people would get downgraded to U – which is just not ‘the grade below E’; it means something else altogether.

The scale of the wave of outrage right now is remarkable – driven by people who have had to counsel distressed students on the receiving end of indefensible outcomes. The technical process has been executed poorly; even if the concept is not unjust in principle – for sure it has delivered a great many injustices. To me it just looks like they weren’t prepared to get into the school-level detail enough. They stopped short once the overall patterns seems ‘close enough’. Given what’s at stake, you have to listen when so many people are this angry about something; the feelings are legitimate. That said, I do think arguments are lost when details become sloganised and people revert to cringey assessment-illiterate wishful thinking. This thread from Sam Freedman is excellent https://twitter.com/Samfr/status/1293979033458417668?s=20 The issue around disadvantage is there in the data – a small net gain overall, but slightly larger adjustment on CAGs. Making it a simple case of rich vs poor winners and losers just undermines the argument – to me.

As Laura McInerney expresses brilliantly in the Guardian, nothing Gavin Williamson has said or done has made it any better and it’s hard to understand what he’s spent the last five months doing on this issue. When it’s time for heads to roll, his will be one, that’s for sure.

With GCSEs next week – and the A level appeals issue to resolve – I would suggest that, as a minimum, various things could be done: (for me, there is zero value in doing a ‘what they should have done in March is X…’ – because, now it’s literally pointless)

1. Examine and respond to the ’rounding down’ issue suggested by Alex Weatherall here: https://twitter.com/A_Weatherall/status/1294012623776817158?s=20 Let’s see what effect rounding up would have.
2. Upgrade all Us to E as a minimum (callous not to)
3. Remove all adjustments of -3 grades (callous not to)
4. Systematically review all adjustments of -2 grades to spot obvious anomalies and cohort discontinuities.
5. For GCSEs, see if a 10% inflation limit at 4+ could be established and reset the grades with that built-in – at least in the full cohort entry subjects (Others would say 5% but I’d go higher)

There’s now a party political tussle around whether to follow Scotland and just use CAGs. On one hand – it makes life simpler. It releases the pressure on appeals; lots of short-term gains for thousands of students. The question then is whether the net long-term devaluation and unstandardised nature of those grades matters enough. They’ll always be the ‘much easier 2020 grades’ – and do we care enough about that happening, mixed in with all the variation between schools? Maybe, it’s ok… given where we are and how messed up it all is. I’d favour easing the moderation over a full-blown ‘revert to CAGs’ approach. I wonder, now Labour has come out in favour of CAGs, whether the government will double down even harder and just focus on their embarrassing triple lock mantra. We shall see.

One think I’m more sure of – the ideas for a National Bacc would address a lot of these issues, putting exams into a wider context of achievements, accumulating credits over time, not just at the end.. maintaining breadth, removing the cliff edge intensity at 16… more to come on that soon.

On a lighter note… I think my Mum wrote the tweet of the week: