I’ve written quite a few posts about exams and assessment in the last year. Some of the earlier ones were a bit of a rant; more recently, I think they are more measured. Through this blog and other related activities such as working with the Headteachers’ Roundtable, I’ve been able to meet some of the people from exam boards and OfQual. It is very interesting to talk face to face, to hear their perspective and to discuss the mechanics of the examination system.
My main contention is that teachers, politicians and the public at large need a much better understanding of the mechanics of the system so that there is more awareness of the inherent limitations and margins of error. Of course people are entitled to demand a system that they can have full confidence in…but that doesn’t mean we can wish away or mask the issue of error as if it isn’t there.
Too often the debate descends into conspiracy theory territory. I firmly believe that everyone involved in the system sets out to act with integrity and seeks to do the best they can for students and schools. However, we are all under intense scrutiny pressure and so I also believe that people are motivated to protect and defend themselves, even when they know they’ve made mistakes or where standards are below what they could be. That only serves to undermine confidence rather than build it.
Fundamentally we’d like to think of the exam system as predictable and linear, with clearly defined goal posts and boundaries; in practice it is non-linear; the boundaries are fuzzy and the goal posts are not static. We want our system to be such that students are competing with themselves – simply gaining credit for doing their best on their own terms. However, in reality, our exam system measures how well individuals perform compared to everyone else. This is how it has always been and is how most tests in most schools are conducted too. Essentially, the national exams system recreates many of the processes that teachers use in schools in designing tests, but scales them up to a very large scale. I’m coming to terms with this, starting to see the flaws as an intrinsic aspect of the system we all operate.
And yet, it feels wrong. We want some firm lines in the sand; some benchmarks that won’t move. But just try doing that; all standards are ultimately defined by comparison as I suggest in my most recent post on this subect: Assessment, Standards and the Bell Curve. Basically, if you don’t want to compare students, then don’t ever give them the same test. There are serious concerns and real issues around labeling children as failures… but that is a fault with a wider values system that goes wider than schools and schooling. Exams are technical and procedural; the value we place on their outcomes is something else. You can’t make the high jump fairer or easier or mask the range of performance in any cohort of high jumpers; it is what it is. We need structures that put exams in context, building self-esteem through a celebration of a wider curriculum; a whole education.
However, as I’ve said before, given how much rides on the outcomes from exams, we should do all we can to get them right. I started getting agitated about exams after a few issues emerged at my school over the last two years. See here:
Sour Grapes on Results Day? I’m getting my excuses in early.
Battles with AQA over English A Level Marking have shaken confidence in the concept of standards
I’ve also written more broadly about how much emphasis is placed on assessment data, far beyond it’s capacity to be meaningful, and protested about our tendency to oversimplify data.
The Data Delusion: On average, it’s a bit more complicated
The Folly of Narrow Newspaper League Tables
I’ve tried to give an alternative perspective on how data, assessment and achievement are viewed – to promote some discussion
Again: Assessment, Standards and the Bell Curve and also That Gap isn’t getting narrower: Now What?
I’ve also tried to present some solutions.
In particular, I’ve tried to promote the idea that assessment in the form of exams can and should be redesigned to allow for progressive achievement within a wider framework that gives value to a rounded education:
Thought for the Week: Exams are only part of the Story
Exams Debate is Far Too Narrow: What are the alternatives?
Towards a Proper English Baccalaureate – which links to the work of the Headteachers’ Roundtable on an idealised progressive qualifications system and a more pragmatic ‘ready to roll’ system.
Summing this up and adding some more, my agenda for exam reform would include the following:
1. Policy for national examinations should be taken completely away from election-cycle politics or the assessment market place where Exam Boards seek competitive advantage. A central independent agency that answers to OfQual, should set out the rules for how exams are to be designed, taking account of the nature of various subject disciplines, research evidence and the views of different stakeholders in the profession. The decisions taken would include the issue of modularisation, the weighting and nature of coursework elements and the structure of exam papers. These decisions must not be taken by DFE under the guidance of amateur politicians.
2. Exams results should we awarded in the form of points, not grades. We must remove artificial cliff-edge outcomes. With points would come an understanding of the concept of margin of error. Different assessment processes could be shown to have varying degrees of certainty that could be referenced. For example, 80/100 and 79/100 are neighbouring marks which would be broadly equal within the margin of error of, say +/- 2. However if 80/100 gains A and 79/100 gains B, assumptions will be made that are not true. Grades have got to go because the boundaries are inherently changeable and blurred.
3. Public examinations should form a part of a wider profiling of student achievement through a national English Baccalaureate system, with online transcripts designed to follow students as they progress through the system. We all recognise that exams don’t tell the whole story of a child’s capabilities so we should back that up by creating a system that gives formal recognition to more than just exams. The English Baccalaureate that I have in mind, could be transforming.
4. All schools should be mandated to release teachers to train and serve as examiners, provided that they meet stringent standards. Examining must be conducted by expert people with good judgement so that the qualitative aspect of learning in a subject is given value beyond the limited confines of a rigid mark scheme which leads to the narrow hoop jumping we’re all so tired of.
5. Examination Appeals should be heard by expert people independent of the original awarding body and re-marking should be done blind – ie without seeing the original marker’s marks or comments. It should be recognised that if localised school-specific cohorts generate anomalous results, a local investigation is the only way to establish the true story of achievement, rather than using national level statistical remedies that affect all students equally. If an exam board sets a bad exam, there should be an acknowledgement and redress.
6. Performance tables must include a broad range of measures that are given parity. They should be designed so that perverse incentives to narrow the curriculum and to operate elaborate window-dressing interventions are eliminated. We must strive to educate the public and the media to embrace the complexity of school outcomes information. Single-figure results tables should be outlawed, regarded as unacceptable, tantamount to public deception!
Many of the issues with our exam system can be traced to the undue, obsessive emphasis on measurable outcomes from the system rather than the quality of inputs and processes. That too must change.
Other policies: I would add a strong line on curriculum scrutiny in any accountability process, ensuring that breadth is maximised and I’d move the UCAS system to be a post-A2 process. More on those things another time.
teacherhead 
Great post, as ever Tom. There’s always a tension between giving responsibility for education to an undemocratic body, and giving it to politicians who are, as you rightly say, quite amateur when it comes to the intricacies of exams. Ofqual is walking this tightrope to an extent but it might help everyone if those relationships were clearer.
One suggestion I gave this time last year was about the possibility of all students receiving their exam papers back in digital form. Now that exams are almost all scanned for online marking, it seems sensible that every student should have an online repository where their exams are ‘returned to’. In doing so they could then make these exams available for viewing by potential employers or universities who can go beyond the ‘blunt’ exam (if they wish) and see what skills the person is actually demonstrating. This would give an incentive for students to be entered into the most challenging exam (because they would want the chance to demonstrate what they can do) and also ensure that the nuances of someone’s skills are more widely available.
Of course, this opens up exam marking to much more scrutiny as everyone would be able to compare their marking. But, really, is that such a bad thing?!
LikeLike
Thanks Laura. I’m sure there are procedural or financial reasons for keeping exam papers locked away but I agree that it should be much more common and much cheaper to get the papers back.
I was really surprised to discover last year that blind re-marking is explicitly ruled out in the Code. Is this about protecting examiners from undue scrutiny? I will try to find out.
Democratic accountability and professional expertise need to find a much better way to co-exist in this area.
LikeLike
“A central independent agency that answers to OfQual, should set out the rules for how exams are to be designed, taking account of the nature of various subject disciplines, research evidence and the views of different stakeholders in the profession.”
Snag is that Ofqual sees itself as in that role in the sense that they could veto any rules that are specified that are not fit for purpose. Traditionally, however, Ofqual tends to be more focussed on procedures than judgements about outcomes. When is a GCSE not a GCSE? When it’s IGCSE. Ofqual’s own conditions of regulation say that qualifications titles and the promotion of regulated quals should not be misleading. How can a qualification have GCSE in its title when it is not bound by any of the GCSE regs? ie it is NOT a GCSE. As a bit of a test I asked if I could rebrand our quals eGCSE. They said that would be misleading as e often means electronic. Well the fact is we provide cloud support and on-line testing and use the quals in Europe so why is that misleading if IGCSE isn’t? OTOH iGCSE is not misleading when these have nothing to do with Apple or the internet and the trademark is with a capital I ? Yet both the DfE and Ofqual use iGCSE. So it’s obvious that this is simply misleading marketing and Ofqual not being inclined to enforce their own conditions. Why? Because the SoS promoted IGCSE as being OK. It would be interesting to get a large number of schools and employers to make complaints to Ofqual about the misleading use of GCSE in the titles of qualifications that are not GCSEs. It is also a monopolies issue. Cambridge International Assessment own the IGCSE trade mark and license it only to guess who? OCR and AQA. So this cartel is being supported by government policy and the regulators turning a blind eye to their own regulatory conditions. Well that’s what it looks like from here.
LikeLike
I’m sure OfQual will have an answer to that point. We’ve taken on iGCSEs at my school including AQA Certificates for a range of reasons, but other subjects use GCSEs. I’ve worked in schools for nearly 10 years where both qualifications were offered and don’t think this is a major issue any more than choosing whether to use OCR or AQA, for example. However, I would prefer there to be less choice, less of a market and more sense of a set of national qualifications. OfQual’s role is to police the system; the new body I suggest would need to drive it.
LikeLike
You are missing the point. To call a qualification a GCSE it has to conform to GCSE criteria. IGCSE doesn’t, its simply a brand. Ofqual know there is a problem its simply the political imperative is more powerful than one guy like me pointing it out to them and I haven’t the money to take it to court – if I had I think I’d win because the logic is clear. If we are saying that GCSE criteria don’t matter, fine then let’s get rid of them but let’s not have rules that simply get ignored to suit political expedience. Why invent another body when there is already one, it simply costs more money and adds bureaucracy. Ofqual is supposed to be independent of government so why would they not be the right organisation to set the rules? Or even the exam itself and let the Awarding Bodies compete on delivery. Like it or not qualifications is a market like any other and until that changes it should conform to the same rules as any other market and if Ofqual has conditions of regulation they should be enforced consistently and fairly to all parties not overlooked because the SoS makes an announcement without understanding the ramifications.
LikeLike
You could be right that OfQual is the body we need – albeit that it would take on a different role, setting out more details on how exams should be structured. They could homogenise all the variants. I’d be more bothered about the iGCSE issue if I felt they were substandard but they are not; if anything they’re more challenging. It is a brand as you suggest and all of that does need to change. But there are much worse things to worry about than iGCSEs.
LikeLike
The thing is why bother with any GCSE criteria if they aren’t needed. If IGCSEs are good enough just regulate whatever the Awarding Bodies come up with and don’t waste everyone’s time with criteria that are not going to get used. It’s not IGCSEs themselves, it’s a principle. Mind having seen some specimen questions from IGCSE computing I can’t agree that they are that good, at least not in that subject. Obscure knowledge required about obsolete technologies that I could find in 2 seconds on a Google search if I ever needed it. Probably GCSE suffers from that too though. But I also fail to see how a valid assessment can be made of practical computing skills with a paper based exam and no coursework at all. As an employer I’d certainly not be confident in theory only technology people. Look what it’s done for government procurement ;-). One reason for having a Baccalaureate is to get a better breadth and more valid assessment of a wider range of competences.
LikeLike
[…] Exam reform. Another blog manifesto. […]
LikeLike
[…] I have also collected posts and added some specific recommendations on reforming examinations: Exam Reform: Another Blog Manifesto […]
LikeLike
Perhaps now is the time to look at the Scottish and Irish models: one national examination board which is highly regarded, respected and regulated.
The IGCSE discussion has only just begun this year and many parents are already questioning independent schools’ decisions to opt for this ‘softer’ option.
LikeLike
Btw how did you get on with the English pre-U at yr centre? Our AQA Lit experience v. Similar to yr own in recent years ( just read yr last yr blog). Be interested to know how things are now. Thanks. Elizabeth
LikeLike
We’ve just had first Y12 through.. so first proper exams next year. All staff love the course structure and it’s going well so far. PreU board offered a mock marking service for a trial exam with Y12 to help with moderation – a form of CPD, not a regular service I think. We”re waiting for the response. Anyway, can’t be worse than AQA LitB3 which has such narrow marking criteria. One of our top students (4A*s at A level,) got a U on LitB3…I think she’s too clever for the markers and they don’t like that.
LikeLike
[…] This week I met Glenys Stacey, the CEO of OfQual. She’d invited me to share views about aspects of the examination system. I had previously met Amanda Spielman, the Chair of OfQual – where she offered advice on our Baccalaureate model. OfQual leaders are keen to engage in dialogue with people in the profession, not least so that they have opportunities to explain their work. (Some of my ideas about exams are summarised and compiled in this post: Exam Reform: A blog manifesto. ) […]
LikeLike