In a previous post I revisited the superb work of Paul Black and Dylan Wiliam in their seminal paper Inside the Black Box, published in 1998.

In 2002, a superb follow-up paper was published: Working Inside the Black Box, by Paul Black, Christine Harrison, Clare Lee, Bethan Marshall and Dylan Wiliam. This reports the findings from a three-year project examining the implementation of the key ideas: the King’s Medway Oxfordshire Formative Assessment Project (KMOFAP). Initially this involved two maths and two science teachers from each of six secondary schools and then extended to include English teachers from the same schools. The pamphlet they produced, now 20 years old, is packed with insights that certainly resonate with what I observe and continue to discuss with teachers in nearly every school I visit.

The findings are presented in four areas related to elements of formative assessment

• Questioning
• Feedback through Marking
• Peer and Self Assessment
• Formative Use of Summative Tests.

Here’s a flavour:

Questioning:

‘Many teachers do not plan and conduct classroom dialogue in ways that might help pupils to learn’. That’s the central critique and challenge teachers need to address. In particular the report focuses on wait time, ‘no hands’, a supportive climate – so pupils are comfortable with wrong answers – and the need for open questions or ‘big questions’ or problem-solving tasks that allow the discourse to yield more information about student understanding.

There are three overall suggestions for teacher actions:

• More effort in framing questions that are worth asking!
• More wait time with answers leading to discussion, the aim being thoughtful improvement rather than getting it right first time.
• Follow-up tasks have to be rich, ensuring meaningful interactions that extend pupils’ understanding.

This is great stuff! It’s a real call to create inclusive dialogic classrooms where ideas are explored, where every student is required to think and teachers engage responsively with students’ ideas – right or wrong.

Feedback through marking.

Here, the overall suggestion is that feedback should cause thinking to take place and that reforms should shift teacher and student attitudes such that assessment is not seen as competitive or a summative judgement but ‘more as a distinctive step in the process of learning’.

• Written tasks (alongside oral questioning) should encourage students to show understanding of key features of what they’ve learned. (ie not just relaying information).
• Comments should identify what’s been done well, what needs improvement and how to make those improvements
• The follow-up to comments should be planned into the overall learning process, within a supportive classroom environment. (to reinforce the idea that improvement is a core element of learning)

Peer and Self-Assessment

The report reinforces the idea that peer and self assessment secure learning aims that cannot be achieved another way; they make unique contributions to student learning. There are three main recommendations I’ll summarise as:

• Make criteria for evaluating learning achievements transparent, with concrete examples and modelled exercises.
• Teach the habits and skills of collaboration in peer-assessment – in part to develop the objectivity needed for self-assessment.
• Encourage students to assess their progress in meeting the learning aims, developing the agency to guide their own work.

(The one area of this report I wouldn’t personally amplify is the idea of traffic lighting. I’ve found this kind of self-reporting too superficial – Red, Amber, Green can’t tell you what students know, only a vague sense of how they feel about their confidence in what they know which can be quite different. I could explore this further elsewhere.)

The Formative Use of Summative Tests.

I think it’s worth stressing the context of this which is that in the 90s, ‘end of topic tests’, in Science for example, were ubiquitous. It wasn’t a response to the weight placed on terminal exams in GCSEs. The report’s overall message is that summative tests should be – and should be seen to be – a positive part of the learning process, actively involving them so:

pupils can see that they can be beneficiaries, rather than victims, of testing because testing can help them improve their learning.

p14.

I love that!

The three main suggestions are:

• Engage students in reflective review of the work they have done to help them plan their revision
• Encourage students to set questions and mark answers to help them understand the assessment process and focus further efforts for improvement
• Use peer and self-assessment to apply criteria to help understand how work might be improved.

The report suggests that KMOFAP teachers found these strategies helped pupils become aware of when they were learning and when they were not; students felt they had more control and more investment in the learning and assessment processes. There’s a fascinating section that explores issues around feedback, motivation and self esteem. Rather than feedback that enhances ego through rewards or grade, a key conclusion is:

Feedback that focuses on what needs to be done can encourage all to believe that they can improve. Such feedback can enhance learning, both directly through the effort that an ensure, and indirectly by supporting the motivation to invest such effort.

p19

The final conclusions relate to actions teachers should take to engineer an effective learning environment. This can be summarised as:

• Plan activities so students express their thinking, so feedback can help develop it.
• Formulate feedback so it guides improvement in learning
• Use activities demanding collaboration, so everyone included and challenged – with the training needed so everyone listens to each other
• Make sure all pupils are active participants in lessons – that their learning depends less on spotting right answers but more on their readiness to express and discuss their own understanding.

That last point is gold, as far as I’m concerned. It feeds into on final point that I’ll share – the idea that, formative assessment requires changing the ‘classroom contract’ –

so that all expect that teacher and pupils work together for the same end, the improvement of everyone’s learning.

Thanks again to the authors for their wisdom. It’s a great, punchy read – highly recommended for revisiting. It’s also fascinating to see how these ideas are developed further in the ‘Five Formative Assessment Strategies, promoted subsequently by Wiliam, Leahy et al.