//
you're reading...
Leadership Issues, Teaching and Learning, Uncategorized

Don’t do ‘Can do’. The problems with can-do checklists and trackers.

In my recent talk at ResearchEd in Birmingham (and more recently at Blackpool) , I explored some ideas about assessment and critiqued various responses to the challenge of getting assessment right.  Sensibly enough, given all the limitations and flaws in grading systems, lots of schools are trying to make their assessment systems meaningful, rooted in the detail of what our students actually know, not simply relying on macro descriptors.

ice

Learning happens at the ‘molecular’ level, not the macro level.

I have seen a lot of schools move towards an approach based on ‘mastery statements’ or ‘can do’ statements.  They sound sensible, look reasonable on paper and certainly offer a plausible solution to one of our central challenges:  how do we capture and track meaningful assessment information that helps students and teachers to know where they are and where to go next?

😊😐😟

In theory, a mastery statement, expressed in studentspeak as an ‘I can’ statement, does the job of looking at the detail, free from arbitrary grade descriptors.  However, in reality  I think this approach is a big mistake for three reasons:

  • The illusion of absolutes.
  • Implausible scale of meaningful tracking.
  • Persistent creation of false numerical scales.

In my talk I use some images of Can Do statements that I found via a google search.  However, I have no desire to focus my critique on specific schools – everyone involved means well and is trying to come up with solutions.  So, let me just share some of the statements I’ve found – they are typical of many others.

Example 1: Computing.

Each of these statements is placed in a grid with tick box where students can self-assess using the self-reflection emoji scale – that staple of reliable school assessment! 😊😐😟

I can explain the term ‘computer system’.  This sounds ok surely? Well, not really.  It depends on what the requirements are for the explanation.  How much depth is required? For example, is this a one mark or four mark answer? We can easily imagine a range of answers some of which would be more detailed and accurate than others.  This statement doesn’t allow anyone, teacher or student, to gauge how good their explanation is or what knowledge they have. And how does 😊😐😟 help?

I know at least three functions of the CPU. This is more like it.  It is precise – three functions.  But, even here this is a heap of ambiguity. Surely some functions are more important to know than others?  Which functions are known and which are not? Hmmm  😊😐😟

I can explain the importance of ethical, environmental and legal considerations when creating computer systems.  Note: this is on the same list as the statements above – ie given the exact same status.  You could probably find a PhD thesis on this topic.   It’s not too cynical to suggest that a simple 😊😐😟 evaluation isn’t really going to get to the heart of our assessment on this rather complex compound Can Do statement; but there it is.

To fully assess this, you would need to state the specific components of the explanation required to demonstrate understanding of the issues at GCSE level and then test whether each students knew them as isolated facts and could then assemble them into a coherent explanation – orally or in writing.  That would require some form of assessment process – probably generating a mark of some kind.  A student’s mark compared to the range of marks scored by the class would tell you more about the degree of ‘can do’ they’ve mastered, on a specific assessment.  Do you need to record it?  Not really – not for everything.  The record is there on the test, in their book, on their assessment.

Example 2: Science 

The example I found is basically a spreadsheet with multiple tabs for each science topic.  There are literally hundreds of statements.   Here are some examples.

I can state that sound is produced by vibrations.  Straightforward enough? Not really. How meaningful is it exactly?  Imagine your student stating “Sound is produced by vibrations”. There you go. Tick. Smiley face.  What does it mean? Do they understand it?  Can they use this information in a context? What about stating ‘Quantum theory introduces probabilistic elements to our understanding of nature’?  I’m not convinced that recording that you can state anything amounts to knowing it.  It’s just such a weird way to record a student’s knowledge.

I can state that sound cannot travel in vacuum.  Imagine a tick against every child’s name on your giant tracking sheet except maybe Michael, so you ask him.   ‘Michael – can sound travel in a vacuum?’ No. ‘Well done, be sure to tick your Can Do statement’. Full house for me.  Move on.  Boy… is this laborious!?  And that’s just one of the hundreds of statements….. And we still don’t really know if Michael understands the nature of sound waves.

In both of these examples, it seems to be that taking an entire syllabus and turning all the factoids into Can Do statements is a false move.  There are already plenty of resources that capture the content – revision guides, the specification itself, text books.  They are packed with content. Why not simply tick and cross these if you want a checklist.  But to add ‘I can’ at the start of a sentence really doesn’t tell you enough to be worth the massive time investment even if it tells you anything at all.

Example 3: History 

I’ve found an example of Can Do statements appearing in a document used to track progress at GCSE against some very generic descriptors relating to London:

  • I can demonstrate that I am developing knowledge of London across the time period.  (KS4 Target Direction 4)
  • I can demonstrate secure knowledge of London across the time period. (KS4 Target Direction 6)
  • I can demonstrate advanced knowledge of London across the time period (KS4 Target Direction 8/9)

How helpful is this? How would any typical GCSE student know if they are ‘developing knowledge’, have ‘secure knowledge’ or ‘advanced knowledge’?  I find myself screaming – Which Knowledge??  Of course, the exact knowledge required to constitute advanced, secure or developing will be stated elsewhere  – so this Can Do exercise is weirdly, unhelpfully removed from anything that informs the assessment.  And as for the link to GCSE grades – that seems monumentally simplistic and vague.  Knowing more = doing better.  Who knew?

Sure enough, this example includes advice for improvement. If you are at the level called Developing, here’s the advice given: You need to add detail; you need to extend your analysis.  Again, this just doesn’t take anyone forward.  In the absence of any actual information about specific knowledge elements, it’s just saying ‘to do better, you need to do better’.

Example 4: Massive Mastery Statement Banks. 

There are lots of systems being developed in the marketplace where various concepts such as flightpaths and minimum target grades are used.  However, in a post-levels world, they are now looking towards statement banks to generate the data.  In some systems I’ve seen, teachers evaluate each statement for every student against a multi-point scale such as Emerging, Developing, Secure, Mastered.  I don’t know what criteria are used by teachers to judge which applies but you can imagine the level of moderation needed to get any kind of consensus on any given statement.  But then multiply that up by hundreds – because the lists of statements are massive.

Here’s some examples from the Art curriculum at KS3:

  • Pupils should be taught to use a range of techniques to record their observations in sketchbooks as a basis for exploring their ideas.  
  • Pupils should be taught to increase their proficiency in the handling of different materials.

Hmmmm.  😊😐😟 ??  Isn’t the first one just a case of getting students to use a sketchbook? That seems to be more about the teacher than the students.  And how do you condense ‘proficiency in handling different materials’ into a selection of ‘developing, secure’ or whatever.?  The teacher evaluation will be so subjective.

And yet…here’s where I really lose the plot with this….. these systems turns the average of the subjective Emerging, Developing, Secure, Mastered evaluations into a number for each student.  Sometimes the numbers are in the form 4.47, 3.6, 5.92….  Yes, THREE SIG FIGS!! I kid you not.  We’ve taken a bunch of Can Do statements, applied the teachers’ “hmmm, roughly, kind of, probably, let’s say Secure” guesstimates and averaged it up to 4.47?  This will look lovely on the system; there will be graphs.  Coloured graphs.  But it’s just so majorly spurious. It’s data porn – seductive, maybe,  but so so wrong.

And still, even if we swallow the Can Do data pill, we have the question of ‘so what’?

And that was the other part of my talk:

We know Amy is struggling. The question is what we do about it.

 

 

 

 

 

 

 

 

 

Discussion

18 thoughts on “Don’t do ‘Can do’. The problems with can-do checklists and trackers.

  1. Very interesting post. I am a Spanish teacher and the World Readiness Standards (in the US) share Can-Do Statements. I had no idea the idea spread or started in other disciplines. I agree with what you wrote above yet I appreciate the Can-Do Statements from ACTFL because they help me remember as well as explain to my students what the path to proficiency looks like (moving from isolated words, to phrases, to short sentences, to series of sentences, to paragraph length, etc.).

    Here’s my question for you: Does your opinion of Can-Do’s change if they address skills (listening, speaking, reading, writing) rather than knowledge acquisition and production?

    Liked by 1 person

    Posted by Amy Pento | February 18, 2018, 2:48 pm
    • I’d have to see the examples. I just think they add a layer that is unnecessary. Also almost impossible to use without exemplars and some process for setting the level of depth. Eg in Spanish – using tenses requires a context and use could use them in a basic or more sophisticated manner; ‘can do’ is rarely definitive.

      Like

      Posted by Tom Sherrington | February 18, 2018, 2:52 pm
      • When my students take my state assessment, I have to judge the level of their language production, whether they’re at Novice or Intermediate level, regardless of what words and structures they use. I believe Europe has a similar system although the name eludes me right now. Might the levels be A1, A2, B1, B2, etc. Since the variables that go into language production are so numerous, I appreciate these targets with which I must work to create lessons that help the students get there. What vocabulary should be taught? What structures and in what order? Those are not aided by the Can-Do’s. I often wish there were more guidance there. There is too much independence permitted here, imho.

        ried to attach a screenshot but I couldn’t so if you’re curious, here’s a link:
        https://www.actfl.org/sites/default/files/pdfs/Can-Do_Statements_2015.pdf

        Like

        Posted by Amy Pento | February 18, 2018, 3:02 pm
      • Hi Amy. That’s useful as a curriculum map setting out the hierachy- I can see that, although writing ‘I can’ is unnecessary and there will be degrees of fluency within each statement. However, imagine a teacher tracking that for a class of 25? It’s huge; way too big to keep on top of I would say. It works here because each statement is a prompt to do something in Spanish – so students would need to then do the thing to demonstrate it so it’s like a list of cues. In English-only contexts, often the statements themselves would require further evidence or subjective evaluation and you quickly hit the buffers of extreme volume of material. Thanks for sharing the detail.

        Like

        Posted by Tom Sherrington | February 18, 2018, 3:17 pm
      • Oh, I don’t have the students actually do the self-rating (feels juvenile to me). I share it with them so they can see where they’ve been and where they’re headed. This is nice for my top students who may have met my class goal yet are capable of exceeding. It also helps my stragglers see why they’re earning full credit and what they’d have to do to get there.

        Like

        Posted by Amy Pento | February 18, 2018, 3:22 pm
  2. Context and application are a million miles away from recall in science. It would indeed be a very long tick list for every conceivable application derived from each topic.

    Like

    Posted by Robert | February 18, 2018, 4:28 pm
    • It’s already too big. Simple quizzes and topic tests and scores have worked for decades; the checklist thing is a crazy waste of everyone’s time.

      Like

      Posted by Tom Sherrington | February 18, 2018, 4:41 pm
      • I do think the checklist can be a good way of showing pupils the knowledge they should be able to recall. I use them with my GCSE pupils and they say it is a valuable guidance tool. I also explain the significance of command words and how they are to use the knowledge to answer the command word.

        I do believe the statements can be given greater significance by placing them in a higher category.

        Can I ask what is the best approach to KS3 grading you have seen since the step away from levels? What approach would you use in your school? Does the approach link to the grading systems used in the new GCSE?

        Like

        Posted by Martin | February 24, 2018, 3:27 pm
  3. Thank you , Tom, for articulating many of my concerns for this approach to assessment and monitoring progress, which doesn’t seem to be going away, and probably will not any time soon.

    Liked by 1 person

    Posted by MaryUYSEG | February 18, 2018, 9:55 pm
  4. Really good points

    Liked by 1 person

    Posted by Mr Egerton | February 19, 2018, 3:19 pm
  5. An excellent blog, Tom. Another problem with can do statements is conflating a one off performance with learning. We have to ask “so what” of everything we do in school.
    Damian

    Liked by 1 person

    Posted by mrbenney | February 20, 2018, 10:42 am
  6. But you have not provided any alternative. If the world of education is doomed, we require a solution.

    Like

    Posted by S Zaman | February 23, 2018, 1:57 pm

Trackbacks/Pingbacks

  1. Pingback: Alphabetical Signposts to Teacher Excellence – C – Teach innovate reflect - February 18, 2018

  2. Pingback: Get assessment right and reduce workload at the same time. #HTRTSummit | teacherhead - February 25, 2018

  3. Pingback: Understanding Assessment: A blog guide | teacherhead - March 1, 2018

  4. Pingback: The can-do conundrum | Kate's Crate - May 11, 2018

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Follow teacherhead on WordPress.com

Archives

Blog Stats

  • 3,973,125 hits

Categories

Follow me on Twitter

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 68,681 other followers

St Jude Songs. And others.

%d bloggers like this: