I’ve recently been reading the ResearchEd guide to assessment as well as re-reading ‘Making Good Progress’ by Daisy Christodoulou as I prepared to write an assessment for my Y10 Economics GCSE class.
I’ve been guilty in the past of relying on exam questions to
get to what Daisy talks of as being the ‘final performance.’ In essence what
pupils need to get to, is to be able to provide judgements about the importance
of a variety of factors on an economy or firm. In order to do this they usually
need some diagrammatical analysis and explain the impacts on different agents
with a logical chain of reasoning.
In a final exam, the way pupils do this is by reading an
extract and then writing an essay explaining their judgement based on what they
have read. The problem with setting this type of a question as an assessment
and then analysing the results is that pupils could get it wrong for a wide
variety of reasons. It might be they struggle to understand the case study, it
could be they don’t understand the economic concept, they can’t explain the
full impact in chains of reasoning or they misunderstand diagrams needed.
Dylan Wiliam talks about construct-irrelevant variance where
pupils get things wrong for factors other than what I am trying to assess which
makes my inferences less valid. Daisy talks of the deliberate practice model
where pupils focus on a component part of a final performance and practice and
get feedback on this and this gradually builds into a bigger mental model to
lead up to the point where the final problem can be solved.
She uses a sporting analogy of a marathon runner practicing
different drills instead of constantly running a marathon. The difficult bit is
to break down what I want my pupils to do to create an assessment that will
give me useful information about what pupils can and cannot do.
I broke down the important component parts that I wanted to
assess as:
·
Key definitions. Pupils need to understand
exactly what terms are. This was a table of definitions. I was able to give
specific feedback on what they got wrong.
·
Diagrams. Pupils had to draw every diagram they
could feasibly have to draw relating to supply and demand.
·
Chains of analysis. I gave pupils a main point ‘productivity
leads to lower average cost’ and pupils then had to complete a table listing several
chains of analysis explaining ‘what happens next.’
This is not every component part that pupils need to master
but it was the fundamentals I wanted to focus on. This is not a reliable
assessment in the sense that it doesn’t sample from all the domain and so doesn’t
tell me about pupils overall understanding of what they have learned in
economics so far. However, I don’t think that is an issue as the trade-off is I
am able to give very specific feedback on crucial areas they need to master and
at this stage in year 10 that is acceptable to me.
At this point, I did some question level analysis to
understand where pupils were going wrong. Usually, I don’t like QLA as pupils
can get the same question or type of question wrong for totally different
reasons. However, because this assessment was so targeted at component elements
of what they needed to do, this analysis gave me much more specific information
than ‘evaluation’ for example that I would have looked at previously.
Their definitions were reasonably well answered. There were a few
exceptions that I explained in feedback and a week later pupils had to do a
recall test on this after I retaught this. I also tried to infer what other
definition they did not know. So, a number of pupils got the definition of ‘production’
wrong and I inferred this meant they struggled with productivity and average
cost too even though this wasn’t in the assessment. This is not a perfect
science but seemed to make sense.
The diagram section was different. Pupils had to draw every
single diagram they need to know in the subject. This is because it is a
crucial element of what they need to do so I wanted to know exactly which
diagrams they cannot draw. I could then give very specific feedback and pupils
were able to respond to this.
I was thinking about Dylan’s comment about a lot of feedback
being true but unhelpful like telling a comedian to ‘be funnier.’ What was
useful about this feedback was that it was very specific, so ‘when showing
elastic supply, your supply curve must be relatively flat’ or ‘draw arrows to show
the change in price and quantity.’
The chains of analysis was my favourite activity. This is
because it is an are where previously I would have given nebulous feedback
about ‘explain your point’ but now the feedback was incredible specific. By
pinpointing exactly where chains of analysis broke down, I was able to give feedback
like ‘explain the impact of a lower price on consumer utility’ when that was
the missing chain.
This is still a work in progress. I need to think more about
how pupils respond to feedback so it is not just performance but that they can
repeat it in the future. Similarly, there are other important component parts
like understanding how economic analysis varies in different contexts that
pupils need to master, and I need to consider how to isolate and asses this.
However, this was by far the most useful assessment I’ve
ever created, both in understanding what pupils know and more importantly in
providing useful feedback to enable them to improve.