Why are we afraid of evaluation?

We, in public service, are all evaluated these days – performance appraisals, inspection, programme evaluations. We live under regimes of judgement – low-trust judgement. It is easy to think that judgements are all about approval and condemnation – they mostly are. We ‘succeed’, we ‘fail’, we ‘get by’. This is what frightens us most, perhaps – a judgement as summary as an execution or a lottery result. But such evaluation is just lazy thinking. There are better uses for evaluation and more useful ways of doing it, as we will see. 

The fact is that most of what we do by way of public action is complex and defies easy measurement. Our work resists simple categories like ‘excellent’, ‘satisfactory’, ‘poor’. Let’s say we care for a vulnerable, perhaps an abused, child. What skills do we need? We need to have a grasp of child psychology, social service, learning theory, child development theory, multiculturalism, gender conflict, conflict theory, attachment theory, institutional politics and the list goes on – we need to have personal resilience and courage, situational insight, tolerance, we need to be a mix of caring and ruthless – we need to be good negotiators. It is simply not possible to be an ‘expert’ at child protection, though we carry the label with hope. Mostly, we do the best we can with what we have.

So what are we to be judged on? And at what point do we assess the success of what we do? At one month after ‘rescue’ of the child? Six months? Six years? Sixteen years? Because our work will have a distinctive impact at each of these stages. 

Being a teacher, a nurse, a police officer, a social worker, a Council officer grows out of who we are and the conditions that face us: women professionals face different conditions from those men face; ethnicity leads to different experiences, not to say different values. Take policing – this demands judgement, the ability to discern and discriminate, to analyse situations, it requires patience, sense of humour, personal courage, a personal ethic, a social commitment, an understanding of how society changes and how people think and respond to events. One day we would arrest that youth, the next day talk to him, another day go see his family. Policing decisions are responsive to the context, the conditions are unstable, they don’t stay still for long enough to ‘measure’ them. We can, and do, make evaluations of policing based on some pretty arbitrary numbers: number of arrests, response times, use of resources and so on.  But this is not where quality lies. Quality in policing lies in how individuals with all those personal characteristics are organised, managed, work together, express the goals and policies of the police organisation. Can we really compare the experience of a young black, woman police officer in Harrogate with that of a veteran, white officer in Brixton? 

The same goes for teachers. Much of what defines a classroom and what goes on in it are the students – and they are unpredictable on a minute-by-minute basis, day-by-day, year-by-year. A teacher who cares for students is distinct from one who is scared by a class of students; a teacher who was raised by an authoritarian father teaches differently from one whose family was inclusive and gentle; one teacher might be brilliant at stimulating and engaging students but have a slippery grasp on their subject – while a colleague in the next room might be a brilliant subject specialist but rubbish at managing interactions with students. We don’t even know what ‘learning’ is or involves – because we cannot answer the most puzzling of all philosophical questions – what is consciousness**. Again, there is no stable base for making a single judgement about a teacher or a police officer – much less for comparing one with another.

But most damning of all is that all forms of professional action are in dynamic relationship with their context. A teacher teaches, feels the impact of what she does and adapts – daily, year-on-year. Same for a nurse and a doctor and a social worker and a lawyer and a politician and a council official and a professor. Our practices are shaped by experience – they are not manufactured by a plan. The idea that there is some abstract and stable measure of what makes a ‘good’ or ‘ineffective’ professional is comfortable, elegant, even – but it is at best a convenient fantasy, at worst, a wilful denial of the realities of life. 

And yet we buy into the idea right across the board.

So perhaps a source of our fear of evaluation is just this, that we are unlikely to be judged in our own terms, or even in terms that make sense to us as professional actors or in terms that make sense of the challenges and unstable conditions we face. And this is made all the more significant and scary where there are potentially punitive consequences for failing the evaluation. A personal performance appraisal or a visit from Ofsted or HMIC to our classroom or our policing beat might well lead to an unfavourable assessment which might cap our career progression or even, in the case of Ofsted, lead to our redundancy or dismissal. 

Our fears of being evaluated are tangible and all-too-often justified. Those who design evaluation are unlikely to have the political, professional and methodological expertise to devise approaches sensitive to these issues – that is just as challenging as being a professional actor, after all. They, too, do what they can under the circumstances and place their faith in improving through experience. Frequently, they will have been commissioned by managers or political bosses who are interested in evaluation for command and control.

Of course, few people who do the evaluations intend this. If you have a spare life moment that you are prepared to sacrifice, do an internet search of conferences for evaluators and. look at their titles. Here are a few I just came up with:

• Evaluation for change: change for evaluation
• Transformations
• Speaking truth to power
• Visionary Evaluation for a Sustainable, Equitable Future
• Shaping better futures
• Co-creation
• Evaluation for more resilient societies
• Exemplary evaluations in a multicultural world

These lean towards a morality and an ethic that are not often enough allowed to be reflected in the way we experience evaluation. They are aspirational, inviting – uplifting, even. ‘Speaking truth to power’ is rarely accomplished in a world in which evaluation is something wielded by the powerful themselves (who else can afford it?) and most often looks ‘down’ the system at people and hardly ever ‘up’ the system at governance and policies.

But these titles do suggest possibilities for how we might, indeed, use evaluation for better purposes:

• to represent rather than ignore the complexities practitioners face
• to look forward to new ways of thinking rather than to look backwards in order to justify what happened
• to encourage people to be open and conversational about their practices rather than defensive and private
• to reward critical reflection on shortcoming rather than threaten it with punitive measures
• to create spaces where people share experiences rather than compete with each other
• to allow policies to be shaped by the wisdom of those who face the challenges of change on the ground – or, at least, to learn from them

There are methods that have been developed to do these things, and occasionally we find examples of them (a previous Blog looked at the work of the Education Review Office in New Zealand which is based just on these principles). But even where we find the political space to go down this route we have mountains of inertia to overcome while people adapt their fears of the evaluation menace into some form of trust. Low trust has been seeded and taken deep root in our public world, and the casualties are many. They are people, practices and organisations distorted by accountability demands. We lose enormous amounts of essential information that just do not fit with the evaluation measure, or which are too dangerous to reveal to the inspector. Democracy – the free flow of argument and information – is corroded. Humanism takes a step back. 

** Want a quick test to show how little we know of learning? Think of something you learned recently. Okay – who was it that learned? Who is it that is thinking right now about that person who learned. Are they the same people? And what you learned – why did you learn that? Is the you before that learning the same as the you after it? How do you know you learned it – did you learn it, or is it just that you can (right now) remember it? Having learned it, have you just acquired a bit more memory, or has it changed who you are – and in what way? Now think of something else that you learned recently – would you answer these questions in the same way?