What Confused the Evaluators

by Rebecca Thomas

Day 2 — Part 2

And so, after all the fishing, came the bread and butter.

Teaching.

ERO wanted to see what actually happened inside our classrooms.

Before they ventured out, they asked us a seemingly simple question: What might we expect to see?

So we told them: WALTs, success criteria, explicit teaching, effective use of teacher aides, feedback and feedforward, power sharing, student agency, engagement, excitement. A curriculum shaped around the children sitting in front of us rather than some imaginary children we wished were there. The list grew. They looked at it. And I looked at them looking at it. There was a pause. Perhaps I imagined it. But after two days of numbers, graphs, attendance figures, achievement data, questions, evidence and more questions, our list must have looked rather ambitious. Because numbers, our numbers, told them a story before they ever walked through a classroom door.

Low attendance.

Low achievement.

And somewhere, surely, there must be a loose thread.

Pull it hard enough and perhaps the whole jumper will unravel.

Maybe leadership?

Maybe teaching?

Maybe curriculum?

Maybe wellbeing?

Maybe expectations?

Somewhere there must be something that explains the numbers.

So off they went, lists and TOF forms in hand. Our doors were wide open, classrooms were visited. Children were watched and teachers observed. And I waited. This was the bit I wasn't worried about. Not remotely. They could open any door they liked. They could walk into the beautifully decorated rooms, or the slightly chaotic ones. They could arrive during explicit teaching, or catch someone kneeling beside a child who wasn't quite ready to learn yet. They could watch our teacher aides and listen to our tamariki. Watch who held the power. Notice who spoke. Notice who listened. Notice whether our children had agency or simply followed instructions. Notice whether te ao Māori lived in our kura or had merely been laminated and attached to a wall. They could look. Really look.

And they did.

Then they returned to the office, and something rather interesting happened. The evidence had refused to follow the script. They had found consistency. They had found the things we said they would find: Explicit teaching, clear learning intentions, success criteria, feedback, feedforward, purposeful support, power sharing, student agency, engagement.

But they had found other things too: Aroha, patience, warmth, beautiful spaces, safe spaces, dry spaces. Teachers who knew their children. Adults who shared power rather than guarded it. And te ao Māori woven through the kura rather than brought out for special occasions.

Suddenly the numbers had company.

And perhaps that is where things become uncomfortable for any evaluation system. Because if poor outcomes automatically meant poor teaching, the equation would be wonderfully simple. Find the broken cog. Replace it. Measure again. Problem solved. But what happens when you go looking for the broken cog… and can't find it sitting in the classroom? What happens when the classrooms are doing many of the things the evidence base says classrooms should do? 

What happens when children are engaged?

When teachers are teaching?

When relationships are strong?

When leadership knows exactly where the challenges sit?

When the culture is visible, lived and breathing?

And yet the numbers stubbornly remain the numbers.

Perhaps then you have to entertain a far more inconvenient possibility.

Maybe some educational outcomes cannot be explained by what happens between 9am and 3pm. Maybe schools don't control every variable in the experiment. Maybe attendance data tells us something about school — but also something about poverty, housing, transport, health, history, trust, transience, systems and the complicated lives being lived beyond the gate. Maybe achievement data is important without being omnipotent. Maybe the numbers don't lie. But perhaps they don't always know why either.

I don't know what the evaluators thought as they returned from their walk.

I only know that what they had seen appeared to complicate the story.

And evaluation doesn't particularly like loose ends.

So where do you look next when the classrooms refuse to provide the answer?

Apparently, you open another cupboard. Policies. Board assurances. Systems. Processes. And then… the SENCO. Lucky me.

The fishing expedition wasn't over.

They had simply moved further upstream.

And then I had my moment. The SENCO drilling had begun. So I explained our Te Rōpū Manaaki approach. I explained our tiers and systems. The children we monitor and wrap support around. The agencies and interventions. The referrals and the meetings. The haunting declines that follow you home for tea.

But mostly, I explained that we are the school that says ‘yes’. Yes to the child another school has struggled to hold and to the whānau who needs another chance. Yes when it would be considerably easier, tidier and perhaps statistically wiser to say no.

Then there was another question. “But that must all come at a cost?”

“Yes,” I replied.

“Our cost is those numbers up there on our wall.” Those red and orange squares by which we are so easily defined. Our untidy numbers. The children who don't make a spreadsheet look pretty. The children other places haven't always managed to hold. The cost is that not every teacher stays. Because when you put that much trauma together in one place, sometimes you get popcorn. It pops.

We know that. We don't pretend otherwise. But we still say yes. Then came perhaps my favourite question of the whole review. “How could you capture evidence of the progress you do make to go alongside the narrative?” And in return I asked a question of my own. Who would I be capturing it for? Would capturing it change anything? Would another six hours sitting at my laptop, carefully turning human progress into evidence, make things better for my kids? Or would it make things better for the people evaluating them? 

We know how that question was answered.

And perhaps somewhere in that exchange we finally arrived at the heart of the problem. Because we can capture more. We can measure more. We can create another tracker, another spreadsheet, another colour. We can turn the child who finally stayed in class for an hour into a percentage increase. We can quantify the whānau who trusts us enough to answer the phone now. We can put a number beside the child who used words instead of fists. We can create a rubric for belonging. A scale for trust. A graph for mana. We can do all of that. But eventually somebody has to ask: For whom? And at what cost? Because evidence gathering is not free. Every hour spent proving that something happened is an hour that cannot be spent making something happen.

That doesn't mean schools shouldn't be accountable. We absolutely should. It doesn't mean our numbers don't matter. They absolutely do. And it certainly doesn't mean we accept them. We don't. But accountability should improve the lives of children, not simply improve the documentation of the adults responsible for them.

In the end, I think they understood.

At least, I think they understood us a little better.

But they still have a job to do. I respect that. They have to take all those conversations, observations, documents, graphs, red squares, orange squares and stubborn pieces of context and make them add up to something. There has to be an explanation somewhere. There have to be things to improve. There has to be something to write. I get that.

The curious thing was that, by the end, there wasn't some great revelation waiting for us. No missing programme. No magical intervention. No single piece of advice that would suddenly make our attendance rise, our achievement soar and all those inconvenient squares turn green. Because perhaps there was never a broken cog to discover in our classrooms. Perhaps what they found instead was something much harder to fit inside an evaluation framework.

A school doing good work in extraordinarily complicated circumstances. A school that knows its numbers. A school that doesn't like its numbers. A school working relentlessly to change those numbers. And a school unwilling to change who it says yes to simply to make those numbers easier to explain.

For two days, we were living proof that education in places like ours does not always add up neatly. The narrative can be strong while the numbers remain stubborn. The teaching can be good while achievement remains low. The aroha can be enormous while attendance remains poor. Both things can be true.

Maybe that was what confused the evaluators. Or perhaps, for a moment, it simply made the numbers harder to believe without knowing the names behind them.

Tomorrow they'll take their notes away. Eventually, our story will become their words. The graphs will remain. The squares will remain. And somewhere, somebody will still try to make it all add up. But tomorrow morning our gates will open again. The same children will walk through them. And we will do what we did before anybody arrived with a clipboard. We will say yes. For us, it will be just another day.

Day 1 Part 1 here