Yes, Schoales says this was a great idea, implemented badly. While I praise Schoales for admitting the scheme doesn't work, he has learned the wrong lesson. The very idea upon which this evaluation scheme was built was so flawed that there was never any hope of it being successful. Others have recounted in great detail how value added measures (VAMs) are hopelessly flawed. Both the American Education Research Association and the American Statistical Association have declared VAMs misleading and of limited use. Audrey Amrein-Beardsley has written a great book about it.
But the real flaw in all these reformy teacher evaluation plans is in a failure to see what teacher evaluation really is built on. Teacher evaluation is not built on value added scores, or rubrics, or student scores on standardized tests or even primarily on classroom observations. The great flaw in these reformy schemes, including those MET Studies promulgated by Bill Gates, is that for all their "data" they fail to recognize the most basic of drivers behind evaluation - trust. Teacher evaluation is built on relationships. It is built on the trusting relationship between teachers and supervisors.
Reformers can't see this very simple and most basic fact of teacher evaluation because they are focused on a fool's errand of seeking objectivity through numbers and a plan designed to weed out low performers, rather than a plan designed to improve performance of all teachers. These folks could have easily found out the flaws in the plan. All they needed to do was spend some time in schools talking to teachers and supervisors. To the extent that current teacher evaluation schemes interfere with teachers and supervisors developing trusting relationships, they are pre-ordained to fail.
I spent 15 years as a public school administrator charged with evaluating teachers. I knew going into the job that my main goal would be to provide teachers with useful feedback for improvement and that if the teachers were going to be willing to implement that feedback, they were going to have to trust me and trust that the feedback I gave was well-informed. I also knew that, while part of my job was to identify poor performers and place them on an action plan for improvement or remove them from the classroom, that the vast majority of the teachers I would be working with, say 95%, were competent professionals who would not be targets for removal. It only makes sense that an evaluation program spend most of its time on professional development rather than on trying to identify low performers.
And so, like most supervisors in public schools across the country in the pre-NCLB/RtTT days, I set about building relationships, listening to teachers, providing information and demonstration lessons, leading book clubs, observing instruction, and sitting down with teachers to provide constructive feedback. To the extent that I could show teachers I knew what I was talking about, the teachers would buy in. Sometimes in these discussions, I would just listen and learn. These evaluations were two-way streets and sometimes the teachers had better understandings than I did. More than a few times I changed an observation report after a conversation with a teacher showed me the thought behind instructional choices that I did not recognize.
When I had to act on a poorly performing teacher, my work on relationships paid off again. As a supervisor who worked to gain the trust of individual teachers, I also worked to gain the trust of the teachers union. Reformers want to paint the unions as the enemy, but coming out of the teacher union movement myself, I knew that it was not in the interest of the union to protect poor performers. Of course, the union's job was to be sure that members got due process and some administrators would see this as obstruction, but I did not find it so. On the several occasions when I needed to recommend the removal of a teacher, the union representative and I worked together to make it happen in as respectful a way as possible for the individual, but with the clear goal of improving educational outcomes for the students.
As I visit schools these days, I worry that the trusting relationship between supervisors and teachers is being undermined by policies that encourage a "gotcha" mentality, rather than a growth mentality. Data can help inform teachers, of course, but when the data is built on arcane statistical formulas that are far removed from the reality of the classroom and built into assessments that are far removed from the instruction in the classroom, that data and the people who deliver it will be seen by teachers as untrustworthy. Here is how I put in my book, A Parent's Guide to Public Education in the 21st Century (Garn Press).
Successful evaluation systems must be built on trust. Teachers do not trust the results of value-added measures for good reason; they have been shown to be unreliable, unfair and invalid. Trust is built through informed supervisors and reflective teachers holding professional conversations with each other around classroom practice and student performance. Once trust is established, the hard work of instructional improvement can begin (p 129).
I am not one to pine for the "good old days" and I do believe in bringing new ideas into the schools and in the importance of having a good teacher in every classroom, but to the extent that corporate education reform policies and practices are destroying the trust between teacher and administrator, I have to say we are moving backward rather than forward. Do we need sound teacher evaluation policies? Undoubtedly. Will we get there with VAMs, rubrics,and standardized tests? Never. Can building trust between all stakeholders help? You bet.
If you do not have a subscription to Education Week and cannot access the Schoales article, Diane Ravitch has a good summary here.
My take on what teacher evaluation should look like from a two part series I wrote four years ago can be found here and here.