Showing posts with label VAMs. Show all posts
Showing posts with label VAMs. Show all posts

Monday, March 10, 2014

Public Education Meets Carrot and Stick: Kids Lose

The only advantage I can imagine to being stuck on the tarmack for 2 hours at Austin-Bergstrom Airport last week while the plane was being de-iced was that I got time to read Greg Anrig's excellent book, Beyond the Education Wars: Evidence that Collaboration Builds Effective Schools, which  has been sitting on my Kindle for a few months now. Anrig, is vice president, programs, at The Century Foundation, author of several books and a former journalist.

Anrig's book begins with a brief history of management theory, focusing on two dominant theories put forward by Frederick Winslow Taylor and W. Edwards Deming. Taylor's thoeries eminated from the idea that workers could not be trusted and so he designed a management style based on punishment and reward. Taylor determined how much a worker should be producing and set workers goals accordingly and then tied their pay to productivity. Modern exemplars of the Taylor approach are bill collecting agencies, customer service call centers and national retail chains (Walmart?).

Decades later, Deming, working in Japan after World War II, argued that long term business success required constant incremental progress, the kind that required workers and management to be deeply engaged in sharing information and ideas with each other. Modern day exemplars of the Deming model include Toyota, Sony, Southwest Airlines and Kaiser Permanente.

Clearly, as Anrig points out, the national political debate over education reform has been driven by the Taylor model of management. There is a direct line from the Taylor model to such reformy ideas as merit pay, standardized testing and tying teacher evaluations to test scores. In other words, education reformers have embraced Taylor's carrot and sticks management style of education that failed the American auto and steel industries and that is currently used for only the most low levels of employment.

Anrig shows that there is ample evidence that this carrot and stick method will not work in schools and plenty of evidence that a more collaborative approach, such as that endorsed by Deming, and used by many of the most successful corporations in the world, is a better way to go.

Anrig cites solid research reports starting in 1990 that support the concept of a collaborative design for school improvement. These studies include those done by the Consortium for Chicago Social Research, The National Center for Educational Achievement and Century Foundation fellow Gordon MacInnes all of which come down to two key findings, strong leadership matters and a strong culture of collaboration between teachers and school leadership matters most of all.

The studies found that change is slow and incremental, but it can be successful in any setting, including in high poverty areas, although change is even more challenging there.

Education reformers do not trust teachers. Because they do not trust teachers, just as Taylor did not trust industrial workers long ago, reformers put forward management plans that seek to control workers, rather than engage professionals in systematic improvement. Education reformers wish to motivate change with the carrot of merit pay and the stick of the highly flawed Value Added Measures (VAMs) tying teacher evaluation to student scores on standardized tests.

We know, and Anrig's book very persuasvley lays out, what works. Collaboration works. Teachers and principals working together to improve schools works. Having teachers compete for merit pay through flawed VAMs, not only will not work, but it will destroy the very collaboration that is so necessary for a highly functioning school culture.

We want teachers working together to share instructional ideas, to puzzle out solutions to persistent problems, to design authentic assessments for students. The concept of the professional learning community, where teachers meet to share ideas and solve problems holds great promise for the continuous improvement of teaching and learning. The test and punish model of the corporate education reformers will inevitably kill collegiality and cooperation and, ultimately, public education.

Concerted group action is the enemy of corporate education reform movement. That is why the reformers demonize teacher unions and support policies that will turn teacher against teacher. By driving a wedge through the profession, the education reformers hope to destroy public education and turn it to a for profit enterprise run like a widget factory. This is one way to distract the public from the real issue facing school improvement - poverty. And poverty is one thing the corporate education reformers do not want to touch. Really dealing with poverty would upset the status quo that is paying off so handsomely for the 1%.

In these times, it is more important than ever for teachers to stick together, not only in the sense of unions, but also in the sense of building and maintaining the kind of collegial culture in the every school  that reasearch has shown is the one true way to effect continuous improvement.

Reading Anrig's book will help arm all in the profession against the false narrative of the corporate education reform movement.

Anrig, Greg. (2013) Beyond the Education Wars: Evidence that Collaboration Builds Effective Schools. New York: The Century Foundation Press.





Sunday, February 16, 2014

All Kinds of Kinds: Evaluating a Teacher


               Evaluating a teacher demands nuance, not numbers
.

The Grammy Award winning singer/songwriter, Don Henry, has a song with a refrain that says, “Ever since the beginning to keep the world spinning, it takes all kinds of kinds.” I think the same is true when we look at the teaching profession.  

As a student in elementary school, secondary school, college and graduate school, I have had a few great teachers, many good teachers, and a poor teacher or two. I remember my third grade teacher, Ms. Miyamoto. Ms. Miyamoto was from Hawaii. She taught us to dance the Hula, how to prepare and eat poi, and other basics of Hawaiian language and culture. I remember mostly how she was unfailingly kind. For an 8 year-old-boy in 1955, her classroom was a valuable lesson in diversity and respect for other cultures. Ms. Miyamoto was a new teacher and she had her challenges with what we now call “classroom management.” I am not sure how she would be rated as a teacher today, but for me she was a good teacher indeed.

In sixth grade I had Mrs. Stout. I can say without equivocation that Mrs. Stout was a great teacher. I don’t know how she did it, but Mrs. Stout made every child in the classroom, including the class clown (moi), believe that they were intelligent and destined for great success. The one thing I remember that Mrs. Stout taught us was how to outline. She said we would need it in high school and college. She always assumed we were going to college, even in this working class suburban neighborhood in 1958 where college was far from a bygone conclusion. She made us all feel good about our potential.

In eighth grade history, I had Mr. Laidacker. I doubt that Mr. Laidacker was any administrator’s idea of a great teacher, but he was for me. Mr. Laidacker was strict (some said mean), aloof and totally involved with himself. He ignored the 8th grade curriculum (American History) and spent nearly the entire year teaching about his passion, The Civil War. And I mean passion. Mr. Laidacker knew more about the Civil War than Abraham Lincoln. And he could tell stories about the war that lit up the mind of at least one 13 year-old in that class. I can still remember in detail the report I did for him on Civil War medicine. Mr. Laidacker was not liked by most of the students in the school, but he is one major reason I became a history teacher. That passion was infectious for me.

I am sure that anyone reading this can name teachers who had similar ompact on them. The point is that teachers who were good for us, as individuals, may not have been good for other kids. But it takes all kinds of kinds in this world and all kinds of teachers to educate a child. Because of this evaluating a teacher demands nuance, not numbers.

The current mania for weeding out “bad teachers” and evaluating teacher effectiveness through student standardized test scores is wrong headed in the extreme. Teaching is one part art, one part science, and one part mystery. One child’s favorite teacher may be the wrong teacher for another child. Evaluating a teacher is a complex process, that certainly includes evidence of student learning, but it also includes many, many more things. This evaluation cannot be reduced to a number on a standardized test or on a rubric. This type of reductionism does a disservice to the complexity of the enterprise.

The evaluation of a teacher requires a knowledge of and sensitivity to this complexity. That is why I consider teacher evaluation to itself be part science, part art, and part mystery. The science is the easy part. Was the lesson well-planned? Were the students engaged? Did the teacher answer questions thoughtfully? Was the objective achieved? Did the students learn?

The art is more difficult. Was the lesson made relevant? Was it differentiated appropriately? Did children feel safe taking risks in this classroom? Did small groups function well? Were routine matters handled briskly? How effectively were problems handled? What is being learned in this classroom beyond the subject of the instruction?

The mystery is even more difficult. Did magic happen in this class? Did students feel cared for in this classroom? Was there an atmosphere of mutual respect? Does the teacher’s passion come through? What students might be inspired by this teacher? Was the joy of leaning evident?

I like to think of the act of evaluating a teacher as informed impressionism. Just as the impressionist painter, Claude Monet, used thousands of brush strokes to form the whole of a painting of a garden, so the skilled evaluator looks through multiple lenses to gain an impression of the whole of a teacher. This evaluation should be informed by student learning data, but that data is only one part of multi-faceted picture. It must also be informed by multiple observations of teachers working with children and working with colleagues. It is the product of the review of lesson plans, but also the product of conversations with the teacher that probe decision making and the ability to improve through reflection.

In other words, teacher evaluation is not easy and it requires considerable resources in personnel, time and money. The current calls for judging the complex act of teaching through reductionist and faulty value added models and theoretical rubrics are an insult to the complexity of the profession. If we are really serious about understanding and improving the evaluation of teachers, then let’s get serious about just how difficult that task is.




Wednesday, February 5, 2014

The Common Core: Is “College and Career Ready” the Right Target?

Why "college and career ready?" Could it be because this fits the corporate reform agenda?

The designers of the Common Core State Standards (CCSS) determined that the purpose of a K-12 education was to insure that children were “college and career ready” by the time they completed high school. Having determined that our students were not “college and career ready”, they looked at what would be required of a graduate in the areas of literacy and mathematics to be deemed ready and built the standards backwards from high school through middle school to elementary school to provide stepping stones that would lead to that final goal. Holding aside for a moment whether this design construct makes any sense, can we say that “college and career ready” is the appropriate target for public education? And further, if it is not the appropriate target, why have the developers of the Common Core put so much stake in it?

Who could be against standards designed to make children “college and career ready?” It sounds wonderful doesn’t it? Perhaps I have become too cynical. Whenever I hear a slogan that sounds too perfect, I get suspect and go into my Orwellian doublespeak mode. A great example of this is the Bush era “Healthy Forests” Act. The Healthy Forests Act was nothing more than a handout to the lumber industry allowing them to cut down trees and earn profits in areas where this had not been allowed before. It is telling that the “Healthy Forests” Act was opposed by every legitimate environmental group in the country, but supported by Bush’s big business friends. Is “college and career ready” nothing more than a gift to the education privatizers and wealthy corporations who are eager to support the CCSS?

For me “college and career ready” is too narrow a goal for public education. The problem with narrow goals is that you may find ways to achieve them and you may even find ways to demonstrate that you have achieved them, but because your goal was too narrow, you never achieve the larger goals necessary. For example, you may set a goal to have children learn to decode words. You may focus on that goal and you may, and likely will, achieve some measurable success. The problem is that you spent so much time teaching decoding, that you failed to focus on the real purpose of reading, and so you have created kids who have improved in decoding, but still can’t read. Getting meaning from reading is the true goal, not decoding. In public education, an informed and participatory citizenry is the goal. Does “college and career ready” get us there?

I would propose a broader target for public education. It is a target that I believe most of us would easily say “amen” to and yet we have watched as the goals for public education have been narrowed to something “measurable.” To my thinking the education of any human being has four dimensions.

1.    The life of the citizen
2.    The life of the mind
3.    The life of the body
4.    The spiritual life

The spiritual life is, and should be, outside of the purview of public education, but all three others are necessary targets of schooling.

The United States of America is the world’s greatest experiment in democracy. In order for that experiment to sustain itself, an informed populace is necessary. This is, of course, the original purpose behind public education since the days of Horace Mann. This has not changed. In a world where “news” organizations have become little more than political organizations flogging partisan viewpoints 24/7 and where all sorts of unsubstantiated opinions (like this one) are readily available on the internet, it is as important as ever to prepare children for informed participation in our democracy. Of course this preparation will include many of the abilities discussed in the Common Core, including critical thinking, problem solving and communication skills. It would also include empathy for fellow citizens or citizens who are less fortunate. Despite CCSS author David Coleman’s disdain for “how you feel or how you think”, how you feel and how you think are critical components of being a contributing member of a democracy.

Despite what appear to be longer and longer work days for many Americans, we can safely say that most Americans will spend at least 1/3 of their waking lives at leisure. The quality of how that 1/3 of a lifetime will be spent is determined, at least in part, by our education. One target of public education, then, should be developing the habit of mind that leads a person to be a lifelong learner. Lifelong learners spend their leisure time in a variety of pursuits, not one of which is the “right” one. The key is how rich and rewarding these pursuits are for the individual.

That is one reason why in public education a dynamic arts program is critical. Despite President Obama’s view of art history majors, the arts are at the center of a quality public education. Children need exposure to all the rich visual and performing arts, the chance to participate in a chorus or a band or a dramatic production. They need to develop the teamwork that is a part of these activities. They need the chance to experiment with a variety of artistic expressions and they need to get a sense of the history of the arts. We are failing our children and our country if we do not provide them with opportunities that will allow them to have a lifetime of continued learning and the joy that comes from being an informed witness or participant in artistic performance. Public education should prepare our children, then, for the life of the mind.

Finally, there is the life of the body. Strong public education includes strong physical education programs and strong sports programs. It includes health education and nutrition education. One of the consequences I have seen from the three decade long push for higher standardized test scores is the continued diminution of the importance physical education programs and recess in schools. I recently visited a school where the children get one forty-minute physical education class a week and recess is a part of a 25-minute lunch that barely gives the kids time to get their lunches and eat. This is not acceptable, but unfortunately it is also not unusual. Children need plenty of opportunities for directed physical education and unstructured play during the school day. In addition the public school should provide organized sports programs, not only for talented varsity level athletes, but intramural programs for all children who only wish to participate in organized sports.

I am reasonably sure that these goals for public education are shared by most Americans. Some might rightly point out that the architects of the Common Core recognized that “particularly in the early grades social, emotional, and physical development are important considerations (CCSS).” Unfortunately, despite such qualifiers as this, with the CCSS tied so closely to standardized tests and teacher evaluation, they are bound to become the defacto curriculum for public schools.

It appears that the authors of the CCSS recognized that there was more to a public education than these standards. Why then the focus on the narrow goal of “college and career ready?” I think the explanation can best be understood if we look at who has funded the CCSS, who are the biggest boosters and what other initiatives the Common Core is tied to.

The development, roll-out and promotion of the Common Core was underwritten to the tune of hundreds of millions of dollars by Bill Gates and The Gates Foundation. Why would this captain of industry want to spend so much money on these standards? If you asked him, I am sure Gates would tell you that he was looking for a way to do good with all his money and this was the way he thought he could do it best. The problem is, of course, that this means the CCSS is representative of Gates’ world view. That world view is colored by his status as a wealthy “self-made” man. What was good for Gates would be good for all America’s children. So we get corporate, privatized education designed to produce a bright and compliant supply of employees for our multi-national corporations.

The true goal of the Common Core State Standards also cannot be understood without viewing them as a part of a total package that includes standardized testing and teacher evaluation. The CCSS had to be narrowly cast because student achievement of the standards was to be subject to standardized testing. It is very difficult to measure empathy, artistic appreciation and physical well-being with a standardized, fill in the bubble test. The CCSS gets rid of all that messiness by simply focusing on narrow, testable standards. Learning is complex and measuring learning is a complex activity that often must be done on the local level; that is, in the classroom, by the teacher who knows the children. The CCSS and the standardized tests they are tied to cannot capture the complexity of teaching and learning.

Why are the standardized tests necessary? Well, one reason is to gauge student progress, but the other reason is to evaluate teachers. The CCSS cannot be separated from the desire to quantify teacher performance. The Common Core leads to standardized tests of children, which leads to “value added” measures of teacher effectiveness. Despite the overwhelming evidence of the fallacious nature of “value added” measures, value added teacher evaluation remains a part of the reform agenda and are inextricably linked to the Common Core. Teachers who like the Common Core, but abhor all the testing and test driven accountability need to recognize that these are all a package.

So, why “college and career ready?” Why design standards that are “standardized test ready?” Why evaluate teachers with a widely discredited value added statistical model? Why take the easy road of an easily quantifiable set of standards that ignores the complexity of learning in all its messiness, all its potential and all its beauty? Why? Because these “reforms” serve the agenda of the corporate privatizers who wish to eliminate the bothersome democratizing impact of public education.






Sunday, October 13, 2013

Responding to TEACHNJ Enthusiasts

Saturday's Trenton Times had this editorial enthusing about TEACHNJ the new teacher evaluation law.
http://www.nj.com/times-opinion/index.ssf/2013/10/opinion_teachnj_reforms_are_ma.html

I wrote the letter below to the editor in response.

To the editor:
I was certainly entertained by reading Janellen Duffy’s rather breathless endorsement of TEACHNJ in Saturday’s (October 12, 2013) edition of the Trenton Times. In these days of rampant cynicism it is nice to read such gosh-golly-gee-willikers enthusiasm in print. Alas, if only Ms. Duffy had saved her enthusiasm for something worthwhile.

TEACHNJ is a law that resulted from the determination of Commissioner Cerf and Governor Christie to use student test scores to assess teacher effectiveness. This effort alone is fraught with problems. First of all, these tests are designed to measure what students have learned, not teacher effectiveness. Using a test designed for one thing to measure another violates the first rule of statistical measures. Secondly, the ability of the Student Growth Percentiles (SGPs) to determine teacher effectiveness has been consistently questioned in the education literature.

 Only 20% of teachers teach classes where students are evaluated using State tests (math and language arts in grades 3-8). It would obviously create discord in a school if a few teachers’ jobs were on the line due to test scores, while the other teachers skated by. The DOE solved this problem by making all other teachers develop Student Growth Objectives (SGOs). While this might serve the purpose of keeping some peace in the school building, there is no evidence that SGOs are an effective way to measure teacher effectiveness. The DOE’s own website points teachers to an article that says, “there are not yet any research-based models for evaluating teachers’ impact on student learning for non-tested groups”(Goe & Holdheide, 2011). This is the kind of measure we are to use to make tenure decisions?

To read Ms. Duffy’s article you would think that teacher evaluation and professional development did not exist prior to TEACHNJ. In my experience as a school district administrator, I have seen rigorous programs of evaluation and professional improvement in many New Jersey school districts. I have seen the vast majority of teachers well prepared and willing to work to improve. I have seen the vast majority of administrators who are knowledgeable about instruction and dedicated to helping teachers do well. TEACHNJ is a slap in the face for every education professional in the State.

Should teacher evaluation and professional development be better? Certainly. Will tying teacher evaluation to test scores and growth objectives improve student learning? The research would indicate that it is not likely. If teachers based their instructional practice on research as shaky as this, they would certainly not receive a good evaluation. Perhaps it is time to evaluate the evaluators.

Wednesday, October 2, 2013

The VAM Moose: Coming to a School Near You



Concerned with the negative publicity surrounding the use of Value Added Models (VAMs)* for teacher evaluation, the Grates Foundation, the Foundation for Excess in Education, RheeFirst and other proponents of such models have decided to take a new approach to promoting the controversial mathematical model. “People are having difficulty grasping the complicated statistical formulas,” said spokesperson Eric “Fuzzy” Numbers. “We needed to come up with an entertaining way to promote the program. We were sitting around crunching incomprehensible data one day, and we hit on the idea of a mascot. We were thinking along the lines of Smokey the Bear or the San Diego Chicken. And so, VAM Moose was born. I think we have really hit the jackpot with this one. At first we were going with a ferret, but decided the moose was cuddlier. His motto is, “Raise those test scores or VAM Moose!" Catchy, huh?

The VAM Moose and his motto will be featured in public service announcements, billboards and internet pop-ups promoting value added assessment of teacher performance. Asked about the long series of reports from respected organizations like ETS, the Educational Policy Institute and the Rand Corporation calling into question the use of VAMs for any high stakes decisions, Mr. Numbers said, “That’s why we need the moose. We need to turn around the conversation and facts won’t help us. Nothing like a soft and cuddly mascot to sell a flawed product. Look what Tony the Tiger did for Frosted Flakes.”

In a related story, Walmart has announced it is launching a line of stuffed talking VAM Moose dolls to go on sale in time for the Christmas season. The cute and cuddly item, manufactured in Guatemala, will have a pull string that when activated has the moose say, “No excuses! No excuses!”

* For the uninitiated, VAMs are statistical models that use student scores on standardized tests to mis-measure teacher effectiveness. They have been very popular with education reformy types. For a fuller, and more sober analysis of this flawed tool please see the following.


Monday, September 9, 2013

Valuing the Professional Educator: Toward an Evaluation Plan that Works, Part 1


By the time a teacher receives the first evaluation of his/her performance, the school district has already invested a great deal of time and money in that individual. First of all there were the recruiting costs involved in finding the candidate. Secondly, there was the time and effort put in through interviews, demonstration lessons and background checks to determine that the candidate was a good fit for the district and the position. Next there was the hiring process itself and the paperwork and man hours involved there. Finally, the new teacher received some sort of orientation training that may vary from a few hours to several days. In other words, the school district has chosen and created an employee of value, an asset to the district as indicated by time and money invested.

Any system designed to evaluate the performance of this employee must first recognize that this teacher is an employee of value. Acknowledging this brings an entirely new orientation to the evaluation process. Instead of an evaluation system that tries to find teacher shortcomings, evaluation becomes a process of recognizing the value of teachers to the organization and helping professionals consistently improve performance.

This gotcha’ model of evaluation is where the education reformers have made a critical error. The main purpose of the evaluation system is not to catch out underperformers (of which there are far fewer than reformer rhetoric would suggest), but to help teachers become better at what most of them already do pretty well. A sound evaluation system, focused on teacher development, will identify those who are not able to improve performance, but its first goal will be to help the valuable professionals in their own pursuit of continuous improvement.

The reformers do have one thing right, evaluation of teachers needs to get better, but not primarily because we must weed out the underperformers. What we need is to provide valuable feedback to the professionals who can use it to improve performance. The darling of the reformers, Value Added Measures (VAMs), fail as an instrument of teacher evaluation for many reasons, mostly because there is no way to come up with a reliable score as Darling-Hammond, et.al have shown here.

Another failure of VAMs is that they do little to move the profession forward. They provide little information that would be useful to a teacher trying to improve performance or an administrator trying to provide meaningful feedback to a teacher.

What would an evaluation scheme look like that valued teachers and provided the feedback they needed? Here is an outline of the basics.
  1. Teachers and school leaders are empowered to jointly customize, adapt, and implement an evaluation framework.
  2. Observations are a mix of a few full period visits and several shorter visits. All observations involve feedback to the teacher in some form aimed at professional development.
  3.  Feedback is provided on all domains of professional practice including planning, instruction, developing a learning environment, meeting professional responsibilities and professional development.
  4. A variety of pupil progress indicators including those designed and scored by the teachers themselves in conjunction with the limited use of some standardized measures.
  5.  Collaboration among teachers as a part of a continuous improvement model.
In a future post, I will expand on each of these ideas and how they can work in a school or school district.