Jim Manzi Earns an A+ Evaluation From Me
I want to thank Jim Manzi for writing a truly excellent post on teacher evaluations. His points 1 and 4 in particular are almost universally not understood.
I do want to add a few additional points of my own:
1. Evaluations establish the principle that there is such a thing as performance in the first place. A great deal of discussion nowadays in education revolves around the idea that what we need to “fix the schools” is great teachers. But if that’s what we need, we’ll never do it. What we need, instead, are mechanisms for getting marginally better performance, year after year, from a teaching pool that remains merely adequate.
One bit of low-hanging fruit for achieving that goal, meanwhile, is the ability to dismiss the bottom 5% of teachers in terms of performance. Not only are these teachers failing comprehensively in their own classrooms, but their mere presence has a corrosive effect on an entire organization – on the teachers, on the students, on the management of the school. But right now, firing these teachers is essentially impossible. For all the difficulty of doing a rigorous evaluation in order to improve teaching performance across the board, I suspect it is a whole lot easier to identify the worst teachers in the school. If that could be done, the pressure to be able to terminate them would be significant, and that could do a lot to improve school performance right there.
2. Value-added metrics wind up punishing perfectly good but not spectacular schools with above-average student bodies. It may be that these schools should suffer reputationally, because the staff is not actually delivering as much value as they should. But high-stakes standardized testing actually pushes these schools to destroy themselves, wiping out the programs that actually do deliver value to these high-aptitude students and instead focusing on teaching to the tests.
That’s not an argument against using value-added metrics as such. It’s an argument that they need to be used intelligently, with some understanding of what “value-added” means at different points on the performance spectrum. But that, in turn, would require admitting that different standards are needed for students with different aptitude, which, in turn, is extremely difficult for our education system to admit. (And, admittedly, it’s a problem in corporate cultures that cross widely different customer bases as well. How well would Wal-Mart manage Tiffany?)
3. Nobody goes into teaching “for the money” – that is to say, teachers in aggregate make significantly less than people with their educational credentials and academic aptitude could make in other professions. So monetary rewards are useful primarily going to prove useful as signaling devices. There’s a lot of evidence coming in from high-performance charter schools suggesting that a monetary reward system tied too closely to evaluations actually degrades performance, because it gets teachers focused on the evaluations rather than on the performance. The evaluations should primarily be used as a diagnostic, to identify correctable deficiencies in teacher performance so they can be corrected through staff development, and to identify gross deficiencies in teacher performance so the teachers in question can be dismissed.
4. Similarly, across a system, what evaluations are useful is for research purposes and to drive market discipline. Evaluations of a school should be very useful to parents seeking to select a school for their child. Schools that consistently achieve high valuations (particularly for value-added metrics) should be objects of study by administrators and others looking to replicate that performance in lower-performing but still basically well-run schools. The least-important use of the evaluation is to directly “reward” or “punish” a school bureaucratically – and, indeed, if that becomes the primary use then the school is likely to start focusing overwhelmingly on the evaluation process and lose sight of actual performance. I’ve seen this happen over and over in New York City schools; it’s not a theoretical question.
As with almost all social reforms, the cause of school reform is best served by under-claiming what any particular reform can achieve. Try to move the needle a little bit with each change in the right direction, and try to line up incentives properly. Wild misrepresentations of what can be done, like the movie Waiting for Superman, actually undermine the very case they are trying to make.
Nice article, thanks for the information.
— sewa mobil · Feb 7, 07:19 PM · #
One of the things that we have to keep aware of as we discuss these issues is that often the teachers who administrators and peers would nominate as the best (or worst) are not the ones who various testing measurements would suggest are. How exactly to navigate discrepancies like that (besides being aware that we need to work on our metrics to make them less likely) is a complicated question, particularly as fair evaluation is a particularly important legal question in response to a firing. Even completely absent the presence of a teachers union, you can imagine the liability a school district would be exposed to if it fired teachers for underperforming if in fact that empirical data/test scores suggested the teacher wasn’t.
— Freddie · Feb 7, 08:16 PM · #
Nobody has explained to me why we couldn’t or shouldn’t fire the bottom 5% of cops or firefighters. Even if they are endlessly valorized and showered with stock phrase praise, wouldn’t removing the worst of them make the forces even better?
As for unions, hated by many education reformers, why do they get special vitriol that police unions don’t? After all, the only time we see police unions in the news is when they rise to the defense of some cop who shot an innocent granny or beat up a guy for looking at them funny.
Instead, when there is perceived to be a problem with crime (or drugs/terrorism more specifically), they get showered with millions of dollars worth of military-grade equipment. It is a rare occasion indeed when a writer with an audience claims that crime, like poor education, is a problem that can’t be solved with more money.
How much of this debate is about sticking it to a supposedly “soft” profession?
— rj · Feb 7, 09:39 PM · #
“Nobody goes into teaching “for the money””
No, but a lot go into it for the good health care insurance and the defined benefits pension. A system of firing the bottom 5% each year lowers the incentives to enter the profession, since they are tied so closely to sticking with the job for an entire career.
This doesn’t seem like an insoluble problem, but one that needs thinking about.
— Steve Sailer · Feb 8, 12:16 AM · #
Noah,
Thanks for the very kind words.
Pretty much everything you say here makes a lot of sense to me.
On your point 4, I’ve written a fair amount about these kinds of measurements as analogous to annual reports for companies to allow families (using funding that follows students) to make informed decisions about schools. This is a element of what I think a real (and very desirable) consumer market for schools would look like.
One question. When you say that – teachers in aggregate make significantly less than people with their educational credentials and academic aptitude could make in other professions. – do you have a source for that? I’ve been looking for good comparisons on this for some time.
Thanks,
Jim
— Jim Manzi · Feb 8, 11:39 AM · #
Jim: sorry, I don’t have a good source for that. It’s bandied about so much – perhaps it’s not true? I’ll ask around.
Steve’s right that, as with other government jobs, there are people who teach not for the cash money but for the benefits. I’m not sure people become teachers so often because of this; rather, I think this is a reason why some people stay teachers, even after they’ve lost their enthusiasm for and even interest in teaching. Flattening the salary scale and making teacher pensions more portable would, indeed, change the incentives – you’d see more turnover. That’s probably a feature, not a bug.
I hear often about people wanting to become teachers for lifestyle reasons. But I haven’t heard of that from anyone I know who actually is a teacher.
As for “hard” professions and the 5% rule – laying off the bottom-performing 5% of cops sounds like an excellent idea. Bad cops can do a lot of damage. No strong opinions about firemen, but my general sense is that the whole country has too many firemen, but nobody likes to lay them off because they are much beloved (and have a very strong union).
5% may even be higher than necessary. I doubt many private corporations would hit a figure like that in the absence of a general “round” of layoffs.
— Noah Millman · Feb 8, 12:42 PM · #
Here’s an anecdote that reflects real data: in 1977, a first year Manhattan lawyer made about a thousand dollars more than a first year Manhattan teacher. By 2007, it was better than one hundred thousand dollars more.
— Freddie · Feb 8, 02:19 PM · #
Teachers generally start out around $27k to $30K and reach an average (in 5-10 years) of around $40K. They usually get pretty good benefits and don’t have to work in summers, but generally work more than 40 hours a week in the school year. Basically, it puts you in the bottom end of the middle class.
I think what is more interesting is to compare the actual job duties, skills, and knowledge required by teaching with that of other jobs.
— cw · Feb 8, 04:15 PM · #
Noah: It would be even easier and fairer to fire underperforming cops. Most cities already have a Compstat-style system in place and the parameters are pretty clear (a cop isn’t expected to “inspire” anyone). The main problem would be selecting and weighting the right metrics so you don’t end up with a lot of junk arrests and pointless ticketing.
In fact, we can probably learn a lot about how to evaluate teachers from how we applied statistical accountability models to crime.
We’ve learned how “hotspots” can be flooded with resources only to pop up elsewhere (and how to stop that). And about decreasing marginal utility of both resources and accountability measures. And that there exists a point where the demand for better numbers turns into stat-jukeing and harassment of innocent citizens.
That is not to say that quantitave evaluation is useless – it clearly isn’t. What it should tell us is that there are limits to statistics and that methods should be designed with goals beyone just singling out teachers for termination.
— rj · Feb 8, 04:35 PM · #
Good discussion. But ultimately, testing is a question about allocation of resources. New York State’s entire “Race to the Top” grant will be used to pay for assessment tools, and the testing companies are in it for the money in a way that teachers certainly aren’t. Kaplan, which owns the Washington Post Company, owns the very lucrative contract to provide test prep materials for the New York City schools, and Joel Klein, great champion of testing, has just moved from the Chancellor of the city schools to the News Corporation, where he will work in educational publishing. In many ways, assessment is a mechanism whereby federal, state, and city education dollars get moved to large media conglomerates—meanwhile arts education and foreign language instruction get cut.
— gb · Feb 8, 04:51 PM · #
Here’s an anecdote that reflects real data: in 1977, a first year Manhattan lawyer made about a thousand dollars more than a first year Manhattan teacher. By 2007, it was better than one hundred thousand dollars more.
Success for the Leftwing-Snothead/Establishment-Republican alliance, whose polices consistently make the rich richer and the poor poorer! Glory be to a regulatory state that fosters uniformity, centralization, and winner-take-all! (To say nothing of winners who become too big to fail.)
— The Reticulator · Feb 8, 05:05 PM · #
What a substantive contribution to this discussion.
— Freddie · Feb 8, 06:37 PM · #
Of course, one big problem is that the very best teachers don’t make much more than the very worst teachers, a factor that only increases the frustration that the very best teachers face. Average teachers’ pay isn’t horrible, and it isn’t grand, even with the “benefits” that Sailor mentions, and what’s the incentive for a great teacher to keep beating his or her head against the wall for the same pay as the teacher who is just on cruise control. With that said, given the Randian outlook of so many conservatives who would never pursue a job for the public good, I find bashing of public school teachers, many of whom do stay in education for pay that is not commensurate with their education and skills, ironic to say the least.
— IronyAbounds · Feb 8, 07:07 PM · #
The focus on teacher performance is, as always in these discussions, misplaced. The majority of teachers will always be average. Many will always be poor performers (however that is measured). Getting rid of 5% of teachers is not going to happen in the schools that actually need improvement the most. Such schools tend to move heaven and earth to make sure that there is an adult warm body (any adult warm body) standing in the front of every classroom.
What is needed is not so much a focus on teachers as some sort of input whose quality we can measure, as if we’re making coffee or something.
Insert high quality teacher. Turn crank. Out pops well educated student!
What we need is a focus on curriculum and teaching techniques that can be replicated by the most mediocre of teachers. There are literally millions of teachers in this country. Every one of them is not going to be a heroic inspiration inventing magical lesson plans that turn the poorest kids with the worst parents into millionaire Wall Street lawyers.
— Rob Mac · Feb 8, 07:23 PM · #
Standardized testing is one of the most useless and detrimental school reforms that exist. And the reason is fairly obvious. Teacher’s cease teaching what they thinking is important and end up teaching the test. And even if I take the position that this isn’t detrimental, it’s a fundamental altering of the teaching environment in an unintended way. That’s why point number 3 speaks so much to me. How do quantitative evaluations magically become neutral on the teacher side? This is all great in the abstract, but any evaluation that’s important necessarily ceases being a metric and becomes a model for behavior.
— Console · Feb 8, 07:25 PM · #
As a High School teacher, I can attest that the big problem with the “low hanging fruit” of firing the bottom 5% is that the administrators who determine exactly who is in that 5% (often through extensive formal observations) have some subjective reign. Without union grievances, administrators at schools that I have taught at in the past would, I’m certain, have found a way to fit anyone who they found to be a nuisance into that 5%, and being a nuisance could mean speaking up against illegal behavior on the part of the administrators. This kind of thing happens all the time in charters. I’ve seen well-written teacher evaluations grossly misused by administrators both intentionally and due to poor training.
It is also a fact that many teachers who fall into that 5% are overburdened, under-trained, lacking proper ELL and SPED support in classes that team with disabilities and English as a second-language speakers, and which take place in classrooms that are always conditioned by administrative choices. For example, I worked in a struggling urban school in which the decision was made to axe gym, history, art, music, and language classes in favor of teaching multiple ACT prep classes to every student every day. There were no extra-curriculars, no field-trips, and a confrontational security detail. You can guess how that affected student motivation, and yet teachers are usually viewed as the only people tasked with motivating students. I’m not saying that teachers shouldn’t be evaluated, or that poor teachers shouldn’t be fired, but I don’t feel like I ever hear similar discussions about administrators, many of whom are far less competent that teachers. While there are many brilliant and inspired administrators out there, there are also many inept former-teachers who cashed-in on their experience by taking a handsomely paying administrative career.The schools that they run are, it needs to be said, probably more important in the production of their teachers than the teacher colleges that those teachers graduate from.
— Dalas · Feb 8, 09:28 PM · #
IronyAbounds:
With that said, given the Randian outlook of so many conservatives who would never pursue a job for the public good, I find bashing of public school teachers, many of whom do stay in education for pay that is not commensurate with their education and skills, ironic to say the least.
That is typical of liberal thinking: that teachers (and all those wonderful folks who work for non-profits) are doing it for the public good. They’re not doing it just to get a paycheck (horrors), or because they have reached an income level commensurate with their abilities. Heaven forbid. They’re doing it because they are good people, sacrificing their
God-given natural abilities because they care so much. God, it always comes down to that: liberals just care more than the rest of us.
— jd · Feb 8, 09:46 PM · #
Most K-12 teachers, given their low average SAT scores, have the intellectual ability and competence to have been social workers, counselors, and other relatively low-paid positions. Whoever thinks that teachers are giving up lucrative careers elsewhere has the burden of proof.
— JD · Feb 9, 12:52 AM · #
Rob Mac wrote: What we need is a focus on curriculum and teaching techniques that can be replicated by the most mediocre of teachers.
This is a good point. This is the first time I’ve heard it from anyone other than the voices in my head.
— The Reticulator · Feb 9, 06:12 AM · #
THere was an article in the NYTimes a while back on a charter school official who was working on exactly this. He has developed a list of something like 25 fairly simple and effective teaching techniques that he got from observing good teachers in the classroom. Many of these techniques are not specific to any subject but deal with management and presentation of information.
Another area that could improve schools is an better understanding of how to teach the actual subject matter. For instance, until very recently almost no one attempted to systematically teach writing. It was either not taught—students were just expected to write—or it was taught as grammar. Most teachers still have very little understanding of what writing is, or the mental processes involved in writing, much less how to teach it. And some of the early attempts at systematically teaching writing are really strange and misguided but still widely accepted (ex. six-traits).
Brain research on how children actually learn is another area that could improve curriculum and teaching methods, as could a better understanding of basic thinking skills (analysis, comparison, etc…) and their use in daily life and how to profitably teach them.
A lot of what and how we teach is based on centuries old traditions. Improving, researching, and distributing better teaching techniques and curriculum is one area with lots of room for growth in this country.
— cw · Feb 9, 06:17 PM · #
1. Here’s an anecdote that reflects real data: in 1977, a first year Manhattan lawyer made about a thousand dollars more than a first year Manhattan teacher. By 2007, it was better than one hundred thousand dollars more.
— Freddie • Feb 8, 09:19 AM • #
Show me the data! Otherwise, I’ll have to assume that you’re positing a parallel fake anecdote and fake data. Besides, whoever heard of a quantitative anecdote?
— Nicholas Stix · Feb 12, 07:02 AM · #