How Can We Know What Public Sector Wages Should Be?
I wrote a post questioning the reliability of a study which claimed to show that Wisconsin public workers face a compensation penalty versus what they would make in the private sector. The motivation behind the study that I criticized is that comparing average compensation for public versus private workers is too imprecise to draw conclusions about whether public sector workers are overpaid, underpaid or paid about right. I agree with this. The method of the study was to control for this selection bias by observing how much more or less people tend to make when they have college degrees or not, are black or white, and so forth, and then to combine these adjustments together into a regression model that should hold these factors equal when comparing compensation for public versus private sector workers. It is this method that I criticized.
One technical problem with this approach is that it’s very hard to correctly adjust for these factors. Another, more obvious, one is that if it doesn’t include some factors that are important, it will be unreliable. This second critique was the one I made in my post. This kicked off a fair amount of back-and-forth and other commentary.
Two conservative scholars (Andrew Biggs of AEI, and Jason Richwine of Heritage) have now claimed that while the study in question was flawed, a better modeling method can be employed to show the opposite conclusion – that public sector workers are overpaid versus what they would earn in the private sector.
Here’s Richwine:
Manzi is referring to “the human capital model,” which holds that workers are paid according to their skills and personal characteristics, like education and experience. Most scholars—including Andrew, myself, and Heritage’s James Sherk—use it to compare the wages of the public and private sectors. If the public sector still earns more than the private after controlling for a variety of factors, then it is said to be “overpaid” in wages. But because we cannot control for everything, Manzi is saying, the technique is not very useful.
His critique is reasonable enough, but overwrought. The human capital model has been around for three decades, and it is unlikely that economists have failed to uncover important variables that would drastically change its results.
So, the reason I’m wrong is that labor economists have been building human capital models this way for thirty years, and it’s unlikely that any important factors would not be in the models at this point. Got it.
Here are the sentences that immediately follow in Richwine’s post:
Nevertheless, there are other techniques that address most of Manzi’s concerns. An upcoming Heritage Foundation report uses a “fixed effects” approach, which follows the same people over time as they switch between the private and federal sectors. By looking at how the same person’s wage changes when he moves between sectors, a lot of unobservable traits—intelligence, extroversion, etc.—are accounted for.
So, in fact, there are obviously important factors – like, say, how smart you are – that aren’t accounted for by the human capital model that I critiqued. They’re also apparently missing from other standard human capital models, because it requires a different “fixed effects” model to address them.
Biggs, in his post, describes more specifically what their fixed effects analysis does:
To address this, we did a second form of analysis that isn’t subject to Manzi’s objections. Rather than comparing different people at one point in time, this second approach – called a “fixed effects model” – follows the same people over time, specifically as private sector workers found new jobs which could be in the public or private sector. If workers get a bigger raise when they switch from private to federal employment than workers who switch from one private job to another, we can infer that the federal government overpays.
This sounds (and is) reasonable, but do you see the problems? First, the people who leave their jobs, voluntarily or involuntarily, are going to tend to be quite different from the people who stay in their jobs. So generalizing from this group of leavers to the broader population of workers will be hazardous, to put it mildly. Second, even within this group of leavers, while there are some characteristics about people that are fixed, lots of things aren’t. Maybe the nature of the average job in the private sector tends to get them to work harder (or easier) after the switch than the nature of the average job in the other sector. Maybe people tend to switch to government employment at a point in life when they wanted to start to take it easy (or work harder) anyway. Maybe switching to private employment disproportionately involves a move to higher-cost urban (or lower-cost rural) area. And so on, just as in the human capital model.
But this is just quibbling, right? Not really. Biggs and Richwine reference the classic papers on this method by Alan Krueger. If you go back to this work, how does Krueger deal with these potential selection bias problems? He builds a regression model with “occupation, human capital and demographic controls.”
Ah.
We’re right back where we started, in the sense that we have the same kinds of questions about, for example, unobserved sources of selection bias between the two groups not accounted for in the model.
As it turns out, I’ve built many fixed effects, regression and similar models for the purpose of estimating the effects of various changes in compensation, pricing and other similar behavior drivers. And in contexts in which accuracy can be validated, fixed effects models are almost always better than the kinds of static models that I originally criticized. The reason is, as Biggs says, that some of the unobserved differences between groups are accounted for by them. So, I agree that that for the purpose of estimating a public-private compensation gap, that it is a reasonable belief that fixed effects models are better than static human capital models.
And a motorcycle is better than a bicycle for jumping the Grand Canyon, but I don’t advise you to attempt it with either one. The important accuracy issue here is not relative accuracy, but absolute accuracy. And the methods of assessing statistical significance and the like that are embedded within any of these approaches do not address the critique I am raising, because the critique is that the assumptions of the analysis itself are plausibly very wrong.
Have I then set up a nihilistic position that we can never know anything tolerably well because I can just keep raising these points that might matter, but are not included in the model? In effect, have I put any analyst in the impossible position of proving a negative? Not really. Here’s how you measure the accuracy of a model like this without accepting its internal assumptions: use it to make predictions for future real world experiments, and then see if its predictions are right or not. The formal name for this is falsification testing. This is what’s lacking in all of the referenced arguments in support of these models.
Human capital models, fixed effects models, and other various pattern-finding analyses are useful to help build theories, but a metaphysical debate about the “worth” of various public versus private sector jobs based upon them is fundamentally unproductive. For one thing, it won’t ever end. And as Megan McArdle correctly put it, the practical question in front of us is whether we the taxpayers can procure the public work that we want at a lower cost (or more generally, though less euphoniously, whether we are at the practical optimum on the cost-quality trade-off). If you want an analytical answer to this question, here is what I would do: randomly select some jurisdictions, job classifications or other subsets of public workers, cut their compensation, and then see if we can observe a material reduction in net value of output in these areas versus the control areas. If not, cut deeper. And keep cutting deeper, until we find our indifference point.
There would be obvious limitations to this approach. First, generalizing the results of initial experiments is not straightforward. Second evaluating output is not straightforward for many areas of government. But at a minimum, and unlike the world of endlessly dueling regressions, this would at least let us see the real-world effects of various public compensation levels first-hand, and allow the public to make an informed decision about whether they prefer the net effect of a change to public sector compensation or not.
(Cross-posted to The Corner)
What about pay differentials between states? Sure, they’re not controlled for things like cost of living and demographics, but it’s something. There has been a chart floating around these last few days that shows the (five?) states that do not allow public school teachers to bargain collectively have among the nation’s lowest performing schools.
There are other factors about those states that make them different from higher performers. At the very least, it doesn’t lend credence to the idea that we’re overpaying teachers.
Manzi will probably introduce enough doubt to allow him to ignore its implications, but it’s a start.
— rj · Feb 25, 05:56 PM · #
People aren’t lab rats, so within a jurisdiction that kind of experiment could only work if you had some way of concealing the fact of the experiment from your workers and even concealing the fact that some workers were being paid less than the control group. In other words, it wouldn’t work.
— MrMandias · Feb 25, 07:53 PM · #
If you want an analytical answer to this question, here is what I would do: randomly select some jurisdictions, job classifications or other subsets of public workers, cut their compensation, and then see if we can observe a material reduction in net value of output in these areas versus the control areas. If not, cut deeper. And keep cutting deeper, until we find our indifference point.
And then what happens to them? The workers, who are having their wages cut? Oh, right— you don’t say. This is the central pathology of modern conservatism: advocating worse conditions for others while refusing to consider what the actual human consequences will be.
— Freddie · Feb 25, 09:46 PM · #
Thanks for the comments.
rj,
The problems with this are the same as with comparing across individual people (lots of things are different between the different states), and worse, you only have 50 of them.
MrMandias,
Lots and lots of randomized experiments have been done by the government to test the impact of various incentives (and by companies as well).
Freddie,
I don’t think that we should conflate what we need to pay to get a service with other potential programs to give people more money if we think this level of pay is inadequate.
Best,
Jim
— Jim Manzi · Feb 25, 10:01 PM · #
Freddie wrote: And then what happens to them? The workers, who are having their wages cut? Oh, right— you don’t say. This is the central pathology of modern conservatism: advocating worse conditions for others while refusing to consider what the actual human consequences will be.
What an amazing coincidence. That’s precisely the problem with the left. They keep demanding more for their own socio-economic class (aka the ruling class) without considering what the human consequences are for the population that has to foot the bill and live with the destruction of society. They’re sort of like the late 19th century industrialists who were proud of themselves for the way they built factories and mills along the Kalamazoo River, and who pointed with pride to the human progress it all exemplified, without considering the human cost of turning the river into a cesspool and of turning workers into drones. But in both cases, it seems impossible to get them to look at the bigger picture, even when all that’s left is a wasted landscape and human destruction.
— The Reticulator · Feb 25, 11:53 PM · #
<i>Lots and lots of randomized experiments have been done by the government to test the impact of various incentives (and by companies as well).</i>
I don’t doubt it.
Whether the experiments worked is a different question.
I’m saying that if the lab rats were aware of the experiment, the results aren’t reliable.
— MrMandias · Feb 26, 01:35 AM · #
I love this post with my heart and it get me many good things.
— Android Tablet PC · Feb 26, 03:00 AM · #
Seems like for some government jobs, pay is low but then you cash in big when you move to the private sector and use your government contacts to finagle things for your company. It seems like from my perspective as an ordinary citizen, I’m better off in that case if the government employee gets paid enough to stick with his or her job for life rather than leaving in his or her prime to go into the influence peddling business.
— Steve Sailer · Feb 26, 08:50 AM · #
Further to what Steve Sailer said, it seems people in government jobs where you can later cash in by moving to the private sector are paid less than people in the private sector with comparable credentials (like when I was a government lawyer I got paid much less than most other Yale/Berkeley grads), whereas people in jobs where you can’t later cash in are paid more. I would guess that “ability to cash in later” is one of the many factors not considered in the EPI study originally critiqued by Mr. Manzi.
I think it highly unlikely that the political process could accept middling successful lawyers like myself being paid a lot more than Supreme Court justices and senators, so Mr. Sailer’s proposed solution is probably not viable.
— y81 · Feb 26, 07:13 PM · #
I do like the way you have presented this problem plus it really does present me a lot of fodder for thought. Nonetheless, from just what I have witnessed, I just hope as the actual feedback pack on that folks stay on point and not start on a tirade involving the news du jour. Still, thank you for this exceptional point and while I can not necessarily go along with the idea in totality, I regard your point of view.
— Hot News · Feb 26, 09:49 PM · #
A few years back I compalined about the two step commenting process here at TAS and was informed that it prevented comment spam. Strangely enough, the only place I see comment spam is TAS.
— cw · Feb 28, 02:35 AM · #
Why do this with just Gov’t wages. Cut Megan McArdle wages until you see a drop in work product quality – then we will try it in other sectors of the economy.
Note: I wish the Atlantic would put Conor on as a full-time blogger.
— danwesjac · Feb 28, 02:54 PM · #
If memory serves, McArdle was hired away from another magazine, so we probably have a pretty good handle (and she has an even more precise handle) on the market value of her work.
— y81 · Feb 28, 04:18 PM · #
Another case of social science struggling to prove what everyone else already well knows by experience or common sense, but some have an interest in denying or obfuscating.
Sure there are some underpaid, overworked positions in the government – but they are few and in pretty high up mostly. The vast majority seek it in the first place for the benefits and cushiness and stability, and those who get it are always shocked by just how criminally easy it is for the pay. They all know it and any of us who have gotten in on the take and seen it for ourselves from the inside know: it is not the same world – it is not really work. That mostly goes for those who live off the govt contract and guaranteed payouts – this is like striking a reservoir of winning lottery tickets. Securing these sweethearts is the real competition that determines class status for the rest of the population.
That is not to say you can compare the two types of employment or ever make them equitable. ‘Full time” and “part-time” dont’ even mean the same thing in gov’t “work.” But I don’t think anyone who is shouting about the issue on one side or the other is basing their position on a perception of compensation equality or inequality.
For many, the fact that the government makes up a sizable interest group, with its own lobbyists, while its workers continue to call themselves public servants and demand the honor of that title, is the true outrage – that equality of servants and citizen laborers and taxpayers is even being broached as a legitimate concern or issue between those honored with serving and those they serve. That’s what’s real, not methods for comparing apples to oranges.
— kemmawn · Feb 28, 08:36 PM · #
How do the more successful large corporations and similar large organizations set pay and benefits for their workers?
— J Mann · Mar 1, 03:59 PM · #
Can’t we take a simpler approach to the question? If government workers are underpaid, shouldn’t there be a bunch of government job openings that no one is stepping forward to fill? And if they are overpaid, won’t there be a surplus of workers pursuing government jobs?
Around where I live, the latter is a more accurate picture of the situation.
— andrew · Mar 1, 10:01 PM · #