Foxes and Henhouses
Ezra Klein has an interesting post up in which he correctly points out the conflict of interest inherent in having pharmaceutical companies that develop new drugs execute the clinical trials that test for their safety and efficacy.
While literal corruption of the results makes for good movies, and will always be present to some extent as long as money is at stake and humans remain human, it appears to be vanishingly rare. What is clearly one huge problem is publication bias in which positive results are much more likely to be reported than negative results. A second problem that he doesn’t really get into in the post is that subtle decisions about study design – in effect, deciding exactly what question is asked by the research – can be manipulated in a fashion that is in the best interest of the pharmaceutical company, rather than what a truly objective researcher would probably do.
Klein recommends a paper that presents a simple solution: the government should fund contract research directly.
But the obvious point that this misses is that a government bureaucracy has its own conflicts of interest. Most directly, bureaucrats and politicians tend to have enormous career risk from an unsafe drug introduction, but almost none from a rejected drug that would have been effective had it been introduced. Publication and study design bias can be pointed in both directions.
All else equal, Klein’s solution would likely result in fewer new drugs being brought to market. The average new drug would probably be safer, and we would likely have fewer “me-too” drugs. We would also likely not get some new, safe and effective drugs that we would get under the current system. Would this be an improvement? It’s not so obvious. Think of the recent debate over HIV/AIDS treatments. It’s a question that requires fairly careful research.
In fact, research and analysis of exactly this problem is what led to many reforms of our drug approval system over the past couple of decades. The classic work here is Sam Pelzman’s Regulation of Pharmaceutical Innovation from 1974. It kicked off a wave of drug regulation research throughout the late 1970s. A more contemporary popular treatment of the general topic of the trade-off between regulation and innovation is Richard Epstein’s Overdose from 2006.
Any delegated task creates agency problems. Pointing out only those on one side of the see-saw is missing at least half the picture.
Thanks for this.
— Klug · Jan 2, 07:49 PM · #
What if companies were forced into a peer review system of some sorts. If companies have the personal in place to run these clinical trials already, why not re-rig the system so that these same groups are doing blind reviews of other companies drugs. The incentive towards approving their own drugs are thus removed, and at no additional cost to a government oversight board (just change of regulation procedures).
This does raise two problems… one, companies are incentivized now to not approve the drugs of other companies (they are of their rivals after all), and two, you still don’t get around the problems of ‘fewer new drugs brought to market.’ I think these could be addressed in two fashion. First, if drugs are randomly assigned to different companies, the differences in rejection rate by companies testing groups should be nill. Therefore, it should be possible to possible to look into investigating companies that have unusually high or low rejection rates. Also, if blind reviewed, it would be more difficult to target rejection of a specific rivals drug.
With regards to fewer new drugs being brought to market, could it be possible to have different levels of approval? That way, you potentially bring similar amounts of drugs to the market, but perhaps at different levels of specified ‘safety’
A major problem that would arise out of this set-up concerns information proprietary rights. Rival companies would be testing the ‘new’ products of their competitors. This could be a major issue. I guess the only way around this issue is to realize that once drugs are at this testing state, it is almost too late to copy, produce and test their drugs.
Anyway, probably wouldn’t work, but worth thinking about alternative solutions that get around the incentive misalignment.
— Peter Boumgarden · Jan 2, 08:56 PM · #
I don’t understand why Klein likes this. I spent a lot of time doing private-sector biomedical research under government contracts, and am looking a little at the other side of the coin now. And it’s damn hard to regulate, not least because frankly the government guys lack the technical competence to judge what you’re doing. At first it seems life gets easier for this kind of thing because you’re funding something a lot simpler than exploratory research — except somewhere along the line somebody has to decide which of the tons of things coming down the pipe, are worth doing clinical trials on, and that takes a ton of current technical knowledge. Having the companies pay for it to some extent aggregates that — what am I able to convince investors is worth gambling on? It doesn’t look like Baker has figured out a way to ensure that you can do that some other way. In the end I think that problem is going to beat out even the [Ionaddis-like] bias problems that concern you for intractability.
— Snajay · Jan 2, 09:23 PM · #
But the comparison isn’t the bureaucrat to a drug executive, but the bureaucrat to generalized risk tolerance of the population as a whole; would someone in the government have a higher risk aversion for his career than the population does for their lives? For people who are terminal, probably. For people who want day-to-day quality of life drugs, not so sure.
I’m pretty pro-drug companies on these issues, but every other week there’s another thing that throws me into Klein’s camp. Wyeth’s ghostwriters may have pushed me over the edge – it seems less about risk-versus-profits tolerances and more about the drug companies having actually captured the approval and recommendation process, akin to a CEO capturing his compensation package.
— rortybomb · Jan 3, 12:03 AM · #
I think the US government running clinical trials is a very, very bad idea. What I don’t think is a bad idea – and would solve some of Kleins issues – is granting the FDA the right to release succesful and failed trial data to the public through it’s own publication.
Companies want to have control over the data from their trials, and up to a point, they are entitled to that control. It is their money and their product. However, the FDA is a public institution that we trust is providing proper oversight in the approval of new drugs and devices. Allowing the FDA to release their own analysis of the data does not infringe on the companies entitlements, as longs as the company gets first publication rights. If the FDA had its own journal, for example, the problem of secondary publication would also be removed. I would guess that this system would deter some companies from pursuing risky drug ventures, or the production of me-too drugs. As for the unintended consequences, who knows.
One thing Klein fails to mention is that the journals seeking new medical research generally don’t want failed trial data. They aren’t going to make money off of those articles (and no doctor is going to make much giving talks about some drug that failed a clinical trial). A non-profit journal focusing on the release of all trial data to the medical community and the public might help solve this problem. At least it would give Klein more stuff to write about.
— mattc · Jan 5, 01:51 PM · #
mattc, while nobody can argue against more transparency in government evaluation and regulation, I don’t see as your journal idea helps much. Manzi’s point exactly addresses it in fact — the issue is how bias shapes what research is done and how it’s done, not whether that research is made available. I agree in part that pushing more availability might in theory reveal the biases Manzi’s worried about, except (1) it doesn’t do so now even with published studies, (2) the competence to make that kind of statistical judgement, while not excessive, is less common than you might think, and probably close to nonexistent among the target population of prescribers for your proposed journal, and (3) I think you can get bogged down in holy wars over how to design studies past the point of their being much use.
— Sanjay · Jan 5, 03:13 PM · #