Factoring in Behavior
New medicines may seem to pop up overnight on pharmacy shelves, but the drugs that make it to market have actually gone through a long period of testing. Today, one of the most important steps in this process—the gold standard for testing the efficacy of a treatment—is the randomized controlled trial, or RCT. By randomly assigning eligible patients to either an experimental group that receives the drug or a control group that doesn’t, researchers try to factor out some of the variables that differ among patients— and glean more accurate information about the actual effects of the drug. But the effects of human behavior can still seep into the results of such trials, says Caltech economist Erik Snowberg.
Snowberg, who first came to Caltech in 2008, is interested in understanding how economic theory can be used to understand human behavior outside the realms traditionally considered by economics. When he arrived at Caltech his research focused on using economic models to predict political behavior, but more recently—along with his collaborator, Sylvain Chassang of Princeton—he has focused on the challenge of using the economic view of behavior to improve the randomized controlled trials commonly used in medicine and public health.
“Over the years, people have come up with different methods to remove bias in clinical trials—biases caused by people’s behavior in the trial,” he says. Randomization was one such method, first implemented to eliminate differences between patients who receive the experimental treatment and those in the control group—that is, those who do not receive treatment. However, as RCTs have evolved, they’ve developed issues of their own, Snowberg says. For example, some patients who end up in the control group may really want the treatment and may subvert the experimental protocol to get it. “We thought that, rather than trying to eliminate the effects of behavior, it would be better to understand these effects—so that we can harness them to develop better therapies,” he says.
To do that, Snowberg and collaborators have developed experimental designs that identify motivated patients, while also randomizing their treatment status. They reason that if the test group of patients is motivated and interested in participating, they will more likely follow the instructions. This will ultimately give doctors a better idea of exactly how well a drug works when taken as recommended, and how well it works when a patient’s behavior does not conform to the experimenter’s desires.
“A randomized controlled trial is like a lottery; in a trial, you might have a 33 percent chance of getting treatment,” Snowberg explains. “But what if there is a patient who really believes in the trial and is really motivated to follow through with the treatment regimen? What if they can get what is essentially an extra lottery ticket?”
In one example of their innovative trial designs, every such “ticket” is drawn at a rate of one out of three. However, patients are given the opportunity to earn another ticket and thus identify themselves as motivated to complete the treatment—even through negative side effects. They might earn this ticket through spending their time in a boring and tedious task, or through a cash payment, but as long as the patients are selected to receive treatments through a lottery, the trial is still randomized.
Snowberg believes this type of design could yield more precise information than more traditional RCTs, regarding the efficacy of a properly used treatment. For example, if the treatment only works for people who earned an extra ticket, it indicates that they believe in the treatment, and this caused them to behave in a way conducive to the therapy, he says.
Although the “earning” aspect of this selection process may seem controversial for trials involving a last- hope treatment for a terminal illness, Snowberg says that it’s important to realize the design is not one-size-fits-all—and RCTs are used for all sorts of experimental trials, many outside of medicine. For example, the researchers are also hoping to learn more about human behavior in experimental trials involving the adoption of a new technology—an improved water pump for agriculture in Africa.
“Right after college, a friend of mine joined the Peace Corps, and her job was to help prevent dysentery outbreaks by convincing people to drink water from a safe, clean water pump rather than from an often- contaminated open well. But she was frustrated because no one wanted to drink from the pump,” Snowberg says. Interested in the situation, Snowberg visited his friend in Mali. “I thought it was weird that an organization just came and installed the pump without making sure anyone in the community wanted or would use it,” he says.
By distributing water pumps to anyone, whether or not they believe the pumps will be beneficial, “you’re giving people the opportunity to reinforce their prior beliefs about what is or isn’t going to be effective,” he says. “If a community is given a pump, but most of the people don’t believe that using the pump will keep them from getting sick, a majority won’t use it—and they’ll continue to get sick. Before, they believed the pump would be ineffective, and now they have evidence, ‘See, everybody is just as sick now as they were before,’” he says.
To see if a new type of trial design could help organizations better allocate resources to where they will do the most good, Snowberg and his collaborators are now testing several of their trial designs using agricultural water pumps in Kenya. In this ongoing study, at least one person in every village in the study area will get a pump, but how that person is chosen will change from village to village. In some villages, the pump will be distributed in the traditional way—randomly. But in others, villagers will have the option to earn another ticket by weeding a field in a neighboring village, or even to receive extra tickets from their neighbors— a form of voting the researchers hope will allow the village to identify who they think would be the best person to experiment with this new technology.
The researchers will then follow up with each village to see if the residents of those villages where a pump was given to someone who earned an extra ticket have more favorable opinions of the pump. The researchers hope that the results from this study will enable more effective distribution models for aid organizations. Snowberg says that almost anything evaluated using a randomized controlled trial is probably also affected by behavior. “There may be differences in behavior based on culture, but economics, and our research, is focused on identifying the behavior that is common to all of us,” he says.
Erik Snowberg is professor of economics and political science at Caltech. His work on the design of clinical trials is supported by the National Science Foundation and Caltech.