Making a Better Match

As the father of two school-age girls, Adam Wierman knew from experience that the Pasadena Unified School District's open-enrollment process was not optimal. "I couldn't help but notice that it wasn't particularly well designed," says Wierman, a Caltech professor of computing and mathematical sciences. "There was a huge opportunity, I thought, to improve."

PUSD's open-enrollment system allows parents to participate in an annual lottery for the chance to get their children into schools outside their neighborhood. The problem, Wierman says, is that the process was both inefficient and prone to gaming by anxious parents. Eager to come up with a better algorithm to support the district and its parents, Wierman collaborated with Caltech social scientists Federico Echenique, the Allen and Lenabelle Davis Professor of Economics; and Laura Doval, assistant professor of economics. Both had experience working on similar problems in the area of school-choice assignment.

This kind of research partnership at the intersection of economics, computer science, and mathematics is supported and encouraged at Caltech, says Wierman, through the Center for Social Information Sciences, which is funded by The Ronald and Maxine Linde Institute of Economic and Management Sciences.

With the team's new and improved algorithm, which rolled out in early 2019, families not only were more likely to get their top match but also were more likely to keep their children in the school district rather than enrolling them in private or charter schools.

"It's not immediately obvious why economists would work on this type of thing," says Echenique. "But if you think about an economy, it's all a resource allocation problem: who gets, who consumes, which goods. This is usually studied through markets, but there are many resource allocations that are not done through markets. Public schools are a good example."

PUSD's previous open-enrollment system involved two rounds of applications and offers, in between which families found out whether they had been admitted to private schools as well. PUSD is not the only school district that runs multi-round algorithms; Boston and New York City do so as well.

The existence of so many nonpublic school options in Pasadena provided an additional wrinkle that made the problem interesting to the Caltech researchers. "Everything we know about how well these algorithms perform is based on the idea that you have access to the whole market," says Doval. "So, if all the schools were participating in this, then theory would say that things would perform well. But here, public schools are just a subset of the whole school system, and there are people thinking, ‘Well, if I don't get the allocation I want, I can have my kid go to a charter school or to a private school.'"

Having this outside option has a real impact on public school enrollment, says Wierman. "In most public school districts, if families participate in open enrollment, they're going to register somewhere in the system," he notes. In Pasadena, however, more than half of the families participating in open enrollment were taking an outside option. This created inefficiency, Wierman says, because slots were being allocated to families that did not end up registering.

In addition, some parents would attempt to manipulate the system. Instead of listing schools in order of their actual preference, they would estimate which schools had a larger number of open slots and then rank those first. This, says Wierman, led to a lack of fairness. "You ended up with disparities between parents who could invest the time to understand the system and figure out the optimal thing to do versus parents who didn't have that time or the knowledge to make those strategic decisions." Or, as Echenique puts it: "It shouldn't take a PhD in computer science to figure out how to participate in the public school system. It should be simple and straightforward and transparent."

To tackle the problem, the Caltech group—in collaboration with four undergraduate students—studied three years of PUSD data on families' participation and preferences, how families were assigned to schools, and whether they ended up registering in the district. And then they came up with an algorithm they could test on the previous year's data. After six months of research, in the fall of 2018, Wierman presented their recommendations to the school board and was able to get approval for a multiyear phase-in beginning soon thereafter.

"A lot of the changes we were making were small things," says Wierman. "It didn't require a fundamental shift in the way things were happening."

One of those "small things" was upping the number of schools that families could list from five to unlimited. "When there's a limitation to the number of schools you can list, you tend not to list schools that have very few slots available," explains Wierman. "Once you go to unlimited, there's no reason not to just list every school in the order that you prefer them."

Another significant change is aimed at dealing with families that might leave the district. Historically, families participated in one round in which they were asked to rank the schools and were given an initial assignment that they could take or leave. If they were unhappy with that assignment, they could enter a second round in which they could resubmit their preferences. The catch was that they had to give up their original assignment.

In the new system, families are asked to rank the schools just once; if they are not fully satisfied with their initial assignment, they can move on to the second round but are guaranteed that they will not lose their original seat, until and unless they are given a seat in a school they ranked higher. This helped encourage parents to stay in the system rather than leaving for an outside option after the first round.

Finally, the new algorithm allows families to sign up for the waiting list for every school they are interested in; the previous method allowed for families to be put on a waitlist for their top choice only and only if they did not receive a match.

Data from PUSD shows that these changes to the algorithm had a significant impact, even in its first year. The number of families matched to their top choice went up by 10 percent overall, with even greater improvement at particular grade levels: 97 percent of high school families and 80 percent of middle school families were matched to their first choice. The number of families that continued to the second round increased by 150 percent. "Everything that we predicted ended up showing through in the data," says Wierman. PUSD is continuing with the changes for its upcoming open enrollment season.

This new program could be helpful to a number of other school districts, says Doval, since the issues it addresses are "by no means exclusive to Pasadena."

Seed funding for this project came from the Carver Mead New Adventures Fund, which supports projects that are at an early stage of development and therefore pre-competitive for industry or government backing.