I wasn’t always convinced that behavioral insights could usefully be applied outside the context of a randomized controlled trial (RCT).
In fact, I decided to go to graduate school primarily because I was a public policy maker disillusioned with the lack of evidence being used in policymaking. I went back to school to gain skills in statistical methods and program evaluation, so that I could better use evidence in my government role in Australia. As someone with a background in science, I’m the first to acknowledge that I’m an unlikely advocate for applying behavioral insights without always experimenting through an RCT.
Heretical? Maybe, but hear me out. Increasingly I have been thinking about how we can apply behavioral insights outside of the scientific method of experimentation. Further, I’ve come to believe that if we don’t apply behavioral insights in contexts where it’s not possible to experiment, we’re actively undermining our ability to use behavioral science to improve policy and achieve better outcomes for citizens.
If we don’t apply behavioral insights in contexts where it’s not possible to experiment, we’re actively undermining our ability to use behavioral science to improve policy and achieve better outcomes for citizens.
Much of what government does involves large systems change: legislation, regulation, compliance. These system-level changes are not generally amenable to real world experimentation for a range of legal, ethical, and practical reasons. We cannot have one version of the world where some people are bound by a certain law or regulation and others are not.
So while I think increased experimentation—creating more evidence in advance of implementation—is a great benefit of the way behavioral insights have been applied in government to date, I also think that we’re constraining our impact if we restrict applying behavioral science only to scenarios where it’s possible to run an RCT.
Behavioral insights to date have largely been used in the later stages of a traditional policy process, at implementation and evaluation to help optimize services or programs. However, at this point many decisions about the policy architecture have already occurred. At the policy analysis stage, policymakers have defined the parameters of the problem and outlined the potential solutions. Policymakers have also likely decided which policy instruments to use, such as incentives, information, or regulation.
If welfare is our guide, we are obligated to apply behavioral insights at the early stages of policy design. To not do so simply because we cannot test beforehand would be to withhold the benefits of current research from citizens.
For example, in 2012 the Australian Government introduced new laws that mandated plain packaging of all cigarette packages. This regulation is prescriptive, requiring cigarette packets to be a drab olive color, with brand names provided in specific font, size and location on the pack, and graphic health warnings to cover at least 75 percent of the front of the packet. The Australian Government implemented this policy with the intention of reducing smoking rates in Australia.
Changing the packaging does not, of course, change the cigarettes themselves. The packaging is a “supposedly irrelevant factor,” to use Richard Thaler’s term; however, we know such factors influence our decision-making.
Two behavioral factors were particularly relevant to this policy decision: salience and attraction. The requirement to increase the size of the health warning and combine a striking image with text made the health risks of smoking more salient to smokers. Regulated “plain packaging” also removed the ability for cigarette companies to make cigarette packages more attractive by using logos and graphic design to attract attention and make the product seem more appealing.
These insights from the behavioral sciences were used to inform the policy design. However, the effect of the redesigned packaging on the desired behavior change—reduction in smoking—was not empirically tested using an RCT before its implementation across the country. How could it be? It would not be legal or practical to legislate that some firms must use plain packaging and some could continue to use existing packaging.
What policymakers considered beforehand was the body of research knowledge, including experimental evidence, that demonstrated the potential benefits of this type of packaging. There was no guarantee the policy would work but understanding and applying the existing evidence provided the Australian government with the best starting point for designing a policy that could not be fully tested before its introduction.
If welfare is our guide, we are obligated to apply behavioral insights at the early stages of policy design.
The evaluation of the effect of the redesigned packaging on smoking rates in Australia indicates that this policy achieved its desired effect. The post-implementation review indicates that smoking prevalence among adults in Australia dropped after the introduction in 2012 of plain packaging by over 0.55 percentage points, equivalent to approximately 18,000 fewer Australian smokers. This is not an RCT, and therefore we cannot conclusively determine that plain packaging caused the reduction in smoking. It is, however, good evidence that the policy produced this effect. Similarly, follow-up research indicates that the policy was effective in increasing smokers’ awareness of toxic chemicals found in cigarettes and the associated health risks.
Our work in the family violence system in Victoria, Australia, provides another example of applying behavioral science early in the policy process.
Information sharing in the family violence system is essential to keep victim survivors safe and hold perpetrators accountable. This includes how family-violence workers share information between organizations, across sectors, and over time. For example, family-violence service organizations should be able to access relevant information held by the police to better assess risks and therefore support a victim survivor of family violence.
In 2016, Victoria’s Royal Commission into Family Violence found that information is not routinely or systematically shared in the family-violence system, exposing victim survivors—the term used in Australia to describe those who have experienced family violence—to risk of further harm. To better understand what contextual and psychological factors affect information sharing, we undertook extensive ethnographic research with frontline workers in the family violence system. Our report, Applying Behavioral Insights: Improving Information Sharing in the Family Violence System, identified a range of factors that influence the accessibility and sharing of information, including physical and systems design, varied understanding by workers of the information other services need, and significant time scarcity.
Findings from the report are being used to inform policy, process, and cultural change around information sharing. For instance, this research has been used to simplify the Ministerial Guidelines on how services should share information to improve efficiency and reduce errors in workers’ decisions.
It’s too early to know whether this application of behavioral insights to policymaking has made an impact on how information is shared across the family-violence system and how this affects long term outcomes for victim survivors of family violence in Victoria. The Victorian family-violence information-sharing scheme will be reviewed two and five years after its implementation. Given the many changes to the system overall, we will not be able to neatly disaggregate any positive or negative outcomes from the behavioral insights applied to the Ministerial Guidelines with a nice bar graph with error bars. However, to not consider human behavior in complex and, in the case of family violence, potentially life-saving policy areas would be to restrict ourselves from considering evidence that we know is relevant to the problem.
“Wicked” problems like family violence, obesity, and climate change—problems that are complex, multicausal, and resistant to change—are some of the most pressing challenges facing governments today. These challenges are rarely amenable to solutions that focus only on implementation of one program in one part of the system. Yet given solutions to these problems all require behavior change, perhaps it is time to lay down our bar charts in favor of a more nuanced approach to the application of behavioral insights. Rigorous testing and evaluation to understand “what works” is fundamental to good policymaking. However, where experimentation in advance is not possible, we must still consider what we know about human behavior to design better policies from the start.
