A recent meeting left us feeling queasy. One of us (Jon) was preparing to partner with a large multinational company on a research study involving the company’s employees. Jon had partnered with the company in the past and was excited by the opportunity to turn their previous collaboration on passion for work into a large-scale intervention. As Jon sat in a planning meeting, the vice president of human resources mentioned that even though the intervention would be evaluated at the group level—meaning an employee’s personal information would only be analyzed as part of an aggregate of all employee data—their sensitive individual-level data, such as the intention to leave and level of burnoutwould be reviewed by each employee’s manager, as well as local human resource teams.
In other words, as part of an experiment examining how to increase their passion for work, employees were expected to provide sensitive information about their experience at the company. Evaluating whether the intervention was a success did not depend on knowing how individual employees responded to a particular question. Nevertheless, the company planned to use that sensitive information in management decisions about specific employees.
This is problematic for several reasons. First, if employees suspected that the information they provided could cost them their job, they’d be less likely to answer honestly, meaning all of the time, effort, and money put into the intervention would be wasted. Second, acting on individual employee responses opens the door for the unethical treatment of employees. Managers, when considering promotions, might pass over an employee who reported feeling burned out or divert resources away from employees who think they might leave, regardless of whether they actually do.
What started as a company’s chance to learn more about their employees and improve their operations quickly devolved into an activity with the potential to undermine employee honesty, erode trust between managers and their direct reports, waste company time and money, and induce skepticism about participating in research.
What started as a company’s chance to learn more about their employees quickly devolved into an activity with the potential to undermine trust, waste company time and money, and induce skepticism about participating in research.
To be clear, there was nothing illegal about how the company planned to use the data. Yet this myopic procedure would have failed even basic ethical conventions in university-based research.
Jon tried to push back against the company’s plan. But without a specific set of ethical guidelines in place, it was his professional opinion against that of the vice president. We believe that a clearly defined set of ethical guidelines could have helped Jon and the company work through this situation. And in speaking with other behavioral scientists, we’ve heard that Jon’s experience is not uncommon.
Simply put, a vice president’s motives are different than those of an academic researcher. Turning a profit and advancing scientific knowledge are very different undertakings, and these differences can lead to ethical standoffs. As more and more companies hire behavioral scientists, increasingly the person leading the research will be an employee of the company, blurring the lines between a company’s motives to generate revenue and a researcher’s obligation to his or her participants.
Yet it’s the differences between companies and researchers that make their partnerships so valuable. Companies have access to one-of-a-kind datasets, specific settings with which to test interventions, and access to large populations of people working toward the same goal. Researchers have the training to design and test ideas, analyze data, and draw actionable insights from the research.
To ensure these partnerships are beneficial to all involved—companies, employees, customers, and researchers—behavioral scientists need a set of ethical standards for conducting research in companies.
To address this need, we created The Behavioral Scientist’s Ethics Checklist. In the checklist, we outline six key principles and questions that behavioral scientists and companies should ask themselves before beginning their research. To illustrate how each principle operates in practice, we provide mini case studies highlighting the challenges other researchers and companies have faced.
The checklist is not yet perfect, nor is it exhaustive. But we feel it’s a good start, as it helps operationalize previous work on applying behavioral science ethically (see the Further Reading & Resources section at the end of the article). We invite academics and practitioners alike to help us add to and refine it. We’ll make the checklist a living document and update it accordingly.
We designed the checklist to play an active role in discussions between researchers and managers and we invite you to try it at your next study design meeting. Our hope is the checklist will lead to improvements in intervention design, decrease uneasiness and uncertainty, and, ultimately, ensure that the work we conduct lives up to the highest ethical standards.
#1 Aligned Interests: Are the interests of our company, researchers, and participants aligned?
In 2016, the life insurance company John Hancock announced an unusual move, one that was later lauded by an article in the Harvard Business Review and that earned John Hancock an award for being the most innovative insurer. John Hancock partnered with Vitality, a wellness program provider, and started offering incentives that rewarded healthy behaviors by their customers. In short, John Hancock planned to encourage a range of behaviors associated with a longer, healthier life.
The motive on John Hancock’s part was clear: the longer people who had a life insurance plan lived, the less likely John Hancock would have to pay out the insurance policy. Longer, healthier lives are therefore clearly in the company’s best interest. What makes this case special, however, is that for those people taking out life insurance, living a longer, healthier life is also in their best interest. In this case, the interests of the company running the study and the customers participating in it are aligned.
Contrast this approach with the way credit card companies bill their customers, particularly the way companies present the minimum repayment fee. Imagine your credit card bill came to $435 this month. A company might suggest that you repay two percent of the total ($8.70) as a minimum payment, or they could leave this suggestion out.
Make no mistake: The credit card company benefits and you lose if you don’t repay the whole amount. You’ll pay interest on any leftover sum.
Does it matter whether your credit card bill includes a minimum repayment fee? A 2009 study by Neil Stewart indicates it does. In his experiment, bills with low minimum repayments made it less likely that individuals repaid their full credit card bill, in comparison to those who received a letter that omitted minimum repayments. This occurs because of the anchoring effect, whereby arbitrary numbers, like the suggestion to pay $8.70, systematically skew people’s decisions, even if they know such anchors are arbitrary. In this case, the intentions of credit card companies and their customers are clearly misaligned. When the company benefits, the customer suffers.
Insights from behavioral science can help people live longer, and they can also induce people to pay more interest. People can be made better off or worse off.
For this reason, behavioral scientists must ask their collaborating companies what their intentions are and how their intentions align with those of the people they’re working with, often employees or customers. When intentions are misaligned, researchers and companies need to place greater care on how behavioral insights are used and for what ends.
Our hope is the checklist will lead to improvements in intervention design, decrease uneasiness and uncertainty, and, ultimately, ensure that the work we conduct lives up to the highest ethical standards.
#2 Transparent Processes: Is our research process transparent to the participants? How can we make the process more transparent?
Online dating has exploded over the past few years. Over 25 percent of American adults 18 to 24 years old have used an online dating site. In 2013, the percentage was just 10 percent. Few organizations collect as much intimate data on as many users as companies that facilitate online dating. These companies use the data to improve their products and ultimately grow their user base. This is how they stay in business.
But online dating companies vary widely in how transparent they make this process. Tinder, though it lists a sociologist on their staff, does not provide users with insights into how their data is being used for research. OkCupid, in contrast, openly talks about how it employs users’ data for research and regularly publishes summaries of the insights it has gained.
When companies conduct research on their customers, they must consider whether customers know that they are part of an experiment. If not, companies must explore the best ways to inform customers that their data is being used for research.
It’s worth noting that most users are unlikely to take advantage of the transparency offered. But the fact that the company offers this transparency signals that the company is conducting business ethically. Transparency also provides an important check on the power companies have over users’ personal data.
Moreover, covertly studying users can severely damage a company’s reputation. In 2017 Facebook conducted a study on teenage users, some as young as 14, to pinpoint “moments when young people need a confidence boost,” but the study was made public knowledge only after the fact. The public outcry following the leaked results was immense: A coalition of 25 consumer rights organizations wrote a public letter to Facebook, criticizing the opaque nature of their research. While the full ramifications of the lack of transparency are still to be seen, it is clear that consumer rights organizations and governments around the world will now keep an even closer eye on Facebook’s data operation.
#3 Rigorous Evaluation: Does our study design and analysis plan allow us to evaluate the effectiveness of the research?
When individuals participate in academic research studies, they give their time and effort in order to advance the state of science (and, sometimes, to make a little bit of money on the side). University researchers are expected to respect participants’ time and effort and design studies with enough rigor that the findings have the potential to benefit the field.
Companies should be held to a similar standard. It is a company’s obligation to design studies in ways that allow for rigorous evaluation, so that the time and effort invested by hundreds, thousands, or even millions of customers or employees has the potential to improve the company’s operations, to the benefit of both the company and its customers or employees.
How does this work in practice? Opower, an energy services provider based in Arlington, Virginia, provides a useful case study.
When it bills its customers, Opower provides people with information about how their energy use compares to that of their neighbors. A wealth of research shows that this type of social norm information can reduce individuals’ energy usage, saving money for the company and customer alike, as well as being friendly to the environment.
In its early days of providing this information to its customers, Opower encountered a problem their academic counterparts had been struggling with for a while. Giving people feedback on their energy consumption tended to reduce energy use for those individuals who used more than others, but increase energy use for those who used less than others—a phenomenon known in the academic literature as the boomerang effect.
In follow-up experiments, Opower, in collaboration with behavioral scientists, found that pairing the social norm information with emoticons, such as a smiley face when energy use was below average and a frowny face when energy use was above average, eliminated the boomerang effect. This insight was then rolled out across Opower’s billing operations, saving the customers and company money and reducing energy consumption.
Opower continues to develop and test interventions. In ongoing work that one of us (Jon; unfunded research partnership) is involved in, Opower is evaluating how customers’ perceptions of their neighbors influence how they respond to the social norm information.
Continuously developing and testing interventions, like Opower has done, is not just good practice: It’s critical for developing effective relationships with your customers and employees. Research by Suzanne Shu and colleagues (in press at Behavioral Public Policy) shows that people are more likely to endorse the use of behavioral insights if they perceive those insights to be effective. That is, the more time and effort companies invest to improve the substance and delivery of interventions, the more likely people are to support them—which in turn, is likely to make them even more effective in the future.
The takeaway? Treat customers and employees with respect. Design and invest in studies that allow for a rigorous evaluation. And if an intervention only benefits one of the stakeholders, it’s likely to cost the company money and their reputation in the long run.
Click here to download a PDF of the table above and here for the full checklist.
#4 Data Privacy and Security: Do we have a data management plan that protects the privacy and anonymity of our participants?
Companies often collect sensitive data on employees and their customers. Consider that for many companies who issue their employees cell phones and computers, it is legal to monitor these devices.
Recently, Microsoft’s Duncan Watts analyzed the email data of Microsoft’s employees. He and his colleagues found that emails sent outside of working hours negatively predicted work-life balance. They also found that employees liked their manager more if the manager responded quickly to emails.
While these insights are intriguing and useful to Microsoft and its employees, it is important to remember that emails also often contain personally sensitive information. Companies that plan on analyzing such information must think proactively about how they will treat this data. Ideally, companies would strip all personally identifiable information for the data set—often a requirement of university Institutional Review Boards (IRB).
In addition, it is important that organizations have adequate procedures to protect sensitive data from being exposed. We lauded OkCupid above, but we also need to note that researchers have exploited vulnerabilities in OkCupid’s web design to collect personally identifiable information. Organizations that conduct experiments on their employees and customers need to consider how they will anonymize existing data and protect it from leaking.
We want to emphasize just how difficult it is to fully anonymize data. It requires far more than getting rid of personally identifiable information, such as individuals’ names, addresses, and dates of birth. (It takes just four data points from an individual’s credit card statement to identify them with 90% accuracy). To preserve confidentiality, researchers could add random variation to indirect identifiers in the data. Detailed logs of all anonymization steps need to be kept.
Minimizing the risk of privacy violations is not a one-sized-fits-all approach: The required level of anonymization may vary depending on the context of the research. Organizations need to proactively and carefully think through their data collection policies to best protect the privacy of the population they are studying. In some cases, it may be wise for researchers and companies to work with specialists in data security.
#5 Ease of Opt Out: Can our participants easily opt out of the study?
Several years ago, a group of researchers from Facebook’s data science team conducted a controversial experiment involving over 689,000 Facebook users. In the experiment, the researchers manipulated the emotional content of the messages appearing in users’ news feeds. They found that emotions can be contagious on the social network. That is, even when people aren’t in the same physical location, people can transfer their emotional states to others.
The scale and nature of the experiment were ground-breaking. But this push for new scientific territory by a company a little more than a decade old and with millions of users highlights some of the new ethical concerns companies and behavioral scientists must take into consideration.
Critics of the study questioned whether or not it was ethical to manipulate the emotional content of a user’s news feed. The journal that published the results, Proceedings of the National Academy of Sciences, issued a rare “Editorial Expression of Concern,” voicing concerns about the lack of informed consent in the study.
At the time of publication in PNAS, the study fell in line with Facebook’s data policy and the legal framework in the United States known as the “Common Rule.” However, it came out later that the clauses regarding research were added to Facebook’s data policy approximately four months after the experiment took place (but before the study was published).
The 689,000 people whose Facebook news feeds were manipulated did not get any advance notice and there was no way for them to opt out from any research activities conducted on the site. This is extremely problematic: Users must be given the opportunity to opt out from research.
When behavioral scientists work with companies, it is important to take into account participants’ agency. People need to be able to clearly understand when their data is being used for “business-as-usual,” and when their data is being used for additional research experiments. An opt-out clause is particularly important when experiments manipulate participants. And opting out should be made easy—viable for busy participants who have many different things on their mind.
As collaborations between academic behavioral scientists and companies become more common, we need to be aware of how different goals may lead to ethical trade-offs.
#6 Cost–Benefit Analysis: Do the potential benefits of our study outweigh the potential harms?
Facebook’s manipulation of people’s news feeds raises another ethical consideration. Specifically, did Facebook need to manipulate 689,000 users’ news feeds to get an answer to their research question? A power analysis—a test determining the number of participants needed to detect an effect—revealed that they didn’t.
It is important to conduct research studies with sufficient power to detect the presence of hypothesized effects, even if they are small—as is the case with most complex behavioral phenomena. But it is equally important to not be wasteful with sample sizes, as Facebook was in this case.
This is important for two reasons. First, as mentioned above, we need to respect participants time and effort. Second, the negative consequences of an experiment are often unknown, meaning a larger than needed sample size could inflict more harm than was necessary.
In pharmaceutical research when side effects remain unclear, a new drug is not immediately tested on thousands of people. Why test unnecessarily in the behavioral sciences? Even though the harm in the behavioral sciences may not be as immediate and lethal, the consequences could nonetheless negatively impact people’s lives. The question then is what the harms of a possible study are, and whether these harms are justified in comparison to the potential benefits. Indeed, one key component of the review process (IRB), standard in colleges and universities, is evaluating the potential benefits of the research versus its potential harm. Businesses lack comparable oversight, which is one reason we feel this checklist is so important.
Finally, it’s important to contemplate how the benefits and costs are distributed. For example, if all of the benefits go to the organization running the study and all of the costs go to the population under study (such as employees or customers), then it may be questionable whether the benefits truly outweigh the costs. Companies may therefore want to consider for whom benefits offset costs, and how a positive tally may be achieved for each group.
Ultimately, we believe that The Behavioral Scientist’s Ethics Checklist will help behavioral scientists and companies work together in more effective and ethical ways. As collaborations between academic behavioral scientists and companies become more common, we need to be aware of how different goals may lead to ethical trade-offs. One way to do this is by developing a shared language and understanding of the purpose and parameters of a research partnership. We feel the checklist helps do just this.
Click here to download a PDF of The Behavioral Scientist’s Ethics Checklist and here to download a table of The Behavioral Scientist’s Ethics Goals.
