Behavioral science is emerging as an exciting discipline in the private sector, but for many employers its novelty brings ambiguity. The most innovative companies know that it can be great for business, but they don’t know how to find the right people to realize that potential. What, exactly, makes someone a qualified behavioral researcher? What technical skills should an in-house behavioral scientist have?
To date, there is no guide to being a behavioral scientist in the business world; across the practitioners in the private sector, there’s a diversity of backgrounds. Nevertheless, patterns emerge, both in people’s backgrounds and in what we see is needed for day-to-day success. At my company, Morningstar, we’ve discovered those patterns through building a Behavioral Insights Team. Here, I’ll share the “talent stack” of skill sets that we’ve found is valuable for an in-house behavioral scientist and how to evaluate those skills in applicants. The exact mix and depth of these skills varies depending on a company’s specific needs, but behavioral roles will generally involve all of them in some capacity. I’ll break down the talent stack into five broad categories:
- Experimental Design
- Psychology Knowledge
- Data Analysis
- Digital Experimentation
- Product Management
What, exactly, makes someone a qualified behavioral researcher? What technical skills should an in-house behavioral scientist have?
#1: Experimental Design
What it is and why it’s needed: First and foremost, an applied behavioral scientist must be versed in designing and interpreting experiments. Running experiments, especially randomized controlled trials (RCTs), is the fundamental value-add of an in-house behavioral researcher.
Why is this so important? Applied behavioral science is more process than theory. While the field has gained notice from popular books filled with noteworthy examples of human behavior, applying findings directly from these books may you leave you disappointed. Success comes by knowing how to identify a behavioral problem, assess potential solutions based on the literature and your experience, and rigorously test solutions. Your customers likely differ from someone else’s, and an in-house behavioral scientist can help you discover what works (and what doesn’t) in a scientific and data-driven way.
What this looks like: An applied behavioral researcher must have experience running high-impact experiments. They should know the ins and outs of experimental design, not just the simple A/B testing that is often found in industry.
This includes experience running real world experiments (either academic field experiments or in a business setting) and a deep understanding of proper experimental design, such as randomization, statistical power, and variance reduction. While formal academic training is the most common route for acquiring these skills, more and more people are learning on-the-job, especially as companies increasingly experiment with growth marketing and conversion optimization.
A behavioral researcher must also be able to analyze the results of experiments and share the findings with a broader audience. This means having the theoretical and technical knowledge to perform statistical analysis, including sampling, T-tests, probability distributions, statistical significance, and confidence intervals. Successfully interpreting and sharing the findings requires strong communication skills.
How to evaluate: Have them provide detailed examples of experiments they’ve run, including how they defined the behavioral problem, the theory behind their interventions, how they chose the sample size, prepared the data, randomized the sample, analyzed the results, and interpreted the learnings for their team. They should be able to justify their experimental design and also explain them in a way that their less statistics-savvy coworkers will understand. Research is only valuable insofar as it’s communicated in a way that allows it to be applied.
#2: Psychology Knowledge
What it is and why it’s needed: Experiments are the core task of behavioral work, but a strong knowledge of psychology is what allows those experiments to succeed. The behavioral science and related disciplines inform the solutions that get tested and, ultimately, positively influence a company’s operations.
An applied behavioral researcher should have deep knowledge of at least cognitive and social psychology. They should be well versed in the works of the field’s leaders, like Daniel Kahneman, Amos Tversky, Richard Thaler, Cass Sunstein, Dan Ariely, and Robert Cialdini. But a good practitioner goes well beyond the core books and frequently dives into academic papers to stay on the cutting edge. The field is evolving regularly, so they should have their finger on the pulse of the latest developments within behavioral science and should note studies that have failed to replicate.
What this looks like: Practitioners should be capable of defining and applying key behavioral concepts such as dual process theory, bounded rationality, the psychology of habits, prospect theory, social norms, and temporal biases.
How to evaluate: To vet their ability to apply insights from psychology, take a problem your business experiences and ask the candidate to walk through their thought process of defining the behavioral problem. Ask them to map possible diagnoses and solutions. Have them detail how they’ve learned these concepts and what their reading habits are. How do they stay appraised of the latest developments in the field? How do they ensure they’re not fooled by poor research practices or studies that haven’t been replicated?
Success comes by knowing how to identify a behavioral problem, assess potential solutions based on the literature and your experience, and rigorously test solutions.
#3: Data Analysis
What it is and why it’s needed: Data skills are crucial for behavioral scientists for two reasons. First, high impact experiments require data collection, cleaning, and analysis. Second, data availability and analysis are often the biggest barriers to rigorous testing in organizations. Poor data practices—like losing track of what data is stored where—prevents effective testing in many companies. So, to make impactful experiments possible, a behavioral scientist must understand how to construct a data set that lends itself to experimentation and analysis.
What this looks like: A behavioral researcher must understand how their customer’s behavior is tracked and collected through analytical tools. The most common tools—such as Google Analytics, Adobe Analytics, Heap, and Kissmetrics—help researchers understand user behavior on websites and web-based software. Mobile products have some unique tools, like Mixpanel. Candidates should at least be familiar with these tools and preferably have high-level knowledge of how they work and when they should—or shouldn’t—be relied upon.
They also need to understand how to access that data. This requires a knowledge of how databases work and how to use SQL, a data management programming language, to extract data sets from them. It’s also helpful for a behavioral researcher to understand what a relational database (the most common form of database organization) is and how data warehouses (systems for reporting and data analysis) work.
A data skillset is rounded out with the ability to analyze and distill data to make it useful. This requires functional ability in a statistical programming package, like R, STATA, SAS, or Python. Which program will depend on the industry and organization, but R and Python are most common in the private sector. Candidates should be able to provide examples of code they’ve written and walk an interviewer through their programming. They should be able to perform common functions like regression analysis, cohort analysis, and pre–post comparisons.
How to evaluate: To evaluate a candidate’s data skillset, have them provide examples of data analyses they’ve conducted, including their code. Ask them to walk through the thought process and explain the findings. They should be able to detail each piece of the process and explain the results to non-experts, just like for experimental design.
#4: Digital Experimentation
What it is and why it’s needed: Experimental design and data skills must be paired with the modern digital experimentation tools that make both possible. Fortunately, technology has made testing within private organizations remarkably simple.
What this looks like: The tools can be divided up by channel. To test a website or a web-based product, point-and-click tools for A/B testing are needed; these include Optimizely, Visual Website Optimizer, Adobe Target, or Google Analytics Content. To test email communications, behavioral researchers will need experience building email campaigns and running tests in email clients like MailChimp, AWeber, or Eloqua. All have built-in A/B testing features that are simple to set up. While the process of actually building and launching tests is relatively straightforward, candidates need some functional knowledge of building lists and campaigns in these tools.
How to evaluate: To demonstrate their knowledge, have candidates walk you through how they’d use one of these tools to set up a basic web experiment. For example, ask them to simulate either an A/B test of email subject lines or two different versions of your website’s homepage.
#5: Product Management
What it is and why it’s needed: Finally, a strong set of product skills can allow a behavioralist to amplify a company’s digital products. A behavioral researcher need not be able to build the product, but they should know how it’s built. Product development is about resources: In order for behavioral researchers to provide input on how those resources are used, they must know the language and process of coworkers who deploy them. To prioritize behavioral interventions in a product development cycle, an in-house behavioral researcher must be able to work within that environment.
What this looks like: A good behavioral researcher must understand how behavioral interventions can fit within the product development process. This means understanding the typical models of product management, such as Waterfall, Stage Gate, and Agile. If the researcher knows how such features fit into a greater product plan, they will be able to help the team reach business objectives.
A behavioral researcher must also understand the language of the product world. This includes key performance indicators (KPIs) and how their organization’s particular product team measures impact. Any experiments have to align with those measurements and improve that impact. They should also know tools of product management—like JIRA and Aha—and have a general understanding of how they’re used.
Finally, a behavioralist should know the basics of qualitative user research. They should understand how to get qualitative feedback from users (or potential ones) before a test or new behaviorally informed feature, especially through user interviews and user testing.
How to evaluate: Candidates should be interviewed by members of the product team to ensure that they can communicate on the same terms. They should also be asked how they would answer a question using qualitative user research, and be able to articulate the trade-offs between different methods.
The skillset of applied behavioral scientists is still evolving. Not every behavioral scientist will have all of these skills, but this framework provides a starting point for evaluating whether they will be able to help your organization achieve its goals. While behavioral science has great potential, without the proper training and technical skills, those impacts can’t be realized. By assessing candidates’ skills using this talent stack, companies can begin to find the right person—or people—to reap the benefits of behavioral science.
