The Potential for Human-Computer Interaction and Behavioral Science

This article is part of our special issue “Connected State of Mind,” which explores the impact of tech use on our behavior and relationships. View the complete issue here.

A few days ago, one of my best friends texted me a joke. It was funny, so a few seconds later I replied with the “laughing-while-crying emoji.” A little yellow smiley face with tear drops perched on its eyes captured exactly what I wanted to convey to my friend. No words needed. If this exchange happened ten years ago, we would have emailed each other. Two decades ago, snail mail.

As more of our interactions and experiences are mediated by screens and technology, the way we relate to one another and our world is changing. Posting your favorite emoji may seem superficial, but such reflexes are becoming critical for understanding humanity in the 21st century.

Seemingly ubiquitous computer interfaces—on our phones and laptops, not to mention our cars, coffee makers, thermostats, and washing machines—are blurring the lines between our connected and our unconnected selves. And it’s these relationships, between users and their computers, which define the field of human–computer interaction (HCI). HCI is based on the following premise: The more we understand about human behavior, the better we can design computer interfaces that suit people’s needs.

For instance, HCI researchers are designing tactile emoticons embedded in the Braille system for individuals with visual impairments. They’re also creating smartphones that can almost read your mind—predicting when and where your finger is about to touch them next.

Understanding human behavior is essential for designing human-computer interfaces. But there’s more to it than that: Understanding how people interact with computer interfaces can help us understand human behavior in general.

Understanding human behavior is essential for designing human-computer interfaces. But there’s more to it than that: Understanding how people interact with computer interfaces can help us understand human behavior in general.

One of the insights that propelled behavioral science into the DNA of so many disciplines was the idea that we are not fully rational: We procrastinate, forget, break our promises, and change our minds. What most behavioral scientists might not realize is that as they transcended rationality, rational models found a new home in artificial intelligence. Much of A.I. is based on the familiar rational theories that dominated the field of economics prior to the rise of behavioral economics. However, one way to better understand how to apply A.I. in high-stakes scenarios, like self-driving cars, may be to embrace ways of thinking that are less rational.

It’s time for information and computer science to join forces with behavioral science. The mere presence of a camera phone can alter our cognition even when switched off, so if we ignore HCI in behavioral research in a world of constant clicks, avatars, emojis, and now animojis we limit our understanding of human behavior.

Below I’ve outlined three very different cases that would benefit from HCI researchers and behavioral scientists working together: technology in the developing world, video games and the labor market, and online trolling and bullying.

Technology empowers people in the developing world, right?

In the developing world, mobile technologies and the arrival of high-speed internet are revolutionizing areas such as finance, education, and science. These new technologies are changing people’s economic and social lives. However, there is little research on how people’s interactions with computers and mobile phones impact their economic and social behavior.

For instance, internet searches in Africa usually bring up sources from the United States and Europe, to the detriment of African websites. One avenue of research might explore human–computer interactions in internet cafes and the potential dominance of foreign-based websites at the expense of local ones. For instance, it may be the case that people who are younger, wealthier, and live in cities benefit disproportionately from these type of search results, relative to those in rural areas.

How could those in HCI and behavioral science collaborate to understand these issues? HCI researchers could gather data on which users are seeing online employment content and how and when that content is presented. Behavioral scientists could explore people’s perceptions of and the norms around certain occupations. This combination of data can give us a picture of both the online and offline factors that influence people’s choices in the labor market. Without this research, we risk overestimating or misconstruing the degree to which new technology and access to information is empowering people in the developing world and impacting economic advancement.

The mere presence of a camera phone can alter our cognition even when switched off, so if we ignore HCI in behavioral research in a world of constant clicks, avatars, emojis, and now animojis we limit our understanding of human behavior.

The unexpected effects of videogames on the labor market

Since 2004, young men have been working less and less, and technology might have something to do with it. On average, these 21–30-year-olds have work between 15 and 30 hours fewer each year. Recently, a group of economists found that video games, with their increasingly sophisticated graphical interfaces, may be responsible. The authors theorize that when deciding how much to work, people compare the benefits and costs of working. The benefit is a paycheck. The cost is the leisure time surrendered to employment. Advanced video games make our leisure time more attractive, and the desire to join the workforce suffers. An article in The New York Times compared this phenomenon to how advances in household technology motivated women to enter the workforce back in the 1960s, but in reverse. “If innovations in housework helped free women to enter the labor force in the 1960s and 1970s,” writes Quoctrung Bui, “could innovations in leisure—like League of Legends—be taking men out of the labor force today?”

What we don’t know from the research is why young men are making this tradeoff. What keeps young men playing video games at the expense of their careers? This is where HCI and behavioral science come in. By analyzing behavior like clicks, views, and eye scans while young men are playing video games, HCI researchers can offer insights into why the demographic finds video games so alluring. Is it the realistic graphics, the immersive storyline, the sense of control and achievement, or a certain combination of these factors?

Behavioral scientists can investigate how playing video games stacks up with searching for jobs. Perhaps the young men feel a lack of self-efficacy in the job-market, which they don’t feel when gaming. Perhaps they know and want to apply for jobs but feel a pull toward the games and aren’t able to sustain effort to their job search. (Scrolling through pages of job posts certainly doesn’t feel as exciting as fighting a dragon or defeating a dark lord.) With gaming data collected by the HCI researchers alongside personality and contextual data gathered by behavioral scientists, we could have a better idea of why there’s a problem—and develop interventions to help solve it.

Dealing with cyber bullying

“Cyber victimization is a problem that’s become pervasive,” writes Brendesha Tynes, an associate professor of education and psychology at the University of Southern California. “The bullies have moved from the playground to the mobile screen, and there is no escaping harassment that essentially lives in your pocket.”

Cyber bullying, particularly instances involving children and teenagers, is a growing concern for technology companies as well as parents, educators, and policymakers. One initiative to prevent cyber bullying on social media is the I am A Witness campaign. This campaign was launched by The Ad Council with supporters including Google, Facebook, and Twitter. It aims to “[activate] the ‘silent majority’ of kids who witness [bullying] each day, transforming them from passive bystanders into an active collective that speak up against bullying.” Kids can “speak up” by using a specially designed emoji (an eye in a comment bubble) and graphic keyboard to signify that they’ve seen bullying and that they don’t think it’s ok.

But how has the campaign, which began in 2015, impacted cyber bullying? Again, this is a question for behavioral scientists and HCI researchers. For one, they could collaborate on a randomized controlled experiment to measure the effects of the emoji on bullying behavior both online and off. The emoji might reduce bullying in some instances, but could it increase it in other scenarios? Without an experiment, we don’t know. Other well-meaning programs, like Head Start, D.A.R.E and Scared Straight!, provide cautionary examples of what can happen when an initiative is not evaluated rigorously. It’s not just time and money lost but also a failure to address the problems they were hoping to solve.

Finally, behavioral scientists themselves are not immune to online misogyny and harassment. For instance, an anonymous economics job board and forum, Economics Job Market Rumors, recently attracted criticism for abuse and misogyny. In fact, the American Economic Association, a major society of professional economists, recently decided to create its own online platform to provide a more professional, less abusive forum for the field. Offline, the field has come under fire for its discrimination of women and minorities.

Online and offline behavior, it seems, are better understood in tandem than apart.

Further Reading & Resources

  • Ward, A. F., Duke, K., Gneezy, A., & Bos, M. W. (2017). Brain Drain: The Mere Presence of One’s Own Smartphone Reduces Available Cognitive Capacity. Journal of the Association for Consumer Research2(2), 140-154. (Link).
  • Aguiar, M., Bils, M., Charles, K. K., & Hurst, E. (2017). Leisure luxuries and the labor supply of young men (No. w23552). National Bureau of Economic Research. (Link)
  • Aker, J. C., Ghosh, I., & Burrell, J. (2016). The promise (and pitfalls) of ICT for agriculture initiatives. Agricultural Economics47(S1), 35-48. (Link).
  • Dix, A. (2009). Human-computer interaction. In Encyclopedia of database systems (pp. 1327-1331). Boston, MA: Springer US. (Link)
  • Nass, C., Moon, Y., Fogg, B. J., Reeves, B., & Dryer, D. C. (1995). Can computer personalities be human personalities?. International Journal of Human-Computer Studies43(2), 223-239. (Link).