Kaiser, B., Wei, J., Lucherini, E., Lee, K., Matias, J. N., & Mayer, J. (2020). Adapting Security Warnings to Counter Misinformation. arXiv preprint arXiv:2008.10772. https://arxiv.org/abs/2008.10772

University of Michigan - Ann Arbor

Researchers studied how social media users interact with warning signs by implementing two new methods of tracking user engagement with warnings against disinformation on social media. They found that contextual warnings (do not interrupt users) are less effective than interstitial warnings (require users to stop and interact). Research questions:
- After encountering contextual and interstitial disinformation warnings, how often do users change their behavior by opting for alternative sources of information?
- Why do some users choose not to change their behaviors after encountering contextual and interstitial disinformation warnings?
- Can interstitial warnings that are highly informative effectively change user behavior?
- Can interstitial warnings that are highly threatening effectively change user behavior? 

Adapting Security Warnings to Counter Misinformation

- Recruited 40 students through Princeton University’s Survey Research Center and to participate in the experiment by taking part in a survey where two questions had no warnings (control) and two questions had either contextual or interstitial warnings (treatment) to study if the participants interacted with the warnings
- Participants were asked to narrate their thought process as they navigated through one of two websites to find information and news from other countries and barely mentioned in the US and chose sources from not well-known sources 
- Clickthrough rate (CTR) calculated by accumulating the number of times participants clicked on a result with contextual warnings and the number of times they clicked on “Learn more” but ignored the interstitial warnings
- Alternative visit rate computed based on whether participants used more than one website to study whether participants confirmed results with both sources or switched to another source when they encountered the warnings
- Interviews conducted to learn whether participants saw and understood the warnings. Interview questions aimed to understand whether participants knew and could recognize disinformation, what warning format was most effective for them, and how they felt when they first saw the warnings 

Adapting Security Warnings to Counter Misinformation - Laboratory Study Methodology

- Crowdworker study was used study the behavior effects of interstitial warnings
- Recruited 238 participants through Amazon Mechanical Turk and used a similar setup as laboratory study where participants had to answer questions about topics that were not talked about in the US, researchers paid attention to CTR and AVR and administered surveys, but treatment consisted of one of eight potential interstitial warnings 
- Using a multi-armed bandit algorithm to increase the chance that participants would encounter warnings that were deemed most/least informative and most/least effective at inducing fear of harm 
- Designed 8 interstitial warnings – 4 targeting information and 4 targeting fear of harm; information warnings contained detailed explanations and harm warnings used threatening and more extreme language to evoke fear
- 4 rounds of surveys asking participants why they decided on a result after each time they enter their answers before moving on to the next task
- Treatment surveys had an extra step that asked if participants understood the purpose and presence of the warnings (choice of malware, information theft, disinformation) and whether they felt fearful or harmed by the warnings
- Changes in behavior effects measured as the difference between the mean scores for informativeness and harm

Adapting Security Warnings to Counter Misinformation - Crowdworker Study Methodology

- Role of user experience friction in disinformation warnings because results indicate that such friction could be the primary reason for changes in behavioral patterns
- Apply interstitial warnings to social media platforms 
- How interstitial warnings along with other factors that influence behavior patterns together affect users’ susceptibility to disinformation 
- Unintended impacts of interstitial warnings 

Learn Before

Related