RAPID: Automated Extraction and Validation of the Gist of Social Media Messages about COVID-19

While the novel coronavirus sweeps the globe, democratic societies face a quandary, namely, how to encourage sacrifices to reduce risk, even when the threat is invisible and seems to be receding. When risk is high, individuals may need to engage in extreme forms of social distancing for long periods, but this choice comes with economic and personal costs. Not engaging in these risk-reduction activities, however, could cause millions of deaths. It is therefore crucial for public health communicators to understand rationales for refusing these actions and to base risk communication on empirically supported principles, including the need to convey the gist (bottom-line meaning) of these actions to individuals making these challenging choices.

This project consists of three studies, each with two samples at different time points, with the ultimate goal in subsequent studies of converting the gists of social media messages into effective risk-communication interventions. The approach is motivated by Fuzzy Trace Theory?an evidence-based account of health decision making under risk?which posits that decisions are based on qualitative gist representations of stimuli that encode basic meaning in context. The scholars test time-sensitive hypotheses using bottom-up unsupervised machine-learning algorithms to characterize the topics in social media messages about COVID-19 and top-down human judgments about theoretically predictable mental representations of the gist of perceived risks and benefits of risk-reduction behaviors. Study 1 quantifies the prevalence of gists pertaining to social distancing and other risk-reduction behaviors (e.g., hand washing) by extracting millions of social media messages from Twitter and public Facebook posts. This involves automatic coding an ongoing corpus of millions of social media messages using probabilistic topic models. Study 2 provides a systematic approach with human judges to interpreting the topics extracted by a topic model at different time points. Study 3 involves a different group of human judges assessing topics for their consistency with a targeted set of gist representations of risks and benefits, along with gist principles that express values. The researchers adapt items used successfully in prior research. Similar gist representations have predicted self-reported risk-reduction behaviors for numerous health conditions. Thus, for Study 3, the researchers validate topic gists by fielding a theoretically motivated survey analyzed for reliability and construct validity. The focus is on perceptions of the gist of risks and benefits, especially gists that can promote the health of individuals and society. Overall, this project introduces novel techniques for eliciting gists from social media that can be used to generate meaningful public health communications. The project directly informs existing attempts by public health communicators to express the risks, benefits, and actions that members of the population should take to mitigate the spread of the COVID-19 pandemic. The measures are validated against human users, enabling the team to achieve both accuracy and scale. This project serves as the basis for a larger effort that can increase the extent to which gist elicitation may be automated, helping public health communicators to quickly understand which gists should be communicated to which communities during outbreaks.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.