Responding to the unprecedented COVID-19 pandemic has been made even more challenging by an extraordinary infodemic, defined as an overabundance of information, including credible and false or misleading information, during a disease outbreak [1]. Information voids (ie, lack of accurate information on a specific topic from credible sources), misinformation (ie, inaccurate information), and disinformation (ie, inaccurate information designed to achieve an agenda) have been hallmarks of the COVID-19 infodemic [2-4]. Furthermore, information voids left by evolving science and rapidly spreading misinformation have served as breeding grounds for public confusion [3], posing a serious risk to compliance with public health prevention and the uptake of mitigation measures, such as mask wearing and vaccination efforts [4,5].

In the fall of 2020, the influence of the infodemic on public confidence in COVID-19 vaccines was evident before any vaccine was granted an emergency use authorization. National polls and surveys conducted in September 2020 and December 2020 indicated wide-ranging levels of public confidence in COVID-19 vaccines and their rollout [6]. In December 2020, before any COVID-19 vaccine was authorized for emergency use, more than a quarter of those surveyed said that they would not get vaccinated, citing concerns about the side effects and safety, lack of trust in the government, and worries about the speed at which vaccines were developed [7]. In addition, in December 2020 and January 2021, two-thirds of adults in the United States (65%) said that the federal government was doing a “fair” or “poor” job of distributing vaccines [7,8]. While these large-scale surveys signaled what Americans felt about COVID-19 vaccines, they did not provide an understanding of the reasons why they felt that way and the social and informational influences that factored into vaccine confidence and uptake.

In late 2020, the leadership of the US Centers for Disease Control and Prevention (CDC) COVID-19 Response recognized that the agency needed to act more swiftly to address the questions, concerns, perceptions, information voids, and circulating mis- and disinformation that affected people’s health decision-making, particularly regarding COVID-19 vaccines. At that time, there was no established, coordinated mechanism for collecting, reviewing, and synthesizing qualitative and quantitative data from multiple CDC-owned and external sources to assess the state of vaccine confidence in the United States.

The Insights Unit was established in January 2021 and became the first infodemic management unit of its kind at the CDC. The Insights Unit was part of the Vaccine Confidence and Demand Team within the Vaccine Task Force (VTF) of the CDC’s COVID-19 Response. The Insights Unit established a novel approach and methodology to rapidly detect trends in vaccine confidence by integrating multiple data sources and using established frameworks to understand the knowledge, attitudes, and behaviors of Americans regarding the COVID-19 vaccines. The aim was to use this methodology to help inform communication and programmatic strategies at a national level through a continual and iterative listening and feedback process [9].

The initial Insights Unit was a 3-person, multidisciplinary team led by a health communication specialist with experience in immunization campaigns and a behavioral scientist, with additional support from a data analyst with a public health background. The unit remained multidisciplinary and ranged in capacity between 1 to 2 full-time unit leads and 1 to 6 full- and part-time data analysts during the first year.

This manuscript aims to outline the Insights Unit’s first year and the establishment, approach, and methodology for COVID-19 State of Vaccine Confidence (SoVC) Insights Reports so that others can replicate and adapt its processes and methods. We hope to help health authorities and public health professionals explore opportunities for establishing insights units or complex social listening mechanisms for any public health area of concern. They can use these units and their findings to examine information ecosystems to gain insights into how their community’s thoughts and feelings affect critical health decisions and use it to enact policies, design and adapt programs, and inform communication campaigns.

The methodology was determined to be non–human subject research through the CDC’s ethics and project approval process. Additional ethics approval was not required as the data being considered were deidentified during the analysis step. Primary data sets, such as social listening data collected via Meltwater, comprised publicly available data falling under the public domain. Secondary data sets, such as survey data, did not contain any personal identifiable information, and data collection and analysis for those data sets underwent their own independent ethics approval processes.

As of August 2022, the Insights Unit has produced 27 full reports, 5 rapid reports, and 2 special reports. Rapid and special reports used the same methodology applied to full reports except with a shorter data collection period. Rapid reports pertain to a specific and time-sensitive issue, such as the emergency use authorization of COVID-19 vaccines in adolescents.

SoVC reports were distributed via email to a listserve of 956 readers, including CDC staff and external partners (eg, federal agencies, professional associations, community-based organizations, and nonprofits, in addition to worldwide health authorities and government representatives). Reports were also published on the web and have been downloaded 22,361 times. Findings and action steps were also shared through presentations to local, national, and international stakeholders from March 2021 to December 2021.

Within the agency, the reports were used by VTF and the emergency operation center to directly inform strategies for communication and community engagement. VTF’s communication team used the reports to enhance the CDC’s COVID-19 vaccine social media content, update digital content on myths and frequently asked questions, and improve search engine optimization. As the reports highlighted challenges faced by specific demographics, VTF staff used the findings to support partner organizations and state and jurisdiction efforts for on-the-ground vaccine demand generation. The reports also served as a unique mechanism for providing feedback to VTF’s vaccine distribution teams by highlighting underlying practical issues impacting vaccine uptake, such as challenges navigating digital systems to schedule vaccine appointments.

Between March 2021 and August 2022, SoVC reports were referenced by groups outside the CDC 22 times on the web. Organizations and agencies who referenced the reports included state and local health departments (n=6), nonprofit organizations (n=5), professional associations (n=3), and congressional committees (n=2). The reports were mostly referenced as a resource for learning about COVID-19 vaccine confidence and demand, with nearly half of the identified references (n=11) linking directly to the reports. Many of these references focused on increasing uptake of the COVID-19 vaccine, sharing the latest information on COVID-19 vaccinations, and improving public health communications. SoVC reports were also cited by research studies (n=4) to justify study goals and findings. For example, one journal article used a report as context to describe how perceptions of vaccine effectiveness can impact vaccine confidence [16-19].

In addition to serving as an informative resource, the reports were used to demonstrate how the CDC monitored and responded to COVID-19 vaccine mis- and disinformation. Then CDC director Rochelle Walensky referenced the reports and the Insights Unit at 2 congressional hearings, describing how the CDC was working to understand the public’s perceptions of COVID-19 vaccines and address the public’s information needs to increase vaccine uptake [20]. Furthermore, we identified 2 sources that applied findings and themes from the reports, one of which was the Africa Infodemic Response Alliance, which issued a report in June 2021 on COVID-19 infodemic trends containing recommendations adapted from an SoVC report.

To better understand the consumers of the reports, the value consumers derive from them, and how to further improve the reports’ utility, we conducted an outcome evaluation in October 2021 via a survey of the audience. We received 53 survey responses (6.4% response rate from the email distribution list) and solicited feedback from 6 stakeholders and 1 focus group session with 4 Insights Unit staff members. We found that approximately 75% (40/53) of respondents said that they found the reports “very” or “extremely” relevant and 52% (32/61) of respondents used the reports to inform communication strategies. The respondents also made suggestions for improvements (eg, creating bulleted summaries of findings and providing resources for readers to operationalize the reports’ recommendations), which led to several changes to make the reports more readable and useful. Subsequent reports, beginning in early 2022, incorporated bulleted summaries of themes rather than descriptive narratives and resources. These were provided within the reports themselves, and links to additional resources for communicating about the themes were also provided.

Our approach and methodology intentionally responded to urgent, interconnected needs. The primary priority was to assist the agency and our partners in acting more swiftly to address the questions, concerns, perceptions, information voids, and circulating mis- and disinformation regarding COVID-19 vaccines and inform communication and programmatic action. The second priority was to establish a coordinated mechanism for collecting, reviewing, and synthesizing qualitative and quantitative data from multiple internal and external sources. Our unit and reports were intended to bridge the gap among the CDC’s current social listening efforts, reports produced by external organizations, and current surveys and polls conducted by the CDC and third parties. Being one of the first integrated infodemic management products of such scale, scope, and regularity to be produced at the CDC, these reports interpreted findings in a novel way through the lens of behavioral theory and public health best practices. Similar approaches often focus on singular data streams rather than multiple data inputs to contextualize findings.

While our methodology was developed specifically in the context of understanding vaccine confidence and demand and the information ecosystem regarding COVID-19 vaccines in the United States, it can be adapted for any public health issue. Furthermore, the varied use of our reports suggests that they met the information needs of a diverse audience, served an informative purpose, and conveyed an urgency to address information challenges, implying that similar reports on other public health topics would support both routine and emergency functions. However, this methodology should not be directly reproduced without contextualization. It is important to conduct needs assessments and landscape analyses to understand the current state of the topic at hand and the population that is going to be examined, in addition to identifying data sources for consideration. Irrespective of these contextualization needs, our integrated analysis and the consensus-building processes are replicable, and others could create similar reports. In addition, the reports were produced by a rotating staff with different backgrounds and changing capacities, which implies that the analysis process and techniques used can be learned and scaled up or down depending on the skills and bandwidth of the staff.

There were several limitations to this methodology both for generating insights themselves and regarding its utility.

The Insights Unit’s methodology is not a solution for infodemics but rather a diagnostic technique that can rapidly identify opportunities for intervention. In addition, identifying mechanisms to act on the findings need to be created at the onset to ensure success. This methodology is also an adaptable process that can be scaled and used in a variety of public health settings for all public health issues. It can be applied beyond acute public health crises, such as within routine monitoring and surveillance. The methodology as outlined provides a road map for organizations focused on improving public health to conduct an integrated analysis of disparate data sources to generate and leverage insights during outbreaks and routine programming to inform population-level health. The insights generated can inform the deployment of effective interventions to fill information voids and counter misinformation, including partnering with trusted messengers.

The results of integrated analyses can be considered within already established, evidence-based conceptual frameworks and leveraged to deliver timely and responsive communication, programming, and policies. The continual listening and feedback process of insights units such as the one detailed in this paper allows health authorities to clarify guidance and improve programmatic quality in near real time based on changing public needs, especially within the context of a rapidly evolving public health emergency. Institutionalizing such units could enable public health messages, guidance, programs, and policies to better resonate and meet the information needs of the public. More responsive public health action could build trust between public health authorities and the public, which could lead to higher sustained uptake of health promotion strategies.

Public health authorities are especially well positioned to use their staff, including behavioral and social scientists, communication scientists, informatics specialists, ethicists, and epidemiologists, to institute insights units for population health promotion. These units can be on the frontlines of not only identifying and diagnosing infodemics but also building and promoting the science to test, evaluate, and scale promising interventions to rapidly respond to community needs, counter misinformation, bolster resiliency, and reinforce trust [25]. These units could benefit from being closely connected to traditional public health functions both within an institution and with external partners, including those focused on public health surveillance, field epidemiology, and health communications, to further contextualize and validate data trends [26,27]. Such collaborations allow health authorities to work more proactively.

The methodology and work of the Insights Unit and similar units are critical to the burgeoning field of infodemic management. This methodology has centered on data within the United States, a high-income and predominantly English-speaking country; these methods should be extended to better incorporate non–English-language digital listening in the future because information ecosystems are not limited by language or geographic borders. The methodology presented in this paper provided a national view of vaccine confidence and demand issues; localized subnational applications will require improved digital listening tools and investigation to determine the best online and offline sources from which to collect data. In addition, the development of new technology and platforms at low cost is essential to moving this work beyond high-income countries that can adapt current tools to perform these social listening and analysis functions. Artificial intelligence and machine learning could help reduce the burden of data collection and preliminary analysis, with promising approaches already being tested, such as using machine learning to help identify bots and measure infodemic risk [28]. Finally, more research needs to be conducted to better understand the impact that infodemics and infodemic management have on health behaviors. Indicators should be explored and examined to allow for quantifiable measurement of the burden of infodemics, which in turn will allow for improved evaluation of interventions and infodemic management itself.