Some qualitative approaches are inherently linked to their broader methodological frameworks (eg, narrative analysis, interpretative phenomenological analysis, and grounded theory) while others are more flexible in terms of belonging to a certain methodological framework (eg, thematic analysis [16] and content analysis [27]). Thematic analysis, qualitative content analysis, grounded theory, and interpretative phenomenological analysis are some of the many forms of pattern-seeking qualitative approaches. In our opinion, grounded theory and interpretative phenomenological analysis, which are highly interpretive and explorative, are less likely to fit the needs of most digital health research exploring participant usability of and engagement with digital tools, and we are not going to discuss them here. (Good guidance on these methods can be found via Strauss and Corbin [28] and McLeod [29].) Instead, we’ve highlighted the 2 most commonly used approaches in digital health research exploring participant experience and user engagement.

Qualitative content analysis, including conventional, directed, and summative approaches [30], uses systematic methods to analyze patterns in text and explore meaning. Sometimes considered an “intermediary” approach between qualitative and quantitative methods [31], content analysis, particularly some types, such as summative, use methods that borrow from quantitative research, such as counting the frequency of words or phrases [30]. Researchers using content analysis may take a deductive, inductive, or abductive (combined) approach and often use it deductively given its utility for exploring predefined categories and/or incorporating an existing behavioral or theoretical framework [27,29]. Conventional content analysis can also be used more interpretively, and shares some similarities with certain types of thematic analysis (such as the framework method, discussed below) [32].

We find qualitative content analysis particularly useful in digital health intervention evaluations that build on existing frameworks or previous research (as most of our work does). We have also found directed content analysis to be an efficient and effective method to analyze qualitative data within mixed methods randomized controlled trials of digital health interventions (such as in Brenton-Peters et al [33] and Serlachius et al [34]).

Similar in some aspects to content analysis, thematic analysis is a “family” of related methods [19] that involve labeling (“coding”) data and then organizing them into themes (patterns of meaning). Reflexive thematic analysis, which directly addresses the researcher’s role and process in the analysis, is Braun and Clarke’s [16] updated approach and seeks in part to differentiate itself from older styles of thematic analysis, such as codebook and framework analysis, which take a more structured, and often combined inductive/deductive, approach.

Due in part to Braun and Clarke’s [16] detailed, step-by-step description of thematic analysis published in 2006, this approach has become widely used. They have written at length on how this approach is frequently misunderstood and misapplied, as demonstrated by a recent study where they reviewed 20 health-related studies that used thematic analysis and found the most common problem was in the creation of themes [20]. In reflexive thematic analysis, a theme should highlight a broader meaning across the dataset, telling an interpretive story. However, many studies confuse themes with “topic summaries,” such as “helpfulness of the intervention” or “ease of use.” Braun and Clarke [20] note that if the theme could have been created before conducting the data analysis (ie, it could be mapped directly to an interview question), then it is a topic summary.

We feel that if your research question and aims align best with topic summaries and deductive analysis, qualitative content analysis or codebook or framework thematic analysis may be a better fit than reflexive thematic analysis or other more interpretative approaches.

Most digital health intervention evaluation studies have pragmatic goals grounded in a realist (ie, “scientifically oriented”) view, building on previous research findings, for example, as part of mixed methods research. If your study fits within this orientation, strongly consider using a method like content analysis or thematic framework analysis, which can flexibly accommodate existing frameworks and build on previous quantitative findings.

While more inductively aligned methods of thematic analysis or other more interpretative approaches such as narrative analysis or grounded theory can be excellent approaches at the development stage of a digital health intervention, when there is more emphasis on exploring in-depth patient experiences, they are frequently less aligned with most research aims around digital health intervention evaluation and exploring questions regarding user experience and user engagement through a realist epistemology. However, there are exceptions; for example, a study by Knox et al [35] explored stakeholder views on virtual pulmonary rehabilitation using a critical epistemology, and a paper by Bleyel et al [36] explored patient perceptions of mental health care through video consultations from a critical realist perspective [36]. These examples demonstrate the importance of articulating your research aims, methods, and methodological perspective to justify your chosen approach, which is not always easily achievable within the tight word-count limits of medical journals.

Also carefully consider whether you want to take a deductive, inductive, or abductive approach in your analysis. While you can use any of these approaches across the different types of content and thematic analysis methods [27,37], you should select an analysis method that works within your research aims and state this orientation in your reported methodology.

Many digital health interventions are built on existing intervention or behavior change frameworks, such as the capability, opportunity, motivation-behavior (COM-B) model [38] and the reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework [39]. If you want to use an existing framework in your qualitative analysis, as many digital intervention studies do, choose an approach designed for this, such as directed content analysis or thematic framework analysis. Do not choose a highly interpretive approach, such as reflexive thematic analysis or interpretative phenomenological analysis, if you intend to map your qualitative findings onto an existing framework. A good example by Szinay and colleagues [40] used the framework method to explore engagement with well-being apps informed by the COM-B model. For further details regarding the framework method, refer to Gale and colleagues [32], who provide a comprehensive outline of the 7 stages of analysis using this method in health research.

Qualitative research is well suited to answer “how” and “why” questions, such as why individuals did (or did not) use a digital health intervention and how they went about it. But different research questions may be better suited to one type of qualitative analysis than another.

For example, let us suppose your digital health intervention for weight loss had a high drop-out rate, and you want to use qualitative methods to explore reasons why. The most suitable qualitative approach depends on your underlying rationale and research strategy (including your methodological/theoretical position) and what you plan to do in response to your findings (ie, changes or iterations to the intervention or exploring a different approach altogether). If you want to explore topics such as participants’ beliefs about weight loss and/or perceived barriers to weight loss, narrative analysis or another more inductive approach may be more appropriate. However, if, as is more common in digital health intervention evaluation research, you have specific questions related to participants’ opinions about the design, content, and activities within the intervention itself that caused them to stop using it, then a more focused and deductive or abductive qualitative approach (ie, content analysis) may be more useful.

According to a 2019 systematic review, frequently used qualitative data collection tools in digital health intervention evaluation and testing and exploring users’ perspectives include “think-aloud” protocols, focus groups, and interviews [41]. We often also use open-ended online survey questions in our mixed methods evaluation studies. It is important to consider the type of data you are collecting alongside your analysis plan.

Stemming from cognitive psychology, the think-aloud method asks participants to share what they are thinking while performing a task [42], which can lead to a focused data set. Because of this, the think-aloud method is popular in usability research and has been found to successfully generate intervention improvements [43]. Think-aloud data are most commonly analyzed using a deductive approach and with methods similar to content analysis, such as counting positive or negative sentiment [44], but these data can also be coded and organized into topics or themes [12].

Focus groups and interviews can generate large datasets that may cover a broad range of topics and experiences. While exploratory questions may be useful in digital health intervention development research, a more focused approach is usually necessary to meet the goals of evaluation research or exploring whether a digital tool met participants’ needs. Therefore, it is critical to develop an interview guide that considers your research questions, theoretical framework, and desired result type (ie, themes or frequencies). Otherwise, you may end up with a wide-ranging and meandering dataset that does not answer your research questions and leads to what Braun and Clarke [20] call a “mish-mash” of ineffective qualitative analysis.

Digital health interventions often require rapid evolution to stay relevant, and researchers may have limited time, resources, and funding [7]. Therefore, it is important to consider how quickly you need to generate your results, as well as the resources needed to complete the analysis.

In terms of the approaches discussed here, reflexive thematic analysis, interpretive phenomenological analysis, grounded theory, and narrative analysis often require significant time to conduct, with multiple rounds of detailed, in-depth coding and thematic review and ongoing exploration regarding reflexivity and how it influences the process and outcomes of the analysis. Content analysis or framework thematic analysis can be done in a more efficient and timely manner, with codebooks often developed in advance of analysis and the frequent use of co-coders with a focus on accuracy.

Increasingly, so-called rapid approaches have been developed and compared against more traditional qualitative approaches due to this need to reduce time and improve efficiency in health care research [45,46]. For example, Holdsworth et al [47] used a rapid form of framework analysis to summarize evaluation data from intensive care unit site visits while still on site. They found this approach delivered significant savings in analysis and transcription costs.