In this article, we review 3 case studies illustrating the pragmatic application of an IS TMF and its associated methods, the PRISM, to LHSs. These examples highlight useful and practical applications of PRISM and serve as a basis for discussing challenges and future directions related to the use of IS methods and EHR data.

Although many IS TMFs exist [22,23], we selected PRISM to concretely illustrate the application of a TMF within an LHS. PRISM is an expanded version of the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework, incorporating 4 context domains and the 5 RE-AIM outcome dimensions [17,24-27], as detailed in Tables 1 and 2. PRISM evolved from a blend of frameworks from agriculture (Diffusion of Innovation), engineering (Plan-Do-Study-Act quality improvement cycles), and health care (Chronic Care Model) [24]. PRISM is designed to be used throughout the life cycle of a study or project to guide the systematic assessment and alignment of the project with its context, aiming to maximize equitable impact on relevant outcomes and sustainability. PRISM underscores the importance of aligning a project with the diverse perspectives and characteristics of various partners, including those who will be directly or indirectly impacted or involved in approving and funding the programs, such as frontline staff, clinicians, clinic or department leaders, system-level leaders, and boards of directors. For effective alignment, it is crucial to ensure the representation of these partners. A key aspect of promoting equity is defining relevant outcomes that are important from diverse perspectives and measuring the representativeness of these outcomes across various demographics (patients) and types of clinics or providers (settings) [27]. PRISM’s RE-AIM outcomes facilitate discussions about relevant and meaningful outcome measures at different levels of perspective (eg, leadership, clinician, patient) while emphasizing representativeness and pragmatic issues such as adoption and uptake. PRISM also takes into account the external context (eg, policies, guidelines) and the support or infrastructure available for initial implementation and sustainability (eg, resources, audit, and feedback processes). This consideration enhances the likelihood that the project will continue beyond the study timeline or funding. The systematic approach provided by a TMF like PRISM, combined with the RE-AIM pragmatic outcomes, allows a project to be adapted and scaled up in ways that are locally relevant to different situations and health systems.

To illustrate the challenges and potential solutions of applying IS TMFs to LHS learning cycles and using EHR data, we present retrospective case studies of 3 applications of PRISM to different LHS projects across diverse health care settings. These case studies were selected based on the authors’ experiences to represent various types of projects and settings, as well as different approaches to applying PRISM and addressing key issues within LHSs. We focus on identifying challenges and solutions related to (1) operationalizing and applying PRISM within the EHR-embedded LHS context; (2) pragmatically adapting PRISM and its RE-AIM outcomes based on available resources, expectations regarding speed, and data availability; and (3) effectively utilizing EHR data. The solutions provided aim to offer guidance and direction on addressing these challenges and improving the practical application of PRISM and other TMFs and IS methods to EHR-driven LHS.

To facilitate a systematic evaluation of how PRISM was applied in each case study and to minimize recall bias, we first adapted our previously published framework for a fully mature LHS to illustrate where and how PRISM and its associated methods can be applied (Figure 1) [10]. The original framework was designed to be agnostic to any specific IS TMF, so we modified it by overlaying PRISM and highlighting where and how it integrates with the broader LHS framework. In this figure, PRISM guides and informs key aspects and activities of an LHS, including representativeness and equity of perspectives and outcomes; achieving local relevance and external validity or generalizability; rapidity of change and impact; and designing for sustainability. This adapted figure of a fully mature LHS was used to stimulate recall and identify challenges and solutions when applying PRISM and using EHR data. For each case study, we thematically reflected on the challenges and both actual and potential solutions at each phase of the implementation continuum (ie, preimplementation or planning, implementation, and sustainment or evaluation) [28]. This reflection was conducted both inductively to identify new themes and deductively by considering preidentified themes. We also identified aspects of the EHR that pose barriers to applying PRISM and other IS TMFs, which are crucial to address in order to achieve the aspirational goals of a high-functioning LHS.

In each case study, we identified several challenges, potential solutions, and considerations for future research when using EHR data, PRISM, and other IS approaches. The detailed evaluation and findings from each case study are described in Multimedia Appendix 1 and are categorized into 6 overarching themes of challenges with corresponding solutions. These themes and solutions are described below and summarized in Table 4. Across these themes, there are interdependencies, which are expected from a systems science perspective. For instance, resource and time constraints can exacerbate EHR data limitations, and varying levels of IS expertise can complicate the appropriate application or adaptation of PRISM and IS methods. For each challenge, we discuss the issues,potential solutions,and future directions.

Team science refers to the cross-disciplinary collaboration necessary to address scientific questions and challenges [37]. Both LHS and IS require multilevel engagement from partners with diverse roles, backgrounds, and perspectives, which introduces complexities. Representatives from various partner groups (eg, patients, community members, clinicians, leaders, nonclinical staff, and researchers from different fields) bring unique histories, perspectives, terminologies, biases, assumptions, and knowledge. Sometimes, communication issues or differences in perspectives are recognized, but they can often remain unnoticed for extended periods.

These team science challenges are manageable and should be addressed during the planning phase and throughout the project [35]. Taking the time to understand and respect each partner’s perspective and developing a shared vision and vocabulary is essential for team effectiveness and efficiency. It is also important to continually emphasize that different perspectives are not only beneficial but also expected [35]. The use of tools such as the iPRISM webtool [17,36] can systematically capture the diverse perspectives on teams and summarize the mean and distribution of scores, which can be used to focus team discussions on areas with lower mean scores (indicating areas for improvement) or where scores vary and perspectives differ. In the future, greater use and availability of tools such as the iPRISM webtool are recommended to facilitate team science principles within LHS and when using IS, including other TMFs [38,39] beyond PRISM.

In an LHS, and within health systems more generally, there may be variable or no IS expertise, which can preclude the application of IS altogether or lead to inconsistent or incomplete implementation. Inconsistent application may result in replication issues or incomplete assessments of context or project alignment, ultimately affecting project outcomes and sustainability. Understanding how to apply IS principles and TMFs such as PRISM can be challenging without training, and it is often impractical to train all team members.

One potential solution is identifying external IS expertise, which may be feasible through existing consultation services [40-42]. Utilizing resources and tools that make IS more accessible to individuals and teams by simplifying its application can also help. Tools such as the iPRISM webtool [17] and other resources [43-50], which can increase the accessibility of IS, are described in Multimedia Appendix 2. In our case studies, we identified specific aspects of applying PRISM and IS generally that would benefit from greater guidance and tools, such as how to feasibly and systematically anticipate, assess for, and mitigate unintended consequences—including those that can exacerbate or create inequities—and how to design for sustainability, including identifying and securing resources. Capacity-building efforts are also needed to train implementation researchers and practitioners [51].

Timely and feasible access to complete, accurate, and actionable quantitative and qualitative data is a universal challenge for LHS and IS [6,10,52-54]. These challenges limit outcome evaluations, contextual assessments, and the capacity to rapidly and strategically design interventions and programs that are equitable and optimally fit within workflows [55-58]. Often considered together under the umbrella of “informatics,” access to the needed technology is also a challenge. Even when technologies are accessible, they do not always have the functionality needed to seamlessly embed within clinical workflows. Furthermore, there may not be enough skilled staff to use or configure the technology to achieve the goals of delivering the “right” information at the “right” time, to the “right” person, in the “right” format, and through the “right” channel [59].

When faced with data and technology limitations, it is important to first recognize the potential issues and how they might impact a project, and then explore workarounds or strategies to monitor for potential downstream consequences. Workarounds to address data access issues include creating proxies or estimates and conducting sensitivity or subgroup analyses, which may not be precise but can provide valuable insights and aid in understanding. Addressing technology functional limitations often requires adopting a “good enough” mindset, provided the benefits are expected to outweigh potential sacrifices in user experience and no harm is anticipated [60,61]. Other strategies include transparently reporting the data and acknowledging technology limitations, allowing the audience to make informed interpretations of the findings. The current reality may compel LHS projects to operate within existing constraints, but it is crucial to advocate for the goals of truly inclusive precision health [62], which requires comprehensive integration of data to support holistic care decisions. Guidance is needed to help current LHS teams optimize their projects and evaluations within these constraints. However, it is also important to challenge and push against current limitations when necessary. For example, when available data are known to produce biased information or fail to include critically important patient or contextual factors, the health system may need to add essential data elements or utilize novel methods [63,64] to achieve impactful and equitable results [56,62,65,66].

In most projects, restrictions on time, availability of skilled personnel, and other resources can slow data access or preclude it entirely. Additionally, the time required from the implementation team and participants can limit the breadth of perspectives engaged and the frequency of qualitative assessments. These limitations can negatively impact the equity and sustainability of projects.

To make progress, LHS teams often need to adapt evaluation plans and partner engagement methods to fit within available time and resource constraints. Finding partners and collaborators who can generate win-win situations and extend resources is also crucial for overcoming these limitations. There is also a growing availability of consultant-type services to support LHS initiatives, offer specific methodological expertise, or connect projects with needed resources [67,68]. Advances in artificial intelligence, such as natural language processing and machine learning, can enhance efficiency and reduce resource usage by automating data collection and analysis. Although these advancements are increasingly present within LHS, improving accessibility to these skills, resources, and software for automation is essential for further enhancing efficiency. Additionally, while the application of artificial intelligence approaches can offer significant benefits, it is important to exercise caution and carefully balance and vet automated processes [69,70]. To further improve efficiency, creating and disseminating “how-to” or implementation guides and recommendations [42,71-73] could help reduce the resources needed to understand, adapt, or apply specific technologies, data, or methodologies.

Not all aspects of IS apply to every LHS situation for a variety of reasons. In some cases, a TMF construct or IS method may not align with a project’s goals, may not fit within resource or timeline constraints, or may need to be adapted to suit the situation [74]. Additionally, when implementing interventions without an established evidence base, flexibility is required in applying IS methods and partner engagement strategies. For instance, the LHS may be implementing a clinical guideline recommendation based on poor-quality or low-strength evidence, or following a new regulatory mandate for an intervention that has yet to demonstrate effectiveness [75].

IS is intended to be practical and pragmatic, and its principles, outcomes, and TMF constructs should be considered as a guide and adapted to the situation at hand, focusing on what is feasible and relevant for the context [74]. When changes are made, it is important to document and report the adaptations along with the rationale to facilitate future scalability. For projects without an established evidence base, some common IS strategies, including how and which partners are engaged, may need to be adjusted. Further, when a project’s evidence base is uncertain or may vary based on contextual conditions (eg, the effectiveness of a CDS alert varying by clinical situation and design), iterative IS and LHS approaches can be leveraged to develop the evidence base and understand the necessary conditions for success.

To increase the uptake of IS, it is essential to promote awareness that IS should be adapted to fit the specific needs and context of each project [76,77]. Current misconceptions that IS cannot be adapted may inhibit its application. Providing guidance on how to adapt TMFs and IS methods, supported by case examples, can help address these misconceptions and enhance the accessibility and use of IS. Additionally, interactive tools such as the iPRISM webtool, which dynamically guides implementers through the process of adapting IS methods for their specific project, could further facilitate this adaptation.

Limitations in the representativeness of documented data or in the range of partners engaged can impede the ability to design equitable solutions. This may stem from difficulties in assessing equity of outcomes and having a limited number of partners to strategically address existing disparities [27,62,78]. Such limitations could exacerbate or create new inequities without the capability to use data to identify and resolve these issues. IS methods, including PRISM, promote representativeness in data and partner engagement, which may not always be achievable within existing constraints. Therefore, when planning LHS or learning cycles, equity should be clearly defined and prioritized from the outset [79].

To the extent feasible, inclusive use of data and engagement of partners across the spectrum of perspectives—not just the average or majority perspective—is important for promoting equity within LHS [27]. Proactively considering the potential unintended consequences of using different types of data is also key to mitigating inequities. When possible, integrating data sources beyond the EHR (eg, social media, patient and staff satisfaction, community forums, community partner data) and using systems science approaches that include patient-reported outcomes and other social determinants and behavioral data can aid in more comprehensive consideration of the data needed to promote health equity [80]. There is a clear need for health systems to access a more inclusive integration of reliable, structured data.