The International Simulation Data Registry: Harnessing the Standardized Data

The International Simulation Data Registry (ISDR) was established in 2014 and is inspired by the increasingly numerous and widely used clinical registries. It aims to enhance simulation-based medical education (SBME) and improve clinical practices and patient outcomes. A recent strategic partnership was initiated between the Society for Simulation in Healthcare and the University of Toledo to update the ISDR to version 2.0. This collaboration aims to leverage a novel multimodal assessment platform, enhancing the ISDR’s capabilities. The project not only boosts medical training and patient care but also lays the groundwork for tackling research hurdles like standardized data sharing, and enhancing education within the simulation community, reinforcing its significance for the future of healthcare education and practice improvement.

DOI: https://doi.org/10.69079/simzine.H24.n14.00011

Relevant Background

The global healthcare simulation community has grown tremendously over the last two decades. Many simulation centers worldwide have contributed to education and research efforts to optimize simulation-based medical education (SBME) outcomes. However, SBME has not realized its full translational science potential by demonstrating that effective SBME approaches lead to improved clinical practices and, ultimately, better patient outcomes[1]. The lack of infrastructure for sharing standardized data from healthcare simulation activities and correlating it with practice worldwide presented a challenge in bridging the gap. 

The Emergence of the International Simulation Data Registry

The International Simulation Data Registry (ISDR) was established in 2014 to drive evidence-based SBME, better clinical practice, and improved patient outcomes[2]. The ISDR was inspired by the growing utilization of clinical registries, which facilitate care quality benchmarking, optimizing health outcomes, controlling costs, and conducting epidemiological research[3]. Accessible data registries are also vital for developing effective tools and technologies across all healthcare fields and specialties, especially as machine learning and artificial intelligence become more prevalent [4]. 

The initial rollout of the ISDR targeted standardized cases with clearly defined performance metrics based on American Heart Association Guidelines, such as pulseless cardiac arrest[2]. The ISDR was launched across 27 institutions, collecting standardized data including (but not limited to) patient simulator type, simulation setting, simulation participant discipline and years in training, and key objective performance measures (e.g., time to initiation of care). There was considerable interest and momentum within the ISDR after its launch; however, this was sidetracked due to the COVID-19 pandemic. 

Enhancing ISDR with PREPARE: ISDR 2.0

In June 2023, the Society for Simulation in Healthcare (SSH) and the University of Toledo (UToledo) initiated a strategic partnership to update the ISDR. The ISDR 2.0 will leverage a novel multimodal assessment platform (PREPARE) developed at the University of Toledo at Dr. Pappada’s laboratory[5]. The simplicity of the user interface enables standardization of curriculum generation and underlying assessments, making PREPARE highly synergistic with the vision and goals of the ISDR. A key feature of the platform is to collect, process, and synchronize data from multiple sources, such as observer-based performance assessments, operational/training environments (e.g., audio, video, simulator data), and learners. 

Addressing Simulation Challenges with ISDR 2.0

With the new functionalities afforded by PREPARE, ISDR 2.0 will address some of the challenges faced by the simulation community. One of these challenges is the lack of standardized scenarios, underlying data, and assessments to be collected during SBME activities.

The holy grail of simulation-based research is to implement outcome measures that can be collected in both simulated and real-world patient care settings.

Standardizing metrics and key performance measures in a way that can be seamlessly applied in both simulated and real environments ensures that key comparisons can be made. This will allow the community to evaluate how knowledge, skills, and practices trained and evaluated in simulated environments translate to real-world care settings.

Another challenge lies in the subjective performance evaluation that accompanies SBME, often resulting in a simplistic pass/fail assessment rather than acquiring richer data. Consequently, learners receive feedback that is often not personalized or tracked longitudinally to encourage reflection and inform repeated deliberate practice for improvement. The ISDR framework provides a standardized way to collect clinically relevant, objective data that can be consistently tracked over time. This data can be used for feedback and benchmarking across learners and simulation centers, optimizing the SBME processes and outcomes. Figure 1 shows the four key areas that ISDR will address. Two of these areas (**denoted) will be available with future development and advancement of the PREPARE platform through collaboration amongst the simulation community. 

The Society for Simulation in Healthcare and UToledo teams are collaborating on the initial deployment of ISDR 2.0, which introduces instructor-level assessment. These features enable standardized curriculum, scenario creation, and real-time assessment during curriculum delivery. Unlike the initial version, ISDR 2.0 allows assessments to be entered as the scenario unfolds rather than retrospectively. Furthermore, all assessments are time-stamped, facilitating the recording of objective performance metrics such as time to intervention or diagnosis.

PREPARE is platform-agnostic and can be utilized across various simulation centers or sites. Being web-based, users can access the platform on any device with internet browsing capabilities.

The instructor-level functionalities of ISDR 2.0 are detailed in an initial publication outlining the multimodal assessment capabilities of the PREPARE platform [6]. Additionally, certain learner-level data acquisition capabilities are available in ISDR 2.0, including customizable pre- and post-assessments, survey generation, and a collection of learner-level demographics and training frequencies.

The SIM SCENARIO CANVAS

PREPARE has additional capabilities that are part of a forward-looking development strategy [4]. The current capabilities of the PREPARE platform are shown in Figure 2, which demonstrates the data collection at the learner, simulation environment, and instructor levels. The automated data collection at the learner and simulation environment levels will not be included in the initial rollout of the ISDR, but may occur in subsequent phased roll-outs. 

PREPARE also enables real-time monitoring of simulation participants’ physiological data, which correlated with learner performance in our initial studies [6, 7]. Additionally, it can capture audio from the simulation environment to align verbalized events/interventions with expected events, employing a speech-to-text (STT) and natural language processing (NLP) module for event detection [8]. Future enhancements involve implementing computer vision for automated detection of procedural interventions and team dynamics.

The Future of Simulation in Healthcare with ISDR and PREPARE

Finally, as artificial intelligence (AI) continues to grow, existing registries are already being used to develop predictive models and machine learning (ML) approaches to optimize and personalize healthcare. The vision for the ISDR is to provide a cohesive and intuitive infrastructure to promote this for the simulation community. Figure 3 illustrates the vision for ISDR and PREPARE in generating the necessary data for this purpose. Although some of PREPARE’s functionalities will not be available in the initial release of ISDR 2.0, accumulating and sharing data through a common platform utilization will bring us closer to realizing this ultimate vision.

READ ALSO

total words:1390 - positions {"top":0.3,"middle":0.6,"bottom":0.9}