Screen-Based Virtual Simulation in Medical Laboratory Science Education: Findings from a National Program

Virtu-WIL

Virtu-WIL is a project led by Colleges and Institutes Canada, the national organization for Canadian publicly supported colleges, institutes, and polytechnics, in collaboration with Simulation Canada.^10,11 The Virtu-WIL project uses virtual screen-based simulations to enhance health care education. VSs were developed for the Virtu-WIL project by expert educators in their respective fields. The VSs underwent extensive usability testing; peer review; and diversity, inclusion, equity evaluation before deployment.^10,11 The newly developed VSs are provided for free, with open access to approximately 200 VSs designed to provide students in nursing, medical laboratory sciences, paramedicine, respiratory therapy (RT), and sonography with experiential learning experiences and practical competencies.

VSs were hosted on several platforms, such as PCS Spark and Affinity Learning.^12,13 At the time of publication, there are 25 VSs available for medical laboratory science (eg, Mass Spectrometry Troubleshooting, Flow Cytometry in a Hematology Analyzer, and A Virtual Simulation Game on the Communication of Laboratory Results). For an example simulation, follow this link: Blood Cultures - Simulation Canada.¹⁴ The full list of simulations is available at the Simulation Canada website.¹⁵

A VS experience will begin before accessing the VS, when the learner is presented with the title, time required, and a brief description of the purpose of the VS. After accessing the VS, learners are provided with a prebrief that follows simulation best practices, including a fiction and confidentiality contract, establishment of psychological safety, learning objectives and competencies, an explanation of how the simulation will work, and scenario background. Depending on the VS, participants may be provided with multimedia resources, knowledge questions, decision points, and didactic moments as they move through a variety of scenes in the scenario. VSs are presented in a variety of ways, such as first-person video; avatar-based content; or a mixture of pictures, videos, and character-based actions. After completing the simulation, the students complete a self-debriefing that includes reflective questions and feedback on performance. Later, an in-person debriefing can occur; how the in-person debriefing is conducted depends on the context and the educator’s choice.

Some students participating in the Virtu-WIL program completed VSs as a part of their curriculum, while others completed it as an optional extracurricular. Implementation and use of the VS followed simulation best practices,¹⁶ which ensured students received a prebriefing before the VS and a debriefing after the VS. Prebriefings were both built into the VS or completed in person, whereas the debriefing style varied from synchronous in-person to asynchronous virtual debriefing. All students were required to play at least 3 VSs, although students were free to complete any individual VS as many times as desired.

Study Design

A cross-sectional mixed-methods analysis was conducted, using quantitative survey data and qualitative open-ended responses and interviews. The survey was cohort based. All students participating in the Virtu-WIL program were asked to complete an exit survey. Completion of the survey questions was voluntary. Participants were from multiple schools located across Canada. Participants were offered compensation for participation in the program and completion of the survey. Data for the MLT students were extracted from the larger sample of health care students for this analysis.

The broader evaluation of the Virtu-WIL program used 2 evaluation frameworks—the Evaluation Framework for the Change Foundation Grants Program¹⁷ and the Kirkpatrick model—focusing on the first 3 levels of evaluation: reaction, learning, and behavior.¹⁸ This study represents a focused secondary analysis of data within the pan-Canadian Virtu-WIL initiative; thus, it deviates from the program evaluation framework, focusing on specific outcomes for MLT participants. The findings for MLT participants were compared with the results of other health programs in the Virtu-WIL program to contextualize and provide a relative comparison for MLT student outcomes. Data were collected throughout the 2023-2024 academic year.

As the project’s goals included evaluation, for quality improvement purposes, the Tri-Council Policy Statement on the Ethical Conduct of Research, which governs research ethics in Canada, determined that survey data for the project were exempt from ethics review. Institutional review and approval for interviews were provided by Centennial College Research Ethics Board (#2022/23–10). Participants were advised of the purpose of the evaluation and that data collected would be kept strictly confidential when completing any surveys or when participating in interviews. Participants were advised that data would only be reported in aggregate and no individuals would be identified when reporting. Informed consent and data collection occurred only after the nature of the procedures was fully explained to participants.

Measures

Student exit survey data were collected using Hosted in Canada,¹⁹ a survey administration tool. The survey comprised custom items and validated tools, including the mini–Player Experience Inventory (PXI)²⁰ and Debriefing Experience Scale (DES).²¹ Custom items were developed through a process of expert review and pilot testing with an initial sample of students. The mini-PXI consists of 11 items divided into 2 subscales. Five items measure players’ perceptions of the functional aspects of gameplay, 5 items measure the psychosocial consequences of gameplay, and a single item measures enjoyment. The mini-PXI scores are reported as a raw total score and subscale scores. The DES measures students’ perceptions of the quality and utility of the debriefing experience and comprises 7 items aggregated as a single global score. All survey items were rated on a 1-to-5 Likert scale. Qualitative data were obtained through open-ended survey questions and interviews with educators and facilitators.

Analysis

Data were analyzed using frequency, descriptive, and inferential statistics. All analyses were conducted in R²² using the psych²³ and dplyr²⁴ packages. All comparative analysis was conducted using analysis of variance. Based on the unbalanced program sample sizes, all inferential statistical analyses were conducted using a randomly stratified equal-n subsample with 30 data points from each program. This approach balances the sample, leading to more accurate and robust estimates of variance, and reduces sensitivity to violations of model assumptions, reducing the influence of large programs.²⁵ A sample size of 30 was selected based on the inclusion of maximal data from the smallest program sample (sonography) while meeting a level appropriate for detecting group differences.²⁵ Based on the central limit theorem, a sample size of 30 is adequate for a sampling distribution whose mean approaches a normal distribution for independent samples.²⁶ A consistency check found no significant differences for each program’s stratified and unstratified scores. Open-ended items were analyzed using a descriptive category coding scheme, which organizes responses into clear, concrete categories for systematic interpretation.²⁷

Learner Satisfaction

Overall satisfaction among MLT students was high, with 87.9% reporting satisfaction or high satisfaction with Virtu-WIL simulations. This was comparable with the overall sample, which rated satisfaction/high satisfaction at 88.3%. MLT students rated psychological safety, inclusivity, and likelihood to recommend the program positively. No significant differences existed across programs on psychological safety, inclusivity, or recommending the program (Table 1).

Skill Development

MLT students rated the Virtu-WIL program’s impact on critical thinking and problem-solving skills as high. Lower, though still high, ratings occurred for time management, teamwork, oral communication, and written communication. The trend of higher scores on critical thinking and problem-solving skills and lower scores on teamwork, oral communication, and written communication was observed in the other 4 programs as well. A significant difference was observed compared with RT students ( = 3.34; SD = 0.97) in written communication skills, with MLT students scoring higher. The same pattern was observed for paramedicine and nursing students compared with RT students (Table 1). It should be noted that VSs were designed to address different learning objectives to varying extents.

Functionality and Engagement

Good reliability was demonstrated for the total mini-PXI (α = .93), and the functional (α = .86) and psychosocial (α = .89) subscales. The total mini-PXI score for MLT students was 40.6/55, which is slightly lower than other professions, though only nearing a significant difference compared with nursing 46.1/55. The functional and psychosocial usability subscales indicated positive perceptions, highlighting ease of use and engagement. No significant differences were observed between MLT and other professions on the subscales (Table 1).

Prebriefing and Debriefing Experience

Prebriefing for the VS was rated highly. Good reliability was demonstrated for the DES (α = .95), and debriefing was positively rated, with no significant differences between MLT and the other 4 programs (Table 1). In open-ended responses, students highlighted the role of the prebriefing and debriefing experience in making meaningful connections for learning and as an essential part of the VS experience (Table 2).

Repeated Use and Impact on Clinical Practice

Four items measured impact on clinical practice: (1) the Virtu-WIL VS scenarios reflected what I see in clinical practice, (2) I learned a lot from the program, (3) I will be able to apply the knowledge and skills in practice, and (4) I now feel more competent and ready for practice. For the 4 questions, MLT mean scores ranged from 4.01 to 4.2 and were comparable with other programs; the 4 other programs’ scores ranged from 3.9 to 4.44. No significant differences existed between MLT and any other programs.

Of the 48 students in the MLT program who had completed VS in multiple terms, most strongly agreed (41.7%) or agreed (45.8%) that repeated use enhanced their learning experience in the program. Similarly, most students strongly agreed (29.2%) or agreed (39.6%) that they had the opportunity to apply knowledge and skills obtained from the VS to clinical practice. Qualitative feedback underscored VS’s value—for example, offering experience with procedures such as phlebotomy and equipment operation. Practicum supervisors echoed the benefits of the VS on student preparation and performance during clinical placements.

Professional Outcomes

MLT students found the Virtu-WIL program helped them to develop a professional network (81.7% strongly agree or agree), gain experience that would help them in the workplace (86.2% strongly agree or agree), and better prepar to find a job in their field (73.1% strongly agree or agree). The frequency of agreement or strong agreement was comparable with the overall sample, wherein agreement or strong agreement on the 3 items was 79.4%, 92.5%, and 73.6%, respectively. Across programs the outcome was comparable as well, wherein agreement or strong agreement ranged from 71.7% to 91.4%, 85.5% to 100%, and 62.5% to 80% for the 3 items, respectively.

Limitations

There are 2 primary limitations to the present study. First, the results are self-reported. Students’ perceptions of their level of development in learning, knowledge, skills and competency, and preparation may be biased, as students may overestimate these abilities in themselves. Second, how debriefing was conducted varied by site—facilitators could choose to conduct debriefing online or in person, vary group size, and use their preferred debriefing model. Guidelines and recommendations were made for debriefing; however, based on considerations of academic freedom, no specific debriefing methods were mandated.

These limitations were partly addressed through interviews conducted with instructors and practicum supervisors. The interviews helped to triangulate findings, identifying consistency in responses between instructors, supervisors, and students. The interviews largely corroborated the findings of the survey and student interviews in the present study and the broader evaluation, providing some validity to the self-reported data.^10,11

Future Directions

To determine the transfer of learning that occurs using a VS, actual behavior and performance during practicum must be examined. Longitudinal studies can assess the impact of VS on practice behavior and objectively assess competency attainment as well as long-term competency outcomes. Longitudinal assessment should also compare practicum outcomes and licensure examination scores of students who use VS and those who do not. Timing of use and repeated use of VS are variables with high potential impact on outcomes that should also be investigated.

With further assessment and evaluation of objective behavior in practicum, VS may present a viable method for replacing certain aspects of clinical practicum or reducing time in practicum. By replacing or reducing parts of practicum with VS, clinical sites can free up time and provide the opportunity to train more students while decreasing the burden on practicum sites and supervisors. Simulation has been previously shown to be a valid replacement for clinical time. Prior work in nursing has shown that up to 50% of clinical time can be replaced with high-quality simulation,²⁹ and multiple other professions have demonstrated that clinical time can be replaced with no detriments to competency development.³⁰

The lower scores in written skills, oral skills, and teamwork highlight opportunities to tailor VS scenarios to the unique communication and collaborative demands of laboratory settings. While, in part, the lower scores in these areas may be an artifact of the medium used, it is possible to enhance these skills through VS development. For example, VSs may be built with a specific focus on developing these skills—for example, VSs with multiple learners in the same virtual space may be used to enhance communication and teamwork skills.

Finally, VS may be used for continuing professional development (CPD) for practicing MLTs. The ease of accessibility provides the opportunity for interactive and experiential CPD, which may be preferable to other forms of CPD. VS may also present a better option for knowledge and skill retention and development than other forms of CPD.