HiQuiPs: How do we evaluate virtual care?

You are a resident coming in to see a patient who has come into the ED complaining of a new, sudden-onset, 9/10 pain headache. You run a CT scan and rule out any hemorrhage or stroke, referring them to the Stroke Prevention Clinic for a consultation. The patient asks if they can receive a virtual appointment for the future, as they live quite far away and would much prefer it.

You wonder if the virtual care provided would be appropriate for the type of care the patient needs. This leads you to think: how would one even determine what quality virtual visits mean to patients?

Introduction

The healthcare system has found a need to shift to virtual care to adapt to the risks introduced by the 2019 novel coronavirus (COVID-19) pandemic to maintain patients’ health needs. In Ontario, Canada, 70% of all ambulatory visits were conducted virtually – over telephone or video – during the first three months of the pandemic.¹ Although it is unclear how long this modality of ‘virtual visits’ will stay, this care model is increasing in popularity due to its many benefits to patients.² For example, virtual visits can improve accessibility of care for patients from rural or more remote areas, as well as those located in areas with healthcare provider shortages. In-person appointments can pose a barrier for some patients due to the high costs of travel and parking, and considerations such as time off work or the need to find childcare.³ Moreover, increased convenience, efficiency, and reduced travel and wait times can all contribute to higher patient satisfaction.² For example, Hamilton Health Sciences in Hamilton, Canada has attempted to quantify the travel saved by virtual appointments, estimating that they have saved patients and families 1.5 million kilometers of travel on average, per month. Although some patients prefer in-person appointments, virtual care is becoming an increasingly common option for certain types of encounters, such as those reviewing non-remarkable lab or imaging results.³ Regardless of the modality offered, the ultimate goal of healthcare providers should be on offering the highest quality, patient-centred care. An integrated approach where providers partner with the patient to utilize the most appropriate option for each encounter is essential.

A quick Google search revealed a myriad of methods evaluating virtual care, but no standardized approach exists.⁴ A tool to evaluate virtual care offerings may be beneficial, as the quality of care can differ significantly amongst healthcare providers, clinics, and institutions.⁴ Recently, through interviews with patients, Hamilton Health Sciences’ Virtual Care Operations Team (VCOT) has been working to drive continuous quality improvement across its sites by updating the virtual care evaluation tools to ‘measure what matters.’ This has included looking at the six dimensions of quality, which ensures that virtual care projects are evaluated from a holistic lens.⁵ The Institute of Medicine (IoM) established these six domains of healthcare quality, which was adapted by Health Quality Ontario to create the Quality Framework.⁶ The six domains include: safe, effective, patient-centered, timely, efficient, and equitable care.⁶ This framework provides guidance for creating indicators, benchmarking and setting targets to ensure that high-quality care is being provided.

Patient Engagement

To understand what quality virtual care means to patients, HHS collaborated with three cohorts of patient advisors to ask the following question: what does virtual care mean to you in relation to the six domains of healthcare quality? The groups that participated include the 1) Family Advisory Council 2) Youth Advisory Council and 3) Patient Advisors (corporate patient partners who contribute to a variety of projects), totalling 16 people. These conversations, despite their differing ages and contexts, demonstrate alignment between all three patient advisor populations.

A summary of the themes from these conversations can be found below:

Quality dimension	Thematic response from HHS patient advisors
Safe	• Provide care using solutions which are secure and safe from a data privacy and security standpoint • Understanding of a level of trust in the provider and the healthcare organization that the due diligence to uphold and protect patient privacy is maintained, as it is for in-person care (i.e., private office space, no one can overhear our conversation)
Effective	• Virtual care as an option for patients only when clinically appropriate: “Don’t give me a virtual appointment when I really need to come in anyway” (personal communication, Patient Advisors focus group) • Beneficial to know what the next steps are after a virtual visit • Expect the same outcomes as in-person care, not an avenue for suboptimal care
Patient-centered	• Virtual care allows for flexibility in scheduling appointments: “It’s easier for me to attend when I don’t have to leave work or school” (personal communication, Youth Advisory Council) • Preferred when the provider actively listens, regardless of visit type, for care
Efficient	• Avoids lengthy non-value added appointments and ensures appointments accomplish goals related to patient care • Anecdotal experience of less waiting for a virtual appointment to start vs. at a clinic in the waiting room • Looking forward to not having to repeat medical history with the new hospital-wide electronic medical record implementation, which will have all patient records in one spot
Timely	• Appreciate knowing when knowledge about care will be shared (i.e., time range when test results will be available to manage associated stressors while awaiting results) • Expect no difference with timely care for in-person or virtual
Equitable	• Be mindful that devices/WiFi is not accessible to all patients for virtual care • In-person options should continue to be offered

The resident listens to the patient, hearing their wishes for a virtual consultation. Because they live far away and have to take time off work to come in-person, it is much easier for them to meet via video virtual visit. The resident also confirms that they have a telephone and/or smart phone/video device with a reliable internet connection to participate in the visit. Although certain types of encounters may be more beneficial in-person, the resident reflects on the six dimensions of quality, and the clinical requirements of the consultation. As healthcare providers, our role is to equip patients with the proper tools and information, and partner with them as to the best way forward in their care.

Conclusion

Virtual care, while here to stay, is not a ‘one size fits all’ solution. When we think about evaluating virtual care, it’s often a multi-pronged approach, which includes understanding perspectives of both the patients and providers, as well as the healthcare organization. There is also a patient population that could be lost to follow-up if the only option is in-person. Further, evaluation should be ongoing so that continuous improvements to patient care can be made to ensure we are providing optimal patient care. These standards should be upheld, regardless of the type of visit – virtual or in-person.

So, we ask you, as a provider: what does it mean to you in regards to safety, effectiveness, patient-centredness, efficiency, timeliness, and equity when it comes to virtual care? What else would you consider when deciding if an appointment should be virtual or in-person?

Senior Editors: Dr. Darius Bägli and Dr. Ahmed Taher

This post was copyedited by Sydney Terry and Tiffany Tse

1.
Stamenova V, Chu C, Pang A, et al. Virtual care use during the COVID-19 pandemic and its impact on healthcare utilization in patients with chronic disease: A population-based repeated cross-sectional study. PloS one. 2022;17(4). doi:e0267218
2.
Gajarawala S, Pelkowski J. Telehealth Benefits and Barriers. J Nurse Pract. 2021;17(2):218-221. doi:10.1016/j.nurpra.2020.09.013
3.
Reed ME, Huang J, Graetz I, et al. Patient Characteristics Associated With Choosing a Telemedicine Visit vs Office Visit With the Same Primary Care Clinicians. JAMA Netw Open. 2020;3(4). doi:10.1001/jamanetworkopen.2020.5873
4.
Sartori D, Lakdawala V, Levitt H, Sherwin J, Testa P, Zabar S. Standardizing Quality of Virtual Urgent Care: Using Standardized Patients in a Unique Experiential Onboarding Program. MedEdPORTAL. 2022;18:11244. doi:10.15766/mep_2374-8265.11244
5.
Institute of Medicine – Six domains of health care quality [1] Institute of Medicine (IOM) . Crossing the Quality Chasm: A New Health System for the 21st Century. National Academies Press; 2001. doi:10.17226/10027
6.
Healthy Quality Ontario . An Overview of Quality Matters: Realizing Excellent Care for All. Ontario: Health Quality Ontario; 2017:9.

Reviewing with the staff

Dr. Bägli is the Associate Head of Urology at The Hospital for Sick Children (SickKids) as well as a world expert in interdisciplinary paediatric urology and translational science. He has intersecting interests and experience in health informatics, innovation, and leading transformational organizational change.

Dr. Darius Bägli

@DBagliMD

Bio
Latest Posts

Tiffany Tse

Tiffany is currently a first year medical student at the University of Toronto. She has an interest in quality improvement, public health, and increasing accessibility to care. In her free time, she enjoys dancing, playing music, and spending time with friends!