FEI logo FINAL

CAEP FEI | Daily Faculty Evaluations

In Featured Education Innovations (FEI) by Alim PardhanLeave a Comment

Dr. Rosenberg prides herself on the effort she places into teaching students on each shift, but she is not sure whether her teaching methods are effective. At the end of the academic year, she reviews her evaluations from students and is disappointed when she finds only a handful of generic evaluations, with the majority of her students not completing the form. She approaches a few of her learners who say the existing preceptor evaluation form is clunky and difficult to complete. Does a simpler evaluation form help faculty receive more feedback from students?

Effective, timely feedback is a cornerstone for ongoing improvement, for both students and faculty. Furthermore, for faculty, learner evaluations can sometimes be tied to academic promotion. One barrier for many students to complete evaluation forms is a lack of time, as both medical students and residents have many competing interests. Streamlining this process may help increase uptake. This Feature Educational Innovation (FEI) was originally posted by the CAEP EWG FEI Team on September 19, 2014. A PDF version is available here.

Name of Innovation:
Daily Faculty Evaluations

Lead Innovator:
Dr. Alim Pardhan ([email protected])

Division Website:
http://fhs.mcmaster.ca/emergmed/

Description of the Innovation

Overview

There is an existing paradigm of learner evaluation in the emergency department, but this has not been consistently paralleled with faculty evaluation.  An online tool was developed that allowed for fast, convenient, anonymous and immediate evaluation of faculty after every shift.  This web-based form is accessible via smartphone or computers, and can be easily tabulated.  This has markedly increased uptake of both evaluations, especially that of faculty.

Project Goals

The goal of this project was:

  • To ensure that there are good quality daily evaluation forms of both learners as well as for faculty working in the emergency department (ED)

Background

Previously faculty had a limited number of evaluations per year. Evaluations were done on a block-to-block basis with the residents completing a single evaluation form for their preceptor. The number of evaluations received was quite variable depending on Senior Preceptor Selection, number of shifts worked, number of times assigned a resident, etc. The intent of this innovation was to increase the number of faculty evaluations received from learners in the ED. In 2013-2014 using the existing system the mean number of evaluations = 6.2  (SD = 5.3) per faculty member.  19% (12/63) did not receive a single evaluation over the course of the year and many did not receive enough evaluations to be able to receive proper feedback (3+ to ensure resident confidentiality).

Methods

A team of faculty developed a list of questions that encompassed CanMEDS Roles specific to Emergency Medicine. This was placed on a 5-point scale to allow for daily evaluation of residents.

The University Senate at McMaster has a standard evaluation form that is used for all clinical teachers that was adapted for the purposes of this project.

The evaluation questions were uploaded onto Medportal, an in-house portal based on a Google platform. An online form-based application was created and then linked to Quick Response Codes. QR Codes enable residents to scan the code to take them directly to the Resident Evaluation form, which is then linked to the faculty evaluation form. These forms are separate forms and cannot allow the resident feedback to be traced back from the faculty evaluation form.

Results

In the blocks that we ran the Resident Evaluation System, on average residents received daily evaluations from ~70% of their shifts which is a substantial increase compared to previous end-of-rotation methods.

For 2013-2014 within the Divisions of Emergency Medicine, the Daily Faculty evaluation process started earlier and resulted in ~ 230 daily evaluations for faculty in just over 8 months, 92% of faculty received at least one evaluation ~50% had 3 or more, allowing for proper feedback to be provided to these faculty members. At the time of this submission, the number of Pediatric emergency medicine evaluations were not available.

Reflection on the Process

This system has worked quite well for collecting additional data, not only about the daily performance of residents but also the daily evaluation of faculty.

Our biggest challenge still remains having the resident’s initiate the process. However this has improved significantly since the start of the new academic year. Linking the Resident and Faculty forms was a new change which improved the uptake and numbers for 2014-2015, suggesting that uptake this year will be significantly better than the initial run last year.

What are the strategies you have employed to encourage evaluation form completion for faculty? Are there examples in which increased evaluations have led to new insights on faculty performance? 

[bg_faq_start]

More About the CAEP FEI

This post was originally authored for the Canadian Association of Emergency Physicians (CAEP) Feature Educational Innovations project sponsored by the CAEP Academic Section’s Education Working Group and edited by Drs. Teresa Chan and Julien Poitras. CAEP members receive FEI each month in the CAEP Communiqué. CanadiEM will be reposting some of these summaries, along with a case/contextualizing concept to highlight some recent medical education literature that is relevant to our nation’s teachers.[bg_faq_end]

Alim Pardhan

Alim Pardhan

Alim Pardhan is an emergency physician in Hamilton, ON. He is an Assistant Professor and Program Director of the Royal College Emergency Medicine program at McMaster University.
April Kam

April Kam

April is an Assistant Professor in the Division of Pediatric Emergency Medicine in the Department of Pediatrics at McMaster Children’s Hospital and the McMaster Evaluations Portfolio Lead for the Pediatric Postgraduate Education Committee.
April Kam

Latest posts by April Kam (see all)

Teresa Chan

Senior Editor at CanadiEM
Emergency Physician. Medical Educator. #FOAMed Supporter, Producer and Researcher. Chief Strategy Officer of CanadiEM. Associate Professor, Division of Emergency Medicine, Department of Medicine, McMaster University.

Daniel Ting

Daniel Ting is an Emergency Physician and Clinical Assistant Professor at the University of British Columbia, based in Vancouver. He is the Editor-in-Chief of CanadiEM and a Decision Editor at the Canadian Journal of Emergency Medicine. He completed the CanadiEM Digital Scholarship Fellowship in 2017-18. No conflicts of interest (COI).