Failure to Fail Part 3 – A Prescription for Better Evaluation in the ER

In Education & Quality Improvement by Nadim LalaniLeave a Comment

Introduction:

In the last two posts (post 1, post 2) I introduced the fact that there exists a culture of ‘failure to fail’ in medicine. Faculty themselves have a large part to play and, apart from being too lenient, also unwittingly introduce bias during learner evaluations.In this final post I want to share my approach to trainee evaluation.

I believe that most clinicians already know how to make judgments about learners – they just don’t know that they know! Furthermore if you follow the six steps below I think you’ll start to make more effective judgments and provide more meaningful feedback to learners. Just maybe – you might even identify [and start the process of helping] a failing learner. Comments welcome.

Fact: Most clinicians already possess skills transferable to trainee evaluation

Screen Shot 2014-02-24 at 7.14.21 AM

  • ER docs routinely diagnose and treat conditions in the ER with little information
    • We have the ability to quickly discern sick  from not sick
    • What’s more – (if you think about it) you also possess the right words to describe why  … pale, cyanotic, listless versus  pink, alert, smiling
    • The same skills can be used to identify competence [introduced self, gave clear question to consultant, provided concise history] and incompetence [pertinent negatives missing from history, inability to generate basic ddx, brusque interpersonal interactions]

Screen Shot 2014-02-17 at 3.29.37 PM

  • Many clinicians also possess transferable “Soft Skills”. We use these daily in difficult patient interactions.
  • Guess what? The exact same skills can be brought to bear in difficult trainee evaluations!
    • listening empathetically
    • breaking bad news
    • dealing with emotions
    • dealing with hostilityFailure-Success

My 6-step Prescription for Evaluation Success:

I have tried to illustrate above that the same skills that make you an astute clinician will also allow you to discern whether a learner may be under-performing and then let them know. Why not stick to what works? Below is my six-step prescription for evaluation success. I am hoping that it is pleasantly familiar to you.

STEP 1: Take a history from the learner at the beginning of the shift.

The ED STAT course run by CAEP [link] teaches Faculty to understand where the learner is coming from at the start of the shift.

  • Ask the learner about:
    • Level of training
    • EM experience
    • Home program
  • Provide an orientation to the ER
  • Ascertain their learning goals
  • Set expectations
    • based on level of training
    • that they will be receiving feedback about their performance
    • [in fact – tell them about these six steps]

STEP 2: Examine the learner’s skills

Use the IPPA approach!

  • Inspection: watch the learner
    • taking a history
    • performing physical exam manoeuvres
    • giving the diagnosis/ddx
    • giving discharge instructions
  • Palpation: perform physical exams together to

    • confirm or refute findings
    • correct errors
    • role model
    • learn from them
      • learners often remember all the special manoeuvres better
      • they ask questions that challenge you to read up – like “which cranial nerve is glossopharyngeal?”
  • Percussion:  use your finely tuned senses to ‘tap out’ gaps in:
    • knowledge
    • clinical skills
    • attitude
    • [Hint: the mnemonic KSALTs can help identify where the problem lies with the learner. Read this [link] to the ALIEM MEdIC case].
  • Auscultation: listen to the learner
    • during consultations
    • during interactions with other providers
    • [HINT: eavesdrop on their histories from behind the curtain]

STEP 3: Diagnose the Learner:

Screen Shot 2014-02-25 at 4.52.23 PM

  • Is the learner competent or not?
  • Think about biases you may be introducing and try to avoid them – especially central tendency bias [link].
  • Use my ABC RIMES approach:
    • Attitude – make comments like “self-directed” vs “unmotivated”
    • Behaviours – “procedural skills above peers” vs “poor attention to workplace safety”
    • Competencies: [I use a modification of the RIME approach [link] with the addition of an ‘S’]
        • Reporting – give feedback on history-taking, written communication and presentations
        • Interpretation – give feedback on interpretation of Xrays, EKG’s and ‘putting information together’
        • Managerial skills – give feedback on efficiency, multi-tasking, stewardship
        • Educator/Expertise – give feedback on knowledge and teaching skills
        • Soft skills – give feedback on communication, team-skills, honesty, reliability, insight, receptivity to feedback
  • For those that are slavish to the CanMEDs brand try the MS CAMP mnemonic:
    • Medical expert
    • Scholar
    • Communicator/ Collaborator
    • Advocate
    • Manager
    • Professional

STEP 4: Provide your diagnosis

  • Think of your favourite uncle or granddad – would you be happy with the care that they received? If so  – great!
  • Provide feedback using the ABC RIMES framework [here’s an earlier post on providing feedback].
  • If the feedback is going to be negative – you need to prepare. Here’s more information on getting ready for a difficult conversation [link].
    • If you’ve primed the learner at the beginning of the shift – they shouldn’t be surprised.
    • Have the courage to practise tough love – trainees are adults, they should be expected to receive feedback like adults.

STEP 5: Document! Document! Document!

Screen Shot 2014-02-25 at 4.57.57 PM

  • ALL verbal feedback needs to be written down.
  • Use strong descriptive terms rather than weak ones. Saying ‘good job’ is good for self-confidence, but learners need specifics.
  • Use the ABC RIMES or MS CAMP framework – effective evaluation forms make this task easier.
  • RANK THE LEARNER. They cannot all be ‘above average’ – some trainees are below average.
  • Even though a learner may be having a bad day this needs to be documented so that patterns can emerge.

STEP 6: Consult them out!

Most of us feel that problem learners are “someone else’s problem” – you’re right! You need to consult them out. But like all consults – you need to give the consultant the right information [see above].

  • Remediation is NOT your responsibility
  • Issues need to be referred to:
    • Rotation coordinator
    • Program director

Summary:

Most clinicians already possess the skills needed to made judgements about failing learners. Furthermore, they also possess the soft skills to give negative face-to-face evaluation. If all clinicians used these skills [together with direct observation and expectation-setting] we may identify more learners that need help. Ultimately we will achieve more success at graduating cadres 100% of whom are competent, caring physicians.

Future Directions:

There needs to be huge emphasis on culture change. This is going to require:

  1. More faculty development and engagement. BOTH faculty and programs need to come together to do this. [I’d appreciate comments on how to bring faculty to the table because at my institution this is problematic]
  2. Learners themselves understand how to receive and act on feedback. 

Note: This post was originally posted on ERMentor. This post was copyedited by Stephanie Zhou (@stephanieyzhou) and Sean Nugent (@sfnugent)

References:

1.
Hauer K, Holmboe E, Kogan J. Twelve tips for implementing tools for direct observation of medical trainees’ clinical skills during patient encounters. Med Teach. 2011;33(1):27-33. [PubMed]
2.
Kogan J, Hess B, Conforti L, Holmboe E. What drives faculty ratings of residents’ clinical skills? The impact of faculty’s own clinical skills. Acad Med. 2010;85(10 Suppl):S25-8. [PubMed]
3.
Kogan J, Conforti L, Bernabeo E, Durning S, Hauer K, Holmboe E. Faculty staff perceptions of feedback to residents after direct observation of clinical skills. Med Educ. 2012;46(2):201-215. [PubMed]

Nadim Lalani

Nadim is an emergency physician at the South Health Campus in Calgary, Alberta. He is passionate about online learning and recently made a transition into human performance coaching. He is currently working on introducing the coaching model into medical education.