A few months back Dr Eddy Lang (@EddyLang1) [Co-editor of the Royal College Research Guide] graced us with his kind and friendly personality and dropped some pearls on retrospective chart reviews.
Medical Record Review [MRR] Research in General aka Chart Reviews
“Chart reviews don’t get the respect they may deserve” Dr Lang
Dr Lang lamented the fact that MRR doesn’t get the street cred it deserves. This is large part because of a historical pattern of:
- Wrong questions
- Poor methods
- Action/Documentation Divide = what happened vs what was documented
- Missing Data
- Case identification
Gilbert et al., in their article “Chart Reviews in Emergency Medicine: Where are the Methods?” showed that 25% of EM publications between 1988-1995 relied on chart reviews1. However, although inclusion criteria were present 98% of the time, important data regarding methodology was generally absent:
- Abstractor training, 18% (95% CI, 13% to 23%)
- Standardized abstraction forms, 11% (95% CI, 7% to 15%)
- Periodic abstractor monitoring, 4% (95% CI, 2% to 7%)
- Abstractor blinding to study hypotheses, 3% (95% CI, 1% to 6%)
- Inter-rater reliability was mentioned in 5% (95% Cl, 3% to 9%) and tested statistically in 0.4%
In their article – Gilbert et al. lay out their solutions1.
The 7 Key Ingredients of good MRR:
1. Abstractor Training: Need to convince the reader that the people pulling the charts
- Describe the Qualifications and Training procedure for the data abstractors
- Before the study begins pull some Trial charts to Test the data abstraction process
2. Case Selection: Needs to be explicit and well described
- Administrative codes is a start but has flaws
- Often this can lead to a substudy [i.e do the ultimate codes reflect the Dx?]
- Clear inclusion/exclusion criteria
- Screening procedures must be solid
3. Definition of the variables: Need to be done well
- Dictionary – define things e.g. vitals signs … at triage? by the Emergency Physician? on reassessment?
- Timing and Source of the info needs to be described
- Adjudication – how are you going to categorize contradictions and inconsistencies?
4. Data Abstraction Tool: Make it good
- Need to have a standardized data abstraction tool – use your research staff here
- Need to have a uniform process of handling missing data – need to think about what to do with missing or unclear data
- Consider using software to manage data [e.g. Using Redcap Software]
5. Blinding:
- Are the abstractors unaware of the study hypothesis? – consider quizzing them afterwards to see!
6. Quality Control
- Regular meetings to ensure standard process
- Need to monitor the abstractors work – consider audits
- Resolution of conflicting assessments
7. Inter-rater reliability: Report inter-rater reliability – it’s eKspected …get it?
- Reported on a sample of charts reviewed by another [blinded] reviewer
Criteria to Follow for Chart Reviews
The next paper Dr. Lang discussed was a landmark article by Jansen et al who developed a guideline on how to conduct chart reviews.2 They are outlined below for your perusal.
Dr. Lang also outlined some questions that chart reviews can help to answer:
- How are we doing? [care practices, quality of care e.g. Look at time to analgesia after intro of new acute pain protocol for say… renal colic]
- What does this condition look like? [e.g looking for key word search “Rugby” … pull charts associated with rugby injuries]
- Derivations Models [e.g. Risk Factors for Hospitalization after Dog Bite injury5]
He also provided some comments on ethics in this type of research, noting that a full ethics process is often not necessary for QI initiatives which can often be approved after “expedited review.” The ARECCI Ethics Screening Tool can guide potential researchers as to what type of review is ethically required for the work that they plan on doing.
Instructive Examples – Chart Reviews CAN change practice!
Dr Lang finished by giving us some examples of chart reviews that changed practice.
- In his 1995 publication on the prognostic value of amylase in the evaluation of the abdominal pain patient3 he pulled lab results, reviewed charts, and showed that there was no difference in the outcomes between patients with intemediate levels of amylase and normal patients. This impacted the way that his group treated patients with these results at the local level.
- The Canadian Adverse Events study was a chart review that had an impact on global practice.4 A retrospective review of 3500 charts from 5 provinces in Canada, it showed an adverse event rate of 7.5% of hospitalizations and shone a powerful spotlight on issues of patient safety.
Note: This post was originally published on the ERMentor Blog on November 14, 2014. It was revised by Rob Carey (@_RobCarey), Sean Nugent (@sfnugent), and Brent Thoma (@Brent_Thoma) before being reposted on CanadiEM in August of 2016.
References
Latest posts by Nadim Lalani (see all)
- CAEP FEI | Advanced Performance in Medicine™: 12 Conversations - April 6, 2018
- Medical Coaching: Don Miguel Ruiz’s “Four Agreements” - August 24, 2016
- CaRMS Rodeo – How to prepare for a crazy ride - August 11, 2016