#TipsForNewDocs | Using spaced repetition to study

The Canadian Royal College licensing exam in emergency medicine has a certain mystique. While the pass rate for Canadian graduates is spectacular [1], exam survivors speak of the test in hushed whispers that have an air of PTSD. Having recently survived the exam myself, I have some understanding of where these whispers come from. Prior to my exam year I had planned to write a blog post to reassure the residents following me that it was not that bad… Except that it was.

Or at least it would have been if I had not prepared for it like it was a battle to the death with the zombie version of Gregor Clegane on bath salts. We are sworn to secrecy regarding the content of the exam so I can not provide any specifics, but suffice to say, no matter how much you prepare you are going to take heavy fire. Any illusion that being a good, competent emergency medicine resident will adequately prepare you should be strongly rejected. The content of the exam is too broad to cram for and too deep to drift through on the skills that you’ve developed as a resident. Perhaps when the Royal College embraces the tenets of competency-based medical education in its high-stakes assessment (which is the plan [2]) this will change, but my advice to future candidates would be to hunker down and memorize everything – especially those things that seem too obscure to be on the exam.

How in the world are you going to do that?

With spaced repetition.

What is spaced repetition?

Spaced repetition is the forced retrieval of previously learned information at increasingly lengthening intervals. This is one of the truly evidence-based ways to enhance your studying. By encouraging active recall of information previously committed to memory, spaced repetition can be used to remember the medical esoterica loved by Royal College exam writers. By lengthening retrieval intervals there is time to add new information into the process while locking old information in the vault. Combining spaced repetition with various other memory devices (consider mnemonics such as rhymes, acronyms, chunking, imagery, etc) increases its efficacy even further. Don’t just take my word for it – there is a ton of literature on spaced repetition that demonstrates its value [3-9].

How did you use spaced repetition during residency? To prepare for the exam?

Unfortunately, I was a late convert to the benefits of spaced repetition and did not incorporate it into my studies until my exam year.

In my final year I worked through a study schedule and stopped every time I hit a word, concept, or list that I didn’t know well (I stopped a lot). That item was transcribed to a flashcard program online (I used cram.com until it quit supporting my flashcard program; I then switched to quizlet.com) that was then synced with the flashcard program on my phone. I used the spaced-repetition flashcard app Flashcards Deluxe on Android (also available on iDevices!) to review my cards and track my responses [Note: I have no conflicts of interest related to Cram, Quizlet, Flashcard Deluxe, or any other programs mentioned in this post]. Each time I reviewed a card I indicated whether I knew it well (swipe up!), okay (swipe left), or not at all (swipe down).

I thought of my flashcard deck like an ectopic brain full of facts that I wanted to transplant into my head. Over time, entering a bit of knowledge to my deck became a commitment to memorizing it because I knew the app would bug me to review it until I did. The review settings on Flashcards Deluxe (and other spaced repetition flashcard programs) can be customized to your needs but I used the default settings. Every new card needed to be swiped up once or left twice before it was be added to the main deck. It recorded the results for each card in detail and used that information to determine when the it would next be due for review using interval multipliers.

As quizzing myself with flashcards is not the most entertaining activity I made it a rule to:

get every card I made into my main deck by the end of each week, and
review every card that was due before the end of each day.

While I did break these rules with some frequency, sticking to them most of the time prevented me from getting too far behind. Using this strategy in my exam year required me to review flashcards for 1-3 hours per week initially before ballooning to 20+ hours per week in the month before the written exam.

The importance of active learning

Flashcards and spaced repetition may not be for everyone. It is also not the only strategy you should use. However, when you are considering how else to study throughout residency in general and for your exams in particular, I would urge you strongly to make sure that you select active methods.

The educational literature has proven time and time again that active learning (instructional methods where the thinking is done predominantly by the learner; e.g. quizzing, interacting with colleagues/material, group discussions, practice exam questions) is vastly superior to any and all forms of passive learning (instructional methods where the thinking is being done predominantly by the teacher; e.g. listening to a podcast, watching a video, reading a textbook chapter, attending a lecture) [10-14]. Hopefully you will have a cohort of colleagues that will be willing to contribute to these other active methods of learning both throughout residency and in preparation for your exam.

As one study put it, active learning is like broccoli: it is good for you but nobody likes it [14].

The challenge of active learning is that it is a lot more work. Providing correct answers to all of the ridiculous flashcards you make is way more cognitively demanding than watching a nice lecture. However, the beauty of flashcards is that they can be used for active learning independently and on your own schedule.

If you did residency over again, what would you do differently?

As I alluded to above, I would start building (and using) my flashcard deck a lot earlier. Every conference, teaching session, and textbook chapter are full of pearls that can be deposited into your ectopic brain. With regular review, those facts will become as attached to you as Iron Man is to his arc reactor. By the time you’re in final year of residency you’ll have so much information memorized that the Royal College exam will be much easier to prepare for.

Downloading my flashcard deck and using Flashcards Deluxe

You are welcome to check out my flashcard deck or use it to prepare for your exam (noting, of course, that making your own flashcards provides a substantial amount of their value). It can be downloaded directly in the Flashcards Deluxe app by going to the ‘Add deck’ page, clicking the ‘Shared Library’ button (it has the Flashcards Deluxe logo next to it), searching ‘BoringEM,’ and adding the ‘Tiny Tips – BoringEM’ deck with 932 cards. If you’d like to use my cards in other programs the app can export it to Quizlet, Cram, Dropbox, and Google Drive from there

There are several ways to create cards for Flashcards Deluxe and related flashcard programs (use whatever program you prefer, but I would highly recommend one that incorporates spaced repetition to make your study sessions more efficient). I recommend building your decks using the very easy to use Quizlet website and syncing them to your spaced repetition flashcard program.

If you use Flashcards Deluxe it can be a bit difficult to figure out how to turn on and edit the spaced repetition functions. To do so you need to open your deck, click the gear icon in the top right hand corner, click ‘card order,’ and turn on ‘spaced repetition’ (if it isn’t on by default). On this page you can also click ‘Spaced repetition settings’ to customize the repetition settings.

Teresa Chan (Managing Editor of BoringEM, and Royal College exam survivor of 2013) would like me to tell you that while I am suggesting the use of electronic flashcards, the education evidence would suggest writing things out repetitively by hand is probably even more helpful. That said, she learned her lesson after getting some level of carpal tunnel syndrome whilst writing out 1400 flash cards in her PGY5 year. Personally, I started to write out the answers to my flashcards in the last month before my exam and do feel that it was beneficial.

Conclusion

Residency and our licensing exams are tough. Active learning is where it’s at both for learning throughout residency and preparing for exams. I would recommend flashcards with a spaced repetition function enabled to embrace principles of active learning during independent study. Best of luck with your exams!

Sincerely,

Brent Thoma, MA, MD…. FRCPC!!

References

Royal College of Physicians and Surgeons of Canada. Pass Rate Percentages. Retrieved from http://www.royalcollege.ca/ on July 1st, 2015.
Royal College of Physicians and Surgeons of Canada. Frequently Asked Questions: Competency-Based Medical Education and Competence by Design. Retrieved from http://www.royalcollege.ca/ on July 1st, 2015
Schmidmaier, R., Ebersbach, R., Schiller, M., Hege, I., Holzer, M., & Fischer, M. R. (2011). Using electronic flashcards to promote learning in medical students: Retesting versus restudying. Medical education, 45(11), 1101-1110.
Caple, C. (1996). “The Effects of Spaced Practice and Spaced Review on Recall and Retention Using Computer Assisted Instruction”. Dissertation for the degree of Doctor of Education, North Carolina State University.
Kerfoot, B. P., Baker, H. E., Koch, M. O., Connelly, D., Joseph, D. B., & Ritchey, M. L. (2007). Randomized, controlled trial of spaced education to urology residents in the United States and Canada. The Journal of Urology,177(4), 1481-1487.
Kerfoot, B. P., Fu, Y., Baker, H., Connelly, D., Ritchey, M. L., & Genega, E. M. (2010). Online spaced education generates transfer and improves long-term retention of diagnostic skills: a randomized controlled trial. Journal of the American College of Surgeons, 211(3), 331-337.
Kerfoot, B. P., & Brotschi, E. (2009). Online spaced education to teach urology to medical students: a multi-institutional randomized trial. The American Journal of Surgery, 197(1), 89-95.
Kerfoot, B. P. (2009). Learning benefits of on-line spaced education persist for 2 years. The Journal of urology, 181(6), 2671-2673.
Bryson, D. (2012). Using Flashcards to Support Your Learning. Journal of visual communication in medicine, 35(1), 25-29.
Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93, 223-232.
Schackow, T. E., Chavez, M., Loya, L., & Friedman, M. (2004). Audience response system: effect on learning in family medicine residents. FAMILY MEDICINE-KANSAS CITY-, 36(7), 496-504.
Minhas, P. S., Ghosh, A., & Swanzy, L. (2012). The effects of passive and active learning on student preference and performance in an undergraduate basic science course. Anatomical sciences education, 5(4), 200-207.
Tan, N. C., Kandiah, N., Chan, Y. H., Umapathi, T., Lee, S. H., & Tan, K. (2011). A controlled study of team-based learning for undergraduate clinical neurology education. BMC medical education, 11(1), 91.
Smith, C. V., & Cardaciotto, L. (2012). Is active learning like broccoli? Student perceptions of active learning in large lecture classes. Journal of the Scholarship of Teaching and Learning, 11(1), 53-61.