Health knowledge made personal
Join this community!
› Share page: Email Digg del.icio.us Reddit icon StumbleUpon Technorati
Go
Search posts:

Michelle Lin's Twitter Updates

@precordialthump I totally forgot about the EKG library! Awesome stuff. Keep up the great work (when do you sleep?!) 257 days ago
@emeducation Perfect, thanks for the suggestions! Turns out one is co-authored by Jeff Tabas. He already gave them all books. 257 days ago
Recommendations for an EKG resource to review bread & butter cases for senior residents? Pre-graduation panic setting in for our residents. 258 days ago
@danipedia Good point. I use the studies to convince the trauma consults NOT to get c-spine imaging on EVERYONE (citing distracting injury)! 263 days ago
@doctorflash Hi there. Just wanted to drop a note to thank you for all the extra traffic you're sending to blog. Much appreciated!! 273 days ago
 

Article review: Evaluating your written evaluation of a learner

Posted Nov 07 2011 12:00am

As a new faculty, one of the first challenges that I encountered was completing evaluation forms for medical students and residents. In our department, a Daily Evaluation Card (DEC) is to be completed at the end of every shift for each learner. These DEC’s are then collated by the program directors to yield a summative final rotation evaluation.

What I wondered was: how can I best use these DEC's to help learners progress as medical professionals and at the same time provide critical information for the PD’s?

Fortunately, I stumbled upon a 2008 Medical Education paper called “Assessing the quality of supervisors' completed clinical evaluation reports” by Dr. Nancy Dudek (University of Ottawa).  This article was what I was looking for. Although this article was intended to evaluate the quality of the summative evaluation, the principles remain applicable to the DEC's.



The article is summarized below:
  • End-of-rotation evaluations usually consist of a checklist/rating scale and written comments. These forms have questionable reliability and validity.
  • End-of-rotation evaluations remain a valuable resource when trying to assess what a trainees "actually do" versus what they "can do" (eg. on an exam).
  • The study attempted to determine the features of a high-quality evaluation and to develop an instrument to assess its quality.
Methodology:
Using brainstorming and a modified Delphi consensus technique, a focus group developed a Completed Clinical Evaluation Report Rating (CCERR) form. This form was then tested nationally and revised to yield a tool which evaluated 9-items each on a 5-point scale. This CCERR tool was found to be a reliable and valid means to differentiate superior from average from poor end-of-rotation evaluations.    


The 9-item CCERR checklist:
How would your own Daily Evaluation Card evaluations fare? 
Use a 5-point scale (1 = not at all, 3 = acceptable, 5 = exemplary).
  1. Checklist/ numeric ratings show sufficient variability to allow identification of relative strengths and weaknesses of the trainee.
  2. Comments are balanced providing both strengths and areas for improvement.
  3. The trainee’s response to feedback and/or remediation during the rotation is described in the comments.
  4. Comments justify the ratings provided.
  5. Clearly explained examples of strengths using specific descriptions (not generalizations) are provided in the comments.
  6. Clearly explained examples of weaknesses using specific descriptions (not generalizations) are provided in the comments.
  7. Concrete recommendations for the trainee to attain a higher level of performance are provided.
  8. Comments are provided in a supportive manner.
  9. Overall, this end-of-rotation evaluation provides enough detail for an independent reviewer to clearly understand the trainee’s performance on the rotation.
Reference
Dudek NL, Marks MB, Wood TJ, Lee AC. Assessing the quality of supervisors' completed clinical evaluation reports. Med Educ. 2008 Aug;42(8):816-22. 

Post a comment
Write a comment: