Gail Sullivan, Deborah Simpson, Thomas Cooney, and Eugene Beresin. A Milestone in the Milestones Movement: the JGME Milestones Supplement. JGME 2013 5:1s1, 1-4
This article is a comprehensive review of the GME movement towards defining behavioral and developmental milestones. It describes the historic, political, and economic factors underlying this movement and gives an overview of challenges, potential implications and remaining unsolved questions around the proposed milestones. In this era of outcomes based physician training, this represents a seminal article for educators, educational administrators, and clinicians who are involved in the assessment of residents.
Background on the Milestones Movement
Both the U.S. health care system as well as the graduate medical education (GME) system are in a state of transformation as a result of landmark studies such as the 2001 Institute of Medicine Report: Crossing the Quality Chasm which demonstrated significant discrepancies in the quality of medical care delivered to the American public. This has led to calls for major reforms aimed at improving quality and safety of healthcare in the United States. Similar reforms in GME were pioneered by the Accreditation Council for Graduate Medical Education (ACGME) through the Outcomes Project; shifting the focus of educational programs from a process oriented approach to rigorous assessment of trainee performance as a means of ensuring safe and high quality practices. This has led to an increased administrative burden on programs in implementing these competency-centered, outcomes-based standards while in the context of ever increasing financial burdens on institutions as a result of ongoing health reforms. As part of this paradigm shift in the assessment of residents, there has been an increasing effort to describe the characteristic progression of resident competence through the concept of educational milestones and entrustable professional activities (EPA’s). An important assumption of competency based training is that residents acquire increasing proficiency along competency-specific continuums, towards the acquisition characteristics/competencies of independent practitioners. Milestones guide assessors in delineating where a resident falls in this continuum towards clinical independence by identifying the specified benchmarks of competency and timelines expected towards their attainment.
The Development of Milestones
Internal Medicine was one of the first specialties to develop milestones, which described behavioral milestones attached to specified time periods when the majority of residents would be expected to demonstrate that milestone. These milestones have been in use by tens of residency programs over the past 3 years. Data from these programs has demonstrated that the milestones are logical and represented “realistic progression of resident knowledge, attitudes, and skills”. However, an important drawback identified by educators was that using the milestones took significant time and that assignment specific residents to specific competency levels (when using composite milestones) was difficult.
It is important to note that as of July 2013, seven specialties have been identified to undergo the ACGME’s Next Accreditation System (NAS) which requires reporting of resident performance every 6 months. These NAS “phase I” specialties have undertaken the initiative to develop specialty specific milestones including Internal Medicine, Emergency Medicine, Orthopedic Surgery, Neurological Surgery, Diagnostic Radiology, Pediatrics, and Urology. This process has consisted of specialties working independently and has resulted in milestones that vary in specificity of content, numbers of milestones, and time assignment. For example, the milestones developed by Surgical specialties are highly specific and separate procedural competencies from behavioral competencies whereas Internal Medicine and Pediatrics milestones are utilize general descriptions presented in narrative format. Accordingly, surgical milestones will likely result in minimal assessor ambiguity (ie. assessors will have a good understanding of what constitutes successful performance at each level) but the increased level of detail for each competency can place excessive burden on programs. Conversely, Pediatrics or Internal Medicine milestones will likely lead to increased ambiguity of meaning requiring ongoing faculty training, interpretive consensus development, and rigorous studies of validity evidence. Notably, milestones for such as interpersonal skills and communication, professionalism, systems-based knowledge, and practice-based learning and improvement are generally not specialty specific and have generic milestones (though some exceptions do exist).
Questions and Challenges Ahead
Upon review of the phase I specialties milestones, several important questions have been posed including whether core competencies (such as professionalism) should employ generic milestones, whether milestones are in alignment with the changing health system needs and physician roles (e.g. increasing team based care models), whether milestone assessments demonstrate acceptable measures of accuracy and inter-rater reliability (validity evidence), and how programs will tackle issues of faculty development around the milestones. By extension, much study will need to be carried out to determine whether NAS mandated reporting intervals will facilitate assessment and demonstrate enhanced validity of assessment. Perhaps most importantly, the authors conclude that the potential academic utility afforded by this form of educational outcomes measurement must also demonstrate practical feasibility, cost effectiveness, and a positive contribution to health systems needs and challenges.
For further reading on these topics consider:
- Lurie SJ, Mooney CJ, Lyness JM. Measurement of the General Competencies of the Accreditation Council for Graduate Medical Education: A Systematic Review. Acad Med. 2009;84:301–309.
- Nasca TJ, Philibert I, Brigham T, Flynn TC. The next accreditation system rationale and benefits. N Engl J Med. 2012:366;1051–1056
- Swing S, Beeson MS, Carraccio C, Coburn M, Iobst W, Selden NR, et al. Educational Milestone Development in the First 7 Specialties to Enter the
- Ten Cate O. Nuts & Bolts of Entrustable Professional Activities (EPAs). J Grad Med Educ. 2013;5(1):157
Reviewed for December 2013 by
Sawsan Abdel-Razig, MD, FACP
Adjunct Assistant Professor of Medicine (UAE University)
And
Hatem Faraj Al Ameri, MD, FRCPC, FCCP
Associate Professor of Medicine (UAE University)