Resident preparation for on-call responsibilities using a formative objective structured clinical examination (OSCE).
Residents routinely carry the bulk of on-call responsibilities on academic hospital services. Despite this, junior resident preparation for call is often suboptimal, informal on-the-job training by more senior residents with little standardization or direct observation by faculty. The objective structured clinical examination (OSCE) is an accepted formative or summative assessment tool that may also be used to generate and deliver immediate feedback. We report our experience in the use of a formative objective structured clinical examination (OSCE) for on-call resident preparation.
Background
Singapore is currently transitioning from a United Kingdom-style system of graduate medical education to one modeled after residency programs in the United States guided by the Accreditation Council for Graduate Medical Education-International (ACGME-I). In May 2010, the National University Health System along with two other Sponsoring Institutions in Singapore welcomed its pioneer class of residents. Under the new system, residents follow a more compact, structured and intensive program, and they are expected to achieve competency in the six ACGME-I core competencies in as little as three years in specialties such as Internal Medicine. The system assumes that medical school graduates will be prepared for the rigors of residency from day one.
Why the idea was necessary
The experience of our first class of residents in their first few months of training informed the rationale to develop of a formative �on-call� OSCE. Even though our residents felt they were prepared to handle clinical duties on a non-call day, they were under-prepared for the rigors of taking call on the medical wards. Some of this feedback from the trenches regarding their first medicine calls included: �Medical school probably prepared us for about 20-30% of the things we were called up for�; �I didn�t know when I had to take blood cultures and initially I did blood cultures for everyone only because I was afraid to be scolded the next morning�; �It would definitely help to be more prepared to handle the scenarios hands-on, e.g. hypoglycemia, rather than just go through the theoretical�; �I had to manage so many conditions at one time and at the end of the night you don�t even know if you have done the right thing�; �[A prep course] would allow people to have standardized training and understanding so the resident can go about his/her duties with more confidence, that the patient is not let down as a result.�
Being on-call is the period of clinical duty that makes the most physical and mental demands on trainees. There is little formal preparation for this and it is often informed by faculty who may have completed their own training (including taking in-hospital call) many years prior and who are thus not in touch with current demands. The majority of junior resident preparation is thus informal, on-the-job training by more senior residents with little standardization or direct observation by faculty.
Aims
Our aim was to develop a formative OSCE as a means to a) better prepare junior residents for taking in-hospital call and b) standardize approach to common on-call scenarios by formal teaching grounded on best practices rather than informal on-the-job training
Methods
We conducted a survey among senior residents to identify common on-call scenarios that they face at our institution. Teams composed of 2-3 senior residents and a faculty member designed eight OSCE stations based on the scenarios identified by the survey. The stations comprised one each of the following: communication with medical team; communication with patient�s family; assessment of hypoglycemic patient; assessment of hyperglycemic patient; assessment of hypoxic patient; assessment of febrile patient; management of transfusion reaction; management of urinary retention. Each station comprised of 10 minutes of testing followed by 5 minutes of immediate verbal feedback given by the examiners. We analyzed differences in mean scores using the two-tailed, Student�s t-test for unpaired data.
Results
Fifty-two junior residents participated in the OSCE at entry into residency. The residency program distribution was as follows: internal medicine 25, transitional year 10, pediatrics 9, emergency medicine 4, psychiatry 2, general surgery 1 and preventive medicine 1. Fifteen residents (28.8%) were in their second post-graduate year (PGY2) or above, while the rest were in PGY1. The mean overall score was 70.7 � 6.4 %. Mean scores for each station were: communication with medical team 56.7%; communication with patient�s family 77.9%; assessment of hypoglycemic patient 70.7%; assessment of hyperglycemic patient 66.2%; assessment of hypoxic patient 78.0%; assessment of febrile patient 79.4%; management of transfusion reaction 77.4%; management of urinary retention 60.2%. Residents in the PGY2 group earned higher mean scores than residents in PGY1 (73.9% versus 69.4%, P = 0.018). All residents received immediate, individualized feedback during the OSCE and performance was reported to their residency program directors for development of individualized learning plans.
| Resident post-graduate year
|
Number of residents
|
Mean overall score %
|
| All
|
52
|
70.7
|
| 1
|
37
|
69.4
|
| ≥2
|
15
|
73.9*
|
*p=0.018
Evaluation
We administered an anonymous online questionnaire via SurveyMonkey� 3 months after the OSCE, allowing residents to reflect on their interim, actual on-call experiences in relation to the OSCE�s mock on-call scenarios. Residents rated their self-perceived effectiveness of the OSCE content and immediate feedback format using a four-point, Likert-like scale (Disagree, Inclined to Disagree, Inclined to Agree, Agree) which we collapsed into two categories (Agree versus Disagree) for analysis. 48 residents (92.3%) completed the survey.
In the domain of content, the majority of residents agreed that the OSCE helped them build confidence (95.8%) and improved their ability (95.8%) in managing common on-call clinical encounters. In the domain of format, residents agreed that receiving individual feedback during the OSCE helped them to direct their learning (93.8%), identify their strengths and weaknesses (95.8%), and incorporate feedback into their practice (95.8%).
Residents gave a positive overall rating (median = 4 out of 5, interquartile range = 4-5) and 95.8% felt that the OSCE should become a regular feature at the start of residency.
Conclusion
In the era of outcomes-based medical education, when direct observation and skills assessment are integral to medical training, a formative OSCE is an effective tool with which to teach and assess resident skills including those required to take call.
References & further reading
- Archuleta S, Goh WP, Aw M. Resident preparation for on-call responsibilities using a formative objective structured clinical examination. Poster Presentation. 9th Asia Pacific Medical Education Conference. Singapore, 11-15 January 2012. Abstract published in Med Edu 2012; 46 (2 Supp):2-3.
- Hodder RV, Rivington RN, Calcutt LE, Haart IR. The effectiveness of immediate feedback during the Objective Structured Clinical Examination. Med Educ. 1989 Mar; 23(2): 184-8.
- Cogbill KK, O�Sullivan PS, Clardy J. Residents' Perception of Effectiveness of Twelve Evaluation Methods for Measuring Competency. Acad Psychiatry. 2005 Spring; 29(1): 76-81.
Author
Sophia Archuleta, M.D.
Assistant Professor of Medicine1
Infectious Diseases Fellowship Program Director2
Affiliations
1Department of Medicine, Yong Loo Lin School of Medicine, National University of Singapore
2University Medicine Cluster, National University Health System, Singapore