Development and Pilot Evaluation of the ECALM Program: A Bedside Clinical Assessment Tool for First-Year Residents in the Emergency Department.

Authors

Siiri Astudillo Kunnas, Lilia Rotari, Farès Moustafa, Daniel Aiham Ghazali

Published in

Advances in medical education and practice. Volume 17. Pages 629117. Epub Oct 06, 2026.

Abstract

To describe the development and pilot evaluation of the ECALM program ("bedside clinical evaluation", in French), a formative bedside assessment framework designed for first-year residents during emergency department (ED) rotations.
The ECALM program was developed prospectively in three stages, in accordance with Stufflebeam's "context, input, process, product" framework: (i) a baseline needs assessment survey, (ii) the development of standardized clinical scenarios via a structured expert consensus process, and (iii) a single-center pilot implementation. Six core clinical themes were selected (chest pain, abdominal pain, limb trauma, syncope, fever, and confusion), on the basis of epidemiological data from French EDs. A pilot study involving 12 first-year residents was conducted at Amiens University Medical Center's ED (Amiens, France). Each resident was evaluated with a single scenario, and each session included a structured briefing, passive bedside observation by paired, independent, senior evaluators, and a formative debriefing. Psychometric evaluation of the ECALM tool was based on inter-rater reliability, using the intraclass correlation coefficient (ICC(2,1)) calculated using a two-way random-effects model for absolute agreement based on a single measurement, the weighted kappa, and a Bland-Altman analysis of inter-observer differences.
In the needs assessment (physician response rate: 27.1%), 97% of the participating senior practitioners reported that residents were insufficiently prepared for independent clinical tasks. During the pilot phase, the mean ± standard deviation total duration of an ECALM session was 43 ± 12 minutes. A preliminary analysis of the scores' consistency yielded an ICC(2,1) of 0.76, indicating good inter-rater agreement between independent evaluators. The weighted kappa (0.74) was consistent with the ICC results. A Bland-Altman analysis demonstrated a mean inter-observer difference of only 1.25%, indicating negligible systematic measurement bias between evaluators. Qualitative feedback demonstrated high acceptability: all 12 residents expressed their satisfaction, reported an increased sense of psychological safety, and perceived the session to be a "one-on-one tutorial". Furthermore, ten residents (83.3%) spontaneously reported acquiring new knowledge during the debriefing phase.
The results of this pilot study demonstrated the operational feasibility, high acceptability, and good preliminary inter-rater reliability of the ECALM program for the formative assessment of first-year residents. Although ECALM constitutes a promising, non-punitive approach for encouraging early clinical reasoning and self-assessment, further well-powered, multicenter validation is required to demonstrate the tool's external validity and long-term impact.

PMID:
42859792
Bibliographic data and abstract were imported from PubMed on 11 Oct 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.