A Study of Variability in Brief Assessment of Transactional Success in Aphasia Story Retelling Performance Using Generalizability Theory.

Authors

Alexis Doyle-Connolly, Jacquie Kurland, Anna Liu, Pauline Stokes

Published in

American journal of speech-language pathology. Pages 1-12. Oct 12, 2026. Epub Oct 12, 2026.

Abstract

This study used generalizability theory (G-theory) to assess the reliability of the Brief Assessment of Transactional Success in aphasia (BATS) and whether reliability can be maintained with more efficient study designs.
G-theory was applied to a data set from BATS test development, where story retells were scored using main concept analysis. Linear mixed-effects models were used to estimate the proportion of variability in scores attributable to storyteller (a person with aphasia [PWA] or familiar/unfamiliar conversation partner [CP]), rater (human or automated), and stimuli. Decision studies (D-studies) were conducted to determine the number of stimuli needed for acceptable reliability. Score distributions and correlations were examined to identify stimuli that disproportionately contributed to measurement error.
Differences in performance between PWA contributed most to variability in scores, whereas rater contributed minimally. Stimuli contributed substantially, primarily through interactions with PWA and storyteller. Two stimuli (Bicycle Boy and Ferguson) emerged as the least and most challenging; when excluded, stimuli were no longer a significant source of variability. D-studies showed that reliable results could be obtained with three to four stimuli, or as few as two when Bicycle Boy and Ferguson were excluded. CP familiarity contributed to variability in scores via its interaction with PWA.
The BATS demonstrated high reliability despite multiple sources of variability in scores. Results support the reliability of automated main concept scoring and show that stimuli-driven error can be controlled with a limited number of stimuli. These results offer guidance for designing efficient, reliable discourse studies.

PMID:
42862813
Bibliographic data and abstract were imported from PubMed on 13 Oct 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.