Distillation Paper: A program can sound good. Can people actually use it?
Distillation Paper | Critical appraisal | 6 November 2026
Weiner et al. (2017). Psychometric assessment of three newly developed implementation outcome measures. Implementation Science, 12, 108. doi:10.1186/s13012-017-0635-3.
The discourse and the gap
Researchers needed practical measures of whether a program seems acceptable, suitable, and doable. Existing measures often blurred those ideas or lacked adequate testing.
The research question
Could brief scales measure acceptability, appropriateness, and feasibility with useful reliability and validity?
The study design
Three studies combined expert assessment, randomized scenarios rated by 326 mental health counselors, and repeat testing.
The methods and their weight
Factor analysis examined how items grouped; reliability tests assessed consistency. Useful checks, not universal validation. In the 326-person scenario study, 91% identified as White and 77% worked in private practice. That sample and the hypothetical task limit transfer to other workforces. A survey design error also limited testing of how clearly the scales distinguished the three concepts.
The finding that matters
Three four-item measures showed promising measurement properties. The important contribution is a brief way to assess different implementation concerns, not evidence that a program improves people’s lives.
Why it matters
My reading: ask whether people welcome a program, whether it fits, and whether they can deliver it. This paper did not establish that these scores predict adoption, and it did not test CRG outcomes.

Comments