top of page

Distillation Paper: A program can sound good. Can people actually use it?

Nov 6
1 min read

Distillation Paper | Critical appraisal | 6 November 2026


Weiner et al. (2017). Psychometric assessment of three newly developed implementation outcome measures. Implementation Science, 12, 108. doi:10.1186/s13012-017-0635-3.



The discourse and the gap


Researchers needed practical measures of whether a program seems acceptable, suitable, and doable. Existing measures often blurred those ideas or lacked adequate testing.


The research question


Could brief scales measure acceptability, appropriateness, and feasibility with useful reliability and validity?


The study design


Three studies combined expert assessment, randomized scenarios rated by 326 mental health counselors, and repeat testing.


The methods and their weight


Factor analysis examined how items grouped; reliability tests assessed consistency. Useful checks, not universal validation. In the 326-person scenario study, 91% identified as White and 77% worked in private practice. That sample and the hypothetical task limit transfer to other workforces. A survey design error also limited testing of how clearly the scales distinguished the three concepts.


The finding that matters


Three four-item measures showed promising measurement properties. The important contribution is a brief way to assess different implementation concerns, not evidence that a program improves people’s lives.


Why it matters


My reading: ask whether people welcome a program, whether it fits, and whether they can deliver it. This paper did not establish that these scores predict adoption, and it did not test CRG outcomes.



Recent Posts

See All

Comments


Commenting on this post isn't available anymore. Contact the site owner for more info.
bottom of page