Peer-reviewed article
Using Real-World Evidence to Estimate the Predictive Value of Screening Questionnaires: Autism Screening as a Case Example.
R Christopher Sheldrick, Jessica L Hooker, Meghan Davignon, Jean Sakimura, Yinge Qian et al. ·
Summary
Written by AI, not yet AICS's view · claude-haiku-4-5-20251001This study evaluated the accuracy of two autism screening questionnaires (POSI and M-CHAT-R) used with children aged 4 and under in a large US healthcare system. The researchers used electronic medical records from over 33,000 families to measure how well these screening tools identified children who received an autism diagnosis. The POSI screener showed reasonable sensitivity and specificity, with predictive accuracy varying depending on screening score.
Key points
- POSI sensitivity was 69.5% at 18 months and 70.3% at 24 months
- POSI specificity was 76.4% at 18 months and 80.8% at 24 months
- Predictive probability of autism diagnosis ranged from 4.9% for low scores to 76.8% for high scores on POSI at 24 months
- Predictive values improved when secondary screening (M-CHAT-R) was used
- Authors recommend replicating findings in other health systems to verify applicability
Limitations
This study measures screening tool accuracy in one US healthcare system and cannot establish whether the POSI or M-CHAT-R directly improves child outcomes or access to services. The findings may not generalise to other healthcare contexts or populations.
Abstract
OBJECTIVE: We used real-world evidence to estimate the accuracy of autism screening questionnaires, including sensitivity, specificity, and predictive probabilities for positive results and for individual screening scores. METHODS: This study utilized electronic medical records (EMR) from 33,490 families of children aged 4 years and under and enrolled in Kaiser Permanente Northern California. Independent variables included results of the Parents' Observations of Social Interactions (POSI) and Modified Checklist for Autism in Toddlers-Revised (M-CHAT-R), which were implemented as primary and secondary screeners, respectively. The primary outcome was an autism diagnosis in the EMR by age 4 years. RESULTS: The POSI displayed 69.5% sensitivity and 76.4% specificity at 18 months, and 70.3% sensitivity and 80.8% specificity at 24 months. Predictive probabilities for positive scores varied; for example, 4.9% of children scoring 3 on the POSI at 24 months were diagnosed with autism by 4 years, compared to 76.8% of children scoring 7. Predictive values were higher among children who completed the MCHAT-R as a secondary screener, and even higher among those who had a positive MCHAT-R screen. CONCLUSIONS: Demonstrating the range of information on screening that can be learned from EMR data, findings support the accuracy of the POSI while providing evidence of predictive probabilities to inform clinical care. Evidence and theory demonstrate that screening accuracy can be influenced by context. Thus, we recommend replication in other health systems to ensure fit-to-context and replication over time to identify potential trends in prevalence and decisions that may influence the utility of autism screening.