Peer-reviewed article
To Collapse or not to Collapse Multiple Measures of the Same Psychological Construct: A Demonstration Across 14 Independent Studies of Autistic and Non-Autistic Children.
Erin Reckner, Naveen Thourani, Drew Parker, John D Herrington, Timothy P L Roberts et al. ·
Summary
Written by AI, not yet AICS's view · claude-haiku-4-5-20251001This is a methodological paper about statistical approaches for combining data from multiple studies that use different measurement instruments. It demonstrates a quantitative method to test whether different instruments measuring the same psychological construct (like cognitive ability or anxiety) can be validly combined in secondary data analysis. The study includes 1,352 autistic and non-autistic children from 14 independent studies. The paper addresses a technical research question about instrument compatibility rather than evaluating an intervention or assessing child well-being outcomes.
Key points
- The study develops a quantitative method using moderation analyses to test whether different instruments measuring the same construct are compatible for data pooling
- Some commonly used instruments within domains like cognitive ability, anxiety, and ADHD symptoms can be acceptably combined, while others show discrepant relationships with outcomes
- The research includes 1,352 participants from both autistic and non-autistic samples across 14 independent studies
- The paper addresses a practical challenge in secondary data analysis where different studies use different measurement tools
Limitations
This is a methodological paper focused on statistical techniques for combining datasets rather than a substantive evaluation of child well-being or any intervention. It cannot provide evidence about the effectiveness of any programme or policy, nor does it evaluate outcomes related to children's survival, development, protection, or participation.
Abstract
Large data repositories provide a valuable opportunity to combine multiple independent studies, creating large samples that can address questions not achievable within a single study. However, independent studies often use different instruments to measure the same behavior or psychological construct, such as cognitive ability, social functioning, or psychopathology. Investigators often combine samples that include different instruments under the assumption that instruments assessing the same construct capture the same underlying phenomenon. Yet this assumption of instrument interchangeability may be wrong. Collapsing different instruments into a single variable could introduce bias. Here, we established a quantitative method to demonstrate compatibility among multiple instruments with moderation analyses. We illustrate this method in domains of cognitive ability, anxiety symptoms, and attention-deficit/hyperactivity disorder symptoms as exemplars that might interact with outcomes of interest (autistic features instruments). A total of 1,352 participants were included from autistic and non-autistic samples across 14 independent studies. Our findings suggest that within this sample, some commonly used instruments within a domain can be acceptably combined, whereas others demonstrate discrepant relationships with outcomes of interest. More importantly, we offer a generalizable approach for scientists seeking to leverage existing datasets for secondary analyses.