In Autism Studies, Keep an Eye on the Definition

A new study on the level of “autism spectrum disorders” in the United States is sober and circumspect. Some of today’s reporting about it has been less so.

The report, based on the federal government’s National Survey of Children’s Health in 2007-2008, finds a higher level of autism and related conditions than a previous NSCH survey found for autism alone. What needs to be clear is that the two are not directly comparable.

The 2003 NSCH survey asked about “autism,” finding it reported in 0.55 percent of American children. The new survey, instead, asked about “autism, Asperger disorder, pervasive developmental disorder or other ASD.” It produced an estimate of 1.1 percent.

It’s entirely likely that the new, broader definition contributed to the higher count; indeed the report’s authors list “more inclusive survey questions” first among their suggested explanations (higher awareness and earlier diagnosis are the others).

There could be other reasons, as well. The survey was done among parents, asking if they’ve ever been told by a doctor or health care provider that a child of theirs had any of the disorders listed. That is not a measure of clinical diagnosis; indeed, the authors note that in 38 percent of the reported cases, parents said their child does not still have the condition now.

The study’s authors say this could mean that autism or its related disorders “may have been initially suspected… but subsequently ruled out and never truly diagnosed.” I also wonder whether, even despite the question’s context, “pervasive developmental disorder” might have been misunderstood by some parents as developmental disorder of any type, perhaps also contributing to the higher estimate.

It would have been a good approach for this survey to measure each of the conditions listed by itself; it could then have aggregated them to get a net total of “autism spectrum disorder” while maintaining the estimate of autism alone for comparison to 2003. That looks like a missed opportunity.

It’s worth keeping in mind that surveys, even big ones, are not laser surgery. The difference between this measure of autism and related disorders, vs. the previous measure of autism alone, is 0.55 percentage points. That’s statistically significant given the sample sizes (the latest was done among 78,037 parents). But reporting it as a doubling of autism incidence takes advantage of a change in small numbers, as well as disregarding the different definition.

Even first-quality data, moreover, also are subject to nonsampling error. The survey was done by landline phone only; while that was customary as recently as a year or so ago, more good surveys these days include cell-phone-only respondents.

And there are the questions themselves. Surveys, fundamentally, are based on what respondents understand the question to be, and are limited by the questions asked. The sizable number of parents in this study who said they were told their child had ASD, but that he or she does not have it now, leaves questions about the questions that were asked, and suggests follow-ups that weren’t.

Other studies of autism and related disorders have produced different estimates, using different methodologies as well as, again, different questions. I reported here on one such 2007 report, a CDC study in which state estimates were used, questionably, to produce a national estimate of prevalence. In response to the NSCH report, the CDC on Friday said it’s updated that study, also producing about a 1 percent prevalence estimate. It said its detailed report was not yet ready.

That’ll be worth reviewing closely, especially in terms of its comparability to previous CDC work. As they go forward, what autism researchers seem to need is an agreed-upon definition of exactly what it is they’re measuring, as well as a consistent approach to measuring it.

The new study is published in the journal Pediatrics. One cou-author, Dr. James M. Perrin, is reported to be “supported in part by a grant from Autism Speaks,” an autism awareness group. I have occasional contact on other matters with the second author, Stephen J. Blumberg, a researcher at the National Institute of Health Statistics who’s also standards chair of the American Association for Public Opinion Research.

Join the Discussion
blog comments powered by Disqus
You Might Also Like...