The MIND Institute's Second Attempt: More of the Same Type of Reasoning
Posted Jan 08 2009 10:16am
The MIND Institute has published a new study titled The Rise in Autism and the Role of Age at Diagnosis (hereby referred to as H-P et al.) Among its conclusions: "Younger ages at diagnosis, differential migration, changes in diagnostic criteria, and inclusion of milder cases do not fully explain the observed increases." As you can imagine, the paper is being cited uncritically in the usual places.
I would like to discuss what it is the paper finds, how it finds it, and whether the findings are accurately characterized. Before I do that, however, I think some background is in order.
Back in 2002, a MIND Institute report to the California Legislature concluded basically the same thing: That a loosening of the criteria had not contributed to the rise in autism diagnoses in California. Back then the MIND Institute report was considered evidence of the "autism epidemic." But it contained a significant error in reasoning, first noted in Gernsbacher, Dawson & Goldsmith (2005).
Basically, the 2002 report found that an earlier cohort of children met DSM-IV criteria at about the same rate as a more recent cohort of children. As Gernsbacher et al. patiently explain, it's not surprising that the more recent DSM-IV criteria is met by nearly all the children, the younger ones and the older ones. What would be more relevant to find out is whether the older children meet some narrower criteria that the younger children do not meet. Gernsbacher et al. illustrate the fallacy using a height analogy and call the researchers' conclusions "imprudent."
There are two main areas of the new paper that I want to discuss: Changes in diagnostic criteria, and inclusion of milder cases (birth cohorts 1990-2006). The paper looks at other areas, but I'd like to focus on the two that are key.
Changes in diagnostic criteria
Finding: Changes in diagnostic criteria have contributed a 2.2-fold increase in the rise of autism incidence in California.
How is this determined? This being the most important area of the paper, my expectation was that I would find new data produced by H-P et al. showing that only a 2.2-fold increase may result from changes in diagnostic criteria. This is not the case at all. H-P et al. rely on prior work.
Specifically, they rely on Kielinen et al. (2000), a study out of Finland based on data "collected from hospital records and the records of the central institutions for the intellectually disabled in the Provinces of Oulu and Lapland in 1996–1997."
The 2.2-fold increase is the difference found by Kielinen et al. when comparing autistics diagnosed with Kanner's criteria vs. those diagnosed with ICD-10 or DSM-IV.
So what's wrong with this result? You need to ask yourself if the results from Kielinen et al. are applicable to changes in the California DDS autism population.
You see, even if we assume that all California autistics from the birth cohort 1990 were diagnosed with Kanner's criteria, why should we assume that all persons matching this criteria were identified and registered with California DDS? Isn't it more likely that only a small minority of such persons were recognized at the time? Furthermore, Kielinen et al. only look at the population with autism or other psychiatric conditions as recorded in known databases. California DDS has a much broader population pool to draw from, which would be more than relevant at the present time.
Also note that the DSM-IV prevalence found by Kielinen et al. is low compared to that of other DSM-IV studies: 20.7 in 10,000.
In essence, I do not believe there is any basis for comparison of Finnish ascertainment done in a 2000 study vs. California DDS ascertainment, either in 1990 or in 2006.
Inclusion of milder cases
Finding: Inclusion of milder cases contributes a 1.5-fold increase in the rise of autism incidence in California.
How is the result determined? H-P et al. again rely on work from a separate study: The Childhood Autism Risks from Genetics and Environment study. In this study, 64% of California cases were confirmed to meet criteria for autism (excluding Asperger's and PDD-NOS) when evaluated using two separate diagnostic instruments.
I was initially confused as to why inclusion of "milder" cases is necessary to take into account, if the paper already claims to have looked at changes in diagnostic criteria. The reason is probably that Kielinen et al. would not have studied PPD-NOS or Asperger's cases.
So what's wrong with the result? If you're trying to account for cases that Kielinen et al. would have missed, I'm not sure that PDD-NOS and Asperger's are enough. What about high functioning autism in general? That's not the same thing.
There's also the question of why only "milder" cases should be considered. Why not more severe cases? Should we assume that Kielinen et al. would've detected all of those? Then there are cases that are neither more severe nor milder but just not diagnosed as autism traditionally. For example, would Kielinen et al. have recorded cases of autism in Down Syndrome or Cerebral Palsy, where California DDS obviously does?
In the end, this particular result does not tell us much. What it says is that if you exclude all autistic children who should not be eligible for California DDS services, the impact on the number of cases is not too great. I could've told you that.
Did They Miss Anything?
One question that comes to mind is why the researchers did not discuss diagnostic substitution, particularly from mental retardation.
Would this be part of the "changes in diagnostic criteria" analysis? Yes and no. To the extent that children previously diagnosed with mental retardation are currently diagnosed with autism because DSM-IV criteria says so, then yes, the analysis by H-P et al. would suffice (assuming that analysis were valid, which I don't believe it was). If there are reasons for the shift that go beyond criteria, then no, it is not enough.
Diagnostic substitution in California is interesting for other reasons. I think there's sufficient data there to do a proper analysis. For example, you can look at cases of mental retardation without autism over time and see if they decline. I have reason to believe such an analysis will be available in the near future.
H-P et al. is a surprisingly poor paper. It does not produce any new data in order to support its two main results. It makes an apples-to-oranges comparison between a Finnish epidemiological study and California DDS ascertainment over time. It tells us the obvious about "milder" cases. In the end, I don't think this is an improvement over the 2002 MIND Institute report to the California Legislature. In fact, it could very well be worse.
The way H-P et al. have gone about trying to show there's a real rise in autism incidence over time is not a very good way to go about doing things, in my view. There are other ways. For example, I've suggested trying to replicate Lotter (1967) in detail. This would not be as easily challenged.