Health knowledge made personal
Join this community!
› Share page:
Go
Search posts:

Think You Have Autism? Don't Take a Quiz.

Posted Oct 27 2010 8:44pm
Over in the autism area on about.com, Lisa Jo Rudy posted something today where she talks about how taking the self-administered  Autism Quotient (AQ) test might be a good first step if you think you have autism -
Of course, there is a disclaimer that these tests are not diagnostic, and they're only to be used for "genuine research." Still, for parents of children on the autism spectrum, adults wondering whether they might fall into an autism spectrum category, or anyone concerned about the symptoms of autism, these tests may be a useful first step in deciding whether or not to seek an evaluation.
This is very bad advice.

First of all, if you seriously think that you may have a disorder such as autism or any other mental disorder then you should seek professional help.  You should not try to take a test you found on the Internet to see if you might "fall into an autism spectrum category" or to give you a "hint".  None of the lists of signs symptoms on the Internet nor any of of the tests out there are going to be able to tell you whether you have autism.  Your "hint" should be that you think there is something wrong and that alone should spur you to seek a professional's opinion.

But lets say that you have decided to take Lisa's advise or that you, like many people before you, have decided that this test is an appropriate way to determine whether you have autism.   The question then becomes if you take the Autism Quotient test and score high enough, how likely are you to actually have autism?

It turns out that the answer is not very likely - the odds are about 1 in 10.

To say that the opposite way, if you had a group of ten people who all scored high (over 32), nine of them would not have autism.  So if you are claiming that you have autism based solely on a self-administered AQ test then you are most likely wrong.

To make matters worse, even if you score below that 32 number, you are still somewhat likely to have autism.  If you took a group of ten people who actually had autism, two of them would likely score below the 32 cutoff.

So either way the AQ isn't going to tell you that much.

To understand the reason the odds are so very much against you, you have to understand what the purpose of a test like the AQ is - and what it isn't.  A test like this is supposed to be used to help find people who are more likely to have a condition.  It is not supposed to be able to tell you that one specific person has the condition.

A test like this can be used as a screening tool in research to help researchers find the people who are more likely to have a condition so that they can focus on those people.  Or, if the test has been validated in earlier studies, researchers can use the results of the test to draw general conclusions about the people in their study.

In either case, to properly use the test you have to know how good of a job the test does at finding those who have a condition and eliminating those that don't.  These concepts are formally called sensitivity and specificity .  The sensitivity of a test tells you have good of a job the test does at finding the people who have the condition while the specificity tells you how good of a job the the test does at telling you which people don't have the condition.

In an ideal world, you would want every test to identify every person that has a condition (sensitivity of 100%) and exclude every person who doesn't (specificity of 100%).  But, in the real world, tests are never that accurate and we have to make do with what we can get.

When it comes the sensitivity and specificity, the AQ isn't the greatest but it isn't too bad.  According to a recent paper that I was reading, the AQ has been demonstrated to find 80% of the target group (sensitivity 80%) while only misidentifying 2% of the rest (specificity ~ 98%).

Which leads me to my next, point, what is it exactly that the AQ is supposed to be finding?  Here's a hint, it isn't what you normally think of as autism - "The Autism Spectrum Quotient (AQ) has been developed to measure the degree to which an adult with normal intelligence has autistic traits".  If you dig a little bit more, you would see that this test is meant to find adults who have "traits" of Asperger's or high-functioning autism.

Or in other words, this test is geared towards finding individuals who fall on a small portion of the autism spectrum but is not going to be very helpful in identifying the majority of people with autism.  Well, it won't be helpful  unless you believe that - all evidence to the contrary - the majority of people with autism are high functioning.

So far the AQ doesn't seem to bad - it should be capable of identifying the majority of its target audience.  But there is one last piece to the puzzle of understanding why the AQ isn't able to tell you much, and that piece is called the  positive predictive value .  The idea is to take the sensitivity and specificity and put them in practical terms, and this value does that by telling you how likely it is that you have a condition if you have a positive value on the test.

This is where the AQ starts running into some serious problems - the positive predictive value is somewhere between 8% and 12%.  This means that even if you have a positive value on the test, you are only about 10% likely to have Asperger's or high-functioning autism.  The other 90% of the time the test is just wrong.

If that doesn't make any sense to you, perhaps an example would make the picture clearer.

Say you are looking at a group of 10,000 people and that we can assume that 1% of the people have any form of autism.  How many people will the AQ correctly identify, how many will it fail to identify, and how many will it falsely identify?

The first thing we need to know is how many of the cases of autism we would expect to be classified as Asperger's or high-functioning autism.  While there isn't a good answer to this question, I think it would be reasonable to assume that 25% of all of the people with autism fall into one of those two groups.  I suspect that the actual number would be less than this, but this number is good enough for this example.

So, with 10,000 people, we would expect 100 of them to have any form of autism and 25 of them to have Asperger's or high-functioning autism. Given what has been found in studies that looked at the validity of the AQ, we would expect that the AQ would correctly identity 20 of that 25 (80% sensitivity) and miss 5 out of the 25.  We would also expect that 2% of the rest of the population would be falsely identified as having Asperger's or high-functioning autism, or roughly 200 people.

If we put that all together, we would find that, out of the 220 people identified by this test, only 20 would actually be correctly identified and another 5 would be missed.  Hence the positive predictive value would be about 9.1%, which translates roughly to a 1 in 10 chance of being correctly identified.

Now, just to nitpick my own argument and be completely honest,  one possible source of error in the above analysis is what the AQ would show for the 75 other cases of autism.  Would it be able to label them as having autism or would it miss them as well?  The short answer is that we don't know because this test has never been validated in this population.  No published research has shown how well the AQ does at detecting classic autism, PDD-NOS, Retts, or CDD in a large group of people and, without that information, it is simply impossible to know how well it would perform.

But, for the sake of argument, lets assume that the AQ works the same for every person with autism rather than just the higher functioning ones.  If you use the same assumptions that I used above, you would find that a positive result in the test would give you a 1 in 3.5 chance (or about 30%) of actually having autism.

The bottom line is that even if you get a high score on the Autism Quotient you are not likely to autism.   So if you think you might have autism, don't take a random test on the Internet - go talk to a professional instead.
Post a comment
Write a comment: