Fog Creek Software
Discussion Board




The 99 percent Accurate AIDS Test

Hi guys,
Been a long time since I last checked here, anyway I just finished reading a game theory book (a really interesting read) and I found this paradox to be interesting so thought I could share here with you.... I'll state the question here and later I could post the answer when you've had your chance at thinking about it :)
So here it goes:

You have just taken an AIDS test. You know a lot about how AIDS is transmitted, and, thankfully, you are almost certain that you don’t have the disease. Anything is possible, however, and you estimate that there is a 1/100,000 chance that you have AIDS.

The AIDS test you took is good but not perfect. The test is 99 percent accurate. This means that if you do have AIDS, the test is 99 percent likely to give you a positive result, while if you don’t have AIDS, there will be a 99 percent chance that the test results will come back negative.

You get back your test results, and they are positive. How concerned should you be?

Christian Kamel
Thursday, October 14, 2004

Concern enough to take the test again, or take a different test.  But not enough to disrupt my sleep; I need the sleep for good health.  Meanwhile, just to be cautious, use condoms until the matter can be resolved.

Is that the type of answer you are looking for?

JHY
Thursday, October 14, 2004

The solution is simple:

Take 100,000 people of whom 1 has AIDS (the possibility if you are not within a group with a higher risk).

Now the test is 99% corrent. So for the one that has AIDS, the test will likely be correctly positive.

For the 99,999 people that do not have AIDS, 1 Percent will get an false positive: Round about 100 people.

So you chance having AIDS is 1 : 100.

Nevertheless, you should of course do a second test. Anyhow, it is important to understand the meaning of such tests. Tests are widly used in modern medicin, but most doctors lack the understanding of the results.

An average doctor would tell you, that your possibility of having AIDS is 99%, which is incorrect but quite shocking. There even are cases, where people killed themself or lost there jobs and friends after getting a false positive. Because they where not even told about that there are more false positives then true positives.

The same thinks holds for less dramatic diseases that are tested with less exact test. Breast cancer, for example.

Some notes:  Actually AIDS-tests are even 99,9% correct, while (as far as I remember) the chance of gettign AIDS as a member of a non-or-less-risk-group is 1:1,000,000 or even higher.

Gerd Riesselmann
Friday, October 15, 2004

Ehem, it should be 1:1,000,000 or even *less* - of course.

Gerd Riesselmann
Friday, October 15, 2004

And another correction (not regarding the millions of typos...):

1 percent of 99,999 is round about 1,000, not 100.
So the likelyhood is 1:1001.


I looked up the real world data (from the end of the 90s):

- The likelyhood to have AIDS for a hetero-male that does not take drugs is 0,01%.
- If a person is infected in 99,9% of all cases, the test will be correctly positive.
- If, however, the person is not infected, 99,99% of all tests will truly state he or she isn't.

Rolled out, this is:
1 of 10,000 has AIDS.
1 of this 1 is tested positive
0 of this 1 are tested negative
9,999 do not have AIDS.
1 of this 9,999 is tested positive
9,998 of this 9,999 are tested negative

2 persons are tested positive of which 1 is a true and one is  a false positive. Luckily, a second test will eliminate the false positives in nearly all cases.

This is little known. A german study (1994) that investigated the information policy of doctors and social workers reagarding HIV tests, found 13 out of 19 experts telling their clients there are literally no false positives. The University of Chicago investigated 21 information brochures about AIDS and none of them even mentioned false positives.

The HIV tests are very very good, other common test aren't. For example:
- For breast cancer, the rate of true to false positives is 1 : 9 (for women older then 50, for younger women it is even higher).
- For cancer - I don't know the english word, let's say "up the ass" -  the rate is 1 : 20 (for men age 50 and above).

Gerd Riesselmann
Friday, October 15, 2004

Well I think Gerd (almost) got it here :)
I'll quote the author of the book for the answer cuz I probably won't be able to explain it as well...
-------------------
You actually shouldn’t be that worried since you almost certainly don’t have AIDS. To see this, imagine that 1,000,000 people, who are just like you, take the test. Each of these people has a 1/100,000 chance of having AIDS. Thus, of these 1,000,000 people, 10 have the disease and 999,990 are free of it. When these people get tested, probably nine or ten of the people with AIDS will get back positive results. Of the 999,990 who don’t have the disease 1 percent will get false positives. This means that there will be about 10,000 false positives. Consequently the vast majority of people who get positive results got false positives. Your chance of actually having AIDS after getting the positive test results are only about 1/1,000.

This result seems very paradoxical since the test is 99 percent accurate. After getting your tests results, however, you have two pieces of information: the tests results and your initial belief that you almost certainly didn’t have AIDS. You don’t lose the second piece of information just because of the positive test results; rather the test results should be used to update your beliefs. These two pieces of information need to be combined. When you (sort of) average the 1/100,000 chance of having AIDS with the 99 percent chance of not having AIDS, you get an approximate 1/1,000 chance of having the disease.
-------------------
Interesting eh? ;)

And to give credit where credit is due the book is Game Theory At Work by James D. Miller

Christian Kamel
Friday, October 15, 2004

This phenomenon happens because you are 99.999 percent sure that you do not have AIDS, and then you go and take a test which is 99 percent accurate.  Therefore, the test is not precise enough for the job.  What you need is a test which is 99.999+ percent accurate.  Then you would be more concerned when you get a positive.

The story would be different for people who are engaging in high risk activities, e.g. drug users, prostitutes, porn stars.  They do not know before hand whether they have the disease or not.  In this case the test results are more meaningful because now you go from completely clueless to 99 percent sure.

If after careful analysis that you found there is 99.999 percent chance that you do not have AIDS, then you do not need to take a test which is only 99 percent accurate.  This is assuming the 99.999 percent is extremely reliable, although how you can arrive at that figure is beyond me.  I said extremely reliable because in your reasoning, you are assuming the 99.999 percent is precise and accurate, and you used it to calculate the number of people who would have the disease.

But in real life how can you be 99.999 percent sure?  Are you living in a bubble on an isolated island?  Maybe the nurse at the clinic accidentally picked up a used needle when giving you the hepatitis B inoculation.  Or some other weird way you can contract the disease without knowing.

This is all about precision.  Here is an analogy.  You have an atomic clock that can tell time good to a nanosecond.  The clock says it is exactly noon, and you try to see if it is really exactly noon by looking at the shadow a sundial.

Here is another analogy.  A super computer calculated pi to the trillionth digit and displays that the trillionth digit of pi is 2.  You know that pi is the ratio of the circumference to the diameter of a circle, so you decided to prove the computer wrong by drawing a circle on a piece of paper and measure its circumference and diameter with a rope and a ruler, then calculate pi by finding the ratio.  On top of that, you did not even use a compass to draw the circle because the compass is hidden somewhere in the garage and you do not want to look for it.  Besides, you figured a hand drawn circle provides enough precision for this simple problem.

You get the picture.  The tests are utterly inappropriate for what you are trying to accomplish.  On the other hand, if you are trying to show that pi is 3, not 30, then it might have worked.

JHY
Friday, October 15, 2004

I took my real world data out of this book:

Gerd Gigerenzer: Das Einmaleins der Skepsis. Über den richtigen Umgang mit Zahlen und Risiken. Berlin Verlag 2002. Paperback: Berliner Taschenbuch Verlag 2004.

The english version is (I think):
Gerd Gigerenzer: Reckoning with Risk. Learning to live with Uncertainty. Penguin Books 2003.

It deals with common misinterpretations of statistical data, mostly in the medical sector - but also before the court (remember the O. J. Simpson trial?). And it offers a quite easy solution to avoid these misinterpretations: Present statistical data as statements about sets, not in percentage. So, rather then to say the chance of having AIDS is 0,01% and the test is 99,99% correct, say: 1 of 10,000 avarage male has AIDS, 1 of 10,000 testes persons not having AIDS will get a false positive test etc.

If statistical data is presented this way, the rate of people (doctors, in this case) that are able to solve a question like the one in this thread is growing from about 10 to extraordinary 100%. Astonishing, isn't it?

Maybe some kind of better user interface ;-).

Gerd Riesselmann
Saturday, October 16, 2004

"Meanwhile, just to be cautious, use condoms until the matter can be resolved."

Why not ABSTAIN until the matter is resolved?  "Sorry, I knew I might have AIDS, and it looks like I gave it to you when the condom failed, but hey it was a really small chance and you couldn't expect me to give up sex for a month."

That leads to justifiable homicide : )

Steamrolla
Thursday, October 21, 2004

The original problem asked for the level of concern with respect to the given percentages.  The made up percentages have little correlation with the real world, but to answer the question we make the assumption that they are true and dependable.

My response, which directly follows the problem, was to address the scenario posed by the problem.  This is not to say that if one tested positive in the real world with completely different percentages, one should or would behave the same.  However, given the figures from the problem, the level of concern should be minimal and certainly does not warrant given up the simple pleasures of life.  The test result should not adversely affect one's mood or desires, should not cause one to lose jobs or friends, should not provoke one to commit suicide, and unquestionably should not propel one to any form of abstinence.

Since HIV is now related to homicide, one must be extra cautious.  I suggest doing some research to find out the most reliable condom, and spend a few more bucks for them.  Now this is really going out of the way to make sure no accidents happen.

JHY
Thursday, October 21, 2004

----" while (as far as I remember) the chance of gettign AIDS as a member of a non-or-less-risk-group is 1:1,000,000 or even higher."-----

Can non-humans get Aids?

Because you'll find it difficult to find many groups with a risk of one in a million or higher.

Going back to the original question, it might be better phrased if you talked about random testing on a sample, where it had been clearly established that the incidence was only one in a hundred thousand. In that case you could be fairly sure that with a 1% false positive rate the chances of the person showing up postive actually being so are around one in a thousand.

Stephen Jones
Friday, October 22, 2004

Can non-humans get AIDS? Not exactly, but monkeys and cats can get a related form of the HIV retrovirus and suffer similar symptoms from it.

Mr Jack
Tuesday, November 02, 2004

I disagree that there are no groups with a risk of Aids that low. If you and your parter have always been monogamous, if you've never used intravenous drugs and never had a blood transfusion or major surgery, then I think your risk level is probably down there.

Low Risk
Friday, November 05, 2004

But what is the probability that your partner(s) never cheated on you?  They could lie about it, telling you how much they love you, and only you.  You might never found out about it because of the intoxicating sweet talk.  I think studies were done on the percentages of wives who have cheated and percentages of husbands who have cheated.  Over fifty percent of people cheat if I remember correctly.

Also, you could inherit the disease.

There are too many IF’s.  IF you were allowed to make that many assumptions, the world would be a utopia.

JHY
Friday, November 05, 2004

So that means fifty percent of people don't cheat. So at worst case 25% of couples have had neither partner cheat (probably more since cheaters are more likely to pair up). I don't know what percentage of people don't have sex before marriage (as opposed to outside it), but I'm willing to bet its 10-20%, and that group is mostly a part of the non-cheaters.

Again make the assumption that that group also doesn't use intravenous drugs, and you've got a group of maybe 5-10% whose risk of AIDS is negligible. Which is a good thing, really. Wish it was higher.

Low Risk
Tuesday, November 09, 2004

> If you and your parter have always been
> monogamous, if you've never used
> intravenous drugs and never had a blood
> transfusion or major surgery, then I think
>  your risk level is probably down there.

That is what I thought too until I was exposed to a co-worker's blood during the course of an industrial accident.

FWIW, the full sequence is 3 tests:  at exposure, 3 months later, and 6 months after that (9 months after exposure).

Luckily all 3 of mine were negative.

Fortunate Son
Friday, November 19, 2004

Well considering that AIDS is not all that infectious I would have been very suprised if you got it from just getting splattered with blood.

You usually need to have a wound (or a failure in the membrane/skin) that the infected blood touches to get infected.

anon from Iceland
Tuesday, November 23, 2004

> Well considering that AIDS is not all that
> infectious I would have been very suprised
> if you got it from just getting splattered
> with blood.

I didn't say "splattered", I said "exposed".  You don't want to know.

Fortunate Son
Tuesday, November 23, 2004

Dear Gerd

The data you got from that book is just plain wrong. In sub-saharan africa the infection rate from AIDS is between 25-30% and most of those are non drug taking heterosexuals. Even in India, with a very low infection rate, there are an estimated 5 million infected at present, again mainly as a result of heterosexual transmission.

Stephen Jones
Friday, December 10, 2004

I find it interesting to find that nobody actually used statistical theory. 

The question posed can be stated as such:

If we define our states as:

s1: you are infected with HIV
s2: you are not infected with HIV

and our results (or outcomes) as

o1: test returns HIV-positive
o2: test returns HIV-negative

then we are given the following information:

P(s1) = .00001
P(s2) = .99999
P(o1 | s1) = .99
P(o2 | s2) = .99

What we're actually looking for is

P(s1 | o1)

which means we need to use Bayes rule to calculate the posterior probabilities. 

First, we need to find the marginal probability of o1 (a positive result, no matter what your state)

P(o1) = P(o1 | s1)*P(s1) + P(o1 | s2)*P(s2)

or

.99 * .00001 + .01 * .99999 = 0.0100098

so now Bayes rule says

            P(o1|s1) * P(s1)
P(s1 | o1) = ----------------
                  P(o1)

=>

.99 * .00001
------------ ~ .000989
  .0100098

which is the same answer most everyone else got (<1/10,000), just done in a more formal way. 

Gosh, I never actually thought I'd use that class for anything...

p.s. since this page doesn't use a monospaced font the fractions might not be quite right, but oh well, you should be able to figure it out. 

Chris Weisel
Monday, December 20, 2004

*  Recent Topics

*  Fog Creek Home