Disease Screening & Base Rate Fallacy

Definition
The base rate fallacy refers to the neglect of prior probability of the evidence that supports the conditional probability of a hypothesis.
(Based on Wikipedia)

(Photo: mayaevening at Flickr)

Commentary
A recent example is the controversy about breast cancer screenings.

Imagine that about 1% of women (1 in 100) have breast cancer. You have a diagnostic test that correctly detects cancer 85% of the time (i.e. if the test is given to 100 women with cancer, it catches 85, but misses 15 of them).

Also, the test sometimes incorrectly detects cancer (when none is present) about 10% of the time (i.e. if the test is given to 100 women without cancer, it accidentally tells 10 of them they have cancer, but correctly tells the other 90 they don't have cancer).

Now the tricky bit: Imagine we give the test to 1,000 women in the population. If the test says a women has cancer, what is the probability she actually has cancer?

This question is hard for many people (including doctors!) because it's hard to make the trade-offs in our head about whether or not the test is accurate for this particular woman. Here's how you would do the calculation correctly:

1. Based on the rate of cancer in the population (1%), how many of the 1,000 women tested do we expect to have cancer?
   Answer: About 10.
   
   
2. Of those 10 who have cancer, about 9 will be told they have cancer, and 1 will missed. (Recall, the test only catches 85% of cancers.)
   
   
3. Now of the remaining 990 women who don't have cancer, about 99 of them will be told they have cancer (10% false-alarm) while the rest (891) will be correctly told they don't have cancer.
   
   
4. So how many women are told they have cancer?
   Answer: 9 + 99 = 108.
   
   
5. How many of those women actually have cancer?
   Answer: Just the 9.
   
   
6. So if you're told you have cancer, what's the chance you actually have cancer?
   Answer: 9 / 108 = 8.3%

Pretty strange, right? What about the people who are told they don't have cancer? What's the probability you actually do have cancer?

1. How many women are told they don't have cancer?
   Answer: 1 + 891 = 892.
   
   
2. How many of those actually have cancer?
   Answer: Just the 1.
   
   
3. So if you're told you don't have cancer, what's the chance that you actually do have cancer?
   Answer: 1 / 892 = 0.1%

That means that it's pretty unlikely for you to have cancer if the test says you don't.

The reason this occurs is because the number of women who have breast cancer to begin with is not that high (10 of 1,000). Therefore, the mistakes the test makes start to matter when applied to the entire population.

Naturally, this has policy implications: if you test more and more people, a large percentage of people will be told they have cancer when they don't-- leading to more invasive testing that has other real side-effects. The trick is either to try to test a high-risk subpopulation (where the prevalence rate is higher) or to improve the test by reducing its false-positive rate.

See Also

• Screening for disease and dishonesty at Understanding Uncertainty for an excellent interactive representation of base rates.
• Screening for Breast Cancer at the U.S. Preventive Services Task Force for a summary of its recommendations.
• The base rate fallacy reconsidered for a counter-argument about base rate neglect.

Shoes & Gait

Summary
A study in the journal The Foot showed that wearing shoes isn't that good for your feet.

(Video: YouTube)

Meta
Suppose you were shown the overwhelming evidence of the harm of shoes on gait-- how would you respond?

See Also

• Shod versus unshod: The emergence of forefoot pathology in modern humans? for the abstract or the author's personal version.
• Why Shoes Make "Normal" Gait Impossible in Podiatry Management for an older study.
• You Walk Wrong at New York Magazine for an article on the subject.
• Feet Hurt? Stop Wearing Shoes at NPR for another article.

Placebo Effect

Definition
A placebo is a fake treatment. The placebo effect is the phenomenon whereby patients feel some benefit even from placebo treatments.

Sweet sugar crystals.
(Photo: Wikimedia)

Commentary
Wired has an excellent article on the increasing effectiveness of placebos. It seems that this increase is correlated with the lifting of the ban on direct-to-consumer marketing of drugs by the FDA in 1997.

Other research is focused on determining how placebos work. In one study, researchers were able to see parts of the placebo effect in action in the spine by telling participants that they were trying out a new painkilling cream against a non-painkilling cream (both creams were inactive). While the exact mechanism needs further investigation, this was the first step towards demonstrating the participation of the spine in the placebo effect.

See Also

• Placebos Are Getting More Effective at Wired for the full article.
• Placebos trigger an opioid hit in the brain at New Scientist for a brief discussion of the real effects that placebos have (i.e. generating chemicals in our bodies).
• The Dilute Homeopathy for an example of a very expensive placebo.

The Mindless Ginkgo

Summary
Ginkgo biloba is a species of tree that is claimed to help prevent memory loss, reverse the effects of dementia, and prevent the onset of Alzheimer's. A recent study showed no such effects.

(Photo: kcm76 on Flickr)

Commentary
The study was the latest product of the National Center for Complementary and Alternative Medicine (NCCAM) -- an organization that uses taxpayer money to design excellent studies to demonstrate the worthlessness of alternative medicines (such as homeopathy).

This particular study is impressive:

[The study is a] randomized, double-blind, placebo-controlled clinical trial of 3069 community-dwelling participants aged 72 to 96 years, conducted in 6 academic medical centers in the United States between 2000 and 2008, with a median follow-up of 6.1 years.

Their conclusion: "Annual rates of decline in z scores did not differ between G. biloba and placebo groups in any domains..." (emphasis added). Which domains did they test? Among them were memory, attention, visuospatial abilities, and language.

See Also

• Ginkgo biloba – No Effect at Science-Based Medicine where Steven Novella points out that some of the effects of supplements (e.g., blood thinning) may pose unnecessary risks (e.g., patients undergoing surgery).

The Dilute Homeopathy

Definition
Homeopathy refers to treatment of symptoms by extremely diluted versions of poisons that are thought to create those symptoms. I'll defer to The Amazing Randi for a nice explanation of why that doesn't make any sense.

(Video: YouTube)

Commentary
My favorite part: "Non habit-forming." Except for the people who have a habit of buying homeopathic "remedies."

See Also

• Mole as a unit of substance.
• Avogadro's constant for the number of atoms or molecules in a mole.

Chiropractic Destroyed from Within

Summary
Chiropractors make numerous claims about their ability to treat symptoms by a method called chiropractic subluxation (not to be confused with medical subluxation with is visible in x-rays). A group of chiropractors has shown that the work of chiropractors provides no benefit over physical therapy.

(Photo: Wikimedia)

Commentary
Like homeopathic remedies (a rant for another day), many have suggested that sublaxation is ineffective. Simon Singh was sued for libel for writing an article highlighting the ineffectiveness of chiropractors (the UK has "interesting" libel laws).

However, Harriet Hall, of the Science-Based Medicine blog, suggests that the "death knell" of chiropractic may have been rung. It is one thing for members of a profession to be introspective; it is another for them systematically refute the entire basis of their own profession. In the study they published, the chiropractors used a standard method for establishing causation known as Hill's Criteria. The criteria show the possibility of a causal relationship; not that it definitely exists. Yet, subluxation fails to satisfy any of the criteria. Sounds pretty bad for a field that's existed for 114 years.

Meta
What other professions should apply similar systematic rigor to their claims?

See Also

• The Dilute Homeopathy