This post is based on a submission by reader Professor Daniel Bitran. Please submit suggestions for posts to metaist.blog@gmail.com.
Definitions
fMRI is a way of measuring blood-flow in the brain or spinal cord and, by extension, neural activity in those areas.
A
false positive is the sort of mistake your smoke detector makes when it goes off, but there's no smoke.
Summary
Despite the widespread use of fMRI, a few false positives may result in inaccurate results.
The fMRI is showing neural activity, but the salmon is definitely dead.
(Image: Courtesy of Prefrontal.org)
The image above is striking because the false positives seem to show neural activity in a dead salmon's brain. According to
researchers at UCLA Santa Barbra, these errors are due to a problem of
multiple comparisons.
Imagine we're playing
One of These Things is not Like the Others with several tin cans of Atlantic salmons. At first, it's hard to tell which one is not like the others -- there's a bunch of canned salmon. They have similar color, weight, shape, etc. But as we add different ways of comparing the cans (or more cans to compare), we increase the probability that there will be
some way in which one of them differs from the rest -- particularly because of small differences, say manufacturing defects.
[
Note: Corrections appreciated.] Now imagine we're collecting data for an fMRI. Each each point (called a
voxel) is measured several times with certain extreme values discarded. Now we want to figure out which of the voxels is not like the others -- that's because that's where we expect to see
differences in blood flow. However, by comparing voxels we're actually comparing multiple measurements of each voxel to multiple measurements of other voxels. This is like adding more ways of comparing the cans. Moreover, because there is a small bit of noise, the measurements for each voxel can be slightly different each time. This is why we occasionally find
some difference between neighboring voxels that isn't really there -- it's a false positive. Luckily there are ways of
correcting for this sort of error, but unfortunately, it is not applied as frequently as it should.
Meta
What are other examples of widespread errors of multiple comparisons or false positives?
Acknowledgements
Thanks to
Craig Bennett of
Prefrontal.org for providing a high resolution version of the Atlantic salmon fMRI.
See Also
indicates a link to a different website.
