Do Biomarkers Improve Diagnostic Accuracy?

It’s a scenario familiar to many clinicians: A patient diagnosed with mild cognitive impairment asks if he or she will eventually develop Alzheimer’s disease. Can fluid or imaging biomarkers help clinicians fine-tune their prognoses? Maybe not, according to a paper in the June 4 BMJ Open. Researchers led by Edo Richard, University of Amsterdam, the Netherlands, retrospectively studied participants in the Alzheimer’s Disease Neuroimaging Initiative (ADNI) and found that neither atrophy of the entorhinal cortex nor the ratio of phosphorylated tau to Aβ42 in the cerebrospinal fluid improved diagnosis. “Clinicians should think twice about whether these diagnostic instruments will increase the accuracy of predicting who will progress to Alzheimer’s disease,” Richard told Alzforum. Other researchers pointed out limitations in the study that may temper that idea.

Since revised diagnostic criteria for Alzheimer's disease co-opted biomarkers, they have been increasingly used in research and in clinical trials (see ARF related news story and ARF news story). There has also been a push to get biomarkers into the clinic. In August 2012, the European Federation of Neurological Societies taskforce recommended that everyone with a potential diagnosis of dementia undergo structural MRI to rule out other disorders (see Filippi et al., 2012) and noted that CSF measures may support an AD diagnosis (see Hort et al., 2010). In April 2012, The FDA approved 18F-florbetapir (Amyvid®) for imaging of amyloid deposition in the brain (see ARF related news story).

However, few studies have examined the usefulness of biomarkers for individuals in daily clinical use, Richard told Alzforum. He added that studies on diagnostics typically treat all biomarkers equally, rather than looking at their predictive value when applied sequentially. Richard and colleagues wanted to simulate the situation in the clinic, where a cognitive test was administered first to see if biomarkers could then improve prognosis about who would or would not progress from mild cognitive impairment (MCI) to AD.

In ADNI, 181 participants with MCI had complete neurological assessments as well as CSF and MRI data. Over an average of 39 months, 81 of those volunteers progressed to AD. To test how biomarkers influenced that diagnosis, Richard and colleagues looked at a trio of measures: the ratio of phosphorylated tau to Aβ42 in the CSF, the entorhinal cortex volume on MRI scans, and Rey’s Auditory Verbal Learning Test (RAVLT), in which participants recollect as many words as they can from a list of 15. Researchers from Richard’s group previously found that these three measures were particularly sensitive for diagnosing AD and MCI (see Schmand et al., 2011).

When analyzed separately, all three measures predicted who would progress to AD with about the same degree of accuracy. However, if the researchers started with the learning test data, the other markers added little predictive value. RAVLT alone correctly categorized 65.7 percent of subjects. The MRI results improved that by 1.1 percent. CSF measures actually detracted from it by 2.2 percent, because they introduced false positives. The authors did not test whether these biomarkers worked together to boost accuracy, since doctors rarely have access to both measurements in daily clinical practice.

What does this mean for biomarker use in the clinic? For many with MCI, these biomarker tests offer no added value over a simple memory exam, Richard told Alzforum. One reason could be the large variability in entorhinal cortex volume and CSF measures among individuals, making it hard to define cutoff values that suggest disease.

The authors point out that the data may poorly extrapolate to the general population. The ADNI participants are a highly selective group and are free from cerebrovascular disease, depression, psychiatric disease, and substance abuse problems. In a more varied population, MRI may offer more predictive accuracy because it can rule out non-AD causes for dementia, for instance, in patients with suspected cerebrovascular damage. Likewise, CSF analysis can rule out AD pathology in unusual cases, such as suspected prion disease or meningoencephalitis, where unusual proteins, glucose levels, blood cell profiles, and pathogens in the CSF signal trouble, Richard said. Though he has no plans to extend this research, he said that a similar study in a less selective cohort would be required to validate the results.

Many researchers contacted by Alzforum were skeptical of the main findings. Clifford Jack, Mayo Clinic, Rochester, Minnesota, cautioned that while the RAVLT did not figure directly into their MCI classification, ADNI subjects were selected on a similar verbal-recall test. It is no surprise, then, that the RAVLT best predicted outcomes, he told Alzforum. “This circularity has significant implications for how the study results are interpreted,” he said. Several scientists echoed that concern in e-mails to Alzforum. Richard maintained that this potential source of bias does not disqualify the results. “This study still answers the clinically relevant question about how to diagnose a patient with MCI in the outpatient clinic,” he wrote to Alzforum in an e-mail.

Jack also noted that the ADNI researchers selected patients into the MCI cohort based on the poorest performance on memory tests in order to observe those closest to dementia. That means these results, and others based on ADNI data, may poorly represent earlier stages of MCI, when memory impairment is less obvious, he said. Stephanie Vos, Maastricht University, the Netherlands, agreed, pointing out that CSF and MRI might be more useful in earlier stages when certain biomarkers begin to change in the absence of cognitive decline. Jack added that a memory test says nothing about the underlying etiology of the MCI, which may be important when the cognitive impairment is treatable, or even if the patient's family wishes to know it. Biomarkers can at least help rule out AD, he said.

Despite these limitations, the overall message of this study is important, wrote Murali Doraiswamy, Duke University, Durham, North Carolina, to Alzforum in an e-mail. “The Achilles' heel of most AD biomarkers is that their predictive value above and beyond a memory test is still not optimal,” he wrote. “We need to develop biomarkers with significant additive value.”

The authors did not include amyloid positron emission tomography imaging because it would have shrunk the available ADNI sample, said Richard. He added that the test is neither practical nor inexpensive enough to use widely on a large scale, so it would not have fit with the pragmatic, clinical aspect of the study.—Gwyneth Dickey Zakaib

Comments

  1. This study by Richard et al. provides important findings on the diagnostic benefit of combining neuropsychological memory assessment with biomarkers for predicting AD dementia in patients with amnestic MCI.

    The authors conclude that biomarkers are not useful to improve predictive accuracy once memory test performance has been accounted for. Since memory tests are less expensive than biomarker assessment, neuropsychological tests might be preferred in clinical practice.

    The study has several strengths and supports the notion that the costs of biomarkers need to be weighed against their real benefit. It should be noted, however, that the current paper focused on a few measures, and none of the prediction models (biomarkers or memory tests) reached clinically relevant levels of accuracy (remaining below 80 percent). Thus, more sophisticated prediction models need to be developed, and biomarkers may still contribute to any improvement in the models.

    Other points to consider are the predictive value with respect to differentiating AD from other types of dementia, which was not assessed in the current study. Furthermore, it remains to be tested whether biomarkers are beneficial at the very early stage of AD, when no or only very slight cognitive impairment is present.

    View all comments by Michael Ewers

  2. The findings of this study are not that surprising—baseline cognition is the biggest predictor of cognitive progression in MCI. The RAVLT is the most sensitive test at the MCI stage simply because verbal recall is the key criterion used to define MCI patients for entry.

    The Achilles' heel of most AD biomarkers is that their additive predictive value above and beyond a memory test is still not optimal. This is where we need to improve our tests. That said, pathology biomarkers have a high negative predictive value. MCI can be due to 20 different types of brain pathology, all yielding memory test scores in the same range; a negative brain scan can help clinicians rule out the likelihood of AD. In addition, memory tests are less useful at predicting future disease in people who have perfectly normal memory, whereas biomarkers can identify silent pathology. For example, in a recent multicenter study, amyloid PET significantly predicted an 18-month cognitive decline in normal subjects, above and beyond their baseline memory test score (see Doraiswamy et al., 2012).

    The overall message of the paper is a good one—that we need to develop biomarkers with significant additive value above and beyond simple memory tests.

    References:

    Doraiswamy PM, Sperling RA, Coleman RE, Johnson KA, Reiman EM, Davis MD, Grundman M, Sabbagh MN, Sadowsky CH, Fleisher AS, Carpenter A, Clark CM, Joshi AD, Mintun MA, Skovronsky DM, Pontecorvo MJ, . Amyloid-β assessed by florbetapir F 18 PET and 18-month cognitive decline: a multicenter study. Neurology. 2012 Oct 16;79(16):1636-44. PubMed.

    View all comments by P. Murali Doraiswamy

Make a Comment

To make a comment you must login or register.

References

News Citations

  1. AD Diagnosis: Time for Biomarkers to Weigh In?
  2. FDA Approves Amyvid for Clinical Use

Paper Citations

  1. Filippi M, Agosta F, Barkhof F, Dubois B, Fox NC, Frisoni GB, Jack CR, Johannsen P, Miller BL, Nestor PJ, Scheltens P, Sorbi S, Teipel S, Thompson PM, Wahlund LO. EFNS task force: the use of neuroimaging in the diagnosis of dementia. Eur J Neurol. 2012 Aug 20; PubMed.
  2. Hort J, O'Brien JT, Gainotti G, Pirttila T, Popescu BO, Rektorova I, Sorbi S, Scheltens P, . EFNS guidelines for the diagnosis and management of Alzheimer's disease. Eur J Neurol. 2010 Oct;17(10):1236-48. PubMed.
  3. Schmand B, Eikelenboom P, Van Gool WA, . Value of neuropsychological tests, neuroimaging, and biomarkers for diagnosing Alzheimer's disease in younger and older age cohorts. J Am Geriatr Soc. 2011 Sep;59(9):1705-10. PubMed.

Other Citations

  1. ARF news story

External Citations

  1. Alzheimer’s Disease Neuroimaging Initiative

Further Reading

Papers

  1. Dubois B, Feldman HH, Jacova C, Dekosky ST, Barberger-Gateau P, Cummings J, Delacourte A, Galasko D, Gauthier S, Jicha G, Meguro K, O'brien J, Pasquier F, Robert P, Rossor M, Salloway S, Stern Y, Visser PJ, Scheltens P. Research criteria for the diagnosis of Alzheimer's disease: revising the NINCDS-ADRDA criteria. Lancet Neurol. 2007 Aug;6(8):734-46. PubMed.
  2. Dubois B, Feldman HH, Jacova C, Cummings JL, Dekosky ST, Barberger-Gateau P, Delacourte A, Frisoni G, Fox NC, Galasko D, Gauthier S, Hampel H, Jicha GA, Meguro K, O'brien J, Pasquier F, Robert P, Rossor M, Salloway S, Sarazin M, de Souza LC, Stern Y, Visser PJ, Scheltens P. Revising the definition of Alzheimer's disease: a new lexicon. Lancet Neurol. 2010 Nov;9(11):1118-27. PubMed.
  3. Schott JM, Warren JD, Barkhof F, Rossor MN, Fox NC. Suspected early dementia. BMJ. 2011;343:d5568. PubMed.
  4. Hort J, O'Brien JT, Gainotti G, Pirttila T, Popescu BO, Rektorova I, Sorbi S, Scheltens P, . EFNS guidelines for the diagnosis and management of Alzheimer's disease. Eur J Neurol. 2010 Oct;17(10):1236-48. PubMed.
  5. Schmand B, Eikelenboom P, Van Gool WA, . Value of neuropsychological tests, neuroimaging, and biomarkers for diagnosing Alzheimer's disease in younger and older age cohorts. J Am Geriatr Soc. 2011 Sep;59(9):1705-10. PubMed.
  6. van Rossum IA, Vos S, Handels R, Visser PJ. Biomarkers as predictors for conversion from mild cognitive impairment to Alzheimer-type dementia: implications for trial design. J Alzheimers Dis. 2010;20(3):881-91. PubMed.
  7. Chong MS, Sahadevan S. Preclinical Alzheimer's disease: diagnosis and prediction of progression. Lancet Neurol. 2005 Sep;4(9):576-9. PubMed.

Primary Papers

  1. Richard E, Schmand BA, Eikelenboom P, Van Gool WA, . MRI and cerebrospinal fluid biomarkers for predicting progression to Alzheimer's disease in patients with mild cognitive impairment: a diagnostic accuracy study. BMJ Open. 2013;3(6) PubMed.

Annotate

To make an annotation you must Login or Register.