23 Aug 2018

Could learning Chinese make a game out of cognitive testing—all the while spotting earlier stages of decline within a week’s time? Perhaps, according to work presented at the Alzheimer’s Association International Conference, held July 20–26 in Chicago. When Jenalle Baker of the Florey Institute, Melbourne, Australia, challenged older adults to memorize the English equivalents of 50 Chinese characters, within one week she started to detect subtle learning problems tied to brain amyloid deposition that would take a year to discern with traditional tests. Also at the meeting, other researchers showed off how smartphone and digital pen technology can simplify and improve cognitive testing aimed at detecting the smallest of dips in mental function.

With the growing use of amyloid PET scanning, researchers discovered that as people accumulate amyloid, their cognitive power wanes long before dementia appears (Feb 2018 news; Baker et al., 2017; Aug 2018 conference news). But the decrements are small, and take more time to detect than is practical for clinical trials. That is a serious problem for early stage therapeutic studies, which need faster cognitive outcome measures.

For example, Baker, who works with Yen Ying Lim at Florey Institute, showed data from the longitudinal Australian Imaging, Biomarker & Lifestyle Study of Ageing (AIBL). In that cohort, amyloid-positive and -negative, cognitively normal people start out performing the same on a learning and memory task. The test asked participants to learn and recall the locations of abstract shapes on a grid, a measure of paired association learning that is sensitive to the disruption of hippocampal function in early AD. In a single session, both amyloid-positive and -negative people learned quickly and got better with practice, improving their scores equally on repeated trials given on a single day. But in the following years, the groups diverged: Amyloid-negative people continued to improve on the test, while scores for amyloid-positive people stayed flat. Clearly, the amyloid-positive people were worse at acquiring new information. Baker’s problem? It took a year for the difference between the groups to become significant.

Baker wondered if she could devise a test to pick up that difference sooner. She developed the Online Repeated Cognitive Assessment (ORCA), a more challenging version of paired association. ORCA asks volunteers to learn to match 50 Chinese characters with their meaning, spoken in English. People did this at home, via the Internet, 25 minutes per day for six days.  The outcome was the number of correct and incorrect guesses. All data went to a central server for analysis.

Baker evaluated the test in 30 amyloid-negative and 20 amyloid-positive, cognitively normal AIBL participants. Both groups were equal in average age, hippocampal volume, and scores on established cognitive tests.

Over the six-day testing period, all participants learned to associate the words with the characters, but the amyloid-positive group made significantly more mistakes than their amyloid-negative counterparts. The difference emerged by the second day of testing, and grew in succeeding days. By day six, the magnitude of the impairment was large, with the amyloid-positive group showing an effect size of 1.69 relative to the amyloid-negative group. That is a far greater and faster difference than traditional pen-and-paper measures, where effect sizes even after a year stay within 0.15 to 0.3 (Baker et al., 2017). “ORCA takes one week, not one year,” Baker said.

The result needs to be repeated in larger and independent samples, but Baker thinks the data look promising. ORCA may be suitable for widespread screening for clinical trial candidates, or to triage people for diagnostic follow-up. Baker said preliminary results with this initial sample of 50 people suggest that the test can discriminate amyloid positivity with 70–80 percent accuracy and a high test-retest reliability of 0.9. “We want to move toward establishing a cutoff score, but need to test more participants first,” she said. With more data, ORCA could also be used to evaluate treatment effects, she suggested.

Baker said it will be important to determine whether a person’s performance on ORCA is influenced by cognitive enhancers or other medications older people take, or by sleep quality, or genetic variants such as ApoE4.

One audience member questioned how the researchers keep people engaged in this regimen, which would seem to take considerable effort on the part of participants. Lim replied that, au contraire, participants reported enjoying taking this test. She believes this is because the test does not rely on abstract symbols. “It has a real-world aspect that drives engagement,” Lim said. One participant told Baker that, while walking in Melbourne’s Chinese district, he recognized the character for noodle on a restaurant sign. “People feel like they are learning something,” Baker said.

This format is only useful in English speakers who know no Chinese, which rules out readers of Japanese, as well, as the language uses many of the same pictographs. Lim told Alzforum her group is trying to develop tests with other pairings, such as birds and their songs.

The test is available online, and a mobile phone app is under development, Lim said, adding that interested researchers should contact her.

Reality Check. Top graph shows tidy, but imaginary, results of cognitive testing of amyloid-positive (black) and –negative (blue) adults, versus the bottom graph of real-world data. Could a smart phone app help clean up the mess? [Courtesy of Jason Hassenstab.]

Jason Hassenstab of Washington University in St. Louis updated the AAIC audience on Ambulatory Research in Cognition, aka ARC, his group’s smartphone-based testing model. Hassenstab devised this app to tame the huge variability seen with in-clinic cognitive testing. He also aims to replace the exhausting and anxiety-producing in-person testing sessions in the clinic with frequent, short bursts of testing embedded in participants’ daily routine. To do that, Hassenstab has developed an app that allows brief bouts of testing multiple times a day for a week, on a participant’s own phone. “Our strategy is to test often and everywhere, to keep it short, and to combine results of many sessions,” Hassenstab said.

The ARC testing battery includes three tasks that assess working memory, processing speed, and associative memory. Participants complete each one, every day, in four short sessions, and do that for a week. Hassenstab has begun to compare the ARC results to in-clinic tests in Dominantly Inherited Alzheimer’s Network (DIAN) members, and finds that the correlations, while weak, go in the right direction. The ARC tests are highly sensitive to age.

As presented previously, this kind of “burst testing” increases the within-person reproducibility between test sessions (Dec 2017 conference news). In-clinic testing yields retest reliabilities ranging from 0.5 to 0.7, which would be unacceptably low for any other biomarker, Hassenstab noted. His goal was a figure of 0.9 or higher. As of last year, the scientists had not quite reached this goal. But at AAIC, Hassenstab presented a few tweaks to the app, and now, after seven days and 20–22 tests, the tests make the mark on all three tasks. “We can now do as well or better than anything in the clinic,” Hassenstab said.

How do ARC scores relate to brain amyloid and tau deposition? Hassenstab showed new data on a small sample of 39 cognitively normal adults, where he found neither the ARC composite score nor in-clinic testing scores correlated with CSF Aβ42. ARC scores, but not in-clinic tests, did correlate with CSF total tau.

As ARC is being refined, a key aspect continues to pose problems, Hassenstab said. Ideally, participants should take the tests on their own cell phones, but standardizing them on a plethora of devices and operating systems is a challenge. To do that, Hassenstab’s group has developed robots to determine a key variable between devices, which is the time it takes for a screen tap to be recorded in the processor. After testing hundreds of different phones and operating systems, they have learned which phones are suitable and which are not. “Now we can say, you can use this phone, you can’t use that phone,” Hassenstab said.

Robots aside, a more important question may be how ARC and ORCA compare. ORCA features a large number of items, while ARC has fewer items but administers them rapidly at higher frequency. Baker said, “The two complement each other, and we think we can get some cool data going if we combine efforts.” Hassenstab agreed, saying the groups are already collaborating. “These tests are measuring different things—one is comparing individual learning curves, the other is testing cognition in different environments and situations,” he said.

Yet another group, represented by Kate Papp from Brigham and Women’s Hospital, Boston, presented new data on the clock-drawing test. This time-honored, if dated, cognitive assessment sparked renewed interest when it migrated from paper to digital capture (Dec 2012 news; Dec 2017 conference news). In the traditional form, patients either free-draw or copy the face of an analog clock, with the hands set at 10 minutes past 11, and clinicians score the drawing. In the digital version Papp used, patients draw the clock with a recording pen, which marks its own position every 1.2 milliseconds, generating a continuous steam of data on drawing speed, pauses, location and so on. The data gets compared to a proprietary database of thousands of other drawings by people in different diagnostic groups, and converted to a composite score, ranging from 0 to 100.

The pen’s maker claims that the test, which takes less than five minutes, can tell cognitively normal people from those with MCI or AD. Nowadays, these three categories are no longer nuanced enough, so Papp decided to explore whether the digital pen detects more subtle impairment, and how scores relate to preclinical biomarkers.

Her study included 98 people from the Harvard Aging Brain Study who were cognitively normal and had PET scans for amyloid (PiB) and tau (flortaucipir). Their digital clock scores were unrelated to sex or education, but did decline with age. Digital clock performance correlated with performance in other tests of multiple cognitive domains, including processing speed, executive function and memory scores. This suggests the test taps into multiple aspects of cognitive function, Papp said.

The digital clock also picked out subtle cognitive decrements, differentiating people with normal from those with impaired performance on the Preclinical Alzheimer Cognitive Composite (PACC), a measure of mild cognitive decline that is used as an outcome measure in the ongoing A4 secondary prevention trial of solanezumab.

Importantly, scores on the digital clock tracked with biomarkers of amyloid and tau deposition. PiB positivity was accompanied by lower clock-drawing scores, where the amyloid-negative people scored 72/100 and the amyloid-positive people 60/100, for an 18 percent reduction in mean score and an effect size of 1.48. Papp also detected a correlation with test scores when she treated brain amyloid as a continuous variable. Increasingly, researchers are searching for ways to measure the consequences of biomarkers as gradually changing factors in Alzheimer’s disease, not only by comparing groups with artificial demarcations. In a sensitivity and specificity analysis, the five-minute digital clock was as good as the 30-minute PACC at picking out amyloid-positive people, Papp said.

How about tau? Clock drawing was linked to regional tau PET signals in sites of early neurofibrillary tangle deposition in this clinically unimpaired group. Worse scores correlated with more tau in the inferior temporal lobe and entorhinal cortex. Moreover, participants who had both elevated amyloid and inferior temporal tau tended to have even worse digital-clock scores.

Time and Space. Tau deposition selectively affects clock-drawing features. Higher tau correlates with lower scores for proper vertical positioning of a copied clock (top), but not horizontal alignment (bottom). [Courtesy of Kate Papp.]

Going forward, Papp wants to dig more deeply into the wealth of data generated by the digital pen, and the more than 700 features analyzed by the software. For example, in this group, incorrect vertical alignment of a copied clock went hand in hand with higher entorhinal tau, but poor horizontal alignment did not. Does this kind of finding hint something about the underlying biology? Papp also found that the command or copy formats of the test are differentially related to tau deposition. That’s worth further study, she believes. “If these scores are related to biomarkers, we expect we are capturing meaningful information,” she said.

>The next step is to use machine learning to teach the data how to pick out amyloid or tau positivity in larger groups of people. This would open up the prospect of digital clock drawing as a screening tool in the general aging population. It may also be possible to use the digital clock as an outcome measure in prevention or treatment trials. Like the ORCA and ARC developers, Papp, too, needs much more data on how the test performs over time, she said. —Pat McCaffrey

Comments

No Available Comments

Make a Comment

To make a comment you must login or register.

References

News Citations

1. No Amyloid, No Memory Problem—Even with ApoE4? 2 Feb 2018
2. Memory Slips as Soon as Amyloid Appears, Two Decades Before Dementia 17 Aug 2018
3. Cognitive Testing Is Getting Faster and Better 9 Dec 2017
4. Will Technology Revolutionize Dementia Diagnosis and Care? 19 Dec 2012

Paper Citations

1. Baker JE, Lim YY, Pietrzak RH, Hassenstab J, Snyder PJ, Masters CL, Maruff P. Cognitive impairment and decline in cognitively normal older adults with high amyloid-β: A meta-analysis. Alzheimers Dement (Amst). 2017;6:108-121. Epub 2016 Oct 18 PubMed.

Further Reading

No Available Further Reading