27 Comments
author

Hi Bob: Thats excellent. :)

Expand full comment

It depends on the measure. Some of the cognitive measures are not PROMIS but over time have been added from other patient-reported outcomes measure. So, some might be diagnostic rather than for monitoring. Prof Gershon is most knowledgeable, and can hook you up with a colleague to help walk you through the process - perhaps even apply for an NIH grant to help fund your effort.

Expand full comment

I would start where the work has already been done and validated; the NIH Patient Reported Outcomes Measurement Information System (PROMIS). All the measurement scales have been highly validates and are in wide use. PROMIS was conceived as a minimally invasive series of questions that apply Computerized Adaptive Testing (CAT) technology. The advantage to CAT is that after the initial item, subsequent items are presented as informed by the previous items. Thus, you get a fast and valid measure with fewer questions and minimal time. That not only increases participation rates in research, but it frees time for more comprehensive measures seldom addressed due to questionnaire length.

PROMIS Health Measures also offer mobile phone, table, PC and paper data collection methods. Of course the paper method is lower in validity as items cannot be adaptive to prior responses. The list of available measures (scales) is available at https://www.healthmeasures.net/explore-measurement-systems/promis/intro-to-promis/list-of-adult-measures .

As I was on the team that conceived of PROMIS (the first ever project funded by all NIH institutes and centers), our goal was to create a standard to enable more valid epidemiological and recovery research by using a common metric. It also, for the first time, enabled long-term tracking of symptoms that not only assessed measures such as "How much does it hurt when you do this?" type items, but also quality of life items seldom included in clinical trials and recovery research such as "To what extend can you shop for groceries on your own?" or "How hard is to put on your socks in the morning?"

PROMIS offers an abundance of measures addressing cognitive function as well as daily-life functioning which should be ideal for detecting early cognitive deterioration.

An additional bonus with PROMIS is that the item pools used to assemble and test measures were derived from a century of research. Items pools were winnowed down by psychometric analyses (often hundreds of items per scale) and reduced via repeated field tests until there was an acceptable balance between reliability, validity, and CAT requirements. The beauty of that expensive strategy is that it means most scales in PROMIS correlate highly with widely-used standardized paper-and-pencil/interview measures - a bonus for people engaged in meta-analyses of large numbers of published studies over the years.

Admittedly, NIH has done a mediocre job of promoting PROMIS and related measures developed with NIH grant support. I can attest that PROMIS involved over 2 dozen research centers across the country, with a list of principle investigators who qualify as an international Who's Who of psychometricians. It is available in a host of foreign language versions. My estimate is that NIH invested about $200 million dollars in PROMIS before its initial release to the health research community at large. So, as I initially asserted, the work has already been done. All that is required is selecting germane measures, assembling a battery, and deciding the best way to administer and computer-score the responses.

PROMIS and related measure are administered by Northwestern University Med School in Chicago. It might be best to start with a call to Prof Richard Gershon (312) 503-3453) to get help in finding the ideal point of contact for your project. Tell him Tom Hilton sent you :-)

Footnote, Prof. David Eagleman at Stanford as he can help identify neurological processes most germane to early changes in adult cognition. Got to https://eagleman.com/about-david-eagleman/ for contact information - he is a bit of a celebrity and has a busy schedule.

Expand full comment
author

Hi Tom: Many thanks for this. It is extremely interesting. We will definitely explore further.

Took a quick look at the links. It looks like there is an iPad App already, which is interesting. It seems it might be targeted at a situation where a clinician "administers it", but perhaps there are ways to make it self administered. Looking forward to learning more.

Expand full comment

We designed PROMIS to be self-report. The only reason to use an interview is if the patient cannot easily self-report. Thus, most scales can self-score in only 5 or 6 items, then move on. There have been numerous use of PROMIS via smartphones for people in recovery status. Tablets seem to be popular in clinical office waiting rooms.

Desktops applications offer the advantage of not having to leave one's domicile. I think for elderly respondents, smartphones are likely not ideal. Not only do they require good vision, but high levels of digital dexterity in addition to knowing how smartphones work besides making phone calls. Desktop is not much different from logging in to email. My late mom used to refer to her desktop PC as "the email." Also, we had the display up 150% so she could easily read it. Somebody at Northwestern - graduate student perhaps - can help set up a system and provide feedback to the respondent and/or their healthcare professional.

Expand full comment

Tom, Let me also thank you for this great information. I just started looking at it and was drawn to a few of the Cognitive Function Short Forms. These could relatively easily be adapted to being self-administered via a vocal interface. The few I've looked at so far seem to rely on the individual's self-awareness of some issues, whereas some of the other tools I've identified ask the individual to perform mental tasks, e.g., repeating a phrase that they are previously told. To your knowledge is one type preferable for a self-administered screening tool?

I will try to reach out to Professors Gershon & Eagleman this week and will be mentioning your name!

Thanks again,

Bob

Expand full comment

Voice Assistant System Research mailto:vas@unc.edu

Expand full comment

I’m proud to announce that Amazon certified my Alexa Skill, Brain Aware, that I had been talking about creating in previous posts within 2 days of submission. It is now available for anyone to use without charge on their Alexa/Echo/Firestick devices by enabling it with the Alexa App on their smartphone. Brain Aware allows the user to self-assess and log their own perceptions of their thinking abilities in six different areas or domains over the previous seven days. The concept is for them to periodically repeat this self-assessment so they can ask to hear score comparisons in the future. If they notice a trend of concern, they can then bring it to the attention of their physician. The user can even set reminders to do those reassessments from time to time.

Many thanks to Tom Hilton for his suggestions and support for my project. Also, many thanks to the staff at the Department of Medical Social Sciences at Northwestern University Feinberg School of Medicine who worked with me and granted me permission to use the PROMIS Item Bank v2.0 – Cognitive Function – Abilities Subset. The process took quite a few weeks, but I believe that was due to the fact that my request was rather unusual.

I hope some of you give Brain Aware a try!

Expand full comment

I just wanted to let you know that the "Braintest" app that you recommended requires a touch-screen device. I jumped through all the hoops to sign up for the "free trial," including giving them my email address and my credit card, only to learn that I cannot use this app with a desktop computer that only has a keyboard and a mouse. I think that you, or the Braintest website, should have made that clear before I provided all this information.

Expand full comment

It’s been a while since I’ve posted, but for those of you who may be interested in my progress with developing a voice application (Skill) that a user can self-administer on an Amazon Alexa or Echo device, I thought I’d post an update.

Initially, I was working with the 6 Item Cognitive Impairment Test and was able to program it to ask 6 questions and score the responses. It worked great when I tested it, however, I was concerned that if there was any hesitancy in responding by the user, it wouldn’t work quite so well. While I haven’t totally given up on that idea, I think that using a tool that asks the user to assess their perceptions of how their thinking has been in several areas/domains may be preferable. So, I have programmed a voice application to administer one of the PROMIS Item Bank v2.0, Cognitive Function, Abilities Subsets. While my understanding is that it would be free to use for the purpose I have in mind, I am waiting for final approval and documentation that I can share with Amazon from contacts I’ve identified through the Healthmeasures.net website. Once I get that, I’ll be submitting it to Amazon for certification. My plan is for it to be free to the user.

Thanks to everyone for your support & input, especially Tom Hilton.

Hopefully, I’ll be able to post something positive soon! Wish me luck!

Expand full comment

Over the last several days I've been developing an application (Skill) for the Amazon Alexa/Echo devices through which the user in their home can take an adaptation of the Six Item Cognitive Impairment Test (6CIT). I still have a lot of work to do to try to ensure that the average user can take the test. One of the challenges is that Alexa doesn't allow too long of a pause before it stops listening to your answer. I have some ideas how to work around that.

In reviewing some of the other tests, especially the PROMIS ones, I noted that, unlike the 6CIT which actually tests a person's cognitive abilities, several of them ask for the user's perception of how certain of their thinking abilities have been within the last 7 days. I am considering incorporating one of those also in the voice application I am developing. What do others think of that idea?

I am hoping to connect with some of the experts in this area that Tom Hilton has suggested.

Looking forward to any feedback. There is still a lot to do. I will keep you updated.

Expand full comment
author
Aug 6, 2023·edited Aug 6, 2023Author

Cool. :)

Sounds like getting some additional experts to weigh in would be very interesting. Look forward to hearing what you learn.

One extra thought, based on something that occurred to us as we tried out the AARP product. I think it makes a lot of difference where the data ends up and how well attached it is to your personal data. I realized I was unenthusiastic about the idea of info about my mental state ending up in the data base of some entity keen to use it to place ads.

With your Alexa app, can you find a way to keep the data out of the hands of third parties?

Expand full comment

Sorry I missed this until now. Bottom line is that the information is stored in a database I control and will not be given/sold to any third party.

The self-assessment results get stored in a MySQL relational database. Let me explain what gets stored - Anyone who uses an Alexa/Echo device must have an Amazon account which has a unique ID. The program I write accesses that ID and stores it. The unique part of that account number is over 200 characters long. In order for more than one person in a household to be able to track their separate results, I ask each user to register with a first name. That name doesn't have to be their real name. It is just the name they will use when using the Skill/Application. As far as scores, the user is asked to reflect over the past 7 days and score how well their thinking has been regarding 6 abilities such as memory & speed. They give a score from 1 to 5 with 5 being the most positive response. They have the option to log their scores so they can check what they were in the future.

As I said, this information is kept private and as part of the submission process, I will be writing a Privacy Policy that the person considering enabling the Skill will be able to view.

I hope this answers any of the questions regarding the privacy of collected information.

Expand full comment

In addition to the comment that I posted earlier, I wanted to ask whether the Explorers are considering self-administered assessments that can be done also via the voice interface. This approach would benefit those who have challenges with manual dexterity. My experience as a retired physician who has created over 30 voice applications for the Amazon Alexa/Echo devices has started the creative juices flowing and I'm researching possible screening tools that may be adapted to the vocal interface. I'm currently looking at the 6-Item Cognitive Impairment Test (6CIT). We'll see where this goes. I'll be anxious to learn of any thoughts the Explorers might have about this approach.

Thanks!

Bob

Expand full comment
author

Sounds intriguing. Dont know of any such efforts yet. Have seen projects trying to use voice itself as an early indicator of dementia, but that is different to your idea. Some of the tests (eg remembering 3 words) on the standard tests seem easily suited to a voice interface. Some, like interpreting visual shapes, or creating a clock face, seem less obviously suited. But maybe with a deeper understanding than we have of the science behind the tests something may present itself. Please keep us in the loop. :)

Expand full comment

Yes, not all the measurement tools would lend themselves to a voice interface, especially ones measuring visuo-spatial ability. The 6CIT seems like a possible candidate. I plan to keep looking. Based on another comment, I'll probably look at PROMIS next.

I'll keep you all updated and will continue checking these posts for any other ideas.

Expand full comment

I really enjoyed this update as well as these comments. I plan to check out what AARP has to offer. I did want to mention what Chat-GPT suggested when I asked it about cognitive assessment tools to see if it identified any that the team at Tech-enhanced Life hadn't considered:

There are several self-assessment tools that can help you detect decline in cognition. Some of them are:

- The **10-point cognitive screener (10-CS)**, which is a brief test that assesses orientation, memory, and executive function¹.

- The **6-item cognitive impairment test (6CIT)**, which is a simple test that measures orientation, memory, and concentration¹.

- The **6-item screener**, which is a short test that evaluates memory and orientation⁴.

- The **Memory Impairment Screen (MIS)**, which is a four-minute test that assesses verbal memory⁴.

- The **Mini-Cog**, which is a three-minute test that combines a clock drawing task and a three-item recall task⁴.

- The **Test Your Memory (TYM)**, which is a self-administered test that covers various cognitive domains such as orientation, language, calculation, and visuospatial skills¹.

- The **Self-Administered Gerocognitive Exam (SAGE)**, which is a pen-and-paper test that evaluates thinking abilities such as orientation, memory, problem solving, and language⁴.

These tests are not diagnostic tools, but they can help you identify areas of concern and discuss them with your doctor. You can find more information about these tests and how to access them online or in print on the websites of the sources I have referenced below. I hope this helps you. 😊

¹: [Assessment | Diagnosis | Dementia | CKS | NICE](https://cks.nice.org.uk/topics/dementia/diagnosis/assessment/)

⁴: [At-Home SAGE Test for Alzheimer's and Dementia Detection - Everyday Health](https://www.everydayhealth.com/alzheimers-disease/all-about-the-sage-test-for-alzheimers-and-dementia-detection/)

Thanks,

Bob

Expand full comment
author

Hi Bob: Glad you liked it. Thx for the ChatGPT inputs. We updated accordingly. Need to get in the habit of checking ChatGPT before pushing publish. :)

Expand full comment

I have never been a great fan of AARP, but they are offering a new service which seems to check almost all of the boxes of what is desired from a cognitive monitoring tool. AARP Staying Sharp is a new AARP program focused on brain health. It offers online cognitive assessment tools, background research, and training to develop specific skills. The tests measure five cognitive attributes: working memory, cognitive flexibility, recognition memory, sustained attention, and processing speed. Your score is based on comparisons with others of similar age, gender, and years of education. Each cognitive attribute is associated with specific daily activities. For example, processing speed helps with

* Driving safely by responding to changes in traffic and road conditions

* Reading quickly and accurately

* Keeping up with a conversation

* Doing complex math in your head

You can retake the self-assessments every 30 days to monitor progress. The site also offers life-style check-ins in six “pillars of brain health”: (exercise, sleep, eating, social, stress, brain engagement). It seems well designed and quite comprehensive. It does not require a touch screen, but a mouse is probably essential. (It’s not clear how you enroll.  I think I was somehow invited through my UHC Medicare Advantage plan and have not been asked to pay anything. It looks like AARP members can enroll for a one-time fee of $14.)

https://stayingsharp.aarp.org/?referrer=https://stayingsharp.aarp.org/

Expand full comment

RE the SAGE results - is there only one broad result reported e.g. Normal, or are there separate numerical scores for each area of cognition tested; memory, attention, visual/spatial, etc.

I would be happy to take the test several times at one sitting if the results could gain better statistical significance.

Expand full comment
author

Ken: You get a "score" (eg 20/22). And in addition you get an interpretation by the clinician of that score (eg "most likely normal").

Expand full comment

I think the effects of inflammation need to be incorporated in these studies.

I developed Long Covid couple years ago with various neurological effects, including, extreme hyperglycemia, dizziness, partial loss of hearing and memory lapse.

Using myself and a few other long haulers, I started to develop a model using 20 protein blood biomarkers to correlate the effects of inflammatory responses on different organs.

Among other things, the model uses the effect of specific proinflammatory responses on degradation of ACETYLCHOLINE and its impact on vagus nerve signaling with organs such as pancreas, kidney, lung and other systems.

We are still in the startup phase but we are looking for others to collaborate with us to expand our model. Feel free to reach me at: AttaPilram@OPTIMYZI.com

Expand full comment

There is a university based research study that has a free online component testing cognitive areas. If you opt into the study (as I did) you can also choose whether to submit a blood sample for DNA purposes. It is called MindCrowd and is sponsored by TGen, and has a primary researcher that is University based. I found the online games fun and easy to do, and my immediate score was given. It also showed me the aggregate average score of others (not identified, just the average score of all others) so I could compare for myself. Some of the repetitive motion on my laptop keyboard using my index finger slowed me down, and I later learned to switch to using thumbs! It is studying the link with the online tests, Alzheimer's, and genetics if I understand correctly. I believe that I will be able to continue to take the tests in the future. Here is the link if anyone wants to look into it.

https://mindcrowd.org/about-mindcrowd-study/

Expand full comment

What can I do that would be meaningful once my brain light come on? I need motivation to do the tests other than to just know the light came on.

Expand full comment
author

Hi Pete: I definitely agree. Stay tuned for future installments as we explore this issue as well as how to measure cognition trends. :)

Expand full comment

The cognitive exams you listed, usually by health providers, all have an associated cost. I would suggest considering the St. Louis University Mental Status Exam. (SLUMS). Developed by Saint Louis U in conjunction with the VA, it is freely available and has good data to back it up. As a geriatrician, I used it almost daily as did the other providers in my department.

https://en.wikipedia.org/wiki/Saint_Louis_University_Mental_Status_Exam

Expand full comment
author

Thx for this suggestion. We will add this to the list. However, I am wondering if this can be done electronically? It looks like it might be limited to being administered by a clinician perhaps? Nothing wrong with that for the purposes you describe of course, but for this exploration we wanted something that could be done remotely.

Expand full comment