SBIR-STTR Award

Fall Detection and Prevention for Memory Care Through Real-Time Artificial Intelligence Applied to Video
Award last edited on: 3/2/2021

Sponsored Program
SBIR
Awarding Agency
NIH : NIMHD
Total Award Amount
$1,137,266
Award Phase
2
Solicitation Topic Code
NIA
Principal Investigator
Glen Xiong

Company Information

Safelyyou Inc (AKA: NestSense Inc)

2935 Mlk Jr Way Unit C
Berkeley, CA 94703
   (713) 822-6924
   N/A
   www.safely-you.com
Location: Single
Congr. District: 13
County: Alameda

Phase I

Contract Number: 1R43AG058354-01
Start Date: 9/15/2017    Completed: 8/31/2018
Phase I year
2017
Phase I Amount
$150,000
In the US, Alzheimer’s disease (AD) is the single most expensive disease, the only disease in the top six for which the number of deaths is increasing. The greatest cost contributors are frequent hospitalizations, where falls are the largest culprit, and frequent need for assistance with the activities of daily living. A fall safety system shows the potential to reduce costs and increase quality of care by reducing the likelihood of emergency events (e.g., detecting falls before a fracture occurs and reducing the number of repeat falls). Unfortunately, current safety devices require wearable or sensor technology not suitable for individuals with dementia and incapable of showing caregivers how falls occur. Our goal is to deploy and demonstrate NestSense (also known as SafelyYou), an online fall detection system with off-the-shelf wall-mounted cameras to passively detect falls for patients with AD and related dementias (ADRD), enabled by a human-in-the-loop (HIL). The HIL will confirm the fall detection alerts provided by our artificial intelligence algorithms. We will demonstrate it for 100 patients in 13 memory care facilities. Memory care facilities can select parameters that matter for specific patients; for ex., some patients wake up frequently during the night while others should be assisted when they attempt to leave the bed at night. It does not require action of individuals / caregivers such as wearing a fall pendant and is therefore well-suited for individuals with ADRD. We leverage our HIL paradigm, in which our deep learning (a subfield of artificial intelligence) approaches identify and pre-filter falls well enough to leave the last check to a human, who will call the facilities in case of detected safety critical events (falls). The human can monitor several facilities at a given time. This project leverages the already recruited 100 patients in our partner 13 memory care facilities, recruited through our previous (IRB approved) pilot. The work will leverage our previous three pilots. • Pilot 1: We demonstrated the feasibility of the system by collecting a proof-of-concept data containing 200 acted falls of healthy subjects and showed accurate fall detection. • Pilot 2: We demonstrated acceptance of privacy/safety tradeoffs by patients, family and staff, through the collection of 3 months of video data at WindChime of Marin, a memory care facility from the Integral Senior Living network, in which we identified 4 total hours of fall data. This led to clinical benefits including a reduction of falls from 13 and 11 in the first 2 months to 2 in the final month, due to video review with care staff. • Pilot 3 (ongoing): We demonstrated scalability and further acceptance by deploying the system in 13 facilities of the Carlton, Integral Senior Living, Pacifica and SRG networks, totaling 100 patients already monitored by our system (offline). The pilot proposed for this SBIR Phase I will translate the 100 cameras in these facilities into a real-time fall detection system which will run online for 3 months with a 24/7 HIL support. Compared to a 3-month baseline from the 100 cameras recording with the detection offline, we hypothesize this real-time detection system will lead to a statistically significant reduction in time on the ground after a fall, fall related hospitalizations, and length of hospital stay following fall incidents based on results described in previous clinical trials with 150+ participants [8,14].

Public Health Relevance Statement:
The proposed NestSense system uses off-the-shelf wall-mounted cameras to perform detection of safety-critical events for Alzheimer’s disease and related dementia patients in memory care facilities. It does not rely on active use from the patients or caregivers (e.g., through wearing any kind of device). The NestSense technology provides the first robust non-wearable fall detection tuned specifically to the privacy-security tradeoffs of dementia care. It provides an answer to today’s technology gaps (far from 100% reliability) by use of a Human-in-the Loop (HIL) paradigm to enable a quick rollout of the technology. Following the acceptance of privacy safety tradeoffs of our system by patients, family, and memory care staff in the Carlton, Integral Senior Living, Pacifica and SRG networks, we have deployed the system in offline mode in 13 facilities of the network, monitoring 100 patients. The research, implementation, and deployment work encompassed in this SBIR Phase I will provide validation that the existing prototype can be rolled out to a proof of concept, operational in real-time mode, with HIL, to reduce fall-related hospitalizations and time spent on the ground. The pilot follows 3 previous pilots, the most recent of which deploys cameras with these 100 patients in offline mode to study the effect on the fall rate of occupational therapist review of fall video in memory care. This study will translate those cameras into a real-time fall (online) detection system which will run online for 3 months with a 24/7 HIL support. Compared to a 3-month baseline from the cameras recording with the detection offline, we hypothesize this real-time detection system will lead to a statistically significant reduction in time on the ground after a fall, fall related hospitalizations, and length of hospital stay following fall incidents based on previous clinical trials with 100+ participants [8,14].

Project Terms:
Accidents; Activities of Daily Living; Adherence; Adult; Aging; Algorithmic Software; Algorithms; Alzheimer's Disease; Artificial Intelligence; Awareness; base; Beds; Caregivers; Caring; Cessation of life; Clinical; Clinical Trials; Cognitive; Collection; cost; Data; Data Collection; Dementia; dementia care; design; Detection; Devices; Direct Costs; Disease; Emergency Situation; Event; Facilities and Administrative Costs; falls; Family; Feasibility Studies; Fracture; Future; Goals; Hospitalization; Hospitals; Hour; Human; human-in-the-loop; Individual; Institutional Review Boards; Intervention; Interview; Lead; Learning; Length; Length of Stay; Managed Care; Measures; Memory; Monitor; Occupational Therapist; Online Systems; Participant; Patient Monitoring; Patients; Phase; prevent; Privacy; prototype; Quality of Care; Quality of life; Recruitment Activity; Research; Running; Safety; Scheduling and Staffing; Security; Sensitivity and Specificity; sensor; Small Business Innovation Research Grant; Specificity; success; Surveys; System; Techniques; Technology; Testing; Time; time use; Translating; Validation; Video Recording; Visit; willingness; Work

Phase II

Contract Number: 2R44AG058354-02A1
Start Date: 00/00/00    Completed: 00/00/00
Phase II year
2019
(last award dollars: 2020)
Phase II Amount
$987,266

In the US, Alzheimer’s disease (AD) is the single most expensive disease, the only one in the top six for which the number of deaths is increasing. The greatest costs are hospitalizations, where falls are the largest culprit, and frequent need for assistance with daily life activities. A fall safety system shows the potential to reduce costs and increase quality of care by reducing the likelihood of emergency events (e.g., detecting falls before a fracture occurs, reducing the number of repeat falls). Unfortunately, no fall detection and prevention technology has been developed specifically for the needs of dementia care where individuals (1) fall more frequently and (2) often cannot tell care staff how they fell, leading to increased use of Emergency Medical Services (EMS) when falls are unwitnessed to ensure affected individuals are safe. Our goal is to perform a randomized wait-list control clinical trial (n=460) of SafelyYou Guardian, an online fall detection system with wall-mounted cameras to automatically detect falls for residents with AD and related dementias (ADRD). The automation is based on algorithms that push the frontier of deep learning, a subfield of Artificial Intelligence (AI), with a human-in-the- loop (HIL). SafelyYou Guardian is designed to primarily operate in memory care facilities (defined herein as assisted living and skilled nursing facilities providing ADRD care). Deep learning has already revolutionized several fields: robotics, self-driving cars, social networks in particular. Our approach is anchored in novel algorithms developed at the Berkeley AI Research Lab (BAIR) and extended by SafelyYou for real-time detection of rare events in video. The HIL is operating from a call center, confirms the fall detection alerts provided by our artificial intelligence algorithms, and places a call to the communities, so an intervention can happen within minutes of the fall detection. Subsequently, an Occupational Therapist (OT) working from our office in San Francisco reviews the fall videos with the front-line staff over video conference and using our web portal to make recommendations on how to re-organize the resident space (intervention) to prevent future falls. We leverage our HIL paradigm, in which our deep learning approach identifies and pre-filters falls with high sensitivity followed by a human who confirms the fall with high specificity and calls the communities in case of detected fall. This project leverages past small scale clinical and technical pilots including 87 residents from 11 partner communities, and our experience with paid commitments for 480 residents from three partner networks. Past pilots leading to this NIH Phase II proposal include: · Pilot 1: Technical proof of concept with healthy subjects (200 acted falls). · Pilot 2: We demonstrated acceptance of privacy/safety tradeoffs by residents, family and staff, through the collection of 3 months of video data at WindChime of Marin, our first partner facility; we identified 4 total hours of fall data. This led to clinical benefits including an 80% fall reduction through the intervention of OT. · Pilot 3: We demonstrated scalability and acceptance by deploying the system in 11 communities, for 87 residents monitored by our system (offline, no HIL intervention). · Pilot 4: Small scale NIH Phase I clinical trial. We demonstrated the ability to perform real-time fall detection, with real-time intervention of the HIL through our partner company Magellan-Solutions which provides the 24/7 monitoring service for the facilities. We demonstrated that 93% of 89 falls were detected, that time on the ground was reduced by 42%, that the likelihood of EMS use was 50% lower with video available, and the that total facility falls including participants and non-participants decreased by 38%. The trial proposed for this NIH SBIR Phase II will provide clinical evidence that the preliminary trends observed experimentally (pilot 2) and at small scale (pilot 4) are true phenomena. It will use a wait-list control population (230 residents) to be compared to the population monitored with SafelyYou Guardian (230 residents). After crossover, the wait-list population will also benefit from the technology and be compared to itself before crossover.

Public Health Relevance Statement:
Narrative The goal of this project is to perform a randomized, wait-list controlled trial (n=460) of SafelyYou Guardian, an online fall detection and prevention system for memory care facilities (defined here as skilled nursing and assisted living facilities providing dementia care). The technology applies breakthroughs in artificial intelligence to video data collected from off- the-shelf, wall-mounted cameras to automatically detect falls from video for residents with Alzheimer’s disease and related dementias (ADRD); it enables care staff (1) to know about falls right away without requiring residents wear a device, (2) to use video review to quickly assess need for emergency medical services (EMS) after unwitnessed falls, and (3) to perform accurate incident review with support from a remote occupational therapist (OT) to assess how to reduce the risk of repeat falls. The Phase I project goal of launching this service at small scale was achieved and demonstrated with 11 memory care facilities (1) 93% of 89 falls detected, generating 1 alarm per camera per 15 days, (2) 37% reduction in EMS use through better understanding of risk when residents with dementia were found on the floor, and (3) 38% reduction in falls through reduced risk of repeat falls following OT video review.

Project Terms:
Accidents; Address; Adult; Affect; Aging; Algorithms; Alzheimer's Disease; Alzheimer's disease related dementia; Artificial Intelligence; Assisted Living Facilities; Automation; Automobile Driving; Awareness; base; Beds; care costs; Caregivers; Caring; Cessation of life; Clinical; Clinical Trials; Cognitive; cohort; Collection; Communities; Control Groups; cost; Data; deep learning; Dementia; dementia care; design; Detection; Devices; Discipline of Nursing; Disease; Emergency department visit; Emergency medical service; Emergency Situation; Ensure; Event; experience; falls; Family; Floor; Fracture; frontier; Future; Goals; Health Care Costs; Health care facility; Hospital Costs; Hospitalization; Hour; Human; human-in-the-loop; improved; Individual; Intervention; Lead; Letters; Life; Measures; Medical; Memory; Monitor; Morbidity - disease rate; mortality; Notification; novel; Occupational Therapist; Outcome; Participant; Persons; Phase; phase 1 study; Phase I Clinical Trials; Population; Population Control; prevent; Prevention; Privacy; Quality of Care; Quality of life; Randomized; Recommendation; Research; Risk; Risk Factors; Robotics; Safety; Sample Size; San Francisco; sensor; Series; Services; Skilled Nursing Facilities; Small Business Innovation Research Grant; Social Network; Specificity; standard of care; Statistical Data Interpretation; Stream; symposium; System; Technology; Time; trend; United States National Institutes of Health; Visit; Waiting Lists; web portal