SBIR-STTR Award

In the absence of a global reference such as GPS, many efforts focus on using IMU sensors or a hybrid approach of using IMU sensors with camera, LiDAR, and Wi-Fi or Bluetooth beacons for accurate indoor positioning and navigation. As these auxiliary sensors bring inconvenience and leads to increased cost, pedestrian dead reckoning (PDR) based on IMU alone has received much attention. Nevertheless, existing PDR based on Kalman Filtering and machine learning techniques fail to predict under varying device placements and human motion especially when riding on an elevator or climbing stairs where steps cannot be accurately predicted. As a solution, we propose to research and develop an innovative, context-aware pedestrian dead reckoning technology called COSINE (Context-aware Opportunistic Sensing for Indoor Navigation Environment) that can predict user positions and trajectories with less than 0.2% error. COSINE relies on raw sensor data streams and advanced analytics to sense the context passively and implicitly around the user, and then predict position and orientations under a given context. The core of our approach stems from the exploitation of edge-friendly temporal deep learning architectures with adversarial learning inspired feature denoising and context-aware sensing to predict user trajectories under unpredictable sensor noises and contexts.

The COSINE STTR Phase II is established with the rigorous goal of realizing human pose estimation and localization with less than 0.2% error rate of the ground truth position in GPS denied and contested environments, without relying on any infrastructure. Unknot.id Inc., proposes to research, develop, and demonstrate an innovative, multi-modal, cost-effective, user and motion-dynamics invariant, artificial intelligence (AI) powered robust human positioning and identification platform called COSINE (Context-aware Opportunistic Sensing for Indoor Navigation Environment). The core of our technology lies on the exploitation of novel sensor modalities and the advancements in deep learning technology from training to learning algorithms on multi-modal noisy sensors readily accessible in a smartphone to accurately localize and identify humans in GPS-denied and contested environments like low RF signal power (tree canopy, buildings, storage containers) and low lighting environments (e.g., dark, smoke-filled room) irrespective of the underlying human motion dynamics and device orientations. COSINE is a Software-As-A-Service, multi-modal pose estimation and localization platform that facilitates every user (aka device) to compute its own six degrees of freedom (6DOF) pose, identity and context in real-time by optimally fusing audio, camera, three-axis accelerometer, three-axis gyroscope, three-axis magnetometer, light, and barometer sensors, without the need of any bespoke positioning infrastructure.