SBIR-STTR Award

With the current flood of surveillance data available to ISR analysts, human attention has become the most valuable resource to ISR systems. Although automated tracking and labeling algorithms are now capable of automatically identifying and roughly classifying targets, the current rate of false alarms and irrelevant annotations makes existing technology unsuitable for wide-area persistent surveillance applications, where analysts are overwhelmed by irrelevant data. What is needed is a system that incorporates a user-trainable relevance/saliency classification algorithm with the best available tracking algorithms to achieve very low clutter rates even in urban environments. Toyon Research Corporation and Penn State Professors David Miller and George Kesidis propose to address this need through a prototype system for automated saliency annotation, incorporating recent results in active learning of semisupervised mixture models, and automated feature extraction from video data with reconstructed 3D models. The proposed system combines the extremely low clutter rates of Toyon’s 3D clutter suppression algorithm, with high-accuracy classification methods using fine-grained mixture models developed by Professors Miller and Kesidis.

Benefit:
The capability generated by the proposed system will be crucial to Air Force ISR systems that rely on real-time processing and mining of wide-area persistent surveillance (WAPS) data, by dramatically increasing the effective surveillance region size per operator. The technology also reduces potential tactical cost in the consequences associated with misidentifying critical targets (either missing terrorist activity, or incorrectly targeting innocent civilians). Active learning has the potential both to greatly reduce the amount of labeling analysts need to do to achieve accurate automated classification/saliency determinations and, by achieving accurate classification, to reduce the risk of adverse tactical consequences. Additionally, the technology has many non-military applications, including manufacturing, construction, security, and automobile traffic monitoring, where the rarity of salient events limits the effectiveness of existing wide-area video systems from being effective, due to shortage of human resources.

Keywords:
Video Analysis, Machine Vision, Image Processing, Active Learning, Saliency

Toyon Research Corporation and the Pennsylvania State University propose to develop a software tool which allows an operator to interactively identify suspicious activities. The tool will be called ALARM (Active Learning for Anomaly Recognition and Mensuration). It will ingest a database of track data and automatically cluster the data and develop statistical models of the tracks. The statistical models will be used to rank the tracks in terms of the most anomalous to the least anomalous. The operator will observe the most anomalous tracks and use contextual information to identify the most suspicious tracks. The ALARM inference algorithms will use the labels provided by the operator to classify the labeled tracks as suspicious and to identify unlabeled tracks that may or may not have been classified as anomalous, and present these tracks to the operator for further labeling and classification. The track database that we will use during Phase II will be derived using stored data collected by the ARGUS-IS sensor. We will improve and run automated detection and tracking algorithms on the ARGUS-IS data to create the tracks.

Benefit:
The successful completion of the proposed research will allow analysts to effectively sift through large volumes of video and track data in short periods of time. This will be made possible by making use of low-level automation for video processing and tracking combined with high-level reasoning by humans to recognize what is suspicious and what is not. A classifier that treats as inputs features derived from the track measurements will be actively learned both to automatically predict whether given tracks are suspicious or not and to rank all tracks for prioritized forwarding to a human analyst for labeling.

Keywords:
Video Processing, Anomaly Detection, Active Learning, Wami, Argus, Tracking