While full motion video (FMV) data contains a wealth of information, its potential for providing actionable intelligence cannot be realized unless the data is provided within its geographic context and is correlated with available intelligence from human assets. To provide the necessary contextual, registration to/fusion with GIS data is a natural choice. This requires both geo- and ortho-rectification of the video frames based on measurements of the sensor position and orientation, knowledge of the natural and man-made terrain from 3D map and LIDAR data, and video data-driven processing. Toyon Research Corporation proposes to develop an automated sensor calibration and pixel geo-registration system using a suite of innovative algorithms which are applicable for various video sensors. The algorithms do not require a priori estimates of the intrinsic or extrinsic sensor parameters or of the terrain. Nevertheless, any available information (for example, initial calibration estimates, GPS and/or inertial measurements, DTED, LIDAR range maps, or geo-registered orthoimages) can be used by the algorithms with the benefit of improved estimation accuracy and/or improved computational efficiency. The processing is performed using an innovative, computationally-efficient particle filtering algorithm that Toyon has developed for this purpose. When completed, the software will provide, among other benefits, FMV windows with optional intelligence overlays displayed in a viewpoint congruent manner.
Benefits: The successful completion of this research and development will result in a net centric, open architecture, operating system independent software solution providing near real-time geo- and ortho-rectification of live full motion video, with fusion and viewpoint-congruent display of LIDAR and 3D mapping data with intelligence overlays. This will allow surveillance and tactical asset operators and commanders, as well as geographically distributed analysts, to maximize the efficiency and effectiveness of sensor data exploitation. For example, this technology would be useful for surveillance over challenging terrain along portions of the US Southern and Northern borders, as well as in law enforcement, disaster monitoring and response management, transportation studies, surveying, and environmental monitoring. The technology could even be used to provide improved realism in personnel training and computer gaming.
Keywords: Automated Sensor Calibration, 3d Reconstruction, Geo-Registration, Control Point Detection, Particle Filtering, Bayesian Estimation, Full Motion Video
