SBIR-STTR Award

This Small Business Innovation Research (SBIR) Phase I project aims to develop methodology and software for highly accurate and efficient semantic video retrieval and summarization. Video Semantics will, provide personalized summaries of video content that meet users' preferences. These summaries will be based on shot granularity instead of the widely used key-frame-based summaries that are oblivious of semantics. Additionally, the proposed technology will significantly enhance online video search by enabling users to retrieve only semantically-relevant shots instead of the entire video. The key component of the software is an automated semantic video annotation system that integrates realtime video shot detection, speech recognition, natural language processing, and logic inference methods to accurately select video shots according to semantic user requests and preferences. Consumers and video content service providers will use the proposed adaptive video messaging technique to efficiently communicate queries, preferences and results using Semantic Video Summary messages (SVS). The proposed software, once commercialized, can affect a shift in the way online video content is searched and retrieved. Moreover, if successful, the software will advance the state-of-the-art of constructing video summaries, which in contrast to current technologies, accurately responds to semantic level user queries. Consequently, the software may be of interest to numerous content providers and consumers to be employed in a multitude of video applications. The software could also be integrated into the ever-popular digital video recorders to enable the owner to search large volumes of archived videos and retrieve specific ones given semantic queries, rather than the usually inaccurate file names. On the other hand, the unique summarization capabilities of the software can be used by content/service providers where personalized, semantic-based summary criteria can be predefined by the user so that the content providers, adaptively (based on network and device capabilities) stream summaries matching users' requirements to their smart phones of other mobile devices. This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).

This Small Business Innovation Research Phase II project will develop contextual video segmentation and automatic tagging technology and software. In long video streams that contain one or more topics, the software automatically discovers the beginnings and ends of Contextually-Coherent Video Segments in each video. Moreover, Video Semantics' technology automatically assigns textual tags to each segment such that these tags describe the topic discussed in that segment. The tags assigned make all parts of the video easily searchable. Large video producers currently depend on manually segmenting their content into small segments and assigning textual tags to these segments in order to make them searchable. A short advertisement is then inserted before each segment. This manual segmentation and tagging process represents a significant pain point for content producers because it is labor intensive and not cost effective. Meanwhile, government agencies, which continuously monitor video content depend on speech recognition to spot specified keywords. This approach inflicts two pain points: (i) analysts have to deal with large number of false detections because the context in which the keyword occurs might be irrelevant, and (ii) if the keyword occurs in an important context, analysts still need to scroll back and forth into the video to find the beginning of the relevant segment. Video Semantics' technology and products have the potential to efficiently address significant market needs. In addition to the commercial applications, the proposed technology will enable media monitoring agencies to perform their tasks more efficiently saving valuable analyst time and resources. Moreover, because Video Semantics? technology is language-independent, media monitoring agencies will be able to monitor more content in foreign languages without the need to develop language-specific technologies. The company will employ an indirect sales strategy via partnerships with software companies that develop media monitoring solutions and metadata generation tools. The company has identified its first customer and is working with them to integrate the contextual segmentation and tagging technology with their current media monitoring solutions.