A Scalable and Real-time Hybrid Multimodal Content Exploitation System Navy SBIR FY2014.2 Sol No.: Navy SBIR FY2014.2 Topic No.: N142-122 Topic Title: A Scalable and Real-time Hybrid Multimodal Content Exploitation System Proposal No.: N142-122-0745 Firm: Intelligent Automation, Inc. 15400 Calhoun Drive Suite 400 Rockville, Maryland 20855-2737 Contact: Kaizhi Tang Phone: (301) 294-5214 Web Site: www.i-a-i.com Abstract: For many analysis tasks, the frame-level contextual understanding including query and content tagging on the multimedia data will provide time-critical actionable intelligence and insights. However, such frame-level capability does not exist for multimedia streams and files. To address this need, IAI, along with our collaborators, proposes to develop a Scalable and Real-time Hybrid Multimodal Content Exploitation System: HExp (Hybrid Exploitation). The key innovation is to seamlessly and systematically integrate information retrieval methods in different information spaces through theoretically sound retrieval models so that intelligence analysts can search multimedia contents in both the structured and unstructured spaces. HExp serves as a software system for intelligence analysts to search, tagging and tracking multimedia frames based on the dimensions of time, location, and event categories. HExp consists of three subsystems in different layers: (1) the distributed video/audio annotating and tagging subsystem that transform raw files into text descriptions and semantic entities and relations, (2) distributed hybrid information retrieval subsystem that index the unstructured and structured information from multimedia data using four different retrieval models, and (3) context-sensitive user interaction system that extends IAI's faceted heterogeneous search interface with a persistent user modeling module. Benefits: HExp can be used as a tool of military intelligence analysis to provide guidance and directions to commanders in support of their decisions by analyzing multimedia data. HExp can significantly expedite the intelligence analysis to meet commanders' mission requirements or respond to focused questions such as entities of interest as part of the operational or campaign planning activity. Some relevant military applications are:  HExp can be useful for handling the huge amount of multimedia data in the Navy/ONR BAA of Autonomous Persistent Tactical Surveillance.  HExp can assist the force protection for Navy, Marine Corps and Coast Guard forces in home and foreign ports and harbor facilities. Force protection requires security and surveillance measures from sources such as emplaced cameras, blog photos, and cell phone and radio intercepts.  HExp can be used to enrich the text analytics in the D2D program to address the text analytics to enrich information network for contextual understanding with a systematic dual-space information retrieval system. HExp can also be applied into many applications in the commercial world that require scalable search and in-depth semantic understanding of high volume of multimodal multimedia data types such as traffic multimedia data and crime scene data. Return