|
Intelligent Retrieval of Surveillance Imagery
Navy SBIR 2006.2 - Topic N06-147 ONR - Ms. Cathy Nodgaard - [email protected] Opens: June 14, 2006 - Closes: July 14, 2006 N06-147 TITLE: Intelligent Retrieval of Surveillance Imagery TECHNOLOGY AREAS: Information Systems, Human Systems ACQUISITION PROGRAM: MARCORSYSCOM, PM Intel OBJECTIVE: Develop the content-based retrieval tools and architecture that enable users of intelligent video surveillance systems to easily conduct video-based forensics and IMINT from video imagery and display entire event histories for a given target or alerting event, regardless of whether the video data was tagged during acquisition. DESCRIPTION: Intelligent video surveillance systems offer potential advantages for force protection and anti-terrorism in terms of automated alerts that improve the effectiveness of responses to threats and reduce manpower requirements for watchstanders. Intelligent video systems can automatically detect, classify and track persons, vehicles, and watercraft and infer threat potential through activity recognition. Such systems also offer the possibility of searching for patterns of intrusion or surveillance over an extended time course. Intelligent video architectures developed for force protection can be enhanced to provide forensic video analysis and IMINT. Such IMINT and forensic applications require the storage of large amounts of appropriate imagery, together with tagging metadata, when available. This metadata can be as simple as timestamps, camera ID and alert messages and or it can include markups on target class and video event ontologies. However, not everything in a scene can be tagged, and some video imagery may also be available with no tags. In order to fully exploit surveillance imagery there needs to be a capability for querying the database by either semantic queries for metadata search or by content-based image retrieval. The content-based image retrieval should be able to handle both those cases where sample images of a specific "target" are presented or where the analyst can iteratively search for a class of related "targets" within the imagery database. Recent research on content-based image retrieval, boosted retrieval and weakly supervised learning of image hierarchies have provided new tools for retrieval and retagging of data that could be employed in a user-centric search. In order to support both Force Protection and Intelligence missions these image retrieval and indexing tools ultimately need to be integrated into a knowledge management architecture that includes cataloging of the sensor data, the ability to handle non-video imagery and geospatial information, context information, event and activity recognition metadata and user-centered tools for retrieval of imagery and activities, image exploitation and decision support. The interface should be intuitive and make the best use of human visual perceptual skills. A number of image retrieval systems have been developed that focus on web content or broadcast video. The focus of this topic is on imagery data obtained outdoors, both visible and thermal, especially non-tagged data. Targets of interest include watercraft, vehicles and persons. PHASE I: Identify, refine and evaluate content-based image retrieval techniques suitable for use in an intelligent surveillance system. Develop a knowledge management architecture that supports forensic ability for both tagged and untagged imagery, and is capable of representing video activity. Develop a querying schema suitable for both imagery and human activity. Test for storage and retrieval of images. PHASE II: Develop a prototype system. Demonstrate forensic ability to retrieve history linked to target images from a large image library. Demonstrate human-in-the loop retrieval of related targets. Demonstrate end-to-end system with data acquisition from multiple cameras and data collection over an extended period of time. Demonstrate forensic retrieval of event history linked to a recognized activity. Develop and present performance metrics on retrieval accuracy, speed, flexibility for different image sources, and scalability for library size. PHASE III: Design interfaces to defense systems for support IMINT within an intelligent video surveillance system. Show rapid reconfiguration changes between Force Protection mode and forensic mode. PRIVATE SECTOR COMMERCIAL POTENTIAL: The commercial potential is in the large security and surveillance industry (facility protection), and in forensics for law enforcement. REFERENCES: KEYWORDS: Intelligent Video Surveillance; Imagery Retrieval; Video Forensics; IMINT TPOC: Thomas McKenna
|