Multimodal Knowledge Acquisition and Management Navy STTR FY2010.A Sol No.: Navy STTR FY2010.A Topic No.: N10A-T019 Topic Title: Multimodal Knowledge Acquisition and Management Proposal No.: N10A-019-0357 Firm: The Friedland Group, Inc. 330 SW 43rd St., Suite K #489 Renton, Washington 98057 Contact: Noah Friedland Phone: (206) 760-9487 Web Site: www.noahfriedland.com Abstract: Related information, particularly in the real word, can come in many forms, like text, images, video, and more. Exploiting that information will require a multimodal approach. The Friedland Group, which led Project Moebius at DARPA, is joined by The University of Rochester and Prof. David Forsyth - respective leaders in knowledge acquisition from text and images, to create a new framework for mutlimodal knowledge acquisition and management (MKAM). MKAM utilizes and expands Episodic Logic (EL), a highly expressive logical representation and reasoning framework that has been successfully applied to model complex events and situations. Capturing multimodal knowledge in EL will make it integrable, inference and unification capable. It will also enable the improvement of knowledge acquisition capabilities in individual modalities by providing more context and reducing ambiguities. Our Phase I work will demonstrate the feasibility of our approach through the development of several concrete examples, utilizing data produced by existing knowledge extraction systems, to show how EL can meld knowledge from different modalities while improving acquisition from individual modalities. Benefits: The ability to jointly acquire related knowledge from multiple modalities will greatly enhance the capabilities of knowledge systems to produce a more comprehensive picture of facts, events and individuals. This new capability will greatly improve existing capabilities, like indexing of knowledge, while opening the door to new capabilities, like near real time event modeling. On the government side, we see this technology improving the way information is accessed and managed. For example, it could help bridge disconnects in current intelligence applications by helping to "connect the dots". It could also be used to improve emergency and first response management. We envision many potential new commercial applications, or enhancements to existing applications. For instance, the ability to better index images and videos would be of significant interest. Return