N092-155 TITLE: Organization, Search and Manipulation of Large Databases of Face Images

TECHNOLOGY AREAS: Information Systems

The technology within this topic is restricted under the International Traffic in Arms Regulation (ITAR), which controls the export and import of defense-related material and services. Offerors must disclose any proposed use of foreign nationals, their country of origin, and what tasks each would accomplish in the statement of work in accordance with section 3.5.b.(7) of the solicitation.

OBJECTIVE: This topic seeks to develop new technologies for organizing and searching very large collections of face images. Of particular interest are technologies that will aid DoD, Homeland Security, and law enforcement communities, where there are existing large collections of diverse facial photos from surveillance images and videos, mug shots, driver’s license photos, passport photos, etc. Computer-based software tools should be developed and tested to allow users in these communities to search and browse large image databases based on the visual appearance of the faces within. Users of these software tools will be able to rapidly search these large databases using simple text descriptions for the search queries. Example queries might include "smiling old white man with mustache wearing glasses" or "young Asian man with mole on left cheek without facial hair outdoors." A second related objective is to develop algorithms for synthesizing photo-realistic sketches (or a family of sketches) from such textual descriptions.

DESCRIPTION: There have been numerous commercial efforts to develop technologies for automatic facial recognition. The applications for such technologies are manifold, including access control to secure facilities, passport control, password-free access to computer accounts, image search and organization, as well as numerous applications in the intelligence community. Yet, a persistent problem with traditional face recognition is that these technologies do not work with high enough reliability. The reasons for this are many, but all have to do with the large variations in the appearance of the same person from image to image, as lighting, pose, expression, hairstyle, camera parameters, etc., change from image to image.

While the exact determination of the identity of a person within an image may not yet be possible with sufficient accuracy, there are many things that can still be determined with high reliability from a face image. In particular, computer vision and machine learning technologies can be used to label – automatically – each face in an image with various attributes such as gender, age, race/ethnicity, expression, hair color, image quality, etc. Such software can be built around commercially available face detectors that have been demonstrated to be highly reliable. The robustness of the automatic attribute labeling depends on the size and diversity of the data used to train the labeler. Recently, large photo-sharing Web sites with publicly available photos have made it possible to obtain tens of millions of face images (spanning gender, race, pose, expression, etc.) taken under diverse imaging conditions (indoor/flash/outdoor lighting, resolution/blur, etc.). Such a large and diverse collection of face images can enable the development of robust attribute classifiers using state-of-the-art computer vision techniques to extract features and machine learning methods to train the classifiers. The learned classifiers can then be used to automatically assign a large number of attributes to each face image. Any large face database can then be organized and browsed by allowing a user to select the attributes of interest.

A major technical challenge involved in the development of the above tools is scalability. The tools are intended to work on databases that include millions of faces. A successful set of tools must therefore address the storage, processing (feature extraction, training and labeling), indexing (browsing/searching) and user interface (control, interactivity and display) aspects of the technology. We emphasize that the purpose of this topic is not collecting face databases; rather it is to develop algorithms and technologies for organizing existing databases to enable rapid and reliable searches based on verbal attributes, as well as using massive face databases to develop algorithms for creating photo-realistic sketches from verbal descriptions.

PHASE I: Develop a detailed technical plan and architecture for managing and labeling of training images as well as the labeling and browsing of faces within an application database. This phase should also include obtaining estimates related to scalability – storage requirement and search efficiency. A prototype system that handles a hundred thousand of faces should be demonstrated.

PHASE II: Develop tools for training, browsing and de-identification that can be scaled to work on a database that includes at least 10 million face images. The system should be demonstrated to end user communities and feedback obtained with respect to how the tools need to be tailored for specific application domains. Develop tools for generating photo-realistic face sketches from verbal descriptions.

PHASE III: The technologies and products developed under this topic will have applications in intelligence analysis, law enforcement, and security. For example, this technology will aid security and law enforcement personnel to search in existing databases for images of a suspect based on verbal descriptions; or generate photo-realistic sketches of a suspect based on verbal descriptions.

PRIVATE SECTOR COMMERCIAL POTENTIAL/DUAL-USE APPLICATIONS: Applications of the developed tools also include specialized image-based search for online sites such as image search engines, social networks, and personalized photo organization, as well as for face synthesis in gaming and entertainment industry.

REFERENCES:
1. B.A. Golomb, D.T. Lawrence, T.J. Sejnowski: Sexnet: A neural network identifies sex from human faces. NIPS (1990) 572-577.

2. P.N. Belhumeur, J. Hespanha, D.J. Kriegman: Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. ECCV (1996) 45-58.

3. B. Moghaddam, M.H. Yang: Learning gender with support faces. TPAMI 24 (2002) 707-711.

4. Y. Yacoob, L.S. Davis: Detection and Analysis of Hair. TPAMI 28 (2006) 1164-1169.

5. S. Baluja, H. Rowley: Boosting sex identification performance. IJCV (2007) 111-119.

KEYWORDS: Large face databases; face detection and recognition; face attributes; automatic labeling; face de-identification; face search and synthesis.

** TOPIC AUTHOR (TPOC) **

DoD Notice:   Between April 20 and May 17, 2009, you may talk directly with the Topic Authors to ask technical questions about the topics. Their contact information is listed above. For reasons of competitive fairness, direct communication between proposers and topic authors is not allowed starting May 18, 2009, when DoD begins accepting proposals for this solicitation.

However, proposers may still submit written questions about solicitation topics through the DoD's SBIR/STTR Interactive Topic Information System (SITIS), in which the questioner and respondent remain anonymous and all questions and answers are posted electronically for general viewing until the solicitation closes. All proposers are advised to monitor SITIS (09.2 Q&A) during the solicitation period for questions and answers, and other significant information, relevant to the SBIR 09.2 topic under which they are proposing.

If you have general questions about DoD SBIR program, please contact the DoD SBIR Help Desk at (866) 724-7457 or email weblink.