Natural Language Dialogue for Supervisory Control of Autonomous Ground Vehicles Navy SBIR FY2010.2 Sol No.: Navy SBIR FY2010.2 Topic No.: N102-177 Topic Title: Natural Language Dialogue for Supervisory Control of Autonomous Ground Vehicles Proposal No.: N102-177-0425 Firm: Soar Technology, Inc. 3600 Green Court Suite 600 Ann Arbor, Michigan 48105-2588 Contact: Glenn Taylor Phone: (734) 887-7620 Web Site: www.soartech.com Abstract: Unmanned Ground Vehicles (UGVs) currently deployed on the battlefield are driven with Operator Control Units (e.g., joystick or point-and-click interface), which requires the full attention of an operator. This means the operator cannot concentrate on maintaining situational awareness, security, or other tasks. To reduce the burden on the operator, SoarTech with our partner Southwest Research Institute propose to develop a natural dialogue interface for interacting with UGVs, where an operator can interact with a UGV using speech and gesture. The UGV will be capable of taking high-level commands, engaging in dialogue with the user to ensure that the command is clear and providing feedback to the user during task execution. To accomplish this, we will design a multi-modal Smart Interaction Device (SID) for UGV control, based on proven Intelligent User Interface system that we have used for supervisory control of tactical simulation entities. We will extend this prior work to a new input modality (e.g., pointing gestures) and a new plug-in to interface with SwRI's UGV. This approach will enable supervisory control of one or more UGVs, and greatly reduce the workload of the UGV operator. Benefits: A natural dialogue system for supervisory control of UGVs has the potential to change the face of robotic systems. Current battlefield UGVs effectively require two people to operate: one to drive and one to supply security. By freeing an operator of the need to work heads-down, and instead allowing them to talk with a UGV as if it were another teammate, will mean the operator can go back to being a warfighter. Furthermore, by enabling an operator to treat a UGV as a teammate - with the UGV able to take high-level commands, execute autonomously, and provide feedback to the operator - the operator will be freed enough to manage multiple such unmanned systems simultaneously. The SID framework will also be capable of managing diverse UGVs though a single natural control interface. Beyond AUGV applications in logistics, supervisory control of autonomous entities has a great deal of potential in a range of fields: managing all kinds of ground and airborne unmanned systems; managing entities in large-scale simulation environments, where hundreds or even thousands of entities play a part; training air traffic controllers with virtual pilots; and managing intelligent virtual characters in immersive cultural trainers. Return