TECHNOLOGY AREAS: Air Platform, Information Systems, Human Systems

ACQUISITION PROGRAM: AACUS Candidate FY12 ONR Innovative Naval Prototype

OBJECTIVE: To develop and demonstrate a scalable human interface and related automation technologies to enable a user at a remote, unprepared site to manage a highly autonomous Vertical Take-off/Landing (VTOL) cargo/casualty evacuation unmanned air system in a simple way that requires minimal dedicated time and minimal skill/training. This will include communicating important high-level directions and spatial and temporal information about the conditions and any potential hazards of the landing site, the approach path, potential threats in the area, and any tactical considerations of the unit that is being resupplied. The approach should be scalable for a wide variety of users with different skill levels and a wide range of environmental conditions in the field. Note that the focus of this topic is on the user interaction approach and related algorithms and not the development of new hardware. Existing COTS hardware should be used to the greatest extent possible.

DESCRIPTION: There is currently interest in the idea of using an Unmanned Air System (UAS) to deliver cargo to marines in the field or to provide casualty evacuation or extraction. This type of UAS might be used to support a range of users from Forward Operating Bases with some level of available resources to support UAS operations to small unit operations at a remote site with minimal manning or equipment that can be dedicated to this purpose. A goal is to be able to operate in a broad range of conditions including night / degraded visual environments, GPS-denied areas, and areas with possible threats, high winds, and complex terrain/landing conditions including landing on slopes and around man-made and natural obstacles, people, water, and soft terrain. Safety considerations will also need to be taken into account with regards to ensuring the operation of this system does not provide a threat to friendly or noncombatant humans on the ground. ONR is currently exploring the potential of developing a very high degree of on-board autonomy that could be implemented as part of a future system. Autonomous capabilities are envisioned to include (1) Fully autonomous or human assisted approach selection and landing zone suitability assessment, (2) Launch / recovery with little or no human supervision, including by a relatively unskilled user with limited time in the field, (3) Highly automated mission planning and fully autonomous dynamic replanning when operating beyond line of sight communications, (4) Obstacle detection and avoidance, (5) Avoidance and evasion of known threats, and (6) Contingency response. However, given limitations in the current and projected state-of-the art in autonomy, it is likely that human users would need to provide some guidance to the system beyond that required by pilots of manned aircraft performing a similar mission. In addition, human users will need to be able to interact with the system to convey their mission objectives, priorities, constraints, and knowledge of the situation at and around the landing area. There are significant challenges involved in such high-level human interaction with systems that have a substantial degree of autonomy and complexity.

This topic will develop and demonstrate a scalable user interface approach to enable a wide range of users to interact with this type of autonomous system at a high level. This will include communicating important high-level directions and spatial and temporal information about the conditions and any potential hazards of the landing site, the approach path, potential threats in the area, and any tactical considerations of the unit that is being resupplied. The approach should be suitable for a range of users with different skills and experience levels and access to hardware interfaces ranging from ruggedized laptop computers to smaller PDA-like devices. The approach should be suitable for non-dedicated users in the field that may not be able to focus all their attention on interaction with this system. Minimizing the extent to which users must be "heads down" watching computer screens is highly desirable. Interaction approaches may include multi-modal input approaches such as sketch-input and speech/natural language. Though, the focus of this effort will be on using such advanced approaches and not developing new approaches to speech recognition, sketch input, etc. Approaches should be developed to ensure adequate situation awareness by the user and ensuring that the human has a sufficiently good mental model of how the autonomy operates to be able to effectively utilize the system. In addition, operator trust will play an important role in the usefulness of these tools and that must be considered in the development of the approach.

PHASE I: Develop an initial version of the proposed approach for a limited set of landing site/environmental conditions with sufficient functionality to demonstrate feasibility and allow some limited evaluation by military operators and domain experts. Ideally, this could include integration with simple, limited-fidelity simulation elements to show closed-loop performance. However, the use of canned data and/or static mock-ups will also be acceptable. Develop metrics to evaluate the system in Phase II and determine how the approach could integrate with particular types of hardware components and a future Cargo Unmanned Air System. Candidate metrics might address workload, command/interaction frequency, decision accuracy, error frequency, error impact, efficiency of use, response time, situation awareness, correlation between system state and the operator�s mental model, task time, usability, training time to achieve proficiency, and trust.

PHASE II: Further develop the proposed approach for a broader set of environmental conditions in a more complex dynamic and unstructured environment and integrate them with a higher fidelity simulation and sufficient autonomy components to perform laboratory operator in-the-loop demonstrations and comparison with benchmarks. Experiments with live assets may be used when of value to validate simulation results, but are not required. Revise evaluation metrics and interfaces as necessary.

PHASE III: Integrate the software with other components in a naval control station and participate in integrated demonstrations of autonomous systems operations

PRIVATE SECTOR COMMERCIAL POTENTIAL/DUAL-USE APPLICATIONS: This capability could be used in a broad range of military and civilian security applications of unmanned systems and in other applications involving management of automated systems, such as industrial applications.

REFERENCES:
1. "Unmanned Air Delivery System Industry Day," https://www.fbo.gov/index?s=opportunity&mode=form&id=87c081f0c38b5a1bf6bea130535614bf&tab=core&_cview=1

2. Cummings, M.L., Bruni, S., Mercier, S., & Mitchell P.J. Automation Architecture for Single Operator, Multiple UAV Command and Control, The International Command and Control Journal, (2007), Vol. 1(2).

3. Cummings, M. L., Clare, A. S., Hart, C. S. The Role of Human-Automation Consensus in Multiple Unmanned Vehicle Scheduling. Human Factors: The Journal of the Human Factors and Ergonomics, 52(1), 2010.

4. Endsley, M. R. (1995). Measurement of Situation Awareness in Dynamic Systems. "Human Factors", 37(1), 65-84.

5. Franke, J., Zaychik, V., Spura, T., & Alves, E. (June 2005). Inverting the operator/vehicle ratio: Approaches to next generation UAV command and control. Paper presented at the Association for Unmanned Vehicle Systems International and Flight International, Unmanned Systems North America. Retrieved from http://www.atl.lmco.com/papers/1261.pdf

6. T. Kollar, S. Tellex, D. Roy, N. Roy. "Toward Understanding Natural Language Directions", Proceedings of the 5th ACM/IEEE International Conference on Human-Robot Interaction, 2010.

7. Olsen, D.R., & Goodrich M.A. (2006). Metrics for evaluating human-robot interactions, Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-Robot interaction, Salt Lake City, Utah, USA pp. 33 -40.

8. Lee, J.D. and See, K.A. (2004), "Trust in automation: Designing for appropriate reliance," Human Factors, 46, 50-80.

9. Skubic, M., Anderson, D., Blisard, S., Perzanowski, D., Shultz, A. (2007). Using a hand-drawn sketch to control a team of robots, Autonomous Robots, 22 (4): 399-410.

10. Y. Wei, E. Brunskill, T. Kollar, and N. Roy, "Where to go: Interpreting natural directions using global inference," in IEEE International Conference on Robotics and Automation, 2009.

KEYWORDS: cargo unmanned air system, scalable human interface, natural language, sketch

** TOPIC AUTHOR (TPOC) **

DoD Notice:   Between November 10 and December 12, 2010, you may talk directly with the Topic Authors to ask technical questions about the topics. Their contact information is listed above. For reasons of competitive fairness, direct communication between proposers and topic authors is not allowed starting December 13, 2011, when DoD begins accepting proposals for this solicitation.

However, proposers may still submit written questions about solicitation topics through the DoD's SBIR/STTR Interactive Topic Information System (SITIS), in which the questioner and respondent remain anonymous and all questions and answers are posted electronically for general viewing until the solicitation closes. All proposers are advised to monitor SITIS (11.1 Q&A) during the solicitation period for questions and answers, and other significant information, relevant to the SBIR 11.1 topic under which they are proposing.

If you have general questions about DoD SBIR program, please contact the DoD SBIR Help Desk at (866) 724-7457 or email weblink.