TECHNOLOGY AREA(S): Air Platform, Human Systems, Information Systems

ACQUISITION PROGRAM: PMA-281 (UAS) Strike Planning & Execution Systems

OBJECTIVE: Develop a machine intelligence system that provides real-time validation and trust in decisions made by unmanned vehicles implementing machine intelligence to control autonomous vehicles.

DESCRIPTION: If an operator was to control multiple autonomous vehicles operating in potentially different domains and executing mission goals from onboard machine intelligent systems, it is important that the operator be provided support in terms of validation that the unmanned vehicle machine intelligence is making the correct decisions regarding their autonomously generated mission parameters. As operators delegate more control to autonomous systems, the risk increases that the system might make an inappropriate decision, before that decision and resulting action is detected by the operator. Therefore, a high-fidelity decision aid is required to provide oversight to the operator that a complex system of multiple vehicles is operating correctly, safely, and within existing rules. In case of inappropriate actions, the operator should be provided recommended corrective actions with probabilities of success to mitigate/resolve issues. The operators must be assured that autonomously operating vehicles would adhere to applicable legal (e.g., rules of Law of Armed Conflicts) and ethical principles [Ref. 8] in the decision processes that are being made by autonomous vehicles. Because of the many, perhaps swarms of, autonomous vehicles that the operator must track, an intelligent process is necessary. This process must be able to observe ongoing autonomous operations and assure that decisions made are appropriate for the situation. The primary goals are to provide real-time validation of the ongoing autonomous operation, and through machine learning techniques establish and develop trust in the autonomously operating vehicle's decision-making process. The tool should alert the operator if the autonomous vehicle is taking inappropriate action, for example as an extremely rare case, readying itself to launch a weapon without operator concurrence/approval. The tool must validate and verify system performance while providing accurate and timely feedback, and ultimately increase operator trust in the autonomously operating system’s behavior.

PHASE I: Develop and demonstrate the feasibility of a conceptual tool that meets the requirements in the Description. Produce prototype plans to be developed under Phase II.

PHASE II: Design and develop a prototype tool based on the Phase I concept and demonstrate the performance in a simulated environment. However, if feasible, a live demonstration would be preferred.

PHASE III DUAL USE APPLICATIONS: Refine and enhance the prototype tool resulting in a final product and demonstrate the capability in an operational setting. Transition the developed technology to appropriate systems such as the Department of the Navy Program Executive Office for Unmanned Aviation and Strike Weapons (PEO (U&W) Common Control System (CCS). Companies such as Amazon are using unmanned aerial vehicles (UAVs) for delivery of parcels would glean benefits from this proposed tool. As companies embed or increase autonomous behavior in the UAV operation, this tool will aid in validation and verification of the embedded autonomy, which in essence will build trust in the operation of the autonomously operating UAVs.

REFERENCES:

1. Finn, R.A. and Scheding, S.J. "Developments and Challenges for Autonomous Unmanned Vehicles." Intelligent Systems Reference Library, January 2010.  https://www.researchgate.net/profile/R_Finn/publication/289726773_Developments_and_Challenges_for_Autonomous_Unmanned_Vehicles_A_Compendium/links/5800386c08aec3e477ead0f5.pdf?origin=publication_detail

2. Clare, A.S. Cummings, M.L., and Repenning, N.P. "Influencing Trust for Human-Automation Collaborative Scheduling of Multiple Unmanned Vehicles." Human Factors, Vol. 57, No. 7, November 2015, p. 1208, https://hal.pratt.duke.edu/sites/hal.pratt.duke.edu/files/u13/Influencing%20Trust%20for%20Human%E2%80%93Automation%20Collaborative%20Scheduling%20of%20Multiple%20Unmanned%20Vehicles.pdf

3. Kuipers, B. "How Can Robots Be Trustworthy?" Computer Science & Engineering, University of Michigan. http://qav.cs.ox.ac.uk/autonomy_morality_trust/img/KuipersMoralityTrustWorkshop17.pdf

4. Hall, B.K. "Autonomous Weapons Systems Safety.” National Defense University Press, Joint Force Quarterly 86. http://ndupress.ndu.edu/Media/News/Article/1223911/autonomous-weapons-systems-safety/

5. Tucker, P. "The Air Force Doesn’t Know How to Test Its Future Robotic Wingmen." Defense One, Oct.20, 2016. http://www.defenseone.com/technology/2016/10/military-unsure-how-test-future-autonomous-drones/132525/

6. Huang, S, et al. "Enabling Robots to Communicate their Objectives."11 Feb 2017. arXiv:1702.03465 [cs.RO]. https://arxiv.org/pdf/1702.03465.pdf

7. Pike, L., Stewart, D., and Van Enk, D. "Unmanned Autonomous Verification and Validation." Position Paper. https://pdfs.semanticscholar.org/5405/e13e7d8fba11ca945e4faf9641e9f89769d8.pdf

8. Stansbury, R.S., Olds, J.L., and Coyle, E.J. "Ethical Concerns of Unmanned and Autonomous Systems in Engineering." 121st ASEE Annual Conference and Exposition, 15-18 June 2014. https://www.asee.org/public/conferences/32/papers/8996/download

KEYWORDS: Autonomous Operation; Verification and Validation; Computational Trust; Machine intelligence; Real-time; Unmanned Vehicle