DIRECT TO PHASE II - Augmented Reality for Live Flight Training

Navy SBIR 23.2 - Topic N232-D07
NAVAIR - Naval Air Systems Command
Pre-release 4/19/23   Opens to accept proposals 5/17/23   Closes 6/14/23 12:00pm ET

N232-D07 TITLE: DIRECT TO PHASE II - Augmented Reality for Live Flight Training

OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Human-Machine Interfaces;Sustainment;Trusted AI and Autonomy

The technology within this topic is restricted under the International Traffic in Arms Regulation (ITAR), 22 CFR Parts 120-130, which controls the export and import of defense-related material and services, including export of sensitive technical data, or the Export Administration Regulation (EAR), 15 CFR Parts 730-774, which controls dual use items. Offerors must disclose any proposed use of foreign nationals (FNs), their country(ies) of origin, the type of visa or work permit possessed, and the statement of work (SOW) tasks intended for accomplishment by the FN(s) in accordance with the Announcement. Offerors are advised foreign nationals proposed to perform on this topic may be restricted due to the technical data under US Export Control Laws.

OBJECTIVE: Develop Augmented Reality (AR) to provide a potential solution for reducing the need for costly, live, multi-ship scenarios by integrating visible constructive entities via AR presentations during live flight training events.

DESCRIPTION: Technology has greatly outpaced updates to aviation training, and though many aerospace corporations are embracing different forms of Extended Reality (XR) for use in aircrew and maintenance training [Ref 1], most of the Navy�s current training syllabi have remained unchanged for decades. With fleet aircraft also becoming more complex, the basic piloting skills being taught at the undergraduate level are not preparing students adequately for the more advanced critical thinking and mission planning required for Fleet Replacement Squadron (FRS) training. With XR technologies improving exponentially, while also becoming cheaper, the traditional focus on the accumulation of flight hours to develop basic airmanship skills is no longer the optimal method to train effectively and efficiently, both in terms of quality and cost. Shifting away from a time-based to a competency-based approach to training with the incorporation of XR technology could provide a higher-level of training that would meet FRS entry-level requirements at a lower cost.

To make this shift, the Navy started Naval Aviation Training Next (NATN), a broad initiative focused on producing higher quality aviators in a more efficient manner. A primary catalyst behind NATN is the use of XR technologies. To date, the effort has been focused on virtual reality (VR) to provide students an immersive, lower cost platform capable of practicing procedures before doing them in the aircraft, better preparing them for flight events, which in turn allows flight events to focus on higher complexity or more difficult scenarios. This crawl-walk-run approach with VR has been demonstrated to successfully train flight procedures in a lower cost platform before demonstrating the same procedures in an aircraft [Refs 2, 3], where resources are scarce, and costs are high. Under this crawl-walk-run framework of training [Ref 4], the VR training is allowing students to shift the historical �crawling� during initial flights to �walk� or �run� training in the aircraft, with the �crawling� accomplished in VR. With the ability to execute any syllabus maneuvers in a VR device, NATN training has rapidly shown to be more efficient while also building higher quality pilots [Ref 5]. A natural extension of the VR training is to incorporate AR into actual aircraft training, as flight time gained in actual aircraft is invaluable and greatly reinforces skills learned during ground training.

AR has the potential to provide more efficient and effective training for undergraduate pilots to increase their capabilities during flight events while reducing resource requirements. An important factor for AR is its ability to �overlay information at the point of need� [Ref 6] making it a potentially very powerful training tool for nearly any flight training scenario by either inserting visible constructive entities, or guiding student attention to specific areas. For example, undergraduate jet training incorporates significant formation training to develop skills that are foundational for fleet assignments and missions. AR could supplement this training by utilizing a visible, constructive, formation partner in early stages of training instead of relying on another live aircraft, improving safety by avoiding possibilities of mid-air collisions while lowering overhead costs associated with utilizing multiple aircraft for training, ultimately reducing overall training time and cost by re-allocating live flight resources to other student naval aviators (SNAs) and events. Additionally, AR can improve training quality by allowing more practice opportunities for students to develop these important skillsets and fit into NATN�s methods for immersive �crawl-walk-run� training by facilitating the �walk� to �run� in live flight: the student is able to practice the basics of formation flying to better prepare for events with actual partner aircraft. Other logical areas in which AR could facilitate training include more advanced tactical formation flying, basic fighter tactics, aerial refueling, weapons deployment visuals, air-to-air engagement, air-to-surface missions, and other mission sets involving interaction with outside entities increasing the training capabilities and ability to introduce more complicated scenarios earlier in training.

In this Direct to Phase II SBIR topic, the Navy seeks an AR solution that would provide high-fidelity, behaviorally accurate, and visible constructive entities for live flights within the training pipeline successfully integrated into a military aircraft. Primary focus will be on demonstrating capability to support training scenarios with constructive entities in a military aircraft by successfully integrating an AR system into a Navy training aircraft and aviator gear for safe use in flight. At this stage, the AR system it is not expected (but is encouraged if meeting milestones) to be flown in military aircraft, but shall be demonstrated as capable for in-flight use by other means to provide evidence of reliability and functionality in the dynamic flight environment. Careful consideration should be given to scenario development and behaviorally accurate models of any constructive entities developed. Other items to consider should be system performance measures and assessment, integration into Navy data and grading systems, and methods for debrief utilizing scenario data from constructive entities. It is anticipated this technology would expand the NATN competency-based instructional model into live aircraft flight training, lowering training overhead while increasing training efficiency and output, by supplementing various training scenarios requiring multiple aircraft.

PHASE I: For a Direct to Phase II topic, the Government expects that the small business would have accomplished the following in a Phase I-type effort. It must have developed a concept for a workable prototype or design to address at a minimum the basic requirements of the stated objective. The below actions would be required in order to successfully satisfy the requirements of Phase I:

Designed a proof-of-concept technology that demonstrates high-fidelity virtual aircraft within an AR environment with high-quality real-world visuals.

Determined the technical feasibility of integrating virtual lead aircraft visuals for an aviator in full aviator gear in an actual aircraft cockpit.

Determined the feasibility of the technology meeting Risk Management Framework guidelines [Ref 7] to support cybersecurity compliance outlined in Defense Federal Acquisition Regulation Supplement (DFARS) and published in National Institute of Standards and Technology (NIST) Special Publication (SP) 800-171 "Protecting Unclassified Information in Non-federal Information Systems and Organizations" [Ref 8].

Determined the technical feasibility to incorporate performance assessment capabilities for After Action Review (AAR).

FEASIBILITY DOCUMENTATION: Offerors interested in participating in Direct to Phase II must include in their response to this topic Phase I feasibility documentation that substantiates the scientific and technical merit and Phase I feasibility described in Phase I above has been met (i.e., the small business must have performed Phase I-type research and development related to the topic NOT solely based on work performed under prior or ongoing federally funded SBIR/STTR work) and describe the potential commercialization applications. The documentation provided must validate that the proposer has completed development of technology as stated in Phase I above. Documentation should include all relevant information including, but not limited to: technical reports, test data, prototype designs/models, and performance goals/results. Work submitted within the feasibility documentation must have been substantially performed by the offeror and/or the principal investigator (PI). Read and follow all of the DON SBIR 23.2 Direct to Phase II Broad Agency Announcement (BAA) Instructions. Phase I proposals will NOT be accepted for this topic.

PHASE II: Develop a prototype of the AR flight training system integrated into a Navy military training aircraft (e.g., T-45) capable of presenting in an aviators visual field accurate and dynamic digital entities. By integration, the AR system should be fully functional and usable by an aviator without impeding any operation of the aircraft by the aviator or limiting access or function of aircrew gear. Major areas to consider include, but are not limited to: power supply; required computer processing; size, weight, and location of components; and interaction with aircrew gear. Consider and adhere to the Risk Management Framework guidelines during the development to support information assurance compliance [Ref 7]. Demonstrate the prototype integrated into the military aircraft in a relevant but safe environment (e.g., ground demonstration).

PHASE III DUAL USE APPLICATIONS: Develop hardened system architecture and complete the Risk Management Framework process to gain cybersecurity accreditation for system deployment. Demonstrate the ability to integrate transition-specific content for initial training capability transition for use during live flight in a Navy military training aircraft. Demonstrate the ability to incorporate product into a learning management system (LMS) for sustainment. Undergo safety of flight evaluations for approval for use during flight.

Development of AR technology for use during flight will present new training capabilities for commercial industry, providing civilian training programs with safer and more immersive training methodologies for scenarios like potential bird strikes, high traffic patterns, landmark identifications, and more. Additionally, once demonstrated as beneficial in an unclassified training context, the AR capability can be expanded to multiple military training platforms to aid not only training but mission rehearsal and planning across all aircraft, significantly reducing flight hour costs and time to train.

REFERENCES:

    1. Bellamy III, W. (2017, August 24). 9 companies using augmented and virtual reality in aviation. Aviation Today. https://www.aviationtoday.com/2017/08/24/9-companies-using-augmented-virtual-reality-aviation/
    2. Severe-Valsaint, G., Mishler, A., Natali, M., Astwood, R., Seech, T., & McCoy-Fisher, C. (2022). Training effectiveness evaluation of an adaptive virtual instructor for naval aviation training (NAWCTSD Public Release 22-ORLO44). Defense Technical Information Center, 2022. https://apps.dtic.mil/sti/citations/AD1170195
    3. McCoy-Fisher, C., Mishler, A., Bush, D., Severe-Valsaint, G., Natali, M., & Riner, B. (2019, September 30). Student naval aviation extended reality device capability evaluation (Report No. NAWCTSD-TR-2019-001). Defense Technical Information Center. https://apps.dtic.mil/sti/citations/AD1103227
    4. Kinney, M. (2008, July 17). What the Army taught me about teaching. Inside Higher Ed. https://www.immagic.com/eLibrary/ARCHIVES/GENERAL/GENPRESS/I080717K.pdf
    5. Mishler, A., Severe-Valsaint, G., Natali, M., Seech, T., McCoy-Fisher, C., Cooper, T., & Astwood, R. (2022, January 20). Project Avenger training effectiveness evaluation (Report No. NAWCTSD Public Release 22-ORL006). Defense Technical Information Center, 2022, https://apps.dtic.mil/sti/citations/AD1162306.
    6. Goel, A. (2018, March 19). Augmented reality in aviation: Changing the face of the sector through training and simulated experience. eLearning Industry. https://elearningindustry.com/augmented-reality-in-aviation-changing-face-sector-training-simulated-experience
    7. Department of Defense Education Activity. (2019, October 29). DoDEA administrative instruction 8510.01: Risk management framework for DoDEA information technology. Department of Defense. https://www.dodea.edu/Offices/PolicyAndLegislation/upload/DoDEA-AI-8510-01-Risk-Management-Framework.pdf
    8. Ross, R., Pillitteri, V., Dempsey, K., Riddle, M., & Guissanie, G. (2020). Protecting controlled unclassified information in nonfederal systems and organizations. National Institute of Standards and Technology Special Publication 800-171 Revision 2. U.S. Department of Commerce. https://doi.org/10.6028/NIST.SP.800-171r2
    9. KEYWORDS: augmented reality; AR; extended reality; XR; aviation training; training systems; aircraft integration; live-virtual-constructive


      ** TOPIC NOTICE **

      The Navy Topic above is an "unofficial" copy from the Navy Topics in the DoD 23.2 SBIR BAA. Please see the official DoD Topic website at www.defensesbirsttr.mil/SBIR-STTR/Opportunities/#announcements for any updates.

      The DoD issued its Navy 23.2 SBIR Topics pre-release on April 19, 2023 which opens to receive proposals on May 17, 2023, and closes June 14, 2023 (12:00pm ET).

      Direct Contact with Topic Authors: During the pre-release period (April 19, 2023 through May 16, 2023) proposing firms have an opportunity to directly contact the Technical Point of Contact (TPOC) to ask technical questions about the specific BAA topic. Once DoD begins accepting proposals on May 17, 2023 no further direct contact between proposers and topic authors is allowed unless the Topic Author is responding to a question submitted during the Pre-release period.

      SITIS Q&A System: After the pre-release period, until May 31, (at 12:00 PM ET), proposers may submit written questions through SITIS (SBIR/STTR Interactive Topic Information System) at www.dodsbirsttr.mil/topics-app/ by logging in and following instructions. In SITIS, the questioner and respondent remain anonymous but all questions and answers are posted for general viewing.

      Topics Search Engine: Visit the DoD Topic Search Tool at www.dodsbirsttr.mil/topics-app/ to find topics by keyword across all DoD Components participating in this BAA.

      Help: If you have general questions about the DoD SBIR program, please contact the DoD SBIR Help Desk via email at [email protected]

      [ Return ]