PacMar Technologies LLC — Department of Defense SBIR Phase II: N193-A01

Description

The Navy seeks new certification and testing solutions for artificial intelligence and machine learning (AI/ML) technologies to increase trust, understanding, and reliability for use of these technologies by the warfighter. Recent advancements for autonomous control, such as deep reinforcement learning, offer the ability to scale to previously unsolvable problems. However, the ability of the Navy to leverage these advancements is limited because methods do not exist to test and certify that these systems will reliably achieve mission goals. An underlying problem is that autonomous agents might act incorrectly in environment states that are poorly represented in the training/simulation data, and their learned behavior is opaque to human understanding. Therefore, the Navy requires way to certify a system is ‘deployable’ or reliable enough given the amount of testing it has undergone and the competency of its performance observed per task. The Navy C4/ISR community seeks to develop AI-based adaptive network routing algorithms to improve communications and data exchange between submarines, shore facilities, and rest of the fleet. In our proposed Phase II effort, we will develop the testing and certification capability that the C4/ISR community needs to ensure that future AI-based adaptive network routing algorithms are deployable and reliably meet mission goals. Our certification approach for these AI/ML systems consists of (1) ensuring satisfactory performance on a collection of intelligently-selected test scenarios and (2) guaranteeing the coverage of test scenarios is broad enough to confidently predict performance in un-tested scenarios. We propose a novel certification approach of AI/ML techniques for autonomous agent control leveraging dynamically constructed Markov Decision Processes (MDPs). An MDP is a mathematical framework for decision making, that when solved, produces a policy and a value function. The policy describes the best action to take in each state, while the value function is a measure corresponding to the likelihood of achieving the goal from each state. We will use execution traces of the AI/ML system in given test scenarios to construct an MDP, then use the resulting policy and value function to (1) evaluate the degree of optimality of the agent’s performance in the given test scenarios and (2) identify gaps in the test scenario’s coverage of agent behavior followed by recommendations for new test scenarios. The use of MDPs in the certification process is significant because it provides a baseline from which to measure the performance of the AI/ML algorithm being tested with a rigorous mathematical basis from which to grade optimality. Due to our innovative use of MDPs, we refer to the certification capability to be developed in Phase II as the AI-based MDP Uncertainty Reduction tool, or AIMUR Certification Tool.