DYNAMIC OBJECT LANGUAGE LABS INC. — Department of Defense STTR Phase II: ST14B-003

Description

The new Phase 2 effort will focus on the role of stories to dramatically support and enhance social interactions, teaming, and dynamic situations in operationally relevant use cases.  We draw from story representation and understanding technology from both CART II, and the ASIST RITA projects.  CART II focused on the ability of stories to capture situational awareness, task monitoring and planning, while RITA uses stories to capture goals, motivations and participant mind states.  The focus of the current effort will be on developing more generalizable agent capabilities to deal with greater complexity and better support teamwork in operational settings. The effort will integrate agent technology across multiple synthetic environments, including the Minecraft testbed and the Unreal Engine, which offers significant scaling up of the complexity while staying within a research setting. Unreal Engine has long been used as a proxy for real-world interactions precisely because it offers greater visual realism, and provides a low cost way to test robotic and agent capabilities that can readily be transferred to real-world applications. This effort will build a message bus interface to Unreal Engine based on existing Unreal Engine research worlds identified within DoD research labs. This will allow technologies developed within the Minecraft world to be generalized to more complex renderings in the Unreal Engine testbed. In this way, more of the capabilities from the CART II level of understanding of what is happening in the simulated world and the team understanding developed in the DARPA ASIST program can be combined and transitioned into a DoD research lab. This effort will leverage technology and testbed from the ASIST program, focusing on assistive agents and coaching human teams to improve teamwork and performance outcomes. In place of a physical robot dealing with real-world vision to track what is going on, the effort will focus on a virtual assistant agent that observes the simulated world using messages on a communication message bus. In place of repair tasks, problems will focus on search and rescue operations, general search problems, and new tasks relevant to transition partners. Synthetic testbeds with different task designs will allow for the experimental development and testing of agents capable of supporting and enhancing human teams. The testbeds produced will allow for the testing of team building and enhancing strategies, as well as the development and testing of agents capable of executing such strategies.  Agent capabilities will focus on social interactions and agent coaching of team processes to improve performance outcomes, concentrating on the planning phase of the task. This effort will enhance capabilities around a cognitive artifact that can be used for team planning to support team coordination, enable machine observation, and enable intervention during team planning, coordination, and prioritization activities.