CLARIFAI, INC. — Department of Defense SBIR Phase II: NGA201-006

Description

The NGA program “GLIMPSE” leverages context and topography to geolocate imagery for further analysis. For this proposal, Clarifai intends to develop and deliver a deep-learning pipeline to reduce the geographical search space for an image that will expedite analysis and reduce computational cost. The objective of this proposal is to provide a system to (a) efficiently identify relevant imagery and extract context and landmark features, and (b) support AI-assisted region reduction. Clarifai's commercial AI platform will provide the National Geospatial-Intelligence Agency with a Computer Vision capability to triage imagery, extract valuable contextual information, and support region reduction efforts. Clarifai will leverage foundational image classification models combined with custom models trained on features identified by the MIT research project “Places365” and any newly identified relevant features, to extract scene information, context, and landmarks from an image. Clarifai’s workflow capability will enable the orchestration of these models along with custom logic operators to perform complex region reduction tasks based on results from multiple feature extraction models.  A major challenge with developing an automated solution for region reduction is developing the feature extraction capabilities and logic to mimic the complex human reasoning used for geolocating ground-level imagery. With current state-of-the-art deep learning capabilities, it would require extreme amounts of data and labelling for every region of interest to train a single model to directly classify the region. However, a human-AI combination approach could dramatically improve analyst efficiency and accuracy. Depending on the image and location, an analyst will use different combinations of contextual information to estimate the location of an image. For example, if a street sign or unique landmark is present in an urban environment, that may be the most important information. In contrast, the imagery of a natural environment, scene, and biome information may play a more important role. These two types of information are most optimally determined in different ways. Scene and biome identification is generally a classification problem. In contrast, there would be too many unique landmarks to train a classification model. A similarity or clustering approach would be more effective for identifying these landmarks. Furthermore, fine-tuning additional classification models for specific tasks would allow the system to rapidly adapt to new mission requirements and features of interest (e.g. hurricane damage, weapons, etc.) to support region reduction and targeting of images for further analysis. By orchestrating models optimized for different tasks and developing logic-based analytic approaches based on reference data and subject matter expertise, an automated system could reduce the overall region search space and provide relevant information with associated confidence.