DECISIVE ANALYTICS CORPORATION — Department of Defense SBIR Phase I: The intelligence value of a document goes far beyond the face value of its content1. Clues

Description

The intelligence value of a document goes far beyond the face value of its content1. Clues to the identity, worldview, and even the psychological state of its author are encoded in features such as word choice, sentence structure, and explicit and implied statements of group membership. Years of research have shown that statistical and linguistic methods can shed light on a substantial amount of information about the identity and characteristics of an author. But traditional analysis techniques have been investigated in isolation, on a small scale, and with limited variety in the target documents. The Author and Group Insight through Linguistic Expression (AGILE) approach to author analysis extends DAC"s text analytics platform by incorporating a variety of extensions to the standard set of stylometric features used to attribute authorship. AGILE uses DAC"s existing semantic and sentiment processing technology to extract discourse-based features that capture the way authors perceive themselves and their relationships with other entities. The Phase I effort demonstrates how discourse features can be extracted from a variety of on-line sources of English and Arabic text. A series of experiments evaluates the power of discourse features to cluster documents and authors according to their social identity and world view.