SBIR-STTR Award

Cyber operators mentally integrate numerous data sources, many whose native format is difficult to comprehend. Visualizations can help, but the raw data must first be transformed. Designing effective visualizations is also difficult because there is a dearth of empirical research on how various visualizations affect the cyber operator’s performance. Secure Decisions proposes to systematically research this problem starting with a knowledge elicitation of cyber operators to establish their information requirements and cognitive challenges, and developing visualizations to meet the operator’s needs. We have chosen to focus on alert triage/escalation, vulnerability assessment and incident response tasks; and more specifically on the mission impact decisions that operators have to make when engaged in these tasks. Establishing the mission relevance of an alert or vulnerability, or assessing the mission impact of an incident response is a hard problem that is not currently automated. The USAF Cyber Vision specifically cites visualization of cyber impacts on missions as a near-term goal. It involves understanding complex dependencies between network entities, users and missions. Because semantic ontologies are excellent at representing complex relationships, we will use that as our data transformation approach. Finally, we’ll design a Phase II experiment to objectively evaluate how visualizations affect operator performance.

Benefits:
The proposed work will make several contributions to the cyber defense domain: 1. A significant step forward in meeting the USAF Cyber Vision near-term goal of being able visualize cyber impacts on missions. 2. Deep insight into how cyber operators currently assess the mission relevance of alerts, vulnerabilities and incident responses and consider mission impact in their cyber defense decisions. This will include information about the analytical questions they ask, information needed to answer those questions, decision processes used, and data sources consulted. 3. Identification of cognitive challenges and other impediments to assessing mission relevance when performing alert triage/escalation, vulnerability assessment, or selecting courses of action in incident response. 4. Prototype visualizations designed to help cyber operators gain awareness of the mission relevance of alerts, vulnerabilities and incident responses 5. Visualization of the complex dependencies between network devices, users and missions that can be applied to emerging Department of Defense programs in cyber mission assurance and mission impact analysis. 6. Enhanced understanding of how visualizations can be used by cyber operators to assess the mission impact of various cyber defense activities. It will shed light on the insights that cyber operators gain and the types of visual patterns they look for when answering questions such as: Which of the myriad of alerts are occurring on mission-critical devices? Which of these vulnerabilities must be remediated before the mission can be assured? What users and operational roles will be affected by closing this port or blocking this IP address? 7. An assessment of the feasibility of using a semantic ontology as a data transformation method to represent complex relationships between network devices, users and missions. The feasibility study will identify changes required to improve its use with various visualizations and new data sources. 8. A fully described experimental design to be conducted in Phase II to assess the effectiveness of various visualizations on cyber defense activities.

Keywords:
cognition, computer network defense, cyber security, human computer interaction, knowledge elicitation, mission impact, semantic ontology, visualization

Cyber operations research typically focuses on technology solutions, with little regard for perceptual and cognitive capabilities of human operators. In our Phase I, we analyzed cyber operator tasks within the DoD’s Cyber Incident Handling Life Cycle (CIHLC), including information and data sources needed to perform those tasks. Integrating Goal Directed Task Analysis (GDTA) methods with CIHLC task decomposition, we modeled the cyber operator’s environment and its corresponding decision making requirements. We then created, and qualitatively assessed prototype transformations of low level data as visualizations that support operator decision making requirements. In Phase II, we will objectively evaluate insight gained during Phase I by conducting an experiment to quantitatively measure the effect interactive data visualization techniques have on operator analytic and decision making performance. We will implement a visualization prototype with data transformations, and an experimental interface that permits remote, asynchronous participation, enabling a disperse and diverse candidate population of experienced cyber operators to join the study. Outcomes from the Phase II effort will contribute significantly to understanding how visualizations can be used by cyber operators, and enable multiple transition opportunities for the visualizations and data transformations, as well as a cognitive architecture for visual analytics support to cyber operations.

Benefits:
WhyViz will add to the base of knowledge required to design and develop usable decision support tools that can enhance the cognitive capabilities of cyber operators. The insight gained can be inserted within data visualization products available in the commercial marketplace, as well as in government developed systems. Other derivative commercially applicable outcomes include consulting, training, and test & evaluation services that are enabled by the cognitive engineering and evaluation methodologies advanced through this work.

Keywords:
cognitive engineering, goal directed task analysis, cyber incident handling, human computer interface, visual analytic data transformation, defensive cyber operations, computer security data visualization, remote asynchronous experimentation