SBIR-STTR Award

Team AIS proposes Model Annotation eXtension With Exploratory Label Learning (MAXWELL), an applied research effort to develop a semi-automated framework to annotate simulation models for increased utility and efficiency of model composition. MAXWELL will achieve this by combining assistive Natural Language Processing (NLP), automated model modification and behavior analysis, and a guided semi-automatic analyst annotation workflow. Simulation models may come from multiple platforms and the model assets available for analysis and information extraction may vary wildly. MAXWELL will overcome the challenges associated with diverse information sources by structuring each model’s unique attributes into a universal model annotation formalism, allowing the analyst to use semantic queries to discover all types of models from various frameworks. This conversion will be performed via automating the extraction of information and annotation as much as possible while keeping analysts in the loop, enabling them to make additional annotations and corrections that inform automation improvements. MAXWELL will efficiently define the attributes of a model in an extensible, integrable model annotation formalism that concentrates on model aspects relevant to semantic understanding and composability while respecting security classification. These attributes will aid in identifying suitable existing models and determining required modifications and compositions to meet a simulation requirement.

Team AIS proposes Phase II of MAXWELL, an applied research and development effort to extend the proof-of-concept capabilities from MAXWELL Phase I. The overall goal of MAXWELL is to create a capability to semi-automatically annotate models with semantic information, including model behavior and components, to streamline scenario composition and solve analytical problems. MAXWELL Phase I developed a proof-of-concept formalism to capture multi-domain (i.e., kinetic and non-kinetic) modeling scenarios, including discrete-event simulation, agent-based modeling, continuous-time modeling, and cyber assets. Additionally, MAXWELL Phase I demonstrated the use of Natural Language Processing (NLP) to deconstruct a basic semantic query and return model results rapidly. The NLP semantic query capability can query against over 1,000 models and return results in 0.00075 seconds, exceeding the Phase I benchmark by over 250 times the goal of twenty seconds. While Phase I provided a suite of proof-of-concept capabilities and technologies, further research and development remains to provide the analyst with a utility that enhances their workflow. This includes additional research for semantic decomposition of queries and diverse modeling assets (e.g., datasets, source code, scripts, etc.), testing and refinement of the formalism, and engineering to tie all the technologies together for integration into the Secure Advanced Framework for Simulation and Modeling (SAFE-SiM) platform. MAXWELL Phase II will continue the research and development performed under Phase I to build a framework and mature the capabilities for integration into the SAFE-SiM platform. Using a newly developed user story, MAXWELL will identify and address shortcomings in the Phase I proof-of-concept technologies, including additional representation of multi-domain and cyber scenarios. Further, MAXWELL Phase II will use NLP techniques to glean higher-level model characterizations that are not explicitly stated (e.g., model type) from low-level model details (e.g., technical details about model parameters). Improving and maturing the NLP capabilities will further enhance the decomposition of semantic queries into decidable parts. Phase II will also extend the model gym beyond proof-of-concept to efficiently exercise simulation models and provide decidable behavioral, time, space, and resource annotations to be used by the sematic queries. Additionally, MAXWELL will use model information to create an evaluation framework at the composition level, making use of annotations from the Smart Model Repository (SMR). For each scenario, the level of effort to build the composition of models will be predicted based on comparing overlapping attributes between models in sets of returned models. This approach will mature the Phase 1 prototype into a full framework capable of integration with SAFE-SiM to improve analyst’s workflow for scenario composition.