SBIR-STTR Award

This proposal is for a Team Oriented Resource Management and Control (TORMAC) solution to the problem of distributed decision-making for resource constrained platforms in complex environments. As autonomous unmanned vehicles (UXV), sensor and manned vehicle capabilities have proliferated, it is more important to find ways of coordinating these assets towards common objectives in an evolving, communications degraded and potentially hostile environment. There is a clear need to be able to coordinate assets across domains, space, time and circumstances to meet common, joint and multiple objectives. The capability must be functional despite factors such as changing environmental conditions, tight and changing resource constraints, and actions with uncertain outcomes. A framework for distributed decision-making towards cooperative goals will have broad applicability to both civilian and military applications including in environmental, construction, health, and disaster response scenarios. The TORMAC approach features two key conceptual ideas: Partially Observable Markov Decision Processes (POMDPs) and Teamwork as formalized by our partners. POMDPs are a formalism and related algorithms for reasoning about courses of action under uncertain action and uncertain knowledge of an environment. Together, the planning provided by the POMDPs and the teamwork to react in a principled way to changing circumstances and will form the basis of the TORMAC framework.

  This proposal is to extend into Phase II for our development of Team Oriented Resource Management and Control (TORMAC), as a solution to the problem of distributed decision-making for resource constrained platforms in complex, adversarial environments. We are joined in this effort by Harvard University’s Center for Research on Computation and Society (CRCS), Teamcore Group. In Phase I we developed a scenario where rangers with UAVs and ground vehicles patrolled a wildlife reserve for poachers.  The rangers were supported by ground sensors listening for vehicles and drones-in-a-box placed strategically in the environment.  The objective of the rangers was to minimize poaching by finding poachers and tracking them when possible.  The adversarial poachers aimed to poach as many animals as possible, without being caught by the rangers.  This scenario features much of the complexity of more general problems, with spatial and temporal constraints on rangers, uncertainty about poachers, adversarial behaviors, and the need to react to stimulus. In Phase II our work will focus specifically on the game theoretic aspects of the cooperative reasoning.  There has been extensive study of adversarial bandit problems, where algorithms like Exp3 and its variants have proven excellent theoretical guarantees of low-regret learning against an adversary.