The 2nd Workshop on Deep Models and Artificial Intelligence for Defense Applications: Potentials, Theories, Practices, Tools and Risks


November 11-12, 2020, Virtual

Proceedings at

(Check for the registration information)

Sponsored by the Association for the Advancement of Artificial Intelligence
In cooperation with the Stanford University Computer Science Department

(See the AAAI announcement

Schedules  (Eastern Standard Time)

AAAI 2020 Spring Symposium Series
(The 2nd Workshop on Deep Models and Artificial Intelligence for Defense Applications: Potentials, Theories, Practices, Tools and Risks)


Nov. 11, 2020:

10:00-10:30 opening remarks (30m, organizers’ view challenges of defense applications, Ying, Bonnie, Arjuna, Doug)

10:30-11:30 invited talk 1 (60m)


Dr. Eric Vorm, Lead of the DARPA’s  XAI Evaluation Team, Aerospace Experimental Psychologist, Naval Research Laboratory


Topic: DARPA's Explainable AI (XAI) Program and beyond: the importance of XAI in the DoD, and real-world challenges the DoD will face in the very near future if this research is not given priority. Human-autonomy interaction, human-machine teaming, trust in sociotechnical systems.


11:30-11:45 coffee (15m)

11:45-13:30 (105m) paper presentations (105m, 4 papers, 25 mins each)

1.      Sean Soleyman and Deepak Khosla: Multi-Agent Mission Planning with Reinforcement Learning

2.      Zhaoyuan Yang, Naresh Iyer, Johan Reimann and Nurali Virani:  Backdoor Attacks in Sequential Decision-Making Agents

3.      Edwin Stevens and Ying Zhao: Using Lexical Link Analysis as a Tool to Improve Sustainment

4.      Stanislav Seltser and Arkady Godin: Towards common integration infrastructure for AI-assisted systems


13:30-14:15 lunch (45m)

14:15-15:15 invited talk 2 (60m)


Dr. Una-May O'Reilly and Dr. Erik Hemberg, Principal Research Scientist and Research Scientist, MIT Computer Science and Artificial Intelligence Lab (CSAIL)

Topic:  Scalable Machine Learning, Evolutionary Algorithms, and Frameworks


15:15-15:30 coffee (15m)

15:30-17:30 paper presentations (120m, 5 papers, 25 mins each )

1.      Bonnie Johnson: Predictive Analytics in the Naval Maritime Domain

2.      Sam Ganzfried, Conner Laughlin and Charles Morefield: Parallel Algorithm for Approximating Nash Equilibrium in Multiplayer Stochastic Games with Application to Naval Strategic Planning

3.      Erik Blasch, James Sung and Tao Nguyen: Multisource AI Scorecard Table

4.      Capt John (Bud) Geldmacher, Applications of Deep Learning


Nov. 12, 2020:
10:00-10:30 opening remarks (30m, organizers’ view challenges of defense applications, Tony, Erik, Treadway)

10:30-11:30 invited talk 3 (60m)


Dr. Mike van Lent, CEO, Soar Technology, Inc.


11:30-11:45 coffee (15m)

11:45-13:15 paper presentations (75m, 3 papers, 25 mins each)

1.      Chris Michael, Dina Acklin and Jaelle Scheuerman: On Interactive Machine Learning and the Potential of Cognitive Feedback (

2.      James Chao, Jonathan Sato, Crisrael Lucero and Doug Lange: Evaluating Reinforcement Learning Algorithms for Evolving Military Games (

3.      Ying Zhao and Bruce Nagy: Causal Learning in Modeling Multi-segment War Game Leveraging Machine Intelligence with EVE Structures

13:15-13:30 lunch (15m)

13:30-14:30 invited talk 4 (60m)


Dr. Josef Schaff, DSc, Mission Systems, NAVAIR Associate Fellow, NAWCAD Lead Autonomy Architect, Principal S&T investigator

Topic: The Holistic Battlespace: Why the Key to Resilience for AI/ML Algorithms is to Leverage Complexity Science.


14:30-14:45 coffee (15m)

14:45-16:15 invited speakers' panel (90m)





Possible post workshop publication format:


1.      AI Magazine

2.      Springer Book

3.      Online via Arxiv



Background and Objectives 

Increasing global threats and competition have underscored the importance of artificial intelligence (AI) research and applications to confront adversaries.  Warfighters in all domains (land, sea, air, undersea and cyber) utilize AI to support smart sensors, robotics, cyber honey pots, virtual swarms, and war games. Furthermore, multi-domain decision makers from central command centers to the distributed tactical edge also seek trusted AI as assistants and automation tools to overcome cognitive overload.  These domains underscore the warfighters need for both perceptive and reasoning AI to help in modern combat.  This need has motivated the US Department of Defense Strategy (2018) to state that AI and resultant defense applications are the “very technologies that ensure we will be able to fight and win the wars of the future.”


In 2017, the first AAAI Fall Symposium titled “Deep Models and Artificial Intelligence for Defense Applications (DMAIDA): Potentials, Theories, Practices, Tools and Risks,” resulted in two volumes of the 2019 Special Issue of AI Magazine (Volume 1 and Volume 2). In addition, discussion and collaboration continued after the symposium. The 2nd DMAIDA workshop will be held at the AAAI 2020 Spring Symposium in Palo Alto, California.


Ubiquitous sensors such as “smart cities,” “smart seas,” “smart homes,” and the “Internet of Things” (IoT) introduce a wide range of industrial and societal challenges with an overwhelming large volume, high speed data, and type variety of data (e.g., databases, image, text, audio, and video). Concurrently with the “explosion” of the data volume, the excitement of handling increasingly larger “big data” has generated significant breakthroughs of data models, cloud computing, parallel and distributed computing, and more importantly, advanced deep analytics including machine learning (ML) and artificial intelligence (AI) algorithms.


Deep analytics algorithms have transformed AI capabilities to near human or super human intelligence and capabilities. AI capabilities are categorized by different types of learning, such as: supervised learning, which requires labeled big data for backpropagation; unsupervised learning for data mining, data discovery, statistical pattern recognition, and anomaly detection; semi-supervised learning for accelerated training, transfer learning, and changing scenarios; and reinforcement learning which requires big data with trials and rewards. The reinforcement learning algorithms are used broadly, including Markov decision process (MDP), genetic algorithms, game-theory, control theory, temporal difference, and sequential learning.  Due to these breakthroughs, Deep analytics and AI support an abundant set of applications in the commercial world which demonstrate enormous potential but there are still many intractable challenges. For example, academic and industrial AI applications focus on machine vision, speech recognition, chat understanding, and autonomous driving that achieve amazing automation and accuracy that surpass human experts.  However, these existing industrial applications may not adequately address or transfer to specific defense problems with intelligent adversaries, sparse data, and unique sensors.


The DMAIMA symposium will explore the potentials, theories, practices, tools and risks related to deep models, AI and networks in defense specific applications. For example, the DoD’s challenges include not only the volume/velocity of big data but also veracity/variation from bad/corrupted or no data. Adequate labeled samples for classification tasks may be lacking and therefore alternatives of synthetic and simulation data are needed. Furthermore, tactical environments often involve shorter time scales and fewer resources for learning. Also, unlike many commercial applications, defense applications access data sources stored in a distributed environment that are fused and analyzed together to form a coherent and holistic battlespace picture.   Defense data analytics from multi-source data include requirements of real-time, high rates, and limited channels; but also subject to strict security across all domains.


In summary, the four of the main challenges in using defense AI include (1) lack of adequate samples for classification learning, (2) short time scales for adaptive learning, (3) less computational resources for multi-source edge learning, and (4) adversarial behavior for robust learning.  A representative scenario task is to identify a rarely observed object for novel mission planning and therefore there is little training data; relatively little time to integrate recent observations into the training, contains only a network of high-powered desktops for training, and adversaries are trying to jam or corrupt the sensors.  For these reasons, fast optimization methods, generative modeling, and transfer learning methods are of particular interest.  Since resources are always limited, the research community needs to discuss how data sciences, ML/AI, and multi-domain fusion can be applied to different levels of defense operations such as strategic, operational, and tactical operations.


While the potential is great, the risks may not be trivial either, which require policy positions and discussions.  What are the risks that challenge or potentially compromise fundamental human capabilities in the long-run by applying AI technologies? How will AI shape the manpower requirements and costs for the future defense organizations and systems? What level is necessary for explainable learning/decision-making, as well as human-in-the-loop AI. What are the ethical and legal consequences of using this technology? Where are the boundaries if any? What quality assurance approaches are relevant for defense applications?

The goal of the workshop or working group is to foster collaborations and form communities for the theories and practices of deep models to defense applications. We solicit unclassified research, papers, and innovative ideas in the following areas (not limited to) for defense applications.


What are the potentials, theories, practices, tools and risks using the following Deep Models (i.e., models with large number of parameters that can be trained by Big Data)?

·         Deep data fusion models

·         Various types of machine learning models (e.g., supervised learning reinforcement learning, and unsupervised learning).

·         Deep learning models such as deep machine vision and image processing models

·         Pattern recognition and anomaly detection algorithms

·         Generative Adversarial Networks (GANs)

·         Network models

·         Graph models

·         Game theory models

·         Link analysis models

·         Parallel and distributed computing models

·         Smart data outputs from deep analytics

·         Visualizations and depictions of smart data outputs

·         Decision making models

·         Cognitive models

·         Using AI and human capabilities fused and optimized together, or is there optimized human-in-the-loop AI?

·         Advanced optimization algorithms and online learning

·         Cyber security,

·         Open AI

·         Legal and ethical considerations

·         Evaluation and Assessment considerations

Format of Symposium

The workshop will consist of keynote talks, invited talks, tutorials, oral presentations, poster/demo presentation, and panel discussions.
We will also invite a panel of experts from defense organizations, funding agencies and contractors to discuss these topics as a part of the workshop activities.



Confirmed Invited Speakers

1. Dr. Eric Vorm, Lead of the DARPA’s  XAI Evaluation Team, Aerospace Experimental Psychologist, Naval Research Laboratory

Topic: DARPA's Explainable AI (XAI) Program and beyond

2. Una-May O'Reilly, Principal Research Scientist, MIT Computer Science and Artificial Intelligence Lab (CSAIL)

Topic:  Scalable Machine Learning, Evolutionary Algorithms, and Frameworks

3. Dr. Josef Schaff, DSc, Mission Systems, NAVAIR Associate Fellow, NAWCAD Lead Autonomy Architect, Principal S&T investigator

Topic: Cyber, autonomy (AI/ML), E.M. Theory, complexity science and emergence.

4. Mike van Lent, CEO, Soar Technology, Inc.

Topic: TBD


Important Dates


Regular papers should be 6-8 pages. Position papers should be 2-4 pages; submitted to


Ying Zhao, Ph.D.
Research Professor
Information Sciences Department
Naval Postgraduate School
Monterey, CA 93943

Organizing Committee

Doug Lange, Ph.D.

Distinguished Scientist for Machine Learning/Artificial Intelligence, SSTM

Naval Information Warfare Center, Pacific


Bonnie Johnson, Ph.D.
Systems Engineering
Naval Postgraduate School
Sandy Spring, MD


Tony Kendall
Information Sciences Department
Naval Postgraduate School
Monterey, CA 93943


Erik Blasch, PhD, MBA

Air Force Office of Scientific Research

IEEE Fellow, SPIE Fellow, AIAA Associate Fellow


Arjuna Flenner, Ph.D.

Advanced and Special Programs

GE Aviation Systems


Bruce Nagy

NAVAIR China Lake


Richard Arthur

Captain, US Navy

NAVAIR China Lake, CA


Symposium URL