DARPA sets sights on making AI self-aware of complex time dimensions

The Defense Advanced Research Projects Agency (DARPA) is setting its sights on developing an AI system with a detailed self-understanding of the time dimensions of its learned knowledge.

DARPA’s Time-Aware Machine Intelligence (TAMI) research program and incubator is looking to develop a new class of neural network architectures that incorporate an explicit time dimension as a fundamental building block for network knowledge representation,” according to the TAMI program solicitation.

The overall goal is to create an AI system that will be able to “think in and about time” when exercising its learned task knowledge in task performance.

The Challenge

Current neural networks do not explicitly model the inherent time characteristics of their encoded knowledge.

Consequently, state-of-the-art machine learning does not have the expressive capability to reason with encoded knowledge using time.

The Proposed Solution

TAMI’s vision is for an AI system to develop a detailed self-understanding of the time dimensions of its learned knowledge and eventually be able to “think in and about time” when exercising its learned task knowledge in task performance.

How and Why

Large amounts of data samples are needed to feed neural networks; however, each data sample exists only in a specific time frame.

To understand what this means and looks like, the solicitation points out:

Consider neural networks designed for inference. Such neural networks derive abstract task knowledge from the analysis of a large number of data samples.

Each data sample exists only in a specific time. For example, features given by a vehicle data sample are associated with that specific vehicle’s age (e.g., rust and dents) and, therefore, are explicitly dependent on time.

Neural networks incorporate such information as static activation weights; however, using the example above, the activation of these weights should ideally be conditioned on time.

What DARPA wants, according to the solicitation, is a learning mechanism that enables “a self awareness of the complex time-conditioned property of neural networks’ knowledge encoding.”

The TAMI research program will have two phases:

1. Feasibility Study: Performers will develop theories and computational methods to answer fundamental questions regarding time cognition in machine learning
2. Proof of Concept Demo: Performers are expected to prototype time-aware meta-learning methods into computational models and demonstrate whether the new model could provide novel machine intelligence capabilities that current state-of-the-art machine learning architectures cannot achieve.

For the feasibility study, DARPA seeks answers to such questions as:

• How can time attributes co-evolve with task learning itself?
• What association mechanisms should be used to represent the interactions between the time dimension and the other dimensions of the encoded knowledge?
• How should implicit time-dependent information not directly observable in the data be captured?
• And more

In a nutshell, the TAMI program will look to develop new, time-aware neural network architectures that introduce a meta-learning capability into machine learning, and this meta-learning will enable a neural network to capture the time-dependencies of its encoded knowledge.

TAMI Inspired by Time Processing Mechanisms in Human Brains

According to the solicitation:

TAMI draws inspiration from ongoing research on time processing mechanisms in human brains.

A large number of computational models have been introduced in computational neuroscience to explain time perception mechanisms in the brain.

TAMI will go a step further from such research to develop and prototype concrete computational models. TAMI will leverage the latest research on meta-learning in neural networks.

TAMI Program Manager’s Background and Experience

While the TAMI program solicitation did not mention how the research would translate into real-world applications for the Department of Defense (i.e. as part of department-wide AI adoption, for use in autonomous vehicles, weapons systems, drone swarms, surveillance, etc.), perhaps some background on the program manager might offer a few clues for the reader to infer.

Dr. Jiangying Zhou is leading the TAMI program, and she has been a program manager for DARPA since November, 2018.

She is also the program manager of at least four other DARPA research programs:

• Revolutionary Enhancement of Visibility by Exploiting Active Light-fields (REVEAL) — to develop a comprehensive theoretical framework to enable the development of new imaging hardware and software technologies.
• Competency-Aware Machine Learning (CAML) — to make AI and Machine Learning systems more trustworthy by programming systems to communicate their decision-making and strategies with their human counterparts.
• Artificial Intelligence Research Associate (AIRA) — to elevate AI to the role of an insightful and trusted collaborator in the scientific process.
• Nature as Computer (NAC) — to “crack computation problems unsolvable by classical models, such as developing simulations for hypersonic flight, materials for massively distributed sensing and control, and robust network optimization and analysis.”

Combined, the programs that Dr. Zhou leads have to do with making AI more robust and trustworthy while pushing the limits of imaging, sensing, and computational technologies, which is pretty much the whole aim of DARPA’s larger Artificial Intelligence Exploration (AIE) program to turn machines into collaborative partners for national defense.

According to Dr. Zhou’s bio, her areas of research include:

• Machine Learning
• Artificial Intelligence
• Data Analytics
• Intelligence, Surveillance and Reconnaissance (ISR) Exploitation Technologies

Previously, Dr. Zhou spent over 10 years as an engineer at Teledyne Scientific and Imaging (a subsidiary of Teledyne Technologies Inc) where she worked on “sensor exploitation, signal and image processing, and pattern recognition,” for public and private entities.

Currently, Teledyne Scientific and Imaging is comprised of:

• Teledyne Scientific Company
• Teledyne Imaging Sensors

The Teledyne Scientific Company specializes in:

• Advanced wireless systems for a multitude of uses: battlefield surveillance and factory monitoring
• 3D video and audio environments for applications in augmented and virtual reality
• Lip reading, eye tracking, and speech recognition to facilitate hands free control of computer functions in battlefields and call centers
• And more

Teledyne Imaging Sensors bills itself as a leader in high performance imaging systems for military, space, astronomy, and commercial applications that include:

• Infrared & visible sensors
• Read-Out Integrated Circuits
• Infrared scientific and tactical cameras
• Camera electronics embedded with advanced algorithms
• Laser eye and sensor protection devices & filters

Last year, another Teledyne subsidiary, Teledyne Instruments, was awarded a $22 million contract to supply the US Navy with autonomous underwater vehicles (AUVs) and related monitoring and communications acoustic systems.