Robustness of artificial intelligence in the face of novelty
Douglas S. Lange · 2022
A critical factor in utilizing agents with Artificial Intelligence (AI) is their robustness to novelty. AI agents include models that are either engineered or trained. Engineered models include knowledge of those aspects of the environment that are known and considered important by the engineers. Learned models form embeddings of aspects of the environment based on connections made through the training data. In operation, however, a rich environment is likely to present challenges not seen in training sets or accounted for in engineered models. Worse still, adversarial environments are subject to change by opponents. A program at the Defense Advanced Research Project Agency (DARPA) seeks to develop the science necessary to develop and evaluate agents that are robust to novelty. This capability will be required, before AI has the role envisioned within mission critical environments.