The Defense Advanced Research Projects Agency (DARPA) recently transitioned newly developed capabilities focusing on the defense of our military AI, to the Chief Digital and Artificial Intelligence Office, a senior official stated.
The GARD Program
DARPA’s Guaranteeing AI Robustness Against Deception (GARD) program, announced in January 2022, has been seeking to establish a theoretical military learning system to identify system vulnerabilities, enhance system robustness, and encourage the creation of effective defenses, said Dr. Alvaro Velasquez, a program manager at the agency.
“That is a program that’s focused on building defenses against adversarial attacks on AI systems,” Matt Turek, deputy director for DARPA’s Information Innovation Office, said Wednesday during a virtual event hosted by the Center for Strategic and International Studies.
The need for defensive capabilities for systems that can be tricked, which include machine learning technologies is finally being recognized by U.S. military officials.
Software defenses
“AI systems are made out of software, obviously, right, so they inherit all the cyber vulnerabilities — and those are an important class of vulnerabilities — but [that’s] not what I’m talking about here,” stated Turek.
“There are sort of unique classes of vulnerabilities for AI or autonomous systems, where you can do things like insert noise patterns into sensor data that might cause an AI system to misclassify,” Turek continued. “So you can essentially, by adding noise to an image or a sensor, perhaps break a downstream machine learning algorithm. You can also with knowledge of that algorithm sometimes create physically realizable attacks.”
According to Turek, it’s possible to “trick” an AI’s algorithm using something as simple as a patch, or a sticker, to misidentify objects. This could cause an AI software to misclassify enemies as allies, and vice versa.
Imagine a bus being identified as a tank, or a tank being identified as a school bus.
The GARD program has allowed DARPA to work with industry partners in order to develop algorithms and other capabilities to deter manipulation or trickery of these systems.
“Whether that is physically realizable attacks or noise patterns that are added to AI systems, the GARD program has built state-of-the-art defenses against those. Some of those tools and capabilities have been provided to CDAO,” Turek said, referring to the Chief Digital and AI Office.
The Defense Department formed the CDAO in 2022 to serve as a hub to accelerate the adoption of AI and related technologies across the DoD.
DARPA’s mission
“DARPA’s core mission [is to] prevent and create strategic surprise,” Turek explained. “So the implication is that we’re looking over the horizon for transformative capabilities. So in some sense, we are very early in the research pipeline, typically.”
“Products that come out of those research programs could go a couple places … Transitioning them to CDAO, for instance, might enable broad transition across the entirety of the DOD,” Turek said. “I think having an organization that can provide some shared resources and capabilities across the department [and] can be a resource or place people can go look for help or tools or capabilities — I think that’s really useful. And from a DARPA perspective, it gives us a natural transition partner.”
Turek also stated that DARPA has other strategies to help other organizations outside of the Defense Department.
Outside the DoD
“We have created new algorithms — some of those actually in partnership both with the research teams that we’re funding but with researchers at Google — and then created open-source tools that we can provide back to the broader community so that we can really raise defenses broadly in AI and machine learning,” Turek said. “But those tools [are] also provided to CDAO and then they can be customized for DOD use cases and needs.”
The GARD program is only one of DARPA’s AI programs. DARPA, created in 1958, has been working on AI since its inception. Recently, the program has gained momentum and roughly 70% of the programs have AI or ML aspects to them.
“There is really broad penetration across the agency. So it’s really difficult to sum up, you know, what the agency as a whole is up to, but from an [information innovation office] perspective, we’re really looking to try and advance … how do we get to a highly trustworthy AI that we can bet our lives on and [ensure] that not be a foolish thing to do,” Turek said.
