The Government Accountability Office’s (GAO) Innovation Lab is building its own generative AI models to help solve some of the problems GAO’s mission teams face, according to the agency’s first chief data scientist and director of the Innovation Lab.
During his keynote speech at an IBM event in Washington, D.C., today, Taka Ariga highlighted four AI projects GAO is spearheading that will help to further the agency’s mission.
The first project he highlighted was Project Holodeck, which is leveraging extended reality to enhance GAO employees’ ability to collect evidence.
“One of the exciting parts for us is the ability to embed a computer vision model inside these goggles so that when we’re visiting hospital settings, for example, we can automatically recognize medical devices,” Ariga said. “So, to us, that’s an exciting evidence collection enhancement capability” especially in foreign areas where GAO does operate on site, he said.
The second AI project the Innovation Lab lead highlighted was Project ToM – a topic modeling tool.
“GAO has a very large body of work that requires us to look at textual data. So, one example is looking at regulations.gov where publics are allowed to submit comments around proposed legislation,” Ariga said.
“There are a lot of good technical comments that we’re interested in; there are probably some pictures of cats that we’re less interested in. So how do we use topic modeling to organize thematic material in a way that allows our analysts to address comments at the highest rapport,” he continued, adding, “An example of something like this places humans squarely in the center. We place our auditors; we’re just trying to focus their attention to areas of highest importance. But the humans are still very much in the loop.”
Project FACET is a tool that focuses on allowing GAO auditors to be forward-looking instead of reacting.
“This is looking at the Federal Audit Clearinghouse – so all of the single audits that are being submitted to GSA, how do we use machine learning models to predict, in a prospective way, likelihood of persistent material weaknesses and significant findings,” Ariga questioned. “This is the kind of thing that I think positions auditors to be less reactive, to be more forward-looking, and that’s where we want to be.”
Finally, Ariga highlighted GAO’s Project Galileo – the agency’s own large language model, which Ariga said they deployed last month.
“We recognize the opportunity for generative AI, but we don’t want to take the position of banning it like some other organizations have,” he said. “We want to deploy our own large language model inside our own computer infrastructure so that we control what comes in [and] what goes out.”
“Then we can actually experiment with a variety of learning techniques on how to point that generative AI to GAO’s corpus – that really rich historical data that we have,” Ariga said.
“We also have the fact that we don’t have the kind of budget to retrain these models – that takes tens if not hundreds of millions of dollars – so we really have to be creative in a way that not only harnesses the power of a pre-trained model, but making sure that they’re pointing at the corpus of data that we want them to be able to respond to,” he added. “And then the complexity here is making sure that something like a generative AI can produce outcomes in a way that are acceptable by auditors.”
The big final piece of leveraging a generative AI tool, Ariga said, is to ensure that GAO employees have the digital literacy skills necessary to accurately leverage the information the tool spits out at them.
“[It] underscores the importance of having the digital skill sets so that we can interpret and implement [AI tools] in a way that’s our mission and not the other way around.”
