The Department of Veterans Affairs (VA) is modernizing its finance and acquisition systems by implementing the Integrated Financial and Acquisition Management System (iFAMS), but a new watchdog report reveals major gaps in usability testing that could jeopardize future implementation.
The VA is implementing iFAMS as part of its $8.6 billion Financial Management Business Transformation (FMBT) program. iFAMS is the VA’s third effort to replace its legacy financial management system since 1998.
As of June 2023, VA completed “five waves” of implementation, which the VA Office of Inspector General (OIG) assessed in a report published on Thursday.
The OIG explained these early deployments represent just 3.8 percent of total anticipated iFAMS users. Seven implementation waves remain, including the Veterans Health Administration (VHA) – which alone comprises more than 92 percent of VA’s total iFAMS users.
“Overall, the OIG determined the system implementation timeline limited the ability to fully test user requirements,” the report says. “If usability testing for key interfaces had been completed before going live, [VA] may have discovered many of the critical interface challenges users experienced after deployment.”
According to the report, when the VA held “validation sessions” to make sure iFAMS worked properly, the examples the agency used didn’t reflect “actual business processes” closely enough for users to identify gaps.
Later, during “user acceptance testing,” not all end-to-end processes were tested either.
As a result, the VA did not have “reasonable assurance the system functioned properly to meet users’ needs for some interfaces,” the OIG said. Specifically, the Electronic Contract Management System, SAM.gov, and the Invoice Payment Processing System interfaces were not included.
The OIG determined that issues with the testing environment also impeded users’ ability to assess the system’s functionality.
“In short, the team concluded that the development process during the audit period did not provide opportunities for users to confirm their needs would be met,” the report says. “Consequently, VA acquisition staff experienced significant limitations using iFAMS that resulted in manual work-arounds.”
The OIG said those lapses in testing could delay user adoption, hinder change management, decrease productivity and efficiency, and increase the risk of not meeting iFAMS’s goals.
“By better identifying and incorporating all business-essential processes within the development cycle, FMBTS can ensure iFAMS meets user needs at go-live moving forward,” the report says.
The watchdog made four recommendations to the VA, including that it identify and incorporate “all business-essential processes within the development cycle.”
It also recommended “enhancing the test plan to incorporate user-testing requirements for functional and nonfunctional business-essential processes related to interfaces, developing a process to confirm that affected administrative offices are aware of needed changes to test environments and have executed the modifications before interface testing, and establishing a method to evaluate whether test deficiencies warrant changes to the deployment schedule to ensure they are properly addressed before going live with another wave and then implement those changes.”
The VA concurred with all of the recommended actions, which the OIG wants the VA to take during the next implementation wave.