Editor’s Note: The following story is taken from a book-length work authored by a senior Federal IT official currently working in government. This is one part of an extensive, firsthand account of how IT decisions are made, the obstacles standing in the way of real change in government technology management, and what one career Federal IT employee really thinks about the way government does IT.

Because the author is a current government employee and is concerned about the impact this may have on their career, we’ve agreed to publish this series of weekly excerpts under the author’s chosen pseudonym—Demosthenes.

MeriTalk has agreed not to make substantive changes to any of the chapters.

— Dan Verton, Executive Editor


The Gift of HealthCare.gov

HealthCare.gov is the gift that has the potential to keep on giving. While I am generally snarky in most of my comments, I genuinely mean that. Healthcare.gov, in all its glorious hellfire of disaster, can be a good thing overall for Federal IT. I suspect that most people won’t agree with me on this point, so please allow me to explain.

I mentioned that early in my career I was the project manager for a $10 million legacy redesign. It was a conversion from a mainframe system to an n-tier one, and it did not go well. We pulled the plug on the failure after spending $6 million. Eventually we hit the finish line with the remaining $4 million, but I was really mad about being taken to the cleaners for the initial failure. I took this setback personally. I was smarter and better than that failure indicated, but it nonetheless failed. As a result, I went back and took the time to understand all the dumb-ass stuff I did so that if I was in that situation again, I would make better decisions. My thinking was now colored by experience.

HealthCare.gov can be this type of gift for everyone else in Federal IT. It was a very public opportunity for you to bear witness to a lot of stuff that went wrong, and learn from those mistakes without making them yourself. There is so much that went wrong and so many common mistakes that I don’t know where to begin. I take that back. I know exactly where to begin. We start by acknowledging the opportunity to learn from this technological disaster. For all the waste, and the pain, and the political toll it cost, we all have a responsibility to LEARN from this and to endeavor to not repeat these mistakes. The real tragedy from HealthCare.gov would be if we then replicated these mistakes another 10,000 times.

I know that this is an impossible proposition. The world is full of noteworthy IT projects that went wrong. A researcher at the University of Virginia assembled a fantastic 12-page article identifying “Infamous Failures, Classic Mistakes and Best Practices.” One of the key big projects that he discusses in this article, which was published in 2007, is the National Health Service (NHS) from the United Kingdom. In it, he says, “The emerging story in this case seems to be a familiar one: contractor management issues. In fact, more than a dozen vendors are working on the NHS modernization, creating a ‘technological Tower of Babel.’ “1

That is a perfect quote. It is a true shame that nobody bothered to learn the lessons from the U.K. version of HealthCare.gov. If someone had, I likely wouldn’t be writing about it now. But because enough people didn’t learn those lessons, HealthCare.gov has been forever burned into the American psyche as the IT version of the Hindenburg or the Titanic.

The reason I call it a gift is because of the opportunity it has presented for so many people to learn these hard lessons by studying this case and not repeating them in their own projects. Just like I had to have a $6 million failure before I figured out the right way to do things, we have the opportunity to avoid hundreds or even thousands of these small (I use small here as a relative term, $6 million is a lot of money) failures if we take the time to really understand all of the stuff that was screwed up in this colossal failure. Thus from that perspective it is a gift and can actually save time, effort and money, but we must learn the lessons.

I have already touched on a few of these classic mistakes in previous chapters. I talked about change management already in the in the Agile chapter. Clearly this was a project in which they had a lot of change, but they didn’t have mechanisms that were effective in digesting that churn. So every time there was a change they had to go back and modify contracts. It is so stupid that we get out our chisels and carve requirements in granite. It’s like we are trying to make life more difficult. We should simply recognize that we lack the ability to describe what the final product should look like in the next two or three years. What we should do instead is describe the process that we will use; how many teams, how they interact with each other, and then iterate rapidly until we achieve acceptable levels of capabilities.

The really good news is that there is an IG Report2 on this system that lists all of the stuff that is worth learning from this project. I think this report should be required reading for everyone supporting IT projects in the Federal government. You should really commit to reading that full report, it is really good. I’ll highlight just a few of what I think some of the biggest issues are.

Page 7, “Integrating into a large organizational structure at CMS brought new challenges to the Federal Marketplace project, primarily caused by unclear project leadership.” This was a situation of too many cooks in the kitchen and no single person responsible for the whole enchilada. In truth, though, there was a single person who was supposed to be responsible for the whole enchilada as this article from Politico points out in June of 2010:

For Todd Park, working these days as chief technology officer at the massive Department of Health and Human Services feels more like working at a frantic Silicon Valley startup.

“In the startup world, you work 24/7, or at least 22/7, definitely nights and weekends,” Park explains. “And that’s exactly what we’ve been doing here.”

Park, who joined HHS in August 2009, is in charge of launching HealthCare.gov, the new Web portal that goes live July 1 and is designed to give consumers a place to research and compare health insurance plans. 3

Yes, more than three years before the eventual implosion of HealthCare.gov, Todd Park, who would later be on the cover of Time magazine for “rescuing” HealthCare.gov, was actually in charge of the program. He would later become the Federal CTO. But let’s not lose this particular issue, Todd was the leader in charge of this program. So, for me, I can relate to the symmetry that the person who was in charge of the biggest IT disaster of all time is also hailed as the savior. I watched the hearings on the system. I did not see the same contrition that I felt after my $6 million disaster. On my project, I accepted total and complete responsibility; it was MY fault, not anyone else. I didn’t hear that from Todd during his testimony.

It burns me a little to see that rather than be the teacher of all the lessons learned from this project, he is instead just recruiting people to USDS and 18F. If there is a person who saw first-hand how the government could and should change to make sure that this type of project never happens again it would be Todd. But rather than go out and be a mentor to the many big development projects we are currently working, he is working to outsource IT leadership to Silicon Valley. He is trying to effect change from the outside in, and I suspect he would be much more effective teaching current Feds who are running these projects right now how to avoid and overcome the obstacles that ruined HealthCare.gov. You aren’t going to replace the entire set of 2210s in the Federal government with people who contributed to an app. Play the game with the team you have; coach them into the team you want.

Read More About
More Topics
Demosthenes is a pseudonym for a senior Federal IT official.