IT Transformation at EPA
The government is full of smart, talented people who have chosen to forego the potential financial rewards of the private sector because they believe deeply in public service and the mission of their agency. We often criticize government employees for their inability to get things done, while failing to recognize the tight bureaucratic constraints that they must work within. This article will discuss some of those constraints and the way EPA is using agile development practices to deliver transformational results. The adoption of agile practice and the associated changes to EPA’s development culture have enabled EPA’s IT teams to deliver solutions that meet user needs faster and at a lower cost than with traditional methods.
Software development in much of the government still follows a model that has proven ineffective at delivering quality results or low cost and, as a result, has been losing favor in the private sector for a while. This is the classic ‘waterfall’ software development model: In this model we define all of our requirements up front and, in the government, typically hire a contractor to build exactly what was defined. The product is built, tested and delivered with little stakeholder or customer interaction. This methodology is ineffective because it’s virtually impossible to know everything we want and all the issues that might be encountered in a complex system before we even start building that system. We end up spending too much time making plans that may be meaningless shortly after development starts, paying the vendor too much to cover the cost of uncertainty and taking too long to deliver a solution that rarely meets user needs. Highly defined processes, enforced by a number of oversight bodies, have prolonged use of the waterfall model. Waterfall has also persisted for so long because government procurement processes require a complete plan years before a line of code is written. It takes substantial will to move away from a model that is reinforced by every control in place in the system in which you work.
In a government organization, creating this product based team and deploying at high frequency may be the greatest cultural change of all
The prevalence of waterfall has been perpetuated by an extremely risk-averse working environment. It is ironic, that our government, an enterprise whose creation was the ultimate risk, has become so risk averse. But it is not surprising. As oversight has increased, so has the perceived cost of failure and, therefore, our desire to know exactly what we’re building and what it costs up front. We believe we are saving taxpayer money and ensuring success, but we are, in fact, doing exactly the opposite. We are increasing both cost and risk.
The Digital Services movement in government is designed to change that model, to move IT organizations from a waterfall culture to an agile culture. I use the word culture very deliberately because both waterfall and agile represent more than a development methodologies. Each one represents a mindset; a development culture. Changing from waterfall to agile without changing the culture is likely to result in nothing more than waterfall masquerading as agile, or as some like to call it, “agilefall” where all the waterfall preparation is performed and the team completes the development in carefully planned sprints that look like agile to the common observer, but aren’t. These sprints aren’t experiments that lower risk and they don’t deliver working deployable code at their completion. They are just closely spaced markers in a waterfall or, at best, incremental delivery project.
At EPA we have embraced several key concepts to help us move to a more agile culture. Our first cultural change was to eschew the idea that we must build everything. We have embraced a decision process that values reuse first. This means we look for shared services and commercial off the shelf products first and re-engineer our processes to fit those tools. We only build when we can’t buy something that meets our needs. When we build, we use shared platforms where possible and only build custom software from scratch when absolutely necessary.
The second cultural change is that our development process starts with our business process. We always streamline the process first, so that we are not automating inefficiencies and so that one process and, therefore one system, can be shared across the agency.
The third cultural change we made at EPA was to embrace risk. Our implementation of agile is designed to reduce lifecycle risk by explicitly taking on risk early in the project. We identify our greatest risks and develop solutions for those in our earliest sprints. That way we can fail fast, learn from our failure and find solutions that work. By dealing with the riskiest items up front, we reduce overall risk and increase the predictability of delivery later in the project. We also reduce the cost and impact of failure. A failed experiment is much less problematic when it is a four week sprint than when it is a four year project. With an agile approach, we can quickly change technical direction or cancel a project if it becomes apparent it will not yield value. We also embraced the idea that in order to know which experiments to undertake, we must talk to our customers and understand their needs. One of the major causes of failure of government IT projects has been the lack of user input. Successful systems are not designed around the needs of stakeholders, often a small percentage of system users or individuals who don’t use the system at all but rather around the needs of individuals who will use the system every day.
Our fourth cultural change saw the implementation of a number of practices that serve as tools in our efforts to deliver software efficiently and effectively. Any one of these changes can also be implemented in a waterfall world, arguably without any change to an organization’s culture, but when implemented together, they reflect a shift in organizational mindset. As a group, these practices help us reinforce the idea that we are fundamentally changing the way we work while also changing the way we think about product development. Further, each one is crucial when implementing true agile. These include an API-first strategy that enables another practice, modular development and redevelopment. These practices enable the practical migration of monolithic systems while eliminating the creation of new legacy systems. We focus on code reuse to improve efficiency and quality and are developing a set of shared services to serve as the foundation for our regulatory platform. Finally, we have embracedcloud-native development and the use of secure open source technologies.
All of these practices lead us to the end game of agile–DevOps. Getting to DevOps requires breaking downthe silos between development, operations and security. DevOps requires the collaboration of all three communities on a product team to enable continuous deployment to a cloud environment. In a government organization, creating this product based team and deploying at high frequency may be the greatest cultural change of all.
All of this is certainly a work in progress at the EPA. When it does work, amazing things happen. Teams experience tremendous moments of clarity about user requirements and those same users are delighted by rapid delivery of products that truly meet their needs.