Thursday, November 28, 2013

Update: DoD Goes Agile

Update November 2013: SEI has posted a report analysing the legal documents that require Department of Defense to use agile practices. Defense contractors that Scrum Inc. is working with propose Scrum for most new projects. See also DoD CIO comments on agile development.

With waterfall failure after failure, even the government has decided to make Agile development a priority. Some people have a hard time believing this but I just want to take you through what happened in one department, the biggest one there is, the Department of Defense. Back in 2009 someone inserted this section into the 2010 Defense Acquisition Bill. These are the rules that the Department must follow when purchasing anything. Here’s the relevant section 804: IMPLEMENTATION OF NEW ACQUISITION PROCESS FOR INFORMATION TECHNOLOGY SYSTEMS.

The key language is this:
(2) be designed to include—
(A) early and continual involvement of the user;
(B) multiple, rapidly executed increments or releases of capability;
(C) early, successive prototyping to support an evolutionary approach; and
(D) a modular, open-systems approach.
Basically, for the DoD at least, Agile became the law. Here's the report the DoD returned to congress on how they would go about it:

Based on my Scrum briefing at the Pentagon in 2011, DoD introduced this language into their requirements for vendors:

•    Deliver Early and Often: This principle is aimed at changing the culture from one that is focused typically on a single delivery to a new model that comprises multiple deliveries to establish an environment that supports deployed capabilities every 12 to 18 months.
•    Incremental and Iterative Development and Testing: This principle embraces the concept that incremental and iterative development and testing, including the use of prototyping, yield better outcomes than trying to deploy large complex IT network systems in one "Big Bang."
•    Rationalized Requirements: User involvement is critical to the ultimate success of any IT  implementation, and user needs must be met. However, this principle also recognizes the need for users and requirements developers to embrace an enterprise focus across a portfolio of capabilities with established standards and open modular platforms vice customized solutions to ensure interoperability and seamless integration.
•    Flexible/Tailored Processes: The Department's IT needs range from modernizing nuclear command and control systems to updating word processing systems on office computers. This principle acknowledges  unique types of IT acquisition and embraces flexible and tailored-and risk-appropriate-IT paths based on the characteristics of the proposed IT acquisition.

And while many DOD departments have not yet come to terms with Agile, this is the language their CIO is using on their modernization plans.
The DoD CIO's 10 Point Plan for IT Modernization targets the most pressing, near-term challenges and presents approaches to efficiently and effectively deliver agile, secure, integrated, and responsive IT capabilities. This plan will enable the DoD to reduce costs and deliver faster, more responsive capabilities, while improving interoperability, user satisfaction, cyber security, and, ultimately, mission success. The primary goal is to enable agile, secure, efficient and effective IT for DoD.
Here's an interesting bullet from a slide deck by the White House CIO, Steven VanRoekel on the administrations 2013 IT budget priorities:
Entrepreneurs in Residence
– Introduce and cultivate innovative best practices and technologies into
the Government
– Assemble agile teams to solve problems using rapid cycle, lean engineering principles
There are other government agencies that have used Scrum to great effect, I know the FBI used it to rescue their Sentinel program from a complete waterfall failure, and I’m sure there are more. Anyone else know of any?

Also, if anyone knows of anybody who wants to take the next step in their career and get trained as a Scrum Master, there are still a few seats left for my July 26-27 course in Boston.


Petri Heiramo said...

Yeah, happy to see one of the most influential software purchasers adopt new practices. I just wonder how long it will take to actually make this a consistent reality, given the long culture both at DoD and at supplier community. I've noticed that many of the big traditional suppliers are very slow in adopting Agile in any way, and the smaller ones are often not privy to larger government software projects.

Unknown said...

Sutherland for President!

Rod Claar said...

Thanks Jeff for pulling this all together. There is hope, but this ship is very big and I am not sure that the ocean it is in is big enough to turn it!

Bob Schatz said...

I have personally worked with NASA, Department of Defense, DoD contractors, Department of Homeland Security, and the Government Accountability Office. There is movement, but it will take time. Lots of baggage to unload. But the pockets that have used it have had great results.

Michael Kutch said...

Hey Jeff. We're beginning to put this Roadshow together - Anyone that would enjoy helping out come on aboard.

Michael Kutch said...

Jeff . . . can you please adjust my last comment . . . our new webpage is

Glen Alleman said...

There are some issue with "going agile" in the Federal IT space. The first is for contracts over $20M require Earned Value Management, per FAR/DFARS flow downs and the OMB Part 7 guides for "FedCiv."

The second is the contract vehicles need to be adapted to how agile does things with requirements, changes, deliverables, progress payments, fee calculations.

Here's the "tip of the iceberg" for integrating Scrum with Federal Acquisition. While Scrum is a wonderful concept and applicable to large majorities of projects, but it's not quite as simple as it might seem.

Implementation of agile in the federal procurement world is the focus of many organizations in and out of the government. Management Concepts is one source of training for agile in the Fed space, there are others.

But in the end this is an acquisition issue. Without changes in the FAR/DFARS clause flow downs that "put on contract" the behaviors mandated by the regulations of how software is developed, acquired, and put into service.

While the agile community has many power tools to improve the problem in the federal government, they need come at this from a acquisition point of view for those benefits to be realized.

Jeff Sutherland said...

There are challenges to saving 100s of billions of dollars by moving to agile contracts. When I briefed the coordinating task force for implementing agile across services for all of DOD at the Pentagon last year it was clear to everyone that the procurement process has to change. This is true is commercial companies as well. The old ways of doing things are dysfunctional and the procurements experts need to fix it! Let's take an example in the commercial world. Your big multibillion dollar company requires six months to execute a purchase order for a new PC. We see this in the real world. We tell these companies that this is one of the reason they are job losers instead of job creators. Only a dysfunctional management team would set up such a system.

Rick said...

Jeff & All:

I'm a developer who works for the government on acquisition and lifecycle issues.

I started an Agile Practices Working Group under the CIO Council's Management Best Practices Committee in January and I've been working as an affiliate with the Software Engineering Institute.

No doubt, guidance like the 804 report, the Defense Science Board's 2009 IT Policies and Procedures report, the National Academy of Sciences recomendations and the 25-Point Plan present a compelling vision.

I suggest that you and the folks in the Agile community review and comment on Contracting Guidance to Support Modular IT Development referenced here [1]. Pay close attention to the Performance Work Statement in the appendix of the document.

I also wanted to direct people's attention to last month's IEEE Computing Now article entitled "Why the FBI Can't Build a Case Management System. [2]" by Jack Israel. The article is relevant both for its history and its currency.

I'm interested in hearing what folks think.

The opinions in my communiations are soley my own. No representation is made by my employer or its affiliates regarding the utility, objectivity, accuracy or integrity of the information published through any of my communications.



curiouscat said...

It is good news. Also remember DoD is huge. There is pretty much always some of great stuff and some lousy stuff going on relating to almost everything. Encouraging agile practices is great. It takes a long time for the majority of what is being done to change. But getting even 5% of the work done at DoD to be agile would be a huge amount of work being done much more effectively.

Chris Fortuin said...

DOD also mandated the Earned Value Management (EVM) method for program management and procurement way back in 1991 ( Empirical Process Control is an integral part of Agile and the most effective EVM implementation in a complex environment. You could consider EVM an early DOD endorsement for going Agile.

Glen Alleman said...

Integrating agile and EVM is straightforward and done on many programs already and more as incremental-spiral and evolutionary acquisiton for the new 5000.02 guidance rolls out