Discussion: Agile not suitable for governmental IT?

The recent article Agile will fail GovIT, says corporate lawyer is rather controversial but very valuable. Its value lays not in its claim that agile cannot work in governmental environment, something I quite disagree with, but in its presentation of how (inaccurately) agile is/may be perceived in this environment and of obstacles posed by such an environment to any (and especially to agile) development.

The article reveals a fundamentaly psychological and social issue. It doesn't question the ability of agile to deliver projects much more successfully than waterfall. The laywer, Alistair Maughan, doesn't speak at all about projects' results. He speaks about fear (to bear the consequences of a failure) and constraints present in governmental environment. Thus it's pointless to argue about benefits of the agile and absurdity of the waterfall approaches. The key to a successful project is to understand those constraints, lift them as much as possible, and create a security structure for both governmental officials and the supplier to be able to work within those constraints safely. We shouldn't forget that there are also smart people in the government agencies who will gladly accept a methodology that leads to better results as long as they are safe from being denounced on the front side of newspapers as bad public servants when something goes wrong.

That, of course, isn't easy. But successful agile projects in the public domain prove that it is possible.

An agile approach will never be able to deliver a large, multi-year project within budget, with agreed upon (detailed) features, and on time. Of course a waterfall approach won't be able to do that either - but, contrary to agile, it can pretend that it can do it. It will fail (or maybe "succeed," producing something that nobody really wants), but everybody concerned will be safe enough - only tax payers will mourn.

Perhaps the only feasible way to realize an agile project in the public domain is to break the task into multiple projects small enough to be estimated with sufficient reliability (and if the estimates are wrong, such short time frame that they cannot go too wrong) so that the supplier can commit to a fixed price and scope¹. The customer would be then obliged to take over the deliverable and pay for it (to avoid "it's great, but we need these two more features to make it usable").

An experienced team, knowing both the domain and technology, can estimate quite reliably in perhaps a three months horizon. So the supplier can make an umbrella contract, which breaks the task into individual 3-months fix-price projects, each starting with a new detailed contract and ending with a working, paid-for deliverable. If the task cannot be broken into such incremental deliverables than it is heading into big troubles and you don't want to be there anyway.

There is a nice example of how to form a contract so that the customer can feel safe in the blog post Selling Software Projects by Demonstrating Value Early. It also mentions the time & material contract, mistakenly taken as an essential characteristics of "agile" by some, including likely the A. Maughan:

We will work time and materials, it will cost you approximately this much per sprint, and we think it will take about this many sprints but are not really sure. Furthermore, we don’t know exactly how long it is going to take, or what we are going to deliver.

Of course most customers - not only governmental ones - would have problems accepting this.

I will quote some interesting parts of the article and its comments, for a full picture I'd recommend you reading them for yourself.

1) The idea of breaking project down into fixed-price (1-)3 months projects doesn't come from me but one clever person who I'd need to ask for permission to name.

The Article

Quotes:

I'm prepared to accept on trust that, if all goes well, Agile may reduce the risk of project failure. But Agile simply won't work in the real world of government ICT. ...

It means customers don't pay a fixed price for a complete project. They pay for a commitment of resources.

...

But the lack of clearly defined project roles and requirements is a problem for Agile.

...

Agile projects rely on decisions based on mutual trust. They are therefore well suited to in-house projects. But the faith they ask customers to have in service providers makes them ill-suited for external developments.

...

You can have an ICT project with a watertight contract, clear deliverables, openly and legally procured, with a fixed price and appropriate remedies if you don't get what you want. Or you can have an Agile project. You can't have both.

According the the laywer, there are four clear reasons why Agile won't work in government ICT

1. Government customers want to know up-front how much a system will cost; "Under Agile [..] can't guarantee a specified outcome for a specific price"
2. Government has to compare different bidders on a like-for-like basis but "Agile can't give you a clear specification of outputs up-front. Nor can it give a definitive up-font price."
3. "Agile offers insufficient means of remedy if things go wrong" - for "Agile makes it hard to apportion blame because the customer is intimately involved in the work." (Wohoo - blame & fear-driven development! :-))
4. Decision-making is centralised in government, every important decision flows up to senior levels

Conclusion

As I understand it, the main challenge in governmental project is the bureaucratic nature of the customer. More exactly the following implications:

1. Requirement of fixed price, time and scope
2. Paralyzed decision-making process - nobody but the top decision makers really dares to make a decision and they are not available and/or too far away from the problem beeing solved to be able to decide properly and timely
3. Need to "apportion blame," absence of trust

These factors are very real and every project has to deal with them, being agile or not. Both the agile development methodology and the governmental environment must be adjusted to make a project possible at all.

Development happens in 4D space: time, money, scope, quality. You can ever fix only 3 of these axis (and it cannot be quality for quick&dirty is a contradiction). Pretending that you can fix all of them is a lie. Agile can guarantee fixed price and time but the scope (expressed as a high-level business values and epics)  must vary - to a certain degree - to provide for  inherent complexity of the domain and to provide flexibility for finding out what the customer really needs. It should be noted that detailed up-front specifications lead to the customers trying to add any possible feature they may think of even though 45% of them will never be used and 19% only rarely (Standish Group Study reported at XP2002 by Jim Johnson, taken from Mary Poppendieck's Lean Software Development slides). Wouldn't it be better to develop only those ~ 20-35% features whose benefit justifies their cost to get the software at a much lower cost and shorter time?

Without some level of trust (which, of course, cannot just "happen," but must be gradually built) and a decision structure that actually works, every project will most likely terribly fail. A "watertight contract" and "clear deliverables" are not a solution as the history of failed waterfall projects has shown because only rarely does the customer really know what exactly he needs and mis-trust based relation hinders cooperation and communication, the very essential precondition of a successfull project. "Appropriate remedies" won't really help anyone, even if the customer gets most money back, he will have wasted many resources. On the other hand, trust is not something you can expect from a bureaucratic ogranization so, as mentioned, you must construct a security structure to get rid of the fear preventing cooperation.

By the way, this reminds me of Mary Poppendieck's explanation while other US airlines cannot reproduce the success of the lean SouthWest Airlines - they are too much stuck where they are and driven by short-term profits and thus visions - due to their dependency on the short-sighted stock holders - rather than (potentially much higher)  long-term ones.

The culture in government agencies is often little suitable for agile development, but it is possible to a feasible way of introducing agile there by breaking projects into small, few-months ones or perhaps by using a hybrid agile approach, as described in the comments by Dylan Jay.

The Comments

I've extracted (for me) interesting parts of the comments; doing so I might have inadvertently changed what the author intended to express.

I've actually copied so much of the comments that the rest of this post feels more like a steal-work than a creation of my own. I hope the authors will forgive me, even though I've violated the basic rule of quoting and decency and failed to include their names for the most part.

You should absolutely check three comments - A. Maughan's reaction to the comments (right below), an overview of the DSDM method by the DSDM Consortium's Technical Director Andrew Craddock (original), and Dylan Jay's description of a successful hybrid agile project (original). You should also consider looking at the follow-up post by Nik Silver, Agile government IT can succeed.

Author's response to the comments

The author, Alistair Maughan, responds to the comments among others by saying:

I don’t actually think Waterfall is the best idea since sliced bread, nor that Agile is the worst idea since the Sinclair C5.

But I really do believe the public sector environment leaves Agile little chance of being adopted properly, let alone applied widely.

[...]

Project staff should be allowed to work without fear of procurement challenge, public humiliation and the blame culture that seems to come with major public sector projects. If that changes, Agile in Government may blossom. Sadly, there’s no chance of that happening, but it's a nice thought.*

*) Emphasized by me

Among other problems, he mentions EU-mandated procurement regime and losing bidders challenging procurement awards, the centralization of ICT governance within (UK) government, the fear of failure caused among others by "baying media, the hyper-critical Parliamentary Accounts Committee, and the National Audit Office", the scrutinization of every penny spent to death andthe requirement of accounting each penny up-front.

The commentators

About waterfall:

From what I've read in research literature (and heard anecdotally about failed government IT projects) I don't think the waterfall method has been particularly successful in guaranteeing a specified outcome either!

Another:

government agencies  are "a system whereby apportioning blame after the fact is more important than ever delivering anything"

On outcome specification (btw, the DSDM case studies include a governmental agency):

With the latter [the DSDM method] it is possible to guarantee a minimum specified outcome for a set price. (As much of a guarnatee as can be given for more tradtional methods at any rate).

At least some government agencies can succeed with agile:

I have personally been involved in delivering a fixed price Agile project to a government customer. The project was a huge success that delivered something a traditional waterfall approach had spent 3 years trying to deliver and failing.

Agile fails (too) simply because the customer is paralyzed by bureaucracy:

I've worked on several "agile" public sector projects, and none of them have delivered any of the promised advantages of Agile. The reasons are probably familiar to anybody who's worked on government IT projects: slow and bureaucratic internal processes, inability to take decisions, inability to decide on requirements (ever), lack of commitment from business users, lack of understanding of Agile in IT departments, inability to deliver rapid iterations, lack of rapid feedback between users/developers, lack of empowered business representatives, [...]

"[...] But you can't guarantee a specified outcome for a specific price."

That is true. It is equally true of the methods governments and others have used in the past, and of the methods recommended for government ICT today. It is true of all software development methods, past and present. Therefore, the statement does not distinguish between any two methods.

About the faulty culture:

I broadly agree with this article. It's clear to me the mismatch in social capital culture between government and the conditions under which Agile is designed to work.

[...]

Rob Knight's comment quoting DeMarco is correct. Managing cost is most relevant where the payoff is closely matched to the cost. Most IT projects have massive asymmetric payoffs. It is better to manage schedule and scope and to launch early in order to realize the opportunity "costs" of delay. Recognizing greater benefit early.

What this article does well is demonstrate that current government procurement procedures and governance are unsuitable for managing IT projects on behalf of tax payers.

It would be better to advocate for this change. [to align the governmental culture with Agile/Lean thinking]

On the contract:

Of course, continuing to use old-style contracts won't work with lean & agile approaches (of course, they mostly don't result in the desired outcomes anyway).

Hence, the move toward outcome-based agreements, predicated on output-based pricing regimes. These aren't new. They've been used for over 15 years to my knowledge. But they may be unfamiliar to specific individuals, naturally.

An 'agile contract' does not have to be an agreement to buy a certain amount of effort at an agreed price. That's not agile. That's 'time & materials'.

Another success of agile @ governmental env.:

[..] agile has been used very successfully in government projects by Valtech, see my blog post http://www.pharmarketeer.com/2010/05/16/agilenorth.html#agile-in-waterfall-case-studies [..]

On the "Agile makes it hard to apportion blame [...]"

Yes its true that agile focuses on heading off problems early when they are still small problems rather than waiting until the end when its too late and you need to find someone else to sue and hold accountable for your disaster.

On non-agility of one of the two failed "agile" project cases:

I worked on a system that was downstream from the one EDS built for Sky and I can assure you, whatever the publicity, that it didn't look Agile. From what I remember, the court case turned on the untrustworthiness of the lead salesman. No Agile project I've worked on could run to completion without that sort of issue being raised and addressed earlier.

Ex-public servant who experienced agile in the public domain:

I am an ex-public servant who has worked in teams that have delivered highly successful agile projects using DSDM coupled with PRINCE2, and in the interests of being balanced, worked on a really bad ‘agile’ project that is currently stalled owing to procurement rules – note the ‘a’, but it still delivered a working product. In fact I learned my agile trade as a civil servant.

Fixed-price, fixed-scope projects results in additional hidden costs (making the described process open actually results in the proposed break down into multiple projects :-) ):

With a typical fixed scope fixed price contract there is a lot of "discussion" between government purchaser and IT vendor to come to an arrangement of agreement that the delivery meets the initial scope requirements. Nobody in government wants to have backed a loser, so its best if the system is agreed to have met its scope. Once this has happened, then a number of expensive change requests are put through to change the system into one that is deployable. These change requests are often budgeted separately, so its harder to see the true cost of the system.

Is it really necessary to blame somebody?

Is it fair to label Agile projects as failed when all they did was surface issues much more quicker than in a traditional approach? And thereby save tons of money that would otherwise be wasted?

Fear-based culture of the bureaucratic agencies (the last sentence of the second paragraph made me really lough):

This blog post isn't about succeeding at delivering useful working systems. It's about limiting the impact of inevitable failure.

I worked with a developer once who devoted most of his time to making sure when we failed he wouldn't get any of the blame. He was absolutely horrified when he realised we were actually setting out to succeed. He now works in the public sector, and is thriving.

On the DSDM method by Andrew Craddock, the DSDM Consortium's Technical Director (including partly as an overviw of DSDM):

Tags: opinion methodology