Agile ROI

Last Friday I listened to the CMIT presentation by David F. Rico on the Business Value of Agile Methods. Overall a good presentation but I left with some concerns and questions. Specifically, slide 13 of his presentation is what has proven to be successful for me in how I approach development projects. The reason SCRUM and Extremem Programming, in my opinion, can't be successful in the government environment, is because the government has an imperative to 'know' that we are on the right track and 'monitor' progress. SCRUM and XP don't lend the visibility necessary to government Project Managers and Contract Officers to know that things are going well. Slide 13 demonstrates the overall model and features that are to be delivered for each iteration. The CO or COR can easily check the box to report that performance is in alignment with the Quality Assurance Surveilance Plan (QASP). With the other Agile methodologies I do not know how I could construct a QASP to effectively monitor progress. So kudos to Mr. Rico for pulling this slide that I was in agreement with, even though he said that most development projects that he has been involved with use SCRUM and XP.

The part for me that was concerning begins in slide 21 in which he begins to use data to build his case. The issue I have with his data is that he is using Lines of Code (LOC) as a common denominator across Agile and non-Agile projects. The issue here is that older projects not using Agile methods are very likely using older development languages. I strongly suspect that the data for the non-Agile projects is relying heavily on COBOL and C projects. I also suspect that the Agile projects is relying more on C++, JAVA and .NET languages. The fact that there are likely differences in the development toolset increses the risk of the findings derived from the studies. In this instance, I see LOC as a high risk because I would expect more lines of code from the COBOL and C projects than the Object-Oriented projects because the OO languages operate more efficiently. For instance you can perform a function in 10 lines of JAVA code that would require 100 lines of COBOL code. As such I have concerns about the findings presented here.

Finally, I ended with some questions. If someone asked me, "What is the return on investment from switching from a waterfall or code-and-test methodology to an Agile methodology" I would probably not start with this type of formula. The first place I would look is to project success. I would begin by digging into overall project success and failure for Agile methods versus non-Agile methods. I'm not a researcher or writing a book on this subject, but I suspect that projects using Agile methods are more likely to be launched or released than projects using other methods. I would also argue that this factor is likely to dwarf other measures.

But if you insist on other measures, I would offer that it offers a significant benefit in the scope dimension when compared to the Waterfall method. In waterfall you exhaustively capture and carve requirements into stone tablets that are then delivered in complete software some amount of time later. The reality is that if the business requirements could lend themselves to stone tablets the world would be a much happier place. But that is unrealistic. As such delivered software in a waterfall project rarely meets the scope of the business needs because the business needs have evolved during the time the team was working to develop the software. Agile allows closer interaction with the business personnel during the entire development process and this helps the final products to be in much closer alignment with the business needs.

An additional measure for comparing Agile practices against the client-server code-and-test practice is on cost. Good Enterprise Architecture practices are very difficult to implement in a code-and-test environment. This leads to a lot of redundant development, more difficult integration, and the most significant component, an increased cost to maintain the finished product.

So overall I agree with the conclusions, that Agile Methods have an increase return when compared to other traditional methods of development. But the details of that analysis are a little uncomfortable for me.