When should you use Agile?

Agile is a set of software development values, principles and practices that help your organization change more effectively. If you need to make any software changes then agile is simply a better way of making these changes than traditional methods.

Some people argue that if a project has a clear, fixed scope and solution then there’s no need to use Agile. I disagree. If the project scope and solution are fixed throughout the project then an agile project can substantially increase your return on investment by delivering incremental benefits early as shown in the diagram below.

Agile ROI

But project scope and requirements are never fixed throughout a project no matter how hard you try. That’s because software development isn’t like making widgets in a factory as many people think; it’s like designing a new car and the factory you need to build it in. In every software project we learn what the real requirements and solution are as we go and it’s not until the end that we understand them fully. Agile understands the need to learn along the way and helps you implement what you learn quickly and effectively.

The only time you shouldn’t do Agile is when you don’t have the capability to do it or the will to resolve the organizational blockers that Agile reveals.

Jonah Lomu rugby superstar leaves competitors in the dust

Jonah Lomu rugby superstar leaves competitors in the dust

For example, if your software project needs a new environment and it usually takes your organization six to twelve months to get one then your Agile project will be blocked in a few weeks. If your senior management are unable or unwilling to provide an alternative environment straight away then the project will stop. As a result any project that depends on a new environment would not be a good candidate for Agile.

Or, let’s say that your organization has a strict waterfall methodology and little experience with Agile. If you start your Agile journey on a multi-billion dollar program of work then you will probably fail because your organizations structure, beliefs, processes and policies will block your Agile program at every point. Agile can be done successfully on large projects using the Scaled Agile Framework or Disciplined Agile Delivery but this requires a high level of Agile capability that you won’t have to begin with. If you haven’t done Agile before you need to start on small projects and remove organizational barriers to progress before you move onto big projects.

Some critics have claimed that Agile is an evangelical fad that won’t work on large, complex, mission critical projects. These people claim that Agile is an open ended, time and materials approach with no clearly defined roles or requirements and no guarantee of a specified outcome for a specific price. The underlying assumption seems to be that Agile = Scrum = Adhoc.

These critics are wrong. The 2011 IT Project Success Survey shows that Agile projects are much more likely than traditional projects to deliver a quality system, meet stakeholders real needs, provide a return on investment and deliver on time and schedule.

Rather than being adhoc, Agile is a highly disciplined approach with clearly defined roles and requirements that can be tailored to your needs as you can see at NetObjectives Lean-Agile Pocket Guide to Scrum teams.

And rather than being open ended, Agile projects can deliver a guaranteed outcome for a specific price if required. In fact Agile projects are better at doing this than fixed price projects because Agile is faster, higher quality and more efficient than Waterfall.

When we do fixed price Agile projects at Kiandra IT we work closely with the client to understand the features (or epics) they want; we get the development team to size each feature (or epic) based on previous examples (with a contingency for risk); and then we work closely with the client to develop a roadmap to deliver the features in order of priority within the client’s time and budget. This roadmap includes a plan to release features every few weeks so that the client can get a return on their investment as soon as possible. If requirements change then we can substitute features of similar point value for the same cost.

In my experience, if you have the capability to do Agile and the will to solve the problems it reveals then you can benefit from using Agile on all your projects.

What is Agile?

What is Agile?

Agile software development is a group of software development values, principles and practices based on incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams.

Agile_Software_Development_methodology

Agile was developed in 2001 by a group of software development writers who got together to find common ground to promote better ways of developing software. This group included representatives from Extreme Programming, SCRUM, DSDM, Adaptive Software Development, Crystal, Feature-Driven Development, Pragmatic Programming, and others sympathetic to the need for an alternative to documentation driven, heavyweight software development processes.

agile methods

What emerged from this discussion was an agreement on a common set of values and principles that are the foundation of the new lightweight approaches to software development. This was the Manifesto for Agile Software Development which says:

“We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.”

It’s important to note that Agile does not advocate abandoning the values on the right. These values do help you develop software it’s just that they have been overemphasized by most organizations.

“We follow these principles:

  • Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
  • We welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
  • We deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
  • Business people and developers must work together daily throughout the project.
  • We build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
  • The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
  • Working software is the primary measure of progress.
  • Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
  • Continuous attention to technical excellence and good design enhances agility.
  • Simplicity–the art of maximizing the amount of work not done–is essential.
  • The best architectures, requirements, and designs emerge from self-organizing teams.
  • At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.”

Many people think Agile is just a set of useful processes and tools. This is not the case. Agile is a set of common values and principles that the processes and tools have in common.

Agile Values

The practices themselves are evolving over time and new Agile practices such as Kanban are emerging. These practices should not be implemented rigidly because they are only a means to an end. You can and should select the practices you need and tailor them for your situation. As long as you’re sticking to the core values and principles you are still being Agile.

Conceptually Scrum is a subset of Agile and Lean principles and practices which overlap. Both Agile and Lean are in turn a subset of Systems Thinking.

Lean Agile Systems Thinking

 

For example the founders of Scrum acknowledge that Scrum was inspired by Lean Product Development practiced by Japanese companies such as Fuji-Xerox, Canon, Honda and NEC in the 70’s and 80’s. In the past few years this has been fully developed in Lean Software Development. Lean in turn is about seeing the whole system and understanding its assumptions and feedback loops which is explored more fully in Systems Thinking.

To help you learn more about Agile I have listed what I think are the classic texts on the field by subject. Start reading and try it out for yourself.

Agile Reading List

Agile Planning

Agile Software Development with Scrum (Series in Agile Software Development) by Ken Schwaber and Mike Beedle

Kanban: Successful Evolutionary Change for Your Technology Business by David J. Anderson and Donald G Reinertsen

Agile Project Management: Creating Innovative Products (2nd Edition) by Jim Highsmith

The Rational Unified Process Made Easy: A Practitioner’s Guide to the RUP: A Practitioner’s Guide to the RUP by Per Kroll, Philippe Kruchten and Grady Booch

DSDM: Dynamic Systems Development Method: The Method in Practice by Jennifer Stapleton and Peter Constable

Rapid Development: Taming Wild Software Schedules by Steve McConnell

Crystal Clear: A Human-Powered Methodology for Small Teams: A Human-Powered Methodology for Small Teams by Alistair Cockburn

A Practical Guide to Feature-Driven Development by Stephen R. Palmer and John M. Felsing

Lean-Agile Software Development: Achieving Enterprise Agility by Alan Shalloway, Guy Beaver and James R. Trott (Nov 1, 2009)

Lean Software Development: An Agile Toolkit by Mary Poppendieck and Tom Poppendieck

Agile Business Analysis

Reengineering Management: Mandate for New Leadership, The by James Champy

Agile Modeling: Effective Practices for eXtreme Programming and the Unified Process by Scott Ambler

Agile Software Requirements: Lean Requirements Practices for Teams, Programs, and the Enterprise (Agile Software… by Dean Leffingwell

Writing Effective Use Cases by Alistair Cockburn

User Stories Applied: For Agile Software Development by Mike Cohn

Agile Design

Designing with the Mind in Mind: Simple Guide to Understanding User Interface Design Rules by Jeff Johnson (Jun 3, 2010)

UML Distilled: A Brief Guide to the Standard Object Modeling Language (3rd Edition) by Martin Fowler

Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and Iterative Development (3rd… by Craig Larman (Oct 30, 2004)

The Object Primer: Agile Model-Driven Development with UML 2.0 by Scott W. Ambler

Design Patterns: Elements of Reusable Object-Oriented Software by Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides

Patterns of Enterprise Application Architecture by Martin Fowler

Agile Development

Extreme Programming Explained: Embrace Change, 2nd Edition (The XP Series) by Kent Beck and Cynthia Andres

Refactoring to Patterns by Joshua Kerievsky

Refactoring: Improving the Design of Existing Code by Martin Fowler, Kent Beck, John Brant and William Opdyke

The Pragmatic Programmer: From Journeyman to Master by Andrew Hunt and David Thomas

Clean Code: A Handbook of Agile Software Craftsmanship by Robert C. Martin

Code Complete: A Practical Handbook of Software Construction, Second Edition by Steve McConnell

 

Agile Testing

Agile Testing: A Practical Guide for Testers and Agile Teams by Lisa Crispin and Janet Gregory

Test Driven Development: By Example by Kent Beck

Fit for Developing Software: Framework for Integrated Tests by Rick Mugridge and Ward Cunningham

Automated Software Testing: Introduction, Management, and Performance: Introduction, Management, and Performance… by Elfriede Dustin, Jeff Rashka and John Paul

Agile Delivery

Continuous Integration: Improving Software Quality and Reducing Risk by Paul M. Duvall, Steve Matyas and Andrew Glover

Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation (Addison-Wesley… by Jez Humble and David Farley

Agile Review

Agile Retrospectives: Making Good Teams Great by Esther Derby, Diana Larsen and Ken Schwaber

Agile Estimating and Planning by Mike Cohn

What’s the evidence that Agile works?

In my experience Agile works very well but it’s helpful to have concrete evidence to convince skeptics.

Case Study 1- Telco Projects

In 2007 I worked on a series of large projects for a large Telco using a heavy Prince 2 waterfall process. During this company’s multibillion dollar IT transformation program, the business and technical people became very unhappy with this approach and in 2010 the COO announced that they were going Agile.

Prior to the Agile announcement my group spent nearly two years using this Prince2 methodology to deliver three large multi-million dollar projects which transformed the company’s online platform for Enterprise customers. As usual these projects took longer, cost more and delivered less than expected.

After these projects were deployed, the business asked us to deliver the most important things that had been de-scoped in the next three months before they lost their remaining budget. We estimated three projects with the usual waterfall methodology and found that none of these projects could be delivered in three months. Since time was short I asked the business, the vendor and IT management to let me use an Agile approach to deliver as much as we could in the time remaining. The result was proof that Agile works.

On the first portal, we delivered the features requested in 50% less time and 10% less cost than originally estimated. On the second portal we delivered the features required in 30% less time and 10% less cost than estimated with half the functionality deployed early. On the third portal we reduced the time it took to place a business order for mobile phones from 10 minutes to three minutes for 80% less than it had taken to do a similar project two years before! This is shown in the table below.

Agile vs Waterfall

Case Study 2- Ericsson Mobile Core

In 2009 Ericsson Mobile Core recognized that their established practices were failing. Projects were delayed. Quality was difficult to maintain. And even with the best project management oversight, they still had problems obtaining a believable picture of where they were.

In Experiences of the Ericsson Mobile Core Agile Transformation Ericsson reported that after a three year Agile transformation they have exceeded customers quality expectations, reduced their product development cycle time and released new functionality ahead of schedule.

Case Study 3 – SAS Agile Transformation

For many years SAS used a waterfall methodology to develop the SAS Platform and implement new solutions for customers; but from 2005 SAS found that they were unable to keep pace with customers’ demands for new features. So in 2008, SAS decided to implement Agile to improve speed to market and trained over 2,000 staff.

In a 2013 paper Agile Adoption: Measuring its Worth SAS reported that teams that implemented agile have been more engaged, more productive, produced higher quality products and delivered faster. SAS found that the return on investment in Agile has been substantial irrespective of project size and length.

The 2011 IT Project Success Survey

Scott Ambler, found similar results in the 2011 IT Project Success Survey of IT industry professionals. This survey is highly credible as more than 80% of the respondents have more than ten years’ IT experience, 51% are IT managers or team leads and more than half are from large organisations with more than 500 people.

The 2011 survey shows that Agile projects (iterative, agile and lean) are much more likely to be successful than traditional and ad-hoc projects. When asked how successful projects were in their organization, respondents said that 67% of agile projects in their organization were successful compared to 50% of traditional projects.

agileevd1jpg

Agile projects are more successful because many more of them deliver a quality system, meet stakeholders real needs, provide a return on investment and deliver on time and schedule. Traditional projects are less likely to deliver a quality outcome and most of them don’t meet stakeholders needs, provide a return on investment or deliver on time.

agileevd2j

Drilling down we can see that more than 70% of Agile projects were effective at delivering a return on investment compared to only 30% of traditional projects.

agileevd3j

Agile methods work. Try it and you’ll see.

If you want some help contact me at Kiandra IT

Becoming Agile

Surfer on a Wave

I’m a big fan of agile. I find it’s faster, more flexible, higher quality, lower cost and lower risk than traditional software development. I also find it a refreshingly honest and adult way of dealing with people. But I didn’t always think this way.

When I first started developing software in the late 80’s for a global IT company, most big software projects used a strict step-by-step methodology. We used an approach called Method/1. It came in 20 white binders and was commonly known as methadone because it put everyone to sleep.

Lightweight agile software development methods evolved in the mid-1990s as a reaction against these heavyweight waterfall methods. Kent Becks “Extreme Programming Explained” came out in 1999 but my first exposure to agile was not until 2000 when an evangelical XP coach talked my ears off at a party. His fervor made me skeptical.

A few years later in 2004, I worked at an online jobs company with a gentle, intellectual software architect who thought agile was a good idea and persuaded us all to give it a go. After training we implemented Agile Development on an online project using all the XP and Scrum practices we knew. We had a cross-functional team of developers, testers, a business analyst and product owner, daily scrum meetings, a feature backlog, progressive elaboration of requirements, automated unit testing, continuous integration, a project wiki and burn down charts. To my surprise and delight this was by far the smoothest, easiest and most successful project I’d ever worked on! Ever since then I have been reading as much as I can about agile, lean and systems thinking and implementing an agile approach whenever I can.

One of the main reasons I joined Kiandra IT was because the software development team were committed to implementing an agile approach. If you’re one of our clients, I am convinced that this will give you a much better outcome than the traditional software development approaches you’re used to.