How to reduce waste in IT – Transport Waste

In manufacturing transport is moving products between operations and locations. It could be moving product from the machine shop to the welding shop, or from a factory in China to an assembly line in America. Transport increases shipping and handling costs, damages product and delays the process without adding value to customers.

trafficchaos

In IT transport waste is the time and cost caused by handing over work from one functional team to another. For example from a build team to a test team, from an internal team to a vendor team or from an onshore team to an offshore team.

When work is handed over from one group to another there are engagement costs, management costs, learning costs and delays. To maximise utilization many functional teams do work in big batches which leads to over production, over engineering and big queues of work in progress. In additional physical, organisational and cultural distance between groups often lead to poor communication and poor integration which cause a lot of defects and rework which increase time and cost further.

handover cost2

Many big IT departments are full of handover costs because they have adopted a strategy of cutting costs by centralizing each specialist function in a shared service, standardizing the work and outsourcing it to the lowest cost commodity vendor. When this is done each central function focuses on cutting costs by reducing team size and skill until they just barely meet minimum acceptable standards and by adopting strict procedures which push as much work as possible onto customers and other teams. Any local savings from doing this are outweighed by a large increase in time and cost to the customer when you look at the system as a whole. As mentioned before this leads to situations where it takes 12 weeks to get a firewall rule changed and 12 months to get new servers in production.

Handover time and cost can be reduced by setting up co-located, cross functional teams that contain all the functional specialists required to do the work from beginning to end just in time to meet customer demand and by mapping out the end to end business process from the customer’s point of view and eliminating any steps that don’t add value. This approach can deliver large savings in time and money with little cost.

The lean approach to reducing transaction costs is well aligned with the Agile Manifesto which says that we value “Customer collaboration over contract negotiation” and the Agile principle that “The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.”

This is not an argument to do everything in house with permanent staff. If your IT organization is not able to work in cross functional agile teams or does not have the capability for a particular project then you may be better off engaging an agile team from a quality local provider to both do the project for you and teach you how to operate in a lean agile way.

In the next article we will look at another cause of IT waste.

How to reduce waste in IT – Over production

In a previous post I talked about the different types of waste in IT. In this article we will talk about the waste of overproduction

In manufacturing over production is making too many products before they are actually needed. Over production ties up capital in work in progress and excess product, takes up space and increases support costs. This reduces profit and the money available to build other products that customers want.

In IT over production means building and supporting features that are rarely or never used. In a 2002 study, the Standish Group found that in traditional waterfall software projects 64% of typical system features were rarely or never used.

feature usage

Over production of features occurs often in traditional IT because stakeholders are only given one opportunity to present their system needs every 5 to 10 years. Recognizing that this is their only chance, stakeholders specify everything they’ve ever dreamed of with the hope that it will deliver value. IT project teams take these requirements without question on the assumption that stakeholders understand what users need.

The principles of Lean Manufacturing require that we make what the customer wants when they want it, pulling only what is ordered through your work flow. We can do this in IT by re-organising our work to deliver small batches of features frequently with cross functional teams that have a short cycle time from feature request to delivery combined with a customer feedback loop to determine if users really want what we delivered or need something else instead. This aligns well with one of the key principles of Agile IT which is that “our highest priority is to satisfy the customer through early and continuous delivery of valuable software”. For guidance on how to implement a customer feedback loop take a look at The Lean Startup.

In my next post I will examine waste from over processing in IT.

How to reduce waste in IT – Excess Inventory

In manufacturing inventory is raw materials, work in progress (WIP) and stocks of finished goods. Money tied up in inventory increases debt, reduces return on investment and is not available for other work. Excess inventory also increases transport, storage and administration costs. Over time excess inventory becomes obsolete and must be written off.

excessinventory

In IT any work done on planning, analysis, design, development and testing for system features that have not been deployed is inventory waste. Most traditional IT groups are full of inventory waste because they spend millions of dollars on waterfall projects that take years to deploy to production. For example a $10 million IT project that delivers in one big bang after two years has $10 million of inventory before it deploys. If your IT division spends $100 million a year on projects like this then you have an average of $100 million dollars tied up in IT inventory all the time. If you could reduce your software development lifecycle from 2 years to four weeks then your IT inventory would drop to less than $4 million which means that you would have an extra $96 million available for profit or other projects.

In IT as in manufacturing the main cause of excess inventory is over production and queues of work within the production process. As described in a previous article over production is caused by doing work in big waterfall projects. It’s made a lot worse by doing the work in specialist teams that throw it over the fence to the next team at the end of a phase. When work enters a team in a big batch it blocks other work and takes a long time to process. When the team is slow, under-resourced or has a long lead time then work sits in a queue for a long time. For example, when an IT Operations team has minimized its costs to the point that it takes several months to provide a server then this causes unfinished work to pile up in software projects that need servers.

The best way to reduce inventory waste in IT Operations and Software Development is to set up cross functional teams to do the work for each requirement Just in Time to meet customer demand. A good approach is to set up a Kanban board for each team with columns for request, analyze, design, build, test and deploy. When you have a Kanban board you can measure the average cycle time from customer request to deployment and you can see where work is building up and take steps to reduce it. It’s not important to ensure that every person is fully utilized. What is important is to implement things that customers value as quickly as they need them and no faster.

kanbanboard

The Agile principle that we “Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale” aligns well with this approach.

Excess work in progress hides a lot of other problems. When you drop your work in progress you start revealing issues that are causing waste in other areas. I will continue this series on reducing waste in IT in subsequent articles.

How to reduce waste in IT – Over Engineering

The next major waste we will look at in this series is over processing. In manufacturing over processing means doing more than the customer requires. For example: painting areas customers don’t see or producing products with unnecessarily tight tolerances. Over processing increases labor and material time and cost without delivering extra value to customers.

mousetrap

In software development over processing is caused by designing and building more complex applications than users require. For example: rebuilding an application on a new platform when the old platform could have been upgraded for a much lower cost, using a complex middleware product that causes a lot of extra work or building an ideal server platform that can cater for far more users than you expect to have any time soon.

Over engineering is common in traditional software development because design is done upfront by a software architecture team when the requirements are over stated, the solution is uncertain and the costs are unknown. As a result traditional software architects often design big, complex solutions using the latest technology with little consideration of the cost.

In IT Operations over processing occurs when you give customers more than they need sooner than they need it. For example: setting up a new employees PC two weeks before they start or getting new servers into production in one week when most projects won’t use them for two weeks. This happens very rarely.

In Lean manufacturing you minimize over processing by developing acceptance criteria that customer’s value and standard operating procedures to meet them with the minimum time and effort possible.

In software development we can minimize over engineering by moving from big design up front to continuous just in time design that evolves as we learn what we really need to build.  We do this making design a role within a cross functional team that develops software from beginning to end in small batches with a short cycle time and a lot of feedback from users. This aligns well with the Agile principle that “Simplicity–the art of maximizing the amount of work not done–is essential”.

In IT Operations we can minimize over processing by asking users what they need and then developing standard services with standard processes and end to measures of customer value to meet those needs. Anything that doesn’t add value to users can be taken out of the process.

Unfortunately many traditional IT Departments apply Lean Principles in a way that cuts IT costs locally at the expense of customer value. For example it’s common for Contract Managers to outsource IT services on a long term contract to the IT provider that promises to meet service level agreements (not user needs) for IT transactions (not end to end services) at the lowest cost (not highest value). To minimize their costs commodity service providers typically apply strict operating procedures that push as much responsibility for the work as possible onto users and other teams. Then they reduce labor costs as much as possible by reducing the team size and skill and sending the work offshore until they just barely meet the minimum transaction requirements. In these arrangements it’s common for new employees to find that they can’t get a PC or login until weeks after they’ve started and for it to take 12 weeks to get a firewall rule changed and 12 months to get new servers in production. This approach is not recommended because it substantially reduces the value of IT to the organization.

The next article will discuss how to reduce inventory waste in IT.

How to reduce wasted time and effort in IT

Toyota is one of the world’s most successful companies with 330,000 staff and a value of $193 Billion.  Toyota leaders often say that their growth is due to the Toyota Production System which is renowned for its focus on removing any activity that doesn’t add value to the customer. Toyota’s method known as Lean Production has spread across the manufacturing industry as companies have realised that they have to adopt it or be left behind.

toyota-model-factory-line

 “All we are doing is looking at the time line, from the moment the customer gives us an order to the point when we collect the cash. And we are reducing the time line by reducing the non-value adding wastes.” Taiichi Ohno, Toyota.

In my experience traditional Information Technology groups have a lot of waste that goes unrecognized because it’s not physical.

“The most dangerous kind of waste is the waste we do not recognize” Shigeo Shingo, Toyota

In this series of articles I will translate the ideas of lean production from the physical world of manufacturing to the virtual world of information technology so that you can see the waste in your process and learn how to reduce it.

The cause of waste

Wasted time and effort is ultimately caused by uneven work flow and unreasonable demands.  If we streamline our work and make reasonable demands then wasted time and effort will fall.

IT groups traditionally think of software development as being like building a bridge in which we need to complete each step in the process in one big batch before we move onto the next step. The problem is that software development is nothing like building a bridge and doing work in large batches in specialist groups creates a very uneven workflow which leads to a lot of wasted time and effort and a high rate of failure.

Types of waste

Uneven workflow and unreasonable demands cause several different types of waste. In the language of IT these wastes are:

7Wastes

In the next post I will examine how over production causes waste in IT.

When should you use Agile?

Agile is a set of software development values, principles and practices that help your organization change more effectively. If you need to make any software changes then agile is simply a better way of making these changes than traditional methods.

Some people argue that if a project has a clear, fixed scope and solution then there’s no need to use Agile. I disagree. If the project scope and solution are fixed throughout the project then an agile project can substantially increase your return on investment by delivering incremental benefits early as shown in the diagram below.

Agile ROI

But project scope and requirements are never fixed throughout a project no matter how hard you try. That’s because software development isn’t like making widgets in a factory as many people think; it’s like designing a new car and the factory you need to build it in. In every software project we learn what the real requirements and solution are as we go and it’s not until the end that we understand them fully. Agile understands the need to learn along the way and helps you implement what you learn quickly and effectively.

The only time you shouldn’t do Agile is when you don’t have the capability to do it or the will to resolve the organizational blockers that Agile reveals.

Jonah Lomu rugby superstar leaves competitors in the dust

Jonah Lomu rugby superstar leaves competitors in the dust

For example, if your software project needs a new environment and it usually takes your organization six to twelve months to get one then your Agile project will be blocked in a few weeks. If your senior management are unable or unwilling to provide an alternative environment straight away then the project will stop. As a result any project that depends on a new environment would not be a good candidate for Agile.

Or, let’s say that your organization has a strict waterfall methodology and little experience with Agile. If you start your Agile journey on a multi-billion dollar program of work then you will probably fail because your organizations structure, beliefs, processes and policies will block your Agile program at every point. Agile can be done successfully on large projects using the Scaled Agile Framework or Disciplined Agile Delivery but this requires a high level of Agile capability that you won’t have to begin with. If you haven’t done Agile before you need to start on small projects and remove organizational barriers to progress before you move onto big projects.

Some critics have claimed that Agile is an evangelical fad that won’t work on large, complex, mission critical projects. These people claim that Agile is an open ended, time and materials approach with no clearly defined roles or requirements and no guarantee of a specified outcome for a specific price. The underlying assumption seems to be that Agile = Scrum = Adhoc.

These critics are wrong. The 2011 IT Project Success Survey shows that Agile projects are much more likely than traditional projects to deliver a quality system, meet stakeholders real needs, provide a return on investment and deliver on time and schedule.

Rather than being adhoc, Agile is a highly disciplined approach with clearly defined roles and requirements that can be tailored to your needs as you can see at NetObjectives Lean-Agile Pocket Guide to Scrum teams.

And rather than being open ended, Agile projects can deliver a guaranteed outcome for a specific price if required. In fact Agile projects are better at doing this than fixed price projects because Agile is faster, higher quality and more efficient than Waterfall.

When we do fixed price Agile projects at Kiandra IT we work closely with the client to understand the features (or epics) they want; we get the development team to size each feature (or epic) based on previous examples (with a contingency for risk); and then we work closely with the client to develop a roadmap to deliver the features in order of priority within the client’s time and budget. This roadmap includes a plan to release features every few weeks so that the client can get a return on their investment as soon as possible. If requirements change then we can substitute features of similar point value for the same cost.

In my experience, if you have the capability to do Agile and the will to solve the problems it reveals then you can benefit from using Agile on all your projects.

What is Agile?

What is Agile?

Agile software development is a group of software development values, principles and practices based on incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams.

Agile_Software_Development_methodology

Agile was developed in 2001 by a group of software development writers who got together to find common ground to promote better ways of developing software. This group included representatives from Extreme Programming, SCRUM, DSDM, Adaptive Software Development, Crystal, Feature-Driven Development, Pragmatic Programming, and others sympathetic to the need for an alternative to documentation driven, heavyweight software development processes.

agile methods

What emerged from this discussion was an agreement on a common set of values and principles that are the foundation of the new lightweight approaches to software development. This was the Manifesto for Agile Software Development which says:

“We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.”

It’s important to note that Agile does not advocate abandoning the values on the right. These values do help you develop software it’s just that they have been overemphasized by most organizations.

“We follow these principles:

  • Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
  • We welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
  • We deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
  • Business people and developers must work together daily throughout the project.
  • We build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
  • The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
  • Working software is the primary measure of progress.
  • Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
  • Continuous attention to technical excellence and good design enhances agility.
  • Simplicity–the art of maximizing the amount of work not done–is essential.
  • The best architectures, requirements, and designs emerge from self-organizing teams.
  • At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.”

Many people think Agile is just a set of useful processes and tools. This is not the case. Agile is a set of common values and principles that the processes and tools have in common.

Agile Values

The practices themselves are evolving over time and new Agile practices such as Kanban are emerging. These practices should not be implemented rigidly because they are only a means to an end. You can and should select the practices you need and tailor them for your situation. As long as you’re sticking to the core values and principles you are still being Agile.

Conceptually Scrum is a subset of Agile and Lean principles and practices which overlap. Both Agile and Lean are in turn a subset of Systems Thinking.

Lean Agile Systems Thinking

 

For example the founders of Scrum acknowledge that Scrum was inspired by Lean Product Development practiced by Japanese companies such as Fuji-Xerox, Canon, Honda and NEC in the 70’s and 80’s. In the past few years this has been fully developed in Lean Software Development. Lean in turn is about seeing the whole system and understanding its assumptions and feedback loops which is explored more fully in Systems Thinking.

To help you learn more about Agile I have listed what I think are the classic texts on the field by subject. Start reading and try it out for yourself.

Agile Reading List

Agile Planning

Agile Software Development with Scrum (Series in Agile Software Development) by Ken Schwaber and Mike Beedle

Kanban: Successful Evolutionary Change for Your Technology Business by David J. Anderson and Donald G Reinertsen

Agile Project Management: Creating Innovative Products (2nd Edition) by Jim Highsmith

The Rational Unified Process Made Easy: A Practitioner’s Guide to the RUP: A Practitioner’s Guide to the RUP by Per Kroll, Philippe Kruchten and Grady Booch

DSDM: Dynamic Systems Development Method: The Method in Practice by Jennifer Stapleton and Peter Constable

Rapid Development: Taming Wild Software Schedules by Steve McConnell

Crystal Clear: A Human-Powered Methodology for Small Teams: A Human-Powered Methodology for Small Teams by Alistair Cockburn

A Practical Guide to Feature-Driven Development by Stephen R. Palmer and John M. Felsing

Lean-Agile Software Development: Achieving Enterprise Agility by Alan Shalloway, Guy Beaver and James R. Trott (Nov 1, 2009)

Lean Software Development: An Agile Toolkit by Mary Poppendieck and Tom Poppendieck

Agile Business Analysis

Reengineering Management: Mandate for New Leadership, The by James Champy

Agile Modeling: Effective Practices for eXtreme Programming and the Unified Process by Scott Ambler

Agile Software Requirements: Lean Requirements Practices for Teams, Programs, and the Enterprise (Agile Software… by Dean Leffingwell

Writing Effective Use Cases by Alistair Cockburn

User Stories Applied: For Agile Software Development by Mike Cohn

Agile Design

Designing with the Mind in Mind: Simple Guide to Understanding User Interface Design Rules by Jeff Johnson (Jun 3, 2010)

UML Distilled: A Brief Guide to the Standard Object Modeling Language (3rd Edition) by Martin Fowler

Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and Iterative Development (3rd… by Craig Larman (Oct 30, 2004)

The Object Primer: Agile Model-Driven Development with UML 2.0 by Scott W. Ambler

Design Patterns: Elements of Reusable Object-Oriented Software by Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides

Patterns of Enterprise Application Architecture by Martin Fowler

Agile Development

Extreme Programming Explained: Embrace Change, 2nd Edition (The XP Series) by Kent Beck and Cynthia Andres

Refactoring to Patterns by Joshua Kerievsky

Refactoring: Improving the Design of Existing Code by Martin Fowler, Kent Beck, John Brant and William Opdyke

The Pragmatic Programmer: From Journeyman to Master by Andrew Hunt and David Thomas

Clean Code: A Handbook of Agile Software Craftsmanship by Robert C. Martin

Code Complete: A Practical Handbook of Software Construction, Second Edition by Steve McConnell

 

Agile Testing

Agile Testing: A Practical Guide for Testers and Agile Teams by Lisa Crispin and Janet Gregory

Test Driven Development: By Example by Kent Beck

Fit for Developing Software: Framework for Integrated Tests by Rick Mugridge and Ward Cunningham

Automated Software Testing: Introduction, Management, and Performance: Introduction, Management, and Performance… by Elfriede Dustin, Jeff Rashka and John Paul

Agile Delivery

Continuous Integration: Improving Software Quality and Reducing Risk by Paul M. Duvall, Steve Matyas and Andrew Glover

Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation (Addison-Wesley… by Jez Humble and David Farley

Agile Review

Agile Retrospectives: Making Good Teams Great by Esther Derby, Diana Larsen and Ken Schwaber

Agile Estimating and Planning by Mike Cohn