Pitfalls of Agile Transformations

  

“We are a conservative company, so we are just starting our agile transformation,” the manager told me. “But we expect big things from it:  faster delivery, easier recruiting, happier customers.”

“Interesting objectives,” I thought to myself. “Something I might have heard ten years ago.” It struck me that the reason an organization opts for late adoption is to learn from those who go first – from the companies that bushwhacked through the agile swamp a decade ago, or the organizations that followed a few years later. I wondered how much of what we have learned in the last decade will inform this budding agile transformation. I sensed that the answer was “not enough.”

Once you get past the sales pitches and confirmation biases, it doesn’t take much research to discover that agile and Scrum don’t have such a great track record. In the First Round Review article  I'm Sorry, But Agile Won't Fix Your Products, Adam Pisoni, co-founder and former CTO of Yammer, contends that “While SCRUM did manage to rein in impulsive managers, it ended up being used more to exert tighter control over engineers’ work.” In The Failure of Agile, Andy Hunt, an original signatory of the Agile Manifesto, writes “Agile methods themselves have not been agile. Now there‘s an irony for you.” Both of these pieces complain that agile does not provide real empowerment – one of several persistent problems we have observed in many organizations as they adopt agile practices.

Every organization undertaking an agile transformation imagines that the problems with other agile implementations will not plague THEIR transformation. If they hire the right consultants and use the best practices, they assume they will be fine. This kind of wishful thinking only lengthens the list of mediocre agile transformations. It would be more useful to understand the most predictable problems with agile implementations and actively help your organization avoid them.

With this in mind, I offer three questions you might ask to expose some of the typical ways in which agile disappoints, along with the best current approaches for avoiding these common agile pitfalls.

Question 1: Should you use Scrum or Continuous Delivery?

This may come as a surprise, but quite frankly, Scrum says nothing about how to develop software, nothing about how to deliver defect-free code and nothing about techniques for faster production releases. Other agile methodologies – especially the long lost Extreme Programming – have more to say on these topics, but most agile transformations reserve little time for improving the actual work involved in generating top notch software. Yet without a solid foundation in the technology that produces great systems, agile is pretty hollow. 

The technical heart of agile is embodied in the practices articulated by Jez Humble and Dave Farley in Continuous Delivery: acceptance test-driven development; automated builds, automated testing, automated database migration, and automated deployment; everyone checks their code into the mainline at least daily (there are no branches!); the mainline is ALWAYS production ready and is deployed very frequently (daily is slow); release is by switch rather than by deployment. If you aren’t heading toward these or similar technical practices and you think you are doing an agile transformation, think again. Agile without a strong technology base is usually a mistake.

Start your agile transformation by acknowledging that software development is a deeply technical endeavor leading to highly complex systems. These systems behave like all complex systems – if you smash them with a big change, all bets are off – you cannot predict the results. The only way to have predictable, stable code bases is to modify them with small probes, observe the results, modify the code and probe again. [Incidentally, a small probe is not two weeks of work; it’s more like two hours of work.] If deploying small probes to live systems is not at the core of your agile transformation strategy, you are missing today’s most reliable tools for delivering stable systems with predictable results.

Yes, this means writing a lot more code. It means tests as code, infrastructure as code, deployment as code. It means no one writes production code until there is an acceptance test for it, written in an executable language. It means teams can pretend they are working in a cloud because the infrastructure they need is always available and can be provisioned as needed. It means that whole teams (which include everyone from product to operations) retain responsibility for their code even after it goes live. And it means that the most common way teams decide what to do next is to examine feedback from the effects of their work in actual use.

The technology enabling Continuous Delivery should be at the core of any modern agile transformation because it has proven to be the safest way for an organization to gain and maintain control of complex software systems. If your agile transition team does not understand this technology, then you are probably trying to switch to agile without adequate technical leadership. This is not a good strategy.

Admittedly, Continuous Delivery is technically challenging, but no more so than the many other challenges that technical teams deal with every day. In fact, we have found that almost without exception, software engineers love to work in a Continuous Delivery environment because of the challenge, the discipline, the clarity, and the immediate feedback. One financial services company told us that in the three years since their (large) IT department switched to Continuous Delivery, they have had zero turnover, except for emigration. Their transformation resulted in the most desirable jobs in the area.

Question 2:  Do you hire Developers or Engineers?

What title do you use for people who solve problems with software? Years upon years ago, I was called a programmer and that was a high status job. But once waterfall processes placed analysts between programmers and their customers, the programmers were no longer expected to analyze customer problems and solve them. The title “programmer” was downgraded to a second class job which mostly involved coding what someone else wrote in a specification. Over time a new term – developers – came into use and referred to a more holistic job. But then, agile processes placed a product owner between developers and customers, so developers were no longer expected to analyze customer problems and solve them. Instead, they were given a prioritized list of relatively small stories to estimate, code, and (hopefully) test.

If you visit Silicon Valley these days you will find that software developers have been replaced by software engineers. We can only hope that those smart people who have this title will be presented with complete problems and expected to engineer a solution. They will not be given specs, because whoever wrote the spec designed the solution. They will not be given stories, because whoever wrote the stories designed the solution. They will be given real problems – customer problems, business problems, technical problems – and asked to engineer a solution. They will be expected to implement the solution within valid constraints and take responsibility for its success. Silicon Valley companies understand that this is the kind of job that attracts the best engineers.

If you want more effective recruiting in today’s very tight talent market, don’t look for software developers or mention your agile transformation. Look for software engineers and reliability engineers and make it clear that you expect them to engineer effective solutions to meaningful problems. Then make sure that your agile transformation makes this challenging work the responsibility of your engineers, because most agile methodologies place it elsewhere.

Question 3: How will you handle dependencies?

I was astonished when I heard that after Amazon completed its switch to services, the company no longer used central databases. How could this possibly work? I thought it was self-evident that a single system of record is fundamental to the success of an enterprise – so how could Amazon possibly survive without a central database? Either the information about abandoning central databases was wrong or Amazon was doing something that defied all conventional wisdom.

It turns out that the second was correct – Amazon had discovered something so obvious that it had escaped us for decades: A central database is one humongous dependency generator. Ouch!  Take a look at Sam Newman’s book Building Microservices – where the case is made that dependencies are among the greatest evils in software development and central databases are among the most pernicious creators of dependencies in the software world. It’s eye-opening.

These days we see a lot of companies building microservices – Netflix and realestate.com.au and Gilt and many more. Why? Because when they experience extremely high volume, the code that handles this volume needs constant attention and tuning. The only way to make that happen at scale is to adopt a structure which allows individual teams to deploy their code – live to production – independently of other teams. A microservice is exactly that – code owned by one (small) team that designs, monitors, maintains, and deploys the service – independent of other teams and other code.

If this sounds a lot like something you’ve heard of before, that’s because independent module deployment has been the dream of software development just about forever.  A couple decades ago, object-oriented programming promised this nirvana, but it never quite delivered. Now microservices are making the same promise, and there are instances of them working pretty well. Of course, microservices are rather new and the jury is still out. (See Martin Fowler’s summary of Microservices.) But we know that for very high volume systems, independent deployment appears to be mandatory and microservices seem to be the architecture of choice. Clearly microservices are a viable way – but not the only way – to handle dependencies.

No matter what kind of system you have, dependencies must be dealt with or else they will eventually haunt you. The Google code base started out as a monolith which rapidly developed many dependencies, but fortunately, Google's engineers understood the danger. So they developed a dependency matrix to keep track of code interactions, and whenever code was pushed to the test framework, the new code and all of its dependencies were tested together – immediately. If the test found problems, the code was reverted and everyone involved was notified. New code was system-tested thousands of times a day, which required a massive environment with a lot of automation. But it worked infinitely better than manually testing large changes because it identified the precise cause of potential problems before they happened. As expensive as it seems, it turns out that testing each small change with its complete stack of dependent code is better, cheaper, safer and faster than testing big batch releases the way we used to in the past.

“But how do we get from our legacy systems to that ideal state?” we are often asked. Well, that is precisely the question your agile transformation should answer. There are plenty of places to look for ideas, because this is a path many companies have taken. To get started, Martin Fowler's Strangler Application provides a general pattern for migrating away from legacy code, and several case studies can be found here. However, there are no canned answers for dealing with legacy code; the problems are quite specific to each situation. You need good engineers to take up the challenge supported by leadership that appreciates the importance of the issue. But the bottom line is that if an agile transformation does not provide a path from smashing your system with big releases to probing it with tiny bits of code, you have more homework to do before you get started. 

We have learned a lot about how to deal with dependencies over the last few years. We can do it with an architecture that isolates dependencies – perhaps microservices – or by automatically testing the complete system of dependent code after every small change. We know we should NOT deal with dependencies by consuming the last third of a release cycle with system testing (and fixing) the way we used to in the waterfall days. And we know it does not make sense to automate tests just to make this back-end testing go faster – a mistake we have seen frequently that you want to avoid. Test automation should be aimed at defect prevention, not defect discovery. Preventing defects as the code is written pays for itself. Many times over. Every time.

Ask the Right Questions

If you are one of those conservative organizations that is just getting around to an agile transformation, be sure you ask the right questions before you take the leap. Remember that typical agile practices are just table stakes. You need to know how to play the complex systems game, a deeply technical game played by very smart engineers. Don’t insult their intelligence if you want to engage them.

Understand that dependencies cause most defects and fragile code bases, and they also lead to tangled organizational structures. Really. If you’re skeptical, check out Conway’s Law. Get your technical and architectural act together, as well as your strategy for dealing with dependencies, before you begin. This may prompt you to consider an organizational change as part of the transformation.

When you are ready to start, be sure to articulate the specific business goals the agile transition will help achieve and how you will measure the agile transition’s contribution to these goals. Then challenge your smart people to figure out how to move those metrics – and your transition will be off to a good start.

As an industry, we know how to do this. Your colleagues have done it. You may as well avoid the pitfalls they have discovered. Start by asking a few questions.

Before There Was Management

Management is a rather recent invention in the history of human evolution – it’s been around for maybe 100 or 150 years, about two or three times longer than software. But people have been living together for thousands of years, and it could be argued that over those thousands of years, we did pretty well without managers. People are social beings, hardwired through centuries of evolution to protect their family and community, and to provide for the next generation. For tens of thousands of years, people have lived together in small hamlets or clans that were relatively self-sufficient, where everyone knew – and was probably related to – everyone else. These hamlets inevitably had leaders to provide general direction, but day-to-day activities were governed by a set of well understood mutual obligations. As long as the hamlets stayed small enough, this was just about all the governance that was needed; and most hamlets stayed small enough to thrive without bureaucracy until the Industrial Revolution.

The Magic Number One Hundred and Fifty
Early in his career, British Anthropologist Robin Dunbar found himself studying the sizes of monkey colonies, and he noticed that different species of monkeys preferred different size colonies. Interestingly, the size of a monkey colony seemed to be related to the size of the monkeys’ brains; the smaller the brain, the smaller the colony. Dunbar theorized that brain size limits the number of social contacts that a primate could maintain at one time. Thinking about how humans seemed to have evolved from primates, Dunbar wondered if, since the human brain was larger than the monkey brain, humans would tend to live in larger groups. He calculated the maximum group size that humans would be likely to live in based on the relative size of the human brain, and arrived at a number just short of 150. Dunbar theorized that humans might have a limit on their social channel capacity (the number of individuals with whom a stable inter-personal relationship can be maintained) of about 150.[1]

To test his theory, Dunbar and other researchers started looking at the size of social groups of people. They found that a community size of 150 has been a very common maximum limit in human societies around the world going back in time as far as they can investigate. And Dunbar’s Number (150) isn’t found only in ancient times. The Hutterites, a religious group that formed self-sufficient agricultural communities in Europe and North America, have kept colonies under 150 people for centuries. Beyond religious communities, Dunbar found that during the eighteenth century, the average number of people in villages in every English county except Kent was around 160. (In Kent it was 100.) Even today, academic communities that are focused on a particular narrow discipline tend to be between 100 and 200 – when the community gets larger, it tends to split into sub-disciplines.[2]

Something akin to Dunbar’s number can be found in the world of technology also. When Steve Jobs ran the Mackintosh department at Apple, his magic number was 100. He figured he could not remember more than 100 names, so the department was limited to 100 people at one time. A team that never exceeded 100 people designed and developed both the hardware and software that became the legendary Apple Macintosh.[3] Another example: in a 2004 blog The Dunbar Number as a Limit to Group Sizes, Christopher Allen noted that on-line communities tend to have 40 to 60 active members at any one time. You can see two peaks in Allen’s chart of group satisfaction as a function of group size – one peak for a team size of 5 to 8, and an equally high peak when team size is around 50.[4]

Steve Job’s limit of 100 people was probably a derivative of the Dunbar Number, but Allen’s peak at 50 is something different. According to Dunbar, “If you look at the pattern of relationships within… our social world, a number of circles of intimacy can be detected. The innermost group consists of about three to five people. … Above this is a slightly larger grouping that typically consists of about ten additional people. And above this is a slightly bigger circle of around thirty more…”[5] In case you’ve stopped counting, the circles of intimacy are 5, 15, 50, 150 – each circle about three times the size of the smaller circle. The number 50, which Allen found in many on-line communities, is the number of people Dunbar found in many hunting groups in ancient times – and three of these groups of 50 would typically make up a clan.

Does this Work in Companies?
One Hundred and fifty is certainly a magic number for W.L. Gore & Associates. Gore is a privately held business that specializes in developing and manufacturing innovative products based on PTFE, the fluoropolymer in Gore-Tex fabrics. Gore has revenues exceeding 2.5 billion US dollars, employs over 8000 people, and has been profitable for over a half a century. It has held a permanent spot on the U.S. "100 Best Companies to Work For" since it’s inception in 1984, and is a fixture on similar lists in many countries in Europe. This amazing track record might be related to the fact that Gore doesn’t have managers. There are plenty of leaders at Gore, but leaders aren’t assigned the job, they have to earn it by attracting followers.

You’ve got to wonder how such a large company can turn in such consistent performance for such a long period of time without using traditional management structures. The answer seems to have something to do with the fact that Gore is organized into small businesses units that are limited to about 150 people. “We found again and again that things get clumsy at a hundred and fifty,” according to founder Bill Gore. So when the company builds a new plant, it puts 150 spaces in the parking lot, and when people start parking on the grass, they know it’s time to build a new plant.

Since associates at Gore do not have managers, they need different mechanisms to coordinate work, and interestingly, one of the key mechanisms is peer pressure. Here is a quote from Jim Buckley, a long-time associate at a Gore plant: “The pressure that comes to bear if we are not efficient as a plant, if we are not creating good enough earnings for the company, the peer pressure is unbelievable. …This is what you get when you have small teams, where everybody knows everybody. Peer pressure is much more effective than a concept of a boss. Many, many times more powerful.”[6]

Like many companies that depend on employees to work together and make good decisions, Gore is very careful to hire people who will fit well in its culture. Leaders create environments where people have the tools necessary for success and the information needed to make good decisions. Work groups are relatively stable so people get to know the capabilities and expectations of their colleagues. But in the end, the groups are organized around trust and mutual obligation – a throwback to the small communities in which humans have thrived for most of their history.

Google’s management culture has quite a few similarities with Gore’s. Google was designed to work more or less like a university – where people are encouraged to decide on their own (with guidance) what they want to investigate. Google is extremely careful about hiring people who will fit in its culture, and it creates environments where people can pursue their passion without too much management interference. For a deep dive into Google’s culture, see this video: Eric Schmidt at the Management Lab Summit

Peer Cultures
Before there were managers, peer cultures created the glue that held societies together. In clans and hamlets around the world throughout the centuries, the self-interest of the social group was tightly coupled with the self-interest of individuals and family units; and thus obligations based on family ties and reciprocity were essential in creating efficient communities.

There are many, many examples of peer cultures today, from volunteer organizations to open source software development to discussion forums and social networks on the web. In these communities, people are members by their own choice; they want to contribute to a worthy cause, get better at a personal skill, and feel good about their contribution. In a peer culture, leaders provide a vision, a way for people to contribute easily, and just enough guidance to be sure the vision is achieved.

Arguably, peer cultures work a lot better than management at getting many things done, because they create a social network and web of obligations that underlie intrinsic motivation. So perhaps we’d be better off taking a page out of the Gore or the Google or the Open Source playbook and leverage thousands of years of human evolution. We are naturally social beings and have a built-in need to protect our social unit and ensure that it thrives.

Example: Hardware/Software Products
“We have found through experience that the ideal team size is somewhere between 30 and 70,” the executive told us. At first we were surprised. Aren’t teams supposed to be limited to about 7 people? Don’t teams start breaking up when they’re much larger? Clearly the executive was talking about a different kind of team than we generally run into in agile software development. But his company was one of the most successful businesses we have encountered recently, so we figured there had to be something important in his observation.

We spend a morning with a senior project manager at the company – the guy who coordinated 60 people in the development of a spectacular product in record time. The resulting product was far ahead of its time and gave the company a significant competitive advantage. He explained how he coordinated the work: “Every 2 or 3 months we produced a working prototype, each one more sophisticated than the last one. As we were nearing the end of development, a new (faster, better, cheaper) chip hit the market. The team decided to delay the next prototype by two months so they could incorporate the new chip. Obviously we didn’t keep to the original schedule, but in this business, you have to be ready to seize the opportunities that present themselves.”

It’s not that this company had no small teams inside the larger teams; of course they did. It’s just that the coordination was done at the large team level, and the members of the smaller teams communicated on a regular basis with everyone on the larger team. All team members were keenly aware of the need to meet the prototype deadlines and they didn’t need much structure or encouragement to understand and meet the needs of their colleagues.

Another Example: Construction
The Lean Construction Institute has developed a similar approach to effectively organizing construction work. The first thing they do is to break down very large projects into multiple smaller ones so that a reasonable number of contractors can work together. (Remember Dunbar’s Number.) For example, they might completely separate a parking structure and landscaping from the main building; in a large building, the exterior would probably be a separate project from the interior. Each sub-project is further divided into phases of a few months; for example, foundation, structure, interior systems, etc. Before a phase starts, a meeting of all involved contractors is held and all of the things that need to be done to complete that phase are posted on cards on a wall by the contractors. The cards are organized into a timeline that takes dependencies into account, and all of the contractors agree that the wall represents a reasonable simulation of the work that needs to be done. This is not really a plan so much as an agreement among the contractors doing the work about what SHOULD be done to complete the phase.

Each week all of the “Last Planners” (crew chiefs, superintendents, etc.) get together and look at what they SHOULD do, and also what they CAN do, given the situation at the building site. Then they commit to each other what they WILL complete in the next week. The contractors make face-to-face commitments to peers that they know personally. This mutual commitment just plain gets things done faster and more reliably than almost any other organizing technique, including every classic scheduling approach in the book.

The Magic Number Seven
George Miller published “The Magical Number Seven, Plus or Minus Two” in The Psychological Review in 1956. Miller wasn’t talking about team size in this article; he was discussing the capacity of people to distinguish between alternatives. For example, most people can remember a string of 7 numbers, and they can divide colors or musical tones into about 7 categories. Ask people to distinguish between more than 7 categories, and they start making mistakes. “There seems to be some limitation built into us either by learning or by the design of our nervous systems, a limit that keeps our channel capacities in this general range [of seven],” Miller wrote.

This channel capacity seems to affect our direct interaction with other people – we can keep up a conversation with 7 or so people, but when a group gets larger, it is difficult to maintain a single dialog, and small groups tend to start separate discussions. So for face-to-face groups that must maintain a single conversation, the magic number of 7 +/-2 is a good size limit. And historically, most agile software development teams have been about this size.

Moving Beyond Seven
The problem is, 7 people are not enough to accomplish many jobs. Take the job of putting a new software-intensive product on the market, for example. The product is almost never about the software – the product is a medical device or a car or a mobile phone or maybe it’s a financial application. Invariably software is a subsystem of a larger overall system, which means that invariably the software development team is a sub-team of a larger overall system team.

In the book Scaling Lean & Agile Development, Craig Larman and Bas Vodde make a strong case for feature teams – cross-functional teams that deliver end-to-end customer features. They recommend against component teams, groups formed around a single component or layer of the system. I agree with their advice, but it seems to me that software is invariably a component of whatever system we are building. We might be creating software to automate a process or software to control a product, software to deliver information or software to provide entertainment. But our customers don’t care about the software; they care about how the product or process works, how relevant the information is or how entertaining the game might be. And if software is a component of a system, then software teams are component teams. What we might want to consider is that real feature teams – teams chartered to achieve a business goal – will almost certainly include more than software development.

Agile development started out as a practice for small software teams, but these days we often see teams of 40 or 50 developers applying agile practices to a single business problem. In almost every case, we notice that the developers are organized into several small teams that work quite separately – and in almost every case, therefore, the biggest problem seems to be coordination across the small teams. There are many mechanisms: use a divisible system architecture so teams can be truly independent; draw from a common list of tasks, which makes teams highly interdependent; send small team representatives to weekly coordinating meetings; and so on. But rarely do we see the most powerful coordination mechanism of all for groups this size: create a sense of mutual obligation through peer commitments.

Mutual Obligation
You can call mutual obligation peer pressure if you like, but whatever name you use, when individuals on a large team make a commitment to people they know well, the commitment will almost certainly be honored. Mutual obligation is a much more powerful motivating force than being told to do something by an authority figure. And the interesting thing is, the power of mutual obligation is not confined to small teams. It works very well in teams of 50, and can be effective with teams up to 150. The time to split teams is not necessarily when they reach 10; team sizes up to 100 or 150 can be very effective – if you can create a sense of mutual obligation among the team members.

There are, of course, a few things that need to be in place before mutual commitment can happen. First of all, team members must know each other – well. So this won’t work if you constantly reform teams. In addition to knowing each other’s names, teammates must understand the capabilities of their colleagues on the team, have the capacity to make reliable commitments, and be able to trust that their teammates will meet their commitments. This process of creating mutual obligations actually works best if there is no manager brokering commitments, because then the commitments are made to the manager, not to teammates. Instead, a leader’s role is to lay out the overall objectives, clarify the constraints, and create the environment in which reliable commitments are exchanged.

For example, the project manager of the hardware/software product (above) laid out a series of increasingly sophisticated prototypes scheduled about three months apart. Having made a commitment to the team, sub-teams organized their work so as to have something appropriate ready at each prototype deadline. When an opportunity to dramatically improve the product through incorporation of a new chip, the whole team was in a position to rapidly re-think what needed to be done and commit to the new goal.

In the case of lean construction (above), a large team of contractor representatives works out the details of a “schedule” every few months. Each week, the same team gets together and re-thinks how that “schedule” will have to be adapted to fit current reality. At that same weekly meeting, team members commit to each other what they will actually accomplish in the next week, which gives their colleagues a week to plan work crews, material arrival, and so on for the following week.

It certainly is a good idea to have small sub-teams whose members work closely together on focused technical problems, coordinating their work with brief daily meetings to touch base and make sure they are on track to meet their commitments. But the manner in which these sub-teams arrive at those commitments is open for re-thinking. It may be better to leverage thousands of years of human evolution and create an environment whereby people know each other and make mutual commitments to meet the critical goals of the larger community. After all, that’s the way most things got accomplished before there was management.

_________________________________
Footnotes:
[1] Technically, Dunbar calculated the relative sizes of the neocortex – the outer surface of the brain responsible for conscious thinking. For a humorous parody of Dunbar's theory, see "What is the Monkeysphere?" by David Wong.

[2] Information in this paragraph is from: How Many Friends Does One Person Need? by Robin Dunbar.

[3] See John Sculley On Steve Jobs.

[4] This figure from “The Dunbar Number as a Limit to Group Sizes” is antidotal.

[5] From How Many Friends Does One Person Need? by Robin Dunbar. Interestingly, while Dunbar finds 15 an approximate limit of the second circle of intimacy, Allen finds a group of 15 problematic.

[6] The Dunbar Number was popularized by Malcolm Gladwell in Tipping Point. Much information and both quotes in this section are from Chapter 5 of that book. See http://nextreformation.com/wp-admin/general/tipping.htm for an extended excerpt.

A Tale of Two Terminals

By any measure, Terminal 3 at Beijing Capital Airport is impressive. Built in less than four years and officially opened barely four months before the Olympics, the massive terminal has received numerous awards for both its stunning design and its comfortable atmosphere. And it escaped the start-up affliction of many new airport terminals when it commenced full operations on March 26, 2008 without any notable problems.

The next day, half way around the world, Heathrow Terminal 5 opened for business. At one-third the size of Beijing Terminal 3, the new London terminal had taken twice as long to build and cost twice as much. Proud executives at British Airlines and BAA (British Airports Authority) exuded confidence in a flawless opening, but that was not to be. Instead, hundreds of flights were canceled in the first few days of operation, and about 28,000 bags went missing the first weekend. The chaotic opening of Heathrow Terminal 5 was such an embarrassment that it triggered a government investigation.

The smooth opening of Beijing Terminal 3 was not an anomaly – Terminal 2 at Shanghai Pudong International Airport opened the same day, also without newsworthy incident. Given the timing just before the Beijing Olympic games, it was clear that China was keenly interested in projecting an image of competence to the traveling public. But of course, the UK was equally interested in showcasing its proficiency, and the British executives clearly expected that the opening of Heathrow Terminal 5 would go smoothly. So the question to ponder is this: How did the Chinese airports manage two uneventful terminal openings? Did they do something different, or were the problems in London just bad luck?

It’s not like testing was overlooked at Heathrow Terminal 5. In fact, a simulation of the terminal’s systems was developed and all of the technical systems were tested exhaustively, even before they were built. A special testing committee was formed and thousands of people were recruited to be mock passengers, culminating in a test with 2000 volunteer passengers a few weeks before the terminal opened. On the other hand, the planned testing regime was curtailed because the terminal construction was not completed as early as planned; in fact, hard-hats were required in the baggage handling area until shortly before opening day. In addition, a decision was made to move 70% of the flights targeted for Terminal 5 on the very first day of operations, because it was difficult to imagine how to move in smaller increments.[1]

Those of us in the software industry have heard this story before: the time runs out for system testing, but a big-bang cut-over to a mission critical new system proceeds anyway, because the planned date just can’t be delayed. The result is predictable: wishful thinking gives way to various degrees of disaster.

That kind of disaster wasn’t going to happen in Beijing. I was in Beijing a month before the Olympics, and every single person I met – from tour guide to restaurant worker – seemed to feel personally responsible for projecting a favorable image as the eyes of the world focused on their city. I imagine that for every worker in Terminal 3, a smooth startup was a matter of national pride. But the terminal didn’t open smoothly just because everyone wanted things to go well. It opened smoothly because the airport authorities understood how to orchestrate such a large, complex undertaking that involved hundreds of people. After all, they had just finished building the airport at amazing speed.[2]

The opening ceremony of the Beijing Olympics was also a large, complex undertaking that involved hundreds of people. It’s easy to imagine the many rehearsals that took place to make sure that everyone knew their part. When it comes to opening a new terminal, the idea of rehearsals doesn’t usually occur to the authorities, but at Beijing Capital Airport, rehearsals started in early February. First a couple of thousand mock passengers took part in a rehearsal, then five thousand, and finally, on February 23rd, 8000 mock passengers checked in luggage for 146 flights. This was the average daily load expected a week later, when six minor airlines moved into Terminal 3. During the month of March, Terminal 3 operated on a trial basis, ironing out any problems that arose. Meanwhile, staff from the large airlines about to move to the terminal rehearsed their jobs in the new terminal day after day, so that when the big moving day arrived, everyone knew what to do. On March 26, all the practice paid off when the Terminal was opened with very few problems.

This certainly wasn’t the approach taken at Heathrow Terminal 5. It’s pretty clear that the opening chaos was caused because people did not know what to do: they didn’t know where to park, couldn’t get through security, didn’t know how to sign on to the new PDA’s to get their work assignments, didn’t know where to get help, and didn’t know how to stop all the luggage from coming at them until their problems got sorted out. Even the worst of the technical problems was actually a people problem: the baggage handling software had been put in a ‘safe’ mode for testing, and apparently no one was responsible for removing the patch which cut off communication to other terminals in the airport. It took three days to realize that this very human error was the main cause of the software problems![3]

In testimony to the British House of Commons, union Shop Steward Iggy Vaid testified:[4]

We raised [worker concerns] with our senior management team especially in British Airways. … [Their response was to] involve what we call process engineers who came in and decided what type of process needed to be installed. They only wanted the union to implement that process and it was decided by somebody else, not the people who really worked it. The fact is that they paid lip service to, ignored or did not implement any suggestion we made.

… as early as January there was a meeting with the senior management team at which we highlighted our concerns about how the baggage system and everything else would fail, that the process introduced would not work and so on. We highlighted all these concerns, but there was no time to change the whole plan.

... [Workers] had two days of familiarization in a van or were shown slides; they were shown where their lockers were and so on, but there was no training for hands-on work.….

The opening of a new airport terminal is an exercise in dealing with complexity. At Heathrow Terminal 5, new technical systems and new work arrangements had to come together virtually overnight – and changing the date once it has been set would have been difficult and expensive. Hundreds of people were involved, and every glitch in the work system had a tendency to cascade into ever larger problems.

If this sounds familiar, it’s because this scenario has been played out several times in the lives of many of us in software development. Over time, we have learned a lot about handling unforgiving, complex systems, particularly systems that include people interacting with new technology. But every time we encounter messy transition like the one at Heathrow Terminal 5, we wonder if our hard-learned lessons for dealing with complexity couldn’t be spread a bit wider.

Socio-technical Systems
Not very far from Heathrow, the Tavistock Institute of London has spent some decades researching work designs that deal effectively with turbulence and complexity. In the 1950’s and 60’s, renowned scientists such as Eric Trist and Fred Emery documented novel working arrangements that were particularly effective in the coal mines and factories of Great Britain. They found that especially effective work systems were designed (and continually improved) by semi-autonomous work teams of between 10 and 100 people that accepted responsibility for meaningful (end-to-end) tasks. The teams used their knowledge of the work and of high-level objectives to design a system to accomplish the job in a manner that optimized the overall results. Moreover, these teams were much better at managing uncertainty and rapidly adapting to various problems as they were encountered. The researchers found that the most effective work design occurs when the social aspects of the work are balanced with its technical aspects, so they called these balanced work systems Socio-technical systems.

In 1981, Eric Trist published “The Evolution of Socio-technical Systems,” an engaging history of his work. He attributes the “old paradigm” of work design to Max Weber (bureaucracy) and Frederick Taylor (work fragmentation). He proposed that a “new paradigm” would be far more effective for organizations in turbulent, competitive, or rapidly changing situations:[5]

Old Paradigm	New Paradigm
The technological imperative	Joint optimization [of social & technical systems]
Man as an extension of the machine	Man as complementary to the machine
Man as an expendable spare part	Man as a resource to be developed
Maximum task breakdown, simple narrow skills	Optimum task grouping, multiple broad skills
External controls (supervisors, specialist staffs, procedures)	Internal controls (self-regulating subsystems)
Tall organization chart, autocratic style	Flat organization chart, participative style
Competition, gamesmanship	Collaboration, collegiality
Organization’s purposes only	Members’ and society’s purposes also
Alienation	Commitment
Low risk taking	Innovation

In the 1980’s the socio-technical paradigm gained increased popularity when team work practices from Japan were widely copied in Europe and America. In the 1990’s socio-technical ideas merged with general systems theory, and the term “socio-technical systems” fell into disuse. But the ideas lived on. These days, it is generally accepted that the most effective way to deal with complex or fast-changing situations is by structuring work around semi-autonomous teams that have the leadership and training to respond effectively to any situation the groups are likely to encounter.

The clearest example we have of semi-autonomous work teams are emergency response teams – firefighters, paramedics, emergency room staff. Their job is to respond to challenging, complex, rapidly changing situations, frequently in dangerous surroundings and often with lives at stake. Emergency response teams prepare for these difficult situations by rehearsing their roles, so everyone knows what to do. During a real emergency, that training coupled with the experience of internal leaders enables the teams to respond dynamically and creatively to the emergency as events unfold.

Design Social Systems Along with Technical Systems
Developing a software system that automates a work system is fraught with just about as much danger as moving to a new airport terminal. There are many things we can do to mitigate that risk:

1. Cutover to any new system should be in small increments. Impossible? Don’t give up on increments too quickly – and don’t leave this to “customers” to decide! The technical risk of a big-bang cut-over is immense. And it’s almost always easier to divide the system in some way to facilitate incremental deployment than it is to deal with the virtually guaranteed chaos of a big-bang cutover.

2. Simplify before you automate. Never automate a work process until the work teams have devised as simple a work process as they possibly can. Automating the right thing is at least as important as automating it right.

3. Do not freeze work design into code! Leave as much work design as possible for work teams to determine and modify. If that is not possible, make sure that the people who will live with the new system are involved in the design of their work.

4. Rehearse! Don’t just test the technical part, include the people who will use the new system in end-to-end rehearsals. Be prepared to adapt the technical system to the social system and to refine the social system as well. Be sure everyone knows what to do; be sure that the new work design makes sense. Leave time to adjust and adapt. Don’t cut this part short.

5. Organize to manage complexity. Structure work around work teams that can adapt to changing situations, especially if the environment is complex, could change rapidly, or is mission critical. At minimum, have emergency response teams on hand when the new system goes live.

Much of the software we write ends up having an impact on the lives of other people; in other words, our work creates changes in social systems. We would do well to consider those social systems as we develop the technical systems. If we want to create systems that are truly successful, the technical and social aspects of our systems must be designed together and kept in balance.
___________________
Footnotes:
[1] “The opening of Heathrow Terminal 5” report to the House of Commons Transportation Committee pages 13-14.

[2] Contrast this with the absence of the BAA management team that oversaw the on-time, on-budget construction of Heathrow Terminal 5; they were replaced after a 2006 takeover of BAA by the Spanish company Ferrovial.

[3] See “The opening of Heathrow Terminal 5” report to the House of Commons Transportation Committee.

[4] “The opening of Heathrow Terminal 5” report to the House of Commons Transportation Committee pages 22-25.

[5] From "The Evolution of Socio-technical Systems" by Eric Trist, 1981. p 42.

Train-Wreck Management

“On October 5, 1841, two Western Railroad passenger trains collided somewhere between Worchester, Massachusetts and Albany, New York, killing a conductor and a passenger and injuring seventeen passengers. That disaster marked the beginning of a new management era." [1] These words open Peter Scholtes classic book on leadership. He goes on to explain how the term "management" was unknown in the days of cottage industries. As business grew and became geographically disperse in the 1800's, a way to run these businesses had to be found. But there were no models outside the church and the military, so investigators into the train-wreck disaster looked to the Prussian army for a model. And there they found the classic organization chart - the one we know so well today. Scholtes calls it the "train-wreck" chart. It was revolutionary at the time.

The purpose of what became today's organization chart was clear: The assignment of responsibility would enable "prompt detection of derelictions of duty... and point out the delinquent." Scholtes says: "A fundamental premise of the 'train-wreck' approach to management is that the primary cause of problems is 'dereliction of duty'. The purpose of the organizational chart is to sufficiently specify those duties so that management can quickly assign blame, should another accident occur."[1].

Blame
Note the thinking here: Problems are caused by people who don't do their job well, so finding someone to blame is the first step to correcting problems. Scholtes notes: "The era of management that began in the mid-1800's can be characterized as "management by results.".... Since managers could no longer do the work themselves or direct others in the doing of the work, managers exercised their authority by holding people accountable for results.... In the 1950's, management by results reached its epitome in MBO (Management By Objectives) and performance appraisals, the Harvardization of train-wreck management."[1] He goes on to say that at the time, this theory of management was the best available, and it succeeded in creating order out of chaos. "People like Whistler, McCallum, Frederick Taylor or Henry Ford in the United States or Darby, the Stephensons, or Brunel in England were pioneers.... they did their best and, by and large, what they did was very good."

"Meanwhile, in Japan...." is the title of the next section Scholtes' book. He chronicles how a better approach to management emerged in Japan in the 1950's assisted by W. Edwards Deming. Deming taught that most of the problems we encounter (perhaps 90%) are the result of multiple influences, they generally cannot be attributed to a single cause. Assigning blame for a problem to the last person involved is worse than counterproductive, it will probably make the bad situation worse. Exhorting people to "be careful," "try harder," and "work smarter" is not useful if individuals have little effect on results. Rewarding or punishing people for outcomes that are not under their control can only result in discouragement - or in gaming the system. Instead, chronic problems must be fixed by finding their underlying causes and addressing these effectively. As Deming points out, this usually involves changing the system - the way things are done. And according to Deming, it is management's job to change the system.

Process or People?
Agile software development places a strong emphasis on putting change into the hands of front-line people on self-directed teams - isn't this contrary to Deming's philosophy? Writing in 1995, Scholtes lists what he calls "fads" for addressing systemic problems: "empower people, put them into self-directed teams, motivate them, offer incentives, reengineer and reinvent them." And then he says: "All of the empowered, motivated, teamed-up, self-directed, incentivized, accountable, reengineered, and reinvented people you can muster cannot compensate for a dysfunctional system.... A well-run organization with well-functioning systems allows people from top to bottom do work of which they can be proud."[1] So where does this leave us? Which is more important - process or people?

It helps if we trade in the overloaded word "process" and use "system."

In the article "Managing a Living System, not a Ledger"[2] H. Thomas Johnson says "Managers at Toyota believe that improving the system is the surest way to improve long term financial results." He points out that Toyota takes lots and lots of measurements, but they do not use these as performance measurements. Johnson writes: "...Toyota makes virtually no use of management accounting targets (or 'levers') to control or motivate operations... Toyota focuses its operations on continuous system improvement through endless rapid problem solving. And they emphasize genchi genbutsu, or 'going to the place,' to see where a problem occurs, firsthand. They don't rely on second-hand reports or tables and charts of data to achieve a true understanding of root cause. Instead they go to the place (gemba) where you can watch, observe, and 'ask why five times.' This attitude shows a deep appreciation that results (and problems) ultimately emanate from, and are explained by, complex processes and concrete relationships, not by abstract, quantitative relationships that describe results in simple, linear, additive terms." Winding up the article, Johnson says: "Financial quantities cannot reveal if a system is improving or not... No company that talks about improving performance can know what it is doing if its primary window on results is financial information and not system principles.... Companies that intend to perform like Toyota should recognize that... they will never get there by trying to motivate and direct 'lean' initiatives with 'lean accounting' and management accounting 'levers of control.'"

Taiichi Ohno on Standard Work
Let's go back to the source of the Toyota Production System, Taiichi Ohno, and see what he had to say about process - how it is established and how it is changed.[3]

There is something called standard work, but standards should be changed constantly. Instead, if you think of the standard as the best you can do, it's all over. The standard work is only a baseline for doing further kaizen. It is kai-aku [change for the worse] if things get worse than now, and it is kaizen [change for the better] if things get better than now. Standards are set arbitrarily by humans, so how can they not change?

When creating Standard Work, it will be difficult to establish a standard if you are trying to achieve 'the best way.' This is a big mistake. Document exactly what you are doing now. If you make it better than it is now, it is kaizen. If not, and you establish the best possible way, the motivation for kaizen will be gone. That is why one way of motivating people to do kaizen is to create a poor standard. But don't make it too bad. Without some standard, you can't say 'We made it better' because there is nothing to compare it to, so you must create a standard for comparison.

Take that standard, and if the work is not easy to perform, give many suggestions and do kaizen.

We need to use the words 'you made' as in 'follow the decisions you made.' When we say 'they were made' people feel like it was forced upon them. When a decision is made, we need to ask who made the decision. Since you also have the authority to decide, if you decide, you must at least follow your decision, and then this will not be forced upon you at all.

But in the beginning, you must perform the Standard Work, and as you do, you should find things you don't like, and you will think of one kaizen idea after another. Then you should implement these ideas right away, and make this the new standard.

Years ago, I made them hang the standard work documents on the shop floor. After a year I said to a team leader, 'The color of the paper has changed, which means you have been doing it the same way, so you have been a salary thief for the last year.' I said 'What do you come to work to do each day? If you are observing every day you ought to be finding things you don't like, and rewriting the standard immediately. Even if the document hanging there is from last month, this is wrong.' At Toyota in the beginning we had the team leaders write down the dates on the standard work sheets when they hung them. This gave me a good reason to scold the team leaders, saying 'Have you been goofing off all month?'

If it takes one or two months to create these documents, this is nonsense. You should not create these away from the job. See what is happening on the gemba and write it down.

Process AND People
Ohno believed that the primary job of team leaders (first line supervisors) is the constant improvement of the way work gets done. Work standards should be written and posted, but this had better not take very long because the standards should change all the time - at least once a month. Standards are not about how work should be done, but how work is being done. You don't want the standard to be too perfect, because that leaves no incentive for workers to improve their standards. If workers are annoyed by a standard, they are expected to change it. They do not drop a suggestion in a suggestion box, they do kaizen. That is, workers - led by their team leader - do many rapid experiments, find a better way, agree on the improvement, quickly document the new way, and use it. When a standard is improved, the decision for the change must be made by the people doing the work, so they won't feel it is being forced upon them.

People like to use effective processes, and they also like to have control over their own environment. The Toyota Production System provides for both. Ohno made it clear that people must be at the center of improving their own processes. Process improvement may be done only "at the gemba" and it is up to the workers to decide whether or not a proposed improvement should be implemented. Workers are expected to keep changing the way they do their job; in fact, it is bad leadership to have a process so perfect that workers have little incentive to improve it!

Assessment and Certification
Scholtes takes process improvement assessment programs such as ISO 9000 to task because even though they seem good on the surface, they have some problems:[1]

1. The pursuit of quality must be guided by a larger context than certification - it requires a holistic, integrated, long term commitment.
2. Certification is not equal to satisfied customers - you can do the wrong thing as long as you do it consistently.
3. Assessment has a tone of paternalism and mistrust - it replaces internal motivation with external motivation.
4. Assessment assumes that inspectors are all the same - but inspections are not standardized.
5. A certified process is difficult to change - Ohno would be appalled.

Conclusion
When Deming said "change the system", he was talking about changing the complex, interrelated processes used to get work done. Deming believed that changing the system is management's primary job, and in order to do this, managers need competency in four areas:

1. Appreciation for the overall system in which work is done
2. An understanding of variation - and the true relationship between cause and effect
3. Constant pursuit of learning (improvement) through designed experiments
4. An understanding of the psychology of people

When all of these areas are balanced and working together, great things can happen.

References
[1] The Leader's Handbook, by Peter R. Scholtes, McGraw-Hill, 1998.

[2] "Managing a Living System, Not a Ledger,", by H. Thomas Johnson, Lean Manufacturing 2007, Supplement to Manufacturing Engineering, SME, August 2007. Johnson also coauthored Profit Beyond Measure: Extraordinary Results through Attention to Work and People, Free Press, November, 2000.

[3] Workplace Management, by Taiichi Ohno, originally published in 1982, from translation by Jon Miller, Gemba Press, 2007.

Screen Beans Art, © A Bit Better Corporation

Zero Defects Mentality

A ‘zero defects mentality’ is a bad thing in the military. “Demanding such a rigid standard produces timid leaders afraid to make tough decisions in crisis, unwilling to take the risks necessary for success in military operations,” Perry said. “This zero defects mindset creates conditions that will lead inevitably to failure ….”[1]

A Military Tradition
William G. Pagonis, director of Logistics during the Gulf War of 1991, wrote about the time he led his small company into crossfire to rescue stranded soldiers, against the orders of his commander: “…following a time-honored tradition in the military, I developed ‘radio trouble’ – that is, I turned the communications gear off…” and led a volunteer team to the rescue.[2]

William McKnight, who created the 3M culture of innovation, once said “Those to whom we delegate authority and responsibility, if they are good people, are going to want to do their jobs in their own way. Mistakes will be made. But if a person is essentially right, the mistakes he or she makes are not as serious in the long run as the mistakes management will make if it undertakes to tell people exactly how they must do their jobs.” [3]

Good organizations understand that in a dynamic environment, the safest course is to develop intelligent, courageous people who understand that they are expected to exercise their own initiative. Most organizations would like to think that they empower people, but their behavior would suggest otherwise.

What Would You Do?
Ponder this: What is your organization’s instinctive reaction when things go wrong?

1. Reorganize.
2. Develop a better plan.
3. Send in a swat team to improve the processes.
4. Work with the front line people to find out what they think is wrong and how it can be fixed.

Collin Powell has said: “Organization doesn’t really accomplish anything. Plans don’t accomplish anything, either. Theories of management don’t much matter. Endeavors succeed or fail because of the people involved.”[4] So you can imagine what his choice would be.

The Paradox: Superior Performance Comes From Low Control
Organizations which tolerate mistakes and overlook disobedience build an organizational culture in which everyone knows that the best way to tackle the really tough problems is through the people who are closest to them. They also build a cadre of front line workers who are not afraid to think and act on their own. Such organizations are the envy of their industries, even as their competitors try to reorganize, plan and improve processes in an attempt to compete.
__________________
Footnotes:

[1] Defense Secretary William J. Perry, Quoted by Linda D. Kozaryn, American Forces Press Service, August 6, 1996.

[2] ‘Leadership in a Combat Zone’ by William G. Pagonis, Harvard Business Review, Volume 79 number 11, December 2001.

[3] William McKnight, President and CEO of 3M from 1929–1966, quoted in Brand of the Tartan by Virginia Huck, Minnesota Mining and Manufacturing, 1955.

[4] ‘Great Military Leaders’ http://www.geocities.com/Pentagon/Bunker/6513/

Lazy Workers

Scientific Management
The workers on the first Ford assembly line spoke more than 50 languages, and many of them could barely speak English.[1] It was in this context that Frederick W. Taylor’s book, The Principles of Scientific Management, was published in 1911.[2]


Taylor believed that laborers were uneducated and lazy, reflecting the prevailing thinking of the time. To increase productivity, he proposed the ‘science’ of decomposing tasks into their smallest components, timing and planning each micro-task, and telling the worker exactly how to do each task. Taylor admitted that his methods were inappropriate for educated craftsmen or even intelligent laborers.[3]

Scientific Management was the beginning of the separation of planning from execution. Prior to Ford’s assembly line, automobiles were assembled and maintained by skilled craftsmen. Ford first developed interchangeable parts, then developed a method to have them assembled by interchangeable laborers. After all, Ford’s turnover was 380% in 1913, so it was necessary to give laborers a job they could learn in just a few minutes.

The NUMMI Experiment
In 1982, GM closed it’s Fremont plant, which had the worst productivity and absenteeism record in the company. In 1983, a Toyota and GM re-opened the same plant and hired back the same workers. New United Motor Manufacturing, Inc. (NUMMI) was managed by Toyota-trained management. They borrowed widely from Frederick Taylor in areas of work measurement, but with one major difference. Instead of industrial engineers, small work teams were formed and trained in work measurement and analysis methods. Workers designed their own jobs, and continually worked to improve their own performance. In two years, the same facility with the same workers was operating at twice the productivity and quality, better of any other GM plant. Absenteeism and drug abuse on the job had virtually disappeared, and the plant was being expanded.[4]

The Problem: Separation of Planning from Execution
Lean Production was the end of the separation of planning from execution. The fundamental change at the NUMMI plant was the involvement of the workers in the design and improvement of their own work. ‘Holding back knowledge and effort (has been) repeatedly noted by industrial sociologists as a salient feature of all mass-production systems.’[5] The critical difference in Lean Production was the direct engagement of the workers in improving the process.

Lean Production does not require extraordinary people and is certainly not without discipline. It is build on the principle of a learning environment, where small, educated teams work toward an objective using the basic scientific method: experiment, measure the results, see if it’s an improvement, and if it is, go with it, if not try something else. Don’t guess, gather data.

The fundamental difference between mass production and lean production is the separation of the planning activity from the execution activity. In the NUMMI plant, work planning was done by the workers, which resulted in an extremely short feedback loop that was continually correcting toward the desired set point. In the former GM plant, work planning was done by industrial engineers resulting in open loop control.

The Solution: Closed Loop Control
Similarly, Just-in-Time provides for work planning at the point of execution, with extremely short feedback loops overseen by the workers, who exercise ultimate control. Contrast this to MRP systems, which is divorced from the workers and provides open loop control (if it provides any control at all). These days, MRP systems are used as overall planning systems, while detailed scheduling is done with the work-level pull systems.

The only way to get closed loop control is to have workers plan the process as well as execute it. The separation of planning from execution comes from a paradigm which regards workers as uneducated and lazy. The integration of planning and execution recognizes that given the proper training, leadership and objectives, workers are more capable of designing and improving their processes than any unengaged organization, be it an industrial engineering office, a materials control office or a project office.
______________________
Footnotes:

1. The Machine That Changed the World : The Story of Lean Production, by Womack, James P., Daniel T. Jones, and Daniel Roos, New York: Rawson and Associates; 1990, page 31.

2. The Principles of Scientific Management, Taylor, Frederick W., first published as an essay in 1911, published by Harper & Brothers, New York, 1919, available as a Dover republication printed in 1998.

3. Taylor, Frederick W., Scientific Management - Comprising Shop Management, The principles of Scientific Management and Testimony before the Special House Committee, 1964, Harper and Row


4. “Time-and-Motion Regained”, Alder, Paul, Harvard Business Review (January-February, 1993) pp97-108.

5. The Machine That Changed the World : The Story of Lean Production, by Womack, James P., Daniel T. Jones, and Daniel Roos, New York: Rawson and Associates; 1990, page 53.

Screen Beans Art, © A Bit Better Corporation

Lean Programming

About the time of the 1980 NBC documentary ‘If Japan Can, Why Can’t We?’, I was the System Manager in a video cassette manufacturing plant, and our management team was asking this question every day.  Our Japanese competition was selling superior products at much lower prices, and we couldn’t figure out how they did it.  We knew we needed to make dramatic changes or close up shop, but we didn’t know what to change.

As far as we could tell, we were doing everything right.  We relied on optimized forecasting methods to determine economic lot sizes, and we used the latest MRP (Manufacturing Requirements Planning) software to launch daily schedules into the plant.  We had a sophisticated computer system that analyzed QC results and process parameters, to pinpoint the causes of defects.

We had some quality problems, and it took a month to fill most orders.  In any given week, we were able to pack out about 60% of the planned line items for the week.  But this was okay, because the other 40% of the week’s packout went into finished goods inventory.  Usually we had plenty of on-hand inventory for shipping standard orders.  Special orders were another matter, however.  The division vice president would often call to expedite special orders for important customers.

We moved in-process video cassettes from one workstation to another on carts, and we had a lot of carts.  There was never enough room to store all the carts at the next workstation, so carts full of inventory would get misplaced.   Sometimes cassettes were stacked on top of carts, and occasionally they would spill on the floor.  Video cassettes piled up in front of testing stations, so whenever a process drifted out of control, it took a while to discover that we were producing marginal product.  We had plenty of rework stations, to be absolutely sure that everything we shipped was good product.

All-in-all, we had about a month’s worth of work-in-process inventory. At the time, we blamed our inability to rapidly fill orders on bad forecasts from marketing. Later we were surprised to learn that the real culprit was our in-process inventory.  Today it is well known that the average shipping time of most supply chains is about the same as the average level of inventory in the supply chain.

Lean Manufacturing
At the end of World War II, Sakichi Toyoda, founder of Toyoda Spinning and Weaving company, dreamed of providing cars for the general public, much like Henry Ford’s dream thirty years earlier.  He chartered Taiichi Ohno to put in place an efficient production system to produce high quality automobiles.  Over the next three decades, Ohno developed the Toyota Production System, now known world-wide as Lean Manufacturing[1].  The foundation Ohno’s system was the absolute elimination of waste.

Ohno studied US manufacturing techniques, and learned a lot from Henry Ford’s pioneer work in assembly line flow.  However, the assembly line produced large lots of identical cars.  Ohno didn’t have the customer base to imitate the US practice of manufacturing in ‘economic’ (ie. large) lot sizes.  He was captivated by US supermarkets, however, where a small quantity of every product was placed on shelves, and as shoppers removed  products, the shelves were rapidly replenished.  He decided to place inventory ‘supermarkets’ throughout his plant, and found that this technique dramatically lowered the ‘waste’ of in-process inventory.  He named these inventory supermarkets ‘kanban’.

Because Ohno was converting a spinning and weaving company to an automobile manufacturer, he already knew how to avoid making bad product.  Founder Toyoda Sakichi had invented an automatic shut-off mechanism that stopped a weaving machine the minute a flaw such as a broken thread was detected.  Ohno moved this concept to car manufacturing, where he insisted that each part be examined immediately after it was processed, and the line stopped immediately if a defect was found.

To maximize product flow, standard work sheets were created, but these were not developed at a desk by engineers.  They were developed on the shop floor by the workers who know the process.  Standard cycle times and kanban shelf space for each item was determined and workflow was leveled.  Production workers were like a relay team, handing off the baton (product) to the next person.  The handoff required 100% quality and tight timing.  If things got delayed, teammates were expected to help each other set up a machine or recover from a malfunction.

Ohno’s aggressive elimination of waste led him to the twin values of rapid product flow and built-in quality.  Over time, Ohno discovered that these two values led to the highest quality, lowest cost, shortest lead time products possible.

Total Quality Management
About the same time, Dr. W. Edwards Deming was teaching Quality Management in Japan.  In fact, the Total Quality Management (TQM) movement cannot be separated from Lean Manufacturing.  Demming’s photo is in the lobby of  Toyota’s headquarters, bigger than the photo of founder Toyoda Sakichi. Demming didn’t find an audience in the US after WW II, because managers at the time thought that poor quality was caused by people who just didn’t want to do a good job.  They didn’t think there was much managers could do to improve quality except exhort employees to do a better job.

Demming’s basic message was that quality is a management responsibility, and poor quality was almost always the result of systems imposed on workers which thwarted people’s desire to do high quality work.  He taught the Japanese managers how to empower production workers to investigate problems and systematically improve processes.  He taught that teamwork and long term, trust-based relationships with suppliers were far better than adversarial relationships.  He emphasized a culture of continuous improvement of both processes and products. 

In the 1980’s, Demming’s fourteen points (See Appendix 1) were studied by virtually every manufacturing manager.  Among these fourteen points are the well known mantra’s:

• Don’t Inspect Quality In.
• Constantly Improve the System.
• Break Down Barriers Between Departments.

But a few of Demming’s fourteen points might seem revolutionary even today, such as:

• Drive Out Fear.
• Eliminate Quotas, Numerical Goals and Merit Ratings.
• Don’t Award Business Based on Price; Minimize Total Cost.

Paradigm Shift
When we first heard about Lean Manufacturing, we thought it was a hoax.  Get rid of safety stock, don’t run machines at full capacity, have suppliers deliver small lots on a daily basis?  This was so counter-intuitive, so against the paradigm of the day, that the Japanese manufacturing techniques were widely discounted.  TQM concepts were more intuitive, but alone they were not enough to lift us out of our dire situation.  Desperate for a change, we decided to give Lean Manufacturing a try, and in the end, it saved our plant.

The critical step in implementing Lean Manufacturing in our plant was a carefully planned changeover from push scheduling to pull scheduling.  We decided that we could not do it part way, we had to switch plant-wide, cold turkey, over a weekend.  We devised a simple simulation which we taught to every one in the plant – managers, shift supervisors, and operators.  Using the simulation, teams of workers designed the layout and flow in their areas, including the kanban cards and rapid changeover methods.  The entire plant held its collective breath as the pull system went into effect, but the workers knew what to do – they had developed the methods themselves.  The first week packout accuracy was 92%, and it got better from there.  We were able to fill special orders in two weeks, so the vice president could stop expediting orders.  In a short time we were down to one week of inventory and could fill any order in the same amount of time.  We had lots of extra space, and quality had never been better.

The most difficult part of implementing Lean Manufacturing was the paradigm shift it required.  Everyone ‘knew’ that large lot sizes were necessary to keep expensive machines running at full capacity.   They also ‘knew’ that machine changeovers took a long time, and every minute a machine was idle its burden rate went up.  In addition, large warehouse inventories were necessary to make sure that when a customer ordered a product it could be shipped immediately.  After all, customers didn’t want to wait the month it took us to produce the product.

One of the reasons why Lean Manufacturing has been so difficult to implement is because people must question established, known truths, and this is not easy.  Another reason is that practices which create local optimization at the expense of the overall system are difficult to recognize, let alone change.  Local optimization points provide attractive points of measurement, and inevitably, what is measured is optimized.

Simple Rules
In a January 2001 article in Harvard Business Review titled ‘Strategy as Simple Rules’, Kathleen Eisenhardt describes how smart companies thrive in a complex business environment by establishing a set of simple rules which define direction without confining it.[2]  She suggests that instead of following complex processes, using simple rules to communicate strategy is the best way to empower people to seize fleeting opportunities in rapidly changing markets.

The 1980’s were a time of profound change in US manufacturing, and the change was guided by a set of simple rules.  Simple rules gave guidance to every level of the organization, and got everyone on the same sheet of music.  They empowered people at all levels of the organization, because the provided guidance for making day-to-day decisions.  With simple rules, work teams were able to continuously improve the processes and products without detailed guidance or complex processes.  

The basic practices of Lean Manufacturing and TQM in the 1980’s might be summed up in these ten simple rules:

   1. Eliminate Waste
   2. Minimize Inventory
   3. Maximize Flow
   4. Pull From Demand
   5. Empower Workers
   6. Meet Customer Requirements
   7. Do it Right the First Time
   8. Abolish Local Optimization
   9. Partner With Suppliers
  10. Create a Culture of Continuous Improvement

These Lean Manufacturing rules have been tested and proven over the last two decades.  They have been adapted to logistics, customer service, health care, finance, and even construction.  The application of the rules may change slightly from one industry to the next, but the underlying principles have stood the test of time in many sectors of the economy.

Lean Programming
Recent work in Agile Methodologies, Adaptive Software Development, and Extreme Programming have in effect applied the simple rules of Lean Manufacturing to software development.  The results, which we call Lean Programming, are as dramatic as the improvements in manufacturing brought on by the Just-in-Time and Total Quality Management movements of the 1980’s.

Lean Rule #1:  Eliminate Waste
The first rule of Lean Programming is:  Eliminate waste.  That is, eliminate anything which does not add value to the final product.  In Lean Manufacturing, waste is identified through a value stream analysis, a process which identifies all activities in the value stream and identifies the specific value they add to the final product. The value analysis process then attempts to find a different, more efficient way to add the same value,

The documents, diagrams, and models produced as part of a software development project are often consumables, aids used to produce the system, but not necessarily a part of the final product.  Once a working system is delivered, the user may care little about the intermediate consumables.  Lean principles suggest that every consumable is a candidate for scrutiny.  The burden is on the artifact to prove not only that it adds value to the final product, but also that it is the most efficient way of achieving that value.

Lean Rule #2:  Minimize Inventory (Minimize Intermediate Artifacts)
In our manufacturing plant, we communicated this message:  Inventory is waste.  Why?  Inventory consumes resources.  Inventory slows down response time.   Inventory hides quality problems.  Inventory gets lost.  Inventory degrades and becomes obsolete.  The ‘benefits’ of inventory are oversold.  The ‘cost’ of inventory almost always outweighs such ‘benefits’.

The inventory of software development is documentation that is not a part of the final program.  As inventory, this documentation should be subject to value analysis.  Take requirements and design documents, for example. How much value do the really add?  How important are they to the final product?  If you compare requirements and design documents to in-process inventory, then it is striking to note that the time it takes to produce these documents probably determines the cycle time of the project.  Just as inventory must be minimized to maximize manufacturing flow, so too requirements and design documents must be kept to a minimum to maximize development flow.

There are many wastes associated with this excess documentation:  The waste of time producing the documents, waste of time reviewing the documents, and the work that goes into change requests and associated evaluations, priority setting, and system changes.  But the biggest waste of all is the waste of building the wrong system if the documentation does not correctly and completely capture the user requirements.

The best approach for minimizing intermediate artifacts is to raise the level of abstraction of documentation.  Instead of a 100 page detailed specification, write a 10 page set of rules and guidelines, and document only the exceptions.  Instead of a few inches of specifics, produce a concise 25 page matrix which summarizes the effort.

We know that users are relatively poor at envisioning the details of a system from most documents, and are even less likely to correctly perceive how it should operate in their environment until they actually use it.  Even if users could predict exactly how the system should operate at the present time, it is unlikely that the way the system is supposed to work months before it is delivered will be exactly the way users need it to work for the rest of its useful life.  All of this must be taken into account when we determine how much value these documents actually add to the final product.

Lean Rule #3:  Maximize Flow (Drive Down Development Time)
During the 1980’s we learned how to make products in hours which used to take days or weeks.  We learned that very rapid product flow resulted in very short cycle times, often one or two orders of magnitude lower than before.  During the 1990’s, e-commerce projects were often able to accomplish in weeks what used to take months or years in the traditional software development world.  Yes, in some sense they cheated.  But the bottom line is, huge amounts of useful software was deployed in the last five years with extremely short cycle times by traditional standards.

In a recent paper titled ‘Reducing Cycle Time,’[3] Dennis Frailey  proposes reducing software development cycle time using the same techniques employed to reduce manufacturing cycle time.  He suggests looking for and reducing accumulations of WIP (Work in Process).  Just as in manufacturing, if WIP is reduced, and the cycle time will be reduced.  To reduce WIP, Frailey recommends using the ‘Small Batch’ principle and the ‘Smooth Flow Principle’, concepts straight from Lean Manufacturing.

Iterative development is basically the application of these principles to programming.  The basic premise of iterative development is that small but complete portions of a system are designed and delivered throughout the development cycle, with each iteration adding an additional set of features.  The cycle time from start to finish of any iteration varies from a couple of weeks to a couple of months, and each iteration engages the entire development process from gathering requirements to acceptance testing.

Lean Rule #4:  Pull from Demand (Decide as Late as Possible)
In our video cassette manufacturing plant, we used to think that it would be ideal if our marketing department could forecast exact market requirements.  A lot of work went into sophisticated forecasting techniques to more accurately predict the future.  Then  one day we realized that we were trying to do the wrong thing.  It would not be ideal if we had a perfect forecast.  Instead, it would be ideal if we could reduce our reliance on forecasts by reducing the system response time so dramatically that the system could respond to change rather than predict it.

In a market where volatile technology requires constant product upgrades, Dell Computer has a huge advantage over it’s keenest competitors because it doesn’t forecast demand, it responds to it by making-to-order in an average of six days.  While Dell holds about six days of inventory, it’s competitors maintain six weeks of inventory.  Dell’s ability to make decisions as late as possible gives Dell a significant competitive advantage in a fast-moving market.

Software development practices which keep requirements flexible as close to system delivery as possible can provide a significant competitive advantage in a changing market.  In a volatile business environment, users are not able forecast their future needs accurately.  Freezing the design early in a software development project is just as speculative as forecasting.  Software systems should be designed to respond to change, not predict it.  In software development, as in building computers, the ability to make decisions as late as possible provides a competitive advantage.

Lean Rule #5: Empower Workers (Decide as Low as Possible)
A basic principle of Lean Manufacturing is to drive decisions down to the lowest possible level, providing both the tools and the authority for people “on the floor” to make decisions. When Toyota took over GM’s manufacturing plant in Fremont, California in 1983, it inherited workers with the worst productivity and absenteeism record in the industry. Those same workers doubled their quality and productivity record in two years. This was accomplished through formation of teams that were trained in work measurement and improvement techniques and expected to develop and continually improve their own work standards and practices.[4]

One of the problems with heavyweight intermediate documentation is that it attempts to make all of the decisions for developers, rather than giving them a set of guidelines.  In general, raising the level of abstraction of intermediate artifacts will give guidance as well as freedom to the developers as they make the detailed design and programming decisions.  It is always better to tell developers what needs to be done, not how to do it.

Developers need to understand the goal of their work and how it fits into the overall flow, what it means to meet customer requirements, and the architectural structure and GUI standards of the system.  They also need to know what they must accomplish, by when, and how to tell when it is complete.  Finally, their work needs to be made visible in short iterative cycles to provide the feedback necessary for continual improvement.

Lean Rule #6:  Meet Customer Requirements (Now and in the Future)
In his 1979 book ‘Quality is Free’, Philip Crosby defines quality as ‘conformance to requirements’.   The Standish Group study of 1994[5] noted that the most common cause of failed projects was missing, incomplete, or incorrect requirements.   The software development world has responded to this risk by amplifying the practice of gathering detailed user requirements and getting user sign-off prior to proceeding with system design.   However, this approach to defining user requirements is deeply flawed.

I  worked on one project in which the customer wanted a complex system delivered in ten months.  Time was of the essence – 10 months or bust.  And yet, being a government agency, the contract required sign-off on an external design document before internal design and coding could begin.  Several users were involved, and they dragged their feet on signing the documents.  Why?  They were concerned that they might approve something which would prove to be a mistake later on.  Since there was no easy way to change things after the design documents were signed, they took two months to approve the design.  An who can blame them?  Their jobs depended on them getting it right.  So half way into a very tight schedule, over two months of time and a lot of paper was wasted producing and getting user sign-off on design documents. 

Instead of encouraging user involvement, user sign-off tends to create an adversarial relationship between the developers and the users.  Users are required to make decisions early in the development process, and are not allowed to change their minds, even when they do not have a full concept of how the system will work or how their business situation may develop in the future.  Users are understandably reluctant to make these commitments, and they will instinctively delay decisions to as late in the process as possible.  Note that this instinct on the part of the users is in line with Lean Rule #4.

The most effective way to accurately capture user requirements is found in the iterative approach to system development.  By developing core features early and obtaining customer feedback in a focus-group demonstration of each iteration, a far more correct definition of customer requirements can be obtained.  In addition, if we accept that the requirements will necessarily change over time, we must start with the essential requirement that the system must be designed to easily adapt to changes over its lifecycle.

Lean Rule #7:  Do it Right The First Time (Incorporate Feedback)
Before Lean Manufacturing arrived at our plant in the early 1980’s, we occasionally had output of marginal quality. We would test to find the good product and rework the bad product. After understanding the “Do it Right the First Time” rule, we closed down rework stations and stopped trying to test quality into the product. Instead, we assured that each component was good at every handoff.  This involved having tests and controls at every point of manufacture to detect a drift toward out-of-spec product and stop production before any bad product was made.

“Do It Right the First Time” did not mean “Freeze the Spec”.  On the contrary, product specs changed constantly, and lean discipline meant being able to flawlessly adapt to changing market conditions.  This was accomplished through a product architecture which facilitated manufacturing change, monitoring techniques which detected errors before they happened, and tests which were designed before manufacturing began.

In 1987 Barry Boehm observed that it costs 100 times more to find and fix a problem after software delivery than to find and fix it in early design phases.[6] This observation and the “Do it Right the First Time” rule have been widely used to justify the overhead of developing a detailed system design before code is written.

The problem lies in the assumption that it is possible to generate a detailed set of documents that correctly define customer requirements, and that those requirements will not change. The fact is that requirements do change, and frequently, over the life of most systems. “Do it Right” has also been misinterpreted to mean “don’t allow changes.” In fact, once we acknowledge that change is a fundamental customer requirement, it becomes clear that what “Do it Right” requires that we provide for change.

If we want to meet customer requirements, and we acknowledge that customers don’t really know what they want at the beginning of development, then we need to incorporate a method of obtaining customer feedback during development.  Instead, most software development practices include a “Change Control Process” which makes it so difficult to respond to user feedback that developers are discouraged from asking for it.  Far from insuring a quality result, these change-resistant practices actually get in the way of “Doing it Right”.

Lean Programming employs two key techniques that make change easy. Just as Lean Manufacturing builds tests into process so as to detect when the process is broken, Lean Programming builds tests into the development process in order to ensure that when changes don’t inadvertently break the code. In fact, the best approach is to write the tests first, and then write the code. An excellent unit and regression testing capability is the best way to encourage change late in the development process.

The second technique for allowing change to happen late in development is refactoring, or improving the design of existing software in a controlled and rapid manner. When refactoring is an accepted practice, early designs can focus on the issue at hand rather than speculate as to what additional design elements will be needed. As the additional features are actually added, refactoring provides a new, simplified design to handle the new reality. When refactoring is a part of the process, we reduce speculation as to what will be needed in the future by making it easy to accommodate the future if and when it becomes the present.

Lean Rule #8: Abolish Local Optimization (Sub-Optimized Measurements are the Enemy)
In the 1980s, the biggest enemy of Lean Manufacturing was often the accounting department. We had big, expensive machines in our plant, and the idea that they should not be run at full capacity was radical, to put it mildly. We compiled daily reports of work-in-process inventory, and the accountants didn’t want these reports abandoned just because there was virtually no WIP to report.

A generation of accountants had to retire before it was “OK” to run machines below their full capacity. Designing machines for rapid changeover rather than highest throughput remains a tough sell even today. After 20 years, Lean Manufacturing is still counter-intuitive to those who lack a broad view of the enterprise.

In this context, let’s examine the role of managing scope in a software development project. Project managers have been trained to focus on managing scope, just as we in manufacturing were trained to focus on maximizing machine productivity. However, Lean Programming is fundamentally driven by time and feedback. In the same way that localized productivity optimization creates a sub-optimized overall process, so too, does focus on managing scope create a sub-optimized overall project management process.

Think about it – holding the scope to exactly what was envisioned at the beginning of a project has little value for the user whose world is changing.  In fact, it adds anxiety and paralyzes decision-making.  It doesn’t add much to the ultimate system, which will be outdated by the time it is delivered. Managing to a scope that is no longer valid wastes a lot of time and takes up a lot of space on issue lists, trade-off negotiations and ultimate fixes to the system to get things right in the end. However, as long as keeping a project within its original scope is a key project management goal, this measurement will continue to be optimized—at the expense of the overall value delivered by the project.

Scope will take care of itself if the domain is well understood and there is a well-crafted, high-level agreement on what the system will do in the domain. Scope will take care of itself if the project is driven in time buckets that are not allowed to slip. Scope will take care of itself if both parties focus on rapid development and solving the user’s problem, and adopt waste-free methods of achieving these goals.

Lean Rule #9: Partner With Suppliers (Use Evolutionary Procurement)
Lean Manufacturing did not remain in the manufacturing plant. Once the idea of partnering with suppliers was combined with an understanding of the value of rapid product flow, Supply Chain Management was born. People began to realize that it took tons of paperwork to move material between companies, and this did not add value to the product. Moreover, the paperwork cost more than one might expect, not to mention the delay in product flow that it caused. Even today, predictions of billions of dollars of savings resulting from business-to-business Web portals are based on cutting the cost of transactions required to move goods between companies.

Supply Chain Management caused companies to take a close look at their business-to-business contracts. All too often, these contracts were focused on keeping the companies from cheating each other. In addition, it was common to pit one vendor against another to assure supply and obtain the lowest cost. Again, Lean Manufacturing changed this paradigm. Deming taught that trusting relationships with single suppliers create an environment that allows optimizing the overall value to both companies.

Throughout the 1980s, companies achieved the highest quality and lowest costs in their supply chains by reducing the number of suppliers and working with the remaining suppliers as partners. The quality and creativity which resulted from collaborating supply chains was demonstrated to far outweigh the apparent (and sub-optimal) benefits that came from competitive bids and rapid turnover of suppliers. Partnering companies helped each other improve product designs and product flows. They linked systems to allow just-in-time movement of goods across several suppliers with little or no paperwork. The long-term advantages of a collaborative supply chain relationships are well documented.

Wise companies realize that traditional software development contract practices generate hidden wastes. As manufacturers discovered in the 1980s, trusted relationships with a limited set of suppliers can yield dramatic advantages. Without the adversarial relationship created by a constant focus on controlling scope and cost, software development vendors can focus on providing the best possible software for customers, fixing requirements as late as possible in the development process and providing the most value for the available money.

Lean Rule #10: Create a Culture of Continuous Improvement
When software development seems to be out of control, one response has been to increase the level of “software maturity” of the organization. This might seem to be in line with good manufacturing practice, where ISO 9000 certification and Malcom Baldridge awards are sometimes equated with excellence. However, these process documentation programs indicate excellence only when the documented process is excellent in the context of it’s use.

In many current software development projects, excellence means the ability to adapt to fast moving, rapidly changing environments.  Process-intensive approaches such as the higher levels of Software Engineering Institute's (SEI) Capability Maturity Model (CMM) may lack the flexibility to respond rapidly to change.  In a recent e-mail advisor from Cutter Consortium, Jim Highsmith highlights the tension between such heavyweight methodologies and lightweight methodologies such as Lean Programming.[7]

The question becomes, do process documentation certification programs stifle, rather than foster, a culture of continuous improvement?  Deming would probably turn over in his grave at the thought of tomes of written processes substituting for his simple Plan-Do-Check-Act approach:

• Plan:     Choose a problem. Analyze it to find a probable cause.
• Do:        Run an experiment to investigate the probable cause.
• Check:   Analyze the data from the experiment to validate the cause.
• Act:     Refine and standardize based on the results.

Iterative development allows the use of the Plan-Do-Check-Act approach within a project. During the first iteration, the hand-off from design to programming or programming to testing may be a bit rough. It’s okay if the first iteration provides a learning experience for the project team, because there are more iterations to come, so the team can improve its process. In a sense, an iterative project environment becomes an operational environment, because processes are repeated and Deming’s techniques of process improvement can be applied from one iteration to the next.

Product improvement is also possible with iterations, particularly if refactoring is used. In fact, refactoring provides a tremendous vehicle to apply the principle of continuous improvement to the programming environment.

However, we need improvements that span more than a single project. We must improve future project performance by learning from existing ones. Here again, Lean Manufacturing can point the way. During the 1980s, a set of practices summarized in the ten rules of Lean Manufacturing were adopted widely across most manufacturing plants in the West. These practices then spread to service organizations, to logistics organizations, to supply chains, and beyond. They have withstood the test of time across multiple domains.

Following the simple rules of Lean Manufacturing has brought dramatic improvements to every industry in which they have been applied. These same rules can and should be applied to software development projects. The resulting Lean Programming practices will lead to the highest quality, lowest cost, shortest lead time software development possible.
_______________

Appendix 1:  Summary of W. Edwards Demming’s 14 points

   1. Create consistency of purpose.
   2. Adopt a win-win philosophy.
   3. Don’t depend on mass inspection; build quality in.
   4. Don’t award business based on price; minimize total cost; build long-term relationships of loyalty and trust with a single suppliers.
   5. Constantly improve the system of production, service, planning, etc.
   6. Train for skills.
   7. Provide leadership:  help people do a better job.
   8. Drive out fear and build trust so everyone can do a better job.
   9. Break down barriers between departments; abolish competition and build a win-win system of cooperation.
  10. Eliminate slogans, exhortations and zero defect targets; the cause of the bulk of problems lie in the system, and are beyond the power of workers to correct.
  11. Eliminate quotas, numerical goals and Management by Objectives; substitute leadership.
  12. Remove barriers that rob people of joy in their work; abolish the annual rating or merit system.
  13. Educate and improve individuals.
  14. Involve the entire organization.

There are many summaries of Demming’s 14 points, which he modified throughout the years, in the spirit of continuous improvement.  The above summary is based on his last version of the 14 points.[8].
_________________

References

[1] The Machine That Changed the World : The Story of Lean Production, by Womack, James P., Daniel T. Jones, and Daniel Roos, New York: Rawson and Associates; 1990

[2] Strategy as Simple Rules, by Eisenhardt, Kathleen M and Donald N. Sull, Harvard Business Review, Volume 79, Number 1, January 2001, pp 107- 116

[3] Reducing Cycle Time, by Frailey, Dennis, Software Development Magazine, August, 2000

[4] Time-and-Motion Regained, by Paul Adler, Harvard Business Review, January-February 1993 pp 97-108

[5]Charting the Seas of Information Technology – Chaos, by The Standish Group International, 1994

[6] Industrial Software Metrics Top 10 List, by Boehm, Barry, IEEE Software, Volume 4 Number 5, September, 1987, pp 84-85

[7] E-Projects in India, by Jim Highsmith, e-Project Advisor, Cutter Consortium's e-Project Management Advisory Service, March 1, 2001.

[8] Gone But Never Forgotten, by Brad Stratton, editor, Quality Progress Magazine, March 1994
______________

Annotated Bibliography (Chronological)

Quality Control Handbook, Joseph M. Juran, originally published in 1951, now in its forth edition
Considered the standard reference in the field of quality. Like Demming, Juran consulted mainly in Japan during the 1950’s and 60’s.
Managerial Breakthrough, Joseph M. Juran, originally published in 1964
Widely ignored when it was first published, this book is now considered a landmark treatise on continuous improvement.
Quality is Free, by Philip Crosby, New York:  McGraw-Hill, Inc. 1979
This is the book we used to launch the TQM program in our plant.  Demming never liked the zero defects approach advocated in this book, and it contains little about statistical quality control.  However, many corporate executives got the quality message from Phil Corsby.
Study of ‘Toyota’ Production System from Industrial Engineering Viewpoint, by Shigeo Shingo, Osaka, Japan, Shinsei Printing Co. Ltd.  1981
The title of this book is an indication of the quality of the translation, but that was not a barrier to our plant.  We studied the book cover-to-cover and got our introduction to Lean Manufacturing from this book.
Toyota Production System, Practical Approach to Production Management, by Yasuhiro Monden, Norcross, Georgia, Industrial Engineering and Management Press.  1983
This is the classic book on Lean Manufacturing.
Zero Inventories, by Robert W. Hall, Homewood, IL, Dow Jones-Irwin 1983.
Robert Hall, a professor at Indiana University, understood the concept of Just-in-Time earlier than most academics.  This book is still considered the definitive work on JIT.
A Revolution in Manufacturing, the SMED System, by Shigeo Shingo, Cambridge, MA, Productivity, Inc.; Originally published as Shinguru Dandori in 1983, English translation 1985.
The title of this book is an indication of the quality of the translation, but that was not a barrier to our plant.  We studied the book cover-to-cover and got our introduction to Lean Manufacturing from this book.
The Goal, by Eliyahu M. Goldratt, First Edition Published in 1984, Second Revised Edition Published in Great Barrington, MA, 1992
This book is a business novel. It is the easiest book to read for an introduction to Lean Manufacturing and is definitely a classic.  You will find this book on the reading list of most Operations Management courses.  Goldratt went on to develop the ‘Theory of Constraints’ and write several related business novels.
Out of the Crisis, by W. Edwards Demming; 1986
This is the book in which Demming outlined his famous 14 points.
The Demming Management Method, by Mary Walton and W. Edwards Demming; 1988
Probably the best book there is on Demming and his management approach.
Toyota Production System – Beyond Large Scale Production, Taiichi Ohno, published in Japanese in 1978 and in English in 1988 by Productivity, Inc.
This is an explanation of JIT by the inventor. The English version arrived after JIT had become widespread in the US.
The Machine That Changed the World : The Story of Lean Production, by Womack, James P., Daniel T. Jones, and Daniel Roos, New York: Rawson and Associates; 1990
This landmark book about the Toyota Production System was the first to associate the term ‘Lean’ with manufacturing.
Lean Thinking, by Womack, James P., and Daniel T. Jones, Simon & Schuster; 1996
A follow-up on the story of Lean Production, this book extends the concept of Lean throughout the enterprise.  It shows how the "lean principles" of value (as defined by the customer), value stream, flow, pull, and perfection can be applied to all areas of the enterprise.  (Software development is implicitly included.)

Originally Published in Software Development Magazine, May and June, 2001.

Screen Beans Art, © A Bit Better Corporation