Train-Wreck Management

“On October 5, 1841, two Western Railroad passenger trains collided somewhere between Worchester, Massachusetts and Albany, New York, killing a conductor and a passenger and injuring seventeen passengers. That disaster marked the beginning of a new management era." [1] These words open Peter Scholtes classic book on leadership. He goes on to explain how the term "management" was unknown in the days of cottage industries. As business grew and became geographically disperse in the 1800's, a way to run these businesses had to be found. But there were no models outside the church and the military, so investigators into the train-wreck disaster looked to the Prussian army for a model. And there they found the classic organization chart - the one we know so well today. Scholtes calls it the "train-wreck" chart. It was revolutionary at the time.

The purpose of what became today's organization chart was clear: The assignment of responsibility would enable "prompt detection of derelictions of duty... and point out the delinquent." Scholtes says: "A fundamental premise of the 'train-wreck' approach to management is that the primary cause of problems is 'dereliction of duty'. The purpose of the organizational chart is to sufficiently specify those duties so that management can quickly assign blame, should another accident occur."[1].

Blame
Note the thinking here: Problems are caused by people who don't do their job well, so finding someone to blame is the first step to correcting problems. Scholtes notes: "The era of management that began in the mid-1800's can be characterized as "management by results.".... Since managers could no longer do the work themselves or direct others in the doing of the work, managers exercised their authority by holding people accountable for results.... In the 1950's, management by results reached its epitome in MBO (Management By Objectives) and performance appraisals, the Harvardization of train-wreck management."[1] He goes on to say that at the time, this theory of management was the best available, and it succeeded in creating order out of chaos. "People like Whistler, McCallum, Frederick Taylor or Henry Ford in the United States or Darby, the Stephensons, or Brunel in England were pioneers.... they did their best and, by and large, what they did was very good."

"Meanwhile, in Japan...." is the title of the next section Scholtes' book. He chronicles how a better approach to management emerged in Japan in the 1950's assisted by W. Edwards Deming. Deming taught that most of the problems we encounter (perhaps 90%) are the result of multiple influences, they generally cannot be attributed to a single cause. Assigning blame for a problem to the last person involved is worse than counterproductive, it will probably make the bad situation worse. Exhorting people to "be careful," "try harder," and "work smarter" is not useful if individuals have little effect on results. Rewarding or punishing people for outcomes that are not under their control can only result in discouragement - or in gaming the system. Instead, chronic problems must be fixed by finding their underlying causes and addressing these effectively. As Deming points out, this usually involves changing the system - the way things are done. And according to Deming, it is management's job to change the system.

Process or People?
Agile software development places a strong emphasis on putting change into the hands of front-line people on self-directed teams - isn't this contrary to Deming's philosophy? Writing in 1995, Scholtes lists what he calls "fads" for addressing systemic problems: "empower people, put them into self-directed teams, motivate them, offer incentives, reengineer and reinvent them." And then he says: "All of the empowered, motivated, teamed-up, self-directed, incentivized, accountable, reengineered, and reinvented people you can muster cannot compensate for a dysfunctional system.... A well-run organization with well-functioning systems allows people from top to bottom do work of which they can be proud."[1] So where does this leave us? Which is more important - process or people?

It helps if we trade in the overloaded word "process" and use "system."

In the article "Managing a Living System, not a Ledger"[2] H. Thomas Johnson says "Managers at Toyota believe that improving the system is the surest way to improve long term financial results." He points out that Toyota takes lots and lots of measurements, but they do not use these as performance measurements. Johnson writes: "...Toyota makes virtually no use of management accounting targets (or 'levers') to control or motivate operations... Toyota focuses its operations on continuous system improvement through endless rapid problem solving. And they emphasize genchi genbutsu, or 'going to the place,' to see where a problem occurs, firsthand. They don't rely on second-hand reports or tables and charts of data to achieve a true understanding of root cause. Instead they go to the place (gemba) where you can watch, observe, and 'ask why five times.' This attitude shows a deep appreciation that results (and problems) ultimately emanate from, and are explained by, complex processes and concrete relationships, not by abstract, quantitative relationships that describe results in simple, linear, additive terms." Winding up the article, Johnson says: "Financial quantities cannot reveal if a system is improving or not... No company that talks about improving performance can know what it is doing if its primary window on results is financial information and not system principles.... Companies that intend to perform like Toyota should recognize that... they will never get there by trying to motivate and direct 'lean' initiatives with 'lean accounting' and management accounting 'levers of control.'"

Taiichi Ohno on Standard Work
Let's go back to the source of the Toyota Production System, Taiichi Ohno, and see what he had to say about process - how it is established and how it is changed.[3]

There is something called standard work, but standards should be changed constantly. Instead, if you think of the standard as the best you can do, it's all over. The standard work is only a baseline for doing further kaizen. It is kai-aku [change for the worse] if things get worse than now, and it is kaizen [change for the better] if things get better than now. Standards are set arbitrarily by humans, so how can they not change?

When creating Standard Work, it will be difficult to establish a standard if you are trying to achieve 'the best way.' This is a big mistake. Document exactly what you are doing now. If you make it better than it is now, it is kaizen. If not, and you establish the best possible way, the motivation for kaizen will be gone. That is why one way of motivating people to do kaizen is to create a poor standard. But don't make it too bad. Without some standard, you can't say 'We made it better' because there is nothing to compare it to, so you must create a standard for comparison.

Take that standard, and if the work is not easy to perform, give many suggestions and do kaizen.

We need to use the words 'you made' as in 'follow the decisions you made.' When we say 'they were made' people feel like it was forced upon them. When a decision is made, we need to ask who made the decision. Since you also have the authority to decide, if you decide, you must at least follow your decision, and then this will not be forced upon you at all.

But in the beginning, you must perform the Standard Work, and as you do, you should find things you don't like, and you will think of one kaizen idea after another. Then you should implement these ideas right away, and make this the new standard.

Years ago, I made them hang the standard work documents on the shop floor. After a year I said to a team leader, 'The color of the paper has changed, which means you have been doing it the same way, so you have been a salary thief for the last year.' I said 'What do you come to work to do each day? If you are observing every day you ought to be finding things you don't like, and rewriting the standard immediately. Even if the document hanging there is from last month, this is wrong.' At Toyota in the beginning we had the team leaders write down the dates on the standard work sheets when they hung them. This gave me a good reason to scold the team leaders, saying 'Have you been goofing off all month?'

If it takes one or two months to create these documents, this is nonsense. You should not create these away from the job. See what is happening on the gemba and write it down.

Process AND People
Ohno believed that the primary job of team leaders (first line supervisors) is the constant improvement of the way work gets done. Work standards should be written and posted, but this had better not take very long because the standards should change all the time - at least once a month. Standards are not about how work should be done, but how work is being done. You don't want the standard to be too perfect, because that leaves no incentive for workers to improve their standards. If workers are annoyed by a standard, they are expected to change it. They do not drop a suggestion in a suggestion box, they do kaizen. That is, workers - led by their team leader - do many rapid experiments, find a better way, agree on the improvement, quickly document the new way, and use it. When a standard is improved, the decision for the change must be made by the people doing the work, so they won't feel it is being forced upon them.

People like to use effective processes, and they also like to have control over their own environment. The Toyota Production System provides for both. Ohno made it clear that people must be at the center of improving their own processes. Process improvement may be done only "at the gemba" and it is up to the workers to decide whether or not a proposed improvement should be implemented. Workers are expected to keep changing the way they do their job; in fact, it is bad leadership to have a process so perfect that workers have little incentive to improve it!

Assessment and Certification
Scholtes takes process improvement assessment programs such as ISO 9000 to task because even though they seem good on the surface, they have some problems:[1]

1. The pursuit of quality must be guided by a larger context than certification - it requires a holistic, integrated, long term commitment.
2. Certification is not equal to satisfied customers - you can do the wrong thing as long as you do it consistently.
3. Assessment has a tone of paternalism and mistrust - it replaces internal motivation with external motivation.
4. Assessment assumes that inspectors are all the same - but inspections are not standardized.
5. A certified process is difficult to change - Ohno would be appalled.

Conclusion
When Deming said "change the system", he was talking about changing the complex, interrelated processes used to get work done. Deming believed that changing the system is management's primary job, and in order to do this, managers need competency in four areas:

1. Appreciation for the overall system in which work is done
2. An understanding of variation - and the true relationship between cause and effect
3. Constant pursuit of learning (improvement) through designed experiments
4. An understanding of the psychology of people

When all of these areas are balanced and working together, great things can happen.

References
[1] The Leader's Handbook, by Peter R. Scholtes, McGraw-Hill, 1998.

[2] "Managing a Living System, Not a Ledger,", by H. Thomas Johnson, Lean Manufacturing 2007, Supplement to Manufacturing Engineering, SME, August 2007. Johnson also coauthored Profit Beyond Measure: Extraordinary Results through Attention to Work and People, Free Press, November, 2000.

[3] Workplace Management, by Taiichi Ohno, originally published in 1982, from translation by Jon Miller, Gemba Press, 2007.

Screen Beans Art, © A Bit Better Corporation

Cause and Effect

Is low cost achieved by focusing on cutting costs?
Is high utilization achieved by trying to utilize resources full time?
Does standardized work mean that work processes are followed without challenge?
Does everyone in your organization agree on the answers to these questions?

Cost
When Jane Beseda took over Toyota’s North American Parts Operation (NAPO), she knew that really dramatic results required breaking down the barriers between departments [1].  So she set three year Stretch Goals that were all but impossible – a 50% reduction in inventory, 25% increase in throughput, 25% reduction in freight costs, 50% reduction in packaging expense, 25% increase in space utilization, 50% decrease in backorders.  After a year of effort, department managers began to realize that that they were not going to achieve the Stretch Goal targets unless they changed their focus to cross-organizational projects.  Project teams had been struggling to coordinate their work through the traditional functional department planning approach. This was changed; the departments started to consider potential impact of their plans on other functions and areas of the broader organization. Only then were truly significant gains in eliminating waste and reducing cost realized. After three years, the results at NAPO were truly amazing; almost all of the Stretch Goals were achieved. But this would never have happened if everyone in the organization had not focused on overall system waste rather than individual department costs.

Low overall costs rarely come from lowering costs in individual departments; they come from lowering system costs. Lean companies have learned that this requires a keen understanding of underlying cost drivers and a system-wide effort to eliminate waste.  Consider Zara, a huge women’s fashion clothing chain headquartered in western Spain. It fills retail store orders twice a week, shipping garments around the world in two days – not folded compactly in boxes – but ironed and hanging on hangers ready to display. Zara realizes that it lowers overall costs by getting the clothing that women are asking for on shelves very rapidly.  Zara has fewer markdowns and unsold clothes, and it drives more business to stores with less advertising than its competitors. These tremendous savings would disappear if Zara focused on reducing shipping costs. This is systems thinking at work.

Over time, individual departments will find ways to drive out costs, but rarely do organizations attack the larger costs that occur between departments, divisions, functions, and companies.  A quick and easy cost reduction approach for a department might be to outsource work to obtain lower labor costs.  However, in most cases, outsourcing simply adds more boundaries to cross, and boundaries typically add 20 to 30% to overall costs [2]. One company I know outsourced the process of engaging contractors.  Since training is included in this outsourcing arrangement, all trainers must work through the outsourced contractor management company.  The amount of time and paperwork necessary to set up a training engagement at this company is about an order of magnitude greater than at any other company I have dealt with. However, the arrangement appears to be a cost reduction for the company, because contractors pay the contractor management firm through a fee imposed on their payment. This financial arrangement hides the tremendous waste in the engagement process, which includes a great deal of problem resolution by employees in the company receiving the training. But because of organizational barriers, these same employees have no way to change or improve a very burdensome process.

Standardized Work
In studying Toyota, many companies notice that standardized work processes are a key element of the company’s success.  What they generally miss is that at Toyota, standards exist to be challenged and changed by the front line employees doing the work.  Toyota actively encourages all employees to question any part of their job that is annoying or gets in the way of doing a good job. Employees are expected to look for a better way, prove with experiments that the new way is better, and then implement a new standard.  Taiichi Ohno, father of the Toyota Production System, wrote: “Something is wrong if workers do not look around each day, find things that are tedious or boring, and then rewrite the procedures. Even last month’s manual should be out of date.”

There are those who believe that standardized processes should be imposed from the outside and followed without question. However, companies with a Lean perspective believe that the essence of eliminating waste is to encourage everyone to attack and change the things that annoy them about their jobs. Only by engaging people in doing their job better on a daily basis can you get the sustainable gains that companies like Toyota enjoy. Work standards in a lean company spell out the current best known way of doing things, and they are a baseline for change. New standards are developed through constant experimentation by work teams, using the current work standards as the baseline against which improvement is measured.

At Toyota’s Georgetown, Kentucky plant the paint department has always been the biggest bottleneck preventing one-piece-flow of vehicles through the plant.  Over the past few years, however, things began to change [3].  First the people in the department devised a way to paint any color in any order – instead of piping paint to a robot through a tube that needs cleaning between colors, they now pipe each paint color to a canister just the right size to paint a car.  The robot picks up the right canister and Viola! the cost and time needed to clean out paint lines is eliminated.  The bottom line:  30% less paint used, a huge drop in the use of cleaning solvents, and cars can now be painted any color in any order. The throughput of the shop has increased from 33 to 50 an hour while the space required for painting was reduced by a third and employees were freed up for other work.

At most auto companies, this achievement would be cause for celebration – but at Toyota, constant improvement is the day-in-day-out job of the paint manager. There are no black belts, no outside process experts – just the people doing the work led by their first line managers. Everyone’s job is to keep on improving their work processes, every day, every week, every month. All workers are engaged in the relentless pursuit of perfection – the elusive goal which keeps everyone looking forward rather than patting themselves on the back.

In software development, we have been led to believe that “maturity” involves documenting best practices and making sure that they are followed. On the contrary, a truly mature organization expects development teams to constantly challenge and improve their processes.  The day an organization settles into believing that it is perfect is the day it invites its people to stop thinking. Real, sustainable improvement on all fronts comes only when all workers are expected to challenge and fix anything about their job that annoys them or keeps them from taking pride in their work.

Utilization
Managers in manufacturing plants, supervisors of computer operations, and highway engineers agree on one cause and effect relationship:  if your resources approach full utilization, response time slows to a crawl. Queuing theory is widely applied to servers and highways and manufacturing equipment.  However, in a development environment, managers seem to be unaware that attempting to achieve full utilization slows response time to a crawl and decreases actual utilization. Development organizations are no more immune from the laws of queuing theory than airlines. We see how constantly full airlines with no slack cause cascading problems whenever a slight perturbation is introduced into the airline system. And weather being what it is, perturbations happen all the time.

Similarly, when a slight perturbation is introduced into a fully scheduled development organization, cascading problems are inevitable. Knowledge work being what it is, there will always be perturbations to the most carefully laid out schedules. The only way to deal effectively with these perturbations is to follow the advice of queuing theory:  work in small batches, minimize the length of queues of work to be done, and never, ever schedule an organization beyond its capacity to deliver.  Do this and utilization will increase.  Focus directly on utilization and it is guaranteed to be sub-optimal.

Many organizations queue up requests for software development into large batches of work to be done, presumably because better decisions can be made if the whole picture is visible before work begins.  Challenge this assumption.  Why is it better to respond very slowly to a pile of accumulated requests rather than respond very quickly to the current most important outstanding request? What good, really, does it do to have a long list of work to do?  Looked at closely, it's hard to make a case for piling up lists of work-to-do that are far longer than you have a hope of accomplishing. You're pretty much wasting your time keeping track of stuff you can never get around to, and you're probably setting incorrect expectations on the part of the requestors. Just say no.

Unfortunately, once software has been around for a while, we often see that it takes longer and longer to run the regression tests as each new feature is added, since every feature increases the regression test load.  We call this increasing regression load the regression deficit, and unless it is systematically tackled and reduced, the code base will become increasingly difficult to change.  You will be tempted to release software in larger and larger batches, because regression testing takes so long. Similar to the paint shop manager, a software development manager’s job should be to help development teams chip away at the regression deficit every day, every week, every month, until throughput is increased and one-piece-flow of small feature sets becomes practical.

Cause & Effect
In order for organizations to perform brilliantly, there are two prerequisites:  First, everyone has to agree on what they want, and second everyone has to agree on cause and effect [4].  Let’s assume that everyone in your organization agrees on the results they want, their values and priorities, and the trade-offs they are willing to make in order to achieve those results.  The question to ask is – does everyone agree on cause and effect?

Is there general agreement on what actions will result in system-wide cost reduction?  Are individual departments expected to reduce costs independently, or is it clear that departments must work together to reduce overall system costs, even at the expense of costs in individual departments? Are the measurements in place to reduce the costs of crossing boundaries?

Is there general agreement on the best way to achieve consistent results through standardization?  Are standards always followed?  If not, do you really understand the root cause – are the standards irrelevant or inaccurate, too complex, or too far from the actual work being done? Are standards maintained by a central process group, or are they continually improved by the people doing the work?  Are front line people expected and encouraged to continually challenge and change standards?

Do you have a project scheduling system aimed at optimizing utilization?  Does the management team believe that the system is in fact optimizing utilization? Do people believe that full utilization is the right measurement to emphasize? Is time-to-market an important factor in your business? Does everyone agree on the relationship between time-to-market and utilization?

Lean thinking is counterintuitive; it flies in the face of conventional wisdom concerning cause and effect. No matter how well proven the results, no matter how intellectually sound the arguments of systems thinking are, it is almost impossible for those who have been successful with conventional wisdom to change their habits. It is very difficult for leaders to address the system costs that flourish between department boundaries if they believe that lowering costs in each department will add up to lower overall costs. When leaders believe that standardized processes are an end rather than a beginning, they have a difficult time leveraging the wisdom of front line employees. For those leaders who believe that focusing on full utilization will increase productivity, the outstanding productivity gains that come from focusing instead on throughput will not be available.  It all comes down to measures – small measures cause small results; global measures promote globally optimized results.

Those who have seen the dramatic results of Lean Thinking have come to believe that concentrating on throughput, making front line employees the center of focus of the company, and eliminating waste across the system are the tools of choice in a competitive environment.  Most management teams that have been threatened by fierce competition and survived have changed their habits and adopted the counterintuitive tenets of Lean Thinking. Virtually all companies facing fierce competition from Lean companies that have failed to adopt similar thinking have failed to thrive in the long run.
_________________
Footnotes:

[1] This story is from The Elegant Solution: Toyota’s Formula for Mastering Innovation, by Matthew E. May, Free Press, 2007, p. 139.

[2] Management Challenges for the 21st Century, by Peter Drucker, HarperBusiness, 2001, p. 33.

[3] From “No Satisfaction” by Charles Fishman, Fast Company, Dec 2006 / Jan 2007

[4] See “The Tools of Cooperation and Change,” by Clayton Christensen and co authors, Harvard Business Review, October 2006.


Screen Beans Art, © A Bit Better Corporation

Toward A New Definition of Maturity

In the mid 90's, a company named Zeos assembled PC's in my home state of Minnesota. I was impressed when Zeos was named as a finalist for the Malcolm Baldrige Award, because competing for this quality award is more or less the equivalent of a software company trying to reach CMM Level 5. At the same time that Zeos was focusing on the Malcolm Baldrige Award, a similar company in Austin, Texas, called Dell Computer, was focusing all of its energy on two rather different objectives: 1) keep inventory as low as possible, because it is the biggest risk in the PC assembly business, and 2) deliver computers just as fast as possible after customers decide what they want. Dell could not possibly meet these goals unless it had the key Malcolm Baldrige capabilities in place, but that was not its focus. On the other hand, in order to be a finalist, Zeos had to spend huge amounts of executive and management attention on Malcolm Baldrige activities. So which of these two companies would you call the most mature?

A company does not rise to the top of its industry by perfecting its normative procedures. While General Motors was busy refining it’s four phase process for product development, Toyota and Honda were developing cars in something approaching half the time, for half the cost. The resulting cars were less expensive to produce and captured a much larger market share. Yet when looked at through Detroit’s eyes, the development approach of the Japanese companies was decidedly immature – why, they even let die makers start cutting dies long before the final drawings were released!

The problem with maturity models is that they foster a mental model of ‘good practice’ that can block out paradigm shifts that are destined to reshape the industry. This has happened in manufacturing. It has happened in product development. Can it be happening in software development today?

Why Do We Do This To Ourselves?
We human beings a tendency to decompose complex problems into smaller pieces, and then focus on the individual pieces, often, unwittingly, at the expense of the whole. One reason for this is that people can only hold a few chunks of information in short term memory at once; Miller’s law claims that this number is seven plus or minus two. Thus we have a strong tendency to take concepts which we cannot get our minds around and decompose them into parts, so we can deal with one part at a time. There is nothing wrong with our very natural tendency to decompose problems, except that we often forget to reconfigure the parts into a whole and take their interactions into account.

The problem is, decomposition only works if the whole is indeed equal to the sum of its parts, and interactions between the parts can be ignored. In practice, this is rarely the case; in fact, optimization of the parts has a tendency to sub-optimize the whole. For example, a manager might think that the best way to run a testing department is to make sure that every single person is working all of the time. In order to guarantee maximum productivity of each individual, he makes sure there is always a pile of testing waiting to be done. Down the hall, the operations manager knows better. She knows that if she runs the servers at high utilization, the entire system bogs down, just like rush hour traffic. If only the testing manager would realize that his policy of full utilization of testing resources creates the same kind of traffic jam in software development!

Decomposition of a problem into pieces is a standard problem solving heuristic, but it needs to be accompanied by regular aggregation of the parts into a whole. It is here that iterative development shines, because it forces us to develop, test, integrate, and release complete threads of the system early and often. Thus we reset our view of the big picture every iteration. However, iterative development requires that we decompose the problem space along dimensions that are orthogonal to those traditionally recommended. Instead of decomposing the problem into requirements, analysis, design, programming, testing, integration, deployment, we decompose the problem into features along the natural fault lines of the domain.

Decomposition vs. Abstraction
There is an alternative to the decomposition of a problem into components, and that is to abstract the problem to a higher level, rather than decompose it to a lower level. The result is the same in both cases – you have reduced the number of things you need to think about to 7 +/- 2. But when you move the problem up to a higher level of abstraction, the interaction of the parts is maintained, so abstractions would seem to be the better approach to solving complex problems.

There is a catch: people can’t create abstractions without understanding the domain, because correct abstractions require that the abstractor knows what is important, what fits together naturally, and where to find the natural joints or fault lines of the domain. For this reason, experts in the domain have a greater tendency to abstract to a higher level, while those who are not familiar with the domain tend to decompose the problem, and that decomposition will probably not be along the natural fault lines of the domain.

It is when we decompose a problem in a manner which does not fit the domain, and then drill down to the details too fast, that important things get missed. We would like to think that a lot of early detail will help us find all the hidden gremlins that might bite us later. This is only true if those gremlins are actually inside the areas we investigate – but the tough problems usually lurk between the cracks in our thinking. Unfortunately we only discover those problems when we integrate the pieces together at the end, and by that time we have invested so much in the details that change is very difficult.

It is safer to work from a higher level or abstraction, but this means we don’t drill down to detail right away. We take a breadth-first approach, keep options open and gradually fill in the details. With this approach we are more likely to find those gremlins while we still can do something about them, but some people might think that we don’t have a plan, aren’t tracking the project, and can’t manage requirements.

Assessment Rather than Certification
Quite frequently CMM is implemented as a certification process, decomposing ‘mature’ into separately verifiable capabilities. The danger is that we may lose sight of forest for the trees. For example, focusing on requirements management can be detrimental to giving users what they really want. Focusing on quality assurance separate from development can destroy the integrating effect of developers and testers working side-by-side every day. Focusing on planning as a predictive process can keep us from using planning as an organizing process.

A better approach to discovering the underlying competence of an organization is to use an assessment process rather than a certification process. Assessments present challenging situations that cannot be dealt with unless a host of capabilities are in place; if the challenge is properly handled, the presence of these capabilities can be inferred. For example, a pilot’s ability to fly a plane can be assessed by observing how the pilot lands a plane in a stiff cross wind.

Consider your hiring process. When a candidate lists Microsoft or PMI certifications, you take that into account, but you are really looking for a track record of success which demonstrates that the certifications were put to good use. One software company I know administers a one hour logic tests to job applicants, a test without a line of code in it. Yet the test does an admirable job of assessing the capability of the candidate to think like a good developer.

Measure UP
If we accept that we get what we measure then, as Rob Austin points out in “Measuring and Managing Performance in Organizations,” we do not get what we don’t measure. If we are not measuring everything, if something important just didn’t make it into our measurement system, then we are not going to get it. To counter this problem, we have a tendency to pile one measurement on top of another, every time we discover that we forgot to measure something. In CMM, for example, each KPA addresses something that caused a failure of some software project somewhere. As people discovered new ways for software projects to fail, a KPA’s was added to address the new failure mode. Nevertheless, all of those KPA’s still don’t cover everything that could go wrong, although they certainly try.

Once we admit that we just can’t measure everything, then we are ready to move to assessment-style measurements rather than a certification-style measurements. For example, in project management, we decompose the measurement system into cost, schedule, scope and defects, and we try hard to make these measurements work because we think they are the measurements we should use. But no matter how hard we try, we are often unsuccessful in measuring true business value by totaling up the yardsticks of cost, schedule, scope and defects.

What if we just measured business value instead of cost, schedule, scope, and defects? When I was developing new products at 3M, we did not pay much attention to cost, schedule and scope. Instead we developed a P&L which we would use to check out the impact of a late introduction date or a lower unit cost. The development team tried to optimize the overall P&L, not any one dimension. It may seem strange that a project team would concern itself with the financial model of the business, but I assure you it is far better decision-support tool than cost, schedule and scope.

The Measure of Maturity
I believe that our industry could use a simpler measurement of software development maturity, one that is not subject to the dangers of decomposition and does not attempt to defy Miller’s Law with a long lists of things to think about. I propose that we use a measurement that has been successfully used in countless organizations, and has proven to be a true indicator of the presence of the kind of capabilities measured by CMM. To introduce this measurement, let’s go back to Dell and see what it measured:

1. The level of inventory throughout the entire system, and
2. The speed with which the organization can repeatedly and reliably respond to a customer request.

Okay, you say, fine for manufacturing, but software development is different.

True, I counter, but these measurements still work.

The level of inventory in your system is the amount of stuff you have under development. The more inventory of unfinished development work you have sitting around, the more you are at risk of it growing obsolete, getting lost, and hiding defects. If you capitalized it, you also bear the risk of having to write it off if it doesn’t work. The less of that kind of stuff you have on hand, the better off you will be.

The speed with which you can responds to a customer is directly proportional to the amount of unfinished development work you have clogging up your system. In truth, the two measurements above are one and the same – you can deliver faster if your system is not clogged with unfinished work. But more to the point, you cannot reliably and repeatedly deliver fast if you do not have a mature organization. You need version control, built-in quality, ways to gather requirements fast and routinely translate them correctly into code. You need everything you measure in CMM, and you need all of those capabilities working together as an integrated whole. In short, the speed with which you can repeatedly and reliably deliver on customers requests is a better measure of the maturity of your software development organization.

Published as “Measuring Maturity” in Software Development Magazine, February 2004.

Screen Beans Art, © A Bit Better Corporation