The Cost Center Trap

In the 1960’s, IT was largely an in-house back-office function focused on process automation and cost reduction. Today, IT plays a significant strategic and revenue role in most companies, and is deeply integrated with business functions. By 2010, over 50% of firms’ capital spending was going to IT, up from 10-15% in the 1960’s.[1] But one thing hasn't changed since the 1960’s: IT has always been considered a cost center. You are probably thinking "Why does this matter?" Trust me, cost center accounting can be a big trap.

Back in the mid 1980’s Just-in-Time (JIT) was gaining traction in manufacturing companies. JIT always drove inventories down sharply, giving companies a much faster response time when demand changed. However, accounting systems count inventory as an asset, so and any significant reduction in inventory had a negative impact on the balance sheet. Balance sheet metrics made their way into senior management metrics, so successful JIT efforts tended to make senior managers look bad. Often senior management metrics made their way down into the metrics of manufacturing organizations, and when they did, efforts to reduce inventory were half-hearted at best. A generation of accountants had to retire before serious inventory reduction was widely accepted as a good thing.[2] 

Returning to the present, being a cost center means that IT performance is judged – from an accounting perspective – solely on cost management. Frequently these accounting metrics make their way into the performance metrics of senior managers, while contributions to business performance tend to be deemphasized or absent. As the metrics of senior managers make their way down through the organization, a culture of cost control develops, with scant attention paid to improving overall business performance. Help in delivering business results is appreciated, of course, but rarely is it rewarded, and rarer still is the cost center that voluntarily accepts responsibility for business results.

Now let’s add an Agile transformation to this cost center culture. Let’s assume that the transformation is supposed to bring benefits such as faster time to market, more relevant products, better customer experiences. And let’s assume that the cost center metrics do not change, or if they do change, process metrics such as number of agile teams and speed of deployment are added. I’ll wager that very few of those agile teams are likely to focus on improving overall business performance. The incentives send a clear message: business performance is not the responsibility of a cost center.

Being in a cost center can be demoralizing. You aren’t on the A team that brings in revenue, you’re on the B team that consumes resources. No matter how well the business performs, you’ll never get credit. Your budget is unlikely to increase when times are good, but when times are tight, it will be the first to be cut. Should you have a good idea, it had better not cost anything, because you can’t spend money to make money. If you think that a bigger monitor would make you more efficient, good luck making your case. Yet if your colleagues in trading suggest larger monitors will help them generate more revenue, the big screens will show up in a flash.[3]

Let’s face it, unless there are mitigating circumstances, IT departments that started out as cost centers are going to remain cost centers even when the company attempts a digital transformation. What kind of mitigating circumstances might help IT escape the cost center trap?

1. There is serious competition from startups.
   Startups develop their software in profit centers; they haven’t learned about cost centers yet. And in a competitive battle, a profit center will beat a cost center every time.
2. IT is recognized as a strategic business driver.
   You would think that a digital transformation would be undertaken only after a company has come to realize the strategic value of digital technology, but this is not the case. IT has been treated as if it were an outside contractor for so long that it is difficult for company leaders to think of IT as a strategic business driver, integral to the company's success going forward.
   
3. A serious IT failure has had a huge impact on business results.
   When it becomes clear exactly how dependent a profit center is on a so-called cost center, people in the profit center are often motivated to share their pain with IT. Smart IT departments will use this opportunity to share the gain also.

Many people in the Agile movement preach that teams should have responsibility for the outcomes they produce and the impact of those outcomes. But responsibility starts at the top and is passed down to teams. When IT is managed as a cost center with cost objectives passed down through the hierarchy, it is almost impossible for team members from IT to assume responsibility for the business outcomes of their work. When IT metrics focus on cost control, digital transformations tend to stall.

Every ‘full stack team’ working on a digital problem should have ‘full stack responsibility’ for results, and that responsibility should percolate up to the highest managers of every person on the team.  Business results, not cost, should receive the focused attention of every member of the team, and every incentive that matters should be aimed at reinforcing this focus.

The Capitalization Dilemma

Let’s return to the surprising assertion that in 2010, over 50% of firms’ capital spending was going to IT.[1] One has to wonder what was being capitalized. Yes, there were plenty of big data centers that were no doubt capitalized, since the movement to the cloud was just beginning. But in addition to that, a whole lot of spending on software development was also being capitalized. And herein lies the seeds of another undue influence of accounting policies over IT practices.

Software development projects are normally capitalized until they are “done” – that is they reach "final operating capability" and are turned over to production and maintenance.[1] But when an organization adopts continuous delivery practices, the concept of final operating capability – not to mention maintenance – disappears. This creates a big dilemma because it's no longer clear when, or even if, software development should be capitalized. Moving expenditures from capitalized to expensed not only changes whose budget the money comes from, it can have tax consequences as well. And what happens when all that capitalized software (which, by the way, is an asset) vanishes? Just as in the days when JIT was young, continuous delivery has introduced a paradigm shift that messes up the balance sheet.

But the balance sheet problem is not the only issue; depreciation of capitalized software can wreck havoc as well. In manufacturing, the depreciation of a piece of process equipment is charged against the unit cost of products made on that equipment. The more products that are made on the equipment, the less cost each product has to bear. So there is strong incentive to keep machines running, flooding the plant with inventory that is not currently needed. In a similar manner, the depreciation of software makes it almost impossible to ignore its sunk cost, which often drives sub-optimal usage, maintenance and replacement decisions.

Capitalization of development creates a hidden bias toward large projects over incremental delivery, making it difficult to look favorably upon agile practices. Hopefully we don't have to wait for another generation of accountants to retire before delivering software rapidly, in small increments, is considered a good thing.

To summarize, the cost center trap and the capitalization dilemma both create a chain reaction:

1. Accounting drives metrics.
   ⇩
2. Metrics drive culture.
   ⇩
3. Culture eats process for lunch.

The best way to avoid this is to break the chain at the top – in step 1. Stop letting accounting drive metrics. Alternatively, if accounting metrics persist at the senior management level, then break the chain at step 2 – do not pass accounting metrics down the reporting chain; do not let them drive culture. When teams focus on improving the performance of the overall business, accounting metrics should move in the right direction on their own; if they don't then clearly something is wrong with the accounting metrics.

Beware of Proxies

This year Jeff Bezos's annual letter to Amazon shareholders[4] listed four essentials that help big companies preserve the vitality of a startup: customer obsession, a skeptical view of proxies, the eager adoption of external trends, and high-velocity decision making. These seem pretty clear, except maybe the second one: a skeptical view of proxies. Just what are proxies? Bezos explains:

“A common example is process as proxy. Good process serves you so you can serve customers. But if you’re not watchful, the process can become the thing. This can happen very easily in large organizations. The process becomes the proxy for the result you want. You stop looking at outcomes and just make sure you’re doing the process right. Gulp.”

“Another example: market research and customer surveys can become proxies for customers – something that’s especially dangerous when you’re inventing and designing products.”

Here are some common proxies we find in software development:

Accounting metrics are proxies, and not very good ones at that, because they encourage local sub-optimization. 

Project metrics – cost, schedule, and scope – are proxies. Worse, these proxies are rarely validated against actual outcomes.  

“The Business” is a proxy for customers. Generally speaking, so is the product owner.

Proxies should be resisted, Bezos argues, if you want a vibrant startup culture in your company. But without proxies, how do you manage the dynamic and increasingly important IT organization? You make a habit of measuring what really matters - skip the proxies and focus on outcomes and impact.

In his excellent book, “a Seat at the Table,”[5]  Mark Schwartz proposes that IT governance and oversight should begin with strategic business objectives and produce investment themes that accomplish these objectives. IT leaders fund teams to produce desirable outcomes that will have impact on the strategic objectives. Note that these outcomes are not proxies, they are real, measurable progress toward the strategic objective. Regular reviews of teams’ progress -- quantified by these measurable outcomes -- provides leaders with insight, flexibility and an appropriate level of control. At the same time, detailed decisions are made by the people closest to customers after careful investigation, experimentation and learning.

Schwartz concludes: "this approach can focus IT planning, reduce risk, eliminate waste, and provide a supportive environment for teams engaged in creating value."[5] What's not to like?

______________________
Footnotes:

[1] From “What is Digital Intelligence” by Sunil Mithas and F. Warren McFarlan, IEEE Computing Edge, November 2017. Pg.9.

[2] The 1962 book “The Structure of Scientific Revolutions” by Thomas Kuhn discussed how significant paradigm shifts in science do not take hold until a generation of scientists brought up with the old paradigm finally retire.

[3] Thanks to Nick Larsen. Does Your Employer See Software Development as a Cost Center or a Profit Center?

[4] Jeff Bezos - Letter to Shareholders - April 12, 2017

[5] "A Seat at the Table" by Mark Schwartz

Before There Was Management

Management is a rather recent invention in the history of human evolution – it’s been around for maybe 100 or 150 years, about two or three times longer than software. But people have been living together for thousands of years, and it could be argued that over those thousands of years, we did pretty well without managers. People are social beings, hardwired through centuries of evolution to protect their family and community, and to provide for the next generation. For tens of thousands of years, people have lived together in small hamlets or clans that were relatively self-sufficient, where everyone knew – and was probably related to – everyone else. These hamlets inevitably had leaders to provide general direction, but day-to-day activities were governed by a set of well understood mutual obligations. As long as the hamlets stayed small enough, this was just about all the governance that was needed; and most hamlets stayed small enough to thrive without bureaucracy until the Industrial Revolution.

The Magic Number One Hundred and Fifty
Early in his career, British Anthropologist Robin Dunbar found himself studying the sizes of monkey colonies, and he noticed that different species of monkeys preferred different size colonies. Interestingly, the size of a monkey colony seemed to be related to the size of the monkeys’ brains; the smaller the brain, the smaller the colony. Dunbar theorized that brain size limits the number of social contacts that a primate could maintain at one time. Thinking about how humans seemed to have evolved from primates, Dunbar wondered if, since the human brain was larger than the monkey brain, humans would tend to live in larger groups. He calculated the maximum group size that humans would be likely to live in based on the relative size of the human brain, and arrived at a number just short of 150. Dunbar theorized that humans might have a limit on their social channel capacity (the number of individuals with whom a stable inter-personal relationship can be maintained) of about 150.[1]

To test his theory, Dunbar and other researchers started looking at the size of social groups of people. They found that a community size of 150 has been a very common maximum limit in human societies around the world going back in time as far as they can investigate. And Dunbar’s Number (150) isn’t found only in ancient times. The Hutterites, a religious group that formed self-sufficient agricultural communities in Europe and North America, have kept colonies under 150 people for centuries. Beyond religious communities, Dunbar found that during the eighteenth century, the average number of people in villages in every English county except Kent was around 160. (In Kent it was 100.) Even today, academic communities that are focused on a particular narrow discipline tend to be between 100 and 200 – when the community gets larger, it tends to split into sub-disciplines.[2]

Something akin to Dunbar’s number can be found in the world of technology also. When Steve Jobs ran the Mackintosh department at Apple, his magic number was 100. He figured he could not remember more than 100 names, so the department was limited to 100 people at one time. A team that never exceeded 100 people designed and developed both the hardware and software that became the legendary Apple Macintosh.[3] Another example: in a 2004 blog The Dunbar Number as a Limit to Group Sizes, Christopher Allen noted that on-line communities tend to have 40 to 60 active members at any one time. You can see two peaks in Allen’s chart of group satisfaction as a function of group size – one peak for a team size of 5 to 8, and an equally high peak when team size is around 50.[4]

Steve Job’s limit of 100 people was probably a derivative of the Dunbar Number, but Allen’s peak at 50 is something different. According to Dunbar, “If you look at the pattern of relationships within… our social world, a number of circles of intimacy can be detected. The innermost group consists of about three to five people. … Above this is a slightly larger grouping that typically consists of about ten additional people. And above this is a slightly bigger circle of around thirty more…”[5] In case you’ve stopped counting, the circles of intimacy are 5, 15, 50, 150 – each circle about three times the size of the smaller circle. The number 50, which Allen found in many on-line communities, is the number of people Dunbar found in many hunting groups in ancient times – and three of these groups of 50 would typically make up a clan.

Does this Work in Companies?
One Hundred and fifty is certainly a magic number for W.L. Gore & Associates. Gore is a privately held business that specializes in developing and manufacturing innovative products based on PTFE, the fluoropolymer in Gore-Tex fabrics. Gore has revenues exceeding 2.5 billion US dollars, employs over 8000 people, and has been profitable for over a half a century. It has held a permanent spot on the U.S. "100 Best Companies to Work For" since it’s inception in 1984, and is a fixture on similar lists in many countries in Europe. This amazing track record might be related to the fact that Gore doesn’t have managers. There are plenty of leaders at Gore, but leaders aren’t assigned the job, they have to earn it by attracting followers.

You’ve got to wonder how such a large company can turn in such consistent performance for such a long period of time without using traditional management structures. The answer seems to have something to do with the fact that Gore is organized into small businesses units that are limited to about 150 people. “We found again and again that things get clumsy at a hundred and fifty,” according to founder Bill Gore. So when the company builds a new plant, it puts 150 spaces in the parking lot, and when people start parking on the grass, they know it’s time to build a new plant.

Since associates at Gore do not have managers, they need different mechanisms to coordinate work, and interestingly, one of the key mechanisms is peer pressure. Here is a quote from Jim Buckley, a long-time associate at a Gore plant: “The pressure that comes to bear if we are not efficient as a plant, if we are not creating good enough earnings for the company, the peer pressure is unbelievable. …This is what you get when you have small teams, where everybody knows everybody. Peer pressure is much more effective than a concept of a boss. Many, many times more powerful.”[6]

Like many companies that depend on employees to work together and make good decisions, Gore is very careful to hire people who will fit well in its culture. Leaders create environments where people have the tools necessary for success and the information needed to make good decisions. Work groups are relatively stable so people get to know the capabilities and expectations of their colleagues. But in the end, the groups are organized around trust and mutual obligation – a throwback to the small communities in which humans have thrived for most of their history.

Google’s management culture has quite a few similarities with Gore’s. Google was designed to work more or less like a university – where people are encouraged to decide on their own (with guidance) what they want to investigate. Google is extremely careful about hiring people who will fit in its culture, and it creates environments where people can pursue their passion without too much management interference. For a deep dive into Google’s culture, see this video: Eric Schmidt at the Management Lab Summit

Peer Cultures
Before there were managers, peer cultures created the glue that held societies together. In clans and hamlets around the world throughout the centuries, the self-interest of the social group was tightly coupled with the self-interest of individuals and family units; and thus obligations based on family ties and reciprocity were essential in creating efficient communities.

There are many, many examples of peer cultures today, from volunteer organizations to open source software development to discussion forums and social networks on the web. In these communities, people are members by their own choice; they want to contribute to a worthy cause, get better at a personal skill, and feel good about their contribution. In a peer culture, leaders provide a vision, a way for people to contribute easily, and just enough guidance to be sure the vision is achieved.

Arguably, peer cultures work a lot better than management at getting many things done, because they create a social network and web of obligations that underlie intrinsic motivation. So perhaps we’d be better off taking a page out of the Gore or the Google or the Open Source playbook and leverage thousands of years of human evolution. We are naturally social beings and have a built-in need to protect our social unit and ensure that it thrives.

Example: Hardware/Software Products
“We have found through experience that the ideal team size is somewhere between 30 and 70,” the executive told us. At first we were surprised. Aren’t teams supposed to be limited to about 7 people? Don’t teams start breaking up when they’re much larger? Clearly the executive was talking about a different kind of team than we generally run into in agile software development. But his company was one of the most successful businesses we have encountered recently, so we figured there had to be something important in his observation.

We spend a morning with a senior project manager at the company – the guy who coordinated 60 people in the development of a spectacular product in record time. The resulting product was far ahead of its time and gave the company a significant competitive advantage. He explained how he coordinated the work: “Every 2 or 3 months we produced a working prototype, each one more sophisticated than the last one. As we were nearing the end of development, a new (faster, better, cheaper) chip hit the market. The team decided to delay the next prototype by two months so they could incorporate the new chip. Obviously we didn’t keep to the original schedule, but in this business, you have to be ready to seize the opportunities that present themselves.”

It’s not that this company had no small teams inside the larger teams; of course they did. It’s just that the coordination was done at the large team level, and the members of the smaller teams communicated on a regular basis with everyone on the larger team. All team members were keenly aware of the need to meet the prototype deadlines and they didn’t need much structure or encouragement to understand and meet the needs of their colleagues.

Another Example: Construction
The Lean Construction Institute has developed a similar approach to effectively organizing construction work. The first thing they do is to break down very large projects into multiple smaller ones so that a reasonable number of contractors can work together. (Remember Dunbar’s Number.) For example, they might completely separate a parking structure and landscaping from the main building; in a large building, the exterior would probably be a separate project from the interior. Each sub-project is further divided into phases of a few months; for example, foundation, structure, interior systems, etc. Before a phase starts, a meeting of all involved contractors is held and all of the things that need to be done to complete that phase are posted on cards on a wall by the contractors. The cards are organized into a timeline that takes dependencies into account, and all of the contractors agree that the wall represents a reasonable simulation of the work that needs to be done. This is not really a plan so much as an agreement among the contractors doing the work about what SHOULD be done to complete the phase.

Each week all of the “Last Planners” (crew chiefs, superintendents, etc.) get together and look at what they SHOULD do, and also what they CAN do, given the situation at the building site. Then they commit to each other what they WILL complete in the next week. The contractors make face-to-face commitments to peers that they know personally. This mutual commitment just plain gets things done faster and more reliably than almost any other organizing technique, including every classic scheduling approach in the book.

The Magic Number Seven
George Miller published “The Magical Number Seven, Plus or Minus Two” in The Psychological Review in 1956. Miller wasn’t talking about team size in this article; he was discussing the capacity of people to distinguish between alternatives. For example, most people can remember a string of 7 numbers, and they can divide colors or musical tones into about 7 categories. Ask people to distinguish between more than 7 categories, and they start making mistakes. “There seems to be some limitation built into us either by learning or by the design of our nervous systems, a limit that keeps our channel capacities in this general range [of seven],” Miller wrote.

This channel capacity seems to affect our direct interaction with other people – we can keep up a conversation with 7 or so people, but when a group gets larger, it is difficult to maintain a single dialog, and small groups tend to start separate discussions. So for face-to-face groups that must maintain a single conversation, the magic number of 7 +/-2 is a good size limit. And historically, most agile software development teams have been about this size.

Moving Beyond Seven
The problem is, 7 people are not enough to accomplish many jobs. Take the job of putting a new software-intensive product on the market, for example. The product is almost never about the software – the product is a medical device or a car or a mobile phone or maybe it’s a financial application. Invariably software is a subsystem of a larger overall system, which means that invariably the software development team is a sub-team of a larger overall system team.

In the book Scaling Lean & Agile Development, Craig Larman and Bas Vodde make a strong case for feature teams – cross-functional teams that deliver end-to-end customer features. They recommend against component teams, groups formed around a single component or layer of the system. I agree with their advice, but it seems to me that software is invariably a component of whatever system we are building. We might be creating software to automate a process or software to control a product, software to deliver information or software to provide entertainment. But our customers don’t care about the software; they care about how the product or process works, how relevant the information is or how entertaining the game might be. And if software is a component of a system, then software teams are component teams. What we might want to consider is that real feature teams – teams chartered to achieve a business goal – will almost certainly include more than software development.

Agile development started out as a practice for small software teams, but these days we often see teams of 40 or 50 developers applying agile practices to a single business problem. In almost every case, we notice that the developers are organized into several small teams that work quite separately – and in almost every case, therefore, the biggest problem seems to be coordination across the small teams. There are many mechanisms: use a divisible system architecture so teams can be truly independent; draw from a common list of tasks, which makes teams highly interdependent; send small team representatives to weekly coordinating meetings; and so on. But rarely do we see the most powerful coordination mechanism of all for groups this size: create a sense of mutual obligation through peer commitments.

Mutual Obligation
You can call mutual obligation peer pressure if you like, but whatever name you use, when individuals on a large team make a commitment to people they know well, the commitment will almost certainly be honored. Mutual obligation is a much more powerful motivating force than being told to do something by an authority figure. And the interesting thing is, the power of mutual obligation is not confined to small teams. It works very well in teams of 50, and can be effective with teams up to 150. The time to split teams is not necessarily when they reach 10; team sizes up to 100 or 150 can be very effective – if you can create a sense of mutual obligation among the team members.

There are, of course, a few things that need to be in place before mutual commitment can happen. First of all, team members must know each other – well. So this won’t work if you constantly reform teams. In addition to knowing each other’s names, teammates must understand the capabilities of their colleagues on the team, have the capacity to make reliable commitments, and be able to trust that their teammates will meet their commitments. This process of creating mutual obligations actually works best if there is no manager brokering commitments, because then the commitments are made to the manager, not to teammates. Instead, a leader’s role is to lay out the overall objectives, clarify the constraints, and create the environment in which reliable commitments are exchanged.

For example, the project manager of the hardware/software product (above) laid out a series of increasingly sophisticated prototypes scheduled about three months apart. Having made a commitment to the team, sub-teams organized their work so as to have something appropriate ready at each prototype deadline. When an opportunity to dramatically improve the product through incorporation of a new chip, the whole team was in a position to rapidly re-think what needed to be done and commit to the new goal.

In the case of lean construction (above), a large team of contractor representatives works out the details of a “schedule” every few months. Each week, the same team gets together and re-thinks how that “schedule” will have to be adapted to fit current reality. At that same weekly meeting, team members commit to each other what they will actually accomplish in the next week, which gives their colleagues a week to plan work crews, material arrival, and so on for the following week.

It certainly is a good idea to have small sub-teams whose members work closely together on focused technical problems, coordinating their work with brief daily meetings to touch base and make sure they are on track to meet their commitments. But the manner in which these sub-teams arrive at those commitments is open for re-thinking. It may be better to leverage thousands of years of human evolution and create an environment whereby people know each other and make mutual commitments to meet the critical goals of the larger community. After all, that’s the way most things got accomplished before there was management.

_________________________________
Footnotes:
[1] Technically, Dunbar calculated the relative sizes of the neocortex – the outer surface of the brain responsible for conscious thinking. For a humorous parody of Dunbar's theory, see "What is the Monkeysphere?" by David Wong.

[2] Information in this paragraph is from: How Many Friends Does One Person Need? by Robin Dunbar.

[3] See John Sculley On Steve Jobs.

[4] This figure from “The Dunbar Number as a Limit to Group Sizes” is antidotal.

[5] From How Many Friends Does One Person Need? by Robin Dunbar. Interestingly, while Dunbar finds 15 an approximate limit of the second circle of intimacy, Allen finds a group of 15 problematic.

[6] The Dunbar Number was popularized by Malcolm Gladwell in Tipping Point. Much information and both quotes in this section are from Chapter 5 of that book. See http://nextreformation.com/wp-admin/general/tipping.htm for an extended excerpt.

Lessons from Planned Economies

Just as a market economy which relies on the collective actions of intelligent agents gives superior performance in a complex and changing economic environment, so too an agile project leadership approach which leverages the collective intelligence of a development team will give superior performance in a complex and changing business environment. However, conventional project management training focuses on using a plan as the program for action; it does not teach project leaders how to create a software development environment that fosters self-organization and learning. Since very few courses with such a focus are available today, this paper proposes a curriculum for agile software development leaders.

Planned Economies
In the middle of the 20th century, dozens of countries and millions of people believed that central planning was the best way to run their economies. Even today there are many people who can’t quite understand why market economies invariably outperform planned economies; it would seem that at least some of the planned economies should have flourished. After all, there are advantages to centralizing economic decisions: virtually full employment is possible; income can be distributed more equally; central coordination should be more efficient; directing resources into investment should spur growth. So why did planned economies fail?

There are two fundamental problems with planned economies: First, in a complex and changing economic system, it is impossible to plan for everything, so a lot of things fall between the cracks. For instance, planned economies usually suffer a shortage of spare parts, because no one plans for machines to break down. Secondary effects such as environmental impact are often ignored. Furthermore, planners do not have control of the purchase of goods, so they have to guess what consumers really want. Inaccurate forecasts are amplified by a long planning cycle, causing chronic shortages and surpluses.

The second problem with planned economies is diminished incentives for individuals. When compensation does not depend on contribution, there is little to gain from working hard. When incentives are tied to meeting targets, risk adverse managers focus on routine production to meet goals. The stronger that rewards are tied to meeting targets, the more disincentive there is for being creative or catching the things that fall between the cracks.

If we look at conventional software project management, we see practices similar to those used in planned economies, and we also see the similar results. Among projects over $3 million, less than 10% meet the conventional definition of success: on time, on budget, on scope. For projects over $6 million the number drops below 1% [1]. Interestingly, the underlying causes of failure of planned economies are the same things that cause failure in software projects, and further, the remedy is similar in both cases.

The difference between a planned and a market economy is rooted in two different management philosophies: management-as-planning/adhering and management-as-organizing/learning. Management-as-planning/adhering focuses on creating a plan that becomes a blueprint for action, then managing implementation by measuring adherence to the plan. Management-as-organizing/learning focuses on organizing work so that intelligent agents know what to do by looking at the work itself, and improve upon their implementation through a learning process.

The Planning/Adhering Model Of Project Management
Conventional wisdom holds that managing software projects is equivalent to meeting pre-planned cost, schedule and scope targets. The unquestioned dominance of cost, schedule and scope – often to the exclusion of less tangible factors such as usability or realization of purpose – draws heavily on the contract administration roots of project management. Therefore project management training and certification programs tend to focus on the management-as-planning/adherence philosophy. This philosophy has become entrenched because it seems to address two fears: a fear of scope-creep, and a fear that the cost of changes escalates significantly as development progresses.

However, management-as-planning/adherence leads to the same problems with software projects that planned economies suffered: in a complex and changing environment, it is virtually impossible for the plan to cover everything, and measuring adherence to the plan diminishes incentives for creativity and catching the things that fall between the cracks.

In the classic article ‘Managing by Whose Objectives?,’[4] Harry Levinson suggests that the biggest problem with management-by-objectives is that important intangibles which are not measurable fail to get addressed, because they are not in the action plan of any managers. Often these are secondary ‘hygiene’ factors, similar to environmental considerations in a planned economy.

In the book ‘Measuring and Managing Performance in Organizations’[5], Robert Austin makes the same point: over time, people will optimize what is measured and rewarded. Anything which is not part of the measurement plan will fade from importance. Austin points out that managers are often uncomfortable with the idea of not being able to measure everything, so they compensate through one of three techniques:

• Standardization. By creating standards for each step in a development process, it is hoped that all steps in the project can be planned and measured, and nothing will be missed.
• Specification. Specification involves constructing a detailed model of the product and/or process and planning every step in detail.
• Subdivision, functional decomposition. The Work Breakdown Structure (WBS) is the classic example of attempts to decompose a project into steps so that all steps can be planned. 

Conventional project management practices have emphasized all of these techniques to help a project manager be certain that everything is covered in the project plan. However, just as in a planned economy, these techniques are insufficient to catch everything in all but the simplest of projects. In fact, drilling down to detail early in the project has the opposite effect – it tends to create blind spots, not resolve them. By taking a depth-first rather than a breadth-first approach to planning, mistakes and omissions become more likely,[6] and these tend to be more costly because of early investment in details. Thus an early drill-down approach tends to amplify, not reduce, the cost of change.

A management-as-planning/adherence approach also tends to amplify, not reduce, scope-creep. In many software development projects, a majority of the features are seldom or never used.[1] Part of the reason for this is that asking clients at the beginning of a project what features they want, and then preventing them from changing their minds later, creates strong incentives to increase the number of features requested, just in case they are needed. While limiting scope usually provides the best opportunity for reducing software development costs, fixing scope early and controlling it rigidly tends to expand, not reduce scope.

Just as in planned economies, management-as-planning/adhering tends to have unintended consequences that produce precisely the undesirable results that the plans were supposed to prevent. The problem lies not in the planning, which is very useful, but in using the plan as a roadmap for action and measuring performance against the plan.

The Organizing/Learning Model Of Project Management
Market economies deal with the problems of planned economies by depending upon collaborating intelligent agents to make decisions within an economic framework. In market economies, it is the job of the government to organize the economic framework with such things as anti-trust laws and social safety nets. Economic activity is conducted by intelligent agents who learn from experience what is needed and how to fill the needs.

Of course, the economies of individual countries dwarf most software projects, so we might look further to find examples of management-as-organizing/learning. We will explore two domains: manufacturing and product development.

Throughout most of the 20th century, mass production in the US focused on getting things done through central planning and control, reflecting the strong influence of Frederick Taylor’s Scientific Management. The climax came when computer systems made it possible to plan the exact movement of materials and work throughout a plant. Material Requirements Planning (MRP) systems were widely expected to increase manufacturing efficiency in the 1980’s, but in fact, most MRP systems were a failure at detailed production planning. They failed for the same reasons that planned economies failed: the systems could not readily adapt to slight variations in demand or productivity. Thus they created unworkable schedules, which had to be ignored, causing the systems to become ever more unrealistic.

As the centralized MRP planning systems were failing, Just-in-Time systems appeared as a counterpoint to Scientific Management. Just-in-Time forsakes central planning in favor of collaborating teams (intelligent agents). The environment is organized in such a way that the work itself and the neighboring teams signal what needs to be done; rather than a central plan. When problems occur, the root cause is sought out and eliminated, creating an environment in which intelligent agents continually improve the overall system. In almost all manufacturing environments, implementing Just-in-Time trumps any attempt to plan detailed production activities using a MRP system. These systems succeed for the same reason a market economy succeeds: intelligent agents are better at filling in the gaps and adapting to variation than a centrally planned system.

An argument can be made that manufacturing analogies are not appropriate for software development, because manufacturing is repetitive, while projects deal with unique situations. Because of this uniqueness, the argument goes, management as planning/ adhering is the only way to maintain control a design and development environment. A look at product development practices shows that the opposite is true: creating a detailed plan and measuring adherence to that plan is actually a rather ineffective approach a complex product development project.

In the late 1980’s Detroit was shocked to discover that a typical Japanese automotive company could develop a new car in 2/3’s the time for half the cost as a typical US automaker.[7] The difference was that product development in Japan used a concurrent development process, which allows for learning cycles during the design process as well as on-going communication and negotiation among intelligent agents as design proceeds.

Just as market economies routinely outperform planned economies, concurrent development routinely outperforms sequential development. Replacing sequential (plan-up-front) engineering with concurrent (plan-as-you-go) engineering has been credited with reducing product development time by 30-70%, engineering changes by 65-90%, and time to market by 20-90%, while improving quality by 200-600%, and productivity by 20-110%.[8]

Based on experience from other domains, management-as-organizing/learning would appear to have a better chance of resulting successful software projects than the prevailing management-as-planning/adhering approach. An examination of the Agile Manifesto shows that agile software development approaches favor the management-as-organizing/learning philosophy. (See Table 1.) Therefore, we can expect that in the long run, agile software development might significantly outperform traditional software development practices. In fact, evidence is mounting that agile approaches can be very effective. [9]

Table 1. Mapping Values from the Agile Manifesto to Management Philosophies

A Curriculum For Agile Software Project Leadership
Existing training for project management appears to be largely focused on the management-as-planning/adhering philosophy. Courses seem to be aimed at obtaining certification in an approach to project management developed for other domains, such as facilities construction or military procurement. Even courses aimed specifically at software development tend to focus on work breakdown and a front-end-loaded approach to managing scope. As we have seen, this is not a good match for concurrent development or agile software development.

It would seem that at a curriculum should be available for leaders of agile software development projects, given the dismal track record of current approaches and the potential of agile software development. Project managers who know how to develop work breakdown structures and measure earned value often wonder what to do with an agile software development project. Senior managers often wonder how to improve the skills of project leaders. Courses on management-as-organizing/learning are needed to fill this void, but there seem to be few project management courses with this focus. To help make agile project leadership training more widely available, this article outlines a possible curriculum for such courses.

Change the Name: Project Leadership
Since this is new territory, we may as well start with a new name, and move away from the administrative context of the word management. All projects, agile or otherwise, benefit from good leaders; that is, people who set direction, align people, and create a motivating environment. By using the term leadership we distinguish this course from one which focuses on the management tasks of planning, budgeting, staffing, tracking, and controlling.

Setting Direction
Planning is a good thing; the ultimate success of any project depends upon the people who implement a project understanding what constitutes success. Planning becomes brittle with it decomposes the problem too fast and moves too quickly to solutions. The best approach to early planning is to move up one notch and take a broader view, rather than decompose the problem and commit to detail too early.[6] A project leader starts by understanding the purpose of the project and keeping that purpose in front of the development team at all times.

Organizing Through Iterations
The idea of management-as-organizing/learning is to structure the work so that developers can figure out what to do from the work itself. The fundamental tool for doing this is short iterations which develop working software delivering business value. Project leaders need to know how to organize iteration planning meetings and how to structure the development environment and workday so that people know what to do when they come in to work every day, without being told.

This part of the curriculum must cover such concepts as story cards, backlog lists, daily meetings, pair programming, and information radiators. It should also stress the importance of organizing worker-to-worker collaboration between those who understand what the system must do to provide value and those who understand what the code does.

Concurrent Development
Strategies for concurrent development are an important tool for project leaders, especially in organizations which are used to sequential development. General strategies include:

• sharing partially complete design information
• communicating design constraints instead of proposed solutions
• maintaining multiple options
• avoiding extra features
• developing a sense of how to absorb changes
• developing a sense of what is critically important in the domain
• developing a sense of when decisions must be made
• delaying decisions until they must be made
• developing a quick response capability

System Integrity
Project leaders must assure that the software developed under their guidance has integrity. This starts with assuring that the basic tools are in place for good software development: version control, a build process, automated testing, naming conventions, software standards, etc. Leaders must assure that the development team is guided by the true voice of the customer, so that the resulting system delivers value both initially and over time. They must assure that technical leadership establishes the basis of a sound architecture and an effective interface. They must make sure that comprehensive tests are used to provide immediate feedback, as well a framework so that refactoring can safely take place.

Leading Teams
People with experience in traditional project management who are about to lead agile software development projects might need some coaching in how to encourage a team make its own commitments, estimate its own work, and self-organize around iteration goals. Project leaders attend the daily meetings, listen to and solve team problems, serve as an intermediary between management and the team, secure necessary resources and technical expertise, resolve conflicts and keep everyone working together effectively; but they do not tell developers how to do their job. Project leaders coordinate the end-of iteration demonstration and the beginning of iteration planning, making sure that work is properly prioritized and all stakeholder interests are served.

Measurements
Feature lists or release plans, along with associated effort estimates, are often developed early in an agile project. The difference from traditional project management occurs when actual measurements begin to vary from these plans; for agile development, it is assumed that the plan is in error, and needs to be revised. By measuring the actual velocity or burndown rate, a far better picture of project health can be obtained than measuring variance from a guesstimate. Creating more accurate estimates becomes easier as developers gain experience in a domain and customers see working software. Leaders should learn how to combine reports of actual progress with increasingly accurate estimates into a tool for negotiating the scope of a project; this can be far more effective at limiting scope that the traditional method of fixing scope and controlling it with change approval mechanisms.

A useful technique for project leaders is to aggregate all measurements one level higher than normal.[5] This encourages collaboration and knowledge sharing because issues are called to the attention of a larger group of people. It also helps to avoid local optimization and the dangers of individual performance measurements. This technique is useful, for instance, for defect measurements.

Large Projects
Various techniques for synchronizing agile development across multiple teams are important for project managers to understand. Some of the techniques that might be covered are: divisible architectures, daily build and smoke test, spanning applications, loosely coupled teams that develop interfaces before modules.

Conclusion
We have often heard that the sequential, or waterfall, approach to software development projects is widely known to be ineffective, but it seems to be quite difficult to do things differently. We also heard that many, many features in most systems, possibly even a majority, are not needed or used; yet limiting scope to what is necessary seems to be an intractable problem. One solution to these problems can be found in concurrent engineering, which is widely used in product development as an alternative to sequential development. Concurrent engineering works more or less like a market economy: both depend on collaborating intelligent agents, both allow the end result emerge through communication, and both provide incentives for individuals to be creative and do everything necessary to achieve success.
_______________
References:
[1] Johnson, Jim, Chairman of The Standish Group, Keynote “ROI, It’s Your Job,” Third International Conference on Extreme Programming, Alghero, Italy, May, 26-29, 2002.

[2] Johnston, R B, and Brennan, M, “Planning or Organizing: the Implications of Theories of Activity for Management of Operations”, Omega: The International Journal of Management Science, Vol 24 no. 4 pp. 367-384, Elsevier Science, 1996.

[3] Koskela1, Lauri, “On New Footnotes to Shingo”, 9th International Group for
Lean Construction Conference, Singapore, August, 6-8 2001.

[4] Levinson, Harry, “Management by Whose Objectives?” Harvard Business Review, Vol 81, no 1, January, 2003, Reprint of 1970 article.

[5] Austin, Robert D., Measuring and Managing Performance in Organizations, 1996, Dorset Publishing House

[6] Thimbleby, Harold, “Delaying Commitment”, IEEE Software, vol 5, no 3, May, 1988

[7] Clark, Kim B, Fujimoto, Takahiro, Product Development Performance; Strategy, Organization, and Management in the World Auto Industry, Harvard Business School Press, Boston, 1991.

[8] Thomas Group Inc., National Institute of Standards & Technology Institute for Defense Analyses, from Business Week, April 30, 1990, pp 111

[9] Weber Morales, Alexandra, “Extreme Quality”, Software Development, Volume 11, No 2, February 2003.

[10] Kotter, John P, “What Leaders Really Do,” Harvard Business Review, Vol 79, no 11, December, 2001, Reprint of 1990 article.

Screen Beans Art, © A Bit Better Corporation

The Leadership Paradox

“No one has yet figured out how to manage people effectively into battle; they must be led,” wrote John Kotter in ‘What Leaders Really Do’[1]. He notes that leadership is about helping people cope with change, while management is about coping with complexity. Leaders set direction, managers plan and budget. Leaders align people, managers organize and staff. Leaders motivate, managers control.

Champion
In this context, it should not be a surprise that each new product development program at 3M is led by a ‘champion’. Innovation at 3M is brought about by excited, motivated teams. They are led by a product champion who probably wrote the initial product concept, gathered management support for the program, and recruited most of the team. The champion interprets the product vision for the team, thus representing the customer who is not, after all, on site. The champion sets the pace of development and determines how decisions get made. A champion is also expected to keep working on a good idea even if the program gets killed by management.

Shusa
Similarly at Toyota, a new vehicle development program is led by a shusa or chief engineer. In contrast to the coordinating role of a new vehicle manager at US auto companies, the shusa has complete responsibility for the vehicle, and has the authority necessary to make all program decisions. The shusa has been called a ‘heavyweight program manager’[2], but this is a misnomer, because a shusa is a leader, much more than a manager.

Perhaps the correct characterization of a champion or shusa is a ‘respected leader’. The emphasis of the role is on setting direction, aligning the organization and motivating the team. At 3M, the champion is largely self-nominating, and in both companies, the role holds great stature. In both 3M and Toyota, the product produced by one of these leaders often bears their name (Fuji-san’s car or Art Fry’s Post-it® notes). These companies seem to understand that if a team is to deal with change and innovation, a ‘respected leader’ is needed; a coach is not enough and a manager (in the traditional sense) is the wrong approach.

Software Development
It has been claimed[3] that software development is similar to new product development, because it is an activity which creates something unique for a customer, something which has not been made before. In this sense, programming is not like manufacturing, which makes the same thing many times over and strives to make each instance the same.

If software development is like new product development, then how should software projects be led? There seems to be some ambivalence surrounding the role of leadership in agile methodologies, possibly stemming from the traditional role of a project manager in software development projects. Often the project manager does not have a technical background and may not feel responsible for understanding the technical aspects of the project. Usually the project manager is trained for and measured against the controls of scope, budget and schedule. This administrative role is quite different than the leadership role exercised by a champion or shusa .

If a software project requires vision and must cope with change, this is not going to come from following the rules taught in project management certification courses. If stakeholders need to be aligned and the team needs motivation, managing scope, schedule and cost is not the place to focus attention.

Scrum Master
What are the alternatives to project managers? Beck suggests the use of a ‘coach’. Schwaber and Beedle propose role of a ‘Scurm Master’: ‘The Scrum Master represents management and the team to each other.’[4] ‘… is in touch with all aspects of the project…’[5] ‘… is responsible for ensuring that impediments are promptly removed and decisions are promptly made.’[6]

A coach or Scrum Master is not quite the champion of 3M or the shusa of Toyota, but then again, there is a difference between new product development and most software development. At 3M and Toyota, the customer is not readily available and must be represented. In fact, it’s pretty well accepted at 3M that asking existing customers what they want is not the way to come up with innovations. Instead, you have to understand the problems that customers have, anticipate future needs, and invent something new to solve their problems. A 3M champion creates and communicates a vision of the new product, just as at Toyota, the shusa writes the description of the new vehicle. To do this, they need a high degree of technical expertise as well as a deep understanding of the customers.

Most agile methods depend upon the ‘customer’ to create a vision of the software under development. Sometimes this will work, but it places a large and sometimes unrealistic burden on the customer. While the champion and shusa are technical experts, the customer usually is not, so the customer’s vision of a solution may be limited. In practice, customers often don’t know what they want, and to make matters worse, there are often multiple stakeholders, making it unclear who the ‘customer’ really is.

The Scrum Master represents the customer to the team, and thus must understand who the customer is and what all the stakeholders really want. However, the Scrum Master is not chartered to come up with the technical vision of how to address the customers’ needs. In XP, the technical vision of the system is expected to evolve throughout multiple iterations.

Design vs. Vision
The biggest criticism of agile methodologies is that they do not provide for overall design prior to the beginning of development. At Toyota and 3M, the product largely emerges from the development process, but the programs start with an overall technical vision of what the product will look like in the end. In fact, the shusa has been equated to a system architect of a new car.[7] Both the shusa and the champion are leaders, not managers, but as very experienced technical leaders, they provide the overall design to the product under development, or assure that an overall design is established by the team. They judge the level of design necessary to allow development to proceed in an emergent manner, while assuring that there are no downstream surprises which should have been anticipated.

If a development process must deal with ‘wicked problems’ (see separate essay), then by definition, the solution emerges as development proceeds. Such projects need someone to make the tradeoffs between early design and emergent development. In world class new product development organizations, this role belongs to the champion or shusa, a ‘respected leader’ with the technical and customer savvy to make such tradeoffs.

Technical Lead
Experience shows that if there is not someone to make similar tradeoffs for a software system, large organizations have a tendency to err on the side of caution. A corporate architecture committee, customer committee or administrative coordinators will usually lack an overall vision of the project, and so will look for comfort in detail, even though the detail is not what is needed to proceed safely. It would be better if there were a technical lead on the program to create and be responsible for maintaining the technical vision. It would not be so good, however, if the technical lead were not a leader. A leader not only sets direction, but also aligns and motivates people. The shusa and champion are expected to secure resources, remove impediments, and lead the team into battle. A technical lead who either sits on a pedestal or on the sidelines will not be able to lead the team.

If becoming a technical guru or a PMI graduate does not make a person into a leader, then how should leaders be developed? First, an organization should understand what it means to develop leaders. The armed forces develops leaders as their primary responsibility. A Special Forces team, composed of a dozen experts and a leader, is expected to adapt continually to very demanding conditions, given only the broadest of objectives. But even the most ordinary army units are composed of teams with a leader who is expected to respond to events on the ground, adapt to changing conditions, make decisions and lead the team. Although military training is not being advocated, it might be educational to understand how the military trains its leaders.

Developing Leaders
At 3M, people with technical competence are encouraged to develop a new product concept based on solving a customer problem. In order to carry out that vision, they find they need to recruit and motivate a team and secure resources from the organization. This is one of the most popular career paths in the technical organization, and so many technical people learn how to follow this path.

Toyota and 3M have learned how to make the role of technical leader one of the most desirable in the corporation, and thus they are able to develop a large number of qualified leaders. On the contrary, project management is frequently considered an undesirable career path for software developers, so few seem interested in learning how to manage. To break this vicious circle, an organization needs to begin by considering how to make the role of project leadership attractive to technical experts. Changing job expectations from administration and management to vision and leadership would be a good start.

Finding Leaders
Where does an organization find people to fill the newly defined role of project leader? In many cases the training ground might already be in house. In the past ten years, supervisors and managers in operational areas such as manufacturing, logistics and customer service have been trained to focus on empowering the first line workers to deal with problems and improve their own processes. Thus many organizations already have leadership development programs in place, they just haven’t moved beyond the operational areas yet.

Good operations managers are likely to make good software project leaders, because they have already learned how to work with people and lead a team. If you are blessed with good operations, then look for respected supervisors and first line managers in operations who have a technical background or aptitude, and consider training them as project managers. Alternatively, but not as good, consider sending everyone who will lead a software team to a good training program for first line supervisors in an operational area. But note that it is easier to develop people who know how to lead into managers than it is to develop people who know how to manage into leaders.

You Get What You Expect And What You Inspect
The bottom line is that people respond to the expectations of their management. Software project leaders do not develop in a position where the primary expectation is change control and the primary measurement is earned value. An organization will get what it values, and the agile methodologies do us a great service in shifting our perception of value from process to people, from documentation to code, from contracts to collaboration, from plans to action.
______________________
Footnotes:

[1] ‘What Leaders Really Do’, John P Kotter, Harvard Business Review, Volume 79, Number 11. Reprint of article first printed in 1990.

[2] ‘Toyota’s Principles of Set-based Concurrent Engineering’, Durward K. Sobek II, Allen C. Ward and Jeffrey K. Liker, Sloan Management Review, Volume 40 #2, Winter 1999, and Product Development Performance: Strategy, Organization, and Management in the World Auto Industry, Clark, K.B. and T. Fujimoto, Harvard Business School Press, 1991.

[3] See, for example, Agile Software Development with Scrum, Schwaber, Ken and Mike Beedle, Prentice Hall, 2001, Chapter 6.

[4] Agile Software Development with Scrum, Schwaber, Ken and Mike Beedle, Prentice Hall, 2001, p 31

[5] Agile Software Development with Scrum, Schwaber, Ken and Mike Beedle, Prentice Hall, 2001, p 142

[6] Agile Software Development with Scrum, Schwaber, Ken and Mike Beedle, Prentice Hall, 2001, p 32

[7] ‘Toyota’s Principles of Set-based Concurrent Engineering’, Durward K. Sobek II, Allen C. Ward and Jeffrey K. Liker, Sloan Management Review, Volume 40 #2, Winter 1999

Screen Beans Art, © A Bit Better Corporation

Zero Defects Mentality

A ‘zero defects mentality’ is a bad thing in the military. “Demanding such a rigid standard produces timid leaders afraid to make tough decisions in crisis, unwilling to take the risks necessary for success in military operations,” Perry said. “This zero defects mindset creates conditions that will lead inevitably to failure ….”[1]

A Military Tradition
William G. Pagonis, director of Logistics during the Gulf War of 1991, wrote about the time he led his small company into crossfire to rescue stranded soldiers, against the orders of his commander: “…following a time-honored tradition in the military, I developed ‘radio trouble’ – that is, I turned the communications gear off…” and led a volunteer team to the rescue.[2]

William McKnight, who created the 3M culture of innovation, once said “Those to whom we delegate authority and responsibility, if they are good people, are going to want to do their jobs in their own way. Mistakes will be made. But if a person is essentially right, the mistakes he or she makes are not as serious in the long run as the mistakes management will make if it undertakes to tell people exactly how they must do their jobs.” [3]

Good organizations understand that in a dynamic environment, the safest course is to develop intelligent, courageous people who understand that they are expected to exercise their own initiative. Most organizations would like to think that they empower people, but their behavior would suggest otherwise.

What Would You Do?
Ponder this: What is your organization’s instinctive reaction when things go wrong?

1. Reorganize.
2. Develop a better plan.
3. Send in a swat team to improve the processes.
4. Work with the front line people to find out what they think is wrong and how it can be fixed.

Collin Powell has said: “Organization doesn’t really accomplish anything. Plans don’t accomplish anything, either. Theories of management don’t much matter. Endeavors succeed or fail because of the people involved.”[4] So you can imagine what his choice would be.

The Paradox: Superior Performance Comes From Low Control
Organizations which tolerate mistakes and overlook disobedience build an organizational culture in which everyone knows that the best way to tackle the really tough problems is through the people who are closest to them. They also build a cadre of front line workers who are not afraid to think and act on their own. Such organizations are the envy of their industries, even as their competitors try to reorganize, plan and improve processes in an attempt to compete.
__________________
Footnotes:

[1] Defense Secretary William J. Perry, Quoted by Linda D. Kozaryn, American Forces Press Service, August 6, 1996.

[2] ‘Leadership in a Combat Zone’ by William G. Pagonis, Harvard Business Review, Volume 79 number 11, December 2001.

[3] William McKnight, President and CEO of 3M from 1929–1966, quoted in Brand of the Tartan by Virginia Huck, Minnesota Mining and Manufacturing, 1955.

[4] ‘Great Military Leaders’ http://www.geocities.com/Pentagon/Bunker/6513/