Monday, February 4, 2002

The Leadership Paradox

“No one has yet figured out how to manage people effectively into battle; they must be led,” wrote John Kotter in ‘What Leaders Really Do’[1]. He notes that leadership is about helping people cope with change, while management is about coping with complexity. Leaders set direction, managers plan and budget. Leaders align people, managers organize and staff. Leaders motivate, managers control.

In this context, it should not be a surprise that each new product development program at 3M is led by a ‘champion’. Innovation at 3M is brought about by excited, motivated teams. They are led by a product champion who probably wrote the initial product concept, gathered management support for the program, and recruited most of the team. The champion interprets the product vision for the team, thus representing the customer who is not, after all, on site. The champion sets the pace of development and determines how decisions get made. A champion is also expected to keep working on a good idea even if the program gets killed by management.

Similarly at Toyota, a new vehicle development program is led by a shusa or chief engineer. In contrast to the coordinating role of a new vehicle manager at US auto companies, the shusa has complete responsibility for the vehicle, and has the authority necessary to make all program decisions. The shusa has been called a ‘heavyweight program manager’[2], but this is a misnomer, because a shusa is a leader, much more than a manager.

Perhaps the correct characterization of a champion or shusa is a ‘respected leader’. The emphasis of the role is on setting direction, aligning the organization and motivating the team. At 3M, the champion is largely self-nominating, and in both companies, the role holds great stature. In both 3M and Toyota, the product produced by one of these leaders often bears their name (Fuji-san’s car or Art Fry’s Post-it® notes). These companies seem to understand that if a team is to deal with change and innovation, a ‘respected leader’ is needed; a coach is not enough and a manager (in the traditional sense) is the wrong approach.

Software Development
It has been claimed[3] that software development is similar to new product development, because it is an activity which creates something unique for a customer, something which has not been made before. In this sense, programming is not like manufacturing, which makes the same thing many times over and strives to make each instance the same.

If software development is like new product development, then how should software projects be led? There seems to be some ambivalence surrounding the role of leadership in agile methodologies, possibly stemming from the traditional role of a project manager in software development projects. Often the project manager does not have a technical background and may not feel responsible for understanding the technical aspects of the project. Usually the project manager is trained for and measured against the controls of scope, budget and schedule. This administrative role is quite different than the leadership role exercised by a champion or shusa .

If a software project requires vision and must cope with change, this is not going to come from following the rules taught in project management certification courses. If stakeholders need to be aligned and the team needs motivation, managing scope, schedule and cost is not the place to focus attention.

Scrum Master
What are the alternatives to project managers? Beck suggests the use of a ‘coach’. Schwaber and Beedle propose role of a ‘Scurm Master’: ‘The Scrum Master represents management and the team to each other.’[4] ‘… is in touch with all aspects of the project…’[5] ‘… is responsible for ensuring that impediments are promptly removed and decisions are promptly made.’[6]

A coach or Scrum Master is not quite the champion of 3M or the shusa of Toyota, but then again, there is a difference between new product development and most software development. At 3M and Toyota, the customer is not readily available and must be represented. In fact, it’s pretty well accepted at 3M that asking existing customers what they want is not the way to come up with innovations. Instead, you have to understand the problems that customers have, anticipate future needs, and invent something new to solve their problems. A 3M champion creates and communicates a vision of the new product, just as at Toyota, the shusa writes the description of the new vehicle. To do this, they need a high degree of technical expertise as well as a deep understanding of the customers.

Most agile methods depend upon the ‘customer’ to create a vision of the software under development. Sometimes this will work, but it places a large and sometimes unrealistic burden on the customer. While the champion and shusa are technical experts, the customer usually is not, so the customer’s vision of a solution may be limited. In practice, customers often don’t know what they want, and to make matters worse, there are often multiple stakeholders, making it unclear who the ‘customer’ really is.

The Scrum Master represents the customer to the team, and thus must understand who the customer is and what all the stakeholders really want. However, the Scrum Master is not chartered to come up with the technical vision of how to address the customers’ needs. In XP, the technical vision of the system is expected to evolve throughout multiple iterations.

Design vs. Vision
The biggest criticism of agile methodologies is that they do not provide for overall design prior to the beginning of development. At Toyota and 3M, the product largely emerges from the development process, but the programs start with an overall technical vision of what the product will look like in the end. In fact, the shusa has been equated to a system architect of a new car.[7] Both the shusa and the champion are leaders, not managers, but as very experienced technical leaders, they provide the overall design to the product under development, or assure that an overall design is established by the team. They judge the level of design necessary to allow development to proceed in an emergent manner, while assuring that there are no downstream surprises which should have been anticipated.

If a development process must deal with ‘wicked problems’ (see separate essay), then by definition, the solution emerges as development proceeds. Such projects need someone to make the tradeoffs between early design and emergent development. In world class new product development organizations, this role belongs to the champion or shusa, a ‘respected leader’ with the technical and customer savvy to make such tradeoffs.

Technical Lead
Experience shows that if there is not someone to make similar tradeoffs for a software system, large organizations have a tendency to err on the side of caution. A corporate architecture committee, customer committee or administrative coordinators will usually lack an overall vision of the project, and so will look for comfort in detail, even though the detail is not what is needed to proceed safely. It would be better if there were a technical lead on the program to create and be responsible for maintaining the technical vision. It would not be so good, however, if the technical lead were not a leader. A leader not only sets direction, but also aligns and motivates people. The shusa and champion are expected to secure resources, remove impediments, and lead the team into battle. A technical lead who either sits on a pedestal or on the sidelines will not be able to lead the team.

If becoming a technical guru or a PMI graduate does not make a person into a leader, then how should leaders be developed? First, an organization should understand what it means to develop leaders. The armed forces develops leaders as their primary responsibility. A Special Forces team, composed of a dozen experts and a leader, is expected to adapt continually to very demanding conditions, given only the broadest of objectives. But even the most ordinary army units are composed of teams with a leader who is expected to respond to events on the ground, adapt to changing conditions, make decisions and lead the team. Although military training is not being advocated, it might be educational to understand how the military trains its leaders.

Developing Leaders
At 3M, people with technical competence are encouraged to develop a new product concept based on solving a customer problem. In order to carry out that vision, they find they need to recruit and motivate a team and secure resources from the organization. This is one of the most popular career paths in the technical organization, and so many technical people learn how to follow this path.

Toyota and 3M have learned how to make the role of technical leader one of the most desirable in the corporation, and thus they are able to develop a large number of qualified leaders. On the contrary, project management is frequently considered an undesirable career path for software developers, so few seem interested in learning how to manage. To break this vicious circle, an organization needs to begin by considering how to make the role of project leadership attractive to technical experts. Changing job expectations from administration and management to vision and leadership would be a good start.

Finding Leaders
Where does an organization find people to fill the newly defined role of project leader? In many cases the training ground might already be in house. In the past ten years, supervisors and managers in operational areas such as manufacturing, logistics and customer service have been trained to focus on empowering the first line workers to deal with problems and improve their own processes. Thus many organizations already have leadership development programs in place, they just haven’t moved beyond the operational areas yet.

Good operations managers are likely to make good software project leaders, because they have already learned how to work with people and lead a team. If you are blessed with good operations, then look for respected supervisors and first line managers in operations who have a technical background or aptitude, and consider training them as project managers. Alternatively, but not as good, consider sending everyone who will lead a software team to a good training program for first line supervisors in an operational area. But note that it is easier to develop people who know how to lead into managers than it is to develop people who know how to manage into leaders.

You Get What You Expect And What You Inspect
The bottom line is that people respond to the expectations of their management. Software project leaders do not develop in a position where the primary expectation is change control and the primary measurement is earned value. An organization will get what it values, and the agile methodologies do us a great service in shifting our perception of value from process to people, from documentation to code, from contracts to collaboration, from plans to action.

[1] ‘What Leaders Really Do’, John P Kotter, Harvard Business Review, Volume 79, Number 11. Reprint of article first printed in 1990.

[2] ‘Toyota’s Principles of Set-based Concurrent Engineering’, Durward K. Sobek II, Allen C. Ward and Jeffrey K. Liker, Sloan Management Review, Volume 40 #2, Winter 1999, and Product Development Performance: Strategy, Organization, and Management in the World Auto Industry, Clark, K.B. and T. Fujimoto, Harvard Business School Press, 1991.

[3] See, for example, Agile Software Development with Scrum, Schwaber, Ken and Mike Beedle, Prentice Hall, 2001, Chapter 6.

[4] Agile Software Development with Scrum, Schwaber, Ken and Mike Beedle, Prentice Hall, 2001, p 31

[5] Agile Software Development with Scrum, Schwaber, Ken and Mike Beedle, Prentice Hall, 2001, p 142

[6] Agile Software Development with Scrum, Schwaber, Ken and Mike Beedle, Prentice Hall, 2001, p 32

[7] ‘Toyota’s Principles of Set-based Concurrent Engineering’, Durward K. Sobek II, Allen C. Ward and Jeffrey K. Liker, Sloan Management Review, Volume 40 #2, Winter 1999

Screen Beans Art, © A Bit Better Corporation