The Two Sides of Teams

Collective wisdom outweighs individual insights

Most of us believe that collective wisdom outweighs individual insights – or do we?

Perhaps the biggest shortcoming of agile development practices is the way in which teams decide what to do. What product should be built? What features are most important? What consumer experiences will work best? These are the most important questions for the success of any product, and yet for the longest time, answering these questions have not been considered the responsibility of the development team or the DevOps team.

Historically, someone with the role of business analyst, project manager, or product manager made the critical decisions about what to build. Or maybe some third party wrote a specification. While the technical team might question or push back on product decisions, too often the ideas and priorities were expected to come from outside. For example, the Scrum Product Owner role is often implemented in a way that favors individual insight over collective wisdom when it comes to critical product ideas and priorities.

Until recently, there hasn’t been a practical process for tapping into the collective wisdom of everyone on the development team when making key product decisions. But now there is: it’s called the “Design Sprint.”[1]  Combining a design thinking approach with the timeboxing of an agile sprint, this is a process that captures the collective wisdom of a diverse group of people. During the five-day process, the group not only makes critical product decisions, it creates prototypes and validates hypotheses with real customers as part of the process.

Design sprints were developed by Google Ventures to help the companies in its portfolio uncover a variety of product ideas and quickly sort out the good ideas out from the mediocre ones. Design sprints have been used at hundreds of companies with amazing success. While the Lean Startup approach starts by building a Minimum Viable Product (MVP) to test ideas, design sprints are a way to avoid building the MVP until you are sure you are starting with a good idea. They help you sort through a lot more ideas before starting to code.

Where do all those good ideas come from? Design sprints do not depend on individuals or roles to generate ideas; the ideas are generated and validated by a diverse team tackling a tough problem. The insights of engineering and operations and support are combined with those of product and business and marketing to create true collective wisdom.

There are a couple of roles in design sprint; one is a “decider.” The decider generally avoids making any decisions unless called upon by teams that do not have enough information to make the decision themselves, yet need to make a choice in order to proceed. In a small company, this might be the CEO; in a larger company it is more likely a product manager. But let’s be clear – the decider is a leader who articulates a vision and strategy, but she does not usually come up with ideas, set priorities or select features. That is what teams do.

Another recommended role is someone Google calls a “sprintmaster” – a facilitator who plans, leads, and follows up on a five-day design sprint. This person is almost always a designer, because the facilitator’s job is to help teams use design thinking and design techniques to answer key product questions. For example, on the second day of the sprint, everyone develops their own ideas through a series of individual sketches; on the third day, teams review the sketches jointly and create a storyboard for a prototype – or maybe a few prototypes. On the fourth day, the prototypes are created, usually with design tools. On the fifth day, the prototypes are tested with real consumers as the team observes. When most of the people on a team have no design experience, it helps to have a designer lead them through the design process.

Really good teams generate a lot of ideas. These ideas are quickly validated with real consumers and perhaps 10 or 20% of the ideas survive. This low survival rate is a good thing; investigating a lot of ideas dramatically increases the chances that one of them will be a winner. The trick is to have a very fast way for teams to generate, validate, and select the ideas that are worth pursuing – and the design sprint provides one good option.

Of course, success requires a lot more than a diverse team and a good process.

Deliberation Makes a Group Dumber

Most of us would be surprised by the idea that deliberation makes a group dumber. But that is the conclusion reached by respected authors Cass Sunstein and Reid Hastie in their sobering book Wiser: Getting Beyond Groupthink to Make Groups Smarter. The two set out to study the cognitive biases of teams, and found that groupthink plays a bigger role in group decision-making than most of us realize.

There is no advantage in diversity on a team if those who are in the minority – those who are different or soft-spoken or are working in their second language – do not feel comfortable about sharing their unique perspective. Yet Sunstein and Hastie note that in most groups, deliberation is very likely to suppress insights that diverge from the first ideas expressed (anchoring bias) or the majority viewpoint (conformity bias).

Brainstorming has come under criticism – for good reason – as a technique that favors talkative and confident team members over thoughtful members and those with undeveloped hunches. Brainwriting[2] is an alternative to brainstorming that gives individuals time to think individually about the problem at hand and come up with ideas based on their unique background. Brainwriting is used during on the second day of a design sprint, when individuals sketch their solution to the chosen problem. This gives everyone the time and space to develop their ideas, as well as a way to have these ideas anonymously presented to and discussed by the group.

After a brainwriting exercise, a group will have generated maybe 40% more ideas than brainstorming. Typically, a technique such as dot voting is used to prioritize the many ideas and select the best ones to pursue. Unfortunately, this is another technique that favors groupthink. Voting is likely to weed out hunches and fragile ideas before they have time to develop, so outlier ideas that come from those who think differently tend to be lost in a voting process.

The lean approach to product development is pretty much the opposite of voting. Instead of narrowing options early, the lean strategy is to pursue multiple ideas that span the design space, gradually eliminating the ones that do not work. In a lean world, teams would not prioritize and select the most popular ideas after brainwriting – selection at this stage would be premature. Instead, teams would identify several very different ideas to explore, making sure to include outliers.

It is important to ensure that the ideas which survive the selection process span a wide range of possibilities – otherwise much of the benefit of brainwriting is lost. One way to do this is to select ideas that have a champion eager to pursue the idea and one or two people interested in joining her. If small sub-teams are encouraged to explore the possibilities of outlier ideas, the group is more likely to benefit from its diversity. By giving those with minority opinions not only the opportunity to present their ideas but also the time and space to give their ideas a try, a much wider variety of ideas will be seriously considered.

Consider this example: Matthew Ogle joined Spotify’s New York office in early 2015. For years he had been working on the problem of helping people discover appealing music, most recently in his own startup. He joined a Spotify team developing a discovery page, but he thought the process involved too much work – he thought discovery should be automatic. This was a radical idea at Spotify – so luckily, Ogle’s team did not vote on whether it ought to be pursued, because it would probably have died.

Instead, Ogle joined Edward Newett, an engineer and expert at deep learning who was experimenting with the idea of a discovery playlist, to explore the possibility. When Ogle realized that algorithms could generate a playlist that was uncannily well matched to his tastes, he knew they were on to a good idea. The next step was to find a way to check out these magic playlists with more people.

They tried an unusual approach – they generated playlists matched to Spotify employees’ tastes and sent them out with an email asking for feedback. Almost everyone loved their playlist, and it became clear that this idea was a winner. Through a lot of quick experiments, the idea was improved, and soon playlists were delivered to a few customers under the name “Discover Weekly.” As it scaled up, Discover Weekly proved to be wildly popular and has become a dramatic success.

The Two Sides of Teams

There are two sides to teams. There is the side that needs to make its own decisions and the side that can turn decision-making into groupthink. There is the side that wants to leverage diversity and the side that tends to ignore the input from team members who are different. The point is this: if you believe in collective wisdom, be sure to collect all of the wisdom that is available. If you look closely and honestly at your current processes and team dynamics, you might be surprised at how much wisdom is locked in the minds of individuals who don’t feel comfortable participating in the give and take of a dynamic team.

____________________________
Footnotes:

[1] See: Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days  by Jake Knapp, John Zeratsky, and Braden Kowitz. For a quick “how to” summary, see: https://developers.google.com/design-sprint/downloads/DesignSprintMethods.pdf

[2] Brainstorming Doesn't Work; Try This Technique Instead

The Product Owner Problem

“We’re really struggling with the Product Owner concept, and many of our Scrum teams just don’t feel very productive.” they told us. “We’d like you to take a look at this and make some recommendations.” The company had several vertical markets, with a Scrum team of about ten people assigned to each market. Each market had a Product Manager, a traditional role found in most product companies. The company was clear about the role of a Product Manager; after all, there are university courses and professional organizations for this role. The Product Managers had a customer-facing job that included business responsibility for determining product direction and capability.

However, there was serious confusion about the Scrum role of Product Owner and its fit with the classic role of Product Manager. In addition to business responsibility, the Scrum Product Owner has the team-facing responsibility of managing the detailed product requirements.[1] In this company, the Product Managers found it impossible to handle both the customer-facing and team-facing jobs at the same time. So most teams had added an additional person to assist the Product Manager by preparing stories for the team, and called this person the Product Owner. The job of these Product Owners resembled the classic role of business analyst or, in some cases, user interaction designer.

Unfortunately, these Product Owners had little technical background in analysis or design, and yet they were expected to prepare detailed stories for the development team. Critical tradeoffs between business and technical issues often fell to these Scrum Product Owners, yet they had neither the first hand customer knowledge nor the in-depth technical knowledge to make such decisions wisely. They had become a choke point in the information flow between the Product Manager and the development team.

We asked the obvious question: How are things organized in the markets where things seem to be working well? It turns out that in the two highly successful vertical markets, there was no Product Owner preparing and prioritizing stories for the development team. Instead, the Product Manager had regular high level conversations with the development team about the general capabilities that would be desirable over the next two or three months. They discussed the feasibility of adding features and the results that could be expected. A real time application was created to show live web analytics of several key metrics that the Product Manager correlated to increased revenue. Then the team developed the capabilities most likely to drive the metrics in the right direction, observed the results, and modified their development plans accordingly.

This is a pattern we have seen frequently: Product Managers who lack the time, training, or temperament to handle both the customer-facing and the team-facing responsibilities of software development have two options. They can appoint Scrum Product Owners for each development team, or they can provide high-level guidance to a development team capable of designing the product and setting its own priorities. We observe that the second option generally works better, because an intermediary Product Owner brings a single perspective and limited time to the complex job of designing a product.

In 1988, Tom Gilb wrote the book Principles of Software Engineering Management, which is now in its 20th printing. One of the earliest advocates of evolutionary development, he has recently reiterated the elements of good software engineering in an article in Agile Record[2], from which I quote liberally:

Principle 1. Control projects by quantified critical-few, results.
1 Page total! (not stories, functions, features, use cases, objects, ..)

Principle 2. Make sure those results are business results, not technical.
Align your project with your financial sponsor’s interests!

Principle 3. Give developers freedom, to find out how to deliver those results.
The worst scenario I can imagine is when we allow real customers, users, and our own salespeople to dictate ‘functions and features’ to the developers, carefully disguised as ‘customer requirements’. Maybe conveyed by our Product Owners. If you go slightly below the surface, of these false ‘requirements’ (‘means’, not ‘ends’), you will immediately find that they are not really requirements. They are really bad amateur design, for the ‘real’ requirements – implied but not well defined.

Principle 4. Estimate the impacts of your designs, on your quantified goals.
….We have to design and architect with regard to many stakeholders, many quality and performance objectives, many constraints, many conflicting priorities. We have to do so in an ongoing evolutionary sea of changes with regard to all requirements, all stakeholders, all priorities, and all potential architectures…. a designer [should be able] to estimate the many impacts of a suggested design on our requirements.

Principle 5. Select designs with the best value impacts for their costs, do them first.
Assuming we find the assertion above, that we should estimate and measure the potential, and real, impacts of designs and architecture on our requirements, to be common sense. Then I would like to argue that our basic method of deciding ‘which designs to adopt’, should be based on which ones have the best value for money.

Principle 6. Decompose the workflow, into weekly (or 2% of budget) time boxes.
….I would argue that we need to do more than chunk by ‘product owner prioritized requirements’. We need to chunk the value flow itself – not just by story/function/use cases. This value chunking is similar to the previous principle of prioritizing the designs of best value/cost.

Principle 7. Change designs, based on quantified value and cost experience of implementation.

Principle 8. Change requirements, based in quantified value and cost experience, new inputs.

Principle 9. Involve the stakeholders, every week, in setting quantified value goals.
….In real projects, of moderate size, there are 20 to 40 interesting stakeholder roles worth considering…. But it can never be a simple matter of analyzing all stakeholders and their needs, and priorities of those needs up front. The fact of actual value delivery on a continuous basis will change needs and priorities. The external environment of stakeholders (politics, competitors, science, economics) will constantly change their priorities, and indeed even change the fact of who the stakeholders are. So we need to keep some kind of line open to the real world, on a continuous basis. We need to try to sense new prioritized requirements as they emerge, in front of earlier winners. It is not enough to think of requirements as simple functions and use cases. The most critical and pervasive requirements are overall system quality requirements, and it is the numeric levels of the ‘ilities’ that are critical to adjust, so they are in balance with all other considerations.

Principle 10. Involve the stakeholders, every week, in actually using value increments.
….I believe that should be the aim of each increment. Not ‘delivering working code to customers’. This means you need to recognize exactly which stakeholder type is projected to receive exactly which value improvement, and plan to have them, or a useful subset of them, on hand to get the increment, and evaluate the value delivered.

The Scrum Product Owner might be a role, but it should not be a job title. Product Owners wear many hats: Product Manager, Systems Engineer, User Interaction Designer, Software Architect, Business Analyst, Quality Assurance Expert, even Technical Writer. We would do well to use these well-known job titles, rather than invent a new, ambiguous title that tends to create a choke point and often removes from the development team its most important role – that of product design.

Discovery of the right thing to build is the most important step in creating a good product. Get that wrong and you have achieved 100% waste. Delegating decisions about what to build to a single Product Owner is outsourcing the most important work of the development team to a person who is unlikely to have the skills or knowledge to make really good decisions. The flaw in many Product Owner implementations is the idea that the Product Owner prepares detailed stories for the team to implement. This does not allow team members to be partners and collaborators in designing the product.

The entire team needs to be part of the design decision process. Team members should have the level of knowledge of the problems and opportunities being addressed necessary for them to contribute their unique perspective to the product design. Only when decisions cannot be made by the development team would they be resolved by a product leader. The main team-facing responsibility of the product leader is to ensure the people doing the detailed design have a clear understanding of the overall product direction.

The concept of single focus of accountability is at the center of this issue. Too often, accountability is implemented as a prioritized list of product details (stories) rather than as communication of intended results (business relevant metrics). As a result, the expertise, creative input, and passion of team members is sacrificed to the false goal of a single point of responsibility.
___________________
Footnotes:
[1] “The Product Owner is responsible for the Product Backlog, its contents, its availability, and its prioritization.” “The Product Backlog represents everything necessary to develop and launch a successful product. It is a list of all features, functions, technologies, enhancements, and bug fixes that constitute the changes that will be made to the product for future releases.” From Scrum: Developed and sustained by Ken Schwaber and Jeff Sutherland; http://www.scrum.org/scrumguides/.

[2] “Value-Driven Development Principles and Values – Agility is the Tool, not the Master.” Agile Record, Issue 3, July 2010, pp 18-25. Available at www.agilerecord.com. Used with permission.

Screen Beans Art, © A Bit Better Corporation