The Messy Middle Ground of Agile Transformation

Many people think of transformation as a linear process. Whether it is a digital or agile transformation, or even a personal transformation, we like to think of it as a straight line from where we are to our goal.

As a coach, and a “change agent”, I’ve been overly enthusiastic about the agile transformation process. Three years ago I published a whitepaper, “How to Transition from Waterfall to Scrum” where I used the image below on the cover.

While moving from waterfall to agile does require a leap of faith, the leap looks like a straight line. And it also looks like going from waterfall to agile is pretty quick. These are not characteristics of agile transformations.

That whitepaper with the agile leap has been downloaded over 500 times. While I hope that at least some of the agile transformation steps I wrote in that paper were helpful, I don’t think the cover image was very accurate.

That image and my description don’t accurately portray the difficulties that organizations experience in trying to adopt agile. That process is not quick and it nothing like a straight line – leaving one thing and arriving at another.

Now with some hindsight, I have come up with a better depiction of how many leaders feel about moving from waterfall to agile ways of working.

This is what I call the messy middle ground of agile transformation. It is where organizations are striving to achieve business agility but they are finding it difficult to progress.

They are stuck.

Stuck somewhere in the middle.

It’s not a leap.

They can’t go back. Can they?

Do they have to move forward? But what does it take?

Where Organizations Get Stuck in Agile Transformation

So where do organizations get stuck with their agile transformation? There are many areas but I think they can be lumped into the six categories shown in the diagram and explained in the sections that follow.

People and Teams

It is interesting to me that one of the key difficulties most organizations have is moving from planning by individuals to planning by teams.

This is partly based on the project paradigm where new work is viewed as a project which means unique and temporary. We then assign people to deliver that project knowing full well that those same people are also assigned to several other projects. And then we wonder why things take longer, or so many projects are started but not finished.

It is pretty typical when you treat work this way that you have a project team that is made up of fractional people. You might have one or two people who are full time, but everyone else is assigned to other projects.

The main driver for starting too many projects and sharing people across initiatives is scarcity thinking.

• Managers cannot say no to new requests
• People believe that starting more projects means more will be finished

These are both false beliefs. In fact, starting more projects at the same time with the same set of resources guarantees that fewer projects will be completed due to context switching and overhead costs.

The solution is simple, but difficult to do. Organizations that are serious about moving to agile establish stable and dedicated teams.

Benefits of dedicated and stable teams:

• More productive since they aren’t continually going through forming, storming, norming
• Stable teams make it easier to co-locate
• Stable teams are able to provide a deeper focus in the business area
• More predictability
• More productivity and less context-switching

Despite the clear advantages of long-term stable teams, many organizations insist on it. Here is why:

• That previously identified belief that more project started = more projects completed.
• Desire to delude themselves into thinking that simply assigning people to multiple initiatives will magically result in more productivity
• Desire to control people like chess pieces and continually redirect them.

That last reason is perhaps the most accurate and insidious. After all, if I am a manager and my people are all assigned to dedicated teams, I don’t have a need to assign them here and there. Isn’t that my job? Will I eliminate my job if I have dedicated teams? That is all about power, and that is the next topic.

Power and Decision-Making

True or false, the managers job is to make decisions?

No matter where you come down on that argument, the reality is that in most organizations today, decisions flow down from the top. Power is concentrated at the top and decision making authority is rarely delegated down to those who are closest to the work and have the most information.

In most organizations, managers are so accustomed to making decisions that they cannot imagine letting team members do it. I had a situation once where new scrum teams were being formed and the manager wanted to pick the team names for the Scrum teams. Why? This is typically something we let the teams decide as it is a good first step in self-determination. This manager wanted to constrain the choices to the names of the Chicago professional sports teams, even though two of the teams were based in Chennai India!

Another example of self-determination is letting team members decide what team they want to work on. Literally, the people are provided with the options for the various teams and given a choice of what they want to work on. I’ve had the opportunity to facilitate exercises like this at different clients. You can read about an exercise that my colleague Ahmad Fahmy conducted at Bank of America.

Take a moment to think about who makes the decisions in your organization. The graphic below lists some really basic examples of decisions that could be delegated to team members. Who makes those decisions in your organization? And why?

Human Resource Policies

As you move from waterfall thinking and ways of working to agile, one thing that should immediately come to mind is the human resource policies. How we recruit, reward and retain people needs to change.

A simple example of this is career paths. If you introduce Scrum, you are going to need Scrum Masters. What is the career path for a Scrum Master? Who should the Scrum Master report to in the organization? Who should report to the Scrum Master?

Most organizations have an HR role for the project manager and they just assume project managers and scrum masters are the same, or close enough. They are not.

Similarly, if you are using Scrum you will need Product Owners. Will you simply assign this to an already busy person who has another full-time job? Would that make sense?

In many cases, moving to agile and Scrum results in more self-organizing teams and a flattening of the organization. There will be fewer “management positions”. Some people who are in management roles will move to being individual contributors.

What happens when the employees think that they only way to advance is to become a manager? And you don’t have many management positions. Have you eliminated opportunities to advance? Or, do you need to provide career paths that allow for growth and development without becoming a manager?

The diagram below shows some of the key HR policy implications that need to be talked through.

One area that falls somewhere between people and teams and HR policies is staffing. Some organizations treat people like widgets. If they need a Java developer, they find the lowest cost person. Even if that person is in Bangalore, Belarus or Brazil.

I’ve seen it happen again and again. Usually, it is a surprise to the team. “Hey, we are adding 2 testers from Bangalore…they will be joining your team meeting this morning.” What impact will that have on:

• Team morale
• Team productivity
• Impromptu discussions or meeting
• Amount of documentation that the team creates
• etc

People are not widgets.

Tools

Another key area to consider when moving to agile is the tooling that teams will need. There are obvious tools like test automation or developer tools. And there are the less obvious tools that teams will need to support collaboration. The diagram below shows some of the key tools to be considered.

Compliance & Reporting

Another area where organizations get messy in their agile transformation is compliance and reporting. Unless they are a startup, organizations have built up systems and processes based on centuries old thinking about control and risk avoidance. Those existing procedures and policies create a lot of friction when trying to move to agile ways of thinking and working.

One that I see commonly is team status reporting. In a traditional world, project managers create weekly status reports that get rolled up and presented to management. Often at the senior management level they become stoplight or RAG reports that are ignored unless they are red.

When you move to agile and your team delivers on a 2-week sprint, does weekly status reporting make sense? No. And who compiles the status report? The team? (No) The Scrum Master? (God no!) The Product Owner? (Maybe)

Some of the key implications that agile will have on compliance and reporting are shown in the diagram below.

At a recent client using Scrum, a new senior manager came in and stated that he couldn’t tell what was going on. Teams were using two week sprints and held regular sprint reviews. But he couldn’t see the work from his office. So this manager asked for MORE STATUS REPORTS. Yep that’s right, like the proverbial tree falling in the woods, it didn’t happen if it wasn’t on a status report somewhere.

This manager could simply begin to attend Sprint Review meetings and go out and observe or talk to his teams. Instead, he wants the Scrum Masters to develop more detailed reports. Yikes!

That managers behavior is exactly the opposite of what one would hope would happen. The following quote from Taiichi Ohno comes to mind.

Budgeting

Budgeting is another point where many organizations remain stuck and their agile transformation gets messy. Most have long established processes for annual funding which results in long cycles of budget preparation, budget gaming, approvals, and then spending. The timing of these bureaucratic activities frequently gets in the way of normal work.

An extreme example of this is in one organization on an annual budget. They had a project approved mid-year, however, the money had to be spent before the end of the year. It was a 9-month project and the team was given 6 months to hire a team and complete the work before they lost the budget. Though everyone knew it was physically impossible, the manager of the area knew they had to try so that they wouldn’t “lose” the money at the end of the year. We’ve all seen this movie before and can predict how it will end.

The solution is to move away from annual budgeting and move toward a model of stable, continual funding. It’s not trivial but it is doable. The Beyond Budgeting book and website provide a framework and examples of how others have accomplished this challenging task.

Organizations Are Stuck in the Messy Middle of Agile Transformation

Now that we have looked at some of the pain points around being stuck in the messy middle, the question is, why don’t organizations recognize the problem and make the shift? The reality is that organizations are actually optimized to resist change.

Organizations Are Optimized to Resist Change

Every year Collabnet/VersionOne publishes the annual state of agile report with various statistics about agility. One question that has been consistent for years is the challenges experienced adopting and scaling agile.

The top 3 reasons continue to be related to organizational resistance to change.

Organizational resistance to change is strong and patient. It may go underground but it won’t go entirely away.

SO WHAT CAN YOU DO TO HELP YOUR AGILE TRANSFORMATION?

If you are stuck in a messy agile transformation, hope is not lost. Perhaps you have to create a way of bridging the gap from tradition to agile.

Whether you are a manager, coach, team member or senior leader, you will have to commit yourself to a tough challenge that will take a long time. It will be more like a marathon. And you may need to put yourself at personal risk. It is one thing to propose new ideas. It is another to be willing to charge up the hill fighting for those new ideas.

Perhaps you have to make a personal sacrifice, like the manager shown below, in order to create a path for others to follow.

Action Steps for Change Agents in Agile Transformation

For those of you who are leading change through the messy middle, here are some specific steps that you can take.

#1 – Get Clear on Why and Communicate It

Agile Transformation is a program of change. Organizational change expert John Kotter says a sense of urgency is essential to a change. That sense of urgency starts with a compelling why. Effective change agents will make sure everyone understands WHY change is needed.

I’ve heard some real bullshit when it comes to agile transformation. A CIO recently stated that he had a goal that ‘75% of our projects will be agile by the end of this year’. That may be a good goal, but that is definitely not a compelling why. Here are some common reasons why organizations pursue agile and scrum:

• Competitive Threats and Organizational Survival
• Delighted Customers
• Happier Workplace
• Reduced Cycle Time
• Respond to Change
• Increased Innovation

Whatever your reasons, they should be clearly articulated and well understood. The leaders of the transformation need to say them again and again.

#2 – Get Outside Help

Without question organizational change must be led by those internal to the company. You cannot outsource that to McKinsey or Boston Consulting. But it is really difficult for leaders to make changes without help from others who are more experienced in agile transformation.

Einstein said you cannot solve today’s problems with today’s technology. Similarly, it will be difficult for leaders to make changes without outside help.

#3 – Leaders Need to Learn & Lead by Example

Leaders cannot simply dictate that the organization change. They must learn what that change means. They need to demonstrate leadership by example.

This can be taking the training with the teams, or better yet, ahead of the teams. I know of one senior technology leader who took the Scrum Master certification training three times with different teams. He walked away with a deep understanding of lean principles, empiricism, and Scrum.

Leaders and change agents need to walk the talk. They need to visibly lead by example.

#4 – Try Skunkworks

In some organization, the bureaucracy is so stifling that it will strangle any attempts to be agile. In these cases, it may make sense to split off a small group that is not encumbered by the organization.

The idea of a skunkworks project was to foster innovation by isolating a team of people and give them a challenge or stretch goal that was nearly impossible. Lockheed is credited with creating the Skunkworks approach to build the first U2 spy plane.

You needn’t call it skunkworks. Perhaps it is setting up a small agile pilot team that is in a separate building. Maybe you need to seed the team with high performers, rebels, or outsiders who will eschew bureaucracy and reporting structures and focus on creating great solutions.

From that skunkworks experiment, you can evaluate what you need to change in the organization to foster more agility, or how you can stand up additional teams outside the traditional controls.

#5 – Create a Visible Transformation Backlog

When attempting an Agile transformation, some organizations have found it helpful to have a large visible backlog of the impediments, challenges and activities that the leaders are working on to drive change and increase agility.

Just like we would expect with an agile team, the leadership team should have transparency on their efforts. The leaders should be visibly tackling these impediments and obstacles. I highly recommend a large visible Kanban board near the transformation leader’s desk.

#6 – Move with Speed and Decisiveness

I was visiting a client this week who is in the early stages of their agile transformation. My last meeting with the leadership team was six months ago when I provided training on the leadership role in agile transformation. This team spent 90 minutes telling me about all the initiatives they had completed since the training. It was amazing!

What I loved was the boldness they demonstrated. They knew that they would make some mistakes but they went through a rapid organizational change that flattened the organization from 6-7 layers of management to just 3. They aligned their organization to their key customer areas. They introduced cross-training and T-shaped skill development. I was so excited for them.

There is an old saying, Time Kills all Deals. It applies equally to sales as it does to transformation.

I think the following quote from Mark Zuckerberg of Facebook is a better way to think about agile transformation. It is not the time to be timid or risk averse.

“Move fast and break things. Unless you are breaking stuff, you are not moving fast enough.”

– Mark Zuckerberg

Agile Transformation is Not a Straight Line

Despite * ahem * any images portraying agile transformation as a quick leap, it is anything but a straight line. And it is anything but fast.

Don’t focus on agile for agile’s sake. Rather, focus on those goals or outcomes you are trying to achieve. Agile may be the vehicle that helps you get there.

I hope that you have found this helpful. I welcome your comments.

This article originally appeared on Viality Chicago’s Blog and has been republished with permission.

Business & Finance Articles on Business 2 Community

(32)