Why Are There So Many Misconceptions Around Agile?

—Andy Hunt

11/20/2020
Interviewed by Malak Bougader

Malak Bougader is a masters student completing a thesis regarding agile software development. Malak encountered a lot of different definitions, expectations and ideas around the concept of agility, and asked Andy Hunt the following interview questions to help clarify.

I think we all agree that what agile is today has nothing to do with what it was conceived to be. Why are there so many misconceptions around agile? Do you think it is because of the way the manifesto was written, or because there was not enough concrete guidance in how to apply agile?

Well that’s the fundamental rub right there: the conflict between concrete rules and abstract principles. The agile manifesto establishes abstract principles for skilled practitioners in a healthy environment. But novices can’t apply such abstract principles directly, they have to start with unambiguous, context-free, concrete rules. So it’s perfectly natural to begin with concrete, hard-and-fast rules such as a two-week iteration or a stand-up meeting with a fixed format.

But while rules are effective to help beginners get started, they also then limit you to beginner levels of performance (see the Dreyfus research on airline pilots).

Agile is not a set of practices or rules. Anyone who tells you any different is mistaken. Agile is a mindset and approach used for working effectively in complex domains. For a good explanation of complex, complicated, chaotic and simple domains, see Dave Snowden’s Cynefin model.

As such, the agile manifesto was really aimed at skilled practitioners in a healthy environment. But most people work in unhealthy work environments with low psychological safety (see Amy Edmonson’s work), and degraded information flows (see the Westrum Continuum). Additionally, many team members have a desire to “just be told what to do”—in other words, concrete rules.

But agile is all about being able to adapt quickly in volatile and uncertain environments, so there will never be concrete rules to tell you what to do. There can’t be. You have to to gain enough expertise to be able to apply the underlying principles in your own unique, context-sensitive situation.

It seems people (whether clients or developers or consultants) have gone from suddenly being completely against agile to having absolutely irrational expectations when on an agile project. Why do you think that is? And how to handle it as an agile consultant?

I think that’s true of any tech or process, from Java to Rails, from to Elixir to Rust. We keep expecting the next shiny new thing to be a silver bullet. But there are no silver bullets; there is no “one size fits all approach”. Not even agile—it’s not suited to Clear and Complicated Cynefin domains. It would be a mistake to use an agile approach outside of the domain in which it is effective.

A complex adaptive system such as a software project and team has to be addressed via exploration and feedback. It takes hard work, skill, and experience. There are no shortcuts. But we humans are riddled with cognitive biases such that we still expect a shortcut, a silver bullet, even when we know better.

As a consultant, leader, or team member, the key is to take very small, incremental steps. That reduces risk—and threat, which helps temper both anti-agile attitudes and irrational expectations.

As an agile expert, what is agile to you very simply and concretely?

My definition from 2008’s Practices of an Agile Developer:

Agile Development uses feedback to make constant adjustments in a highly collaborative environment.

Most problems you see on any project stem from lack of feedback or lack of collaboration. Lack of feedback can come from many poor practices, such as not integrating to main with trunk-based development, or having long-running code branches. Poor or missing unit testing or lengthy release cycles also deprives a team of fast feedback.

Lack of collaboration is often seen when you have little to no stakeholder or user involvement, or artificially separated “stovepipe” teams that duplicate or ignore other team’s work, and so on.

Even if you are getting feedback, you have to act on it. That’s the “constant adjustment” part. If you are getting indications that something is wrong, either from the users or from the unit tests, you need to address it. If you ignore it, you may as well be doing a naive, plan-driven, “waterfall” style approach.

In a complex environment, such a plan-driven approach is virtually guaranteed to fail.

According to you, what do we need to do in order to restore the true definition of agile the way it was meant when the manifesto was written?

For starters, perhaps consider moving away from Scrum. Scrum is a lightweight project management method. It does not include the technical practices that a team requires, and it’s concrete practices are akin to “training wheels.” if you’re coming from an even less flexible approach, it’s a great start. But as with a bicycle, at some point the training wheels need to come off.

Don’t even think about using some abomination such as SAFe. It’s neither “agile” nor “safe.” The approach behind SAFe is not appropriate for projects in the Complex domain.

Instead, focus on rock solid delivery practices with continuous integration and continuous potential deployment in order to get fast feedback and make quick changes while working closely with users and stakeholders.

That’s all it takes to keep software soft. And win.

According to the Westrum Continuum model, most companies are still in either a “Pathological” or “Bureaucratic” organization and mindset. But they try nonetheless to forcibly practice agility in their projects or day-to-day activities, which might explain the fact that the whole concept of agile in these work environment is distorted. Is that correct?

Yes.

And not only just distorted; agile techniques simply will not work in those environments. Again, the agile manifesto is aimed at skilled practitioners in a healthy environment. In a Pathological organization information is not shared (and can be jealously hoarded), so developers can’t get the information or the feedback they need. You won’t get the cross-pollination and growth of ideas which is absolutely critical to software success.

Bureaucratic organizations are stifled by turf wars and hide behind layers of “this is how we do things around here,” which clearly is the wrong approach in a complex, rapidly changing environment. Again, a recipe for certain failure.

So these organizations take the vocabulary of agile, and perhaps the flavor of some of the practices and techniques, but twist it to fit their own dysfunctional context. The results, unsurprisingly, are disappointing.

What does it take to become an agile coach/expert?

You need a solid set of learnings from the pioneers in the field, combined with lots of practice and experience. Sadly, many “agile coaches” I’ve run into have never even read the agile manifesto, nor any of the seminal works in this area. They literally don’t know what they’re talking about. In a more mature field, they’d be run off as quacks.

If agility is about collaboration and feedback, then those are the areas where you need expertise. How can you set up experiments to get better and faster feedback? How can you improve collaboration and working together across disparate communities and skill levels of users and stakeholders? And how can you teach others to do that?

That’s what it takes.

Which takes me to your answer about Scrum and SAFe. I was quite surprised by your statement as Scrum and SAFe are the most widely known “agile methods” out there, it’s the first thing they teach you at school, or the first thing they check on your resume etc… but I completely agree that restricting yourself to the disciplined application of a set of rules or a method is the against agility by definition of the term itself.

Exactly. I hear people refer to “canonical agile.” It’s an immediate tell that they’ve missed the entire point.

Which actually inspires me a question (just for my mere curiosity) how was the term agility chosen when you were working on the manifesto??

When we first got together, we called it the “light weight processes” conference. But we collectively felt that “light weight” had a connotation of “not serious,” or “easily defeated by a heavyweight.” My favorite other choices were “responsive” or “adaptive,” which I think are closer to the original spirit.

But the group settled on “agile,” to try and emphasize the idea that we can rapidly respond to change. But not even just respond to change, to actively embrace change: to harness change for competitive advantage.

I want to go back to your definition “Agile Development uses feedback to make constant adjustments in a highly collaborative environment.” If I rely on this and the rest of our conversation, do you think we might say that agility is a state of mind and a state of being that needs to be incorporated to the company/project/team DNA?

Yes. And that’s a really good point, in that an agile approach requires more than just the development team. For example, if your organization requires annual budgets, then your ability to be agile can be crippled (see the Beyond Budgeting movement.)

You don’t necessarily need the entire company to be agile—that’s falling into the “one size fits all” trap. Different parts of the organization may need to follow a different model. For example, a manufacturing department (which is not in the Complex domain) would be better served with Six Sigma and the like. But you have to be careful with seemingly unrelated departments such as accounting or HR that actually have a big impact on development teams.

We’ve been working on these ideas at the growsinstitute.com, trying to generate detailed, concrete practices according to a pattern language to help get novices started on the right foot, but with the full knowledge that concrete practices are only a starting point.

So in order to achieve agility we need to stop trying to force ourselves and our teams to “be agile” (by forcing them to apply rules such in Scrum, for example) and can we simply say that agility by definition is at the end of the day just finding that resonance point within your team or company that will allow you to achieve high performance by communicating, giving and taking feedback, collaborating with stakeholders etc… which eventually just comes back to intuitively incarnating the four values of the agile manifesto and reaching the level of awareness in the Dreyfus model?

Yes. That’s the secret. That’s actually all there is to it.

/\ndy

Keep up to date with my low-volume newsletter and don't miss another article or fresh idea:

Your subscription could not be saved. Please try again.

Your subscription has been successful.