Embrace agility, not fragility
Summary
The agile movement was about freeing developers from the baggage of Dilbertesque corporations. But in the 2020s, “doing agile” often comes at the cost of agility. Teams and companies sacrifice common-sense at the altar of a hustle culture, that looks agile, but is far from the spirit of the movement. To run effective distributed software teams, we must embrace scoping, planning, documentation and role and process clarity. This may, on the face of it, feel like they violate some agile principles. But when you dig deeper, you’ll notice that they lend agility to the way you deliver software.
Agility is a broad and often misused term. After 20-odd years of the agile movement, I feel like many of us use the term as shorthand for including agile practices in the way we work. Whether it helps our teams and companies be resilient and responsive to change and adapt quickly, remains an open question. What’s worse, I see common sense becoming a casualty of a narrow view of agile.
Let me give you a few examples.
We read that agile values customer collaboration over contract negotiation. So that becomes an excuse for teams to aim at moving targets. Why bother with an initial backlog if it’s going to change, anyway? Definition of done? That’s a contract, isn’t it? To hell with that. Plans? What plans? Watch us respond to change, baby!
Working software matters more than comprehensive documentation, doesn’t it? Do we need to document your process then? Naah, we’ll do what it takes - processes be damned. What about team handbooks, decision records, onboarding docs, and meeting minutes? Meh, you’ve got to be some foggy old documentation pedant to worry about that nonsense.
And look, individuals and interactions matter more than processes and tools. So don’t worry about roles and responsibilities. We’ll take “collective responsibility” for everything. We’ll scoff at the idea of modern tools, because hey - tools don’t matter. Watch us scribble on whiteboards, and place flip-charts and sticky notes on our walls while we also power through our manual workflows. No sweat.
Is it just me or is this way too common to be funny anymore? When did agile become an excuse to live life in a giant Dilbert comic? We must shake off this dysfunction.
Regular readers of this blog may see this as preaching to the choir. That said, it’s the end of the year. I guess it’s as good a time as ever to reflect on some common dysfunctions in the name of agile, and the common sense practices to address them.
If you have a contract, don’t live in denial
I work in the IT services industry. Every project I work on, runs off a “negotiated contract”, regardless of whether I like it. Often, these contracts are fixed-price as well. And even if they aren’t fixed-price in the way they read, every stakeholder has a fixed budget. It’s just that we may not know what that fixed budget is. More likely, every stakeholder wants to deliver some value to their customers by a certain time, at a certain cost. And they make an implicit contract with their dev team to achieve this goal or at least get reasonably close.
Scoping allows all stakeholders to arrive at documented, shared understanding
Regardless of how much you believe you can negotiate the stakeholder’s notion of “must-have” value, you can begin that negotiation only when you document what that initial assumption is. This is as true for operational work, internal gigs and minor projects as it is for multi-million dollar partnerships.
Before you start any piece of work, however small, a scoping exercise is invaluable. At the risk of patronising experienced practitioners, I must clarify that scoping is not “big, upfront design”. It’s a lightweight exercise that helps you understand the boundaries of the project, what the business seeks to achieve, how complex certain parts of the work might be, and where the pitfalls may lie. The biggest benefit of scoping your work well is that between your business stakeholders and your team, you’ll have a shared understanding of the challenge that lies in front of you. You can be transparent about negotiables and non-negotiables. The team can be honest about contingency plans. Nobody’s beholden to the plan, but the plan represents a starting point from which you can pivot whenever necessary.
Document with haste or repent at leisure
On a distributed team, nothing creates clarity faster than clear and concise writing. The practice of writing underpins almost every practice I evangelise on this blog. On the flip side, as GitLab says, “…the need for documentation increases in parallel with the cost of not doing it.”
Yes, the agile manifesto values working software matters far more than comprehensive documentation. It also says that individuals and interactions are more important than processes and tools. Many people point me to a principle of agile that says, “The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.”
I don’t disagree with these statements, but I also think that the world has changed considerably since 2001 when we first read these words. My employers, Thoughtworks, were amongst the pioneers of the agile movement. Thoughtworks teams practised agile with many clients and could nuance the ideas of the manifesto. When handling low-complexity projects, few dependencies across teams, co-located colleagues and on-site customers, it was easy to get away with little documentation. It’s another matter, that the lack of documentation made software ownership transfer a rather painful task. But, let’s not digress down that path right now.
The moment projects became distributed, which was the case for almost everything the company delivered from India and China, it was a different story. As Martin Fowler noted almost two decades back, distributed projects need more documentation. And remember, distribution adds only one level of complexity.
If you have dependencies across teams, expect more documentation.
When your clients and stakeholders are remote, expect more documentation.
If you employ a sophisticated system architecture, you must document it so people understand how each component or service interacts with the others.
To make life easy for your developers, you must document your APIs, coding standards and the templates and tools that you reuse on the team.
Factors that drive the need for documentation
The less you document, the more people have to keep in their heads while contributing to your team. The more effective your documentation is, the lower the cognitive load. And that, as we know by now, enhances your developer experience.
So if your project ignores documentation, then I suggest you address this as an urgent problem. I know many of us baulk at the thought of documentation. It’s daunting and on the face of it, less valuable than producing software or other outputs that stakeholders pay us to create. But the value you lose from the lack of shared understanding is hurting you. If you’ve never experienced good documentation, you probably don’t know how much.
Take the following three steps to get yourself going.
Create a project handbook to catalogue the most obvious information people need, to contribute to your project. While building out this handbook, spend some time thinking about your documentation strategy. Think about ownership, review mechanisms, how to archive dated content, and how to improve the speed of retrieval. Use the information architecture of the handbook to nudge team members to support this strategy.
Streamline your onboarding process with a checklist. A meaningful test for your documentation is if a new team member can follow a set of steps to make a day-one contribution. The simplest way to do this is through an onboarding checklist. Pare it down to the bare minimum and test it on the next person you bring to your team.
Recognise the in-flow documentation triggers. Team knowledge is rarely one-and-done. It becomes richer as everyone contributes to it. For everyone to contribute, everyone must recognise the triggers to create documentation. I use the mnemonic DEEP, to nudge my colleagues about the documents they must create.
D for Decisions. Each time you decide something, document it for everyone’s benefit. Architectural, general or business decision records - when you document them, you help people who weren’t part of the decision to understand its rationale.
E for Events. Meetings, workshops, town halls—they’re all events. By documenting these, you persist the outcomes and knowledge from these interactions. That way you avoid the FOMO that causes everyone to attend every meeting.
E for Explanations. Instead of repeating explanations to different people, write them down. Or record them. Create once, share many times. This is most useful for onboarding.
P for Proposals. Whenever you have an idea, document the thought process and the details of what you’re proposing. Think about idea papers and design docs. Not only will it foster decision hygiene, but it’ll also build the collective memory of the team.
Think about maintaining your documentation much like you maintain your codebase. It’ll never be complete. You’ll continue to refactor it if it has to serve a meaningful purpose. And when I say “you”, I mean each person on the team!
Purchase calm with process and role clarity
Ask yourself if you’ve noticed this on a team.
Tasks originating, flying around, and getting completed on Slack or Teams.
People discussing work topics in long IM or email threads. These threads take as much time as an all-day meeting.
Ad hoc meetings to solve ad hoc problems. A bunch of people get on the call, but only a few take part actively.
On the face of it, this hustle seems very agile, but the chaos it generates leaves little room for deep work. It also disrupts every team member's state of flow - which we also know is a driver of developer productivity. The problem that I illustrated here, is a lack of processes around where work originates, the stages it flows through, and how you prioritise it. When you don’t formally agree on this process, people will imagine it for themselves. Don’t be surprised then, if everything feels urgent and if rogue asks from inboxes side-track the team from more valuable work.
Every distributed team must have a centralised task board or project management tool, where all work originates. No task, however trivial, should live outside this system. Any exception you make will normalise several future deviations. There’s no team I know that doesn’t already have access to a system of this sort or isn’t using such a system. The trouble is, that they don’t use the system diligently. Here’s the least you must do, to facilitate a calm workplace.
Document the stages that any piece of work goes through, from the time it enters the system, until you label it as “done”. Let there be no shadow processes and underground gate-checks. If it’s part of your process, it’s part of the documentation. Don’t forget to document the process to identify and address truly urgent issues.
Collective responsibility is fine, but accountability must be singular. Document the roles in your team and use a RACI matrix to clarify who is responsible, accountable, consulted or informed for a certain stage or type of work. This gives people the confidence to stay away from certain work, and to protect their focus.
There are many ways to achieve this process and role clarity. You can conduct a collaborative whiteboard exercise if you like. If you’re a team lead though, I suggest writing things down first, as a solo exercise. You can then ask your teammates for feedback. Even if you get things wrong with the first draft, you’ll notice that your colleagues will point out your mistakes and blind spots to agree faster than if you were to start from a blank slate.
Imagine a new world with this shared understanding in place.
If a task originates on instant messaging, ask the requester to log it into the central tool. It should go through the same prioritisation process as any other work.
When a discussion relates to work-in-progress, then that work is surely part of the central task board. That discussion should become part of the related issue on the task board, instead of living in chat or email.
If a new task comes up, concerning work-in-progress, you must address the question of scope. Is this a new scope? If yes, should you analyse it before you address it? Can it wait until you finish the work on hand? These considerations must be transparent on the task board.
There will be inevitable, ad hoc meetings to address work-related problems. But if you’ve documented your process clearly, they shouldn’t happen for random tasks that originate on IM or email. They’ll often relate to work-in-progress. And if so, your RACI should tell you who must be part of the meeting. Everyone else can consume a summary of the meeting, to stay up-to-date.
Unless you’re expecting people to monitor IM and email all day, you should’ve agreed that phone calls are (still) the best way to get someone’s attention on an urgent matter. We all know how reluctant we are to pick up the phone and call someone. By making this the only medium for urgent communication, you help people decide if something is urgent enough to endure the discomfort of phone calls.
I encourage you to read DHH’s fabulous post about managing processes before people and how it helps him and Jason Fried to be “moonlighting managers”. Indeed, a clear process benefits everyone. Managers avoid being annoying supervisors. Team members can experience focus and flow. Will that not help you be agile?
As I round off this piece, I know I’m ignoring a space where dysfunctional agile occurs more often than on software development teams. It’s in operational and support teams. If you’re part of such a team, the same disciplines apply to your work as well. I know a lot of operational and support work seems ad hoc, but on closer examination, most work needs prioritisation, analysis and tasking. So don’t give yourself a free pass. Whichever kind of team you work in, think about embracing agility more than embracing the hustle. You can only do that when team knowledge is easily accessible, when you’re not under constant stress from a barrage of incoming work, when you can work at a sustainable pace and when you enjoy deliberate practice and a state of flow. With the new year being right around the corner, I bet that’s a meaningful shift for all kinds of teams.