Hansel and Gretel - 5 audit trails from the flow of our work
Like the pebble trail in the story of Hansel and Gretel, our projects need audit trails for us to keep track of changes, communicate on a daily basis, and to onboard and align people. We discuss the five most important trails in this article. In the context of distributed teams these represent communication and documentation in the flow of work.
- Meeting minutes to get everyone up to speed with the outputs of any meetings that a subset of the team has attended.
- Commit messages help build a history of code commits, so that it becomes easier to investigate the root cause of issues.
- Architectural decision records build up an archaeology of how the project's architecture has changed until a certain point in time.
- Business decision records keep track of every business driven decision on the project, including but not limited to prioritisation, roadmaps, migrations, budgets, target groups and experiments.
- Pull requests are controversial, but if your team's using them, you can leverage them as an effective communication tool to share feedback about code and build an engineering culture.
You might remember a story from our childhood. That of Hansel and Gretel. I won’t attempt to tell the entire story here. That’s for another day. What you might remember from the story is the trail that Hansel leaves so the children can find their way back home. While the trail of white pebbles endures the day, they lose the breadcrumb trail to birds and animals. On projects as well, you’ll need similar, enduring trails. Why, you may ask. Well, for the same reason - to find your way back if you need to. You need ways to resolve a bug, understand a decision, or identify the rationale for a code change.
Back in the day, when we all sat around a table, you could “just ask” and get answers to questions. Can’t resolve a bug? People would just huddle around and tell you what to do. Don’t understand an architectural decision? Just shout out. Someone will explain it to you. Frustrated with a code change? Say it out loud and someone will give you the reason - however unsound. It’s a different matter that this was a very distracting and unproductive environment to work in. The fact is that you got your answers. Yes, you could “just ask”.
The “just ask” pattern breaks down when you’re working remotely. There’s no one to huddle with, no one to holler at. If you’re following this site, then you are one of those people who cares about minimising interruptions for yourself and everyone else. There has to be an async way to backtrack, hasn’t there? So today’s post is all about learning from Hansel. We’ll discuss the trails we create in the flow of work.
1. Meeting notes
Now that you don’t have a table that you can shout at, think about the number of communication paths on a team. If you just took the simplest communication path possible; i.e communication between two people; then the formula for the number of different paired paths in a team is nC2. Here ‘n’ refers to the size of the team. For a team of three people, that’s three paths. Make it a team of four, and now you have six. For a team of 10, you have 45 options for paired communication. The graph below shows you how paired communication explodes with team size.
Communication complexity increases with team size
Those numbers by themselves may seem staggering, but they don’t show the complete picture. Paired communication is just one type of combination that exists in a team. To derive the true number of communication paths in a team, the formula is as follows.
nC2 + nC3 + .... + nCn
This is where things get properly crazy. For a team of 10, you now have a whopping 1013 possible communication combos. In fact, you have so many combinations in which people can communicate with each other, that it makes the paired communication graph look flat!
Total number of communication pathways dwarf the number of paired pathways
Why am I telling you this? Well, that’s so you know how verbal communication between teams can break down. If three people in a 10 person team have a meeting and they just rely on a verbal relay of information to share the output of that meeting, the best-case scenario is another meeting just to convey the information. We’ve already discussed that conveyance in a team should be asynchronous. The worst-case scenario can be a pretty huge number of additional meetings; again, just to convey information. Before you know it, you can have a game of Chinese whispers going.
A better approach is to write a good, old-fashioned meeting summary. For those familiar with the initial promise of Java - this is a “write once, run anywhere” pattern. A succinct summary can help people consume the gist of a long meeting in a few minutes. Popular calendaring tools like Google Calendar and Outlook already build in nudges for you to make meeting notes. Confluence has a very practical template - any actions you decide from your meeting also become to-dos on your team’s space. It helps you track due dates and action-owners get notifications from the system. If your organisation’s willing to spend the extra bucks, there are tools that’ll help you make this even easier - Fellow, tl;dv, Hugo and Otter come to mind. The tools aren’t the problem; discipline is.
It’s not enough to just write meeting notes. You need to make them easy to find and reference. Tools like Confluence make this easy - all your meeting notes stay in one place. You can also tag people if you need their attention. If you use Office 365 or Google Workspace, you’ll need to decide where to host these notes. Don’t expect everyone to keep up with the notes from every meeting. We don’t want people drinking from a firehose, trying to consume every artefact the team produces. Instead, we want it to be easy for people to reference these artefacts, if they need to.
2. Commit messages
If you’re a developer, you will empathise with what I’m about to say. Do you remember a time when you had to fix a bug, and you had to:
check a huge number of logs with unhelpful commit messages, such as “fix variable type”;
and understand the code corresponding to each commit, so you could find the root cause?
This is where effective commit messages come into play. Sure, at some point someone has to dive into the code to fix a bug. Well written commit messages just make it easier to do a fast, first-level investigation to identify where the root cause may be. The anatomy of a commit message is beyond this post, but there are a few things I’d like to discuss here.
There’s no reason not to write a good commit message. You just need a <50 character title and a description, where each line is <72 characters. More on the 50-72 rule here. If you’re struggling to meet these constraints, it may mean that your commits are too big. In which case, make your commits more atomic.
The title of the message should explain the purpose of the commit. This helps someone examining multiple messages identify which commit corresponds to the change they’re looking for.
Describe the commit so anyone looking at it in the future can understand the scope of the change you’ve made, without getting into the code itself.
Use integrations so your commits reflect on your task board. These are excellent proxies for status updates. For example, Jira integrates with GitHub, GitLab and BitBucket.
I find Hoorvash Nikoo’s guide to commit messages very useful and I’m sure you can find a dozen other useful guides. Commit messages are the most fundamental units of communication in a distributed project. Take some time to examine the quality of commit messages on your project and figure out how you can improve your own messages if you’re writing code as well. Oh, and here’s a hot tip.
“There is a command-line tool available on GitHub named Commitizen which makes it a little bit easier. When you want to commit, you just type ‘git cz’ and it asks you a couple of questions and then creates the commit with a proper message for you.”
No excuse to not get better, eh?
3. Architectural decision records
One of the central principles of extreme programming is “simplicity” and, by extension, simple design. In the recent decade, the idea of “evolutionary architecture” has gained popularity. Teams identify the fundamental technical requirements for a system up front, but delay other decisions till the “last responsible moment”. The benefit? You avoid up-front guesswork, and allow yourself to decide when you have the necessary information.
This means that your architecture will change over time and there will be times when you’ll need to make sense of why your architecture is a certain way. Architectural decision records (ADRs) help by providing an archaeology of your project. Here are some benefits of using them.
You can align partner teams and stakeholders at speed by using these artefacts to drive consensus, promote framework and code reuse, and to even remove duplicative efforts.
ADRs can help onboard current and future team members by helping them understand how the project’s architecture has evolved and the rationale behind specific architectural decisions.
Project ownership transfer becomes easier with ADRs. The new team taking over the project can use these decision records to understand how the system’s architecture has developed until this point.
As the Thoughtworks tech radar says,
“For most projects, we see no reason why you wouldn't want to use this technique.”
Joel Henderson’s guide to ADRs is one I recommend - there are lots of templates that you can use to get started. I won’t go into the details of ADR writing, but I do have a few pieces of advice.
Like meeting notes, ADRs should live in a central location for the project. You can use collaborative file sharing tools like Google Drive or wikis like Confluence. A sensible default, however, is to store them in source control, with your code.
No decision is too small to document. In fact, the cost of your undocumented change is just as high as that of the first duplicated effort in your team or organisation. ADRs are lightweight, so take a few minutes to write the change down.
If you come across an undocumented decision, be a good citizen. Collaborate with your team to document it, so you avoid confusion in the future.
Last but not the least, write ADRs in a way that everyone on the team can understand. You know your ADR is impactful when both techies and business folks can understand the rationale behind your decisions and the solutions you’ve arrived at.
4. Business decision records
Agile is all about “responding to change over following a plan”. A part of that premise is that business needs and imperatives change over time, and software development should respond to these changes. As any agile practitioner will know, this is what we experience day in and day out. The business makes decisions all the time - about prioritisation, roadmaps, migrations, budgets, target groups, experiments and whatnot.
The larger the team, the more the decisions, and the harder it is to keep track of them in our heads or through conversations. So, just like ADRs, distributed teams need to adopt the discipline to document business decisions as well, in business decision records (BDRs). All heroes don’t wear capes and Joel Henderson sure is one of those heroes. He has an excellent guide for general decision records, which I’m sure you’ll find useful. If you use Confluence, it has a built-in template that also allows you to populate all decisions in one place in your team space.
5. Pull requests
Finally, pull requests (PRs). I left this last trail for the very end, because it’s popular and controversial in equal parts. The feature, which most source control systems provide (GitLab calls it a “merge request (MR)”) allows anyone who wants to contribute code to a project, to package their changes so someone can review their code before merging it into the codebase. In theory, it’s a systemic approach to facilitate code reviews.
No doubt, I know many teams use PRs or MRs as the means for code reviews. On the other hand, I know that most of my colleagues prefer a trunk based development approach; combined with pair programming. My colleague and author, Kief Morris, considers this latter approach more aligned to continuous integration. Undoubtedly, PRs introduce a delay in the system. Consider this informal study by my colleague Ryan Boucher.
“91% of PRs had zero comments (about 7,000 in 2020) and the total time spent waiting (from creation to resolve) was about 130,000 hours in 2020”
Those figures highlight that code reviews driven by PRs can become a time-consuming tick-box activity. The overhead of pull requests also has inadvertent consequences. For example, it’s common practice for developers to refactor adjacent code when they’re fixing a bug or writing a new feature. PRs seem to discourage this behaviour. Here’s what my colleague Dan Mutton says.
“On every project I've been on which uses PRs, the overhead has all but stopped people from doing small "just while I'm in the area" improvements (eg. fixing a misnamed test, a readme change, a minor refactor). Refactoring in the feature branches still happens, but only really in places the feature is touching. Who really wants to go through the whole process of creating a branch, code and PR, then chasing people for a minor change? Where they have done so, there are many examples of these PRs lying dormant, becoming quickly out of sync and then deleted in a half-yearly cleanup.”
So that’s the controversy. That said, PRs are popular and you may not have a say on whether or not your team uses them. You may see more value in them than my colleagues do. If your team does not pair, then this may be an effective option to ensure code quality. And if so, you need to communicate well in your PR. Keavy McMinn’s article is a simple guide to writing pull requests. Here are a few good practices I’d like to highlight.
Communicate purpose clearly. Explain not just what has changed, but why. For example, if you’re adding new functionality, reference the user story you’re implementing.
Use integrations with your task board. As I’ve mentioned earlier, Jira’s integrations with GitHub, GitLab and BitBucket make it easy to reference the tickets in question.
Explain your implementation, especially if some parts of it may not be clear by just examining the code. Use videos, sketches, or diagrams to clarify your explanation. Ideally, these should already be part of the task on your task board.
If you’re reviewing a PR, be respectful when providing feedback. Do your best to understand before you offer suggestions. Help the author understand the rationale behind your suggestions.
If you’re the author of the request, be open to feedback - this is your opportunity to not just improve your code, but also to create a healthy engineering culture.
PRs are notorious for the lag they introduce. Reviewers and authors need to collaborate with each other in a thoughtful and timely fashion so you keep the waiting time minimal.
The five audit trails you should pay special attention to
So those are the five audit trails your team needs to build in the flow of your day-to-day work so you can make sense of where you are at any point in time. You will not build Rome in a day, so I suggest examining your practices one at a time. Make a team commitment to improve at least one or two practices every two weeks or every sprint if you’re following scrum. Find a champion for each of these practices so they can hold everyone accountable. As those audit trails become more apparent, you’ll notice it’s easier to find your way back home!