One day last June I was moved to share an opinion on Twitter.
And as the following exchange shows, at least one reader, my former colleague SengMing Tan, expressed a desire to know more.
As I foreshadowed, it has taken me a while to get around to elaborating.
However, the time has come. After nearly a year has elapsed I am fulfilling my promise.
Here is a description of how the team I’ve been working in at The Conversation uses Kanban, Trello, GitHub, Buildkite and Babushka to develop, review and ship software in a way that encourages flow.
Whilst there are several tools that we use which combine to help our team feel that we are constantly making progress towards shipping software, it is the Kanban system which underpins them.
As I understand it, the team started out with a low-fi approach by using a physical card Kanban wall. But by the time I joined, they had moved that wall to Trello. Which was just as well, because I was the first remote member of the development team.
Nevertheless, I think it is the Kanban style of only allowing a specified maximum number of cards in each “swim lane”, that is crucial to the overall feeling of progress that our team has.
As anyone who has used Trello knows, the general style is to represent progress by aiming to move cards from left to right across the columns. Whilst there are other columns on our development board, the following image depicts those that are essential to the way Trello supports our Kanban style of development.
I’ve blurred out the details of the cards but the key things to notice are the column headings. Notice that we have upper limits set for how many cards should be Queued, In Progress or under Review. This helps us each individually focus on completing a piece of work. As a team, it draws attention to ensuring that work is reviewed in a timely manner. If more than six cards are in the Review column, we consider that to be a broken state. It is a prompt to the team to give more focus to reviewing pull requests until we can merge enough of them and move the corresponding Trello cards to the Ready column.
There are other columns on our development board such as Confirmed to the left and Complete to the right of those shown in the screenshot. And we have other boards. However, in the context of how we use Trello to help the team achieve flow, the four columns shown demonstrate what is at the heart of how we use Trello.
One of the things I like about the paid version of Trello is the various Power-Ups. I find the GitHub Power-Up particularly useful. As I have written elsewhere, when I’m working towards a solution I prefer to share code via a pull request as early as possible. Fortunately our team has a strong culture of providing respectful feedback via pull requests.
There are times when I feel the need to gently prod my teammates to provide feedback on my pull requests. However, once the conversation within the context of a GitHub PR starts, it is usually very helpful. I like the way the tone of our comments tends to be questioning and curious rather than judgemental.
Once a Trello card, with a linked PR, is designated as available for review, it is important for the team to give it timely attention. Occasionally attention is diverted elsewhere. For this reason our team programmed our Slackbot to inform us if a card has been in the Review column for too long. To me this is a helpful prompt to keep contributing to the effort that will result in shipping software. Speaking of Slack, its integration with GitHub is an obvious boon. Being able to see via our main #dev Slack channel when a PR is created or merged certainly helps teamwork.
Being able to easily trace code changes in a PR that result from a Trello card is wonderful for maintaining flow. Did I mention how much I love the Trello GitHub Power-Up?
The ways in which Buildkite can assist teams achieve flow is worthy of a post in itself. For the purposes of this discussion, I’ll confine myself to the way Buildkite is integrated into our team workflow.
We use Buildkite to automate our continuous integration builds. Unsurprisingly it is integrated with GitHub so that it’s easy to see whether or not the build has passed for the latest commit on a PR. Then there is the Slack integration, which I find useful as another prompt about the success or failure of builds. There are times when the first place I’ll notice a build failure is via our #buildkite Slack channel.
Obviously part of achieving flow is ensuring that the build for a PR is successful before that PR is merged. And, of course, the build for the master branch must be before we can deploy.
Whilst we have not yet reached the point where we continuously deploy our software, we do typically deploy applications several times each day. The tool that helps us software is Babushka, courtesy of an alumnus of The Conversation, Ben Hoskings.
Babushka may not be as well known as other tools which support deployment but it has served our team well so far. Once a master build is for an application and we are ready to , it is a simple matter of entering babushka ‘SHIP IT’ at the command line and the defined dependencies will enable babushka to deploy our software.
It’s all about flow. As emphasised by psychologist Mihály Csíkszentmihályi, in a personal context flow is “the mental state of operation in which a person performing an activity is fully immersed in a feeling of energized focus, full involvement, and enjoyment in the process of the activity.”
Translating this concept into a software team, we can see that it is important to remove as many barriers as we can to the team being fully immersed in shipping quality software. The Kanban approach supported by various integrated tools can certainly help in this regard.
And, in my experience, it can be fun too!
A few years ago I wrote a brief post about committed stand-ups in which I drew on the work of Jonathan Rasmusson and posed the following questions:
How effective is your daily stand-up meeting?
Is it energising the team or does it have a “going through the motions” feel to it?
Part of Jonathan’s answer was to, rather than simply let the team know what you plan to do today, tell them “how you are going to crush it today”.
The change of emphasis that Jonathan suggested can only work if individuals in the team genuinely feel like “crushing it” for the sake of the team. Unsurprisingly, barriers can exist which prevent effective stand-ups.
The reason for standing up whilst meeting is to keep the discussion short and to the point. If the meeting drags on for too long, participants are likely to become distracted. Obviously, the larger the team, the harder it is going to be to enable each member of the team to focus on what others are saying.
In my view, any more than about six people in a standup reduces its effectiveness.
Good stand-ups typically involve participants who are all working on the same project. If someone else in the stand-up starts talking about an activity that is unrelated to what the another in the group is doing, it is only natural that the listener will be inclined to tune out.
Stand-ups can become less effective if always led by a manager. In my experience, they work better if the role of convening the stand-up is shared throughout the team.
Whilst stand-ups can effectively include remote participants, of course they will break down if the technology fails. When this happens, those that are remote can be left laughing or thinking, “what was the point of that?”
In my experience, the most effective stand-ups have been in small teams focussed on the same project.
I recall that during my early experiences of eXtreme Programming in about 2001, the Daily Stand-up practice was one of those that I found most effective. In my view, it can still be very valuable, provided barriers aren’t inadvertently put in its way.
How effective is your daily stand-up meeting? Is it energising the team or does it have a “going through the motions” feel to it?
In his book, The Agile Samurai, Jonathan Rasmusson recommends that each person tell the rest of the team:
It’s a change of emphasis that I think is worth trying. As Jonathan says, it is a demonstration of commitment to the team and “dramatically increases the chances of you getting it done.”
However, if being so positive feels uncomfortable or forced, this probably indicates that there are other problems affecting the team’s morale that need to be dealt with.
Before you turn off, I realise that the term “agile” in the context of software development has been overused to death. My aim is not to write a buzzword-compliant essay. Rather, having endured many projects run under the misapprehensions of outmoded assumptions and having endeavoured to get the best out of so-called “agile” approaches to software development, I feel I have a worthwhile perspective to offer.
I think the crux of discussing “agile” is to acknowledge the differing points of view of the developer and the customer. This may seem obvious but, in my view, it’s a crucial point.
As a developer, I’ve had a lot of time to reflect upon shortcomings of the more traditional approach to building software. I well remember attending in-house courses on “Requirements, Planning and Estimating” and “Project Management” at IBM, early in my career, in the mid-80s. At the time there was much emphasis on documentation. The requirements, the external design and the internal design all had to be documented and formally approved before one line of code was written.
To be fair, in some cases, projects were divided into stages but, by and large, my experience of project management in the 1980s and 1990s was dominated by the waterfall model. The idea of iterative development had yet to hit the mainstream.
Central to the problems of the waterfall model is the difficulty in estimating accurately when grappling with initial requirements and designing an application. Usually, throughout the course of a software development project, understanding of requirements and design changes significantly. So it is very unlikely that an accurate estimate can be given at the outset. No doubt this is why estimation games have been well practiced in the software industry.
As described in Jonathan Rasmusson’s excellent book, The Agile Samurai, all projects are bound by four variables: Time, Budget, Quality and Scope. The problem with basing a software development project on the waterfall model is that this often leads to an expectation that all four variables will be fixed. Given the inherent difficulty in accurate software project estimation, this is obviously unrealistic.
The agile approach addresses these problems by treating scope as the only variable and delivering software in frequent iterations. As shown in Extreme Programming Release Planning, which I first experienced about ten years ago, requirements are broken up into “user stories”, each of which is estimated and prioritised.
A key point is the unit of estimation. In the agile approach, it is not a measurement of actual work hours which may be translated into a monetary amount. In original XP planning, the unit was an ideal week. The name of the unit may be arbitrary. It doesn’t matter what you call it. What really matters is that each user story is estimated relative to others. This enables allocation of user stories to releases more accurately as the developers discover roughly how many units they can complete in each iteration. Also, since iterations are kept small in duration, user feedback allows timely modification and exploration of user stories that will comprise subsequent iterations.
All the elements of agile planning make sense to me as a developer. However, a customer, not necessarily used to software development, may be less easily persuaded.
As a customer, speaking generally, I expect value for money. For most goods and services I am used to having a fixed-price quotation honoured for clearly agreed specifications.
Why should I expect any different for software development? If I ask how much it will cost for a specific software application to be built, why should I not expect the supplier to let me know in advance?
What’s this talk about estimating requirements in terms of arbitrary points? And what’s this I hear about pair programming? Do you seriously expect me to pay for two developers working on the same piece of code at the same time?
At this point I should emphasise that I am trying to think of various customers that I have dealt with over the years and put myself in their shoes. As I do that I am reminded that these customers have had various degrees of acquaintance with the complexities of developing software. I think that this gets to the heart of the matter. As a customer, if there is a good reason why I should not expect a fixed-price quotation then the developer must be able to explain that reason clearly to me.
After all, the customer is paying the bills. Any argument for embracing an agile approach must be compelling in monetary terms.
When we consider the agile approach to software development, I think it is clear that there is a gap between what proponents see as benefits of this approach and the expectations of many customers.
If you’re still with me, you may be expecting that at this point I will reveal my brilliant answer to the conundrum. I am sorry to disappoint you. If there was a brilliant answer I am sure someone else would have capitalised on it before now. Obviously, it is critical for an agile software developer to clearly describe to their customer at the very beginning of an engagement what to expect. As with so much in life, setting expectations is crucial. It is a lesson I need to keep reminding myself.
Every customer is different. Every relationship will differ. And relationships change over time. I forget where I read it, but the essence of the quote was that “in a good agile relationship, trust improves with every iteration”. I like the spirit of that statement.
It is also important to allow your approach to evolve. As Ben Webster said recently, “embracing as many mistakes as possible on the job” is a useful resource as an agile practitioner.
To borrow from The Mythical Man-Month by Fred Brooks, there is no silver bullet. Making the most of the agile approach to software development requires an individual response. If you would like to share elements of yours, I look forward to your comments.