Achieving Flow in a Development Team

June 1st, 2017

One day last June I was moved to share an opinion on Twitter.

And as the following exchange shows, at least one reader, my former colleague SengMing Tan, expressed a desire to know more.

As I foreshadowed, it has taken me a while to get around to elaborating.

However, the time has come. After nearly a year has elapsed I am fulfilling my promise.

Here is a description of how the team I’ve been working in at The Conversation uses Kanban, Trello, GitHub, Buildkite and Babushka to develop, review and ship software in a way that encourages flow.

Tools and Techniques

Kanban

Whilst there are several tools that we use which combine to help our team feel that we are constantly making progress towards shipping software, it is the Kanban system which underpins them.

As I understand it, the team started out with a low-fi approach by using a physical card Kanban wall. But by the time I joined, they had moved that wall to Trello. Which was just as well, because I was the first remote member of the development team.

Nevertheless, I think it is the Kanban style of only allowing a specified maximum number of cards in each “swim lane”, that is crucial to the overall feeling of progress that our team has.

Trello

As anyone who has used Trello knows, the general style is to represent progress by aiming to move cards from left to right across the columns. Whilst there are other columns on our development board, the following image depicts those that are essential to the way Trello supports our Kanban style of development.

I’ve blurred out the details of the cards but the key things to notice are the column headings. Notice that we have upper limits set for how many cards should be Queued, In Progress or under Review. This helps us each individually focus on completing a piece of work. As a team, it draws attention to ensuring that work is reviewed in a timely manner. If more than six cards are in the Review column, we consider that to be a broken state. It is a prompt to the team to give more focus to reviewing pull requests until we can merge enough of them and move the corresponding Trello cards to the Ready column.

There are other columns on our development board such as Confirmed to the left and Complete to the right of those shown in the screenshot. And we have other boards. However, in the context of how we use Trello to help the team achieve flow, the four columns shown demonstrate what is at the heart of how we use Trello.

GitHub

One of the things I like about the paid version of Trello is the various Power-Ups. I find the GitHub Power-Up particularly useful. As I have written elsewhere, when I’m working towards a solution I prefer to share code via a pull request as early as possible. Fortunately our team has a strong culture of providing respectful feedback via pull requests.

There are times when I feel the need to gently prod my teammates to provide feedback on my pull requests. However, once the conversation within the context of a GitHub PR starts, it is usually very helpful. I like the way the tone of our comments tends to be questioning and curious rather than judgemental.

Once a Trello card, with a linked PR, is designated as available for review, it is important for the team to give it timely attention. Occasionally attention is diverted elsewhere. For this reason our team programmed our Slackbot to inform us if a card has been in the Review column for too long. To me this is a helpful prompt to keep contributing to the effort that will result in shipping software. Speaking of Slack, its integration with GitHub is an obvious boon. Being able to see via our main #dev Slack channel when a PR is created or merged certainly helps teamwork.

Being able to easily trace code changes in a PR that result from a Trello card is wonderful for maintaining flow. Did I mention how much I love the Trello GitHub Power-Up?

Buildkite

The ways in which Buildkite can assist teams achieve flow is worthy of a post in itself. For the purposes of this discussion, I’ll confine myself to the way Buildkite is integrated into our team workflow.

We use Buildkite to automate our continuous integration builds. Unsurprisingly it is integrated with GitHub so that it’s easy to see whether or not the build has passed for the latest commit on a PR. Then there is the Slack integration, which I find useful as another prompt about the success or failure of builds. There are times when the first place I’ll notice a build failure is via our #buildkite Slack channel.

Obviously part of achieving flow is ensuring that the build for a PR is successful before that PR is merged. And, of course, the build for the master branch must be green_heart before we can deploy.

Babushka

Whilst we have not yet reached the point where we continuously deploy our software, we do typically deploy applications several times each day. The tool that helps us ship software is Babushka, courtesy of an alumnus of The Conversation, Ben Hoskings.

Babushka may not be as well known as other tools which support deployment but it has served our team well so far. Once a master build is green_heart for an application and we are ready to shipit, it is a simple matter of entering babushka ‘SHIP IT’ at the command line and the defined dependencies will enable babushka to deploy our software.

Summing Up

It’s all about flow. As emphasised by psychologist Mihály Csíkszentmihályi, in a personal context flow is “the mental state of operation in which a person performing an activity is fully immersed in a feeling of energized focus, full involvement, and enjoyment in the process of the activity.”

Translating this concept into a software team, we can see that it is important to remove as many barriers as we can to the team being fully immersed in shipping quality software. The Kanban approach supported by various integrated tools can certainly help in this regard.

And, in my experience, it can be fun too!

Tags: agile teams flow | Comments (1)

Reflecting on Ruby Conf AU 2016

February 18th, 2016

A few weeks ago, James Healy, my colleague at The Conversation, alerted the team to the following tweet from Kent Beck.

Fast forward to the present and I find myself reflecting on Ruby Conf AU, held on the Gold Coast recently.

My gut feeling is that the conference would score fairly well if evaluated against Kent’s criteria. But us programmers are analytic beings, so let me reflect on the talks and see how well they stack up.

The Talks

Whilst there were many excellent talks amongst the 24 that were included, I’m going to pick some of them that resonated with me and place them each in one of Kent’s categories.

Working in Small Increments

Gradual refactoring, as illustrated by Katrina Owen in her One Undo talk, is a vital aspect of working in small increments. Katrina showed us an example of how to initially place some poorly optimised code under test and then progressively improve it by teasing out the abstractions. To me, it was a brilliant example of how to “remove duplication and improve names in small cycles”, as Joe Rainsberger would say.

To work in small increments, refactoring as you go, requires effective testing strategies. To this end, Tom Ridge gave a thoughtful talk which focussed on the readability of RSpec code. In Explicit Tests Tell a Better Tale, he challenged us to consider how our choices in our RSpec usage affect our cognitive load.

Being Kind

I could have categorised the talk by Elle Meredith as honest, which undoubtedly it was. However, my first inclination about Elle’s talk entitled Feedback Matters was that it was foremost about being kind. As Elle’s talk emphasised, giving and receiving feedback in the right spirit with careful attention to how it affects people is of utmost importance to software development teams.

Ernie Miller struck a chord with his Humane Development – Empathy talk. I’m looking forward to seeing his slides and watching the video to see what else I can glean from what Ernie had to say. But I do recall that I found myself thinking, how good is it to hear someone talking about empathy in the context of software development!

Adam Cuppy may have posed the question, What If Shakespeare Wrote Ruby? and arguably provoked thoughts about abstract commonalities between Shakespearian and Ruby patterns. However, to me there were overriding factors that lead me to unquestionably consider his talk as kind. For one thing, this professionally trained actor provided the opportunity for the organisers to schedule his talk as the second last in the conference. A masterstroke! Secondly, Adam describes himself as a Master of Smile Generation. I rest my case.

Being Honest

Jeff Casimir opened the conference with a talk that unquestionably fitted the “honest” bill. Sharing his experiences in the context of 10 Years and 10 Mistakes set a nice tone for the conference. I found it to be a refreshing approach. We all learn from our mistakes but it takes a certain degree of intestinal fortitude to get up on stage and honestly talk about all the ways you have goofed up.

Debugging Diversity, presented by Dan Draper and Catherine Jones was without a doubt an honest appraisal of the challenges that face the Australian tech community with respect to increasing the opportunities for people who don’t fall into the “white male” stereotype that is predominant.

Given the seriousness of the diversity challenge in tech, it was pleasing to also listen to Jess Rudder give her perspective on the topic. Her presentation, Diversity in Tech – It’s About More than Just the Hiring Process, hit home, focussing on a critically important aspect. The fact that so many women choose to leave the IT industry points to a problem that the community needs to address.

We were privileged to hear Senator Scott Ludlam present the closing keynote of the conference. Honesty is a word that leaps out when I reflect on Scott’s talk, How the government broke the internet. The importance of an honest approach to the Internet and democracy, that is. As Scott illustrated, we certainly cannot assume government honesty when it comes to privacy for individuals and transparency of governments.

Details

I guess it’s fair to say that Paulo Perrotta presented about “details”. After all, his talk delved into Refinements, a Ruby 2 feature. However, when you consider that Paulo’s talk was entitled Refinements – the Worst Feature You Ever Loved, you get an idea that this Italian has a devious sense of humour. And so it proved. To me, this was a great example of the importance of presenting a technical talk as an entertaining story.

When I reflected on what André Arko had to say, I admit that I hesitated about where to place his talk, or talks. His official talk was entitled Security Is Hard, But We Can’t Go Shopping, in which he shared with the audience the importance of handling security vulnerabilities. I guess it’s fair to say that’s an explanation of details. However, that wasn’t the end of his message. André went on to talk about Ruby Together, which to me sits squarely in the honesty category.

The Social Activities

Of course, the talks were only part of the conference. I love the way RubyConf AU has evolved to feature plenty of social activities. My impression is that our international guests particularly appreciate these, especially those on the Saturday morning. Kudos to the conference organisers, Jo Cranford, Rob Jacoby and Trish Jacoby, together with the volunteers for their thoughtfulness and kindness.


Richie Khoo, one of the Ruby Australia founders, enjoys a refreshing swim in Springbrook National Park

In Closing

I think my selection of talks bears out my contention that the success of Ruby Conf AU 2016 coincides with the fact that many of the talks fitted in with Kent’s idea that a good conference focusses on small increments, kindness and honesty.

Tags: conferences | Comments (0)

Digging deeper into Rails scopes

October 8th, 2015

Background

Recently I wrote a post about Rails scopes and lambdas in an attempt to clarify why it is so common to see the use of lambdas with Rails scopes. I now realise that I could have gone a bit further in my explanation.

A question that is sometimes asked is:

Why are lambdas used with Rails scopes? Why not procs or blocks?

To answer that question it is useful to first look at the implementation of the Rails scope method in ActiveRecord::Scoping::Named::ClassMethods.

Implementation Details

The implementation is as follows:

def scope(name, body, &block)
  unless body.respond_to?(:call)
    raise ArgumentError, 'The scope body needs to be callable.'
  end

  if dangerous_class_method?(name)
    raise ArgumentError, "You tried to define a scope named \"#{name}\" " \
      "on the model \"#{self.name}\", but Active Record already defined " \
      "a class method with the same name."
  end

  extension = Module.new(&block) if block

  singleton_class.send(:define_method, name) do |*args|
    scope = all.scoping { body.call(*args) }
    scope = scope.extending(extension) if extension

    scope || all
  end
end

Now let’s consider this implementation in conjunction with the following scope:

class Article < ActiveRecord::Base
  scope :created_before, ->(time) { where("created_at < ?", time) }
end

In this example, :created_before is interpreted as name and ->(time) { where("created_at < ?", time) } as body.

Block, Proc or Lambda?

Notice that body must be callable. So that rules out a block, which is simply a syntactic structure. However, it does still allow body to be a lambda or a proc.

Whilst it is technically possible for a proc to be used in conjunction with a Rails scope, a lambda is more useful because of the constraint that it must, unlike a proc, have a specific arity. For example, in the example above, Article.created_before must be called with one argument.

Summing up

Hopefully, that explains why lambdas are used with Rails scopes.

Of course, you’ll notice that the implementation of the scope method uses metaprogramming via :define_method to create a class method that could have been programmed directly.

Tags: rails ruby scopes lambdas | Comments (0)

Rails scopes and lambdas

October 6th, 2015

Recently I was asked a searching question that left me pondering. The questioner mentioned that he had been following The Rails Way and was puzzled by it’s emphasis on the use of lambdas in conjunction with Rails scopes.

At the time I could only offer a vague answer along the lines of it being a common idiom. I was not satisfied with my reply so committed to following up with a more definitive response.

Background

Ruby Lambdas

Firstly, let’s recap and clarify exactly what a Ruby lambda is.

The lambda method belongs to the Kernel module. Since Ruby 1.9 introduced an alternative syntax, a lambda can also be identified with the characters ->. So, the following two code blocks are equivalent:

succ = ->(x) { x + 1 }
succ = lambda { |x| x + 1 }

Another thing to note about lambdas is their arity, or how many arguments are required when they are called. Every lambda has a precise arity, which can be tested using the arity method as follows:

->(x) { x + 1 }.arity # => 1

There is much more that can be said about lambdas but I’ll leave that to others. For example, Chapter 6, Methods, Procs, Lambdas, and Closures within The Ruby Programming Language by David Flanagan and Yukihiro Matsumoto gives a thorough treatment despite the age of the book.

As a taste, let me quote:

Blocks are syntactic structures in Ruby; they are not objects, and cannot be manipulated as objects. It is possible, however, to create an object that represents a block. Depending on how the object is created, it is called a proc or a lambda. Procs have block-like behavior and lambdas have method-like behavior. Both, however, are instances of class Proc.

Rails Scopes

At the time of writing, the latest Rails version is 4.2.4. On the subject of Rails scopes it is worth reading what is said about them in the Rails guides.

It is also worth keeping in mind that Rails has undergone many changes in it’s history. Currently the Rails guides illustrate that this:

class Article < ActiveRecord::Base
  scope :published, -> { where(published: true) }
end

is exactly the same as:

class Article < ActiveRecord::Base
  def self.published
    where(published: true)
  end
end

Furthermore, the Rails Guides section about passing in arguments advises:

Using a class method is the preferred way to accept arguments for scopes.

In other words, this:

class Article < ActiveRecord::Base
  def self.created_before(time)
    where("created_at < ?", time)
  end
end

is preferable to:

class Article < ActiveRecord::Base
  scope :created_before, ->(time) { where("created_at < ?", time) }
end

If you look carefully and test the two styles, you will realise that the important part of the code is the where clause. So long as the scope or class method returns an object that is an ActiveRecord::Relation, it can be chained together with other scopes or class methods.

In essence, with Rails 4.2 there is no reason to use scopes together with lambdas in preference to class methods. It is a matter of personal preference.

So, why the emphasis on using scopes with lambdas?

To answer this we need to consider the history of Rails. It was around the time of version 2 that named scopes were introduced. They allowed the convenience of chaining scope calls together. Later, if my memory serves me correctly, in version 3, Arel was introduced. This reduced the importance of named scopes when it came to specifying a variety of conditions to return a collection of model objects.

Fast forward to today and, as the Rails Guides state, there is no need to use scopes with lambdas instead of their equivalent class methods, which are arguably easier to read. However, given that Rails has been around for over ten years now, there are lots of legacy Rails codebases, many of which contain a plethora of scopes. Also, there are plenty of seasoned Rails programmers who have formed the habit of using Rails scopes.

In Summary

Hopefully this post has done a better job of clarifying the use of lambdas in Rails scopes than my original fumbling reply during a conversation at last week’s Melbourne Ruby meetup.

Tags: rails ruby scopes lambdas | Comments (0)

Effective Collaboration using GitHub PRs

September 19th, 2015

Background

In her excellent talk about Prefactoring at RubyConf Australia 2015, Coraline Ada Ehmke emphasised her preference for collaboration over code reviews:

I don’t like pull requests. By the time code comes up in a pull request, a thousand micro-decisions have been made about how it’s going to look and how it’s going to function. If you have an alternate idea, maybe even a better idea or a different idea, or questions about why something was designed the way it was or suggestions about how to make it better, by the time it comes up to a pull request, you may be reluctant to share that kind of feedback because somebody has put a lot of work into that already. You don’t want them to feel bad about the work that they’ve done. You don’t want them to feel defensive about the code that they’ve written. So we end up accepting code that is technically sufficient instead of excellent, code that doesn’t necessarily live up to our architectural vision or our standards of quality, and thus begins a slide into entropy.

In my own talk about Loving Legacy Code I touched briefly upon GitHub pull requests, recommending them wholeheartedly, especially when compared with the “structured walkthroughs” that were in vogue in earlier decades. I prompted the audience to ponder whether they were using GitHub PRs to their full potential.

Am I at odds with Coraline on this topic? Not necessarily.

Whilst I agree with Coraline’s sentiments, I believe that, with a thoughtful approach, GitHub pull requests can be used advantageously without falling into the traps she speaks of.

Thoughtful Collaboration using Pull Requests

Early feedback about design

For all non-trivial programming tasks, it definitely makes sense to check your intended approach with one or more colleagues before getting too deeply into coding. How this is effectively done depends on aspects such as the proximity of your colleague. You may be remote but in the same time zone. Or you may be in a very different time zone. The important thing is to make good use of the many different types of tools that now facilitate collaboration.

For co-located collaborators, design discussions around a white board are often effective.

For remote collaborators, in days gone by, your choices may have been limited to the telephone and email. However tools such as Slack, Trello and Skype now provide plenty of scope for collaborating about design approaches.

Once thoughts are closer to code, collaboration via a GitHub pull request can be beneficial. Obviously the creation of a pull request requires a difference in the code. However, that can simply be a TODO comment, for example. From that point on, a valuable discussion can be had via comments that can include code snippets, attached diagrams and all sorts of artifacts.

Early feedback about code

Coraline is right to highlight the risks of creating pull requests only once the programmer believes their change is finished.

By contrast, the creation of a PR early in the development process allows more visibility to colleagues and should, in my view, go hand in hand with a culture that encourages early feedback.

The earlier a PR is created the better. Certainly, as soon as thought about a change can be conveyed by code, it makes sense to create a PR to promote discussion with your peers. Even if you’re not ready to commit code changes, GitHub Flavored Markdown offers an ideal way of sharing thoughts in code in comments within a PR. Resulting comments from your peers can steer you towards a good solution.

One practice I would encourage is to use the prefix [WIP] in the title of the PR to indicate that it is a work in progress rather than one ready for final review.

For all but trivial PRs, my ideal PR is one which is created early in the thought process, undergoes collaboration via comments with colleagues and eventually leads to a solution that the affected developers are all prepared to accept.

Avoiding over-protectiveness

Obviously, if a developer puts a lot of effort into a solution and, once they think it’s finished and ready to be shipped, they open a PR, they are likely to be protective of their efforts. It is human nature.

This is not the fault of pull requests but the choice of the developer to delay the decision to invite feedback via a PR until the last moment. So it’s important to acknowledge that a consequence of opening pull requests early in the development process is a reduced likelihood of over-protectiveness.

Attitude is also important. Even if a pull request is created late in the development process, it is possible to avoid over-protectiveness with a good culture that encourages constructive criticism and teamwork. There should be an expectation that each pull request receives close scrutiny from peers. Equally, when feedback is provided, encouraging language should be used.

Who should merge the PR?

There are various practices that are followed when it comes to merging pull requests. In some teams, a convention is followed whereby receiving one or more shipit comments gives the right for the author to merge. An alternative practice dictates that the author can never merge. Only when their teammates have approved the PR does one of them merge it. Obviously the complexity of the change dictates how many people need to be involved in giving the +1 for a merge.

Whichever approach is followed, it is important that at least one other set of eyes sees the change and approves it before the merge is carried out.

Conclusion

So, there you have it. That’s my take on using pull requests to collaborate effectively. I think they offer a fantastic opportunity provided they are used intelligently within a culture of encouraging teamwork.

Tags: pull requests collaboration | Comments (0)