Saturday 7 March 2015

Scrum - Part VIII : Long-term planning

 

Introduction


This is the eight part of the Agile blog saga. This time I'd like to turn to long-term planning and talk about how it can be reconciled with constantly evolving Agile priorities.

For the sake of this post, let's assume we're all project managers; if not by title, then by responsibility.

We have a requirement to deliver (and I'll dig out examples shortly), crossing several teams, with many technical dependencies.
As expected, all teams run Scrum, and duly perform all the ceremonies. However, if we look at the list of these ceremonies (to refresh your memory: it's backlog grooming, demos, sprint planning, retrospective and daily stand-ups), none help that much with our task.

The challenge here is that Scrum, as a discipline, tends to focus on iterations. There is a distance, however, from iterations and individual stories to something that customers, sales, marketing and execs will care about.

You might say - but these are user stories, and they have acceptance criteria! True, but they also have to fit into sprint. Thus, they need to be quite small. Thus, they have to be fairly trivial to be customer-facing. Let's throw a few examples of mini-projects that would not fit into a single sprint/team:

  • Add telemetry on customer usage of a widget
  • Implement two-factor authentication for a web service
  • Allow resellers to co-brand the product's UI
  • Add periodic back-ups of blogs to a cloud storage (this one is personal wishful thinking for the platform I'm using to type this blog)
None of these are huge, many person-year, projects. Nevertheless, I'd be surprised if any would fit to a single team and sprint. They cross front-end, and back-end, they do have a few weeks or more of development and QA, so at least some coordination and framework on top of Agile ceremonies is required.

Specifically, with our project manager hat on, we need to answer at least these basic questions:
  1. How do I draw the line between sprint items to customer deliverables?
  2. How do I track dependencies across teams' backlogs and ensure they stay in sync?
  3. Is the project on track? What is the ETA?
  4. What is the impact on commitments from changes in Agile backlog priority order?
The last question is especially important, since we cannot go back to drafting Gantt charts and doing waterfall-like plans. The world has changed, and it moves underneath our feet, so we need to anticipate and figure out quickly priority shifts in all teams involved. 

Of course, I don't delude myself in having the ultimate answer. Moreover, there is probably no such answer to start with - specific organisations and teams would find different techniques effective.
The rest of this write-up would go over techniques that worked for our teams with no guarantees or illusions of grandeur (which, actually, can be said about the entire series).

Epics

Let's start with question 1. To save you scrolling up and down, here it is copy/pasted:
How do I draw the line between sprint items to customer deliverables?

Well, I gave away the answer in the heading, but it's longer than just the word epic. There's any number of long and short definition of those. I'd go for:

Epic is a project that crosses several backlogs and/or several sprints.

To make things less theoretical, let's pick up the first example above, i.e. adding telemetry.
At a very high level, we want to track usage of our online multimedia converter; e.g. which countries/IPs they came from, what formats they converted to, and what browsers they've used.
When we break it down to teams and user stories, we get something like this:

  • Middleware team - Record IPs (1), client-side headers (2) and formats (3) requested via a new API.
  • Back-end team - Create a REST-ful service for adding new telemetry records (4). Create a new table for storing telemetry data (5). Add another API for summarising and fetching. (6)
  • Front-end team - Provide a new internal page which fetches telemetry data from back-end (7) and displays it in a format that can be sorted (8) and exported (9). 

There are other (and better) breakdowns, but our main topic here is epics and not architecting internal reports, so I'll cut a few corners.
Each of the number in parentheses maps to a single user story, and each can have well defined acceptance criteria.

In fact, there may be well more than nine user stories, for example "Define a REST-ful for posting telemetry records", a precursor to (4)".


Combining all these to a single epic for Telemetry at the very least enables each team to see the big picture, and allows us (remember, we're all project managers) to quickly figure out the work involved.


Dependencies

Let's move on to the next question: How do I track dependencies across teams' backlogs and ensure they stay in sync?

This is one area where, to my opinion, "classical" project management techniques still outperform Scrum. It's easy to ridicule huge Gantt charts with an incestuous labyrinth of dependency arrows, but the ability to prod a dependency at the bottom and see how far the house of cards flies is useful.

JIRA and other Scrum management tools do provide relationships between tasks. However, these are more akin to documentation, and deriving critical paths and impact from delays is, for better or for worse, left to us, humans.


Technique I've been using on some projects is sprint pre-planning (yes, it's an oxymoron and against classic Scrum practices, but bear with me). 

Taking our sample project, we could tentatively define the following:

Sprint 42:
  • Back-end team: Create a REST-ful service for data provisioning (4)
  • Front-end team: Design internal page user interface and prototype using mock interfaces
 Sprint 43:
  • Middleware team: Record IPs (1) and client side-headers (2) using the REST-ful service
  • Back-end team: Create table for storing telemetry data (5), and create API for fetching statistics (6)
  • Front-end team: Develop sorting (8) and data export (9).
 Sprint 44:
  • Middleware team: Record requested data formats (3)
  • Front-end team: Integrate with retrieval API (7), and validate end-to-end functionality.
  • System QA and release. 

To avoid all this complexity, you might suggest taking a developer from each group and creating a virtual feature team. Definitely an option, but it comes with strings attached, and I'll cover it in a future post.

Of course we operate in an Agile environment, and any of these tasks can be pushed back by the respective Product Owner(s). Even for such a minor project, the likelihood of everything happening as desired is small.
However, having these allows the (virtual) project manager to re-plan and see what the new release/target sprint would be.


Tracking could be done in a similar way: talk to the teams and agree on an "ideal" plan, and then track and adjust progress.

For example, let's say the back-end team has been diverted for production issues in Sprint 43, and could only create the table (5), but not the API (6). 
This is a perfectly legitimate situation and reason, so we re-adjust (assuming sprint 42 went to plan):

 Sprint 43:
  • Middleware team: Record IPs (1) and client side-headers (2) using the REST-ful service
  • Back-end team: Create table for storing telemetry data (5), and create API for fetching statistics (6)
  • Front-end team: Develop sorting (8) and data export (9).
 Sprint 44:
  • Middleware team: Record requested data formats (3)
  • Back-end team: Create API for fetching statistics (6)
  • Front-end team: Integrate with retrieval API (7), and validate end-to-end functionality.
  • System QA and release. 

 Sprint 45:

  • Front-end team: Validate end-to-end functionality.
  • System QA and release. 

Yes, it is all painfully obvious, and yes, it is a poor-man's version of traditional project management. Nevertheless, I've seen many people rely entirely on backlogs and spontaneous coordination, with not too great results.

When planning an ideal plan, I'd always budget for eventualities such as the one we had above. It would depend on what happens around the teams, but it would take a lot of convincing to commit on Sprint 44 to important people.

What epics are not

Occasionally epics are used just as themes or work buckets. For example, the team might have epics called: Refactoring, widget A, operational support etc.
These are not projects: they can go on indefinitely, and there is no target. But, hey - who said that my definition of epic is the right one, and the bucket definition is the wrong one?
Some explanation is required, and I'll give my reasons on why this can be done better and elsewhere.

1) Such buckets are hard to track. Let's say I have a mini-project: "Enhance widget A to provide telemetry". 
The widget A epic does not have a target release, so I can't track it there. I also can't create a new epic, since a single user story can't be assigned to multiple epics.
The widget A epic leaves me without any attractive options.

2) These buckets are like the undead. There can be only more of them - they never die. The team's board becomes a mausoleum of epics and it becomes harder to see the projects/epics that do matter.

3) Figuring out how much of team's work falls in each area (a noble goal) can be done using other means such as labels.

Summary

Scrum does not prescribe how projects are coordinated and managed. Some form of framework is required, and there is a variety of those. I went over epics, their pre-planning, adjustments and tracking. We also went over what epics are not.


In the next post, I'd like to talk about another long-term planning technique: task pinning.

No comments :

Post a Comment