Project grammar

I sometimes wonder if the field of Project Management spends too much time emphasizing management of the noun vs management of the verb.

PROject (n)
A temporary group endeavour undertaken to create a unique product, service or result.

proJECT (v)
To thrust forward.
To estimate or predict based on present data or trends.
To direct one's voice so as to be heard clearly at a distance.
To cause an image to appear on a surface.

Isn't project management really about managing the actions/decisions necessary to achieve a desired outcome?  Focusing on projections of the desired outcome, based on present data or trends, is how we truly manage risk on a project; the risk of shipping the wrong product, the risk of shipping a low quality product, and the risk of shipping late or not at all.  The first key to mitigating these risks, of course, is to ensure that the present data and trends used to make the projections are both valid and valuable.  The second key is to update the present data regularly, accordingly update the projections, and then make decisions.  These concepts describe two of the primary elements of iterative/incremental development.  Successful proJECT management is much more akin to causing an image to appear or hearing clearly at a distance than it is to simply endeavouring to achieve a result.

When you're up to your ass in alligators ...

Most teams use some sort of defect management tool which allows them (and other interested parties) to record defects along with meta-data about the defect such as severity and priority.  Severity is usually an objective value but priority is subjective.  For instance, severity is usually defined in terms like 'High - Results in crash or data loss', 'Medium - Work around available', or 'Low - Cosmetic' etc.  High severity defects are sometimes low priority and sometimes low severity defects are high priority.  For instance, the misspelling of a company name may be low severity but high priority while an application crash generated from a situation that is unlikely to be encountered by the customer may be low priority due to a cost/benefit assessment.  Team members can assign severity but usually only a Product Owner is responsible for assessing the value of addressing (or not) a defect.  This is because usually those people who are responsible for fixing defects are the same as those responsible for adding value via new functionality.  Product Owners need to be able to prioritize the value of fixing a defect against adding new functionality.

When your team is faced with 'draining the swamp' of legacy defects, they must face the need for effective defect prioritization.  The first step in addressing this issue is assessing whether the 'defects' are indeed defects (functionality that does not behave as previously agreed upon by all involved parties) or enhancements (behaviour that has not yet been designed/implemented/tested).  In my view, if the team did not agree that some behaviour was expected, designed, implemented and tested, the behaviour is an enhancement to the current functionality.  Once the enhancements have been distinguished from the true defects, those enhancements can be turned into Stories and prioritized just like any other Story which adds value to the application.  The remaining defects then need to be prioritized in terms of the value they prevent the application from maintaining.  Something of value used to work properly and now does not do so.  How important is that to the success criteria of the product and/or release?

In order to mitigate risk on a software development project, one of the principles of Scrum is that teams try to focus on delivering the next most valuable functionality while keeping the product potentially shippable.  We are to work on the next most valuable functionality in order to insure that if we run out of money or time (and we will) that we have created the most value for the money and time expended.  This should apply in the world of defects as well as enhancements.   Often the difficulty with doing so is that the number of defects in various priority queues are so large that it is difficult to assess whether the team is working on addressing the most valuable defects at any given time.  If 100 defects are denoted as High Priority but we can't address them all in one iteration, which ones shall we address to accrue the greatest value?

In most defect management tools there are usually priority choices like Must Fix, High, Medium and Low.  These classifications are perhaps arbitrary in that the only important thing about them is that each is related to the other by a higher or lower value. However many 'buckets' exist, treating them as static is not an effective mechanism for executing against priority.  Prioritization is a subjective exercise and usually prone to changes based on business conditions and newly reported issues from the field.  To that end, the highest priority queues should constantly be being emptied by the end of any given iteration.  This means that Product Owners must be vigilante about either 'promoting' defects from Medium to High and from Low to Medium (which seems like busy work) or simply limiting the highest priority bucket to a queue size that the team is likely to be able to completely address.  The key here is always making it apparent to the team which defects are the most important to fix in any given iteration.  Very often I see queues of 100's of High priority defects and 10's of Low priority defects.  This is usually the exact reverse of what we'd like to see!  We are much better at managing smaller queues … for instance queues that we can see and contemplate in their entirety.

In order to keep a product potentially shippable at the end of each iteration, some teams adopt a Task Priority list describing a working agreement about the team's default task priorities:
a) Fix any build/install issues (if we don't have a build/install, we can't test)
b) Fix any automated tests (if our tests are broken, we don't know what works and what doesn't)
c) Fix any regression defects (if we have open regression defects then we have likely regressed in value)
d) Fix any current iteration Story defects (standard practice to meet acceptance criteria)
e) Implement new Stories

a) and b) above certainly keep the product from being in a known potentially shippable state while c) keeps the product from maintaining a known value.  Issues associated with d) and e) above are about adding incremental value to already valuable software.  On large multi-team projects, distinguishing those issues keeping the product from being potentially shippable from issues of maintaining or adding value can help with queue size.  For instance Must Fix can incorporate a) and b) while c) can be distributed across High Medium and Low priorities.  Note that in this context, Must Fix is not associated with a value judgement, it is associated with a fairly common Definition of Done that teams use to help them keep a product in a known state.

Of course the best solution to the problem of how to effectively drain the swamp is to prevent the swamp from forming in the first place.

Pragmatic Moneyball

So much of the premise of the movie 'Moneyball' is rooted in the concepts of successful product management leadership.  Once Billy Beane has announced his need (as the General Manager of the Oakland A's) to 'adapt or die', the crux of the solution to his problems rests in discovering the REAL problem.  

After losing their star players to larger contracts offered by richer teams, the scouts and coaches of the Oakland A's begin to discuss how to replace those star players using their standard, time honoured criteria.  When Billy questions their methods the following conversation occurs:   

"We're trying to solve the problem."
    "Not like this you're not. You're not even looking at the problem."
"We're very aware of the problem."
    "OK good, what's the problem?"
"We have to replace 3 key players."
    "Nope.  What's the problem?"
"Same as it ever is, we have to replace these guys with existing players."
    "Nope, What's the problem?"
"We need 38 home runs, 120 rbis and 37 double plays to be replaced."
    "Nope.  The problem we're trying to solve is that there are rich teams and there are poor teams … its an unfair game."

The remainder of the film deals with a product strategy to address the real problem at hand; competing in a league where the payrolls of teams are vastly disparate.  The scouts and coaches were trying to solve the apparent symptoms of the problem but not the problem itself.  The strategy Billy Beane employed is spoken by his assistant, Peter Brand:

"Baseball team owners think in terms of buying players.  Your goal shouldn't be to buy players but to buy wins and in order to buy wins you need to buy runs."  

This product strategy leads to a different way of thinking about what's valuable about a baseball player.  Billy and Peter acquire players whose primary value lies in their ability to get on base (by walk or by hit) and therefore be more likely to score runs.  They are able to ignore other negative factors about those players and as a result acquire players not attractive to the big market teams still using traditional selection criteria.  This in turn allows them to field a competitive team at a fraction of the cost of their competitors' teams.  While not winning the World Series that year, the Oakland A's implementation of this product strategy changed the way Major League Baseball teams managed their teams from that year forward.

Applying the Dreyfus Learning Model to Focus Your Coaching Approach.

Agile2011-badge.jpg

Over the past five years, Jaron Lambert (@jaronlambert) and I have helped several companies and dozens of teams transition to agile product development processes.  In working with the myriad of teams and people, we've learned that the best way for us to help people learn agile principles and techniques is to use the Dreyfus Skills Acquisition Model.  This model employs the usage of non-situational rules for novice practitioners.  The adherence to these rules promotes the transition to an advanced stage of learning by providing the student with a foundation for recognizing patterns and principles for application in new situations.  We've discovered (as the model predicts) that skipping this stage of learning can lead to problems absorbing and implementing the philosophies and principles of the agile manifesto.  To that end, Jaron and I have collated a set of rules we have found useful in working with Novice teams.  This is a living collation and we'd be very happy to hear what other coaches and ScrumMasters (or others) think does and does not help their teams' learning process.

Jaron and I will be speaking about the 5 stages of learning that teams traverse (not just Novice) according to the Dreyfus model as they learn agile concepts and philosophies.

Environment

  • Co-locate where ever possible, set up your teamʼs space for optimal teamwork.
  • Maximize face-to-face communication; minimize email communication within the team.
  • Minimize distractions. Remove or minimize anything that distracts the team from finishing the work that they committed to (i.e. completing all the Stories in the iteration plan). Inform the Program/Project Manager of any distractions that canʼt be removed directly.

Roles

  • The Product Manager is the “messenger of the market” and articulates why we should spend time/money on any development. Product Managers describe what the market needs in a Market Requirements Document (M.R.D) and are responsible for describing those needs in User Stories.
  • Product Managers set the business priority and help define acceptance criteria during iteration planning, answer questions from the team during the iteration, and accept stories before the end of the iteration.
  • Program/Project Managers (Scrum Masters) shepherd the process. They provide the Product Manager with all currently available information so that the Product Manager can make informed and timely decisions. Program/Project Managers answer process questions from the team.
  • Program/Project Managers facilitate removal of any/all roadblocks for the team, ensuring that someone on the team is responsible for any roadblock and keeping track of unresolved roadblocks.
  • All other team members are responsible for delivery: reliably delivering quality software solutions (i.e. implementing stories).
  • All other team members provide the Product Manager with solution options (alternative ways to implement solutions to stories) with associated costs.

Release Planning

  • Review the MRD to ensure that the entire team has an understanding of the goals of the release (i.e. Problems, the users who have them, and the situations under which they have them)
  • Write stories using the template: “As a {persona} I want to be able to {do something} so that {some goal is achieved}”, See “Writing Stories”, Chapter 2 from User Stories Applied by Mike Cohn.
  • Size all the stories in the backlog using a modified Fibonacci sequence (0,0.5,1,2,3,5,8,13,20,40,100). The size of a story represents the overall complexity, difficulty, and effort to complete the story.
  • Add up the story sizes to get the total story points in the release.

Backlog Grooming

  • Product Manager grooms the Release Backlog for their product. They keep it organized and prioritized, and add or remove stories so the Release Backlog always describes expectations for the release.
  • Team Members are responsible for sizing the stories in the release backlog, and breaking stories from the Product Manager into smaller stories that deliver value within an iteration (see “Twenty Ways to Split Stories”, http://xp123.com/xplor/xp0512/).

Iteration Planning

  • The Product Manager decides priorities for the team to work on in the next iteration. Team selects the stories they can complete within the iteration and decides how they work on the tasks during the iteration.
  • Only plan to work on sized stories.
  • Only plan to work on stories with agreed upon acceptance criteria (discuss and agree on the criteria, and document it during the planning). Team breaks each story into tasks and clearly defined acceptance criteria.
  • Agree on the definition of ʻdoneʼ for the team (update it whenever necessary).
  • Agree on ʻRules of Engagementʼ (update them whenever necessary).

Iteration/Sprint Execution

  • The teamʼs priority of work:
  • Keep the build/install working and testable (getting a brokenbuild/ install, and getting broken automated tests working, is always top priority).
  • Keep existing functionality working (fixing defects with functionality that worked before the current iteration is #2 priority).
  • Keep the functionality being built this iteration working (fixing defects on this iterationʼs stories is #3 priority)
  • Build out current stories (implementing new functionality is #4 priority)
  • Philosophies to continually keep in mind:
    • The fewer stories in progress the better (3 things shippable and 2 things unstarted is better than 5 things ʻalmost doneʼ).
  • The fewer tasks in progress thebetter.
  • Donʼt work on anything unless youʼve already agreed with your team mates how it will be tested and how you will know if youʼve been successful.
  • Donʼt add a new story to the current iteration mid-iteration (unless every other story is complete).
  • Donʼt split a story mid-iteration. Teams need to consistently complete the stories they committed to and establish a velocity. While they are gaining experience doing that, they will learn better habits faster by recognizing 0 points for incomplete stories and reflecting on how to better deliver on the iteration plan.

Daily Standup

  • Each member of the team updates the other team members on progress against the stories they are working on:
  • What did you accomplish yesterday (since the last standup)?
  • What do you plan to accomplish today (before the next staandup)?
  • What is getting in your way?
  • What is your latest estimate of how much time is left on your current task(s)?
  • Only people with work assigned in the iteration should speak.
  • Topics outside these questions should be addressed outside the Daily Standup.
  • Does the plan need to change as a result of the above? If so, change it now!
  • If anyone doesnʼt have enough to do today, decide what they will do at the standup.
  • All members of the team watch for “bad smells” in their scrums, and mention any apparent infractions. All members of the team work together to remove “bad smells” from their daily scrums.

Demos

  • Demonstrate only what the team accomplished (i.e. Stories the Product Manager has already accepted).
  • Record any issues, bugs, or enhancements that come up (and assign them or add them to the backlog after the demo).
  • Product Manager accepts (or decides not to accept) any remaining stories.
  • Avoid troubleshooting, discussing solutions, brainstorming ideas, exploring functionality, or anything else that takes away from clearly demonstrating the work that was completed this iteration.

Retrospectives

  • What was the teamʼs velocity this iteration? Has the team established a consistent velocity? If so, what is it? If not, what is preventing the team from establishing a constant velocity?
  • What did we say weʼd improve / stop doing last retrospective? Did we?
  • Each member of the team has a chance to say (focus on the process, not the people):
  • What went well / what should we keep doing?
  • What could be improved / what should we stop doing / what is holding us back?
  • Team identifies the most important item(s) or issue(s) to focus on next iteration (1 or 2 items / issues is enough).
  • Change or add to “Rules of Engagement”?
  • Modify the teamʼs definition of “Done”?

Agile at the Masters

Watching today's Masters Golf Tournament in Augusta, Georgia I was struck again by something I've noticed before.  In the game of golf, when more than one person has the same score, the person who has completed the most holes is listed as 'ahead of' or 'above' the person or people with that same score.  Evidently, while having more holes remaining is surely an opportunity to get a better score, golf aficionados know that those extra holes are more often than not an opportunity to lose a stroke.  They seem to place more value on what has been accomplished for certain rather than what the potential for the future might hold.  

Thoughts on 'Potentially Shippable'

Scrum calls for the delivery of a 'potentially shippable' product increment at the conclusion of each iteration.  The reason it is 'potentially shippable' (rather than simply 'shippable') is that ideally it should only be a pure business decision as to whether enough value has been accrued to warrant actually shipping.  Therefore, the functionality that is exposed to the user works as intended based on the implemented Stories/Acceptance Criteria which in turn presumes that the quality is fit for purpose.

The value of keeping software in a 'potentially shippable' state at regular intervals is twofold: a) a real/tangible indication of progress for use in making date vs scope decisions and b) the ability to garner meaningful feedback at regular intervals.

If the software is in a 'potentially shippable' state, progress towards the end-goal is based on working, tested workflows/functionality implemented in software and not based solely on overall task estimates.  If the software is in a 'potentially shippable' state, feedback from existing customers, potential customers, and internal stakeholders can be meaningful.  Otherwise feedback can be, at worst, invalid, and at best confusing.

One of the goals of iterative/incremental development is to minimize the difference between 'potentially shippable' and 'shippable'.  Ideally it is simply a business decision whether there is enough value to actually warrant shipping.  In practicality, however, for many teams there are activities that they need to perform prior to actually releasing that they are unable to perform every iteration.  In some cases, the totality of all manual, automated and performance acceptance tests possible and/or necessary to execute and analyze in order to fully assess whether a given build is 'shippable' takes on the order of weeks to months.  In other cases, there is just too much legacy code which is not covered by automated testing to allow for the creation of something considered 'potentially shippable' within any given iteration.

With this in mind, teams need to be able to focus on those activities that they can accomplish inside an iteration which will best lead them to having confidence that the iteration backlog Stories work and that previously implemented workflows still function correctly.  Doing so will lead to a smaller gap between 'potentially shippable' and 'shippable'.  If a significant part of the cost of change in a code base is the uncertainty created by the change and our inability to validate (in a timely manner) that our workflows have not been inadvertently effected, then we should always be striving to minimize the time it takes to do that validation.  Validating quicker leads to finding and fixing problems quicker and cheaper.  Automate, automate, automate.

IDEALLY:

- acceptance criteria outline the circumstances under which each new workflow functions and the associated expected results
- if the acceptance criteria are met, and we have proved that the acceptance criteria for previously accepted workflows continue to be met, then we are potentially shippable.

PRACTICALLY:

- acceptance criteria for any given Story need to include regression tests for previously working functionality (either manual or some subset of long-running automated tests) which the team has assessed are likely to have been effected by the code changes necessary to complete the Story in question.

- alternatively, the Definition of Done can be altered to include a statement about the inclusion of relevant, focused regression tests which are either performed manually or are a subset of an existing long-running test automation suite.

- those manual regression tests then need to become an ongoing part of the automated test suite

The usual objection to this approach is that it means that teams apparently deliver less in an iteration.  This of course is a red herring as the teams were never actually delivering as much as they thought in an iteration because the regression testing necessary to deliver functionality was hidden in the 'stabilization/hardening' period prior to release.  Moving that regression testing forward moves teams closer to the ideal and should lead to shorter stabilization/hardening periods.

I'm often asked "How do we measure if we are 'potentially shippable'?"  My response to this is generally the same, "You're 'potentially shippable' if the software behaves the way you say it does."  Without some way to adequately describe (and ultimately test) this behaviour, it is difficult to know if you are 'potentially shippable'.  This behaviour is, of course, described in stories and their respective acceptence criteria.

Spike ... Do the Right Thing. [Redux on Aug. 30 post]

Doing the right thing is often context dependent.  I've had occasion with my current clients to discuss  the concept of Spikes and how they are treated by numerous teams.  I've realized that where a team currently resides in the continuum of learning iterative/incremental development has quite an effect on how I discuss this topic.  The current industry norm is to treat a Spike as a type of Story.  This has been reinforced by web-based tools limiting trackable items to Stories, Tasks or Defects.  I've seen novice teams struggle with this notion and here's why:

  • Summarily, the purpose of a Story is to describe some piece of functionality valuable to, and testable by, a customer.
  • The purpose of a Spike is to investigate the solution options and feasibility of addressing a particular problem space or Story.

Stories should have direct customer value as well as a size estimate (representing effort, uncertainty etc) from the team, while Spikes have no direct customer value and are generally time-boxed to no more than 2 days.

From this perspective, Spikes should not be called Stories because they violate the spirit of the principal purpose of a Story (to express something of value to a customer).  I think much of the confusion on this topic stems from our desire to want to name things based on how we manage them.  I think people would like to manage Spikes in the same manner they manage Stories but then assume that means the Spike should be called a type of Story.

I've seen novice teams want (naturally) to do things like give the 'Spike Story' a size and then directly equate that size to the length of the time-box.  This tends to negate the value of using Story Points for sizing Stories in a backlog.

Conversely, if a team treats a Spike as a Story and gives it a size of zero, that is inconsistent with the fact that they are spending time on it.  It also grates against those who look for Stories which we "get for free" (and therefore give a size of 0) as a result of some overlapping work on another Story.  These are subtleties which a mature team understands but a novice team, new to Stories, can find challenging.

So how about Spikes as Tasks?

I believe Spike activities have much more in common with Tasks than Stories.  However, there are inconsistencies and pitfalls here as well.  If a team treats a Spike as a Task inside a relevant story for an iteration, when the Task is complete, then the relevant Story should technically be removed from the iteration and returned to the release backlog (unless it is determined that the entire story can be completed before the end of the iteration) for future prioritization along with at least some of the resultant child/replacement Stories.  Depending on the the size of those resultant Stories (and the team's velocity) some of them may remain in the current iteration.  If we emphasize striving for completing all stories initially planned in an iteration we end up having to make exceptions for Stories containing Spikes.  Another pitfall is that when a team employs "yesterday's weather" for their next iteration, they need to ignore the size of a Story containing the Spike task.   Again, these things may be easily comprehended by a mature team but perhaps are contentious for teams transitioning to Agile methods, inexperienced in the use of Stories, and looking for absolutes (and THAT is a subject for a future post!).

How about neither Story nor Task?

In my experience, Spikes are generally an extension of Backlog Grooming and as such could be considered release overhead.  Spikes are generally not included as part of the release backlog but they likely result in a set of more defined Stories which may be part of the release backlog.  Perhaps it would be better for some teams to simply reduce team member availability during an iteration to account for the Spike time-box?  After all, we do this for things like Backlog Grooming meetings.  The one drawback to this approach is that there would be no explicit/transparent way for the team to manage the time spent on the Spike activity.

So why isn't a Spike just a Spike?

While the notion of a Spike overlaps the notion of Story in that both have a goal; and the notion of a Spike overlaps the notion of a Task in that both represent an activity, I think the differences between the 3 are signficant enough to warrant maintaining/managing a separate entity - 'Spike'.  This is easily accomplished on a traditional index card wall, but is usually explicitly prevented when utilizing a web-based management tool.  I believe this is one of the mistakes we as a community made when we transitioned to using these tools.  When we were using index cards we simply called it a Spike and off we went.  We didn't have to choose to create that entity as a Story or a Task so there was less confusion. 

Of course this is all just semantics ... Story, Task, Spike, Backlog Grooming ... what's the big deal?  The answer to this lies in the fact that we use words to communicate intent and without some consistency in the meanings of these words, it is difficult for people new to the concepts to keep it all straight.  The English language itself is full of inconsistencies and similarly, it is not until one is familiar with the patterns that one can grasp and remember the exceptions.

In the end what do we care about?

I really only care that the size of the release backlog is consistent with the team's current understanding of the backlog, and that the team's velocity actually represents the rate at which they may be able to reliably and sustainably address prioritized items in that backlog.  As Spikes occur, the resultant child/replacement stories will likely affect the cumulative size of the release backlog and the product owner can make the relevant business decisions with current information.

So depending on the individual team and their circumstances, any one of the 4 options might be the right thing.  If you're using a web-based tool (that limits your trackable items to Stories, Tasks and Defects) to manage your iterations and your team is relatively new to the use of Stories, consider treating Spike activities in the same way you treat other iteration overhead activities.

The Test

I've spent a lot of time over the past year thinking about and working with the Dreyfus learning model and the concepts of Shu-Ha-Ri as they apply to the evolution of agile software development teams.  During that time, I've also seen that much has been written about the Nokia Test.  This test, developed by one of the co-creators of Scrum, aims to assess a team's level of adoption of Scrum practices and principles.  Specifically this test was developed to help teams avoid the traps of 'Scrum but'.  Scrum is a fantastic approach to software development that I heartily endorse but I'm not sure that assessing adherence to the approach should come before assessing a team's ability to deliver what the business needs.

Rather than worry about whether 'we are agile', teams should primarily be concerned with whether they are providing the business with what it needs: a continuous flow of potentially shippable functionality and information pertaining to a realistic rate of progress which can be used to make timely business decisions.

How about asking these three questions to start instead:


  1. Does the team regularly deliver potentially shippable and incrementally valuable functionality at the end of each 2-4 week iteration?

  2. Can the team's product owner, given the team's historical velocity, predict with a satisfactory level of confidence, what value from the Release Backlog is likely to be shipped by a particular date?

  3. Is the team actively improving on 1 and 2 above?


If a team cannot answer 'Yes' to all three of these questions, using Agile techniques (and specifically following the Scrum framework) will likely lead the team in a direction consistent with one of the primary reasons to use agile techniques - project risk mitigation.

If the team can answer 'Yes' to all three of these questions, applying Agile techniques (and specifically following the Scrum framework) will likely lead the team to an increase in velocity.

Golf, Poker and Software

Last week I heard an interview with a  golf psychology coach who had recently added poker psychology coaching to his repertoire.  Golfers and poker players alike hire him to help them deal with the psychological aspects of their respective games.  Predominantly his clients are dealing with how to prevent golfers from 'choking' and poker players from 'tilting'.

His approach in both cases is to narrow the skill gap a player experiences between their 'A' game and their off-game which they can experience under stress.  Players who must perform under pressure tend to revert back to what is known as procedural memory - actions which are performed when the brain is no longer functioning normally.  The skill level captured at the procedural memory level governs how you will react when stress shuts down normal functioning of the brain.

Essentially, the best players under pressure are those whose skills in their off game are streets ahead of their competitor's.  How these players achieve this gap reduction is through repetition of the necessary skills until they become second nature and therefore part of procedural memory.  An example of things that move to procedural memory are activities like driving a car.  How many times have you arrived home, and realize that you can't really remember the details of the trip?  You were using skills at the procedural memory level which allowed your mind to be elsewhere.

This is interesting to me due to the work I've been doing with the Dreyfus skills acquisition model for assessing the skill level of agile teams.  It struck me that software teams (and individuals) react under date-related pressures in a manner that is governed by similar forces - if quality agile techniques are not second nature to the team, they tend to revert back to old, less than desirable ways.  For most people this transition of skills to procedural memory takes disciplined, directed effort.  Sports and game players must practice endlessly before they are expected to perform during a game.  Software developers must practice as a normal course of action; it's always game day in software development.

The Bus

In Jim Collins' "Good to Great" he touched on great leaders taking steps to ensure that the wrong people were off the bus and that the right people were in the right seats on the bus.

In his follow up, "How the Mighty Fall", Collins explores how once-great companies disappear into oblivion and how some manage to turn their decline around.  Once again, the first step that leaders take in trying to resurrect a company involves the concept of 'the bus'.  Here's how you know whether the right people are on the bus:

  • The right people fit with the company's core values.

    • You don't figure out how to get people to share your values ... you HIRE them based on your shared values.



  • The right people don't need to be tightly managed.

    • The right people are self-motivated and self-disciplined.



  • The right people understand that they don't have 'jobs'; they have responsibilities

    • These people can articulate that "I am ultimately responsible for ...".



  • The right people fulfill their commitments.

    • They take commitments seriously and thus are careful not to over-commit.



  • The right people are passionate about the company and their work.

    • Nothing great happens without passion.



  • The right people display 'window' and 'mirror' maturity

    • When things go well they point out the window to others, when thinks don't go well they point to the mirror.




My experience has shown me that having the right people on the bus is the most critical part of creating and maintaining a successful software development organization.  Keeping the wrong people on the bus has enormous risks associated with and it is always more cost effective to remove people from the bus as soon as possible rather than keep them around for perceived short term gains.

The Case for Competition.

Leo Laporte (@LeoLaporte) and Jason Calacanis (@JasonCalacanis) recently hosted an interesting podcast (#TWIT Episode 206) wherein Jason asserted that Apple should be vilified for having a business model based on its closed platform and anti-competitive nature.  He encouraged people to give up use of their iPhones in order to protest what he saw as an abuse of power on Apple's part.  Jason followed that up with an article entitled 'The Case Against Apple' where he essentially describes his evidence of the anti-competitive nature of Apple.  He finishes the article with:
Apple will face a user revolt in the coming years based upon Microsoft, Google and other yet-to-be-formed companies, undercutting their core markets with cheap, stable and open devices. Apple’s legendary comeback ability will be for naught if they don’t deeply examine their anti-competitive nature.

What I think Jason is missing here is that Apple 'fanboys' (I am admittedly one of them) have been prepared to pay a premium for a good user experience and we're prepared to live in a closed environment in order to preserve (for now) that good user experience.  The problem here is that we don't seem to be able to get that same level of user experience elsewhere.  Is that anti-competitive?  Windows has, in the past, been a miserable experience.  There are some technical reasons for this but there is also the primary reason that Apple appears to place more of a premium on producing products that WORK.  Do they work 100% of the time?  No, but they do a damn sight better than most of their competitors.  Is that anti-competitive?  I understand, for now, that part of the price for this is a closed platform.  I wish that wasn't the case, but currently I value gear that works over open platforms.  If that's part of the kool-aid then so be it.  That situation will change when a good user experience becomes a commodity.

I look forward to the day when 'Microsoft and Google undercut Apple with cheap, stable and open devices' WITH GREAT USER EXPERIENCES.  Until then I will continue to reward the company who apparently has realized how much I value the fact that my IT gear works the way I want and need it to work.  Isn't that what competition's all about?

It's ALWAYS been the problem!

When I started working full-time with Agile development techniques in 2004 I also had the good fortune to start working with the concepts of Pragmatic product management.  What quickly became apparent to me is that the two approaches in their respective fields were beautifully suited to one another.

By articulating the market problems, the nature of the people who have them, and the circumstances under which they experience those problems, Pragmatic product managers communicate to development teams what is really necessary to build great solutions; context.  We've all heard the stories of how the software industry's predilection for waterfall methodologies from the 1970's through the 1990's led to an inordinate amount of 'failed' projects.  Early proponents of agile development techniques ascribed those failures to the waterfall process itself, rightly arguing that performing development tasks (specifically testing) in phases and a lack of iterative feedback were the biggest causes of these 'failures'.  However, I would contend that just as large a reason for those 'failures' was product management's inability to communicate what was actually required for those projects to be successful.  Too much emphasis was placed on building to a specification which had often been created and disseminated without providing sufficient context to the people responsible for providing solutions.  Because of this lack of context there was no framework for negotiation with a customer or stakeholders and therefore much of what was built was not actually required to meet the customer's needs.  This resulted in unnecessary delays and cost overruns at a minimum and often the creation of ineffective solutions which quite rightly were considered 'failures' regardless of whether they met the specification.  Even if the solutions were built iteratively, the focus on the specification, rather than the problems in context, might very well have led to the same ineffective solutions.

Many software development companies do not have the luxury of having real-time access to actual customers.  Nor is the concept of building a product based on a single customer's requirements necessarily attractive.  Instead these companies rely on their product management organizations to represent the needs of customers within a market.  There is a hierarchy of proxy which is created as a result.  The customer is a proxy for the market and the product manager is a proxy for the market.  If the product manager is too busy performing strategic tasks and does not have time to fulfill his product ownership duties for the team then a proxy of a proxy is incurred with all the risk that entails.

All too often, agile projects rely on an internal 'product expert' to act as the team's Product Owner.  The danger of this is that the product expert understands the existing incarnation of the product very well but may not have the viewpoint of a customer who has issues with the product.  Properly implemented agile development techniques usually provide the transparent and tangible progress necessary to make informed business decisions.  However, without a clear sense of the relative value of the problems to be solved it is difficult for the team to understand and embrace those business decisions.  This is where the value of the MRD resides; providing the context necessary for a Product Owner to make sound decisions throughout the development process on behalf of the product manager.

In the end agile product ownership is a subset of all that a product manager is responsible for.  From a Pragmatic perspective, product ownership entails the 'Product Planning' tasks of the product manager: Market Requirements, Road Mapping, User Personae, User Scenarios, and Release Milestones.

The Pragmatic Market Requirements Document (MRD) focuses on market problems, the people who have them, and the situations under which they experience them (scenarios).  It also focuses on valuing/prioritizing those problems, people, and scenarios.  Hmmmm ... sounds suspiciously like a good start on a backlog of stories!  A story used in an Agile project usually takes the form of: A <type of user> needs to be able to <perform some action> in order to <get some value>.  This is essentially a statement of the circumstances under which a user experiences a problem.  I suppose technically it is an anti-problem as the real problem is the users inability to get the required value by performing the desired action.  Nonetheless there is a direct correlation between the primary elements of a Pragmatic MRD and a product backlog used in an Agile project.  Each scenario is prioritized using the following additional context: impact, frequency, and criteria.  The value to the persona of solving the scenario, the frequency with which the persona experiences the scenario, and the value to the to the Product Manager /Business of solving the scenario, all provide valuable context necessary for establishing business priority.  Once these scenarios have been prioritized, the product manager has a solid starting point for a product backlog.  Further, the entire team has a solid understanding of what problems they are trying to solve and for whom, and what the relative value of those problems is to both the market and the business.  With this context, the team is much better equipped to be able to produce solutions appropriate for the target personae and any decisions around solution options/modifications can be made in the context of the user and the business value.

agile2009_webbadges_speaker

Self-organizing teams at Quarry Bistro

My wife and brother-in-law own and operate a French/Italian bistro in our little mountain town.  They've been in the restaurant industry for 2 decades and are in their 6th year at Quarry.  Naomi runs the front of the house and David runs the kitchen.  Over the years, I've picked up a lot of knowledge about what it takes to run a successful restaurant.  It is a labour of love and not an endeavour to be taken lightly or for dreams of huge monetary returns.  I have been fortunate enough to be on the periphery of this and, apart from the odd dish washing shift, have had the luxury of being the unofficial taster of countless culinary gems paired with just enough wine.

During the peak season, when Quarry is busy and tables are turning quickly and constantly throughout the evening, Naomi has often said that the reason is that the team has an innate ability to do two things: prioritize their work and look out for each other.

In the dining room, the staff need to execute on priorities as inputs continue to change: new customers at the door, food ready to be served, tables in need of resetting, patrons ready to order, patrons telephoning for reservations, patrons finishing their first or second course, tables need to be cleared, patrons ready to pay.  All this has to be looked after in a timely manner while keeping the ambiance of the room relaxed and enjoyable.  While there are specialists on the team, each team member must be able to look after any of these tasks.  Further everyone has to be on the look-out for gaps in service which have occurred simply because at any given moment, only the top priorities are being looked after.  Upon noticing a gap, what separates the great team members from the ordinary team members is their ability to seamlessly pick up small tasks in the seconds in between their own top priorities.  There is an understanding that the overall room efficiency and success is everyone's responsibility.

Similarly in the kitchen, cooks glide past each other in tight quarters, knowing what motions to expect from each other and trusting that each person will be where they need to be given the current state of upcoming orders.  The sense of timing necessary to serve up to 60 patrons with 3 cooks is astonishing.  It means that those cooks need to be constantly evaluating priorities, executing on those priorities (usually in terms of time at temperature and time between courses), and when necessary helping a specialist with too many high priorities while tending their own tasks which, for a short moment, are better spaced than their colleagues'.  David has stated that when this culinary ballet is on form, the adrenaline rush is the reward.

One of the reasons all of this works is the way the team is compensated.  Tips are pooled as the philosophy is that if the team is working well together, all patrons' experiences will be improved and thus all will be more likely to tip more.  Further if the team feels one of its members is underperforming the team has a vested interest in either helping that team member or recommending that he/she be replaced.

A lot of what I've learned about great teams, I've learned while eating carbonara at the bar ...

But WHY, dad?

Asking 'why' is perhaps the most valuable of all questions both in life and software development / product management.  Being the father of two toddlers, I get asked 'Why?' a lot.  I only wish the team mates on some of the teams I coach were so inquisitive.  Knowing the 'Why' is often the difference between success and failure.  This is, of course why User Stories take the form they do: As a <type of user> I need to be able to <perform some function> so that I can <get some value>.

Both the type of user and the 'so that' clause provide valuable information in the form of 'Why'.  Without this information, a team could go horribly astray.  Asking 'Why?' helps provide valuable context.

Here's a simple example:

The user needs to be able to enter surface positional data.  Why?

The directional drilling engineer needs to be able to enter surface positional data so that he/she can perform anti-collision analysis.  Why?

The directional drilling engineer needs to be able to enter surface positional data so that he/she can perform anti-collision analysis, avoid drillstring collisions with offset wells and therefore avoid tremendous expense and/or injury/death of drilling personnel.

Clearly knowing who needs to enter this data, and why, puts a different context in front of the team around the requirement of simply being able to enter surface positional data.

My sons make me answer this question a lot and software development teams could learn a great deal if they if they continually did the same!

The fifth element.

The most overlooked of the 12 agile principles described alongside the agile manifesto is perhaps the hardest to come by - building teams around motivated individuals.

Build projects around motivated individuals.
Give them the environment and support they need,
and trust them to get the job done.


There it is ... that term four words in to the first sentence ... 'motivated'.

A very subjective word.  Some people are motivated by money, some people are motivated by the recognition of their peers, others by visions of climbing the corporate ladder.  What this term means in the context of agile principles though is something much more specific: 'motivated' by pride in their chosen profession and the desire to learn and improve.

All too often in large traditional organizations, software development team members simply have a job.  Occasionally they've simply been beaten into submission and can be resurrected by agile techniques.  Very often however, the people that survive for long periods in large organizations are those that are mediocre and are valued because they don't rock the boat.   These are the people who will, at best, go through the motions of implementing agile techniques while giving no thought to the principles and philosophy behind them.  At worst these people will simply figure out ways to 'game the system'.  Of course the ultimate responsibility for tolerating this behaviour lies with management.  Either way, having a motivated cast of characters is crucial to the success of an agile project.

Political waterfall

It's always amazed me the emphasis so many politicians put on 'staying the course' and not 'flip-flopping'.  Being able to 'stay the course' (regardless of what presents itself) is viewed as a sign of strength and resolve ... very macho and confidence inspiring.  Apparently, if you change your mind you must be weak and easily manipulated.  Yikes!

If only politicians (let alone the people who have the responsibility of voting them into office) realized that processing new information regularly, and changing course (or indeed 'flip flopping') based on that new information, may be EXACTLY what's required in order to continually deliver value to constituents.

I suppose my naivete is showing ...

But, Mousie, thou art no thy lane,
In proving foresight may be vain;
The best-laid schemes o' mice an 'men
Gang aft agley,
An'lea'e us nought but grief an' pain,
For promis'd joy!

till thou art blest, compar'd wi' me
The present only toucheth thee:
But, Och! I backward cast my e'e.
On prospects drear!
An' forward, tho' I canna see,
I guess an' fear!

Robert Burns (1785)

Adventures in distributed 'Agile'

The most successful implementations of offshore agile development are those where most of the team is co-located offshore.  As long as the developers and testers are co-located and have reasonably continuous access to the Product Owner/domain expert, there is a chance of success.  With that in mind there are still many factors which should be considered when contemplating utilizing off-shore development teams in a distributed agile environment.

COMMUNICATION BARRIERS

The time difference between your team and the offshore team is critical to establishing communication patterns.  If there is not sufficient overlap between the onshore and offshore working hours, at least one onshore team member may have to expand the overlap by working earlier in the morning or later in the evening.  Sometimes all that is required is someone moving their lunch hour 1-2 hrs.  Many employees of offshore companies rely on fixed-departure transportation to get to and from work (and travel great distances) and thus are not able to change their working hours.  Negotiate the possibilities with the offshore company upfront.

Many countries have statutory holidays which differ vastly from North America or Europe.  Further, some of these statutory holidays last for many days at a time rather than just the occasional Monday.  Of course some countries also have different work weeks.  This can either work in your favour or against you depending on the overlap.

Probably the largest barrier to success on a distributed agile project is the likelihood of language /cultural differences which adversely affect the flow of communication.  If you intend to use Scrum methods (daily stand-ups, sprint planning sessions etc) then it is important that you have confidence in the speaking/listening skills of both teams.  Working with offshore teams that emphasize ESL for their employees is a great advantage.

Video conferencing greatly enhances the communication experience for teams.  Being able to see faces and body language helps with the interpretation process that goes along with listening to team mates who do not speak your native tongue fluently.  Obviously the quality of the video conferencing makes a difference and to a large extent it is reliant on the network bandwidth on both ends.  Don't forget to investigate network reliability and bandwidth in the country/city of the offshore company.

However good the verbal communication is, understand that you will be relying more on Email/IM than you would otherwise like to on a standard agile project.  The good news is that many employees of offshore companies have stronger written than verbal communication skills.  Also, because of the prevalence of Instant Messaging outside of the workplace, having the team connected this way means that both parties can take advantage of asking quick questions over IM on either parties' off-work hours.  Being able to do this can save many hours for the project as there are fewer '15 hr' delays in having questions answered.

Nothing can replace the value of face-to-face communication.  Given that the offshore team has a reasonable level of verbal communication skills, it will still be necessary to work in-situ with the team with some frequency.  This will mean at least traveling to the offshore team's campus several times during the project.  Certainly early instances of release planning and iteration planning will benefit from having all parties co-located.  Further, having one or more members of the offshore team travel onshore periodically creates stronger relationships which are paramount to good communication when the team is separated.

While I routinely work with distributed teams where the Product Owner is geographically dispersed from the rest of the team, the domain knowledge of the team is highly critical to the project success.  Some offshore teams specialize in particular domain knowledge while others' are open to developing that knowledge in-house.  In either case, you need to be aware that your Product Owner's travel time and time dedicated to answering questions in a full and timely manner is inversely proportional to the domain knowledge of the off-shore team.  Ideally, offshore teams should have direct co-located access to their own domain expert.

AGILE MECHANICS

Practicing iterative/incremental development in a distributed environment requires modifications to some of the basic agile techniques.  Scrums, demonstrations, retrospectives, and planning sessions usually have to be altered to fit the situation.

In situations where the Product Owner is geographically separated from the rest of the team, but the time-zone overlap is sufficient, I've been able to hold daily scrums by speaker phone in front of a scrum wall.  The speaker phone in conjunction with a a webcam directed at the Scrum wall is sufficient for the Product Owner to follow and indeed participate in the Scrum.  In situations where the time-zone overlap is insufficient, the Product Owner has had to call in to the offshore Scrum on off hours and view a virtual Scrum board (e.g. Rally).  Both scenarios have proven successful.

In fully co-located projects, I like to perform Demos and Retrospectives on the same day as iteration planning.  Those activities generally take up most of a day.  As one has to factor in time zone differences in distributed agile projects, it becomes clear that these activities need to be split up over multiple days.  The demo is best done 'live' but certainly the iteration demo should not be the first time the offshore team receives feedback on functionality developed in the iteration.  Similarly the retrospective should be a chance to review feedback on the iteration already collected in the closing day or two of the iteration.  This way both the demo and the retrospective can be completed in under 2 hrs.  If the team can then be made aware of the subsequent priorities, they can work on the iteration planning in the onshore off-hours and the following day the team can reconvene for 2 hrs to review the iteration plan, review questions and answers for the Product Owner, and re-jig the plan if necessary.  During both of these sessions, Video Conferencing is invaluable.  Often the audio quality of a VC unit is superior to that of standard telephone audio due to the bandwidth dedicated to the VC unit.

In many cases, distributed agile teams cannot effectively use a standard Scrum wall covered in index cards.  A virtual scrum board (e.g. Rally, VersionOne, Team Foundation System) can prove to be almost as engaging for the distributed team.  Using a virtual scrum board, goes against many of the agile principles (mainly the fact that physical artifacts like the wall and the cards foster face-to-face discussion) but it is a necessary sacrifice once one embarks on distributed agile development.

In the end the most important aspect of any model for software development revolves around the ability to make appropriate business decisions based on the most current information at hand.  Companies should always be asking themselves 'Why do we want to offshore our development?'  Clearly cost reduction is the chief reason, but it is a common mistake to grossly underestimate the real cost of implementing successful distributed offshore development.   There needs to be budget for a moderate amount of travel, virtual backlog/scrum wall tools and ideally video conferencing.  The good news is that once you have made the investment in the teams, tools and techniques the cost of delivering projects declines.

Mis-adventures in Off-shoring

Many companies have dabbled in the practice of 'off-shoring' some of their software development.  More and more companies may be looking in that direction as resources become tighter and tighter in this currently shrinking world economy.  Many of the companies who've given it a try have made some pretty basic mistakes and others have found success.  Those that have made mistakes have generally missed the mark by NOT actually off-shoring development ... just off-shoring some element of the development process; usually testing.  The problem with off-shoring just the testing can be that unless the offshore resources are presented with either the test cases, a detailed functional specification, continual access to a domain expert, or some other form of impossibly detailed acceptance criteria (in conjunction with the relevant domain knowledge), it is difficult for the off-shoring team to meet their customer's expectations.  This is not to say that off-shoring testing cannot be successful, just that since meaningful testing relies on communicating the intent of the myriad of different usages of an application, any impediment to that communication degrades the delivery of meaningful testing; either in quality or timeliness.  Depending on the nature of the product development, if only one element of the SDLC is to be off-shored, the likely best candidate is test automation.  Having on-shore testers (working in conjunction with the Devs/Product Owner) producing test cases that can then be automated by others presents a situation where the reliance on off-shore domain expertise is diminished.

Games of Incomplete Information

Agile Software Development as a hand of No Limit Texas Hold'em?

Ever since I started playing various forms of poker "seriously" about 5 years ago, it has struck me that there are definite similarities between these card games and agile software development.  For the past couple of years I've been mulling this over and trying to see if the analogy works.  If so, perhaps it would provide another mechanism to illustrate the value of iterative/incremental development.  Here's what I've come up with:

In an agile software development project, the object of the exercise is to continuously deliver as much value as possible given the current conditions.  After every iteration, the Product Owner can see the value created and make decisions on whether to continue (and in what direction) or cancel the project.  In a hand of NLHE the object of the exercise is to continuously make decisions based upon incomplete information such that one can profit from the hand.  After every street, the rounder can see the value accrued, assess the risk, and decide to continue (and in what direction) or fold the hand.  All of these decisions are based on a myriad of inputs which affect the perceived value of proceeding.  As we shall see, Agile projects are analogous to individual hands of poker.

Pre-flop (Release Planning) Action


Prior to seeing any community cards (streets), one must make a decision based on several things: hole card value, position, chip stack, and opponent characteristics.  In order to proceed with the hand, one must wager at least the amount of the big blind.  Of course there are other possible actions:

Limp: This is equivalent to simply paying the 'cost of doing business'.  Enough information is present to justify starting the project. Based on what is known, while there are still significant uncertainties, there is some undetermined likelihood of success.

Raise: Enough clarity is present to warrant raising management expectations regarding the level of success that‘s possible.

Call a raise: While significant risk is apparent, there is enough information (hand value, relative position, participant characteristics) in place to warrant accepting the increased risk.  On a project, the fact that we have a  seasoned team, known technology and a known domain might cause us to call a raise.

Re-raise: This would occur only when the team believes that their estimation accuracy is sufficient to overcome most likely risks and is a signal of significant confidence.

(Re)Raise All-in: This translates to risking the entire project success on incomplete information regarding the hands of the opponents and the community cards. Committing to a specific combination of schedule, scope and resources at this time is analogous to raising all-in pre-flop.  This is fairly common in 'waterfall-based' software development projects.  While this action does occur in poker, it occurs only in specific circumstances involving calculated risk/reward ratios.

Fold: The project is canceled because the likelihood of receiving enough value from the project based on the starting information is extremely low.  If you're playing poker correctly, this event happens much more often than in software development although sometimes I think we should fold much more than we do in software development.

Flop, Turn, and River (Iteration Planning) Action


After the flop, the majority (3/5) of the common cards have now been exposed.  Some more information is now present for all to see and it is reasonably unambiguous.  Every exposure of a community card is analogous to an iteration planning session.  Whatever the plan was prior to the flop, turn or river, there may be need to adjust the plan based on the new information.  As with most things there is always some amount of risk (unless you've hit a royal flush ... 20,000:1).  The amount of risk is relative to hand value, position and chip stacks.   Further, taking into account the information (bet amounts and other non-verbal inputs) gathered on prior streets complicates or simplifies the issue depending on your point of view.  The options here are to check or bet in the face of no overt threats, or to call, raise or fold in the face of overt risk.

Check: Based on the newly presented information we are unsure of where we stand and may not be willing to invest more in the project if more adversity arises.  We are certainly not bullish about our strength.

Bet: Based on the newly presented information we believe we are strong enough to invest more in the project given that no added adversity has arisen yet.

Call: Based on the newly presented information, and a new overt risk (someone else is representing strength)  the team believes that it is worthwhile investing more in the project in the face of the new risk.  In poker, this is actually usually deemed a sign of weakness.  To some extent it is viewed as ignoring reality ... if you were really strong, wouldn't you re-raise?

Raise: Based on the newly presented information, and an overt risk (the previous bet), the team believes they are strong (confident enough in their knowledge of the situation, skills and domain) enough to overcome the risk.

Fold: Based on the newly presented information the team realizes that the risks far outweigh the likelihood of success.

The Showdown (Retrospective)

At the end of the hand, if we've seen it through, we have a clear indication of success or failure.  In either case we are able to look back at the information taken in during the hand and learn something about people's patterns and communication styles.  This can be very useful for future projects.

In the end, it's the experienced teams and card players that are able to raise their stakes and their games as they are able to continually adjust their plans mid-play.