Product Management

What's in a Title?

Much is made about the title of 'Project Manager' and the need (or not) for that position in agile software projects.  I've always felt that the role of Project Manager is different than the role of a Scrum Master even though the roles are often attempted to be filled by the same person.  While there may be some overlap in the roles, the Scrum Master is generally focused internally toward the team and the Project Manager is often focused on activities that affect the team but are external to the team.  A Scrum Master would be focused on guiding the team through the Scrum framework, helping the team improve their effectiveness, and removing impediments for the team.  A Project Manager is often tasked with ensuring that regardless of what is transpiring within the team, the team is aligned with current corporate protocols and constraints.  Ideally, of course, the entire organization is using an agile mindset and the role of Project Manager is subsumed by that of Scrum Master. In reality, at least as a transitional move, the two roles are still required during the early stages of an agile transformation.

The role of Product Owner and/or Manager on a sofware project should have the responsibility for making all the business decisions related to scope, budget and schedule because they are supposed to have the most understanding of the value created and the business priorities for the marketplace.  Other members of the team (and indeed of the organization) may have different opinions but it is the role of the Product Management discipline to make product business decisions.

In Oil and Gas Facilities construction, the Project Manager is the arbiter of ALL of those same business decisions.  It is from the roots of construction/manufacturing that software development took its first organizational steps and so it is not surprising that the title of Project Manager has been confused.

An Agile Mindset in House Construction

New Construction

One of the arguments purporting to negate the logic of agile principles comes from the analogy of building a house.  It is said that no one in their right mind would build a house incrementally - finishing a bathroom, then finishing a kitchen, then finsihing a bedroom etc.   There's no value in the house until the entire house is finished, which is why they are constructed based on task sequence effeciency: all structural followed by all plumbing followed by all electrical etc.

After a conversation with a colleague, Brent Walker, it became clear to me that building NEW software is not analagous to building a house, it's analogous to building a housing subdivision.  The subdivision is a system of houses just as software is a system of functionality.

Developers buy land and install basic infrastructure for the subidivision: roadworks, electrical, and waterworks.  They then build and market houses one at a time to accrue value on their investment, tieing in the basic infrastructure to each house.  They do NOT set out to finish all houses in the subdivision at once when the last finishing task (painting for instance) is completed on all houses.  They serially accrue value one finished house at a time.

In continuing the anaology, the finished software application is the housing subdivision and the incremental valuable story is the house.  Agile principles stress the importance of accruing incremental value by finishing stories one (or a few) at a time as early as possible rather than having them all partially finished at any given time (thus delaying value accrual).  The bedroom or kitchen in any one of those houses is more akin to a task necessary to meet the acceptance criteria of the story.


Many software development projects are similar to house renovations; you want to add value while still accessing the existing value.

In June of 2013 a major (>100 year) flood in our town caused us to consider renovating our basement.  We had to remove the flooring and strip the walls to the studs.  In so doing, we also decided to reconfigure the space from a project room and large common room to two bedrooms and a smaller common room.  Each area to be renovated required some framing, some electrical (lighting), sound insulation in the ceiling, drywalling, mud/taping, painting and cement floor polishing in order to be considered done.

After completing the electrical in all rooms and hanging the drywall in the first bedroom, the question arose as to whether effort should be spent finishing hanging drywall in ALL areas (both bedrooms and common room) before moving on to mud/taping, painting and floor-finishing or whether each room should be taken to completion in series.  Mudding/taping all at once, for instance, would minimize the incidents of mess and dust inherent with the drywalling process.  However, with only 6 weeks until Christmas, the entire basement could not be finished.  It was decided it was more important to have both bedrooms ‘usable’ (drywall hung and providing at least visual barriers) before Christmas to allow for the bedrooms to accomodate likely holiday visitors; the Minimum Marketable Feature given the time constraint.  The state of the common room in-between was less important and therefore effort was not spent hanging drywall in it.  After Christmas, it would be more important to finish the bedrooms prior to the common room even though that would likely require more more instances of drywall dust/mess.

Performing one task across all rooms has the apparent value of being more efficient as there would be less context switching and less instances of disruption but taking that route also DELAYS the principle value of each of the bedrooms; having private spaces ready for use by holiday visitors.

When you're up to your ass in alligators ...

Most teams use some sort of defect management tool which allows them (and other interested parties) to record defects along with meta-data about the defect such as severity and priority.  Severity is usually an objective value but priority is subjective.  For instance, severity is usually defined in terms like 'High - Results in crash or data loss', 'Medium - Work around available', or 'Low - Cosmetic' etc.  High severity defects are sometimes low priority and sometimes low severity defects are high priority.  For instance, the misspelling of a company name may be low severity but high priority while an application crash generated from a situation that is unlikely to be encountered by the customer may be low priority due to a cost/benefit assessment.  Team members can assign severity but usually only a Product Owner is responsible for assessing the value of addressing (or not) a defect.  This is because usually those people who are responsible for fixing defects are the same as those responsible for adding value via new functionality.  Product Owners need to be able to prioritize the value of fixing a defect against adding new functionality.

When your team is faced with 'draining the swamp' of legacy defects, they must face the need for effective defect prioritization.  The first step in addressing this issue is assessing whether the 'defects' are indeed defects (functionality that does not behave as previously agreed upon by all involved parties) or enhancements (behaviour that has not yet been designed/implemented/tested).  In my view, if the team did not agree that some behaviour was expected, designed, implemented and tested, the behaviour is an enhancement to the current functionality.  Once the enhancements have been distinguished from the true defects, those enhancements can be turned into Stories and prioritized just like any other Story which adds value to the application.  The remaining defects then need to be prioritized in terms of the value they prevent the application from maintaining.  Something of value used to work properly and now does not do so.  How important is that to the success criteria of the product and/or release?

In order to mitigate risk on a software development project, one of the principles of Scrum is that teams try to focus on delivering the next most valuable functionality while keeping the product potentially shippable.  We are to work on the next most valuable functionality in order to insure that if we run out of money or time (and we will) that we have created the most value for the money and time expended.  This should apply in the world of defects as well as enhancements.   Often the difficulty with doing so is that the number of defects in various priority queues are so large that it is difficult to assess whether the team is working on addressing the most valuable defects at any given time.  If 100 defects are denoted as High Priority but we can't address them all in one iteration, which ones shall we address to accrue the greatest value?

In most defect management tools there are usually priority choices like Must Fix, High, Medium and Low.  These classifications are perhaps arbitrary in that the only important thing about them is that each is related to the other by a higher or lower value. However many 'buckets' exist, treating them as static is not an effective mechanism for executing against priority.  Prioritization is a subjective exercise and usually prone to changes based on business conditions and newly reported issues from the field.  To that end, the highest priority queues should constantly be being emptied by the end of any given iteration.  This means that Product Owners must be vigilante about either 'promoting' defects from Medium to High and from Low to Medium (which seems like busy work) or simply limiting the highest priority bucket to a queue size that the team is likely to be able to completely address.  The key here is always making it apparent to the team which defects are the most important to fix in any given iteration.  Very often I see queues of 100's of High priority defects and 10's of Low priority defects.  This is usually the exact reverse of what we'd like to see!  We are much better at managing smaller queues … for instance queues that we can see and contemplate in their entirety.

In order to keep a product potentially shippable at the end of each iteration, some teams adopt a Task Priority list describing a working agreement about the team's default task priorities:
a) Fix any build/install issues (if we don't have a build/install, we can't test)
b) Fix any automated tests (if our tests are broken, we don't know what works and what doesn't)
c) Fix any regression defects (if we have open regression defects then we have likely regressed in value)
d) Fix any current iteration Story defects (standard practice to meet acceptance criteria)
e) Implement new Stories

a) and b) above certainly keep the product from being in a known potentially shippable state while c) keeps the product from maintaining a known value.  Issues associated with d) and e) above are about adding incremental value to already valuable software.  On large multi-team projects, distinguishing those issues keeping the product from being potentially shippable from issues of maintaining or adding value can help with queue size.  For instance Must Fix can incorporate a) and b) while c) can be distributed across High Medium and Low priorities.  Note that in this context, Must Fix is not associated with a value judgement, it is associated with a fairly common Definition of Done that teams use to help them keep a product in a known state.

Of course the best solution to the problem of how to effectively drain the swamp is to prevent the swamp from forming in the first place.

Pragmatic Moneyball

So much of the premise of the movie 'Moneyball' is rooted in the concepts of successful product management leadership.  Once Billy Beane has announced his need (as the General Manager of the Oakland A's) to 'adapt or die', the crux of the solution to his problems rests in discovering the REAL problem.  

After losing their star players to larger contracts offered by richer teams, the scouts and coaches of the Oakland A's begin to discuss how to replace those star players using their standard, time honoured criteria.  When Billy questions their methods the following conversation occurs:   

"We're trying to solve the problem."
    "Not like this you're not. You're not even looking at the problem."
"We're very aware of the problem."
    "OK good, what's the problem?"
"We have to replace 3 key players."
    "Nope.  What's the problem?"
"Same as it ever is, we have to replace these guys with existing players."
    "Nope, What's the problem?"
"We need 38 home runs, 120 rbis and 37 double plays to be replaced."
    "Nope.  The problem we're trying to solve is that there are rich teams and there are poor teams … its an unfair game."

The remainder of the film deals with a product strategy to address the real problem at hand; competing in a league where the payrolls of teams are vastly disparate.  The scouts and coaches were trying to solve the apparent symptoms of the problem but not the problem itself.  The strategy Billy Beane employed is spoken by his assistant, Peter Brand:

"Baseball team owners think in terms of buying players.  Your goal shouldn't be to buy players but to buy wins and in order to buy wins you need to buy runs."  

This product strategy leads to a different way of thinking about what's valuable about a baseball player.  Billy and Peter acquire players whose primary value lies in their ability to get on base (by walk or by hit) and therefore be more likely to score runs.  They are able to ignore other negative factors about those players and as a result acquire players not attractive to the big market teams still using traditional selection criteria.  This in turn allows them to field a competitive team at a fraction of the cost of their competitors' teams.  While not winning the World Series that year, the Oakland A's implementation of this product strategy changed the way Major League Baseball teams managed their teams from that year forward.

The Bus

In Jim Collins' "Good to Great" he touched on great leaders taking steps to ensure that the wrong people were off the bus and that the right people were in the right seats on the bus.

In his follow up, "How the Mighty Fall", Collins explores how once-great companies disappear into oblivion and how some manage to turn their decline around.  Once again, the first step that leaders take in trying to resurrect a company involves the concept of 'the bus'.  Here's how you know whether the right people are on the bus:

  • The right people fit with the company's core values.

    • You don't figure out how to get people to share your values ... you HIRE them based on your shared values.

  • The right people don't need to be tightly managed.

    • The right people are self-motivated and self-disciplined.

  • The right people understand that they don't have 'jobs'; they have responsibilities

    • These people can articulate that "I am ultimately responsible for ...".

  • The right people fulfill their commitments.

    • They take commitments seriously and thus are careful not to over-commit.

  • The right people are passionate about the company and their work.

    • Nothing great happens without passion.

  • The right people display 'window' and 'mirror' maturity

    • When things go well they point out the window to others, when thinks don't go well they point to the mirror.

My experience has shown me that having the right people on the bus is the most critical part of creating and maintaining a successful software development organization.  Keeping the wrong people on the bus has enormous risks associated with and it is always more cost effective to remove people from the bus as soon as possible rather than keep them around for perceived short term gains.

The Case for Competition.

Leo Laporte (@LeoLaporte) and Jason Calacanis (@JasonCalacanis) recently hosted an interesting podcast (#TWIT Episode 206) wherein Jason asserted that Apple should be vilified for having a business model based on its closed platform and anti-competitive nature.  He encouraged people to give up use of their iPhones in order to protest what he saw as an abuse of power on Apple's part.  Jason followed that up with an article entitled 'The Case Against Apple' where he essentially describes his evidence of the anti-competitive nature of Apple.  He finishes the article with:
Apple will face a user revolt in the coming years based upon Microsoft, Google and other yet-to-be-formed companies, undercutting their core markets with cheap, stable and open devices. Apple’s legendary comeback ability will be for naught if they don’t deeply examine their anti-competitive nature.

What I think Jason is missing here is that Apple 'fanboys' (I am admittedly one of them) have been prepared to pay a premium for a good user experience and we're prepared to live in a closed environment in order to preserve (for now) that good user experience.  The problem here is that we don't seem to be able to get that same level of user experience elsewhere.  Is that anti-competitive?  Windows has, in the past, been a miserable experience.  There are some technical reasons for this but there is also the primary reason that Apple appears to place more of a premium on producing products that WORK.  Do they work 100% of the time?  No, but they do a damn sight better than most of their competitors.  Is that anti-competitive?  I understand, for now, that part of the price for this is a closed platform.  I wish that wasn't the case, but currently I value gear that works over open platforms.  If that's part of the kool-aid then so be it.  That situation will change when a good user experience becomes a commodity.

I look forward to the day when 'Microsoft and Google undercut Apple with cheap, stable and open devices' WITH GREAT USER EXPERIENCES.  Until then I will continue to reward the company who apparently has realized how much I value the fact that my IT gear works the way I want and need it to work.  Isn't that what competition's all about?

It's ALWAYS been the problem!

When I started working full-time with Agile development techniques in 2004 I also had the good fortune to start working with the concepts of Pragmatic product management.  What quickly became apparent to me is that the two approaches in their respective fields were beautifully suited to one another.

By articulating the market problems, the nature of the people who have them, and the circumstances under which they experience those problems, Pragmatic product managers communicate to development teams what is really necessary to build great solutions; context.  We've all heard the stories of how the software industry's predilection for waterfall methodologies from the 1970's through the 1990's led to an inordinate amount of 'failed' projects.  Early proponents of agile development techniques ascribed those failures to the waterfall process itself, rightly arguing that performing development tasks (specifically testing) in phases and a lack of iterative feedback were the biggest causes of these 'failures'.  However, I would contend that just as large a reason for those 'failures' was product management's inability to communicate what was actually required for those projects to be successful.  Too much emphasis was placed on building to a specification which had often been created and disseminated without providing sufficient context to the people responsible for providing solutions.  Because of this lack of context there was no framework for negotiation with a customer or stakeholders and therefore much of what was built was not actually required to meet the customer's needs.  This resulted in unnecessary delays and cost overruns at a minimum and often the creation of ineffective solutions which quite rightly were considered 'failures' regardless of whether they met the specification.  Even if the solutions were built iteratively, the focus on the specification, rather than the problems in context, might very well have led to the same ineffective solutions.

Many software development companies do not have the luxury of having real-time access to actual customers.  Nor is the concept of building a product based on a single customer's requirements necessarily attractive.  Instead these companies rely on their product management organizations to represent the needs of customers within a market.  There is a hierarchy of proxy which is created as a result.  The customer is a proxy for the market and the product manager is a proxy for the market.  If the product manager is too busy performing strategic tasks and does not have time to fulfill his product ownership duties for the team then a proxy of a proxy is incurred with all the risk that entails.

All too often, agile projects rely on an internal 'product expert' to act as the team's Product Owner.  The danger of this is that the product expert understands the existing incarnation of the product very well but may not have the viewpoint of a customer who has issues with the product.  Properly implemented agile development techniques usually provide the transparent and tangible progress necessary to make informed business decisions.  However, without a clear sense of the relative value of the problems to be solved it is difficult for the team to understand and embrace those business decisions.  This is where the value of the MRD resides; providing the context necessary for a Product Owner to make sound decisions throughout the development process on behalf of the product manager.

In the end agile product ownership is a subset of all that a product manager is responsible for.  From a Pragmatic perspective, product ownership entails the 'Product Planning' tasks of the product manager: Market Requirements, Road Mapping, User Personae, User Scenarios, and Release Milestones.

The Pragmatic Market Requirements Document (MRD) focuses on market problems, the people who have them, and the situations under which they experience them (scenarios).  It also focuses on valuing/prioritizing those problems, people, and scenarios.  Hmmmm ... sounds suspiciously like a good start on a backlog of stories!  A story used in an Agile project usually takes the form of: A <type of user> needs to be able to <perform some action> in order to <get some value>.  This is essentially a statement of the circumstances under which a user experiences a problem.  I suppose technically it is an anti-problem as the real problem is the users inability to get the required value by performing the desired action.  Nonetheless there is a direct correlation between the primary elements of a Pragmatic MRD and a product backlog used in an Agile project.  Each scenario is prioritized using the following additional context: impact, frequency, and criteria.  The value to the persona of solving the scenario, the frequency with which the persona experiences the scenario, and the value to the to the Product Manager /Business of solving the scenario, all provide valuable context necessary for establishing business priority.  Once these scenarios have been prioritized, the product manager has a solid starting point for a product backlog.  Further, the entire team has a solid understanding of what problems they are trying to solve and for whom, and what the relative value of those problems is to both the market and the business.  With this context, the team is much better equipped to be able to produce solutions appropriate for the target personae and any decisions around solution options/modifications can be made in the context of the user and the business value.


But WHY, dad?

Asking 'why' is perhaps the most valuable of all questions both in life and software development / product management.  Being the father of two toddlers, I get asked 'Why?' a lot.  I only wish the team mates on some of the teams I coach were so inquisitive.  Knowing the 'Why' is often the difference between success and failure.  This is, of course why User Stories take the form they do: As a <type of user> I need to be able to <perform some function> so that I can <get some value>.

Both the type of user and the 'so that' clause provide valuable information in the form of 'Why'.  Without this information, a team could go horribly astray.  Asking 'Why?' helps provide valuable context.

Here's a simple example:

The user needs to be able to enter surface positional data.  Why?

The directional drilling engineer needs to be able to enter surface positional data so that he/she can perform anti-collision analysis.  Why?

The directional drilling engineer needs to be able to enter surface positional data so that he/she can perform anti-collision analysis, avoid drillstring collisions with offset wells and therefore avoid tremendous expense and/or injury/death of drilling personnel.

Clearly knowing who needs to enter this data, and why, puts a different context in front of the team around the requirement of simply being able to enter surface positional data.

My sons make me answer this question a lot and software development teams could learn a great deal if they if they continually did the same!