Thursday, 28 August 2014

Agile Project Management - Complete Refrence

Agile Lifecycle

All software development methods, including the Agile ones, have some sort of underlying project lifecycle. System Development Lifecycle (SDLC) is the common name for a software development process, but in the Agile world I prefer calling it a “heartbeat” reflecting the organic nature of an Agile project. Some of the big Agile methods don’t make a big deal of the heartbeat and others do. Some have such abstract lifecycles that it is actually hard to know what activities to schedule. And they all use different terms for the same thing. I have pulled out the common activities to create a generic agile lifecycle.

The key elements are a Project Initiation followed by one or more Releases made up on one or more Time-boxes (although usually 3), and ending with a Project Closure.


Agile Pre-project

There are some important questions to be addressed before the start of any project like “Why are we doing this project?” and “Should we do this project?” Project Initiation is the phase where these questions are answered but some work has usually been done before.

Find/Write Project Brief

Projects don’t happen in isolation. The key question during project initiation is “Should we invest more?”, i.e. is the project worth going ahead with?  There must be a reason for the project. The first step during Agile Project Initiation is to find this reason. Often with software development there are just too many choices for what the team could do. A Project Brief can help remove this confusion by pointing the general direction in which the team is meant to head.
The format of a Project Brief document might be a Project Vision, in part of a Product Road Map or a Project Mission Statement but the key thing is knowing why we’re spending this money on this activity.
Marty Cagan of the Silicon Valley Product Group suggests we provide good answers to nine questions to know if we are pursing a good opportunity. The nine questions of the Opportunity Assessment are:
  1. Exactly what problem will this solve? (value proposition)
  2. For whom do we solve that problem? (target market)
  3. How will we measure success? (business metrics)
  4. What alternatives are out there? (competitive landscape)
  5. Why we are best suited to pursue this? (our differentiator)
  6. Why now? (market window)
  7. How will we deploy this? (gentle deployment strategy)
  8. What is the preliminary estimated cost? (small/medium/large)
  9. What factors are critical to success? (solution requirements)
I encourage Product Owners to answer these questions before launching the project proper.

High Fidelity Prototype

The Silicon Valley Product Group recommend a a High Fidelity Prototype rather than using documents to describe the target system.  If the Project Brief gets the go ahead then invest some time in putting together a click through prototype of the system.

Agile Project Initiation

Traditional Project Initiation

Initiating is one of the five project management process groups in the Project Management Body of Knowledge (PMBOK) from the Project Management Institute (2004).
In the UK project initiation usually means writing a Prince2 style Project Initiation Document (PID) or the equivalent.  Once you have the PID the Requirements Capture, Analysis and Design activity can begin.  All of which is rather time consuming.

The Project Initiation Document (PID)

The defining feature of the Prince2 Project Management method is the PID. It can take a considerable amount of effort to write this document. OGC: Project Initiation Document (PID) gives the suggested content:
  • As a minimum the document should answer the following fundamental questions about the project:
  • What the project is aiming to achieve
  • Why it is important to achieve it
  • Who will be involved in managing the process and what are their responsibilities
  • How and when the project will be undertaken
  • The PID has to answer the above questions to a sufficient level of detail to maintain control of the project. The document should cover the following areas:
  • Background, explaining the context of the project, and how we have arrived at the current position of requiring a project.
  • Project Definition, explaining what the project needs to achieve. Under this heading will be:
    • project objectives
    • defined method of approach
    • project scope what is included and what not
    • project deliverables and/or desired outcomes
    • exclusions
    • constraints
    • interfaces
    • assumptions
  • Project organisation structure, detailing who is going to be involved and what their responsibilities are (team management structure and job descriptions)
  • Communication plan, describes how the project stakeholders will be kept informed during the project
  • Project quality plan
  • Project controls, laying down how control is to be exercised within the project, and the reporting and monitoring mechanisms that will support this
  • Business case, covering the estimated costs, risks and benefits. The Business Case will require regular review throughout the project and may require updating
  • Initial project plan. The plan will be reviewed and further developed at regular intervals during the project
  • Risk register containing details of the identified risks so far. The Risk Register will be reviewed at regular points during the project to assess progress on managing risks and to identify new risks that may have appeared

Requirements Capture, Analysis and Design

These phases are all about identifying the requirements and then transforming them into a format suitable to hand to developers.  The Requirements are usually in business language.  Analysis, usually done by a systems or business analyst, extracts the logical requirements.  Design attempts to show how the logical requirements which be met technically.  These phases are all time consuming, happen in sequence, and involve different people at each stage.

Agile Project Initiation

The Agile methods are beginning to acknowledge that some upfront work is required to ensure project success. Project Initiation includes all the work necessary before full scale coding begins.
Agile Project Initiation includes these steps:
  1. Find/Write Project Brief
  2. Form the team
  3. Initial Plan
  4. Initial Requirements
  5. Initial Infrastructure
The difference between Project Initiation and the equivalent in a traditional waterfall process is that Project Initiation is time boxed and the Agile Team do the work.  a couple of days is all you need for a small project and a month is fine for a large project. Given the short time period the initial deliverables will not be perfect but perfection is not the aim. The idea is to do barely sufficient work to begin developing a solution. For some deliverables, e.g. Release Plan and Risks and Issues Log, these will be only the first version of many created during the project.
For further information on activities to include in Project Initiation, check out the Business Study in DSDM (2002) and Iteration Zero in XP (Beck 2000).

Find/Write Project Brief

Strictly speaking the Project Brief should exist before the project but in reality this often hasn’t been done and the Agile Project Manager should to write/find it.

Form the Team

The Agile Project Manager mobilises the Agile Team and ensures the Agile Team has an appropriate work space and support.
An Agile Team should contain the mix of skills required to deliver the product. Common skills are Project Managers, Product Owner, Designers, Developers and Testers.
Agile promotes close collaboration between team members so advocates co-location of the team members. Co-location makes for better channels of communication and counters Conways Law which suggests poor channels of communication may be reflected in the structure of the end product (Wikipedia: Conway’s Law). If the team have a dedicated work area then they can use their workspace to display visible progress towards their Timebox Goal.
This is a good opportunity to do the following.
  • Identify and arrange any necessary training
  • Set up recurring meetings and book rooms for Timebox Planning, Demos (also book a projector) and Retrospectives
  • Arrange for the team to sit together
  • Order a board if you need one
  • Prepare for the first Timebox planning ( See: Timebox Pre-Planning)
  • Create a project space and decorating it with material, staking it as the informative workspace as defined in the Agile glossary.

Initial Plan

The Release Plan is a reflection of features to be developed over time. The desirable features of a product are prioritised and then divided by Timebox with the most important features developed first. See the section Release Plan for more details. The division by Timebox is a result of the current velocity of the team being mapped against the current story point backlog for the project.
This is a good time to start the Risks and Issues Log.

Initial Requirements

The Product Owner is responsible for delivering the initial requirements during Timebox Zero.
There is no need to provide detailed requirements for subsequent Timeboxes at the start of the project. This will allow requirements to develop during the project with minimum overhead.
In terms of requirements that means doing:
  • High level requirements for the entire project
  • Detailed Requirements for Timebox One
The high level requirements are for the entire project.  Facilitated workshops are an effective mechanism for eliciting the requirements from the stakeholder group fast. The high level requirements drive the the Release Plan.
Agile is about doing the barely sufficient documentation to facilitate the next step in the process.  That means only enough detailed requirements are specified, agreed and understood to keep the team busy for the first one or two Timeboxes. Ideally the detailed requirements include Visual and User Experience Designs.
Silicon Valley Product Group recommend a a High Fidelity Prototype rather than using documents to describe the target system.

Initial Infrastructure

the technical staff set up the infrastructure for the project during Timebox Zero. This includes:
  • Environments
    • Developer and tester machines
    • Servers (development, continuous integration, test, UAT) including hardware and operating system, web servers, application servers, databases and permissions
    • Team Wiki
    • Set up and configure a bug tracker, agree bug process
  • • Version control and builds
    • Source Code Control repository (e.g. SVN) with access for developers and tester(s)
    • Tools for Continuous Integration (e.g. Cruise Control, Nemesis)
    • Continuous Integration up and running on your project in your source code repository
  • • Test Driven Development frameworks
    • Tools for automated testing (e.g. JUnit, JMock, JSUnit, Selenium, FIT, Fitnesse, DBUnit, XMLUnit)
    • Agree method(s) and depth of unit testing
    • Agree methog(s) and depth of acceptance testing
    • Agree the high level scope and approximate timing or frequency of other tests, for example accessibility, performance, security
    • Decide, set up and configure a framework for acceptance test automation
    • Write just enough code to support the automated build, continuous integration system, and automated test framework in use.
The technical staff, led by the Technical Lead, also do the first high level version of system architecture.  The systems architecture document outlines both the hardware and software of the intended solution. It describes the target platform and (if different) the development platform. It also explains the software structure including major components and their interactions. [Some Agile methods are very keen on emergent design, but you have to start somewhere.]

Environments

Not really Agile per se but I thought it worth having a mention of the environments a development team will need. There are quite a few.
 
Promotion Path to Live
EnvironmentPurposeWhen to useRelease ControlWho Controls
DevelopmentWhere the developer does their thing.AlwaysControlledDeveloper
Continuous IntegrationWhere the continuous integration build appearsAlwaysUncontrolled
Automatic System TestWhere the automatic tests run if the Continuous Integration box can’t handle the load.If have to split test because they run too slowUncontrolled
Manual System TestWhere the Testers check out stable releases.AlwaysControlledTester
Automatic Integration TestTo integrate all of the components when there are multiple development teams working on a shared product.  Kicked off by an automatic process.Multiple teams integratingUncontrolled
Manual Integration TestDitto but for the integration testers to use.Multiple teams integratingControlledIntegration Tester
Hot Fix DevelopmentTo maintain a live copy of the software using Hot Fixes, whilst developing new functionality in parallelWhen system is liveControlledDeveloper
Hot Fix Continuous IntegrationDitto; this is were the continuous integration build goesWhen system is liveUncontrolled
Hot Fix Pre-productionDitto; this is were the Hot Fixes are tested before going live.When system is liveControlledProduct Owner
Pre-productionThis is where manual user acceptance testing usually takes placeAlwaysControlledProduct Owner
ProductionLive.AlwaysControlledProduct Owner or Operations
  

Agile Project Scope

According to Wikipedia: Project Management project scope is:
The overall definition of what the project is supposed to accomplish, and a specific description of what the end result should be or accomplish.
We can talk about what is “in-scope”, i.e. what the project is meant to delivery, and what is “out-of-scope”, i.e. what won’t be delivered.
Typically the project scope is a subset of the scope of the product under development.

Traditional Project Scope

Project Scope Management is one of the nine project management knowledge areas in the Project Management Body of Knowledge (PMBOK) from the Project Management Institute (2004).
Traditionally the scope of a software project has been defined in a specification document of some kind. Terminology varies considerably and the document might be called a requirements specification, functional specification, or program specification. Regardless of the name the document describes the function requirements, i.e. the intended behaviour of the system being developed, and the non-functional requirements that constrain the design or implementation (e.g. performance requirements, quality standards, etc). (See  Wikipedia: Software Requirements Specification (SRS) orWikipedia: Functional Specification). These documents tend to be quite large.
Traditional requirements documentation can be characterised as both wide and deep, i.e. it covers the entire breath of requirements in considerable detail. The trouble with this approach is that generating such comprehensive documentation takes a lot of effort, and by the time the specification document is complete, it is also out of date.
And ironically in traditional project management any changes to the scope are seen as a threat to the project. Change Management is used to combat changes to scope.

Agile versus Traditional Project Scope

In an Agile project, as with traditional approaches, the scope defines what software to build and deliver. There are, however, several differences between the two approaches including the:
  • Attitude towards change in Agile
  • Documents defining the Agile project scope
  • Timing of producing the Agile requirements
  • Emphasis on high level requirements in Agile
Unlike the wide and deep requirements of the traditional approach, I characterise Agile requirements as high level and focussed. The scope of an Agile project is defined by high level requirements, in the form of User Stories, scheduled in the Release Plan. Detailed (or deep) requirements are still necessary but they are only created when they are needed – this is the focussed bit. That means that in Agile Project Initiation, the Product Owner produces User Stories for the entire project, but only produces supporting documentation for those User Stories scheduled into the first Timebox. During the first Timebox detailed supporting documentation is produced for the User Stories in the second Timebox, etc. This means that if/when the scope changes we have invested a minimal amount of energy in specifying out the bit that is no longer needed.
Whereas change is seen as the enemy of a traditional project and traditional Change Management is used to combat changes to scope, in the Agile world change is expected and Agile Change Management facilitates change. Seeing change as likely has an impact on the nature of the documents that define Agile project scope. Basically the documents have to be easy to change. In addition detail is left as late as possible to facilitate change. In the absence of detail the Agile Project Manager must use high level requirements, i.e. User Stories, as the basis of their Agile Project Planning.

Role of the Project Brief in Defining Scope

The Project Brief is a prerequisite for Agile Project Initiation. The Project Brief provides the context in which the User Stories are written.

Role of the Release Plan in Defining Scope

The Release Plan is the main deliverable from Agile Project Planning. It lists the User Stories that are in-scope and when they are scheduled for development. By implication it also indicates what is out-of-scope.
I find it very useful to have all User Stories, whether in-scope or out-of-scope, listed on the Release Plan. I flag the out-of-scope items as such as it is clear that they will not be built. This is mainly to avoid the inevitable discussion along the lines of
Product Owner: “But I asked for X”
Project Manager: “Yes you did, but you then deprioritised it to make room for story Y. We developed Story Y. Story X is out-of-scope”
This is an example of why Agile Change Management is important.

User Stories = High Level Requirements

User Stories seem to have become the defacto method for expressing high level user requirements in the Agile world. They originate in the XP world (Beck, 2000; Beck & Fowler, 2001), but the definitive work on writing user stories is “User Stories Applied” by Mike Cohn (2004).
User Stories are usually brief descriptions of the intended software behaviour from the user’s perspective. They can be brief because they are a place holder to a conversation between the Product Owner and the Agile Team and/or to supporting documentation.
User Stories are the base planning units used in Agile Project Planning.

Guidelines for Good Stories

Rachel Davies of Agile Experience advocates a user story format which is gaining increasing acceptance:
Story Title
As <role> I want <feature> so that <benefit>
This format not only tells the Agile Team what the feature is but which user needs it, and most importantly, why they need it.  The title is useful as a short hand for the rest of the story. I give each User Story a unique numeric ID as another form of short hand.
User Stories should avoid unnecessary detail, for example, they shouldn’t have details of the specific user interface. On the other hand it is essential that each story has at least one acceptance test associated with it before it reaches a Timebox to be developed.

Vertical versus Horizontal Slicing

Traditionally systems are planned and built by slicing up the work horizontally, component by component. As the work progresses the team adds extra building blocks to the product be they modules or classes.  This behaviour means that usable product features must wait until all the building blocks are in place before being ready, which might take several increments of the product.  This is problematic because the Product Owner cannot provide immediate feedback about the software until the full feature is delivered, long after the work started on it.
Agile uses vertical slicing. The software is built feature by feature, i.e. User Story by User Story. The Agile Team builds enough of each component (module or class) to meet the needs the User Story currently being worked on. That means the end of the Timebox, in the Timebox Review Meeting, there are more working features to be reviewed.  Instead of incrementally adding components the team is incrementally adding value. In this approach result, frequent feedback is possible to ensure that the software can evolve almost as quickly as the customer’s vision, maximising the chances that the customer actually gets the product they want.

Technical and Non-functional Requirements

This User Story format is useful for user oriented requirements but not for technical or non-functional requirements (performance, accuracy, portability, reusability, maintainability, interoperability, availability, usability, security, capacity). In these circumstances use whatever format is appropriate. For example, “The system must support peak usage of up to 50 concurrent users” or “Every page should load in less than 3 seconds”.

Splitting a User Story

There are at least three reasons to split a User Story:
  • the story is an Epic, i.e. to too big to fit in a Timebox
  • part of the story can’t be estimated
  • part of the story has is higher priority (more value and/or more risky) than the rest
An Epic story is a User Story that is too large to be developed in a single Timebox. It is normal to have a few epic stories in the Release Plan, particularly at Agile Project Initiation, but these have to be split into smaller stories before the User Story reaches a Timebox for development.
Often User Stories include several separate functions. If there is some uncertainty about part of the User Story then cut this part out for investigation. The understood part of the User Story can get scheduled normally.
Similarly if part of a User Story is higher priority than the rest then the high priority element can be split out as a separate User Story for early scheduling.

Defects

Defects need to be logged just like enhancements which means as User Stories. The specifics of how teams do this varies. You have a choice:
  • Log each Defect as a User Story
  • Batch up several Defects as a single User Story
You then have the choice of scheduling the Defect User Stories like other User Stories or of having a reduced Velocity to cater for doing Defects. I tend to do the latter.

Where to Keep User Stories

Most commonly User Stories are written on index cards and are then pinned to a board. This approach has several advantages:
  • It keeps the descriptions short
  • It is cheap and easy to update existing stories, write new stories and/or discard outdated ones
  • Anyone can write one, which gets everybody involved in planning meetings
  • The board and the User Stories it contains becomes the focal point of discussion in the team
There are many Agilists who are happy to leave the User Stories on index cards but I prefer to have duplicate systems. Three in fact:
  • As index cards on the board
  • Titles and estimates in the Release Plan which is usually a spreadsheet.
  • the titles at least in a task tracking database.
There is an overhead to this but the different formats are for different reasons and I could never quite eliminate any one of them.
My Release Plans are always in a spreadsheet. Not only does this allow me to do some calculations on the data it contained (e.g. levelling team scheduled user Stories and Velocity/capacity) but it also means I can email the document to my customers. It is hard to email index cards.  If I had a specialist Agile project management tool such as VersionOne, Mingle, ScrumWorks, etc, I would consider abandoning the spreadsheet.
I also use a task tracking database containing the the tasks for each story. This facilitates sending out emails to individual developers and to the team as a whole, allows me to generate automatic release reports for the operations team and my customer, and gives us a history of what happened when which is important for support. Those specialist Agile project management tools (VersionOne, Mingle, ScrumWorks, etc) are ideal for this but there are other ways to achieve the same effect. At one company we just modified our call tracking system to be the task tracking database. Some of the other Agile teams I’ve managed prefer using a fault tracking database to the specialist tools because the fault tracking database has hooks into their source code repository (specifically Trac and Subversion).

Supporting Documentation

User Stories are not the only information on requirements. User Stories are a place holder to a conversation between the Product Owner and the Agile Team and/or to supporting documentation. If the Agile team doesn’t have sufficient information to estimate the work involved then the User Story needs further elaboration. In such a situation the Product Owner must provide additional documentation or models to help clarify particular User Stories. This documentation should be kept to a minimum but it shouldn’t be forgotten; XP recommends at most four pages of supporting documentation per User Story.
Typically this supporting documentation includes User Interaction Designs (wire frames), visual designs, or a combination of the two in a high fidelity prototype. Financial calculations also warrant being documented separately.

References and Further Readings: 
Beck, K. (2000). Extreme Programming Explained: Embrace Change. Reading, Massachusetts: Addison-Wesley.
DSDM Consortium. (2002). Framework for Business Centred Development:Handbook for DSDM Version 4.1.. Author.
Project Management Institute. (2004). A Guide to the Project Management Body of Knowledge (PMBOK Guide) [3rd Edition]. Author.
Beck, K, & Fowler, M. (2001). Planning Extreme Programming. Boston: Addison-Wesley.
Cohn, M. (2004). User Stories Applied. Addison-Wesley.
Steven Thomas Blog: http://itsadeliverything.com