The Nevex Process

A Happy Medium

• Very few individuals or teams stick to a textbook process
  • A process is just a tool
  • You should always adapt tools to local needs…
  • …and personalities
• This lecture is a slightly fictionalized account of what Nevex did in 1999-2005
  • A computer security startup (acquired by Baltimore, then by Hewlett-Packard)
  • Shipped several major versions of a complex product, and many patches
  • Met deadlines without lots of late nights and weekends
• Starting point:
  • Version 3.2 of GRACE shipped last week
  • Version 3.3 has to ship in approximately eight months
  • Personnel available are:
    • One overall manager
    • Nine developers (two junior, four intermediate, three senior)
    • One developer support (installers, build, etc.)
    • Four quality assurance (QA)
    • One technical writer
    • One system administrator
    • One help desk
    • One product manager (marketing, sales support, etc.)
    • One secretary/HR/office admin
• Project lifecycle:

  • Figure 28.1: Project Lifecycle

• Goal: meet schedule and quality goals without heroic effort
  • No point getting something flaky to the trade show if half the development team quits the week after
• Comment on this slide

All the Vision You'll Ever Need

• Important to have everyone on the same page
  • Because common sense isn't
• Exercise: have everyone on the team fill out the following form independently, and compare answers
  • Bits in italics are typical answers: have your team members replace them with their answers

(Problem statement)	The problem of	not being able to access the BIND molecular database remotely
(Target market)	affects	scientists studying protein interactions.
(Impact)	The impact is that	they can only inspect/update single records at a time via a browser.
(Solution)	Our solution is to	let programmers search and update the database programmatically using multiple languages.
(Specific audience)	For	programmers
	who want to	incorporate BIND into a larger application
(Technical details)	our product is a	web services API based on SOAP
	that	supports remote calls into the core BIND API.
(Competitive comparison)	Unlike	screen-scraping
	our product	can be used directly and reliably in many languages.

• Comment on this slide

Analysis & Estimation

• “Plans are useless, but planning is invaluable.” (Dwight D. Eisenhower)
• Product manager gathers requirements from customers
  • Customers often say, “We want feature X.”
    • Often means, “I heard someone mention this while I was waiting for a flight.”
  • Product manager turns that into, “We want the product to be able to do Y.”
    • What do customers want to be able to do that they can't right now?
    • What does the product make them do that they don't want to have to?
• Developers analyze the feature requests provided by the product manager
  • How can the capability be implemented?
    • Often involves researching new technologies, competitors' products, etc.
  • What are the risks and unknowns?
    • Best way to find out is often to build some throwaway code
    • This goes under version control like anything else…
    • …but should not then be “grown” into the product
  • How long will it take?
    • We'll return to this topic below
  • Usually takes 3-6 weeks to do A&E
• What's everyone else doing while this is going on?
  • Providing estimates of their own, of course
  • Every feature must be tested and documented
  • Must also be added to the daily build, and to the installer
  • So QA, documentation, and developer support need to add time estimates to the A&E too
• When all A&E's are done, the product manager collects the estimates
  • Total time is (for example) 9000 work-days
  • But only 2000 work-days are available
    • Fifteen people in development, QA, tech writing, and technical support
    • Eight months of five days a week, minus statutory holidays
    • Minus 20% for sick days, emergency bug fixes for existing customers, etc.
    • No team ever works at 100% capacity, so don't schedule that way!
• So now the product manager has to prioritize
  • Which features are most cost-effective to develop?
    • I.e., will satisfy the most customer needs for the least effort?
    • Developers don't know enough (don't have enough customer contact) to make this decision
  • Usual way to do this is to build a 3×3 grid
    • Rank each feature low-medium-high on importance and time required
    • More honest than the false accuracy of a 1-10 scale

    • Figure 28.2: Feature Rankings

  • Product manager is not allowed to shave developers' time estimates
    • If she is, developers will start padding their estimates…
    • …or supplying random numbers, secure in the knowledge that they won't be able make the deadline anyway
  • Letting managers make up “science fiction schedules” is one of the most common mistakes companies make
    • The other is not doing enough testing early on
  • Yes, customers (and your marketing department) will complain if their feature doesn't make it onto the schedule
    • But putting it in the schedule when you know that schedule is fiction doesn't improve the chances of it getting implemented on time
    • It's better to live up to small promises than break big ones
• End result of A&E process is a schedule that everyone believes is achievable
  • Yes, it contains a lot of guesswork…
  • …but it's better than nothing…
  • …and estimates improve with practice
• Comment on this slide

What Goes Into an A&E

• No template is ideal for all A&E's
  • Every A&E is different, because every problem is different
  • Requiring team members to express their ideas in a fixed way just leads to fewer ideas
• Typical A&E has these sections:
  • Title, author(s), revision history
  • Executive summary (abstract)
    • One paragraph
    • Tells someone browsing through dozens of A&E's whether this is one they should read
  • Background
    • Aim is to provide a context for evaluating decisions
    • Introduce the problem to a reasonably knowledgeable developer
    • Don't explain what the Internet is…
    • Imagine writing for who you were when you started the project
  • User stories
    • What will people be able to do (or do better) when this feature is implemented?
    • Often the starting point from which QA develops test cases, and the technical writer develops documentation
  • Often find there's an easy way to give customers half of what they want, and a harder way to do it all
    • Estimate both, and discuss pros and cons…
    • …because the product manager may decide to go with the cheap 50% solution when putting the schedule together
    • “Not everything worth doing is worth doing well.”
  • For each alternative:
    • How long will it take to create? To test? To document? To add to the build and installer?
    • What impact will it have on other features?
    • How risky is it (i.e., how certain are you of your estimate)?
  • Do not gloss over the hard bits
    • This is when you do your hard thinking (while you're still fresh)!
    • “We parse the XML configuration file, and then a miracle happens” is not a good A&E
  • References
    • These days, usually a list of URLs
    • Include pointers to prototypes stored in the version control repository
• Time estimates come from experience
  • You can usually estimate how much code you'll have to write to implement something
    • If you can't, you should go back and think it through in more detail
  • You should know how long it takes you to develop working code
    • Just as runners keep track of their times for running 1000 meters, or how much they can bench press
    • The most elaborate version of this is Watts Humphrey's Personal Software Process (PSP)
    • In practice, enough to keep track of how many days you spend on each module you write…
    • …where “days spent” includes debugging time
  • Your first estimates will be wildly wrong
    • But the more you estimate, the better your estimates will become
• Expect to revise this document several times
  • Project manager assigns task of reviewing A&E to specific developer(s)
    • Ensures that reviews actually get done…
    • …and that knowledge and decisions are communicated
• Comment on this slide

Getting It Built

• Developers then finish designs
  • Purpose is to convince people (including themselves) that they can actually build things the way they intend to
    • Very easy to believe incompatible things when it's all in your head
    • The act of writing it down (explaining it) forces brings these issues out into the open…
    • …and ensures that other people know what you think they think you think
    • Also helps you remember later why you're doing something a particular way
  • Note that design overlaps analysis
    • After all, it's impossible to estimate how long it will take to build something if you don't know how you're going to build it
• Level of detail varies widely
  • The better developers understand the product and the technologies it uses, the less risk there is of a lightweight design leading to nasty surprises
  • The greater the cost of delivering the wrong thing, the more detailed the design should be
    • But it doesn't help to do a detailed design that customers can't understand
• Up-front design is a tool for managing risk
  • The later you change or fix something, the more expensive it is
  • But the more time you invest up front, the longer it is until you discover that you need to make changes
• Write the code
  • Also overlaps analysis and design
    • Build throwaway prototypes while doing analysis
    • Expect to refine design during early stages of construction
    • If you're still refining the design a week before you're due to ship, something has gone wrong
  • Remember that coding includes writing unit tests
    • These are the developer's responsibility
    • Include coverage analysis in the daily build & test to ensure that new code is being tested
    • In theory, QA should refuse to test features that aren't exercised by unit tests
• Often refactor old code while adding new stuff
  • Your skills (and coding style) improve over time
    • Or the person working on the feature in Version 3.3 knows something the Version 3.2 author didn't
  • The problem changes over time
    • A good solution to last year's requirements may not be a good solution to this year's
• As features are completed, update the design to reflect what was actually built
  • If you don't, then:
    • The next person who has to work with the code will have a harder time
    • You'll have to start from further behind the next time you need to modify this section
  • Making sure this happens is the project manager's job
• Maintenance usually starts before the product goes out the door
  • Fix bugs left over from the previous cycle while adding features in this one
  • Start fixing bugs in new features as QA discovers them
  • Last quarter of a project cycle is usually nothing but bug fixing
• Comment on this slide

Other Activities

• Quality assurance starts at the same time as development
  • As soon as feature specifications are firm, QA can:
    • Start making up lists of tests to run
    • Start writing utility programs
  • And of course, there's always bug fixes to the previous version to test…
• QA's usual role is to find bugs in the finished product
  • Yes, developers should do this as they're coding…
  • …but in practice, they are usually too focused on particular features to find wider issues
  • Developers also tend not to be good at testing
    • Subconsciously don't want to find problems
    • Focus on the parts they're most familiar with, rather than the product as a whole
• QA's real job is to figure out why bugs are happening
  • “Trying to improve the quality of software by doing more testing is like trying to lose weight by weighing yourself more often.” (Steve McConnell)
  • So that developers stop making that kind of mistake
• Documentation also starts before product is available
  • Describe new GUI screens based on mock-ups drawn for A&E
  • Write new on-line help
  • If the technical writer can't figure out how to explain a feature, the developer should redesign it
• Developer support starts work on installers the same day developers start on new features
  • Makes sure the daily build & test keeps running
    • And chastises developers who've broken things
  • Everyone has a status report in their mailbox when they arrive each morning
• Schedule is always up to date
  • Developers provide progress reports every week (at least)
    • Doing this at 9:00 a.m. Monday works better than asking for it at 4:45 on Friday
  • Always based on binary milestones
    • If you allow percentages, then many tasks will be 90% done for 90% of the lifetime of the project
    • Instead, break tasks down into subtasks that are at most a few days long, and either completed or not completed
    • Describe milestones in terms of verifiable deliverables (i.e., things that someone else can check)
  • The real purpose of a schedule is to help the project manager cut corners when things go wrong
    • How far off track are we?
    • What can we cut (or scale back) to get on track?
• Comment on this slide

After the Party's Over

• The last thing you do is write the release log and assign the release a version number
  • Leave this until the end because:
    • You may discover that what you thought was going to be one release actually needs to be several
    • Marketing may change its mind based on what the competition is calling their releases
  • Refer to the release by a codename (like “Beryllium” or “Munich”) while it's under development
• Always do a post mortem on the project after shipping
  • What went right (that we want to do again)?
  • What went wrong (that we want to avoid next time)?
  • Usually best to bring in a neutral outsider to facilitate
    • Feedback is only as useful as it is honest
• And then start the next round of A&E's
  • Because while the development team has been wrapping up this release, the product manager has been preparing for the next one
• Comment on this slide

Exercises

Exercise 28.1:

Does your manager know when you expect to complete your current task? How inaccurate the schedule currently is?

Exercise 28.2:

Can you find out when your manager expects you to complete your current task (without asking her directly)? When team members expect to complete their current tasks (without asking them directly)? Who would be affected if you slipped a week?