Software estimates that suck less (Henning Koch, makandra GmbH)

Software estimates
that suck less

henning.koch@makandra.de
@triskweline

My context
makandra GmbH

We develop and maintain ambitious web applications
Since 2009
12 Ruby devs, 5 operations engineers
~ 100 applications shipped
Project lengths range from 2 person-months (start-up MVPs)
to 5+ person-years (AUDI, ProSiebenSat.1, Siemens, Volkswagen)

University vs. work

At university, you are mostly limited by energy and focus.
You can fix many problems by throwing more time at it.
At work, a lot of tasks are time-constrained.
You will talk a lot about how long something will take.

Hey quick!

How long is this going to take?

Every developer has conflicting emotions when asked this question:

It's a giant question for any non-trivial task
but customer/boss expects you to answer it on the spot
You want to help customer/boss
but you don't want to write cheques your future self can't cash

Also you were hurt in this situation before:

People get anchored to the first estimate they hear
People will hold you to your estimate, even when they say they won't
People will hold you to your estimate, even when requirements change

I estimated ~20 person-years of development
I used to hate estimates
I still don't enjoy software estimation
But I made my peace with it

Why estimates are useful

Customer/boss need them for budget planning
They enforce a process for requirements analysis and resource planning
Developers don't naturally care for time constraints

Your estimates are too low!

Larger estimates are more likely to cover actual effort (duh!)
Don't "let's just 10x every estimate"
Let's take a deeper look why those numbers are off

Common issues with estimates

Incomplete
Not taken seriously
Dishonest
Thrown overboard once the project begins

10 reasons why your estimates suck

#10

You are not taking the time required

Estimates take a lot of time
Easily 5-10% of the total project volume
It's the first (and hardest!) part of the project

How we estimate

Bottom-up estimates
Long, nested list of cost drivers
Insane detail

Example

Users should have a new required field "country"

Users should have a new required field "country"     1h

Users should have a new required field "country"
  Add database column                              0.1h
    Fix existing data with a default value         1.5h
      Infer country from e-mail domain
  Change views                                       1h
                                                  ------
                                                   1.7h

Users should have a new required field "country"
  Add database column                              0.1h
    Fix existing data with a default value         1.5h
      Infer country from e-mail domain
    Add index for fast lookup
  Views
    Find and install country picker library        0.5h
    Add country field to sign up form              0.1h
    Add country field to profile settings          0.1h
    Whitelist new form param in controller         0.1h
    Fix integration test for the sign up form      0.1h
    I18n                                           1.0h
      Field label
      Country list
  Model                                            0.5h
    Validate presence of country
      Add to model
      Unit test
    Fix tests that create users                    0.2h
                                                  ------
                                                   4.3h

Users should have a new required field "country"
  Migration                                        1.0h
    Fix existing data
  Views                                            1.5h
    Use nice country picker
    I18n
  Model                                            0.5h
  Fix tests                                        1.0h
                                                  ------
                                                   4.0h

Estimate for 3 person months

Estimate reviews

Hide the numbers
Let a colleague have a go in a separate column
Discuss major differences
Surprisingly few differences in tight-knit teams

#9

You are not getting money (or time) for estimates

Getting dedicated money (or time) dissolves a lot of tension surrounding estimates
It's the first 5-10% of the project. It's the hardest part of the project. Why would you not get time for it?
Get into a habit of never giving a snap estimate for anything non-trivial.
Always schedule a workshop for new projects or milestones

Workshops

Usually 3-5 days
Scope diet
Mockups
Detailed estimate

#8

You are estimating things you've never done before

You can't do that!
Require time (or money) for research & prototyping

#7

You are forgetting organisational overhead

Fleshing out requirements
Code reviews
Development process
Acceptance testing
Deployment

We add 25% to every estimate to cover organisational overhead:

Requirements analysis	10%
Project management & QA	10%
Acceptance testing & deployment	5%

Scale those numbers by customer and organizational complexity

#6

You are scared of confronting your customer (or boss) with the cost of their requirements

Every developer I know is struggling with this
Pathological customer behavior reinforces this
- Focused on cost
- Low perceived value of your work
- Often has tight budgets herself

You just need to get over this
Trust the objectivity of your numbers
Don't write cheques your future self can't cash!
Work on getting better customers (or a better boss)

#5

You are working in an
interrupt-driven workplace

No priorities
No culture of letting people concentrate
Change it or leave it

#4

You're not fixing skill gaps
in your team

Always have mixed junior/senior teams and estimate for a mid-level persona
Junior developers can be very productive when you talk them through stories and unblock them often
Developers who learn on the job cannot count > 50% against the budget
Always be working at closing the skill gap
(code reviews, pairing,dedicated time for training)
Make seniors easy to reach for unblocking (e.g. chat channel #help)

#3

You have misconceptions about
agile methods

"We don't commit to dates. It's not the agile way."
Self-fulfilling prophecy: "Everything will be different than expected, so we don't plan anything"
You can and should plan out projects that take ≤ 4 MMs
Split larger projects into milestones
(and only estimate the next milestone)

#2

Your project is a mess

Estimates assume that you can get 1 unit of change for 1 unit of time.
For this you need:

Workable level of technical debt
Zero bug policy
Tests
Tests
Tests

But my project is a mess!

Then fix it!

Write integration tests for the happy paths
(they don't know that your code sucks)
Setup exception notifications
(tools like Sentry help with the initial volume of notifications)
Get customer/boss on board
Dedicate two days a week to fixing bugs, adding tests, refactoring code

#1

You don't manage changes

This is a talk of its own and there are many valid approaches.

My personal approach (YMMV):

Prefer milestones ≤ 3 person-months
Plan out what you can
Only do T&M projects
From day 1 scare your customers about
their project not getting done in time
Prioritize ruthlessly
It's the cheat code for software project!
Be a broken record about scope and prioritization
Update estimates every few weeks

10 reasons why your estimates suck

You're not taking the time required
You're not getting money (or time) for estimates
You're estimating things you've never done before
You'Re forgetting organisational overhead
You're scared of confronting your customer/boss with the cost of their requirements
You're working in an interrupt-driven workplace
You're not fixing skill gaps in your teams
You have misconceptions about agile methods
Your project is a mess
You don't manage changes

Vielen Dank!

Du hast Spaß am Programmieren und möchtest dein
Studium mit Praxiserfahrung kombinieren?

Wir bieten in den Bereichen Webentwicklung und Linux-Operations:

Praktika
Werkstudentenstellen
Bachelor- und Masterarbeiten
Trainee-Programm

makandra.de/karriere (oder mich ansprechen)