Monkey See Monkey Do aka Agile WTF (Way To Fail)

Here's the talk Naresh and me did at Agile Bangaluru 2010.

This was the talk's abstract:

We don't have all the answers. We don't know the best way to build software in the right way. But we do know one thing: the right way doesn't involve mindlessly following practices just because some "expert" says you need to.
In this workshop we'll take a critical look at various "agile" practices and try to highlight the dogma and ceremony that has creeped in. We'll also question if the practices defined a decade ago are still applicable? If yes, have they evolved since? What are some of the original creators of these processes practicing today? And so on..

It's almost 2 hrs, and the first 25 mins of the presentation is a spoof of how agile projects are run. We eventually get around to discussing some agile practices and the tradeoffs they involve:

Here are the slides:

Designing Your Startup

This is the video of my talk at Agile India 2012 (it's missing about 5-10 minutes from the beginning). I'm totally winging it, so I've goofed up a lot, but in the spirit of shipping, here it is:

Why I prefer Prioritylog over Backlog

Backlog has negative connotations for me; of not keeping up (falling behind), of making slow progress, etc.

I prefer Prioritylog instead, because it emphasizes the fact that the list should be prioritized (so that it can be pulled from).

Continuous Software Maintenance aka Agile Software Development

Its widely accepted that about 40-90% of the total cost of an application is incured after it goes into production, the phase that is traditionally called maintenance.

When I'm doing Agile, not the practice-oriented farce kind, but the values-principles-oriented real kind, I don't build software in phases. I develop software in a continuous flow of activities. When I'm doing this, it feels like I'm continuously maintaining the software instead of only maintaining it after its done, and that makes all the difference.

We Don't Need No Version Number

When your delivering working software in a continuous flow, versioned in a DVCS, do you really need version numbers?

Stories

This post by Joel made me smile and got me thinking about an interesting way to highlight the differences between the various ways we software developers perceive what it is we do. I haven't gotten around to describing the perceived benefit/reason of the As a user, I want user story template yet, but here goes:


As an engineer,
I want to build software
so that ...

As a coder,
I want make software
so that ...

As an author,
I want to write software
so that ...

As a craftsman,
I want to create software
so that ...

As a developer,
I want to evolve/grow software
so that ...

Estimating in Ideal Time might not be ideal

Ideal Time estimates are almost inevitably perceived as estimates of duration leading some (especially managers) to compare the Ideal Time estimates with actual Elapsed Time. Velocity then becomes a measure of efficiency, used to determine how close they are to removing the impediments that the developers encounter everyday. This might make sense, except that Ideal Time is not really about time and the good intentions might actually be detrimental if the developers find themselves explaining why they only got x Ideal Time of work done in x+y Elapsed Time.

The root of the problem is the assumption that the x in the x+y Elapsed Time is the same as the Ideal Time x.

Relative Measures of Size

Story Points
NUTs (Nebulous Unit Of Time)
Gummi bears
T-shirt Sizes

Velocity, and deriving an estimate of duration

By measuring the team's rate of progress (velocity) in terms of the number of size units of work [1] done in an iteration [2], you can arrive at an estimate of duration in terms of the number of iterations required to complete the remaining tasks: remaining size units of work divided by the teams velocity.

Instead of having to think about and account for the inevitable overheads that occur everyday, velocity gives us a way to map ideal time into elapsed time based on historic data.

1. ideal time or a relative measures of size
2. fixed length, comprised of working hours of working days

Ideal Time, an estimate of size

Ideal time (or ideal development time of effort) is the amount of time it will take you to finish a task if you had no interruptions or impediments. It is an estimate of size or effort and has little to do with time, since you rarely ever get contiguous blocks of productive time.

Estimating in ideal time is affected by various factors like the developers own cognitive biases, their skill and experience levels, complexity of the problem, the design of the software, and available tools, to name just a few.

Ideal time can be used to derive an estimate of duration.

See also:
Estimating in Ideal Time might not be ideal

Elapsed Time, an estimate of duration

Traditionally, software developers are expected to give estimates of duration in elapsed time. Elapsed time is the actual time (working hours of working days) it takes to complete a task. This includes time spent on things (inevitable everyday overheads) other than the task at hand.

Estimating in elapsed time is inherently difficult because, apart from the various factors that affect the estimation of effort for a given task, you also need to worry about: distractions and interruptions [1], unexpected events or obstructions [2], and just about anything else.

1. meetings, emails, interviewing candidates, phone calls, reviews, task switching, training, bug fixes, etc.
2. other projects, absences, developer turnover, non-cooperative team members, sabotage, etc.

Accurate Estimates

Accurate estimate is an oxymoron. You might get estimates that are better, never accurate.

Deliver When Done

When I was first introduced to Unit Testing, I didn't like it because it was doing things to my code that I didn't like. Over time I realized that it was my code that was the problem and not the tests. I seem to be in that same place again with wanting to continuously delivery working software without the artificial boundaries of iterations and releases. Where you make very thin slices of your tasks and deliver a slice when its done. Where every commit is also a deploy.

Note to self: Look up incremental development-and-delivery model

Estimation and Targets

For some, estimates serve as targets. Targets set the pace at which work gets done. They change one's relationship with time (you get a lot more work done on the day before going on vacation than on a regular day).

From Fundamentals of Software Engineering (Second Edition) by Carlo Ghezzi, Mehdi Jazayeri, Dino Mandrioli:

A surprising finding was that, in a controlled experiment, the subjects who had no schedule at all finished their project first, beating out both the group that had an aggresive schedule and the one that had a relaxed schedule. Experiments have shown that engineers, in general, are good at achieving the one tangible goal that they have been given: If the primary goal is to finish according to a given time schedule, they usually will-but at the cost of other goals such as clarity of documentation and structure.

From Software estimation considered harmful by Peter Seibel:

Sometimes we set targets in order to convince others, or ourselves, that something can be done. We may set targets to inspire ourselves to do more, though it’s not clear that’s a winning move, and even less so when managers set a target to “inspire” the folks who work for them. (See DeMarco and Lister’s Peopleware, chapter 3 and the discussion of Spanish Theory managers.) We may also set targets to give ourselves a feeling of control over the future, illusory though that feeling may be. After the fact, a target hit or missed can tell us whether or not we did what we set out to do. However if we missed a target, we can’t know whether that’s because the target was unrealistic or because we didn’t perform as well as we should have. Setting and hitting targets does make it look like we know what we’re doing but we need to keep in mind that targets rarely encompass all the things we care about — it’s much easier to set a target date for delivering software than a target for how much users will love it.

If the goal is simply to develop as much software as we can per unit time, estimates (and thus targets), may be a bad idea. In chapter 4 of Peopleware, DeMarco and Lister discuss a study done in 1985 by researchers at the University of New South Wales. According to Peopleware the study analyzed 103 actual industrial programming projects and assigned each project a value on a “weighted metric of productivity”. They then compared the average productivity scores of projects grouped by how the projects’ estimates were arrived at. They found that, as folks had long suspected, that programmers are more productive when working against their own estimates as opposed to estimates created by their boss or even estimates created jointly with their boss, averaging 8.0 on the productivity metric for programmer-estimated projects vs 6.6 and 7.8 for boss-estimated and jointly-estimated. The study also found that on projects where estimates were made by third-party system analysts the average productivity was even higher, 9.5. This last result was a bit of a surprise, ruling out the theory that programmers are more productive when trying to meet their own own estimates because they have more vested in them. But the real surprise was that the highest average productivity, with a score of 12.0, was on those projects that didn’t estimate at all.

Estimation and Closure

A lot of us are uncomfortable with doubt and uncertainty. Estimation is a way for us to put a fence of certainty around the inherent uncertainty of software development. However, coming up with a date for when a task will be done, does not remove the uncertainty, it just hides it.


From Fundamentals of Software Engineering (Second Edition) by Carlo Ghezzi, Mehdi Jazayeri, Dino Mandrioli:

Estimation is always accompanied by the risk of being wrong.

Silver Bullet?

Is minimizing repetition the single most important adaptive design (Probe-Sense-Respond vs. Sense-Analyze-Respond) technique?

From Lean Development & the Predictability Paradox by Mary Poppendieck

A key technique for absorbing changes easily is to avoiding repetition like the plague. If you have to say the same thing in more than one place – either in design documents or code – then refactor the design to consolidate the capability into one place. One of the most effective ways to facilitate change is to localize every potential change in only one place.

From Extreme Programming Explained: Embrace Change (2nd Edition) by Kent Beck, Cynthia Andres:

If small steps are how to design, the next question is where in the system to improve the design. The simple heuristic I have found helpful is to eliminate duplication. If I have the same logic in two places, I work with the design to understand how I can have only one copy. Designs without duplication tend to be easy to change. You don't find yourself in the situation where you have to change the code in several places to add one feature.

IMO, if you couple minimizing repetition with writing code that is communicative, you should be in a fairly good place.

Estimation vs. Guessing

Without information and experience you are not estimating, merely guessing.

Perfect Process, Perfect Design and Perfect Stories

From Extreme Programming Explained: Embrace Change (2nd Edition) by Kent Beck with Cynthia Andres:

In software development, "perfect" is a verb, not an adjective. There is no perfect process. There is no perfect design. There are no perfect stories. You can, however, perfect your process, your design, and your stories.