Extreme Programming Explained

While I've been sitting on this for a little while, chromatic has been patient. Yes, it's his review of Kent Beck's Extreme Programming Explained. The publisher is Addison-Wesley, and the book is for all those people out there who need to do programming but don't have time to do the engineering phase. Interesting book - click thru to read more.

Extreme Programming Explained
author	Kent Beck
pages	179
publisher	Addison Wesley, 09/1999
rating	7/10
reviewer	chromatic
ISBN	0201616416
summary	Extreme Programming Explained explains the virtues of the Extreme Programmer and shows you how to develop them.

The Hook

Want to write better code? How about working less overtime, getting along with your team better, meeting customer demands more quickly and accurately, spending less money, and having more fun?

Extreme Programming may be for you.

Be prepared to make some adjustments and sacrifices. Individual code ownership? Gone. Programming for the future? Slow down, cowboy. Working on your own? Grab a partner and dance.

What's the Scoop?

Extreme Programming is a way to improve software development by focusing on what really matters. If it will cost you $50,000 to implement a feature now that may not be used for two years, and it will cost you $55,000 to implement it in two years, hold off. If running test suites is good, write tests for every significant piece of the system. If multiple pairs of eyes make bugs shallower, program in pairs. If you enjoy meeting deadlines (and not working your fingers to the bone every night for weeks to do so), make shorter deadlines.

It sounds simple, even deceptively so. It may also set your teeth on edge at first.

Imagine that your customer has the time and the manpower to send a representative to sit with your programming team. He is actively involved in the design, writing 'stories' about how the system works for the end users. Every morning and afternoon, your programmers meet to decide which tasks to tackle, and they pair off, sharing one computer between them. One person codes and the other watches, and they switch off as necessary.

With every change to the system, the previous tests are rerun until they work perfectly, and new tests are added to test new functionality. Changes are not commited until all tests run successfully.

Releases are started early (six months, for a big programming project) and continue quickly after that (every couple of months). With a customer sitting in with the programmers, feedback can be instantaneous. The initial investment pays off quickly, while expenses are spread out over a greater period of time.

With no one owning a particular section of code, and with everyone working with different partners from day to day, everyone should have a good overview of the system as a whole. This can lead to better programming, from less bugs to very quick refactoring. New programmers can also be brought in and up to speed much more quickly.

What's to Like?

The book is clear and readable -- even funny. Chapters are short and to the point. Beck uses the metaphor of driving to bring his point across. (Driving is not about pointing in the right direction and maintaining that course, it's about making slight corrections all of the time.)

The bibliography is a great place to find some classic works (including books by Brooks and Knuth and even the movie 'The Princess Bride' -- no, really!).

Extreme Programming itself has a lot of promise. Some of the principles (programming for today, releasing early and often, peer review, community code ownership) fit in pretty well with open source/free software. Some of the other ones would be nice to see....

What Might Annoy You?

It's not clear where Extreme Programming fails. To the author's credit, he mentions this and gives some guidelines, but the choice and the implementation ultimately rest with the managers and bean counters. There will be some resistance at first, but Beck's enthusiasm is infectious and his clarity of explanation might be enough to overcome it.

The Wrapup

If you're a member of or a manager of a moderate programming team, you ought to read this book. It will go nicely on the shelf next to "The Mythical-Man Month". If you're curious about new ways to look at programming (especially in a group), you'll want to pick it up.

Purchase this book at fatbrain.

Table of Contents

1. The Problem
   1. Risk: The Basic Problem
   2. A Development Episode
   3. Economics of Software Development
   4. Four Variables
   5. Cost of Change
   6. Learning to Drive
   7. Four Values
   8. Basic Principles
   9. Back to Basics
2. The Solution
   1. Quick Overview
   2. How Could This Work?
   3. Management Strategy
   4. Facilities Strategy
   5. Splitting Business and Technical Responsibility
   6. Planning Strategy
   7. Development Strategy
   8. Design Strategy
   9. Testing Strategy
3. Implementing XP
   1. Adopting XP
   2. Retrofitting XP
   3. Lifecycle of an Ideal XP Project
   4. Roles for People
   5. 20-80 Rule
   6. What Makes XP Hard
   7. When You Shouldn't Try XP
   8. XP at Work
   9. Conclusion

end user v. manager feedback

I like the part about customer feedback (i.e., end user feedback). That is often hard to come by in projects I work on. The tricky part is balancing it with management feedback. A typical situation is one where the users of the software want ultimate flexibility, but management wants software that adheres to certain business rules. As in "you can't set a ship date for whatever you want, it has to be within five weeks of the order date."

BTW, all this is from the perspecitive of "business apps," which some twit said wasn't "real programming" here a while back. Whatever.

Sounds Promising

This method sounds like a programming corollary to the Cabal [gamasutra.com] system of design. (The method used to design Half-Life.)

If it's anywhere close to being as effective, it's definitely worth a look.

Spiral rehash?

This sounds much like the "sprial" lifecycle model, which has been around for some time. Risk management is very important, and with the spiral model you want to identify the risky-but-critical parts of the project and tackle those early. If you can't implement those critical areas, your project isn't going anywhere.

Comments about user feedback are right on the money. Frankly, if you're developing a mission-critical app and you *don't* have constant feedback from the users during development, you're asking for failure. Software engineers aren't domain experts, and we will screw it up we lock ourselves away from the users. Yes, you'll have management constraints but you must always present options to the user. "We can implement the 100% solution using X time/money, or we can give you the 80% solution in X/2 time/money and also have resources to attack Y and Z. You pick."

Neutron

Re:Planning is good!

There is an excellent interview with the XP creator in some recent issues of _C++ Report_ (which is itself a very respected journal). Some very good questions and answers are brought up by (IIRC) John Vlissides, the interviewer...

...whose last name I just horribly misspelled, probably. :-(

Extreme Programming web site

I can't believe that no one else has posted a link to the Extreme Programming web site [armaties.com] yet. A former coworker pointed it out to me. It is worth the time to read. Some of the ideas are excellent. The one I particularly liked which I have seen used by a group I work with is coupling an automated regression test suite with a requirement that all code changes must pass that suite before they can be committed.

Too many eyes

Reminds me of a time back in college when three of us were assigned projects in an AI class. One guy had no experience, so he did nothing. Two of us spent all nighters working on the code. One night, around 4am, our Lisp code wasn't working and we were stuck. We'd stared at it for hours, no luck.

"Wait a second!" shouted my partner, leaning over my shoulder and beginning to type.

"WHAT?" I asked, watching him move the cursor.

He cursored over to a piece of Lisp code, and pressed the space bar. "Sorry," he said, "That was bugging me. Now it's indented properly."

Where Extreme Programming Fails!

While I can see it working well in a number of circumstances, most of the projects that I've worked on it wouldn't work very well.

Project type 1 where is wouldn't work is any large project, say over fifteen developers (plus management). Any system that big and complex will not be understandable by everyone. Cross-fertilisation is a nice idea, but when things get too complex you need an expert or a keeper.

I guess you could have several experts, but you come up against the cost aspects very quickly.

Project type 2 are those with varying technologies. One I worked on recently used Oracle, Unix, NT, LotusScript (Domino), NotesPump, VBScript, JScript and HTML. I would argue that you can't be an expert in all of them. I do Oracle and Unix and have an appreciation of the others, but I couldn't really add much when I reviewed the work in other teams.

Again you could switch around each team once in a while, but any good team-leader would do that if they could anyway.

And that's before you even get into the 'egoless' bits, and the fact that management like people to be responsible. How can you have a scape-goat if it's everyones fault?!

Anyway, that's enough. Nice idea, but it wouldn't work in real life.

Save some cash....

This book is available at Amazon for $20.97, instead of the $29.99 Fatbrain wants for it.

If you are so into the whole patent thing, well go ahead and pay more I guess. I just saved 8 bucks, myself.

where it fails

I see a some big problems with the technique:

• no individual code ownership: this seems to suggest that the design would either be imposed from the top, which implies that coders would be treated as unspecialized labor, or that the entire team would be involved in the design of every piece of the code.
  
  Treating coders as unspecialized labor tends not to work because coders are not like factory workers but more like artisans or craftsmen. When you treat programmers like cattle they tend to resent it and deliver substandard code, at best.
  
  While getting the entire programming team involved with the design of every piece of the code might sound good, it results in design by committee, which doesn't tend to work in practice. The best designs come from inspired individuals or from very small teams of people.
  
  Finally, individual code ownership minimizes the need for communication accross the entire group (the N^2 communication problem noted by Brooks in The Mythical Man Month). By requiring a very high degreen of communication between all coders in the group, this technique is almost certain to fail for any group larger than a half dozen or so.
  
• new team members: the review asserts that "New programmers can also be brought in and up to speed much more quickly." but doesn't specify how. This is a similar problem to N^2 communication problem and was shown by Brooks to be one of the primary factors in killing large projects. The fact that the reviewer brushes this off so lightly, and that the table of contents doesn't show a chapter specifically dealing with it, suggests to me that it is not addressed.
  
  Ignoring N^2 communication issues is inexcusable in any methodology postdating The Mythical Man Month.
  
• debugging in pairs: debugging in pairs is a good idea, and should be used by any programming team that has the personnel to spare, however, it does not address the central problem of debugging. The problem with debugging is that it tends to take up a huge amount of the project time, and XP seems to want to add even more debugging to the mix (through the use of automated debugging tools). Just pairing your coders during debugging sessions won't reduce the total debugging time very much since it can't increase the number of cases covered or the range of debugging skills present by a significant amount.
  
• design to throw one away: The early release schedule seems to imply that the first coded design would be used as the production system. This is not a flaw solely in XP, but is rife in the current programming market. Many systems are built, essentailly, from the initial prototype, with no redesign. Brooks identified this as a problem back in the early seventies, but it doesn't seem to have made much difference to actual practice.
  
  A methodology that doesn't address the need to redesign the production system, however, will suffer from cost overruns and reliability problems (both of which XP is explicitly trying to avoid) due to the need to work around mistakes and compromises made when designing the prototype.
  
• unwarrented assumptions: the review seems full of unreasonable assumptions. Most programming projects don't have the latitude to set prices for each feature, or even to pick and choose which features to implement (at least not after the proposal has been made and the contract accepted). Even if you can set prices for individual features or components of the design, most features will not fall into the easy choice presented in the review. Any feature worth worrying about is likely to have effects across a large part of the design and leaving it out early one will pretty much exclude it later on as well.
  
  If XP is really dependant on some of these assumptions then it seems unlikely that it is a generally applicable methodology.
  

So, it sounds a good piece, but I don't see anything that sets it apart from standard practice or from design methodologies that have already been elaborated elsewhere. Some of the ideas are good, but others just seem to be naieve or illconceived: some even seem to smack of outright voodoo software engineering.

A personal experience...

A foundational part of XP is the mutability of code: you write the code for what you know now, and be willing in the future to totally change it. One of the best technical books I've read recently discusses the how-tos of this: "Refactoring [bestbookbuys.com]" (by the way, the link directs you to a price comparison site which I find REALLY useful).

I tried to use this method is my latest project, a test framework, and I was extremely pleased by the results; not only did the design morph as I discovered the requirements (so the final result was a design I was pleased with), but the final result snapped into place around the item being tested and no bugs have been discovered in it!

I can highly recommend at least that part of XP: refactoring and coding for today. I'm going to try some of the other parts in small bits at a time, but from what I see they look very useful.

I certainly wouldn't describe XP as being for someone who doesn't have time to design; I would describe it as something for someone who doesn't expect to get it right the first time, and who wants to get it right anyhow. I would also describe it as being for someone who wants to be useful to customers without hurting his own productivity -- see the Bill of rights [xprogramming.com] on the Extreme Programming website. In short, XP is about never having to tell a customer that you can't make that change now -- and it's also about always being able to make changes when you need to.

-Billy

Have the scoffers actually tried coding like this?

I have, and it works beautifully. In fact, the nastier and more complex the coding problem the better. Two people watching pointer arithmetic are always more careful than one.

That said, a few commonsense caveats must precede its adoption. Namely:

• Management must allow it
• If coders are of wildly differing skill levels, then they shouldn't be paired for long periods of time or swaths of code
• For shorter sections, an individual can take a "first cut" at coding something while somebody else tracks him with test code. Then there should be a trade off and review period.

I know of no piece of code that I've written this way that hasn't been FAR less expensive than its more traditionally coded counterparts, once debugging and QA expenses are factored in. The interesting part is that design and refactoring remain part of the process long after they would normally, as ones assumptions must be continually re-examined as people new to the code come in.