Book Review: The Economics of Software Quality 83
First time accepted submitter BenLinders writes "The Economics of Software Quality provides solutions to quantify software quality, helping you to manage software development and maintenance. It contains software quality data that you can use to build a business case to improve the quality of your software, and decide upon processes and techniques that can help to implement the needed improvements in your organization." Read below for the rest of Ben's review.
Quantifying software quality is not an easy thing. Several measurements exist, for instance estimating and tracking the number of defects that are found (both within development/maintenance and from customers), measuring software quality with static analysis tools (complexity, fan in/fan out), or measuring the effectiveness of software development methods and techniques (like inspections, test, and Cost of Poor Quality). This book covers software quality factors that influence the quality of software products as perceived (and believed!) by customers. An extensive list of factors is provided, where the authors have selected those factors that they consider most significant to achieve quality.| The Economics of Software Quality | |
| author | Capers Jones and Olivier Bonsignour |
| pages | 587 |
| publisher | Addison-Wesley |
| rating | 8/10 |
| reviewer | Ben Linders |
| ISBN | 978-0-13-258220-9 |
| summary | To build your Business Case for Software Quality Improvement |
Many software development processes and techniques are covered in this book, from a quality and economic point of view. This also includes agile methods, where a body of data is available about the effects of agile techniques like user stories, Test Driven Design, Scrum Sessions, Measuring Technical Debt, and Pair Programming. For instance, about agile user stories the book states "... the user story method seems to be concise and fairly trouble-free, User stories average below 0.5 pages per function point and seem to contain fewer than 0.5 defects per function point`. This kind of information can be very helpful to build a business case for using agile methods in your organization.
Most of the data on software quality that the book provides is in "Defects per Function Point". A backfiring table is also provided, to translate language statements/lines to function points. So if you are not using function point, but programming in Java, Ruby, C++ or any other popular programming language, the data can still be used.
There is a full chapter covering defect prevention. Methods like Reuse, Formal Inspections and Quality Function Deployment are the most effective in preventing defects, and also techniques like Root Cause Analysis and PSP/TSP are claimed to be very effective. Given that the top ten techniques reduce defects with 40% — 85%, makes it interesting for many organizations to investigate the business case to improve the quality of their products, using these methods and techniques.
Additional information is provided on how to measure the effects on quality from a given method or technique. The book also provides warning for quality measurements that can be unreliable. An example is measuring cost-per-defect. When the quality of your development activities increases, for instance by improving requirements practices and implementing defect prevention for design and coding, the number of defects that testing finds will go down. Since test case preparation is a fixed cost, the cost per defect for testing will go up when the software has fewer defects. This makes such a measurement potentially unreliable. I believe that the main benefits will come when you can reduce your testing activities, based upon measurements that quantify the quality of your products before testing starts. Techniques like risk based testing can also reduce your testing hours, thus saving time and money on tests that are not needed.
Defects measurements and tracking are used in more then 55% of the military and defense software applications (using CMMI, TSP, QFD, etc), but in less then 15% of IT, commercial, web or embedded applications. Given their prevention effectiveness of -35%, and removal effectiveness of 25%, it is still surprising to me that this is not used more often. The data needed for these kinds of measurements is usually available in the defect management systems, though some addition effort is needed to classify defects and to do Root Cause Analysis. The benefits of using these kinds of measurements, combined with estimations of the expected quality at release, to decide and steer software development and prevent defects during the development and before release are significant.
The book also gets into methods to quantify structural quality issues that are not exactly "defects" but have an important impact – "Technical Debt" being one of these methods of quantification. These kind of measurements help to manage the quality of your code base, being able to see the impact on quality from changes, and take action to get quality back on the desired level when needed.
Reviews and inspections are very effective ways to remove defects before testing. Several techniques are described, both informal and formal techniques. Several of them are also usable within agile methods, supporting teams in developing better quality software. Applying these techniques effectively requires training, and arrangements within your company that enable employees to use them. The book makes clear that if you want to reduce post release defects and lower your maintenance costs, the work needs to start with early software development activities, like using better techniques for managing requirements, software modeling and design, reviews and inspections, and automatic code analysis. Testing alone is not sufficient to improve quality, and is also very costly.
The relationship between quality and risks is also explored. Many major software problems are related to the quality of the software products, e.g. outages, data loss, security issues or regulatory non compliances. Investigating such issues, for instance with Audit or Root Cause Analyses, and taking action to prevent similar problems in the future can be essential for your business. Measuring the losses and estimating potential benefits from preventive actions helps you to select the right improvements, and acquire commitment and funding to implement them.
The capabilities and skills of the staff that develops the software have significant impact on the quality. The benefits of training, skill development, and sharing of experiences to develop a learning organization can be huge. Software methods like Agile and RUP include mechanisms to continuously evaluate, learn and improve the capabilities of your staff. E.g. using retrospectives and scrum boards, to identify and follow up with improvement actions.
Overall the book covers the economic perspective of quality. The information provided can be overwhelming for some readers. If you need to improve your product quality, and are limited in time and money to do it, this book helps you to select effective quality methods and techniques, and to measure and track your progress when implementing improvements.
Ben Linders is a specialist in quality, process improvement and organizational development.
You can purchase The Economics of Software Quality from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Agile programming is a lie (Score:4, Informative)
There, I said it.
"This programming philosophy will allow you to develop high quality software really quickly, and on the cheap" is the equivalent of a politician promising to fix every problem in the country with no sacrifices required, and put chocolate milk in all the water fountains to boot.
It's always the old thing: fast, cheap, or quick--pick any two.
Re: (Score:2)
umm...
fast as in good performance and quick as in rapid development?
I heard it before but now I can't remember it.
Re:Agile programming is a lie (Score:4, Informative)
Aren't fast and quick the same thing?
Re: (Score:2)
There's a slew of ven diagrams out there on google in regards to the comparison :)
Re:Agile programming is a lie (Score:4, Funny)
After consulting my MBA decoder ring, I will pick "cheap". Twice.
Re: (Score:1)
Re:Agile programming is a lie (Score:5, Funny)
My post was fast and cheap. If you wanted me to proof it better, you should have given up one of those two.
Re: (Score:3)
Agile is the interruptor the business side has always wanted: a leash for engineers.
But it gets much abused, resulting in needless rewrites, and scattered or stunted architectures - seen it at three tech-centric companies now.
"Refactor" is a banished term in Agileland. And you never really get to refactor, to design patterns or anything else, because you're too busy replacing hand-wired code to fit the latest redesign or business strategy change.
Tech management everywhere have lost their collective spines,
Re: (Score:3, Informative)
Should be:
Fast, Cheap, or Good - pick two.
Re:Agile programming is a lie (Score:4, Informative)
Should be:
Fast, Cheap, or Good - pick two.
And expect at most one.
Re: (Score:2)
And if you work for certain developers, spend several years working to end up with none.
Re: (Score:3)
Agile programming is not a lie. It works, and the company I work for has the customers, profits, and happy non-burnt-out engineers to prove it. But it's not easy to do right, and certainly isn't cheap if you're hiring the right people.
Re: (Score:1)
Oh, this brings back memories, long forgotten, painful memories. [slashdot.org]
Re: (Score:2)
Read the linked comment about XP. On his setup of "two 90/hr contractors"
not being better then one: how about 1 70/hr and a 35/hr one? I have seen
*very* good coders routinely spend forever on simple issues, spending the
the night working to find things that were almost proofreading in nature. Not
the way I want to time spent. The second coder can help keep the
first developer sane in approach,, focused, and with a venue to verbalize approaches,
not least of which is because only when you teach a thing, can you
Re: (Score:3)
Which error? Are you referring to the "then/than" error, the sentence fragment, or the doubled comma, or the complete mess that the last sentence becomes after "approaches,"?
I mean, that's all I found in 30 seconds, there might be more.
Re: (Score:2)
Not needed you say? Okay, can you find the error i made in the paragraph
above? (I'm not joking, start a timer, how long did it take you to find? It's
there!).
Less than a second, probably less than half a
second. But I know what kind of error is easy
to slip into a paragraph of text like that, so I was
was looking in the appropriate places and it stood
out obvious.
Re: (Score:2)
Same here. The oldest non-mistake in the book.
Re: (Score:1)
can you find the error i made in the paragraph above
I assume you are not talking about your multiple grammar errors (like double commas), so it must be that you think that good coders spend forever on simple issues, or maybe it's that you believe one must spend time teaching others to be a quality coder.
None of that defines of a good coder. Having a $70/hr and a $35/hr coder on the same thing is also stupid. It's good for the contractors, good for the agency (or whatever company that provides them), but it's stupid for the end-customer, who ends up being b
Re: (Score:2)
I still can't believe they actually named the obvious "common sense" into a programming model. I've seen a project go from waterfall to agile in its dev cycle and go from abstract to relatively meeting business needs through the communication. I don't know anybody that can devise a complex system on paper and factor in for all the little sutleties that require decisions from the system creator as well as compensate for the business practices. It's just a fast track to getting overwhelmed fast when using
Correlary (Score:5, Insightful)
Everyong knows: Fast, Cheap, or Good - pick two.
But not everyone knows its correlary:
Slow, Expensive, or Wrong - pick one.
-Rick
Re:Correlary (Score:4, Insightful)
Counterpoint: Non-trivial software is never cheap. It will either cost more up front for quality or more after delivery when it doesn't work the way it's supposed to.
Re: (Score:2)
Its most critical to realize that neither of those numbers is fixed; the first is "at most" and the last is "at least".
I've certainly seen projects where, in regard to the last trio, all three were chosen (or, at least, achieved.)
Re:Correlary (Score:5, Funny)
Re:Agile programming is a lie (Score:5, Insightful)
A typical software process innovation happens like this:
1. Group of highly skilled and motivated developers create a new process (agile, code review, team programming...)
2. They see the results are great and start writing about it.
3. Other skilled and motivated developers take note of the new developers and start implementing it. The new process gains acceptance.
4. Consultants and the general software community picks up on the idea and starts implementing it.
5. Projects fail as usual as unskilled and unmotivated developers and organization can't execute it.
The key to software development is that is starts with the people. Success in software is 95% people, 5% process.
http://slashdot.org/comments.pl?sid=2560806&cid=38280588# [slashdot.org]
The team of highly skilled and highly motivated developers that create the new and cool processes would have been able to build a successful project using almost any process... or even no process.
Software processes can enhance a highly skilled team and highly motivated team. They can't do much if you don't have the talent or motivation across the organization.
Re: (Score:1)
Motivated means well paid. No manager will ever agree to this.
Re: (Score:1)
Motivated means well paid. No manager will ever agree to this.
I would say motivation means way more that that, but yea it starts there.
Re: (Score:2)
It's not just money. I'm pretty happy with my salary.
What I'd like
1. stability to know my investment in a company is long term. It doesn't help when they lay off senior staff...
2. Professional development, mentor ships...
3. Well run management, projects... but that is highly tied to 2.
Re: (Score:2)
The team of highly skilled and highly motivated developers that create the new and cool processes would have been able to build a successful project using almost any process.
Excellent point.
Re: (Score:2)
Yep. I'm not against process by any means, but it has to be productive.
Considering how much of the success of a project is based on people... it's amazing to see how little 'people process' there is. Processes to make sure you hire/keep skilled people. Processes to develop skilled people (mentorships...)
Once you have the right people developers, then you can make use of the other processes to enhance things.
Fields like medicine, law.... have extensive people processes. Internships, residency, credential
Re: (Score:3)
It would appear that you don't understand agile.
When done right, agile is the formalization of good work habits (realizable short term goals with something to show at the end of each sprint in order to reach the overall long term goal). It also eliminates a lot of overhead time spent in planning that gets thrown away, unnecessary gant charts, etc.
The long and the short of it is that agile saves time by making the process that a good dev would impose upon himself the formal process for the product.
If you th
Re: (Score:3, Insightful)
Unfortunately, while Agile may formalize good work habits, it does not magically turn poor or mediocre programmers into good ones. No matter how much Agile you throw at a problem, if you have average developers, you still get average software.
Re: (Score:3)
However, if you have great programmers, but saddle them with a really badly implemented waterfall approach, by way of example, you'll also wind up with poor to average software in all likelihood.
Unless your devs manage to overcome the methodology. But you don't want your devs to be battling both the problems of the code and the management approach at the same time, now do you?
Re: (Score:1)
Re: (Score:2)
Jesus, that HAD to have come from an engineer. An engineer could make peeling an apple a pain in the ass.
Re: (Score:1)
There, I said it.
"This programming philosophy will allow you to develop high quality software really quickly, and on the cheap" is the equivalent of a politician promising to fix every problem in the country with no sacrifices required, and put chocolate milk in all the water fountains to boot.
It's always the old thing: fast, cheap, or quick--pick any two.
You beat me to it. Agile is growing like a cancer, every job description at least mentions it if not requiring some experience in it. It seems to me that managers believe that it's 'Agile or die'. It isn't, if you're poor development practices and processes already in place suddenly 'becoming Agile' won't help anything. I've read employee comments about startups like Zygna, while they probably proudly proclaim to be Agile their developers consider it chaotic.
Re: (Score:2)
You can have fast, cheap, and quick if the scope is tiny. You can't have fast, cheap, and quick if the scope is meaty. Scope is the magical fourth factor when comparing the other sides of the triangle.
Something something Diamond is the en vogue chart for the popular engineer trades related to this. And guess what? I am not spending time on this post to go lookup a reference to the chart I'm talking about that's somewhere on Wikipedia. This would be out-of-scope for me. Maybe somebody else will add i
Bullshit. Agile programming is a manifesto. (Score:2, Informative)
The Agile Manifesto doesn't make any promises about quality, speed or price. The only lie here is that your claim that it does.
www.agilemanifesto.org
Manifesto for Agile Software Development
We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
T
Re: (Score:2)
It might be, but II've never seen that as the main promise of agile . The big value I've seen from agile is that small, independent iterations reduce the cycle time from the investment of resources to getting some return, and manage r
Re: (Score:3)
Agile is not really a lie. As originally formulated, it was really just a rehash of the common sense approach that was practiced by many in any case, just without all the fancy labels. But, as with any brand, consultants hopped on the train selling "the Agile" as a silver bullet. Many of the same people also write the books, so in the end it ended up as yet another snake oil. But it doesn't mean that basic tenets such as having relatively small iterations with specific deliverables don't work.
As with any ot
Re: (Score:2)
Just because you are spending a lot of money/time, it doesn't mean the thing you are spending it on is useful. Rarely anything runs in an optimal way*, thus nearly every time there is rom for increasing some of those variables without reducing the others.
Agile programming is an aswer to iterated design. Altought iterated design was an answer to the wastefull beast that is waterfall development, it still didn't get ride of all the waste. Agile improved it. Now, it won't make your software write itself, or ta
Re: (Score:2)
Nah, agile provides gains compared to many other methodologies, but for two reasons:
-It does its best at avoiding retarder requirements processes that lead to spending way too much time rewriting code or programming secondary features
-Provides practices that help team cohesion, which in turn lower turnover.
Too many employers have insane requirement processes and have nobody that works on the same team for more than a year. By just avoiding said demons, agile will provide improvements. If you already have a
Re: (Score:1)
Yes, Agile facilitates development of high quality software. The team I work on develops and maintains a distributed sensing and processing system including multiple CPUs, OS installations, something like 100 purpose-built software components, several hundred off-the-shelf software components, and custom hardware. These systems are deployed worl
Re: (Score:2)
Are you saying the Agile developers I've worked for aren't true Scotsmen?
Re: (Score:1)
Actually sounds interesting... (Score:2, Interesting)
But at 600 pages, I just don't have the time to sit and read it. What I do have is a 45 minute commute twice a day that make for some excellent listening time. If I could get this as an audio book on CD, I'd pick it up tonight.
-Rick
Re: (Score:1)
Can you just hire an Indian programmer to read it to you?
Re: (Score:3, Informative)
Re: (Score:2)
Nice! Thanks for the link, I'll check it out.
-Rick
don't forget the organization itself (Score:5, Informative)
While it's popular to focus on code metrics (defect count, test-based metrics, etc.), when it comes to how to improve software quality, don't forget that it's strongly related to organizational characteristics. Whether you look at it via Conway's Law [wikipedia.org], via Fred Brooks's analysis [amazon.com], or via recent empirical research [pdf] [umd.edu], it's pretty clear that software developed in an organization isn't independent of the organization itself, and sometimes the way the company is structured/managed is the problem.
Re: (Score:1)
Re: (Score:3)
So long as we stick to capitalism, the only ultimate code quality metric is "how well does it sell?" - since that's the only thing actually relevant to the company (note, this includes side effects such as customer dissatisfaction from shipping crap once leading to dwindling sales on later releases).
Re: (Score:2)
Peopel dont buy quality, they buy Brand (Score:2, Informative)
"If it compiles, ship it. If it sells, patch it."
"Nobody ever got fired for buying IBM."
THAT is the software sales model. That, and having people who know braindead CTOs who buy from their golf buddies, creating a giant naked Emperor.
Re: (Score:2)
It's an interesting idea.
What I find shocking is the DBA being close to the janitor.
I bet they want to be with the architects!
Just sayin.
Re: (Score:1)
well, they are in front of the support, and though in many places they do have some say in the architecture, I think in most shops they are delegated about the same amount of power as the support.
Re: (Score:1)
Oh, right after the ACs, I think you take priority in any line up of that nature.
Re: (Score:1)
Re: (Score:1)
Wrong, I am the janitor.
Six Sigma (Score:2)
I felt like I was studying for the Green Belt exam reading that review.
Cost (Score:2)
$57.47 Hardcover
"Software quality" even exists? (Score:2)
I'll believe that when I stop getting monthly Critical security updates from Microsoft, Apple, Oracle and Adobe for products which already passed their entire battery of industry-standard best-practice software QA tests.
Re: (Score:2)
On the Suse linux machine I have handy,
Back in the day, the BSD version was a shell script. I know IBM had to update it for licensing reasons. As far as I know, all the bugs have been removed. (But there's still a chance someone will introduce on with an update)
Re: (Score:2)
Yikes! (Score:5, Insightful)
If you have to "build a business case to improve the quality of your software," you are in deep shit. Don't buy the book, change jobs instead.
Re: (Score:2)
Not everybody believes in "spend money to make money". There are other business models as well. And there are many lemons in the market, as you probably already know.
Re: (Score:2)
Not everybody believes in "spend money to make money".
Or even "first do no harm", which in the software world translates to "first install no botnets on your customers' machines". Certainly Microsoft, Apple, Adobe and Sun/Oracle don't, since they're the top placers for drive-by botnet installs. But none of the open source projects are immune either - Mozilla and Linux still push out monthly security updates.
What worries me most about the present (let alone the future) of software quality testing is that we still don't even have a way to tell if we're doing har
Re: (Score:2)
Re: (Score:1)
It contains software quality data that you can use to build a business case to improve the quality of your software
If you have to "build a business case to improve the quality of your software," you are in deep shit. Don't buy the book, change jobs instead.
Amen.
As Deming said "Most of the costs of quality are unknown and unknowable."
Mod UP! (Score:2)
Here. This is it. Too bad I've already posted.
Somebody please mod the parent up.
Re: (Score:2)
actually, I think you're quite wrong.
Everything in life is a cost/benefit tradeoff. The notion of building perfect software "at any cost" is similar to how Rolls Royce thought about building cars. Making something the absolute best possible it can be is a praiseworthy ideology, but there is obviously an associated cost for doing so.
The world is fortuneate that the Henry Ford's of the world _also_ got into the car business, and decided NOT to compete with Rolls Royce on quality, functionality, etc.
Supposin