Why New Systems Fail 140
Posted
by
samzenpus
from the read-all-about-it dept.
from the read-all-about-it dept.
bfwebster writes "Over the last forty years, a small set of classic works on risks and pitfalls in software engineering and IT project management have been published and remained in print. The authors are well known, or should be: Gerry Weinberg, Fred Brooks, Ed Yourdon, Capers Jones, Stephen Flowers, Robert Glass, Tom DeMarco, Tim Lister, Steve McConnell, Steve Maguire, and so on. These books all focus largely on projects where actual software development is going on. A new book by Phil Simon, Why New Systems Fail, is likewise a risks-and-pitfalls book, but Simon covers largely uncharted territory for the genre: selection and implementation of enterprise-level, customizable, off-the-shelf (COTS) software packages, such as accounting systems, human resource systems, and enterprise resource planning (ERP) software. As such, Simon's book is not only useful, it is important." Read on for the rest of Bruce's thoughts on this book.
Phil Simon has written a long-needed and long-overdue book. Most risks-and-pitfalls book in the IT category focus primarily on projects where actual software engineering is the principal activity. However, many of the large, expensive and often spectacular IT project failures over the past 20 years have little to do with software design and development. Instead, they involve a given organization selecting and implementing — or trying to implement — a commercial off-the-shelf (COTS) software package to replace existing legacy systems, either homegrown or also commercial. The reasons for such a move can be many: standardizing IT and data management across the enterprise, seeking new functionality, retiring systems that are no longer supported or supportable, and so on. By so doing, the firm (usually rightly) thinks to avoid the risks and expense of from-scratch custom software development. However, the firm (usually wrongly) thinks that such a project comprises nothing more than installing the software, training some users, converting some data, and turning a switch. A quick search on the terms "ERP" and "lawsuit" shows just how mistaken that idea can be.
| Why New Systems Fail: Theory and Practice Collide | |
| author | Phil Simon |
| pages | 251 |
| publisher | AuthorHouse, 2009 |
| rating | 8/10 |
| reviewer | Bruce F. Webster |
| ISBN | 9781-4389-4424-1 |
| summary | Risks and pitfalls of enterprise COTS projects |
Simon's book is far more informative and instructive than a Google search and should be required reading for all CIOs, IT project managers, and involved business managers prior to starting any such enterprise COTS project. He covers the complete lifecycle of such projects, starting with the typical expectations by upper management ("Fantasy World") and following it through system selection, implementation, and production, along with a final section on how to maximize the chances of success. Along the way, he uses several real-word case studies (with names changed), as well as a few hypothetical ones, to demonstrate just how such efforts go wrong.
What Simon writes is spot on. For roughly 15 years now, my primary professional focus has been on why IT projects fail. I do that both as a consultant (brought in to review troubled projects to get them back on track) and as a consulting or testifying expert (brought in to review troubled or failed projects now in litigation). I have reviewed hundreds of thousands of pages of project documentation and communication; I have likewise traced or reconstructed project histories for many major IT projects, including enterprise COTS projects. It's clear that Simon knows exactly what he's talking about and knows where all the bodies are buried.
The book itself is very readable. Simon's tone is conversational and a bit humorous; he occasionally dives into technicalities that would be lost on upper management, but always comes back to basic principles. The real-world and hypothetical case studies will have those of us who have been on such projects nodding our heads even as we occasionally wince or shudder. His coverage is exhaustive (and at times a bit exhausting), but his goal appears to be to give those managing and overseeing such projects the information they need to navigate the shoals. He goes into detail about COTS pitfalls such as project estimation, vendor selection, use of consultants, group responsibility, integration with legacy systems, data conversion, and report generation.
The first section of the book covers how and why firms decide to initiate a major COTS project. Besides the "Fantasy World" section that compares management expectations to what really happens, the book also covers why firms hold onto legacy systems, why they buy new (replacement) systems, and how they can (or should) make the decision among building a custom system, buying a COTS system, and "renting" enterprise software via a web-based software-as-a-service (SaaS) vendors such as Workday and Salesforce.
The second section covers COTS system selection. The book divides current ERP and COTS vendors into four different tiers based on company size and use (e.g., SAP, Oracle and BaaN are all Tier 1) and warns of the, ah, enthusiasm of vendor salespersons. (Old-but-still-timely joke: What's the difference between a used car salesman and a software salesman? The used car salesman knows how to use his own product and knows when he's lying.) The book then raises up front an issue often left (by customers) until much later: how will business processes change as a result of the COTS system we're acquiring? It then talks about selecting, if necessary, a consulting firm to help with the installation and project management.
The third section covers the actual COTS implementation process, including the overall strategy, roles and responsibilities, providing the necessary environments, data migration, testing, reports, and documentation. This section is a bit exhausting at times, but it is critical for exactly that reason: far too many firms launch into a major COTS acquisition without fully realizing just what it will take to get the system into production.
The fourth section briefly deals with life after implementation. In theory, one of the reasons a firm buys a COTS system is to avoid doing its own maintenance and support; the reality is that the firm often doesn't like paying those large annual maintenance fees and instead goes off on its own path, which is seldom a good idea.
The fifth and final section talks about how to maximize the chance of success in a large COTS implementation. This section builds upon the rest of the book, which has provided suggestions along the way. In particularly, it talks about how to deal with a troubled project mid-course in order to get it back on track.
Throughout the book, Simon puts a significant focus on human factors in project success and failure. He identifies issues such as internal politics, kingdom-building, reluctance to learn new systems, internal project sabotage, end-user resistance, and staff allocation. Simon divides firm personnel assigned to work on the COTS project into four groups — willing and able (WAA); willing but not able (WBNA); not willing but able (NWBA); and neither willing nor able (NWNA) — and talks about how each groups helps or hurts. Similarly, he identified four dangerous type of project managers: the Yes Man, the Micromanager, the Procrastinator, and the Know-It-All. Again, those of us who have been on major IT projects, particularly those involving COTS implementations, will recognize both sets of categorization and the risks they entail.
While Simon is himself a consultant, he is also quite frank about the role consultancies can play in COTS project failures. In particularly, he notes the tendency of consulting firms to underestimate project duration and cost in order to win business, as well as the frequent unwillingness to point out risks and pitfalls to the client, particularly if they represent something the client wants to do.
My few complaints with Why New Systems Fail are mostly production-related. Simon self-published the book; as such, the book's internal layout and graphic design leaves something to be desired. Likewise, his organization and prose could use a bit of editing in spots; he has a propensity for throwing in terms and abbreviations without clarification, and the technical level can vary within a given chapter. Almost all of his footnote references come from Wikipedia; his bibliography is small (just four books) and cites only Brooks from the cadre of authors listed above. None of this makes the book's content any less important or useful, but some of the very people who should be reading this book might well skip or skim it for those reasons. My understanding is that Simon is working on finding a publisher for the book, which will likely solve all those problems.
In the meantime, if you or someone you love is about to embark on an enterprise-level COTS project, get this book; I've added it to my own short-list of recommended readings in software engineering.
You can purchase Why New Systems Fail: Theory and Practice Collide from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Alot of Software is Shit (Score:1, Insightful)
Let's be real here for a moment...
People who make alot more than I do look at a list of features. They don't tend to ask peons like me if these features are implemented in a reasonable way.
I'm not given the opportunity to warn of an impending clusterfuck until it's to late. By then it's not just my problem, it's everybody's.
Of course by the time it gets that far it is to late to turn around.
By the way, I heard second hand of very senior people at a company being fired because of an SAP implementation gone awry.
Me - I cram good websites in to shitty content managements systems. Generally i could personally develop most of the features that are in these CMSes if instead of dicking around with them I was just writing code.
I have personally spend over 3 days implementing a drop down list...
Re:Didn't need a book to know this (Score:5, Insightful)
I've had two projects fail ignominiously. One was my fault for not getting much more concrete requirements, and getting caught up in the "oh, and can you add this to it too?" The second was because I was basically lied to by a supplier who claimed their own product could do what it ultimately could not, and since it was a core feature of the system we were putting in place, the whole thing died, but not before consuming heap loads of money.
I learned a few things. The first is to get exact specifications. Let there be no wiggle room, no "well I thought it would do that" crapola. Extras can be added on once the core system is demonstrated to work, not before. Have a design philosophy and stick to it. As to lying suppliers, well, it's a lot easier to assess these things nowadays than when the one project failed in the mid-1990s. Still, I always keep in the back of mind "if software/library/whatever doesn't work, is there something that can".
Re:Software Projects vs. Traditional Projects (Score:5, Insightful)
how many bridges never get finished compared to the number of software projects that never get finished?
All bridges are essentially "open source". Plenty of bridges have failed, but the failures are right out there in the open, ready to be studied by anyone who wants to build another bridge.
In contrast, when a company's software project fails, the only people who learn from it are the ones involved with the project.
Re:Software Projects vs. Traditional Projects (Score:4, Insightful)
Not quite, you could do the same with a bridge, if parts of it collapse you can rebuild it.
The reason why software is so buggy is no one is being hold responsible. Software is the only product out there where catastrophic failures are accepted and people happily sit around waiting for new stuff.
If a bridge fails, some contractor is going to lose a lot of money, if someone is killed in the process they will be out of money (most likely). If software fails people just get a new update.
Re:Software Projects vs. Traditional Projects (Score:3, Insightful)
Indeed. With a bridge, the requirements are simple and obvious - you want a structure that permits transit from one side of some geographical divide to the other. All the detail is just detail, the end requirement is invariable.
With a software project, the requirements are often poorly understood or even unknown - a nebulous sense that things could be better if only we had better software. Often the software itself will reveal the real requirements.
Tolstoy's version (Score:5, Insightful)
People have written oodles of books on this subject, because there are oodles of different ways to screw up a project.
The best insight on this subject comes from Tolstoy, not Brooks. He was talking about families being functional, not software, but the principle is the same.
All happy families are alike; every unhappy family is unhappy in its own way.
A far better method of approaching this issue is to study projects that don't fail, not ones that do.
Re:Didn't need a book to know this (Score:5, Insightful)
What is different now from the 1990s? I've been involved with one software project that failed because the vendor promised functionality that they couldn't deliver. The client spent a significant amount of money on the project. Once it came out that the software couldn't do what the vendor promised it could do, the client sued the vendor and recouped all of their money plus legal fees. The client was able to sue because the vendor put it in writing.
Getting things in writing from the vendor is of paramount importance. Doing a needs analysis with the client before shopping around for software vendors is key. With a needs analysis in hand, you can present that to the vendors and ask them point blank whether or not their software fits the needs. If they say it does, make them sign a contract to back up their claims. Then they either deliver what they promised or they get sued.
The bigger the rollout, the harder the crash (Score:5, Insightful)
Also heard as "Why, don't you have confidence in your project"
Putting aside the sheer commonsense approach of not giving a porsche to a newly passed driver, most projects are run in a state of panic. Panic that the timetable is slipping (although this is almost always due to poor time-estimating, it seems to get presented as being due to slothful or untalented techies), Panic that it's costing too much - again due to poor cost estimation, rather than ovespending. Panic about bugs, Panic about training (ha!). Panic about compatibility with other systems. Panic about all the little patches, workarounds, working practices and hacks that have developed in the old system - that everyone knows about, but have never been documented.
All these, could have been identified and most of them fixed just by running a small scale prototype in parallel to the existing system. However by the time the project is halfway through, most of the directors are firmly engaged in either "buyers remorse" or utter denial. They become deaf to bad news and generally take full aim at the messenger, while leaving the culprits of all the problems unscathed. This is usually because all the biggest mistakes are made right at the start - in the design stages. However, these have been completed and signed off, so by definition cannot be at fault. The blame gets transferred down the line, to the people who have their hands-on right at the time the deadline is due. It's the original smoking gun: "The project ran over time / budget today - you were working on it when that happened, therefore you must be to blame". It's simplistic, always wrong and always starts off the finger pointing part of the process. You can't get away from it.
Although the biggest problem I see is "seagull" consultants. They fly in, make a lot of noise, crap over everything and fly off. The trouble usually only surfaces once they've disappeared.
classes of problems (Score:4, Insightful)
Technical - Large systems tend to be very complicated and it's difficult and expensive to make them fault tolerant and build in the sort of redundancy, validation, and security that make critical systems reliable.
Leadership - Decision makers on the client and supplier side often don't know enough details about various parts of the project to really know what they want much less what they need.
Organizational - Setting deadlines before defining the scope of the project, belligerent coworkers and other HR issues, uncooperative clients, cutting testing time to meet deadlines, and other general issues within the organizations can lead to death march development and other undesirable situations.
Re:Software Projects vs. Traditional Projects (Score:3, Insightful)
People are taken out of the equation when it comes to bridges. You don't have to teach people how to use *your* bridge since the use of a bridge is the same regardless of which bridge they use.
People are the main reason why I see projects fail. Incomplete/incorrect requirements, artificial deadlines, glory seekers, scope creep, poor training, process change, resistance to process change, etc. These are all variables that don't have to be considered when building a physical structure.
And unfortunately they're also variables that aren't given enough consideration in the project from start to finish.
I'm currently waiting for my company to replace the system at the heart of our IT support functions. The one we've built around our business processes for the last 9 years. The one they honestly believe can be replaced by a Big System in about 2 months. And they wonder why I drink.
The really sad thing is that the system they're dead set on replacing isn't broken, is still in active use out in the wild, is off the books, and has no real reason for being replaced except for a manager who wants to put his name on the replacement project and his manager who is convinced that since the system isn't one of The Big Three we shouldn't be using it any more.
Re:Didn't need a book to know this (Score:2, Insightful)
I think you need to take it one step up the abstraction chain. First, find out what the goals of the users are: what real-world problem are they trying to solve? Forget traditional "requirements" and such, which just leads to discussion about the "system" in the minds of the user and the "system" in the minds of IT. Which, by the way, are greatly different. Once you first figure out what they are trying to accomplish well above requirements, only THEN can you start to figure out the requirements to meet their goals.
The major problem is that everyone in IT projects forget to first figure out the users' goals, but rather go straight to requirements. Requirements depend on goals.
Re:Software Projects vs. Traditional Projects (Score:4, Insightful)
A better book would be "How New Systems Succeed".. (Score:4, Insightful)
Because why they fail is not all that interesting. A project specified mostly by people who don't know what the system is supposed to do, implemented by people who don't understand the business, replacing a legacy system containing within its labyrinthine bowels the combined knowledge of tens or hundreds of expert users past and present. What could possibly go wrong?
Add on top of that a COTS requirement, so it's a matter of making the requirements fit the software's limitations (while still fitting the business), and you have an almost guaranteed recipe for failure. Particularly when the users _won't_ adapt.
Software is hard (Score:3, Insightful)
You can't see it, touch it, smell it, taste it etc. Most of it is an intellectual abstraction many programmers, not to mention the general population, isn't very good at.
Doing software well requires being an expert in complex problem domains. The domains may require knowledge of complex financial, legal, engineering and manufacturing systems. It may require modeling human relationships. Or combinations of all of the above.
Where people get things wrong is they do not take the time to understand their problem domain. They look for magic bullets. They need to spend time with their end users and understand the work processes. A little business process modeling goes a long way.
Re:Tolstoy's version (Score:3, Insightful)
All happy families are alike; every unhappy family is unhappy in its own way.
That's interesting. I think just yesterday I said exactly the opposite.
Everyone I know who's furiously pusued the American married-with-2-kids nuclear family ideal is miserable. Everyone I know who has some alternative-y lifestyle (without marriage, kids, lucrative job, white picket fence, or some combination) seems a lot happier.
In reality everyone is miserable.
Re:Didn't need a book to know this (Score:3, Insightful)
You make a good point, but it is totally unrealistic. Let me demonstrate where it will fail:
A more realistic approach is to get the best requirements you can, and build enough time into the project to handle 1.5-2 years worth of scope creep because that's what's going to happen with any huge system.
If you overbid by 1.5-2 years, you are sure to be outbid by a competitor who will stick to the "rigid requirements" method. So you will never receive a contract in the first place.
If you try to hardline your users by forcing them into a corner with rigid up front requirements that they cannot possibly help you formulate, they'll simply go outside the company and work with someone who knows how to run a project better and you'll get laid off.
Yes, but if you let the schedule slip those very same users will suddenly remember that you have a contract and force you into a corner with rigid contract stipulations about deadlines. If you want to avoid that, you'd better act first.
Don't forget, "the users" is not a single homogenous group, they are a collection of individuals, each with their own ideas and agenda, about half of which will hate your guts on principle (you mess with their computer, their routine, and their certainty about the future for no good reason they can think of). If you listen to them, they will tug you into a hundred different directions, roughly half of which are on the wrong side of a tall cliff.
I've been doing this for 20 years and I've seen the approach you are talking about fail over and over even with PM's that have 30 years experience. They knew better but corporate policy forced them to operate this way.
They just know that rigid requirements don't work. But do they have an actual alternative? You've apparently chosen to work on failing projects for the last 20 years, that tells me something about how difficult it is to find a project that's managed differently.
Inevitably the requirements were hopelessly incomplete and the users were pissed off when they had to sign off the project as complete because of what they agreed to, and in the end, the product did not meet their needs. The whole idea is to give the users the product they need. So even if you succeed in beating them on paper, and they are forced to sign off complete, you've failed.
Actually, the whole idea is to make money building something to specification. That is (apparently) what you were hired to do. If you don't like it, set up your own company, make your own rules, and fail to get _any_ customers because you are consequently overbidding and customers cannot tell the difference between you and the huge number of penny-pinching nitwits that define the rest of the industry.
Know what happens when you do this to your users? They hire contractors, who will be more flexible and give them what they want, and fire you.
Contractors will either work on a fixed budget, in which case they will either demand fixed requirements or stop working once the budget has run out, or they will work on time and materials basis, in which case they will be happy sitting on your premises and drinking coffee (and the occasional bit of programming) until the sun goes out.
You are better off with a "Look this is a big system and it's going to take a while to get it right. Lets figure out what you think you need now, we'll build it, and use that as a starting point to flesh out your system."
And you really claim to have 20 years of experience? First of all, a lowly peon will never get to sit down with the people who make the decisions and have this sort of talk with them. If you somehow managed to do it anyway, they will smile and say that they cannot budget for an open-ended development project (which is what you are proposing), so they are just going to go with a fixed set of requirements and the associated f
Re:Didn't need a book to know this (Score:3, Insightful)