Extreme Programming for Web Projects 197
| Extreme Programming for Web Projects | |
| author | Doug Wallace, Isobell Raggett, Joel Aufgang |
| pages | 165 |
| publisher | Addison Wesley |
| rating | Poor |
| reviewer | PinglePongle |
| ISBN | 0201794276 |
| summary | A book about applying the Extreme Programming methodology to web projects. |
Can good ideas dominate the buzzwords?
This risk -- the authors contend -- is the reason many web development projects fail in one way or another. The client's objective is to obtain maximum value, the developer's to incur the least cost possible without getting sued.The authors show a way in which this risk can be shared fairly between the client and the developer, by using XP and iterative development cycles, alongside a release plan, to acknowledge the risks inherent in a development project, and manage them rather than try to pretend they don't exist. The project team -- client and developer -- work together to create an iteration plan, and use this shared understanding of the requirements to guide the project.
The book is structured into 4 parts: Part 1: XP and Web Projects explores the problems associated with web development projects. Part 2, Working on Web XP Projects explores some of the practicalities of the authors' process - iterative development cycles, the development environment, team roles, and the graphic design process. Part 3: XML and Web XP is a bit of an oddity in a methodology book -- it focuses on some technology-specific issues which the authors claim can be addressed by using XML. Part 4: Web XP Best Practices discusses planning, design, coding and testing issues.
What's good about this book?
Well, there are some insights into the relationship between suppliers and customers in development projects. (I don't believe, though, that they're as specific to web projects as the authors seem to claim).
What's bad about this book?
It seems to be a sales brochure for the author's web shop -- "we do things thusly, and it yields fantastic results every time." The text is full of fairly broad, even sweeping statements ("Many programmers put SQL code right on a web page" -- when was the last time you saw a select statement on a web page ?).The authors do not really seem to be able to identify those aspects which make web development projects different from other types of development. Some of the team roles they recommend are bizarre -- the authors identify the role of "Strategist" who seems to help those poor idiot customers to understand their own business. This may be necessary on some projects, but I find this attitude very condescending -- the days when web development was portrayed as a cross between alchemy and spiritual enlightenment are long gone. Many of the sections are very superficial, but the book is littered with footnotes saying "Chapter X discusses this in detail."
In short, I'd say this book is too lightweight for people who understand XP already and want to learn how it applies to web projects, and novices are likely to get hung up on the largely redundant side tracks (CVS versus MS Sourcesafe -- Huh? How did that get past the editors?) to be able to see the extreme wood for the trees.
You can purchase Extreme Programming for Web Projectsfrom bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
I'd rather see Extreme Grocery Bagging (Score:5, Funny)
This books sounds like buzzword fluff.
Re:I'd rather see Extreme Grocery Bagging (Score:2, Insightful)
This is a rather short-sighted view. In 1997, I was faced with the same choice: Experience, or degree. I chose both. Finished high school at the end of 1997, started doing a Bachelor's degree in Information Technology part time, in 1998, looked for work, had a few odd jobs but nothing major, and in my 2nd year, 1999, started an e-billing company with 2 partners. Finished my degree with honours in 2001, and also had a moderately successful company. (Moderately successful back then, substantially more successful now). Good luck to you with your degree only, though.
Admittedly, it has a lot to do with luck, but I'm pretty sure that most people doing their degrees right now could improve their situation significantly, instead of getting their degree and then wondering if it's going to be all that useful in the ultra-dynamic I.T marketplace.
My hiring policy is simple: if you have a degree, great, if not, also fine. It all boils down to how you do in the technical test that my colleagues and I have devised. It's based on knowledge of the following subjects: Operating systems, Python programming, C programming, Zope usage, and Network knowledge. It doesn't matter how many degrees you have, if you want to be a programmer with us, you need to do well in this test. Most of the degreed applicants' CV's look something like this:
Look, degrees are impressive and all, but most of the CV's that have them also contain large amounts Microsoft, and that's not acceptable in firms that use non-Microsoft technology for scalability and stability reasons (our custom e-billing app needs to serve millions of requests per day and send large amounts of mail without crashing, freezing, BSODing, or whatever the case may be).
As for other organizations, from all I've heard and experienced, people with degrees are more likely to get laid off because they tend to cost more than non-degreed people, and in business, it's often the bottom line that counts, unfortunately.
From a skills perspective, people who have been programming since their teens or earlier tend to be the best technically (I was one of these people), and degrees do indeed help to sharpen and augment those skills, but are by no means the end-all and be all of programming, experience is far more important.
Hope this helps, and I hope that you manage to find a job.
Re:I'd rather see Extreme Grocery Bagging (Score:2, Insightful)
Re:I'd rather see Extreme Grocery Bagging (Score:2, Interesting)
An online auction company I recently worked for has a massive hang up about MBAs. If you want to do anything technology related that's not actually coding, for example Project Manager, then you'd better get yourself an MBA.
Also during recent reading of job advertisements I've seen a lot of ads that say something like "must have good degree from a red brick university."
real xtreme programming... (Score:5, Funny)
Re:real xtreme programming... (Score:3, Funny)
Re:real xtreme programming... (Score:2)
OK, so this particular book fails: iterate, then.. (Score:1)
What do you know-of that is effective in knowing and communicating what this book purports to be?
reliability (Score:5, Insightful)
Re:reliability (Score:5, Insightful)
If a program passes all the tests, but doesn't work the way you want it to, your tests are not good, i.e. you requirements are wrong or incomplete.
The main argument against XP is that some systems are so complex that you cannot build tests to cover all the functionality. Take for example a Quake engine. The combinations of objects, positions, orientations and actions are virtually infinite, so it is impossible to write tests to cover everything. For a system like this XP is not the right approach, and the creators of XP would tell you that.
Re:reliability (Score:2, Insightful)
I'd expect many real-life systems to be complex enough that exhaustive testing is not practical. My point is that there needs to be a formal approach to s/w development. As Dijkstra said, testing reveals the presence of errors but not their absence. In safety critical environments, while testing is necessary, it cannot be and shouldn't be deemed to be sufficient.
Again, I'm not passing a blanket statement on ``XP'', I'm merely asking if there is enough statistical evidence of its reliability.
Re:reliability (Score:5, Interesting)
I guess my issue is that the XP approach does not dictate anything about design. You can and should design well even when doing XP. In fact, in the original book there is a lot of talk about continually reworking the design to simplify and clarify it, and avoiding the temptation to add complexity that is not necessary at the moment, just in case it's needed in the future.
So, I think of XP as a set of tools, none of which restricts or prevents good design.
It would be interesting to collect stats on projects are compare reliability produced by different design methologies, platforms, languages, tools, etc., looking for correlations, but XP is just one piece, and does not equate to a design methodology.
Re:reliability (Score:3, Interesting)
Actually, it explicitly does. One of its practices is Testy Driven Design: every element of the programmer's design is expressed in the form of a test, written in automatically executable form before anything is added to the code.
Another rule XP requests is to minimise Big Design Up Front: don't do design on something else else until you've proven the design for what you just did. In other words, don't design something and then leave it unimplemented; let every implementation be completed as close to its design time as possible.
I don't use XP, since my company has its own methodology (CMM level 3); I do use Test Driven Design. It works extremely well, and has the pleasant side effect of producing tests to hand in with my code for no additional effort. Given what I've experienced with TDD, I would take up XP in a heartbeat.
So, I think of XP as a set of tools, none of which restricts or prevents good design.
XP is not a set of tools, although there are one or two practices in XP which make sense outside of XP (for example, pair programming can be imagined in another context, and TDD works very well without XP). Rather, it's a complete methodology, much like the Spiral methodology or the much-feared Waterfall (and no, waterfall isn't a myth; I've seen it used a couple of times, and most of the rest of the time it's a good approximation).
-Billy
Re:reliability (Score:5, Insightful)
The practices try to make it easier to produce good code design, but they could be applied to any design methodology. You could do XP Object-Oriented or not, with Java, COBOL, perl, basic, or assembler. There is nothing to make your software design good or bad except your skillz and the developer sitting next to you.
So, I'd say XP doesn't enforce any coding methodology, but tries to create an environment that encourages good habits.
Maybe we are having semantic issues. I was talking about the methods for designing at a fairly low implementation level the actual code that makes an application work - the parent comment here expressed concern about code made under XP being sloppy. That's what I meant by design. If you talk about RUP or Waterfall as a design methodology then obviously they are incompatible with XP.
Re:reliability (Score:2)
You do know that those aren't design methodologies, right? XP is a lifecycle methodology, and thus includes a design methodology, as well as systems and practices which attempt to ensure that the methodology is followed.
There is nothing to make your software design good or bad except your skillz and the developer sitting next to you.
Wait, don't forget about all the other developers who'll be refactoring your code, and the tests which serve as much to document the original intent of changes as to test the continued validity of the changes, and the constant integration which keeps your code polished, and weekly or biweekly product deliveries that keep your customer in the feedback loop, and the planning game that makes sure important requirements are found and developed for, and...
So, I'd say XP doesn't enforce any coding methodology
You'd be right, but this is the first time you've said that; you've been talking about design methodologies before.
, but tries to create an environment that encourages good habits.
Nope, not XP's job -- that's for the team to do. You could say that XP describes a set of "good habits", and how they interrelate to reinforce each other.
I was talking about the methods for designing at a fairly low implementation level the actual code that makes an application work
Do you know the difference between "design" and "implementation"? I ask because in the above sentance you mix them up.
the parent comment here expressed concern about code made under XP being sloppy.
The parent comment poster should speak for himself, but I have to say that I did NOT see that in his post at all. What I saw was that he felt that code produced using XP would be unreliable: in other words, it would fail to meet requirements all or part of the time, either due to bad understanding of the requirements, bad solutions to the requirements, or bad implementations of the solutions.
That's what I meant by design.
Well, that's not design. That's coding standards. XP addresses those too, but leaves all the details to the team doing the actual work.
If you talk about RUP or Waterfall as a design methodology then obviously they are incompatible with XP.
So are you agreeing that XP includes design methodology?
BTW, minor point: RUP is a flexible system which can include XP. See Rational's own website [rational.com] for what I'd consider dramatic proof.
-Billy
Test-First is too simplistic (Score:4, Interesting)
To start with, let me say that the XP approach utterly failed in this case, but it was probably in good part to the people we were working with still learning Java at the time and also having terrible design skills (or rather I should possibly say, design experience in other languages but not in Java which led to ill-fitting design). So, they probably would have generated drek no matter what approach they used.
However, as I was around from the start of this project I was able to make some interesting observations. The first is that you are correct, if you build code that passes all tests but still does not do what you want, then your requirements used to build the test are wrong. However, I've always been confused by this aspect as I thougt a requirement-poor environment was supposed to be where XP was helpful...
The larger issue I have noticed through my own experience as well as this project was that test-first is too simplistic a strategy. Instead I would break testing into two sorts - "scaffolding" and "regression".
The original project did not break up tests in this manner. As a result, the core of the project became encrusted in tests. After a while more work was being done on tests than on the actual project... at which point the buisness owner for the project raised hell, and testing was dropped altogether. Obviously that was a bit too far, but it did get the project moving, and ended in a state where it worked (though I still have a terrible mess to clean up and a very fragile system).
Back to the breaking out of tests... I think they would have been much better served by "scaffolding" tests that are created during construction of a system, but ultimatley are thrown away when the system works well so you do not have the overhead of maintaining tests while adding to your system. Then of course there would be true "regression" tests that managed to cover most of the application testing, and ensure large portions still worked after changess... but they would be few enough in number that enhancements of fixes would not mean more work fixing tests than code.
I've used a scaffolding approach in my own work and designs, and find it woorks really well. It gives you enough tests to get much of the touted test-first benefits, but does not leave you with a system that cannot be changed for fear of cascade fixes on multiple, ancient, tests.
I would be interested to hear what criteria other people use to abandon a test when doing test-first programming.
Tests (Score:3, Insightful)
On this occasion, I did what you suggest: I built unit tests during development and threw them away when they were costing more time to maintain than they were saving in bugs that would not have been found another way. Once it was possible, I developed tests from the user's API level (its a middleware product), and these will be maintained for the life of the project. These API level tests include regression tests for specific bugs.
The XP folks seem to suggest maitaining unit tests at a level I consider excessive. I think they suggest one test per non-trivial method on all classes. This just seems too much, and very hard to achieve, since even in the best designed projects, individual methods are hard to test without also testing a bunch of other stuff.
It sounds like your project was suffering mainly from a lack of design skills. Spending a lot of time maintaining tests implies too much coupling between components: the only way that can come about is if changing one component affects many others, so there tests need updating too.
Re:Tests (Score:2)
There are two common phrases to sum up what to test in XP:andWhat that actually maps to in your code depends a lot on your code. But I think the most important rule is that if a bug gets through despite my tests, it's a sign that I guessed wrong about what to test.This just seems too much, and very hard to achieve, since even in the best designed projects, individual methods are hard to test without also testing a bunch of other stuff.
It's true that it's hard, but there are a variety of techniques to make it easier. The more I do test-driven design, the more I discover that when an object is hard to test, it's a sign that I should improve the design generally by reducing coupling and simplifying my objects.
It sounds like your project was suffering mainly from a lack of design skills.
Yeah, XP gets beat with this stick a lot: "We took a bunch of novices and they produced a system that was poorly designed! It must be the fault of XP!"
I think that no matter what level of design skill a team has, XP is a great way to make the best use of that skill. But people shouldn't fool themselves: making complex software requires substantial design expertise, and a team without a good percentage of grizzled veterans is on the the short road to critical mass [developerpit.com].
Re:Tests (Score:2, Insightful)
Re:reliability (Score:2)
Reliability is the point. (Score:5, Insightful)
Then you don't understand the whole approach. XP evolved by taking good programming practices (automated testing, peer code review, "integration builds", client communication, etc.), and kind of going overboard with them. As a result, it typically generates higher quality deliverables. For example, in XP, before you write code, you write the test first [c2.com]. Instead of weekly code review meetings, you code with someone else [c2.com]. Instead of everyone writing their code in isolation, then slamming it together to see if it all works, you see if your code works together [c2.com] all the time. Instead of waiting for one or two business days to get a requirement clarification from the customer, you get that information [c2.com] from them much sooner.
There are more practices such as these that make up XP. I've worked on many projects, particularly with the big consultancies, that use waterfall-type methodologies that fail. A 600 page design document [c2.com] is useless when the requirements change during the project, which they always do, due to market demands, time constraints, etc. XP is no silver bullet, but it makes a lot of sense after you've been through lots of projects that go over budget, without all the desired features, even after working 80 hour weeks [c2.com].
Re:Reliability is the point. (Score:3, Insightful)
There is generally no such thing as a true waterfall in a real project. When one occurs is usually an indication of design by committee and bureaucracy, where the project is doomed regardless of methodology.
I really think latching onto a branded methodology, such as XP or RUP or Waterfall or whatever, for a project is the wrong approach, because it is only good management that makes XP or RUP or Waterfall or whatever work in the first place. The methods themselves are for informational purposes only. They can influence a managerial approach, but anyone exclaiming, "We follow XP methodologies," just looks silly and will invoke resistence from the more cynical team members.
Good managers avoid thinking and speaking in terms of such buzzword methodologies, but they also freely admit that they learned something from those methodologies and applied what they learned to their own approach. Good arguments beat buzzwords when trying to convice the skeptics and cynics among us.
Re:reliability (Score:2, Interesting)
You are one of the many who have never read more than two sentences about XP, and jump to the conclusion that because eXtreme is in the name it must be hacking.
I think your assumptions are the exact opposite of what is actually true; "extreme programmed" programs should be more reliable, because they don't guess for tomorrow. If XP is done right, what you have is what you need...nothing more. This is the essence of KISS. If you can't make it reliable by doing the simplest thing possible that solves the problem, I shudder to think what you will produce if you are guessing for tomorrow and adding things you don't need.
There are plenty of studies... just google for them.
Scott
Re:reliability (Score:5, Interesting)
Would you ever trust any other program in this sense? Just curious.
I feel (perhaps just a prejudice) that
Sounds that way. Unless you have some facts that you haven't shared with us...
extreme programming is a "do now, think later" kind of approach.
No, it's the other way around -- "think now, do now." You're supposed to do things as you think of them -- when you come up with a clever design, don't write it down and walk away; instead, codify it as a test and then implement it. All the practices of XP are supposed to come together to prevent premature or delayed action: pair programming ensures that every idea put into code is understood enough to be explained to your peer; TDD ensures that every change made to code actually _adds_ some functionality; and so on.
I'd be interested in knowing if there have been studies with respect to reliability of extreme programmed projects.
A very interesting question! I think that we'll have answers on that in about five years. XP as a movement is a little new; very few companies have had a chance to adopt it, and only a few formal projects have been run with it.
Until then, any XP shop wishing to perform a high-reliability-required task would be wise to adopt some of the known reliabilty practices -- formal reviews, for example.
-Billy
Re:reliability (Score:4, Interesting)
It's actually difficult to track because the original project didn't monitor error rates particularly well. We do know that they had more than 1000 errors over 14000 "processing cycles" (a defined chunk of work.) Our current system has had 7 errors over 7000 "processing cycles".
The vast majority of these improvements come from Pair Programming and Test-Driven Development.
And - of course - we now have tests for those 7 errors!!!
(For those who care, a "processing cycle" has about 10 feature-points that are executed - so the original error rates are somewhere around 1% (pretty dang high) and ours is around
Re:reliability (Score:2)
This matches my experience, too. I'm well below one shipped bug per man-month.
Re:reliability (Score:2)
I don't know a lot about it myself, but in certain domains it tends to produce good results.
Re:reliability (Score:2)
One of the guys is now the leader of a programming team and has been trying to get his team to use XP - if not all of it, at least some of it. As one of the books on the subject has said: you have to have the right people with the right personalities and working style to make XP work.
Just like anything, no method is perfect - it has to be adapted for the project/team/etc. Pure XP might not be the perfect answer, but certainly it has some good ideas that can be integrated at a team, project, or even sub-project level.
Buzzword compliant. (Score:3, Insightful)
Surely not another book with a buzzword in the title that doesn't actually focus on the alledged subject at hand?!
wtf (Score:5, Insightful)
About 10 seconds ago on my current project. I fail to see why I should make a seperate file and include it for one SQL statement, or even for the 20 or so I use...
Can anyone explain the logic behind this? If they mean "when was the last time you saw SQL directly put in the html", then the answer would be never apart from the mysql manual...but then surely thats obvious...if they mean "when was the last time you saw SQL in a page-generating script", then I don't get the problem.
Re:wtf (Score:3, Insightful)
Re:wtf (Score:5, Insightful)
Many web programmers don't know the difference between the presentation layer, business logic, etc...and they just throw it all together. You end up with a site that works great in its current version...but is a bear to add functionality to or redesign visually.
Re:wtf (Score:3, Interesting)
While I do know the difference between the presentation layer, business logic, and data storage - what if the customer doesn't actually know the full requirements of the site (or indeed a specific section of the site) prior to them wanting it to go live?
I personally love working on projects where the presentation logic is separated from business logic, but unfortunately the majority of my customers need it in production yesterday, and have no time themselves to actually go through the requirements, nor are willing to commit to requirements that will no doubt change in 3 months time.
And for them, I believe, its just a matter of being in the business they are in - their core business has practically nothing to do with I.T. (short of perhaps the checkout scanners at the front of each of their stores). Its only that they are a nation-wide company, and therefore have an image to uphold.
Re:wtf (Score:2, Interesting)
The Jr. members of my team don't understand how I can make major changes to some of my programs in hours when it takes them days to make simple changes. Well - it's all about the rewrite. I don't find myself rewriting my code as much as I used to...it was like the day I learned that by writing object oriented code, I didn't need to completely reinvent processes to make modifications...just add new methods or - in cases where requirements change - new methods+properties.
And - what I always tell my coworkers - if they needed it yesterday, it's already late. Take 2 hours and come up with a good plan...
Re:wtf (Score:3, Insightful)
Re:wtf (Score:2, Interesting)
a presentation layer, eg jsp over a presentation package, eg tags
and
a business logic layer, eg java classes over a data abstraction layer, eg bc4j
If you do this, you get re-usable, maintainable code. If not, and you're building anything bigger than a pretty noddy app, you're not going to be able to do anything with it six months after the initial build.
Hell, I've even built noddy apps badly and had a hard time maintaining them six months later.
Incidentally, wtf is extreme programming? Somebody?
The last time I saw a select statement ... (Score:5, Interesting)
Almost every ASP project I've seen has embedded SQL in the presentation pages (.asp files). Yuck.
Re:The last time I saw a select statement ... (Score:1)
Same as Java or any other multi tiered system. Presentation-->Code behind(for
The 'better' way - abstraction (Score:2)
Basically, rather than call 'select field1 from table1 where field2=value2' you call some sort of function. That function contains the information needed to get the information out of the database.
This way, if you ever need to change your backend, you don't have to go picking through every web page on the system to see where that data object might have been referenced.
Now, for one-off projects, this is a cumbersome annoyance, which only gets in the way of the end product. If you're putting something online for a week, and it's going to then go away, this probably isn't worth it. If it's going to stay around for years, and you may have to make changes to your data model or add some wierd functionality later, it's better to have the abstraction, as it keeps you from having to essentially rewrite everything.
In some cases, the extra level of abstraction may save you processing time, as well. Oracle will have to maintain multiple execution plans if there's even a difference in capitalization of field names [even though they're not case sensitive], but by making sure that everone calls the same function, which calls the same exact SQL query, you avoid this problem.
This also makes it much, much easier to optimize your code, oracle or otherwise, as there are times when just the order of the items in the query will affect the execution time. (as it may affect how it does joins, etc).
Now, in your case, you have one layer of abstraction, to keep you from retyping the exact query each time, you may still want to have seperate functions that get a specific set of information, so that you can optimize or otherwise modify them as needed.
Re:The 'better' way - abstraction (Score:2)
When a custom tag already provides an easy-to-use and easy-to-setup abstraction to access the database, to create some bean/utility-class/extra-code just to get a resultset seems cumbersome and useless. The data access layer duplicates functionality and gets in the way in small JSP projects, it seems, because the tags can do that work in two lines.
Something I do when the project starts to grow and I start to feel uneasy about those SQL statements is to use JSP fragments as my data layer, and call them as functions through jsp:include tags.Since the JSP tag I typically use translates the resultset into collections, nothing stops me from switching data sources and data fetching methods without touching presentation code. All I have to do is make sure the data layer JSPs store the expected object where I expect it in the request object, and assign it to a local bean/variable if I must.
Some may consider it hackish, and some may have some prejudice to use JSP for anything but presentation, but it makes it very easy to move from a prototype with a bunch of repeated-and-embedded SQL to something cleaner without redesigning, or reimplementing, that much.
Then, if you really need a cleaner abstraction than that, with your own data access classes, your data access JSPs are only another layer and moving there is even easier, and doesn't touch any other code.
More than just for changing the type of database (Score:2)
Now, under some situations, you can define a view that mimics the original table, but when you're adding extra interfaces to an existing product, and you don't have control of the underlying structure, you have to deal with what might happen when it's upgraded.
It'll normally occur when someone gets on a kick of normalization, or they decide that there's a need to handle journaling on a table.
Obviously, you know what sort of an environment you're dealing with, and how likely the odds are of something like this in happening. The more recently you're taken on new staff, or changed management, the more likely this is to happen.
Re:The last time I saw a select statement ... (Score:3, Insightful)
Re:The last time I saw a select statement ... (Score:2)
And most software projects also fail. There's a correlation.
Cutting corners with software is generally a false economy. You may get to the first release faster, but cut corners add up exponentially. By starting out cutting corners, you're betting that the project will fail.
Instead of cutting quality, wouldn't it be better to cut out useless work?
I don't think I've ever seen a requirements document that isn't at least half fluff. In XP, every week you force the product manager to pick the very most important features to work on. On an XP project, most of fluff never rises to the top of the pile. Once you cut that crap out, there's plenty of time to do quality work.
And if you're doing test-driven design and pair programming, your bug counts will likely drop by an order of magnitude. If you spend hardly any time on debugging, that frees up a lot of time, too.
Re:The last time I saw a select statement ... (Score:2)
I agree with you on the rest of your comments. I don't think PHP or ASP are any good. We use server-side Java/JDO to do most of our development. Large project. (Once a) Fortune 500.
Re:The last time I saw a select statement ... (Score:2)
Re:The last time I saw a select statement ... (Score:2)
Re:The last time I saw a select statement ... (Score:2)
Usually followed by some crazy deadline - good people do bad things all the time. God knows I've hacked out some ugly stuff while the sales guy is doing a demo....
Re:The last time I saw a select statement ... (Score:2)
Re:The last time I saw a select statement ... (Score:2)
Usually when you take "a lesson" on MVC, theinstructor tells you it's the 'good' way to do things.
Re:The last time I saw a select statement ... (Score:2)
OK this one isn't great, but is there a great one? (Score:2, Interesting)
Does anyone know of any good suggestions?
Re:OK this one isn't great, but is there a great o (Score:2)
"Performance Solutions" by Smith et al is probably the best modern (i.e. recognizes OO)treastise on how to design a system for performance.
For the Java world, I would suggest "Advanced JavaServer Pages" by David Geary and/or "Expert One on One: J2EE" published by Wrox.
There isn't much on integrating XML services out there, and there's a lot of freedom.
As for SQL services, the book I found most helpful is Expert One On One: Oracle by Thomas Kyte. It's oracle-specific, but a very high signal/noise ratio.
SQL in web pages (Score:3, Insightful)
I think the author mean in php/asp/other embedded languages pages, where finding plain sql code (instead of calling a library that hides for portability the sql code) is very common yet.
Of course, that don't display in the visitor browser, at least, not unless is intended (.phps) o "accidental" (the old ::$DATA that showed the .asp code), or is part of the debugging process (visible or in the html generated code)
XP is a manager's dream (Score:5, Insightful)
Re:XP is a manager's dream (Score:2, Informative)
Re: (Score:2, Insightful)
Re:XP is a manager's dream (Score:2)
That's a blanket statement that's not entirely true -- our shop uses it quite a bit... and the Manager are just fine with it. And no, things don't ship late.
Re:XP is a manager's dream (Score:2)
I'm not sure that's the case. XP's scope control techniques have cut costs substantially in the projects I've done. Ditto for the defect reduction. Is your experience different?
Next week's review: X for Y (Score:5, Funny)
Ask someone what "they do." (Score:2)
"I'm a web developer," or "I'm a Java programer."
People who answer that question in that manner will generally buy these books, in fact, insist on them. It never occurs to someone who thinks of himself as "a suit" that there is such a thing as *cluefulness,* detached from what he "does."
Publishing a book on cluefulness would smack of "theory" to these people, and theory, to these people, means not practically useful.
Cluefulness for Suits for Dummies, however, the suit can recognize as describing himself to a tee ( although the irony of the accuracy of that description often eludes him). It smacks of being useful, to *him*, without all that extraneous stuff about cluefulness in general.
I mean, really, who would want to waste their valuable time just being clueful, in general?
KFG
Ignorant Review (Score:5, Interesting)
I think the author was refering to something like this:
<html>blah blah blah
<?php
$sql = "select
db_query($sql);
?>
blah blah
</html>
and not actually displaying the SQL in the output of the page.. both are bad, but showing the user a query is worse..
Too bad the reviewer doesn't seem to know enough about the subject to actually catch on to this.
S
Re:Ignorant Review (Score:2)
Right now I'm trying to (re)learn AxKit, which can be used to develop such; I provide an XML document (whether static or from some CGI request), use XSL stylesheets to take the XML to XHTML, then use good CSS practices to mark that up correctly. Other engines, like the Struts engine for JSP, do similar type of things. There's numerous libraries for Perl, Java, Python, etc, that allow one to adapt to existing services to help with that separation (for example, I'm thinking of things like the Template Toolkit for perl).
I would think that the point that is being missed is that it's easy to write do-everything pages that don't separate content and form, but web apps that have good content/presentation separation are typically much easier to maintain and much easier to augement with new features.
Now, as to how this can be done with XP (without having read the book), I question, since I would imagine that the advice suggests that the designer and the programmer are sitting at the same terminal by the usual XP philosophy. But that's a mystery left to the book.
The point isn't the techie stuff... (Score:2)
Have I seen a lot of embedded SQL in my time ? Sure - I've even written some. Is Web XP the only cure for this evil ? Uh...no. In fact, does any process shield you from poor design or poor implementation ? Well, no, probably not.
Another quote that annoyed me : while talking about the quality of web site code, "Few developers used an object oriented approach to development. Most used procedural languages, which made refactoring code or making changes to it much more difficult". Well, I'm a bit of an object bigot, but I would contend that the key tension here is not OO versus procedural, but rather time versus quality. I've worked on code in procedural languages that was very easy to work with, and OO systems which were an unholy mess. The reason many web sites are brittle is not that they don't use the right inheritance tree, but rather that they are put together in a very short space of time, and with ever-changing requirements.
How much XP is there in the real world? (Score:5, Informative)
XP was all the rage for a little while there. There was talk of it everywhere, especially here. I read about it some and became very interested in it. I think many of the core ideas are valid, and it seems like it would be a fun way to develop.
Now that the hype has died down, my question is this: How many people out there are really doing XP? How much has this really caught on? Is it just a bunch of talk?
If you are actually doing XP, tell me a little about:
* the industry you are in
* the kind of project
* how it was done before
* what prompted you to make the switch to XP
* how that switch work and how long it took
* and how things have been since moving to XP
* do you know others doing XP, if so how many
Thanks for sharing your experience.
Re:How much XP is there in the real world? (Score:2)
* the kind of project
Web based publishing systems, cross publishing systems, search engines, log analysis.
* how it was done before
Pretty much like XP, but not so formalised. I.e. the customer was in charge, no hard contracts but rather a relationship and mutual trust with customer, delivering a quick bottom line solution first and then add features on request..
* what prompted you to make the switch to XP
It made good sense judging from my own experience. My degree from University is in Systems Development methodologies, and I have always found the ones we were taught to be very bureaucratic. I also worked as a project leader in a research environment were we used a lot of prototyping.
* how that switch work and how long it took
We're not yet ready with any smooth running unit testing framework, but it took maybe one month to get pair programming and stories to function well. The company is Webworks [webworks.se] (in Swedish), BTW.
* and how things have been since moving to XP
Very relaxed, no worries, although I have a tendency to burn out when I'm driving in pair programming since I can go so fast now. Customers are happy too.
" * do you know others doing XP, if so how many"
Nope
Re:How much XP is there in the real world? (Score:4, Informative)
To answer your questions one by one:
* the industry you are in
Financial Industry ( focusing on real-time trading application in the options, futures, and equity industries )
* the kind of project
Mostly applications that either provide infrastructure for doing electronic trading, or applications that implement particular trading strategies.
* how it was done before
The sorts of applications I would work on were often quick prototypes that were meant to exploit an opportunity in the market. This would often mean lots of shortcuts were taken because chances are the code would be thrown away anyway. Problem is that once in a while an application will become become succesful and grow out of one of these prototypes. Since little time was available for proper design, such applications generally became a maintenance nightmare as they kept growing. XP solved this problem by provideing a framework that allowed me to only design what I need when I need it, but enabled me to radically alter the design if ever necessary.
* what prompted you to make the switch to XP
I was exposed to it by some people who were working on a project at my company. I found many of the practices allowed me to create "prototypes" just as quickly as before, but with a higher degree of reliability and a framework that allowed them to grow as requirements changed. I was really sold on it when I saw how simply I was able to implement pieces of functionality that had previously just seemed more complicated.
Pair Programming [c2.com] gets a lot of flak. But really I fully credit it for getting me interested in these techniques. Because I generally paired with someone who knew these practices better than me, I was able to actively see the value of these practices while writing production code. It was far more of an eye opener than reading any book or article.
A lot has been said about Pair Programming that is just not true. It does NOT mean that there is one computer for every two people. Everybody has a machine. Just because you try to write all production code in pairs doesn't mean that everything is done in pairs. Also, you do not get paired up with someone like a ball an chain. Pairs switch, sometimes several times a day. Pairs are NOT assigned. If I need to write an ordermatching algorithm and I've never done it before, I'm going to try to pair with someone who has. By the time we are done with the task, we now have two people who know how to write an ordermatching algorithm. It's also untrue that one person is the designated typer while the other one just sits back and criticizes. Both people type, switching sometimes several times a minute while designing a class or method. Code is typed as a way to communicate concepts. If I want to convey the direction I think something should go, I write a little code. If the pair has another idea, he'll start tapping his ideas in the keyboard. It's a very actively collaborative process.
* how that switch work and how long it took
I haven't yet seen a project switch to an XP style process without problems, and from what I understand is that it is possible but slow. Part of the problem is that people who switch often do so from a less test-centric approach. This makes it difficult as XP is so reliant on aggressive testing to be succesful. It is also a new concept for many developers to take responsibility of some of the testing that has traditionally been soley in the domain of QA.
* and how things have been since moving to XP
Like I said, I have not been involved in a switch, but can speak about the difference I've seen between similar projects. One problem that I've solved in both a non-XP approach and an XP approach is in implementing the FIX [fixprotocol.org] protocol.
This protocol is relatively simple, but it is rife with various possibilities for boundry errors, off by one errors, and complicated resend logic. The big difference I found is that I never broke old logic when modifying the code. Generally when doing development, you are only testing the piece of functionality you are working on, so you get that working, but you break something else. This might not be discovered for quite some time. With an XP approach, while developing, you test everything you've ever tested. You find out immediately if your changes break old code.
Not only this, but despite the overhead of writing tests, I found functionality was implemented quicker, more reliably, and far simpler. Some of this was because I learned from previous mistakes, but primarily it was due to a different way to approach a problem.
* do you know others doing XP, if so how many
Yes. But they all do it a little differently. I have not seen two XP projects that look exactly the same. The core value I always see is the emphasis on repeatable tests.
Re:How much XP is there in the real world? (Score:2)
The summer 2002 XPUniverse conference was circa twice the size of 2001. I expect that this year's growth will be at least 50%.
But there was certainly a difference in tone between the two years. There was still a lot of evangelistic fervor, but there were a lot more average joes who were using it because it worked for them.
Regarding your questions, I've been using it for dynamic web sites and application provided via the web. It works for me stunningly well. I make better software. I get more sleep. Both I and my clients are happier and more relaxed.
For more info, drop by the XP mailing list [slashdot.org]; there are a lot of friendly people there.
Re:How much XP is there in the real world? (Score:4, Informative)
With the help of Google I found the article here:
http://www.informationweek.com/story/IWK20020111S0 046 [informationweek.com]
Authors' Site (Score:4, Funny)
Not a bad book (Score:5, Informative)
Although I'm not a fan of extreme programming (it seems counter-intuitive to my highly structured mind), the aspects mentioned by this book have accurately reflected the last five years of experience I've had as lead architect and developer at a custom development firm.
Let me give an example. The reviewer condescends the book for assuming a "Strategist" role is necessary in a successful project, since the customer undoubtedly knows his or her own business.
In my experience, which may not be the gospel truth, but is valuable nonetheless, the customers often do not know their own business. The individuals of an organization sometimes know nothing more than the rote daily routine with which they've been guided over the years. Ask them an insightful question about why or how a process came to exist, and they might give contradictory or vague answers. It is the role of the strategist to exhume the truths and necessities of an organization, which are not always superficial or easily understood.
The reviewer also disbelieves that SQL code is ever embedded within web pages. Many quick and dirty (or under-engineered) web sites do use some form of embedded SQL, however. I'm often called in to clean up such sites, and make them more secure and modular.
The book is admittedly light on related topics, and perhaps a more academic treatment of extreme programming would have been more useful. And I do agree with the reviewer in that many statements within the book seem like advertisements for the author's own company.
Nonetheless, Extreme Programming is a practice understood by few (comparitively speaking), and this book serves as a good bridge between Extreme Programming and more structured development methodologies.
Re:Not a bad book (Score:3, Insightful)
Or even worse, they expect that just putting their processes online will magically solve all their problems. So a "Strategist" is definately needed to help set goals and expectations.
OK book perhaps, but mediocre review (Score:2)
As for assigning a "strategist"--that can be invaluable to a project. The end customer in most cases knows his business as it presently runs, and most often they are not a tech business. I have clients in manufacturing, distribution and food processing/agribusiness. Their business is knowing how to make wigits or food or chemicals or how to send stuff all over the continent--and all they want out of computers are ways to make these tasks work better--and more often than not they must rely on "strategists" to show them HOW and WHAT computers can do to achieve that. That's the case for ANY component of a client's business that is ancilliary to their primary goals.
I don't always use a strategist--but I will if the job is big enough and/or to establish a relationship with a new client. In my opinion (and experience), a good "strategist" should be the following:
1. The strategist should NOT be a past or present employee or long-standing affiliate of the client--that might seem counter-intuitive however someone with intimate knowledge of some or all of the processes in a business standa a chance of being blind to change. Viewing "from the outside in" is most often the best way to spot and change counter-productive practices.
2. If possible, a person with industry knowledge (if you're going a web project for Pepsi, then someone who knows the soft drink industry). This can be more important than having an advisor or strategist that knows technology...which brings us to:
3. The strategist should most certainly NOT be a programmer on the project (or indeed be involved in project development AT ALL beyond the requirements phase--or in XP, I suppose developing the tests). If there are questions about feasibility or time/resource requirements to achieve functionality, it is not the strategist's job to answer them--the answers should be provided as feedback from the developers upon receiving the specs/tests. Not only can programming be an iterative process--so can (and often should be) determining project specs. The last thing a client wants is to get mired in an alphabet soup discussion of SQL, ASP, PHP, etc. on what is required to satisfy a reporting requirement--all they want is to state a list of wants/needs--then they want you to think about it a bit and tell them how long and how much $ for them so they can prioritise. Having a non-programmer strategist as a go-between helps immensely in avoiding that trap.
SWM, XP curious, seeks Strategist (Score:2)
I must agree on your comment about a Strategist. Whether you follow XP or not, you need this. A client might well know their business inside and out, but it's unlikely they know that much about the internet (or they probably wouldn't need the likes of us), or how this tool relates to it. And we've had plent of clients where even key people in the company hadn't really given much thought to certain critical business issues. A good part of our job is discovering and exploring this.
It's a lot of work, but a good strategist will keep you sane.
Strategists may well be useful (Score:2)
At best, this is consultant mumbo jumbo - "just you leave all the difficult stuff to me, I have a brain the size of a planet and will prove it by saying the word "synergistic" a lot".
At worst, it's precisely why we techies tend to get a bad reputation in many businesses. Many business people may find it hard to express their knowledge in the terms that we techies like; much of what they do may be hard to encode in nice UML diagrams, but they do not deserve to be treated like idiots. And that is the distinct impression I got from this book - the authors seem to imply their knowledge is infinitely superior to that of their customers, even when it concerns the customer's own business.
If you read the review again, you should find I do not claim SQL is never embedded within web pages; I simply object to the many unsubstantiated, sweeping statements that litter this book, often left dangling with very little context.
Re:Not a bad book (Score:3, Insightful)
Not only do customers often not know their own business (or at least haven't thought about it in any systematic way), they generally have no idea of a) what's possible, and b) what's cheap, and c) what's good on the web. They get really caught up in features.
I groan when I'm in a requirements meeting and I hear a client start, "Now, can there be a 'button' that..." Generally they get this image of a button and various form elements in their head, and don't really have the training to think it through, or even better, step back and think about what they're trying to do and what kinds of services can fit them--let the details work themselves out later.
Clients who have this kind of blinder focus on details tend to have more unrealistic expectations and greater disappointment when the magical mind-reading system doesn't appear. They are very frustrated with the process of fixing this system in development.
This isn't their fault. If this were easy, everybody would do...oh wait, everybody would do it well. They don't know that the system requires mind reading--they just assume several steps in between.
Your average developer, head deep in code, doesn't have the business process experience to chain the processes together and see the problem until they get into the code and realize somebody forgot Phase Two:
Those who are really good, and they are rare, at Internet strategy tend to have experience in organizations (not necessarily businesses) and technology. Too little technology and they plunge off the cliff with the client (we have one of those in our company). This is why domain experts with tech experience are as valuable on Web projects as they are on traditional client-server projects.
Fair ? (Score:2)
I don't understand what is unfair about the supposed existing system. One of two parties, both of whom willingly enter into a contract, supposedly exposes itself to more risk: so what ? If the company is smart, it takes on that extra risk in the hopes that it will be able to realize a larger reward.
The point is that both parties enter into the contract willingly and freely. So what can be unfair about that ?
Re:Fair ? (Score:2)
That's certainly fair, but that's not the point.
If I can find a way to better share the risks and rewards with my clients or contractors, then generally strikes both me and them as more fair.
Risks (Score:2)
This book seems to assume that customers
a) Know what they want
b) Are capable of helping to formulate a realistic iterative dev plan
In my experience, customers tend to want fixed price projects because they know how much they'll pay up front, but then they also want changes made at a whim if they don't like something, they often expect the developers to be able to read their minds regarding ill defined requirements and they expect it all to be defined, developed and delivered yesterday.
The key to successful fixed price development is to make sure that the client understands their own requirements and understands that anything outside of that understanding is a Change Request for which they will pay over and above the originally decided project cost.
Regarding Extreme Programming - one cornerstone of XP seems to be two developers working at one machine developing one Unit together. You'd have a hard time trying to convice someone managing a fixed price project to sign up to this as instantly, your costs for each unit double (managers can't see time savings).
I think that if a customer can be persuaded to go Time and Materials and a realistic agreement can be made re: milestones between the developer and the customer, then you have the best of all worlds.
Optional Scope is a non-starter (Score:5, Insightful)
Well, I worked at a web shop for a few years (though, that was during the bubble, maybe things have changed now) and I looked into using XP because many aspects of it seemed to fit our needs. But one aspect always hung me up and that was that XP projects are basically optional scope contracts.
Basically, the clients always wanted to see the whole site (at least mockups) before they would sign off on the work. Even if we all knew that there was likely to be significant changes along the way.
Saying something like "let's get broad agreement on the general nature of the site and then work in iterations to refine the details. Now please sign this contract for three months of work for four developers" just didn't work.
Now, XP proponents will claim that this can be overcome by educating the client. I'll just say that that wasn't my experience. Optional scope contracts just wouldn't fly. Other XP proponents might say to hide the process from the client and make an XP project look, to the outside, like a waterfall with very flexible change management, but I wasn't happy with that sort of methodological dishonesty.
I think that this problem with optional scope is a problem not just with web sites but with any project where contractor and client are different companies. Most of the XP success stories I've heard are on projects where the client is an internal division of the same company, so things are a little less confrontational and more flexible.
Re:Optional Scope is a non-starter (Score:2)
For this to work, you already need to have a relationship with the client. Once they trust that you aren't going to screw them, they're much more willing.
Also, optional scope contracts have big benefits for the purchaser. For a long project, the ability to kill it with two weeks notice drastically reduces risk on their part.
And if I'm doing optional scope, I can charge them less because I have to set aside less cushion for risk. One company I know offered an optional scope contract at $X and a fixed-scope contract at $1.5X.
So it's certainly not impossible, although I agree it's hard.
Other XP proponents might say to hide the process from the client and make an XP project look, to the outside, like a waterfall with very flexible change management, but I wasn't happy with that sort of methodological dishonesty.
What's dishonest? If I sign a contract for a fixed spec and fixed price with change fees, then I don't have a problem telling the client that internally I'm using XP if they ask.
If I've already recommended an optional-scope contract and they turn it down, the fact that I will make a killing on the change fees is their problem, not mine.
Re:Optional Scope is a non-starter (Score:2)
Ah, that's a good point. My negative experiences with trying to get people to agree to an optional scope contract were with new(ish) clients where we didn't have a pre-existing relationship of trust.
I think that if a suitable existing client comes along with a suitable project I might just try it again (optional scope contract that is, I still have other problems with XP, but that's another story).
Thanks for your insight.
Extreme Programming – A Two Way Street (Score:2)
We feel it's only fair to also have the customer appoint one of their own people we like to call the "hygienist."
They help the poor idiot programmers understand the daily value of brushing the back of your tongue, that wax paper from discarded pizza boxes is not a replacement for clean underwear, and keeps our dew-induced flop sweat upwind of the secretarial staff at all times. They often do breakdown and get the recommended VPN installed to lessen the direct contamination from our biohazards.
Lazy developers? (Score:4, Interesting)
XP is the only way to program for the Web... (Score:4, Funny)
This is not a joke.
But there is an important question here... (Score:4, Insightful)
I've seen plenty of this and I believe it's nearly always true. And I think the reviewer is correct in stating that this is not unique to web projects.
So, with that in mind, I'll assert that it would be overall _more efficient_ (less waste of money and resources) if both parties were able to manage uncertainty and risk in projects in a less adversarial manner.
Call me on that assertion if you want, but risk management is an important part of managing the software development process for just that reason. So, why _not_ come up with a better way to manage risk across organizations??
I don't think contracts are bad things (just the opposite) and I don't have The Answer...but I'm imagining a better contractural toolkit and a better set of development and project methodologies that allow some uncertainty and flexibility and assigns risk at a more granular level than 100%-0% or vice versa...
For an analogy that's pretty far afield, I saw a report recently that said something like 50% of mergers and acquisitions fail to add value, but if the M&A was contested or if there were multiple bidders, it goes up to 70-80%. Demonstrating, I think, that while people enter into contracts freely, those entities or contracts that are more adversarial are more wasteful of resources
OK, let's all get out there and fix the consulting business so it's more fun to work on projects for clients...:^) Comments??
RUBBISH !!!!! (Score:2, Informative)
Basically, "Extreme" (God - it makes me cringe just to type it!) programming means "doing it right" ! ie - writing a bit of code that (a) works, (b)
ie - it's meaningless bollox.
Re:RUBBISH !!!!! (Score:2)
Doh... That was supposed to say Extremity Progomaxing... wait, that doesn't even make sense.
extreme, ex-stream, X-wing... dude, this is hard
I know... Xtrememey Rogrammingpay!
See! It is better... Eat that Pessamist!!!!
Re:RUBBISH !!!!! (Score:2)
I agree that it's a stupid name. On the other hand, XP gets more ink and chatter than all the other Agile methods combined. So stupid or not, the name worked.
Basically, "Extreme" (God - it makes me cringe just to type it!) programming means "doing it right"
This got modded as "informative"? Please.
XP has a number of practices that aren't common, certainly not in combination. The name may be stupid, but the practices kick ass. They've made my software better; they've made me and my clients happier. If you want to let the name keep you away from that, go right ahead.
An aside (Score:2, Insightful)
Successful software development has always been about people - not development philosophies, and not about which language you use. If you're working with people who are winners, you're going to find a way to succeed.
That said, I'm sure there are many who feel very strongly about operating systems, development environments, programming languages, etc. (myself included), but none of it matters if you don't have the right talent in place.
Just my master-of-the-obvious aside ...
Extreme Programming brings teamwork to the cubicle (Score:2, Informative)
Re:Extreme Programming brings teamwork to the cubi (Score:2)
It's probably some stupid thing hardcoded in my brain, but if I'm not actually the one at the keyboard, my mind tends to drift within minutes and I'm thinking of something else completely unrelated. For some reason I just don't remain focused if I'm looking over someone's shoulder. Same with meetings - I have a hard time staying awake in a meeting unless I'm the one presenting or actively discussing.
Automatic Unit Testing For Web Applications (Score:3, Insightful)
Automated unit testing is at the core of XP. All the lightweight agile stuff with "write code first, refactor later" doesn't really work if you don't use automated regression tests to make sure your agility don't break anything.
I earn my living doing web applications with a database backend and I still haven't figured out how to write simple lightweight unit tests for this kind of application.
There are two major problems with testing database driven web sites:
1. How do you write unit tests for the database access code that are
* lightweight (i.e. fast), so that they can be run all the time,
* don't have any side effects and
* don't rely on a dedicated test database with predefined known state?
2. How the hell do you write unit tests for a web UI? JUnit has a HttpUnit class that simulates a servlet client, but how does an automatic unit test for a servlet procedure look like? Do you put the expected HTML into a file, capture the output from the servlet and compare? What if the sevlet code accesses the database, so that the contents of the page get determined by the database?
My point here is that the main topic in applying XP to web projects got omitted from this book. All the process crap gets broad treatment, but the really tough stuff is relegated to a five page treatment at the end of the book.
Books like this discredit a methodology that brought up a number of really good ideas for the first time. But apparently those guys are no different than all the other method gurus hawking their wares at conferences, who haven't coded in years.
Re:Automatic Unit Testing For Web Applications (Score:2, Informative)
It is impossible. To do XP with database devlopment, you do use a dedicated test database with predefined known state. In fact, to do any automated testing, you need a predefined known state for you data. This is no different from using XML, CSV files, or BCD files on a mainframe.
I'd investigate and see! There are two major projects for doing automated testing with Java web development. They are HttpUnit [sf.net] and Cactus [apache.org]. HttpUnit simulates a servlet container, while Cactus places the tests on you container with the rest of the servlets/JSPs. Check out each project and see what will work better for you.
Any Books Actually Deal With This? (Score:2)
You have just made sure I don't buy this book, or waste my time looking for it.
Not that the review had me convinced, but I was going to check it out at the bookstore to see if I it covered exactly that: Automated Unit Testing.
I'm starting to get used to the testing methodology of XP for non-web programming, and to develop non-web code with JUnit and Ant. Of all the ideas in XP, testing discipline is the one I'm seeing the benefits of.
The problem is I'm not doing that much non-web programming these days, and I can't figure out how to apply disciplined automated testing to web applications.
It's not just the GUI aspect. Much algorithmic functionality in JSP is abstracted into JSP tags, and moving this into your own custom abstraction in normal Java classes so you can test them outside the web-app would remove most of the development advantages of JSP.
The database code issue you mention applies, I think, to all database-access code. So far the only way I've seen to deal with this is to depend on known-states of the database, which works well enough if you're willing to spend more time writing identical tests for different database-states than actually developing.
Re:Automatic Unit Testing For Web Applications (Score:3, Informative)
As for your second request as to what unit tests look like for web UI, look at HttpUnit's manual [sourceforge.net].
Re:Automatic Unit Testing For Web Applications (Score:3, Informative)
For the lowest layer, you can either test the emitted SQL or you can test against a database. You should aim to keep that low layer as thin as possible. Or better, move it off into a separate library, one that just does generic O/R mapping stuff.
Then for the higher layers, you can test against Mocks.
How the hell do you write unit tests for a web UI?
You test the things that you are worried might break. For end-to-end testing (what XP calls the customer tests) I generally mark all the interesting elements with id attributes and just check those. So if you aren't worried about most of the HTML, don't test it.
For unit tests, then you write your objects in such a way that they are testable. For example, if you have a object that takes a InventoryItem and produces HTML, then test that in isolation, without having a whole servlet container around.
What if the sevlet code accesses the database, so that the contents of the page get determined by the database?
This is one of those, "Doctor, doctor! It hurts when I do this!" things. The right answer is, "Well, don't do that then."
You have to write your objects in such a way that they are easy to test. A good way to get that for free is test-driven development. Start with a simple test. Then write an object to make it pass. Then add another couple lines of test code. And then write production code until it passes. And so on.
When I started doing this, not only were my objects testable, but making them testable forced me into doing much better design.
Re:Automatic Unit Testing For Web Applications (Score:2)
1. How do you write unit tests for the database access code that are
* lightweight (i.e. fast), so that they can be run all the time,
* don't have any side effects and
* don't rely on a dedicated test database with predefined known state?
And what's wrong with the last idea? You not only put the database into a predefined known state, but you also insert test cases that your program can insert and mimic the sort of thing that your users would have in there.
Then, as defects arise, you insert more data into your baseline test database to cover even more situations.
Tedious? Maybe. Doable? You bet!
Re:Jesus Saves! (Score:3, Funny)
Re:Jesus Saves! (Score:2)
In many ways, that's the failing of this book... (Score:2)
The stuff which differentiates web projects - in my experience - is
they tend to have very compressed schedules
they are usually very visual - stakeholders can usually see both their own site and a bazillion other sites on the web; a lot of requirements discussions seem to to go something like "make it like amazon when you see the homepage, and like dell when you buy something, but make it nice and light like google and...". This is pretty hard to capture in a requirements document...the visual nature of web projects tends to lead people to assume the underlying stuff is easy to understand too ("I know we're selling books, but if you just make it do auctions like ebay we could earn an additional bazillion dollars")
many of the techniques that help you develop solid code are not easy to adapt to web development, e.g. unit testing, automated system & regression testing, there are comparitively few design patterns catalogued (the "Core J2EE patterns" book is about the bestI've seen thus far, and that is platform-specific), etc.
A lot of web projects are exploratory in nature - both for the business and in the technology used. It's hard to build a solid code base from a prototype.
The shared risk issue is not really specific to the web - it's a common issue with consulting style contracts.