The Art of Scalability 63
Martijn de Boer writes "Creating high performance growing networks is really a special skill managers and network architects should possess to be ready for the future. The Art of Scalability is a book written for these kinds of functions, and prepares you for the present and the imminent future. Scalability is achieved by principles that work on many levels within enterprises, whether it's processes, organizational structure or setting up your project, this book covers it all." Read on for the rest of Martijn's review.
The sub-title for this book is "Scalable Web Architecture, Processes and Organizations for the Modern Enterprise" which is basically a catchy title for project managers and decision makers. It's just catchy enough to grab the book from a shelf and start digging into the table of contents, which is exactly the spot where you get hooked and decide to get the book. The book is written by two experts on the subject; both of them have a strong background in large corporations dealing with scalability. Throughout the book the authors shine a good light on tools and cases used for making a project scalable.| The Art of Scalability | |
| author | Martin L. Abbott and Michael T. Fisher |
| pages | 592 |
| publisher | Addison Wesley |
| rating | 8 |
| reviewer | Martijn de Boer |
| ISBN | 0-13-703042-8 |
| summary | Explains everything from organizational processes to the techniques used to accomplish a scalable high performance project. |
The book is divided into four sections which follow the process of starting a scalable project. First off you will need to know everything about strategy, organizational structure, the kind of people your organization needs and how to manage your team. The second part focuses on the how's and why's of scalability, aspects of planning for continuity, crisis and incident management. The later chapters in this second part talk about risks, how to value risk and the importance of testing to make educated conclusions about how far to scale your project, a notable mention here is chapter 19 which focuses on trade-offs in development speed or doing things the right way, often an underestimated point in these businesses. Architecting scalable solutions is the third part of the book, mainly taking a more technical approach to the matter. This part talks about containing faults and breaking up applications so they scale well. This goes all the way to scaling the database backend for your application to caching objects and making synchronous versus asynchronous calls to your application.
While most of the book is good for preparation, I think the fourth part of the book crosses the boundary from preparation to being very useful during the process as a fallback to your knowledge. It's titled accordingly "Solving other issues and challenges" and mainly focuses on things you will come across during the first and perhaps even later stages, providing solutions for unforeseen costs of data storage to monitoring your application for user experience, speed and the processes you will have to implement for this.
If you have read this far, and are thinking about how complex scalability really is, then you have found the right book. The clear writing style and detailed writing of numerous pitfalls for such big projects make this book a valuable asset to your knowledge base. It's also written in a challenging way, and between the lines there is also some humor (evident for instance in the reference to Rain Man in chapter 18), making it a fun way to learn or build upon a great skill set needed by organizations in the not so distant future. I feel very positively about the writing style of the book, but despite there being some illustrations there could be some more diagrams of organizational structure and the architecture of projects as I didn't find the illustrations adding much to the context.
The short appendixes to the book are mostly about calculating capacity for cpu power, bandwidth, etcetera. These appendixes actually provide sample calculations which are useful to backup your analysis when you need to make a bill of materials for your superiors and can also be used to better grasp the size of a project. Numbers mostly provide a context for the people above you, and I think these extra pieces of content could have deserved another part in the book, but perhaps it's better off as some food for thought.
Before starting with this book, I had not much prior knowledge about scaling projects other then some technical ideas of implementation. I always hate it when I know I'm just scratching the surface of something, and need to satisfy my urge to learn more about the subject. The Art of Scalability really helped me accomplish that, and provided much more background information then I expected. I was really surprised by the pieces about cloud computing, it's such a buzzword nowadays but the texts give it a real context.
If you are looking to set up a project or are generally interested by the concepts of scalability, then this is the right book for you. It's an appropriate recommendation for most businesses, because this knowledge can only be used to your advantage and just takes a little bit of time to read trough. It's a heavy subject, but once you finish the book you will be able to make a decision about architecture based on a good foundation of background information rooted in real situations.
You can purchase The Art of Scalability from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
this book covers it all (Score:4, Funny)
http://science.slashdot.org/article.pl?sid=10/01/22/1715229 [slashdot.org]
So if this is so hard for humans and they need new books to work it out, why can slime mold do it?
Is there a slime mold chapter in this book or is that coming in next release?
Re: this book covers it all (Score:5, Funny)
The problem is finding a network administrator who's smarter than a slime mold.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
How Marketable Will That Skill Be? (Score:2)
That said, any admin worth their salt who focused on scalability shou
Re:How Marketable Will That Skill Be? (Score:4, Insightful)
Re: (Score:3, Insightful)
Re: (Score:2, Insightful)
Did you even look at the title of the book?
"Scalable Web Architecture, Processes and Organizations for the Modern Enterprise"
This has more to do than just processing power. Good Grief...
Re:How Marketable Will That Skill Be? (Score:4, Insightful)
I'm not sure that scalability will be all that relevant as long as Moore's Law continues to hold.
Ask Twitter or Facebook.
Moore's Law is helping them, I'm sure, but for really successful new websites, it's nothing compared to the influx of new customers to deal with. (And yes, it's Slashdot so we all must hate Twitter, but you have to admit they grew damned fast.)
I think you're overestimating the value of CPU power here. The real bottleneck is always data-- storing data in such a way that access to it is fast, but keeping it in a consistent state at the same time. Nobody would use Facebook or Twitter if it took a couple minutes for your status update to be visible by your friend.
Re: (Score:2)
Like asian carp in the great lakes, and just about as welcome.
Re: (Score:1, Offtopic)
Wow, despite the disclaimer somebody still posts the "OMG I hate Twitterz so much!" post. I guess there's just no avoiding it on this site. Yes, we get it. You hate Twitter. We. Get. It. Stop posting it. Please.
Re: (Score:3, Funny)
CHAPEL HILL, NC–Area resident PopeRatzo does not use Twitter, a fact he repeatedly points out to friends, family, and coworkers–as well as to his mailman, neighborhood convenience-store clerks, and the man who cleans the hallways in his apartment building.
"I, personally, would rather spend my time doing something useful than tweeting," PopeRatzo told a random woman Monday at the Suds 'N' Duds Laundromat, noticing the establishment's wall-mounted TV. "I don't even own an account."
Re: (Score:3, Funny)
I'm glad to see that press agent I hired is earning his money.
Re: (Score:2)
Sorry, friend, but the answer is "no".
You can have my twitter-hatred posts when you wrest them from my cold, dead hands.
Are you also fed up and sick and tired of the Windows-hating posts? Or IE hating posts? Or Apple-hating posts? I must have missed your impassioned pleas for those to stop.
Interesting that with all the griping and carping that goes on here, with all the rants against this product or this technology or that company, the one thing you
Re: (Score:3, Insightful)
Are you also fed up and sick and tired of the Windows-hating posts?
Yes.
Or IE hating posts?
Only the ones that don't acknowledge IE7 and IE8 exist, which is most of them. So... yup.
Or Apple-hating posts?
Yah.
I must have missed your impassioned pleas for those to stop.
Well, ok. I did a Windows one just a little bit ago... here it is: http://slashdot.org/comments.pl?sid=1531200&cid=30967046 [slashdot.org]
Is it really that wrong to want the level of discourse to rise above "moron" levels on this forum? Is it really so wrong
Re: (Score:2)
OK, you're passion to defend twitter from my hatred is greater than my desire to continue to declare my disgust.
I will never, ever again express my distaste for Twitter in any way. Not on the Internet and not personally. Nor will I cast aspersions on Windows, Microsoft, Sony or Apple. Yet I reserve the right to express negative opinions of users thereof (especially Apple, but only for the entertainment derived from their outraged reactions).
You have convinced me of the error of my ways, friend.
Re: (Score:2)
Unfortunately, they have multiple datacenters and they are running cached mysql with some magic glue so it often does take several minutes for changes to propagate from coast to coast, become cached, etc. That's why the default feed is not live, but you can click it to get it--they would be brought to their knees doing all live updates.
They do use a technique called sharding which keeps the row counts down. Basically they break up all the users into different tables, databases or even servers. Then they
Re: (Score:1)
twitter wasn't scalable at all even long after it became famous.
don't buy this book. only 12.5 people on the earth need to know scalability, and they don't need to read this book either.
Very Marketable (Score:2)
If you think the solution to scaling issues is to 'throw more silicone' at the problem, you are about the worse programmer in the universe. This talent is incredibly handy, as any site shut down by Slashdot's ravening hordes would attest. What you are suggesting is akin to 'Bubble sorts are a great al
Re: (Score:1)
What you are suggesting is akin to 'Bubble sorts are a great algorithm if you do them an a fast CPU'.
This is an interesting analogy, because the fastest possible way to sort 3 elements is bubble sort. It's a good example why reductionism often yields poor algorithms. It's also relevant to scalability of implementation versus scalability of architecture. The former is more about software design, running MT hot, clustering, db schemas, etc, while the latter is more about partitioning the problem and processes. Many business processes are much too big for any one person to comprehend in depth, and it's im
Re: (Score:1)
As computational resources on the market continue to expand, I would suspect that we could soon come to see the opposite of scalability becoming more important - getting more done with less space and energy.
If you think the solution to scaling issues is to 'throw more silicone' at the problem,
I have no idea how you could have possibly derived that from what I said. Indeed I was saying the opposite of that - I was questioning whether scaling itself is relevant.
you are about the worse programmer in the universe
I am not a programmer by profession, nor do I play one on TV.
Re: (Score:2)
Remember Moore's law is about number of transistors. Historically there was a correlation between Moore's law and processor performance. A mulitcore approach breaks that correlation.
Re: (Score:2)
I'm not sure that scalability will be all that relevant as long as Moore's Law continues to hold.
If that were going to be true, it would have happened long ago. Remember the traffic for a popular website can double a lot faster than once every two years.
Re: (Score:2)
First two sections (Score:3, Interesting)
Nobody ever knows everything about the strategy or organizational structure, and the people and managers change places and responsibilities faster than any project can proceed.
And continuity, crisis and incident management are only longed for when the systems are down and eveybody is expected to work continuously until they're fixed, but never when it's time to pay for it.
Conway's law (Score:5, Funny)
Re: (Score:2)
First two sections are not going to happen in large organizations.
What you say is sad but true in my experience at the companies I have worked for.
The first two sections are essentially: (1) the business strategy for your particular product, and knowing your human resources (team); (2) /really/ thinking about the future, not just how you will sell it; why you would need to scale.
The following question is somewhat on topic as this book appears to be targeted to decision makers at the company, which usually means high-level product/project managers and "directors" or "VPs"
Eerie (Score:2, Offtopic)
This reminds me of the original Zelda where you go in the cave and pay money...
Gates: Give us all your personal information and we will use this to make recommendations for you /me gives
User:
Gates: We recommend that you be less gullible
Try again (Score:2)
ERRRRRR
Sorry buddy, better luck next time. Wrong thread.
Re: (Score:1)
Warning (Score:5, Insightful)
Unless you are actually working on a web site that needs to be scalable, thinking about scalability is both a waste of time and harmful. A totally unscalabe web app where each page view takes 1 second, can sustain 86k page hits per day or 2.5 million page hits per month. Will your web site see that much traffic? Really? If not, your main objective should be to get the site out of the door as soon as possible to keep development costs down. Hopefully, users flock to your site, melting the servers under huge loads of traffic. Good for you, you'll have a very profitable job designing the next scalable version of the site.
Or you get no traffic because the site idea was stupid to begin with. That sucks, but much less so than if you had spent months of effort making it scalable. "Scalable architecture" is merely a different form of premature optimization, and it is just as harmful.
Re:Warning (Score:4, Insightful)
"Your simplified model only holds true if all of those one second requests appear strictly one after the other. If all your eighty six thousand requests land over lunchtime, you might need to think about scalability."
A fairly good definition of "being slashdotted"
Re: (Score:2)
Talk about missing the point.
In the tech world (and even more so in the land of Web 2.0 and web apps), getting stuff out the door quickly and improving it is way more important than having it work perfectly from day one. Because if the idea is any good, someone else is also working on it, and the first one out the door tends to win. It's widely acknowledged that with web startups, if you want to take on an existing market, you need to be 10x better than your competition. So if you're first to market with s
Re: (Score:3, Insightful)
Wrong, because 80k of those hits actually come during a 8-12 portion of the day, sometimes much smaller depending on the focus of the site.
Yes, its fine if you're never going to get anywhere near that kind of load, but load planning doesn't work in 'days' it works in hourly peaks and valleys if you want to look at it from a real rough perspective.
Ask some sites that get slashdotted how well capacity planning works when you get 120k hits in the first hour its on slashdot, and 10k for the rest of the day afte
Re: (Score:2)
I should have known slashdotters always have trouble getting the point of simplified calculating examples. To put it another way, a simple, shitty $20/month VPS with a standard LAMP stack can easily handle 4 hits/second, which translates to 14k hits/hour.
There are millions of sites on the internet, thousands are added each day. The chance of your site getting on slashdot is extremely small. You'd be a fool to waste time, money and effort planning for that.
throughput vs latency (Score:3, Informative)
A totally unscalabe web app where each page view takes 1 second
That's latency, not throughput. Scalability is throughput. Latency is how shitty it is to use.
I can scale your 1 second web app by adding 1000 processes per server and 1000 servers. That's a million page views per second throughput. But it'll still be a 1 second latency piece of shit application.
Re: (Score:2)
IINM latency is how bad it starts out, throughput is how many simultaneous users it can handle before it gets worse.
Re: (Score:1)
Re: (Score:2)
He's obviously talking about the page rendering time, not hosting a fast app from a dial-up connection. We're not serving static pages anymore (and if it takes a full second to serve a static page, you're doing something seriously wrong).
Re:Warning (Score:4, Interesting)
Once you know how, making the solution fairly scalable is usually not a whole lot of extra work. Furthermore, if you don't bother learning how to do stuff right, you'll continue to make subpar products until you get your shot and then you waste that shot because your solution wasn't up to the task. Ahh.
Re: (Score:2)
Re: (Score:2)
Unless you are actually working on a web site that needs to be scalable
I'm sure they wrote Facebook thinking they'd only get a few hundred hits per month... few years and several billion dollars later, guess what - they're writing PHP compilers to fix the problems with the initial design.
Take a little time to make yourself a better developer, learn the necessary knowledge and apply it to all you do, and you'll be known as the guy who writes good systems. Keep thinking you can just hack it all together and y
Re: (Score:2)
When your website has 350M+ users and 150M+ uniques/day, you're well past the point of traditional scaling solutions. If Facebook was a country, it would be the third largest in the world, so to say the amount of data they have to deal with is monolithic is a terrific understatement. At this point, I'm surprised they're not getting custom silicon; custom compilers seems like they should have come a hundred million users ago.
Don't get me wrong - I'm sure Facebook's original code was awful. Do you think they'
Re: (Score:1)
Unless you are actually a contender for the Olympics, thinking about being stronger and faster is both a waste of time and harmful. Will you actually be a finalist? Really? If not, your main objective should be to fit out the door. Hopefully, sponsors flock to you. Good for you, you'll have your work cut out for you training in the pre-season.
Or you watch the Olympics on TV like the rest of us. That sucks, but much less so than if you had spent months of effort cutting your times by a hundredth of a second.
Martijn (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Re: (Score:1)
It actually means "stuff" but it's close :).
Ofcourse I meant "things", gah! posting this early on a friday when the coffee still is boiling..
Scalable? (Score:2)
Creating high performance growing networks is really a special skill managers and network architects should posses to be ready for the future.
Most software/network designers will never work on any project that has so much traffic that it needs to scale to the levels discussed in this book. In fact, most (note I say most) businesses don't actually need the amount of scalability currently built into their current out-of-the-box hardware and software solutions. Just saying...
"This book is much more than you may think it is" (Score:2)
"This book is much more than you may think it is. Scale is not just about designing Web sites that don't crash when lots of users show up. It is about designing your company so that it doesn't crash when your business needs to grow. These guys have been there on the front lines of some of the most successful Internet companies of our time, and they share the good, the bad, and the ugly about how to not just survive, but thrive." -- Marty Cagan, Founder, Silicon Valley Product Group
From the somehow omitted s [theartofscalability.com]
Re:Book cover... Copycat (Score:2)