Automating Unix and Linux Administration

nead writes "If you are disciple in the church of Wall, or like me you believe that laziness is the father of invention, or if you simply have more than a couple *nix machine to administer, Kirk Bauer's new book Automating Unix and Linux Administration is definitely for you. From the creator of the popular open source projects AutoRpm and LogWatch comes a thorough - and believe it or not entertaining - look at how one can leverage the power of a few common tools to significantly reduce the time and effort system administrators spend doing their jobs." Read on below for the rest of nead's review.

Automating Unix and Linux Administration
author	Kirk Bauer
pages	547
publisher	Apress Inc.
rating	8.0
reviewer	Nick Downey
ISBN	1590592123
summary	Tools and methods for automating *nix administration for a couple (or a few thousand) computers.

From the outset, Bauer takes a straightforward and principled approach to problem analysis. Usually starting with anecdotal example scenarios (many of which will have you saying "been there before") and progressing through ideals, goals and consequences, he examines many of the common issues facing system administrators with candor and realism. Almost nowhere in the book does the author assume an authoritarian stance; he questions his own decision making process and encourages the reader to come up with exceptions to his rules. Fundamentally Bauer has one goal -- to develop a comprehensive system for reliably automating the tedious but important tasks that all system administrators face on a recurring basis.

Admittedly, it would be a fallacy for any book to claim complete and comprehensive coverage of all things related to system administration and Bauer does no such thing. When the author touches on topics that obviously require more depth than a single chapter can afford, he is certain to include at least one reference (and in many instances more) to alternate publications without bias to any particular publisher or author. Having said that, the book's scope and depth of topic coverage is impressive. Starting with an exhaustive examination of SSH and progressing through cfengine, NFS, LDAP, RPM and Tripwire (just to name a few) Bauer provides carefully detailed instruction on how to automate tasks ranging from simple network management and software packaging to security, monitoring and backups. The author even goes so far as to suggest methods for efficiently front-ending automation systems for the less technical of users.

Although not expressly stated in the text, the overall theme of the book is walk on the shoulders of giants. Starting with simple example scripts (in both Bash and Perl) and many single-line commands, Bauer builds on the content of each previous chapter as the book progresses. Examples shown in early chapters are incorporated into more complex systems one step at a time. Following along is easy, each script or command is detailed on a line-by-line basis, and because of Bauer's principle-based approach the reader is rarely left wondering why the author has chosen a particular tool or implementation. More often than not the elegance of how Bauer pieces together methods and procedures will excite you about the possibilities for automation of your own systems.

Although Bauer explicitly states that readers are presumed to have more than a modicum of experience in system administration, even the novice administrator, as well as those that are responsible for only a handful of machines, will find this book invaluable. Also included are three appendices which provide an easy introduction to basic shell tools, creating your own RedHat distribution and how to package software as RPMs. These portions of the book alone justify the less than $40 price tag, but for those who run clusters or data centers, this book stands to save you countless hours of repetitive headaches. Published by apress and boasting nearly 600 pages, this lively read has made itself a permanent addition to at least one reference library.

---

You can purchase Automating Unix and Linux Administration from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Book reviews on slashdot

RANT

Have book reviews on slashdot become about who can get the earliest links to their amazon.com partner site?

This books looks interesting (to me) and I might actually take a trip to the book store to check it out. But the comments (so far) aren't about the book.

/RANT

Another goodie in this area

Obligatory O'Reilly plug:

Perl for System Administration [amazon.com].

Just wait until YOU have a mortgage, jr!

Goddamit, I'm just trying to make a buck here, since I automated our system administration and put myself outta a job!!!

Lazy ?

That just sounds so bad. I prefer the term "minimal keystroke solution". -B

Here's how I learned it

1. Decide that automating takes too much time
2. Do everything by hand
3. Fuck up once too often
4. Decide that automating is necessary

Don't know about the rest of you.

Re:Here's how I learned it

1. Type the same thing three times in a row
2. Decide that this task could be replaced by a
shell script.
3. Spend the afternoon perfecting and documenting
the 400 shell/perl/expect/... script so that I
can save 30 seconds a day for the next few
months.
4. Find a better solution on sourceforge

Little rants..

• If this book obviously doesn't have any downsides (at least you didn't mention any) - why did it get only eight points? (assuming maximum would be 10, as usual. Or do you mean eight out of eight points?;)
• 547 pages - I'd say that's nearer 500 pages than 600 pages. Or simply around 550 pages. But certainly not nearly 600 pages.

At all - thanks for the review!

Autocrash

Call me a cynic, but I am under the impression that without knowledged personnel (i.e. who don't need autowhatever) there will be, almost as sure as a natural law, a corrupt server or an autocrash. Don't do away yourself with knowledge - see what happened in the Windows world.

Re:Autocrash

And then reality smacks you in the face. If your in a shop with a staff of 10 sysadmins and you have 2000 servers to look after, you NEED automation. Anyone who thinks automation puits sysadmins out of work is full of it. Who do you think has to write/customize these scripts? SYSADMINS! Do you as a sysadmin really have time to properly pour over the logs by hand or would writing a script to do this for you help you do the other things you need to do like:

Patching
Fixing user passwords (unless you have a help desk)
Working on upgrades and installs.
Planning for future growth
Work on your disaster recovery plan
Possible Machine Room moves
etc
etc

Sysadmins do more then watch over a system. We need to realize that automation is NOT a panacea, but yet another tool in the sysadmin bag. Besides....if everything was supposed to be done by hand why was cron created??

I bought the book

I own the book and have been using it for a couple of weeks now. All in all, I think it's a great resource if you already have a fair amount of linux knowledge. I purchased it primarily because of its coverage of cfengine [cfengine.org] but found it useful for other purposes as well.

Definitely not for the newbie system administrator (nor does it pretend to be). But it is a great resource if you're looking to administer more boxes with less bodies.

Unix/Linux ratio??

Simple question, that isn't really answered in the review. How much of this book is generic Unix/Unixlike information, how much is specific to a single vendor OS, and how much is specific to Linux?

I'd like to think that most of this stuff is fairly transportable, but when I hear about "bash scripts," I wonder if it's the reviewer or the book that's pushing Linux-centricisms. (and yes, I know that bash is available everywhere, blah blah blah. It still doesn't make it a valid replacement for /sbin/sh, for admin scripting)

Re:Unix/Linux ratio??

Disclaimer: I am the author of this book.

The book is aimed towards all Unix variants (as is Cfengine, which is a big part of the book). But I prefer Linux and use Linux for many of the examples... but all that usually means is it begins with #!/bin/bash at the top instead of #!/usr/local/bin/bash or #!/bin/sh.

One appendix is on RPMs (which is used on other systems besides Linux) and another is on Red Hat Linux specifically.

Nothing new here

This is nothing new. Unix admins have been automating machines since before Linux was even a glimmer in Trovald's eye. The only difference I think, is that there are a great deal more admins today who don't know their craft very well. To many fuzzy GUI widgets (that invariably screw things up) getting in the way.(You hear me RedHat?!)

Use Knoppix. No maintenace, no problem.

I know it doesn't apply in all cases, but if you're just running a web server my experience is that running it all out of Knoppix RAMDisk just makes sense in every way. It's faster, it's cheaper and if it screws up, just start from scratch. But since it's so cheap why not run redundant servers? It's a winner from every angle.
Yeah, you need to make a few little scripts to automate your rebuilding process, but once you've done that it's about as maintenance free as you can possibly imagine.
Of course a web server is a limited example and I assume the book covers more than just such a simple case, but for a lot of net server tasks it's the way to go and yet I get the impression a lot of people don't quite get it yet.

If RPM weren't such a mess ...

If RPM weren't such a mess, it might be more convenient to make RPMs of the packages you build from source but want to install the same exact binary on all the other machines. I just make Slackware style tarballs, so it's real easy; no spec file needed.

just what I need right now, another Book!

i haven't been getting much bonuses or raises lately. why waste money on a book?

i get most of my stuff from reading periodicals while sipping chai at Borders. and websites like the Linux Documentation Project.

http://www.tldp.org

fathers & mothers

RE: >

and necessity is the mother of invention, does this mean laziness and necessity get together and have nasty sex before inventing something :)

entire contents of the book:

man cron

automation is easy...

I have just about any task down to issuing one command:

"Brian, go overto server X and do such-and-such"

Multiple Machines

One of the problems we have, is when you have clusters with 100+ machines, and need to push configs, or gather stats off each box.

On solaris, we run a script called "shout" that does a for/next loop that ssh's into each box and runs a command for us. We also have one called "Scream" which does some root privilege ssh enabled commands.

Nortel has a nice program called CLIManager (use to be called CLImax), that allows you telnet into multiple passports and run commands. Same idea, but the program formats data to display. Say you wanted to display "ipconfig" on 50 machines, this would format it, so you have columns of data, easy to read and put in reports.

Also, has a "Watch" command that will repeat a command, and format the data. Say you want to display counters.

I have not seen an opensource program that does the same as "CliManager" but its has to be one of the best idea's that should be implemented in opensource. Basically, it logs into multiple machines, parses and displays data, and outputs all errors on another window to keep your main screen clean.

Think of logging into 10 machines, and doing a tail -f on an active log file. Then the program would parse the data, display it in a table, and all updates would be highlighted.

I havnt spoken to the author of CliManager, but I guess he also hated logging into multiple machines, and running the same command. This program has been updated over the years, and is now the standard interface to the nodes. It just uses telnet and a command line, but you can log into 100's of nodes at once.

Wish I could post pics and the tgz file, maybe someone from Nortel can comment. (Runs on Solaris, NT and linux)

Learn to script

Certainly I haven't read the book but it looks as if Kirk is offering examples of how to write scripts to handle everyday gruntwork. Good idea.

But I say to those that call themselves sys.admins, Learn how to script!!!

I work at a large bankrupt telcom :) and it's amazing the amount of admins that don't have the slightest idea how to write the simplest loop. Or use ksh, bash, or csh's cmd history. Or vi.

Maybe this is just a corporate thing. They were raised, in a sense, in a setting where all they had to do was add users and replace disks. Maybe they never learned how to do anything else.

Back in '83 I took manuals home and poured over every page, every weekend for months. That didn't make me a good admin but it gave me a good foundation. From there I had to just halfway use my head (imagination?) and start writing scripts. Ugly? Sure. Did they get better? Of course!

Now I play admin on 110+ machines, and I stay bored. Why? Because I've written a response engine in Expect that handles most of my everyday problems. I call it AGE, Automated Gruntwork Eliminator.

There's no way I could have done this if I had just sat back and floated, not put in a bit of effort to learn new things.

T.

It's in the mindset

If you're thinking of this computer or that computer then you won't make an effective systems administrator. You have to see the network of all of the computers as a single whole and treat them as such.

Once you've got the mindset change sorted, 10, 100, 1000 systems it makes no difference, it's just as simple to manage. You aren't managing individual computers, you're managing an infrastructure.

Course, you actually have to be competent as well... Obviously.

It looks like a worthwhile purchase.

I thoroughly browsed the book at my local B&N cafe, and recommend it highly. It is a well-written, knowledgable book for admins/techs of intermediate sysadmin skills. I mean truly intermediate, for there are no lengthy chapters on Installing Linux, The History of Unix, The History of the Internet, or any such thing. Just useful instruction, insight into the application and usage of certain software packages, and enough scripts to keep one happy. The author's tone is similarly refreshing, as it avoids the blandness of other (good) tech books I've read.

It is definately on my list of Expensive Books (50. Am I cheap?)to Buy.

=============

Might be useful...

This might very well be a book I'll pick up sometime. I'm always looking for more ideas.

I maintain about ~170 remote Linux boxes (in our company's retail stores and warehouses), as well as our ~30 or so inhouse servers.

I went through a lot of work to enable our rollout and conversion to go more smoothly. The network and methodology for users, printers, etc. is extremely simplified and patterened.

For each of the 3 'models' of PCs we use, I have a master system that I produced. I used Mondo Rescue [mondorescue.com] to produce CD backups of these systems. These systems act as serial terminal controllers, print spoolers, routers, desktop system usage (OpenOffice, Mozilla, Kmail under KDE), and other functions as needed.

When we need to replace a system, or rollout a new location, we grab a system, pop in the Mondo CD, and do a nuke restore. When done, we have a standard configuration user that we log in as. It runs a quick implementation script where you answer anywhere from 3-8 questions (depending on the system type and options), and it configures everything. All networking, users, sets up Kmail, configures all printers and terminals (we use Comtrol Rocketport serial boards), and so on.

If the system is physically ready, we can have it ready software-wise in about 20 minutes (2 CDs to restore).

Updates are done via a couple different methods. I use SSH (over our internal VPN, using key authentication) in scripts to do most updates. If I need to do anything major, such as recently updating Mozilla, we do a CD distribution. The users have a simple menu to take care of running the update for them, even with autorun under KDE. Just pop in the CD, and it automatically takes them into the menu they need.

All logs are duplicated across the network to a central server, but intrusion is less likely as these systems sit on a private frame network. They do, however, have fully secured network setups, as we use cheap dial-up internet access as a backup in case the frame circuit goes down.

I can't help but feel every day like this is just one big hack/kludge, but it works, works damned well, and was about half the cost of any other solution (i.e. higher end Cisco routers to handle various functions, and using Equinox ELS-IIs or the like...those pieces of crap never would work right, we finally pulled only 2 we had in use, and they are currently collecting dust in a storage cabinet).

Needless to say, I am *always* looking for ideas to improve upon this.

More info from the publisher

Full details, including sample chapter, here [apress.com].

I see the /.gods have already got to this thread: "Duh, it's easy, just use cron/telnet/syslog!" Do any of you people have more than a home PC to maintain? Come to that, would anyone trust you with more than that?

Ade_
/

Doing the same with *gasp*Windows*gasp*

On a similar topic, I am a longtime Unix/Linux admin who has inherited a large farm of Windows servers (don't ask, I'm not happy about it either). This is probably about the worst place to ask this, but I'll give it a shot:

Do any of you have recommendations for books/URLs on how to effectively manage a large Windows cluster using automated methods?

Thanks in advance for any useful information.

Re:Amazon Link

Or you could purchase from a store that doesn't support a ridiculous software patents.
(From the link you posted): Boycott Amazon!
Why would I want to? It has given me great service for many years, should I care if they fill in stupid patents? I'd rather they didn't, but it won't stop me shopping there.

Re:$5 cheaper and free shipping

You people are just so pathetic.

If you have a complaint against the USPTO, get to the source. Don't whine about the companies who just take advantage of the existing loopholes like any company would.

Re:Amazon Link

You know that when you call for a boycott and then link to the silly rant on gnu.org you're just like a Jehova's Witness?

My question to you is: why are you so obsessed with Amazon? Is it really that hard for your GNU-soul to accept that companies make products fundamentally to make money - not to "serve the public" or whatever.

If you have a complaint, direct it at the USPTO - not to a company who's just using the legal loophole.

Re:$5 cheaper and free shipping

www.bookpool.com HAD this book for $31.95, but they
are now out of stock. Must be the ./ effect!

puncuation nazi

sorry to be this guy but otherwise...
i'd think this book is about working for dummies....instead of "Making Management Think You're Actually Working, for Dummies"

unless you really do mean "working for Dummies", in which case, if you make management think that...you might not be working for the dummies anymore.

Re:$5 cheaper and free shipping

Dang, Windows crashed while submitting my detailed price analysis. Summary: due to free shipping Amazon.com has the best price, UNLESS you are in a state where Amazon charges state taxes OR are in a state that charges no (or very low) sales taxes. In that case, the best price is likely (believe it or not) Walmart.com. Check the Walmart shipping charges to be sure.

Re:Alternative Review

Apparently we are stupider - it's +5 now. Wish I had some mod points, I'd take it down a peg...

This is right up there with that fucker that rewords the same paragraph and FPs it on every review. Haven't seen that yet this round.

Non-Referral Amazon Link

[amazon.com]
here

I wish you would stop posting amazon links to make referral money. It's not like you're actually contributing anything to the discussion of the book. you're just abusing the fact you got to post early on a book review.

Re:Alternative Review

That's not a review, that's the book description off Amazon. A description isn't a review, especially when it looks like it was written by the publisher. (The description was presumably written before the book's release, thus explaining the cryptic references to what the book will do instead of what the book does.)

Furthermore, why copy something instead of just linking to the original [amazon.com]?