Book Review: Linux Shell Scripting Cookbook 173
vellorean writes "I have been reading Linux
Shell Scripting Cookbook by Sarath Lakshman, published by Packt,
for a while. While most people I know learn shell scripts themselves,
I was looking to refresh my concepts a little as well as have a
reference lying around on the table for fast access." Read below for the rest of vellorean's review.
First of all, let me remark by saying that shell scripting is
something learned more on a need basis than as a tool to solve the
main problem. People would seldom write shell scripts as standalone
programs (exceptions exist). However, what makes shell scripting
invaluable to know is the fact that knowing some tricks can save
several minutes, or hours, of work by automating and simplifying
certain tasks, generally (but not restricted to) file management and
data processing. Linux
Shell Scripting Cookbook does go quite far in pursuing this goal,
and is appropriate for both beginners who are looking to gain
dexterity in shell scripting, as well as intermediate users who wish
to polish their skills. The book also can double up as a quick
reference, though I would argue that the "Advanced Bash Scripting
Guide" would suit that more.
| Linux Shell Scripting Cookbook | |
| author | Sarath Lakshman |
| pages | 360 |
| publisher | Packt Publishing |
| rating | 9 |
| reviewer | Kumar Appaiah |
| ISBN | 1849513767 |
| summary | A book for beginners and intermediates, which introduces shell scripting and proceeds to provide several practical real-world recipes of useful shell scripts |
At the outset, the author clarifies that the focus will be on Bash. This, people may or may not like, but the fact that bash has become ubiquitous in terms of the available shells on Unix-like systems today, starting out with bash is not a bad thing to do. Besides, learning other shell scripting languages while knowing bash isn't too hard, since the paradigm remains the same.
The book is organized into chapters based more on utility than scripting concepts themselves, although the language aspects are brought onto the reader gradually. For instance, the examples in the first chapter focus more on the basic data elements (variables, arrays, functions etc.) as well as operators (for numbers, files etc.), and all the examples demonstrate simple usage of these concepts, and he further chapters build upon these in a gradual manner.
At the same time, if he reader has some familiarity with shell scripting and needs to only refresh or learn a certain concept, he/she needs to just read the relevant chapter. It is not too difficult to grasp the examples of the later chapters, provided some basic shell knowledge is assumed.
A positive trait in the presentation of this book is that it is all based on practical everyday examples which, with minor adaptation, can be used by many for their own daily tasks. For instance, there are several examples which describe searching for and processing files, which, I'd imagine, many users would want to do on a regular basis. Thus, providing realistic examples allows the book to double its utility. The language and approach used is simple and conversational, and the presentation is very clear, with each idea being described as a problem statement followed by a "How to do it" section with the actual code, and ending with a discussion of the nitty-gritties of the code. It is easy to go for a quick scan for those in a hurry, while those who with to read in more detail will not be disappointed either.
The book also covers a wide array of applications. For instance, there are examples on automating fetching web pages and processing them, demonstrations of parsing and simplifying and even some queries around databases wrapped around in shell. It also spans to utilities and tasks connected to statistics, backups, compression, version control and many more.
The book goes into a fair amount of detail in terms of describing the shell scripting concept under consideration. The examples used go into a fair amount of detail in order to describe to the user all the aspects involved in the method or command being used. The concepts described are fairly complete, and would be sufficient for the reader to use immediately or with just a little bit of fine tuning. In terms of breadth, the book covers most of the features of shell scripting while also describing the various facilities the shell provides access to in a Unix-like environment. Thus, the book does not disappoint in this front either.
In summary, probably the only thing I'd have liked to see more of is some emphasis on how to write more efficient shell scripts. Granted, most of the shell scripts described in the book are very simple and succinct, but a some words on how loops can be made better, or how to spot situations where pipes are not needed to solve a problem etc. might have been a nice addition. Some explanation of differences with dash, tcsh, zsh etc. might also have been nice, since a lot of users have different default shells. But all this isn't going to prevent me from giving this book a high rating, since it delivers quite well on the promises it makes at the beginning.
This is definitely a good book to have near your desk, and kudos to the author for having taken the effort to put it together. I would highly recommend it to the beginner and occasional shell user for a thorough read, and to an intermediate to have on his/her desk for borrowing the cool scripting ideas and applications the author has written in this book.
You can purchase Linux Shell Scripting Cookbook from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Why are there still shell scripts anyways? (Score:2, Informative)
Re:Why are there still shell scripts anyways? (Score:5, Insightful)
wow. how many ways can i answer that?
- there is no one true way
- people learn a skill and want to continue using it
- concise syntax / small footprint / fast load times
- high confidence shell scripts will run on unknown systems
- can be layered on years of useful scripts
- distrust of your options for doing work
Re: (Score:3)
- too lazy to learn another language or...
- not enough time to learn another language
Re: (Score:2)
- high confidence shell scripts will run on unknown systems
I guess it depends what you consider a "high confidence shell script". Avoiding GNU extensions to programs you run and sticking as strictly as possible to POSIX options and behavior would probably get you most of the way there, but that's a pretty big limitation.
The way I see it, the strengths of a shell for scripting are that it's an environment truly made for running other programs and interfacing them together, and it's an automation (scripting) environment that matches your interactive session (and thu
Re: (Score:2)
Sure you can use a wrench to hammer in a nail but a hammer is much more appropriate. Modern scripting languages have concise syntax, small footprints and fast load times. There is a high confidence that such languages will be on unknown systems and they have been around for years sporting vast libraries cough... CPAN.. cough that rival and exceed most anybody's bag of shell tools.
The real point is that modern scripting languages are languages and as such offer sophisticated and proper design for engineering eloquent and useful solutions to real world problems. There's a reason why languages are designed and there are reasons for each and every bit of functionality and concepts that they put in them - even if you fail to grasp them. As such there is good reason to move to them instead of arcane shell incantations that do not promote sound software engineering principals.
That said I will whip out a quick bash script to do some research or investigative type stuff but if I ever find it being run frequently, growing bigger or being put into production I usually take the time to re-write it properly in Perl.
It amazes me that you can write all that about proper design in the same message as you're advocating Perl... :)
Re:Why are there still shell scripts anyways? (Score:5, Insightful)
Re: (Score:2)
Rah! A really good comment, saves time, works with all shells.
Re: (Score:2)
Re: (Score:2)
Not all systems have perl or python installed, but all of them have a shell. Lowest common denominator is a powerful thing sometimes.
Not all systems have any particular shell installed (UNIX/Linux systems have bourne shell, but nobody uses that). Lots of Solaris systems do not have bash (I assume the book is about bash).
I certainly cannot count on any UNIX/Linux shell to be running on Windows, but perl runs fine on Windows.
Re:Why are there still shell scripts anyways? (Score:5, Insightful)
Sysadmins who can't work in plain shell scripts are dangerous. What will you do if you lose your /usr filesystem ? Be stuck with no automation tools at all as you fix by hand ? Shell scripts have little or not dependencies, and can function where other languages aren't available.
More-over they are ubiquitious. Perl, python and ruby are common in Linux, but what will you do if you're on a different Unix altogether ? A sysadmin who can use shells efficiently can easily transfer his skills to solaris, hp-ux or several others (I did just that, and now I'm paid quite a lot more than a Linux sysadmin earns as a Unix sysadmin because the skillset is rarer). Of course there are other skills you'll need to learn (package management for example is vastly different between various unixes and most are frankly primitive if you are used to Linux) but the core Unix fundamentals remain the same, and if you don't know powerful and flexible shell scripting, in my book, you're no sysadmin at all.
Re: (Score:2)
You phrase the question differently than I would. I would ask why is perl not the default shell language.
I have a hard time understand the need for bash when perl is around. I can't quite make the same argument for Python and Ruby. THose are arguably better programming languages but not better administrative script languages. Perl on the otherhand is still very close to the common shell languages while being much more powerful. It's also faster than shell for almost every purpose where time matters.
Re: (Score:3)
You phrase the question differently than I would. I would ask why is perl not the default shell language.
Because Perl is huge and sh is tiny. Would you be able to port Perl to BusyBox without dramatically increasing the size of the executable?
Re: (Score:2)
Even my phone has 256MB of ram, and its replacement will have at least 1GB. Remind me again why this is a problem?
What is up with embedded devices having less than 64MBs of ram?
Re: (Score:2)
Because I don't want to pay for more RAM in my device just because you're too lazy to use the right tool for the job? :)
Re:Why are there still shell scripts anyways? (Score:4, Insightful)
OK, how much RAM does your router have? Your cable modem? A small NAS device? Your concept of embedded devices may be only a small fraction of what's out there.
Re: (Score:3)
Re: (Score:2)
I mean consumer embedded devices. I would probably just stick with actual compiled code on devices like that.
Re: (Score:2)
In my shop it is either bash or python take your pick, perl is not to be used on company systems.
Re: (Score:3)
I take option 3, not working there.
Python could be nice, if it ignored whitespace/indentation. Also if it had its own version of CPAN.
Re: (Score:2)
I keep hearing this as the #1 complaint about Python, but isn't Python open source? Why doesn't someone just modify it to ignore whitespace and use braces instead of indents? I don't get it.
The core developers do not want that. They had an opportunity - a mile wide - with python 3.x; but instead decided to implement a lot of nearly useless features.
There is also the issue of backward compatibility.
Re: (Score:2)
I keep hearing this as the #1 complaint about Python, but isn't Python open source? Why doesn't someone just modify it to ignore whitespace and use braces instead of indents? I don't get it.
If you create your own in-house branch of Python that behaves differently from the mainline version, then you've just given yourself more work to do - maintaining this piece of software. Presumably you'd also want to keep it up-to-date with all the changes in mainline Python, so you don't find yourself in an awkward situation years later where you're still using that in-house branch and maybe you've lost the source code with your modifications but you have tons of stuff that relies on it and not a whole lo
Re: (Score:2)
I'm curious as to the reasons behind this. Is there some inherent problem with perl that it's banned or is it just a requirement to maintain consistency?
[John]
Re: (Score:2)
You phrase the question differently than I would. I would ask why is perl not the default shell language.
I have a hard time understand the need for bash when perl is around.
[snip] Larry's mantra: simple things should be simple, and hard things possible.
Perl isn't simple enough for basic scripting. Most shell scripts are little more that lists of commands with minimal control flow. For that, Perl is too verbose. The extra syntax means more places for bugs and it gets extra messy if you are generating scripts programatically. Now, I agree that more complex scripting should be done is something like Perl. "Advanced Bash" makes about as much sense to me as "Advanced duck tape application"
It's too bad that Unix shell scripting languages are so dreadful.
Re: (Score:2)
I have a hard time understand the need for bash when perl is around.
Conversely, I have a hard time understanding why anyone would use perl when bash, Ruby and Python are around.
A shell should be (needs to be) light-weight. Perl is not. Perl's syntax is horrendously complicated. Bash (and sh) syntax is not.
Re: (Score:2)
You phrase the question differently than I would. I would ask why is perl not the default shell language.
For one reason, you wouldn't want to have to type in something like
instead of
for everything you do.
However, the exact same syntax features that make sh good for interactive prompts make it a horrible programming language. IMO, it's tied with COBOL for the worst language still in general use. I'm familiar with a couple of dozen languages, and sh is the one I find hardest to use without constantly referring back to the documentation. I can never remember its crazy variable expansion
Re: (Score:2)
Use IPython [scipy.org], and not only do you get an interactive Python console with tab completion, code highlighting, simple introspection, logging, macros, profiling and more, but it also acts as a shell where you can do things like ls -l, and treat the output as a Python type that you can pipe to variables and functions if you so desire. Awesome software.
Re: (Score:3)
Perhaps because perl's about an order of magnitude larger:
# quickpkg bash perl /usr/portage/packages/app-shells/bash-4.1_p9.tbz2 /usr/portage/packages/dev-lang/perl-5.12.2-r6.tbz2 /usr/portage/packages/app-shells/bash-4.1_p9.tbz2 /usr/portage/packages/dev-lang/perl-5.12.2-r6.tbz2
# du
1120
12372
Re: (Score:2)
Try running bash without grep and awk some time. bash is smaller because it calls other programs. that makes it fragile if those other programs change.
Re: (Score:2)
Try doing anything useful in Perl without loading modules.
And I'm saying this as a Perl fan. I use it for everything that can't be done in a 10-line shell script, but for some simple automation tasks it is just overkill.
Mart
Re: (Score:2)
So nothing serious, just the stuff no business can live without holding everything together. Ok got it.
Re: (Score:2)
If you are talking about actual commercial enterprise software that is sold for money, perl probably never had anything "serious" developed using it. It's not really a language for that.
Perl is the scripting language you use when you want more than you get with shell, but don't really care if it is optimized to the point it can be used for an enterprise app. Just like bash, perl is on every UNIX-like host out there and if you write your scripts using only perl and perl modules, the code is entirely portab
Re: (Score:2)
If you're running a bunch of commands on a system and just verifying return codes, a shell script is the right tool for the job. It's also the right tool for those times when you're working on a very stripped down machine that may or may not have your favorite interpreter installed.
Re: (Score:2)
Because I've always got two or three shells open anyway. If I'm using Bash all day anyway, why not write my scripts in it?
There are definitely better scripting languages, but I get a lot more Bash practice than I do Perl or Python practice. That's all there is to it.
Re: (Score:2)
Since I use the shell as my preferred work environment, rolling up commands into scripts simply multiplies my existing effectiveness with very little extra effort.
Correspondingly, what I learn from scripting improves my ability to act quickly in a 'live' shell.
Since I work in PHP (mostly), I use it for anything that BASH can't do, since then I have access to my application libraries. If my app was in Ruby I'd use that.
Re: (Score:2)
initscripts. If you're not writing them, you aren't a software developer.
Perl server pages. (Score:2)
<<date +%d/%m/%Y>>
works on my pages. ya it prints the date... Powered by PERL.
PS. I can read an write SQL on my web pages. no programing needed.
Re: (Score:2)
echo "Lot of the time it's just quick and easy"
Re: (Score:2)
print 'I see your point. Perl is just too unwieldy in this case.';
Re: (Score:2)
/usr/local/bin/perl: bad interpreter: No such file or directory
Shoot, perl is in /usr/bin and not in /usr/local/bin. So I'll have to ln -s /usr/bin/perl /usr/local/bin/perl and hope it works.
[John]
Re: (Score:2)
Try working in an embedded environment. When you have 4 MB of flash and 8 MB of RAM, and python (even a barely usable stripped python) takes 3 MB, you use shell. Most of my programming is on embedded platforms; if I have 32 MB of flash I don't know what to do with it. I use a lot of shell and awk.
Don't even get me started on PERL; it works for some and for others it is the most obtuse incomoprehensible glop you can imagine. Instant cruft. And it takes megabytes of storage.
Just because you can't see it
Why do people thing there's only Linux distros? (Score:4, Interesting)
The script receiving stdout from telnet would watch for input lines, keeping track of what response was expected, and send either a USR1 or a USR2 back to the script outputting commands into telnet's stdin, which would then either bail or send the next command. It was rudimentary pass/fail error checking, but much better and faster than echo a; sleep 1; echo b; sleep 1; echo c...
I often prefer #!/bin/sh as I know it's a common denominator on all *nix systems, and often my scripts work even if the shell is busybox. Yes, perl or python may handle the task more efficiently and with less code, but keeping fresh on Bourne shell makes it easier to fiddle with initramfs images and embedded systems with limited space and no advanced shells. In fact, I'll often stick with it just for the challenge. I like making the original Bourne shell truly sing...
Open your eyes. There's much more to the world than just "every major (and most minor) distros." Even more than just Linux and the BSDs.
Crikey. If nobody learned assembly, we wouldn't even have operating systems.
Re:Why are there still shell scripts anyways? (Score:5, Insightful)
In particular Bourne shell makes gluing other programs together far easier, cleaner, and more reliably then can be done with any of the languages you list...and I say that even given that I'm extremely fluent in at least two of them.
Correctly and reliably handling even a simple "foo | bar | baz" construct in the languages you list can be done, but you're talking about an LOT of non-trivial systems level programming to do it, effectively "coding C inside perl/python/ruby". No, really. Sure, it's very easy to do it wrong and fragile (just call system(), what's so hard right?), but if you actually want to handle it as correctly and reliably as Bourne does out of the box...it's going to take a hell of a lot more code and detailed systems programming knowledge then even most "Sr" Unix sysadmins tend to have. Almost without fail ever attempted I've ever seen in the industry gets it wrong...typically very, very wrong.
Bourne is built to run other programs, to manage the interactions between them, programming logic being the exceptional use. All of the languages you list are built to be self-contained, programming logic being the primary task and communicating with external programs a very secondary use.
Now of course...if your script/program is self-contained and not just wrapping a bunch of exec()s of other programs, then sure Bourne shell is one of the least favorable options.
There's also nothing saying you can't freely mix and match. Most anyone worth their salt does so all the time. To do otherwise would be like saying a web coder could only work in HTML or only work in JavaScript or only work in PHP. We're talking about complimentary languages here, with some overlap.
Use the right tool for the job.
Re: (Score:2)
Shell scripting lets you turn things you do on a shell command-line into scripts, without changing them.
That you can then cruft them up with reams of flow-of-control syntax, variables out the wazoo, optargs for days, and trap statements, is just a bonus.
Sure you would have done it better in Python. But if you don't know Python you take what you type into the shell more than once, and make a script out of it.
Re: (Score:2)
I still use Bash for most automation tasks because there's no guarantee that the next person in my job is going to be familiar with Perl, Ruby, and Python, but every unix admin is going to know Bash.
If all I need to do is start up a job, make sure it created a non-zero length output file, and log a message if it ends with a non-normal return code, bash is simple and easy and has no real drawbacks.
For anything non-trivial (i.e. a monitoring plugin to do a transaction on our website) than I would use Perl (
Re: (Score:2)
Why does anyone still use shell scripts anymore?
Because they are the simplest and quickest way to do a lot of things one needs to do when using a computer. Try this in python, ruby, or perl:
the disk is nearly full, find who is the hog
in bash:
du -x / | sort -n
Now do it in another language
Re: (Score:2)
It's trivial to do it in perl!
#!/usr/bin/perl
system("du -x / | sort -n");
Re: (Score:2)
It's trivial to do it in perl!
#!/usr/bin/perl
system("du -x / | sort -n");
As a career Perl hacker, I sympathise, but you're enthusiasm is misplaced in this situation.
First, system() only gives you the return value of the command you executed, so you haven't got anything useful. Backticks would work, but they're horribly insecure if you're doing variable interpolation.
Second, a more realistic example would look like this:
for DIR in `find /home/ -maxdepth 1 -mtime -7 -type d`; do du -sh ${DIR} | grep '\d+G'; done;
This, too, is easily done in Perl, but why bother when you're just
Re: (Score:2)
It's trivial to do it in perl!
#!/usr/bin/perl
system("du -x / | sort -n");
As a career Perl hacker, I sympathise, but you're enthusiasm is misplaced in this situation.
First, system() only gives you the return value of the command you executed, so you haven't got anything useful. Backticks would work, but they're horribly insecure if you're doing variable interpolation.
That's like saying that the original poster's shell command didn't give him anything useful since it just wrote the output to stdout (i.e. his terminal).
My perl script did the exact same thing since all system does is fork() and exec(), so the stdout/stdin are inherited by the child process(es). In this case, since there were shell metacharacters in the command line, my system() argument was actually passed onto the shell for execution, so in that respect you're right that I haven't gotten anything useful -
Re: (Score:2)
It's trivial to do it in perl!
#!/usr/bin/perl
system("du -x / | sort -n");
As a career Perl hacker, I sympathise, but you're enthusiasm is misplaced in this situation.
First, system() only gives you the return value of the command you executed, so you haven't got anything useful. Backticks would work, but they're horribly insecure if you're doing variable interpolation.
That's like saying that the original poster's shell command didn't give him anything useful since it just wrote the output to stdout (i.e. his terminal).
But in shell that's exactly what you need for the next step of processing. In Perl you'd want to capture this in a variable, or split it on newlines to produce an array value, or something. That's... what, backticks? I am no Perl master.
Re: (Score:2)
But in shell that's exactly what you need for the next step of processing. In Perl you'd want to capture this in a variable, or split it on newlines to produce an array value, or something. That's... what, backticks? I am no Perl master.
He was debugging a disk space problem and wanted to see what was using the disk space. No need to put it into variables, no need to build an array, he wanted to see where his disk space was going. Sometimes even in perl, you don't want to do any complicated data manipulations, you just want to see the output.
Though admittedly, there'd be no reason to run his bash commands in Perl, but hey, he asked, so I provided.
He said this:
in bash:
du -x / | sort -n
Now do it in another language
Nothing about storing it in variables, nothing about building an in-memory data st
spellening (Score:2)
It's trivial to do it in perl!
#!/usr/bin/perl
system("du -x / | sort -n");
As a career Perl hacker, I sympathise, but you're enthusiasm is misplaced in this situation.
"you're" is a contraction. What do you suppose it's a contraction of?
OK, 5 seconds and here it is in perl (Score:2)
You left yourself wide open. Here's my answer:
#!/usr/bin/perl
exec("du -x | sort -n");
Would I use it in place of
#!/bin/sh
du -x | sort -n
No. But it's not something I have to "try" very hard to do in the same number of lines and not many more characters.
Re: (Score:2)
You listed the options backwards. Most UNIXes (you might be shocked to find that Linux isn't the only Unix variant people use) do not ship with Python, and I do everything I can to avoid having Ruby installed on even my Linux systems. And while I love perl, you are the first person I've heard describe its syntax as sane. :)
Re: (Score:2)
Why does anyone still use shell scripts anymore?
Because I'm a Unix Sysadmin?
Shell scripting is something I haven't felt the need to do in 5 years.
I'm guessing you're not a Unix Sysadmin any more?
If we were interviewing someone who said they hadn't written a shell script in 5 years, I'd have to spend some extra time querying the reasons and then think twice about hiring him. See, we use shell scripts. For init scripts, for simple tasks, for data gathering; loop through 300 systems getting the uptime for instance. Someone we hire would need to be able to perform maintenance on the scripts and hopefully not break an existing
Re: (Score:2)
Shell code is much easier to write then any other language, thats why we type it into our xterms after all instead of Python code. So naturally shell one-liners grow into shell scripts. There is also a mismatch between what Python/Ruby provides and what your shell provides, thus converting shell scripts into Python/Ruby isn't exactly straight forward and its easier to keep things as shell instead of doing a complete rewrite.
Re: (Score:2)
Why does anyone still use shell scripts anymore? Every major(and most minor) distro ships with python, ruby, and perl either built in or a trivial statement away. These languages give you access to basic OS functions and much more sane syntaxes. Shell scripting is something I haven't felt the need to do in 5 years.
Then your tasks are different from mine. There are things that can be done much better in a shell script than in Perl or Python. Anything that executes a lot of other programs for example. Anything that pipes them together. I use Perl and/or Python a lot for many things, but shell scripts still have their place. (And if you choose to use bash rather than /bin/sh the language isn't *that* primitive.)
Re: (Score:3)
Maybe because bash is installed by default on most Linux/UNIX systems and is portable across all yoru installed base.
perhaps they exist but in 20 years I've not see a Linux distro without Perl.
Or maybe your security requirements disallow python, or ruby or whatever to be installed.
if they disallow perl then shell needs to be disallowed too
Or that running shell scripts using many built-ins run faster than the interpreted languages?
this is patently false. Perl is vastly faster than shell even taking into account load times when you do anything significant. Bash scripts that do anthing important generally are calling things like awk and grep and sed to parse things and these invocations are dramatically slower in a loop than a single invocation of perl. Additionally, perl I/O is in many case faster t
Re: (Score:2)
finally invoking and disposing a command like awk many times in a row makes for very poor memory management and system resource utilization compared to keeping one program like perl resident.
Poor memory management? The awk binary and its libraries will stay resident in the pagecache between invocations, and awk+bash has a lower memory footprint than perl (on my system, running perl uses around 19MB of Virtual memory, awk+bash is around 14MB).
So, it could be argued that running awk+bash is better for memory overhead.
fork+exec overhead is so low that unless the script is calling awk thousands of times, it's not worth the time to even think about it. It takes around 2ms for me to call awk from
Re: (Score:2)
I agree with you, right up until you claim perl is remotely close to bourne syntax...
Sure, some of the latest bash' isms look perl' ish, but that's about it. Perl gets -1 trillion points for the very existence of the "unless" function... You could read through an entire program, line by line, looking for any possible issues, only to find an unless statement that is always false... Meaning none of that is ever executed anyhow.
Perl is notorious for how horrendously unreadable and un maintainable it is. Bo
Re: (Score:2)
Maybe because bash is installed by default on most Linux/UNIX systems and is portable across all yoru installed base.
perhaps they exist but in 20 years I've not see a Linux distro without Perl.
I've seen many. Usually they are embedded systems, where memory is at a premium.
Or maybe your security requirements disallow python, or ruby or whatever to be installed.
if they disallow perl then shell needs to be disallowed too
Nonsense. From a security point of view there's a lot of sense in minimising the amount of code on a system. If you have bash and perl running, and someone discovers a vulnerability in perl, you're wide open. If you just have bash, you're not. Of course, the same applies the other way round too - but there is a strong security argument for having "less stuff" installed and running wherever possible.
Or that running shell scripts using many built-ins run faster than the interpreted languages?
this is patently false. Perl is vastly faster than shell even taking into account load times when you do anything significant. Bash scripts that do anthing important generally are calling things like awk and grep and sed to parse things and these invocations are dramatically slower in a loop than a single invocation of perl. Additionally, perl I/O is in many case faster than standard unix operations that move large blocks of data.
finally invoking and disposing a command like awk many times in a row makes for very poor memory management and system resource utilization compared to keeping one program like perl resident.
So don't try to argue this on speed or system resource utilization.
You're right that it's not a h
Re: (Score:2)
When I need to rescue a server, I boot with a recovery USB flashdrive, and then I have a full linux environment at my disposal Including Perl (and probably Python and Ruby, but I've never checked).
Gone are the days when you only get what's in /sbin and if you can't fix it with cat and sed then you're out of luck.
Re: (Score:2)
Try again... I'm not taking a flight to the data center to plug in a usb flash drive.
Re: (Score:2)
Or you could use perl and `` to run whatever you want.
I use bash all the time, but sometimes perl is the way to go.
Useful (Score:2)
Re:Useful (Score:5, Informative)
Re: (Score:2)
reviewer isn't a sysadmin (Score:5, Insightful)
Seriously? So you write your application's
Re: (Score:2)
csh if on solaris
Speaking of csh on solaris, I had a devil of a time figuring out why my csh scripts never worked until I realized that I needed to always put a newline character on the last line (the system wouldn't parse the last line without it). I slowly learned to hate csh's oddities as I learned tcsh (and later, [ba]sh).
Re: (Score:3)
csh on Solaris? All scripts are generally in /bin/sh on my Solaris systems.
[John]
i can forkgive if you can forkget (Score:3)
So the bash fork bomb got popular back while I was at UNH. If you're not familiar with it, it's similar to this: ;(){ :|:& };: (I changed one character so that you dont paste it on recommendation) The way it works is trivial, and it's mystique is only in that it uses punctuation marks in lieu of letters for its own name: make a new function called :, run it and force the output into a new fork of itself in the background; then a final call to the new function. h4rdc0r3 1337.
So, we had this UNH policy that if you hung a shared unix dev box, you got evicted from CEPS (the college for the engineering/cs kids). They could give you a warning or go straight to evict. Their call.
I wrote to their head unix admin once, and with some humility, cause I knew they knew this, pointed out that they could just change the system's ulimit values to disallow casual fork bombing. The default system ulimits on the distro back then (still?) allowed a single user to consume all memory. So we saw a couple kids run the fork bomb and they never actually got evicted, cause maybe 40 grand tuition outvotes a cranky unix admin's wanton lust for cruelty.
Anyways, as a student, my issue was this - set the damned ulimit so that my editor session doesn't get wiped with my homework -- you can't argue for frequent backups when more than 0 seconds of hard work are arbitrarily on the line solved by a simple config change.
But the UNH unix admin guy replied to me and told me that yeah, he knew about ulimit back when I was in diapers, but he wasn't going to change it up. And a few more times that semester, I lost a couple lines of uncommitted code due to some clown pressing enter on a dare.
Fork ya later,
Comment removed (Score:5, Insightful)
Re: (Score:2)
There's a reason we use /bin/sh (Bourne) to write scripts, and it's the same reason we use vi to edit them - mainly it's available on ALL UNIX systems. Despite widespread adoption, bash is not universally available - sorta like EMACS.
If I stick it in root's cron on, say, an AIX system and it pukes I'm not interested in rewriting somebody's bash script to handle running under sh, csh or ksh. I want it written by its original author in sh. And I don't want to hear about a shebang line (#! /bin/bash) - if I haven't installed bash, it'll still puke.
That makes good sense if you are writing scripts that you will have the need to run from maintenance mode, but that's a pretty small amount of administrative scripting when it comes down to that.
However, if you are writing all of your shell scripts with the assumption that you will not be able to access something as widely available on all UNIX platforms as bash, even when those scripts are not needed in maintenance mode and/or make reference to things that wouldn't be available in maintenance mode anyway,
Re: (Score:2)
> mainly it's available on ALL UNIX systems
Mainly it doesn't matter since this book is about LINUX shell scripting.
Besides Bash, a lot of tricks and utilities are only available if you assume a Linux environment.
Re: (Score:2)
There's a reason we use /bin/sh (Bourne) to write scripts, and it's the same reason we use vi to edit them - mainly it's available on ALL UNIX systems. Despite widespread adoption, bash is not universally available - sorta like EMACS.
But not all GNU/Linux systems. Gentoo is a prime example. Nano instead of vi. And as sibling stated: almost no GNU/Linux system has true sh, they symlink sh to bash.
Oh, and don't stick things in a crontab on Mac OS X systems. Mac OS X prefers XML plist files for scheduling and startup.
Re: (Score:2)
You, sir, understand the Unix philosophy.
Re: (Score:2)
Not all scripts are meant to be re-usable.
I often find myself writing a script directly in my terminal, without going through a text editor. A simple for i in `find the_files_i_want_to_do_something_with | preprocess_the_input`; do what_I_want_to_do_on_those_files $i; done.
In this particular, I'm just using the shell I happen to be running (bash). I don't have to care about compatibility. Whatever works to get my job done is enough.
Actually, whenever I want to write a re-usable script, I write it in Python.
Re: (Score:2)
A longstanding pet peeve of mine are systems that symlink /bin/bash to /bin/sh
Doesn't bash emulate sh if it's called as sh?
Re: (Score:2)
Perhaps but if you're in single user mode and can't log in because the shared library isn't available, what do you do now?
At least that's where my pet peeve would come from. It doesn't happen often but when it does, it's a pain to get the system working.
[John]
Re: (Score:2)
Re: (Score:3)
bash uses shared libraries and sh doesn't. If you lose the /lib directory (or /usr/lib, or whatever), you lose the ability to log in if a system crashes to single user mode and the library file system didn't mount. Then it's boot to CD to recover which is great fun on a remote system (but not always impossible with virtual mounts, etc).
[John]
Re: (Score:2)
This link has a tabular description of the differences between sh & a small raft of other shells [faqs.org], including bash. Bash syntax is very similar similar to sh, but there are a few gotchas, which I'm too long out of the game to remember.
This book gets the details critically wrong. (Score:4, Informative)
I've just had a look through the book via Amazon's Look Inside.
I simply cannot fathom how so many authors manage to get a book published on a subject they don't even master properly. "Shell scripting" is by far the worst in this category. So much that I recommend anyone to run screaming in the other direction when you see mention of "shell scripting" in any title. If it mentions "Linux", it's usually also an excellent indicator of junk. Not to take away from the beauty of the Linux kernel (which obviously has nothing at all to do with shell scripting, but let's ignore that fact).
I obviously haven't read the book, but when I skim through it and I see failure to quote parameters, failure to recognize the difference between executing a script with /bin/bash in the hashbang and running it by passing it as an argument to sh, and more, I know for a fact that I'm going to be sitting in #bash trying to re-educate poor misguided souls for as long as I can suffer it.
To those interested in the subject: Do not read this book. Do not read this "Advanced Bash Scripting Guide" mentioned in this review. You will end up writing junk.
Go see Greg Wooledge's wiki, it's got an extensive FAQ of actually useful issues (http://mywiki.wooledge.org/BashFAQ), it's got a great newbie guide (http://mywiki.wooledge.org/BashGuide), it's got a great cheat sheet (http://mywiki.wooledge.org/BashSheet), and much more. Best of all: Unlike that broken ABS, they're all community supported and unlike this book, the knowledge is free.
Re: (Score:2)
What's wrong with the Advanced Bash Scripting Guide at tldp.org?
It's not perfect, but it will get you along way.
Re: (Score:2)
The main issue I have is that the "Advanced Bash Scripting Guide" seems to be fascinated with listing every possible way to do the things it covers in bash. But missing from that tutorial is any notion of which of those approaches is the best one in any context. Certain shell constructs are best avoided, in favor of ones that are either more portable to other shells, more resilient in the face of bad data, or easier to maintain. But you get no sense of this from the guide; just a set of ways to hack the
Re: (Score:2)
I'm of the opinion (just an opinion, mind you) that shell scripting is a bit of a black art and very quirky and unpredictable. I do my shell scripting by "hack and test, repeat."
I think it's very dangerous when people claim that shell scripts can be portable (some people abhor bash and say that you should stick with sh) especially since they rely on the commands and utilities installed on any particular system. Are they GNU, BSD, SysV, POSIX? What versions are they? What directories are they in etc.
I thin
Re:This book gets the details critically wrong. (Score:5, Funny)
Go see Greg Wooledge's wiki
Just not all at once, please.
Thank you! (Score:2)
I had been limping long with the Advanced Bash Scripting Guide and Google searches for a while. The Wooledge guide looks like a vast improvement. Thanks.
Re: (Score:2)
This might be the appropriate place to mention the venerable Unix Power Tools [oreilly.com].
It goes through most of the common Unix commands (including Bash, but also sort, uniq, awk, sed, etc.). It teaches you about redirection and piping, customizing your shell prompt, xterms, X, remote X, Unix file times and permissions, find/exec, diff, head/tail, grep, vi, emacs, jobs, cron, time, regexes, sed, bash scripting, tar, rcs/cvs, perl/python, /dev, symlinks, tty's, ssh, users/groups.
In short, everything you ought to know
yahoo (Score:2)
The only interesting thing about this book was to find the author's web site and see how many hoops there are nowadays to get through the Yahoo interview process. I've a friend who managed to get a senior sysadmin role at Yahoo a few years ago who can barely program and has no computer science knowledge whatever - his only previous experience was some ISP admin work, although he was a tremendous suck-up to^W^W^Wgood friends with a few core FreeBSD developers. Anyway, I had the impression that the company wa
Boy Goatsex is out in force today... (Score:3)
Every topic is littered with them...
Re: (Score:2)
Re: (Score:2, Interesting)
Re: (Score:2)
It really should just expand them when the comment is submitted.
Re: (Score:2)
The problem is then these folks will link to blog pages that they setup to host this material. Some have already started doing that.
Re: (Score:2)
Agreed!
This book is probably interesting, but in my opinion if you are doing anything beyond a basic set of commands run in order (with _maybe_ some very primitive control statements) .. you are far better off using a more powerful scripting language... or even a real programming language.
Personally I don't like shell scripts at all for anything more complex than a temporary convinience. From what I've seen they are usually poorly written (lacking proper error handling and logging), are usually riddled with
Re: (Score:2)
Is this what you were thinking of?
ls -l | awk {'print $5'} | sort | tail -n1
Re: (Score:2)
ls -l |sort -k3|tail -1
Re: (Score:2)
Two processes.
Re: (Score:2)
Re: (Score:2)
That's somewhat clever, but some of us do know what base-64 encoding is.