Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
Programming Books Media Book Reviews IT Technology

Regular Expression Recipes 258

Posted by timothy from the prune-talkin' dept.
r3lody writes "If you spend time working writing applications that have to do pattern matches and/or replacements, you know about some of the intricacies of regular expressions. For many people they can be an arcane hodgepodge of odd characters that somehow manage to do wonderful things, but they don't have enough time (or interest) to really understand how to code them. Nathan A. Good has written Regular Expression Recipes: A Problem-Solution Approach for those people. In its relatively slim 289 pages, he offers 100 regular expressions in a cookbook format, tailored to solve problems in one of six broad categories (Words and Text, URLs and Paths, CSV and Tab-Delimited Files, Formatting and Validating, HTML and XML, and Coding and Using Commands)." Read on for the rest of Lodato's review.
Regular Expression Recipes: A Problem-Solution Approach
author Nathan A. Good
pages 289
publisher Apress
rating 8/10
reviewer Raymond Lodato (rlodato AT yahoo DOT com)
ISBN 159059441X
summary A cookbook of useful regular expressions for Perl, Python and more.

Regular expressions are not restricted to just the Perl or shell environments, so Nathan offers variations for Python, PHP, and VIM as well. In most cases the translation is relatively straight-forward, but in a few cases a different environment may have (or lack) additional facilities, prompting a different expression to do the same task.

Before you even read chapter 1, Nathan provides a quick summary course on regular expressions, with detail given to each of the five environments you might utilize. He has written the syntax overview in a highly-readable format, making it easy to understand the gobbledy-gook of the most bizarre concoctions you might encounter.

The first chapter (Words and Text) starts simply enough. He gives examples of how to find single words, multiple words, and repeated words, along with examples of how to replace various detected strings with others. In each case he gives an example of its use for each platform, followed by a bit-by-bit breakdown of how it works. Not every environment is given on every example, and in many cases the "How It Works" section refers to the first one, as most REs are identical between the platforms.

The next chapter (URLs and Paths) offers various methods of doing commonly needed parsing. Pulling out file names, query strings, and directories, as well as reconstructing them in useful fashions is covered in the 15 offerings given here. Validating, converting, and extracting fields of CSV and tab-delimited files are handled in chapter 3, while chapter 4 is concerned with validating field formats, as well as re-formatting text for the fields. Chapter 5 handles similar tasks for HTML and XML documents. The final chapter covers expressions that facilitate the management of program code, log files, and the output of selected commands.

First, I must admit that there are a number of useful solutions provided, especially for someone who is concerned with application and web development. However, I did feel a little cheated by the fact that several chapters covered essentially the same task, with only minor variations. It almost seemed as though the author was trying to pad out the solution count to the magic number 100. A simple example: three solutions in chapter one cover (a) replacing smart quotes with straight quotes, (b) replacing copyright symbols with the (c) tri-graph, and (c) replacing trademark symbols with the (tm) sequence. In each case, the expression was simply "s/\xhh/ rep /g;". Did we really need three separate chapters for that? I don't think so.

Another quibble revolves around some of the coding of the expressions. Nathan has made liberal use of the non-capturing groups (that is, (: expr )) to insure only the items that needed replacement were captured. While a worthy idea, in some cases the expression may have been simplified for understanding. Another issue is a slight error in searching for letters. In a number of expressions, Nathan uses [A-z] to capture all letters. Unfortunately, the special characters [, \, ], ^, _, and ` occur between upper-case Z and lower-case a, making it match too much. Either [[:alpha:]] or [A-Za-z] should have been used.

Despite these quibbles, Regular Expression Recipes does provide a useful compendium of solutions for common problems developers face. Presenting the information in a cookbook fashion, along with ensuring that those using something other than Perl don't have to sweat translating the expressions to their target language, makes this a handy book to have. I wouldn't hesitate to recommend it.

You can purchase Regular Expression Recipes: A Problem-Solution Approach from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
This discussion has been archived. No new comments can be posted.

Regular Expression Recipes More

Regular Expression Recipes

Comments Filter:

  • Curious (Score:3, Funny)

    by LiquidCoooled ( 634315 ) on Tuesday March 22, 2005 @02:48PM (#12014937) Homepage Journal
    I was performing a strange custom regular expression on the book review, and discovered that it outputted the following:

    "Regex coders are in league with the devil"

    Who woulda thunk it!

  • Points (Score:5, Informative)

    by 2.7182 ( 819680 ) on Tuesday March 22, 2005 @02:49PM (#12014946)
    I really liked this book, but

    1. the binding broke
    2. the index has a lot of typos.
    • 2. the index has a lot of typos.

      No problem, the website issued a global regex and a pot of tip-ex for all customers.
    • This seems to be typical for tech books: Way overpriced (although this one seems more reasonable), incredibly crappy binding, and less than aggressive proof reading.

      • You may be right, but its my experience that tech books are surprisingly reasonable in price. My average textbook in any other field costs >$100, and my tech books are usually in the $40 range. Of course, this could just be that there's less demand for "Financial Accounting" than my PHP or Java books. Dunno, just my experience.

      • Re:Typical... (Score:4, Insightful)

        by Monkelectric ( 546685 ) <slashdot AT monkelectric DOT com> on Wednesday March 23, 2005 @12:20AM (#12020487)
        I had a brief skirmish with the tech book publishing industrty (and believe me thats the right word). The real problem is they pay authors BY THE PAGE so their incentive is to write flowery, lengthy language which conveys as little information in as much space as possible. This in turn justifies high book prices and higher author royalties.
    • 2. the index has a lot of typos.

      Yeah, but in a book about regexes, you have to study the index VERY CAREFULLY to determine whether there are any typos or not.

  • Bran... (Score:3, Funny)

    by Anonymous Coward on Tuesday March 22, 2005 @02:49PM (#12014950)
    ...is the best regular recipe.

  • Another one? (Score:2, Insightful)

    by cmstremi ( 206046 )
    Isn't there already enough coverage for Regex's? With all the existing books and the nearly endless availability of free information and sites (including many using the 'recipie' format) online, who will want this book.

  • Regular expressions in a cookbook? (Score:5, Informative)

    by DeadSea ( 69598 ) * on Tuesday March 22, 2005 @02:51PM (#12014962) Homepage Journal

    Sounds like good eating. ;-)

    Regular expressions are great, but once you know them and you think you can conquer the world, I find they occasionally let you down. The text editor I was using had a rudementary regular expression search that did not support non-greedy matching. I found that writing a regular expression that finds C style /* comments */ to be quite tricky with only greeding matching [ostermiller.org]. I wrote it up as an article where I build the expression piece by piece showing common things you might try that won't work.

    If you want more of a challenge, try writing a regular expression that find any <script></script> tags along with anything in between using only greedy matching. You will find that the length of your regular expression goes up exponentially with the length of your ending condition.

    --
    Calculator for Converting Currency [ostermiller.org]

  • REGEX (Score:5, Funny)

    by null etc. ( 524767 ) on Tuesday March 22, 2005 @02:54PM (#12015003)
    Another quibble revolves around some of the coding of the expressions. Nathan has made liberal use of the non-capturing groups (that is, (: expr )) to insure only the items that needed replacement were captured. While a worthy idea, in some cases the expression may have been simplified for understanding.

    I'm not sure I understand what your quibble is - do you dislike the fact that he uses non-capturing groups, or the fact that he disposes of them at certain points?

    Another issue is a slight error in searching for letters. In a number of expressions, Nathan uses [A-z] to capture all letters. Unfortunately, the special characters [, \, ], ^, _, and ` occur between upper-case Z and lower-case a, making it match too much. Either [[:alpha:]] or [A-Za-z] should have been used.

    This seems like a relatively novice mistake, and I'm surprised it would show up in a book on regular expressions.

    Despite these quibbles, Regular Expression Recipes does provide a useful compendium of solutions for common problems developers face. Presenting the information in a cookbook fashion, along with ensuring that those using something other than Perl don't have to sweat translating the expressions to their target language, makes this a handy book to have. I wouldn't hesitate to recommend it.

    It's nice that he covers five environments for regular expressions. I'm sure everyone has heard of Mastering Regular Expressions [oreilly.com], published by O'Reilly. The Perl Cookbook [oreilly.com] also does a good job at solving common problems with Regular expressions.

    This is just my opinion, but I think what the world needs is a book on Regular Expression Design Patterns.

  • Unacceptable mistakes (Score:5, Interesting)

    by gniv ( 600835 ) on Tuesday March 22, 2005 @02:55PM (#12015008)
    In a number of expressions, Nathan uses [A-z] to capture all letters.

    How can this be a good book when it makes such mistakes? If this book is for beginners (as it seems) the editing process should have been much better.

    • *Technically*, [A-z] does capture all letters. It does not, however, capture *only* letters.

      (Just to be pedantic.)
    • In a number of expressions, Nathan uses [A-z] to capture all letters.

      How can this be a good book when it makes such mistakes? If this book is for beginners (as it seems) the editing process should have been much better.

      I guess the author was a novice and used Word to type the book. Word is notorious of automiscorrecting technical documents.

  • Minor variations (Score:5, Funny)

    by pocari ( 32456 ) on Tuesday March 22, 2005 @02:55PM (#12015009)
    However, I did feel a little cheated by the fact that several chapters covered essentially the same task, with only minor variations.

    I can relate. I have cookbooks for food that have all these recipes that are nothing but flour, butter, eggs, and sugar. Do we need all these recipes for pancakes, cupcakes, cookies, crepes, waffles, popovers, bread, quick bread, bread sticks? Won't people figure out eventually to put a little less sugar in waffles with savory ingredients?

    Japanese cookbooks are even worse. Soy sauce, sake, mirin...boooooooring!

    • I can relate. I have cookbooks for food that have all these recipes that are nothing but flour, butter, eggs, and sugar. Do we need all these recipes for pancakes, cupcakes, cookies, crepes, waffles, popovers, bread, quick bread, bread sticks?

      If you think there's only a minor variation between cookies and bread, let me adopt you. You'll be the easiest kid ever to take care of.

      Yum, peanut butter and jelly cookies'mich.

  • I personally... (Score:5, Informative)

    by BlueCodeWarrior ( 638065 ) <steevk@gmail.com> on Tuesday March 22, 2005 @02:55PM (#12015012) Homepage
    ...use 'Mastering Regular Expressions [oreilly.com] . It's a good book on the topic as well.

    • Re:I personally... (Score:3, Informative)

      by Bryson ( 112202 )
      > use 'Mastering Regular Expressions . It's a good book on the topic as well.

      I'm one of the few people who doesn't like Friedl's /Mastering
      Regular Expressions/. (I have the first edition.)

      First, he says that extended regexp engines, such as Perl's, use
      nondeterministic finite automata (NFA). Not true; NFA's can
      accept exactly the same languages as DFA's (deterministic finite
      automata). The extended regexps use search-and-backtrack
      engines.

      Friedl gives some examples of (extended) regexps that have
      catastro

  • add this book to your list (Score:3, Informative)

    by yagu ( 721525 ) <yayagu.gmail@com> on Tuesday March 22, 2005 @02:55PM (#12015014) Journal

    While I can't vouch for the quality of the reviewed book,if you want something definitive on regular expressions, Mastering Regular Expressions, Second Edition [amazon.com] by Jeffrey E. F. Friedl is an absolute must for your professional library. Jeffrey breaks down and then builds back up what regular expressions are and how they work, and offers an entire matrix breakout of the slightly different implementations among the most common utilities (grep, sed, awk, perl...). Not to shill for amazon, but if you select the reviewed book, the "buy this book too, and you get this great price" deal actually includes the Mastering Regular Expressions, Second Edition. . Get 'em both, you won't be sorry.

  • two problems (Score:4, Funny)

    by EphemeralPhart ( 107572 ) on Tuesday March 22, 2005 @02:56PM (#12015026)
    Some people, when confronted with a problem, think ``I know, I'll use regular expressions.'' Now they have two problems.

    Jamie Zawinski

  • Linda Richmond says... (Score:5, Funny)

    by Anonymous Coward on Tuesday March 22, 2005 @03:05PM (#12015146)

    I'm feeling a bit verklempt!

    Talk amongst yourselves!

    Alright, I'll give you a tawpic:

    "Regular Expressions are neither regular nor expressions."

    Discuss.

  • a Cookbook eh? (Score:3, Funny)

    by chiapetofborg ( 726868 ) on Tuesday March 22, 2005 @03:08PM (#12015187) Homepage
    Anyone have any good recipies for [cookies]+ ?

  • Regexes are overused (Score:5, Informative)

    by ryantate ( 97606 ) <ryantate@ryantate.com> on Tuesday March 22, 2005 @03:14PM (#12015263) Homepage
    Anyone who drops in regularly on a Perl discussion forum (like perlmonks.org) knows that programmers tend to over-use regular expressions.

    Regexes are actually a pretty poor way to extract information from comma-delimited or tab-delimited files, for example. By the time you're done dealing with escaped commas, escaped tabs, quoting characters (which many CSV and TDT exporters use in addition to commas and tabs), escaped quote characters, escaped newlines, and escaped escape chars, you end up with a super-complicated regex.

    HTML is even more complicated. You have HTML comments and nested tags on top of everything else.

    To validate a simple email address, Jeffrey Friedl in his Mastering Regular Expressions book for O'Reilly writes an *11-page* regex.

    Most of the time the correct answer is not "here is a regex recipe" but rather "here is a simple library to do the job property with a parser", like Text::CSV or HTML::Parser in perl.

    • Of course, the compiled regex will likely be faster than any parsing library you write. So it all depends what you're doing.

      For some sort of system that processes umpteen billion transactions per second, they can be a godsend. For parsing a .conf file once every six months when the machine is rebooted, it's a waste of time.

      It's all about knowing how and when to use the tool. A pneumatic nailgun can save a carpenter hours on a jobsite, but it's a waste of time to set it all up if you only need to knock

    • Re:Regexes are overused (Score:4, Insightful)

      by Black Perl ( 12686 ) on Tuesday March 22, 2005 @03:29PM (#12015431)
      Yes, exactly. Any good book on Regexes should have a chapter on when NOT to use them.

      I see many people trying to use regexes to do parsing, when they should be using a specialized parser.

    • Re:Regexes are overused (Score:4, Funny)

      by Anonymous Coward on Tuesday March 22, 2005 @03:37PM (#12015541)
      > *11-page* regex.

      I think that's a sure sign of insanity. Or autism at the least.
    • I agree -- many parsing jobs are much simpler doing basic character-at-a-time C code, especially validation.

      If you're searching for occasions of something or other in a long document, grep is obviously going to be an easy way (with regex's), but if you want to extract the hostname from a URI, just code it.
    • To validate a simple email address, Jeffrey Friedl in his Mastering Regular Expressions book for O'Reilly writes an *11-page* regex.

      That's not quite fair. That regex validates any RFC822 address, and the syntax allowed isn't simple. Validating things that are currently used is fairly easy, but there's a lot of historical baggage in RFC822 addressing.

    • Re:Regexes are overused (Score:4, Insightful)

      by 2short ( 466733 ) on Tuesday March 22, 2005 @10:30PM (#12019672)
      "an *11-page* regex."

      That's insane. My feelings on Regexes were set early in my career. I discovered them, and like many started using them everywhere. Then in a code review, my boss pointed to one particularly complex one and said "See, there's why you shouldn't try to do such complex things with regular expressions, this one has a bug" "Where?" says me. "Let's leave that as an exercise for the student. Come ask me if you can't figure it out in an hour or so." Well, I certainly wasn't going to admit defeat, even though it took me several hours to find the rather subtle problem. So I went back and demanded to know how he had spotted it so fast. And he said "I didn't. It was a regex 3 lines long. It had to have a bug."

  • Regex Coach helps building Regexp (Score:5, Informative)

    by uss_valiant ( 760602 ) on Tuesday March 22, 2005 @03:18PM (#12015294) Homepage
    Regex Coach [weitz.de]

    This program assists you building regular expressions. I've never used it (real men code regexp at once and it works). But some friends recommend it.
  • The vi-style regular-expression substitution technique might help: :-)

    "If you spend time working writing applications that have to do pattern matches and/or replacements, you know about some of the intricacies of \(regular expressions\). For \(many people\) \1 can be an arcane hodgepodge of odd characters that somehow manage to do wonderful things, but \2 don't have enough time (or interest) to really understand how to code \1."

  • Different flavors? (Score:4, Informative)

    by dpbsmith ( 263124 ) on Tuesday March 22, 2005 @03:30PM (#12015448) Homepage
    In an average month, I use regular expressions as implemented in Microsoft Visual C++ 6.0, BBEdit Lite, TextWrangler, Apple MPW, and REALBasic. Every single one of them has _significant_ differences in syntax and semantics.

    My understanding is that even the UNIX world sports several different flavors of regular expression in grep, egrep, fgrep, etc.

    The biggest barrier to _my_ use of regular expressions is that every time I switch from one regular expression context to another, it takes me a good half hour to refresh my memory of what does and doesn't work in each environment.
    • My understanding is that even the UNIX world sports several different flavors of regular expression in grep, egrep, fgrep, etc.

      Er, well, not exactly. grep, extended (egrep) and fixed (fgrep) allow for different feature/speed tradeoffs, but they are consistent in their use of regular expressions. Where you will find differences is between the regex syntax of vi, perl, sed, grep, etc.

      After ten+ years, I still consult a reference for all the escape codes and such. Used to be a book, now it's google.

  • HTML, XML, CSV, but why? (Score:4, Interesting)

    by AGTiny ( 104967 ) on Tuesday March 22, 2005 @03:33PM (#12015499)
    Of course everyone should know how to build a regex, but why take time discussing how to parse common formats such as HTML, XML, CSV, and so on? Every language likely has a good standard module/library/package that does it all for you, hopefully in the most efficient way, and gives you an easy API. I write Perl, and have used XML::*, HTML::*, DBD::CSV, Text::CSV, the list goes on. No need to write a single regex there. Another good set of modules is Regexp::Common, giving you correct regexes for parsing semi-hard things like IP addresses, MAC addresses, phone numbers, etc.

  • Free Alternative (Score:4, Informative)

    by MudButt ( 853616 ) on Tuesday March 22, 2005 @03:48PM (#12015661)
    This is free... And interactive...
    http://www.regexlib.com/ [regexlib.com]

  • About 279 pages too long (Score:4, Insightful)

    by natoochtoniket ( 763630 ) on Tuesday March 22, 2005 @03:57PM (#12015764)
    I have a huge, 1000+ page Betty Crocker cookbook which I hardly ever use. It gives detailed recipes for particular dishes, but nothing that helps me to just throw a dinner together. And nothing that helps me to create anything new.

    My very favorite recipe book is a tiny little thing of about 40 pages. For each kind of meat and each kind of vegetable, it lists what spices and sauces go well with it, how long and how hot to cook it, and how to tell when it is done. There is a little section on how to make about a dozen differnet sauces. That's it.

    A programming language has syntax and semantics. For regular expressions, Chomsky gave both fully in his original paper on the subject. The added conveniences that some utilities provide are all listed in their respective man pages. The entire subject, if it were collected together, should be about 10 pages. With some explanation of language theory, grammars, and such, the whole might be worth a chapter. Get out an undergraduate compiler-theory book (such as Aho/Sethi/Ullman). They have less than a chapter on regular expressions, and they cover the topic fairly well.

    But, I suppose, there is a difference between a cookbook that is made for cooks to use as a reference, and a cookbook that is made for non-cooks to follow by rote. Learn how to cook. You will be surprised how seldom you actually refer to the 1000+ page cookbooks.

  • Whenever I need to use some regex, I google for a regex reference and try to figure out how to do what I want to do. Then the next time I need to use regex, I have to do it again. I literally cannot hold regex in my head for more than a day or so.

  • check out regex coach if you want to learn (Score:2, Interesting)

    by Anonymous Coward
    I found this tool while doing my undergrad. Having this tool and playing with it showed me how to understand and how to sucessfully write regexs. 5 minutes of playing with it and you be enlightened.

    http://www.weitz.de/regex-coach/ [weitz.de]
  • That's a phrase a co-worker once tossed out to differentiate regex wranglers from lowly code cowboys. The implication being that real programmers use REs. At that time in my life I knew a dozen or so programming languages, but had avoided learning REs. That little quip prompted me to start learning, first in AWK, then via Perl. Today, I'm proud to say that I can fumble my way around a regex pretty good. I'm still a little fuzzy when it comes to concepts like "negative look-ahead" and "positive look-beh
  • Rather than relying only on regular expressions, it would be beneficial to use regexps along with sed/(g)awk/perl. If the incantation that you use using regexps is obscure to you, how will the next guy who will support your stuff feel? Break up your uber regexp into a simpler combination of regexp(grep)/sed/awk combination.

    With that, I almost always use anchoring via ^ or $.

  • :help pattern (Score:4, Informative)

    by digitect ( 217483 ) <digitect@Nospam.dancingpaper.com> on Tuesday March 22, 2005 @09:23PM (#12019047)

    Of course, if you use the one true text editor [vim.org], all you need to know about regular expressions is:

    :help pattern

    :)

Slashdot Top Deals

Ocean: A body of water occupying about two-thirds of a world made for man -- who has no gills. -- Ambrose Bierce

Close