The Art of Computer Virus Research and Defense

nazarijo writes "I think by now we're all familiar with viruses and worms. It may have been a term paper diskette chewed up by a virus back in college, a family member's computer infected with the latest worm, or your email inbox clogged with a mass mailer of the week. But how do AV researchers dissect such malware, especially when virus writers have devoted so much time to avoiding detection and perfecting their craft with self-decrypting viruses, polymorphic shellcode, and obfuscated loops. Haven't you wanted a peek into how that's done, and how you would analyze such a monster that landed in your computer? Well, Peter Szor's book The Art of Computer Virus Research and Defense (TAOCVRD) has been gaining lots of critical acclaim lately for filling that gap, and rightfully so. (Before we begin, however, I should make one thing perfectly clear: I was a technical reviewer of this book. I enjoyed it when I read it originally, and I'm even more pleased with the final result. And now on to your regularly scheduled review.)" Read on for the rest.

The Art of Computer Virus Research and Defense
author	Peter Szor
pages	713
publisher	Addison Wesley Longman and Symantec Press
rating	9
reviewer	Jose Nazario
ISBN	0321304543
summary	Clear, sweeping coverage of virus history and technical details

TAOCVRD opens with Part 1: Strategies of the attacker. Here we get to start to think about malicious code from the original ideas and viewpoints of its makers. Chapter 1 opens up with various games of the classic computer science world, including Conway's Game of Life and Core Wars, which is still fun after all of these years. From this we can start to think about computer viruses as a natural extension of other self-replicating computer structures. What's great about this chapter is that you can actually understand, and share in, the fascination of replicating code. It's as if you can understand the pure world that some virus writers live in.

Chapter 2 starts off the virus-analysis section, including some of the basics (like the types of malicious programs and their key features), as well as the naming scheme. Chapter 3, "Malicious Code Environments," serves as a lengthy and complete description of how various viruses work. The dependencies that you would expect to see, including OS, CPU, file formats, and filesystems, are all described. Then Szor goes on to describe how viruses work with various languages, from REXX and DCL to Python and even Office macros. Not all of the descriptions are lengthy, but you get to see how flexible the world of writing a virus can be. What I most enjoyed about the book overall is represented in this chapter, namely Szor's command of the history of the virus as well as his technical prowess, which he drops in as appropriate.

Chapter 4 gets a bit more technical and now focuses on infection strategies. Again, Szor isn't afraid to delve into history or technical meat, including a lengthy and valuable section "An In-Depth Look at Win32 Viruses." If you don't feel armed to start dissecting viruses by this point, you're in luck: there's so much more to read. Chapter 5 covers in-memory strategies used by viruses to locate files, processes, and sometimes evade detection. Szor has a list of interrupts and their utility to the virus writer, providing a comprehensive resource to the virus analyst.

Chapters 6 and 7 cover basic and advanced self protection schemes, respectively, used by viruses. TAOCVRD's completeness of information in a usable space, together with very functional examples and descriptions, is again evident. Szor walks you through a basic decryptor routine, for example, showing you how a self-contained virus can be both evasive and functional at the same time. Sadly little attention is given to various virus construction kits at the end of chapter 7, though.

Chapters 8 and 9 get a little less technical and somewhat more historical. These chapters cover virus payloads and their classification (ie benevolent viruses, destructive viruses, etc) and computer worms, respectively. The overview of payloads is almost entirely historical, giving a great overview of how virus writers have used their techniques to cause havoc or just have "fun" from time to time. Chapter 9 gives a concise and valuable overview of computer worms, almost boiling about half of my worms book down into just one chapter in a clear and easy to use fashion.

Part 1 concludes with chapter 10, which covers exploits and attack techniques used by worms and viruses. Again, Szor's clarity of explanation shines as he artfully gives a concise overview of how a buffer overflow attack works (including stack layout and address manipulation), heap-based attacks, format string attacks, and related methods. He then discusses these techniques in light of various historical examples, clearly explaining how they operated and were successful. If you've been yearning for a short overview of attack techniques and how malware has used them, this chapter is for you.

Part 2 covers the defender's strategies. Chapter 11 serves as a nice introduction to this section by describing many of the current and advanced defense techniques such as some of the first and second generation scanners, code and system emulation, and metamorphic virus detection. This is all covered in nice technical detail, always at a reasonable level to not leave everyone in the dust. Through it all small examples are constantly given, which reinforce the text nicely. Chapter 12 is very similar, this time focusing on in-memory scanning and analysis techniques.

Chapter 13 covers worm blocking techniques, focusing on host-based methods which can prevent the buffer overflow from being successful or the code from arbitrarily gaining network access again. Chapter 14 complements this with network specific defenses, including ACLs and firewalls, IDS systems, honeypots, and even counterattacks. These two chapters are a lot less technical than the previous two, but still quite valuable.

By this point I'm sure you're ready to try your hand at virus analysis, and Szor is eager to help you out. In chapter 15 he gives you a great setup for virus analysis, including various tools and examples of how they work and what kind of information they give you. Finally, in chapter 16 you have the obligatory (and valuable) resource roundup which complements the references given in every chapter, as well.

Overall I find Szor's book to be amazing, both in terms of its technical prowess over so many specifics in the field but also for its presentation. Without dumbing it down, Szor's able to communicate to most readers with clarity in a manner they'll understand, learn from, and be able to use. I think that many of us, especially those of us who get plundered in our email inboxes with malware, are curious to spend some time dissecting these beasts using techniques AV professionals use, and Szor's book does an exemplary job of introducing that world to us all. I consider this to be one of the most important computer security books I own due to it's clarity and completeness of coverage.

---

You can purchase The Art of Computer Virus Research and Defense from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Is it just me...

Is it just me or does this sound like it would be a really cool job--especially if you like sleuthing.

Too Much Joy

I think you have hit the nail on the head here.

Reverse engineering malware is so much fun, and appeals to techie and tech-savvy manager types so much that it has been a terrific and terrible distraction. I've seen the effect firsthand -- companies waste precious limited mitigation and response talent and time trying to analyze malware when they should be taking immediate action to contain the spread of a worm.

Corporations and government agencies have been so thoroughly trained by the AntiVirus industry that they have a hard time coping in an age of the zero day worm, flash worm, or even the boring ordinary retread worm with 800 variants that do different things and propagate through a dozen different old defects. In fact, in the last year it's become clear that worms targeting many old defects can spread widely, slipping in under the radar of AntiVirus definitions with dozens of daily variants. (It's hard to patch a large network, and the industry hasn't woke up to the fact that it's also hard to keep it patched.)

What does it matter, which of the 800 strains of Spybot or Rxbot is smacking your PC's around? Well, if it were possible to quickly assess exactly what a given strain might do on a computer, it might be. But typically it's not possible.

In fact, it's gotten to the point where the AntiVirus vendors themselves have all but given up on detailed analysis of the many variants emerging each hour. Sometimes critical features of a strain (what ports does it probe, etc.) are missing entirely from the public analysis of the strain for weeks after it was first detected. Sometimes one vendor will describe a feature while others don't. Obvious cut-and-paste errors in the analysis of major vendors can also be observed, if one pays close attention.

The AntiVirus industry can't keep up the analysis of every minor strain, but they do continue the practice because it's a proven effective strategy for keeping mindshare. To their credit, they do a pretty reasonable job of rapid analysis and signature development on quite a few variants every day. Unfortunately, the stakes are pretty high and getting higher.

The bottom line for big networks: focus on prevention and containment. Cleanup is very costly, so do your own analysis if you must, but don't let it delay or sap resources from containment efforts when a worm hits. Other damages might be mounting while the mitigation effort stalls out because an incident response team is bogged down trying to answer the question: "Does the variant that hit your network today have a keystroke logger?"

With several variants of various worms released each day, are you *sure* that you've been hit with only one variant?

Even if you think you are sure, in fact, you typically can't be sure quickly enough. Well staffed, well funded, and highly experienced labs at the major AntiVirus vendors can't keep up with detailed analysis of the zillions of variants. Neither can the overburdened IT staffs of the world. They need to stop trying.

Disclaimer: As the founder of Intrinsic Security [intrinsicsecurity.com] I am clearly convinced enough in the limitations of the AntiVirus approach that I started a company and developed an alternative (complementary) approach. All of my opinions, well reasoned and otherwise, are my own, although they may be shared by others.

Avoid The Obvious Punctuation Error...

But how do AV researchers dissect such malware, especially when virus writers have devoted so much time to avoiding detection and perfecting their craft with self-decrypting viruses, polymorphic shellcode, and obfuscated loops.

I have a theory that probably 90% of the worms we see are written by the AV companies themselves.

Either that, or they're REALLY DAMN GOOD at getting hold of some fledgling outbreak, no matter how obscure, and reverse engineering it and learning all its minute details. Sometimes they claim to do this within a couple of hours of its first known incident.

I dunno.. maybe i'm a conspiratorist... I still say that Norton Internet Security is the most effective piece of malware out there.

Conspiracies?

I'm an old-schooler, I've read Norton's books like "Inside the IBM PC", when he spoke about bits, bytes, sectors, clusters, etc.

I remember using the famous Norton Utilities for say, defragging my HD or repairing the DOS FAT table.

Norton didn't enter the antivirus business until much later. The de-facto standard for cleaning up viruses was McAffee viruscan for DOS.

And I was shocked at the massive amount of viruses being written... or actually, the amount of viruses that the McAffee antivirus SAID had been written (this was BEFORE the internet as we know it; we used to get the antivirus from BBS's or in issues of computer magazines). I'm talking about 300 or more viruses being written PER MONTH.

The rumour of McAffee hiring virus writers was pretty extended.

Today is very different. Antivirus companies DON'T NEED to hire virus writers (they don't need to create their own market, Microsoft has done them the favor :). With websites dedicated to virii and similar stuff, irc channels, mailing lists, and specially the number of Windows vulnerabilities, it's almost as if virii wrote themselves.

Also, the jerks in the world seem to be multipying. And virus-writing tools are relatively easy to find. All it takes is a script kiddie and a virus writing toolkit. The real geniuses writing virii, are rare. However, all it takes is one original virus for several variants to appear in the next months.

So, conspiracy theories? I don't think so.
A bunch of self-organizing lamers? Very possible. Just look at the wikitorial invasion.

Practice makes perfect

But how do AV researchers dissect such malware, especially when virus writers have devoted so much time to avoiding detection and perfecting their craft with self-decrypting viruses, polymorphic shellcode, and obfuscated loops.

It is important to note that virus authors who have perfected their craft create viruses that are not found.

I have an idea

With this knowledge, someone could start a whole new gang of internet vigilantes. First you write "malware" and sell it to seedy companies, phishers, spammers, and various other jerks. Then it is revealed that the software is actually "beneware" which cleans the target computer of all spyware and viruses and reports to the perps to the authorities.

The only virus I've ever caught

is one I wrote myself..

Question...

What happens when someone finds a loophole and patents some sort of virus and then sues any AV companies that try to reverse engineer it? Yay for our intellectual property dogma!

The funny part is on the Mil side

we figured out how to do all of these back in the mid-80s. Yup, self-decrypting viruses, polymorphic shellcode, and obfuscated loops - all of it.

The amazing thing is that it took so long for people to actually put them into practice.

wormholes

I remember the early 1980s, when a funny little VMS program called "worms" would carve circular holes in text files, like a bookworm would in paper. It didn't selfreplicate or anything, and it destroyed the wormy file. But it was a real laugh when it wasn't your file. FWIW, the purely funny version backed up the original file in its own subdirectory.

bookreview

Book certainly looks interesting, but all the comments so far have been moderatly off-topic.

Anybody actually read the book and can tell me wether it actually is good, or is this is another case of an auther/publisher writing a review for their own book?

Uh huh.

But how do AV researchers dissect such malware, especially when virus writers have devoted so much time to avoiding detection and perfecting their craft with self-decrypting viruses, polymorphic shellcode, and obfuscated loops. sic.

They don't. All they need to do is watch the thing go by on the wire and pick out something that vaguely looks like a unique signature for their dumb as dirt detection engines. And that assumes that such techniques are commonly used, which they're not.

All this analysis and scanning is dumb

Virus scanners/blockers are a scam; they have caused more problems in my network of tech-unsavy users than viruses have by a long way. Just last night my sister's McAffee took her harddrive out back for a thrashing. Another person I know runs 3 firewalls and 2 virus blockers at the same time, computer is practically useless.

There is a better way people! Either boot from a read only media or restore an image of the system every few boots--much faster and practically invulnerable. Put your documents (non executable formats only!) onto removable media and leave them removed except for saving.

This way even if an internet worm exploits a hole in your OS or email, its gone the next boot--even if it is undetectable!

Not only is this more effective and faster, you don't have to pay for pattern updates.

Missing something fundamental

From the review, it sounds like the book's missing something critical.
1. The fact that viruses even exist today is a testament to crappy OS Design. OS X and Linux don't even get viruses. And then if you put crappy application design on top of crappy OS design, you get viruses you don't even have to execute, like Outlook and Word macro viruses.
2. Worms and viruses are totally different. Worms attack you from the outside. But the odds of you getting a worm on a patched, up-to-date system that's behind a firewall is practically nil. (see for example, Apple Software Update.)
In other words, in a "sane" world with decent operating systems and applications, viruses and worms wouldn't even exist.

This book was absolutely terrible.

Once again, what whitehat sources call detailed descriptions of "advanced" techniques really barely makeup a decent FAQ on an infosec subject. The infamous PHC Production "The Real Scriptkiddies" never fails to be relevant: http://seclists.org/lists/fulldisclosure/2002/Aug/ 0482.html [seclists.org] I really wish Slashdot would mark it's article-based advertisements with some sort of flag so that I may filter them out in the same manner I do it's bannerads.

stop the horrible acronyms

what is this? AOL Instant Messenger? Stop With All The Annoying Acronyms For Things That Do Not Need Them (SWATAAFTTDNNT)

The Art of Computer...

The Art of Computer... by Knuth?

Re:How to defend against computer viruses...

Never heard of 'driveby downloads' have you?

Re:How to defend against computer viruses...

Don't download random crap and execute it.

That's easy to say, but harder to do. Any non-trivial program that connects to the Internet is going to download something... that's what makes it useful. And if the program wasn't 100% correctly written, there may be a way to make it execute the thing it downloaded. Voila, all the conditions are there to catch a virus, without the user ever realizing he was "downloading random crap" at all. (For examples, see: every web browser ever written)

Re:How to defend against computer viruses...

easier method - don't let companies profit from "fixing" virus programs so that their isn't an incentive for "helping" bored developers to create new ones.

that would probably cut them in half.

Re:How to defend against computer viruses...

On a Windows machine, you don't need to download anything. Just plug it into the Internet with a publically accessible IP address sometime. I'm not even being paranoid right now.

There are BOOTP attacks, buffer overflows for every type of service, even exploits against the network stack.

On my old company honeynet, we couldn't keep our machines up for more than a week. All recent "SP2 blah blah" patches. Both Windows XP and 2000. We even turned on the Windows "Firewall."

It's not a totally hopeless situation. You definately need a *HARDWARE* firewall with Windows. Relying on your ISP to block ports is unwise. Using Outlook is unwise. Opening Word documents from email is unwise.

I've even gone as far as to remove the VB & J Script engines from my machine. Less components = less to break. Who really even scripts MS Office documents anyway? When you connect your machine to every other person in the world, take some precautions for heavens sake.

Good advice, but not sufficient

It's likely that most home computers at least are infected by worms rather than careless double-clickery. You can buy a computer as a novice home user, and it can get infected before you have time to patch it. Here's an amusing example, just one of many recent stories on the phenomenon: Jacques' Hack Attack [bbcworld.com]

The same is probably also true for most infected corporate computers, even though those are some what better protected.

Re:How to defend against computer viruses...

Even that is not enough sometimes. What happens when the virus comes on the shrink wrapped distro disk from a software company? I've had this happen before and it was a good thing I scanned the disk before running the install.