OpenGL Shading Language 96
OpenGL Shading Language | |
author | Randi J. Rost |
pages | 608 |
publisher | Addison-Wesley Publishing |
rating | 8/10 |
reviewer | Martin Ecker |
ISBN | 0321197895 |
summary | A solid introduction to developing shaders in the OpenGL Shading Language. |
Because of its orange cover, the book is also called the "Orange Book," and together with its siblings, the classic "Red Book" (aka OpenGL Programming Guide) and the "Blue Book" (aka OpenGL Reference Manual (see this earlier review), it is a member of the OpenGL family of books from Addison-Wesley. Although it has a short overview of the basic features of OpenGL, it is intended for an audience that is already somewhat familiar with OpenGL and with 3D graphics programming in general. The interested reader should probably have read the "Red Book" or at least have a good understanding of how to use the OpenGL graphics API before attempting to tackle this book.
Rost, as well as the co-authors on some of the chapters, John M. Kessenich and Barthold Lichtenbelt, all employees of the graphics hardware vendor 3Dlabs, were driving forces behind the inception of the OpenGL Shading Language. They are also core contributors to the final language specification as well as the OpenGL extensions that provide the framework for this new shading language. So the information in the book actually comes from the people that created the language, which is a definite plus.
The book consists of 17 chapters and two appendices which can be roughly categorized into four major parts: An introduction to the basics of OpenGL and GPU programmability; a description of the OpenGL Shading Language and the associated OpenGL extensions; a number of chapters that show the shading language in action; and finally a reference section on the language grammar and the entry points introduced by the new OpenGL extensions. Each chapter of the book has numerous interesting references to get further information on the presented topics. I can only recommend taking a closer look at some of them.
The first two chapters of the book describe the basics of the OpenGL graphics API, followed by an overview of the new programmable processors in the graphics pipeline and an overview of the shading language used to program them. The introductory chapter on OpenGL basics is very well written and worth the read even for more experienced OpenGL programmers. However, as mentioned above, the reader should have enough expertise in using OpenGL to be able to understand the more advanced parts of the book. The introductory chapter won't be enough in my opinion.
The third chapter, written by John Kessenich - one of the main authors of the OpenGL shading language specification - presents the language definition. This chapter is where the basic data types as well as the available control structures are described in detail. For people interested in writing a compiler for the OpenGL Shading Language, Appendix A also contains the entire language grammar in BNF.
Chapter four moves on to describe the programmable graphics pipeline, which was first introduced in the second chapter, in more detail. The programmable vertex and fragment processors and their interaction with OpenGL's fixed functionality are presented. In chapter five, the description of the shading language concludes with the available built-in functions. Chapter six offers the first simple example that shows the shading language in action - a shader to procedurally create a brick texture.
Until this point, the book doesn't talk much about how to integrate shaders into the host program running on the CPU. New OpenGL extensions were created for this purpose, in particular GL_ARB_shader_objects, GL_ARB_vertex_shader, and GL_ARB_fragment_shader. Chapter seven contains detailed descriptions of the entry points provided by these new extensions. Among other things, it describes how shader objects are created, compiled, and then linked to form shader programs that can then be used to render objects. Appendix B also has a reference section on the new entry points similar in style to the "Blue Book." Chapter seven concludes the dry, technical part of the book that introduced both the shading language and the necessary infrastructure to use it from a host program running on the CPU.
The remaining chapters delve into the really interesting topic: shader development. These chapters offer multiple ideas on what can be done with shaders and how to effectively use them in graphics programming. Standard techniques, such as bump mapping, procedurally creating textures, using noise, and others, are presented. Chapter nine deserves special mention because it presents shaders that mimic the behavior of the OpenGL fixed-function pipeline. Many developers new to shader programming are faced with re-implementing certain features offered by OpenGL's fixed functionality. This chapter addresses this.
Chapter fourteen also deserves mention. Shaders that procedurally create textures usually suffer from aliasing artifacts. This chapter shows a number of anti-aliasing techniques to diminish these artifacts. In my opinion, this important topic has not received the attention it deserves -- it's good to see such a chapter in this book.
Closing this section of the book, chapters fifteen and sixteen describe some interesting non-photorealistic shaders and shaders for doing image processing. (For more ideas on what can be done with shaders I also recommend the book "GPU Gems", which I have read and reviewed some time ago.
The final chapter of the book (chapter seventeen) is a language comparison with other high-level shading languages such as the RenderMan Shading Language, SGI's Interactive Shading Language from the OpenGL Shader package, Microsoft's HLSL, and NVIDIA's Cg. Although I am quite familiar with most of these languages, I found this chapter to be an interesting read because it attempts to look at the languages objectively, listing advantages and disadvantages of the various approaches.
The book contains many diagrams and images, all in black and white, except for 16 pages containing 30 color plates in the middle of the book. Most of the images are not overly "flashy" but do give a practical idea of the types of rendered images a particular shader can produce.
There is also a website for the book where you can find an errata list and download a sample chapter (chapter six). As mentioned above, this chapter develops a simple brick shader to show the basic features of the shading language. The website also has all the shaders presented in the book available for download. Because the book does not come with a CD-ROM this is the only means of getting shaders code without having to type them. At the time of this review, the site appeared to be in a transitional state.
Rost's OpenGL Shading Language succeeds at giving a good introduction to shader programming with the OpenGL Shading Language. Not only does it provide the necessary technical instruction to allow the reader to write his/her own shaders as well as integrate them with the host program, it also demonstrates a number of practical applications for shaders and tries to encourage exploring the new dimension of real-time graphics programming opened up by the OpenGL Shading Language. Since there is no other book currently available on this topic, it is hard to say whether the "Orange Book" will stand the test of time and actually become the reference book on the OpenGL Shading Language, but I believe it will.
Ecker has been involved in real-time graphics programming for more than 9 years and works as a arcade game developer. He also works on a graphics-related open source project called XEngine. You can purchase OpenGL Shading Language from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, carefully read the book review guidelines, then visit the submission page.
The main problem with GLSL (Score:5, Insightful)
The core competencies of graphics IHVs are generally not in compiler writing -- writing a good optimizing compiler is still a "black art" significantly more difficult than writing a hardware driver, and it will be very annoying if compiler bugs show up on some vendors' drivers and some don't, forcing developers to work around them -- or different compilers optimize things differently.
At least with DirectX, there's guaranteed to exist one common compiler that's written by a company with years of experience in optimizing compilers.
Of course, the philosophy of OpenGL is counter to DirectX in that there's no one Big Company controlling it all, but
at the very least, there needs to be some standard token bytecode defined and standarized by the ARB, and a reference compiler design, as well as a compliance suite to verify compiler correctness and language compliance.
Re:The main problem with GLSL (Score:2)
They are two different things from Microsoft.
Re:The main problem with GLSL (Score:1)
Re:The main problem with GLSL (Score:1)
Re:The main problem with GLSL (Score:2)
Re:The main problem with GLSL (Score:2, Interesting)
Re:The main problem with GLSL (Score:5, Interesting)
For one thing, GLSL is an incredibly easy language to compile compared to C++ or C#. That aside, 3Dlabs has written the source code to the GLSL compiler and released that source code for anyone to use, including competitors like ATI or NVIDIA.
But my main disagreement is with the idea of having an ARB-specified intermediate language. I already described my position in another post, so I won't repeat it all here.. but I think your position is short-sighted. I think allowing the IHVs to do the full compile/optimization stuff will provide them with greater flexibility and greater potential for innovation.
Re:The main problem with GLSL (Score:5, Insightful)
Exactly.
I haven't worked with GLSL et al, though I have done a lot of work with RenderMan shading language compilers, including writing two of them from scratch. I definitely agree that these shading languages are much easier to compiler compared with C++ or C. (The lack of user-defined types and recursion helps a LOT --- no fancy stack frames.) Similarly, the target platforms (usually some kind of virtual machine in the case of RenderMan) are invariably quite simple.
However, the problem is that even though the target platforms are simple, they're all extremely different. Most of them are SIMD architectures, but some are not. Some use stacks and others use register transfer. One or two (most notably Pixar's PRMan) execute annotated abstract syntax trees (called "shade trees") directly. Many compile to C++, but even then, they often have very different capabilities and limitations.
Take, for example, the following piece of code, where all of the variables are "varying" (that is, they vary over the grid being shaded):
If the target machine supports SIMD addition, then it pays to use it. So we might use a temporary, and generate code like this, scheduling the two additions separately:
If, however, you're compiling to C/C++, then you're better off scheduling the additions together:
This increases the effective size of each basic block and hence gives the C/C++ compiler a better chance of performing low-level optimisations.
And this is just showing the difference between two software VMs. I can only imagine what the differences are between the capabilities between two generations of NVIDIA hardware, let alone between NVIDIA and ATI. Leaving optimisation up to the IHV was definitely the right thing to do.
Re:The main problem with GLSL (Score:1)
w = x;
w += y;
w += z;
??
Re:The main problem with GLSL (Score:2)
Re:The main problem with GLSL (Score:2)
There is no "the" preferred code. Your code may be better on some platforms, but on others it may not be.
However, you do raise a good point. A coalescing register allocator, particularly one that was sensitive to target paths, could easily optimise the temporary version to:
It's one fewer temporaries, but the same number of operations.
Re:The main problem with GLSL (Score:5, Interesting)
The big advantage of passing the high level code straight to the driver is that it allows for easier optimizations, simply because the conversion from high level -> assembly/bytecode inevitebly loses some of the context.
For instance (I don't know HLSL, but I have used Cg, which I'm told is almost identical to HLSL), the smoothstep() function provides a nice, smooth interpolation between two values. When I compile my Cg shader down to arb_vp (the assembly language), smoothstep has to be expanded to perform all the necessary arithemtic instructions sequentially, because there is no comparible instruction.
If the high level code were passed directly on to the driver, the driver could determine if for instance, this was a new piece of hardware that didn't exist when the game was made, and this hardware has dedicated smoothstep functionality, allowing the operation to be performed much faster than sequential instructions. The same information could be recovered from the bytecode, but it would be much more difficult.
The same general principle holds in other languages. Higher level doesn't necessarily mean slower or more difficult compilers. Look at Apple's extensions to C to allow easy utilization of the PowerPC's vector processor. A little hint from the programmer can make the job of writing an optimizing compiler much easier.
Re:The main problem with GLSL (Score:1, Interesting)
I disagree (Score:3, Insightful)
But for what it's worth, the parent raises some interesting points, although I disagree that a bytecode/pseudo asm route is a better method.
In reality, the ASM-like bytecode used probably bears little resemblance to the machine code instruction sets actually used by the various cards. Forcing vendors to use such a low level instruction set allows very little room for optimisation, whereas allowing vendors to write their own c
Re:The main problem with GLSL (Score:2, Interesting)
The fact is, however, that the language spec is down, it has already undergone revision, and the compiler is open source. Support for the language will undoubtably grow with time. OpenGL's core strength is its stability.
One of the main reasons, so I understand, for the lang
I see that as its main strength (Score:2)
If you remember figures starting coming out about very poor performance with nVidia hardware and Half-Life 2 a while back. If HL2 had been OpenGL, nVidia could have tweaked their driver to workaround the suboptimal way Valve was driving their hardware, presumably to the benefit of the frame rate. ATI could have done the same thing.
Re:The main problem with GLSL (Score:1)
I agree with the other posters that the GLSL way could help with optimizations. However, given the problems I've had with ATI's drivers, I'm more than a little worried about the quality of ATI's shader compilation.
Just out of curiosity, does anybody think that the extra effort to do a driver with GLSL support will affect how often we'll see drivers for platforms other than Windows? I'm thinking back to when it was hard to
Re:The main problem with GLSL (Score:2)
Re:The main problem with GLSL (Score:2)
So I think in the end this solution is superior, provided the card manufacturers use the open-source front e
Re:The main problem with GLSL (Score:1, Funny)
-- writing a good optimizing compiler is still a "black art" significantly more difficult than writing a hardware driver...
You have obviously never written a hardware driver for Windows 2K/XP, or you wouldnt say that ;-)
issue: shader copyrights (Score:2)
Since the app sends the shader source code to the driver, it would be easy to see all the shaders an app is using. I remember Randi saying that this is one of the issues they are having to deal with. Microsoft doesn't have this problem because the shaders are compiled before shipping.
while not an issue for me, it will be for many companies planning on competitively deploying shaders in their apps.
-metric
Re:issue: shader copyrights (Score:2)
2) it should be easy to spot rip-offs, then.
The basic idea can still be stolen, so yours was a very insightful observation.
Re:The main problem with GLSL (Score:2)
If so, you still can't guarantee IBF code will compile to "the same result" on a different hardware implementations. ( Just ask the Java guys about this problem
Granted there's less room for bad compiler design, but its there. What do you think?
What are people's opinion of comparisions (Score:5, Informative)
I spoke to a friend who is a developer for visualization applications. He feels that with Open GL 1.5 and some of the new ratified standards from the OpenGL ARB, OpenGL is back on par with Direct X 9.0.
Anyone have other insights ?
Re:What are people's opinion of comparisions (Score:5, Interesting)
It will take a while for drivers supporting OpenGL's SL to be widely distributed, and existing artist-centric shader design tools to be updated for the new standard, but things seem to be moving very quickly.
There are still a few areas where I wish the OpenGL ARB would follow DirectX's lead -- a library like D3DX and a generic mesh format similar to DirectX X files would be really nice (there are various Open Source projects that fit this bill, but AFAIK, none of them are official or have any sort of critical mass).
Of course, a developer can easily create his/her own code that does similar things to D3DX using OpenGL, but having such a functional officially-supported utility library really helps when you're prototyping and just want to get things up and running quickly and also helps if you're just beginning with 3D programming, so you aren't completely overwhemled by everything.
Re:What are people's opinion of comparisions (Score:2)
Re:What are people's opinion of comparisions (Score:2)
Re:What are people's opinion of comparisions (Score:2)
There are loads of libraries out there that you can use for those things. It's also not too hard to roll your own - you'll also learn a hell of a lot more in the process.
The beauty of OpenGL is that it sticks to doing one thing and doing it well. The choice of not having an intermediate bytecode for GLSlang is a good one too... it allows compilers to directly optimise the shader program for the archit
Re:What are people's opinion of comparisions (Score:2)
(there are various Open Source projects that fit this bill, but AFAIK, none of them are official or have any sort of critical mass).
I'll take this opportunity to promote my favorite open source 3d project: OGRE [ogre3d.org]. It's really a very complete rendering system with a great set of plugins for Maya and 3dStudioMax to work with it's mesh format. The tutorials and documentation are also very well written. It, of course, does not do everything, but it does a lot of what I need it to do. It was written around
Re:What are people's opinion of comparisions (Score:1)
Re:What are people's opinion of comparisions (Score:3, Insightful)
Re:What are people's opinion of comparisions (Score:3, Informative)
You can do that with DX9 too, using HLSL and the 'FX' effects file system.
This is a very good step in the right direction, I think. Also, HLSL specifically compiles its code into a DirectX-specified shading assembly. Microsof
Re:What are people's opinion of comparisions (Score:5, Insightful)
Re:What are people's opinion of comparisions (Score:1)
Actually, no meaning is lost. Have you ever actually done shader programming? Even with PS3.0, which adds support for looping and branching logic, the higher level language is really
Re:What are people's opinion of comparisions (Score:2)
But there is no longer a good reason to have this intermediate assembly. It only makes it more difficult for the IHVs to optimize their drivers.
Consider a hypothetical high-level shader function foo() that compiles into, say, six instructions in SM2.0 four instructions in SM3.0. For the driver to try to do some crazy-cool
Re:What are people's opinion of comparisions (Score:2, Funny)
Except in the area of the existance of real implementations.
Re:What are people's opinion of comparisions (Score:1)
True enough, and the forthcoming DX9 Summer Update 2004 nudges DX9 a bit further yet. While there are certainly some benefits to having a committee like the OpenGL ARB, there are some drawbacks (like slower forward movement) too.
Re:What are people's opinion of comparisions (Score:4, Informative)
RenderMonkey is not a language, but an IDE for several shading languages, such as HLSL, Cg and GLSL.
Graphics? (Score:2, Insightful)
Re:Graphics? (Score:1)
FINALLY! (Score:1, Offtopic)
Re:FINALLY! (Score:3, Funny)
Yeah but nobody'll call it 'News for Studs'.
Re:FINALLY! (Score:3, Interesting)
Re:FINALLY! (Score:2)
Useful book (Score:5, Informative)
Somewhat familiar? I hope so! (Score:5, Interesting)
Although it has a short overview of the basic features of OpenGL, it is intended for an audience that is already somewhat familiar with OpenGL and with 3D graphics programming in general
Boy, I hope someone looking to learn OpenGL at least knows enough about 3D graphics to realize this isn't likely the right book to start learning OpenGL with. That said, the Red and Blue books are indispensable. I certainly hope this "Orange" book lives up to the family reputation.
Anyone know if Doom 3 uses these newly ratified extension? I would imagine it does - but I of course have no facts to back this up.
Re:Somewhat familiar? I hope so! (Score:5, Funny)
I had to read that TWICE and I'm still in shock.
A
A sign of the apocolypse if I ever saw one.
Congrats Goyuix, you may now add "Harbinger of the Apocolypse" to your sig.
Re:Somewhat familiar? I hope so! (Score:2)
-- --- ---
You know the world is going crazy when the best rapper is a white guy, the best golfer is a black guy, the tallest guy in the NBA is Chinese, the Swiss hold the America's Cup, France is accusing the U.S. of arrogance, Germany doesn't want to go to war, and the three most powerful men in America are named 'Bush', 'Dick', and 'Colon'. Need I say more?"
-Chris Rock
--- --- ---
Somewhere in there, the Slas
Re:Somewhat familiar? I hope so! (Score:2, Informative)
That said, I do remember JohnC saying at some point that he is committed to doing a codepath for this, so it's extremely likely that it will appear in a future patch. I am very excited to see how it performs.
You take the Red Book... (Score:3, Interesting)
Yes, it's that good.
Re:Somewhat familiar? I hope so! (Score:2)
Yes [webdog.org]
Re:Somewhat familiar? I hope so! (Score:1, Insightful)
Apple seminar on GLSL at WWDC (Score:3, Interesting)
Re:Apple seminar on GLSL at WWDC (Score:1)
Re:Apple seminar on GLSL at WWDC (Score:2, Interesting)
One interesting aspect of it is that it appears to be designed not only for processing images for output to screen (ie realtime visual effects), but also for use elsewhere (ie application of video / photo filters, output to RAM / disk). Nice to see someone thinking of all that mostly unused GPU grunt, and putting it to use.
Judging by the list of cards that support it (on a PC they'd be called "DX9 c
Re:Apple seminar on GLSL at WWDC (Score:3, Interesting)
Re:Apple seminar on GLSL at WWDC (Score:1)
Re:Apple seminar on GLSL at WWDC (Score:5, Informative)
Apple has a great tool called OpenGL Shader Builder that comes with the developer tools on Panther. I don't remember how long Shader Builder has been around but I know it has been at least a couple of years. It even existed before the standard shader language existed, for writing to NVidia and ATI's individual vendor-specific shader languages before there was a standard. Apple made Shader Builder extensible so they could just add the standard language as an additional shader language to the tool.
Shader Builder is cool - you can edit your shader program directly as shader code mneumonics or whatever, run it and debug it interactively, and if I remember right it even generates the C code with the appropriate OpenGL calls to create and load the shader program for you. You just copy and paste right into your C file.
I don't know of anything else like it on any other platform; there may be an equivalent tool on windows but I have never seen it.
Re:Apple seminar on GLSL at WWDC (Score:2)
I should mention that I never wrote a line of opengl before last august, and all that I know came from the Red Book, which doesn't cover shading languages.
So, I'm curious, what shading language is Shader Builder using? Is it a custom language? Is it HLSL? What kind of standards does it follow?
And
Re:Apple seminar on GLSL at WWDC (Score:2, Informative)
Re:Apple seminar on GLSL at WWDC (Score:1)
PDF available at OGL website (Score:5, Informative)
Save youselves a bob or two.
-S
Re:PDF available at OGL website (Score:2)
Re:PDF available at OGL website (Score:5, Funny)
Orange Book (Score:1)
Thank Carmack... (Score:5, Interesting)
Why do I care about this? Because M$ only produces/copies acceptable technologies when heavy pressure is applied to them or if they are playing catch-up. Look at IE for example. I speculate that DirectX would have dominated as the graphics API but would have followed a similar fate to IE if it hadn't been for OpenGL racing along with decent drivers all because of this thing called Quake.
Thinking short-term as a developer, I would have loved if the world had one standard, even for shaders, but I take off my hat to Darwin and accept a little hardship implementing for two or more APIs to let them evolve each other.
Re:Thank Carmack... (Score:3, Funny)
Re:Thank Carmack... (Score:2)
MS still would have invented DirectX because it's something they can controll. They controll the most prevelant development environment for the PC, Visual Studio, and they can use that control to influence developers to use their stuff rather than an open standard or other SDK.
Overview of Shading (Score:5, Insightful)
Here is brief overview of shading, in case you don't know.
Shading is the process by which a renderer assigns color values to pixels on the screen. There are currently three popular rendering methods: Rasterization, REYES, and Ray-tracing. Rasterization simply projects planar polygons onto a 2D plane and discreetizes them to pixels. REYES is slightly more complicated in that it takes mathmatically defined patches, slices and dices them into micro-polygons which are then rasterized. Ray-tracing point-samples the scene by tracing rays through it.
Rasterization is the method of rendering that is implemented on your commodity graphics accelerator. It is commonly considered the most adept at handling real-time graphics. In the past, the type of shading that you could perform in real-time was rather limited. What people did was take a dial and switch approach to shading. Users would tell their graphics API (like OpenGL) what features they wanted to use and pass in parameters. That was basically it. Unfortunately, the more bells and whistles you add to the rendering pipeline, the more unweildly your graphics API becomes. This is because, users deal with the most shading algorithm possible, no matter how simple the task.
Hence, the programmable pipeline is born. Instead, of adjusting dials and switches, you just write little procedures telling the renderer exactly what you want it to do at a certain stage of rendering pipeline. There is some history behind this approach coming from REYES renderer. Renderman is the shading language that tells Pixar's REYES renderer what to do. It is popular, and it is pretty standard in the graphics industry. People have even implemented Ray-tracers capable of using the same shaders that were used for the REYES renderer. But, in all this is pretty inefficent. Shaders are intimately tied to the rendering method that you are using. Therefore, a new shading language needed to be developed.
In all, the shading languages will look like Renderman, which in turn looks like C. Real-time shading languages differ in their specifics. But, in rasterization there are two procedures you can write. A vertex shader which allows the procedure to manipulate points of your planar polygon in some fashion, and a fragment shader which is invoked after the discreetization to actually color the pixels. Cg and HLSL are actually the same shading language. OpenGL Shading language (glslang) is an alternative. Probably the biggest difference at this point is HLSL/Cg dictates a particular instruction set architecture on the hardware. GLslang doesn't, but it requires that your graphics driver has a compiler that compiles the shading language to something the graphics card can use.
In all, I think the money is on HLSL/Cg to win. It has been out considerably longer, and I think it has already picked up developer mind-share. I also think that it makes things considerably easier for graphic driver writers although it might be more limiting. However, we probably won't feel the pains of it being limiting in the next 5 years, and by then the battle will be over.
As for those that mention rendermonkey. Rendermonkey isn't a shading language, it is a suite of tools that help you produce shaders. It uses HLSL/Cg or a general graphics assembly language underneath. It is pretty independent of the whole shading language war.
That said, this book might be good to pick up independent of the actual language that it discusses just because it goes over important issues that you face when writing these shaders.
Re:Overview of Shading (Score:2, Interesting)
I disagree. The RenderMan shading language is not at all tied to the rendering
Re:Overview of Shading (Score:2, Interesting)
Inefficencies: surface shader, you have to break it down to a vertex shader and pixel shader to make it map well on to modern hardware. Displacement shaders: you can't do it the same way using rasterization as you do with REYES. In fact, you can do displacement mapping using the surface shader if you are doing REYES. You need to separate it out if you are doing ray-tracing. This is just one way that the rendering system leaks into shading. Light shaders, there is no equivalent mapping to current hardwa
Re:Overview of Shading (Score:1)
Re:My Money is on GLSL (Score:2, Interesting)
Probably the most eagerly awaited feature for OpenGL developers, everything in the OpenGL state is directly accessible using built-in variables in the shader and need not passed explicitly from the program before executing the sha
Re:Overview of Shading (Score:4, Informative)
REYES is a method, PRMan is a program (Score:2, Informative)
Re:REYES is a method, PRMan is a program (Score:1)
Write this up in wikipedia (Score:2, Informative)
Thanks!