Collada

Tony Parisi writes "Remi Arnaud and Mark C. Barnes' Collada: Sailing the Gulf of 3D Digital Content Creation is a great first book on a new technology for entertainment applications. Collada is a file format for the exchange of 3D game and interactive content, developed by a consortium of companies including Sony and leading digital content creation software vendors. Collada acts as an intermediate format between DCC tools and end applications — thereby promising to solve ages-old productivity issues in the 3D content pipeline. The authors have done a thorough job explaining this problem, and how Collada solves it, in an introductory book that is well suited for a technical audience building advanced 3D tools and applications." Read the rest of Tony's review.

Collada: Sailing the Gulf of 3D Digital Content Creation
author	Remi Arnaud, Mark C. Barnes
pages	240
publisher	A K Peters
rating	9
reviewer	Tony Parisi
ISBN	1568812876
summary	Describes Collada, a new format standard for 3D graphics content

When I first heard about Collada in 2004, I was skeptical. I am no stranger to 3D file formats: I co-created VRML in 1994 and continue to push that rock up the hill with X3D, an XML-encoded 3D format for the web. After a decade of getting arrows in my back blazing this trail, it felt like the last thing the world needed was Yet Another 3D File Format. Developers in the game world, and to a lesser extent, the DCC tools world, had largely turned their backs on standards of any kind — the proverbial tough crowd. However, the laser-focus of the Collada design team seems to have paid off: the industry now has a common exchange format for game assets, and there is hope yet for developers who would rather spend their time building games than building tools to make games. This clarity of focus on the design comes through in the book as well. Arnaud and Barnes, principal designers of the Collada standard, have written a detailed and lucid book that explains Collada from concept through implementation.

Any first book on a new technology has the unenviable task of explaining the *why* of it, as well as the what and the how. The big risk there lies in devoting too much of the book's content to the background and overview materials before getting to the meat. The book excels in this regard, by clearly laying out a compelling case for an intermediate format, but quickly, so that experienced engineers and managers can get the point, and then move on to the useful information that comprises the bulk of the material. I suspect that Collada will serve as the seminal text on the subject for some time to come, and therefore the overview material is essential. However, if this had only been a survey or introductory book it would have missed the mark with its target audience — some of the busiest and results-oriented developers on the planet, people for whom what-is-it and how-do-I are where the rubber hits the road.

Most of Collada is devoted to explaining the format's core concepts: geometry, scenes, animations, and physics. These are the building blocks of all interactive graphics content. The book walks through these in a straightforward progression; I found it very easy to follow. After covering the basics, it combines them into useful end-to-end examples that give prospective implementers a sense of what it would take to build Collada tools. It worked for me anyway: we will be building Collada support into our products based what I learned in here.

Midway through, the book goes into a bit of a jag on the subject of effects. In its simplest form, the concept of effects simply refers to an object's basic visual appearance: color, textures, shading etc. Unfortunately, in modern graphics, nothing is ever that simple. What begins as a discussion about those basic concepts quickly cascades into a deep-dive on programmable shaders, effects-packaging techniques, multi-pass rendering, and profiles — the latter presenting a glimpse into the sausage-making stuff of standards and hardware consortia, things mere mortals were not meant to know. This is not really the fault of the chapter's author (contributor Daniel Horowitz from NVIDIA), but more a side-effect of the intensity of the subject matter. I am wondering if this all could have been handled better in a separate book or an appendix. The saving grace here is that it's at this exact point in the current printing of the book that we get the color plates — 8 gorgeous images courtesy of the biggest names in graphics: Epic, NVIDIA, Alias and the like. But that's the beauty of 3d graphics: when the math gets hard... go to demo!

Collada gets back on track with 2 excellent chapters on animation and physics — again, at just the right level for a seasoned 3D toolmaker like me. The final chapter explores the Collada content pipeline: how different Collada-aware tools can be mixed and organized to create the final content from all the original assets. Here the orientation of the book moves from a focus on specific features to a system view of how to assemble disparate software tools into working production solutions. Managers and decision makers reading the book will benefit most from this chapter, which I found to be not only thoughtful but provocative, as it touches on the issues at the heart of Collada's mission: how to make robust, open, cost-effective systems for creating rich content that lasts the test of time. The book also contains a handful of informative appendices on Collada plug-ins for the major DCC tools such as 3ds Max and Maya. The plug-in appendices cover build steps, supported features, practical use scenarios, and known bugs and issues.

Other than that effects digression, my only critique — a minor one — is that it is a bit heavy on the XML. From the second chapter, the book dives into details of XML syntax, document types, schema and such, and never looks back. I understand where the authors are coming from: rather than invent a pseudo-code to illustrate the concepts (which then would have to be translated into XML for the working examples anyway), it's more economical to just get the reader up the ramp with XML and get on with it. I am willing to believe that graphics-savvy readers who are not conversant in XML — and I am betting there are still many, many such 3D geeks out there — will be able to get over this quickly. You should: XML is *the* way to store and transmit data, period, full stop. Just in case you are naïve about XML, the authors have actually put in a short primer on it in the second chapter. It's good, and it should be enough to get you through.

I highly recommend this book. If you are a programmer with a reasonable understanding of 3D graphics programming and/or authoring using 3D scene graphs, you will be able to learn the basics of Collada. If you are an experienced 3D toolmaker, Collada contains everything you need to get up and running to build your game, game engine or content pipeline tool. If you are a manager or decision maker, you get the added comfort of knowing that — finally — you just might be able to write that tool once and not over and over again.

Tony Parisi is a software architect and entrepreneur who loves to solve hard problems. Tony's company, Media Machines, is building an open source, open standards-based platform for delivering 3D virtual worlds on the web. Tony is the co-creator of VRML, the original standard for 3D on the web, and is a principal developer of X3D, the XML-based successor to VRML for building web-based 3D virtual worlds.

---

You can purchase Collada: Sailing the Gulf of 3D Digital Content Creation from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

XML

XML is *the* way to store and transmit data, period, full stop.

Sure, it's a good way, but could we please have an end to silly and blatantly false assertions like this one?

One main contributor not mentioned...

is Feeling Software http://www.feelingsoftware.com/ [feelingsoftware.com]. I'm using Collada on a daily basis, and the company makes some of the best Collada tools, importers and exporters and are used by many DCC users. They're either freeware or open-source and downloadble on their website, so heads up for any interested developer out there. They were also involved in the writing of that book and most of the color plates the author mentions are actually theirs.

Collada

as in Piña? Sounds like a good idea.

Mentioning that you were involved with VRML...

...does not lend credibility to your review. Does collada support bones/skinning information or does it just support an extension mechanism (disaster)? Are there exporters/importers available currently? Now, I LOVE XML, but I would never suggest that XML is the best way to go for data 'full stop.' Such statements show inexperience in software engineering, especially given that his speciality is supposed to be 3D over the web LOL. Maybe he was referring to compressed and/or encrypted XML. Probably not, not if he had anything to do with that disgusting blob of a file format called VRML. Maybe I should say what I really think...? ;)

Re:Mentioning that you were involved with VRML...

"People much much more involved in the 3D industry will avoid this format on the simple grounds that it is a generic solution to a problem best solved by specific art pipelines."

Collada is rapidly being adopted because it works great as an open exchange format between 3D content creation tools. Maya, 3DS Max, XSI, Lightwave, Houdini, Blender, all have Collada support - indeed good Collada support is a major selling point of the latest release of XSI.

Since Collada is the native format supported for Sony on the PS3 everyone is also motivated to support it well.

LetterRip

If Collada is ....

If Collada is a file format for the "exchange of 3D game and interactive content", what's the file format for a descriptive submission title?

Now they publish

While Collada is a way neat concept, the specification/documentation sucks big time.
Having implemented a parser/importer in Python from scratch I can attest to a huge amount of ambiguity in the 1.4 spec.
That said, once you figure it out COLLADA is really sweet.

COLLADA Physics

In case you are interested: Blender 2.42 and later supports basic COLLADA 1.4 export/import including physics information. Also CreateDynamics and ColladaMaya supports such physics information. The open source Bullet physics engine uses COLLADA-DOM and libxml to import/export such physics files. A less bloated approach is using TinyXML, that's the approach of CreateDynamics tool. Erwin

Collada Academic, not only games

Beside big players, like Maya, 3dsmax and Blender, I would like to remark that Collada is gaining a momentum even in the academic environment. MeshLab ( http://meshlab.sourceforge.net/ [sourceforge.net] ) , for example, is a Collada enabled open source tool aimed to the processing of large 3D scanning meshes for Cultural Heritage applications, like documentation, reproduction, restoration planning and didactical uses of 3D technologies.

XML is the best way to store data?

I worked on a game that was created using a middleware package that used XML as a file format for level layout. As we got into full production, and the levels got populated, it would often take 20-30 minutes to load a level into the editor, most of which time was spent opening a ton of XML files and parsing strings.

Another company I worked for had a proprietary geometry/animation format that was ASCII based and very verbose. The tools to process this data were also very slow, probably at a minimum 10x slower than they needed to be. Most time was spent, yes, loading and parsing ASCII.

Besides this, your file sizes will balloon amazingly, as XML/ASCII doesn't have a very good information density when compared with binary data, which doesn't care about formatting and keywords. This becomes very important when you have game levels with millions of vertices in them.

ASCII format data is great for human readability, but if you want quick iterative development on 3D assets, it is NOT the way to go for production tools. String parsing is very expensive for a variety of reasons. I know you can speed it up with hashing and tricks, but still it's never going to be as fast as a binary format, especially not one that can be loaded into memory and ready to work on with just a few pointer fixups. If you want human readability for debugging, have your tools generate some good log files when a verbose flag is passed in.

COLLADA

I worked on the specification a little and have advocated use of the standard at several development studios I've worked at (primarily for PS2, PSP, and PS3 titles but it would help for simultaneous cross-platform projects too).

I've seen lots of complaints that the format is not necessarily any better than Microsoft's .X format because most DCC tools can import either one just fine. This misses the *exchange* emphasis of COLLADA (i.e., that it can be successfully *exported* by each tool too!).

Of course XML necessarily has overhead and markup that would be assumed and invisible in any reasonable binary format but the benefits have seemed to outweigh the downsides. Just about every programming and scripting language available today has mature XML parsers which means that it can be increasingly easy to write conditioners as part of any asset pipeline in the future. Conditioners might pre-process surface normals or tessellate curves or tri-strip meshes according to particular constraints in preparation for certain platforms. Such tools can make for an incredibly modular pipeline comprised of highly specialized operations in a similar fashion to the Unix Philosophy of piping and filtering standard I/O.

The specification was designed to encompass the major needs of professional game developers and to leave plenty of room for extension both in future iterations of the standard itself and within each instance document according to each project's needs.

Some trade-offs have of course been made. Most DCC-specific construction histories were not represented last time I looked and efficiency can always improve... but the project had many ambitious goals and I think it has accomplished a noteworthy degree of success. The HTTP://Khronos.Org Group has adopted the standard (alongside OpenGL|ES and others). Sony has been promoting the standard through the PlayStation Developer's Network with each SDK and had sessions at their PS3 Developer's Day gathering early this year to introduce lots of "in the trenches" game programmers to the technology. I think increasing numbers of game studios will be adopting COLLADA for at least some portion of their art asset pipeline since the momentum of increasingly interoperable construction, viewing, and tuning tools is picking up.

Of course the standard can be useful to "amateur" game developers too... and we're likely to see repositories of both free and commercial art resources grow around the standard to better facilitate reuse and outsourcing of that work. It can even streamline parametric asset construction in the future.

COLLADA is an important standard for game development today and is the best candidate to become a foundational piece of any future truly Stephensonian Metaverse. Even HTTP://SecondLife.Com would be radically different if it supported import and export of COLLADA data. Arnaud and Barnes are brilliant guys who've done a great job on the whole project and their book is probably going to be an invaluable resource to anyone who cares about the future of entertainment and communication.

I'm admittedly biased though.

Sincerely,
-Pip

How will this affect X3D?

Hi Tony (if you're still reading comments), I think we met at the X3D Symposium last April.

How do you think Collada will affect people's interest and involvement in X3D? It's evident that VRML/X3D has failed to gain traction in the 3D entertainment sector, and that Collada's focus here means that it will probably (hopefully) be much more successful. Also, from a gut-reaction level, Collada is much cleaner and doesn't have some of the silly baggage that you find in X3D (such as delimiting lists with "-1" and storing list data in xml attributes) that probaly turns off some programmers. VRML/X3D has partially filled the gap as a common interchange format between modeling programs, but it doesn't work beyond static models (vertices/polygons/texture coordinates) If Collada can be an effective interchange format for animation, physics properties, etc, that's going to be a big win all around -- except maybe for X3D.

Thanks

Thanks for reminding me of the collada project, I'd forgotten about it and it's good to see how well it's coming on. It should be easy to implement a reader and writer into my own software and it's really refreshing to see a human readable format that is being developed by and for the game authoring community: the VRML effort always seemed out of touch as far as what was going on in 3D games, perhaps through ignorance, snobbery or maybe just that 3D in games were quite immature back then.
It's also refreshing to see such sensible focus: it's not being billed as a new XML format that's going to revolutionize 3D (3D!) on the web and allow us all to live in gibsonesque cyber fantasies of the future, it's a being billed much more soberly as a standard medium of data exchange between 3D apps, something that's sorely needed (as anyone who's tried to write tens of 3D file import/export filters and had to make compromises with each and every one will tell you).
One thing this is not is a web format. 3D files, expressed as XML (or other HR formats) that specify models by vertices and polygons (or tri-strips/fans/edge-breaker-if-you're-lucky) are always going to be unwieldy. It's a bit like HTML specifying images pixel by pixel by using tags: e.t.c. In other words, a bit verbose for most non-trivial cases and in any case unreadable.
Tim.

COLLADA Review Should Have Been Posted In Games./.

I know this is a book review so HTTP://Books.SlashDot.Org makes sense but this standard is going to be most relevant to more people who are consistently reading HTTP://Games.SlashDot.Org than those frequenting any other section of SlashDot. Oh well.

-Pip

no

Silly lameness filter requires me to type something...

maybe

added for completeness