3D User Interfaces

Martin Ecker writes "Two-dimensional user interfaces (UIs) have been around for a long time, and people are accustomed to using them. However, 3-dimensional user interfaces have not yet received as much exposure. Only a select few applications utilize 3D UIs, some with more success than others. The book 3D User Interfaces: Theory and Practice (published by Addison-Wesley) attempts to give a broad overview of the field of 3D user interfaces. It discusses the hardware devices and the software techniques required to build successful 3D UIs and gives a number of design guidelines to follow when having to develop new 3D interaction techniques for specific applications." Read on for the rest of Ecker's review.

3D User Interfaces: Theory and Practice
author	Doug A. Bowman, Ernst Kruijff, Joseph J. LaViola Jr., Ivan Poupyrev
pages	478
publisher	Addison-Wesley Publishing
rating	7/10
reviewer	Martin Ecker
ISBN	0201758679
summary	An extensive overview of 3D input and output devices, 3D interaction techniques, and 3D user interfaces.

The book contains 13 chapters, divided into five parts. The first part contains two short chapters that introduce the basic concepts of 3D user interfaces, give a bit of history of 3D UIs, and define the scope of the book.

The second part discusses hardware input and output devices that are useful when developing 3D user interfaces. The first chapter in this part is on output devices and it presents various visual and auditory displays. Haptic devices are also discussed in this chapter. The following chapter presents 2D and 3D input devices that can be used with 3D user interfaces. The devices discussed include not only the classics, such as 2D mice, keyboards, and joysticks, but also 3D mice, tracking devices, and various forms of direct human input, such as via speech or via bioelectric signals.

The third and largest part of the book is on 3D interaction techniques. The first chapter of this part discusses the various ways that have been devised in the past to perform 3D selection and manipulation of objects. A vast number of techniques are presented in this chapter, from various pointing and virtual hand techniques to widgets for rotating an object. The following chapters discuss techniques to allow navigation through virtual worlds and user interfaces, in particular techniques for traveling and pathfinding. The following chapter is on system control and it discusses how to control the system via commands, such as using graphical menus, voice and gestural commands, or real-world tools. Finally, this part of the book contains a chapter on symbolic input, i.e. communicating text or numbers to the system, in the context of 3D UIs.

Part four of the book deals with designing and developing 3D user interfaces. For me, this was the most interesting part of the book because it shows how to put together the various input/output devices and interaction techniques presented in the previous chapters. This part also contains a chapter on evaluation of the design and implementation of user interfaces, an important aspect in order to ensure the usability of a user interface.

In the book's final section, the author takes a look at the future of 3D user interfaces with a focus on the combination of the virtual world with the real world -- so-called augmented or mixed reality. This area has received quite a bit of attention from academic research in recent years.

Throughout the book, there are useful guidelines on designing usable user interfaces. Following these guidelines will probably not give you a perfect 3D user interface, but it will definitely help you avoid the common mistakes and pitfalls. It would have been nice if all the guidelines in the book had been put all together in a separate appendix in addition to having them spread out all over the book.

The book also has a number of images and illustrations. The figures throughout the book are in black and white, apart from a four-page color insert that depicts various hardware input and output devices.

This book contains a lot of information and is probably the most comprehensive book on 3D user interfaces I have seen to date. Pretty much every aspect of 3D UIs is covered in the book somewhere, with some topics being covered in more detail than others. If you're not familiar with 3D UIs at all, this book gives you an excellent introduction to this active field of research. If you are already somewhat familiar with the topic, this book offers you a comprehensive overview of the field and gives you many references to more detailed research articles and papers.

Martin Ecker has been involved in real-time graphics programming for more than 9 years and works as a games developer for arcade games. In his rare spare time he works on a graphics-related open source project called XEngine.

---

You can purchase 3D User Interfaces: Theory and Practice from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

This is UNIX

I know this!

Re:This is UNIX

It's called FSN, and it's right here [sgi.com]

Re:Look

Chances are that you run linux so download it from sourceforge [sourceforge.net] originally it was developed on SGI and people posted links to that, the sourceforge version is probably more up to date. Enjoy, I tried it but didn't find it that useful. Reason is that the display is flat, i.e. 2D and twisting and turning in 3D using odd combinations of 3 mouse buttons and 3 keys (Ctrl,Alt,Shift ) just to open a folder is somehow too much work, a simple command line or double click works better.

3d interfaces

My UI is already 3D (the third dimension being time).

Re:3d interfaces

I may be wrong but I thought Time was the Forth dimension.

A dimension based on Forth? No wonder Time is so hard to understand...

Re:3d interfaces

Forth understand hard not is!

Define "Interface"

The last time I tried to expose *my* 3D user interface, I was escorted to the nearest holding facility for psychological review. /zing

I'll stick to 2d

As long as the "3d" is still displayed on a 2d-field, I'm sticking to good ol' 2d.

Wake me up when they have the gear available that is being used in Minority Report : There seems to be more thought put into that than just to give Cruise a cool way to look for information.

Re:I'll stick to 2d

0 dimensions = point (e.g. single dot)
1 dimension = line (e.g. line of LEDs)
2 dimensions = plane (e.g. a screen of pixels)
3 dimensions = space (e.g. holographic projection)

In all cases, for information to be transferred to a viewer, the display must be capable of changing over time, so there is an implicit extra dimension (time) that is usually just assumed to exist, unless you're specifically talking about motion blur, MPEG encoding, or some other interaction between space and time on the display.

Also note that displays have properties like quantization (discrete pixels) and boundaries (edge of the screen) that are not normally assumed in the context of mathematical/physical dimensions. Further, the notion that there is a "color depth" to the display is a poor choice of words and doesn't correspond to an actual dimension.

Hand Waving

I'd definitely prefer the current point-and-click interface over all that hand waving in Minority Report. Why do so many "futuristic" interfaces (as seen in movies) require the users to move more? That's a step backwards, not a step forwards. Nobody wants to break into a sweat just trying to use a computer.

Re:Hand Waving

depends on what web site you're accessing...

SphereXP

I've tried out SphereXP [hamar.sk] and I have to say it's pretty kludgy. It bogged my machine down quite a bit, and just seemed rather counter intuitive. The idea of "parking" a window somewhere, and not being able to interact with it unless you restore it seemed to be a pretty big limitation to me.

Re:Why is more dimensions "better"

> this will improve anyone's computing experience in any way.

That's ridiculous. Let's talke real 3D, glasses and all. This would completely change everything and for the better. Putting things in a real background, 3D video, parking windows, 3D representations of CD cases instead of ID3 tags, 3D website deisgn, remote control of real world objects, etc.

>This is just another fantastic way to waste the CPU

So is anti-aliasing, so is even having a windowing system that isn't completely and utterly bare bones, etc. Some of us buy our CPUs to use them, not coddle them.

Then again 640k is enough, eh??

Shameless plug

I am working on a 3D UI project, theres still a lot of work to do but I think it has potential. I'd love to get some feedback, positive and negative. I currently have a techdemo but I plan to have a virutal desktop expose like system sometime early next year.
I will probably buy this book just to see if if actually has any good ideas I can incorporate.
logiccubed.com [logiccubed.com]
Jason

3d File Browser

A while ago I wrote a 3d file browser for Linux. It's available here:

http://www.freshmeat.net/projects/3dfb/ [freshmeat.net]

It was a fun project and I wish I had the time to move on with it. I wanted to start adding support for textures and such, but alas school got in the way.

It was an interesting look into the 3d world. I still use it from time to time just to fly around my file system.

My favorite 3d user interface

I like to be INSIDE a 3dlevel when designing it. I haven't coded the technology, maybe never will, but the ability to 'dig' holes or 'raise' up the walls/floor, or place objects inside the level is CLEARLY SUPERIOR to 2d CAD level design.

Underexposed

Two-dimensional user interfaces (UIs) have been around for a long time, and people are accustomed to using them. However, 3-dimensional user interfaces have not yet received as much exposure.

If you mean, "3D UIs haven't been widely accepted", that's true. But if you mean, "We're stuck in a 2D rut because nobody pays attention to 3D UIs," then you have a short memory. Every couple of years there's some new "breakthrough" in 3D technology that's supposed to change the way we use computers -- everything from simple 3D GUI metaphors to "immersive" user interfaces. These have a few applications (mostly having to do with simulating the real world), but mostly people aren't interested.

And why should they be? Adding a third dimension adds an order of complexity to the interface. The challenge of user interface design is to make things simpler.

3D on 2D Still 2D

Just because you arrange your stuff in 3D does not mean that you do your work in 3D. There's just no usefulness other than eye-candy, didn't RTFA, but as long as my monitor is 2D I don't know how usefull it is to have my Word Processor or my spreadsheets in 3D. 3D is best left for games and simulations. I like my file manager, browser, desktop to remain 2D, can't find the crap as it is. Id be cool if I can "reach" into my monitor and strangle the virtual avatar of that moron thou.

Java 3D Desktop

If you're interested in 3D interfaces, you should take a look at Project Looking Glass from Sun:

Project Looking Glass [java.net]
This was demo'ed at JavaOne this year, and really had some catchy visual features. Window contents can be saved to a backing pixmap and then applied to (wrapped around) objects of any shape.

Windows could be rotated (for example, post-its or config info was stuck to the back of a flat window in several cases)

This is still in the prototype stage, but the developer's release is open-sourced and available at java.net.

Nuh uh

I had a good chance to play with Project Looking Glass at Linux Expo in London this year, and while I was certainly impressed, I realised that it's nothing more than a very very pretty extension of a standard KDE/Gnome-like desktop.

First off, you can shrink things down to the bottom of your window. This is basically a clone of the MacOS X dock. You can also shove things off to the left or right of your workspace, which is the same thing, but sideways. The impressive twist to this is that you can still see what the windows are doing when they are in this state, so if, for example, you have a movie playing, it will continue to play in it's docked state. Basically an up-to-date reworking of an existing concept.

Secondly, you can rotate n degrees clockwise or anti-clockwise to get a fresh workspace. Now bear in mind that the number of workspaces is finite, and you always rotate the same amount of space round, it's not an "analogue" rotation. So basically this is the concept of multiple desktops (as KDE and Gnome and various other WMs have had for years) but made much more pretty. The inclusion of a number of specially created "panoramic" desktop wallpapers help enhance the illusion.

You can't move windows along the Z axis, ie change their "depth" in space, nor can you travel vertically around your 3D environment (think Doom vs Quake here).

So basically, project looking glass is a very impressive, very pretty extension of your standard WM. There will be some next generation desktop features that will be taken from it, but noone's ever going to be able to *use* it.

Think of it as the latest Vivenne Westwood creation strolling down the Milan catwalk. Many of the years line in clothes will be based on elemnents of the design, but noone's ever going to wear it to a business meeting.

why 3d is desirable?

why everybody seem to think that 3d will improve usability? information has stayed on plain paper for centuries. and screen will stay 2d for a bit still.

3D interfaces - the Uncanny Valley of UI

To me, 3D interfaces strongly resemble the efforts to produce realistically rendered humans - with an even deeper "Uncanny Valley" [msn.com].

To summarize, this Valley is where when you get closer to the target (realistically rendered huamns) the more of a problem you have with the small remaining portion of data being "not quite right" to the human eye and as a result being much more disturbing to the viewer, contributing to a feeling of "creepyness" or disbelief in the result.

3D interfaces seem to have very much the same problem, exactly because we are such spatially orientend beings and used to real 3D manipulation of objects everyday. Thus the closer 3D interfaces get, the better the 3D inputs get, the more clunky they seem to use - because you know exactly how you would do something in real life and you are constrained in some artifical way by the technology from doing what seems natural. There are a few speciailized problems solved will by 3D inputs, but no good general use that I have seen or read of.

I would never say never - 3D GUIs may well one day become useful. I would say getting the technology out of this valley and into common use is a long ways off - possibly longer than real honest to god grey-goo nanotechnology!!

There could be a reason for this...

3D interfaces have never caught on for a couple of reasons. First and foremost is that the majority of end-user applications, from web browsers to word processors to spreadsheets are simply digital reimplementations of paper documents. The second reason is that there is no hardware that provides three-dimensional imagery that isn't either hideously expensive, causes headaches, or uncomfortable and awkward. What we casually refer to as 3D games, for example, are really projections of 3D structures onto a two-dimensional screen.

Until it is possible to inexpensively provide a convincing illusion of depth -- which is arguably barely possible even with the expensive stuff -- 3D interfaces will require the user to perform 3D actions with a 2D representation. This is a needless complication in most cases.

Been there, done that .. partially

Some years (5 already?) ago I started developing a server and a 3D widget set based on Direct3D. I started with OpenGL, but found Direct3D more fully featured and a better fit for my goals at that time.

Applications shared the server to display their objects. All interprocess communication was COM. You could easily write a 3D app in Visual Basic.

Navigation between applications (or their components) was made easy by having each application offer a set of camera positions and orientations for the user to travel to (using the Alt-Tab and Ctrl-Tab conventions); but the user still could roam freely if he wanted.

Unfortunately my interest waned out before I could do anything really useful. I've still got a 3D piano keyboard object, controlled by an app playing midi...

While some say that a 3D UI doesn't add value, I think there is much to be discovered. Imagine programming in 3D, where each class and function is a labeled box which you can enter to see its code. If-then-else and case constructs could also be presented interestingly. Lame maybe - for now - but I believe that by leaving the linear one-dimensional text model we'll get a completely new perspective which we haven't grasped yet because of the lack of a useable and non-trivial framework to play with. It'll come.

3D Zooming Interfaces

Most of the 3D user interfaces that people usually mention are designed for 3D file system visualization [slashdot.org]. As others have pointed out, it is not clear that 3D adds any value for navigating the hierarchical structure of current file systems.

It gets much more interesting when you combine 3D navigation with Zooming User Interfaces [nyu.edu] (ZUIs). For example, Zoom Quilt [cyphic.net] is a collaborative art project based on Macromedia Flash [macromedia.com] that illustrates what a 3D ZUI might look and feel like. ZUIs work by creating an intuitive information landscape. The user moves "further away" to get an overview, or "closer" for more detail, while keeping a sense of orientation and structure that traditional pop-up windows and dialogs can't match (see research papers [umd.edu] and Java demo [nyu.edu]). Zoom Quilt was assembled from different frames of content [nikkki.net] contributed by various participants. For another Flash-based example of a 3D zooming experience, see also the older Christmas Zoom [dinosaurdesign.com].

A few comments on 3D UIs

A) For a slash site covering next-gen UI issues, 3D and otherwise, check out Nooface [nooface.com]. There's not a lot of discussion, but it has pretty good high-quality no-fluff content and occassional posts from people who really know their stuff or are actively working in the field.

B) The contrast between 3D FPSes (fun, fairly easy, compelling) and VRML/virtual worlds (often pretty awkward) always struck me as interesting and illustrative of the following point. Too many degrees of freedom makes an interface awkward and highly confusing to someone who hasn't had extensive experience with 3D... a loser at the "mother test." id Software and the 3D FPS genre have always benefitted a fair bit imho from architecting the world such that even though it was 3D, you only had 2 directions to go most of the time; forward and backward.

Wake me up when someone has a (non-bogus) study finding that users can actually be more productive in manipulating information with whatever 3D paradigm is being proposed. Eye candy helps but it's pretty easy to lose productivity going 3D imho.

--LP

Maybe a book can help

I've seen a lot of projects take stab at 3D and many of them go about it in a very unnatural way.

SphereXP that someone mentioned earlier, for example, takes regular windows apps and has paper thin windows floating around arbitrarily in 3D. I mean, that just doesn't work. Then you have all these 3D file browsers that cram so many files into this vast 3D mess that unintelligble. You can't read the filenames because there's so much stuff in the way (usually other filenames, but sometimes representations of files or folders) and that's just not natural either.

And I'm not claiming to be an expert on 3D design. I don't know how you'd do a good 3D file browser off the top of my head, nor a 3D desktop. But I can definitely spot the ones that aren't remotely natural or intuitive.

Part of the reason windowed user interfaces work is because the paradigm of a "desktop" makes sense to users. And a desktop is flat. So is a window. So, if you want 3D UI to work, you need to come up with a 3D paradigm that seems natural to the user, and frankly, I just don't know what that paradigm would be.

3D On The Monitor Doesn't Cut It

The only way a 3D UI is going to take off is if you're immersed in a 3D working environment. Once you're in the environment and can overlay information on top of the real world, then it will start being useful. I seem to recall that there are already some applications for this, mostly in the military and with airline mechanics.