Recovering normal and displacement maps
using bakeAir

(Renderman shaders, and sample rib files)

If you've been around for a while you surely have noticed I have used various techniques to try and recover nice looking displacement maps from within Maya or Mental Ray for Maya.

Alas with all these methods I've been unable to get rid of unwanted artefacts caused by the baking process yet. Mostly I made use of the normal maps to « cover » them.

However I recently tried out a Air and bakeAir licence, and been making some tests and writing a port of my plugin as a Renderman shader. And wow! The quality of bakeAir generated maps is way above all I was able to get so far. In fact it's nearly good enough to be used without the help of additional normal maps (though I'll still cover how to generate them after we deal with displacement maps).

To use the techniques described below you'll need a version (or demo) of bakeAir, and a way to output your models to RIB files (the Renderman scene description format) that the Air renderer and bakeAir utility can read.

As this renderer supports subdivision surfaces, I'll use them for the example below, using nurbs patches or subd will give you the best result as the displacement map you'll eventually get depends on how well defined the high resolution mesh is, and how exact are the normals on the low resolution mesh.

If you need to get these models in your own tools to export them to RIB files, you can find the control cages for these surfaces in Wavefront .obj format here :
TorsoObj.zip

Nowthere are the shaders, and all RIB files « ready to bake » : bakeTorso.zip

Note there are specification of some s,t texture coordinates (likely to be quite goofy too) for the high rez subd in the generated RIB file I use, but the technique used here doesn't need them, you only need a proper UV maping of the LOW version of the object.

To be able to bake with bakeAir, there are some specific additions that were made to the RIB syntax by Sytex, check their documentation to get the meaning of each, just note how I had to add "bakemap_rgb" ["distanceTorso.tif"] at the end of each SubdivisionMesh sub-objects definition.

The baking process will generate a 16 bit tiff file named (until you tune parameters to your liking) : distanceTorso.tif, it should look like the images presented below.

Now you'll need one more rib file and sl shader, so you can render the low rez subdiv, displacement by the map you just obtained and see the result :

Compile the shader, and render that RIB file with Air or any Renderman compatible renderer : torsoDisplaced.zip

I'd say not bad at all compared to the results I had so far with other baking methods, but it could maybe be fine tuned a bit.

Now the parameters listed under the distance map image are the ones that are passed to the rayGetDistance renderman shader I provided. Some of them will be already familiar if you used the Maya/Mental Ray versions of the plugin :

Kd, or Kdout and Kdin are used to normalise the distances returned to the 0-1 color component range. They should be set by trial and error to what you expect the maximum distance to be. Providing only Kd means outside and inside distances will be equivalent (Kdin = Kdout = Kd). Note that the good thing with using 16bit maps, is you don't have to optimize color resolution as much as with 8bit maps and you can set values for Kdout and Kdin that are « confortably above the expected maximum ». Kdout is for the max distance « outside » the model (in the direction of its normals) and Kdin for the max distance « inside ».

Now these parameters can be left to where they are, as for the specific model we are using in this example. When you use the technique on your own models, though, you'll likely have to change them according to the scale and shape of your objects.

Then additionnal parameters are :

SplitChannels which can be 0 to output a grayscale map (default), 1 to output a dual channel red and green map.

Blur, jitter and nsamples which are used to shoot several rays to « smooth out » the results (defaults are blur = 0, jitter = 0, nsamples = 1)

Beware that increasing nsamples of course will directly increases map calculation times...

Another parameter of the displacement map generated if of course the resolution and the color depth used, these are set in the bakeTorso.rib file at the beginning :

Format 256 256 1.0 indicates the 256x256 resolution

and "float[4] quantize" [0 65535 0 65535] is for a 16bit color depth

People familiar with old versions of rayDisplace will note the « bias » parameter has disappeared, it's no longuer needed if you specify in the rib file :
Attribute "reflection" "float bias" [0.0]

Blur, jitter and nsamples are not efficient to increase the generated map quality, provided the high resolution object is of good quality. For that purpose you should instead tune the performance using Air/bakeAir rendering parameters:

So we might wonder, are these blurriness options of any use at all? Well not to increase map quality when the high res object is satisfactory as we saw, but they can be great in case there are issues with the high resolution surface. Let's take the exemple of a heavy polygonal mesh, with some missing facets as a high res object (like the result of a poor 3d scan) :

As usual you are most welcome to write and give me some feedback, and to send your finest samples to the user gallery!

You might also check the excellent Spiraloid forums where other tools than mine are available, and a great discussion is going on on the general subject of using calculated displacement maps in the modeling process.

Olivier Renouard

16/03/03 – updated 23/03/03

olivier AT drone DOT org