Images in 3D Graphics

Texture Maps

2D images are heavily used in 3D graphics. The primary use is for texture mapping 3D surfaces.

Briefly, 3D graphics is based on geometric surfaces described by mathematical (floating point) coordinates. This is much more like 2D "vector" drawings such as those produced with Adobe Illustrator rather than working with individual pixels. (But in 3D "vector graphics" has a different meaning, so don't use the term.)

Plain 3D surface models look like origami or perhaps molded plastic sheet. 2D images are used to texture these surfaces in the artistic sense of adding surface detail to make the image more interesting, without complicating the underlying 3D geometry. The 2D image is "shrink-wrapped" or "painted" over the 3D surface.

As an example, consider a brick wall. The different appearance of the bricks and mortar could be modelled in 3D by creating hundreds or thousands of tiny box-shaped models representing each brick and strip of mortar. This wastes memory and computer power: nearly all the time the wall is viewed from a distance and appears as a flat surface.

Instead, we create a simple geometrical model of the wall and a 2D image of a few bricks. This 2D image is then texture mapped over the geometrical model. Here is the wireframe geometry of the 3D wall (viewed from an angle) and the texture map:

Wall geometry and brick texture

And here is the same wall with texture mapping:

Texture mapped wall

Note that the texture has been repeated (tiled) across the surface of the wall, and that the texture map bricks become smaller as they are drawn further away from the viewer. If you look carefully at the edges from very close up they won't be quite right, as the mortar isn't really recessed from the bricks, but this will hardly ever be noticed.

Texture maps are not limited to flat surfaces. Here is the same texture being used on a sphere (it is a sphere, even if it looks like a flat circle):

Texture mapped sphere

You can see some distortion occurring around the poles of the sphere, caused by the automatic calculation algorithm used.

The "skins" for 3D avatars or characters in computer games are just texture maps. Since the underlying humanoid 3D model will be more complex than a rectangle or sphere, the mapping of the texture to points on the 3D model will be done by hand, a process known as UV mapping.

The power of texture mapping is that no changes need to be made to the original 2D image, no matter how complex the 3D surface or where it is viewed from. The 3D graphics hardware performs all the necessary stretching, repeating, and distorting at runtime. Most "2D" graphic effects are now carried out using 3D hardware and texture maps. This is why every Macintosh computer has a 3D graphics chip to run the animated graphical user interface, and the new Microsoft Vista will only run on PCs with 3D acceleration.

Alpha channels in texture maps

Often we want to use texture maps to create the appearance of natural forms rather than artificial. The problem with these is that their shapes are not simple geometric forms. Here is a photograph of a butterfly to be used in a 3D world:

Original butterfly photo

A simple geometrical model is just two quadrilaterals, one for each half of the body:

Butterfly wireframe

When we texture map this with the original butterfly image, we get this:

Butterfly without alpha channel

We could very carefully construct a 3D model that exactly matches the edges of the image, but this defeats the whole purpose of texture mapping: to create interesting images with simple geometry. Instead we load the photograph into a paint program and use the alpha channel to mark all the white pixels around the butterfly as transparent. When the texture map is applied to the 3D geometry, those parts of the model that have transparent texture map pixels will not be drawn at all:

Butterfly with alpha channel

(The edges of the butterfly are bit "hairy" because the lecturer was lazy and used the magic wand select to do the alpha erase without a final cleanup pass.)

Like the bricks, this butterfly won't look quite right when viewed very closely, but most of the time it won't matter. This technique of displaying images as "flat" shapes is called billboarding and is heavily used for forests, clouds, smoke, flocks of birds, and even crowds of people; anything that needs to be drawn often, but only in the background.

Images as data

Terrain in 3D flight simulators is created from Digital Elevation Maps, which list the ground height above sea level at points evenly spaced on a grid. Normally these are produced by aircraft or satellites with ground mapping radar.

To the computer, a DEM is a 2D array of numbers...just like a 2D grayscale image. (A grayscale image needs one byte per pixel, not three.) The image below was painted with an airbrush tool. Black pixels will have value 0, pure white pixels 255, and other shades of gray in between.

Heightmap image

A 3D program can extract the individual pixel values from the image and use these as heights for a fantasy map. This is the same image data drawn in 3D with a point corresponding to each pixel:

Heightmap in 3D

(A game would draw it as a surface, probably with texture mapping.) Changing this map is easy in a paint program: darkening areas makes the terrain lower, adding white raises it.

This technique is also used to generate indentation details on individual 3D surfaces, for instance the pattern of grooves on an orange or soccerball. An image used for terrain is a height map, an image for surface detail a bump or displacement map.