Creating normal maps for your own Skyrim character mods

Hey all! I'm the creator of Ethereal Elven Overhaul and the Dagi-Raht Khajiit race and I'm here to tell you all my secrets. This is an advanced-level tutorial and will assume existing knowledge of all the software packages involved - this is just a concise walkthrough across my procedure.

My race mods tend to stand out for two big reasons - custom face morphs and purpose-built normal maps. We're talking about the latter today.

Stuff you need

A BSA extractor

Blender 2.49b with NIF scripts and the .Tri importer/exporter tool

3ds Max (optional but useful)

Zbrush 4R2 and above(or any sculpting software you prefer, but I will only instruct Zbrush use here)

Pen tablet (such as the Wacom product line)

xNormal (free download)

Adobe Photoshop or Gimp with the nVidia DDS plugin and Normal Map Converter

I repeat, this is an advanced-level tutorial for people with prior 3D modeling experience. This won't be a guide to using any of the aforementioned software packages. This is a guide on how you can use all of them in unison to make your own normal maps for your characters and the few required tricks for getting the maps baked and brought into the game without bad compression artifacts.

Importing and peparing the head mesh

How you start off will depend a little on what you're doing. If you want to create maps based on an existing Skyrim race, extract femaleheadraces.tri and/or maleheadraces.tri from Data\meshes\actors\character\character assets\. If you're working with a custom race that has its own associated race morph TRI in existence, skip straight to booting up Blender. Import the race morph file you want. Don't rotate the head file when prompted. Select the head and navigate to the Shapes tab as shown in the picture. Choose the race morph you want to work off on and delete all the other morphs using the X button. Make sure only the morph you want is left and export the head as an OBJ file with default settings.

Open up 3ds Max and import the head using Blender settings. Choose to import as an Editable Poly. Rotate it 180 degrees on the Z axis to make it face Front. Enable the mesh and open the Unwrap UVW modifier on it. When you examine the UV, it will be upside down - flip it vertically from the Tools tab and collapse the UV to the model.

Now select the whole mesh in Vertex mode and weld it with 0.1 sensitivity. Examine the model to make sure no geometry was accidentally collapsed together. Now finally add a Cap Holes modifier and collapse it again to seal the neck hole and any remaining gaps in the mesh. We want it to be airtight.

So right now, your head mesh should be:

-Facing front

-UV flipped

-Holes capped (to see if there are any stragglers, subdivide the mesh)

-Vertices welded

Export the mesh as an OBJ with Zbrush settings and overwrite the original Blender export.

For the record - all of the above is possible to do in Blender just alike. I just highly prefer using Max for its more sensible interface. You can close Max and Blender now. We don't need to visit them again anymore.

Sculpting the face

Fire up Zbrush and grab your tablet pen. This is where we get creative. If you haven't used Zbrush before, now's your time to read up, it has a fairly steep learning curve.

Import the mesh and draw it in the viewport, toggle X axis symmetry on from Transform and enable Edit mode. Subdivide the mesh once, delete lower subdivisions and DynaMesh the model at resolution 130-180. This will give us better, quad-based mesh topology to work with. Sculpting with triangles exported from the game produces less satisfactory results.

I usually do most of my sculpting using Clay Buildup, layering thin strokes to establish facial bone structure and musculature and dominant elements. Work as far as you can in low subdivisions and avoid smoothing all the time. Work towards creating definition and volume and keep photo references at hand. Don't change the general shape of the head much - you'll need it to stay as close as possible to the initial exported model, so don't alter the width of the jaw or the nose and such. Focus on amplifying the facial features.

Subdivide whenever you feel you've got the most out of the polygons allocated to your current level. Most of the work should be done at Subdiv 2 for a point of reference. That's as much as you'll need for the most important parts of the procedure.

At Subdiv 2 I also start varying my brush use. Dam_Standard is really great for sculpting in more tight and finessed lines in the skin, like the eyelid creases and lip definition. Again, once the sculpt looks tight and you start being blocked by a lack of polygons, go up to level 3. Here I left the eye area and ears and lip detail to this level. Notice how the sculpt has both large soft areas and tighter creases and subtle pinched skin folds. This keeps the face looking organic and varied and gives the appearance of an adult person.

Climbing into subdiv 4, sculpt out the small details and brush in skin pores. Little wrinkles in the skin of the forehead and under the eyes make the character look more living. Even younger people get these folds to some extent - completely smooth skin tends to look fake. Finish your sculpt at this stage. Export the high poly as an OBJ (this may take time as it has millions of polygons now) and name it something like yourmodel_hipoly.obj in the same folder as the lowpoly you built this sculpt on top of.

Close Zbrush (save your tool just in case) and fire up xNormal.

Baking the normal map

Under high definition meshes, load up your sculpt file. In Low definition meshes, put your Max-exported low poly mesh in. Navigate to Baking Options and enter your filepath, then set your map size to 2048x. Open up the normal map render settings and set the coordinates to X-, Y+, Z+ (Editor's note: in some cases should be set to X+, Z+, Y+ to make it look properly) and make sure Tangent space is unticked. Hit Generate Maps. Your results should look as below:

RED should be cast from the left. BLUE should be cast from the front. GREEN should be cast from above the head. We're almost done! Close xNormal and open your newly baked map in Photoshop. You'll need to clean it up a little.

Pull up an existing human head normal map from Skyrim. Blow it up to 2048x scale - it'll be a blurry mess but that's fine. Layer your new normal map on top of the blurry one and clean up your map's messy borders by erasing with a soft brush so the old map shows up underneath. You'll also want to tidy up the lips as they tend to bake poorly. Sometimes you also broaden the nose too much while sculpting, so it can be useful to adjust it - and I've noticed the bridge of the nose also goes off-kilter occasionally. If you want to distribute a 512-1024 normal map, this is where you can call it a day. But as I like to push my normals even further, I tend to add high-detail skin pore overlays on top of my texture out of an archive of high-res face photographs.

These are great for super closeup screenshots.

Let's take it ingame! Save your map with the following settings in the resolution you deem right:

8.8.8.8 ARGB produces completely uncompressed texture maps. They're large in filesize (my 2048 maps are a whopping 21mb each) but have zero artifacting. These are also punishing for low-end computers, so distributing your maps in 1024 and allowing an optional download at 2048 is smart.

Looks great!

Enjoy your new original map!