Monday, March 15, 2010

Don's Workflow steps for making a high poly game asset.

I am constantly bombarded with issues of "My normal map bake didn't work!" or the final result being messy.  If you want to succeed at baking normal maps, start with these step-by-step rules FIRST, and if you decide to deviate from them, have a good reason for doing so.

There are more than enough ways to skin a cat, but these methods work for me and I very rarely ever have any issues with my normal maps or high poly objects.  ALWAYS be saving versions and iterations of your files.  3ds Max has a mission to crash on you when you least expect it, so SAVE OFTEN.  This list assumes you are using UE3 or UDK as well.
  1. Block in your high poly object FIRST.  It's a lot easier to get the silhouette and scale issues taken care of before you start committing to making high poly work.  This is a rough pass to block in the overall shapes and scale of the majority of your surfaces.
  2. Make an awesome high poly object, keeping the Turbosmooth modifier on your stack when using it.
  3. Finish it up with creative details as needed.
  4. Make a new layer in 3ds Max and add all of your high poly to it.  Name it Original High or something so you can keep track.
  5. Select ALL of your high poly work, hold down Shift, and Move it in a direction in Orthographic view to copy all of it.  This will make it easier later when we want to rebuild our low poly.
  6. Now that you have a second version of your work, add this to a new layer in Max and call it Exploded HIGH.
  7. Hide the Original HIGH layer.
  8. Next up, "explode" your high poly pieces into manageable, individual chunks.  This step is to address each chunk of what pieces you will be making a low poly object out of eventually.  Obviously the more pieces you have the more triangles the final versions will be, but the more accurate it will be to the final object.  Use according to your discretion and budget.
  9. Next, copy each of these chunks (Control+V) and remove the turbosmooth modifier from each of them.  As you are doing this, add each of these new objects to a new layer named Exploded LOW.  This way, we can use the geometry we created from our high as the base for what will be our final low poly objects.
  10. Clean up all of the edge control edges from your newly created low poly objects, and attempt to match the silhouette of the low according to the high poly.  Remember - the closer you can match them, the more likely you are to get a better normal map.  Keep your triangle budgets in mind.
  11. When you are done at this point, you should have 3 layers - Original HIGH, Exploded HIGH, and Exploded LOW.  The exploded layers should contain two types of objects - a nice clean high poly model and a low poly object underneath of it matching the silhouette as best as possible.
  12. Make sure you are addressing any polygon with more than 4 vertices.  If you don't do this, Unreal will do it for you which can be bad.  Avoid Ngons or "T-bones" as well.
  13. Once all of your low poly objects are complete and you have closely matched the silhouette as best as possible, hide the Exploded HIGH layer.
  14. Now that you have JUST the exploded low poly objects onscreen, select ALL of them and Right Click > Convert to Editable Poly just to ensure there are no modifiers left behind.
  15. UVW Unwrap each of your individual low poly pieces.  As you go through unwrapping each of them, move your UV chunks randomly off to the side anywhere in the UV window.
  16. Select ANY of the low poly objects and use the Attach function to attach ALL of your low poly objects together.
  17. Now you should have 1 single low poly object.
  18. Reset X-Form on the object.
  19. Select every face and make sure it has Material ID #1 only as well.
  20. Apply an Unwrap UVW modifier again and pack your UV's. The usual rules apply about UVW mapping.
  21. Once your UVs are packed, make sure you don't have ANY inverted or overlapped faces on the 0,1 space.  See my blog below for how this process works and some workarounds for overlapping / inverted faces.
  22. Collapse to an Editable Poly.
  23. Unhide the Exploded HIGH layer.
  24. With your low poly object selected, bake normal maps as normal selecting all of the objects in the Exploded HIGH layer.
  25. I prefer the Catmul-Rom filter with Supersamples turned on, using Max 2.5 Star for tests, and switching my Supersampler to Hammersley for my final bakes.
  26. Once you have baked everything, it is safe to hide the Exploded HIGH layer.
  27. From here, I duplicated the low poly object one more time into a NEW layer called Final LOW and move it off to the side along the same axis of my Original HIGH geometry.
  28. Explode your single object back out into individual pieces once again.  I highly suggest the "Explode!" script that I have listed below in this blog.
  29. Apply your normal map to the object and check or any weirdness or smoothing on the pieces of this object.
  30. The reason for re-exploding our object back out, is that now we can easily use the Align tool in 3ds Max, unhide our Original HIGH objects and quickly align our newly exploded low poly pieces to the original high poly model without having to re-create it with element mode.
  31. Once all is aligned, hide the Original HIGH layer and you are done.  You should have a perfectly aligned normal baked low poly object in the same worldspace as the original high.
  32. As an added step, I will copy all of the re-assembled low poly pieces into yet another layer, and call that Final Collapsed LOW and attach all of the final finished geometry together before baking AO / exporting it to the game engine.
  33. Drink a beer. Make some textures.