Note: This is my personal workflow as of now. It is tailored to my preferences, so you can always deviate a bit and do what works best for you. Some terms I use are specific to Blender or some other tools I use. The equivalent for other programs will be noted at the end of this post, if I know them. Build your high poly model Build your high poly model first. Do it via box modeling or sculpting, whatever you prefer. A general rule is to keep every part that moves separately separate. This is especially true if you want to bake out Ambient Occlusion and/or Cavity Maps later on. You don’t want to have baked shadows on a moving part. Another thing is that you don’t have to make duplicate parts multiple times. You can just use overlaying UVs in the end. This saves you some work and texture space. If you use box modeling, you will likely want to use a Subdivision Surface modifier on your model. A good way to do that without having the model look like it melted and at the same time keep it easy to work with and able to change, is to use a Bevel modifier beforehand. Most of the time the default setting for the modifier will be sufficient. But as soon as you got round objects on your model, which you want the Subdivision Surface modifier to enhance further, you will need to switch the Bevel to “Weight”. If you do that, you will have to specify the weight for each edge. You will want to mark edges that are comparably “hard”. That way you still get a fairly hard edge, which still gives you nice reflection details. So you select edges you want the Bevel to apply to and hit Ctrl+E. Select “Bevel Weight”, press 1 and hit enter. For the rest of the options, I usually set the amount of edges to three and the size/distance to something corresponding to the model’s size. The iterations for the Subdivision Surface modifier are usually set to 3 for my models. If you are working with Substance Designer/Painter to create your textures, you might also want to apply different materials to the parts of your model, that are supposed to have different materials in the end. I usually use RGB 255, 0, 0; RGB 0, 255, 0 and RGB 0, 0, 255 for the material color. Those three colors are the easiest to distinguish for every color filter in image processing, so they work nicely. If you need more than 3 materials, you can either use some other colors (combinations like RGB 255, 255, 0) additionally, or you can use the same colors again and render them out to a different mask later on. Using more than just the 3 colors might lead to you having to do a little more tinkering in the image processing later on. This method works with every kind of color filter, so you could also use it with Photoshop to set up your masks for painting. Build your low poly model Once you are done with your high poly, you will want to make your low poly. There are several parts to this for optimal baking performance and results. For the most basic, a lowpoly that doubles as your baking target and game mesh and a high poly is enough. If you want to optimize your baking, you want to have a target mesh, a separate game mesh, a high poly and a cage. Now a target mesh has some really special use cases where you want to use one, usually you don’t have to go that far. A cage is not always needed, but is mostly useful for meshes with concave parts where differing ray distances are important. A convex mesh will never have to use a cage. General Starting Point What is the same for all of those meshes is the topologie (how triangles are arranged on the surface) and the UV coordinates. So what you need to do before making separate meshes is to make a low poly and unwrap it. If you used my method with Bevel and Subdivision Surface modifiers, you should still have a fairly usable mesh in your high poly, once you go to edit mode. Copy your high poly objects and remove both modifiers. Now go into edit mode and start optimizing your model. That means remove any unnecessary polygons and edge loops. You also don’t have to worry about only keeping quads. Both quads and tris are fine now. You also want to remove any nGons, especially on cylinder caps. Replace them either with triangle fans or quad strips. (Triangle fans are more easily constructed, but in most cases quad strips save 2 triangles per round surface.) You don’t have to triangulate your mesh right away manually, you want to do that later on with a modifier. The important thing is, that you end up with a mesh that won’t lose or gain any vertices from now on. Once you are done with that, start marking UV seams(Ctrl+E->“Mark Seam”) on your mesh. If you just simply copied over your objects from your high poly, you can join them together for the next step. It will make the unwrap easier, as you can simply unwrap everything on the same UV map with one command, instead of having to manually push UVs around for every single mesh and additionally try to not make them overlap. Unwrap them (U->”Unwrap”). Afterwards separate them again. (Mark the part you want to separate and hit P->”by Selection”). Target Mesh Use Cases and Setup One of the special use cases for a target mesh is when you have round parts, which are smoothed by your Subdivision Surface modifier. Sometimes it can happen, that your negative ray distance doesn’t reach far enough and will leave holes in your bake. This usually occurs where your edges in the lowpoly are. Those are the farthest away from the high poly mesh. To fix that, you can modify your target mesh in that you move the culprit edges further inside your high poly, BUT only along their normals. Otherwise you will get weird normal results. In Blender you can easily do this with Alt+S. The second use case is when you have a modular model where multiple elements are the same. An example is a cube I made for Space Engineers. It only had three differing parts: a bottom, a top and a side, which was the same on all 4 sides. In that case you don’t want to bake the same part over and over. You only set up a high poly once for the duplicate part, then do the same with the target mesh. Set up a target mesh as stated in the corresponding use cases, if you need one. Make sure you copy your unwrapped low poly. You can delete vertices, but you should never add new ones. Cage Mesh Use Case(s) and Setup The most common use case for a cage is when a constant ray casting distance while baking isn’t sufficient or would lead to artifacts. A good example is the space between two fingers. If you have a uniform ray distance, you might bake the side of the next finger on the finger you are currently projecting with. A cage MUST have the same topology as your baking target. (So either the target mesh or the game mesh, depending on which you use.) The vertices of the cage basically draw a line to the corresponding vertices of the target. Those line are the ray casting distance and the normals of the target vertices. So the same as with the target mesh: Only move along the normals. (Alt+S) What you want to end up with is a cage that wraps everything of the high poly and low poly model. Everything outside will not be considered in the bake. Target and cage vertices must not be on the same spot in 3D space. You need to move the cage vertices at least a bit. Generally you want to keep the cage as tightly wrapped around your models as possible. Export Before you export your models, you want to use a triangulation modifier on all of them and make sure they are set to be shaded smooth. In the export dialogue, make sure that “Apply modifiers” is checked. You can export to either .obj or .fbx. I would recommend .fbx, as it saves material color objects properly. (Use the binary format version). Bake and Texturing Now bake every separate part. Normal bakes should include just the normal map, ambient occlusion and cavity. But if you plan on generating some the textures for the model with image processing, bake out the maps you need. Then make your textures. Further steps There are some things you can additionally do to take things further. For example skinning the model to bones and animate it, make collision models, whatever your target engine requires. If demand exists for me to explain these steps, I can elaborate further, but this is my basic workflow for now. Program Tool Equivalents Color Code Blender 3DS Max Substance Designer/Painter Photoshop Tool Equivalents Subdivision Surface~Turbosmooth/MeshSmooth Bevel~Sadly no equivalent, Bevel is a destructive function. You can however simply use supporting edges. (use “Connect”)
