Mapping optimization tutorial

Discussion in 'Mapping' started by Varbles, Feb 20, 2012.

  1. Varbles

    Varbles Simply Maptastic. Staff Member

    Messages:
    2,093
    Likes Received:
    26
    Trophy Points:
    0
    In this article we'll examine every facet of building an optimized map for Empires. Part tutorial and part analysis, this is aimed at increasing every mapper's awareness and knowledge of the process of optimization, and the techniques and tools at their disposal. I'll be adding to this in parts, starting with the most important methods and moving on to more niche applications and information later.



    First, a little bit about the concept of optimization itself. Dictionary.com defines it as "an act, process, or methodology of making something (as a design, system, or decision) as fully perfect, functional, or effective as possible." A very vague definition, but it holds true for mapping. More specifically, however, optimization in mapping centers on one thing: The minimization of waste.

    Every tool available for optimization is there to reduce waste. Waste can appear in many aspects, such as wasted compile time, wasted rendering, even wasted file size. Many mappers assume there is a trade-off between optimization and detail, that you have to somehow choose between the two. The nature of optimization is actually the opposite. Optimization allows and encourages for MORE detail, by virtue of reducing the wasted processing that would discourage more detail. A properly optimized map can have much, much more detail than any unoptimized map and perform better to boot.

    However, the nature of optimization is that it strives for perfectly optimal scenario. In contrast, there is practically no such thing as an "optimal map." When we optimize, we can never take a map to it's fully optimal extent. This is acceptable though, because we don't try to. We only need to optimize to the point that waste is negligible, "negligible" being the point where the mapper deems it acceptable. There is always a way to further reduce waste in a map, but we only need to bring it to the point where we're happy with it.

    So why, then, is optimization so often ignored or misused?
    It's effects are not directly seen by the mapper. It is done by manipulating mechanics most mappers are not aware of, let alone familiar with. Even experienced mappers are not aware of or intimate with ALL the tools at their disposal. Optimization is seen as unnecessary or trivial by mappers with modern hardware. It is seen as too hard to be worth learning about.
    None of these are legitimate reasons to ignore optimization though, and hopefully none of these will apply to you after learning a bit more.


    Part 1: The Visibility System
    Since the Visibility (vis) system used in source maps is arguably the most important and confusing part of optimization for Empires, we’ll start with that.

    Understanding the vis system.

    First we’ll take a look at the vis system optimization in some official maps. You can take a look at the visleaves in any map and see what’s being rendered and what’s not with a few commands in console. These were taken by spawning in, walking to parts of the map, entering r_lockpvs 1 in console and then taking an overhead look.

    These shots of the official emp_canyon show what is being rendered when the player is standing in the rax at the BE or NF start:
    [​IMG] [​IMG]

    It’s clear that a lot of the ground on parts of the map the player can’t see isn’t rendered, as well as the rock models and such on the other side of the map (the trees do not appear because their LOD Models can’t be viewed from above). This is a great example of the desired effect of vis optimization. Since a player obviously wouldn't be able to see the other side of the map, vis optimization has been used to save the engine the trouble of rendering it all. It isn't just the terrain that's being culled, it's also applied to anything going on in that area, be it cannon shells and bullet tracers flying around, or buildings and turrets placed there. It seems fairly effective but there is still a bit of waste. There are some smaller areas that the player can't see but are still being rendered. All of the vertical displacements making up the cliffs can be seen, as well. This is a direct result of what we’ll call the “open sky” system that we use in most empires maps.


    This shot of the official emp_coast shows what is and isn’t rendered when the player is standing on top of the bunker in the middle.
    [​IMG]

    We can see that much of the far away ground geometry isn’t rendered, but again, there is still a bit of waste in the form of models and terrain that the player renders but can’t see. The separate vertical cliff displacements are again always rendered because of the nature of the system we use.

    This last shot of the official emp_slaughtered shows the large difference between what the player sees and what they have to render.
    [​IMG]

    We can see that although there is at least some basic optimization there is also a large amount of waste. Slaughtered is different from the previous examples because instead of using separate vertical cliff displacements, the whole map is one mesh of displacements. We’ll look a little closer at the different approaches and how they relate to our vis optimization later.




    For these next parts I’ve put together an example map that demonstrates the implementation of Optimization concepts in a map layout mockup that’s easily related to most other map designs. I strongly urge you to download it and follow along in it because it’s far easier to learn directly in hammer than follow along on just pictures and text. The map uses displacement geometry from Omni’s Silo, and it is only for example purposes.
    DOWNLOAD IT HERE



    As you hopefully know, the Source engine uses “visleaves” as a basis for it's vis optimization (unrelated to "visgroups" in hammer). A map is divided into a number of visleaves, at first into a horizontal grid of 1024x1024 visleaves by the VBSP compile program, and then each leaf is further divided using the faces of every World brush or hint plane in it. Then the VVIS compile program determines, for each visleaf, which other visleaves are directly “visible”. This is called the Potentially Visible Set, calculated using only World brushes (all non-entity brushes, excluding those textured with alpha textures) as “visblocking” brushes. When the player walks around the map, the engine loads the PVS for the current visleaf, and only renders objects in or touching it’s PVS.

    Since most Empires maps are very large maps constructed completely or mostly with displacements, this leads to two issues:
    The number of visleaves increases with map area, so exponentially more time is required to calculate the PVS for every leaf, meaning extremely long unoptimized VVIS compile times for fullgrid maps.
    Virtually every visleaf can see every other visleaf, since displacements or brush entities are ignored by VVIS, and unoptimized Empires maps generally don’t include large world brushes which block visibility.

    Manipulating the Visleaf system to correct these problems, however, is a simple task that can be done with any map more complex than emp_moors. The way we do this is a technique we’ll call the “open sky." This method involves designating the large open area above the actual playing field as the "sky", with the playing field below divided up by brushes under the terrain blocking visibility between areas of the map that shouldn't be able to see each other. As with any other method, this method has it's advantages and disadvantages; most importantly we have to concede that any player viewing the map from in the "sky area" can see every part of the map. Naturally this is very relevant to the Commander view, however since Commander view chiefly looks down upon instead of across the map, there is less potential use for visleaves, and more benefit from other culling methods such as Frustum culling. The open sky also allows for much easier movement in the Commander view, so all in all the main pitfall of the open sky method is not a problem for us. The advantages of this method are that it is simple, quick to implement, and does not require changing the form or any visual aspects of the map.

    The open sky system involves adequate use of three things: visblocking, hints, and visclusters. First, we use the technique of visblocking: using world brushes underneath all the major hills or cliffs to block visibility for the PVS calculations.
     
    Last edited: Jun 10, 2012
  2. Varbles

    Varbles Simply Maptastic. Staff Member

    Messages:
    2,093
    Likes Received:
    26
    Trophy Points:
    0
    Basic Visblocking


    Open the example map and disable every User Visgroup except “Visblockers” and “Map boundaries”, and you’ll see the world brushes underneath the displacements. Constructing these visblockers in your own map is simple, if you keep in mind the concepts illustrated in the example map. Visblockers should be large nodraw brushes, placed under every major view-obstructing hill or cliff, reaching from the bottom of the map to a height well above the highest level the player would be able to see over. The visblockers don’t all have to be the same height, but they should reach near the top of their respective hills without poking out anywhere. They should be simple brush collections, not much more complex than the ones in the example, but they need to follow the contours of the hill well. Take extra care to follow outside corners as closely as possible, inside corners are less important. They should take up as much area as possible under the hills, reducing the pointless wasted space there (the inside of world brushes is treated as outside the map by the BSP compiler).


    visblockers underneath main cliffs
    [​IMG]


    We want to have visblockers under every major obscuring cliff in the map, but beyond that, we get diminishing returns from them. If we look at the example map, visblocking is done under the cliffs circling the center and around the outside. Making visblockers for the smaller bits of cliff of course would not hurt, but since they would obscure only a bit of the map even if the player is all the way behind them, we can ignore them. While a good mapper would probably put in the 10 seconds required to visblock these hills, the example map shows what we’ll consider an acceptable level of visblocking. The visblocks under the outside cliffs, although they don’t obscure anything, are included because they reduce the wasted space under them, and block unneeded lighting coming in from the sky texture under the hill.

    The construction of the bottom of the map matters too, it should be as close to the ground as it can to reduce wasted space. If any one part of the map is lower the rest, we want to only lower the bottom of the map for that specific area. We won't try to reduce any of the extra space in the sky, because that space isn't wasted, as it’s used by artillery shells and the commander camera.

    These visblockers are the first and most important part to our “open sky” technique for optimization, but they are for the most part useless on their own. The technique requires we implement two more components to be effective, Hint planes and Visclusters. We’ll examine those next, but first we’ll look directly at the visleaves as we have them so far using a lesser known feature in hammer: Portal files. These are generated whenever the map goes through VBSP. First we’ll disable every User Visgroup except “Displacements” and “Map Boundaries”, then we’ll put the VMF through only a BSP compile. When it’s done, under the Map dropdown menu, we’ll hit “load portalfile” and click yes to load the default one.


    large, unhelpful visleaves
    [​IMG]


    We can now see our horizontal grid of very tall visleafs generated by VBSP, outlined by blue lines. This tool is a very good learning tool since the visleaves that appear are the actual visleaves in the map. Since we compiled the map with only the displacements and no forms of optimization, none of the visleaves are cut beyond the automatic grid, and no PVS is calculated for any of them. We can see that these visleaves are useless anyway, though, because they’re all so large, and they’re all visible to each other. Next let’s try a BSP compile with just the “Displacements”, “Visblockers”, and “Map Boundaries” User Visgroups enabled. When we load the new portal file we see that the visleaves are largely the same as before, only there are cuts in the leaves containing the visblockers. This is for the most part absolutely useless, because the main areas of the map are still contained within the very tall grid visleaves, and the player would have to get very close to the visblockers to enter a leaf that has been cut properly.


    the visblockers cut only the visleaves they touch
    [​IMG]




    Hint planes


    This is where the use of Hint planes becomes essential. Hint planes are made by applying the tools/toolshint texture to any face of a brush. Hint planes should be thought of in terms of the faces they’re applied to, instead of the brush itself. Every other face of a brush used for a Hint plane should have the tools/toolsskip texture applied to it, so the face gets discarded by the compiler. Each face acting as a Hint plane will be used to cut every visleaf it touches along the plane of the face, the same way the visleaf would be cut if instead of a Hint plane there was a World brush face there. The Hint planes get discarded by the compiler after they’ve cut their visleaves. Understanding and getting a feel for how these work can be tough, and learning where to implement them can be tougher, fortunately it only takes one hint to have our “open sky” system working moderately well.

    If we enable the “Sky Hint” User Visgroup (under Hint Planes), we can see the one most important hint for our “open sky” system. This Hint plane is a horizontal plane stretching over the whole map, exactly at the level of the top of the visblockers. If you have visblockers with varying heights, this map wide Hint plane should be level with the top of your lowest visblocker, one level with each different visblocker. This Hint determines the area of the map that will be our “sky area” and which areas will be ground level. The concept is that each ground level visleaf will only be as high as the top of the visblocker, so no ground visleaf will be able to see over a visblocker into any other visleaf. If we do a BSP compile with the “Displacements”, “Visblockers”, “Map Boundaries”, and “Sky Hint” User Visgroups enabled, and then load the new portal file, we can see that all of our visleaves are now cut along the top of the visblockers.


    the large horizontal hint bisects every visleaf at the top of the visblockers
    [​IMG]


    This also explains why when we looked at official maps to examine their optimization, we saw the cliffs that the visblockers were under around the whole map. Since in most cases all faces of the cliffs will be touching the “sky area” visleaves, they will be rendered from anywhere in the map. This is generally okay because the displacements are fairly cheap and simple to render. It becomes more of a problem on maps like Slaughtered, which use only a large mesh for hills, rather than separate vertical displacements. On maps using a mesh, a lot more of the terrain will be rendered rather than just the tops of the cliffs, especially if large displacement tiles are used. This is not something to be very worried about however, because optimization comes second to good map design.

    This setup is moderately effective in game, but still needs some refining to be acceptably optimized. The automatically generated grid of leaves is too rough to work well around corners and in areas where the player might be able to see into the middle, and our compile time is still going to be unnecessarily long. We willl add some more advanced Hint planes to the ground visleaves to try and minimize the amount of extra terrain they have to render, especially focusing on hiding the middle from people on the outer part of the map. The placement of these ground level Hint planes is a lot more open ended and is different for each map design, so it can be a bit confusing. Just remember that the objective of these Hint planes is to make our visleaves better suited to the corners and angles of our visblockers, instead of relying on the default square grid. If we examine the North West corner of the example map, we see the Hint brush with two Hint plane faces covering up the Brenodi barracks. The edges of the Hint planes line right up with the edges of the large diagonal visblocker, and the angles of the Hint planes make it so no leaf inside the volume can see around the visblocker. If we look towards the middle, we see the large X made by Hint planes, which serves to divide the square grid leaves into leaves that will be more quickly hidden by the visblockers as the player leaves the center. When it comes to these ground level Hint planes, there’s really no wrong answers; virtually every cut you make will help improve performance in some way. At the same time, there’s always going to be something more we can do to improve performance, but all we really need to do is optimize to a decent extent. A few good ground level Hint planes is all we really need, anything beyond is icing on the cake. If we do another BSP compile with our displacements, visblockers, Hints, and map boundaries enabled, and load up the portal file, we see convenient cuts exactly along the Hint planes, making the visleaves a bit more suited to our visblocking.


    vertical hint planes at ground level cut the visleaves to make more use of our visblockers
    [​IMG]



    we see the cuts in the visleaves made by the hints
    [​IMG]
     
    Last edited: Jun 9, 2012
  3. Varbles

    Varbles Simply Maptastic. Staff Member

    Messages:
    2,093
    Likes Received:
    26
    Trophy Points:
    0
    Visclusters


    These techniques have formed our basic “open sky” optimization structure, reducing waste by a sizeable amount, allowing for more detail and higher framerates. But this still hasn’t addressed our problems with with wasted compile times. This is where the entity called Func_viscluster integrates into our “open sky” technique. A Func_viscluster, or just called a viscluster, is a brush entity that for all practical purposes melds every visleaf in it or touching it together into one visleaf for the purpose of calculating PVS. VVIS assumes every leaf inside the viscluster can see every other leaf inside, and every leaf outside that can see the viscluster, can see every leaf inside. For this reason, it is a very useful tool to reduce VVIS compile time, but we need to be careful with how they’re implemented; the viscluster is best used primarily on the “sky area” we segregated with our first Hint plane, and other areas like the 3d skybox.

    The “sky area” viscluster should reach all the way up to the top of the skybox and come down almost to your horizontal Hint plane, over the whole area of the map. The purpose of this is to take every visleaf in this sky area and tell VVIS that they should all be treated as one large visleaf. This saves the compiler from spending an incredible amount of time trying to tell which other visleaves can be seen by each of these leaves in the sky, when we've already determined that everything in the map is visible from the sky area. By including this sky area viscluster we have shaved hours and hours off of the VVIS compile time without sacrificing any in-game performance. Another similar placement of a viscluster you should consider is in your 3D skybox area, which by nature is separate from the rest of the map and doesn't stand to gain any performance from the VIS system. A viscluster brush in there will shave off a bit more wasted compile time, but not nearly as much as your sky area viscluster.

    If we do a BSP compile now with our all our displacements, Hints, visblockers, map boundaries, and the sky viscluster enabled, we can see that the vertical lines between all the visleaves in the sky area are gone, making it appear that they have all become one visleaf. This is not exactly the case, but for practical purposes this is how we can think of them.

    notice that the vertical divisions are gone from the sky area
    [​IMG]

    The next step in the use of visclusters is to place some in areas around the main gameplay areas of the map, i.e. ground level. However, when we start adding the main ground level leaves to visclusters, we will inevitably start sacrificing in-game performance. Whereas the sky area viscluster we added cut down compile time without performance loss, adding visclusters to the main ground level of the map is in every respect a trade-off between lower compile time and in-game performance. If fact, since their placement is a very map dependent endeavor, and since you can undo the benefits of your previous optimization easily while attempting to gain a slight improvement in compile time, it might be a good idea to save yourself the effort and not try to add visclusters to the main gameplay area of your map. That is not to say that it should be avoided altogether, but that we need to do it carefully and deliberately, keeping in mind that there is only a little bit of benefit to gain while there is a lot to lose.

    When we look at the visclusters in the example map, we can see they’ve been fit to the inside of the large hint brushes, to preserve the way the Hints cut the leaves while letting VVIS ignore the smaller automatically generated leaves. Consider the viscluster in the Northwest, where the BE barracks is. This viscluster takes the area between the two Hint planes, the Northwest corner, and the visblocker under the hill and combines it all into one visleaf. This is advantageous because we're informing VVIS that every leaf in that area can see every other leaf in the area, and that if an outside leaf can see a leaf in the area, it can most likely see all the other leaves in the area. If we look at the layout of the map, we can see that this is true; someone looking from the Southwest corner of the map can see most of that area, as can someone from the Northeast, while someone in the center can't see it at all. So we've saved the compiler a little bit of time and not given up much in-game performance in the process.

    the viscluster is inside the hints, without touching or crossing them
    [​IMG]



    the vertical divisions inside the area are gone
    [​IMG]

    On the other hand, there are no visclusters in the center of the map because, although every leaf in the middle can see every other leaf in the middle, we don't want to force someone in the northwest who can only see part of the middle to render all of it if they don't have to. Remember that the larger a visleaf is, the more places it can be seen from, and visclusters take several smaller visleaves and combine them to make larger ones.


    Keep in mind that the example map actually has more visclusters than you'd want in a standard map, sacrificing a considerable amount of potential performance for just a few minutes shaved off of an already low compile time. Compile time increases exponentially the more visleaves and open space there is in a map, so to bring compile time down to an acceptable level you don't need visclusters everywhere, you're best off with just your sky area viscluster and a few visclusters in good places around the map.


    Important things to note when using visclusters:
    NEVER allow a viscluster to intersect a water brush in any way
    Don't make visclusters too large as they can seriously undermine your optimization
    Each brush should be it's own viscluster entity
    Stick to using the Nodraw tool texture for viscluster brushes
    The viscluster entity is discarded by the compiler after it's used so it does not appear in a running map
     
    Last edited: Jun 11, 2012
  4. Varbles

    Varbles Simply Maptastic. Staff Member

    Messages:
    2,093
    Likes Received:
    26
    Trophy Points:
    0
    still a WIP, a lot more screenshots to upload, trickster said to post it here so it can get consolidated into one post, since its more than 11000 chars over the limit for one post atm.
     
  5. Trickster

    Trickster Retired Developer

    Messages:
    16,576
    Likes Received:
    46
    Trophy Points:
    0
    Yeah, just tell me when you're ready to go public and we'll try to arrange it into one post. Somehow.
     
  6. Beerdude26

    Beerdude26 OnThink(){ IsDownYet(); }

    Messages:
    7,243
    Likes Received:
    13
    Trophy Points:
    0
  7. Silk

    Silk Mapper

    Messages:
    3,147
    Likes Received:
    36
    Trophy Points:
    0
    You can't imagine the amount of hours i wasted adding testblocks strategically to maps for full vis compiles just to see if they were rendering ingame with the mat_wireframe command.

    And now, with your method, all i needed was 2 minutes tops to get the utmost perfect test.
    http://dl.dropbox.com/u/234836/vis_blocking.jpg

    I'd be thrilled if i wasn't so frustrated over wasting all that time in the past 5 years.

    For the love of god, keep making these detailed tutorials!
     
  8. bitchslap

    bitchslap Member

    Messages:
    1,217
    Likes Received:
    0
    Trophy Points:
    0
    Ty for this, it helps a lot to those desiring better quality maps.

    When done, you could always just repost his posts into a new thread and sticky and close it so it won't get cluttered up but will be available to the whole community. Not that many of them will bother, but gotta try. :)
     
  9. Varbles

    Varbles Simply Maptastic. Staff Member

    Messages:
    2,093
    Likes Received:
    26
    Trophy Points:
    0
    yeah it will be posted all in one post on the mapping forum or something when i can get it all together
     
  10. Varbles

    Varbles Simply Maptastic. Staff Member

    Messages:
    2,093
    Likes Received:
    26
    Trophy Points:
    0
    updated and moved to mapping section
     
  11. [lodw]keef

    [lodw]keef Hobbit

    Messages:
    1,336
    Likes Received:
    3
    Trophy Points:
    0
    you probably want to mention model fade distance, displacements take up a pretty small amount of power to render. Where as models take quite a bit to render, specially when the modeler doesn't put in some LOD lower quality models as you get farther away(a problem empires has a bit of). I generally do fade out distance of 10240, about when empires entities fade away. You dont want a long fade out distance though cause fading takes more to render than just the model. If you can lower the fade distance though do it, for instance some models that are inside partially open buildings that cant fully visblock off the whole inside of the structure but you cant see any of the models inside from a distance, go ahead and make the fade distance of the models inside like 1000 or something(depends on geometry and angles you can view it at etc).

    The technique of map to 3D skybox rendering is a debatable topic, I have found it quite useful for really open maps though. It has a thorough rendering cutoff point(again I do 10240 units) that works like model fade distance for all the rest of the little things, bullet/cannon decals, water reflections, res node steam, any other item that isnt optimized for distance, allows for LOD of map geometry(the lower quality copies you put into the skybox). I've heard some complaints over the years that i'm not gaining anything if i'm rendering the map displacements twice, however displacements are cheap and I've found through comparing a map compiled with 3D skybox cutoff and without, using the 3D skybox duplicate map is cheaper. You just have to be aware of your texture locking and scale locking when you do it, also to have a seamless scale you have to scale the lightmaps down the same amount manually. (you can also not scale the lightmaps down on areas with small shadows for a little shadow detail LOD). basically this technique replaces maps having fog where you cant see anything of the map.

    also the newer FGD should have options for fade distances on empires entities like ammo boxes etc now, use them.
     
  12. Trickster

    Trickster Retired Developer

    Messages:
    16,576
    Likes Received:
    46
    Trophy Points:
    0
    emp_optimization looks like a fun map Varbles. I wonder which wonderfully attentive optimiser mapped it originally.
     

Share This Page