Tutorial: Compiler charts and limits Last edited 3 months ago2024-09-02 16:13:51 UTC

This article tries to shed some light on the building blocks of the BSP file format, and inner workings of the Half-Life compile tools which assemble it.

In General

When you compile your map it actually goes through four stages or compilation: You can use the -chart parameter to each of the different compilers to see an overview of the current resource usage. Here’s the resource chart for a very simple box map:
models              1/512           64/32768    ( 0.2%)
planes             20/32768        400/655360   ( 0.1%)
vertexes            8/65535         96/786420   ( 0.0%)
nodes               6/32767        144/786408   ( 0.0%)
texinfos            3/32767        120/1310680  ( 0.0%)
faces               6/65535        120/1310700  ( 0.0%)
* worldfaces        6/32768          0/0        ( 0.0%)
clipnodes          18/32767        144/262136   ( 0.1%)
leaves              2/32760         56/917280   ( 0.0%)
* worldleaves       1/8192           0/0        ( 0.0%)
marksurfaces        6/65535         12/131070   ( 0.0%)
surfedges          24/512000        96/2048000  ( 0.0%)
edges              13/256000        52/1024000  ( 0.0%)
texdata          [variable]         48/33554432 ( 0.0%)
lightdata        [variable]       2082/50331648 ( 0.0%)
visdata          [variable]          1/8388608  ( 0.0%)
entdata          [variable]        371/2097152  ( 0.0%)
* AllocBlock        1/64             0/0        ( 1.6%)
Each of the resources listed above (not marked *) directly corresponds to a BSP lump, and the limits are almost always limits the lump structures impose upon each other, because lump entries reference other lump entries by their indices in some way and the data type used puts a hard limit. The following table puts it in more detail:
Resource Limit Referenced by Using data type (bits) Remarks
models 512 entities String (variable) Arbitrary limit
planes 32768 nodes, clipnodes, faces signed short (16) Positive values
vertexes 65535 edges unsigned short (16) -
nodes 32767 other nodes signed short (16) Positive values > 0
texinfos 32767 faces signed short (16) Positive values > 0
faces 65535 models, leafs unsigned short (16) -
clipnodes 32767 other clipnodes signed short (16) Positive values > 0
leaves 32760 nodes signed short (16) Positive values - ?
marksurfaces 65535 leaves unsigned short (16) -
surfedges 512000 faces unsigned long (32) -
edges 256000 surfedges signed long (32) modulo of value, value > 0
Taking clipnodes for example, it's limited to 32767 because that's the number range a 16-bit signed short can hold for positive values over 0.
Now, let’s go through the different resources and see what does what. I will also determine if you are likely to reach this limit and what can be done to prevent running into problems.

Models

Risk level: high

Other than you might expect, the ‘models’ here do not refer to weapons, monster or NPCs but rather to solid brush entities, such as func_wall, func_breakable, etc. This also applies to any brush tied to a trigger entity, such as trigger_once, trigger_push, etc.
User posted image
The only exception is func_detail. It technically isn’t an entity but rather a trick that the compiler uses to stop world geometry from cutting into each other. It is merged into world geometry when the BSP compiler finishes.

Even if you have no solid brush entities, the compiler will always use one slot for worldspawn. Essentially, it is the first [and if the map has no brush entity, the only] BSP model, and all world geometry that's not part of any brush entity is assigned to it.
User posted image
To save on the model limit, you can combine separate object together into one entity. To the engine, they are now considered as one model. This also means that if you look at one object, then the other objects tied to the same entity will be rendered too. That’s why you should make sure that you tie objects together that are relatively close to one another.

When tying objects together, they will receive a new origin point [incorrect? origin is (0 0 0) by default]. Make sure this point is not outside the world or inside a world brush. If it is, then your objects will always be rendered, even through the visibility system. You can determine if this is the case by using the gl_wireframe 2 mode. This is also often the case when you see objects through the skybox.

The compiler maxes out at 511 brush entities (+ worldspawn), which is mostly enough even for larger maps. However, it’s more likely that you will run into a different limit that is directly connected to the model limit: edicts (entity dictionaries).

Edicts can be seen as slots where entities are stored. The edicts limit is the combination of brush entities and regular point entities, so if you’re making large and complex maps it’s very likely this will become your first problem. If you pass the limit, you’ll be greeted with the 'ED_alloc: No free edicts' message.

For retail WON Half-Life the limit is 900 but that was raised to 1200 in the Half-Life 25th anniversary patch in November 2023. The game will likely also precache a few extra entities through the code, so realistically your budget for your map will be less than 1200. The compile chart doesn’t display the amount of entities in the map, so you’ll need to use external tools such as BSPguy.

If you’re making a MOD of your own, you can raise the edicts limit by adding the following line to your liblist.gam file:
edicts "2000"
You cannot set the value higher than 2048, because that’s the hard limit of the entity index.

Planes

Risk level: low

Planes are two-dimensional sheets that run along an axis. When compiling your map, CSG creates these planes to determine where the faces of your brushes are. Even with a medium-sized map, you will max out the planes limit in the CSG step. Luckily, BSP then greatly optimizes the planes by removing all outer planes that touch the void.
User posted image
This hollow cube, for instance, will have 92 planes in CSG, which is then optimized to 20 in BSP. Looking at the shape, you would expect that this cube has 6 planes for each side but the compilers also add a number of extra planes for clipnode generation, so you will always have more planes than expected.

This makes it hard to predict what kind of effect your brushes have on this limit. If you can maintain a clean mapping style, you can expect BSP to do all the necessary optimizations for you. The risk of running into this limit is therefore quite low.

Vertexes

Risk level: low

Vertexes (or vertices) are the corner joints that connect lines together. You’ll need two vertices to connect one line, four vertices to create a square and eight to make a cube.
User posted image
During the BSP compile process all exterior vertices are removed leaving us with a simple hollow cube. These are then counted towards the limit. The only way to lower your vertex amount is to make your geometry more simple. However, the risk of reaching this limit is rather low. You’re more likely to run into other limits before.

Nodes

Risk level: low

The BSP compiler performs Binary Space Partitioning to cut the map into segments and determine where there is open space for the player to move around in. It uses your world geometry as a guide on how to split the world in the most effective way.

The place where this split is made is called a split plane. This split plane has two sides: a front and a back side. It will check if any of those sides are pure solid or contain open space. If there is open space, it’ll continue looking for the next place to split and redo the process. Every time this occurs it creates nodes. The node network that BSP generates is also referred to as the BSP tree. You can see this in the image below.
User posted image
Split plane #1 has completely solid wall on the backside (indicated in RED) and open space on the front (GREEN). It’ll generate two child nodes for both options. One of the options is completely solid, so that path doesn’t need to be examined any more. The player won’t be able to go there. You can see this indicated in the node network as a solid blue square.

The other option is still open and the compiler will look for another place to put a split plane. In this example it picks the opposite wall running along the same axis (#2). There is solid in the front and open area in the back of this plane. From this spot, two new child nodes are created. It then picks a spot on a different axis to split (#3). There is full open space in the back but also partial space in the front. It creates two new child nodes, #4 and #5.

Let’s follow split plane #5 first. There is solid in the back and open area in the front. With all previous splits combined, this now creates a new sector of which the compiler knows that this is an area that can be traversed by the player (indicated in yellow). It will need to do two more split planes (#4 and #6) to determine the same for the rest of the map. The area at the end of the node (and what it contains) are referred to as a ‘leaf’. Refer to the section about leaves for more info.

Next to world geometry, nodes will also apply to brush entities such as func_wall since they also need to be rendered. Even if you turn them into non-solid brushes with NULL textures, the BSP compiler will still generate nodes to determine if this brush exists in your game world.

The risk of running into this limit is rather low. You can only keep the amount of nodes low by keeping your geometry very simple. As soon as you start making larger and more complex maps, you will run into other limits first.

Texinfos

Risk level: low

Texinfo (texture info.) stores information regarding the orientation of textures that are applied to your brushes. Not every individual brush creates a new texinfo. Brushes that have similar world orientations and use the same shift/scale/rotation values can share the same texinfo. Brushes that use tool textures such as User posted image NULL, User posted image AAATRIGGER, etc will not generate texinfos.

Using NULL textures on unseen faces is a good way to reduce the amount of texinfos but you’re not very likely to ever hit this limit. There are more texture related limits that you’ll hit first.

Faces / worldfaces

Risk level: medium

Every surface that has a texture applied to it is considered a face. A small cube has six sides and therefore six faces. However, by default the compiler will split faces every 240 pixels, when the texture scale is set to 1.00. This is called ‘face subdivision’. This was done to keep the lightmap size per face at or below 16x16 lightmap pixels (luxels). If you want to know more, please refer to the Total Map Optimisation tutorial, section 3.4.

Every brush that has a texture will generate faces, including brush entities such as func_wall. Worldfaces refers to faces on world geometry, basically everything that isn’t an entity. This includes func_detail, because that’s technically not an entity.

An exception is made for tool textures, such as User posted image AAATRIGGER, User posted image CLIP, User posted image SKIP, etc. These are removed during the compile process. Therefore it’s important to use the AAATRIGGER texture on your triggers. You can use regular textures on trigger entities but they will then count towards the face limit.

The risk of passing the threshold of this resource is medium. If you use higher resolution textures such as 256x256 or 512x512, you’ll often need to reduce the texture scale to 0.50 or 0.25. This then sets the face subdivision to 120 and 60 Hammer units respectively. In turn, this will generate four or even sixteen times the amount of faces compared to a 128x128 texture on 1.00 scale.

Working with a low texture scale will cost a lot of face resources but ultimately you’re more likely to run into the AllocBlock limit first. Refer to that segment for more info.

A good practice to optimize this limit is to use NULL on faces players cannot see and use higher texture scales on textures that are far from the players sight. The face subdivision also goes up with higher texture scale: 480 units on scale 2.00, 960 on 4.00, etc.

Another factor that contributes to face generation is leaf splitting. Because faces are "owned" by leaves and can't be shared between them, everywhere that the compiler split leaves, the faces are split along the plane. This is compounded when you have detail brushwork such as pipes that intersects with basic world brushes (walls/floors/ceilings), cutting the affected faces by as many sides as the detail intersecting brushwork. Therefore it is very critical to tie detail brushwork to func_wall or func_detail which removes or suppresses/controls face cutting.

The shape, size, and texture scale of faces contributes to AllocBlock limits which will be discussed further down this page.

Clipnodes

Risk level: high

Clipnodes are similar in structure to the previously-discussed Nodes, but much simpler. It is used to simplify collision detection in GoldSrc. Each plane can have three clipnodes for each of the three hulls that the player and monsters use: standing, crouching and large. So a simple six sided cube will end up with 18 clipnodes.

A symptom of exceeding the clipnode limit is that you start not having collision with parts of the map's brushwork.

If you want to know more, please refer to the Total Map Optimisation tutorial, section 6.2.

Leaves

Risk level: high

As mentioned in the ‘nodes’ explanation, BSP cuts the world into segments and creates a tree.
User posted image
At the end of each branch you’ll find a what’s called a leaf. The compiler divides those into leaves and worldleaves. The yellow boxes in the image above are determined to be traversable space and will generate a worldleaf. You can see two of these worldleaves in the map.

Every worldleaf will also count towards the general leaves limit. The other leaves are created by placing brush entities. For instance: If I create a six sided func_breakable crate, then the amount of leaves will go up by six.
User posted image
If you turn the crate into func_detail then it becomes part of the world geometry. It will be partitioning the space around the crate into new square blocks and this create new worldleaves.
User posted image
The worldleaves limit is only 8192, so if you use func_detail a lot you’ll end up reaching the limit relatively fast. Especially with complex objects, the space around it will be partitioned into many small compartments. To fix this, you can turn these objects into func_wall instead and put the load on the normal leaves instead.

However, func_wall is considered an entity and those are very limited as well. You’ll have to balance func_detail and func_wall if you want to create a large and detailed map. Rule of thumb: complex = func_wall, simple = func_detail.

The worldleaves will also increase if you manually add User posted image HINT brushes to improve performance, because you are adding more split planes. If you want to know more about HINT, read the Total Map Optimisation tutorial, section 4.2.

Regardless of what you do, the compiler will always start with one regular leaf.

Marksurfaces

Risk level: low

Marksurfaces connect the visible surfaces in your map to the leaves in the BSP tree (see Nodes and Leaves for more info). Some surfaces can be connected to multiple leaves if they border a split plane, so there will always be more marksurface entries than actual surfaces.

The risk of running into this limit is low, as the maximum amount of entries is 65535. There is no good way to reduce your marksurface usage, except for trying to keep to a simple and clean style of mapping to minimize the number of leaves generated.

Edges / Surfedges

Risk level: low

Edges are defined by connecting a straight line between two vertices. The surfedges resource is used to store the direction of the edge, which can be either from vertex A to B, or from B to A. Both values are used to help the engine render the faces in the most efficient way.

It’s very unlikely that you’ll run into issues with edges as the limit is 256000 and 512000 entries.

Texdata / lightdata / visdata / entdata

Risk level: low

These resources are used to store texture, light, visibility and entity data in memory. All limits can be variable, because they do not directly rely on resources within the engine or BSP format but rather on your computer hardware. The default settings should be plenty for any system. In the early days, setting a high lightdata value could cause trouble for video cards with small amounts of video memory but this hasn’t been the case for many years.

Entdata (entity data) stores all entity data in text format. It has its own limit for the parsing of the data, namely 8194 entities total, 32 characters for key names, and 1024 characters for key values. In particular, worldspawn's "wad" value can exceed the keyvalue length limit when WADs with long paths are used, so keep the location of loaded WAD files in a short path.

Texdata (texture data) stores information on textures used in the map. Each texture entry has two parts, metadata (name and dimensions) and actual pixel and palette data. The latter part is filled when the textures are -wadincluded or using -nowadtextures, otherwise the engine looks for the textures in wad files defined in worldspawn. Also, using zhlt_embedightmap keyvalue on brush entities will quickly increase texdata as unique textures with baked lightmap are generated for each face of the entity, but luckily you can control its resolution using zhlt_embedlightmapresolution keyvalue.

Visdata (visibility data) is very compactly stored and its absence severely impacts framerate in game, so there is little concern on this data, other than running VIS in fast mode to quickly test out the map.

Lightdata contains the lighting data for all faces (except faces marked special). In the last stage of compilation (RAD), all the lighting is calculated and stored in this lump as a 1D array of values. Its size is inferred from the texture scale of the face. (Refer to the following section on the lighting pixel (luxel) scale). There can be 0 to 4 lightmaps (aka light styles) per face. At runtime, the engine unwraps the 1D values into a 2D texture based on the size of the face, modulates them in the case of animated lights, combines them in case of multiple light styles, and puts them onto the allocated AllocBlock assigned for the face. There is some inefficiencies as the lightmaps of some faces (e.g. liquids) are discarded. See below for more on AllocBlock.

If you place too many dynamic (animated or switchable i.e. named) lights in one area, you can hit the light style limit resulting in blotched lightmap patches. More info about light styles is available at: Tutorial: The Complete Guide to Lighting, section 3.17.

AllocBlock

Risk level: medium

The Allocation Block is like a canvas that lightdata gets painted on. To put it another way, it is the texture atlas for lightmaps. Surfaces that are not sky, !liquids, or marked as special, gets allocated a place into one of the AllocBlocks regardless of the number of light styles assigned to the face.

The engine has 64 of these blocks allocated to place all this lightmap data. Each block holds 128x128 lightmap pixels (luxels). Each luxel spans 16 normal texture pixels (texels). You can see the center of luxels mapped on top of the following 128x128 texture:
User posted image
Because there are far less luxels than actual pixels, the light and shadows effects in Half-Life can become quite jagged. The lightmap scale is directly connected to the scale of the texture, so you can increase the relative amount of luxels by using a smaller scales of 0.50 or 0.25 on your textures. This, however, will drastically increase the amount of resources you’ll need to store all this light information. Using 256x256 textures on 0.50 scale by default makes the risk of running into the AllocBlock very high once your map starts to become larger. With 512x512 textures on 0.25 scale you will only be able to make small maps. If you generally stick to using the default HL textures on scale 1.00, you are not likely to run into this limit.

A very common mistake that contribute to maxing out AllocBlock limit is resizing brushes in Hammer/JACK with the texture scale lock or UV lock enabled, producing faces with very small scales and inversely many, many large luxel allocations.

If you need to free up lightmap resources, you can set textures that are far from the players sight to a higher scale, which reduces the amount of luxels used. Using User posted image NULL to remove unseen faces will also remove lightmap data from that face. The compiler (or rather, the engine) isn’t extremely efficient when it comes to mapping the lightmaps onto the allocated blocks but, as a mapper, there isn’t much you can do to optimize this.

A face's shape, and the texture orientation on it, does have some effect on its lightmap size. Faces that are rectangular generates more efficient lightmap than faces with diagonal edges. For the latter, aligning the texture to the edges help produce more efficient lightmap. You can further help prevent inefficient (i.e. non-rectangular) lightmap generation on faces by preventing diagonal cuts on level geometry, particularly by detail brushwork.

Lastly, consider using detail textures which can increase detail on faces from up close while keeping the base textures on a reasonable scale.

1 Comment

Commented 11 months ago2024-01-09 22:56:28 UTC Comment #105855
I wrote a short journal entry on how all 15 lumps relate to each other. I used the relational model to visualize it.

Because the referencing of entries by other entries is done with indices into the lump using mostly 16-bit shorts, it places a hard limit on how many of the target lump's entries can exist before their indices becomes out of bounds for the data types used. In addition, Carmack makes a few of these data types do double (sometimes triple) duty e.g. positive values refer to one type, negative values means entirely different type, and 0 means null, so that halves the capacity of a type's max index. clipnodes suffer the most from this; an entire half (the negative half) of addressable values in 2^16 is used to assign only 2 significant values -1 and -2.

I've incorporated the information into the first section of this page. You can read the table as:
The resource A has a limit of X because it is referenced by JKL with data type T, using [Remark]
The resource clipnodes has a limit of 32768 because it is referenced by other clipnodes with data type short, using positive values > 0

You must log in to post a comment. You can login or register a new account.