This tutorial will show you the basics of using newbspguy to edit already compiled BSP files. We will use UnrealKaraulov's fork of wootguy's bspguy due to the many added features, and the decoupling from a strictly Sven Co-op focused workflow of the original. We also assume that you have some experience in mapping the usual way, as there are differences when editing with this program, as noted later.
Preface
The workflow of editing already-compiled maps is totally different from making a map from scratch.
In the past, it was easier to just decompile a BSP back to .MAP, and start building up from there, because the typical modding workflow assumes you start from a map in an editor. However, there are several drawbacks to this method:
- Decompilation is a messy process. Due to how the BSP data is structured, the entire level is chopped up across planes, which goes all over the place irrespective of the original brush shape. Decompilers faithfully follow the planes structure, resulting in brushes that likewise go all over the place. Many of them would also be invalid solids and need to be recreated.
- You lose all texture lighting data. 9 times out of 10 a custom map would be using custom texlights entries that are different, or absent, from the stock valve.rad. Your option is to recreate texlights by estimation and guesswork, which won't give you 100% of the original texlight values.
- You lose all clipping data, because no decompiler as of time of writing is able to recreate "Clip" brushes from the clipnode trees since clipnode trees are not technically part of the level geometry.
However, as of 2023, there now exists tools that are capable of editing many aspects of already compiled BSPs directly, namely, bspguy and its fork newbspguy. The original tool was created exactly to port GoldSrc maps over to Sven Co-op, but both tools, especially the latter, can be used just as well to edit maps for any GoldSrc game/mod.
[new]bspguy is invaluable as a study tool, to learn how level designers map their maps. For well known games/mods, it can reveal to you tricks mappers use to achieve certain effects with just clever map work and no code. For obscure games/mods, it shows you how custom entities are set up to implement core game mechanics; and you have a sample size of... the whole game as opposed to a few basic example map sources the developers might give you.
You no longer have to decompile a map and load them in VHE/JACK, or run the map in game with hooks like BunnymodXT to show invisible triggers, whilst enemies are shooting at you, to study it.
Pros, Cons, and Caveats
The advantages of editing BSPs in place are:
- Lightmaps are preserved.
- No messing with messy and invalid decompiled brushwork.
- No need to recreate clip brushes.
- Trivial brush entity duplication, that normally doesn't add to BSP model count.
The disadvantages are as follows:
- You cannot edit world brushes whatsoever.
- You cannot do crazy texture scaling since it's tied to the lightmap size.
- You cannot edit clip nodes.
- There's no easy way to add complex brushwork. You need to rely on models.
Do note of these differences for compiled BSPs when editing:
func_detail
is not a thing in compiled BSPs; they're transformed into normal brushes during compilation. You need to use func_wall
or func_illusionary
.
clip
brushes are also not a thing; they're transformed during compilation into clip nodes which you cannot edit. You can however use the default BSP solid models the program creates to artificially add collision. Do note that this adds to the BSP model count and clipnode count so be conservative.
origin
brushes are not a thing; the BSP models already has its own origin on creation, which you can move.
- The program is unstable and can crash when doing certain random things. Save often.
- Created BSP models immediately adds clipnodes and cannot be optimized. For already clipnode-strickened maps this could bring it over the threshold.
- Don't use the merge command of the original program as that's specifically for Sven Co-op porting. Merging maps will easily exceed the engine limits of vanilla GoldSrc. Map merging is out of scope of this tutorial.
Anyway, let's get on with it.
Setting up newbspguy
- Download the latest release of newbspguy.
- Extract newbspguy into a folder.
- Run the program.
- Open File > Settings.
- Under General, click Reset All Settings.
- Under General, check Make map backup.
- Under General > Game directory, set to location of your Half-Life installation.
- Under FGDs, add the FGD of your target mod.
- Under Asset paths, add relative paths in the following format:
<mod folder>/
. There should be at least these entries:
valve/
<the target mod>/
(if that isn't valve)
- Click "Apply settings". Close the options window.
General settings FGDs Asset paths
Editing maps
Provided you've set up the correct FGD and asset path(s), you are now set to edit the maps for your target mod. Here are some tasks that you should be able to master:
Moving entities
- Select an entity.
- Moving the entity can be done in three ways:
- Grab the axis handles and drag into position.
- Edit the
origin
property directly (e.g. for sub-unit positioning)
- Grab the entity by right click > Grab. The entity will move with you as you move around the map. Ungrab by right click > Ungrab.
- To edit the angles, edit the associated
angles
/angle
+pitch
properties.
- Most brush entities have their origins at the center of the map (0 0 0).
- Most brush entities don't rotate to face the value in
angles
but instead move in that direction e.g. func_button
and func_door
. See §Adding brushes at an angle for static rotation of brush entities.
Adding point entities
- Create > Entity.
- Right-click on the new entity > Properties.
- Click on class, and in the dropdown menu, select the target entity.
- Edit the entity as you would in Hammer/JACK.
Adding and sizing solid entities
- Create > BSP Solid Model
- Right-click on the new brush entity > Transform
- Set
Target: Object
and 3D Axes: Scale
- Grab the edge handle and drag it to the desired size. Repeat for all directions as required.
- Reset to
3D Axes: Move
. Close Transform window.
- Right-click on the new entity > Properties
- Click on class, and in the dropdown menu, select the target entity.
- Edit the entity as you would in Hammer/JACK.
Duplicate entities
Simply copy and paste them. Both point and solid entities are supported.
Duplicated solid entities inherit the original's model. It's the same effect as using
zhlt_usemodel
in regular mapping. It also means that decals on one will magically appear on all other copies.
To remedy this, you can duplicate the BSP model of solid entities. Duplicating a BSP model adds towards the BSP model count, so don't go wild on it.
This is also useful if the duplicated entity is in a different lighting condition from the original. You can then apply a different lightmap from the original BSP model. However do note that lightmap editing feature can be unstable.
Editing keyvalues
Editing keyvalues is the same as in Hammer or JACK. These notes might be useful:
- Be aware that compiler keyvalues e.g.
zhlt_*
& light values for light*
entities do nothing.
Fixing or changing textures on a face
There are a lot of cases where BSP files are shipped with misaligned textures. You can fix this pretty easily with newbspguy.
- Switch to face editing mode by clicking on the second icon on the top right corner.
- Click on a face.
- To change the texture, edit the texture name value. You should only use texture names from WADs that are referenced in
worldspawn
and probably have the WAD file open in a separate WAD viewer program for cross-reference.
- To reposition the texture, edit the X and/or Y shift values.
- To flip a texture, edit the scale value and change only the sign (i.e. between positive and negative).
- To rotate a texture, edit the angle values while having "Lock angles" checked.
- To skew a texture, turn off "Lock angles" and edit the X/Y angle values.
- Switch back to Object selection mode when you're done (first icon on top left).
Do not make edits to a face's scale value (other than the sign) since the face's lightmap is intrinsically tied to the texture scale, and any slight change can corrupt the lightmaps.
Adding a sign
- Create a solid model as shown earlier. Set the size to match the sign's size in pixels, and set the entity to be
func_wall
or func_illusionary
.
- Assign the texture to the front face of the entity.
- Shift the texture to fit the face. Oftentimes it'd be easier to scale the model to fit the texture instead, then move it back to the intended position.
- Uncheck "Special" flag. The face will turn black, because there is no lightmap data on it.
- Select the face of the wall behind the sign. Be sure to pick a face roughly the same size or slightly larger as the sign (preferably the latter) in both dimensions.
- Right click > Copy lightmap.
- Select the sign face. Right click > Paste lightmap.
Adding brushes at an angle
You can either start by duplicating existing brush models
*, or creating a new one for scratch. For the latter, follow the previous method of applying a texture and lightmap to all faces.
Now the issue of turning the solid entity at an angle... The hard way would be to jump into the vertex editing mode and moving the vertices into position, with a lot of math if you want to keep the scale. A simpler way would be to turn the brush into a
func_tank
and set its angle and origin
**. it'd face the given direction at spawn. The default spawn flags should be sufficient; obviously don't set it to active. A named
func_door
with
Starts open
flag could also work but you cannot see the orientation inside newbspguy.
* CAVEAT: Most brush entities have its origin at (0 0 0). You need to follow the instructions below to change the origin of rotation.
Brush entities rotate around their origin. For most entities these are set at the map origin (0 0 0). You'd want the origin to be close to the brush entity, if not at the exact center so that the rotation don't send it flying off. To change the origin:
- Right-click on the brush entity > Transform
- Set
Target: Origin
and 3D Axes: Move
. The handle might vanish at this point, so try look around the center of the map. Flying away helps since the handle's size is fixed on screen.
- Grab one of the axis handles and carefully drag it around. The brush entity might move the same amount. This is a graphical glitch. Deselecting the entity should snap it back to its original position. Select it again and repeat this process until the origin is where you want it to be (close enough to the entity).
- Reset to
Target: Object
. Close Transform window.
Brush entities marked INVALID SOLID can't have its origin adjusted
Common problems and solutions
Invalid FGDs
Fixed in the latest release: newbspguy now supports JACK's FGD format.
Newbspguy doesn't support J.A.C.K.s extended FGD. You need to use FGDs compatible with Hammer 3.x.
If you don't have one, make a copy and edit the FGDs with a text editor. The offending bits are the "in-editor help texts" usually at the end of a class or attribute entry.
Missing textures
If the loaded map is showing missing textures (purple/black checker pattern)...
- Open File > Settings > Asset paths
- Add the path of the mod your map is in. This includes the <mod>_addon and <mod>_downloads
- i.e. if the map is downloaded from a HLDM session and resides in
valve_downloads/maps/
, add valve_downloads/
.
- Click "Apply settings".
If missing texture problem persists, check that you have the required WADs in at least one of the asset paths. In other words, you should be able to load the map in game and not get missing textures there.
In rare cases, there could be a mismatch between the texture entry in the BSP and the texture entry in the WAD it references. This error can't be fixed with newbspguy, but usually the game engine will load the texture just fine in game.
New entities aren't behaving as it should
Level editors (not just [new]bspguy, but JACK as well) can have bugs where it doesn't load the default FGD values when creating new entities. You need to check that the required keyvalues are present in the Raw edit tab of the Properties window. Filling the text boxes in the Properties window, then blanking them, should coerce the program to add the associated keyvalues to the entities.
If the Raw edit tab shows the proper keyvalues yet you still have problems with the entities, then try:
- check for misspellings
- check the entity guide of the mod/game for specific known bugs
- recreate your setup in a fresh test map, compile, and test. If it works there you can in turn open that map in newbspguy and copy over the setup back to the target map.
Worldspawn model vertices can be edited using Face tool, but it not easy, and map crash if do big changes.
Whole map can be exported to .obj with all textures (for example convert part of map to .MDL to bypass BSP limits) but without lightmaps.
Also part of map can be compiled as func_wall and exported to .BSP model with working lightmap and collision to bypass BSP limits.
Also can export .wad from embedded textures. Import textures back to map. Can export files needed for HLRAD.exe for recompile lightmaps. Any bsp solid entity can be exported as .BSP model with working collision and lightmap.
New update has many bugfixes. Please create issue on GitHub if have problems or feature requests
This tutorial definitely needs a part 2 for the more advanced tasks e.g. vertex manipulation, bsp import/export, lightmap editing.
textures can be scaled as need because now lightmap can be recalculated after any changes.
Clipnodes can be edited only for non worldspawn models via Vertex Transformation widget.
As for vertex editing, I just don't recommend it at all too. It's way more cumbersome, and way way harder to get satisfactory result than to get completely broken result. I honestly don't know how a face whose vertices are non-planar would act inside the game. If there's face splitting ability in world faces and easier vertex shifting via handles vs textboxes then it might be feasible, but still not recommended because of the non-planar thing. *
The main takeaway is that just because it's doable with the BSP structure doesn't mean the game engine that's going to parse and run the BSP would like it. If the game engine doesn't like it, don't do it.
I still don't have any draft for a part 2 of this tutorial. 😅
EDIT 1:
* Tested face vertex editing. It messes up lightmaps elsewhere on a map.
aaalso added cull face feature
full VIS editor (but button create new leaf not works because still don't know how to add new leaf, game cant see new leaves)
fps optimization
updated cmd line options