Tutorial: Editing Compiled BSPs With newbspguy: The Basics Last edited 2 months ago2024-09-10 23:33:18 UTC

This tutorial will show you the basics of using newbspguy to edit already compiled BSP files. We will use UnrealKaraulov's fork of wootguy's bspguy due to the many added features, and the decoupling from a strictly Sven Co-op focused workflow of the original. We also assume that you have some experience in mapping the usual way, as there are differences when editing with this program, as noted later.

Preface

The workflow of editing already-compiled maps is totally different from making a map from scratch.

In the past, it was easier to just decompile a BSP back to .MAP, and start building up from there, because the typical modding workflow assumes you start from a map in an editor. However, there are several drawbacks to this method: However, as of 2023, there now exists tools that are capable of editing many aspects of already compiled BSPs directly, namely, bspguy and its fork newbspguy. The original tool was created exactly to port GoldSrc maps over to Sven Co-op, but both tools, especially the latter, can be used just as well to edit maps for any GoldSrc game/mod.
An invaluable study tool
[new]bspguy is invaluable as a study tool, to learn how level designers map their maps. For well known games/mods, it can reveal to you tricks mappers use to achieve certain effects with just clever map work and no code. For obscure games/mods, it shows you how custom entities are set up to implement core game mechanics; and you have a sample size of... the whole game as opposed to a few basic example map sources the developers might give you.

You no longer have to decompile a map and load them in VHE/JACK, or run the map in game with hooks like BunnymodXT to show invisible triggers, whilst enemies are shooting at you, to study it.

Pros, Cons, and Caveats

The advantages of editing BSPs in place are: The disadvantages are as follows: Do note of these differences for compiled BSPs when editing:
Important caveats
  • The program is unstable and can crash when doing certain random things. Save often.
  • Created BSP models immediately adds clipnodes and cannot be optimized. For already clipnode-strickened maps this could bring it over the threshold.
  • Don't use the merge command of the original program as that's specifically for Sven Co-op porting. Merging maps will easily exceed the engine limits of vanilla GoldSrc. Map merging is out of scope of this tutorial.
Anyway, let's get on with it.

Setting up newbspguy

  1. Download the latest release of newbspguy.
  2. Extract newbspguy into a folder.
  3. Run the program.
  4. Open File > Settings.
  5. Under General, click Reset All Settings.
  6. Under General, check Make map backup.
  7. Under General > Game directory, set to location of your Half-Life installation.
  8. Under FGDs, add the FGD of your target mod.
  9. Under Asset paths, add relative paths in the following format: <mod folder>/. There should be at least these entries:
    • valve/
    • <the target mod>/ (if that isn't valve)
  10. Click "Apply settings". Close the options window.
General settingsGeneral settings FGDsFGDs Asset pathsAsset paths

Editing maps

Provided you've set up the correct FGD and asset path(s), you are now set to edit the maps for your target mod. Here are some tasks that you should be able to master:

Moving entities

  1. Select an entity.
  2. Moving the entity can be done in three ways:
    • Grab the axis handles and drag into position.
    • Edit the origin property directly (e.g. for sub-unit positioning)
    • Grab the entity by right click > Grab. The entity will move with you as you move around the map. Ungrab by right click > Ungrab.
  3. To edit the angles, edit the associated angles/angle+pitch properties.
Brush entities, origin, and rotation
  • Most brush entities have their origins at the center of the map (0 0 0).
  • Most brush entities don't rotate to face the value in angles but instead move in that direction e.g. func_button and func_door. See §Adding brushes at an angle for static rotation of brush entities.

Adding point entities

  1. Create > Entity.
  2. Right-click on the new entity > Properties.
  3. Click on class, and in the dropdown menu, select the target entity.
  4. Edit the entity as you would in Hammer/JACK.
User posted image User posted image User posted image User posted image

Adding and sizing solid entities

  1. Create > BSP Solid Model
  2. Right-click on the new brush entity > Transform
  3. Set Target: Object and 3D Axes: Scale
  4. Grab the edge handle and drag it to the desired size. Repeat for all directions as required.
  5. Reset to 3D Axes: Move. Close Transform window.
  6. Right-click on the new entity > Properties
  7. Click on class, and in the dropdown menu, select the target entity.
  8. Edit the entity as you would in Hammer/JACK.
User posted image User posted image User posted image User posted image User posted image User posted image User posted image User posted image User posted image

Duplicate entities

Simply copy and paste them. Both point and solid entities are supported.

Duplicated solid entities inherit the original's model. It's the same effect as using zhlt_usemodel in regular mapping. It also means that decals on one will magically appear on all other copies.

To remedy this, you can duplicate the BSP model of solid entities. Duplicating a BSP model adds towards the BSP model count, so don't go wild on it.

This is also useful if the duplicated entity is in a different lighting condition from the original. You can then apply a different lightmap from the original BSP model. However do note that lightmap editing feature can be unstable.

User posted image User posted image User posted image

Editing keyvalues

Editing keyvalues is the same as in Hammer or JACK. These notes might be useful:

Fixing or changing textures on a face

There are a lot of cases where BSP files are shipped with misaligned textures. You can fix this pretty easily with newbspguy.
  1. Switch to face editing mode by clicking on the second icon on the top right corner.
  2. Click on a face.
  3. To change the texture, edit the texture name value. You should only use texture names from WADs that are referenced in worldspawn and probably have the WAD file open in a separate WAD viewer program for cross-reference.
  4. To reposition the texture, edit the X and/or Y shift values.
  5. To flip a texture, edit the scale value and change only the sign (i.e. between positive and negative).
  6. To rotate a texture, edit the angle values while having "Lock angles" checked.
  7. To skew a texture, turn off "Lock angles" and edit the X/Y angle values.
  8. Switch back to Object selection mode when you're done (first icon on top left).
Do not make edits to a face's scale value (other than the sign) since the face's lightmap is intrinsically tied to the texture scale, and any slight change can corrupt the lightmaps.
User posted image User posted image User posted image

Adding a sign

  1. Create a solid model as shown earlier. Set the size to match the sign's size in pixels, and set the entity to be func_wall or func_illusionary.
  2. Assign the texture to the front face of the entity.
  3. Shift the texture to fit the face. Oftentimes it'd be easier to scale the model to fit the texture instead, then move it back to the intended position.
  4. Uncheck "Special" flag. The face will turn black, because there is no lightmap data on it.
  5. Select the face of the wall behind the sign. Be sure to pick a face roughly the same size or slightly larger as the sign (preferably the latter) in both dimensions.
  6. Right click > Copy lightmap.
  7. Select the sign face. Right click > Paste lightmap.
User posted image User posted image User posted image User posted image User posted image User posted image

Adding brushes at an angle

You can either start by duplicating existing brush models*, or creating a new one for scratch. For the latter, follow the previous method of applying a texture and lightmap to all faces.

Now the issue of turning the solid entity at an angle... The hard way would be to jump into the vertex editing mode and moving the vertices into position, with a lot of math if you want to keep the scale. A simpler way would be to turn the brush into a func_tank and set its angle and origin**. it'd face the given direction at spawn. The default spawn flags should be sufficient; obviously don't set it to active. A named func_door with Starts open flag could also work but you cannot see the orientation inside newbspguy.

User posted image User posted image

* CAVEAT: Most brush entities have its origin at (0 0 0). You need to follow the instructions below to change the origin of rotation.
** Changing the origin of rotation
Brush entities rotate around their origin. For most entities these are set at the map origin (0 0 0). You'd want the origin to be close to the brush entity, if not at the exact center so that the rotation don't send it flying off. To change the origin:
  1. Right-click on the brush entity > Transform
  2. Set Target: Origin and 3D Axes: Move. The handle might vanish at this point, so try look around the center of the map. Flying away helps since the handle's size is fixed on screen.
  3. Grab one of the axis handles and carefully drag it around. The brush entity might move the same amount. This is a graphical glitch. Deselecting the entity should snap it back to its original position. Select it again and repeat this process until the origin is where you want it to be (close enough to the entity).
  4. Reset to Target: Object. Close Transform window.
Brush entities marked INVALID SOLID can't have its origin adjusted

Common problems and solutions

Invalid FGDs

Fixed in the latest release: newbspguy now supports JACK's FGD format.
Newbspguy doesn't support J.A.C.K.s extended FGD. You need to use FGDs compatible with Hammer 3.x.
If you don't have one, make a copy and edit the FGDs with a text editor. The offending bits are the "in-editor help texts" usually at the end of a class or attribute entry.

Missing textures

If the loaded map is showing missing textures (purple/black checker pattern)...
  1. Open File > Settings > Asset paths
  2. Add the path of the mod your map is in. This includes the <mod>_addon and <mod>_downloads
    • i.e. if the map is downloaded from a HLDM session and resides in valve_downloads/maps/, add valve_downloads/.
  3. Click "Apply settings".
If missing texture problem persists, check that you have the required WADs in at least one of the asset paths. In other words, you should be able to load the map in game and not get missing textures there.

In rare cases, there could be a mismatch between the texture entry in the BSP and the texture entry in the WAD it references. This error can't be fixed with newbspguy, but usually the game engine will load the texture just fine in game.

New entities aren't behaving as it should

Level editors (not just [new]bspguy, but JACK as well) can have bugs where it doesn't load the default FGD values when creating new entities. You need to check that the required keyvalues are present in the Raw edit tab of the Properties window. Filling the text boxes in the Properties window, then blanking them, should coerce the program to add the associated keyvalues to the entities.

If the Raw edit tab shows the proper keyvalues yet you still have problems with the entities, then try:

10 Comments

Commented 11 months ago2023-12-11 09:21:49 UTC Comment #105713
Now problems with parsing J.A.C.K fgd files solved.
Worldspawn model vertices can be edited using Face tool, but it not easy, and map crash if do big changes.
Whole map can be exported to .obj with all textures (for example convert part of map to .MDL to bypass BSP limits) but without lightmaps.
Also part of map can be compiled as func_wall and exported to .BSP model with working lightmap and collision to bypass BSP limits.

Also can export .wad from embedded textures. Import textures back to map. Can export files needed for HLRAD.exe for recompile lightmaps. Any bsp solid entity can be exported as .BSP model with working collision and lightmap.

New update has many bugfixes. Please create issue on GitHub if have problems or feature requests
Commented 11 months ago2023-12-11 09:23:04 UTC Comment #105714
And now it support languages using language.ini files.
Commented 11 months ago2023-12-18 21:37:16 UTC Comment #105764
^ Thanks for the plentiful added features! 😁

This tutorial definitely needs a part 2 for the more advanced tasks e.g. vertex manipulation, bsp import/export, lightmap editing.
Commented 10 months ago2023-12-23 09:26:55 UTC Comment #105780
The disadvantages are as follows:
You cannot edit world brushes whatsoever.
You cannot do crazy texture scaling since it's tied to the lightmap size.
You cannot edit clip nodes.
in newbspguy world vertices can be edited via face tool (but without collision changes).
textures can be scaled as need because now lightmap can be recalculated after any changes.
Clipnodes can be edited only for non worldspawn models via Vertex Transformation widget.
Commented 10 months ago2023-12-24 16:42:33 UTC Comment #105783
Great tool! Especially when map sources were lost. I with texture and lightmap editing was easier. E.g. making textures "aligned to face" or scaling to -1. For lightmaps I see there's much planned ahead. Like exporting multiple faces, or adding flood fills. I'm looking forward to see more features in the future! Great work and especially nice to see small, fast and native code!
Commented 10 months ago2023-12-24 22:52:25 UTC Comment #105784
You can select and export multiple faces lightmap, or all lightmaps atlas. For filling flood I can't understand
Commented 10 months ago2023-12-25 16:44:55 UTC Comment #105787
night builds can be unstable, check older versions in releases if one not working.
Commented 10 months ago2024-01-01 07:44:43 UTC Comment #105827
@karaulov
textures can be scaled as need because now lightmap can be recalculated after any changes.
You still need to be careful because the maximum lightmap size for a face is apparently 16x16 luxels. A lot of faces you'll find in maps are already at 16 luxels on either (or both) sides. GoldSrc doesn't support lightmaps larger than that size, and scaling down a texture that results in a face with lightmaps over that size crashes the engine. (IDK if it's supported on svengine). Hence my verdict is don't do it.

As for vertex editing, I just don't recommend it at all too. It's way more cumbersome, and way way harder to get satisfactory result than to get completely broken result. I honestly don't know how a face whose vertices are non-planar would act inside the game. If there's face splitting ability in world faces and easier vertex shifting via handles vs textboxes then it might be feasible, but still not recommended because of the non-planar thing. *

The main takeaway is that just because it's doable with the BSP structure doesn't mean the game engine that's going to parse and run the BSP would like it. If the game engine doesn't like it, don't do it.
I still don't have any draft for a part 2 of this tutorial. 😅
EDIT 1:
* Tested face vertex editing. It messes up lightmaps elsewhere on a map.
Commented 9 months ago2024-01-25 21:06:47 UTC Comment #105928
added bugfixes, now lightmap resize works
aaalso added cull face feature
full VIS editor (but button create new leaf not works because still don't know how to add new leaf, game cant see new leaves)
fps optimization
updated cmd line options
Commented 9 months ago2024-01-26 05:31:51 UTC Comment #105932
And new feature added : map can be scaled in one click. (For example possible make map bigger in 2x size, etc)

You must log in to post a comment. You can login or register a new account.