Progress update:
Custom entity data parsing and entity instantiation
I've figured out a way to take control of entity data parsing as well as entity instantiation. These are two things that will go a long way to making mods more secure and allows for new features to be added.
Entity data parsing is the process by which the entity data in a BSP file is converted into actual entities. This code is now in the server dll which means you can have more control over it.
Entity instantiation is the process of creating the entity that the classname refers to. Having this in the server dll allows for more control and improves security, because it
doesn't involve calling arbitrary functions based on untrusted input.
Potential uses of custom entity data parsing include implementing
point_template, loading default keyvalues from a file, changing the entity data string to another format like JSON (albeit with some restrictions, see below) and adding input/output support.
Changing the data format requires the map compiler to be changed as well. Features like I/O support require map editor support as well and requires modifications to be made to the .map format since it doesn't have the flexibility needed to specify that kind of data (if you need to change it, you may as well switch to a format like JSON).
Potential uses of custom entity instantiation include marking entities as internal so they can't be spawned by mappers (for safety purposes), "synthetic" entities defined by config files instead of in code and custom entities defined in scripts (which you wouldn't be able to restore otherwise).
Here's an example of a synthetic entity config file:
{
"BaseEntityName": "ammo_generic",
"EntityName": "ammo_9mmclip",
"EntityNameAliases": [
"ammo_glockclip"
],
"KeyValues": [
{
"Key": "model",
"Value": "models/w_9mmclip.mdl"
},
{
"Key": "ammo_amount",
"Value": 17
},
{
"Key": "ammo_name",
"Value": "9mm"
}
]
}
This is part of the proposal i wrote, it hasn't been implemented yet. Ammo entities can be described in terms of
ammo_generic
entities, so this moves a bunch of code into user-editable config files.
Combined with per-map config files that can override default files you can control what each entity turns into.
The next step after this is implementing reflection and replacing the old save data with that.
Technical information
The entity data string typically looks like this:
{
"wad" "\quiver\valve\halflife.wad;\quiver\valve\decals.wad;\quiver\valve\xeno.wad;\quiver\valve\sample.wad"
"chaptertitle" "C0A1TITLE"
"message" "Anomalous Materials"
"classname" "worldspawn"
}
{
"origin" "-424 280 -160"
"message" "ambience/crtnoise.wav"
"health" "2"
"spawnflags" "2"
"classname" "ambient_generic"
}
Entity data parsing is normally done in the engine, but there is an edge case that allows you to fool the engine into thinking it's already done parsing.
The first call into the server dll done by the engine during map loading is a call to
DispatchKeyValue.
This keyvalue is always the
classname
key for
worldspawn
, the first entity in any map. Though it is possible for the classname to differ, this is forced back to
worldspawn
for security purposes.
At this point we can assert that the engine's parser is currently positioned right after the first
{
in the entity data string.
We can obtain a pointer to this string by first acquiring the
server_t
instance. This is possible by taking the
string_t mapname
member in
globalvars_t
and getting the pointer to the name using
STRING()
. This is the address of the
name
member in
server_t
. Subtracting the offset of that member in
server_t
gets a pointer to
server_t
.
server_t
's
worldmodel
member is the BSP model, which contains a member
entities
. This is the entity data string.
Modifying it at this point to become
{}
will cause the engine to stop parsing the string. It will still try to call the
worldspawn
function which is an empty function. If it doesn't exist the engine will remove the world entity, so it has to be there.
The server will parse the entity data string and handle creation of all entities by itself. This behavior is currently identical to the engine, except it handles errors more gracefully. It won't stop parsing when it encounters bad data, instead it will try to skip past it to continue. This will likely fail since corrupted entity data is probably missing too much information.
This covers everything that happens when loading a new map. However entities are also created when loading a save game, so this also had to be handled.
When a save game is being loaded the first call into the server dll is a call to
SaveReadFields to read in the data contained in a save game.
It will load all of the entity data stored in the save game, create all entities and then initializes them. If the save game is being loaded because the player is going through a changelevel then entities from adjacent maps that were marked for transition are also created from separate save game files.
The engine creates entities by using the
classname stored in the save game's entity table block.
It then restores the entity by calling
DispatchRestore.
This behavior is overridden by changing the classname to
custom
in
DispatchSave.
The classname is also forced to
custom
in
SaveReadFields
for every
ETABLE
read in.
The engine will call the function
custom
for every entity it wants to create. This function is empty just like
worldspawn
and does not actually create anything. The name
custom
is special because the entity data parsing code in the engine also uses it to spawn entities it can't find a function for. This serves as a fallback to make sure that all works properly, but it should never be needed for that.
When
DispatchRestore
is called the server checks if it has previously seen a particular
SAVERESTOREDATA
instance before (identified by the map it was made for, and the landmark name associated with it). If it hasn't, it creates all of the entities that would have been created by the engine for that save game data.
The actual classname is stored in each entity's save data so it can be recreated from that.
This is needed because when entities are restored they also have to set up connections to other entities. If the entities don't exist yet this will fail and break things.
The result is the server dll handles creation of all entities by itself. The engine function
CREATE_NAMED_ENTITY
is also handled by the server dll so the engine does not create entities anymore, it only thinks it does.
Changing the entity data string format
The entity data string format can be changed to something else since it's parsed by the server dll.
The only restriction is that the string must start with data the engine can understand since it still parses that.
For example here's a version that uses JSON:
{
"classname" "worldspawn"
}
{
"Entities": [
{
"ClassName": "worldspawn",
"KeyValues": [
{
"Key": "wad",
"Value": "\quiver\valve\halflife.wad;\quiver\valve\decals.wad;\quiver\valve\xeno.wad;\quiver\valve\sample.wad"
}
]
}
]
}
Other stuff
I've found a program that can be used to edit JSON files:
https://github.com/json-editor/json-editorThis'll make editing config files a lot easier.