Enhanced Half-Life (custom SDK/mod) Created 2 years ago2021-04-28 17:36:55 UTC by Solokiller Solokiller

Created 2 years ago2021-04-28 17:36:55 UTC by Solokiller Solokiller

Posted 2 years ago2021-04-28 17:36:55 UTC Post #345569
Enhanced Half-Life is a custom SDK/mod designed to replace my old Half-Life Enhanced SDK/mod.

It's not ready for a public release and the code is still in flux (400+ commits of refactoring and cleanup on top of Half-Life Updated) so i'm keeping that private for now.

The purpose of EHL is to do the following:
  • Provide an updated SDK that uses modern source control principles, including a proper directory structure and use of CMake to generate the project files
  • Third party dependencies are included instead of requiring manual setup by programmers (i'm looking at you, HLE's XercesC dependency)
  • Ease of use: can make a new mod in ~5 minutes by cloning the repository, configuring the CMake project to install to a mod directory, generating the files and building the INSTALL target. Also installs liblist.gam and delta.lst (part of the source code repository) to ensure smooth setup
  • Uses C++20, requires Visual Studio 2019 or newer. No Windows XP support.
  • Cleaned up codebase to ensure source code is the highest quality:
    • Obsolete files and APIs have been removed
    • Documentation has been upgraded to use Doxygen style annotations
    • Include guards have been converted to #pragma once and headers now always use #pragma once unless they need to be included multiple times
    • #define has been mostly replaced with constexpr constants
    • Global variables removed whenever possible
    • Duplicate code merged
    • Cleaned up C-style code to use C++ style, use const correctness, Vector instead of float[3]
    • Fixed old hacks needed because of backwards compatibility
    • Routed all filesystem access through IFileSystem, all references to the game directory name removed
    • Safer strings functions used to avoid buffer overflows
    • string_view used to make for better code (works well with non-terminated strings and substrings)
    • Defined constants for many magic numbers
    • Converted many constants to enum class which helps to catch invalid conversions
    • Lots of other stuff that's still in progress
  • The overall goal is to make the smallest SDK possible while maintaining full functionality, and even expanding on features by using newer language/library features, and using code generation to help eliminate redundancy in things like save game code
  • All of HLEnhanced features should eventually be re-implemented in this project, along with merging the unique features from Opposing Force Updated (weapons, NPCs, game modes) and Blue Shift Updated (handful of entities). This should make EHL the one-stop-shop for making a mod that mimics any of these 3 games, aside from the issue of Op4 weapons having grunt hands only
  • The UI should be replaced entirely with a VGUI2 version. The code for that exists in HLEnhanced and will be ported over later on when i tackle that task. Eliminating the use of VGUI1 entirely will help to simplify things.
  • All files loaded by the game codebase are to be converted to XML for consistency and to eliminate issues with invalid parsing, error handling, memory leaks, etc. I considered a few different formats (INI, TOML, JSON, and others) but XML is the format that provides everything needed, though i haven't yet started upgrading the codebase to use it yet. I'm planning to use RapidXML for parsing so the heavyweight XercesC dependency won't be a problem this time
  • Backwards compatibility with existing maps/models will likely be broken to improve entity keyvalue APIs and internal logic. For example some models made for barney and scientist NPCs will likely break due to changes in how submodels are changed (now done correctly). This will help to remove some of the cruft from the codebase that was needed due to Half-Life's maps having older keyvalues still in use. Breaking compatibility now in a big way can help avoid breaking changes later on
So far 53 files have been removed (not counting Ricochet and DMC files which have also been removed) and every file has been moved at least once. Engine headers have had internal engine stuff removed, lots of legacy stuff has been tossed out like the old security module stuff that the client used to have.
Compare this: https://github.com/ValveSoftware/halflife/blob/c7240b965743a53a29491dd49320c88eecf6257b/engine/APIProxy.h
To this: https://i.imgur.com/FVeCt0n.png

(that header was split in 2 so the client API part is only included by files that need to know about it)

I'd like to show something i finished up today. I improved the health and hev chargers:
video
Features:
  • Fixed dmdelay keyvalue not working and renamed it to recharge_delay, and allowed the use of floating point values
  • Allows instant recharging (warning: sound spam)
  • Allows mappers to control initial capacity, total capacity, charge per use and sounds to play
  • Allows chargers to start out of charge (initial capacity of 0)
  • Allows infinite capacity (initial/total capacity of -1)
  • Allows changing charge interval(time between charge uses, basically how quickly it gives you health or armor)
  • Prevents chargers from jamming up if they are constantly +used while recharging
  • Custom sounds
  • Sound to play on recharge (similar to sound played when items respawn in multiplayer)
  • Custom sound pitch setting (100 == default, 0 == minimum, 255 == maximum)
Keyvalues:

recharge_delay(string) : "Recharge delay"
charge_per_use(integer) : "Charge per use"
charge_interval(string) : "Interval between charges"
initial_capacity(integer) : "Initial charger capacity (-1 == unlimited)"
total_capacity(integer) : "Total charger capacity (-1 == unlimited)"
sound_pitch(integer) : "Sound pitch"
charge_on_sound(sound) : "Charge start sound"
charge_loop_sound(sound) : "Charge loop sound"
refuse_charge_sound(sound) : "Charge refuse sound"
recharge_sound(sound) : "Recharge sound"


Potential uses:
Making chargers that never run out and/or give tons of health/armor in an instant. Useful for co-op spawn areas, bunkers, etc.
Making chargers that start off empty but recharge after a long time to have some charge (or unlimited).
Making chargers that have little charge, but recharge quickly.
Chargers that look and sound alien.
Silent chargers (use common/null.wav).
Making chargers that give players their entire charge at once, or however much the player can take (health always expends charge_per_use, armor is conservative and only gives what the player can take). Like Condition Zero Deleted Scenes health stations.
Making chargers that charge really slowly, but charge a lot (forces player to expose themselves).

I'm still working on code cleanup so new features are few and far between. I decided to update the chargers because i was merging the code for both and i wound up adding a bunch of stuff.

Feedback and ideas are always welcome.
Posted 2 years ago2021-04-28 19:22:41 UTC Post #345570
This is very interesting, looking forward to it! Keep up the good work :)

I have a few questions:
  • The overall goal is to make the smallest SDK possible while maintaining full functionality, and even expanding on features by using newer language/library features, and using code generation to help eliminate redundancy in things like save game code
  • All of HLEnhanced features should eventually be re-implemented in this project, along with merging the unique features from Opposing Force Updated (weapons, NPCs, game modes) and Blue Shift Updated (handful of entities). This should make EHL the one-stop-shop for making a mod that mimics any of these 3 games, aside from the issue of Op4 weapons having grunt hands only
Are you going to provide the ability to have these extra features toggle-able through preprocessor defines (like HLEnhanced's USE_OPFOR) or separate repositories (like Half-Life Updated) to get a "HL SDK but just cleaned up"? I know this would require a lot of work but that would be nice for total conversions which do not rely or very little on existing entities.
The UI should be replaced entirely with a VGUI2 version. The code for that exists in HLEnhanced and will be ported over later on when i tackle that task. Eliminating the use of VGUI1 entirely will help to simplify things.
By "the UI should be replaced entirely with a VGUI2 version", I'm guessing you are talking about all the existing VGUI elements (class menu, team menu, scoreboard, observer's control panel...) or are you extending this to other UI stuff (HUD based menu, ammo, crosshair, health...)?

I have an idea: a few days or weeks ago, I was talking with some other GoldSrc programmers about skill CVARs and how it's tedious to add them. As you know already, when you add a skill CVAR in the game code (not talking about updating skill.cfg), you need to add a float variable to the skilldata_t structure, create the 3 CVARs themselves (cvar_t), register them (CVAR_REGISTER), "link" the variable and the CVARs using GetSkillCvar and optionally provide a multiplayer override. This is a pain in the ass because:
  • If you want to add a 4th difficulty, you need to manually add a 4th CVAR and register it for every set of skill CVARs and don't forget to update the "hardcoded clamp".
  • Prone to human mistakes in my opinion, especially for beginners, also known as the "did I updated the number correctly in that cvar_t->name syndrome?"
I was wondering if it would be possible to simplify all of these tedious things by having some kind of CSkillCvar class that would create the appropriate number of cvar_t, register them automatically?
Posted 2 years ago2021-04-28 19:57:35 UTC Post #345571
Are you going to provide the ability to have these extra features toggle-able through preprocessor defines (like HLEnhanced's USE_OPFOR) or separate repositories (like Half-Life Updated) to get a "HL SDK but just cleaned up"? I know this would require a lot of work but that would be nice for total conversions which do not rely or very little on existing entities.
Most of the stuff unique to Op4 and Blue Shift is opt-in for level designers and useful regardless so i don't see much point in making them conditionally included. On the programming side of things you can usually exclude these by removing the files from the CMakeLists files. I'm hoping to decouple the code so there aren't any hardcoded references to other entities like Op4's code currently does (e.g. player TakeDamage references allied grunts). Only the weapons have always-precached assets so maybe a way to eliminate that overhead is better than conditional inclusion.
By "the UI should be replaced entirely with a VGUI2 version", I'm guessing you are talking about all the existing VGUI elements (class menu, team menu, scoreboard, observer's control panel...) or are you extending this to other UI stuff (HUD based menu, ammo, crosshair, health...)?
Well i've tossed the class and team menus out of the codebase since they're TFC-specific, but i want to implement VGUI2-based versions of all of them. Ideally even the Hud should be VGUI2, that means converting the hud sprites to TGA which i call an improvement (no blurring when upscaling i hope).

Having only one UI for everything makes it easier to make new content for things. No more "is it a hud sprite, a VGUI1 TGA or a VGUI2 TGA?".
I was wondering if it would be possible to simplify all of these tedious things by having some kind of CSkillCvar class that would create the appropriate number of cvar_t, register them automatically?
Yeah i was thinking about that too. I'm thinking i'll just chuck skill.cfg out the window and use an XML file:
<Skill>
    <SkillValue name="sk_9mmAR_bullet" level="2" value="4"/>
</Skill>
No more cvars, just strings:

pEntity->TraceAttack(pevAttacker, SkillValue("sk_9mmAR_bullet"), vecDir, &tr, DMG_BULLET);

And then all of the variables are handled like so (pseudocode):
struct SkillValues
{
    std::array<std::optional<float>, SkillLevelCount> Values;
};

class SkillData
{
public:
    void LoadFromFile(const char* fileName)
    {
        auto xml = LoadXML(fileName);

        for (auto skillValue : xml.SkillValues)
        {
            auto it = _skillValues.find(skillValue.Name);

            if (it == _skillValues.end())
            {
                it = _skillValues.emplace(skillValue.Name).first;
            }

            it->Values[skillValue.Level] = atof(skillValue.Value);
        }
    }

    float GetSkillValue(std::string_view name, float defaultValue = 0) const
    {
        if (auto it = _skillValues.find(name); it != _skillValues.end())
        {
            return it->second.Values[_level].value_or(defaultValue);
        }

        return defaultValue;
    }

private:
    std::unordered_map<std::string, SkillValues> _skillValues;
    SkillLevel _level;
};
(That's actually most of the code needed for this)

Just have a CBaseEntity member to forward to the global and you're done.

This scales with any number of skill levels (just update the SkillLevelCount constant), any new skill values are automagically loaded in. There is still the skill level clamping being done but it's trivial to update that. The biggest problem is the hardcoded "new game" dialog, but i'm sure there are ways around that too.
Posted 2 years ago2021-04-28 20:01:07 UTC Post #345572
Looks great. I have some suggestions.
  1. Provide access to UI interfaces: IBaseUI, IGameUI, IGameUIFuncs, IGameConsole, IEngineVGui, etc. (an ability to manipulate the UI without accessing the engine / GameUI).
  2. Add missing interfaces inside server dll: NEW_DLL_FUNCTIONS / SV_SAVEGAMECOMMENT.
  3. Make a new generic mathlib that can suit all solution projects. You mentioned adding vgui2 support to client.dll, which is great. But putting it all in there is a pain in the ass. For example vgui2_controls.lib uses mathlib.h from Source SDK which also includes its own vector.h header file, for client and server we already have vector.h and util_vector.h and they will all hate each other. This could also be related to the tier0's platform.h, which will conflict with Platform.h from vanilla SDK. Creating common header files will not confuse people and take the project one step further.
I have many more suggestions (for example hacking the engine data like server_t / server_static_t etc), but I need to see what will be added next, so I will not suggest anything that could potentially go beyond the scope of the project. Best of luck with it.
Posted 2 years ago2021-04-28 20:29:46 UTC Post #345573
Somebody on Knockout requested this:

New keyvalues:
fire_on_recharge(target_destination) : "Fire on recharge"
fire_on_empty(target_destination) : "Fire on empty"

spawnflags(flags) =
[
    1 : "Fire on spawn" : 0 //If set, fire_on_recharge or fire_on_empty will be triggered on map spawn depending on initial capacity
]
I made it so "fire_on_recharge" sends a USE_ON and "fire_on_empty" sends a USE_OFF so you can directly enable and disable the sprite just by triggering it. Otherwise you can use a relay or multi_manager to handle it. The spawnflag is used to have the appropriate target triggered when the map starts. I made it opt-in so you don't have entities getting triggered by default, and you may want to set things up manually. But for simple cases this automates the work.
Provide access to UI interfaces: IBaseUI, IGameUI, IGameUIFuncs, IGameConsole, IEngineVGui, etc. (an ability to manipulate the UI without accessing the engine / GameUI).
Those are engine interfaces. They're also part of the VGUI2 API.
Add missing interfaces inside server dll: NEW_DLL_FUNCTIONS / SV_SAVEGAMECOMMENT.
Yeah i'll get to those sooner or later.
Make a new generic mathlib that can suit all solution projects. You mentioned adding vgui2 support to client.dll, which is great. But putting it all in there is a pain in the ass. For example vgui2_controls.lib uses mathlib.h from Source SDK which also includes its own vector.h header file, for client and server we already have vector.h and util_vector.h and they will all hate each other. This could also be related to the tier0's platform.h, which will conflict with Platform.h from vanilla SDK. Creating common header files will not confuse people and take the project one step further.
I know there will be conflicts, i'll be merging that stuff in as needed when i get to it. I've already cleaned up most of the duplicate stuff in the Half-Life SDK so the Source SDK won't be too much trouble when i get to it. I've even gotten rid of the Windows.h dependency messing up every file, it's only used in a few files now.
Posted 2 years ago2021-05-06 19:15:57 UTC Post #345589
Minor progress update:
  • I've implemented all of NEW_DLL_FUNCTIONS and provided a default implementation of SV_SaveGameComment with the hardcoded list of title names.
  • I've refactored the codebase so all references to entities use CBaseEntity. Only engine<->game calls deal with edict_t and entvars_t pointers directly now. I hope to eliminate as many of those as i can to streamline things.
  • virtual CBaseEntity methods that take a lot of parameters now use parameter structs. To use the above example for using skill values: pEntity->TraceAttack({attacker, SkillValue("sk_9mmAR_bullet"), vecDir, tr, DMG_BULLET});
    • On the callee's side: void TraceAttack(const TraceAttackInfo& info) override;
    • The purpose of this is to allow for changes to made to the data being passed. I've already adjusted KilledInfo to also include the inflictor so the global used in CBasePlayer::Killed is gone now.
  • Persistent pointers to entities are all EHandles now. Previously entities like the Alien Slave maintained raw pointers to effects entities. If those entities were to be removed for any reason the game would crash if they were accessed.
  • EHandles are now type-safe. You can use EHandle<class deriving from CBaseEntity> to specify a more specific type to use, which eliminates casting. I've also added methods to get the pointer to eliminate the need to cast to CBaseEntity, and a method to call UTIL_Remove on it if it's valid to simplify cleanup.
  • Entities that create effects entities now also remove them when killtargeted or otherwise removed (UTIL_Remove). This does not happen if the entity is destroyed through other means (anything that doesn't call UpdateOnRemove but does call the destructor) to prevent edge cases that could break things (e.g. removing entities when the map is changing, where the engine already does that and doesn't change the serialnumber on edicts).
  • The Alien Slave no longer creates beam entities that can be orphaned if the slave is caught in a transition volume during a level change. See https://github.com/ValveSoftware/halflife/issues/3104 for more information.
  • Fixed ~500 warnings that the V142 toolset reports concerning arithmetic overflow (casting float to double after performing arithmetic on them, nothing buggy in practice), uninitialized class and local variables and null pointer access.
  • Added precompiled headers to the client and server projects. These dramatically speed up compile times, i can compile both in 10 seconds flat. This also seems to reduce memory usage and the size of the Intellisense database files (~5 GB -> ~1.5 GB).
  • Simplified CMake setup even more. The game directory is now extracted from the CMake install prefix just like the mod directory which reduces the amount of required variables to just one, and allows it to work on all platforms instead of requiring manual configuration on non-Windows platforms (which used the registry value for the Half-Life directory). This also means mods targeting other engines will be set up more quickly, but no support is provided for other engines regardless.
  • Simplified multiplayer logging. Code that uses UTIL_LogPrintf in the SDK tends to follow the same pattern (if teamplay log this, else log that). Now gamerules handles this with a special logging function. Only one use requires this check due to logging player kills which requires logging 2 players worth of data.
  • PLAYBACK_EVENT_FULL calls have been simplified by using parameter structs and designated initializers to eliminate the need to specify unused variables: UTIL_PlaybackEvent(flags, m_hPlayer, m_usMP5, {.fparam1 = vecDir.x, .fparam2 = vecDir.y});
  • Added ent_remove developer cheat command: calls UTIL_Remove on the entity the player is looking at. Intended to be used to remove obstructions and unwanted entities, and to test what happens when an entity is removed at a specific point in its lifetime (e.g. while attacking, creating effects, etc).
  • Any math functions that exist in the codebase have been moved to mathlib.h/.cpp to help keep things centralized.
I've also been thinking about configuration file support some more to work out how to handle it. I chose XML because it allows for structured data and comments, but given how often people edit these without knowing how they actually work i want to make a front-end that makes it easier by doing as much work as possible for the user.

However, a front-end would load and save these files which would wipe out the comments anyway, so that advantage is negated. As such JSON is the better option since it's smaller in size, simpler to read, edit and parse and the parsers available support the entire format specification (RapidXML only supports a subset of XML).

I figure any comments that are needed can be embedded using a comment value that gets ignored during parsing. This comment can be shown as an editable field in the front-end which solves the problem of comments being wiped out by loading and saving.

The best C++ library for JSON parsing is nlohmann JSON which like RapidXML is a single header file (or a few headers depending on how you configure it). The advantage that this has over RapidXML is that it uses C++11 syntax.

This makes it easier to use, for example:
for (xml_attribute<> *attr = node->first_attribute();
     attr; attr = attr->next_attribute())
{
//Operate on node attributes
}
Versus:
for (auto& element : j)
{
//Operate on array element
}
So for the programmer this library is easier to use than RapidXML.

I was thinking of making this front-end complete configuration-driven, meaning it lets you specify a list of JSON files that describe each configuration file format (skill.json, listenserver.json, etc) so that it can construct an appropriate UI for each format. That would certainly simplify things for the user.

Beyond that i'm also thinking about how i can bypass the engine for certain things, like entity data string parsing, save/restore and other things. The more engine features i can take control of the easier things will become.

Most of the code cleanup stuff is done now so i can focus on upgrading things some more.
Posted 2 years ago2021-05-17 15:24:35 UTC Post #345627
I've partially implemented a code generator to replace the SDK's save game code.

This is how you have to define information about entity variables that need saving in the SDK:
User posted image
This is how it's done now:
User posted image
This generates the same code:
User posted image
All variables marked with the Persisted attribute will be added to the save game data. The Type attribute can be used to signal that a variable needs specific support like saving it as a time or position variable.

Once the codebase has been updated to use this i can rework how the save game system works to make this more flexible, for example supporting std::vector out of the box, enabling lookup of functions without having to go through the engine (which relies on platform-specific behavior to work) and more.

You can also see some other work i've been doing in these screenshots: i've split up cbase.h into separate files for each class, each header now includes their dependencies as well. Logic that used to be in Restore is now in a separate method called PostRestore, called if Restore returned true.

I've also been reworking the items code (healthkit, battery, ammo, weapons) but more on that later.
Posted 2 years ago2021-06-06 21:05:43 UTC Post #345682

Progress update:

Item updates

item updates
Item entities have been updated with new features. When i refer to items i mean entities classified as one of these types:
  • Pickup items (health kit, battery, suit, long jump, antidote)
  • Ammo items
  • Weapons
For all features they default to the game mode's defaults if not specified.

New features:
  • Item respawn behavior can be overridden. It can be set to always respawn (like in multiplayer) or never respawn (like in singleplayer).
  • Respawn delay can be overridden. Anywhere from instant (delay 0) to some time in the future that could be seconds, minutes, hours, etc.
  • Respawn position can be overridden. By default it respawns where it was when picked up, but using the "Original" respawn position it will respawn where it was placed. This matters when the item is placed above or on top of another entity that may not always be there, like a func_wall_toggle. When using "Current" position (the default) it will respawn below the other entity, when using "Original" it will respawn on top of the entity. Very useful for hidden items like a weapon hidden in a toggle-able ceiling.
  • The method used to make the item fall to the ground can be changed:
    • PlaceOnGround: The item is placed on the ground by making the engine drop it to the floor. If the item falls through the world (e.g. clipping into a brush) it will be removed.
    • Fall: The item falls to the ground by applying gravity.
    • Float: The item floats in the air and does not drop. Unlike the other two modes the item's pitch and roll are not zeroed, which allows the item to be posed as desired.
    • Pickup items use PlaceOnGround, ammo and weapons use Fall (carried over from vanilla).
  • The ability to pick up items that are falling can be controlled. By default "Fall" mode items cannot be picked up until they have fallen on the ground. If the item is also respawning in the air using "Original" position mode this can make a difference.
  • The item "clatter" mode can be overridden. This is the sound played when an item bounces on the ground. Normally it only plays when dropped by a player, this setting allows it to always or never play this sound.
    • The clatter sound can also be changed.
  • The item can be set to stay visible during the respawn delay. Normally it is made invisible until it respawns, this allows it to remain visible, useful for e.g. armories where players can see the items while they're respawning.
  • The item can be set to flash on respawn. This is enabled by default and causes a muzzleflash effect to be applied to any nearby studio model entities. It can be disabled to make a respawning item less conspicuous.
  • The respawn sound can be changed. Setting it to a sound like common/null.wav effectively disables it.
  • Two new trigger targets have been added to trigger something when an entity respawns (aka materializes) and when it has been picked up and becomes invisible (aka dematerializes). Seen in the video when the isotope ammo box toggles the green sprite on and off.
  • All items can have their world model changed to a custom one.
Now the features specific to each type of item.

Pickup items:
  • Health kits:
    • Health kits have a custom capacity option now. Any positive value including 0 (e.g. fake healthkit) can be used.
  • Batteries:
    • Batteries have a custom capacity option now. Any positive value including 0 (e.g. fake battery) can be used.
  • HEV Suit:
    • The "Short Logon" spawnflag has been changed to a keyvalue to control the type of logon. Available options are:
      • No Logon: No message is played, this is now the default.
      • Long logon. The entire logon message is played, this was the previous default.
      • Short logon. The first part of the logon message is played.
  • Long jump:
    • The pickup sentence can be disabled.
Ammo items:
  • The amount of ammo given can be changed. One "unit" of ammo is one of whatever the ammo type is. One unit of 9mm ammo is one 9mm bullet, one unit of RPG ammo is one rocket, etc. The amount can be 0 (e.g. fake ammo).
  • The ammo pickup sound can be changed.
  • New ammo entity: ammo_generic. This ammo entity lets you specify the name of the ammo to give, and does not set a model by default. Useful for invisible ammo items. The ammo names used are internal names, which is something that will be changed later on.
  • New ammo entity: ammo_all. This ammo entity gives the player all of the ammo types known to the game. The amount of units of ammo to give can be set, including a special value of -1 which will give the maximum amount. Useful for quickly topping off a player's ammo.
Weapon items:
  • The default primary ammo amount can be changed. Can be 0 to give the player an empty weapon. If the hornet gun is given with no ammo it will not regenerate ammo since it has to be switched to first to do so. Useful for giving a fully loaded MP5. Does not apply to MP5 grenades.
  • The weapon pickup rule can be changed:
    • Default: you can pick up the weapon if you don't have the maximum amount of ammo for this weapon, or if you don't have the weapon.
    • Always: you can always pick up the weapon, even if you wouldn't get anything out of it. Useful for making sure a player has a weapon without leaving them extra ammo.
    • Never: you can never pick up the weapon. Basically turns the weapon into a prop, there are probably better and cheaper ways to do that though.
    • NoDuplicates: you can pick up the weapon if you don't have it, but you can't pick it up otherwise. Useful for weapons that should only be picked up if the player doesn't have it, without giving them any ammo, while leaving the weapon lying around. Potentially very useful for making maps where players can lose their weapons and can backtrack to restock on missed weapons, as well as multiplayer maps where you don't want players hoarding weapons.
  • Weapons will now also retain their sequence value on respawn. The Crossbow and Gauss gun models in particular have a second "on side" animation that avoids clipping through the floor, this lets you set it and keep it after it's been picked up once.
The video shows some of these new features in action.
Posted 2 years ago2021-06-06 21:06:20 UTC Post #345683

Class hierarchy updates

Part of the item updates also involved refactoring the class hierarchies. All items now derive from a new CBaseItem class that controls shared behavior such as pickup and respawn logic. The CItem class is almost empty, the CBasePlayerAmmo class has been stripped of most of its code and the CBasePlayerItem class's code has also been stripped down since all 3 had very similar code.

Further entity class refactoring summarized as follows:
  • CBaseDelay delayed triggering has been merged into CBaseEntity, giving all entities the ability to delay triggering of the "target" keyvalue, but only if the entity uses SUB_UseTargets for this. This eliminates the use of a non-virtual override which could cause inconsistent behavior.
    • What remains of CBaseDelay has been reworked into CDelayedUse, which handles the delay triggering logic as before. To be removed when delay triggering is moved to a dedicated list of queued events.
    • Master entity string management has been moved from CBaseToggle to CBaseEntity. Duplicate master entity strings have been removed from other classes. This eliminates the use of a non-virtual override which could cause inconsistent behavior.
  • Door-specific toggle state functionality has been cleaned up to eliminate the presence of a useless virtual function. SetToggleState was never used and has been removed.
  • Entity-specific members originally stored in CBaseToggle have been moved to their respective classes, such as the counter variable for trigger_counter. CBaseToggle now only contains members for linear and angular movement, which is its intended purpose.
  • CFuncMortarField, CFuncIllusionary, CBaseCharger now inherit from CBaseEntity instead of CBaseToggle
  • CScriptedSentence now inherits from CPointEntity instead of CBaseToggle
  • CSqueakGrenade (snark) now inherits from CBaseMonster instead of CGrenade
  • CGrenade now inherits from CBaseAnimating instead of CBaseToggle
  • CBaseTrigger now inherits from CBaseEntity instead of CBaseToggle
  • CMultiManager now inherits from CPointEntity instead of CBaseToggle
  • CGenericCycler has been merged into CCycler (CCyler used to be a base class for obsolete HL alpha cine_ entities which used non-existent models)
  • CBaseTrigger members have been moved to the classes to which they belong
  • CBasePlayerItem has been merged into CBasePlayerWeapon
  • CBasePlayerWeapon has been renamed to CBaseWeapon
  • CBasePlayerAmmo has been renamed to CBaseAmmo
  • trigger_counter has been removed since it's an inferior, buggier version of game_counter. (it could break entirely if its master entity were locked since it still counts triggers regardless, and only triggers when it hits exactly 0)
  • The debug-only entity trip_beam has been removed. This entity was never used anywhere.
CBaseMonster still inherits from CBaseToggle for now, and CBaseToggle still inherits from CBaseAnimating. Neither should be the case (CBaseToggle is for brush entities, CBaseAnimating is for studio model entities) but this is needed because a single entity func_guntarget is a monster to make itself target-able by monstrs (inherits from CBaseMonster) but also uses CBaseToggle's linear movement code (it's basically a door that moves between two points).

It's used for the Hazard Course target range and can't be changed right now. Hopefully if i can implement parenting support this can be changed (e.g. like npc_bullseye).

I've also removed all uses of pev->noise, pev->noise1, pev->noise2 and pev->noise3. These are used internally to store off the sounds made by certain entities. The change is part of the "CBaseEntity everywhere" goal and involves the removal of all uses of edict_t and entvars_t.
These variables were being used in confusing ways (multiple #define aliases that made it difficult to find where the values were coming from) so this improves readability as well, and eliminates some of the last remaining #defines.

New and updated entities

Some entities have been updated:
  • game_player_equip now saved and restores its members, allowing it to be used in singleplayer.
  • player_weaponstrip has new keyvalues:
    • Strip Weapons Yes/No (Default Yes): Whether to strip all of the player's weapons
    • Strip Suit Yes/No (Default No): Whether to strip the player's HEV suit. If the flashlight is on it will be turned off.
    • Strip Long Jump Module Yes/No (Default No): Whether to strip the long jump module.
  • New entity: player_sethealth. Sets the player's health and/or armor. Can apply to the activating player or all players. Can set health to any value from 1 to 100, can set armor to any value from 0 to 100. Does not deal damage, shows no HUD icons or damage indicators, cannot kill the player. Useful for setting an initial health value and/or giving armor without playing sounds. Shown in the video when the button is used to set the player's health to 1.

Bugs fixed

  • CWreckage (cycler_wreckage) saved a time value that was actually an int, which would result in corrupted values on restore if the timebase differs. Now saved as a float.
  • Weapons would try to drop the weapon that was picked up instead of the newly created weapon when the weapon gets picked up. Now the weapon being respawned will be dropped, as it should be.
  • When the server is near the edict limit (< 100 free edicts) weapons would be respawned with an incorrect delay. It should be current time + delay, but was current time + current time + delay.

Entity list improvements

I've added a new class that provides an API for creating and destroying entities, as well as iterating over them. It's still a work in progress, but it solves a few problems. For one the classname string is now managed by it so there's no need to worry about strings not outliving the entity, and it sets the classname automatically so there's no need to set it manually when creating entities through code.

To make this work LINK_ENTITY_TO_CLASS now also registers entities in a global dictionary to allow name based lookup. I've also added support for "alias" names that ensure aliased entity names such as weapon_glock not only map to weapon_9mmhandgun but also automatically change the classname to the intended name.

The GetClassPtr function has been removed since it is badly designed and doesn't handle creation properly (classname is never set). The CREATE_NAMED_ENTITY engine function is now obsolete and has also been replaced by the new entity list API.
Posted 2 years ago2021-06-06 21:06:34 UTC Post #345684

Code generation

I've improved the code generator's performance and speed. Code generation is now done on a per-file basis, meaning the generator assumes each file contains at most one class and will generate code for that class if required. This is done because there is no way to know what kind of changes have been made in a file unless the file is parsed, which is the slowest part of the code generation process by far. Instead the generator checks for the last modification time on the file to determine if it needs to reparse the file.

As such all entity classes have been moved to their own header and source file pair. This in turn slows down compilation but due to the increase in efficiency the code generator is much faster when doing partial rebuilds. Typically you'll be modifying only a few files at a time, so compilation takes only a few seconds. A full rebuild on my system (SSD with quad core) takes a few minutes but after that it's fast enough, much faster than the previous approach of using precompiled headers in the code generator (which still had to process a lot of code every time, and always regenerated all files forcing a full rebuild).

The LINK_ENTITY_TO_CLASS macro is no longer used, the code generator takes care of that part now as well.

Additionally i've changed the format used to specify attributes to use JSON. This is because the use cases for the code generator have changed to the point that i was implementing a worse version of JSON, so i switched to it to take advantage of a widely used format with many existing parsers.

It now looks like this:
#pragma once

#include "CBaseEntity.hpp"
#include "CEnvSpark.generated.hpp"

constexpr int SF_SPARK_TOGGLE = 1 << 5;
constexpr int SF_SPARK_START_ON = 1 << 6;

class EHL_CLASS("EntityName": "env_spark", "EntityNameAliases": ["env_debris"]) CEnvSpark : public CBaseEntity
{
    EHL_GENERATED_BODY()

public:
    void    Spawn() override;
    void    Precache() override;
    void    EXPORT SparkThink();
    void    EXPORT SparkStart(const UseInfo& info);
    void    EXPORT SparkStop(const UseInfo& info);
    void    KeyValue(KeyValueData* pkvd) override;

    EHL_FIELD("Persisted": true)
    float m_flMaxDelay = 0;
};
The code generator will now also raise an error if you use unknown attributes or incorrect attribute values (e.g. a string value for Persisted, which is a boolean).

Error handling is designed to tie into Visual Studio's error handling system: the format matches that used by MSBuild, so the errors are listed in the Error List panel and will link to the right file, line and column number when double clicked in either the Error List panel or the Output panel. This makes debugging codegen errors much easier.

The code generator is fairly robust when it comes to handling compilation errors. If you compile your projects and there's an error the files generated that time will be regenerated the next time you compile to ensure any invalid codegen (e.g. missing/invalid attributes) is corrected. There are still a few edge cases where it could fail to do this correctly but i'll try to account for those.

In the event that bad codegen isn't detected, you can wipe all generated code by building a special project. Each project that has codegen enabled gets a CLEAN_CODEGEN_<project name> project that does this. They're also organized under a solution folder in Visual Studio that lets you build all of them at once to wipe all codegen in one go.

Additionally, for Visual Studio i've added a cpp.hint file to tell Intellisense how to deal with the codegen macros. Without this file it can't properly parse class declarations and lists functions as having no definition, which is pretty annoying.

Other stuff

For Visual Studio i've added a .natvis file to tell the debugger how to display entity classes in the debugger. This lets you easily see the classname, targetname and target of any entity you have a pointer of:
natvisnatvis
You'll have to click the refresh icon since these are functions. This makes debugging a lot easier, especially when an entity has an unknown class (e.g. AI code where you want to know the target entity's class).

The weapon selection client command has been changed to use an actual command name. Previously this worked by sending the weapon name and then checking for a "weapon_" prefix. If you were to name a weapon with a prefix other than that you'd never be able to select it through the weapon HUD. Now it works like all other client commands.

CGameRules has been given a virtual destructor to ensure that they can clean up properly on deletion. If a gamerules object destructor has to do any cleanup like freeing memory then this would not work properly in vanilla.

I've fixed a bunch more compiler warnings as well.

All headers now use the .hpp extension for consistency. This also means that all code is now expected to be compiled as C++ code only. All headers now also include their dependencies so each individual header should compile when included by itself. This also means that the include order is not as important as it was before (you had to include extdll.h, util.h and cbase.h in that order).

Including cbase.h will include a bunch of common headers now. Since precompiled headers are in use many source files don't include anything anymore but i may change this to allow the projects to be compiled without precompiled headers if needed for some reason.

Lastly, here's a look at the current project structure in Visual Studio:
project structureproject structure
You can see both CLEAN_CODEGEN projects, the projects that CMake always includes (building INSTALL deploys dlls to the mod directory) and the client and server dlls.

The directory structure makes finding entities pretty easy, though there are some cases where entities had to be put somewhere else to organize things properly.

The warnings indicated by the green squiggles involve the Activity enum, which is an unscoped enum. That hasn't been changed since the name based lookup still relies on the old names (ACT_ names).

That about covers it for work done. I have a bunch of stuff planned for the future but i'll discuss that when i've got some more work done:
todotodo
Posted 2 years ago2021-06-06 21:14:14 UTC Post #345685
I forgot to add this:

Entity batch tool

I made a tool to perform batch operations on .rmf, .map, .bsp and .ent (Ripent) files. It can be used to find things in a map like specific entities, but it can also be used to modify them.

It uses CSharpScript to let you write scripts to apply to maps.

This script collects information about all official maps, specifically the number of items, weapons and monsters:
#load "Bootstrapper.builtin"

public class Diagnostics : ScriptProcessor, IDisposable
{
    const string DiagnosticsFile = "Diagnostics.txt";

    private readonly EntityCounter _itemsCount = EntityCounter.StartsWith("item_");
    private readonly EntityCounter _weaponsCount = EntityCounter.StartsWith("weapon_");
    private readonly EntityCounter _monstersCount = EntityCounter.StartsWith("monster_");

    private readonly StreamWriter _writer;

    public Diagnostics()
    {
        //Overwrite the old file if it exists
        _writer = File.CreateText(DiagnosticsFile);
    }

    public void Dispose()
    {
        _writer.Dispose();
    }

    public override void OnProcessFile()
    {
        if (IsAnyOfficialGameCampaignMap(SourceFileName.Name))
        {
            _itemsCount.AddFrom(Entities, out var itemsCount);
            _weaponsCount.AddFrom(Entities, out var weaponsCount);
            _monstersCount.AddFrom(Entities, out var monstersCount);

            _writer.WriteLine($"File {SourceFileName.Name}");
            _writer.WriteLine($"\t{itemsCount} items");
            _writer.WriteLine($"\t{weaponsCount} weapons");
            _writer.WriteLine($"\t{monstersCount} monsters");
        }
        else
        {
            Logger.LogInformation("Ignoring map {Name}", SourceFileName.Name);
        }
    }

    public override void OnEndProcessing()
    {
        _writer.WriteLine($"Total: {_itemsCount.Count} items");
        _writer.WriteLine($"Total: {_weaponsCount.Count} weapons");
        _writer.WriteLine($"Total: {_monstersCount.Count} monsters");
    }
}
The line #load "Bootstrapper.builtin" is a bit of magic that makes this class stateful across invocations. IsAnyOfficialGameCampaignMap is a built-in function that matches map names up to official map names through regular expressions. There could be false positives if somebody named their map in a specific way, but that almost never happens.

This script updates vanilla Half-Life maps to still work with some of the breaking changes made so far:
void ReplaceWorldItems(MapEntity entity)
{
    //Convert world_items entities to their respective entities
    if (entity.ClassName == "world_items")
    {
        switch (entity.GetInt("type"))
        {
        case 42:
            entity.ClassName = "item_antidote";
            entity.Remove("type");
            break;

        case 43:
            //Remove security items (no purpose, and has been removed from the codebase)
            Entities.Remove(entity);
            break;

        case 44:
            entity.ClassName = "item_battery";
            entity.Remove("type");
            break;

        case 45:
            entity.ClassName = "item_suit";
            entity.Remove("type");
            break;
        }
    }
}

void UpdateSuits(MapEntity entity)
{
    //Set the logon type to the original default
    if (entity.ClassName == "item_suit" && !entity.ContainsKey("logon_type"))
    {
        entity.SetString("logon_type", "LongLogon");
    }
}

foreach (var entity in Entities.ToList())
{
    ReplaceWorldItems(entity);
    UpdateSuits(entity);
}
The tool is still a work in progress so no release yet.

It can process a single file, a directory or a set of directories. I use it to process Half-Life, Opposing Force and Blue Shift maps (processed by the BS Updated conversion tool) to quickly check for uses of entities and to update them to make them work in EHL.

It can technically also just copy maps from one directory to another, but that would be overkill.
Posted 2 years ago2021-06-11 08:46:39 UTC Post #345690

Progress update:

Custom entity data parsing and entity instantiation

I've figured out a way to take control of entity data parsing as well as entity instantiation. These are two things that will go a long way to making mods more secure and allows for new features to be added.

Entity data parsing is the process by which the entity data in a BSP file is converted into actual entities. This code is now in the server dll which means you can have more control over it.

Entity instantiation is the process of creating the entity that the classname refers to. Having this in the server dll allows for more control and improves security, because it doesn't involve calling arbitrary functions based on untrusted input.

Potential uses of custom entity data parsing include implementing point_template, loading default keyvalues from a file, changing the entity data string to another format like JSON (albeit with some restrictions, see below) and adding input/output support.

Changing the data format requires the map compiler to be changed as well. Features like I/O support require map editor support as well and requires modifications to be made to the .map format since it doesn't have the flexibility needed to specify that kind of data (if you need to change it, you may as well switch to a format like JSON).

Potential uses of custom entity instantiation include marking entities as internal so they can't be spawned by mappers (for safety purposes), "synthetic" entities defined by config files instead of in code and custom entities defined in scripts (which you wouldn't be able to restore otherwise).

Here's an example of a synthetic entity config file:
{
  "BaseEntityName": "ammo_generic",
  "EntityName": "ammo_9mmclip",
  "EntityNameAliases": [
    "ammo_glockclip"
  ],
  "KeyValues": [
    {
      "Key": "model",
      "Value": "models/w_9mmclip.mdl"
    },
    {
      "Key": "ammo_amount",
      "Value": 17
    },
    {
      "Key": "ammo_name",
      "Value": "9mm"
    }
  ]
}
This is part of the proposal i wrote, it hasn't been implemented yet. Ammo entities can be described in terms of ammo_generic entities, so this moves a bunch of code into user-editable config files.

Combined with per-map config files that can override default files you can control what each entity turns into.

The next step after this is implementing reflection and replacing the old save data with that.

Technical information

The entity data string typically looks like this:
{
"wad" "\quiver\valve\halflife.wad;\quiver\valve\decals.wad;\quiver\valve\xeno.wad;\quiver\valve\sample.wad"
"chaptertitle" "C0A1TITLE"
"message" "Anomalous Materials"
"classname" "worldspawn"
}
{
"origin" "-424 280 -160"
"message" "ambience/crtnoise.wav"
"health" "2"
"spawnflags" "2"
"classname" "ambient_generic"
}
Entity data parsing is normally done in the engine, but there is an edge case that allows you to fool the engine into thinking it's already done parsing.

The first call into the server dll done by the engine during map loading is a call to DispatchKeyValue.

This keyvalue is always the classname key for worldspawn, the first entity in any map. Though it is possible for the classname to differ, this is forced back to worldspawn for security purposes.

At this point we can assert that the engine's parser is currently positioned right after the first { in the entity data string.

We can obtain a pointer to this string by first acquiring the server_t instance. This is possible by taking the string_t mapname member in globalvars_t and getting the pointer to the name using STRING(). This is the address of the name member in server_t. Subtracting the offset of that member in server_t gets a pointer to server_t.

server_t's worldmodel member is the BSP model, which contains a member entities. This is the entity data string.

Modifying it at this point to become {} will cause the engine to stop parsing the string. It will still try to call the worldspawn function which is an empty function. If it doesn't exist the engine will remove the world entity, so it has to be there.

The server will parse the entity data string and handle creation of all entities by itself. This behavior is currently identical to the engine, except it handles errors more gracefully. It won't stop parsing when it encounters bad data, instead it will try to skip past it to continue. This will likely fail since corrupted entity data is probably missing too much information.

This covers everything that happens when loading a new map. However entities are also created when loading a save game, so this also had to be handled.

When a save game is being loaded the first call into the server dll is a call to SaveReadFields to read in the data contained in a save game.

It will load all of the entity data stored in the save game, create all entities and then initializes them. If the save game is being loaded because the player is going through a changelevel then entities from adjacent maps that were marked for transition are also created from separate save game files.

The engine creates entities by using the classname stored in the save game's entity table block.

It then restores the entity by calling DispatchRestore.

This behavior is overridden by changing the classname to custom in DispatchSave.

The classname is also forced to custom in SaveReadFields for every ETABLE read in.

The engine will call the function custom for every entity it wants to create. This function is empty just like worldspawn and does not actually create anything. The name custom is special because the entity data parsing code in the engine also uses it to spawn entities it can't find a function for. This serves as a fallback to make sure that all works properly, but it should never be needed for that.

When DispatchRestore is called the server checks if it has previously seen a particular SAVERESTOREDATA instance before (identified by the map it was made for, and the landmark name associated with it). If it hasn't, it creates all of the entities that would have been created by the engine for that save game data.

The actual classname is stored in each entity's save data so it can be recreated from that.

This is needed because when entities are restored they also have to set up connections to other entities. If the entities don't exist yet this will fail and break things.

The result is the server dll handles creation of all entities by itself. The engine function CREATE_NAMED_ENTITY is also handled by the server dll so the engine does not create entities anymore, it only thinks it does.

Changing the entity data string format

The entity data string format can be changed to something else since it's parsed by the server dll.
The only restriction is that the string must start with data the engine can understand since it still parses that.

For example here's a version that uses JSON:
{
"classname" "worldspawn"
}
{
    "Entities": [
        {
            "ClassName": "worldspawn",
            "KeyValues": [
                {
                    "Key": "wad",
                    "Value": "\quiver\valve\halflife.wad;\quiver\valve\decals.wad;\quiver\valve\xeno.wad;\quiver\valve\sample.wad"
                }
            ]
        }
    ]
}

Other stuff

I've found a program that can be used to edit JSON files: https://github.com/json-editor/json-editor

This'll make editing config files a lot easier.
Posted 2 years ago2021-06-11 09:31:12 UTC Post #345691
Last update sounds veery neat actually. Wating for further progress! :heart:
FranticDreamer FranticDreamerR.I.P. Benny Harvey. Miss you big man. Gone but not forgotten.
Posted 2 years ago2021-06-12 00:01:53 UTC Post #345692
Wow, this is some really good work, even for you! The synthetic entities would allow people to make some really cool stuff per-map, I'd imagine... Kind of makes me want to reinstall JACK or Hammer 3.5!
Notewell NotewellGIASFELFEBREHBER
Posted 2 years ago2021-06-12 09:39:16 UTC Post #345695
Wow, this is some really good work, even for you! The synthetic entities would allow people to make some really cool stuff per-map, I'd imagine... Kind of makes me want to reinstall JACK or Hammer 3.5!
Synthetic entities are just a way to assign a new classname to a set of default keyvalues, really. Scripts with custom entities would be much more powerful than this, but i don't want to add scripting support until the codebase is stable.

On the plus side because i can use code generation i can automate most of the work needed to make custom entities work.

I've been brainstorming on how to design the config file system, and i've come up with some ideas. None of what i'm writing here has been implemented yet.

I've pinned down the definition for template entity configurations some more. I referred to this as "loading default keyvalues from a file" above.
{
  "Name": "TemplateName",
  "Include": [
    {
      "FileName": "some_other_path_under_cfg.json"
    }
  ],
  "KeyValues": [
    {
      "Key": "Key1",
      "Value": "Value1"
    },
    {
      "Key": "Key2",
      "Value": "Value2"
    },
    {
      "Key": "Key1",
      "Value": "Value3"
    }
  ]
}
Included in a map or server configuration file like this:
{
  "EntityTemplates": [
    {
      "FileName": "path_to_template_file.json",
      "Type": "EntityTemplate"
    },
    {
      "FileName": "path_to_other_template_file.json",
      "Type": "DefaultEntityTemplate",
      "EntityNames": [
        "some_entity_name"
      ]
    },
    {
      "FileName": "ammo_9mmclip.json",
      "Type": "SyntheticEntityTemplate",
      "BaseEntityName": "ammo_generic",
      "EntityNames": [
        "ammo_9mmclip",
        "ammo_glockclip"
      ]
    }
  ]
}
A template marked as default will be used to initialize all entities of the types listed in EntityNames. This occurs after entity creation, before any keyvalues are added.

A template marked as synthetic allows mappers to create instances of entities by the given name. Separate synthetic entity configurations are no longer a thing since the syntax is practically identical to templates.

Non-default templates can be referenced by the entity through a keyvalue. The template will be applied after a default template, before any keyvalues are added (possible since we can now access the entity data before keyvalues are handled).

For non-default templates the Type key can be omitted.

So we've got:
  • Synthetic entities: tie a set of default keyvalues to a new entity name
  • Custom entities: define a new entity through script code
  • Default templates: apply default keyvalues to all entities of specific types on creation
  • Entity-specific templates: apply default keyvalues to entities that refer to a template by name
None of this has been implemented yet, it's all proposal-level stuff i'm thinking of so i can design a proper configuration file format and plan for scripting support.

Config files are being designed to be easy to use, i've defined a general-purpose file inclusion syntax that makes it easy to share configurations between maps.

With what i have in mind you could define a hierarchy of default template files to set defaults for every entity, for example default health, model and classification for every monster. This will require code support to make sure mapper-defined settings aren't overwritten. Ideally i could just move the hardcoded values to config files and let the template system do the rest.

If you wanted to make a map pack you'd make a shared config file that includes all of the templates and ties them to their respective entities. You would then have a map-specific config file that includes the shared one:
"Include": [
  {
    "FileName": "some_path_under_cfg.json"
  }
]
Map and server configs can both do the same things, but map configs can have different filters applied to cvar setters and stuff (blacklisting certain cvars):
{
  "Sections": [
    {
      "Name": "Pretty name for debugging",
      "Condition": "Multiplayer && !Deathmatch",
      "InitializationOrder": "PreMapSpawn",
      "Actions": {
        "Cvars": [
          {
            "Name": "mp_flashlight",
            "Value": true
          }
        ],
        "Include": [
          {
            "FileName": "some_other_path_under_cfg.json"
          }
        ]
      }
    }
  ]
}
Sections can be used to conditionally include configurations based on game state, so the same config can be used for singleplayer and multiplayer, or deathmatch and teamplay. The only problem with this is that config files can change the game mode, so this will need to be well defined somehow.

InitializationOrder controls when each section is applied. Server config sections are always applied before map config sections that have the same InitializationOrder. You can have a server config applying something as a default, then a map config overriding things, then a server config forcing a setting back (e.g. time limit). Servers get a special order value that lets them set things after map configs to force things.

As much as possible config data should be unloaded after map start to free up memory.
You must be logged in to post a response.