SharpLife - Dot Net Core based modding p Created 6 years ago2018-05-06 19:15:02 UTC by Solokiller

what about 24 bit textures? ok if only v2 supports this. i cant wait to see v2

V2 can support 32 bit just fine. I'm using a library made for image loading so eventually it could support any kind of image that that library supports.

and there is one more probleb about goldsrc. i dont want to use milkshape. its expensive and limited. i can use blender but its only supports hl2 smd files. so maybe you can make a compiler for hl2 smds to hl1 mdls.

I've never dabbled in Source programming and from what i know about it the studiomdl compiler for it is closed source. Maybe an easier to use format could be made for it one day though.

Better use hornetgun... But the entity limit is a problem (BIG PROBLEM) as you say. How much entities will support SL at a time before struggling?, also, and remember that I am quite an ignorant in coding, will it depend of the PC specs or this number will adapt itself independently of the system on which SL will run on?

SharpLife supports server-only entities which avoids the whole problem of networking entities to the client and having bandwidth issues. Projectiles don't need to be visible all the time, but networking is also much better so the client can handle more of it on the client, whereas GoldSource just does everything server side and then sends it to the client.

The system requirements are pretty high compared to GoldSource, but they haven't been set in stone. What i know right now is you'll need Windows 7 or newer, and a GPU capable or supporting fairly new versions of OpenGL (4.2 or newer). It may be high but do remember that hardware will catch up so the engine should have a long shelf life, longer than GoldSource considering how poorly the old renderer is supported now.

Pretty much all of the requirements can change depending on future development, and if Valve get their heads out of their collective asses and open source the engine i could do 64 bit development which would drastically increase the options available for bigger features.

i dont think hornets any more useful than another entities. and projectile bullets dont necessary with current tiny maps xD my mod is just was an experiment. ill post my youtube channel here if someone like to see my projectiles

there is too much things to request.

the crowbar gibbing of corpse is buggy. i guess valve intentionally broke it to lower the gore.

some corpses can gibbed with one hit. others are too much hits and they broke the hitting animation so without swinging the crowbar gibbing the corpses. (so politicians or media or something dont want to see a game where player constantly whacking a corpse. it seems so they removed animations.)

and you cant hit the corpse if that part of corpse is out of the otiginal monsters hull (bounding box)

btw im feeling like im flooding the thread with necessary but tiny requests. can we post bug fix request like this?

edit:

I've never dabbled in Source programming and from what i know about it the studiomdl compiler for it is closed source. Maybe an easier to use format could be made for it one day though.

if hl2 smd to hl1 mdl compiler isnt posible, at least hl2 smd file to hl1 smd file can be made. the hl2 smd file format is easy to find, i guess. and there is info about hl2 smd at valve developer site.

some corpses can gibbed with one hit. others are too much hits and they broke the hitting animation so without swinging the crowbar gibbing the corpses. (so politicians or media or something dont want to see a game where player constantly whacking a corpse. it seems so they removed animations.)

Depending on the NPC the health for corpses is set to half the original health value, so it could be quite high sometimes. Corpses are still subject to hitgroups so that might have something to do with it if you're essentially hitting a monster's feet.

The animation not playing is because the weapon prediction code doesn't run think functions on the client, i already fixed that in HLEnhanced. If you turn it off using cl_lw it should work as before, but in multiplayer it'll be laggier.

and you cant hit the corpse if that part of corpse is out of the otiginal monsters hull (bounding box)

That can probably be fixed by changing the bounding box upon death to match the shape when the NPC has come to rest. That depends on how sequences store hull sizes and stuff.

if hl2 smd to hl1 mdl compiler isnt posible, at least hl2 smd file to hl1 smd file can be made. the hl2 smd file format is easy to find, i guess. and there is info about hl2 smd at valve developer site.

It's possible to convert them, but information will be lost. That's why it may be better to use a new format that supports what you need before writing any converter tools.

What mods can SL run right now?, I´m curious.

For now, nothing, it's not even mature enough to start building your own game/mod on top of it

Also it's written in a different language and has a different API so existing mods can't run under it anyway.

Crap. So, what should a mod creator do to his/her mod to make it be run under SL? If this require to learn C# and start all from scratch, will not it disencourage people to port their mods to SL?, maybe I don´t understand very well what Shepard said "building your own GAME (I have this clear) /MOD (What!?) on top of it". We are talking about a program made to take profit of a better written code and all the capabilities of modern hardware, right?.

"If this require to learn C# and start all from scratch, will not it disencourage people to port their mods to SL?"

In my opinion, it may do the opposite. It may encourage people. I've never used C#, but from what I hear and see here, it seems easier than C++, and that could lead to potentially more people coming than people getting discouraged. Besides, learning a new language isn't supposed to be a bad thing.
Then there's the question of doing it from scratch. If the HL SDK is ported to C#, do we replicate the changes which we made to our C++ HL SDK code over to the C# HL SDK, or something else?

@hfc
There has to be a plugin for Blender that supports GoldSrc SMD. You can find out here: http://the303.org/tutorials/gold_mdl.htm

In my opinion, it may do the opposite. It may encourage people. I've never used C#, but from what I hear and see here, it seems easier than C++

It is, in high programming languages (C#/Java), you don't have to deal with pointers (but you still need to worry about "null references"), memory allocations, memory leaks and more, the language does that for you. The only problems (which aren't really problems if you ask me) is you need a "Common Language Runtime" (.NET Core Runtime in the case of Sharp-Life) to run the code and it can be easily reverse engineered (yeah I know about obfuscation). C/C++ being low level programming languages are "closer to the machine", it's the opposite.

Then there's the question of doing it from scratch. If the HL SDK is ported to C#, do we replicate the changes which we made to our C++ HL SDK code over to the C# HL SDK, or something else?

My guess is that you will need to start from scratch, there are already so many differences that you can't just move your C++ code over to C#. Let's take an example: to save/restore entities data, the C/C++ way is to declare and use a save/restore table. Sharp-Life through the power of C#'s annotations handles the save/restore table already for you and you just need to annotate the variables you want to be saved/restore.

C/C++:

// Might be wrong here
TYPEDESCRIPTION CTWHL::m_SaveData[] =
{
    DECLARE_FIELD( FIELD_BOOLEAN, CTWHL, m_bStatusSent )
};
IMPLEMENT_SAVERESTORE( CTWHL, CBaseEntity );

C# (Sharp-Life):

// Might be wrong here too (feel free to correct me Solokiller if I messed up)
class CTWHL
{
    [Persist]
    bool StatusSent;
}

Yeah your example is pretty much how save/restore should work.

C# is much easier to use than C++. I'll show a couple SDK examples to show this.

This is some code that shows up a lot in gamerules classes:

UTIL_LogPrintf(
	"\"%s<%i><%u><spectator>\" joined team \"spectator\"\n",
	STRING( pPlayer->pev->netname ),
	g_engfuncs.pfnGetPlayerUserId( pPlayer->edict() ),
	g_engfuncs.pfnGetPlayerWONId( pPlayer->edict() ) );

The equivalent in SharpLife would be:

_logger.Information( "\"{PlayerName}<{UserId}><{SteamId}><spectator>\" joined team \"spectator\"", player.PlayerName, player.UserId, player.SteamId );

Changing WONId to SteamId since WONId is always -1 now.

This is for logging, for normal string formatting you can just use string interpolation:

sprintf( text, "* you are on team '%s'\n", team_names[ ( int ) pPlayer->m_iTeamNum - 1 ] );

var text = $"* you are on team '{TeamNames[(int) player.TeamNumber]}'\n";

No possibility of buffer overflows here.

EHandles can also be made safer:

CTFGoalFlag pFlag = ( CTFGoalFlag* ) ( CBaseEntity* ) pPlayer->m_pFlag;

if( pFlag )
{
	pFlag->ReturnFlag();
}

Becomes:

EntityList.GetEntity<CTFGoalFlag>(player.Flag)?.ReturnFlag();

You don't have to do the double cast anymore (needed since EHANDLE can only be cast to CBaseEntity through its conversion operator), and the cast is safe instead of potentially creating an invalid object that can crash the game when used. Use of ? here is the same as the if check, if the object is not null call this method.

I've also had to write this code recently:

edict_t* pKiller = INDEXENT( 0 );

entvars_t* pEntKiller = pKiller ? &pKiller->v : nullptr;

None of that is needed in SharpLife, you can just use BaseEntity directly:

var killer = EntityList.GetEntityByIndex( 0 );

Since this just gets the world you can also get it through Context.Entities.World.

This is all pretty basic stuff, it gets easier once you deal with entities more:

pPlayer->pev->effects |= EF_NODRAW;
pPlayer->pev->flags |= FL_SPECTATOR;
pPlayer->pev->solid = SOLID_NOT;
pPlayer->pev->movetype = MOVETYPE_NOCLIP;
pPlayer->pev->takedamage = DAMAGE_NO;
pPlayer->m_iHideHUD |= HIDEHUD_HEALTH | HIDEHUD_WEAPONS;
pPlayer->m_afPhysicsFlags |= PFLAG_OBSERVER;
pPlayer->m_iNewTeamNum = CTFTeam::None;
pPlayer->m_iCurrentMenu = 0;
pPlayer->m_iTeamNum = CTFTeam::None;

player.Effects |= Effect.Nodraw;
player.Flags |= EntityFlag.Spectator;
player.Solid = Solid.Not;
player.Movetype = Movetype.Noclip;
player.Takedamage = TakeDamageMode.No;
player.HideHUD |= HideHud.Health | HideHud.Weapons;
player.PhysicsFlags |= PhysicsFlag.Observer;
player.NewTeamNumber = CTFTeam.None;
player.CurrentMenu = Menu.None;
player.TeamNum = CTFTeam.None;

Though i expect some of these variables will become obsolete since they're used for networking physics specific stuff. There's overlap here and there that can be eliminated.

Network messages will also be easier to use, right now they're just Protobuf messages so you can easily create and send them without having to worry about getting it wrong.

Debugging is also much easier since it'll tell you where a problem occurred, for example here's the log entry for a fatal error where it failed to load a sprite:

2019-01-10 00:34:47.640 +01:00 [ERR] A fatal error occurred
System.IO.FileNotFoundException: Could not resolve absolute path
File name: 'sprites\sewerfog.spr'
   at SharpLife.FileSystem.DiskFileSystem.GetAbsolutePath(String relativePath, String pathID) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.FileSystem\DiskFileSystem.cs:line 134
   at SharpLife.FileSystem.DiskFileSystem.Open(String relativePath, FileMode mode, FileAccess access, FileShare share, String pathID) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.FileSystem\DiskFileSystem.cs:line 222
   at SharpLife.FileSystem.FileSystemExtensions.OpenRead(IFileSystem self, String relativePath, String pathID) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.FileSystem\FileSystemExtensions.cs:line 254
   at SharpLife.Models.ModelManager.LoadModel(String modelName) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Models\ModelManager.cs:line 65
   at SharpLife.Models.ModelManager.InternalLoad(String modelName, Boolean throwOnFailure) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Models\ModelManager.cs:line 87
   at SharpLife.Models.ModelManager.Load(String modelName) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Models\ModelManager.cs:line 119
   at SharpLife.Engine.Server.Resources.ServerModels.LoadModel(String modelName) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Engine.Server\Resources\ServerModels.cs:line 75
   at SharpLife.Game.Server.Entities.BaseEntity.KeyValue(String key, String value) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Game.Server\Entities\BaseEntity.cs:line 228
   at SharpLife.Game.Server.Entities.Effects.EnvSprite.KeyValue(String key, String value) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Game.Server\Entities\Effects\EnvSprite.cs:line 45
   at SharpLife.Game.Server.Entities.ServerEntities.LoadEntity(List`1 block, Int32 index) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Game.Server\Entities\ServerEntities.cs:line 267
   at SharpLife.Game.Server.Entities.ServerEntities.LoadEntities(String entityData) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Game.Server\Entities\ServerEntities.cs:line 197
   at SharpLife.Game.Server.Entities.ServerEntities.MapLoadBegin(ITime gameTime, IMapInfo mapInfo, GamePhysics gamePhysics, String entityData, Boolean loadGame) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Game.Server\Entities\ServerEntities.cs:line 170
   at SharpLife.Game.Server.API.GameServer.MapLoadContinue(Boolean loadGame) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Game.Server\API\GameServer.cs:line 207
   at SharpLife.Engine.Server.Host.EngineServerHost.InitializeMap(ServerStartFlags flags) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Engine.Server\Host\EngineServerHost.cs:line 240
   at SharpLife.Engine.Engines.ClientServerEngine.FinishLoadMap(String startSpot, ServerStartFlags flags) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Engine\Engines\ClientServerEngine.cs:line 442
   at SharpLife.Engine.Engines.ClientServerEngine.StartNewMap(ICommandArgs command) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Engine\Engines\ClientServerEngine.cs:line 426
   at SharpLife.CommandSystem.Commands.Command.OnCommand(ICommandArgs command) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.CommandSystem\Commands\Command.cs:line 43
   at SharpLife.CommandSystem.CommandQueue.Execute() in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.CommandSystem\CommandQueue.cs:line 95
   at SharpLife.CommandSystem.CommandSystem.Execute() in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.CommandSystem\CommandSystem.cs:line 103
   at SharpLife.Engine.Engines.ClientServerEngine.Update(Single deltaSeconds) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Engine\Engines\ClientServerEngine.cs:line 196
   at SharpLife.Engine.Engines.ClientServerEngine.Run(String[] args, HostType hostType) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Engine\Engines\ClientServerEngine.cs:line 181
   at SharpLife.Engine.Host.EngineHost.Start(String[] args, HostType type) in C:\Users\Sam\Documents\GitHub\SharpLife_full_dev\SharpLife-Engine\src\SharpLife.Engine\Host\EngineHost.cs:line 36

It's a bit much but you get everything you need to know.

Compare this with GoldSource or even just C++ debugging where something like this is little more than a message box telling you there was a fatal error, without actually pointing you at the code that triggered it.

Once you get used to the language you'll find it's much easier, you won't want to go back to C++ after this.

So, it is not a question of doing anything from scratch, just knowing the "sintaxis" and "translating" waht´s already done to C#, right?. So, Solokiller is doing a "translation" of the HLSDK if I am not wrong, applying of course updates and making the code cleaner and optimized.

Well, what´s your advice for people like us who´re searching for a GoldSource compatible engine capable of more "horsepower" to move bigger and complex mods? I mean that if we could use, as Solokiller said, his engine to move GS mods or it is oriented to newer mods created on the SL plataform. Probably my explanations are not accurate, but I think from the perspective of a car user with little knowledge about car maintenace, and you are car´s motor and electronics designers, Idk if you catch the idea.

I'm not translating it, i'm making a new engine that's also a mod that lets you make Half-Life mods and load original content.

If you want a better engine other than Xash then you'll have to wait.

If you want a better engine other than Xash then you'll have to wait.

You caught me!! But, If it can load original content like the original HL does I will wait for sure. Also It appears to work without vgui.dll, which´s the major limitation of Xash when it comes to go greenlightened on STEAM, right? , I´m with ZWC for the last 14 years, one more will not harm me.

In what a media playing is about, can we use other formats than AVI for the intro and sierra videos? also, I remember that in EHL you wanted to add background maps for the game menus, is this still present in SL?

You caught me!! But, If it can load original content like the original HL does I will wait for sure. Also It appears to work without vgui.dll, which´s the major limitation of Xash when it comes to go greenlightened on STEAM, right? , I´m with ZWC for the last 14 years, one more will not harm me.

The original engine references vgui.dll, but my code doesn't use it.

In what a media playing is about, can we use other formats than AVI for the intro and sierra videos? also, I remember that in EHL you wanted to add background maps for the game menus, is this still present in SL?

The original engine can't play any videos, and SharpLife can't do it either. I'm not going to look into that any time soon either.
Background maps will be a thing for later, when the engine is stable.

is it posible to use alternative algorithms over csg, bsp, vis and rad?

because map leaks and, convex and simple brush only mapping is very hard and bottlenecking on map design.

and rad compilings learning curve is too steep. I still dont know how to use expert compiling

so more modern collision dedection and more modern visibility calculation would be nice

i hope your sharp lifes mapping learning curve would be less steep too. as less as coding side

edit: i mean polygon based mapping rather than brush based mapping, is more powerful.

edit2:lightmaps have too low resolution

While I think it is possible to have different map formats, I believe you are missing the point. The idea is to have a drop in replacement engine that is compatible with existing assets and later build on top of it as Solokiller said, so having legacy compatibility is the main focus now.

What you are suggesting is having completely different formats and methods for rendering, basically requiring making an engine completely from scratch.

GS mapping is pretty simple for new people to get going with, but as with everything it takes time to master. Same with Source and any other engine (the newer the more complex it is).

You're getting way ahead of yourself here. SharpLife doesn't even support all of the original engine's features yet, talking about adding a bunch of new stuff now is far too soon. At this rate it'll end up being some mythical super engine that can do anything but doesn't actually work yet.

We can talk about this stuff later on, when the initial version is finished. For now i'd like to focus on getting it all to work as it should.

Ok, no avi for SL

What are the entities did you tested successfully among, info_player entities, env entities, lights (I saw something about those), scripst,etc.? apart from what´s descibed in Post #339709.

Also, what programs are you using to edit-compile the source?

Sorry for saying the "translation" thing, but, in your experience, what will be the major problems a non expert code will find when trying to "port" his/her code to SL?, many of us use tutorials all written in C++...

Sorry for my too basic questions, I know that maybe this is not the place to ask them because the info here reached a high level, but I am very interested in SL as I was in EHL and PS before, and I want to gather as much info as possible before making the decission to jump from C++ to C#, because if C++ was sometimes like chinese to me, I think I will not be able to learn korean.

@abbadon:

descibed in Post #339709.

heres post #339709

btw can we make another thread for SL v2 for not to forget cool ideas. and not to flood this thread with weird ideas.

cause i have one more idea to implement: unbroken coop mode and seamless level transition in multiplayer server.

so solokiller dont bother with forwardthinking ideas for now. he can look the other thread when hes ready to implement

What are the entities did you tested successfully among, info_player entities, env entities, lights (I saw something about those), scripst,etc.? apart from what´s descibed in Post #339709.

That's the prototype for SharpLife, which ran under the original engine and only implemented game logic. It was essentially a converted SDK, but i scrapped it in favor of a complete rebuild, so just ignore all that.

Also, what programs are you using to edit-compile the source?

Visual Studio 2017 with C#7.3. The native wrapper uses C++17, but you'll probably never touch it since it's very simple and only does one thing.

Sorry for saying the "translation" thing, but, in your experience, what will be the major problems a non expert code will find when trying to "port" his/her code to SL?, many of us use tutorials all written in C++...

No problems really, you'll just have to know which pev members map to which members in the new version. That's not really hard to figure out.

Sorry for my too basic questions, I know that maybe this is not the place to ask them because the info here reached a high level, but I am very interested in SL as I was in EHL and PS before, and I want to gather as much info as possible before making the decission to jump from C++ to C#, because if C++ was sometimes like chinese to me, I think I will not be able to learn korean.

You shouldn't make any decisions yet, this project is far from complete. I'd suggest checking out tutorials for basic Hello World stuff first, then maybe WPF since it requires the use of many language features.

If C++ is chinese then C# is dutch. It's way easier to learn and use. C++ has a lot of complexity and gotcha's that don't exist in C# and IntelliSense and the compiler will tell you how to fix most problems in C#. I'm using extensions that find code that can be improved which also helps out a lot.

btw can we make another thread for SL v2 for not to forget cool ideas. and not to flood this thread with weird ideas.

That's probably years away at this point, so i wouldn't bother just yet.

cause i have one more idea to implement: unbroken coop mode and seamless level transition in multiplayer server.

I was already going to implement co-op (Opposing Force has a co-op gamemode in it), what do you mean by unbroken?

Seamless level transitions basically revolve around 2 things: not showing the changelevel dialog and transitioning entities, which singleplayer code already does. For multiplayer all that's needed is to track which player is which since they don't remain connected during transitions. That's not very hard to do.

what do you mean by unbroken?

i was thinking hl's coop mode is broken. i didnt know it wasnt

and i also trying to say, trigger changelevels arent working in singleplayer maps when playing MP, are they?

That's probably years away at this point, so i wouldn't bother just yet.

Then im gonna wait. and i will make projects about my own profession. because time wont pass when waiting something will not be released soon

and i also trying to say, trigger changelevels arent working in singleplayer maps when playing MP, are they?

They're disabled for multiplayer, but it's easy to re-enable them. The hardest part about seamless transitions in the vanilla engine is that it wasn't designed for multiplayer use, so save games don't handle it. Also a fair amount of code in the SDK assumes the first player is in charge of gameplay, so things can break in weird ways. Easy to fix if you know what you're doing though.

i was thinking hl's coop mode is broken. i didnt know it wasnt

Half-Life itself doesn't have co-op at all, beyond an IsCoOp member in gamerules that's only used to find info_player_coop spawn points. Valve didn't finish it in time for Half-Life, but Op4 does seem to have it. Which brings me to: https://twhl.info/thread/view/19673

You shouldn't make any decisions yet, this project is far from complete. I'd suggest checking out tutorials for basic Hello World stuff first, then maybe WPF since it requires the use of many language features.

Ok. I´ll finish all the graphics left undone (the code is 99,99% done) and I´ll keepseeing how SL grow and evolves. Thanks Solokiller (as always)

I've been doing some research on how modern engines handle entities and i've found that there are some options for SharpLife, at a cost.

GoldSource uses the older model of having each entity be a class inheriting from an entity base class. This means that when you create an entity, it will always be the class it is when it's first created. So you can't turn a door into a breakable, for example.

Newer engines allow you to add components to entities to separate out optional parts. Source does this, for example it has a VPhysics component to represents its physics mesh, if it has one.

This still uses the same rigid class based structure as before, but it avoids having variables in the class that aren't always used.

A more advanced version of this uses a sealed base class (can't be inherited from) and adds functionality to entities by adding components. This lets you change entities after they've been created, but adds a lot of overhead to update each component.

The newest kid on the block is called the Entity-Component-System, or ECS for short. You still have a sealed base class and components, but the components contain only state and not behavior. Behavior is provided by systems which can operate on a specific component type, or all entities that have a set of component types.

An ECS is much more efficient than a regular component system because it eliminates the overhead needed to update components. I'll illustrate this with some pseudo code.

A GoldSource style entity system works like this:

for each entity in entity_list do
    entity.think()

A component based system works like this:

for each entity in entity_list do
    for each component in entity do
        component.think()

An ECS works like this:

for each system in system_list do
    system.update()

There are far fewer systems than there are entities, and there are far fewer entities than there are components.

This makes an ECS more efficient since it reduces execution overhead. It also allows for more efficient memory usage by adding and removing components as needed.
Networking wise networking individual components is cheaper than networking an entire entity which can save a lot of bandwidth.

Unity for example networks components by networking up to 32 variables per component. By restricting the number of variables they can optimize networking to reduce dynamic memory allocations, something that isn't possible with a traditional hierarchical entity design like GoldSource and Source use.

Now comes the part that's specific to GoldSource: in GoldSource entities think during physics updates like this:

for each entity in entity_list do
    if entity is not world and entity is not player
        entity.move()
        entity.think()

The world never moves or thinks, and players are handled elsewhere.

The problem with this is that entities will sometimes check other entities near themselves while thinking. For instance teleporters and spawners will check if there are entities at the destination to see if they can move entities or perhaps need to telefrag them.

Because this can happen while other entities haven't moved yet, it's possible for inconsistent behavior to occur. Depending on whether an entity is in the entity list before or after another entity its behavior may change slightly as a result.

When using an ECS the physics update logic necessarily has to be changed to this:

for each entity in entity_list do
    if entity is not world and entity is not player
        entity.move()

for each system in system_list do
    system.update()

The result is that instead of executing each entity's think method after the last, all component systems think after all entities have moved. This is more correct than what GoldSource does, but it will change game behavior slightly in these cases where order of execution mattered.

I'm not aware of any cases where game logic is timed to happen in the correct order in the same frame, but i expect some minor behavioral changes to occur as a result of such a change in design.

So i have a decision to make. Do i go with the older entity design, use a basic component based system (remember, this increases overhead) or switch to an ECS?

For backwards compatibility purposes the original entity based approach will need to be emulated to support the limited entity data format. So a func_breakable would be an entity that has a brush model component and a breakable component, and perhaps an item spawner component that triggers when the breakable component triggers.

Once we can make a new map format it should be possible to use a more powerful entity creation process that would allow you to specify components to add to entities so for example you could make a breakable door relatively easily. This wouldn't happen any time soon though, since that requires a lot of changes.

Interesting. The benefits of an ECS seem to make it the obvious choice, but I'm not really sure what little behaviour differences you can expect. Too soon to tell?

If you're up for what seems like a lot of work, I say go for it. The potential down the line for more complex entity setups sounds brilliant.

I order same-frame game logic by calling functions in the order I want, couldn’t something comparable be done? I do this a lot in SMJ when maps load.

The idea behind ECS is that every system executes one at a time so tightly controlling order of execution goes against that.

I'm not sure if there are any practical differences in Half-Life, that's something that can be found by testing it. I can only say that it will be different in terms of when exactly it executes, but it might not be different enough to matter in the end. I'd consider any code and entity setups that are that reliant on this behavior to be bugged, since it's inconsistent.

Building an ECS doesn't seem to be terribly difficult, i've already got a small prototype going to see what it would take. I don't have to consider things like multithreading support so it's simpler than Unity's ECS. In the long run this might actually be simpler than just porting the code since it's so much easier to reuse stuff, and the networking side of things is actually simpler than what i was working on before.

For starters i'll build a non-networked system, but i don't expect there'll be many changes needed to make it networkable. If i apply what i've learned by studying Unity i can eliminate the need for separate client and server versions (much easier since state and behavior are decoupled) which was starting to complicate things a bit.

Save/restore is probably not much different from networking since they both deal with taking variables from components and converting them for something else. I might be able to reuse the code needed for networking to be used for that as well.

I've done some research on ECS and a regular component system and i've found that using ECS would make it much more difficult to implement Half-Life.
For example things like triggering entities/inputs becomes more difficult because there's no object to trigger.

I think it's also more difficult for the average modder to learn how to use ECS, and if plugins are to be supported then it would be impossible to modify behavior on an object without duplicating the code for it since a component or group of components will be affected by systems that are not under the control of the plugin.

A regular component system like what Unity's had for years doesn't have these problems. While it does have lower performance it can do what's needed while still working like the original. The SDK's code can be converted to a component based design without having to change much, whereas using ECS requires a complete redesign.

I've also done some research to see if the greater overhead of a component system can be reduced, and i've found that it is actually possible: https://blogs.msmvps.com/jonskeet/2008/08/09/making-reflection-fly-and-exploring-delegates/

This article shows that's it's possible to get performance close to direct method calls using Reflection. Using Reflection instead of virtual method calls should make components much cheaper to use. I'll illustrate this with an example.

Here's a naive way to call Update on every component:

foreach (var entity in _entities)
{
    entity.Update();
}

//Entity's Update method
void Update()
{
    foreach (var component in _components)
    {
        component.Update();
    }
}

Not every component actually overrides Update but the cost of the call still exists.
Here's a way to optimize this using Reflection and delegates:

//Entity's Update method
void Update()
{
    foreach (var component in _components)
    {
        component._metaData.Update.Delegate?.Invoke(component);
    }
}

So now instead of a virtual method call it's a delegate invocation, but only if the delegate exists. The delegate is created when the component type is first used, as metadata that's generated once and then stored in each instance's _metaData field. This reduces the overhead, and since components rarely implement these methods (Update is called every game frame, hardly ever needed in Half-Life) it means the overhead involved in managing components is reduced.

It can be further reduced by moving from a purely object oriented approach to something more effective:

foreach (var component in _activeComponents)
{
    component._metaData.Update.Delegate?.Invoke(component);
}

Instead of calling the entity's Update method and letting that handle the updating of the components, all active components can be put into a single list and updated in one go by the game every frame. This eliminates the overhead involved with calling Entity.Update and iterating over each individual list so often.

So i think that this approach will work best for this particular project. In the future it may be possible to implement ECS, but that's something to consider after SharpLife V1 is all done.

Also, since this can allow the same code to run on client and server i can probably refactor the existing projects to merge client and server code some more, reducing complexity even more in both engine and game code.

I did some research on how Unity handles commands and remote procedure calls (documentation for those here: https://docs.unity3d.com/Manual/UNetActions.html)

Unity rewrites the code to call a generated method that sends a message to clients (or servers, for Commands) containing the arguments. This makes it incredibly easy to add and use network messages.

Let me give an example by comparing it to GoldSource's approach and the approach i was planning on using with Protobuf.

Here's how you declare a network message and implement it:

//In player.cpp
int gmsgMyMsg = 0;

//In LinkUserMessages

//Maximum name length is 12 bytes
//Sending a string, so variable length message (-1)
//Maximum message size is 192 bytes
gmsgMyMsg = REG_USER_MSG( "MyMsg", -1 );

//In hud.h
int _cdecl MsgFunc_MyMsg(const char *pszName,  int iSize, void *pbuf);

//In hud.cpp
int __MsgFunc_MyMsg(const char *pszName, int iSize, void *pbuf)
{
	return gHUD.MsgFunc_MyMsg(pszName, iSize, pbuf );
}

//In CHud::Init
HOOK_MESSAGE( MyMsg );

//In hud_msg.cpp
int CHud:: MsgFunc_MyMsg( const char *pszName, int iSize, void *pbuf )
{
	BEGIN_READ( pbuf, iSize );

	CenterPrint( READ_STRING() );

	return 1;
}

This is just for a message in CHud, it's a bit different for a message in another Hud element.

And here's how you use it:

extern int gmsgMyMsg;

MESSAGE_BEGIN( MSG_ALL, gmsgTextMsg );
	WRITE_STRING( "my message" );
MESSAGE_END();

Assuming you're sending it to all clients reliably.

The Protobuf approach would be like this:
Declaring the message:

package SharpLife.Game.Networking.Messages;

message MyMsg
{
    string text = 1;
}

Receiving it in code:

public class SomeClass : IMessageReceiveHandler<MyMsg>
{
   public SomeClass(MessagesReceiveHandler handler)
    {
        handler.RegisterHandler(this);
    }

    public void ReceiveMessage(NetConnection connection, MyMsg message)
    {
        //-1 means centered for an axis
        _graphics.Text.DrawText(message.Text, -1, -1);
    }
}

And using it:

Context.Messages.SendMessage(new MyMsg{ Text = "my message" });

And now the Unity approach:

//Singleton component on some entity used to communicate from server to client for the hud
public class HudText : Component
{
    [ClientRpc]
    public void CenterPrint(string message)
    {
        _graphics.Text.DrawText(message, -1, -1);
    }
}

//Some other code in some other component
World.Instance.Hud.CenterPrint("my message");

Where World is a component used to identify the game world, placed on the entity that represents the world, that has a property public HudText Hud { get; }.

No registration is needed since that's all automatically generated behind the scenes, there's no need to define separate message types, no manual serialization or sending of messages, and the destination does not need to explicitly register itself since it is identified through its network identity.

This approach is way better and allows for optimizations that are not possible when user code has to manually register things. Since the game codebase is the same assembly for client and server there is no need to send the metadata to validate it either, as long as you've verified that the client is running the same assembly as the server, which is easy to do by comparing the AssemblyIdentity (strong name signing required to prevent spoofing).

It's also more efficient than Protobuf because it doesn't require the creation of objects to serialize.

Also unlike GoldSource and i think Source as well, Unity allows these sort of calls from client to server. In GoldSource the only way to communicate with the server is to send string commands, which is very restrictive. Being able to directly call the server with a specific command on a specific object can make things much easier.

As far as the technical side of things goes, i've found two libraries that could be used to do this: Reflection.Emit (part of .NET) and Mono.Cecil (also works on Core).
Since i've never worked with either i've asked for more information here: https://old.reddit.com/r/dotnet/comments/apg4qb/reflectionemit_vs_monocecil_which_is_best_suited/?st=js0fdqxt&sh=8db88032

I know that Cecil can do this, but if Reflection.Emit can do the same there is no reason to use a third party library.

As for how this works, it would be a post build step that modifies the assembly. The engine will then load the assembly and use the generated metadata to handle the remote calls.

This article indicates that it should be possible to debug assemblies that have been modified: https://johnhmarks.wordpress.com/2011/01/19/getting-mono-cecil-to-rewrite-pdb-files-to-enable-debugging/

So it shouldn't affect that.

This process of modifying assemblies is called weaving, and i've found a framework that does it. However, it's expected that you become a patron of the project if you use it, and that would mean everyone who uses SharpLife would also have to be a patron. Since this is supposed to be completely free i can't use that.

I don't expect that it will be too hard to implement a basic weaving tool by myself since it's largely the same as using Reflection, the only major difference being that i'll have to write IL instructions, which i'd probably have to learn anyway when using Fody: https://github.com/Fody/Home/blob/master/pages/addin-development.md

I think it may also be possible to use the same framework to optimize parts of the component system's internals, like the calls to non-virtual API methods (Update, Activate, etc), so that could prove useful as well.

I've merged the client and server assemblies into single larger assemblies.

SharpLife.Engine.Client & SharpLife.Engine.Server have been merged into SharpLife.Engine.
SharpLife.Game.Client, SharpLife.Game.Client.Renderer.Shared, SharpLife.Game.Server, SharpLife.Game.Shared have been merged into SharpLife.Game.

I've also updated dependencies to the newest version. Nothing worth mentioning.

I'll probably also merge the model format assemblies into a single assembly.

As far as the rest go, i'm thinking about merging a bunch of assemblies into a SharpLife.Core assembly containing SharpLife.Utility, SharpLife.Engine.API and some parts of SharpLife.Game.

SharpLife.Networking.Shared should be merged in as well, but i'll probably remove all networking code first and add it back later once i've redone the engine and game code structure. Otherwise it'll turn into a mess.

I've reworked the command system:

• Any number of contexts can be shared with any number of other contexts, but circular dependencies are not possible. The internal command context created by the command system is implicitly shared with all contexts (provides the "wait" command), all other contexts must be specified when creating contexts
• The command system and contexts now implement the Dispose pattern
• Variable change event handlers can now veto the change
• Filters are now implemented by aggregating all filters under a change event handler that is always handled first
• Variables can be made read-only
• Variables are now strongly typed, and conversion between string and the type is handled by a type proxy object that provides conversion functionality. The command system constructor takes a format provider used to controlling the formatting of numbers (e.g. 123.45 or 123,45 depending on culture settings). Variables can have custom type proxies specified on a per-variable basis, proxy commands can have custom type proxies specified on a per-parameter basis
• Variables can be virtual (existing version) or proxies to real fields or properties. Fields and properties can be read-only (or have a non-public setter) to implicitly be made read-only)
• Commands can be proxies to methods that take parameters. Command input will be converted to the target types. Methods can have default values to allow for optional command arguments
• Added the find command to search for commands using a keyword, potentially containing wildcards
• Added the help command to display the help info for a named command

Some examples:

//Virtual variable
private readonly IVariable<uint> _fpsMax;

//Creates a modifiable variable, constrained to the range [0, MaximumFPS] (uint does not allow negative values)
_fpsMax = EngineContext.RegisterVariable(
    new VirtualVariableInfo<uint>("fps_max", DefaultFPS)
    .WithHelpInfo("Sets the maximum frames per second")
    .WithChangeHandler((ref VariableChangeEvent<uint> @event) =>
    {
        @event.Value = Math.Min(@event.Value, MaximumFPS);

        var desiredFPS = @event.Value;

        if (desiredFPS == 0)
        {
            desiredFPS = MaximumFPS;
        }

        _desiredFrameLengthSeconds = 1.0 / desiredFPS;
    }));

//Proxy variable
public DateTimeOffset BuildDate { get; }

//Creates a read-only proxy variable of type DateTimeOffset
//Type is printed as 03/08/2019 17:21:41 +01:00
EngineContext.RegisterVariable("engine_builddate", () => BuildDate, "The engine's build date");

//Command
commandContext.RegisterCommand(new CommandInfo("echo", arguments => logger.Information(arguments.ArgumentsString))
                .WithHelpInfo("Echoes the arguments to the console"));

//Proxy command
private sealed class FindCommands
{
    private readonly ICommandContext _context;
    private readonly ILogger _logger;

    public FindCommands(ICommandContext context, ILogger logger)
    {
        _context = context ?? throw new ArgumentNullException(nameof(context));
        _logger = logger ?? throw new ArgumentNullException(nameof(logger));
    }

    public void Find(string keyword, bool searchInHelpInfo = false, bool searchInValue = false)
    {
        var builder = new StringBuilder();

        builder.AppendLine("Find results:");

        var flags = FindCommandFlag.None;

        if (searchInHelpInfo)
        {
            flags |= FindCommandFlag.SearchInHelpInfo;
        }

        if (searchInValue)
        {
            flags |= FindCommandFlag.SearchInValue;
        }

        foreach (var command in _context.FindCommands(keyword, flags))
        {
            command.WriteCommandInfo(builder);
            builder.AppendLine();
        }

        _logger.Information(builder.ToString());
    }
}

commandContext.RegisterCommand(new ProxyCommandInfo<Action<string, bool, bool>>("find", new FindCommands(commandContext, logger).Find)
    .WithHelpInfo("Finds commands and variables"));

The find command is a proxy command with optional arguments, it can be used with just a keyword or with the additional arguments. Finding out which arguments there are is straightforward, using the help command will show you:

help find
find: Proxy command find(String keyword, Boolean searchInHelpInfo = False, Boolean searchInValue = False)
Finds commands and variables

Find works much like help does but can find do partial searches, show multiple commands and perform wildcard searches:

find e*
Command exec
Executes a file containing commands
Command echo
Echoes the arguments to the console
Read Only Proxy Variable DateTimeOffset engine_builddate = 03/09/2019 17:09:58 +01:00
The engine's build date

find * will return all commands.

I'm still reworking the engine so it'll be a while before it gets back to where it was before. I decided to rework the command system now since most of the code that uses it was removed for the redesign.

I've completed the engine redesign. Here's a list of what's changed:

• The engine is now designed to use a shared codebase for client and server. Instead of separate types and assemblies everything is together to make it easier to write and use code that needs to be used on both client and server.

• Model formats are now mostly encapsulated by a single interface called IModelFormatProvider. This interface allows a loader and renderer to be provided for a particular format. This design will need some reworking later on to support renderers that don't use a specific format (i.e. beams).

• Maps are now represented in-engine as a scene, divided into the entities and graphical parts.

• Model loading has been greatly simplified. Loading is now done by using the ModelManager instance for a given scene, and graphics resources are loaded and uploaded to the GPU as needed. Models loaded at runtime load just fine. For studio models the error model will be used, but support for the error model for brush and sprite models does not currently work. The concept of a model index has been completely removed, as is the concept of precaching models.

• World state has been centralized in a WorldState class to make access easier. The renderer can be accessed from here as well, and for server code a dummy renderer will exist to make writing code easier.

• The old inheritance based entity system has been replaced with a component based system. Entities are now comprised out of zero or more components. Functionality like position, angles and velocity is now contained in the Transform component, models are subclasses of the RenderableComponent component, physics is handled by the Collider component.

Specific functionality is being reworked as discrete components to reduce complexity and make code reusable. For example, func_door's movement logic (inherited from CBaseToggle in the original SDK) is now its own component called LinearMovementLocomotor. Code specific to func_door (automatic return, touch, etc) is contained in the LinearDoor component.

• For backwards compatibility components can specify the original keyvalue name to use for a public member, as well as a custom converter should that be necessary. Converters handle conversion between string and the keyvalue type, with support for common types provided already.

In similar fashion spawnflags can be automatically converted to booleans. The SpawnFlag attribute allows you to specify which bit maps to a boolean.

• Game code is now reduced to just actual game logic. The renderer as well as physics code has been moved into the engine. It is expected that modders modify the engine itself as needed to suit their needs, although the game API should allow for most basic tasks.

• The game assembly is loaded dynamically as before in order to make it easier to work with plugins. The game assembly is listed in the engine configuration file as a plugin, and treated as such. This means plugins will be able to do everything that game code can do. Due to how these assemblies are loaded, plugins can override entity factories defined in the game assembly, allowing you to replace entities.

Since the component based design removes the use of specific entity classes this is no longer a serious problem and potential crash issue, but game code that works with classnames (should be nothing beyond logging) might not work as expected.

Plugins can be server, client or both client and server, although i don't plan to allow downloading of plugins. I do plan to support the use of CSharpScript scripts loaded at map start time, which can do server-only tasks like defining non-networked components, new entity factories and replacing existing factories. At present, plugins are loaded at startup time only, to allow all data related to them to be made immutable.

• The event system has been redesigned to simplify it and make it easier to use. It is no longer necessary to define events ahead of time, it is no longer required to inherit from the EventData class when defining custom event data, and all system operations will work even when in an event dispatch. As such the workaround for this, post dispatch callbacks, have been removed. To avoid state corruption any operations executed while in an event dispatch are queued up for execution after the dispatch has completed.

The event system is more memory and CPU efficient as a result.

• The object editor has been updated to support editing component based entities.

Some examples of the new stuff:
Defining a component with keyvalue name changes, spawnflags:

/// <summary>
/// A door that uses a <see cref="LinearMovementLocomotor"/> to move
/// </summary>
public sealed class LinearDoor : Component
{
    private LinearMovementLocomotor _locomotor;

    private ToggleState _toggleState = ToggleState.AtBottom;

    private Collider Activator;

    public Vector3 Position1;
    public Vector3 Position2;

    [KeyValue(Name = "wait")]
    public float Wait;

    [KeyValue(Name = "lip")]
    public float Lip;

    [SpawnFlag(1 << 0)]
    public bool StartsOpen;

    [SpawnFlag(1 << 3)]
    public bool Passable;

    [SpawnFlag(1 << 5)]
    public bool NoAutoReturn;

    [SpawnFlag(1 << 8)]
    public bool UseOnly;

    [SpawnFlag(1U << 31)]
    public bool Silent;
}

A basic light entity factory:

/// <summary>
/// Non-displayed light.
/// Default light value is 300
/// Default style is 0
/// If targeted, it will toggle between on or off.
/// </summary>
[LinkEntityToFactory(ClassName = "light")]
[LinkEntityToFactory(ClassName = "light_spot")]
public class LightFactory : EntityFactory
{
    protected override void GetComponentTypes(ImmutableHashSet<Type>.Builder types)
    {
        types.Add(typeof(Transform));
        types.Add(typeof(Light));
    }

    public override bool Initialize(EntityCreator creator, Entity entity, IReadOnlyList<KeyValuePair<string, string>> keyValues)
    {
        if (!creator.InitializeComponent(entity.GetComponent<Transform>(), keyValues))
        {
            return false;
        }

        if (!creator.InitializeComponent(entity.GetComponent<Light>(), keyValues))
        {
            return false;
        }

        return true;
    }
}

Defining a keyvalue converter for ints:

[KeyValueConverter(typeof(int))]
public sealed class IntConverter : IKeyValueConverter
{
    public object FromString(Type destinationType, string key, string value) => KeyValueUtils.ParseInt(value);
}

The next task is once again to get player physics working so interaction in the world is possible.

Amazing work, Solokiller.

Thanks

So i've been tinkering with WPF a bit to make a mockup of what an IDE for Half-Life modding could look like:Nothing functional, but the idea is to have a program that can manage all the assets in a mod, and access the tools for each one. It would provide shortcuts for doing things like making various assets, checking if you missed anything or if you did it the wrong way, previewing the results, etc.

For instance creating a sky amounts to taking 6 images with specific names, specific dimensions and file formats. This IDE could verify that it's all correct while you're creating it, and preview the skybox in a 3D window to see the orientation.

I was thinking of making this a single instance app with a system tray icon to allow it to run in the background, with a custom protocol to launch various tools, for example: sleditor://run/modelviewer/C:\Program Files (x86)\Steam\steamapps\common\Half-Life\valve\models\barney.mdl

This would launch the model viewer tool, passing the given filename along to view it. You'd then have a shell command registered that will execute that for any *.mdl file.

Since everything would be part of this single program they can share resources and keep resources loaded so it doesn't have to reload everything when you close the window. It would keep track of your current settings so if you changed your current mod, every active tool instantly knows and adapts.

Tools could hook into a global "check for problems" to see if anything is wrong with a mod's assets to find any problems that need fixing, e.g. models that have incorrect sequence filenames embedded in the main model.

The design for the program would be a small core program + extensions for each tool. An extension would work like Visual Studio extensions, which can add new commands that operate on specific file types and the like. That way it doesn't tie the tools in too closely with the IDE (and avoids requiring the SDK license on the entire codebase) and new tools can be added easily. Ideally they'd all be written in C# to directly interface with things like the filesystem, but extensions could just be wrappers around external tools or command line programs.

Once this is set up each tool can be written, designed to be both standalone and integrated with the same code to back both up.
I'd like to make this separate from SharpLife as much as possible, so i'd prefer a standalone BSP viewer that can do map editing much like what the object editor does right now. That way it will work for every mod.

It would still use the same code for that (engine renderer) but since it would be an extension, the licensing for the IDE wouldn't be affected.

The mockup uses WPF for NET framework, so until WPF Core is ready i can't start work on this. But if anybody has any ideas or feedback for such a program i'd love to hear it.

Man this is truly awesome!

You'll certainly receive feedback from me these days.

Posting this here for future reference: https://github.com/antlr/antlr4

Antlr is a library that can be used to parse structured text, like console input and the various text files used by the game.
Could be really useful for when all those files need to be loaded/converted.

Also, Valve is fixing stuff and interacting with us on the Half-Life Github tracker for the moment, so some stuff is getting done.

all these issues are based on reverse engineering code or have I miss something ?

Some of them, some are just based on user experience.

i still have no idea what this project is. Cant you just be straight? its tl;dr for me. Is this a custom engine based on the goldsrc code you got from valve?

It's a custom engine, but it's still loaded as a mod to remain compliant with the SDK license. It uses code from the SDK, but no engine code was given by Valve.

Mike from Valve did say he was looking into open sourcing so maybe we'll get something soon.

Heh, Valve decided to get some work done for a change.

The recent activity of mikela-valve sounds like a good news !!!

I really hope valve open sources goldSRC under a license like the MIT or GPL opposed to something like what the HL SDK End user license agreement.

where is detailed news about recent open sourcing?

None yet, Mike from Valve said he was looking into it and will create a new issue if more information becomes available.

The latest VS 2019 preview added support for XAML Designer for NET Core, so i ported my prototype for HLMV:This is just an early mockup but it's working so far, so i can start working on the new version now.

How are things coming along solokiller?

I've been mostly focusing on Half-Life maintenance on Github and working on Model Viewer (which helps get the shaders and shader invocation code finished). Nothing i can show right now, but i should have something to share soon.

Thanks for the update!

I've made the old SharpLife repositories private to avoid people trying to run it and running into various issues. I have a plan to make it easier to use but that will have to wait until HLMV is done.