SharpLife - Dot Net Core based modding p Created 1 month ago2018-05-06 19:15:02 UTC by Solokiller Solokiller

Created 1 month ago2018-05-06 19:15:02 UTC by Solokiller Solokiller

Posted 1 month ago2018-05-06 19:15:02 UTC Post #339507
I've been doing some research on porting Half-Life to C# using the Dot Net Core framework. I've got a working prototype that has C# code that can interface with the engine, though it's still far from complete.

I'm calling this SharpLife, since it's implemented using C#. It isn't limited to that language since it uses Dot Net but that is what i'm working with.

The goal is to port the SDK to C# and then upgrade it with Half-Life Enhanced's bug fixes and additions. Once that's done i can complete the planned changes for that.

Here's a diagram showing the overall design of the framework itself:
User posted image
Here are some examples of how it's easier to work with C# than C++:

[quote]
//Save restoring a field in a class in the SDK:
CBaseEntity* m_pGoalEnt;

static TYPEDESCRIPTION m_SaveData[];

TYPEDESCRIPTION CBaseEntity::m_SaveData[] =
{
DEFINE_FIELD( CBaseEntity, m_pGoalEnt, FIELD_CLASSPTR )
};

//Save restoring a field in a class in C#:
[Persist]
public CBaseEntity m_pGoalEnt;
[/quote]
//Getting the name of a class in the SDK:
const char* pszClassName = STRING( pEntity->pev->classname );

//Getting the name of a class in C#:
var className = pEntity.ClassName;
[quote]
//Loading an XML file in HLEnhanced:
https://github.com/SamVanheer/HLEnhanced/blob/346a9889f7da589f72cc66a71ee1202fc434714a/game/shared/CWeaponInfoCache.cpp#L154

//Loading an XML file in C#:
try
{
using (var stream = new FileStream("SharpLife/cfg/SharpLife-Wrapper-Managed.xml", FileMode.Open))
{
    var serializer = new XmlSerializer(typeof(ServerWrapperConfiguration));
var config = (ServerWrapperConfiguration)serializer.Deserialize(stream);
LoadMod(config.ModInfo);
return true;
}
}
catch(Exception e)
{
Log.Message($"{e.GetType().Name} - {e.Message}");
Log.Message(e.StackTrace);
return false;
}
[/quote]

It's much easier to manage dependencies than in C++ as well. To use XercesC in HLE you need to download, extract, compile and install it before referencing it in your CMake configuration.

In C# you only need to reference the NuGet package System.Xml.XmlSerializer. Once that's done everybody who checks out the codebase will be able to get the packages by restoring all NuGet packages for the solution.

For less experienced programmers (most modders) the language is easier to use as well. One mistake i've seen made a lot is string comparisons:
const char* somestring = ...;

if("ON" == somestring )
{
//Do something if ON
}
This compares memory addresses, which will always be false.

In C# it works like it does in most modern languages:
string somestring = ...;

if( "ON" == somestring )
{
//Do something if ON
}
And it does what you'd expect.

For scripting there's no need for Angelscript because you can just load assemblies that can directly access mod code. A simple plugin system would be a list of Dot Net assemblies that are loaded on mod startup that then behave as though they were part of the mod itself.

The language is very similar to Angelscript which is intentional, since Angelscript is based off of both C# and C++.

In addition, there's a scripting language called CSharpScript that is essentially C# code compiled at runtime, much like Angelscript: https://github.com/dotnet/roslyn/wiki/Scripting-API-Samples

By moving to this it'll be easier and faster to continue development, modders can focus on implementing game logic instead of working around language limitations and quirks.

It will take some time to implement, but right now the biggest issue is getting the native<->managed code interop working. I have the necessarily functionality done for that, so it's just getting the interfaces for the server done, then re-implementing everything piece by piece.

The priority is making sure the game can load and run with a minimal codebase, so most entities won't work yet for some time.

I'm hoping once that's done i can implement the physics code re-implementation, which will allow for parenting, and then if i have time and motivation implementing a new networking system to transfer data to clients. The client itself will also need to be converted, but it's a bit smaller than the server so that should be simpler.

Best case scenario i can skip the engine entirely and just do everything in C#, which would really make things easy but that's a lot of work and i don't know if i'll still have time and motivation for that.
Posted 1 month ago2018-05-06 23:10:59 UTC Post #339509
Wow. Hope this works out epically.
Posted 1 month ago2018-05-07 00:07:33 UTC Post #339511
Nice - this is something that I've wondered about for a while.
I guess it's possible but apparently not without its challenges.
Posted 1 month ago2018-05-07 02:18:29 UTC Post #339512
Yeah, very cool!
Posted 1 month ago2018-05-08 11:18:23 UTC Post #339519
Putting anything to do with Half life into a C# environment gets a big ol' thumbs up from me. Very happy to see C# get popular, Facepunch are doing a "gmod 2" by putting a layer of C# over UE4 for ease of use purposes. Would be great to see!
Instant Mix Instant MixTitle commitment issues
Posted 1 month ago2018-05-10 08:44:56 UTC Post #339538
Aha why do you not say me before C# is bad now you tried. I don't believe that. James said me I shouldn't use C#. Now Solokiller made it. And what are you using libraries?

MonoGame, OpenTK, OpenGL.net or SharpGL?

But it is really bad news: :cry:

Because I have already tried to use wavefront but wavefront looks for latest OpenTK damaged like you have used bad brush in Valve Hammer Editor with unlimited error-face.

Example my result of CrazyCore:
User posted image
For SharpGL works fine with wavefront files but it looks common incompatible for latest display cards.

You can't fix for mdl / smd because it has high calculation of vertices, normals, texturecoords, faces and indices...

If you have success. I will see your work - if you have luck.
Posted 1 month ago2018-05-10 09:10:30 UTC Post #339539
It looks like it's just the client and server code being ported - not the engine, so I imagine it's just plain old C# and the .Net standard library mostly.
Posted 1 month ago2018-05-10 09:27:39 UTC Post #339540
I'm only using Dot Net Core with compatible libraries, and native interop to run under the engine. I do want to eventually implement engine code in managed code to skip the engine entirely but that's not going to happen any time soon.
Posted 1 month ago2018-05-10 11:34:26 UTC Post #339542
Good for you - I will see if you have goal with models. I think you need assimp net ( it is old but it can't work for Dotnet Core 2.x :/ I saw assimpNet is still outdated.

Why do we need assimp? because it is important for loading unsupported model format example mdl of Half-Life or Quake 1/2/3 .

I think you need write via Dllimport with assimp.dll ( x86 / x64 ) into DotNet Core 2.x.

Good luck for next release!
Posted 1 month ago2018-05-11 08:22:53 UTC Post #339552
I'm surprised C++ is so verbose and isn't equipped with a dependency manager.

I understand it's less verbose, easier to manage dependencies, and run on all platforms without recompiling. I hope it's not too much off topic, but why C# specifically? :) For instance, why not Java? (I'm just curious :p)
Posted 1 month ago2018-05-11 09:17:28 UTC Post #339553
I can just write unsafe code to access model data, worst case scenario i can write my own loader and do what i've done in C++ before. It's not that hard.

Here's what i did to access edict_t: https://pastebin.com/DQ1qeXus

I haven't been able to test it yet, but it should work. This lets me access the native data while still using managed code in the game itself.

C++ actually does have a dependency manager, but as with most things C++ it's new and not terribly widely used yet: https://blogs.msdn.microsoft.com/vcblog/2018/04/24/announcing-a-single-c-library-manager-for-linux-macos-and-windows-vcpkg/

Even so it's still not easy to add dependencies, you still need CMake and old projects like GoldSource aren't built for it so you need ugly hacks to make it work.

I chose C# because it's the best tool for the job. The language itself is very easy to use and lets you design simple interfaces, like you can make a property that returns a string that internally could do a lot of work without exposing that detail. In Java you'd need a method for that.

You could do this:
//In BaseEntity
[Persist]
public string ClassName => pev.ClassName;

//In entvars_t
//Data is a pointer to the native entvars_t instance
//ToString accesses the string pool containing native string data
public string ClassName => Data->classname.ToString();
If i can ever get the engine out of the equation that can be simplified to:
[Persist]
public string ClassName { get; private set; }
In Java you can't do that. They've been playing catch-up to C# for a few years now, but C# is definitely the better language and the tooling support (Visual Studio) is much better than Eclipse/IntelliJ.

Native interop in Java isn't nearly as powerful as it is in C#. I read that they're planning to implement C++/CLI like support in Dot Net Core sometime, so perhaps next year i can simplify the interface even more by writing the native access in C++ instead. That would make things even easier, though by that time i may already have made the native parts obsolete.

That part's a bit tough, since there's a lot of engine code that accesses entvars_t and edict_t. There's 57 known accesses to entvars_t members that represent strings (string_t), the majority of which are classname checks.

Those can be avoided by handling entity creation entirely in managed code, since these calls are part of map spawning, entity creation, etc. If you override specific engine functions using the Detour framework then you can handle this directly in managed code.

If i can do that, and unregister certain console commands the engine won't touch it anymore. Eventually the idea is to not let the engine do anything, at which point the native interface would go away and the managed version would have its own implementation, or would be removed (edict_t and entvars_t really have to go).

I'm designing the game interface to make sense and be sane so most code wouldn't need updating. Direct access to edict_t and entvars_t would be handled in properties in common entity base classes (see example above), making the switchover easy and painless.

edict_t would probably be re-purposed to do what it does in Source, which is to store networking data and provide a way to represent players before their class instance is created (which i can fix if i have control over those particular engine functions).

If i can get this all done i can implement that custom networking i've talked about, which would let me implement uniform entity networking. It would look much like the persistence example:
[Networked]
public float Health { get; set; }
Should this not work and a notify-on-change type be needed, it wouldn't change the public interface:
[quote]
[Networked]
private Networked<float> NetHealth { get; } = new Networked<float>(0);

public float Health
{
get => NetHealth.Value;
set => NetHealth.Value = value;
}
[/quote]

All Networked instances in an entity would be hooked up to notify their owning entity so it can track the changes and network them at the next opportunity.

That would mean networked entities would exist on both the client and server side, could network any amount and type of variables and probably not cause reliable channel overflows anymore. cl_entity_t wouldn't exist anymore, the hacky weapon prediction system would be integrated into the rest of the system and we could actually build some neat things without worrying about engine limits.

Unfortunately this is all a lot of work and i have to finish up the native<->managed interface first, then port the entire server and then make the client version of all this. I can probably speed the client conversion up by writing a code generator for the interfaces, since it's largely the same thing over and over.

I'd still have to write a new GUI library either way since interfacing with VGUI(2) is not an option here. Any OpenGL based library should do here, as long as i can take the result and put it where VGUI goes it'll work, but there aren't many for Dot Net Core right now. I should be able to use the VGUI2 interfaces to render directly to the engine here though.

As far as content development goes, things should be a lot easier for you. I can implement a plugin system that loads assemblies that reference the mod, letting you access the game's code directly so you could add new weapons that way.
That is probably one of the biggest problems with Sven Co-op's scripting system, and one people have asked me about a lot, so i'm happy that this problem is now solved.

If i can implement the server<->client bridge i could add a debug UI that can directly access the server to take care of things that would otherwise be done through restrictive console commands. More expansive AI and scripting debugging would certainly help.

It's a bit too early to go into details about that, but i'm hopeful that this will let me finish the planned features for HLE and beyond.
Posted 1 month ago2018-05-11 10:20:41 UTC Post #339554
@SoloKiller, No, Java can work more than C#. I cancelled C# programming because I have tried since March 2018. No success SharpGL, Neo Kabuto's volume obj and other different ways. = No success.
My results: I really want create cube in C# but I want load from obj without material.
User posted image
It is using from Vector3 from OpenTK ( latest development myget )
User posted image
It is painful if I want load obj. It looks bad for C#.

I have made lessons / tutorials but I am very despairing. I have more ways like C++ and ThinMatrix's tutorial series with lwjgl ( Java )

Java is richer than C# because C# is poor like example Math class

ToRadiant(float angle) hasn't in Math of C#.

Java has more Math methods but I saw like example Quake and Half-Life works fine 100 % in Java with LWJGL 3.1.6/3.1.7 and it has collision but you need create own so close wavefront ore different formats.

Example FloatBuffer, IntegerBuffer etc... has not for C# and has for only Java.

C# is sensitive for only Vector3[] or Vector2[] - maybe it can't show if it doesn't know Vector3, Vector2 or float. It has sometimes problem.
Java can work whatever same to float[], Vector3f[] or Vector2f[]... = No problem.

Example:

You want load and read wavefront in Half-Life with C# but you forget. C# can't

Java works fine almost than C#.
Example: Lwjgl by ThinMatrix: Video

That is reason - Why I cancel C# programming? Because C# can't work for wavefront and other formats.

I wish you have to start with Java because Java is powerful since initial Quake 2 in lwJake2 with Lwjgl 2.9.3 initial test. I have downloaded and compiled and I have tried and it works fine like C++. Java =/= C++ and C# < C++ or C# < Java. But if you make sphere like small faces than LwJake2 crashed since I have tested with JackHammer and LwJake2

Check LwJake2

And it was initial test

That is proof. I don't see lwjgl 3.1.x ( latest version ) because it is very stable for games no lagging or no crashing.
Posted 1 month ago2018-05-11 12:07:55 UTC Post #339555
If you're having issues loading data or rendering it then it's not caused by the language, that's the library you're using or the code you've written. I've had issues with that in C++ as well, it doesn't make C++ crap.

Java's way worse than C#. I've worked with it for years and it is missing a lot that C# has, and i wouldn't use it for something like this. C# has much better native code support than Java which by itself is already a reason to use it here because i need to do a ton of marshalling and native code access.

Package management is pretty much automatic in C#, whereas Java doesn't have that. I wouldn't want people to juggle dependencies all the time, which is a problem that HLE is suffering from due to the CMake based approach requiring a lot of manual building of dependencies.

My college is even switching their courses from Java to C# because it's much better and easier to teach and use.
Posted 1 month ago2018-05-11 13:40:20 UTC Post #339556
Okay your decision with C# I hope you have to prepare OpenGL-Wrapper like OpenTK ,OpenGL.net etc...

Good luck! :crowbar:
Posted 1 month ago2018-05-12 16:59:39 UTC Post #339561
Like... I'm struggling to learn and and apply a programming language (currently mainly using Java) to make a living, and many times I feel like an impostor.

But after seeing this...
User posted image
*goes out for a walk Escobar style
Striker StrikerI seriously doubt myself
Posted 1 month ago2018-05-12 19:54:26 UTC Post #339566
Java and C# are MOSTLY similar when it comes to the syntax, the main differences are the keywords ("final" -> "const", "package" -> "namespace"...) and other minor things. The transition from Java to C# should be fairly easy.
Shepard62700FR Shepard62700FRHalf-Cat is watching...
Posted 1 month ago2018-05-15 16:52:28 UTC Post #339586
Okay your decision with C# I hope you have to prepare OpenGL-Wrapper like OpenTK ,OpenGL.net etc...

Good luck! Headcrab Basher - :crowbar:
I won't have to touch OpenGL for some time, there seem to be half a dozen libraries available so i'll take a look when it's time to work on it.
Like... I'm struggling to learn and and apply a programming language (currently mainly using Java) to make a living, and many times I feel like an impostor.

But after seeing this...
This is really advanced stuff that most programmers will never have to do, so don't be discouraged by it.
Java and C# are MOSTLY similar when it comes to the syntax, the main differences are the keywords ("final" -> "const", "package" -> "namespace"...) and other minor things. The transition from Java to C# should be fairly easy.
C# can do anything that Java can do, but Java can't do everything that C# can do.

Events, properties, directly calling delegates from native code, Java either can't do it or is much worse at it. I'd much rather be using C# than Java if i had a choice.

I've implemented the Detour framework so i can improve performance in some areas. Entity creation was really slow so i hooked it and re-implemented it in managed code. Mods can now control the entire process, it allows the creation of templates that can be spawned with a point_template like entity.

This means there is no maximum length limit on keys and values, no limit to the number of keyvalues that an entity can have and no limit to how many entities are defined in a map, although the engine's limit still applies during map spawn time.

I've also implemented support for linking classes to their map name, handling multiple names (some weapons need this), and partial support for keyvalues declared as properties, fields or methods.

I'll be implementing engine functions so i can implement the game interface, which will allow networking and such to work. Then i can implement some entities so maps work and i can see what else i need to implement before it's functional.

I'll probably need to detour more engine functions for performance reasons, which means physics will be re-implemented in managed code as well eventually. That also means parenting support.

Saved games will also need to be detoured to avoid problems with saved games being too big since the maximum size is hard-coded, which will allow the game to control all of the data being saved.

I'll get all of this stuff on Github soon so i can show the code. It's not ready for use yet and the framework seems to be triggering Windows Defender in a way that causes it to use up to 6 GB of memory at times. I don't know why this happens but it may be because it detects a custom CLR host.

It also seems to be causing crashes on startup (before my code is even loaded) so i'll have to figure out how to prevent this before it can be released.
Posted 1 month ago2018-05-17 18:51:38 UTC Post #339640
Once upon a midnight dreary there was 8-bit. It was invented of the technological limitations of its day. As technology grew more powerful and advanced, so did everything around it. Graphics changed and polycounts increased 20% by the year. Yet the diehards stay behind and give their favorite titles the love they deserve before they get buried in the sands of progress.

Along comes the indie developer. Mr. Indie doesn’t have the tools, talent, processing power, or even the man power to compete with AAA title quality. He lives by the only rule he knows; do what you can, with what you have, right here, right now. And thus he studies his 8-bit ancestry. There is a fundamental difference between then and now as he creates his game. He has better tools and far more computing power than his ancestors. With this, he creates with less limits, so his game naturally has less technical limits. He doesn’t have to consider palettes, or color count, z-fighting, or fitting his game into a 2mb cartridge. Out of a lack of necessity, a new style had been created, this is high-bit.

It is time we see what we can do with goldsource with better tools, better computers, and without limits. It’s come a long way so far, and it still has a long way to go. Let’s high-bit this bitch.
Posted 4 weeks ago2018-05-26 20:18:25 UTC Post #339709
I've put the source code for the wrapper and the game on GitHub:
https://github.com/SamVanheer/SharpLife-Wrapper
https://github.com/SamVanheer/SharpLife-Game

There is no documentation on how to use it yet (you'll need it), and it's still in development so it's subject to change.

Here's the list of changes since my last post:
-No more crashes on startup, unknown what caused it but i haven't experienced it anymore. Defender still uses a ton of memory though

-Implemented a bunch of entities, the training map can be loaded and it has triggers, doors, ladders, letting you move to the end of the map. No changelevel yet, that's partially implemented but it's missing the code to transition entities, which is part of the persistence system

-Player physics is implemented and seems to be mostly working, with the exception that movement on ladders seems to have some issues. There might be missing code in some other area, i'll look into that

-Most of the base classes have been implemented, aside from logic that handles bullets in BaseEntity everything up to BaseToggle is finished

-multi_manager and multisource have been implemented and support an unlimited number of targets now

-You can register cvars that are integrated into the engine, it's nothing fancy but it lets you work with it

-Entity data loading works properly and can handle most inputs quite well

-With the exception of networking code nothing accesses EntVars directly, it's all done through properties which will make switching over much easier

-Edict is avoided wherever possible, most references are in code that accesses the engine and some are wrapped to reduce the number of direct uses

-The node graph and func_train use EHandle to reference entities, which makes the code more reliable since it can't refer to other entities by accident

-The materials file is loaded, all supported materials are defined and contain all relevant data defined in one place: https://github.com/SamVanheer/SharpLife-Game/blob/3af4cda9fc5ec3ed74fd0df38268a41cf6883ab8/src/Server/Game/Materials/MaterialsSystem.cs#L29

Currently hard-coded, but this can be updated to load materials data from config files. It eliminates most of the redundant code and simplifies the logic when it's used

-The sentences file is loaded, its behavior is slightly different but should still be compatible with the engine's version. Once the client is implemented i can handle it there and remove some of the backwards compatibility stuff

-Unused code has not been converted, there are unused functions, class members, as well as logic meant to protect against engine behavior that has since changed. This simplifies things a bit

There are random crashes in the engine that happen from time to time, i'll have to debug that and see what exactly is happening.

I've looked into existing persistence support in the Dot Net framework, it seems that the best solution in this case is to use XML serialization. This will be mostly automated and should work like the original, but without the complicated binary logic.

Here's an example of how an entity is defined in the SDK and the C# version:
https://github.com/ValveSoftware/halflife/blob/5d761709a31ce1e71488f2668321de05f791b405/dlls/plats.cpp#L628
https://github.com/SamVanheer/SharpLife-Game/blob/3af4cda9fc5ec3ed74fd0df38268a41cf6883ab8/src/Server/Game/Entities/Plats/FuncTrain.cs#L22

If you look through the code you'll see some patterns. Spawnflags are defined as a static class named SF that contains constants, that makes it easy to find and reference them.

I've looked into the engine's save game code, it looks like i should be able to override it by handling a few key functions, but i'll need to grab an engine data structure to initialize it properly when saved games are loaded.

I've also looked into the physics code, most of it is handled by one function: https://github.com/id-Software/Quake/blob/bf4ac424ce754894ac8f1dae6a3981954bc9852d/WinQuake/sv_phys.c#L1507

I can take the code from Quake and modify it to match what the engine does, then i can start moving properties from EntVars to BaseEntity. That'll improve performance and reduce memory usage a bit.

Key physics code is only referenced outside of that function hierarchy and engine functions in the player physics code, which i can also override.
That code is what converts edicts to physents, so i might be able to convert all of that to BaseEntity as well at some point.

I won't be able to do that until i've got control over the client as well, since it uses the same code.

Once i do have the client done i can make a networking system to override the engine's version. Then i can start overriding the big stuff like precaches and entities. That's probably going to take some time though.

The big question is whether i should focus on framework stuff like that, or implement all of the game's code first. The former lets me get the entire framework up and running, but leaves it unusable as a modding platform until it's complete, the latter gets it operational but limits functionality to what the SDK can do, with a few additions.

I think the former is the best solution, since it lets me get the client done sooner as well, but if anybody has an opinion, i'm all ears.
Posted 3 weeks ago2018-05-27 19:44:59 UTC Post #339710
Former definitely seems like the best option, if you rush to support operationality people will observe it as sort of "Spirit of HL 2" rather than fully realise the potential of what's at hand. Feel like people need stuff to work and play with from the get-go rather than trickle down over time.

Having an initial framework also allows the opportunity for there to be decent documentation from the get-go, something which UE4/UDK seriously suffered without in its infancy.
Instant Mix Instant MixTitle commitment issues
Posted 3 weeks ago2018-06-02 18:57:36 UTC Post #339776
I've been working to get the client operational, i've got everything up until the managed wrapper done, now it's time to get the actual game code ported over.

Unfortunately i've hit a problem: SDL2 names its threads in a way that causes the game to shut down if it's launched with a managed debugger attached.
This behavior can be disabled, but not in the SDL version used by Half-Life. I've contacted Valve and asked them to update the SDL version, but i doubt they'll do anything about this.

This could also explain the seemingly random crashes i've experienced, if a thread is started and named it would cause the game to exit immediately if ran with a managed debugger so there's that. I'll keep an eye on the exit code to verify this.

Fortunately i can avoid the issue on startup by launching without a debugger and then attaching it, but it is a pain in the ass. Eventually i can ignore the engine and then this problem will go away, but for now it's an issue.

When i ran it without the debugger it seemed that it works without crashing, the player's view is stuck at world origin and nothing is rendered because visibility testing and physics isn't present yet, but that should be relatively easy to implement.

Once i've got that done, i can start working to add a networking layer to build on top of. That'll be the basis of the engine replacements, starting with sending large amounts of data, such as weapon and ammo types, sending a copy of the precache lists, etc. I've got some ideas about optimizing networking that could help but that remains to be seen.

I'll also be looking at available Dot Net Core GUI libraries, there aren't many from what i've heard so i may have to build one. It only has to be able to do what VGUI(2) can do, so a custom one wouldn't be that large. As long as it can render to an OpenGL texture/buffer it should be good.
Posted 2 weeks ago2018-06-03 15:42:06 UTC Post #339795
I've implemented enough to load a client game project now, it's using the same player physics code as the server but nothing's rendering now as before.

I'm considering merging the server and client libraries since they'll be sharing a lot of code, including entities. It makes sense to use the same code with server and client specific providers for things like networking to make things easier, it enforces good design and separation.

I know Minecraft does this, mods are both client and server capable but you can tell it whether to enable the interaction on the client and server sides.

If i were to do this i could probably optimize singleplayer servers by not networking anything and using the same entities for both client and server. This would reduce memory usage and time spent "networking" (sending over loopback) things but could cause changes in behavior when data that is not sent at 100% accuracy is accessed on the client.

I'll have to think it over some more, i could just as easily use a library that shares entity code but one library means it's all in one place. That could be very useful for things like tools which could load the library and query for any data that they need, for instance it should be possible to generate FGD files with the metadata present in the server library.

Right now i'm going to add the code needed to render the game world correctly (telling the engine that entities are visible so it renders them), and then making sure input is handled. This code will need to be updated if i move away from the engine's code but having a working implementation should make it easier to convert.
That way any bugs added during conversion can be fixed without needing to look through my own version of the engine's code.

If all goes well i should have a functioning - if bare bones - C# based modding platform by the end of the week.
Posted 2 weeks ago2018-06-03 17:36:59 UTC Post #339797
I think that this is an awesome project.

Always happy to see somebody else working on a project idea that's so ambitious it seems impossible, but slowly coming up with results. :D
Posted 4 days ago2018-06-19 14:50:12 UTC Post #339964
I experimented with Protobuf a bit to see how easily i could make a messaging system:

//Tell the client to change its name to Solokiller
var message = new StuffText
{
    Text = "name \"Solokiller\"\n"
};

//Represents the server's outgoing message buffer for the given client
var outboundPacketStream = new MemoryStream();

//Queues this message for sending to the client
//First we add a list of messages so the receiving end knows what's coming, and how to parse the messages
var messages = new MessageList
{
    Messages =
    {
        new Message
        {
            //The Id is not defined as an enum in a .proto file because we have both server-to-client and client-to-server messages
            //Using the same enum is a waste of bandwidth (variable length encoding would be larger for no reason) and duplicating the message class is wasteful
            //This cast is needed in only 2 or 4 places (sending messages, receiving them) depending on whether we'll use a generic handler type or one that's hardcoded to the Id type
            Id = (int)ClientCommandId.StuffText,
        }
    }
};

//Write the list, then each message
messages.WriteDelimitedTo(outboundPacketStream);
message.WriteDelimitedTo(outboundPacketStream);

//The client's buffer for processing
var inboundPacketStream = new MemoryStream();

//Would be a net packet send to the client, read by the client into a buffer for processing
outboundPacketStream.WriteTo(inboundPacketStream);

//WriteTo sets the position to the end, so this is needed to read from the start
inboundPacketStream.Position = 0;

//Read the message
var messagesResult = MessageList.Parser.ParseDelimitedFrom(inboundPacketStream);

//Process each message
foreach (var messageResult in messagesResult.Messages)
{
    //Could make a dictionary mapping id to a type containing the parser and a handler method to process the incoming message
    switch ((ClientCommandId)messageResult.Id)
    {
        case ClientCommandId.StuffText:
            var result = StuffText.Parser.ParseDelimitedFrom(inboundPacketStream);

            Console.WriteLine($"Command is: {result.Text}");
            break;
    }
}
So this code makes a StuffText message, sends it, parses it on the "client" and gets the text.

The Quake equivalent looks like this:
https://github.com/id-Software/Quake/blob/master/WinQuake/host.c#L322
https://github.com/id-Software/Quake/blob/master/WinQuake/cl_parse.c#L800

This approach is safer and easier to use. Messages are sent with their size so malformed messages will never cause problems (svc_bad is an indicator of this sort of problem), you can also do validation by marking fields in the .proto file.

Protobuf has no overhead, all it sends is the message data. I'm using the delimited write/parse methods here so a variable length integer is prepended to each message, but that's necessary because the parser wouldn't know where messages end otherwise. If you were to convert all network messages to use this it would be more efficient because Protobuf can encode data more efficiently, for instance you can send entity indices with fewer bits if it's a low number since it can use variable length encoding.

You can read more about encoding here: https://developers.google.com/protocol-buffers/docs/encoding

Eventually the idea is to let you send messages to clients by creating these messages and passing them to the engine's networking system, somewhat like this:
Networking.SendMessage(client, myMessage);

Where Networking is an instance of an object that can send messages, SendMessage is a method that can send to a specific client, client is the target client (index or client instance, probably the former) and the message itself.

Adding new messages is easy, the engine's current limitations won't be a problem here (192 byte maximum for user messages).
It would probably look like this:
Networking.AddMessageType(typeof(MyMessage), MyMessageHandler);

Where MyMessage is the generated class that it can get the parser and metadata from, MyNetworkingClass is the class where you handle your incoming messages and MyMessageHandler is a method with this signature:
void MyMessageHandler(MessageData<MyMessage> message)

Where the MessageData type contains the message itself as a member Message, along with anything else that's relevant (probably nothing, but it allows for changes).

The ability to remove and re-add handlers should also be possible to allow for changing the handlers for cases such as multiple HUDs, where some handlers will depend on the current HUD instance to process it.

So the idea behind this is to make networking a lot easier. Variable length encoding and packed fields will reduce message size, optional fields makes it easy to handle messages with multiple options (e.g. SVC_TEMPENTITY has over 100 possible options, some of which have optional members), and because it's based around code generation it's all strongly typed. No more screwing up a temp entity message because you sent a byte as a short or anything like that, it'll tell you if you made a mistake like normal programming.

It's also possible to send messages to programs that use a different language, so this could be used to send messages between C++ and C# if the need arises. I doubt that will actually be needed, but this could be used to send messages between the game and any tools, like Hammer. That's necessary if the tool is running as its own process, in which case inter-process communication is needed, which makes sending data a lot harder since you can't just pass data around like you normally would.
You could for instance serialize objects to protocol buffers to pass them around, though for that JSON might be a better solution.

It's also easy to log entire messages for debugging since the message size is known, as is the list of all messages in a packet. You could use this information to dump an entire packet for analysis if the game detects something is wrong (e.g. svc_bad errors, assuming those can happen with this design).

The next step will be to look into a good networking system to use, then build something that combines these two things.
Posted 1 day ago2018-06-22 11:20:25 UTC Post #339996
I've looked at a few networking libraries and it looks like Lidgren is the one to go for: https://github.com/lidgren/lidgren-network-gen3
It's recommended by most and provides everything that's needed to replicate the original system.

It also supports DotNet Core, albeit requiring a manual build. It should be possible to distribute the NuGet package itself and load it up locally, as i've done. That's relatively simple to do compared to the C++ dependency management approach.

Unlike GoldSource Lidgren doesn't impose any packet size limitations, but hardware and software limitations in the network infrastructure still apply here. Once a packet grows beyond a certain size it will be fragmented and reassembled on the other side, which means high amounts of packet loss can still cause problems. I may have to look into creating something that can send data in discrete packets to limit the effects of packet loss, but that will require further investigation to see if it's a real problem or not. I'm not too familiar with that stuff so it may not be a problem.

In any case it means the "reliable channel overflow" errors should be a non-issue. Timeouts, high ping and excessive packet loss should be the only reason to disconnect clients.

As far as GUI libraries go i've found that Imgui has a .NET Core wrapper: https://github.com/mellinoe/ImGui.NET
This should make implementing debug overlays simple, i know suXin from Facepunch has done this before so it should be simple to implement.

There's also a library for graphics programming: https://github.com/mellinoe/veldrid

It's low-level, cross platform and supports multiple backends, so i think this is a good way to support Vulkan and D3D.
It uses SDL2 for window management, and the library names are the same as Half-Life's so there is a need to modify the source. This is because Half-Life uses an older SDL2 library that may be missing functions. I may need to make a custom build for this to work out. I see that Veldrid uses a library to load native libraries, i may be able to provide a custom SDL2 path to load it.

For keyboard and mouse input i can also use SDL2, but i'll need to use a .NET wrapper here that also works together with Veldrid.
The wrapper i found uses DllImport which may not cooperate with manual library loading: https://github.com/flibitijibibo/SDL2-CS

Normally any given library is loaded once, so the goal will be to make sure the right SDL2 library gets loaded. If i can control where it looks for native libraries i can use a custom one and bypass the entire issue, otherwise i'll need to modify the source code of all involved libraries.

One option is to use this: https://msdn.microsoft.com/en-us/library/system.runtime.interopservices.defaultdllimportsearchpathsattribute(v=vs.110).aspx

But that only affects the assemblies that it's applied to. What i'm thinking is i can do this: free the original SDL2 library, load the correct one in the engine assembly, marked with that attribute, and then rely on the other load calls to use that library. This depends on the SDL2 library used by the original engine being freed, which may not be possible since it's a load time linked library. I'll have to verify this, otherwise using renamed libraries is the way to go.

That should cover networking, (debug) graphics and input. Sound can be handled through FMOD, that has a C# wrapper. The engine doesn't use it so there's no problem there.

I already have the filesystem code written up, though some changes are needed here and there. I'll be ditching the registry part of its initialization since that forces it to have Windows only dependencies, it'll be stored in a general purpose engine config file, along with things like the default game directory ("valve").

I've also built a command system, i've taken into account the problems that Valve had with engine upgrades to the classes. The problem there was that the class layout couldn't be altered without breaking mods, i've solved this by not making modders create instances of the classes. Instead, you provide info objects that describe commands, which are then used to construct the classes. Combined with C#'s delegates allowing class member functions and the compatibility between libraries, it should solve all of the problems that the command system has had since Quake.

As an example, here's the alias command:

RegisterConCommand(new ConCommandInfo("alias", arguments =>
{
    if (arguments.Count == 0)
    {
        Console.WriteLine("Current alias commands:");
        foreach (var entry in _aliases)
        {
            Console.WriteLine($"{entry.Key}: {entry.Value}\n");
        }
        return;
    }

    // if the alias already exists, reuse it
    _aliases[arguments[0]] = arguments.ArgumentsAsString(1);
})
.WithHelpInfo("Aliases a command to a name"));
Compare with the original: https://github.com/id-Software/Quake/blob/master/WinQuake/cmd.c#L341

The registration methods return interfaces to the commands to allow you to reference them, add more handlers, directly query variables, etc.
Commands and variables both allow you to add multiple handlers, so for instance if you have your cheat cvar "sv_cheats" defined somewhere, you can do this anywhere:

var svCheats = commandSystem.FindCommand<ConVar>("sv_cheats");

svCheats.OnChange += (ref ConVarChangeEvent conVarChangeEvent) =>
{
    if (!conVarChangeEvent.Boolean)
    {
        players.ForEach(p => p.DisableCheats());
    }
};
This adds a handler that checks if cheats have been disabled, and if so disables cheat properties on all players.

Variables also have filters that are checked before their values are changed. It's a pretty simple thing, if the filter returns false the change is not allowed. Filters can also modify inputs to for instance clamp values to a range, similar to Source's cvars. I've added filters that can do a few basic things, like testing if inputs are in a list, allowing only numbers, inverting filters, etc. For example, sv_cheats can have a boolean filter applied that restricts all inputs to 0 or 1.

Filters can be added and removed as well. Since both filters and change handlers are events, removing handlers you don't have a reference to is impossible, at least unless you use reflection to get the underlying event handler. It is thus difficult to accidentally break existing handlers.

Once all of this stuff is set up, the essentials should all be there. When i can draw stuff onscreen debugging will be easier, and i can start writing libraries that can load the various file formats used by the engine (bsp, mdl, spr). These will be standalone to allow for re-usability, hopefully useful to make new tools with.

If i can find the time i should be able to set this stuff up soonish, but i can't promise anything. I'm hoping there won't be any more unexpected problems, but you never know with these sorts of workarounds.
Posted 19 hours ago2018-06-23 09:54:22 UTC Post #340003
The new SharpLife engine is running under Half-Life as a mod now, having taken over execution. Now all further development should be done in C# only.

The engine codebase is now on Github: https://github.com/SamVanheer/SharpLife-Engine

I've worked around the SDL2 issues by renaming the library used by the managed libraries. This means there are 2 SDL libraries being used.
It doesn't appear to be causing problems so far though.

EDIT: spoke too soon, using multiple libraries causes problems because the engine creates a window as well, the library attaches data to the window which both libraries recognize so it ends up calling into the wrong one. I had to change the source code so it uses a different tag so it doesn't conflict.

There appears to be a bug with SDL2 on Windows 10, i emailed Valve about it: https://github.com/ValveSoftware/halflife/issues/1887

This should also let me use the same library, thus solving the problem entirely for this project.
You must be logged in to post a response.