SharpLife - Dot Net Core based modding p Created 4 months ago2018-05-06 19:15:02 UTC by Solokiller Solokiller

Created 4 months ago2018-05-06 19:15:02 UTC by Solokiller Solokiller

Posted 1 month ago2018-08-10 12:59:32 UTC Post #340469
I've added a table of engine components that are required and their status, it should make things a little easier to keep track of: https://github.com/SamVanheer/SharpLife-Engine/wiki/SharpLife-engine-components

I've been doing some experiments with making a good entity list design, since the one used in the SharpLife interop version wraps the original version.
I've come up with one that's pretty simple and efficient and that avoids global variables, so that part should be good to go once the networking system is done.
Posted 1 month ago2018-08-11 15:54:45 UTC Post #340487
You like commandline? You can use Mono's Console ( No Net core only net framework :/ )
Posted 1 month ago2018-08-12 18:39:30 UTC Post #340502
So Solo,

tell us none coders more about this new thing of yours. The way I understood it you switched codes but it's essentially still the same Hl: Enhanced is that correct?
I understand that the new code basis provides even more possibilities later on, or at least for coders like yourself. But tell us how will us the simple mappers benefit of these changes and how did it affect your total schedule regarding a full alpha or beta release?

Will this switch won't set you back yet another few years? Or is it actually the opposite. So that you're actually managing to get things done and implemented a lot faster than with c++ before?
In either case I do hope you will get a build out that SP mappers can actually use with hl1,op4,custom npc content usage. For me it seems you're stuck and limiting yourself with MP features at the moment. A thing that's totally fine of course . Yet I still do I wonder about your ultimate goal here.

Sven Coop is an established IP by now. It is doubtful that anything can push it away these days. So naturally I fail to see the focus on MP based content not that I complain. Competition is always a good thing for business. How ever only if it makes sense.

Also keep in mind that TWHL. is mostly focused on SP scenarios and so are the remaining HL1 fans. People that do love Half-Life and it's MP usually only play counter strike global offensive nowadays and that isn't even Gold Source anymore. Instead it is Source 2 or Source 2.5 as some people call it.

Tell us about your current plans and goals... Anyway I do appreciate you and all your efforts.. So don't get this the wrong way in any aspect. :)
Posted 1 month ago2018-08-12 20:36:21 UTC Post #340505
@23-down Solo works with C# and library with veldrid for Net Core 2.0
He works hard and is very busy. If you want know how do you write C#? If you know it than I respect you. :)
Posted 1 month ago2018-08-12 21:26:01 UTC Post #340506
The idea behind the project is the same, it's a different approach though.

C# is a much easier language to work in, i don't have to worry about some issues that make API design hard in C++. I've made much more progress working on this than i did working on HLE, and that has to do with there being much better third party libraries to provide functionality. The .NET framework itself also covers a lot of features that are missing in the C and C++ standard libraries, which eliminates the need to provide those myself.

The language also provides much better debugging features, if you crash you'll be able to see what happened in the log, unless the crash was so serious it never makes it to logging the error.

At the rate i'm progressing i should have the essentials for entities up and running in a few weeks, which means i'll have model loading done which should let me add the minimum required entities to make a map.

I'm not limiting myself to MP features, these things are necessary even in SP because that's how the engine works. Using the same way to send data to the client makes things a lot simpler.

Sven Co-op has nothing to do with this project. It's just another mod that lets people do stuff, i'm working on this for the same reason that i joined that team in the past, which is to make something nice for people to play around with.

Getting a working build is not just a matter of implementing the minimum required engine features, without the entities it's not going to let you do much.
Once i've got the object networking code done the networking part of things is largely finished. It will require fine-tuning and optimization, but if it works it's enough to start building on top of it.

I've actually covered a large part of the engine already, the biggest missing feature is a proper GUI, which is a fairly low priority since the console provides enough control to get things done for now.

Feature wise this engine eliminates most of the engine limits. The design i've got planned for networking allows for up to 65535 entities to exist at any one time. This limit is dictated by the size of the data type used to store entity indices, and is easy to change. The rest of the codebase should adapt automatically to increasing numbers of entities. I can probably do the same for map sizes, where i can change the networking configuration for coordinates based on the size of the map to optimize bandwidth usage.

The limits you know about the original engine are things like texture size, map size, entity count, player count, etc. All of those are either gone or configurable so it's not really a limitation anymore.

Because this engine uses shaders graphics performance is much better. That means r_speeds is no longer something to keep in mind, so you can make bigger and more detailed maps. I can add things like Source's env_fog_controller to let you control visibility (and add fog properly). Beyond that i can't say what you'll be able to do since i'm still working to implement the required components.

I'll try to explain the benefits of new features in simpler terms in the future so it's easier to understand.
Posted 1 month ago2018-08-13 10:11:10 UTC Post #340509
@SoloKiller,
Really you like add env_fog_controller to Half-Life? Wow it is great news! But please please add more shadow mapping like mod AM by SysOp.and realistic reflection for water and more.. I hope you can write more. Yeah shaders like they should embed in net assemblies without

Like my code:
Shader myshader = new Shader(MyVertexShaderSrc, MyFragmentShaderSrc)
Or
Shader myShader = new Shader(Light.VertexShaderSrc, Light.FragmentShaderSrc, Light.GeometryShaderSrc);
etc

It is really easy to write like strong and powerful performance.

Link to Glist

That is example Shader with shaders - you don't need to worry for "embedded" shaders. like SharpLife works without forgotten shader files. If you like my shader or you have to prepare better shader with UniformLocation binders.
Posted 1 month ago2018-08-13 12:22:54 UTC Post #340514
Are there going to be additional capabilities like a user level scripting api to allow mappers to control more of the gameplay? Like maybe better storing and altering of game data like max health/armor or max bullets amounts and custom monsters that actually work?
Rimrook RimrookGoldsource Guru
Posted 1 month ago2018-08-13 14:09:29 UTC Post #340518
That should be possible using CSharpScript: https://github.com/dotnet/roslyn/wiki/Scripting-API-Samples

Those examples are pretty basic, but it should be possible to build a scripting API that enables you to create your own entities and access other entities as needed.
Custom monsters should work fine, but networking custom data may not be as flexible as it would be if written in C#.

For plugins, custom assemblies can be used instead. You write C# code and tell the game to load it as a plugin, you can then provide entities, hook into other objects and do what you want to do.

Custom monsters largely need support for schedules. I'm going to use a new design for that instead of using Half-Life's approach, it should be more modular so you could do something like this:
public Barney()
{
   this.AddSchedule(new Follow());
}
Follow would be a class that implements an ISchedule interface, providing a list of tasks to complete in that schedule, as in Half-Life. The big difference is that the schedule and task code would reside in the schedule object, and be flexible enough to avoid knowing the actual NPC type most of the time.

A schedule that has an NPC attack something would instruct the NPC to attack using the preferred weapon for a range (close range: don't use weapons that deal splash damage, long range: use accurate weapons). Barney would always use his pistol, so it would look something like this:
public virtual void AttackTarget(BaseEntity entity, float range)
{
    //DefaultWeapon is a property that returns the default weapon for an NPC. NPCs can override it to provide logic to decide what the default weapon is at a given time, but complex logic belongs in this method
   var weapon = DefaultWeapon;
    //NPCs that don't have weapons don't attack
    //They shouldn't have schedules that would instruct them to attack, but if they do, it can be handled here
   if (weapon != null)
   {
      AttackTarget(entity, weapon);
   }
}

public virtual void AttackTarget(BaseEntity entity, BaseWeapon weapon)
{
    //Figure out where to aim, tell the weapon to fire, etc
}
Implemented in the base NPC class so any NPC will try to attack if they have a weapon.

Most of the hard logic would be handled by base classes, weapon classes will take care of actually firing the weapon (weapon_9mmhandgun fires a 9mm bullet fired by Barney) and handling magazine size and reloading. Other weapons, like the Alien Slave electrical attack would also be a weapon for consistency, meaning it's possible to turn it into a player usable weapon (a la Decay).

Making a basic NPC like Barney would mostly be combining existing schedules shared between NPCs, setting the correct ally settings ("i am Black Mesa personnel") and the weapons the NPC can use. This could be turned into a configuration based setup, so an NPC would be defined by loading a config containing the settings. You'd then override the config by inheriting from that and setting values:
<!-- The Base attribute tells the loader to first load the given config file, then merge the values with values loaded from this config -->
<EntityConfiguration Base="NPCs/barney">
    <!-- ConfigAction instructs the loader how to handle merging, in this case clearing and overwriting the old values. Default is merge, overwriting duplicates -->
    <Weapons ConfigAction="ClearAndOverwrite">
        <!-- This Barney uses a shotgun and it's his default weapon. Note that this requires Barney to have animations to fire shotguns and provide attachment points to put the model onto -->
        <Weapon ClassName="weapon_shotgun" IsDefault="true"/>
    </Weapons>
</EntityConfiguration>
It's called EntityConfiguration because it could be used for entity configurations in general.

I'd like to point out that Half-Life only supports up to 4 attachments. This is a limitation in the client, mostly the renderer, and there is no need to impose that limit here. The studiomdl format can store up to 512 attachments if using the default compiler, custom compilers can extend that limit to theoretically unlimited.

That makes adding attachments for weapons pretty simple, the only issue is that while attachments do have names, the default compiler doesn't let you specify the name. I'll have to modify it allow for that, otherwise attachments will need to use their index to identify them.

Unfortunately there is a backwards compatibility check in the studiomdl compiler that makes adding the name a bit complicated. I could make a separate program that sanitizes existing QC files to remove the obsolete tokens, then i can add the name as an optional last argument for $attachment. I'll have to look into that at some point in the future.

For anyone interested, the backwards compatibility check is to parse and ignore the tokens "X" and "1" or "-1". The Half-Life SDK on Steam has model sources containing these, i don't know what they were for but if i had to guess it was some kind of axis specific scale or something.

Getting back to the config files: at this point it's just an idea, until we get to the point where entities can be fully implemented this is just a dream. But if we get there, i'll definitely make it as easy and powerful as possible. I want you to be able to take SharpLife and modify only config files to produce a different looking mod, so no programming experience should be required.

For mappers, you'd just set the entity config filename to something like "maps/my_map_name/NPCs/MyBarney" to use the script for a specific Barney, it would also be possible to specify a per-entity class default using a config file for maps:
<MapConfiguration Base="maps/DefaultMapConfig">
    <DefaultEntityConfigurations ConfigAction="MergeAndOverwrite">
        <DefaultEntityConfiguration ClassName="monster_barney" ConfigFileName="maps/my_map_name/NPCs/MyBarney"/>
    </DefaultEntityConfigurations>
</MapConfiguration>
It's ambitious, but unlike the HLE version i had planned, this one doesn't require a complicated third party dependency setup, and it's much easier to load XML files due to being able to map them onto C# data structures. Text encoding is handled by C# so it's a lot simpler to work with than Xerces was.

Right now the focus is on the engine. I'm finishing up the networking code now, once i'm sure it's working properly i'll integrate it into SharpLife and see if it works well in the existing networking system (i haven't tested it with actual network packets). Then i can integrate my entity list prototype and see if a basic entity can be networked over. Then i'll make a worldspawn entity along with the logic to parse out entities easily.

Once i'm sure that's all working, i can focus on model loading. I've already got a partial prototype for studiomdl loading so it's making progress. I'll need to load the BSP file (already done), studio models and sprites. The server has to do 2 things: add the filenames to the precache list (easy) and load the models (hard). BSP loading is done and most of the rendering code is there. Sprite loading is pretty straightforward to do, but studiomdl is complicated because the format has unused features.

I need to make sure everything that's needed is loaded and converted properly to make it efficient enough, and avoid loading data that isn't needed. Texture data isn't needed on the server side, but the server and client can share data so making sure it loads correctly is important.

Once model loading is done, and the interface to get models for entities is there, it should be possible to render an entire map, including brush and point entities. Models will be precached by the client and rendering settings will be networked so clients can correctly do things like render { textures or sprites with transparent parts.

Getting studio models to render will be the hardest part, since i don't yet know how to handle the uploading of the vertex positions. I don't think re-uploading them every frame is good, so a way to do the vertex position calculations in a shader would solve that problem. I'd probably have to pass the bone positions as a uniform for that, but i don't know if that's a good way to do it. If anybody has experience doing that, i'd love to hear about how you did it.

I've covered a lot now, so i'll focus on getting this all done rather than talking about it more. Hopefully i can show this all off by the end of the month, but that depends on how much work i've got for college.
Posted 1 month ago2018-08-13 16:40:51 UTC Post #340527
Wow. Thanks for letting us know about this in such great detail @Solo.
That indeed sounds like a better deal for you - for all of us. I'm happy that this new project makes things so much easier for you. It certainly sounds like a dream come true. Sounds even better than source engine does regarding it's freedoms. As always best of luck to you. :)
Posted 1 month ago2018-08-13 16:50:51 UTC Post #340528
@23-down,
Sounds even better than source engine does regarding it's freedoms.
But C# can work with Source Engine too because Source Engine can work with PDR Format like high-tech textures from Substrance Designer for Example 2048 x 2048 or advanced features like Unity 3D's movements or map range infinity ( like more than -/+ 65536 ) and add more features like rouge and specular or emission etc like Unity 3D or Unreal Engine. But I don't know if Source Engine would like port to C# than it looks better than old Source 1 and 2. Goldsource and Quake Engine port to C# like SoloKiller made SharpLife.

That is reason: Source Engine and Goldsource Engine with C# look better than c++/+
Posted 1 month ago2018-08-15 09:27:51 UTC Post #340545
SourceSkyBoxer said:But please please add more shadow mapping like mod AM by SysOp.and realistic reflection for water and more.

DON'T

  1. The concerned code uses "legacy" also known as "immediate" OpenGL which is totally different from the "shaders based" OpenGL used by Veldrid (and by extension SharpLife) and modern AAA games (modern Wolfenstein games).
  2. The concerned code has a HARD dependency on PARANOIA's OpenGL renderer which uses the same "legacy/immediate" OpenGL.
  3. Even if you do waste your time porting the code as-is, you'll get a massive performance loss.
  4. It pains me to say this because I'm a developer of Arrangement but please for the love of God, don't use any of it's code from any public source code(s), the quality is so poor that rewriting the entire thing by yourself will be better.
  5. IIRC, Solokiller said that SharpLife's goal is to provide a "better vanilla GoldSrc modding/playing experience". If you want any kind of graphical enhancements, then fork the project and DIY (Do It Yourself).
Shepard62700FR Shepard62700FRHalf-Cat is watching...
Posted 1 month ago2018-08-21 23:26:36 UTC Post #340625
What kind of new tools would there have to be to meet the new requirements?
Rimrook RimrookGoldsource Guru
Posted 1 month ago2018-08-22 12:48:09 UTC Post #340630
Right now the new limits can be used with existing tools, but there are hard-coded limits in the tools that came with the SDK. For instance, QLumpy sets the maximum lump size to 327680 bytes, or about 327 KB. Each textures is stored in a lump, so that's a limitation. Removing the limit is pretty straightforward, just replace the tool with another that can store any size texture.

The compile tools and BSP format impose limits, some of which can be raised or removed by modifying the tools, but the BSP format itself presents a hard limit on its own. A new BSP format is needed to deal with that, which also means new compile tools that can produce those BSPs, and a new .map format to handle any changes in how data is passed from the editor to compile tools.

For now the plan is just to let you use existing maps in SharpLife, and where possible the limits are removed or raised.
The maximum number of entities as well as the maximum size in bytes of the data is imposed by the compile tools as well. VHLT34 supports up to 16384 entities, and a maximum data size of 2097152 bytes, or about 2.1 MB.
This limit can be entirely removed by using dynamic allocation and not imposing any maximum limitation, since the hard limit is defined by engine devs (65534 for now), and is solely a networking and memory usage limitation.

I doubt anyone will hit the compiler limits any time soon, so you can make more entities if you want. Textures can be larger than before without hitting lump size limits, but eventually you'll get there. A way to reduce that chance is to not store mipmaps, since the engine can regenerate those, but that requires modifying the wad creation tool of your choice. I don't know if Wally has any limitations so i can't say anything about that, but i can reverse engineer it and take a look if you want to know.

When it comes to upgrading or replacing tools i'd wait until SharpLife is capable of doing everything the current engine can do before looking too much into it. It may be easier to deprecate wad files entirely rather than try to remove its limits, same for other file formats.

I think the biggest limit is going to end up being the fact that SharpLife is 32 bit, required because it's hosted by a 32 bit process. I can't get around that unless Valve helps us out here. Fortunately that doesn't apply to tools, so compile tools can be 64 bit if needed.

Does that answer your question, or did you need to know more?

EDIT: i've committed the entity list code, the world is now instantiated from the map's entity data. Other entities will be instantiated once they've been ported over.
The world is networked to clients, so they'll be able to get any networked members that it has (currently origin, angles, flags).

Entities can be networked as a base type when there is no need for a specific type (e.g. NPCs can probably use a common base type), allowing them to be networked without requiring you to create client versions for every single one.

Now i can implement model precaching and start working to implement model loading and rendering.

There is currently an issue where starting another map after one has been started will cause an exception due to duplicate network string lists, i'll fix that shortly.
Posted 1 month ago2018-08-23 12:35:33 UTC Post #340641
Wow, this all seems so ambitious. I certainly hope you have help, or at least the time to complete it. The end product sounds like the engine we all need.
Rimrook RimrookGoldsource Guru
Posted 1 month ago2018-08-23 12:39:21 UTC Post #340642
Yeah it is quite a bit of work, but it's way easier to do this in C# than it would have been in C++. IMO C# has a more complete standard library and open source community than C++, so it's easier to get things done without having to reinvent the wheel (just take a look at how Unicode and internationalization is handled in C/C++).

I've fixed the issue i was talking about, i've also refactored the networking code to move most of it to NetworkServer and NetworkClient. I've also reworked the server's setup code for clients to centralize the logic for each stage, but i might rework that later on to use objects that can handle each stage, that's more robust and reduces the amount of variables that are only used during setup.

I've reworked networking objects to use the builder pattern to ensure that all networking data structures are immutable after they've been initialized. This should help prevent accidents that could cause unexpected errors.

Now i can implement model precaching, and model loading after that.
Posted 1 month ago2018-08-23 19:00:43 UTC Post #340645
So awesome to hear that it’s going well.

Also, drag and drop mod support? Like turning mods on and off, and playing smaller self contained ones with all sorts of new stuff. Doesn’t svencoop do something like that? It’s kinda like that on L4D2.

Edit: what if we Minecraft this bitch? What if mods could easily stack, like if I download a new weapon, it adds the weapon to the list rather than replace it? Modders and modelers would have a field day with self contained files and scripting. Same with items and such.
Rimrook RimrookGoldsource Guru
Posted 1 month ago2018-08-23 20:07:05 UTC Post #340647
The plan is to have plugin support, that will allow you to have entities added without having to modify the mod itself.
That also includes weapons, but i'm not sure how safe it would be to allow those to be client predicted since that would require downloading the code to the client.

It should be perfectly possible to have plugins that can both replace and add entities, same with map specific scripts. I don't think we should go overboard with that though, otherwise people will spend more time downloading various scripts than actually playing.

Those entities can use the config feature i talked about before.
EDIT: i've completed the essentials to load models, the map is now loaded using a model manager. The old map manager has been removed and its features merged into the model manager, or moved elsewhere.

Right now the following features are available:
  • Models can be loaded at any time by calling IServerModels.LoadModel or IClientModels.LoadModel (these interfaces may be merged since they're currently identical)
  • There is no limit to the number of models you can load, aside from hard memory limits (RAM and GPU)
  • Model data is shared between client and server, so listen servers are guaranteed to use the same data, though graphics data is stored separately
  • If a model fails to load, the fallback model "error.mdl" (renamed can.mdl from Half-Life for now) is loaded. If the fallback fails to load the game will exit with an error, but i may change it to instead create an empty model. That will have to wait until studio model loading is complete. I'd prefer to avoid having hard exits if at all possible, so this would be nice to have
  • The client can load models without having them loaded on the server. Vanilla GoldSource will silently fail to do anything when calling gEngfuncs.CL_LoadModel and returns null in this case. If the server loads the model later on, it will simply have both a positive index (networked) and a negative index (client specific). There is almost no overhead involved with this, it's just an additional entry in a dictionary
  • The map is now loaded into graphics memory after all resources have been received, matching the original engine's behavior
  • Map information can be retrieved from the engine using IServerEngine.MapInfo or IClientEngine.MapInfo. This provides a read-only object containing the name of the current map, the name of the previous map and the map's world model
  • Game code can directly access the model data by calling IServerModels.GetModel or IClientModels.GetModel. This returns an IModel, which can be cast to the specific model type that contains model data. Type identification replaces vanilla GoldSource's modtype_t here
C#'s pattern matching makes dealing with different types of models a bit easier:
switch (model)
{
    case BSPModel bspModel:
        {
            break;
        }

    case StudioModel studioModel:
        {
            break;
        }
}
It's easy to handle specific types of models this way.

This will need to be stress tested to make sure it's all working properly, but that'll have to wait until studio & sprite loading is done. I'm going to start with sprites since those are the easiest to work with.

Once that's done i'll have to implement the uploading of late precached models to the GPU, since the current design is optimized for loading everything in one go after resources are received.
Posted 1 month ago2018-08-24 21:02:47 UTC Post #340650
Wow great job and please add static mesh like collision :)

If you use GUI with Model Viewer - I recommend for Gtk Sharp 3.22 by @Cra0zy for net core. - Good work. Please stay your work!
Posted 4 weeks ago2018-08-26 13:05:10 UTC Post #340680
I'll probably use Core 3.0's UI support or Avalonia, not GTK.

[TC]Čiαиєz asked if the HUD can be scaled. This is possible in vanilla GoldSource if you modify the HUD code, and i'll keep it in mind when designing the HUD code for SharpLife.

https://github.com/MiYanni/xBRZ.NET may be a good library to use for scaling the images while retaining a sharp appearance.
Posted 4 weeks ago2018-08-26 13:24:41 UTC Post #340681
Please support for cl_dlls for linux and mac Check my post in TWHL about Linux compilation.
Posted 4 weeks ago2018-08-26 14:08:46 UTC Post #340682
That has nothing to do with this project.
Posted 4 weeks ago2018-08-26 19:55:52 UTC Post #340686
Solo okay... You continue your work until SharpLife's release :)
Posted 3 weeks ago2018-08-29 13:32:56 UTC Post #340733
I've been reworking the renderer and model loader to prepare for the other formats. I've merged the server and client interfaces for models and it handles path separator characters properly so for example maps/bounce.bsp and maps\bounce.bsp are considered the same.

Graphics resources are managed separately from rendering state and rendering is now handled by the game, which means when the renderer wants to render models the game can handle the entire thing.

In addition entities can now set their model, which is networked to clients. The model is set as an IModel, rather than an index. This means you can just use the model object, and the networking system will handle conversion to and from the index for networking purposes. This simplifies things and eliminates the need to look up the model whenever it's needed.

I've removed the temporary code to render all BSP models as this is now handled by the game, rendering the world and a basic version of func_wall:
User posted image
The grates and the doors are func_wall.

No render modes yet, but it's easy to add. Due to how it's implemented it's possible to let the game issue render calls and then handle it on another thread if needed since all relevant data is passed into the renderer by the game. This should allow the renderer to run on its own thread without needing to lock for access to the game; the game can collect its render calls and then pass that along, minimizing thread sync time to a single operation. That's going to have to wait for later on though.

I fixed a bug in the BSP loader where submodels were off by one due to a misunderstanding in how the engine loads them. Even the Quake codebase says it's confusing, so it really was confusing. It's all good now though.

Pretty much everything's in place to add sprites and studio models into the mix now.
Posted 3 weeks ago2018-08-31 22:07:54 UTC Post #340754
Looking good
Posted 2 weeks ago2018-09-06 14:10:38 UTC Post #340799
I've mostly implemented sprite rendering:
User posted image
I've implemented sprite loading, image conversion to RGBA32 using the alpha channel to store transparency without modifying the original color data, rendering sprites under all render modes, including a bug that makes Color not apply a color, and apply the right angles based on sprite type.

I reported this bug: https://github.com/ValveSoftware/halflife/issues/1903

I fixed a bug with the "facing upright" sprite type: in GoldSource this uses the model origin from another entity so it will snap around depending on which entity last set it before the sprite is rendered. I could reproduce the original behavior, but since it depends on render order it would never really be the same.
Given how rarely this type is used, and the fact that the bug was fixed in Source, i figured it wasn't worth replicating the original behavior.

I reported this bug as well: https://github.com/ValveSoftware/halflife/issues/1904

Neither of these bugs will ever be fixed in vanilla because some maps may rely on it, i reported them so people can find them more easily.

All that's left as far as sprites are concerned is depth sorting to make sure transparent sections don't block other transparent sprites from rendering.
You can see this problem in the image: the green lambda icon on the right is blocked by the stop sign sprite.
Depth sorting is also required for brush and studio models that have transparent parts.

Once this is done i'll implement render modes for brushes, fix skybox rendering to make sure transparent objects don't cut holes in it and to make sure sprites always render on top of it, take a shot at lightmaps and then finally handle studio models.

Once that's done i can implement render FX, which should wrap up all things model rendering related.

I also had to implement some entity keyvalues to make this all work enough to test things out, and i found a bug in the networking system's converter for vector types where it didn't decode them correctly.

Hopefully this can be wrapped up soon so i can start implementing physics, that's needed to make things like glow sprites work (it tests if a sprite is blocked by something else).

Thanks to The303 for making tutorials and test maps to display the various sprite types and render modes.
Posted 2 weeks ago2018-09-06 15:24:29 UTC Post #340800
Wait is it buggy? But Half-Life has not Alpha. It seems like RGB8 - I remember Quake on XNA 2.0 I have readden about models and sprites
Maybe DrawSpriteModel?

I hope you have to resolve... I think It has not bug.
Posted 2 weeks ago2018-09-06 15:57:28 UTC Post #340801
Quake's SPR and Half-Life's SPR format's spec also support variable frame-rates, which is fun and could be useful in some niche cases.
Tool developers didn't feel like exposing it, so no mod known to me takes advantage of it. I doubt stock Half-Life even cares, as GLQuake/GLQuakeWorld don't and Half-Life's sprite format is essentially the same, but with the palette baked in.
eukos eukosOriginal Cowboy
Posted 2 weeks ago2018-09-06 17:08:37 UTC Post #340802
The vanilla engine doesn't handle frame groups (a frame that contains a variable number of actual frames), it just complains about missing frames ("R_DrawSpriteModel: couldn't get sprite frame for <model name>"). I haven't implemented it either since there is little point in adding that kind of overhead, and no existing sprites would have it anyway. Given that Quake 2 does away with it, it likely never saw widespread use in Quake either.
Posted 2 weeks ago2018-09-06 20:49:52 UTC Post #340803
Uh @SoloKiller did you not see sprite of Quake? I have explained you DrawSpriteModel from Github SharpQuake and did you find RenderSprite? I hope you have to get chance
Posted 2 weeks ago2018-09-07 07:25:42 UTC Post #340807
I can't legally use Quake code in any way, shape or form. We went over this a while back in this very thread.
Posted 2 weeks ago2018-09-09 13:25:20 UTC Post #340831
User posted image
Lightmaps are working, including styles. It's not terribly efficient yet, but i'm working to make it better.

I'm going to implement render modes as well to make sure BSP rendering is mostly done.

After that i'm going to take a look at figuring out what could be causing memory leaks, since the game is crashing after a few maps with a lot of memory usage. Since it's not a C# exception it must be native memory somewhere.

EDIT: looks like it is a C# exception, so something is using up memory somewhere.

EDIT2: found the cause: https://github.com/mellinoe/veldrid/issues/118

A memory leak in Veldrid's OpenGL backend causes the problem.

EDIT3: the leak has been fixed. I've upgraded to the latest Veldrid build and i've also upgraded to the latest ImageSharp beta (since Veldrid needs it). ImageSharp is now pulled from NuGet, Veldrid is still a custom build since there is no full release containing the needed fixes.
Posted 2 weeks ago2018-09-10 19:36:34 UTC Post #340842
I've optimized BSP rendering as much as i can for the time being. Bounce is hitting close to 1300 FPS now, singleplayer maps like c1a0c (Unforeseen Consequences), c2a5 (Hydroelectric Dam), c3a2c (Lambda Core portal jumping puzzle) run at around 600 FPS without any visibility testing.

I also tried this map: https://twhl.info/vault/view/6174
It runs at about 250 FPS.

I did this by reworking a couple things:
  • Shared BSP resources are allocated once, reducing the number of state changes
  • The light styles are passed as a single uniform buffer (only 64 actual values are needed, reducing memory requirements). Each vertex also stores the styles used for it, eliminating the need to group surfaces by which styles they use
  • A single vertex & index buffer is used for an entire model
  • Lightmaps are pooled together in larger images like vanilla GoldSource does. These pools are larger than vanilla to reduce the number of state changes required to switch lightmaps. Unlike vanilla, there is no limit applied to how many pooled lightmaps there are, and the pools are not shared between models. I can change this to reduce GPU memory requirements, but it shouldn't affect performance unless a lot of nearly full pools are reused, which would cause a lot of state changes to happen when rendering BSP submodels (brush entities)
  • All of the surfaces that use a texture are grouped together, ordered by number of surfaces descending and then added in sequence, allowing surfaces with the same texture to have their lightmaps in the same texture to reduce state changes. Ordering by number of surfaces ensures that maximum efficiency is attained in grouping lightmaps together, reducing the number of state changes required a little more
Light style changes barely affect performance now, i applied random modulation as a test and the FPS drop is manageable. Bounce suffers a drop of around 100 FPS, but c3a2c only drops by about 60 FPS. Considering every single light style (64 styles) is changing at once every frame, that's pretty good. Most of the time only one or two change, and they usually don't change as frequently as this. In practical situations it shouldn't affect FPS that much, certainly not as much as people get with the vanilla engine.

Dynamic lighting will probably lower things a bit, but those are usually temporary effects so the FPS cost shouldn't be that bad.

EDIT: I implemented support for Vulkan, here are the results:
  • Bounce: 1950 FPS
  • c2a5: 1000 FPS
  • c3a2c: 900 FPS
  • dm_crossedwire: 360 FPS
Pretty good, i don't think there'll be any performance issues.
Posted 2 weeks ago2018-09-11 13:04:23 UTC Post #340846
amazing... that sounds almost unbelievable... Good job mate.
Posted 2 weeks ago2018-09-11 15:20:13 UTC Post #340850
Just to clarify, these are rendering the maps with no visibility testing, ie. No form of blackface culling, visleaf rendering etc?

If so, that's bloody impressive. Wonder how much of an improvement visibility testing will have if added in
Instant Mix Instant MixTitle commitment issues
Posted 2 weeks ago2018-09-11 15:32:51 UTC Post #340851
It is doing backface culling, no visleaf checking yet. We talked about maybe not using visleaves because it might actually lower performance. I'll have to try it out and see what the results are.
Posted 1 week ago2018-09-12 10:52:22 UTC Post #340854
User posted image
I've implemented render modes for brushes. From left to right:
  • Normal
  • Color
  • Texture
  • Alpha test (normal on top for comparison)
  • Normal and alpha test side by side
  • Additive
The 4 other brushes are a test to see if rotation works properly.

Here's alpha test up close:
User posted image
To make alpha test work i had to implement texture re-sampling and rescaling during upload. The engine actually does a fair amount of work here, first changing all palette colors by running it through a gamma correction table, then changing the color of invisible pixels so sampling doesn't try to blend black. Otherwise the texture ends up having black borders around the transparent parts.

Finally, the image is rescaled to a power of 2. It isn't necessarily upscaled, the cvar gl_round_down affects this and is used to make texture biased towards scaling down instead of up. For instance, the grate texture seen above is 48x48, but is rescaled down to 32x32.

This affects the look of the texture since the number of pixels for each bar in the grate is reduced. Without scaling, each bar is 3 pixels wide. Scaled up, it's 4, and scaled down it's 2. Combined with the transparency color changes this can drastically change the look of a texture.

Here's the original texture, without any of these changes:
User posted image
The scaling of the image is done for all textures to ensure power of 2 sizes, the alpha color balancing only applies to alpha tested and index alpha textures.

Textures used by models and sprites seem to use yet another scaling algorithm that uses a different code path. Setting fs_perf_warnings to 1 and developer to 1 will print out warnings only for those textures. This path is apparently used for textures that don't use mipmapping. I'll have to replicate that later to make textures for those load properly as well.
I've implemented gamma correction and brightness override to (almost) match the original. Since this is done in the shader the results are slightly different from the original, but you won't notice it unless you're comparing the results.

The settings use cvars, and the values are uploaded to the GPU. This means it's now possible to adjust the values in real-time:
User posted image
I've also added an FPS slider, although ImGui is bugged and doesn't display the value correctly.

As you can see the settings are all dialed up to the maximum, showing how it changes gamma.

Like the original render mode Color brushes are unaffected by gamma correction, and i fixed it using texture colors and lightmap data.
Posted 1 week ago2018-09-13 11:35:52 UTC Post #340867
User posted image
I've implemented gamma correction in sprites now as well.

I've also reworked shader compilation to use glslc instead of glslangvalidator.
This tool is built on top of the other tool, but adds support for #include. I've used this to share gamma correction functionality.
Posted 1 week ago2018-09-14 02:30:52 UTC Post #340873
Incredible work!
Posted 1 week ago2018-09-16 16:20:02 UTC Post #340896
Thanks!

I implemented gl_overbright since i was asked about it.

Without overbright:
User posted image
With overbright:
User posted image
Posted 1 week ago2018-09-16 16:31:27 UTC Post #340897
Could you implement HD skyboxes?
Posted 1 week ago2018-09-16 16:47:05 UTC Post #340899
You mean skyboxes that have a higher resolution? That's already available. It doesn't support TGA skyboxes yet, that's going to have to wait until ImageSharp supports it.
Posted 1 week ago2018-09-16 17:47:54 UTC Post #340900
Yes i mean higher resolutions. Especially very very high ones for example 2048x2048

And if you want to add bug fixes it would great to add these things below. :)

When you shoot sky it will be gunshot particles coming out from the sky brush, can you fix it.

A problem i noticed in hl sdk. It isnt posible to change colors of blood decals. I guess decal colors are hardcoded.
I can change color of blood sprites. but blood decal colors stays the same.
User posted image
User posted image
It would also great if you manage to seperate blood sprite color from palette.
I mean i dont want to use #define BLOOD_COLOR_RED (BYTE)247
Is it possible to use like this below,
blood_color human;
human.r = 256;
human.g = 0;
human.b = 0;
And this would affect both blood color and the blood decal color at the same time.

It would great if you add hammer modding support.
For example add a weapon_generic and monster_generic who can change their models and attributes in hammer.
It would be way easier for modders since they dont code new guns.
They would add a weapon_generic and customize their damage,clip etc.

EDIT

And It would great too if you could implement 3d skybox support like source does.
So people like me who cant map organic stuff like clifs, can map a boxy play area and surrounding .mdl clifs :)
Goldsource is easier to mod and suitable for smaller development teams. And more suitable to amateur modders.

And I think this skybox would not interfere with retro style of vanilla goldsource. But i dont know is it posible to implement in gameside programming.
Posted 1 week ago2018-09-16 18:14:31 UTC Post #340901
The first one is in game code, all it needs is a check on material type. A separate material type "none" would be needed to disable effects, that's easy.

Blood color is that way because it's legacy Quake code, i plan to make these use RGB color values instead so that isn't a problem.

Mod support will have to wait until everything else is done, but i'm sure it's possible to add at some point.

I was already going to add 3D skybox support at some point. I don't see why this shouldn't be done since it's fairly simple and can add a lot to a map without costing much.
I've also just implemented fullbright support, since that goes hand in hand with overbright.

I've also added a cvar mat_powerof2textures to disable textures rescaling to power of 2.
Posted 5 days ago2018-09-20 14:52:19 UTC Post #340924
I've refactored the code that handles model loading and rendering, each model type is now completely in its own library.
The file format, the code used to convert it into an IModel instance and the code used to load graphics resources and render it is all in it.

This isn't finalized yet, i want to make the graphics part only handle the loading of resources while leaving the rendering up to the user to allow the same code to be used to render models in-game and in separate viewer programs. IModel handling should eventually be moved out to SharpLife.Game.Shared since it's primarily used by the game.

The renderer has been moved into the client, allowing it to control how everything is drawn. This means all graphics related code is now handled by the client game library.

The engine no longer handles specific model types as map loading is fully handled by game code now, so in theory you could completely replace it with new formats without the engine having to know.

These changes make it easier to handle the remainder of the model rendering code since each type should ideally get only the data it needs to render itself. Since the game handles everything it's easier to handle specific types.

I've also refactored the model manager to remove the explicit handling of bsp submodel loading. The bsp loader now does this, though there is still a need to explicitly add the models to the string list later on so they're in the list. I may change this to allow models to specify what their submodels are so it can be handled automatically there as well.

Another important addition is the bridge between the game client and server, which allows them to share data and communicate directly.
Currently it shares a helper object that's used to format bsp filenames, but i have plans to use this to add debugging features. I'll probably be able to show what i have in mind in the next few days. It should be pretty interesting to see, but i'd like to see if it's possible before saying more.

There is more refactoring left to do to move code from the engine to the game. I'd like the map command to be handled as a shared game command, removing the need to reference saved games in the engine as well as removing the need to pass additional state around when changing levels in singleplayer.

The SharpLife.Renderer library contains types that i'd like to move as well, which should turn this library into a utility library containing interfaces and helpers for rendering and managing resources.

I'm also going to rework the object list networking system so that its delta networking system doesn't have any loss in precision, which was an issue when networking the studio model Frame variable.

When all this refactoring is done the model rendering code should be easy to use in a model viewer. A pretty important bug was reported a few days ago about skin families being missing when the model has a separate texture file, so i'd like to get the new model viewer done soon. To that end i've looked into WPF a bit to see if it has what's needed to make a replacement viewer, and it looks like it's good enough, though i did have to get a third party library (https://github.com/xceedsoftware/wpftoolkit) to cover common controls.

Dot Net Core 3.0 will include support for WPF, but that won't be released until 2019, and when it does the library i used above may not be available right away. i'll try to convert the SharpLife libraries from Net Core to Net Standard so they can work in the Net Framework WPF environment, so until WPF on Core is ready to handle everything it can still work like that. Unfortunately this may result in an excessive number of libraries being loaded so that's something to keep an eye on.

We've also been talking about adding an I/O system to SharpLife. This feature would be purely in game code and would allow the same kind of functionality that Source's I/O system has. I've looked into how Source's system works and it looks like the same could be done in SharpLife without requiring changes to the .map and bsp format.

Penguinboy and i have talked about it a bit and if editor level support can be added it should all work fine. Since Sledge is projected to support Source as well the same UI can be used, and the only difference would be in how the data is stored on disk in the .map format.

For backwards compatibility purposes the old trigger system will still be supported, but i have been working on a tool to convert maps that could also convert the old triggers to the new system. This tool is meant to convert older, more limited keyvalues (like func_door's index based sound keyvalues) to a new format, and outputs are stored the same way so this should work out.

For now the focus will be on reworking the model rendering code to use separate data types to pass around rendering data, this will allow me to pass studio model specific data around without the other formats also getting it. After that the debugging tool i mentioned above should be implementable.
Posted 5 days ago2018-09-21 00:44:07 UTC Post #340927
WPF will run on .NET Core 3 but it'll still only work on Windows. A model viewer doesn't have an overly complicated UI so maybe one of the cross platform UI frameworks is viable?
Penguinboy PenguinboyHaha, I died again!
Posted 5 days ago2018-09-21 06:39:07 UTC Post #340928
I tried out Avalonia, it's similar to WPF but it's not quite there yet. The toolbox doesn't show Avalonia controls but other than that it seems to work.
It currently targets NET standard, the problem is that some of NET Core 2.1's features aren't available in standard, and i'm already using them. So regardless of whether i'm using WPF or Core, it won't be able to work out of the box unless i either remove the features or use the OOP packages that contain the newer features and hope it all works out.

EDIT: i'm not 100% sure if Avalonia can use Core libraries or not, will need to check.

Microsoft's post on Core 3 said they were going to release a preview in 2018, so it should be available soon. I'll put something together once that happens so i can try things out, and then upgrade later on.

There's an article released just today that says NET framework is done: https://medium.com/@andy.watt83/the-net-framework-is-done-8aec3bbae12d
So i would assume that MS will eventually focus on cross platform GUI support. I'm guessing Core 4 will bring support, but there's no telling how long that'll take.

For now i'll stay focused on the engine itself, but i could always use ImGui for a basic GUI if a model viewer is needed sooner.
Posted 3 days ago2018-09-22 16:48:15 UTC Post #340936
So here's that debugging feature i was talking about:
User posted image
This lets you edit entities directly when you're hosting a listen server. It isn't perfect, but it's good enough to let me test different sequences, frame rates, etc without needing to set up the user input system or use console commands.

It does have some limitations, and i discovered a bug in the networking system where objects that haven't updated in a few frames try to send a delta when they shouldn't, but it's going to make implementing studio models a lot easier. It'll also come in handy down the line when tweaking entities.

I've also finished reworking the renderer so now each model type has its own input data type. This will let me separate out the keyvalues to entities that actually need them, and it avoids having to fill in values that aren't used for some types of models.
Posted 2 days ago2018-09-23 11:23:33 UTC Post #340941
User posted image
Skins and bodies work, i've also implemented the game's frame animation logic so it's smoothly animating now (it wasn't before).

Still need to implement lighting, chrome and modify the bone controller logic a bit to match the original. Render modes and fx is next, that should just about cover studio models.
Posted 6 hours ago2018-09-26 01:37:59 UTC Post #340949
Are you doing origin lighting like GldSrc or going with fancy vertex lighting?

Amazing work btw, every post impresses me.
Tetsu0 Tetsu0Original Cowboy
You must be logged in to post a response.