SharpLife - Dot Net Core based modding p Created 6 months ago2018-05-06 19:15:02 UTC by Solokiller Solokiller

Created 6 months ago2018-05-06 19:15:02 UTC by Solokiller Solokiller

Posted 1 month ago2018-09-28 08:58:58 UTC Post #340960
User posted image
User posted image
User posted image
User posted image
User posted image
From top to bottom:
  1. Testing studio lighting. Only includes lightmap light data & world dynamic lights (dlight). Point entity lights (elights) not yet implemented but should be relatively simple. Different color rooms used to test the effect of different colors, showing that sampling is working correctly.
  2. Testing shadows.
  3. Additive render mode for studio textures. The mask goggles are black but are transparent to allow the green color behind it to be visible. The goggles are chrome, but that isn't implemented yet.
  4. Masked render mode for studio textures. Alpha tested transparency.
  5. All 6 render modes, from left to right: Normal, Color, Texture, Glow, Solid, Additive. Color, Texture and Glow all use the same settings and behave as Texture. The upside down grunt is a test to see if negative model scales work. This is important because Counter-Strike's cl_righthand flips models along the Y axis, and requires culling to be changed.
It's not quite finished yet but it's looking good. This runs at ~400 FPS without optimizations (vis leaf culling and bounding box culling).

To make lighting work i've also implemented light_environment. This is needed because it sets the sky color and normal vector.

What's left to implement is the correct lighting values for all render modes (minor adjustments needed), elights, chrome, glow shell and perhaps flat shading as implemented by glShadeModel(GL_FLAT). This last one is very specific since only normal render mode with additive texture render mode sets it, but only after it has drawn the mesh that uses it. I doubt it's actually required since it seems more like incorrect behavior, but i'll look into it all the same.

Models used:
  • Vanilla hgrunt.mdl
  • Sven Co-op 5 hgrunt.mdl (additive render mode, not included in SharpLife repository)
  • Condition Zero props/antenna_urban.mdl (masked render mode, not included in SharpLife repository)
I've also found a presentation detailing the design of a high performance renderer from Valve: http://on-demand.gputechconf.com/gtc/2016/events/vulkanday/High_Performance_Vulkan.pdf

This could be very useful for designing a threaded renderer.
Posted 1 month ago2018-10-04 09:14:54 UTC Post #340982
Good progress... Seems you're 2/3rds done with it.
Posted 1 month ago2018-10-04 13:14:45 UTC Post #340983
User posted image
User posted image
I've been doing some backend work the past few days, mostly figuring out how the physics code works so i can implement it.

To test settings more easily i've added editable fields for vectors.

Above: Origin, Angles and RenderColor are vectors that use different representations. Origin is a plain edit field, angles is 3 angle edit fields in degrees, and RenderColor is a color picker. The antenna model is rotated to show that it does work.

Below: light_environment's settings can be edited, and are then transmitted to the client. The sky color picker is open, set to bright red. The sky normal is set to a direction that causes most studio models in the scene to use the floor color instead. The model standing on a sky brush and the scaled model use the sky color because the normal is pointing from a sky surface for both of them.

You can use this to move objects around to wherever you want, you can rotate them, change render modes, colors, etc. This means instead of having to adjust the setting in Hammer, recompiling (onlyents or otherwise), restarting the map and moving back to where the entity is you can adjust the setting in real-time and then adjust it back in Hammer.

Two uses that i personally think are really good for this are getting a good rotation angle for func_door_rotating, and movement distance for func_door.

The origin fields should be lined up horizontally, but there's an issue in ImGui.NET that prevents this from being done easily so for now it's vertical.

The sky color and normal don't affect much, as far as i can tell only studio models are affected by controlling the sky color when they're standing in the path of sky lighting. In theory you could disable lightmaps and use this to do real-time lighting, but without light data (light, light_spot, texture lights) anything indoors would be black.

I'm not sure if the compile tools strip unnamed lights, the lights in this map are still in the bsp along with the original color and brightness values, but old maps might not. light_environment handles the case where light data only contains a single color value and no brightness, so really old maps (retail maps probably) might not have all the data needed for this.

Even if that's all there, texture lights would require the original lights.rad to even try to handle them, and i don't know how expensive all this would be to calculate in real-time.

I've been asked if there is a way to modify the values in the level editor; this requires the use of Inter-Process Communication (IPC) and unique identifiers for each entity, so that the entity can be properly identified between the game and the editor. Sledge may get this as a plugin someday, but it's not going to happen any time soon.

Now that this is done i'm going to implement physics for normal entities. Once that's all there entities can move, collide, etc, and trace functions can be implemented on top to finish up some renderer visibility testing code. I'm leaving the rest of the studio model features for later because they're not critical to have right now and i'd prefer to be able to fully interact with the entities to do more testing.

After that, if i can get user input to the server working and player movement physics implemented it should be possible to move around, use and shoot stuff, etc.
Posted 1 month ago2018-10-12 16:39:38 UTC Post #341026
I was working on the prototype for SharpLife's audio subsystem and i discovered that the engine handles cue points in wav files in a really strange way.

People always assumed that having 2 cue points would cause the sound to be looped between those two cue points after the first loop (which plays the whole file).

In reality only the first cue point is considered and what everybody thought was the second cue point is actually a bug in the engine dating back to Quake 1 that causes the range of a looped sound to be limited when the sound file has more than 8 bits per sample.

It doesn't seem to be possible to use a second cue point. It is however possible to limit loops if the "purpose ID" of the first cue starts with "mark". I don't know if it's actually possible to set the purpose ID using audio editing tools (Audacity can't edit cue points at all from the looks of it).

It's very limited, the Quake source code mentions a program called "cooledit" so that might let you do it.

I can fix this behavior for SharpLife, it's pretty easy to do. But some sounds in Half-life use cue points, like for example sound/doors/doormove6.wav has a cue point named MARK964. I can see in my sound system that it has a purpose ID "mark", and 964 is the "Cue Point ID" used to map the cue elsewhere.

I'm not sure if the name is actually what's used here or if GoldWave is just combining the two IDs, but when i modify it the cue is saved as a label instead of as label text and a label. The label text contains sample length information that controls how long the loop section should last for, but the file i tested has it set to the length of the file.

The cue point ID is set to 0 by GoldWave, and the cue position becomes 964 which seems to indicate that it's supposed to be the offset in the audio file where the cue point is located. I don't know why the file originally had it different, since the position is 0 there. The engine doesn't read the cue point ID at all from what i can tell, so it basically isn't using the cue point for looping at all.

I assume Valve made their sounds using this cooledit program, but it doesn't explain what their intentions were with these looping sounds. Since this design dates back to Quake 1 they may not even have known about the different behavior at all.

I checked another file, sound/plats/rackmove1.wav and it also has a cue point in it, named MARK514. It also doesn't store the actual cue position where it should and so doesn't loop either.

Neither sounds really need a cue point anywhere other than the start (they are needed to loop the sound), so they probably didn't even notice anything was wrong. For anyone modifying the sounds the cue point positions will matter since GoldWave and probably other programs will convert the cue points to correct this mistake.

Does anybody know more about this? I've seen the discussions on the VDC ( https://developer.valvesoftware.com/wiki/Talk:Looping_a_Sound) but those are inconclusive and only apply to Source. I'm sure it's very interesting to know how looping is actually done but if no tools exist to set the sample length property for a cue point then it isn't really useful.

I also found that GoldWave writes an additional LIST chunk that follows a really old (less than 2 months younger than me) spec: https://www.aelius.com/njh/wavemetatools/doc/riffmci.pdf

The List info chunk can contain information such as copyright. it tripped up my loader and probably breaks NAudio as well.
Posted 1 month ago2018-10-12 18:06:42 UTC Post #341027
I think you've hit the nail on the head. Nobody really knows about looping sounds in GS; this makes sense as I've tried multiple cue points in those files and doesn't have the same effect it does in source.
I've always just used goldwave with two named loops of start and end, but now I'm enlightened with the knowledge this does almost nothing at all.

Embedded cue points are really odd, the half life games are the only thing I'm aware of that use them and support
for them is so rare hence that only cooledit / goldwave are the only programs that support it. A sound VMT style thing would be perfect as a replacement
Instant Mix Instant MixTitle commitment issues
Posted 1 month ago2018-10-14 01:49:10 UTC Post #341035
Looks like "cooledit" is now Adobe Audition: https://en.wikipedia.org/wiki/Adobe_Audition

The one time I tried playing with looping/non-looping sounds it didn't seem to work so I moved on to playing with something else.
Posted 4 weeks ago2018-10-22 17:43:35 UTC Post #341061
A sound VMT style thing would be perfect as a replacement
I could do something that's similar to Source's soundscripts: https://developer.valvesoftware.com/wiki/Soundscripts

Given that they implemented stuff like operators it may be easier to let these be defined using scripts instead, so you can express properties and events programmatically.
I've been working to implement physics support, so far i've got the majority of regular entity physics implemented, with func_door mostly implemented for testing.

Since there's no entity triggering or user interaction yet i added a feature to the object editor to invoke a named parameterless method so i can trigger the door movement methods directly. It works, but it's choppy so i'll need to figure out why that is. It could be related to the framerate or some value being used as int instead of float, but it could also be a networking issue.
I've also made a small prototype IO system. You can define inputs and outputs, trigger them and pass values just like Source's version. Here's what the code for it looks like:
[Output("OnPass")]
public OutputEvent<float> OnPass { get; } = new OutputEvent<float>();

[Output("OnFail")]
public OutputEvent<float> OnFail { get; } = new OutputEvent<float>();

[Output("OnPrint")]
public OutputEvent OnPrint { get; } = new OutputEvent();

[Input("TestValue")]
public void InputTestValue(in InputData inputData, float value)
{
    Console.WriteLine($"Firing {value}");

    if (value != 0)
    {
        OnPass.Fire(EventQueue, inputData.Activator, this, value);
    }
    else
    {
        OnFail.Fire(EventQueue, inputData.Activator, this, value);
    }
}

[Input("Print")]
public void InputPrint(in InputData inputData)
{
    Console.WriteLine("Print");

    OnPrint.Fire(EventQueue, inputData.Activator, this);
}
Inputs are public methods marked with an Input attribute, which takes the name of the input as used in the map editor or console commands. They can optionally take a single additional parameter that is the value that outputs can pass to it (or the parameter override if specified). Inputs can also access the value as an object instance through the InputData instance passed to the method, which allows for variant type inputs.

If the wrong type is passed to an input and it can't be converted, the input won't be called, just like in Source.

Outputs are public properties or fields marked with an Output attribute, which takes the name of the output as used in the map editor or console commands.

Outputs can optionally return values, in which case you have to specify the type in the member declaration. This also enforces the requirement that you pass the correct type when you fire the output.

Firing outputs is straightforward: you pass the event queue used to track events for entities (there's usually only one), the activator, the caller, and a value to pass to inputs if the output returns a value.

The event queue is similar to Source's version: a linked list sorted by event execution time, designed so that new events are added to the end of the list of events fired a a certain time so that events with delay 0 added while an event is being fired will fire in the same frame.

As far as supported return types goes, if you register a converter it'll work. Converters are subclasses of the C# class TypeConverter with ConvertFrom and CanConvertFrom overridden. This way, the USE_TYPE enum can be directly used without having to add it to a variant type like Source requires (Source uses a hack to pass the USE_TYPE value along instead).

This also means that you can easily add support to convert between types without having to edit a massive Convert method.

I'll be integrating this IO system once i've got physics completely done so i can implement +use support. +use relies on the physics code to pass along which buttons are pressed, finding entities in a sphere and then triggering the entity's Use method.

In SharpLife this will be replaced with the firing of the entity's Use input. It will be largely the same since input parameters can be part of the method signature, unlike Source. It should look like this:
public void Use(BaseEntity activator, BaseEntity caller, UseType useType)
{
    //Value not passed to InputData to avoid cost of boxing
    Use(new InputData(activator, caller, null), useType);
}

//Usage:
entity.Use(this, this, UseType.Toggle);
GoldSource actually uses USE_SET when +using entities, so i'll need to figure that out.
Posted 3 weeks ago2018-10-25 12:01:48 UTC Post #341076
Cross posting this bit from the HLE thread:

I estimate it will be at least 6 months before SharpLife can deliver the same functionality as the vanilla SDK, though given that i'm incorporating HLE's features from the start it will provide much of the same at that point in time. This means things like entvars_t members being accessed through properties (SL doesn't have entvars_t), customizable entity relationships, config files, etc will be available as they are for HLE.

How long it takes depends on how difficult it is to implement original functionality.
I'm getting close with physics but it will eventually require both lag compensation and prediction to be implemented, which relies on accessing old data. Easily accessing this will likely require restoring the old state, so i have to make sure non-networked members can be restored as well, which they currently can't.

Once physics is implemented i'll have client input working, which means the user can press their use key, have that converted from +use to a usercmd_t, sent to the server and processed there so it can find and trigger entities. I should also be able to rework the camera movement system to instead use player position in the world at that point.
Everything needed to test physics will be available by that point, so i can verify that it all works.
I also need to make sure the networking system can handle infrequently updated objects properly. Currently it will still send updates for objects that haven't changed so the client can track them, this needs to be improved.
I also started working on a prototype for a GUI, starting by getting the drawing code operational. I studied Doom 3 and ImGui's code for this, they both use a similar method where all drawn primitives are stored as a list of vertices and drawn with a texture. Simple colored primitives use a white texture, text is ultimately also a texture (one character is typically referred to as a "glyph" in this context), so if i can get that working it should be good to go.

VGUI1 can only draw text, filled and outlined rects and textured rects, which my current implementation should be able to handle. Things like rounded corners can be done by using partially transparent textures (alpha channels masking the rounded part), so it shouldn't be that difficult.

The actual widgets like buttons require a good widget management design and class hierarchy, but that's not hard to figure out.
Once this is all working then the engine should be mostly there. Some features are unfinished (brush models don't render water properly, don't do random or animated textures yet, studio models are missing chrome and glow shell, sprites could use some optimizing), but the essentials needed to play a map should be working at that point.
As far as audio goes, my prototype should cover the basics so i can integrate that at some point to do menu music playback and basic per-entity sounds, after that individual features can be implemented.

The stuff that's needed should be there relatively soon if i keep at it, once the engine level stuff is there i can shift my focus to game features. Most of that involves re-implementing SDK code (mostly entities), but i can keep feature requests in mind while i'm doing this.
I do have a request for people wanting to use this for their own mod: if possible could you give me some ideas/feedback on a good way to configure GUI settings? VGUI1 uses resolution specific scheme files, VGUI2 uses resolution independent resource files, so i'd like to make sure i get this part right from the start since the widget design involves getting these values during creation.

Features like interactive building at runtime like VGUI2 does would be nice, so a way to store/update these settings is nice to have. Knowing how the configuration needs to be done plays a big part in designing such an editor.

I know that some people would like to have 3D rendering support in the GUI, i have never done this before but i figure it should be relatively simple by using render-to-texture, which would let it function with the current design for GUI rendering. This is probably a well-known approach but i haven't done any research on it yet.

One good use of this would be in the multiplayer model selection window, which would make the older image based approach obsolete and allow you to view the actual model in real-time.

If anybody has ideas or suggestions i'd love to hear them.
Posted 3 weeks ago2018-10-25 12:26:12 UTC Post #341077
I don't know if this is the best way to do this but how about "per aspect ratio" config. with scaling?
Shepard62700FR Shepard62700FRHalf-Cat is watching...
Posted 3 weeks ago2018-10-25 12:38:23 UTC Post #341078
That's possible, i could make config values take effect for specific ratios. Kinda of like this:
<Property name="Position.X" value="10" condition="AspectRatio == 16/9"/>
The condition could be evaluated using CSharpScript, though in this case a constant value for 16/9 may be preferable. This would make it more flexible, for instance allowing you to support multiple ratios or different conditions (e.g. Game == "Half-Life", but that's pretty hacky).

EDIT: configs could maybe even be whole scripts. Much more efficient, and you can express things more easily by using the actual code. It would unify the GUI code in the game codebase and config files, meaning you can use the same API.
Posted 3 weeks ago2018-10-25 20:25:07 UTC Post #341079
A lot of apps use a web engine for their GUI (I do NOT suggest this for your project) and I think a big part of the attraction is the access to all the CSS units and properties which make it easy to produce UIs that scale for any resolution and aspect ratio. Units like vw, vh, (r)em, and percentages. Maybe they can serve as inspiration somehow?
potatis_invalid potatis_invalidh̴̅̾̓̊ͤͪ̂͑̀̏̒ͤ͒̂́̀͘i̢̧̎̒͆̃͐ͤ̈́̓͑ͪ͂͒͆ͧ͑͘
Posted 3 weeks ago2018-10-25 20:37:28 UTC Post #341080
Yeah we discussed this, i can add support for different units. It may be best to implement things like CSS styling, margin and padding for flexibility.
Posted 2 weeks ago2018-11-01 12:25:40 UTC Post #341113
Avalonia announced that they can recommend its use in production: http://avaloniaui.net/blog/2018-10-30-avalonia-0.7-release

I'll be taking a look at it sometime soon to see if i can make a model viewer replacement with it. According to some of the comments they don't yet have OpenGL support, but DirectX is supported. Figuring out if and how they support integration with Veldrid will be an important part of determining whether it's suitable for use at this time.
I've made a basic prototype for plugin management. It covers the essentials needed for assembly based plugin loading to work.

The manager loads plugin configuration from a file, much like Sven Co-op's plugin system does and similar to HLE's proposed configuration syntax: https://github.com/SamVanheer/HLEnhanced/wiki/Plugin-manifest-(temp-page)
<?xml version="1.0"?>
<PluginListFile xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
	<Plugins>
		<PluginConfiguration>
			<AssemblyName>TestPlugin.dll</AssemblyName>
			<PluginClassName>TestPlugin.TestPlugin</PluginClassName>
			<Arguments>
				<TestPluginArguments>
					<Name>Test Plugin</Name>
				</TestPluginArguments>

				<cvar name="developer" value="2"/>
				<cvar name="sv_cheats" value="1"/>
			</Arguments>
		</PluginConfiguration>
	</Plugins>
</PluginListFile>
A plugin is loaded based on assembly name, the plugin is instantiated based on the given class name.

If the class implements IPlugin, it will create an instance by using dependency injection to provide any objects needed by the plugin.
This could be used to get interfaces like the entity manager, networking, etc. Each plugin can also request the arguments given to it in the config file by specifying a constructor argument of type PluginConfigurationArguments.

Here's a basic example of a plugin:
using PluginHost;
using System.Xml;
using System.Xml.Serialization;

namespace TestPlugin
{
    public sealed class TestPlugin : IPlugin
    {
        public string Name => "TestPlugin";

        public TestPlugin(PluginConfigurationArguments arguments)
        {
            var serializer = new XmlSerializer(typeof(TestPluginArguments));

            var myArguments = serializer.Deserialize(new XmlNodeReader(arguments.Elements[0]));
        }

        public void Initialize()
        {
        }

        public void Shutdown()
        {
        }
    }
}
This plugin takes only the config arguments, and then converts it into an object of type TestPluginArguments, which looks like this:
namespace TestPlugin
{
    public class TestPluginArguments
    {
        public string Name { get; set; }
    }
}
This allows you to specify complex arguments and read it with ease. Note that the example does not do error handling, but that's not hard.

This can easily be used to pass configuration around, like requiring cvars to be set or even created dynamically.

Interface versioning is an important part of supporting plugins properly, breaking changes would require the assembly that provides the interface to change its assembly version, which will cause assembly load to fail with an error noting the version conflict. That makes it relatively easy to avoid cases where plugins rely on interfaces that have changed or have been removed.

For changes that don't break the API, like new methods it won't be necessary to change the version. I'm not 100% sure how C#'s interfaces work in this type of situation, but following the general rule of adding to the end of interfaces might be good enough, or alternatively adding new interfaces that extend from them.

Accessing plugins in game code should only rarely be needed, and can be done in a couple different ways.
Most of the time you won't need to access a single interface, just all of them, so iteration is the common case:
pluginManager.ForEach(plugin => plugin.DoSomething());

foreach (var plugin in pluginManager.GetPlugins())
{
    plugin.DoSomething();
}
The first option is the simplest and won't break easily. It does however create delegates which can add overhead in critical per-frame game logic like RunFrame. The latter option avoids that cost.

Most plugin access in SC revolved around hooks. The way it accesses hooks accounts for the fact that plugins can be reloaded. in C# assemblies can't be unloaded individually, only the app domain in which they've been loaded can be unloaded. Using app domains means direct access to plugins isn't possible, instead it would use proxies which adds too much overhead here, since plugins will define and access entities, among many other things.

This means that once loaded, a plugin will stay loaded.

As such, the hook system is best implemented by extending the event system currently used in the engine. Making each event always pass an Event object around that has properties to support preempting further processing of an event. This is similar to features that exist on most GUI frameworks such as JavaScript's preventDefault.

This would make preemtable hooks such as SayText pretty easy to implement and use, and avoids the direct reference between the code that emits the event and the plugin system.

Registering such an event handler would be done by the plugin:
public sealed class TestPlugin : IPlugin
{
    private readonly IEventSystem _eventSystem;

    public string Name => "TestPlugin";

    public TestPlugin(IEventSystem eventSystem)
    {
        _eventSystem = eventSystem ?? throw new ArgumentNullException(nameof(eventSystem));
    }

    public void Initialize()
    {
        //The event system can deduce the event name from the event object type
        _eventSystem.AddListener(OnSayText);
    }

    public void Shutdown()
    {
        //Remove all listeners registered by this plugin object (but not other objects created by the plugin!)
        _eventSystem.RemoveListener(this);
    }

    private void OnSayText(SayTextEvent @event)
    {
        //Chat text starting with this command is hidden and suppressed from further event handling
        if (@event.CommandLine.FirstOrDefault() == "/hide")
        {
            @event.Visible = false;
            @event.Cancel = true;
        }
    }
}
This way plugins can plug into the engine and game without ever needing to expose the concept of plugins anywhere.

By making it register event handlers in Initialize and removing them in Shutdown plugins and the game itself can be rebooted, which can flush all past state.

In similar fashion plugins can also create and acquire console commands and variables by taking an ICommandContext. The same goes for registering entities; taking the entity factory and registering itself as providing a set of entities that override the normal entities so it can override built-in entities when needed (that isn't implemented yet).

The only downside compared to Sven Co-op and HLE's versions is that you can't reload plugins. To solve that problem CSharpScript based plugins may be used. There is ultimately very little difference between the two once the IPlugin instance has been created, and scripts can be loaded again, although if i understand it correctly the script will stay in-memory even if it isn't used anymore.

CSharpScript does have some downsides but i haven't used it enough to know how much difficulty those add. Handling the difference in the plugin manager is pretty straightforward, and supporting both here means map-specific support can be added by using a map-specific plugin manager. The same config file structure can be reused for everything, the only issue would be that assemblies would stay loaded in-memory.

I don't know how much overhead this adds in memory usage and other costs, but given that the SharpLife assemblies are between 25-80 KB in size when built as Debug, i don't think it's that much. Worst case scenario servers would need to reboot every so often to flush its memory, which they often need to do due to crashes anyway.

Since most maps that use scripts also tend to become popular the memory cost would only have to be paid once, so that does make things better.

This is only a basic prototype of a plugin system, i'll probably rework it to avoid directly depending on XML serialization so it can be used independently.

I would like to use this for HLMV to let people add functionality to it themselves more easily. Since people request all kinds of features, making it possible to easily add it yourself would really open up model customization if you could modify the model data from a plugin.

If anybody has any opinions about this i'd love to hear it.
You must be logged in to post a response.