.vsh / .psh - HLSL/ASM Shaders - SWGalaxies - I'm desperate

I've been working on a mod for SWGEmu, but I'm having extreme problems with the shaders. I'm totally inexperienced when it comes to HLSL and some of it seems to be in ASM.

What I mainly want to do is add depth of field if possible (It should be, a guy on an old mod forum said it was possible by modding the psh/vsh files), probably in the fog .vsh, but I'm not sure. Another thing is understanding the possibilities with the given files.

If anyone could take a few moments and look at it, I would be extremely grateful, I've been trying to figure this stuff out for hours and been researching, but with not too much progress. I've uploaded all of the psh/vsh and their .inc files, including 3 pastebin samples of it.

Re: .vsh / .psh - HLSL/ASM Shaders - SWGalaxies - I'm desperate

Posted 17 June 2012 - 12:13 PM

I haven't done any shader programming outside of copying code from tutorials. But I can tell you that there are two different ways to do shaders, which will probably explain part of what you are seeing.

Originally, shaders were programmed in machine language. You were writing code directly to the graphics card and there was no high level language that you could use. This became too much of a problem apparently, because the graphics cards companies quickly decided this was not going to work.

So, HLSL (High Level Shader Language) was introduced in order to make it "easier" to program shaders. I think very few people actually use machine code/assembler to write shaders anymore, but there's nothing to prevent you from doing it.

HLSL resembles C quite a bit, and most people prefer to use it or one of it's cousins; I believe OpenGL has their own high level shader language that is very similar.

Learning shaders is on my "to do" list, but I haven't gotten around to it. Suffice it to say, shaders is really an intermediate to advanced topic.

Since I haven't really done it myself, I can't really tell you the best way to learn HLSL, make sense of shader code, or find code to do something specific.

However, I bought two books that you might find useful if you can't find anything else. The first is an XNA programming book that turns out to really be a HLSL book coded in XNA. It's about 90% HLSL and 10% XNA, but it's all written to be used in XNA.

That's the book I intend to learn HLSL from when I get around to it. I've read parts of it, but haven't actually tried to write a shader from it yet.

Also, I bought a book in anticipation of eventually learning HLSL. This book is really about writing Shaders and not XNA. In fact, I "think" it's written around C++ and OpenGl rather than C# and XNA or even C++ and DirectX. Because of that, it actually doesn't have any HLSL code in it. Instead it uses the high level shader language for OpenGL or whatever it is (GLSL I think it is). But the book will tell you right in the beginning that the concepts of writing a shader are the same, no matter what language it's in. So, they say that it's not that hard to take a shader from there and port it over to HLSL or whatever.

Re: .vsh / .psh - HLSL/ASM Shaders - SWGalaxies - I'm desperate

Posted 17 June 2012 - 12:16 PM

Well, basically just get a feel or an idea what's actually possible. I don't really have a clue about it at the moment, programming isn't my strong suit, so HLSL is like a foreign language for me at the moment. I'm just trying to wrap my head around it slowly.

Maybe the question can't even be answered, I don't know, but it was worth a shot, I didn't get anywhere alone.

So to maybe rephrase, if it is answerable even, would it be possible to add newer shading functions to the game (I'm modding, I don't have access to the source file), such as HDR, Depth of Field, perhaps water reflections, etc, or is what the game gave me too limited to really mod anything?

The way the game handles the files is basically .sht (Shader, links to texture.dds and a effect.eft, the .eft then links to the .vsh/.psh.

Re: .vsh / .psh - HLSL/ASM Shaders - SWGalaxies - I'm desperate

Posted 17 June 2012 - 12:54 PM

It seems like the game will be able to handle a lot of different effects as it uses HLSL. I didn't see any assembly in those files you linked. You really need to get a book so you can start to learn how shaders work before you can actually create these effects.

The blur shader is pretty simple really

#include "pixel_program/include/pixel_shader_constants.inc"
#include "pixel_program/include/functions.inc"
// this is the texture that you will be sampling for this effect
sampler s0 : register(s0);
// the shader will be passed the current coordinates of the texture, it creates a variable called textureCoordinates consisting of 2 floats that make up
// the position of the texture.
float4 main
(
in float2 textureCoordinates : TEXCOORD0
)
: COLOR // this is what will be returned from the pixel shader, in this case the colour we want the pixel to be drawn
{
float4 result = 0.0f; // a variable made up of 4 floats that make up a colour r,g,b,a to be returned
float2 textureCoordinateOffsets = 0.0f; // a variable to hold the position of the pixel you are going to sample
// sample 16 surrounding pixels weighted by the values in userConstants
for (int i = 0; i < 16; ++i, textureCoordinateOffsets += userConstants[16].xy)
// the result is the current colour in s0 at texture coordinate textureCoordinates + textureCoordinateOffsets multiplied by the current value in userConstants
result += tex2D(s0, textureCoordinates + textureCoordinateOffsets) * userConstants[i];
return result;
}

This code is executed on every pixel which gives the blur effect.

If the userConstants doesn't make much sense then it is just an array like this, containing the amount you want amplify the current colour by multiplying it in the shader.

Re: .vsh / .psh - HLSL/ASM Shaders - SWGalaxies - I'm desperate

Posted 17 June 2012 - 01:12 PM

Yes, this is definitely getting me closer, I really needed that, thank you! Vsh works the same way I presume? For the most part I mean.

I don't plan on going full board into HLSL, but I'll take your advice and grab a book on it. This is really just for a mod, a big one I might add, but it's just a mod, I do want to add certain things in the end though, so I guess it's worth it.

I don't have time tonight to dig into it, but if you don't mind I'll probably have a bunch more questions tomorrow, I really appreciate it!

Re: .vsh / .psh - HLSL/ASM Shaders - SWGalaxies - I'm desperate

Posted 18 June 2012 - 03:10 PM

I've been messing around with both the .psh and .vsh, copied certain shaders on the others (like pink effect) to see where they get used, and then figure out what they actually do, with the main goal being to find where fog was handled. Well I found out a lot of stuff, but fog wasn't one of them.

The fog I'm talking about is used in the game to 'mask' the view distance, there is a slider for it ingame, here is an example how it looks at minimum, 50% and 100%:

What I'm trying to achieve is faux DoF, which a modder did a few years ago, but never released. He has disappeared since then, so I can't ask him how. He mentioned he modified some .vsh and .psh, but didn't go deeper than that. Here are 2 examples of what he did:

He also mentioned he was working on bluring the fog to create it and in the screenshots above, it's exactly what it looks like, that he used the fog for it.

After I went through the .psh and .vsh twice, I looked into the fog.inc, fog_setup.inc, and lighting_fog_setup.inc but to no avail, it didn't do anything at all, but then I DID finally find it in the vertex functions.inc, I do understand what it does, it tells the game where to render the fog based on the players distance, here is the code:

What I can't understand is, and I've talked to someone about it, it doesn't appear anywhere in the .psh files, and the only times it appears in the .vsh files, it's as output but doesn't end up affecting the fog in anyway. Most .vsh files have calculateFog included, here is an example:

My first thought was, that I'd copy the calculateFog code into the pixel shader functions.inc, and then try to maybe build it into the 2d_blur.psh, but the guy I talked to said it wasn't possible since the pixel shader doesn't see the fog (based on the fact that no .psh files does anything with the fog), and he doesn't even know how the game actually makes the fog in the end.

I'm kind of at a loss now, I thought I was close now, but it seems like I'm not, does the fog get handled hidden away somewhere then maybe? Or do you maybe have any ideas on how you could pull it off?

Re: .vsh / .psh - HLSL/ASM Shaders - SWGalaxies - I'm desperate

Posted 19 June 2012 - 05:51 AM

And it's so strange, I edited and used the 2d_blur as basis for something I wanted to try out, made my changes, saved it as 2d_bloom (so I can toggle it ingame via options), but it totally skipped the code and went straight for the normal blur effect upclose, then I copied the same code and used the original 2d_bloom, and it skipped the code and used the bloom code still in the .tre (archive) it seemed, or at least, since it's compiled and only editable in hex, the 'PEXE" information at the end of the file.

What the hell...

On a different note, someone told me I the 2d_blur can/should work for a DoF blur, only that I'd need to find a way to access the depth buffer to change the scale the offsets per pixel , hmm...

Re: .vsh / .psh - HLSL/ASM Shaders - SWGalaxies - I'm desperate

Posted 19 June 2012 - 07:18 AM

Friend found something about what the guy said how he did it:

"It also allows for camera blurring when focal on "centric" objects. "

"Nothing. Didn't touch sandstorms or fog. The only exception is that the draw distance now has a parameter in the Vertex/Pixel shader files that can force Depth of Field on some cards - but that won't affect sandstorms or natural in-game fog."

Hmm, any ideas?

P.S. Is there an edit function on this forum? I can't see it, sorry to keep double posting.