I just started making the effect files for my game,the first one is gonna be the terrain texture blender,but it kinda confuses me.What exactly are the in and out keywords?I downloaded a sample blending effect that blends 2 textures according to a blendmap.My idea is to make it blend tile textures on a grid,according to a tilemap(each square on the grid being some kind of terrain type-grass,rock,dirt,sand).Here is what i got so far:You declare variables at the top that you later link to the variables you want to operate on in your C++ code,you make a struct for vertex shader and for pixel shader with members corresponding to your custom vertex declaration in C++,you make functions that operate on the structs/variables/whatever and then you compile the shader.I saw a main function in only a few of the sample .fx files I downloaded,so i guess it's not manditory.Does this mean that when you use an .fx file ALL functions in it are called?

The only two functions in an .FX file that are guaranteed to be called are the two functions that you list at the end (in "pass P0"), "vertexShader" and "pixelShader" in your case. You can, however, have any number of helper functions etc. above these and call them from either of the two 'main' functions (if you want to call them that).

It's important to keep in mind how the data flows through the shader - I personally use four structs in my HLSL shaders to keep the code readable, your example shader uses only two (the output structs, these are mandatory). For the input to both vertex and pixel shader it just lists the variables as function parameters. I replace that mess with two additional structs so in total I have "vertex_in", "vertex_out", "pixel_in" and "pixel_out", each a struct holding exactly the data that the vertex or pixel shader passes in or out.

This is how the two vertex shader structs would look like for your shader:

You don't have to compile the shader yourself before running your game, D3DX can do this for you via "D3DXCreateEffectFromFile". Make sure to read any error messages if that function (and the shader compile) fails, it'll tell you when there's an error in your shader and where it is!

It's practical this way because I can have the .FX file open in a text editor or my compiler IDE, make a change to the shader and run the .exe again, no need to re-compile or run the HLSL compiler or anything.

Apart from that, I don't quite get your texture blending plan quite yet. The easiest way to do this usually is have the blendmap be just different shades of gray where the brightness of a pixel determines how much of texture A or texture B is mixed in. Looks like the shader you posted should do exactly that!

If you had a vertex or pixel shader that used the same structure type as the input and an output then you would use the "IN " and "OUT" keywords as a prefix to determine which var in the structure you were referring. I would assume this had to do with older versions having significant memory limitations and the IN and OUT functionality allowed for flexibility in this scope.