Developers

Everything posted by danromeo

hi.
I'm trying to create a simple stencil mask on a RenderTarget2D from a bunch of primitives, and then later draw pixels from that render target to another rendertarget in a shader based on the stencil test pass/fail. Code is below. The results that I'm getting seems to be that the stencil test either always passes every pixel or always fails every pixel, regardless of which settings I try for DepthStencilStates.
The idea is to create an overhead view of a forested world and then lay that view over the terrain when viewed from overhead rather than redrawing the forests on every frame, BUT my question is about Stencil Buffers..
I set up the following resources:
MyRenderTarget = new RenderTarget2D(graphicsDevice, mapSize, mapSize, true, SurfaceFormat.Color, DepthFormat.Depth24Stencil8, 0, RenderTargetUsage.DiscardContents);
NewRenderTarget = new RenderTarget2D(graphicsDevice, mapSize, mapSize, true, SurfaceFormat.Color, DepthFormat.Depth24Stencil8, 0, RenderTargetUsage.DiscardContents);
DepthStencilState writeStencil = new DepthStencilState()
{
StencilEnable = true,
DepthBufferEnable = false,
ReferenceStencil = 1,
StencilFunction = CompareFunction.Always,
StencilPass = StencilOperation.Replace,
};
DepthStencilState stencilMask = new DepthStencilState()
{
StencilEnable = true,
DepthBufferEnable = false,
ReferenceStencil = 0,
StencilFunction = CompareFunction.NotEqual,
StencilPass = StencilOperation.Keep,
};
During initialization to create my overhead render target with stencil I set the DepthStencilState to stencilMask and draw the forests to the rendertarget, which SHOULD give me a stencil buffer containing 0's where there are no trees and 1's where there are trees.
graphicsDevice.SetRenderTarget(MyRenderTarget);
graphicsDevice.Clear(ClearOptions.DepthBuffer | ClearOptions.Stencil | ClearOptions.Target, Microsoft.Xna.Framework.Color.Black, 1.0f, 0);
graphicsDevice.DepthStencilState = writeStencil;
foreach (EffectPass pass in effect.CurrentTechnique.Passes)
{
pass.Apply();
graphicsDevice.DrawUserIndexedPrimitives<Position4Texture>(PrimitiveType.TriangleList,
Vertices, 0, 4, Indices, 0, 2);
}
graphicsDevice.DepthStencilState = DepthStencilState.Default;
And then at render time I render my terrain, and then in a second pass I set the DepthStencilState to stencilMask and render a quad over the terrain pulling pixels from MyRenderTarget based on stencil test pass/fail:
graphicsDevice.SetRenderTarget(NewRenderTarget);
graphicsDevice.Clear(ClearOptions.DepthBuffer | ClearOptions.Stencil | ClearOptions.Target, Microsoft.Xna.Framework.Color.Black, 1.0f, 0);
graphicsDevice.DepthStencilState = DepthStencilState.Default;
< DRAW TERRAIN TO NewRenderTarget >
graphicsDevice.DepthStencilState = stencilMask;
effect.Parameters["Texture"].SetValue(MyRenderTarget);
foreach (EffectPass pass in effect.CurrentTechnique.Passes)
{
pass.Apply();
graphicsDevice.DrawUserIndexedPrimitives<Position4Texture>(PrimitiveType.TriangleList,
Vertices, 0, 4, Indices, 0, 2);
}
graphicsDevice.DepthStencilState = DepthStencilState.Default;
And in the simple pixel shader I am returning:
return = tex2D(Texture,input.TexCoord);
I've tried various settings in the DepthStencilStates, and the end result is the stencil test either always passes all pixels, giving me overhead forests with black terrain, or always fails, giving me terrain with no forests. I've never used stencil buffers before but would like to make extensive use of them. Can somebody tell me what I'm doing wrong?
THANKS

Phil,
Very helpful, thank you very much. Simply put, when I set a new rendertarget I lose the stencil buffers on previous rendertargets. Is this correct? So stenciling needs to be done in a multipass inside of a single SetRenderTarget, yes? What a PITA....

Phil,
You answered with "So you can't set a new render target and use the stencil buffer from a previous one.", and then asked exactly what I was trying to do. I was unable to come back to this for several days so I started a new thread, including code explaining exactly what I'm trying to do.
Maybe I misunderstand you answer, but I'm not setting a new render target and using the stencil buffer from a previous one. Are you saying that I need to set the rendertarget, write to the stencilbuffer, and then immediately send the rendertarget to the pixel shader for the comparison operation? This makes no sense.....you can't send an active rendertarget to the pixel shader, it will error out. Or do you mean that I can only perform a stencil test on the active rendertarget? In this case if I lose my stencil buffer as soon as I set the rendertarget, how can stencil buffering be in any way useful at all? Would I have to create the stencil and perform the comparison test all inside one draw call?
SO if what I'm trying to do isn't possible maybe I could trouble you to explain exactly how I can do this? Really all I'm trying to do is mask certain pixels out of a rendertarget with an early stencil test.....seems pretty simple. This is driving me nuts, and I've found several other examples of people with pretty much the same problem who haven't found a resolution, or people who made it work in XNA 3 but couldn't make it work in XNA 4. I found one guy who says you need to use preservecontents on a rendertarget in order to write to the stencil buffer, but still no dice, although my tests do indicate that the stencil buffer is never being written to.
I can't even find a decent explanation of how stencil buffering works. I might be conceptually completely out of the ballpark. For example, I *think* what I'm doing is comparing the stencil buffer of a rendertarget to a simple reference value. Is this correct? Am I comparing the contents of the rendertarget stencil buffer to the stencil buffer of the back buffer?, noting that I'm not drawing to the back buffer? Is drawing to the backbuffer the only circumstance where stencil buffering will work?
Or maybe you could help me with simple code describing how simple a stencil mask works, that actually does work?
Much Confusion. I really appreciate your help.

Hi.
Can somebody explain to me or point me to a resource on using a basic stencil buffer on a render target in XNA 4? I'm finding a lot of XNA 3 resources and I know that it's changed in XNA 4, and a lot of articles on XNA4 that are overkill and confusing.
I want a rendertarget with a stencil buffer, which is drawn to during initialization. Then at draw time I want to pass the render target to the pixel shader, selecting pixels from the passed render target to render to the current render target, but discarding pixels based on the original render target's stencil buffer.
I THINK, correct me if I'm wrong:
The graphics device has a stencil buffer which is zeroed out when you set the rendertarget.
I have to enable the stencil buffer.
I can set the game state to compare the graphics device stencil buffer and the render target stencil buffer at draw time.
BUT, how do I set the values in the render target stencil buffer?
Also I'm using Multiple Render Targets, if this makes any difference.
I know this is probably simple stuff but I'm not finding any resource that gives a simple explanation of What It Is and How To Use It.
Thanks

How can I see the contents and allocations of my video card memory, either in real time or in Pix or any other (free) diag program? How can I poll the available video memory from a program before making allocations? What is the best way to fine tune allocations to video memory? I just realized that allocating beyond the available memory seems to crash the computer with no warning. How do I avoid this while still taking advantage of the full available memory? I'm using XNA at the moment but am porting to Sharp DX.
Thanks

How do I determine camera view angle to an object with the object's rotation factored in, to display a 2d "imposter" based on the actual view angle of the object?
I can get the camera angle to the object with:
half3 center = mul(input.inPos, xWorld);
half3 EyeVector = normalize(center-CameraPosition);
float lookYaw = atan2(EyeVector.x, EyeVector.z);
But I'm not having any luck determining the view angle with object rotation factored in either by adding or subtracting the EyeVector with the ObjectRotation vector or by atan2'ing the ObjectRotation vector and adding or subtracting the result with lookYaw. All vectors in question are normalized.
Hope this makes sense. Thanks!

Hi.
I have a version 3.0 vertex shader for (what I call) Static Imposters that essentially decides which image in a texture atlas should be used and sends that info to the pixel shader.
I am confident that the shader is written correctly but it always displays the wrong image! If I run the shader through a debugger, the numbers are all correct, but the wrong image displays. I have isolated this to a single variable, whose value is assigned from the program with Effect.Parameters["ImagesPerView"].SetValue(ImagesPerView);
Running the shader through the Pix debugger, the value of ImagesPerView is always set correctly to 12, but the program displays the image as if the value of ImagesPerView is set to 11. If I hardcode the value of ImagesPerView in the vertex shader to 12, overriding the program assignment, the program displays the correct image! WHY am I getting different results even though the value of ImagesPerView is always 12?
The program is assigning the value of 12 to ImagesPerView on every draw call. I don't think this is due to any sort of implicit conversions as I have checked the code thoroughly, and even did an (int)ImagesPerView conversion on every operation to make sure. All of the other variable assignments in the shader appear to be correct and functioning correctly. Stared at it until I was crosseyed.....What The Heck am I doing wrong?
Any ideas welcome.
THANKS

This topic is old because I've been involved with other things. I've seen conflicting info re: use of int's, seems I saw a MS recommendation to use int's to avoid rounding errors. Regardless, I get better results when replacing ints with floats. Thanks!

Hi. Did you run scandisk with both checkboxes checked, or with both the ./F and /R switches (chkdsk /f /r)? The /R switch, or the bottom checkbox, repairs bad sectors by copying the data to another location and marking the sector so it won't be accessed anymore. Check your logs after running, if it found any bad sectors run it again and see if it finds anymore.....or if it finds a lot of bad sectors.....replace the drive. The /R switch may run for a long time depending on how many bad sectors it finds....maybe hours. BACK UP YOUR HARD DRIVE BEFORE SCANNING WITH /R.....if your boot sector is bad the /R switch can cook your operating system and cause much sorrow.
Also scandisk isn't 100% reliable, for example it won't find problems with the drive motor, which would still cause errors if it's failing. If you can't get past your error message you should clone the drive and replace it. IMO you should replace the drive now.....I'm not a DX expert but I don't think hardware errors are a result of bad code because code never directly accesses hardware, so you can't write any code that would access the hard drive in the wrong way, etc...all of this is negotiated by DX and the bios. Also if you suddenly have bad sectors then more will probably follow, and keep in mind that you're only seeing bad sectors that your program is trying to access.....there may be many more. .
If you're dealing with a drive failure you'll never find the source of the problem until you repair or replace the drive. If the drive crashes before you replace it recovering will be a lot more difficult, or impossible, or very expensive, so make sure you're backed up.

I thought for sure that unbird must be right with rounding errors and/or int behavior, but I think I've covered this and I'm still getting incorrect results.
Declaring all globals as floats in the shader. Rounding values up to the nearest integer and truncating. Still the same results and still seems to come down to this one single global. Again, the number I'm sending to the pixel shader displays in PIX as 0.00 but the program behaves as if it is a different value.
More Facts: If I hard set the global value in the .fx file and never throw a value from the program at all I still get the same incorrect results. BUT if I declare a variable in the vertex shader and assign the same value everything displays correctly. This must be a significant fact....not sure what to do with it though. What's the difference between a global and a variable in an .fx file? How are they treated differently?
I tried changing the declaration order in the .fx file, still same results. I keep thinking it must be a syntax error somewhere in the .fx file but I can't find any problems and obviously it compiles.

Indexing, definitely a good idea but I've already checked this.
Also if I throw a hard value from the program (Effect.Parameters["ImagesPerView"].SetValue(12)) it still seems to munge the value, but if I hard set a variable value in the shader (ImagesPerView=12) everything displays correctly. What's the difference??
I'm not an expert but I know shaders make some assumptions about variables. For example in different parts of the program I'm sending different values to ImagesPerView....but I still don't understand how the debugger can display all of the correct values and yet the program seems to be doing something entirely different.
Rebuilt the entire project....still same results.
Before I post code I'm going to remove all of the instances of throwing different values to ImagesPerView and then strip the shader code down to it's bare minimum. This looks like some sort of munging to me.....for example hard coding the variable value in the shader results in a lot of code being skipped, so the problem could be in any line of the skipped code. There is a small bit of branching in the shader that I'm suspicious of, although I've used branching in shaders lots of times without any problems. it stands to reason that if all the vertex shader did was accept a value from the program and send it to the pixel shader that it would send the correct value!
If any of this inspires any ideas please speak up! THANKS for your suggestions.