Hi all,I honestly dont know where else to post this, since it probably is a n00b question, it has its home here. In any case...

I am using EXT_packed_depth_stencil to render to DEPTH24_STENCIL8 texture target and I wish to access the stencil value from fragment shaders in order to do some funky shadow volume/map mixing of their ambient factor.

Anyway, how can I access the stencil? The specs for EXT_packed_depth_stencil say that a sampler2D has an undefined operation on a DEPTH24_STENCIL8 format, only sampler2DShadow has defined operations. How does the sampler2DShadow deal with stencil values?

The idea was to access the stencil buffer to unify shadow maps/shadow volumes in their ambient factor so that I can mix and match them. As you probably know, the ambient factor of shadows in maps is just a constant you add at the end of the lighting equation, whereas in shadow volumes, the alpha of the full screen quad changes the ambient factor of the shadows.

I was hoping to be able to access both the shadow map and the stencil buffer from fragment shader so that I can treat both colour values the same...

Render a big-white quad using the stencil to a texture, then sample the texture?

It's slower, but... it's obvious you can't access sstencil-values from the fragment-shader, as the fragment-shaders are not run if the stencil-test fails. Besides... stencil-buffers are plain bits, and i don't think there is a sample-function for bits out there (at least not in DX9 cards)

Hi, appreciate more people! Σ ♥ = ¾Learn how to award medals... and work your way up the social rankings!

I am using EXT_packed_depth_stencil as a texture format for the stencil buffer to be rendered to as a target. The texture should hold the bits of the stencil. When passing this texture to a fragment shader as a sampler2D, you cannot access the stencil information through any current means. Your slightly right in the sense that there isn't a sample function out there, but the fragment shaders that I need will run as they are not stencil tested.

I was hoping to pack the shadow map inside the 24bits of the texture and the stencil shadows in the other 8 bits and use 1 texture for both shadow maps and shadow volumes. But It seems like im going to have to forget about EXT_packed_depth_stencil for the time being as it doesn't give me enough functionality to work with.

whereas in shadow volumes, the alpha of the full screen quad changes the ambient factor of the shadows.

There's no full screen quad in any shadow volume algorithm I've seen.

Quote from: darkprophet

The idea was to access the stencil buffer to unify shadow maps/shadow volumes in their ambient factor so that I can mix and match them.

That's easy. Render an ambient only pass first (that also inits your depth buffer, nicely and fast), then render each light using either shadow maps or stencil volumes or both, without worrying about ambient (the shader should only add more light to the scene).

Yup, i know . I would render into the depth attachment from the light's perspective. Then from the camera's perspective render the to the stencil attachment and send to the fp shader. The sampler2DShadow would then figure out which pixel is in shadow regarding the SM. With the stencil, any stencil bit/pixel thats not 0 is in shadow. You then combine the values of the stencil (clamp to between 0 and 1) and add to the SM value. You then obtain a final value to say whether this pixel is in shadow or not.

Quote

There's no full screen quad in any shadow volume algorithm I've seen.

Umm, there is. After you have the stencil buffer ready and before you render all all shadow volumes, you render a fullscreen quad so that you store it to give some ambient after wards.

Quote

The final section of this function draws one blended rectangle over the whole screen, to cast a shadow. The darker you make this rectangle, the darker the shadows will be. So to change the properties of the shadow, change the glColor4f statement. Higher alpha will make it more black. Or you can make it red, green, purple, ...!

I will be very surprised if rendering the entire scene with colours and depth as an ambient pass is faster than rendering a fullscreen quad with alpha. But I guess your right about the specular highlights...

I still think that my way is faster and better than both, but its a shame that I need to render the stencil texture to another texture to get values...

java-gaming.org is not responsible for the content posted by its members, including references to external websites,
and other references that may or may not have a relation with our primarily
gaming and game production oriented community.
inquiries and complaints can be sent via email to the info‑account of the
company managing the website of java‑gaming.org