I'm pretty new to osgPPU and I'm working on a port of Jimenez's MLAA full scene antialiasing algorithm.

It's a 3-pass post processing effect:

- The first step detects the edges of the RTT'd scene and makes an 'edge texture'.
- The second step uses the previous texture and a precomputed one to calculate the area which pixels belonging to an edge would cover from an ideal trapeze.
- The final step blends each pixel in its 4-neighborhood according to the weights calculated in the previous step.

I got it fully working in osgPPU, however, an important optimization can be done by using the stencil buffer. The idea behind this is:

- Initially the stencil is cleared out to 0

- The first step of the post effect (edges detection) creates a stencil mask writing 1's to the stencil wherever there's an edge pixel. The pixels not belonging to an edge are discarded in the shader, so no writting to the stencil will occur in that case.

- In the subsequent steps (weights calculation and final blend) the stencil test discards every pixel not covered by the mask, hence only the edge pixels are processed.

And this is how I tried to achieve it:

- I created two osg::Stencils, one for creating the mask and other for using the mask and discarding everything else.
- I attached the stencils to their corresponding osgPPU::Unit's.

(now, I need to clear the stencil at the beginning of the postFX chain, but not between each pass)
- I'm not sure about where glClear (with the clear mask of the camera the osgPPU::Processor is using) is called, but my guess is when rendering the scene to texture, not between units so I think that setting the clearMask in the camera is pointless.
- I tried to call glClear(GL_STENCIL_BUFFER_BIT) in a notify callback set in the first unit of my chain.

This has been driving me nuts for a couple of days now. It seems that the stencil buffer is not being written, or cleared prematurely between passes. And the stencil test only seems to be done if I enable it in the last unit's stateset.

Only registered users can see emails on this board!Get registred or enter the forums!

)>

Quote:

Hi,

I'm pretty new to osgPPU and I'm working on a port of Jimenez's MLAA full scene antialiasing algorithm.

It's a 3-pass post processing effect and I got it working on osgPPU.

- The first step detects the edges of the RTT'd scene and makes an 'edge texture'.
- The second step uses the previous texture and a precomputed one to calculate the area which pixels belonging to an edge would cover from an ideal trapeze.
- The final step blends each pixel in its 4-neighborhood according to the weights calculated in the previous step.

I got it fully working in osgPPU, however, an important optimization can be done by using the stencil buffer. The idea behind this is:

- Initially the stencil is cleared out to 0

- The first step of the post effect (edges detection) creates a stencil mask writing 1's to the stencil wherever there's an edge pixel. The pixels not belonging to an edge are discarded in the shader, so no writting to the stencil will occur in that case.

- In the subsequent steps the stencil test discards every pixel not covered by the mask, hence only the edge pixels are processed (weights calculation and final blend).

And this is how I tried to achieve it:

- I created two osg::Stencils, one for creating the mask and other for using the mask and discarding everything else.
- I attached the stencils to their corresponding osgPPU::Unit's.

(now, I need to clear the stencil at the beginning of the postFX chain, but not between each pass)
- I'm not sure about where glClear (with the clear mask of the camera the osgPPU::Processor is using) is called, but my guess is when rendering the scene to texture, not between units so I think that setting the clearMask in the camera is pointless.
- I tried to call glClear(GL_STENCIL_BUFFER_BIT) in a notify callback set in the first unit of my chain.

This has been driving me nuts for a couple of days now. It seems that the stencil buffer is not being written, or cleared prematurely between passes. And the stencil test only seems to be done if I enable it in the last unit's stateset.

What am I missing?

- Have you create an opengl context with a stencil buffer ?
- Have you try to use osg:ClearNode to clear the stencil buffer ?
- Have you check opengl call order in a debugger like gDEBugger ?

- Have you create an opengl context with a stencil buffer ?
- Have you try to use osg:ClearNode to clear the stencil buffer ?
- Have you check opengl call order in a debugger like gDEBugger ?

- Yes, the context that osgViewer creates when realizing has a working stencil (I tested it).
- No, I didn't know that one was allowed to put nodes such as osg::ClearNode inbetween osgPPU::Unit's
- No, but I was planning to use GLintercept as a last resort

Quote:

yes please, could be more simple to find the bug.

Sure!, here it is:

This function generates the whole pipeline and attaches its processor to an osg::Group which is returned

// This stencil is intended to create the mask by writting a 1 to the stencil wherever a fragment is not discarded in the shader of the unit it's
// attached to. However, the stencil test has to pass always and osgPPU disables the Z test, so I don't know if this is really working as I want!
osg::Stencil * createMaskStencil = new osg::Stencil;
{
createMaskStencil->setFunctionRef(1);
createMaskStencil->setFunction(osg::Stencil::ALWAYS);
createMaskStencil->setWriteMask(1);
createMaskStencil->setOperation(osg::Stencil::REPLACE, osg::Stencil::REPLACE, osg::Stencil::REPLACE);
}

// This one is intended to discard every fragment not masked by a 1 in the stencil
osg::Stencil * useMaskStencil = new osg::Stencil;
{
useMaskStencil->setFunctionRef(1);
useMaskStencil->setFunction(osg::Stencil::EQUAL); // TODO: TEMP!!, Should be osg::Stencil::EQUAL!
useMaskStencil->setWriteMask(1);
useMaskStencil->setOperation(osg::Stencil::KEEP, osg::Stencil::KEEP, osg::Stencil::KEEP);
}

// This unit just loads up a texture which is needed by the next shader
osgPPU::UnitTexture* areaTexture = new osgPPU::UnitTexture();
areaTexture->setName("MLAA.areaTextureUnit");
{
osg::Image *areaImg = osgDB::readImageFile("Textures\\AreaMap65.dds");

weightsCalculation->getOrCreateStateSet()->setAttributeAndModes(weightsShader);
weightsCalculation->setInputTextureIndexForViewportReference(1); // Choose to have an output the same size as edgesMap has

// Attach the 'usueMask' stencil here
weightsCalculation->getOrCreateStateSet()->setAttributeAndModes(useMaskStencil, osg::StateAttribute::ON);
weightsCalculation->getOrCreateStateSet()->setMode(GL_STENCIL_TEST, osg::StateAttribute::ON);
// Just to make sure that the stencil is not being cleared anymore
weightsCalculation->setBeginDrawCallback(dontClearStencilCallback.get());

}

// Last step: Blend the edge pixels with their 4-neighborhood according to the blending weights
// (again, only the masked pixels have to be processed)
osgPPU::UnitInOut* pixelBlend = new osgPPU::UnitInOut();
pixelBlend->setName("MLAA.pixelblendingUnit");
{
osgPPU::ShaderAttribute* blendShader = new osgPPU::ShaderAttribute();
blendShader->addShader(osgDB::readShaderFile("Shaders\\blend_frag.frag", fragmentOptions.get()));
blendShader->addShader(osgDB::readShaderFile("Shaders\\offset_vert.vert", vertexOptions.get()));
blendShader->setName("BlendShader");

// It looks like for some reason no stencil testing is performed at all unless I enable the GL_STENCIL_TEST in this unit
// Let's recall that the testStencil does always pass
unitOut->getOrCreateStateSet()->setAttributeAndModes(testStencil, osg::StateAttribute::ON);
unitOut->getOrCreateStateSet()->setMode(GL_STENCIL_TEST, osg::StateAttribute::ON);

/**
* Copyright (C) 2010 Jorge Jimenez ()
* Copyright (C) 2010 Belen Masia ()
* Copyright (C) 2010 Jose I. Echevarria ()
* Copyright (C) 2010 Fernando Navarro ()
* Copyright (C) 2010 Diego Gutierrez ()
* All rights reserved.
*
* Adaptated by Miguel Angel Exposito ()
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
*
* 2. Redistributions in binary form must reproduce the following statement:
*
* "Uses Jimenez's MLAA. Copyright (C) 2010 by Jorge Jimenez, Belen Masia,
* Jose I. Echevarria, Fernando Navarro and Diego Gutierrez."
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ``AS
* IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
* THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL COPYRIGHT HOLDERS OR CONTRIBUTORS
* BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*
* The views and conclusions contained in the software and documentation are
* those of the authors and should not be interpreted as representing official
* policies, either expressed or implied, of the copyright holders.
*/

As promised, here is the source code for the Jimenez's MLAA demo for OSG using OSGPPU. I haven't managed to figure out the stencil buffer thing yet, so if any of you guys is up to the challenge, any help would be greatly appreciated!

You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot vote in polls in this forumYou cannot attach files in this forumYou cannot download files in this forum