So, despite many years of programming experience, I never had anything to do with shaders. I'm more of a game mechanics guy. But I found myself forced to try to write a shader recently, because I want to subtract one texture from another, and on my Tegra-3 based Android device, ALLEGRO_DIST_MINUS_SRC simply don't work at all.

But the problem is, whenever I attach a source for a vertex shader, any kind of output disappears!

Nothing is visible. However, if I omit attaching vertex shader, then something is drawn at the necessary point.

If I set targetBitmap as a target, instead of sub-bitmap, things seem to work even with vertex shader, but this will lead to problems later, so I would prefer this code to work with sub-bitmap of targetBitmap.

What am I doing wrong here?

SiegeLord

I'm pretty sure you can't replicate blenders just like that, so I'm surprised that one of your setups actually worked for you. From my understand of how shaders work, to replicate SRC_MINUS_DEST you'd have to have 3 bitmaps: a destination bitmap (could use your backbuffer), your source bitmap (I imagine that's the targetBitmap) and then the bitmap you will subtract (m_damageImage). Then you'd do something like this:

Oh, but the problem is, I didn't even get to the point where I try to replicate the subtraction! For now, my shader just paints everything red And it doesn't work when I use it with sub-bitmap, as described above.

SiegeLord

Oh I see. I tried a setup like yours in ex_shader, and it seemed to work ok. Could you paste your pixel shader (I imagine it's pretty simple, but still). Alternatively, also try ex_shader yourself.

Max Savenkov

OK, I tracked down my problem (problems, actually). First, sub-bitmaps are not very useful with shaders, since they do not change geometry passed to shader. Second, I did not take OpenGL's coordinate system into consideration properly

SiegeLord

Is this a recent Allegro 5.1? A week or two ago I landed the projection reform which switched to using viewports for sub-bitmaps, which should have made them operate predictably.

Max Savenkov

I haven't updated my copy to the latest git yet. Maybe I'll try, though I have a nice working version already. Almost In reality, I'm still missing something: the whole render gets corrupted after I use my newly written shader outside of test in a real game.