Alpha in Starling Filters and Basic Branching in AGAL

Back in September, a Chris posted a comment on this blog asking about a Starling filter that would create a circular mask over an image. Being the lazy sort I am, I never got back to the commenter (Sorry, Chris). Some stuff I was doing at work the other day reminded me of the question, though; and, although it’s a relatively simple thing to do, the task raises two interesting problems I thought deserved a whole post rather than just a quick response.

In plain English to create a circular mask we would want to do something like this: Find the distance of every pixel in an image from the center of our circle. If that distance is greater than the radius of our circle then we should set its alpha to zero. Well, to find distances we can turn to the good ol’ Pythagorean theorem – distance equals the square root of distance x times distance x plus distance y times distance y (I’m still amazed I use information I learned in high school in my day to day life. Who’d a thunk those teachers were on to something?). In a sort of pseudo/AS3 code, our operation may look something like this:

ActionScript

1

2

3

4

5

6

7

8

9

10

vardistancex=pixel.x-center.x;

distancex=distancex*distancex;

vardistancey=pixel.y-center.y;

distancey=distancey*distancey;

vardistance=distancex+distancey;

distance=Math.sqrt(distance);

if(distance>radius)

{

pixel.alpha=0;

}

We can start to port that pseudocode over to AGAL like so:

Assembly (x86)

1

2

3

4

5

6

7

8

9

10

11

12

; Assume v0 contains original pixel position

; Assume fc0 contains our circle definition:

; fc0 = [centerX, centerY, radius, 1]

subft0.x,v0.x,fc0.x

mulft0.x,ft0.x,ft0.x

subft0.y,v0.y,fc0.y

mulft0.y,ft0.y,ft0.y

addft0.z,ft0.x,ft0.y

sqtft0.z,ft0.z

texft1,v0,fs0<2d,clamp,linear,mipnone>

; ...

But there lies the rub… We now have our distance value in ft0.z and our texture information (red, green, blue, and alpha) in register ft1.xyzw, but how do we do the if conditional? Ideally, we would like to write something like:

if (ft0.z > fc0.z) mov ft1.w, 0

Unfortunately, AGAL doesn’t provide such explicit branching statements. It does, however, provide four nifty little ‘set’ operations which, with a little ingenuity, can handle such conditionals: SGE (“set if greater than or equal”), SLT (“set if less than”), SEQ (“set if equal”), and SNE (“set if not equal”). These operations will set a register’s component to either 1 or 0 depending on whether or not the conditional passes. Let’s take a look at SLT for a moment. SLT (like the other 3 set operations) takes two arguments. If the first argument is less than the second the result will be 1, otherwise the result will be 0. In Actionscript, the operation would look something like this:

var result:int = arg1 < arg2 ? 1 : 0;

So, how is that helpful? Well, we can use that 1 or 0 in a multiplication operation to get either 0 or the original result. If we go back to our pseudocode and re-write it using a ternary statement similar to the one above, it could look like this:

ActionScript

1

2

3

4

5

6

7

8

9

vardistancex=pixel.x-center.x;

distancex=distancex*distancex;

vardistancey=pixel.y-center.y;

distancey=distancey*distancey;

vardistance=distancex+distancey;

distance=Math.sqrt(distance);

varconditional=(distance<radius)?1:0;

// alpha is now either its original value or 0 if the distance is greater than the circle's radius

pixel.alpha=pixel.alpha*conditional;

We know we can do that in AGAL, so now our complete fragment shader can look like this:

Assembly (x86)

1

2

3

4

5

6

7

8

9

10

subft0.x,v0.x,fc0.x

mulft0.x,ft0.x,ft0.x

subft0.y,v0.y,fc0.y

mulft0.y,ft0.y,ft0.y

addft0.z,ft0.x,ft0.y

sqtft0.z,ft0.z

texft1,v0,fs0<2d,clamp,linear,mipnone>

sltft0.w,ft0.z,fc0.z

mulft1.w,ft1.w,ft0.w

movoc,ft1

So, the main point here:

If you need to do some basic branching in AGAL, look for a way to use a “Set” operation and multiply the result with another value to get either a 0 or the original value.

If we plunk that AGAL into a Starling filter now, it might look like this:

ActionScript

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

/**

* Copyright (c) 2012 Devon O. Wolfgang

*

* Permission is hereby granted, free of charge, to any person obtaining a copy

* of this software and associated documentation files (the "Software"), to deal

* in the Software without restriction, including without limitation the rights

If you try that filter though, you’ll notice something very odd. The circle is there all right, but instead of the outside being transparent, it has kind of a ghosted screen look. An interesting effect, maybe, but not what we wanted. That’s because when dealing with shaders in Stage3D, in order to get alpha values, you have to set the blend factors of the Context3D instance. If you check out the Adobe documentation, they even, very helpfully, tell you what blend factors to use. Since, in our case, we want to to use Alpha, back in the activate method of our CircleMaskFilter, just before we set the program of the context, set its blend factors like so:

A little gotcha though – as soon as we’re done running our filter, we need to reset the context3D’s blend factors back to No Blending. Luckily, the good folks who put together the Starling framework made that simple enough. Just as there is an activate method in the base FragmentFilter, there is also a deactivate method. Simply override that method and reset the blend factors to no Blending:

And that’s really all there is to it to handle alpha in a Starling filter. Once you get that down, you can do all sorts o’ stuff. With a second texture you can easily create complex masks. Or instead of finding the distance between two pixels as we just did, you can calculate the distance between two colors. Why (you ask)? Well, imagine you have a target color and you calculate the distance between it and each pixel in your texture. If the distance is below a give threshold, you set its alpha to 0, and suddenly you have a basic greenscreen application.

3 Comments:

Nice one!

Just a heads up though:

ife Jump if source1 is equal to source2
ine Jump if source1 is not equal to source2
ifg Jump if source1 is greater or equal than source2
ilt Jump if source1 is less than source2
els Else
eif Close an if or else block

these have been added to AGAL(2?), woooo branching, making shaders like this even easier

That’s funny. Just Friday I was reading through AGALMiniAssembler.as getting annoyed at all the opcodes there that weren’t actually used by AGAL.. I guess now they are.. I wish Adobe would keep up the documentation on AGAL – or maybe I just completely missed it..