Vector graphics on the GPU with Stage3D

Hey guys and gals,

Started playing with vectors on the GPU last year but it sort of ground to a halt really, I didn’t like the ugly aliasing and let face it vectors should look smooth as a peach. Anyway, skip forward until flash player 11.6 beta was released supporting new shader op codes (namely ddx and ddy) woo, now I had the ability to add in that sweet sweet anti aliasing by leveraging the screen space derivatives!

Cool so now that I was able to render filled triangles and bezier curves (and soooooo close to cubic curves too – need help on this one though), I thought it should be easy to tie it in to the new readGraphicsData command as seen here:Query Graphics Data

Sadly not quite as straight forward as you might think!
The readGraphicsData method returns all the information to serialize the graphics of any object with graphics in it in flash! But there are still a few difficulties with parsing this data for use on the GPU.

Here are some of the issues:

Determining if a bezier curve is concave or convex (this depends on what side of the closed region the curve is on)

(of the two curves on the left shape one is concave and one is convex, blue and red respectively and the shader needs tweaking dependant on which one it is)

Determining if two curves overlap (i.e. in long sweeping thin curves… in this case the curve may need to be broken down into smaller curves)

Gradients fills! A couple of problems here, 1 is reversing the gradient matrix and the other is to replicate it on the GPU. Simple gradient fills are possible but can get complex easily as you start added more than two colours (maybe at this point a 1×256 pixel texture could be used as a lookup) . That said I have not got round to this yet so might not be to bad.

TRIANGULATION!!!!!! This is the real problem here… (at least to me it is). So from the output of the readGraphicsData we have extracted all the curves and along the way we have collected a series of points. These points make up the triangles that we use to fill the solid sections of the shape. Things get tricky however because these points do NOT automatically make up a nice sequence of triangles, you will get overlapping issues and a whole host of other problems. So this is where the triangulation comes in. At first I tried Delaunay triangulation but it was too greedy making triangles outside of the actual shape, so no good. Then I tried some ear clipping examples that I found but only 1 of the 3 I tried kind of worked and I say “kind of” because it goes into a number of infinite loops that I had to hardcode exceptions for and as such it misses a few triangles. (Also every now and again it would reject a complete path for no apparent reason). Not only that but the ear clipping algorithm is SLOW and doesn’t scale very well.

True vector graphics are great, they are something we love about the platform and will miss when the next major release of actionscript comes about. This is the reason why I though it would be great if we could emulate it on the GPU and I am sure it’s possible. It would however be 100 times easier if Adobe could extend their api to expose the result of the internal triangulation that they have already implemented in the player. That way they are still leaving it up to us to handle the rendering but we won’t have to spend an AGE trying to do work that they already have done a long time ago.

Anyway am bored of typing so I will post a small demo video (best viewed @ high resolution + full screen):

..Interactive demo coming soon.

If anyone wants to know more, discuss the topic, contribute, point me at an awesome triangulation library for as3, or anything else.. just drop me a message or reply to this post.

In an ideal world I would like this to become a small Open Source project. One that is not geared to any specific engine or renderer just a simple tool that can be used to generate the required data from any flash display object. If anyone is interested get in touch, I will be more than happy so share the code once its a bit more optimized and would love to see this become something useful not just for the developers of a particular engine but for all flash devs

This entry was posted
on Thursday, January 10th, 2013 at 1:07 pm and is filed under Uncategorized.
You can follow any responses to this entry through the RSS 2.0 feed.
You can leave a response, or trackback from your own site.

This looks like a really great way to preserve legacy assets and even continue using Flash Pro as an asset production tool. Using ddx and ddy produces much better results, but could there still be a mode where they’re not used for times where the extended profile isn’t available? Even if the quality is worse it’s better to have something than nothing.

There will still be the vector graphics… just no display list to render them. The idea being you can still create nice graphics in the IDE or with code and then render the result to a bitmap for use on the GPU. So no more native display list as we know it.

Awesome work dude! I can’t wait until things settle down with the platform. I can envision the entire displayList converted to the GPU in the near future. If FlashPro could just export it, this would be amazing!

Yeah, would also like to hear more about that “no more classic display list” statement, as that means most UI frameworks out there will be useless without extensive rewrites when AS 4 will be out late this year.

I came across this blog post and what you have demoed here looks amazing. I am playing with creating Flex components with GPU based rendering and thought I could take a look at your code on how you rendered vectors using the GPU. Can you please post your code somewhere or drop me an email?

I belieive the Flash rendering engine is not triangulating anything – it’s rather a “draw this line until you reach another shape” approach – hence it’s speed and some memorable glitches

about triangulation, as you mentioned, Delaunay won’t be of any help.
to perform a constrained triangulation, you’ll need to compute the Voronoi diagram and check wether each edge belongs to (or intersects) both sets in which case, it would be part of the “original” polygon (or so I vaguely remember).
quite an overhead anyway, there are articles on how to compute a triangulation on the GPU but I think it’s not what you’re after

Thanks for the comment/links/info! Yeah I am not trying to triangulate on the GPU just provide an easy way to convert graphics data to triangles that can be more easily stored and reused at a later date, i.e. for game ui’s, assets, animations and stuff like that.

A lot of what I did was indeed based on that paper but they don’t go into enough detail on the triangulation side of things for what I still need to figure out.

I will check out that poly2tri lib, it looks quite extensive and will probably be better than what I am currently using.
I have been INSANELY busy at work these last two weeks but things should calm down soon, and I will try to integrate that library and lob everything so far on github – see if I can’t get some fellow flashers to dive in and have a play.

It will be happening soon! Have made a start but am mental busy at the mo. Was hoping to have something ready for last weekend but got overrun. I will get there though! This post will be updated when it is there.

I’m very interested in this! Would love to contribute as soon as you have time to make the project available. I’ve implemented many 3D exporters and triangulation routines and found this post by searching to see if anyone had done it for Flash yet. Thanks for your work on this!

There really isn’t that much, so no excuse for me not having sorted this out sooner. My only problem is that it makes use of the “ddx” and “ddy” shader opcodes and I wasn’t sure if adobe was dropping support for them or not. Without them the vector graphics cannot be anti-aliased as and far as I am concerned that is an absolute must!!

I will get it on github this week in some form or other so folks can grab it and work on improving it.

I’ve been digging into graphics development for a while now and I’m sure
I can contribute something to your project as soon as it goes open source.
Could you perhaps give me access to the source code of your demo?
(I don’t mind if it’s not polished yet.)

I just found this post while looking for “stage3d vector graphics”. It is very interesting, so thanks a lot for sharing it.

I noticed you created a repository for it on github: https://github.com/bwhiting/b3d but the code is missing. It would be great if you could push the remaining files for others to continue your work on it. (Thanks a lot in advance).