A couple great articles from O’Reilly on using social graphs to visualize data. Andrew Odewahn revisits and older project in Visualizing the Senate social graph, revisited – OReilly Radar. The basic notion is to go through senate voting sessions with each senator as a node. When there is a pattern of frequently having similar votes a node is drawn. He does this over an number of different periods giving insight into the political climate. Definetly watch the video.

It is really powerful to be able to connect your Processing applet to external data sources. This enable art infused with data and other great mash-ups. I have done a couple of different projects that connect to Amazon Web Services, YQL and Google Spreadsheets. The Google solution is perhaps the most flexible, because if the data does not reside in your spreadsheet you can import it via the spreadsheet functions.

Writing the Processing code to connect is pretty straight forward. You can use functions like loadStrings to reference URLs that have your data.

This works really well in the IDE… but when you upload it you will see that it will not work. The problem is that the Java applet’s security does not allow it to access urls and content that are not on the same server. If it could there could be viruses that downloaded malicous content to your machine.

Standard Approach – Sign the Applet

The standard option is to sign your applet and create a certificate for the user to approve. To do this you must make sure you have the Java KeyTool and Jarsigner downloaded. Then you need to issue the follow commands in ms-dos or the terminal on a Mac:

Alternative Approach – Redirect to a Local File

As an alternative you can use an intermediate script file to bring the external sites data to your server. This is the approach I took in building my Flickr Kaleidoscope. To create the same result as above, but without the signing the app I created a small php file that I can reference from Processing:

There is an additional advantage to this approach is that if the data needs to be further processed and “massaged.” In this case you can use the PHP functions to parse the data, parse xml, create\sort arrays and much more.

Whichever approach is pursued their is a lot of power in connecting Processing with data from the web. The Processing visuals can be driven from data and have a dynamic element that comes from refreshable data.

“Vedea is a prototype of a new experimental language for creating interactive infographics, data visualizations and computational art.”

Microsoft has noticed what the guys at MIT have been up to with Processing and is preparing to launch Vedea, which they describe as “a new visualization language.” Just knowing Microsoft many of us may have a little skepticism… not that the tool won’t be good or exciting, but will it be open, and free? Being a strong believer in competition this entry can only make the space better, with more minds working to improve the creation of data visualization.

On the Microsoft Research Blog there is a write up with some detail, including some code snippets. In reviewing the code it looks like the language will offer some predefined GUI Components and Data Visualizations built in. This could actually be very valuable including these in the core (as long as it is extensible with external libraries). Processing today is open and there are libraries that cover these areas, but integration can be cumbersome.

It looks like launch is planned for early 2010.

Update: 12/5/09

Here is a blog post detailing a demo of Vedea with the data, code and an image of the data visualization it created.

I remember getting my TRS-80 when I was about 10. I flipped through the manual that came with it and was entranced at the kaleidoscope program at the end of the book. I was amazed to see that a computer could reproduce these patterns and colors in beautiful synchronization. This is my first real memory of programing and computer generated art.

I decided to revisit that memory and create my own kaleidoscope. It allows you to search Flickr titles and tags to return a set of images. You can then flip through the images and adjust the number of mirrored slices. If you are interested in how I built it please read on.

I have been developing with Processing for a couple of years and wanted to see what could be done to make a Kaleidoscope. The technology stack that I came up with was to using Processing to access a dynamic php page that looked up images on Flickr via a REST call to YQL. So let me explain what that looks like…

As you can see it is pretty straight forward. I use YQL (there may be a # of call per day limit on this) to conduct a search based on a term passed to the php page. I then parse the results into variable that are then used to build urls. The generated urls are then output as text.
While the program is running I call this page from processing by:

And that is pretty much it. Sounds easy, but it did take some time to get it to work right. I want to work to add some features around saving images and possibly a gallery, but since I have a working v1 I had better get it out (perfection is the enemy of good enough).

So from some of my earlier post it is clear that I am a fan of the Processing IDE for create artistic programs and applets. I have also been struggling with finding the right language (Javascript, Actionscript, ?!?!) to build out some new ideas with for the web. I think my prays have been answered. John Risig has ported Processing over to Javascript, call his creation Processing.js. This means that “most” of the cool things that you could do in Processing, you can now do on the web without a java applet and much better integration with html.

The biggest down side I see at this point is limited and inconsistent browser support for the javascript canvas required for Processing.js to work. So if your project is for commercial use it may not meet compatibility standards. Of course Internet Explore is the least compatible. Firefox is great and Chrome seems to work pretty good.

Processing.js has been around for a little over a year… it came out in May of 2008, but it really seems to be getting some traction. And I just discovered it this month, so it is new to me. Below are some resouces that I have found helpful:

I am sure there is going to be a ton of cool stuff that can be made with this…

The I think some charts and graphs would be great (again no text rendering for now). Have you seen Ben Fry’s book on data visualization: Data Visualization by Ben Fry . A lot of the examples in this book should be portable.

Of course there is the art of processing

Games anyone?

Processing tool for creation and enabling it with Javascript would seem to open up possibilities to new audiences and new uses (I particularly like the sketch in the background of Robert O’Rourke Blog).

As you can tell I am super excited by this… my enthusiasm is only tempered by the fact that IE does not support the JavaScript Canvas object that is required to render. 😦

So I just read an article regarding the cost to build a Kindle. I won’t give it away, but it also brought up the key difference of a Kindle vs other e-readers (or books for that matter). The difference is the Whispernet wireless service that allows you to get content on your device wirelessly.

I cannot stress enought that this difference changes everything. If you are thinking about a Kindle you have to understand this feature! Basically for no monthly fee you have access to the Internet and the Amazon store (and yes other websites). It is terrific being able to think about buying a book or magazine and then have it appear. Instant gratification!

I am also finding other ways to utilize the service. For a $.10 fee you can email a pdf to Amazon and they will convert it to their format and send it to your Kindle. I am starting to use this quite a bit when someone sends me something at work that I do not want to stare at my computer screen and read. This is a great way to get through any “white paper” reading for your job.

If you have not see or heard of Processing the IDE for art using Java you should really check out Processing.org. Basically a couple of really smart guys from MIT came up with a way to make Java programing much easier for artist. Processing just reached 1.0 and it is a ton of fun for the technically challenged but artistically inclined. Check out www.OpenProcessing.org to see some more examples of what people are doing with it.