Using Social Graphs to Visualize Political Factions

21 06 2010

A couple great articles from O’Reilly on using social graphs to visualize data.  Andrew Odewahn revisits and older project in  Visualizing the Senate social graph, revisited – OReilly Radar.  The basic notion is to go through senate voting sessions with each senator as a node.  When there is a pattern of frequently having similar votes a node is drawn.  He does this over an number of different periods giving insight into the political climate.  Definetly watch the video.

Then he goes into how to improve upon the effort.  The second post shows how to code this using Processing for a more interactive social graph analysis.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Advertisements




Importing Google Spreadsheet Data into Processing

17 02 2010

It is really powerful to be able to connect your Processing applet to external data sources.  This enable art infused with data and other great mash-ups.  I have done a couple of different projects that connect to Amazon Web Services, YQL and Google Spreadsheets.  The Google solution is perhaps the most flexible, because if the data does not reside in your spreadsheet you can import it via the spreadsheet functions.

Writing the Processing code to connect is pretty straight forward.  You can use functions like loadStrings to reference URLs that have your data.

size(300,300);
background(0);
String items[];
String[] pieces;
PFont  fontA = loadFont("StylisticSFBold-48.vlw");
//load from Google Spreadsheet Url
items=loadStrings("http://spreadsheets.google.com/pub?key=tV-tDRoZvFAdHJ_7fbKxDxw&output=txt");
int itemCount=items.length;
println(itemCount);
for(int a=0;a<itemCount;a++){
println(items[a]);
pieces = split(items[a], '|');
//println(pieces[0]);
//println(pieces[1]);
//println(pieces[2]);
textFont(fontA, 32);
text(pieces[0], 50, 25+a*(height/itemCount));
}

This works really well in the IDE… but when you upload it you will see that it will not work.  The problem is that the Java applet’s security does not allow it to access urls and content that are not on the same server.  If it could there could be viruses that downloaded malicous content to your machine.

Standard Approach – Sign the Applet

The standard option is to sign your applet and create a certificate for the user to approve.  To do this you must make sure you have the Java KeyTool and Jarsigner downloaded.  Then you need to issue the follow commands in ms-dos or the terminal on a Mac:

  • “C:\Program Files\Java\jre6\bin\keytool” -selfcert -keystore pKeyStore -alias yourKeyName
  • “C:\Program Files\Java\jre6\bin\jarsigner” -keystore pKeyStore GoogleImporter2.jar yourKeyName
  • “C:\Program Files\Java\jdk1.6.0_06\bin\jarsigner” -keystore pKeyStore GoogleImporter2.jar yourKeyName

You will need to replace the keytool and jarsigner location with the right ones on your machine, and replace “GoogleImporter2.jar” with your applets name.  Finally make sure to use your own key name.

If all goes well you will have successfully signed your app.  While attempting this for the first time I found the following two links to be very useful:

Alternative Approach – Redirect to a Local File

As an alternative you can use an intermediate script file to bring the external sites data to your server.  This is the approach I took in building my Flickr Kaleidoscope.  To create the same result as above, but without the signing the app I created a small php file that I can reference from Processing:

<?php
$link=$_GET['link'];
$response = file_get_contents($link.'&output=txt');
echo $response
?>

To call a Google spreadsheet from this you simple call a url like the following:

http://www.perlitalabs.com/Google_Importer/Get_Link.php?link=http://spreadsheets.google.com/pub?key=tV-tDRoZvFAdHJ_7fbKxDxw

There is an additional advantage to this approach is that if the data needs to be further processed and “massaged.”  In this case you can use the PHP functions to parse the data, parse xml, create\sort arrays and much more.

Whichever approach is pursued their is a lot of power in connecting Processing with data from the web.  The Processing visuals can be driven from data and have a dynamic element that comes from refreshable data.


Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Microsoft Vedea comes after Processing

4 12 2009

Microsoft Visualization Language The Vedea Project Create. See. Understand.

“Vedea is a prototype of a new experimental language for creating interactive infographics, data visualizations and computational art.”

Microsoft has noticed what the guys at MIT have been up to with Processing and is preparing to launch Vedea, which they describe as “a new visualization language.”  Just knowing Microsoft many of us may have a little skepticism… not that the tool won’t be good or exciting, but will it be open, and free?  Being a strong believer in competition this entry can only make the space better, with more minds working to improve the creation of data visualization.

On the Microsoft Research Blog there is a write up with some detail, including some code snippets.  In reviewing the code it looks like the language will offer some predefined GUI Components and Data Visualizations built in.  This could actually be very valuable including these in the core (as long as it is extensible with external libraries).  Processing today is open and there are libraries that cover these areas, but integration can be  cumbersome.

It looks like launch is planned for early 2010.

Update: 12/5/09

Here is a blog post detailing a  demo of Vedea with the data, code and an image of the data visualization it created.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Flickr Kaleidoscope built with Processing

17 11 2009

I remember getting my TRS-80 when I was about 10.  I flipped through the manual that came with it and was entranced at the kaleidoscope program at the end of the book.  I was amazed to see that a computer could reproduce these patterns and colors in beautiful synchronization.  This is my first real memory of programing and computer generated art.

I decided to revisit that memory and create my own kaleidoscope. It allows you to search Flickr titles and tags to return a set of images. You can then flip through the images and adjust the number of mirrored slices.  If you are interested in how I built it please read on.

Flickr Kaleidoscope Examples

I have been developing with Processing for a couple of years and wanted to see what could be done to make a Kaleidoscope. The technology stack that I came up with was to using Processing to access a dynamic php page that looked up images on Flickr via a REST call to YQL. So let me explain what that looks like…

Let’s start with the PHP:

<?php
$Limit=10;
$request=”http://query.yahooapis.com/v1/public/yql?q=select%20farm%2C%20server%2C%20id%2C%20secret%2C%20title%20from%20flickr.photos.search(0%2C”.$Limit.”)%20where%20text%3D%22″.$Term.”%22%20&format=xml”;
sleep(1);
$response = file_get_contents($request);
$parsed_xml = simplexml_load_string($response);
foreach($parsed_xml->results->photo as $current){ //stopped here need to start inserting
$farm = $current['farm'];
$id= $current['id'];
$secret=$current['secret'];
$server=$current['server'];
$title=$current['title'];
$imageURL = ”http://farm”.$farm.”.static.flickr.com/”.$server.”/”.$id.”_”.$secret.”.jpg”;
echo $title.”|”.$imageURL.”n”;
}
?>

As you can see it is pretty straight forward.  I use YQL (there may be a # of call per day limit on this) to conduct a search based on a term passed to the php page.  I then parse the results into variable that are then used to build urls.  The generated urls are then output as text.
colorful
While the program is running I call this page from processing by:

public void imageLoader(String term){
loading =0;
String[] lines;
lines = loadStrings(“http://www.PerlitaLabs.com/Kaleidoscope/PicturePull.php?term=”+term);
lineCount=lines.length;
for(i=0;i<lineCount;i++){
text(“.”, 50+(3*i), 20);
String[] pieces = split(lines[i], ‘|’);
println(pieces[0]);
println(pieces[1]);
loadedImages[i]= loadImage(pieces[1]);
fileCount++;
a=loadedImages[int(random(0,lineCount))]; }

You can see the code to the entire app at http://www.perlitalabs.com/Kaleidoscope/index.html

And that is pretty much it.  Sounds easy, but it did take some time to get it to work right.  I want to work to add some features around saving images and possibly a gallery, but since I have a working v1 I had better get it out (perfection is the enemy of good enough).

I have posted some of the created images to flickr.  Also worth checking out is the Kaleidoscope Krazy Dad made.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Discovering Processing.js and All the Amazing New Possibilities

25 05 2009

 

Screen shot of HasCanvas

Screen shot of HasCanvas

So from some of my earlier post it is clear that I am a fan of the Processing IDE for create artistic programs and applets.  I have also been struggling with finding the right language (Javascript, Actionscript, ?!?!) to build out some new ideas with for the web.  I think my prays have been answered.  John Risig has ported Processing over to Javascript, call his creation Processing.js.  This means that “most” of the cool things that you could do in Processing, you can now do on the web without a java applet and much better integration with html. 

 

The biggest down side  I see at this point is limited and inconsistent browser support for the javascript canvas required for Processing.js to work.  So if your project is for commercial use it may not meet compatibility standards.  Of course Internet Explore is the least compatible.  Firefox is great and Chrome seems to work pretty good.

Processing.js has been around for a little over a year… it came out in May of 2008, but it really seems to be getting some traction.  And I just discovered it this month, so it is new to me.  Below are some resouces that I have found helpful:

The site from the John Resig: http://ejohn.org/blog/processingjs/

A blog entry from John with some ideas on how to get started and some reflecting: http://ejohn.org/blog/processingjs-aftermath/

A blog post with some good examples: http://felipe.wordpress.com/2008/05/09/processingjs/

An online Processing.JS IDE:  http://obsessing.org

An online Processing.JS game by Casey Reas: http://reas.com/twitch/

A good post on getting started with Processing.JS: http://lethain.com/entry/2008/may/10/getting-started-with-processing-js/

This is an online Processing.JS IDE and Gallery… super slick: http://www.hascanvas.com/

Using Processing.JS for sparklines: http://willarson.com/code/sparklines/sparklines.html#simple7

A lot of great examples: http://www.hyper-metrix.com/processing-js/docs/index.php

I am sure there is going to be a ton of cool stuff that can be made with this…

  • The I think some charts and graphs would be great (again no text rendering for now).  Have you seen Ben Fry’s book on data visualization:  Data Visualization by Ben Fry . A lot of the examples in this book should be portable.
  • Of course there is the art of processing
  • Games anyone?

Processing tool for creation and enabling it with Javascript would seem to open up possibilities to new audiences and new uses (I particularly like the sketch in the background of Robert O’Rourke Blog).

As you can tell I am super excited by this… my enthusiasm is only tempered by the fact that IE does not support the JavaScript Canvas object that is required to render.  😦





The Kindle Difference

22 04 2009

So I just read an article regarding the cost to build a Kindle.  I won’t give it away, but it also brought up the key difference of a Kindle vs other e-readers (or books for that matter).  The difference is the Whispernet wireless service that allows you to get content on your device wirelessly.

I cannot stress enought that this difference changes everything.  If you are thinking about a Kindle you have to understand this feature!  Basically for no monthly fee you have access to the Internet and the Amazon store (and yes other websites).  It is terrific being able to think about buying a book or magazine and then have it appear.  Instant gratification!  

I am also finding other ways to utilize the service.  For a $.10 fee you can email a pdf to Amazon and they will convert it to their format and send it to your Kindle.  I am starting to use this quite a bit when someone sends me something at work that I do not want to stare at my computer screen and read.  This is a great way to get through any “white paper” reading for your job.





What is Processing?

21 04 2009

If you have not see or heard of Processing the IDE for art using Java you should really check out Processing.org.  Basically a couple of really smart guys from MIT came up with a way to make Java programing much easier for artist.  Processing just reached 1.0 and it is a ton of fun for the technically challenged but artistically inclined.  Check out www.OpenProcessing.org to see some more examples of what people are doing with it.