Using Social Graphs to Visualize Political Factions

21 06 2010

A couple great articles from O’Reilly on using social graphs to visualize data.  Andrew Odewahn revisits and older project in  Visualizing the Senate social graph, revisited – OReilly Radar.  The basic notion is to go through senate voting sessions with each senator as a node.  When there is a pattern of frequently having similar votes a node is drawn.  He does this over an number of different periods giving insight into the political climate.  Definetly watch the video.

Then he goes into how to improve upon the effort.  The second post shows how to code this using Processing for a more interactive social graph analysis.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Advertisements




Mapping TIGER Stimulus Grants with Simile

19 02 2010

Today the U.S. Department of Transportation announced the cities that would be awarded with Transportation Investment Generating Economic Recovery (TIGER) Grants.  This seemed like a great opportunity to apply some visualization tools to a new data set.  The data was manually pulled out of the official PDF announcing the TIGER Grants and put into a Google Spreadsheet.  Then it was quick to together a SIMILE Exhibit that pointed to the data.

You can find the resulting map and table at Blldzr.

—————————-

I create a similar TIGER map using Tableau Public.

http://public.tableausoftware.com/views/TIGER_Grants_by_www_blldzr_com/Sheet1?:embed=yes&:toolbar=yes

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Importing Google Spreadsheet Data into Processing

17 02 2010

It is really powerful to be able to connect your Processing applet to external data sources.  This enable art infused with data and other great mash-ups.  I have done a couple of different projects that connect to Amazon Web Services, YQL and Google Spreadsheets.  The Google solution is perhaps the most flexible, because if the data does not reside in your spreadsheet you can import it via the spreadsheet functions.

Writing the Processing code to connect is pretty straight forward.  You can use functions like loadStrings to reference URLs that have your data.

size(300,300);
background(0);
String items[];
String[] pieces;
PFont  fontA = loadFont("StylisticSFBold-48.vlw");
//load from Google Spreadsheet Url
items=loadStrings("http://spreadsheets.google.com/pub?key=tV-tDRoZvFAdHJ_7fbKxDxw&output=txt");
int itemCount=items.length;
println(itemCount);
for(int a=0;a<itemCount;a++){
println(items[a]);
pieces = split(items[a], '|');
//println(pieces[0]);
//println(pieces[1]);
//println(pieces[2]);
textFont(fontA, 32);
text(pieces[0], 50, 25+a*(height/itemCount));
}

This works really well in the IDE… but when you upload it you will see that it will not work.  The problem is that the Java applet’s security does not allow it to access urls and content that are not on the same server.  If it could there could be viruses that downloaded malicous content to your machine.

Standard Approach – Sign the Applet

The standard option is to sign your applet and create a certificate for the user to approve.  To do this you must make sure you have the Java KeyTool and Jarsigner downloaded.  Then you need to issue the follow commands in ms-dos or the terminal on a Mac:

  • “C:\Program Files\Java\jre6\bin\keytool” -selfcert -keystore pKeyStore -alias yourKeyName
  • “C:\Program Files\Java\jre6\bin\jarsigner” -keystore pKeyStore GoogleImporter2.jar yourKeyName
  • “C:\Program Files\Java\jdk1.6.0_06\bin\jarsigner” -keystore pKeyStore GoogleImporter2.jar yourKeyName

You will need to replace the keytool and jarsigner location with the right ones on your machine, and replace “GoogleImporter2.jar” with your applets name.  Finally make sure to use your own key name.

If all goes well you will have successfully signed your app.  While attempting this for the first time I found the following two links to be very useful:

Alternative Approach – Redirect to a Local File

As an alternative you can use an intermediate script file to bring the external sites data to your server.  This is the approach I took in building my Flickr Kaleidoscope.  To create the same result as above, but without the signing the app I created a small php file that I can reference from Processing:

<?php
$link=$_GET['link'];
$response = file_get_contents($link.'&output=txt');
echo $response
?>

To call a Google spreadsheet from this you simple call a url like the following:

http://www.perlitalabs.com/Google_Importer/Get_Link.php?link=http://spreadsheets.google.com/pub?key=tV-tDRoZvFAdHJ_7fbKxDxw

There is an additional advantage to this approach is that if the data needs to be further processed and “massaged.”  In this case you can use the PHP functions to parse the data, parse xml, create\sort arrays and much more.

Whichever approach is pursued their is a lot of power in connecting Processing with data from the web.  The Processing visuals can be driven from data and have a dynamic element that comes from refreshable data.


Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Flickr Kaleidoscope built with Processing

17 11 2009

I remember getting my TRS-80 when I was about 10.  I flipped through the manual that came with it and was entranced at the kaleidoscope program at the end of the book.  I was amazed to see that a computer could reproduce these patterns and colors in beautiful synchronization.  This is my first real memory of programing and computer generated art.

I decided to revisit that memory and create my own kaleidoscope. It allows you to search Flickr titles and tags to return a set of images. You can then flip through the images and adjust the number of mirrored slices.  If you are interested in how I built it please read on.

Flickr Kaleidoscope Examples

I have been developing with Processing for a couple of years and wanted to see what could be done to make a Kaleidoscope. The technology stack that I came up with was to using Processing to access a dynamic php page that looked up images on Flickr via a REST call to YQL. So let me explain what that looks like…

Let’s start with the PHP:

<?php
$Limit=10;
$request=”http://query.yahooapis.com/v1/public/yql?q=select%20farm%2C%20server%2C%20id%2C%20secret%2C%20title%20from%20flickr.photos.search(0%2C”.$Limit.”)%20where%20text%3D%22″.$Term.”%22%20&format=xml”;
sleep(1);
$response = file_get_contents($request);
$parsed_xml = simplexml_load_string($response);
foreach($parsed_xml->results->photo as $current){ //stopped here need to start inserting
$farm = $current['farm'];
$id= $current['id'];
$secret=$current['secret'];
$server=$current['server'];
$title=$current['title'];
$imageURL = ”http://farm”.$farm.”.static.flickr.com/”.$server.”/”.$id.”_”.$secret.”.jpg”;
echo $title.”|”.$imageURL.”n”;
}
?>

As you can see it is pretty straight forward.  I use YQL (there may be a # of call per day limit on this) to conduct a search based on a term passed to the php page.  I then parse the results into variable that are then used to build urls.  The generated urls are then output as text.
colorful
While the program is running I call this page from processing by:

public void imageLoader(String term){
loading =0;
String[] lines;
lines = loadStrings(“http://www.PerlitaLabs.com/Kaleidoscope/PicturePull.php?term=”+term);
lineCount=lines.length;
for(i=0;i<lineCount;i++){
text(“.”, 50+(3*i), 20);
String[] pieces = split(lines[i], ‘|’);
println(pieces[0]);
println(pieces[1]);
loadedImages[i]= loadImage(pieces[1]);
fileCount++;
a=loadedImages[int(random(0,lineCount))]; }

You can see the code to the entire app at http://www.perlitalabs.com/Kaleidoscope/index.html

And that is pretty much it.  Sounds easy, but it did take some time to get it to work right.  I want to work to add some features around saving images and possibly a gallery, but since I have a working v1 I had better get it out (perfection is the enemy of good enough).

I have posted some of the created images to flickr.  Also worth checking out is the Kaleidoscope Krazy Dad made.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to StumbleUponAdd to TechnoratiAdd to Twitter