Using Social Graphs to Visualize Political Factions

21 06 2010

A couple great articles from O’Reilly on using social graphs to visualize data.  Andrew Odewahn revisits and older project in  Visualizing the Senate social graph, revisited – OReilly Radar.  The basic notion is to go through senate voting sessions with each senator as a node.  When there is a pattern of frequently having similar votes a node is drawn.  He does this over an number of different periods giving insight into the political climate.  Definetly watch the video.

Then he goes into how to improve upon the effort.  The second post shows how to code this using Processing for a more interactive social graph analysis.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Mapping TIGER Stimulus Grants with Simile

19 02 2010

Today the U.S. Department of Transportation announced the cities that would be awarded with Transportation Investment Generating Economic Recovery (TIGER) Grants.  This seemed like a great opportunity to apply some visualization tools to a new data set.  The data was manually pulled out of the official PDF announcing the TIGER Grants and put into a Google Spreadsheet.  Then it was quick to together a SIMILE Exhibit that pointed to the data.

You can find the resulting map and table at Blldzr.

—————————-

I create a similar TIGER map using Tableau Public.

http://public.tableausoftware.com/views/TIGER_Grants_by_www_blldzr_com/Sheet1?:embed=yes&:toolbar=yes

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Blldzr: What are they Building?

16 01 2010

I have written before about the power of applying mash-up and web 2.0 technologies to government data.  For me there was a bit of put up or shut-up to the whole effort as well.  What could I do to make my community better using my development, analytical and visualization knowledge.  I wanted to find something that I had an interest around and was relevant to my community (Tucson, AZ).

Blldzr

What I came up with was Blldzr.com (pronounced Bull-Dozer).  Blldzr is a wiki that allows members of a community to enter information about development and construction projects in their area.  In a addition to the factual information there are also comments to express opinions about the developments.

Blldzr:  Answering the Question... What are they building?

After being added each development has a page that can be updated and subscribed to.

Blldzr:  A page for each development

Being informed and having conversations about what is being built in your neighborhood is one way to create community.  My hope is that Blldzr can foster more of that.

There is still much to do.  The site needs content, activity, moderators and to be promoted.  If you are interested in any or all of this feel free to pitch in (e.g. add some entries for you city).  If you would just like to be kept up to date on progress feel free to follow Blldzr on Twitter.

I will post more in the future on progress and the technology that was used in building the site.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Interactive Resume using Simile Exhibit

11 01 2010

Simile Exhibit is a JavaScript framework that enable a data sets to be visualize, sliced and embedded into any web page.  It is fun and easy to quickly see your data brought to life with all of Exhibit’s features.  Some of the included views are a basic table, timeline, map, gallery and more.  It is a great solution for data with less that 1,000 rows.  The data can be integrated in a number of ways, including Google Spreadsheets, JSon files, etc.

In addition to plugging the same data set into multiple visualizations, Exhibit provides Facet filters.  These automatically provide multiple choice filter list that dynamically modify the visualizations.

With all of these features and easy data integration I plugged it into my resume which I set up in a Google Spreadsheet.  First I came up with a basic table view (I applied a little extra CSS):

Garys Resume in an Exhibit Table

Then I added the map view to see all of the locations that I have worked at:

Garys Resume as Exhibit Map

And lastly I added a timeline view:

Garys Resume in an Exhibit TimelineAs you will notice on each of the pages there are a list of filterable facet list on the left and right side (this is configurable as well).

For more examples check out this Exhibit Examples 1Exhibit Examples 2, and Exhibit Examples 3.

So far I have only identified 3 real downsides to the software.

  1. First since the content is published through Javascript you cannot tell what the page is about by looking at its source code.  This will impact some spiders and search engines, when crawling, who cannot call the Javascirpt.  There are work-arounds as the content can be output as html, xml, tab delimited, json, etc, but the dynamic data and filtering is lost.
  2. One of the reasons the software is so easy to deploy is that the server side of Simile is hosted at MIT.  So there is no need to host the library on your site.  This can be limiting since you become dependent on MIT’s uptime and it may not be possible to deploy on an intranet, or with sensitive internal data.  The good news is that is a way to set up your own server, so it can be done, but looks to be a bit of work.
  3. Exhibit is not for large data sets.  It works well for small list that can be managed in a text file or Google Spreadsheet, but it is not something you would want to plug into a data warehouse.

Overall, Exhibit is a great way to interact and visualize data and allows you to be up and running quickly.  Check out my resume, some examples and go build your own.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Data Activism: Geeks for Good

21 12 2009

Data Activism is a new an emerging trend with designers, developers and artist utilizing technology to help society understand and address the issues of our times.

The new U.S. Administration has established initiatives to make government data available to the public.  This is converging with the growing number of developers and designers that are familiar with mash-up’s and data visualization technologies.  The convergence of these trends create the ability for developers, designers and artist to create apps, sites and art that educate and provide transparency to the government.

Creating Government Transparency

The Open Government Directive is a memo sent by the White House on December 8th, 2009 directs federal agencies to:

  1. Publish Government Information Online
  2. Improve the Quality of Government Information
  3. Create and Institutionalize a Culture of Open Government
  4. Create an Enabling Policy Framework for Open Government

Each of these are follow with specific actions that must start to show results in the next couple of months.  The main hub for this information will be Data.gov, providing Federal data sets, and links to State and local efforts as well.
Data.gov Screenshot
This is amazing progress for the government that will bring both innovation and efficiency.  Possibilities will be opened up as great designers get creative on representing the governments data.  The government agencies will be pressured to improve as their performance becomes public knowledge.  All of this is a win for Americans with more services, useful information and government efficiency.

Everyone can participate by using the data, giving feedback, building visualizations, building reports, etc.  You can get involved by yourself, or join a team of like minded individuals.  What are you waiting for?

Resources:

Examples:

If you have other resources or examples you recommend, add them as comments and I will update this article.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Flickr Kaleidoscope built with Processing

17 11 2009

I remember getting my TRS-80 when I was about 10.  I flipped through the manual that came with it and was entranced at the kaleidoscope program at the end of the book.  I was amazed to see that a computer could reproduce these patterns and colors in beautiful synchronization.  This is my first real memory of programing and computer generated art.

I decided to revisit that memory and create my own kaleidoscope. It allows you to search Flickr titles and tags to return a set of images. You can then flip through the images and adjust the number of mirrored slices.  If you are interested in how I built it please read on.

Flickr Kaleidoscope Examples

I have been developing with Processing for a couple of years and wanted to see what could be done to make a Kaleidoscope. The technology stack that I came up with was to using Processing to access a dynamic php page that looked up images on Flickr via a REST call to YQL. So let me explain what that looks like…

Let’s start with the PHP:

<?php
$Limit=10;
$request=”http://query.yahooapis.com/v1/public/yql?q=select%20farm%2C%20server%2C%20id%2C%20secret%2C%20title%20from%20flickr.photos.search(0%2C”.$Limit.”)%20where%20text%3D%22″.$Term.”%22%20&format=xml”;
sleep(1);
$response = file_get_contents($request);
$parsed_xml = simplexml_load_string($response);
foreach($parsed_xml->results->photo as $current){ //stopped here need to start inserting
$farm = $current['farm'];
$id= $current['id'];
$secret=$current['secret'];
$server=$current['server'];
$title=$current['title'];
$imageURL = ”http://farm”.$farm.”.static.flickr.com/”.$server.”/”.$id.”_”.$secret.”.jpg”;
echo $title.”|”.$imageURL.”n”;
}
?>

As you can see it is pretty straight forward.  I use YQL (there may be a # of call per day limit on this) to conduct a search based on a term passed to the php page.  I then parse the results into variable that are then used to build urls.  The generated urls are then output as text.
colorful
While the program is running I call this page from processing by:

public void imageLoader(String term){
loading =0;
String[] lines;
lines = loadStrings(“http://www.PerlitaLabs.com/Kaleidoscope/PicturePull.php?term=”+term);
lineCount=lines.length;
for(i=0;i<lineCount;i++){
text(“.”, 50+(3*i), 20);
String[] pieces = split(lines[i], ‘|’);
println(pieces[0]);
println(pieces[1]);
loadedImages[i]= loadImage(pieces[1]);
fileCount++;
a=loadedImages[int(random(0,lineCount))]; }

You can see the code to the entire app at http://www.perlitalabs.com/Kaleidoscope/index.html

And that is pretty much it.  Sounds easy, but it did take some time to get it to work right.  I want to work to add some features around saving images and possibly a gallery, but since I have a working v1 I had better get it out (perfection is the enemy of good enough).

I have posted some of the created images to flickr.  Also worth checking out is the Kaleidoscope Krazy Dad made.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to StumbleUponAdd to TechnoratiAdd to Twitter