New Posts Coming

16 01 2012

Ok, so I have been a bit neglectful over here at Perlita Labs. I got a little distracted with work, life, other projects, etc… but I plan on some more posts this year.

Over the past year I spent a bunch of time working with Drupal and Flash. All of my creations were being put on my  Family Art Studio site where you could use the Flash tools to create art.

My interest is now turning to Android development and I figured I would resurrect Perlita Labs for this work.  Here is the link to my first app: The Legend of Chess.  I will add a post on it as things progress.





Using Social Graphs to Visualize Political Factions

21 06 2010

A couple great articles from O’Reilly on using social graphs to visualize data.  Andrew Odewahn revisits and older project in  Visualizing the Senate social graph, revisited – OReilly Radar.  The basic notion is to go through senate voting sessions with each senator as a node.  When there is a pattern of frequently having similar votes a node is drawn.  He does this over an number of different periods giving insight into the political climate.  Definetly watch the video.

Then he goes into how to improve upon the effort.  The second post shows how to code this using Processing for a more interactive social graph analysis.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Is MS Pivot a New Tool or a New Paradigm

17 03 2010

Microsoft Pivot is a new data exploration\visualization tool that allows rapid interaction with enormous sets of information.  The tool’s capability is impressive, but some may say inevitable with the convergence of large data volumes, powerful databases, and web technologies.  What is just as interesting is that Pivot, and tools like it, will change how we navigate, explore and come to understand information (why data visualization is important).

Screenshots of Microsoft PivotPivot uses the metaphor of a collection.  It seems each item in the collection has attributes that can be used to filter, or to distribute (think bar chart) the items in that collection.  Collection items can also be linked to one another to rapidly navigate to similar items.  Words really do not do this justice and you should look at the Pivot Demo or Gary Flake’s TED Talk on Microsoft Pivot.  This tool moves at the speed of thought and when you follow a hunch you get a response back quickly that lets you continue without interruption.

When a user is able to work with data without their thought process being interrupted the “tool” starts to disappear.  We have all heard the cliché that a car disappears and the driver is one with the road.  This happens to users and analyst working with data as well.

This is not a complete new concept.  Corporate analyst have had Cubes and other Data Warehousing systems provide them with similar capabilities.  In general these have been expensive and developer intensive solutions.  What Pivot and tools like offer is a way to bring this to the masses.

We will all become comfortable immersing ourselves in profession and personal data.  It may be how you navigate your bank transactions or choose a school for your children.  Your employer may expect you to use this skill for your job, or at least to clean out your inbox.

Data exploration with the help of new technologies will become mainstream in ways we cannot imagine today.

Links

Pivot Demo

TED Talk on Microsoft Pivot

Sea Dragon Showcase


Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Mapping TIGER Stimulus Grants with Simile

19 02 2010

Today the U.S. Department of Transportation announced the cities that would be awarded with Transportation Investment Generating Economic Recovery (TIGER) Grants.  This seemed like a great opportunity to apply some visualization tools to a new data set.  The data was manually pulled out of the official PDF announcing the TIGER Grants and put into a Google Spreadsheet.  Then it was quick to together a SIMILE Exhibit that pointed to the data.

You can find the resulting map and table at Blldzr.

—————————-

I create a similar TIGER map using Tableau Public.

http://public.tableausoftware.com/views/TIGER_Grants_by_www_blldzr_com/Sheet1?:embed=yes&:toolbar=yes

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Importing Google Spreadsheet Data into Processing

17 02 2010

It is really powerful to be able to connect your Processing applet to external data sources.  This enable art infused with data and other great mash-ups.  I have done a couple of different projects that connect to Amazon Web Services, YQL and Google Spreadsheets.  The Google solution is perhaps the most flexible, because if the data does not reside in your spreadsheet you can import it via the spreadsheet functions.

Writing the Processing code to connect is pretty straight forward.  You can use functions like loadStrings to reference URLs that have your data.

size(300,300);
background(0);
String items[];
String[] pieces;
PFont  fontA = loadFont("StylisticSFBold-48.vlw");
//load from Google Spreadsheet Url
items=loadStrings("http://spreadsheets.google.com/pub?key=tV-tDRoZvFAdHJ_7fbKxDxw&output=txt");
int itemCount=items.length;
println(itemCount);
for(int a=0;a<itemCount;a++){
println(items[a]);
pieces = split(items[a], '|');
//println(pieces[0]);
//println(pieces[1]);
//println(pieces[2]);
textFont(fontA, 32);
text(pieces[0], 50, 25+a*(height/itemCount));
}

This works really well in the IDE… but when you upload it you will see that it will not work.  The problem is that the Java applet’s security does not allow it to access urls and content that are not on the same server.  If it could there could be viruses that downloaded malicous content to your machine.

Standard Approach – Sign the Applet

The standard option is to sign your applet and create a certificate for the user to approve.  To do this you must make sure you have the Java KeyTool and Jarsigner downloaded.  Then you need to issue the follow commands in ms-dos or the terminal on a Mac:

  • “C:\Program Files\Java\jre6\bin\keytool” -selfcert -keystore pKeyStore -alias yourKeyName
  • “C:\Program Files\Java\jre6\bin\jarsigner” -keystore pKeyStore GoogleImporter2.jar yourKeyName
  • “C:\Program Files\Java\jdk1.6.0_06\bin\jarsigner” -keystore pKeyStore GoogleImporter2.jar yourKeyName

You will need to replace the keytool and jarsigner location with the right ones on your machine, and replace “GoogleImporter2.jar” with your applets name.  Finally make sure to use your own key name.

If all goes well you will have successfully signed your app.  While attempting this for the first time I found the following two links to be very useful:

Alternative Approach – Redirect to a Local File

As an alternative you can use an intermediate script file to bring the external sites data to your server.  This is the approach I took in building my Flickr Kaleidoscope.  To create the same result as above, but without the signing the app I created a small php file that I can reference from Processing:

<?php
$link=$_GET['link'];
$response = file_get_contents($link.'&output=txt');
echo $response
?>

To call a Google spreadsheet from this you simple call a url like the following:

http://www.perlitalabs.com/Google_Importer/Get_Link.php?link=http://spreadsheets.google.com/pub?key=tV-tDRoZvFAdHJ_7fbKxDxw

There is an additional advantage to this approach is that if the data needs to be further processed and “massaged.”  In this case you can use the PHP functions to parse the data, parse xml, create\sort arrays and much more.

Whichever approach is pursued their is a lot of power in connecting Processing with data from the web.  The Processing visuals can be driven from data and have a dynamic element that comes from refreshable data.


Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Book Review: The Visual Miscellaneum

24 01 2010

So I did get the books on my holiday wishlist and have had a great time going through all the amazing thinking on data visualization.  The first one I have completely finished is The Visual Miscellaneum by David McCandless.

This book is a great sources of inspiration and entertainment.  David McCandles shows off that he is able to take any data set and make it into a beautiful infographic or data visualization.  His layouts and color selections make the information really pop.

The Visual Miscellaneum - Rising Sea Levels

Topics cover many areas including science, pop culture, history, music, the internet, food and more.  The variation helps to make the topic of visualization stand out.  By seeing the techniques and visuals applied to different data sets in different areas, it helps the reader realize that data visualization has value in everything from dealing with climate change (page 74) to ordering the right cup of coffee (page 156).The Visual Miscellaneum - Simple Part I

On the down side is that some of the beauty is only skin deep.  I found certain illustrations would have been served better by a simple table or alternative visualization.  There were also a few cases where a typo existed or it seemed a legend must have been left out.

I would still say this book great addition to a coffee table or library.  If you are a data geek you will love to study the visualizations.  If you are not a data geek you will enjoy the interesting facts and comparisons represented by beautiful images.

Pros:

  • Beautiful book with great graphic design, color schemes, etc.
  • A lot of variety in how data can be presented… some are the same old standards and others are really creative new approaches.

Cons:

  • The illustrations are not always an improvement in understanding the data.

I will keep this book close at hand for best practices in making data look awesome and as a source of inspiration when stuck for a way to represent information. Click here to get it from Amazon.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter





Blldzr: What are they Building?

16 01 2010

I have written before about the power of applying mash-up and web 2.0 technologies to government data.  For me there was a bit of put up or shut-up to the whole effort as well.  What could I do to make my community better using my development, analytical and visualization knowledge.  I wanted to find something that I had an interest around and was relevant to my community (Tucson, AZ).

Blldzr

What I came up with was Blldzr.com (pronounced Bull-Dozer).  Blldzr is a wiki that allows members of a community to enter information about development and construction projects in their area.  In a addition to the factual information there are also comments to express opinions about the developments.

Blldzr:  Answering the Question... What are they building?

After being added each development has a page that can be updated and subscribed to.

Blldzr:  A page for each development

Being informed and having conversations about what is being built in your neighborhood is one way to create community.  My hope is that Blldzr can foster more of that.

There is still much to do.  The site needs content, activity, moderators and to be promoted.  If you are interested in any or all of this feel free to pitch in (e.g. add some entries for you city).  If you would just like to be kept up to date on progress feel free to follow Blldzr on Twitter.

I will post more in the future on progress and the technology that was used in building the site.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter