Using Social Graphs to Visualize Political Factions

21 06 2010

A couple great articles from O’Reilly on using social graphs to visualize data.  Andrew Odewahn revisits and older project in  Visualizing the Senate social graph, revisited – OReilly Radar.  The basic notion is to go through senate voting sessions with each senator as a node.  When there is a pattern of frequently having similar votes a node is drawn.  He does this over an number of different periods giving insight into the political climate.  Definetly watch the video.

Then he goes into how to improve upon the effort.  The second post shows how to code this using Processing for a more interactive social graph analysis.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Is MS Pivot a New Tool or a New Paradigm

17 03 2010

Microsoft Pivot is a new data exploration\visualization tool that allows rapid interaction with enormous sets of information.  The tool’s capability is impressive, but some may say inevitable with the convergence of large data volumes, powerful databases, and web technologies.  What is just as interesting is that Pivot, and tools like it, will change how we navigate, explore and come to understand information (why data visualization is important).

Screenshots of Microsoft PivotPivot uses the metaphor of a collection.  It seems each item in the collection has attributes that can be used to filter, or to distribute (think bar chart) the items in that collection.  Collection items can also be linked to one another to rapidly navigate to similar items.  Words really do not do this justice and you should look at the Pivot Demo or Gary Flake’s TED Talk on Microsoft Pivot.  This tool moves at the speed of thought and when you follow a hunch you get a response back quickly that lets you continue without interruption.

When a user is able to work with data without their thought process being interrupted the “tool” starts to disappear.  We have all heard the cliché that a car disappears and the driver is one with the road.  This happens to users and analyst working with data as well.

This is not a complete new concept.  Corporate analyst have had Cubes and other Data Warehousing systems provide them with similar capabilities.  In general these have been expensive and developer intensive solutions.  What Pivot and tools like offer is a way to bring this to the masses.

We will all become comfortable immersing ourselves in profession and personal data.  It may be how you navigate your bank transactions or choose a school for your children.  Your employer may expect you to use this skill for your job, or at least to clean out your inbox.

Data exploration with the help of new technologies will become mainstream in ways we cannot imagine today.


Pivot Demo

TED Talk on Microsoft Pivot

Sea Dragon Showcase

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Mapping TIGER Stimulus Grants with Simile

19 02 2010

Today the U.S. Department of Transportation announced the cities that would be awarded with Transportation Investment Generating Economic Recovery (TIGER) Grants.  This seemed like a great opportunity to apply some visualization tools to a new data set.  The data was manually pulled out of the official PDF announcing the TIGER Grants and put into a Google Spreadsheet.  Then it was quick to together a SIMILE Exhibit that pointed to the data.

You can find the resulting map and table at Blldzr.


I create a similar TIGER map using Tableau Public.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Book Review: The Visual Miscellaneum

24 01 2010

So I did get the books on my holiday wishlist and have had a great time going through all the amazing thinking on data visualization.  The first one I have completely finished is The Visual Miscellaneum by David McCandless.

This book is a great sources of inspiration and entertainment.  David McCandles shows off that he is able to take any data set and make it into a beautiful infographic or data visualization.  His layouts and color selections make the information really pop.

The Visual Miscellaneum - Rising Sea Levels

Topics cover many areas including science, pop culture, history, music, the internet, food and more.  The variation helps to make the topic of visualization stand out.  By seeing the techniques and visuals applied to different data sets in different areas, it helps the reader realize that data visualization has value in everything from dealing with climate change (page 74) to ordering the right cup of coffee (page 156).The Visual Miscellaneum - Simple Part I

On the down side is that some of the beauty is only skin deep.  I found certain illustrations would have been served better by a simple table or alternative visualization.  There were also a few cases where a typo existed or it seemed a legend must have been left out.

I would still say this book great addition to a coffee table or library.  If you are a data geek you will love to study the visualizations.  If you are not a data geek you will enjoy the interesting facts and comparisons represented by beautiful images.


  • Beautiful book with great graphic design, color schemes, etc.
  • A lot of variety in how data can be presented… some are the same old standards and others are really creative new approaches.


  • The illustrations are not always an improvement in understanding the data.

I will keep this book close at hand for best practices in making data look awesome and as a source of inspiration when stuck for a way to represent information. Click here to get it from Amazon.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Interactive Resume using Simile Exhibit

11 01 2010

Simile Exhibit is a JavaScript framework that enable a data sets to be visualize, sliced and embedded into any web page.  It is fun and easy to quickly see your data brought to life with all of Exhibit’s features.  Some of the included views are a basic table, timeline, map, gallery and more.  It is a great solution for data with less that 1,000 rows.  The data can be integrated in a number of ways, including Google Spreadsheets, JSon files, etc.

In addition to plugging the same data set into multiple visualizations, Exhibit provides Facet filters.  These automatically provide multiple choice filter list that dynamically modify the visualizations.

With all of these features and easy data integration I plugged it into my resume which I set up in a Google Spreadsheet.  First I came up with a basic table view (I applied a little extra CSS):

Garys Resume in an Exhibit Table

Then I added the map view to see all of the locations that I have worked at:

Garys Resume as Exhibit Map

And lastly I added a timeline view:

Garys Resume in an Exhibit TimelineAs you will notice on each of the pages there are a list of filterable facet list on the left and right side (this is configurable as well).

For more examples check out this Exhibit Examples 1Exhibit Examples 2, and Exhibit Examples 3.

So far I have only identified 3 real downsides to the software.

  1. First since the content is published through Javascript you cannot tell what the page is about by looking at its source code.  This will impact some spiders and search engines, when crawling, who cannot call the Javascirpt.  There are work-arounds as the content can be output as html, xml, tab delimited, json, etc, but the dynamic data and filtering is lost.
  2. One of the reasons the software is so easy to deploy is that the server side of Simile is hosted at MIT.  So there is no need to host the library on your site.  This can be limiting since you become dependent on MIT’s uptime and it may not be possible to deploy on an intranet, or with sensitive internal data.  The good news is that is a way to set up your own server, so it can be done, but looks to be a bit of work.
  3. Exhibit is not for large data sets.  It works well for small list that can be managed in a text file or Google Spreadsheet, but it is not something you would want to plug into a data warehouse.

Overall, Exhibit is a great way to interact and visualize data and allows you to be up and running quickly.  Check out my resume, some examples and go build your own.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Data Activism: Geeks for Good

21 12 2009

Data Activism is a new an emerging trend with designers, developers and artist utilizing technology to help society understand and address the issues of our times.

The new U.S. Administration has established initiatives to make government data available to the public.  This is converging with the growing number of developers and designers that are familiar with mash-up’s and data visualization technologies.  The convergence of these trends create the ability for developers, designers and artist to create apps, sites and art that educate and provide transparency to the government.

Creating Government Transparency

The Open Government Directive is a memo sent by the White House on December 8th, 2009 directs federal agencies to:

  1. Publish Government Information Online
  2. Improve the Quality of Government Information
  3. Create and Institutionalize a Culture of Open Government
  4. Create an Enabling Policy Framework for Open Government

Each of these are follow with specific actions that must start to show results in the next couple of months.  The main hub for this information will be, providing Federal data sets, and links to State and local efforts as well. Screenshot
This is amazing progress for the government that will bring both innovation and efficiency.  Possibilities will be opened up as great designers get creative on representing the governments data.  The government agencies will be pressured to improve as their performance becomes public knowledge.  All of this is a win for Americans with more services, useful information and government efficiency.

Everyone can participate by using the data, giving feedback, building visualizations, building reports, etc.  You can get involved by yourself, or join a team of like minded individuals.  What are you waiting for?



If you have other resources or examples you recommend, add them as comments and I will update this article.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Why Improving Data Visualization and Interaction Matters

15 12 2009

There is growing interest in visualizing and interacting with data for good reason.  Improving the speed and intuitiveness of data representation is required as people interact with more and more information.
Data Overload
A UC San Diego study reported on by the Huffington Post article states that “The researchers calculated that we consumed, on average, 10, 845 trillion words in 2008, or about 100,000 words per American, per day.”  And it is not just words, there are images, audio, games and more giving the average American consumption of 34 gigabyte per person, per day.

So you will have to note the word is “consume,” but one may want to argue that only some consumption really has value.  Meaning just because it was streamed in a person’s direction was meaningful information transfered from the broadcast media to the person.  And that answer is that very little of that data really becomes information and knowledge for the person who experiences it.

This is where the study and practice of Data Visualization and Interaction becomes important.  How can developers and designers make there information the easiest, quickest and most intuitive to comprehend?  Those that are good and great at reducing the interface and letting people work with the data will be the winners.

A number of web sites and applications are investing and exploring how data visualization can help organize their content.    Digg Labs has 6 different visualizations to navigate and uncover interesting news and content.

Digg 365 has a number of visualizations they use to help shoppers find what they are looking for… even if the shoppers don’t know it (The mind behind a lot of the Etsy visualizations is Jared Tarbell who is know for some great computer generate art using flash and processing).Etsy Shop by ColorAnother site known for its great traditional graphs is (disclosure: I work for Intuit which recently acquired Mint).  Mint is excelling at helping people understand there finances and budgeting trends.

While the above examples are diverse there is a common theme of helping the users of the sites better navigate content and data.

The capacity for humans to consume information cannot increase as rapidly as our systems are making it available to us.  People will focus on the information that is important and easy to consume.  Good data design can add tremendous value to sites and business that get it, making them more usable and engaging to their users.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Microsoft Vedea comes after Processing

4 12 2009

Microsoft Visualization Language The Vedea Project Create. See. Understand.

“Vedea is a prototype of a new experimental language for creating interactive infographics, data visualizations and computational art.”

Microsoft has noticed what the guys at MIT have been up to with Processing and is preparing to launch Vedea, which they describe as “a new visualization language.”  Just knowing Microsoft many of us may have a little skepticism… not that the tool won’t be good or exciting, but will it be open, and free?  Being a strong believer in competition this entry can only make the space better, with more minds working to improve the creation of data visualization.

On the Microsoft Research Blog there is a write up with some detail, including some code snippets.  In reviewing the code it looks like the language will offer some predefined GUI Components and Data Visualizations built in.  This could actually be very valuable including these in the core (as long as it is extensible with external libraries).  Processing today is open and there are libraries that cover these areas, but integration can be  cumbersome.

It looks like launch is planned for early 2010.

Update: 12/5/09

Here is a blog post detailing a  demo of Vedea with the data, code and an image of the data visualization it created.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

My Data Visualization Holiday Wish List

1 12 2009

As I have been writing this blog I have come across a number of great minds in the Data Visualization space and some of their books.  My Amazon wishlist has actually gotten quite long, but I narrowed it down to 3 books I want to add to my Data Visualization library:

Beautiful Data by Jeff Hammerbacher and Toby Segaran has caught my eye for a while, but it was not until I saw it in Barnes and Noble that it made my list.  There are 39 contributors that share some of their greatest data visualization with a large number of color pages showcasing their work.  This is the kind of book that can get you out of a mental block and inspire you to create.

Running the Numbers: An American Self-portrait by Chris Jordan made my list because of his Ted talk.  If you have not seen Chris Jordan pictures some shocking stats please do.  It is an inspiring video demonstrating how Art and Visualization can help the human mind make sense of the biggest issues facing society today.  I came aware motivated to make the world a better place and inspired that data visualization can play a role in creating change.  I am really excited to all of the “portraits” that are included in the book.

The Visual Miscellaneum by David McCandless proposes to be a collection of great data visualisations, diagrams and infographics.  I have seen the book promoted on the well designed site of David’s Information is Beautiful.   You can just tell from the site that this book will be a good one.  I recently used one of the infographics he contributed to for a political conversation with my wife, regarding the traits of the left vs the right.

So there are the 3… and yes, my order is in and they are on their way.  Lots of good reading and sources for inspiration.

Features of 13 Data Visualization and Chart Libraries

30 11 2009

The great news is that there are a large number of terrific libraries and tool kits available for visualizing data on the web.  Some of the best of them are even free.  In my post providing a list of data visualization tools, I briefly covered about 30 of these tools, but there was something more to do here.  Each of these libraries have different charts, graphs and diagrams that they focus on and the implementation and language often differ.  To provide something for myself and the community to keep track I have prepared a cheat sheet (grid).

Included are the broader libraries that cover a lot of visualization types.  I may not have done justice to all of them, as the type of charts are those that there were demos for.  There may be cases where other types of charts are possible with a little work.  Also provided the language the library is designed to work with\built with.  There maybe a few cases where implementation can actually be done in a number of different languages.  For example the Open Flash Charts can be implemented in PHP, Perl, Python, Ruby, .NET, Google Web Toolkit and JAVA.

For a larger clickable image go to Perlita Labs.

Comparison of Graph and Chart Libraries in PHP, Java, ActionScript and AJAX

Thumbnail of Comparison Grid for Data Visualization Tools


Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter