Using Social Graphs to Visualize Political Factions

21 06 2010

A couple great articles from O’Reilly on using social graphs to visualize data.  Andrew Odewahn revisits and older project in  Visualizing the Senate social graph, revisited – OReilly Radar.  The basic notion is to go through senate voting sessions with each senator as a node.  When there is a pattern of frequently having similar votes a node is drawn.  He does this over an number of different periods giving insight into the political climate.  Definetly watch the video.

Then he goes into how to improve upon the effort.  The second post shows how to code this using Processing for a more interactive social graph analysis.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Is MS Pivot a New Tool or a New Paradigm

17 03 2010

Microsoft Pivot is a new data exploration\visualization tool that allows rapid interaction with enormous sets of information.  The tool’s capability is impressive, but some may say inevitable with the convergence of large data volumes, powerful databases, and web technologies.  What is just as interesting is that Pivot, and tools like it, will change how we navigate, explore and come to understand information (why data visualization is important).

Screenshots of Microsoft PivotPivot uses the metaphor of a collection.  It seems each item in the collection has attributes that can be used to filter, or to distribute (think bar chart) the items in that collection.  Collection items can also be linked to one another to rapidly navigate to similar items.  Words really do not do this justice and you should look at the Pivot Demo or Gary Flake’s TED Talk on Microsoft Pivot.  This tool moves at the speed of thought and when you follow a hunch you get a response back quickly that lets you continue without interruption.

When a user is able to work with data without their thought process being interrupted the “tool” starts to disappear.  We have all heard the cliché that a car disappears and the driver is one with the road.  This happens to users and analyst working with data as well.

This is not a complete new concept.  Corporate analyst have had Cubes and other Data Warehousing systems provide them with similar capabilities.  In general these have been expensive and developer intensive solutions.  What Pivot and tools like offer is a way to bring this to the masses.

We will all become comfortable immersing ourselves in profession and personal data.  It may be how you navigate your bank transactions or choose a school for your children.  Your employer may expect you to use this skill for your job, or at least to clean out your inbox.

Data exploration with the help of new technologies will become mainstream in ways we cannot imagine today.


Pivot Demo

TED Talk on Microsoft Pivot

Sea Dragon Showcase

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Book Review: The Visual Miscellaneum

24 01 2010

So I did get the books on my holiday wishlist and have had a great time going through all the amazing thinking on data visualization.  The first one I have completely finished is The Visual Miscellaneum by David McCandless.

This book is a great sources of inspiration and entertainment.  David McCandles shows off that he is able to take any data set and make it into a beautiful infographic or data visualization.  His layouts and color selections make the information really pop.

The Visual Miscellaneum - Rising Sea Levels

Topics cover many areas including science, pop culture, history, music, the internet, food and more.  The variation helps to make the topic of visualization stand out.  By seeing the techniques and visuals applied to different data sets in different areas, it helps the reader realize that data visualization has value in everything from dealing with climate change (page 74) to ordering the right cup of coffee (page 156).The Visual Miscellaneum - Simple Part I

On the down side is that some of the beauty is only skin deep.  I found certain illustrations would have been served better by a simple table or alternative visualization.  There were also a few cases where a typo existed or it seemed a legend must have been left out.

I would still say this book great addition to a coffee table or library.  If you are a data geek you will love to study the visualizations.  If you are not a data geek you will enjoy the interesting facts and comparisons represented by beautiful images.


  • Beautiful book with great graphic design, color schemes, etc.
  • A lot of variety in how data can be presented… some are the same old standards and others are really creative new approaches.


  • The illustrations are not always an improvement in understanding the data.

I will keep this book close at hand for best practices in making data look awesome and as a source of inspiration when stuck for a way to represent information. Click here to get it from Amazon.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Blldzr: What are they Building?

16 01 2010

I have written before about the power of applying mash-up and web 2.0 technologies to government data.  For me there was a bit of put up or shut-up to the whole effort as well.  What could I do to make my community better using my development, analytical and visualization knowledge.  I wanted to find something that I had an interest around and was relevant to my community (Tucson, AZ).


What I came up with was (pronounced Bull-Dozer).  Blldzr is a wiki that allows members of a community to enter information about development and construction projects in their area.  In a addition to the factual information there are also comments to express opinions about the developments.

Blldzr:  Answering the Question... What are they building?

After being added each development has a page that can be updated and subscribed to.

Blldzr:  A page for each development

Being informed and having conversations about what is being built in your neighborhood is one way to create community.  My hope is that Blldzr can foster more of that.

There is still much to do.  The site needs content, activity, moderators and to be promoted.  If you are interested in any or all of this feel free to pitch in (e.g. add some entries for you city).  If you would just like to be kept up to date on progress feel free to follow Blldzr on Twitter.

I will post more in the future on progress and the technology that was used in building the site.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Interactive Resume using Simile Exhibit

11 01 2010

Simile Exhibit is a JavaScript framework that enable a data sets to be visualize, sliced and embedded into any web page.  It is fun and easy to quickly see your data brought to life with all of Exhibit’s features.  Some of the included views are a basic table, timeline, map, gallery and more.  It is a great solution for data with less that 1,000 rows.  The data can be integrated in a number of ways, including Google Spreadsheets, JSon files, etc.

In addition to plugging the same data set into multiple visualizations, Exhibit provides Facet filters.  These automatically provide multiple choice filter list that dynamically modify the visualizations.

With all of these features and easy data integration I plugged it into my resume which I set up in a Google Spreadsheet.  First I came up with a basic table view (I applied a little extra CSS):

Garys Resume in an Exhibit Table

Then I added the map view to see all of the locations that I have worked at:

Garys Resume as Exhibit Map

And lastly I added a timeline view:

Garys Resume in an Exhibit TimelineAs you will notice on each of the pages there are a list of filterable facet list on the left and right side (this is configurable as well).

For more examples check out this Exhibit Examples 1Exhibit Examples 2, and Exhibit Examples 3.

So far I have only identified 3 real downsides to the software.

  1. First since the content is published through Javascript you cannot tell what the page is about by looking at its source code.  This will impact some spiders and search engines, when crawling, who cannot call the Javascirpt.  There are work-arounds as the content can be output as html, xml, tab delimited, json, etc, but the dynamic data and filtering is lost.
  2. One of the reasons the software is so easy to deploy is that the server side of Simile is hosted at MIT.  So there is no need to host the library on your site.  This can be limiting since you become dependent on MIT’s uptime and it may not be possible to deploy on an intranet, or with sensitive internal data.  The good news is that is a way to set up your own server, so it can be done, but looks to be a bit of work.
  3. Exhibit is not for large data sets.  It works well for small list that can be managed in a text file or Google Spreadsheet, but it is not something you would want to plug into a data warehouse.

Overall, Exhibit is a great way to interact and visualize data and allows you to be up and running quickly.  Check out my resume, some examples and go build your own.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Data Activism: Geeks for Good

21 12 2009

Data Activism is a new an emerging trend with designers, developers and artist utilizing technology to help society understand and address the issues of our times.

The new U.S. Administration has established initiatives to make government data available to the public.  This is converging with the growing number of developers and designers that are familiar with mash-up’s and data visualization technologies.  The convergence of these trends create the ability for developers, designers and artist to create apps, sites and art that educate and provide transparency to the government.

Creating Government Transparency

The Open Government Directive is a memo sent by the White House on December 8th, 2009 directs federal agencies to:

  1. Publish Government Information Online
  2. Improve the Quality of Government Information
  3. Create and Institutionalize a Culture of Open Government
  4. Create an Enabling Policy Framework for Open Government

Each of these are follow with specific actions that must start to show results in the next couple of months.  The main hub for this information will be, providing Federal data sets, and links to State and local efforts as well. Screenshot
This is amazing progress for the government that will bring both innovation and efficiency.  Possibilities will be opened up as great designers get creative on representing the governments data.  The government agencies will be pressured to improve as their performance becomes public knowledge.  All of this is a win for Americans with more services, useful information and government efficiency.

Everyone can participate by using the data, giving feedback, building visualizations, building reports, etc.  You can get involved by yourself, or join a team of like minded individuals.  What are you waiting for?



If you have other resources or examples you recommend, add them as comments and I will update this article.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter

Why Improving Data Visualization and Interaction Matters

15 12 2009

There is growing interest in visualizing and interacting with data for good reason.  Improving the speed and intuitiveness of data representation is required as people interact with more and more information.
Data Overload
A UC San Diego study reported on by the Huffington Post article states that “The researchers calculated that we consumed, on average, 10, 845 trillion words in 2008, or about 100,000 words per American, per day.”  And it is not just words, there are images, audio, games and more giving the average American consumption of 34 gigabyte per person, per day.

So you will have to note the word is “consume,” but one may want to argue that only some consumption really has value.  Meaning just because it was streamed in a person’s direction was meaningful information transfered from the broadcast media to the person.  And that answer is that very little of that data really becomes information and knowledge for the person who experiences it.

This is where the study and practice of Data Visualization and Interaction becomes important.  How can developers and designers make there information the easiest, quickest and most intuitive to comprehend?  Those that are good and great at reducing the interface and letting people work with the data will be the winners.

A number of web sites and applications are investing and exploring how data visualization can help organize their content.    Digg Labs has 6 different visualizations to navigate and uncover interesting news and content.

Digg 365 has a number of visualizations they use to help shoppers find what they are looking for… even if the shoppers don’t know it (The mind behind a lot of the Etsy visualizations is Jared Tarbell who is know for some great computer generate art using flash and processing).Etsy Shop by ColorAnother site known for its great traditional graphs is (disclosure: I work for Intuit which recently acquired Mint).  Mint is excelling at helping people understand there finances and budgeting trends.

While the above examples are diverse there is a common theme of helping the users of the sites better navigate content and data.

The capacity for humans to consume information cannot increase as rapidly as our systems are making it available to us.  People will focus on the information that is important and easy to consume.  Good data design can add tremendous value to sites and business that get it, making them more usable and engaging to their users.

Add to DeliciousAdd to DiggAdd to FaceBookAdd to Google BookmarkAdd to RedditAdd to StumbleUponAdd to TechnoratiAdd to Twitter