Graphical Integrity and Social Media

Today Razorfish released its 2009 FEED report, its annual study on technology and brands. While I believe the motivation behind the report to be worthwhile, I was shocked to find charts that violated the basic principles of graphical integrity.

"Graphical Integrity" is the title of the second chapter Edward Tufte's book The Visual Display of Quantitative Information, a must-read for anyone in the data business. He writes, "For many people the first word that comes to mind when they think about statistical charts is 'lie.'" The continued skewing of data, whether its a "1 out 5" headline, or the charts that I found in this report, do not help increase confidence in statistical charts. The practice of skewing data is particularly rampant among Twitter fans and those wishing to promote their careers or services in social media. My years in advertising led me to a keen sense of data misuse - I was more often called upon to find data that supports the creative strategy, rather than do research that led to a target-right creative strategy. 

In the spirit of Tufte's statement "Deception must always be confronted and demolished," I present these two "charts" from the Razorfish FEED study:

If, as Tufte states, "Graphical excellence begins with telling the truth about data," do these charts tell the truth?

No.

Why do 25.5% and 40.01% get bigger circles than 74.5% and 59.9%? The circle for 74.5% should be 3X as large as the 25.5% circle, not the other way around. Is the 25.5% of the respondents who have followed a brand on Twitter 3X more important than those who don't? For an ad agency trying to promote its social media services, perhaps. But the charts egregiously mislead the rest of who want the truth, particularly the truth about where to invest our marketing dollars.

It confounds me how those charts got made and approved, when there are accurately proportioned pie charts in the "Data" section. These "Yes" and "No" circle charts abound in the report, but the circle sizes are the same whether its a 97/3 or 65/35 split. Did the Republicans present the presidential election results in a chart that showed a big circle for McCain's 45% of the popular vote and a small circle for 55% of Obama's popular vote to make themselves feel better that the electoral votes were 365 to 173? How would we feel if this was done with frequency on the news? Even Fox News couldn't get away with that. So why can Razorfish?

I won't go further about how the report takes its statistically unrepresentative sample of "connected consumers" 18-55 who live in major cities and spent more than $150 online in the last 6 months as representative of ...what? One of the interesting findings for me was that 44% of this "connected" sample did not own a smartphone, while 16.5% owned a Blackberry and 11.6% owned iPhone. (29.5% and 20.07% respectively of the 56.3% that owned a smartphone, according to the data section). It puts investing in an iPhone app in a different light if you're a brand marketer.

Data is a tricky business, since it can be so easily skewed. Because of reports like this one, you should look closely at the numbers in charts, not the shapes or sizes, as well as voraciously read methodology sections. And the next time your graphics team comes back with a chart, ask them what software they used to make it if it didn't look anything like the one you made in Excel.

Comments

[Brisa]

If you need more info on this issue, I recommend making use of http://www.pdfqueen.com pdf books searching service

[Lisa Tweedie]

Have a look at the comments that myself and others put on the welcome page of this report.
http://feed.razorfish.com/2009/11/welcome/#comments

The comments point out it is not just the graphs that are slightly dodgy - the report also states that the data from their sub sample can be generalized to mainstream - with no evidence to show this. Slightly unusual!!

Lisa Tweedie (@lisatw)