The biggest a-ha moment that came from my dissertation was discovering what shaky ground we stand on in data visualization.

When my friends heard I was going to study data visualization, they filled my desk with books from Edward Tufte, Stephen Few, and even Garr Reynolds. I was thirsty for resources and references because this was back in 2009, my young reader, before data visualization was so firmly established as a field. So I went down the reference rabbit hole. Oh, you know what I mean. You get one reference and you look up their references, read those articles, look up THEIR references, and so on until you know everything there is to know on the topic. Let me tell you: I didn’t get far.

Tufte, Few, Reynolds, and others who could be considered the elders of our field don’t have much to support their declarations and assertions – the ones we hear echoed, almost taken as “common sense,” still today. They cite incredibly little actual research-based evidence to back up their claims. Tufte has 5 in his famous The Visual Display of Quantitative Information. The sources they do cite are shared – meaning, they cite the same people. Few cites Tufte. It’s cute until you realize the implications.

You might think that they are excused because we didn’t have any data visualization research back then. But we did. Sure, less of it, but it was there. And in places it wasn’t, perhaps they could cool down the aggressiveness of their “standards.”

And they could make reasonable assumptions based on research from related fields. That’s what I had to do for my dissertation. I combed through literature in survey design, user interface testing, graphic design, typography, presentations, and more.

The fact is our heroes and history stand on shaky ground. Even modern books on data visualization lack research-based references.

One of my mentors is infamous for saying that not everything needs a randomized controlled trial to prove it is true or good. He said, some things are so obviously good or true, they have “interocular validity” – the kind that hits you right between the eyeballs.

But my mentor has something in common with Tufte, Few, and Reynolds. He was a privileged white guy, so his view of what looks good to his eyeballs is not necessarily shared.

The problem with interocular validity is that it means: If I think it’s right, it’s right. This puts us into difficult situations, where we are left tossing opinions. And opinions conflict. The Branding team at Nike may have had the opinion that this chart is good…

… and you might think they have a green obsession. It is no longer just a difference of opinion when the deep research on color by Dr. Cynthia Brewer gives us evidence to the contrary. Research gives you a backbone.

Communications may have thought this Tesla chart looks right…

…even if you are the internal data person trying to argue that bar charts should start at 0%, especially in this case, where a truncated axis makes them look safer than reality. When you have research to back up your recommendations, it turns your shaky ground a little more solid.

Now, research conflicts, too. But at least actual data gives us something to stand on (and this is why all Academy tutorials come backed by research).

Let’s knock down a few other pillars of viz history.

Ever seen this classic backbone as to why we should visualize?

The oft-cited 60,000x justification sounds sexy but has no basis in research. The reference rabbit hole on this one leads to a memo produced by 3M’s Visual Systems Division. Convenient that the Visual Systems Division would make such a conclusion about visuals, huh?

And this classic viz, produced by John Snow, is heralded as an example of the power is visualization. You probably know the story – during the cholera epidemic people thought the disease was spread by air. Snow marked cholera deaths on a map – one black line per person – and noticed they circulated around a water pump on Broad Street so they just removed the handle from the pump and the epidemic was over. Hero.

In reality, the notion that cholera spread through the air was accepted thinking until after Snow’s death. This viz may have been one drop of evidence but it didn’t have the heroic impact we like to think.

Let us question the foundation of our work because what may look legit to your eyeballs could be full of myth and conjecture.

Evidence, even if it is still evolving, informs decision-making. Heck, that’s why many of us are interested in data visualization in the first place, right? So let us not accept anything less in who we refer to for our data visualization guidance.

This is why my books have references at the end of every chapter. I am not messing around. My books are a good starting point. To nerd out more, investigate IEEE and Multiple Views.