Guest Post: Word Cloud Dog Vomit, An Illustrated Rant

My colleague Humphrey Costello delivers the funniest, snarkiest Ignite sessions at the American Evaluation Association’s annual conference. I’m so happy to give him this platform to articulate the answer to one of the questions I am asked the most, about the phenomenon known as word clouds.

1st ingredient: Long ago, I promised Stephanie a blog entry on word clouds.

2nd ingredient: AEA 2013 just happened.

3rd ingredient: It was a thrill to find the latest copy of New Directions on data viz in my mailbox. Congratulations to Tarek, Stephanie, and everyone else involved!

Blend: Here is a word cloud of all AEA 2013 conference program session titles. I made it using Wordle. (The idea of playing around with conference session titles is not original. My partner, a sociologist, showed me a blog post that examined what sociologists were up to at ASA 2012—lots of examining implications and transgressing boundaries, IIRC.)


Through no fault of Wordle (which is easy and fun), this word cloud is crap. If you’re wondering what will go on at AEA 2013, this world cloud shows that there will be sessions on evaluation—not very helpful.

Deleting the terms ‘evaluation,’ ‘rotation,’ and ‘roundtable’ from the text yields this:


In their article in New Directions, Henderson and Segal say “when evaluators hear of the idea of visualizing qualitative data, a word cloud may be the first things that comes to mind.” Henderson and Segal are not advocating the use of word clouds; that sentence is immediately followed by “However,” and a litany of reservations.

Frankly, I think Henderson and Segal work too hard to find redeeming qualities and possibilities for word clouds. (They generously suggest that word clouds might be made useful if one could click on the words and be taken to relevant text.) No qualitative evaluator I know—not even those who maintain that quantitative methods are part and parcel of the oppressive positivist-corporate/capitalist patriarchy—would assert that counting words constitutes decent analysis.

Drawing on their erudition, here are a couple of cheap shots on word clouds:

  • Word clouds don’t show words’ relative importance accurately. Frequency = font size, regardless of word length. To my eye ‘Approaches’ looks bigger, weightier, than ‘Social’ even though both words are in the same font size and appear with the same frequency in the text. ‘Approaches’ takes up more real estate, and grabs more attention, just because it is a longer word.
  • Word clouds ignore words’ contexts. Without context, it is hard to know what is meant by words in the cloud. Is ‘Capacity’ evaluators’ capacity, their clients’ capacity, or the maximum amount their tummies can contain? (Gratuitous distraction: What are words for?

The bigger problem that this word cloud is still near useless as an illustration of what took place at AEA. I challenge you to use this word cloud to come up with a single clear sentence about important themes of the conference.

Have you seen a useful word cloud? Link to it in the comments!

In the meantime, check out my new book, which doesn’t have a single word cloud in it. Or indulge me with an rant of your own at one of my upcoming events.

Rants are so fun, but we also need to draw inspiration from great examples – so check out the good stuff at thumbsupviz!


  1. I used a word cloud in a training segment to illustrate how confusing financial terminology is and how disconnected the average employee feels to their company’s bottom line. I thought it did EXACTLY what I needed it to do – be overwhelming and unclear.

  2. Ok, so I just watched that video. Be honest Humphrey, was that a younger you on the drums?

    To your point though. I think word clouds only exist because people want something to decorate their reports (decorate being the operative word, not design) and most people are uncomfortable drawing snazzy pictures.

  3. Megan Walker Grimaldi

    It depends on how you define “useful”. If you are looking to do rigorous qualitative analysis, you shouldn’t use word clouds. If you are using a word cloud to interest laypeople and begin to engage them in your evaluation, I have found it to be useful.
    At the end of interviews in a study we’re conducting, we ask, “What is the one word you would use to describe the impact that [our organization] has on your community?” Once word clouds are made, the size of the individual words represent the frequency with which stakeholders mentioned a particular theme when thinking of our organization. We cut the data we collected over interviews into groups of stakeholders or groups of affiliate organizations so that they can directly see what their words look like. Once stakeholders see this, it spurs interest in the more rigorous findings. It’s an “in” – it’s not fancy, but it’s something.

    1. I like this idea because you’ve given greater structure to the data collection – so you intentionally only get one word and it better suits the algorithm of something like Wordle.

      1. I also use Word Clouds in the similar way Meghan described and in part for purposes of particpatory evalaution. At the end of a conference, I asked for 2 words to describe their experience and it becomes an entry to making more meaning of the other data and personal experiences at the conference. We are also able to look at these over several years and see if anything changes in reaction to agenda changes (it hasn’t!)
        Three years ago, I asked the planning team at the beginning of the planning process to imagine the end of the conference and tell me what two words they hoped participants would say about their experience. We then looked at the planning team “hopes” with participant experieces.
        Having said all this, my disclaimer is that this is the ONLY way I am willing to use for all the excellent reasons mentioned. Natalie’s use is intriguing though…..

  4. Both word clouds and articles with “dog vomit” in their title accomplish the same thing: getting people’s attention.

    I have seen word clouds used by youth involved in evaluating their own SAMHSA-funded mental health systems-of-care. While not the most robust form of evaluation, it does increase youths’ appetite for learning about and conducting evaluation.

    It was at least as effective as my attempt to involve youth in our local system-of-care evaluation, which included my singing “Let the Bodies Hit the Floor.” (Both the original and my remake sounded just like dog vomit!)

  5. Word Clouds can have quite valuable uses if used judiciously. For example, it’s possible to limit the number of words included (say 50 words in all, each with three letters or more) so that a very rapid ‘assay’ of a body of material can be gauged. It’s never the same, of course, as actually reading the material but it’s a way of gaining a very quick overview so that the reading can become focused on answering tentative little speculative questions. In addition, Word Clouds can be used for presentation purposes but again, there is merit in not going overboard with the quantity of words. And finally, there’s merit in scrutinizing a Word Cloud not only for what appears infrequently and is represented by little sized words(again so that speculations and attendant questions can be sparked), but also to think about what might be missing altogether.

  6. Interesting topic. I like to count occurrences of the words “you”, “your” (etc) as a clue about how audience-focused a piece of text is. I hadn’t thought of presenting the result as a word cloud, but it could be helpful, depending on how the cloud’s created.

    Maybe word clouds are most useful when only a few target words are included. For instance, putting only your own organisation’s name and the words “you” and “your” in the cloud might be very telling about your writing’s focus!

    Last month, Dave Paradi published a word cloud of the words or phrases that survey respondents often hear used to describe typical presentations. As well as being telling, the result was predictable – in more ways than one!

    By the way, one time I had to take a finance-related elearning course for “compliance” purposes. For some bizarre reason, the course decided to present definitions for several everyday words, like “customer”. Despite being for a non-technical audience, the definition of “customer” was a single sentence of 50 words, and nine of those were the word “or”! Needless to say, that 50-word sentence was complete legalese nonsense. If you’re interested, I’ve made a Wordle of it below:

    Anyway, thanks for provoking thought about the usefulness of word clouds. Good stuff!

Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

This site uses Akismet to reduce spam. Learn how your comment data is processed.