Blog

Guest Post: Word Cloud Dog Vomit, An Illustrated Rant

Guest Post: Word Cloud Dog Vomit, An Illustrated Rant

My colleague Humphrey Costello delivers the funniest, snarkiest Ignite sessions at the American Evaluation Association’s annual conference. I’m so happy to give him this platform to articulate the answer to one of the questions I am asked the most, about the phenomenon known as word clouds.

1st ingredient: Long ago, I promised Stephanie a blog entry on word clouds.

2nd ingredient: AEA 2013 just happened.

3rd ingredient: It was a thrill to find the latest copy of New Directions on data viz in my mailbox. Congratulations to Tarek, Stephanie, and everyone else involved!

Blend: Here is a word cloud of all AEA 2013 conference program session titles. I made it using Wordle. (The idea of playing around with conference session titles is not original. My partner, a sociologist, showed me a blog post that examined what sociologists were up to at ASA 2012—lots of examining implications and transgressing boundaries, IIRC.)

wordle1

Through no fault of Wordle (which is easy and fun), this word cloud is crap. If you’re wondering what will go on at AEA 2013, this world cloud shows that there will be sessions on evaluation—not very helpful.

Deleting the terms ‘evaluation,’ ‘rotation,’ and ‘roundtable’ from the text yields this:

 wordle2

In their article in New Directions, Henderson and Segal say “when evaluators hear of the idea of visualizing qualitative data, a word cloud may be the first things that comes to mind.” Henderson and Segal are not advocating the use of word clouds; that sentence is immediately followed by “However,” and a litany of reservations.

Frankly, I think Henderson and Segal work too hard to find redeeming qualities and possibilities for word clouds. (They generously suggest that word clouds might be made useful if one could click on the words and be taken to relevant text.) No qualitative evaluator I know—not even those who maintain that quantitative methods are part and parcel of the oppressive positivist-corporate/capitalist patriarchy—would assert that counting words constitutes decent analysis.

Drawing on their erudition, here are a couple of cheap shots on word clouds:

  • Word clouds don’t show words’ relative importance accurately. Frequency = font size, regardless of word length. To my eye ‘Approaches’ looks bigger, weightier, than ‘Social’ even though both words are in the same font size and appear with the same frequency in the text. ‘Approaches’ takes up more real estate, and grabs more attention, just because it is a longer word.
  • Word clouds ignore words’ contexts. Without context, it is hard to know what is meant by words in the cloud. Is ‘Capacity’ evaluators’ capacity, their clients’ capacity, or the maximum amount their tummies can contain? (Gratuitous distraction: What are words for? https://www.youtube.com/watch?v=IasCZL072fQ)

The bigger problem that this word cloud is still near useless as an illustration of what took place at AEA. I challenge you to use this word cloud to come up with a single clear sentence about important themes of the conference.

Have you seen a useful word cloud? Link to it in the comments!

In the meantime, check out my new book, which doesn’t have a single word cloud in it. Or indulge me with an rant of your own at one of my upcoming events.

Rants are so fun, but we also need to draw inspiration from great examples – so check out the good stuff at thumbsupviz!

10 thoughts on “Guest Post: Word Cloud Dog Vomit, An Illustrated Rant
  1. Natalie Nestor says:

    I used a word cloud in a training segment to illustrate how confusing financial terminology is and how disconnected the average employee feels to their company’s bottom line. I thought it did EXACTLY what I needed it to do – be overwhelming and unclear.

  2. Chris Lysy says:

    Ok, so I just watched that video. Be honest Humphrey, was that a younger you on the drums?

    To your point though. I think word clouds only exist because people want something to decorate their reports (decorate being the operative word, not design) and most people are uncomfortable drawing snazzy pictures.

  3. Megan Walker Grimaldi says:

    It depends on how you define “useful”. If you are looking to do rigorous qualitative analysis, you shouldn’t use word clouds. If you are using a word cloud to interest laypeople and begin to engage them in your evaluation, I have found it to be useful.
    At the end of interviews in a study we’re conducting, we ask, “What is the one word you would use to describe the impact that [our organization] has on your community?” Once word clouds are made, the size of the individual words represent the frequency with which stakeholders mentioned a particular theme when thinking of our organization. We cut the data we collected over interviews into groups of stakeholders or groups of affiliate organizations so that they can directly see what their words look like. Once stakeholders see this, it spurs interest in the more rigorous findings. It’s an “in” – it’s not fancy, but it’s something.

    • Stephanie Evergreen says:

      I like this idea because you’ve given greater structure to the data collection – so you intentionally only get one word and it better suits the algorithm of something like Wordle.

      • Lisa Richardson says:

        I also use Word Clouds in the similar way Meghan described and in part for purposes of particpatory evalaution. At the end of a conference, I asked for 2 words to describe their experience and it becomes an entry to making more meaning of the other data and personal experiences at the conference. We are also able to look at these over several years and see if anything changes in reaction to agenda changes (it hasn’t!)
        Three years ago, I asked the planning team at the beginning of the planning process to imagine the end of the conference and tell me what two words they hoped participants would say about their experience. We then looked at the planning team “hopes” with participant experieces.
        Having said all this, my disclaimer is that this is the ONLY way I am willing to use for all the excellent reasons mentioned. Natalie’s use is intriguing though…..

  4. Jon says:

    Sometimes your audience will benefit from a Word Cloud. See the post I wrote about this here: http://www.allanalytics.com/author.asp?section_id=3072&doc_id=268097.

  5. Carolyn Sullins says:

    Both word clouds and articles with “dog vomit” in their title accomplish the same thing: getting people’s attention.

    I have seen word clouds used by youth involved in evaluating their own SAMHSA-funded mental health systems-of-care. While not the most robust form of evaluation, it does increase youths’ appetite for learning about and conducting evaluation.

    It was at least as effective as my attempt to involve youth in our local system-of-care evaluation, which included my singing “Let the Bodies Hit the Floor.” (Both the original and my remake sounded just like dog vomit!)

  6. Dr Jens J. Hansen says:

    Word Clouds can have quite valuable uses if used judiciously. For example, it’s possible to limit the number of words included (say 50 words in all, each with three letters or more) so that a very rapid ‘assay’ of a body of material can be gauged. It’s never the same, of course, as actually reading the material but it’s a way of gaining a very quick overview so that the reading can become focused on answering tentative little speculative questions. In addition, Word Clouds can be used for presentation purposes but again, there is merit in not going overboard with the quantity of words. And finally, there’s merit in scrutinizing a Word Cloud not only for what appears infrequently and is represented by little sized words(again so that speculations and attendant questions can be sparked), but also to think about what might be missing altogether.

  7. Craig Hadden – Remote Possibilities says:

    Interesting topic. I like to count occurrences of the words “you”, “your” (etc) as a clue about how audience-focused a piece of text is. I hadn’t thought of presenting the result as a word cloud, but it could be helpful, depending on how the cloud’s created.

    Maybe word clouds are most useful when only a few target words are included. For instance, putting only your own organisation’s name and the words “you” and “your” in the cloud might be very telling about your writing’s focus!

    Last month, Dave Paradi published a word cloud of the words or phrases that survey respondents often hear used to describe typical presentations. As well as being telling, the result was predictable – in more ways than one!

    By the way, one time I had to take a finance-related elearning course for “compliance” purposes. For some bizarre reason, the course decided to present definitions for several everyday words, like “customer”. Despite being for a non-technical audience, the definition of “customer” was a single sentence of 50 words, and nine of those were the word “or”! Needless to say, that 50-word sentence was complete legalese nonsense. If you’re interested, I’ve made a Wordle of it below:
    http://www.wordle.net/thumb/wrdl/7270522/Definition_of_a_customer

    Anyway, thanks for provoking thought about the usefulness of word clouds. Good stuff!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

RT @ImADataGuy: Anyone else see something wrong with this? #datavisualization @evergreendata Good cause, bad math http://t.co/wcQMJS87QE

2 of the worst fonts, combined into 1 http://t.co/5YH662i8xP

@clysy @laurabeals @SheilaBRobinson @AnnKEmery this http://t.co/CAVld759Ej also handouts are going to be discussed on the podcast soon!

RT @LegoAcademics: Green initiative: Dr Black has only used recycled presentation slides for the past 3 years. http://t.co/MqzxL5xjWc

@seanlahman how do you find the time?

RT @leeclowsbeard: If your PowerPoint presentation also works as a leave behind, you’re doing both wrong.

Missed flight = sketching my 2014 personal annual report on the only paper available (my spare workshop handout) http://t.co/fQZg0gj3fa

I'm suddenly amazingly sleepy.

Wow, heading home after my last workshop of the year. Look for the total number I've given on my annual report, out soon.