Developing an analysis plan (ethnographic data)

The other day, I found myself overwhelmed by the amount of data I had collected over the past four or five years for my global surgery project. I’m trying to finish up the book proposal, and I realized that there were bits of data that I hadn’t taken into consideration: should that stuff be in the book, or should I write articles? Should I just assume that none of those materials would be used at all? After mulling it over, I decided I would create an analysis plan. Back in the consulting days, when I designed and coordinated survey-based data collection (and charged cash-strapped NGOs a daily rate for my services), I had very little time to collect and analyze data and write up my findings. To stay within budget, I would usually take a couple of planning days to develop an analysis plan — not only to get clients on board with the scaled back version of their overly ambitious terms of reference, but also to make sure I could write a useful report quickly and as thoroughly as these short timelines allowed.

I’ve since moved on to “academic research” using primarily ethnographic data (which, yes, can include surveys, but in my case, usually does not). But I think the analysis plan template remains the same. Here, I’ll use my Twitter data as an example, even though it is probably the “rawest” and under-theorized part of my data set. (In other words, I think there has to be more said about what Twitter as social data and as social practice can tell us about specific, evolving social phenomena. But that’s another story for another day.)

  1. Decide on an overall timeline for preliminary analysis. I would like to complete preliminary analysis in 8-12 weeks. This means that each of the column headings below will correspond to a week or set of weeks.
  2. List all sources/types of data (can be two columns, e.g. Twitter, tweets). I have “scraped” twitter on my topic for over four years, using a google script. The script has produced a fairly large spreadsheet full of tweets, with some basic information about each of those tweets (RTs, Faves, etc). I also have collected odds and ends on Evernote (screenshots, radio interviews, photos, maps/images),  compiled field notes during participant observation at meetings, written detailed notes on interviews, created a database of documents from various archives; and transcribed and translated interview transcripts. Each data type/source corresponds to a set of research questions that I came into the project with. As the project evolved, I added more (sub)questions to add texture to observations.
  3. Link data source/types with specific research questions. For my Twitter data, for example, I have the following questions: How, if at all, do my interlocutors on this topic (hashtag) use Twitter? What topics do people tweeting under this hashtag write about? What types of people are in the networks coalescing around this hashtag? How has this group changed over time? Who are the nodes in these networks? Has this changed?
  4. Add new sheets as needed. Other issues with analysis and collection may arise as you sort data piles and link them to their proper set of research questions. In the case of my Twitter data, I realized that I may need to reach out to more people for interviews so I added an “interviewees” sheet.
    • This question came up as I scanned the list of tweets and the most prominent Twitter handles: have I talked to these people, or researched the material they have made available about themselves?
    • The interviewee sheet includes columns like ID, Name, Title, website, twitter handle, link to interview notes or transcripts. (I also have an “interview” data source in my general list, so this is helpful for tracking those data).
  5. Finalize duration allotted for this piece of analysis. As far as timeline goes, I have decided that I can handle this set of Twitter questions in three weeks. (if you’re fancy, you might create a Gantt chart, or link this to an Asana project, so that you’ll have a nice visual representation of progress on the project). 
  6. Follow your plan and update as necessary. Flexibility is key!

So, this is a rough sketch of my plan. I believe it could have better/more detail that would allow for tracking themes, but I’m relying on my work in Dedoose for this. Do you have an analysis plan template? What does it look like?

Using short-form writing and audio to build critical skills

This semester, I’ve been testing out several new kinds of assignments. Reading themes, podcasts, blogs. All are to get the students thinking about how to be observant, critical thinkers and how to communicate their analyses. Each of the exercises builds into one big final project: a This American Life style podcast. (Yes, I know. High expectations). For this reason, our discussions and mini-assignments are largely focused on building skills that allow them to read difficult material and listen carefully, recall and recount what they read and listened to, and to find links among stories, events and processes that are not always obvious upon first glance. Ultimately, they should be able to carefully observe their social worlds and analyze what they see in new and exciting ways. Here’s an overview of two of the assignments:

Reading themes. Each week, the student submits a word or phrase that illuminates one of the following:

    • A keyword in a conceptual or theoretical framework used by the author. For example, we read Tim Burke’s Lifebuoy Men, which grapples with the utility of Marxist theory for historiography (use-value, exchange-value, false needs, commodity fetishism) and the ‘social life of things.’ Students would then post some of these keywords and be expected to discuss and define them in class.
    • A theme that reveals the kinds of ‘conversations’ the author is in. For example, in this same book, it was clear that Burke was also dealing with a social science literature that sought to follow a commodity and the cultural meanings associated with it: in short, the social biography of a commodity.
    • A keyword or phrase about which the authors are not explicit, but which raises new questions for you and the reading. This is that sophisticated layer of analysis that takes a bit more time, and often a larger reading repertoire, but something I hope will happen by the end of the course.

The reading themes are then put in a word cloud. Frequency of terms reflects the issues that students tackled as they read the text. These word clouds guide the lecture and discussion. As you can imagine, there are also lots of words that never make it into the cloud. It’s my job to figure out what got omitted and why, and to address these in the discussion. 

Week-in-review podcasts. I got the idea for this assignment from a communications and rhetoric professor. Each week, a group of students pair up to create a five-minute audio summary of what we discussed in class. It serves two purposes:

    • Students learn how to take a relatively large amount of information and condense it, making it digestible for an audience of their peers. In theory, they are also learning to make decisions about what is important and how to organize various insights into themes. (This has been the most difficult part so far, but I am working on shifting that a bit). 
    • Students have a lower stakes opportunity to experiment with developing audio content from writing to recording to editing.

So far, I am thinking about how to better integrate students’ week-in-review podcasts into the teaching and to ensure that the word clouds become more than just a pretty depiction of word frequency for the students. I hope to post more information as we get deeper into the semester.