Eval Academy

View Original

Interpreting themes from qualitative data: thematic analysis

Stephanie Jones

This article is rated as:

Methods such as interviews and focus groups can be an excellent way to collect qualitative data for your evaluation. Check out our previous article on how to conduct interviews. However, there are lots of methods for making sense of qualitative data. These include narrative analysis, discourse analysis, grounded theory, constant comparison analysis, interpretive phenomenological analysis (IPA), and qualitative content analysis. Thematic analysis is one such method that can be used to break down the process into manageable steps. It can also help you to represent your data as honestly as possible.  

This article supports evaluators who are new to qualitative data analysis. We start by defining thematic analysis, then give you a 5-step process to complete your own analysis. We end the article by highlighting some common challenges with thematic analysis. 

Our top tips for Thematic Analysis

  1. Read, read, and read your data again! Coding is only efficient when you are completely familiar with your entire data set. 

  2. Although this is an exploratory method, keep the purpose of your research in mind. This will stop you from getting carried away and over-coding data that is not relevant. 

  3. Don’t rush! Leave yourself plenty of time for qualitative data coding and analysis. It takes practice and you won’t get it right on the first go. This is definitely a learn-by-doing method. 

  4. If possible, walk through your process and codes with a colleague or team member. Everyone looks at qualitative data slightly differently. Having someone review it will support validation of your codes and themes, and reduce bias. 

  5. Transparency is key! Your methods for analysis should be clearly described and documented. This ensures that your readers are aware of the process you followed. Make sure you manage your data by keeping a master list of codes (if coding by hand) and backing up any work you complete in analysis software. 

What is Thematic Analysis?

Completing a clear and organized analysis of qualitative data is extremely important. Thematic analysis provides you with the opportunity to go through your data in a methodical and thorough way to identify themes and patterns. 

In short, thematic analysis is a way of producing themes from texts such as interview or focus group transcripts. The method makes sense of large amounts of information so that responses to a research question can emerge. 

This method is very flexible. It is also a good method to follow when you want to find out people’s views, opinions, knowledge, or experience on a topic. The most common method of thematic analysis follows a 5 or 6 step process: 1) familiarization; 2) coding; 3) generating themes; 4) reviewing themes; 5) defining and naming themes; and 6) reporting. These steps were defined by Braun & Clarke (2006) in this article which is paywalled.

The method is suitable for both inductive and deductive studies which are described below. However, some steps will not be as long in a deductive process. 

Deductive Studies:

Deductive is coming to the data with predetermined themes that you expect to find based on existing knowledge or established evaluation questions. In our experience, deductive analysis is more common than inductive analysis in evaluation. It starts with a predefined set of codes which are then assigned to the qualitative data set. These codes might come from previous research, or you might already know what themes you are interested in looking for. It is important that other ‘unexpected’ themes are not missed. These can be captured by creating a code such as “Unexpected info” or “Misc. theme.” Deductive coding can save time and help guarantee your areas of interest are coded, but you also need to be careful of bias. 

Inductive Studies:

Inductive is a method of coding that allows the data to determine your themes. This is also known as ‘open coding.’ This is an iterative process that involves lots of refinement and multiple rounds of analysis. It often takes longer but can be more thorough and exploratory than deductive coding. If you’ve ever used a grounded theory approach, you’ve probably done inductive analysis.   

It is important that you decide which method (either inductive or deductive) to use before you start the thematic analysis! 

How to use Thematic Analysis

Step 1: Familiarize yourself with the data 

Having a large qualitative data set can be overwhelming. So where are you supposed to start?  

 The first step is to become familiar with the data. If you collected the data, you may have already started to make notes on areas of interest discussed by the participants. Another great way to become familiar with the data is to complete the transcribing process. (See our tips on how to transcribe interviews like a pro.) But if you outsourced the transcribing or maybe weren’t the one completing the data collection, it is important to spend time reviewing (either reading or listening to) your data. Take notes, memos, and start to jot down some ideas of potential codes. But don’t start coding just yet! At the end of this process, you should be very familiar with the entire body of data.  

Step 2: Generate an initial set of codes from a first review, and code your data 

What exactly is a code? A code is a brief description of what is being said in the interview or focus group extract. It’s a description, not an interpretation (we save this part for later!). 

A code is a word or a short phrase that captures the meaning of specific quotes. What codes you use depends on what is being said within your data and on the purpose of your research. It also depends on whether you are performing an exploratory analysis (i.e., inductive) where the themes depend on the data. Or a deductive analysis where you search for specific themes. At this stage, your codes might look something like this: 

  • Job security 

  • Learning resources 

  • Horizontal violence 

  • Unions 

  • Wages 

Overall, you should start to identify codes by reading through your data and applying the same code to sections of the text that represent the same meaning. You should also start to create a codebook to keep track of the codes. If you are coding by hand, you can use sticky notes and colour coding to start to organize your data in a meaningful and systematic way. 

There are also different software packages that can support coding such as NVivo (my personal favourite) and MAXQDA. Be thorough at this initial stage and don’t be afraid of over-coding. We will refine our codes in the later steps. Be careful not to lose context by coding too little of your data. 

In my own experience, coding at this general level first is a step towards organizing the data into meaningful categories. It also allows me to discover reoccurring concepts that could be further refined. Coding in this exploratory way enables me to recognize and recontextualize the data to give a fresh perception of what was already visible. 

Consistent coding is accurate coding, so establishing coding procedures and guidelines from the start is crucial.

Step 3: Start to search for themes in your codes across the entire data set 

This step is all about organizing your codes and starting to identify reoccurring themes. If doing this by hand, cut out all of the data extracts pertaining to the specific codes and start to group the codes together. If using software, the software will automatically do this for you. At this stage, you might decide that some codes are unclear or not relevant to your study. Now is the time to modify and update codes that may not align with the purpose of your project. You can also reflect on whether there is any missing data. For example, is there anything that wasn’t discussed which you found surprising? 

What is a theme? Themes are broader than codes (often combining several codes into a theme) and involve interpretation of the codes and the data. A theme captures something important about the data in relation to the research purpose. It also represents a pattern or relationship across the data set. Searching for common themes across codes is an iterative process where you move back and forth between the codes to identify commonalities.  

Once you have a list of main themes you can start to reflect on the relationships between them and how the themes fit together to tell a bigger story. 

Step 4: Review and refine your themes 

Review your themes by (once again) reading through your data excerpts and ensuring that there are identifiable differences between your themes. Themes need to be clear and distinctive, as well as tell a story. This review process will help to identify new themes you might have missed and make sure that your themes are useful and accurate representations of the data. Ensure that each of your themes have enough data to be convincing and show patterns across the data set.  

Some overarching questions can help you to review your themes: 

  • Do the themes make sense? 

  • Does the data support the themes? 

  • Is the theme too broad or too narrow? 

  • If themes overlap, are they really separate themes? 

  • Are there themes within themes (subthemes)? 

  • Am I missing any themes? 

Define exactly what you mean by each theme. You can also construct a thematic map like the one below to help refine your themes. Discussing your themes with a colleague or your team can help to ensure agreement across findings. You can also use intercoder reliability i.e., having a colleague code the data set using your codebook to see whether you are capturing the same thing. The general rule of thumb is that each theme should not have more than 4-5 codes. It is also better to have 6-10 broad themes with sub-themes rather than lots of really detailed themes.

Step 5: Reporting 

It is crucial that you clearly communicate the methods and steps that you took in your thematic analysis. You may also want to share a copy of your codebook and related themes in an appendix of the report. 

Reporting needs to go beyond just describing your data and should include your own analysis to make an argument for the claims and the story you present. Don’t just paraphrase your data. Make sure you tell a coherent story about your data and choose quotes that back up your points. You can also create an “exemplar quote” code to highlight quotes that you might want to use that help to capture the essence of themes. This helps to make quotes easy to find and will save you re-reading the excerpts another time! 

Ensure any ethical processes are followed e.g., removing identifiable features of the quotes. In projects with small populations, we’ll sometimes even remove any unique turns of phrase or colloquialisms that seem like they could identify the speaker.  

Be aware of the following challenges in thematic analysis: 

  • Confirmation bias: thematic analysis is subjective. It often relies on the evaluators/researcher’s judgement. Reflect on the bias of your own interpretations. Document what these might be and how they might affect the results. 

  • If you’re completing a deductive process, look explicitly for contradictory evidence in the data to minimize bias or missing any important points. 

  • Don’t just organize, make sure you actually analyze the data! Don’t use the main interview questions as themes, a deeper level of analysis is required. 

This can be a long process, especially for first-time coders! So make sure you leave plenty of time in your evaluation plan/timeline to complete the thematic analysis. 

Why not give it a go and let us know how you got on! 

See this form in the original post

Sources: