Trochim’s Research Methods Knowledge Base-Online [Troy Tenhet]

(Trochim) Research Methods Knowledge Base-Online


Troy Tenhet

Qualitative Measures

The purpose of this document is to introduce you to the idea of qualitative research. There are a number of questions that you should consider before doing such research.

  • Are you trying to generate new theories/hypotheses?

You are supposed to be familiarizing yourself with the phenomenon that you are interested in. Too often, researchers do a lit review and then write a research proposal. Try to experience what you are examining before your research starts, then you can formulate your own ideas and hypotheses.

  • Do you need to reach a DEEP UNDERSTANDING of your issue?

This qualitative research is effective in investigating complex and sensitive issues. Quantitative is fine when it’s appropriate, but qualitative has an in-depth interviewing component where you can dig deep.

  • Are you willing to trade detail for generalizability?

Qualitative research is about details. Quantitative analysis is detailed too, but the data tends to shape and limit the study. Quantitative research is also fairly straightforward when generalizing it. Qualitative research is different. Its data is raw and rarely in a set category. Be ready to put things in categories (about a million ways to do this) as you attempt to generalize your research. The detail side of qualitative is a blessing. You can really describe your phenomenon in detail, unfortunately, with such detail; general themes are difficult to flush out. Therefore, try to mix quantitative and qualitative together (mixing immense amounts of data with the real story).

  • Is funding available for the research?

Try not to propose research that isn’t funded. Qualitative research takes LOTS of time and is labor intensive (and may yield results that cannot be generalized). Be advised that it IS POSSIBLE to estimate how much research will cost. Break it into pieces and set funding/timelines accordingly.

The Qualitative Debate

There is a lot of energy spent on which is better (quantitative or qualitative). Both do great, especially together (mixed-methods approach). Qualitative data is words, and quantitative data is numbers (both are data). Just know that all qualitative data can be coded quantitatively. This means anything that is qualitative can be assigned numerical values. Open-ended questions on surveys can have their answers sorted into themes. Check out the “ten responses to five themes” picture below. The first is qualitative; the second is quantitative (same data).

Note—Above data is EXACTLY THE SAME.

We can do a lot with this, and the line between qualitative and quantitative is hazy at best. Based on above data-look at the correlation matrix (theme 2 and 3 are negatively correlated—meaning that if they chose 2, they didn’t choose 3, and vice versa).

We can look at similarity too…among respondents (see below).

Persons 1 & 3 and 4 & 8 are perfectly correlated (r=+1.0). Now look for perfect opposites (r=-1.0). Remember that both types of data (qualitative and quantitative) are similar and this works to our advantage. Additionally, all quantitative data is based on qualitative judgment. Now look at the next Likert scale question:

Because this person circled the “2” what does that mean? Did respondent understand capital punishment? Was respondent aware of what the “2” means? What was the setting? What was the respondent’s history? Etc. The choosing of “2” means judgments (possibly incorrect ones) were made. Try to rid yourself of these myths:

  1. Quantitative research is confirmatory and deductive in nature.
  2. Qualitative research is exploratory and inductive in nature.

Neither of the above is completely true or false. The real debate between the two methods is philosophical. Many qualitative researchers operate under different epistemological assumptions. They think the best way to examine something is to immerse yourself in it. Look at the whole thing in its context. Allow questions to emerge (rather than setting a fixed question). They see quantitative as limited and looking at only one section/portion. Also, many qualitative researchers operate under different ontological assumptions. They don’t believe in a single idea of reality. We all experience and examine things based on our own perspectives. Each individual is unique. Researchers are biased. Good luck establishing validity since all we can do is try to interpret the world from our perspective. In the end, both methods together are the best bet.

Qualitative Data

This type of data is varied in nature. It includes any information that can be captured.

  • In-depth interviews: both individual and group. Data is recorded, taped, written down, audio, video. The idea of an interview is to probe the interviewee for information about the phenomenon of interest.
  • Direct observation: involves watching and not questioning respondents. Could include field research where researcher is immersed in a culture or context. Data is collected by same way as in-depth interviews. Could include drawings (like in a courtroom).
  • Written documents: refers to existing documents including transcripts, newspapers, magazines, books, websites, memos, reports, etc.

Qualitative Approaches

When we reference the “approach,” it refers to how we will conduct our research. It describes purpose of research, role of researcher, stages of research, and method of data analysis.

  1. Ethnography: emphasizes studying of entire culture. This includes virtually any defined group or organization. Very broad in scope. The idea of “participant observation” where the researcher is immersed in the culture as active participant is common.
  2. Phenomenology: also considered a philosophical perspective. Focuses on people’s subjective experiences & interpretations of the world (looks at how world appears to others).
  3. Field research: a broad approach that collects qualitative data. Researcher goes into the field to observe phenomena in natural state. Extensive field notes are taken and coded/analyzed.
  4. Grounded theory: developed by Glaser and Strauss to develop theory based on observation (called rooted or grounded in observation). First you raise generative questions that guide the research but don’t restrict it. Then core theoretical concepts are flushed out. Takes time. The key is to analyze results into a core category. As you analyze, coding is used. Also, memoing is used to record your thoughts/ideas as you develop theories. Lastly, use integrative diagrams to put a graphic to the words and numbers. Helps with clarification (think concept maps, etc.).

At the end, you approach “conceptually dense theory” where the core concept is finally identified. When you finish the process, you should have a good explanation for the phenomenon of interest.

Qualitative Methods

Just know that the use of these methods is limited (mostly) by your imagination.

  1. Participant Observer: very common and very demanding. Researcher becomes participant in culture that is being observed. Be mindful of how you enter the culture/context, of your role as a researcher, of how you store your data/notes, and of how you analyze the data. You have to be accepted as a natural part of the context before you can even hope that what you observe is even real.
  2. Direct Observation: doesn’t become part of the context. Strives to be unobtrusive. This perspective is hopefully detached. Researcher is WATCHING and not TAKING PART. Researcher is observing for specific concepts/behaviors (rather than looking at everything at once). Think about looking through a one-way mirror for a specific behavior and nothing else.
  3. Unstructured Interviewing: involves direct interaction between researcher and respondent(s). No formal structured instrument or format. Researcher can move the interview in whatever direction that seems appropriate at the time. Good for exploring a topic broadly. Difficult to analyze (no structure) and more difficult to synthesize across respondents (no set of questions).
  4. Case Studies: these are specific and intensive studies of specific individuals or specific contexts. It is a combination of methods discussed above. An example could be Piaget’s case studies of children in order to study their developmental stages.

Qualitative Validity

It is not uncommon to hear a qualitative researcher reject the idea of validity, as this is usually a commonly accepted idea in quantitative research. The qualitative researcher may say that there is a reality that is different from our perception of it. Guba and Lincoln give us some alternative criteria for judging the soundness of our qualitative research.

Credibility involves establishing that the results are credible from the perspective of the participant in the research (since the qualitative concept is to understand phenomenon through the participant’s eyes, then only the participant can judge credibility).

Transferability refers to an ability to transfer the results to other contexts or settings. The key to the ability to transfer results is a thorough description of the research context and the assumptions central to the research. Anyone trying to transfer results to their context is then responsible for judging how appropriate (or sensible) the transfer may be.

Dependability is compared with reliability (on the quantitative side). Reliability is all about replicability and repeatability. With dependability, the qualitative researcher must account for the changes in the context during the course of the study. Therefore, if the setting changed, it must be described and then the way it affected the research must also be discussed.

Confirmability is about the research and if it can be confirmed or corroborated by others. In order to enhance confirmability, qualitative researchers document procedures for checking and re-checking their data. Data audits of data collection and analysis are also conducted to help with this.

In the end, more work needs to be done to apply traditional quantitative validity criteria to the qualitative domain. Unfortunately, quantitative criteria match up with quantitative research. Qualitative researchers will usually say that the validity issue is irrelevant (how can you judge reliability of qualitative data when you don’t know how to get the true score).

Remember, a true score is a replicable feature of a concept being measured (the mean score of multiple attempts).

Final note for this section…perhaps validity in quantitative circles is a researcher concern. But in qualitative circles, the participant in the research can only truly gauge it, right? Maybe I’m wrong.


✤Discuss prior CSUF fieldwork that could have benefited from qualitative analysis. Which method (participant observer, direct observation, unstructured interviewing, case study) would you have used?
✤Which is best, quantitative, qualitative, or both? Discuss CSUF fieldwork that matches specifically to EACH designation only.
✤Create a survey question that demonstrates the issue of “quantitative data is really the product of qualitative judgment.” Explain.
✤In terms of qualitative data types, discuss the CSUF fieldwork context in which each type is an appropriate method (in-depth interview, direct observation, etc.).
✤Weigh in on the qualitative/quantitative validity argument.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s