(Barley) For a Definition of What Ethnography is Not [Nan]

PARC Forum:  Stephen R. Barley, Center for Work, Technology and Organization, Stanford School of Engineering at    http://www.parc.com/event/1162/ethnography.html
View minutes 4:00-16:00 of this talk (end on the “ignorance of expertise”)

Summary of presentation by Stephen Barley, a Stanford professor who teaches courses on technology and work, the management of R&D, social network analysis, and ethnographic field methods and researches organizational culture in engineering.

In defining ethnography, both what it is and what it is not are needed to differentiate from other forms of qualitative research methods.

Ethnography is Ethnography is NOT
A type of research that is qualitative, but does not preclude quantification or statistical analysis A synonym for qualitative research
Intensely and acutely empirical Particularly subjective
Counting of incidents of indicators observed that are relevant to the setting and topic of study
Observing behaviors, speech acts, interactions between people filling roles, information from docs produced by informants for their own purposes
Using surveys with questions developed from understanding the setting Using surveys based on others’ research
Utilizes quantitative data to substantiate claims about differences and distributions of behaviors or beliefs in the setting where ethnographer is studying. Based on general theory
Testing hypotheses about phenomena emerge in the field work Test testing hypotheses associated with an abstract theory

There are two ways the term ethnography is used: refers to a document produced following observation, as in a book or paper, and refers to empirical techniques used to collect data.  Those techniques include not only how the researcher collects the data but also the attitude of the ethnographer, the agenda, purpose, perspective of the research.

Ethnography documents situated patterns of action, capturing what people really do over a relatively long period of time through observations.  It is a study of meaning systems, how members of some groups make sense of their world and their behavior.  Informants allow an ethnographer to describe social world from the perspective of an insider and to develop grounded theories, based on observations of a set of behaviors.  Observing how people use a technology or product in daily life can lead to development of other products that fit users’ worlds.

Techniques to collect data can be open ended interviewing, ethnosemantic interviewing (eliciting linguist codes in a hierarchy), using field notes, tape recordings, video, audio tapes, and archives.  Gaining understanding of how insiders make sense of their world provides valuable information that overcomes the ignorance of expertise and authority that assumes an understanding in all aspects of a setting.

Ethnography is a way to build better empirical information about the everyday occurrences and organizational culture that is typically overlooked.  Discovery of the obvious should allow better design of technologies.

Thought provoking questions

  1. Given the one year time frame for studies in industry presented by the speaker, how can an ethnographer establish unbiased research attitudes that can be sustained throughout the project?
  2. How could quantitative data of differences and distributions of behaviors or beliefs in your work setting over an extended period of time inform your own professional practice?
  3. Technology development of products based on ethnographic data in industry leads to products that better fit the user’s world.  What are correlations of products that better fit the user’s world in the field of education?
  4. Compare intensely and acutely empirical data gathered from observing behaviors, speech acts, and interactions between people filling roles with empirical data collected from standardized English Language Arts assessments.
  5. In what components of your work might your expertise create an ignorance of what people are really doing everyday?
Advertisements

27 thoughts on “(Barley) For a Definition of What Ethnography is Not [Nan]

  1. In answer to question #2 – one of the ways this is used in my setting is through the collection of student voices. If we really listened and studied our system from the students’ point of view over time, I think we would be able to make significant changes not only to our teaching practices, but also to the practices surrounding the initial entrance to college, orientation, registering for classes, declaring a major, and taking classes. However, I find that we rarely ask for or study student behavior. We usually sit around a conference table and deliberate on what we think is best for students or “programs” that will make things better for them.

    • If you were to start a new trend in collecting student voices, how could you evaluate whether or not you have heard the voices of students who are not accustomed or prepared to advocate for themselves?

      • I’m not sure, Nan. One way we have started collecting student voices is to video tape focus groups. We start the conversation with an open ended question and then allow the students to discuss and answer with no interference from the “adult”. We have seen this technique to be valuable, but it does not meet the definition that he gave of ethnography. It is simply one qualitative method. I think to be a true ethnography, one would simply hang out where students are and observe their behavior while on the college campus.

      • This is an important issue, Nan. Some student populations are incredibly suspect of these activities (to collect student perspectives) and will not participate. In addition, how do we learn about students who are apathetic? The reasons for their apathy should be important in collecting student research.

    • I agree that many are adult-centered. It seems easy enough to ask students, but it’s so time consuming that adults/educators revert back to making decisions in isolation.

      • Yes, ignoring the student perspective and voice has been apparent in my professional experiences also. Yet I was still surprised when encountering this phenomenon when working on the International student project. Reflecting back, it was absurd for me to be surprised that the university had never before asked the students about their experiences.

  2. 1. The speaker believes much like the early scientists of our time that, “If a phenomenon has not been observed, than it cannot have been proved to happen.” Therefore: Participant-observing, Open ended interviewing and Ethnosemantic interviewing are types of fieldwork that can be utilized throughout a year long study to ensure the researchers can overcome the ignorance of expertise, and remove the likelihood of people assuming answers of the world without observing it with their own eyes. He also points out which tools can be useful in assisting a researcher during this process: Field notes, Tape recording, Video tapes, and Archives.
    2. Patterns will generally emerge over longer periods of time, which will often highlight the true nature of the reality without the biases of person beliefs, morals or researcher perspectives. These patterns can point a researcher in the direction in which to use qualitative research to describe the reasoning for the highlighted patterns.
    3. Textbooks are much like the cars the speaker discusses in the video… as soon as a car leaves the lot it instant depreciates in value. Textbooks in the same sense become obsolete the moment they are printed, as new ideas continuously emerge or are discovered. Technology allows educators to have access to the most current versions of the topic and can direct that knowledge straight to the students. This would be an example of how products better fit our changing world.
    4. The problem with any standardized test (especially ESL assessment) is that often the validity of these assessments can be in question from the start. Without an extensive study into what the true needs/accommodations of these populations, it is difficult to administer an assessment and say with complete certainty that the results are an accurate portrayal of the student’s ability.
    5. Often I struggle when writing Behavior Plans for students transitioning out of my school and into another setting. As required under IDEA, any student changing settings must have a 30 change of placement meeting to ensure all the services are being met. The biggest challenge I often face is creating a behavior plan that another educator is capable/willing to incorporate into their existing system. I can use all the expertise and knowledge I have developed over the years and create a plan in the best interest for the student, but some tools/methods are not replicable for educators in other settings due to lack of experience or knowledge of those methods used.

    • Although I concur that one of the challenges of BSP’s is getting the adults in the student’s life to follow it, their (teacher’s) compliance should not be the criterion for how we write one. I hear with some frequency that a student needs a BSP – as if a BSP is magic dust that will be sprinkled on their ill behaving student to fix him/her. One of the first things I say to teachers is that a BSP is not about correcting a student’s behavior, it is about all of the things the adults in the student’s life are going to do to make the student’s interfering behavior less likely to occur. There’s no two ways about it, BSP’s are a ton of work. Therefore, in response to this question, it is part of our job to follow up with a teacher or the new receiving school to see if the BSP is being implemented…. because, ignorance of what they are really doing cannot be an excuse when it comes to helping our students with significant emotional needs.

  3. Having never done long term qualitative research, I am so curious about what a researcher has to do to keep their own biases from creeping in over time. Have you ever been intensely involved in something for an extended period of time and stayed completely on the outside and remained objective?

    • I think that in the ethnography studies that he presented it wouldn’t be as difficult. As he presented it the researchers were not part of any discussions, but stood to the side and recorded what they saw & heard.

      I think that biases really creep in when you start doing interviews. I see this as being one of the issues that I run into while doing my research.

    • I just can’t see how one would remain completely objective when participating in long term qualitative research. I have worked and trained teachers and administrators in co-teaching for 13 years. I know my own biases when it comes to the subject and see that they color my involvement. I think you need to examine your biases so that you can recognize them for what they are. This helps the researcher to put those aside and be more objective. I do think that it is almost impossible to be completely objective.

  4. As a beginning researcher my experience is from k-12. Ongoing research through a long period of time is relative to what we do as administrators. Data collection, mapping and ongoing discussions of preferred outcomes provide resources to maitain objectivity and focus. Conversely, incorrect data gathering can lead to the wrong path which lends itself to ignorance.

    • Your work as an administrator is while you are already part of the system and you are performing many other duties. This researcher goes to observe at a company for an entire year, collecting data from all components of the system. He looks at very different things than we typically do in the educational system. What would happen if you were assigned to another school and all you did was record interactions between the administrators and students, the teachers and students, and between students for one year? How would that data be different than standardized test data and walk through data? The presenters supposition is that we are missing the most obvious data needed to inform our practice. I am so curious to see what unintended messages are sent through daily interactions. How are the things instituted to measure learning negatively impacting student learning? what are the messages one child hears from all sources in a year? Fascinating!

      • Great framing question. I can speak to that because I am currently in that role. For my own pedagoy and metacognition of management and instruction, I am the observer immersed closely in a system where I can be objective and reflective. I see the bigger and clearer picture while not being a threat to the current system. Hind site, practicioners in the leadership role can be persuaded easily to a certain direction that may not be conducent to their own thoughts and values. Your question is hits the mark as it validates my current role. The process, allows me to reflect on what I was before to what I am “Now” through my observations of data and interactions. Especially now that I am in an adminsistrator role where I can be objective and see the bigger picture. I do not report my observations per se to district leaders, the data I have gathered will benefit my next staff and system whole heartedly. It is interesting to see the intended versus lived interactions on a daily basis with several sites.

      • One thing that is beginning to happen in many schools in the central valley are instructional rounds. We delved into this a little bit in our class with Dr. Boris. I see this as a great way for teachers and administrators to go to other school sites and study a problem of practice and evaluate it from a neutral perspective. If this is done on an on-going basis it can lend itself to significant changes in practice. It also helps the people observing reflect on their own practices. The down side is that no one has the time or money to keep up the practice over an extended period.

  5. 1. I do not think it is truly possible; in a year long project, a researcher is going to build a number of views and biases. These can be controlled by fully acknowledging them, and being cognizant of them when interpreting data.

    2. Quantitative data can be used in a number of ways. As Trochim (2006) states, qualitative data can often be coded quantitatively; this can be analyzed to find patterns and differences.

    3. Ethonographic studies can look at the benefits and shortfalls of uses of various technologies in the classroom.

    4. Both involves a level of interpretation: empirical data from observations requires the observer to make observations, while data from standardizes assessments requires interpretation at the test creation and scoring level. Thus, there is still a level of ambiguity involved with quantitative analysis.

  6. How could quantitative data of differences and distributions of behaviors or beliefs in your work setting over an extended period of time inform your own professional practice?
    I immediately thought of suspension data when reading this question. There is a definite pattern in the number of suspensions of students across school years. I have began to utilize the differences in behaviors to plan interventions at specific times of the year prior to the spikes in behavior that warrant suspension in an effort to reduce the number of those behaviors.
    Technology development of products based on ethnographic data in industry leads to products that better fit the user’s world.  What are correlations of products that better fit the user’s world in the field of education?
    I would say the use of the electronic wireless tablet for math teachers that allows them to be mobile in the classroom while demonstrating sample problems.
    Compare intensely and acutely empirical data gathered from observing behaviors, speech acts, and interactions between people filling roles with empirical data collected from standardized English Language Arts assessments.
    The empirical data from stadardized assessments provides you with “if” or the “what” of student learning and the data from observing provides you with the “why” and “how.”

  7. Stephen Barley talks about overcoming the ignorance of expertise. Not everything fits in theory. In education, we base a lot on observation, such as walkthroughs and formal evaluations. Formal teacher evaluations may create an ignorance of what people are really doing every day. We sometimes base things simply on observation. There may be other useful (or not so useful) teaching techniques that a teacher may be using in and outside of the classroom. A teacher might be using the internet to reach out to their students that a 3-5 minute walkthrough does not allow for the observer to observe. Our perceived expertise might tell us that these walkthroughs are effective in showing us what goes on every day.

    • Sophia, I think you are right on in this area. Interestingly we will see this issue of “expertise” and it’s effect on teaching in the Heath brothers book “Switch”. They point out that teachers speak expertise, but student’s don’t. This is a very important issue in creating change and improved teaching practices. The more entrenched we are as a part of the system, the more “expert” we become and the more of an expert we become, the less aware we are of the true needs of those we are supposed to be educating. I think that if we are constantly aware of our own expertise and reflect on our practices that we can perhaps overcome this. I think in doing an ethnography, we could overcome the ignorance of expertise by being aware of it’s presence and actively pursuing practices that would overcome it.

      • I need to correct myself here….the issue of “expertise” with the Heath brothers is in their book, “Made to Stick” when they discuss the “Curse of Knowledge”. Sorry.

  8. Lori – I was trying to think about the “product” of a student who learns in comparison with a new version of a device. I couldn’t think of any particular examples of how education is delivered that has changed based on data of how students interact with the system. All the metrics I thought about were student outcome based, measuring compliance to standards rather than a measure of interaction. Using spikes in suspension data to guide planning is exactly the kind of information that seems to be lacking in assessing the teacher student interactions. Is there something in the teacher student interactions that we are not measuring yet?

  9. Several years ago, I attended the California Council for the Social Studies annual conference. During one of the breaks, I toured the social studies vendor fair. At the fair, a colleague and I were comparing texts from two competing vendors. A sales representative from one publishing company asked us several questions about the format. The rep specifically wanted to know what we liked about their text and what we did not like about it. I also remember my colleague stating that the competing company was far more advanced in highlighting the essential state standards on the side margins of the page and at the beginning and end of each chapter (Yes, this was some time ago). The rep quickly took notes and then stated that she would be sharing our responses –as well as those from others—with the chief editor(s) and marketing managers.

    On another occasion, at the annual Association of California School Administrators conference, representatives from Microsoft shared what was then the beta version of Microsoft Vista. They gave complimentary copies to those who sat through the presentation and who provided onsite feedback. Here too, the reps took notes on our responses to the features that Vista offered, as well as concerns that were expressed.

  10. One way Barley defines ethnography is by the techniques the researcher uses to collect the data. “Those techniques include not only how the researcher collects the data but also the attitude of the ethnographer, the agenda, purpose, perspective of the research.”

    I would not call what we do at my district “qualitative research” (although that is what we are trying to achieve). In fact, now that I have read this information, I realize how little I know about true qualitative research. When we “survey” students, we (professionals at my community college) do so with a strong set of ideas already in mind… we automatically know what they are going to present (who knows better than faculty what students are about??) and when we get responses that don’t fit our preconceived ideas, we toss them aside as “outliers” or we tell ourselves that students do not know what they think or want.

  11. 1, Given the one year time frame for studies in industry presented by the speaker, how can an ethnographer establish unbiased research attitudes that can be sustained throughout the project? — To meet Barley’s requirement of “intense objectivity and acute empiricism,” an ethnographer should create a list of study objectives, and establish a set of activities, behaviors, and other sorts of professional practices that are measurable. This will ensure that all activities are consistent and implemented with fidelity. Additionally, feedback from a non-interested 3rd party should be obtained.

    2. How could quantitative data of differences and distributions of behaviors or beliefs in your work setting over an extended period of time inform your own professional practice? — Very few activities in my work setting are quantified in any way. That data which is collected is done so for the purposes of an accreditation self-study, and even then, those numbers appear to have little utility when informing daily practices. There is a currently some discussion about moving in that direction, though, to tying individual performance evaluation to profit generating activities.

    3. Technology development of products based on ethnographic data in industry leads to products that better fit the user’s world. What are correlations of products that better fit the user’s world in the field of education? — Targeted interventions to specific populations, services relating to first-semester intervention (as well as “first year” and “sophomore” retention). There exists some controversy about identifying populations and tagging them as ideal for certain products, but the fact remains that this is a win-win for both the institution as well as the the individual student.

    4. Compare intensely and acutely empirical data gathered from observing behaviors, speech acts, and interactions between people filling roles with empirical data collected from standardized English Language Arts assessments. — This is the essential distinction about the values and uses of qualitative vs. quantitative types of data. How do we make decisions that will affect a large population from a focus group with a small sample? In the absence of other information, how do we make decisions that will affect a small, unique subpopulation based on findings that apply generally? There is no way to easily answer these kinds of questions, which are largely context specific.

    5. In what components of your work might your expertise create an ignorance of what people are really doing everyday? — The use of clearly defined “duties” and the creation of fiefdoms around the boundaries of those actions puts us into positions of studied ignorance. While, on the one hand, this allows us as individuals to focus on specific areas and gain expertise, on the other it pushes us into a situation where the function of our organization then becomes a mystery.

  12. In looking at technology that better fits the user’s world, I think about educational data systems such as edusoft and illuminate. In order to determine which product to go to from edusoft, the people developing illuminate conducted focus groups and used pilot groups to listen to feedback in order to make improvements in their product. I see updates constantly that are based on user feedback. Couple this with the advances in scanning equipment, and it is so helpful for teachers and administrators to be able to scan assessments into illuminate in order to analyze the data. This is helping us inform our instruction in a more timely manner.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s