Photovoice

We had a great guest presentation on the use of photovoice in an ongoing study. You can see the slides in PDF format.

Katie is just one of the generous visitors to our seminar. I am happy to report the LIS 3600 spring 2006 PhD Seminar in Qualitative Research Methods has been truly blessed with a steady supply of interesting, talented, and very edgy (as in cutting edge) guest speakers. My deep thanks on behalf of the class go out publicly to Rebekah Hamilton, Gwendolyn Hawk, Charli Carpenter, Sue Sterret, Katie Fitzgerald and Eva Marie Shiver, who who have so far given us incredibly valuable insights into their projects, and to Carrie Farmer Teh and Michelle Upvall, who are lined up for upcoming classes.

It is my sense these presenters have gotten something valuable out of the class feedback. The sessions have more closely resembled brainstorming sessions than traditional academic chalk talks. The students have loved it; it spares them having to hear too much from me.

Getting Started with ATLAS.ti

There are a number of usefull resources available at the ATLAS.ti web site. For example, you can download a free trial version or a quick start guide.

For specific answers to questions about using ATLAS, try searching their online forum or joining their listserv.

Available Data

For those in the class who are having a tough time finding data to use in the lab tomorrow, I have posted two of my public comment samples as .zip archives. You can get a saple of mercury emails or CAFE (corporate average fule economy) public comments at

http://qdap.ucsur.pitt.edu/data/cafe.zip
(CAFE comments)

http://qdap.ucsur.pitt.edu/data/ntf.zip
(mercury comments)

Is Knowing "A Lot" a Downside?

I was running my eyes over the Executive Summary of the report from a National Science Foundation-funded "Workshop on the Scientific Foundations of Qualitative Research" and found this interesting quote:

"The cornerstone of good qualitative research is in-depth knowledge. Qualitative researchers who already have background knowledge are more likely to identify promising leads than those who start from scratch. The downside of 'knowing a lot' at the start is that researchers may enter the field or archive with preconceptions that interfere with the development of new insights."

This generalization left me puzzled. What is the logical strategy in light of such a paradox? It is certainly true that familiarity with a phenomenon is likely to allow a researcher to situate specific obseravtions, or groups of observations, in a more theoretically rich and informed context. At the same time, this pre-knowledge that frames an observation will also likely push out some competing frames that either are not favored or have not been developed yet.

I notice this issue when reviewing the results of coders working in my QDAP lab. In some cases, the students with a disciplinary background in line with the coding project seem better able to distinguish subtle differences in the text and therefore make observations that are consistent with the principal investigators goals and instructions. In other cases, however, the PI will want to have a mix of students with no specific knowledge about the research domain so that they can report on the "unanticipated" observations, those untethered to any particular analytical frame.

Perhaps it boils down to this. In projects that are essentially content analysis exercises driven by the knowledgeable presuppostitions of the PI, coders are best suited when they know some of the theory and jargon of the PI's discipline. When the project employs a grounded theory approach, the need for "knowing a lot" about the subject may be diminished.

Validity and Circumstances

"Validity is not a commodity that can be purchased with techniques...Rather, validity is like integrity, character, and quality, to be assessed relative to purposes and circumstances"

(Brinberg & McGrath, quoted in Miles & Huberman, p. 39)

I liked this quote; it seemed somehow reassuring. Our quick ramble through various approaches to validity in qualitative research left me feeling like it was a hodge podge of or wish list of "ideal types" that were each in their own way trying to define a multi-dimensional, historically contingent, ideosyncratic process as a codifiable regime of trustworthy inferences. That's a tall order.

The typologies of validity are useful, however, foremost for directing our attention to the many ways in which our research can fall short in one or more dimensions. Thinking through the details of what makes for authetic findings is critical to the research enterprise. Valid inferences are the goal, but the pitfalls loom everywhere (errors of omission or characterization, for example).

I do wonder about the refrain that holds qualitative studies are doomed to have limited generalizability. This overstates the case, don't you think? What kinds of qualitative studies might result in findings that are generalizable to a wider popluation?

KKV: "The procedures are public"

"We seek not dogma, but disciplined thought."
(KKV, Designing Social Inquiry, p.7)

Item 2, "The procedures are public," in the 4 point list in section 1.1.2 states concisely one of the central pillars of scientific inquiry. Whether it accurately describes how scientists do their work varies from case to case. As we explore what it means to be a good qualitative scientist, it will be important to discipline our thought by constantly asking exactly how far this maxim can be taken. KKV are correct that "investigators often take down the scaffolding after putting up their intellectual buildings," and I believe I can personally be more fluent with qualitative methods if I attend to the full and orderly preservation of just enough of the scaffolding to make my inferences more compelling to a wider audience. The tough question is: how much is enough?

Where practical, I have tried to make the data on which my major e-rulemaking stakeholder report and other scholarly papers are based available. To a minimal extent, the procedures used to reach inferences found in my work have been described in turgid terms, usually reflecting an over-arching tension that my methods are never going to be good enough for some in my profession or even defensible enough to merit the best of my scholarly prose.

But should I do more? I teach my coders and clients working with QDAP to memo extensively about the process and struggle of coming to grips with a qualitative project. My own memoing has always been rather limited nonetheless and I sometimes find it tough to read all the memos on process that a single project generates. It is even more difficult to know exactly how to incorporate that "procedure" in a public manner.

One idea gaining currency is to share not only the primary data with other researchers but also the completed coding and entire memo trail. One can imagine posting archives of coded data for other researchers to examine or attempt to replicate. I wonder: Would they do so?