Is Knowing "A Lot" a Downside?
I was running my eyes over the Executive Summary of the report from a National Science Foundation-funded "Workshop on the Scientific Foundations of Qualitative Research" and found this interesting quote:
"The cornerstone of good qualitative research is in-depth knowledge. Qualitative researchers who already have background knowledge are more likely to identify promising leads than those who start from scratch. The downside of 'knowing a lot' at the start is that researchers may enter the field or archive with preconceptions that interfere with the development of new insights."
This generalization left me puzzled. What is the logical strategy in light of such a paradox? It is certainly true that familiarity with a phenomenon is likely to allow a researcher to situate specific obseravtions, or groups of observations, in a more theoretically rich and informed context. At the same time, this pre-knowledge that frames an observation will also likely push out some competing frames that either are not favored or have not been developed yet.
I notice this issue when reviewing the results of coders working in my QDAP lab. In some cases, the students with a disciplinary background in line with the coding project seem better able to distinguish subtle differences in the text and therefore make observations that are consistent with the principal investigators goals and instructions. In other cases, however, the PI will want to have a mix of students with no specific knowledge about the research domain so that they can report on the "unanticipated" observations, those untethered to any particular analytical frame.
Perhaps it boils down to this. In projects that are essentially content analysis exercises driven by the knowledgeable presuppostitions of the PI, coders are best suited when they know some of the theory and jargon of the PI's discipline. When the project employs a grounded theory approach, the need for "knowing a lot" about the subject may be diminished.
posted by Stu @ 6:31 AM
1 comments
1 Comments:
That is an interesting paradox -- "How much is enough?" and "When do you have too much?"
My knee-jerk reaction to the quote is that this is where the quantitative-oriented researcher will jump all over the qualitative researcher and pointedly note that quantitative methods have perhaps "solved" the problem of introducing bias into any results. And, because qualitative methods don't explicitly try to exclude the extraneous variables, they must be "inferior" methods.
Well, I done enough reading about qualitative and quantitative methods to believe that such an extreme position is nonsense. I also belive that quantitative researchers can take the good things about those methods and carry them too far as well. You can program in so many controls to eliminate bias that you can loose any insight or significance that your study might produce.
Qualitiative researchers need to meet this issue head on and deal with it as directly and completely as possible. The researcher should spend as much time and energy as necessary to identify possible biases and address them.
What I'm beginning to see is that a single qualitative researcher may not be able to handle all of this completely alone. Perhaps we need to look at all qualitative research in much more collaborative terms. The observation about the coders seeing different things in the data is significant. Different sets of eyes will see things differently, not just because they are different people, but because they have different experiences that inform their coding. Depending on the study, this could be something to be embraced and will help help make the results richer and more significant. Different eyes should not only make independent observations but it probably wouldn't be a bad idea to actively discuss and even challenge observations. Such a dialog could also serve to identify potential biases of the individuals and, if not eliminate them, at least give them a proper airing so that the researcher(s) can address them.
Post a Comment
<< Home