Saturday, May 13, 2006

Thinking about analysis and empiricism

I am getting ready to work on the chapter that is supposed to be somewhat empirical in nature. I had initially decided (and my advisors agreed) on doing a case study or two. Then I started looking at all the data I had--hundreds of blog posts, rough drafts, final papers, interviews, self evaluations--and I thought, surely there's more I can do with this. So I sounded out the empiricist on my committee. The problem is, I'm not an empiricist. The methods that he suggested I am unfamiliar with and I'm afraid I'd misapply them. Further, I'm skeptical of an empirical approach to teaching writing. I don't think, even if I had designed an experiment appropriately, that I could prove that blogging improves students' writing skills. There just isn't any objective concept of good writing skills. Yes, there are schemas that one could use, holistic scoring, primary trait analysis and the like, but I don't buy any of them. I think that my reason for rejecting some of these approaches is that they articulate an outcome that I just wasn't looking for from my students, so it doesn't make sense to apply them. In a way, I was an outlyer among the other faculty teaching the course. They all complained about the lack of theses and topic sentences while I was excited about the ideas my students were presenting and didn't care so much about a thesis as long as the central idea was clear.

So, I'm back to a case study approach. It is suggestive and descriptive rather than empirical and that may be problematic to some, but it just feels like the right way to go. I definitely think that one could apply something more empirical to the data I have, but right now I don't have the time to figure out what that approach might be and it's not where my research is. So the potential for future research is huge, which I think is positive. I just hope it won't be held against me that I can't include that approach in this work.


Cats & Dogma said...

I don't know if this will get you where you want, and it may mean doing too much ground work just to begin a case study, but I wonder if the concept mapping that Bob Broad discusses in What We Really Value might help. As a member of the assessment committee in the last writing program I worked in, we thought about what Broad was doing as a way of ferreting out what we already value when we teach writing, and it may be a way for you to develop a system of measuring the progress in the writing that you've already elicited from students.

I feel your frustration, though. But as much as I resist the notion, it is a political reality of writing pedagogy that quantifiable assessment is a major element driving how administrations view our work. I guess my take on it is that I'd rather be able to design a better assessment, even if the mere idea of a perfect empirical assessment doesn't hold water. Anyway, Having just come over from another of your blogs, I'm happy to have discovered your work here. I'll be sure to keep an eye on it!

Steve said...

I think the key solution to your short term problem--getting the dissertation done--is to consult with your dissertation committee. If they're okay with case studies then you are good to go. You can save the more complete treatment for future research.