Evaluating evaluations

In my last job, student were asked to evaluate instructors in two ways: first, by choosing a number that ranked the instructor based on questions provided ("On a scale of one to five, how would you rank the intellectual challenge of this class?") and second, by developing on those numbers in written comments. The questions were at the top of the page, giving students ample space to reflect, criticize, and judge. Some students merely write down numbers, but most of them actually wrote several sentences about the course and the instructor (me).

As nerve-racking as it always was to get copies of those evaluations and pull them out for a read, it paled in comparison to the experience at my new job. When I got ready to pass these evals out to my students, I noticed that they looked like a scan-tron test-- lots of bubbles and almost no place to actually write. And when I got a chance to look at the results, I got several pages of bar graphs and statistics that told me how my averages related to departmental averages, averages for similar courses, and a whole lot of other confusing crap. What I did not feel I got was a sense of how my students felt about my courses.

I have very conflicting feelings about course evaluations and their usefulness. I absolutely think that reflecting on a course and its usefulness is a valuable exercise for all students. I also think that any input an instructor can get about the course and her methodology is equally invaluable. Unfortunately I don't see course evaluations in their current form as helpful to either students or faculty. My older evals are rife with conflicting opinions ("She's very organized and gave great comments," "She's disorganized and doesn't really comment on papers"), nonsensical comments about other class members, and the all-too obvious diatribe written by the kid who clearly felt entitled to an 'A' from the beginning. While those evals were actually pretty good at pointing out my strengths as a teacher, they were less successful at helping me identify and correct the flaws in my pedagogy.

Unsurprisingly, these new evals are even more useless and frightening than the old ones. If I felt that I could once assess the value of each evaluation based on the comments, these new evals simply present a sea of numbers. Almost none of the students wrote down comments, so I can no longer distinguish between the student with a genuine problem with my pedagogy and a student who simply disliked me, the subject, or his classmates. As confusing as the older evals could be, at least I got a sense of what issues students had. Now, I have no idea. Clearly this kind of evaluation isn't about helping the students or the instructor, and I really resent the fact that my teaching ultimately boils down to nothing but a couple of averages.


Postscript: The obvious solution to this issue is, of course, to not read those damn evals. But I can never help myself. I just wish I could go back to the pre-scantron days.