So today I’m getting topical and taking on teaching evaluations. It’s in the air, and over the past week my colleagues and I have poured over student feedback from our various classes. Unfortunately, not all of that feedback is always of the same caliber, and some comments are more helpful than others. Consider the following, which are actual transcriptions of comments that I’ve received over the years:
“She talks in a belittling manner in lecture to her students. Or maybe I just dislike her voice”
— Duly noted. I will look into voice modification software
— How… I just… really…? Did you actually write that?
“What I cannot comprehend is why in this course, and the other one I took with you, while studying such interesting material you pick the most god-awful and brutally boring essay topics. Sorry for the harsh words, but it’s true”
— Apparently I have bad taste historical documents. Also noted.
“Great fashion sense”
— But great taste in clothes?
“I have no suggestions for improvement which I feel I am at liberty to make”
— This message will self-destruct in 30 seconds.
Let me start by saying I was horribly unsure if I should publish actual student comments when I began writing this post. This is not because I’m self-conscious about the criticisms; in fact, I have a horribly sarcastic (some would say twisted) sense of humour, and I find this sort of thing incredibly amusing. It’s because I have a deep respect for each and every one of my students and I never want them to feel like they are stuck in some sort of unhealthy relationship and can’t approach me with concerns, or make mistakes without being made fun of. In the end, however, I decided this post was about mocking myself and not about mocking them, so it was ok. I also decided that it might be good for students to run across this post because they can sometimes forget that professors are mere mortals as well. If the situation were reversed, and a professor said something like that to a student, said student would be horrified. Not all my colleagues, after all, share my snarky sense of humour when it comes to evals, and I’ve seen more than one evaluation-induced fit of self-doubt prompted by these “helpful observations.”
And yet, bizarre comments like the ones I listed above are actually the ones which are easier to deal with! You might be temporarily shocked when a student declares that you should be taken out behind the shed and shot to put the rest of humanity out of its misery, or temporarily elated when you receive that marriage proposal, but everyone knows that such hyperbole is best ignored. Leaving aside the gendered nature of a lot of these comments (that’s a separate issue for another day), the really troubling problem is that for every student who says “great Powerpoint slides,” you get another one who says “I thought these were the worst slides I’d ever seen.” Likewise, half your students will love the readings and think they’re the right length, the other half will hate them and think the workload is excessive. Some will want more weight on exams, while the rest (and this one never ceases to amaze me) want you to increase the number or value of tests. Really, I swear, they actually say this. I could not make this stuff up.
Begin rant (apologies to those readers who disapprove of my superhuman capacity for rage – SHEHULK SMASH). While preparing this post, I decided to do a little research, and see what sorts of resources were already available to people stymied by their teaching evaluations. In the process, I discovered that McGill – the institution with which I am currently affiliated – has a 40-page guide to interpreting evaluations. FORTY PAGES! In what universe is this helpful?!?! And believe it or not, it actually gets even more delicious. They provide the following chart to help professors sort through their comments:
Now, I had 29 typed pages of student comments this year. Do they really think I’m going to tally everything up and fill out a chart!??! It took me a week just to find the time to read them all. Not to mention their categories had me in stitches. That’s right, dear reader, I actually LOL-ed. Here’s why. Sample positive comment: “This is the best course ever!!! I love the Prof so much!” Sample Negative Comment: “Her voice annoys me and she’s a disorganized cow.” Right about now, you’re realizing why I’m annoyed. Not only do I not have time to count all these things in the first place, but there are shades of positive and negative comments, some more helpful than others. Is “the lectures were clearly organized” really in the same category as “this was the most amazing class ever”? What about “the course was better than most, but I found that the professors spoke too quickly at times”? For that matter, does that last one go in the negative or the positive count? What does my esteemed institution have to say about this problem, you ask? Not a whole heck of a lot.
I do need to acknowledge that in several places throughout the weighty tome, McGill does state that some comments can be exuberant and some are even hurtful, and they encourage you to talk it out with a “trusted colleague.” Presumably, this will help to avoid destructive patterns of self-loathing and to salvage any useful criticism from the dungheap, but the specifics of the procedure remain unclear. The document also provides helpful stats for determining if enough of your students actually filled out the evaluation to give you a useful sampling. This is because at McGill, like many other North American institutions, we’ve gone to online evaluations, making it harder to coax students into filling them out because they are time consuming and are usually done outside of class. If your class is under 30 students, McGill therefore suggests you should aim for a 40% response rate. If you class is 150 students (like mine), you should aim for a 30% response rate. That’s right, 30%! Aim high people. On the bright side, this explains why they think I have the time and inclination to fill out the chart.
OK, rant done, I promise. Now for the constructive part. If you’re not going to read this absolutely gripping 40-page manual, here are some more accessible tips from yours truly. First of all, perhaps I’m just a perfectionist, but I’d aim for a higher than 30% response rate. This is mostly for those of you also struggling with online evals, but it does pertain to some extent to the old paper system as well. Get your response rate up by announcing to students, in advance, when the online evaluation period begins. Better yet, since students now generally have laptops, and most classrooms are wifi-enabled, give them time in class to fill out the survey – this works even better if you give them enough time for those without a computer to share with a friend. Also, use class list-serves to circulate the link to the evaluation site and to advertise an in-class evaluation day. Finally, make it clear that you actually care, and will actually read and consider the feedback – especially the comments. These type of tactics will dramatically improve your response rate; I had 75% of my class take the time to fill out the survey, and many of them left comments. This is by no means an exhaustive list of strategies. I know profs who give bonus marks to the class for every 5% the response rate is over the 70% mark, and I even have a colleague who drinks a shot of espresso for every 5% they make it up the response-rate scale at the end of the course. Personally, I no longer have the intestinal tract for the 14 espressos he consumed this year, but it seems to work for him.
As for interpreting the data, let’s start with comments. The first thing you need to do is jettison the outliers. Your greatest fans and those who would happily dance on your grave both need to go, unless they appear in such overwhelming numbers that you can actually see a pattern. Even then, I’d be leery. What you’re left with, should be a much more grounded sampling of the pros and cons of your course. Respond to these as best you can. For example, my biggest flaw is that I talk too fast, and my greatest strength seems to be organization. I keep working on slowing it down, but these things are easier said than done. I am making some progress through, as the frantic tone discernible in early-career evaluations has now lessened to the annoyed but largely blasé concern about pace which I encountered in my most recent feedback.
Other comments, however, are harder to respond to. This is especially true of those things which I do that my students don’t like, but which I feel are important and necessary. By this I mean those pedagogically sound but somewhat distasteful things we all do that students just don’t appreciate. In my classes, this often has to do with the value at which I weight seminars. I feel it’s absolutely fundamental to have a high participation mark, while student think it’s absurd, tyrannical, and even unethical. This is something I am strongly committed to and so, instead of changing it, I have responded to the issue by having a conversation with new classes about why I weight seminars the way that I do, the value that they have for a person’s education, and the fact that it levels the playing field because not all students do well in exams. So to sum up, I either adapt, or explain.
Moving on to the numeric portion. The McGill handbook says that when you look at your numeric results, if the breakdown in each section is polarized as opposed to sloped, then you have a problem. I’m not entirely convinced this is always the case. For some categories, like “overall, this is an excellent course,” that may be true. But if your institution has a numerically evaluated section for something like “the readings were interesting,” it’s much more likely to be split. I have no good evidence for this, or theoretical justification. It’s merely been my observation when I’ve polled smaller classes informally that the answer to “what is your favourite book” and “what was your least favourite book” tend to turn up the same damn book. Which brings me to my last useful suggestion: if you work at an institution that doesn’t leave space for comments, or you want feedback before the course is over, offer an informal evaluation at the appropriate time (early if you’re unsure about how to progress, later if you just want more extensive feedback). This has to be voluntary and anonymous, but I’ve found that it can be extremely helpful, and there are various guides online to help you draft it. Many universities and colleges also have a teaching center that will guide you through the process.
I’ll end off, then, with some of the more helpful links that I turned up while researching for this post. Since several of my students have suggested I try out a faux British accent, pretend I have a thick Scottish lilt or perhaps that I’m dressed as Eliza Doolittle, while reading out the following information: