Monthly Archives: July 2011

Peer review – painful or practical?

Posted by: Dr Karen Cullen Dr Karen Cullen is part of the L+T team in the School of Media and Communication at RMIT, Melbourne. A new arrival in Melbourne, she was previously a research active historian teaching in Scottish universities primarily utilising online and blended learning techniques.

Copyright © RMIT University. Photographer: Margund Sallowksy.

A few years ago I took part in an exercise which involved a peer review of my teaching. Upon learning of this requirement, indignation immediately set in. Peer review? Me? It was one thing to undergo peer reivew as a teaching assistant when I was a PhD student, but now I was a fully fledged faculty member!

As a postgraduate student I had felt reasonably comfortable undergoing this level of scrutiny, but I always connected it with my position as a junior member of staff. I didn’t feel awkward about this, but tended to view it as a ‘check-box’ exercise. The review was undertaken to ensure that my teaching was satisfactory, then everything carried on as before. I never experienced a bad review, so didn’t even have to consider what I would do to remedy such an outcome. I neither placed any great importance on the event, nor particularly valued its outcomes.

So why was I so bothered about it now that I was more experienced? As a research active academic peer review is part and parcel of writing articles for journals, giving papers at conferences etc. But that is research I thought, the same doesn’t apply to teaching ………………or does it? Don’t students evaluate our teaching effectiveness on a regular basis through surveys, through staff/student committees and informal feedback?

Having talked myself round to thinking that perhaps peer review of my teaching wasn’t such an alien concept after all, I decided to ‘get it over with’. Split into pairs we had to attend three of each other’s classes and discuss our thoughts and ideas afterwards. This observation experience opened up a new meaning of peer review. It was much more balanced – we each played both the role of the reviewer and the reviewed. Rather than receiving feedback from a senior member of staff, the process provided me with an avenue to critique my own teaching, by comparing my skills and methods with those of another tutor. I considered the way in which my partner taught, why he chose a particular approach and what examples of best practice I thought that I could absorb from him. I no longer felt like the junior member of staff whose teaching was being scrutinised by a senior academic for ‘approval’.

Peer review of publications is not something that disappears once an academic is considered an established expert in their field. Why should peer review of teaching be confined largely to observation of junior staff? Review of written work often involves discussion and debate – that is what I concluded I wanted from peer and student review of my teaching.

I have to say that the observation exercise threw up as many questions as answers, challenging me to reflect on my practice by looking at it through the eyes of someone who understand what I am trying to achieve. This experience has not only provoked me to think differently about my teaching but also to consider peer review in more positive light.

Large classes and student interaction – it can happen!

Post by Felicity Prentice

I am not sure what your view of CATs are – I’ve never seen the musical, but I own a 20 year old moggie who rules the house with an iron paw. But I digress, let me start with a confession.

I used to love large class teaching. I loved the sheer performance of it, strutting up and down, holding forth, having ultimate control over the PowerPoint presentation as it slowly and surely clicked its way through 50 slides of dubious quality. Appallingly teacher centric, I know, but so very safe and comfortable. I didn’t know if the students were following me, let alone travelling along side me. So what broke the spell?

Reality usually rears its ugly head at two points. First – your assessment of them:

“These students know nothing! I taught them, why didn’t they learn?”

And then, their assessment of you:

“The CES scores are low, but that is because they were a bad bunch of students, they never did the work I expected of them, and now they are taking it out on me”.

I am sure you have never been as vigorously delusional as I have. Perhaps I should take a look at the situation:

• In large classes students can feel anonymous and voiceless. The threat of exposing their ignorance is often sufficient to keep their heads down (and focussed on some serious texting). (Carnegie Mellon has some ideas for this)

• Teachers in large classes often feel compelled to focus on content delivery. As a solo performance, the emphasis is shifted to the ‘knowledge’ rather than the understanding, evaluation and synthesis (thankyou Bloom, Krathwohl and Anderson).

• First year students can be overwhelmed by the apparent lack of organisation of the lecture environment (who is telling them what to learn, how to learn and when to learn?). The volume of information can often drown out the key ideas and concepts you are trying to reveal.

• We need to offer students an anonymous voice, a chance to tell us; what is muddy, unclear, an unanswered question, or even a quick view of what they believe are the most important things they have learned. We can learn from this, adjust our approach, respond to their feedback – before it is too late.

So, let’s try CATs, or Classroom Assessment Tasks (thanks Angelo and Cross). These are simple, quick, non-graded, anonymous tasks carried out in class, lasting about 2 to 3 minutes. Students are asked to pause, reflect and write down their response to an enquiry about their learning. This can be a response to a variety of prompts, such as:

• What is the most important thing you have learned today?
• What is the muddiest point (the idea that is least clear to you)?
• Write a short 3 sentence summary of [key concept].
• Write a 5 mark exam question based on today’s work.
• Identify the three most important concepts from today’s lecture.
• What information from last week have you used today?
• How could this class be improved?

The pause gives students a chance to switch from listening to thinking, reflecting and responding. The notes (and it is easy to hand out index cards) get passed to the front, and you can scan through them at your leisure to get a snapshot of what is happening in that amorphous body of eager students. Most importantly, at the next opportunity, you respond to the students:

“I had a look at your responses, and I think we might need to review [concept].”
“It was great to find out that you are understanding [concept], but I also hear that you need more time to get a grasp of [concept].

The students have a voice, you have some insight, a feedback loop has been established. You don’t need to do a CAT every time, just dip in and out, let the students know you care about listening to them (as much as I care about listening to my own voice).

And, if you are feeling really bold – try it with Personal Response Systems (“Clickers”). Put up some multiple choice questions, find out what they know and don’t know. Be even bolder, ask them to tell you how the classes are going, what you can do to improve. And if you use the feedback from student survey questions you might have a chance to celebrate, and/or address, issues before the end is nigh.

Want to know more?
Five Classroom Assessment Techniques: A Handy Handbook
An Introduction to Classroom Assessment Techniques by Diane M. Enerson, Kathryn M. Plank, and R. Neill Johnson
Using Classroom Assessment to Change Both Teaching and Learning by Mimi Steadman

Silence and giving students space to think

Thomas Struth, Hermitage 1, St. Petersburg, 2005

Image from mariangoodman.com

Post by Angela Clarke

As the teaching and learning advisor for the School of Art, over time I have observed many peer feedback sessions. What I have noticed is that often students are not consistently good at giving constructive and useful feedback to their peers and even when they do provide insightful comments they often do not go deep enough.

However, in one session I saw last year, I observed students offering extended and unusual feedback to one another. It was a painting studio, five students had displayed their work around the room, the class (approx. 20 students) and two lecturers were there to give formative feedback (ie they were not being assessed). The lecturers began the session by asking students to spend 10 mins observing the works – this was done in silence. Students spent this time slowly milling around the works moving in for closer inspection and standing back for broader perspectives. Something about the extended silence and slow movement evoked a progressively deeper sense of focus in the room.

The lecturers then called on particular students (not the artists) to discuss each work. During this process the lecturers also allowed a great deal of silence and the students seemed to be relatively comfortable with it. Occasionally, after some time had passed and nothing had been said, one of the lecturers would gently provide a prompt like ‘tell me about the vanishing point…’ or ‘describe what you are looking at…’ or ‘talk about the materials used in this piece….’. Once the students got started they offered each other sensitive and thought provoking comments in an almost stream of consciousness way such as ‘objects in space, paint on canvas….take my attention to…. slowing down my thoughts…only looking at what’s there I see….the artist draws my attention to…’. These comments provided a rich source of material, future directions and possibilities to the artist’s whose work was on display.

What is noteworthy about this approach is the lecturers’ use of gentle suggestions that emerge out of silence rather than direct questions such as ‘where is the vanishing point?’ It seems the lecturers were creating a learning environment that actively allowed for slow inner processes to emerge. It reminded me of an article by Anna Weiser Cornell called ‘Questioning Questions’. When working with people to facilitate an inner experience Cornell prefers to use suggestions rather than questions. Cornell finds that questions can be abrupt and can interrupt a person’s inner process because “they place the questioner in a subtly one-up position. They narrow the options of the questioned to those defined by the questioner” (Cornell, 2001, p.3). Cornell points out that even not answering the question is not really a choice as this will be socially interpreted as a refusal or an inability to answer. In a learning context both these responses are generally perceived as unacceptable.

Cornell goes further and argues that questions actually demand fast answers and therefore people tend to answer them using head-processes where ready-made answers are stored. I would argue that responding to art works is an inner experience which requires a quiet attentiveness to a body-process that does not yield immediate results. The body often has intuitive responses to things and accessing this kind of information is slower and less definitive particularly because visceral responses do not immediately have words attached to them. The tone used by the lecturers suggested that they had no doubt the students were able to respond to their prompts. I witnessed the lecturers be present and comfortable with silence, which in turn helped the students take the time they needed to articulate their response – a rare gift in our fast-paced, instant-gratification culture.

Subsequent to this classroom event, I worked with the two lectures to see if we could further enhance this teaching technique. As a result we developed a tutorial that encourages students to actively work in this way. We are trialling this in a peer learning project this coming semester. I’ll keep you posted.

*Thanks to Dr Robin Kingston and Rhett D’Costa and their students for allowing me to observe their class and for the subsequent work we have done together.

there’s something in my inbox

Post by Ruth Moeller.


Sitting in my inbox is an email from the university’s survey centre. It’s been there for a week. I am working up to opening it. I know what it contains; it’s the results of the Course Experience Survey (CES) for the subject I teach.

CES (HE & VET Vocational Education & Training) is the official feedback survey from my students, comprising the Good Teaching Scale (GTS) and Overall Student Satisfaction (OSI). The six questions that make up the GTS really cover the aspects of teaching practice as they ask about: motivating, commenting on work, understanding student difficulties, giving feedback, interesting delivery and clearly explaining content. On reflection, all of these are fundamental to quality teaching.

On a general level, the CES provides us with a statistical snapshot of the student view of their subject, it is a useful tool to review what is working and what could be worked on – I know that the stats can be cold and there is an opportunity for comments, but it’s a clear starting point for discussion and reflection. In fact, when discussing the CES with other teaching staff, I encourage them to work with it rather than rail against it, though there can be something satisfying and cathartic about bagging the instrument and its calculating manner.

Each time I receive my CES feedback, I find it thrilling and depressing in equal measure. Thrilling, as it is a way of finding out if the students’ experience was what I hoped it would be. And depressing as it is such a cold measure, a percentage that represents a semester’s teaching (work).

A common frustration I hear from colleagues is that although they have given plenty of feedback on students’ work, when it comes to the GTS, they are rated as not providing enough. As one colleague said, “we discuss all their ideas, do pin-ups where we have peer and teacher input and at the end we do an overall critique. What more do they want?” Potentially two things are at play here, quantity of feedback not equating with quality, and/or students being asked about ‘feedback’ doesn’t automatically equate with the language of ‘discussion, pin-ups and critiques’.

I know there will be people out there thinking, “surely they know what feedback is” but if you think about it, for us, the GTS has great meaning and purpose (both for better & worse), whereas for many students, it can be seen as an impost. If they cannot link their experience with what is being asked in the survey, there would be a disconnect. With this in mind, the practical advice I can offer is, if it looks like feedback, sounds like feedback, then call it feedback- educate them in the discourse (language) that is important. You can take this a step further by linking the language of the discipline’s discourse, with the language of education discourse; “We are going to conduct critiques, where you will be giving and receiving feedback ……”.

GTS irritations come in many ways; a colleague tells of being rated 44% for the statement “The staff made a real effort to understand the difficulties I might be having with my work.” When he reviewed the stats, the students either rated him as positive (44%) or not applicable (53%) – he has drawn the conclusion that 53% had no issues and accurately responded to the question; ie. they didn’t hypothesis what he would do. Understandably frustrated by this as it had a negative impact on his overall GTS, he has employed the tactic of overtly checking in with his students at key points in the semester; seeing if they are having any difficulties and if they do, asking if he can help. Now some may say that this is manipulative, but others, including myself say it is making the implicit, explicit. He has always been a thoughtful and caring teacher, and is now making sure that his students are reminded of that, with the side benefit being if a student was unsure about whether they should approach him about an issue, they have an explicit invitations to do so.

Teaching to the GTS could be seen as calculating but for me it is good professional practice. If, when teaching I address all aspects of the GTS, I would by definition be a ‘good’ teacher, and by ensuring that the language I use reflects the language of the GTS, this gives students the fair opportunity to reflect and give feedback on my practice in an informed way.

All that being said, the email is still unopened. The weekend is coming up; I will take a quiet moment and open it then.

Follow

Get every new post delivered to your Inbox.

Join 1,413 other followers

%d bloggers like this: