Monthly Archives: May 2013

Student feedback: What it can and can’t tell us

Posted by: Associate Professor Andrea Chester, Deputy Pro Vice Chancellor, Learning and Teaching, Design and Social Context, RMIT University.

As we move towards the end of semester we begin the process of collecting student feedback via the Course Experience Survey (CES). Student feedback on teaching is a complex topic and it typically raises a range of issues for academics.

Get any group of teaching staff together to discuss student feedback and you will be guaranteed a lively discussion. In addition to the many hours clocked up in our staff rooms on this topic, it has generated thousands of articles examining the validity of student evaluation tools; the best time in the semester for such feedback; how to most effectively close the feedback loop and how to communicate with students about changes made as a result of their feedback.

Lecturer showing a mindmap on an overhead projector.

Copyright © RMIT University. Photographer: Margund Sallowsky.

Previous tomtom posts like this one and this one have effectively captured the ups and downs of the process and both make mention of the importance of putting the CES in context for students.  The phenomenon of “survey fatigue” too (as we know from our own lives) is a risk in any drive to increase response rates, particularly as we move to online administration of the survey.

There is one issue, however, on which there is widespread agreement: student feedback is only one source of information available to us about our courses and our teaching. Triangulation is crucial. This means complementing student feedback with information from:

  • assessment tasks, giving due consideration to the learning your students demonstrate
  • peer observation, such as via Peer Partnerships, in which you invite colleagues to experience your teaching and provide feedback and your own reflections on what seems to work and not work and why.

The CES can provide us with useful information, but we do need to remember what it measures, namely student experience. In his useful summary of research on student evaluations, Terry Doyle (2004) reminds us that while student feedback can provide valuable information, there are a number of aspects about which students are not well qualified to provide feedback including:

  • if the teaching methods used were appropriate for the course
  • if the content covered was appropriate for the course
  • if the content covered was up-to-date
  • if the assignments were appropriate for aiding student learning
  • if what they learned has real world application
  • if what they learned will help them in future classes
  • if the type of assistance, help or support given to students was appropriate to the learning goals of the class
  • if the difficulty level of the course material was at an appropriate level.

What Doyle also provides here I think is a structure for a teacher or lecturer to speak to towards the end of her or his course. A quick reminder about each of the elements above would also be an appropriate introduction to students before they complete their survey.

RMIT TAFE Students in class.

Copyright © RMIT University. Photographer: Margund Sallowsky.

Before making changes in response to student feedback, we need to be confident in the validity of the data provided and this brings us to response rates. This semester the Survey Services Group has developed a reliability band calculator. During the administration period of the survey (May 6 – June 2) you will be able to check how your own response rates are tracking against the reliability bands (good, sufficient and insufficient). You can check the response rates by program and school here (RMIT Staff login required). Contact your L&T group if you’d like to use a short presentation that has been designed by the Survey Centre to be displayed in a class so that students can follow the links and complete any outstanding surveys.

The RMIT Academic Expectations have set expected and aspirational targets for the Good Teaching Scale. In the coming years there will be more pressure on academics to provide reliable snapshots of the student perspective on their teaching. The vast majority of academics have always used the surveys as a tool for self-reflection.

I’m confident that we can continue a culture at RMIT that puts an appropriate emphasis on major surveys like the CES as one way in which we identify both evidence of excellence and areas for improvement.

Resources:

  • Read more about Terry Doyle’s research into surveys and teacher effectiveness at his blog Learner Centered Teaching.
  • For more on the CES, read this FAQ published by the Survey Services Centre.

Share your thoughts about the CES in the comments section below!

Wherever you go, there you are

Posted by: Ruth Moeller, Lecturer in Education and Senior Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

Every time I pass the billboard for a certain university advertising an overseas student experience involving elephants, I get irritated. I don’t have anything against students, the university or elephants for that matter but really, how many students will actually go overseas as part of their studies? Although I agree that it would be a wonderful learning experience, I have difficulty with the premise that, for a student to be a global citizen, they need to study abroad.

Don’t get me wrong, ‘They’ say that travel broadens the mind and ‘They’ are right. The opportunity to work or study offshore would enhance any student experience and a highlight of a student’s experience at university.

But in my opinion this should be the icing on the cake, not the main focus. ‘I went overseas, now I am global’ — I don’t think so. So to do justice to the idea of global competence, we need to think more broadly.

RMIT has a sophisticated view when it says graduates will be ‘Global in outlook and competence‘. What that commits us to is providing graduates with ‘…opportunities to acquire professional [and] cultural skills that enable them to engage thoughtfully and effectively with the great diversity of people and situations they encounter at work and socially.’

This is saying that there are personal and professional skills and knowledge that need to be developed in all of our students. As educators, we need to ask: ‘How do we do this?’

As a starting point, the question I think we need to ask is: What does an ‘internationalised’ student look like in my discipline? How can we claim our students will be global in outlook and competence if we don’t actually know what this means within our discipline?

I have tried to do this in my discipline, tertiary teaching. Using the Australian Qualifications Framework criteria of knowledge, skills and application of knowledge and skills, I started by imaging what I would expect if someone came to me for a teaching job claiming that they were ‘global’. What would I be looking for? In doing, this I developed a framework of the knowledge and skills that helps students develop their global competence and outlook.

Some of the knowledge I would expect includes an appreciation of educational philosophies and different education systems to get a sense of the expectations of their students and how these philosophies might be enacted in classes. An added benefit of this could be the help it gives them in finding employment opportunities and navigating the various educational systems that operate across the world. Also of importance would be knowledge of the cultural views of education; the role of student/teacher, group/individual in different contexts.

When thinking of skills I would include a proficiency with different teaching strategies and the use of technology to engage diverse learning styles and cultures as well as the ability to research resources in an international context. The skills that help them identify what is available for them in regards to enhancing and internationalising their curriculum are, as educators, the same ones that will help them localise their curriculum should they wish to deliver content offshore or to deliver at a distance to global learners.

In thinking about the application of knowledge and skills, on a practical level I would incorporate how to design assessment for diverse learners and contexts, as well as the strategies that they, as teachers, could use to make their students ‘globally aware’.

In a broader sense, I would expect that person to be able to listen to, appreciate and synthesise other points of view as perhaps the key ability to operate within diverse cultures and environments.

Now the question is, does this just happen? Or do I need to create learning opportunities for this? Miracles do happen, but usually student learning is based on hard work and good design and that is what I am going for.

As my course is being reviewed, I am currently working on ways to integrate the skills and knowledge required to allow my students to have a global outlook. I found an excellent set of resources The GIHE Good Practice Guide to Internationalising the Curriculum at Griffith University to help with the planning involved in internationalising a course. They encourage you to look at programs and courses holistically, integrating an internationalised approach into aspects of curriculum design, assessment, learning resources and extracurricular activities.

Being global in outlook and competence requires far more than boarding a plane. Recently on the blog (here  and here) we’ve showcased student mobility opportunities that focus on the learner and their discipline. Thinking about the knowledge and skills we want to instil in graduates to give them a global education (and how will they apply these in any setting) is crucial to a genuine engagement with the world.

Resources:

Griffith University: The GIHE Good Practice Guide to Internationalising the Curriculum

http://www.griffith.edu.au/data/assets/pdf_file/0006/345291/Internationalising-the-Curriculum.pdf

Curriculum Review Tools for QAA – Quality Assurance of Assessment, Part 3 – Assessment for internationalisation of the Curriculum.Duncan D. Nulty, Brona Farreley and Michelle Barker

http://www.griffith.edu.au/__data/assets/pdf_file/0008/344384/Curriculum-Review-Tools-for-QAA-Part-3.pdf

Share your thoughts about a global outlook for students in the comments below!

A post from the archive: Native or immigrant – Exploring foreign territory in online learning

Did you know that there are over 80 teaching tomtom posts searchable and available through the tools at the right of the page?

You can also use the tag cloud and categories link to bring up relevant posts.

At this time of year you might be heading into the pointy end of your unit(s) and looking for advice on assessment (17 posts)student engagement (14 posts) or feedback (21 posts).

We’re always interested to hear what you’re wrestling with in the tertiary sector too – drop us a line at theteachingtomtom@gmail.com and request a topic or write a post for submission!

Native or immigrant – Exploring foreign territory in online learning

Posted by: Meredith Seaman, Senior Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

There is sometimes a perception that ALL tertiary students have grown up with technology and are natives of the online environment, and that teaching staff, well, they just have to catch up.

I beg to differ…

Working with staff as they prepare for teaching, I come across a vast range of different styles and views in relation to using newer educational technologies, some keen, proficient and eager to experiment and others overwhelmed, nervous or disinterested.

A recent study found that there was significant diversity in both staff and students in terms of technical experiences and proficiency in Australia universities. Students were not always ‘digital natives’ and academics were not always ‘immigrants’ as has sometimes been claimed. Given my experience, it doesn’t surprise me that they found great diversity across ages and groups, and a wide range of perceptions about the advantages of using technology for learning and teaching. Even if we don’t buy into the immigrant/native analogy, both students and staff can at times feel foreign and lost. As reflective journals, lecture capture, web conferencing, twitter, blogs and video (some explored in recent blog posts to TTTT) become more common, more students and colleagues will be exposed to an increasing range of technologies in learning and teaching. So how can we support better learning and teaching through technology and enable both colleagues and students from a range of backgrounds and technical proficiencies to flourish?

My personal understanding of what it might be like to be in ‘foreign territory’ in an educational online context, comes from my own recent experience as a distance education student. Thrown in the deep end with two other students, who had had very minimal exposure to Web 2.0 technologies but were keen to learn, we were asked to use an emerging educational technology to develop and present an assignment about education and technology. A fellow team member suggested a wiki and we were off. In our case, the technical aspects (setting up and navigating wikispaces which was very new to the other students), and visual and instructional design aspects of the task completely took over from the content and intended learning outcomes of the assignment in our interaction as a group. On top of additional time constraints which we faced as mature aged students with young families and/or in full time employment, the challenges of working and being assessed as a group, the assignment almost derailed. We ended up using email to communicate outside of the wiki and got back on track. The difficulties weren’t because wikispaces was difficult to use, but because of the challenges in sustaining good group work and communication while interacting online in an unstructured, unfamiliar space, in this case with others we hadn’t even met.

I learnt a lot from this activity, and apply it in my work with teaching staff. Like Clare suggests in her recent post, there needs to be a clear sense of purpose as to why to adopt technology for a particular tasks, and clear attention paid to the motivation for students (to foster the kind of willingness and ‘good attitude’ which is so important to successful learning). For our assignment the benefit that we should learn about wikis for education to inform our role as educators was clear, yet it still felt like an unnecessarily add on to an assessment task, and very time consuming in itself. While technology can support communication between peers for distance students, the dry unfamiliar territory of the wiki was not ideal for this in our case. We tended to develop content separately, and then publish, rather than truly collaborate and develop ideas relevant to the assignment as a group. The superficial design and technical aspects unfortunately took over. Other tools, like chat or skype or google docs (or even email which we ended up resorting to) would perhaps have been better for timely communication and collaboration, and would have supported the development of the wiki. But the solution to such challenges isn’t using other tools or technology training (though time and support to learn new technologies is terribly important), but in good teaching practice and design.

So what did I learn about good teaching practice and design using educational technologies from that experience?

That we should:

  • provide time for students to play and explore technologies in advance of the ‘meat’ of the assignment work
  • provide clear structure/scaffolding to support how we were expected to work with the online tools (and most importantly AS A GROUP if that’s a key aspect of the task)
  • make an explicit link between the learning outcomes, assessment criteria, and the process of developing new technical skills

and, the benefits of being:

  • required to work in a group with different levels of ability, and with different individual strengths and weaknesses
  • encouraged to explore new technologies
  • able to experience the technology from a student perspective as an educator.

__________________________

More on recent research into ‘immigrants and natives’ and attitudes about technology in learning and teaching:

Educating the Net Generation: Implications for Learning and Teaching in Australian Universities
Open University research explodes myth of ‘digital native’
The impact of web-based lecture technologies on current and future practices in learning and teaching
Teaching, technology and educational design: the architecture of productive learning environments

 

Follow

Get every new post delivered to your Inbox.

Join 1,394 other followers

%d bloggers like this: