Tag Archives: student feedback

We are listening. Strategies to increase survey responses rates on teaching.

Picture credit iStock.com

Picture credit iStock.com

Posted by: Meredith Seaman, Senior Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

The problem…

We talk quite a lot about student feedback here on the blog, but the specific aspect I want to consider here is appropriate, fair and reliable ways to administer institutional surveys on good teaching to students. In particular I want to examine what could be done to increase survey responses so the results are more meaningful. The challenge in getting students to participate is more significant since surveys moved online, despite the obvious benefits of ‘anywhere/anytime’ for students to complete the survey instruments, and associated efficiencies for the Institution.

With low response rates it can be just a waste of everyone’s time as we need a certain sample size to viable and useful.

The research….

Investigations from the period of transition when many surveys moved online in the United States emphasise the following:

  • … “an important factor in response rates is students’ belief that rating results are used for important decisions about courses and faculty.” (Ballantyne, 1999)
  • Institutions should “encourage instructors to show a personal interest in students completing the forms (e.g., instructors could mention the forms in class, let students know that they pay attention to student responses, or send personal emails to students reminding them to complete the forms)” (Johnson, 2002)

While I’d definitely think twice about sending a personal email to your students, research is consistent that it is important for teachers to set up the context for students to complete the survey. For response rates to increase strategies are required at a range of levels. There is however, consensus that teachers have a role.

Given that the challenge is bigger than an institution sending out disembodied emails or offering iPads. How might we make it a meaningful process, a better experience for our students, and get more reliable data?

Strategies…

Pulling together different ideas from Schools across our College, to do it well, we need to show we are listening. The in-class process might go something like this:

Sowing the seed

Before survey time sow the seed early on about the importance of their ‘feedback’. You could highlight earlier in the course that such surveys inform a range of activities and decisions from university management down to the classroom. You could highlight a couple of specific changes that you have made in response to past survey responses and Student Consultative Committee discussions and so on.

Spending time with your students unpacking the notion of ‘feedback’ more generally is another idea. You might want to emphasise the different types of feedback that students get in your course, such as: from peers, on learning activities, on assessments tasks etc. You could also build on work you are doing with students on their skills to provide constructive feedback, such as giving peer review and feedback on learning activities and assessment tasks etc.

Summarise the Course

At survey time you could summarise the course to date.

e.g. In week one we…  in week 3 we found such and such a concept difficult…, in week 6 we … and finally, reiterate where you’re heading in the remainder of the semester. Helen’s recent post on ‘Going with the flow’ provides a model for this kind of activity. You might want to highlight how have you listened to your students (current and previous) and adjusted your learning and teaching plan.

Set aside time in class: (or in an online space)

Delivery in class time might be tricky, but in terms of getting response rates up it is well worth the time investment. It will show generosity if you allow class time, and that in itself emphasises that you take the process seriously, and are listening. It has been reported that 84% of Australians now have a smart phone, and 71% of those also have a tablet (Horizon Report 2014), and surveys can be completed on these devices online. If these stats seem inflated for your cohort, you could allow time for students to go to the library or build into an existing lab class.

Leave the Room:

It’s important that after all that, you leave the room. It highlights that the process is fair, provides thinking time, and creates a space for their comments to come to the fore.

________________________

At RMIT City Campus the Course Experience Survey (CES) survey is open to students from the 20th of September.

More on the CES at RMIT

Ballantyne, C. (1999). Improving university teaching: Responding to feedback from students. In Zepke, N., Knight, M., L&ach, L. and Viskovic, A. (Eds), Adult Learning Cultures: Challenges and Choices in times of change,  WP Press, Wellington, pp.55-165.

Johnson, T. (2002). ‘Online Student Ratings: Will Students Respond?’ Paper presented at the annual conference of the American Educational Research Association, New Orleans, 2002.

NMC Technology Outlook for Australian Tertiary Education - A Horizon Project Regional Report 2014

Share your thoughts on strategies to increase survey responses rates in the comments section!

_________
Find us on:

Discourse diversity across L&T cultures

This week’s post comes to us via Karen Dellar, Barbara Morgan, Alison Brown and the team at the Study and Learning Centre at RMIT University.

Making the implicit, explicit – discourse structures across cultures

Most students initially find the transition to academic discourse and writing styles challenging. This transition can be more problematic when students are new to the Australian educational environment. Many international students are unfamiliar with lecturers’ expectations and visit the Study and Learning Centre’s busy drop in service for assistance with academic writing.

One student who is now receiving credits and distinctions recently reflected on her previous failure in a number of courses. We asked her what it was that helped her achieve better marks. She said quite simply: ‘I now know what they want!’

Acknowledging diversity – similarities and differences

This student’s experience typifies the challenges often faced by students in not knowing what is expected when they transition to study in a new culture. The first step towards creating a truly inclusive learning environment is to acknowledge the similarities and differences across learning cultures (and there are many) and then to make the differences explicit. Key differences relate to  discourse styles and expectations upon the reader-writer relationships. The way we shape or tell a story depends upon our cultural and linguistic context and the unspoken patterns we have learnt.

Across cultures there are different expectations of how texts are structured. We expect that all texts have a similar framework to our own…until we are faced with something different. It is surprising to realise how much of our knowledge of text organisation is implicit and culturally specific. English language readers have an expectation about how quickly you get to the point, and when you do, how explicitly this is done; the order that concepts are introduced; the amount of preamble or background that is necessary; the amount of repetition that is effective or the extent of digression that is permissible. As teachers we expect our students to conform to these expectations.

In the words of an international student:

In the Subcontinent (Pakistan and India) the way we are taught to structure our essays is, we don’t come to the point directly: we have to develop this major build-up, before coming to the point…otherwise our lecturer won’t think we have put in enough effort…but over here the thing was that ‘bang’ – go to the point directly and then you can start explaining… (Learning Lab International Student Stories)

model1_image

Model 1: Direct, linear approach. Here’s the flower. These are the main points…

Assignment structure is culturally specific

It is often assumed that students whose first language is not English have difficulty with their

Model 2. Here's the garden. Let me take you for a walk and I'll show you something..

Model 2: Explanation before getting to the main point. Here’s the garden. Let me take you for a walk and I’ll show you something…

studies because of their language. This is not the full picture. What is commonly overlooked is that students from different cultural backgrounds are often unaware of the academic expectations of their new educational culture. In particular, the discourse structure can be unfamiliar and different to what they know.

The models here give a visual representation of two common ways of structuring academic writing across cultures.

Model 1 is a simple representation of the approach to organising material in the Australian educational context. The student is expected to present the main point first, followed by explanation and analysis.

Model 2 is another representation of what is common in many other cultures. In this model the student builds a case through background information, explanation and analysis and finally presents the main point.

Put simply, the measures for ‘good writing’ in an Australian university are often quite different from the students’ previous experience. International students are often disappointed at their poor marks in first year as they take some time to work out what is required. In the case above, the student repeated first year having learnt this lesson the hard way. A better solution would be for students to be explicitly taught what is expected and for it to be acknowledged that their understanding of academic writing on arrival may be different to what we expect.

For more on helping your students structure their assignments visit RMIT’s Learning Lab. For diverse cohorts a good starting point is International Student Stories. As well, you will find resources on assignment genres that you can use in-class and direct students to for self-study.

References:

Arkoudis, S & Tran, L 2010, ‘Writing Blah, Blah, Blah: Lecturers’ Approaches and Challenges in Supporting International Students’, International Journal of Teaching and Learning in Higher Education, vol. 22, no. 2, pp. 169 -178.
Fox, H 1994, ‘Listening to the World. Cultural Issues in Academic Writing ‘, National Council of Teachers of English, USA.
Ramburuth, P 2009, ‘The impact of culture on learning: exploring student perceptions’, Multicultural Education and Technology Journal, vol. 3, no. 3, pp. 182 – 195.
Ryan J & Carroll J (eds) 2005, Teaching International Students: Improving Learning for All, Routledge, London.

Share your thoughts about discourse diversity and your tips for conveying your expectations in the comments section!

_________
Find us on:

Being bold, being true – Knowing what you’re best to do

Guest Post: Associate Professor Debra Bateman, Faculty of Arts & Education, School of Education, Deakin University.

Late last year, as a guest of Dean of the School of Global, Urban and Social Studies (GUSS) Professor David Hayward and as part of an LTIF project on Next Generation Learning Spaces, Associate Professor Debra Bateman spoke to teachers and lecturers at RMIT about her own teaching identity and journey as an academic with a Q&A. The project team was keen to capture some of her thoughts and share them with a wider audience. In this post Deb outlines a few of the principles that underpin her teaching. The session was extremely well-received with many survey responses mentioning that Deb’s words energised and affirmed what these lecturers and teachers were doing in their teaching. 

We hope this post can have a similar effect on you at this busy time of the year!

Being bold, being true – Knowing what you’re best to do
(or my thoughts on Teaching and Learning in Higher Education)
 
SuperheroesThe best teachers in higher education are the ones who know what they are doing. Of course, that sounds quite simple, however to know what one is doing, in this context, is complex. You need to be able to juggle multiple pedagogical and curriculum balls in the air at any time, including:

·      Disciplinary knowledge.

·      The broken up bits of that disciplinary knowledge that are accessible to everyone, and subsequently scaffold deeper learning.

·      An approach that not only engages students, but also professionally and intellectually engages oneself, in order to participate in the learning process in a meaningful way.

·      Confidence in one’s academic identity.

·      An ability to articulate, from another different disciplinary knowledge such as education, the reasons you are doing what you are doing and how you are doing it. (Did you know, by the way, that in higher education we should be talking about andragogy rather than pedagogy?)

·      Opportunities to witness, assess and evaluate what is learned, and to make judgments about learning that affect how students are able to enter the world.

In my presentation at RMIT, I suggested that higher education teachers need to know themselves, and situated that idea in the works of Pat Thomson (2002) who talks about the ideas of virtual schoolbags. Whilst she was referring to the ‘stuff’ that students bring into a learning environment, which influence how and what they do, I suggest that teachers carry virtual school bags also. On top of the content they are expected to teach, they bring loads of experiences of the world and sophisticated thinking which ought to capitalise on the connections that are able to be made for students when delivering big ideas or abstract concepts. 

The fact that before I was a teacher and academic, I was a hairdresser might account for some of the very creative endeavours I pursue in teaching and learning. My background as a teacher in some challenging schools offers me a diverse set of strategies for engagement of resistant learners (and sometimes colleagues). Working to our strengths, offers students of all pursuits an enthused and committed experience in their learning.

I’m also pretty interested in Shulman’s notion of pedagogies of uncertainty (2005) and Rowan and Beavis’ notion of ‘playful pedagogies’ (never published but I heard them say it first). The idea of enabling students to take risks in their learning is powerful, and more importantly  exciting.

For different programs of studies, ‘risks’ can look quite different. In one of my classes, when teaching Civics and Citizenship, I ask students to come dressed as superheroes, and through a range of life-world related experiences they undertake a deep critical analysis of many of the taken-for-granted constructs of the caped wonders (though we also critique the capes from an OH&S perspective). The risks are many, including exposing a value or position of a societal issue, feeling like a fool as people stare at you on public transport (if you don’t do a quick change in a telephone booth), and participating in an embodied analysis, which draws upon cultural knowledge. However, the excitement provides the titillation to learn.

I’ve also thought a lot about ‘furtive teaching’. That’s taking risks with new strategies or using strategies in ways other than they were intended, with the fear of being caught. Part of furtive teaching is undertaking the act of teaching knowing that the risk is countered by the amount of preparation, consideration, research or rehearsal that you have done. In my experience, students have loved being part of my risks. And increasingly, other staff are enjoying them too. One staff-related risk was our representation of scholarship materials Screen Shot 2014-05-01 at 11.54.00 amthrough a blend of culture jam, flash mob or parody (see https://vimeo.com/38401190).

Be proud of who you are, and where you come from. Connect the who you are with the what you need to do. Teaching and learning are important aspects of higher education, which cannot be guided by a proliferation of best practice guides. Whilst scholarship can inform and enhance what it is that we do, what we do is part of who we are, what we know, and how we know. But, in a climate of increased accountability, be able to articulate all of those things, and more importantly publish them. Give licence to others to explore their teaching and learning perspectives in ways that up until now, have been fairly limited and narrow.

References:

Shulman, L. S. (2005). Pedagogies of uncertainty. Liberal Education, 91(2), 18-26.
Thomson, P. (2002). Schooling the Rustbelt Kids. Making the Difference in Changing Times. Sydney: Allen and Unwin.

Deb tweets @debbateman
Share your thoughts about pedagogies and andragogies  in the comments section!
_________
Find us on:

Planning learning design through storyboards

Guest Post: Professor Gilly Salmon, Pro Vice-Chancellor, Learning Transformations, Swinburne University of Technology. 

Professor Gilly Salmon is one of the world’s leading thinkers in online learning. She researches and publishes widely on the themes of innovation and change in Higher Education and the exploitation of new technologies of all kinds in the service of learning. The Learning Transformations Unit is responsible for the exploration and exploitation of learning technologies; the resourcing, preparation and scholarship of staff; and the development of partnerships that increase and extend Swinburne’s online provision and presence. This year Gilly will be a guest speaker at the upcoming DEANZ Conference and EduTECH National Congress & Expo.

Gilly tweets @gillysalmon.

Opens in a new window.

Click to see the details of Gilly’s latest book: ‘E-tivities: The Key to Active Online Learning, 2nd Edition’

Late last year I was invited to speak to academics at RMIT and we had a great afternoon together working on ideas around building scaffolds for learning using newer technologies.

Of course the time we had together  went by too quickly!  When I looked at the feedback, I noticed several participants had commented that they liked the idea of storyboarding for planning learning and wanted to know more about it. From the Learning Transformation Unit at Swinburne, we’re in the middle of running a MOOC for professional development around the Carpe Diem process.

I’ve made a series of little videos for the MOOC and one is about storyboarding — essentially representing the sequence or journey of your learners through the time you have together – and how helpful I’ve found it for planning forward-looking learning and teaching. So it’s here for you to have a look at and maybe get together with a course or program team and try!

For me, the focus on learning design is a key shift in the way we need to consider creating the future in our various disciplines and domains.

I would be interested to know how it goes for you.

Best wishes,

Gilly

Share your thoughts about learning design in the comments section!

_________
Find us on:

The art of questioning

Posted by: Associate Professor Andrea Chester, Deputy Pro Vice Chancellor, Learning and Teaching, Design and Social Context, RMIT University.

Over the years I have read quite a bit on the use of questions in education. There are a number of useful websites that describe the importance of questioning in teaching and defining different types of questions. The UNSW L&T Unit has one, as does Cornell’s Center for Teaching Excellence.

make-just-one-change-smAs part of the toolkit of a student-centred teacher, questioning has long been considered a core skill for all teachers from primary through to the tertiary context. So central is the skill that Mary Jane Aschner (1961) described educators as ‘professional question makers’. I know from experience how a well-chosen question can open up a rich discussion and how the wrong question can close it down.

What I hadn’t spent much time thinking about was how to encourage students to ask their own questions.  That is, until I read Make just one change: Teach students to ask their own questions by Dan Rothstein and Luz Santana.

Rothstein and Santana argue teaching can be transformed if students, rather than teachers, assume responsibility for questioning. They argue that asking questions is fundamentally empowering and that all students can and should learn to formulate their own questions and that all teachers can integrate development of the skill into their regular teaching practice. While the book is pitched at teachers of K-12, the ideas can easily be applied to tertiary teaching.

The book and associated online resources available through the Right Question Institute set out the seven-step Question Formulation Technique (QFT™) summarised below.

Step

Description

Principle

Approximate time allocation

  • Begin with a question focus (QFocus)
This is the work you do to develop a stimulus, which can be a topic, a sentence, an image or an object, depending on your purpose, but generally not a question itself.  -  (varies)
  • Share and discuss the rules for producing questions
There are four rules:

  • ask as many questions as you can
  • do not stop to discuss, judge or answer questions
  • write down each question exactly as it is stated
  • change any statements as questions 
Meta-cognition 5-7 minutes
  • Produce Questions
In small groups students produce as many questions as they like within the available time. Divergent thinking 5-8 minutes
  • Categorise Questions
In this step students are encouraged to improve their questions by labeling each as either open or closed and discussing the differences between these two types of questions. Students are encouraged to change some questions from one type to the other to explore the difference in possible responses. Analysis and convergence 5-10 minutes
  • Prioritise Questions
Students prioritise their questions, articulate their rationale and select the three most important questions. They note where these questions occurred in their original list of questions, at the beginning, middle or end. - 5 minutes
  • Next Steps
Here students consider how they will use their questions. This depends on your purpose for engaging in the activity. Application Will depend on purpose of QFT™
  • Reflection
Finally students reflect on what they learned and how they can use it. Meta-cognition 5-8 minutes

The entire sequence can be completed in 30 minutes, with a reduction as you and the students become more familiar with the process.

When you might want to use the QFT™
The QFT™ can be used to:

  • generate interest at the start of a course

    Click on the image to see Dan Rothstein's TEDx presentation: "Asking questions is the single most powerful renewable source of intellectual energy: It's in our minds we can create it, we can create it continuously..."

    Click on the image to see Dan Rothstein’s TEDx presentation: “Asking questions is the single most powerful renewable source of intellectual energy: It’s in our minds we can create it, we can create it continuously…”

  • introduce a new topic
  • assess and/or deepen understanding
  • stimulate new thinking
  • prepare for an assessment task
  • conclude a topic or course.

What to know more?

References:

Aschner, M. J. (1961). Asking questions to trigger thinking. National Education Association (NEA) Journal, 50, 44-46.

Share your thoughts and questions in the comments section!
_________
Find us on:

What will the campus of the future look like?

Guest Post: Jo Dane is a designer, educator and researcher with a passion for educational transformation enabled through research-based design practice. Jo works at woodsbagot.com and tweets @WB_JoDane.

Jo_Dane_Twitter_PicI’m a design educator and someone who’s regularly tasked with putting together a vision of physical spaces for students. Ideally, these new spaces are supposed to be ‘future-proof’. So it can be fun to do some crystal-gazing about the future of the university campus.

Here are some observations, speculations and predictions that I’ll commit to the blogosphere in 2014:

1. Students will be empowered with choices of how, when and where to learn.
It will be increasingly possible to get a degree at University X which includes undertaking core subjects at University Y or via accredited MOOCs. If the quality of the learning experience (and facilities and spaces will be part of that equation) doesn’t stack up, students will shift their allegiance to another institution. And the funding will follow the student.

2. Hybrid learning experiences will be the new norm.
On-campus delivery will increasingly incorporate online components such as response software in lectures, multimedia content, group collaboration and teacher consultation. Digital platforms will continue to improve and enable both synchronous and asynchronous learning encounters.

3. Learning will be social and happen with other students IN REAL TIME.
For too long learning has happened in isolation in students’ homes while studying for exams, or preparing essays and assignment work. It has long been recognised that learning is a social experience. A room full of students is also a room full of teachers. Interaction between students broadens each student’s perspective and provides an opportunity to share and reinforce important concepts.

Click to see more pictures of MUSE, a Woods Bagot project completed this year.

MUSE – Macquarie University Spatial Experience, Sydney, 2014

Real time learning will happen in the classroom when a) the teacher facilitates the interactive learning experience and b) the classroom is designed to enable such encounters.

4. The notion of a 24-7, ‘sticky campus’ will endure.
Students (especially undergraduates) will be encouraged to stay on campus for longer periods of time. They will continue to blur boundaries between learning, socialising and working. The campus, therefore, will provide ‘sticky’ spaces where students can undertake both serendipitous and asynchronous activities. These will include media hubs for small groups to collectively engage in online material, or to Skype subject experts/overseas peers.

Click to see more pictures of MUSE, a Woods Bagot project completed this year.

MUSE – Macquarie University Spatial Experience, Sydney, 2014

5. Mobile devices, ‘Bring Your Own Device’ and cloud computing mean that students can access specialist software anywhere, anytime.
Students need no longer be tethered to the dehumanising lab computer, but can choose where and with whom to study, whilst accessing critical digital infrastructure.

6. Say goodbye to lecture theatres and computer labs!
While this might seem to counter to the ‘sticky campus’ idea (but really it should clarify the purpose of bringing students together) students are voting with their feet and where possible opting to tune into lectures online rather than face-to-face. Not only this, the prevalence of high quality (free) content, through YouTube, TED Talks, MOOCs and a plethora of other online repositories means that students are finding expert content from alternative sources rather than from the prescribed teachers. Universities will increasingly share exemplary content rather than rely on academics reinventing content every year.

7. Augmented learning, wearable technologies, 3d printing and gaming experiences are coming.
These are recognised trends on the horizon. We might not know exactly what they will look like, nor the impact they will have on the campus environment. Get used to this feeling. The better you adapt to change, uncertainty and the unforeseeable, the more agile you are. Agility is a key trait needed for the emerging knowledge economy.

Screen Shot 2014-03-16 at 5.16.10 pm

Media & Communication staff at RMIT discuss learning spaces in the Swanston Academic Building.

8. Academics will work increasingly in teams, sharing and collaborating in teaching and research activities.
The academic workplace will need to provide for a younger generation of academics who are more collaborative and connected than any previous generation. The next generation of academics won’t be hidden away in confined offices. The campus will include ‘third spaces’ — extensions of the workplace where workers can seek alternative environments to promote innovation and problem-solving.

9. Academics will be more accessible to students, but will connect through digital means moreso than face-to-face.
For teachers and lecturers, the skills of delivering remotely and facilitating online discussions will be as crucial as your in-class toolkit. This means your potential reach increases (and so does your profile) but of course that there’s another set of skills that are currently seen as optional.

10. This one’s a fill-in-the-blank, left for you, the reader…
Posts like this can often live on through the comments thread — why not make your own prediction (or disagree with/clarify one of my own) by commenting below.

Share your thoughts and predictions in the comments section!
_________
Find us on:

First impressions

Posted by: Jon Hurford, Senior Advisor, Learning and Teaching & Andrea Wallace, Educational Developer, College of Design and Social Context, RMIT University.

Andrea is a member of the Inclusive Teaching and Assessment Practices Project working to develop resources and deliver professional development to staff.

Sign in a school bus reads: 'Keep Out. Please put on seatbelt + be quiet + behave. Thanks'

A sign in a school’s excursion bus at the Old Melbourne Gaol this week.

As the saying goes, ‘You don’t get a second chance at a first impression.’

This week, across Australia, thousands of lecturers and tutors will be meeting their new students or welcoming back continuing students.

In Vocational Education, classes have been back for almost a month, but still, it’s early days.

It’s obvious that getting off on the right foot and creating the right environment for students has a special importance in tertiary education. For one thing, in a 12-16 week delivery schedule, the feeling that time is precious is understandable.

In wanting students to take our course seriously, in the feeling that we’re competing for the mindshare of their course load, in rushing about, is there a risk of putting up a sign (metaphorically) like the bus driver (or the staff who share driving duties) in the picture to the left? Note the ‘Keep Out’ in red and the tiny ‘Thanks’ at the right. What messages are we sending students in their first classes?

So this post is just a quick reminder that in the midst of all the organisational and administrative tasks we should still hold our personal philosophy of education front-and-centre and be enacting the strategies and principles that brought us through university as learners, and that brought us to university to teach.

The Inclusive Teaching and Assessment Practices Project has a set of principles that might help you see that big picture (or the jigsaw pieces of the bigger picture that are your courses) and we can’t think of a better time of the year (at least for our southern hemisphere audience) than now to put them in front of readers. Each of the following links has an associated page with key questions, resources and examples of the principle in use:

Getting a piece of writing from your students in a class early in your teaching schedule is an easy diagnostic tool. You’ll get to know a key aspect of their learning skill set and coupled with a quick survey you can get an impression of what your students expect from the course. Perhaps you teach online (or you’ve taken these elements of your course online) and you use a discussion board or blog for this. You can probably see how this simple task hits many of the principles above– if you’ve asked students what was their inspiration to study a certain discipline; if you’ve read the responses and turned them around to the students quickly; if you’ve then provided the means for them to share their responses and maybe organise themselves in study groups based on this for the first assessment, you’re establishing an environment that is ‘feedback rich’.

But what about longer pieces of writing? What about supporting your students in documenting their progress in your course?

Next week, there’s an opportunity for all staff at RMIT as Associate Professor Mary Ryan (School of Education, QUT) delivers a lecture and workshop on the Teaching and Assessing Reflective Learning (TARL) model that she and her team developed in a recent OLT project.

Professor Ryan will explain how the systematic approach can be used to embed the pedagogy of reflective writing across courses in different disciplines.  The workshop will explore the suite of pedagogical patterns and accompanying resources for systematically teaching and assessing reflective practice underpinned by the TARL and EPC models.

Lecture: Teaching and Assessing Reflective Writing
Thursday 13 March 2014
11.30 am – 12.30 pm
Building 80, Level 1, Room 2
Workshops: Teaching and Assessing Reflective Writing
Thursday 13 March 2014
12.30 – 2.20 pm
Building 13, Level 3, Room 5 (City)
Or
Friday 14 March 2014
11.00 am – 12.30 pm
Building 514, Level 1, Room 2 (Brunswick)

Registration for the workshops is essential. Space is limited. Click here to register (RMIT Login required).

Oh, and on the topic of first impressions, we’d also like to mention two blogs that have made good impressions on us during the break and will be of particular interest to casual, sessional and part time staff:

We recently added them to our blogroll (right of screen)– go visit their site for more perspectives on starting the year.

Share your thoughts on first impressions, inclusive teaching and reflective writing in the comments section!
_________
Find us on:

Assessment, Grade, Holidays…

Posted by: John Benwell, Principal Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

HD rubricIn Melbourne, it’s the last week of scheduled classes and nearing the long break over the southern summer. Whilst thoughts may be wandering towards holidays, sun, the beach and all those great ‘Aussie’ pastimes, it’s time to assess and grade our students.

All the formative assessment has been done; we have provided vast quantities of feedback to our students and maybe provided avenues for feedback from their peers; now is the time to give each one a grade.

So I thought for my last post for the year, and before running to the beach to go sailing, I’d do a light refresher on assessment and point to some resources on rubrics to help you through.

At this stage of the year, we already know how most students are going. We have been giving them formative assessment tasks, and providing feedback to them so they can learn from their mistakes, fill the gaps and polish their performance. There have likely been summative assessments, which have been building towards a final mark.

Maybe you’ve read (and found useful) other tomtom posts that have discussed aspects of assessment (like Thembi’s post on Active Learning Strategies, Meredith’s post on Academic Integrity, Alex’s post on Peer and Self-Assessment and my previous post, Keeping Watch on using assessment to track our students’ progress), so here is a post with some handy references on assessment and rubric development.

In the old days, students would now start cramming, revising knowledge and processes, going to the library and doing old exam papers hoping their lecturer would just revise the last year’s paper for this year.

Nowadays, we hope those student expectations are well behind us and the last assignment encapsulates the skills, knowledge and the application skills and knowledge into a capstone assessment experience to confirm the student has achieved the course (subject) learning outcomes.

In pondering assessment we should never lose sight of what assessment is, and its purpose. As a reminder, here are the core principles of assessment from The University of Melbourne’s  Centre for the Study of Higher Education:

  1. Assessment guides and encourages effective approaches to learning
  2. Assessment validates and reliably measures expected learning outcomes, in particular the higher–order learning that characterises higher education
  3. Assessment and grading defines and protects academic standards.

At this stage of the year, principle 1 should have provided students with tasks that permitted them to test their learning and understanding in their passage towards the achieving the learning outcomes. The best scenario would be student-lecturer negotiated, multiple learning and assessment tasks that were designed to increase in complexity over the semester.

Principle 2 reminds us that we should not be simply testing students’ knowledge, but more their application of skills and knowledge and their ability to independently think though increasingly complex problems associated with their intended discipline. And we must grade each student in a reliable and repeatable manner. More on how we do that later.

Principle 3 helps maintain our standards. Not only do we have to assess if they have achieved the course learning outcomes, but also how well they have achieved it with reference to industry standards and moderation across institutions. Painfully for a teacher, we also must decide if a student has not achieved the learning outcomes.

Using the results from several forms of assessment during the semester, we need a framework to grade effectively. We need to have a considered series of statements that allow us to assign an overall grade to each student. They are like performance indicators. These statements are incremental performance levels of the learning outcomes. The levels are based on professional judgement, industry expectations and the quality standards of the university.

Commonly referred to as rubrics, you should develop a set of guidelines for marking and grading. They are not rules, but a framework to help you and your co-assessors be consistent across the group of students, from year to year and to maintain the academic quality standards expected by your industry/discipline and the students’ future employers. Your institution also relies on your professional judgement to uphold the standard of its awards.

Some argue that rubrics are restrictive, but with a well-developed set of rubrics, time is saved, consistency is improved, standards are upheld, and the course remains constructively aligned — the rubrics being generated from the learning outcomes. Levels of attainment between are  documented and described for the students to see. These can be a little bit fuzzy in their generic form but discussion with students, providing exemplars and using the same rubrics for peer and self-assessment can enhance all participants’ understanding of what a rubric is trying to do. Rubrics can be applied to all types of assessments: essays, drawings, pictures, models, presentations, designs and films.

The links below have some useful reading on rubrics, their purpose, value and how to write them:

http://aadmc.wikispaces.com/file/view/Assessment.pdf

http://www.edutopia.org/assessment-guide-rubrics

http://www.cmu.edu/teaching/designteach/teach/rubrics.html

Unfortunately some students, despite all attempts by us, fail to provide us with the evidence they have achieved the learning outcomes. The result is then a fail. There wouldn’t be a lecturer or educator who does it lightly, but it’s part of upholding the professionalism of our discipline, and the standards of our university. The determined learner who fails will return and do the course again, learning from their mistakes, and hopefully will achieve a better mark the next time around.

So before you start marking this semester try developing your own rubrics. Start by writing your learning outcomes on the left of a tabjboceanle, and then use grade descriptors of what you would expect to see from the students in the boxes. Your learning and teaching advisor can help you create them and your results will be fairer and more consistent. Rubrics are also a great help when marking online or if you have several tutors performing the assessment. What’s more, you can save time!

Well my last student is graded, so I’m off to a BBQ tonight and out into the ocean (Bass Strait) for a sail on the weekend.

I hope you have an enjoyable break.


Find us on Facebook http://www.facebook.com/teachingtomtom and Twitter @teachingtomtom

Games people play Part 2: Let’s pretend

Posted by: Jon Hurford, Senior Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

Metrics

Fitbit Screengrab

Author is comfortably mid-table at 10000 steps a day. Less than Andrea W, more than The Phantom.

Last week was quite a fun post to write and I’m back with the semi-promised ‘Part 2′…

We do often get between 100 and 300 daily hits on a published post at the tomtom. But my post, the 100th post fell a little bit short. But since then there’s been quite a bit of activity and as a number of people pointed out in the comments, my threat to send it to blog-post-Heaven made it seem:

  • like a waste of my effort/their invested time
  • like an arbitrary requirement
  • unfair to demand a group of people to reach an aggregated target
  • churlish for me to take my ball and go home
  • that perhaps at 1400 words I just didn’t know how to end the post?

All true!

What a savvy readership!

And all so relevant to games- Are the rules clear? Are the rules fair? Who thought up these rules? Who’s the umpire? How do I win? How do I quit?

On to what I’d promised, first, extrinsic motivations and measurements.

There’s already a game in place in TAFE and HE that our students play. The game called, for instance, ‘Bachelor of X’ runs for three years, it’s assessed by former players and it will cost you upwards of $30 000 AUD. Insert as many asterisks as you’d like, but essentially you quit by dropping out, you win by graduating, you can cheat in a number of ways and at the end you’re awarded a badge called a degree.

A cynical sketch of the tertiary experience, yes. Shoulder angels should rightly counter with the intangibles and the intrinsic benefits that come with a tertiary experience. University is where students can grapple with ideas, create new knowledge, speak truth to power etcetera.

Click here to visit GEElab. Opens in a new window.

‘Trouble Tower’ app from RMIT’s GEElab.

It would be depressing (or worse) if we held the first view front of mind and didn’t encourage the myriad benefits related to what learning institutions offer. But you’ll find plenty of posts on the tomtom where you can read about graduate attributes, lifelong learning and that sort of thing!

Caveats

My point though is that adding another layer of achievements as instructors is problematic for a system that already has clear thresholds, ladders and badges. Completion of credit points, Competent/Not Yet Competent, Pass to High Distinction: these are the real points of the game. Universities bear the legacy of ranks and hierarchies in their inherited Latin and medieval terms. Just as more recently we have had the AQF imposing a kind of metric system of levels 1-10 on these old terms from guilds, knights and churches.

So an easier conclusion to this post would be for me to write about what could go wrong with adding a game element to your course. That it’s ‘pointsification‘, that it’s infantile, that there’ll be unintended consequences. That it makes university into (even moreso) a token economy. That carrots and sticks are for donkeys. Adding gameful design to your course won’t make up for opaque course outcomes or dated course materials.

But I did promise to explain why an individual instructor might still be interested in adding some sort of a game mechanic to their course.

Click to read the article at news.Discovery.com

A 5000 year old Bronze-Age game: “According to distribution, shape and numbers of the stone pieces, it appears that the game is based on the number 4.” Haluk Sağlamtimur, Ege University İzmir, Turkey. Click above to read the full article.

Case 1: You enjoy games (boardgames, word games or computer games) and recognise that games can add an element of fun to tasks. You want to allow students to learn, track their learning or to present the results of their learning in a modality that’s closer to one in which they’re spending some of their leisure time.

Case 2: You recognise that there’s an element of your course that’s an ‘eat-your-vegetables’ proposition: it needs to be done and many of your students find it difficult. As a result, it’s often skipped over by students or it becomes a point where their performance dips or where they disengage. It might be something like acquiring the appropriate vocabulary for a unit, or acquiring a set of technological skills that are required that can be applied later in the unit. You think that maybe some healthy competition or a bit of incentivising could do the trick.

To address the latter case, it’s worthwhile noting that psychologists talk about the overjustification effect, where extrinsic rewards reduce intrinsic motivation. As one of the examples in Alfie Kohn’s very quotable ‘Punished by Rewards’ goes:

Asked about the likely results of Pizza Hut’s popular food-for-reading program, educational psychologist John Nicholls replied, only half in jest, that it would probably produce “a lot of fat kids who don’t like to read.” (Kohn, 1999)

Kohn’s book has a bigger target of praise and gold stars in schools and performance bonuses in the workplace but his arguments that this type of vegetables-for-dessert bargaining is essentially coercive (and stacked in favour of society’s dominant power structure) and that the results are either counter-productive or short-lived (they end when the reward ends) are certainly worth keeping in mind. You’d better be careful about incentivising an aspect of the course that part of your cohort actually enjoys already for instance.

In response to the first case, I think it’s important to recognise the range of games and the types of players you’re likely to be teaching. It will be impossible to design a semester-long game that engages all of the players, all of the time. Even a leader in the field, Kevin Werbach (whose videos and articles will point you in the right direction) shies away from using an actual game mechanic in his MOOC: Gamification (run through The University of Pennsylvania on Coursera).

Conclusions

So why, as a teacher or lecturer, would you be looking to introduce game-mechanics to your classroom? The short answer is that I think it gives you another way to experiment with your teaching in a way that brings students along for the journey.

If you begin by asking ‘What is the problem that I’m trying to solve in my class?’ you might end up with a dilemma like:

*Survey results indicate that students don’t feel I’m giving enough feedback to them.
OR
*I’m not getting the quality of answers/creative output that I’d expect from this level of students.

What I think game design opens up here is the possibility for you not to simply answer ‘I’ll work harder.’ Making a game of it means you will work with students and you will help them to work with each other to solve problems.  For you, this problem of practice may or may not be openly shared with them- you’ll find a way to link it appropriately to their real success in their course or program.

I think that’s one of the powerful things about games. The ‘let’s pretend’ aspect of them imagines a world where things are simpler and clearer. Where things work. Where there are bright lines, winners and losers but also camaraderie among the players (and the umpires). It’s where achieving 10000 steps with a pedometer or staying ahead of an opponent in a ladder can be the askew goal that keeps you on track for the ‘real’ goal.

I’ll post my ‘feedback game’ ideas in the comments but to bring it back to Skinner (the Principal from The Simpsons, not the behaviorist) his error was not in the silliness of the game (all games are silly) but in assigning a game that he didn’t play himself, that he didn’t play alongside his student. And that’s the great opportunity of games in higher education — more time playing alongside instead of umpiring. Let’s continue this particular game in the comments section…

Share your thoughts about games, gameful design and gamification in the comments!

Games people play

Posted by: Jon Hurford, Senior Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

This post has been split into two parts:

Part 1: In which I outline some of my own feelings about games in the classroom.
Part 2: Which may be published tomorrow and will cover the why and how of introducing a game mechanic to your class or course…

Screenshot at 99 posts

Screenshot on the eve of 100 posts.

Games to kill time

First, this scene from an episode of The Simpsons:

[Bart has forgotten his permission slip for the class excursion to the chocolate factory and has to stay at school.]

Principal Skinner: Here’s a whole box of unsealed envelopes for the PTA.
Bart: You’re making me lick envelopes?
Skinner: Oh, licking envelopes can be fun! All you have to do is make a game of it.
Bart: What kind of game?
Skinner: Well, for example, you could see how many you could lick in an hour, then try to break that record.
Bart: Sounds like a pretty crappy game to me.
Skinner: Yes, well… Get started.  -’Bart the Murderer’ (Writer: John Swartzwelder, 1991)

Games bloggers play

For a while I’ve wanted to set down some thoughts about games and their place in the classroom. If you blog with WordPress you’ll know that you get a little badge and some words of encouragement each time you publish a post.

It’s not necessarily an earned reward either. The person who pushes the button usually isn’t the same one who wrote the post and nevermind that I only joined the team a year or so ago — in this case I’m the author and the lucky duck that gets to see 100 posts tick over. Similarly with ‘followers’ and site statistics, these two metrics of the online world are easily gained, easily gamed, but addictive regardless. At last count the tomtom has a few hundred followers spread across WordPress, Twitter and Facebook.

Fitting then that I flag the importance of fairness and that with the 100th post from the tomtom team, I weigh in on games and ‘gamification’, a topic that we haven’t really dealt with explicitly.

Paper-based games

A few years before that episode of The Simpsons, I was in 6th grade. At some point in that year, one afternoon, my teacher brought out a blue ice-cream container with cut-up pieces of paper and announced that we were going to play ‘The Fractions Game’. I got along well with my teacher and I sat at a group table near the front of the class.

But this game sounded boring and it sounded like something I wouldn’t be very good at. Plus I probably felt like this was my time to score a point in the more important social game called ’6th grade’.

I didn’t do anything elaborate: I just groaned dramatically and said ‘Not this game.’ (I’d never actually played ‘The Fractions Game’, but the title was a giveaway: these were vegetables masquerading as dessert.)

In my memory this next bit is in slow-motion. Mrs P. shouted something like ‘Right!’ and tossed the bucket of cut-up paper into the air. The pieces rained down on our group’s table and on her head and shoulders. A bit like confetti or ticker tape. But more like something very bad had just happened.

I’d never had this effect on a teacher. So unexpected and such a literal explosion. My group and I started gathering up the paper — but the ‘Right!’ was just the start of the sentence sending me to the principal’s office. And to make clear that this story is not about rewriting my history to represent me as anything like a cool and calm kid, I was definitely in tears at this point.

I was sent home that day with a note (more tears!) asking my mum to come in the next morning. I remember apologising, I remember Mrs P. explaining that she’d spent a lot of time on preparing that game.

I was an enthusiastic participant in any game Mrs P. suggested for the remainder of the year.

Computer games

The games we really enjoyed were on the Apple IIgs at the back of the classroom: Gold Dust Island and Where in the World is Carmen Sandiego. We got to play in pairs or trios I think. Gold Dust Island – especially good — had you marooned and managing water, food, treasure-hunting and shipbuilding. Looking back on the two games they’re both pretty meagre fare educationally. Carmen Sandiego was a bunch of trivia about the flags and currencies of countries and I remember that digging for gold in Gold Dust Island usually prevented us from spending the necessary time on woodcutting and shipbuilding. Still there were early lessons in opportunity costs and logic in both.

Time on the computers was probably based on some sort of behaviourist carrot and dependent on our ability to coöperate sotto voce and get off that island/find Ms Sandiego while the rest of the class were reading or doing maths problems.

I could go on about the computers and other games we played. We had the usual typing tutors and ‘drill & kill’ games. Students of my generation can usually count at least a few skills gained solely through games. Off the top of my head, for me, it would be typing speed and a smidgin of music theory. At home (or at a friend’s house) we played the increasingly sophisticated simulation games of the ’80s like SimCity. For many of my friends, games were their pathway into programming, through a language like Logo.  Many can trace a path from these experiences through to their current professions.

But If Mrs P. is reading this, she should take heart that it was her teaching strategies, her passion and the class conversations that I remember most about 6th grade and not a computer screen. ‘The Fractions Game’ was an off-day. I have vivid memories of her reading to us (Thurley Fowler’s The Green Wind (probably more tears)) and that she was a ruthless critic of my juvenile writing for instance. See how far I’ve come!

She should know too, that as an English and History teacher years later, I would stay up late making revision crosswords or flashcards that stumped my students and made me question what progress we’d made. That I ran in-class games that were unappreciated by most, or that simply crashed and burned. That I set the creation of games as assessments with very mixed results and that we played these games-of-variable-quality (set maybe in Ancient Egypt or Rome) in the final days of a term and that yes, they usually left the participants cold.

So games are attractive. As educators, it’s natural that we should see them as containers that we can sneak knowledge into, perhaps a foreign language or some critical thinking skills. Which is a roundabout way of bringing this post to ‘gamification’ and its place in TAFE and Higher Education.

But first more disclosure

I’ve recently participated in the ‘Global Corporate Challenge’. I wear a fitness monitor to track steps every day. I work on a project that is trialling badges and quests to lift the engagement of users in a professional learning approach.

All three of these activities are trying to use a game mechanic (ie points, leaderboards, quests) to increase the level of engagement/’stickiness’/personal commitment or fun. There are many more examples of gamification being used by institutions, corporations and by governments to alter behaviour.

So gamification often tries to take something most of the population experiences as tiresome or time-consuming or not intrinsically satisfying (exercise, professional development, sorting your recycling or paying your tax) and attaches an extrinsic reward to it.

And now, a game!

If this, the 100th post of the teaching tomtom gets more than 100 hits today, Thursday 3 October, I will publish Part 2 on Friday. If not, I’m following Mrs P.’s lead and throwing it to the winds! Lost to the ages!

In Part 2 I will discuss some caveats of extrinsic rewards; why and how one still might like to introduce game mechanics to a HE or TAFE course and I’ll share another anecdote about me as a learner. This time, a lecturer at university takes me to task for doing the crossword before his lecture begins.

Stay tuned/click refresh/leave a comment.

Share your thoughts about Part 1 in the comments…

Follow

Get every new post delivered to your Inbox.

Join 1,410 other followers

%d bloggers like this: