Category Archives: assessment

Reflecting on reflection: Part 2

Posted by: Mary Ryan, Associate Professor and Higher Degree Research Coordinator in the Faculty of Education at the Queensland University of Technology, Brisbane. 

A/Prof Mary Ryan at the Inclusive Conversation Series March 2014 © RMIT University, 2014, Photographer: Margund Sallowsky.

A/Prof Mary Ryan at the Inclusive Conversation Series March 2014 © RMIT University, 2014, Photographer: Margund Sallowsky.

The project team thanks the Office of the Dean of Learning and Teaching and Associate Professor Andrea Chester, Deputy Pro Vice-Chancellor, Learning & Teaching, DSC for supporting the Inclusive Conversation Series.

(This post has been broken into two parts- click here to go to the previous post.)

4Rs of Reflection

Reporting and responding

Reflective learning is a wicked skill (Knight, 2007). It has slippery definitions, is seen differently by different people, and is often treated as omnipresent rather than teachable. My response to this issue has been to find out a bit more about it. I needed evidence about things like… What constitutes reflective learning? What are the conditions under which it can happen or is taught? What do people do in different disciplines? How can it be expressed? Is it assessable?

Relating

I started by drawing on my own experiences. I realised that in my own teaching I was making assumptions about students’ knowledge of how to write an effective text, their abilities to analyse and weigh up a situation, and their skills in identifying a key issue (for them) upon which to reflect. I soon became conscious of the need for a teaching intervention. I couldn’t leave this to chance – particularly for those students for whom English was not their first language or for those who were first in their family to attend university or who had entered university through pathways other than senior schooling. So I decided some serious research was needed to help me work out how this could be improved for students in higher education. Fortunately, the Australian Learning and Teaching Council (ALTC) agreed that this was an important issue and they funded a project over two years.

Reasoning

I turned to learning theories such as Kalantzis and Cope’s Science of Learning through knowledge processes, Bloom’s taxonomy and others. A common factor across these theories was the view that learning was an active rather than passive process and that students can move from basic understandings to quite complex thinking skills of critical analysis and reasoning.  I scoured the literature on reflective learning and practice and found that it is generally accepted that there are levels of reflective thinking or learning, moving from basic identification of an issue, to dialogic thinking back and forth, to deep, transformative reflection that can change ideas or practice – hence the 4Rs that I’m using to reflect here (nothing like practicing what you preach). Plus I started to annoy plenty of colleagues at QUT – asking them about their own practices in teaching reflective learning. My colleague (Michael Ryan) and I developed the Teaching and Assessing Reflective Learning (TARL) Model: (See Figure 1: Populating the Pedagogic Field). The model considers the pedagogic field of higher

Populating the Pedagogic Field

Figure 1: Populating the Pedagogic Field (click to enlarge).

education as a space that enables increasingly more complex ideas and professional attributes to be attained (vertical axis) as students move through their degree (horizontal axis). It suggests that students can:

  • begin by reflecting on their own views and practices as a novice in the field,
  • incorporate the views and practices of others in the field,
  • reach the final goal of critically reflecting on self in relation to experienced colleagues and clients as a beginning professional.

The practical aspects of the model are the teaching patterns that are mapped to show at which point in a program they have been successful and the level of complex thinking that they can achieve. The benefits of this approach include minimising replication of activities across a program, ensuring that reflective activities are increasingly more sophisticated across a program and introducing a shared language for staff and students.

Reconstructing

I’ve learnt that a smorgasbord of reflective activities is not useful to develop levels of complexity across a program. I’ve learnt that we can’t make assumptions about students’ skills in this area. I’ve realised the importance of a shared language across programs and consistency in language within a course. Most importantly, I’ve learnt that higher education teachers really make a difference. If they prioritise and explicitly teach reflective learning, students can progress to those deep levels of self-reflection. New applications of this work have been in areas of peer review – teaching students how to write a reflective and useful review and how to respond reflectively to peer feedback; as well as teaching students how to evaluate university teaching and courses in a more reflective way as co-contributors to the learning experience. From here, I think I need to work more with academic staff in helping them to implement some of the great resources and strategies from the project.

References:

Knight, P 2007, Fostering and assessing ‘wicked’ competences, Milton Keynes, Open University.

Murphy, KR, 2011, ‘Student reflective practice – building deeper connections to concepts’, ASCD Express, Vol. 6, No. 25

Ryan, ME & Ryan, MC 2013, ‘Theorising a model for teaching and assessing reflective learning in higher education’, Higher Education Research & Development, vol 32, no. 5, pp. 244-257.

Ryan, ME, 2014, Teaching and Assessing Reflective Learning in Higher Education, Inclusive Conversation Series, RMIT, March 2014 presentation.

Share your thoughts on reflection in the comments section!

_________
Find us on:

 

Reflecting on reflection: Part 1

Posted by: Mary Ryan, Associate Professor and Higher Degree Research Coordinator in the Faculty of Education at the Queensland University of Technology, Brisbane. 

The project team thanks the Office of the Dean of Learning and Teaching and Associate Professor Andrea Chester, Deputy Pro Vice-Chancellor, Learning & Teaching, DSC for supporting the Inclusive Conversation Series.

(This post has been broken into two parts- click here to go to the next post where I apply the 4Rs to my own experiences.)

Reflective practice is often described as being as much a state of mind or attitude as it is a set of activities. It requires educators to assess themselves and their practice and as a result of this process become, “conscious agents in their own pedagogy” (Griffiths: 2010).

Screen shot 2014-04-15 at 11.45.45 AM

A/Prof Mary Ryan at the Inclusive Conversation Series March 2014 © RMIT University, 2014, Photographer: Margund Sallowsky.

My work as a teacher of undergraduate and postgraduate Education students for many years has shown me how much students can benefit from explicit teaching of critical reflection to improve their learning. This has motivated my work on developing students’ reflective learning capacities over several years — first as a teacher working directly with students and in the past few years supporting other teachers, program coordinators and support staff to develop a systematic curriculum model with practical strategies and resources that builds students’ capacities in reflective learning.

At the end of this page I’ve provided some resources that could help you in your teaching and in Part 2 I’ll share my reflections (on reflection) using this 4R model.

When educators reflect on their teaching, their practice improves. Students can also benefit when they reflect on their learning experience or practice. Murphy (2011) states the act of reflecting on an experience or critical incident, leads to students making deeper connections to the concepts they are learning beyond the rote memorisation or simple completion, resulting in students experiencing an ‘a-ha’ moment.

Reflective learning is a way for students to:

  • develop problem-solving and critical thinking skills

  • consider different possibilities and actions

  • link old ideas with new ones

  • stimulate creative solutions

  • encourage life-long learning

  • draw on evidence to plan future actions

  • improve practice

  • create cohesiveness across a course/program. (Ryan, 2014)

How can I integrate reflection into teaching and assessment?

Designing a practice of reflection means both clarifying the purposes it needs to serve and identifying opportunities for reflection in students’ work that are realistic and yet occur at the right intervals with sufficient depth to be meaningful (Murphy 2011).

At RMIT, there are a number of teaching staff who have introduced reflective practice into their curriculum in courses such as Fashion and Textiles, International Development (GUSS), and Media and Communication to name just a few. And there would be countless staff who use the word ‘reflection’ in a task or in assessment criteria. What makes our project special is the real examples of reflection shown within their discipline. These patterns have been linked (where appropriate) to professional standards for the accrediting bodies and plotted on a graph to

Populating the Pedagogic Field

Populating the Pedagogic Field

show how they increase in complexity, or how they move from simulated to real experiences. (See the figure: ‘Populating the Pedagogic Field’ to the right and click to expand.)

Some examples have been selected here from the Developing Reflective Approaches to Writing (DRAW) Wiki to illustrate how you could introduce reflective practice into the course. The patterns include teaching resources including annotated examples of reflective writing, and student blogs:

Analysing Reflective Texts (ART),

Mapping Critical Incidents – Foundation (MCIF)

Reflections Around Artefacts (RAA)

Reflection as a Professional Activity during Service Learning (RPA)

Resources:

The Developing Reflective Approaches to Writing (DRAW) Wiki: holds the teaching patterns and common resources for over 20 patterns that are being used in different disciplines. The DRAW website (http://www.drawproject.net)provides a short summary of the project and references.

References:

Knight, P 2007, Fostering and assessing ‘wicked’ competences, Milton Keynes, Open University.

Murphy, KR, 2011, ‘Student reflective practice – building deeper connections to concepts’, ASCD Express, Vol. 6, No. 25

Ryan, ME & Ryan, MC 2013, ‘Theorising a model for teaching and assessing reflective learning in higher education’, Higher Education Research & Development, vol 32, no. 5, pp. 244-257.

Ryan, ME, 2014, Teaching and Assessing Reflective Learning in Higher Education, Inclusive Conversation Series, RMIT, March 2014 presentation.

Share your thoughts on inclusive teaching and assessment in the comments section! Click here to read Part 2 of this post!

_________
Find us on:

Peer Partnerships: the professional development program that really resonates

Posted by:

Angela Clarke, Senior Research Fellow,
A/Prof Andrea Chester, Acting Deputy Pro Vice-Chancellor, Learning and Teaching
&
Dallas Wingrove, Senior Advisor, Learning and Teaching, Property, Construction and Project Management, 
College of Design and Social Context, RMIT University.

Rebekha Naim and Shayna Quinn are peer partners from the School of Media and Communication.

Rebekha Naim and Shayna Quinn are peer partners from the School of Media and Communication.

The RMIT Peer Partnerships program has been running for two years. Over that time the program has been successfully implemented in 17 Schools/centres and units across campuses in Australia and Vietnam. There have been over 195 peer partners and 24 leaders who have participated. The response from academic staff has been overwhelmingly positive, as evidenced by the following comments:

*One of the most positive and enriching professional development experiences I have had in nearly ten years at RMIT. 

*Best PD ever! To be great, you have to want to be great, there is always room for improvement as an academic…this is a great structure for reflective practice in teaching and learning.

*Very positive and insightful. Helped me conceive and expand possibilities for my own teaching practice. Sparked new ideas.

*This was a valuable program, which fostered teaching skills in ways both practical and theoretical.

*I found peer partnerships to be a really effective way of learning and reflecting on practice. It was particularly rewarding as a sessional teacher as many other opportunities aren’t open to us so it made me feel part of the RMIT team – for possibly the first time – which is really important to me.

These responses from our teaching staff suggest that the RMIT Peer Partnerships program is truly resonating with our staff and is having a positive impact. This is due to many factors including the structure of the program and the underpinning principles which foreground voluntary participation, reciprocal exchange and confidentiality. Many academics feel that this program is filling a professional gap in their academic work.  The process, which focuses on reciprocal observation of teaching practice, has generated meaningful professional conversations about teaching and learning and is fostering collegiate communities of practice within Schools/units.  For some staff who are seeking promotion or a teaching award, Peer Partnerships have offered a safe way in to the experience of peer observation and feedback for continuous improvement.

The program is open to all academic teaching staff, including sessionals and caters for early, mid and experienced professional development needs. The program has been specifically devised for implementation within the local context of a School.

We are now in our third year of implementation. This year we are very pleased to announce that two College representatives will co lead Peer Partnerships with us: Laurine Hurley in the College of Science, Engineering & Health and Tom Palaskas in the College of Business. We would like to welcome anyone who is interested in Peer Partnerships to contact dallas.wingrove@rmit.edu.au or angela.clarke@rmit.edu.au.

For further information visit our website www.rmit.edu.au/teaching/peerpartnerships.

Share your thoughts and questions in the comments section!
_________
Find us on:

The art of questioning

Posted by: Associate Professor Andrea Chester, Deputy Pro Vice Chancellor, Learning and Teaching, Design and Social Context, RMIT University.

Over the years I have read quite a bit on the use of questions in education. There are a number of useful websites that describe the importance of questioning in teaching and defining different types of questions. The UNSW L&T Unit has one, as does Cornell’s Center for Teaching Excellence.

make-just-one-change-smAs part of the toolkit of a student-centred teacher, questioning has long been considered a core skill for all teachers from primary through to the tertiary context. So central is the skill that Mary Jane Aschner (1961) described educators as ‘professional question makers’. I know from experience how a well-chosen question can open up a rich discussion and how the wrong question can close it down.

What I hadn’t spent much time thinking about was how to encourage students to ask their own questions.  That is, until I read Make just one change: Teach students to ask their own questions by Dan Rothstein and Luz Santana.

Rothstein and Santana argue teaching can be transformed if students, rather than teachers, assume responsibility for questioning. They argue that asking questions is fundamentally empowering and that all students can and should learn to formulate their own questions and that all teachers can integrate development of the skill into their regular teaching practice. While the book is pitched at teachers of K-12, the ideas can easily be applied to tertiary teaching.

The book and associated online resources available through the Right Question Institute set out the seven-step Question Formulation Technique (QFT™) summarised below.

Step

Description

Principle

Approximate time allocation

  • Begin with a question focus (QFocus)
This is the work you do to develop a stimulus, which can be a topic, a sentence, an image or an object, depending on your purpose, but generally not a question itself.  -  (varies)
  • Share and discuss the rules for producing questions
There are four rules:

  • ask as many questions as you can
  • do not stop to discuss, judge or answer questions
  • write down each question exactly as it is stated
  • change any statements as questions 
Meta-cognition 5-7 minutes
  • Produce Questions
In small groups students produce as many questions as they like within the available time. Divergent thinking 5-8 minutes
  • Categorise Questions
In this step students are encouraged to improve their questions by labeling each as either open or closed and discussing the differences between these two types of questions. Students are encouraged to change some questions from one type to the other to explore the difference in possible responses. Analysis and convergence 5-10 minutes
  • Prioritise Questions
Students prioritise their questions, articulate their rationale and select the three most important questions. They note where these questions occurred in their original list of questions, at the beginning, middle or end. - 5 minutes
  • Next Steps
Here students consider how they will use their questions. This depends on your purpose for engaging in the activity. Application Will depend on purpose of QFT™
  • Reflection
Finally students reflect on what they learned and how they can use it. Meta-cognition 5-8 minutes

The entire sequence can be completed in 30 minutes, with a reduction as you and the students become more familiar with the process.

When you might want to use the QFT™
The QFT™ can be used to:

  • generate interest at the start of a course

    Click on the image to see Dan Rothstein's TEDx presentation: "Asking questions is the single most powerful renewable source of intellectual energy: It's in our minds we can create it, we can create it continuously..."

    Click on the image to see Dan Rothstein’s TEDx presentation: “Asking questions is the single most powerful renewable source of intellectual energy: It’s in our minds we can create it, we can create it continuously…”

  • introduce a new topic
  • assess and/or deepen understanding
  • stimulate new thinking
  • prepare for an assessment task
  • conclude a topic or course.

What to know more?

References:

Aschner, M. J. (1961). Asking questions to trigger thinking. National Education Association (NEA) Journal, 50, 44-46.

Share your thoughts and questions in the comments section!
_________
Find us on:

What does ‘good’ look like?

Posted by: Ruth Moeller, Lecturer in Education and Senior Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

Series of sticky notes all saying 'good'.Let me begin with an anecdote.

Several years ago, I returned to study to do a post graduate course in Organisational Behaviour. I remember the first assessment clearly, we had to write an analysis of a group situation, I think it was about 1500 words.  I remember the anxiety, I had no idea of what I was doing. I had done the reading, attended the classes, consulted fellow students but in writing my analysis I made the best go of it I could but really had no idea of what was required.

I missed the class when the work was returned, so had to catch up with the lecturer at another time – I still remember the nervousness and trepidation I felt in waiting for my paper, and I did ask her, ‘Just tell me if I passed or if I have to do it again.’ When I got the paper back I got an HD, I still don’t know how, and I suspect that the lecturer regretted the mark, when she realised that I didn’t really know what I was doing!

The purpose of this anecdote was not to tell you I got an HD or to share my neuroses, but rather to make the point that when assessing and grading students they need to know what is expected and to what standard. Or to put it another way, ‘What does “good” look like?’

This is particularly important for students transitioning: from school to tertiary studies, from vocational to higher education or from one year level to the next. Expectations can be different, so we shouldn’t assume that students will understand what is required of them.

To help, consider these three questions:

  1. What criteria are you using? Are you assessing a product, application of theory, diverse reading, critical analysis, spelling and grammar, team work? Make this clear to the students and then they can aim to demonstrate what they can (or can’t) do, rather than try to guess what you want.

  2. What does ‘good’ look like?  You may have assessment criteria but when you are grading, could you explain to a student the difference between a Credit and a Distinction?  “It’s just the vibe of the thing…” (Dennis Denuto in The Castle) isn’t a satisfactory explanation.  This is often highlighted when a student questions their grade and asks what was missing. What did they need to do to get a higher grade?  Rubrics can help here.

  3. Can you explain what students have to produce? Even better, are there examples they can look at?  Students like to see what is required. You think you have clearly articulated the requirements but nothing beats a physical example. I get my post grad students to write wiki posts, and until I provided sample posts, I was always fielding questions about what was wanted, even though I thought it was clearly explained in the course guide.

It is Week 2 for Higher Education and Week 5 for Vocational Education, so it’s not too late to review your assessment tasks and see if there are ways to make them student-friendly rather than ‘guess what I want’ tasks.

Resources that can help:

Assessing student work
Rubrics for assessing English language and academic literacies

Share your thoughts on what ‘good’ means in the comments section!
_________
Find us on:

First impressions

Posted by: Jon Hurford, Senior Advisor, Learning and Teaching & Andrea Wallace, Educational Developer, College of Design and Social Context, RMIT University.

Andrea is a member of the Inclusive Teaching and Assessment Practices Project working to develop resources and deliver professional development to staff.

Sign in a school bus reads: 'Keep Out. Please put on seatbelt + be quiet + behave. Thanks'

A sign in a school’s excursion bus at the Old Melbourne Gaol this week.

As the saying goes, ‘You don’t get a second chance at a first impression.’

This week, across Australia, thousands of lecturers and tutors will be meeting their new students or welcoming back continuing students.

In Vocational Education, classes have been back for almost a month, but still, it’s early days.

It’s obvious that getting off on the right foot and creating the right environment for students has a special importance in tertiary education. For one thing, in a 12-16 week delivery schedule, the feeling that time is precious is understandable.

In wanting students to take our course seriously, in the feeling that we’re competing for the mindshare of their course load, in rushing about, is there a risk of putting up a sign (metaphorically) like the bus driver (or the staff who share driving duties) in the picture to the left? Note the ‘Keep Out’ in red and the tiny ‘Thanks’ at the right. What messages are we sending students in their first classes?

So this post is just a quick reminder that in the midst of all the organisational and administrative tasks we should still hold our personal philosophy of education front-and-centre and be enacting the strategies and principles that brought us through university as learners, and that brought us to university to teach.

The Inclusive Teaching and Assessment Practices Project has a set of principles that might help you see that big picture (or the jigsaw pieces of the bigger picture that are your courses) and we can’t think of a better time of the year (at least for our southern hemisphere audience) than now to put them in front of readers. Each of the following links has an associated page with key questions, resources and examples of the principle in use:

Getting a piece of writing from your students in a class early in your teaching schedule is an easy diagnostic tool. You’ll get to know a key aspect of their learning skill set and coupled with a quick survey you can get an impression of what your students expect from the course. Perhaps you teach online (or you’ve taken these elements of your course online) and you use a discussion board or blog for this. You can probably see how this simple task hits many of the principles above– if you’ve asked students what was their inspiration to study a certain discipline; if you’ve read the responses and turned them around to the students quickly; if you’ve then provided the means for them to share their responses and maybe organise themselves in study groups based on this for the first assessment, you’re establishing an environment that is ‘feedback rich’.

But what about longer pieces of writing? What about supporting your students in documenting their progress in your course?

Next week, there’s an opportunity for all staff at RMIT as Associate Professor Mary Ryan (School of Education, QUT) delivers a lecture and workshop on the Teaching and Assessing Reflective Learning (TARL) model that she and her team developed in a recent OLT project.

Professor Ryan will explain how the systematic approach can be used to embed the pedagogy of reflective writing across courses in different disciplines.  The workshop will explore the suite of pedagogical patterns and accompanying resources for systematically teaching and assessing reflective practice underpinned by the TARL and EPC models.

Lecture: Teaching and Assessing Reflective Writing
Thursday 13 March 2014
11.30 am – 12.30 pm
Building 80, Level 1, Room 2
Workshops: Teaching and Assessing Reflective Writing
Thursday 13 March 2014
12.30 – 2.20 pm
Building 13, Level 3, Room 5 (City)
Or
Friday 14 March 2014
11.00 am – 12.30 pm
Building 514, Level 1, Room 2 (Brunswick)

Registration for the workshops is essential. Space is limited. Click here to register (RMIT Login required).

Oh, and on the topic of first impressions, we’d also like to mention two blogs that have made good impressions on us during the break and will be of particular interest to casual, sessional and part time staff:

We recently added them to our blogroll (right of screen)– go visit their site for more perspectives on starting the year.

Share your thoughts on first impressions, inclusive teaching and reflective writing in the comments section!
_________
Find us on:

Learning Analytics: What does it all mean?

Posted by: Erika Beljaars-Harris, Educational Developer, Learning and Teaching, College of Design and Social Context, RMIT University.

Ever heard of the term ‘Learning Analytics’? If you haven’t, then you will. The 2013 Horizon Report describes it as the “[f]ield associated with deciphering trends and patterns from educational big data, or huge sets of student-related data, to further the advancement of a personalized, supportive system of higher education.”  What does this all mean? It means that we can gather student data to uncover trends, patterns and issues. It’s what we do with that data and how we can support the student that is the key.

Click on the image to explore educause.edu's resources on learning analyticsThe report also leads us to believe that it will take 2-3 years to adopt. However I believe it’s already here.

For example, in Blackboard you can access the ‘Performance Dashboard’ (from the Control Panel) to ascertain when a student last entered the course and drill down to the exact date and time they entered. As an instructor you can also view the last date and time that you accessed the course. This means that you (as an instructor) can confirm the amount of interaction the student is having with the online course. As I am a Blackboard gal, I presume that this is all possible with other learning management systems (LMS). Regardless of what LMS you use, there is already the capacity to obtain some basic data on students and instructor navigation within an online course.

Click on the image to explore educause.edu's resources on learning analyticsUseful? You betcha. Think of it this way, you are able to determine those students who have not accessed the course in the first week, this is a red flag. One possible intervention method is to contact the student and notify them that they haven’t accessed the course and you want to ensure that they are not having any technical issues, access issues, or any other issues. Then, the student emails you back with ‘thanks for your email I had problems accessing my course as I am located in a remote part of Australia/America/Afghanistan’ (wherever). Problem solved.

And this is only the beginning of what learning analytics can do. It can predict the learning route of a student, it can assist in personalising the student’s learning, and it can recommend and apply interventions. As an instructor (with some setup) Blackboard can present the results of your assessment with full item analysis, meaning that you can look at what aspects of a course or topic your cohort found difficult and what they have mastered. You can use this data to modify your teaching after (or even during) the semester.

There are already criticisms to learning analytics including: ethical issues on the collection of data, who owns the data, the sharing of data, privacy and legal issues too. These are all valid concerns that need to be navigated carefully. Regardless of the route, learning analytics is here, and it’s only gaining ground.Screen shot 2013-12-05 at 1.50.51 PM

If you’re still not quite sure what learning analytics is, take a look at the infographic “Learning Analytics” produced by Open Colleges. It provides an excellent breakdown of what it is. If you still have more question, as we all do. Try www.educause.edu and do a search on learning analytics. You will find plenty of resources.

References:

Horizon Report. 2013 Higher Education Edition. Retrieved from http://net.educause.edu/ir/library/pdf/HR2013.pdf

‘Learning Analytics 101. Leveraging Educational Data.’ Open Colleges. Retrieved from http://net.educause.edu/ir/library/pdf/HR2013.pdf

Share your thoughts on learning analytics in the comments!

Don’t forget you can subscribe to have the tomtom delivered to your email as soon as it’s published and you can follow us on facebook: www.facebook.com/TeachingTomTom.

Procrastination

Posted by: Meredith Seaman, Senior Advisor, Learning and Teaching, College of Design and Social Context, RMIT University. 

Academics can sometimes hold very negative perceptions of students as lazy and question their ability to meet deadlines and submit assignments on time. Academics can even be a bit gleeful in their enthusiasm in coming up with rules and penalties for late submission. Reinforcing deadlines is critical if we are trying to teach students about the realities of the workforce, but can’t we all relate to students who struggle with time management or procrastination? If we’re being honest, don’t we all struggle with deadlines and more specifically procrastination on difficult tasks like writing articles? I’ve noticed for myself that writing can prompt anxieties and very similar avoidance strategies that I had sadly practised back as a student. Of course, very real issues including illness, unrealistic goals and workloads get in the way, but we’re also all well aware of a host of procrastination techniques in ourselves and observed in others.

I surveyed colleagues and friends to find out a bit more about how they procrastinate and whether they have any useful (online) tools or strategies that might help them avoid procrastinating. Finally I begin to consider how we might better support students to stop procrastinating and submit on time.

I wonder if procrastination needs much introduction…

I don’t know if there is anyone who doesn’t sometimes procrastinate. For me, when procrastinating, I go to the internet and social media tools. The most mundane of internet games and the worst of television shows all take on new importance. Research has demonstrated that technologies can similarly tempt students to procrastinate and it shouldn’t surprise us that they’ve also demonstrated links between Facebook and procrastination.

Yet it isn’t a new phenomenon

I used to bake cookies before Facebook and iPads, Candy Crush and Pinterest. I have the recipe written down in my recipe book as “procrastination cookies”. So a Facebook restriction will result in other types of procrastination. Cooking, cleaning, sleeping… — Erica

file000184731991

Before the internet, as my friend Erica says, there was cleaning and baking. Tax returns might even get done if avoiding writing or marking papers. If these can be kept in hand, tasks (like setting up a conducive working space) can be appropriate precursor activities before sitting down to get through some marking for instance. Many people have rituals they have to go through before sitting down to work.

In ‘Waiting for the motivation fairy’ Hugh Kearns and Maria Gardiner (2011) also remind us that procrastination can be ‘far more subtle, and can even be taken for productive work’ such as digging up elusive references, starting new projects or experiments, chasing up elusive but perhaps unnecessary references, checking emails.

Is it necessarily a bad thing?cleaning

We certainly need breaks. Breaks are essential for deep thinking and assimilation of ideas and concepts, critical for creativity, to occur. Walking, gardening or other simple repetitive tasks not taking much concentration can help the creative process. Productive and creative ‘types’ throughout history have often taken dramatic steps to increase their productivity and avoid procrastination, some common elements include daily set periods of work, clear targets of how many words to achieve, but they also had breaks such as time out for day jobs and long walks.

It’s important to think about why you might be procrastinating and not be too judgemental or hard on yourself:

Reward yourself for work done. Punishment never works, it just creates more procrastination. Sometimes laughing helps to, to take the pressure off: I love PhD comics. Oh, and getting to know the difference between avoiding because you’re lazy, and avoiding because you’re actually on the brink of a brainwave… — Lisa

I think the unconscious aspects of self sabotage often need to be addressed carefully rather than becoming stentorian with oneself… — Fiona

Sometimes we may just be stuck on something or need to approach a different way. Other times a task may be overwhelming or crippling, and strategies are needed to address the procrastination.

filing

Are there any solutions?

One colleague successfully uses Pomodoro as her procrastination avoidance tool. In brief, it’s a pre-set strict period of work, using a timer, where interruptions are carefully managed with breaks interspersed. Another finds it really useful to get up very early in the morning at the same time each day to write. The lack of interruptions and being a bit less awake may actually be a benefit to productivity in his case. Some people have joined support networks such as “Shut up and Write” where interested people meet at a cafe and write in short bursts and then have a chat to each other as well.

Kearns and Gardiner identify three techniques which provide a good summary of key practices to hold procrastination at bay:

1) big projects need to be broken down into steps (perhaps even tiny ones)

2) set a time deadline by which to perform that tiny step

3) build in an immediate reward.

Implications for assessment design

If we think about assessment design in the context of the conditions that may contribute to procrastination, then as academics, we would want to avoid setting unclear tasks; tasks without any progress points or milestones and tasks that feel too big and complex to get started. They may all affect student motivation and their ability to make a start. If ‘action leads to motivation, which in turn leads to more action’ (Kearns and Gardiner, 2011) then designing assessment that encourages students to get started makes sense. So think about breaking up some big assessments into smaller components with earlier due dates to get students started and on the right track. Provide them with feedback, early on. Even better: work with your students to help them to break up the assessment tasks. Also, think about rewards versus the perceived ‘threat’ or pressures associated with assessment tasks.

coffeeIf redesigning your assessment for the next semester seems like a big task at the moment, don’t put it off! Break it down, set some dates and reward yourself!

Thanks to Fiona Collins, Lisa Farrance and Erica Walther for their input into this post.

References and more information:

Share your thoughts and strategies in the comments!


Find us on Facebook http://www.facebook.com/teachingtomtom and Twitter @teachingtomtom

Assessment, Grade, Holidays…

Posted by: John Benwell, Principal Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

HD rubricIn Melbourne, it’s the last week of scheduled classes and nearing the long break over the southern summer. Whilst thoughts may be wandering towards holidays, sun, the beach and all those great ‘Aussie’ pastimes, it’s time to assess and grade our students.

All the formative assessment has been done; we have provided vast quantities of feedback to our students and maybe provided avenues for feedback from their peers; now is the time to give each one a grade.

So I thought for my last post for the year, and before running to the beach to go sailing, I’d do a light refresher on assessment and point to some resources on rubrics to help you through.

At this stage of the year, we already know how most students are going. We have been giving them formative assessment tasks, and providing feedback to them so they can learn from their mistakes, fill the gaps and polish their performance. There have likely been summative assessments, which have been building towards a final mark.

Maybe you’ve read (and found useful) other tomtom posts that have discussed aspects of assessment (like Thembi’s post on Active Learning Strategies, Meredith’s post on Academic Integrity, Alex’s post on Peer and Self-Assessment and my previous post, Keeping Watch on using assessment to track our students’ progress), so here is a post with some handy references on assessment and rubric development.

In the old days, students would now start cramming, revising knowledge and processes, going to the library and doing old exam papers hoping their lecturer would just revise the last year’s paper for this year.

Nowadays, we hope those student expectations are well behind us and the last assignment encapsulates the skills, knowledge and the application skills and knowledge into a capstone assessment experience to confirm the student has achieved the course (subject) learning outcomes.

In pondering assessment we should never lose sight of what assessment is, and its purpose. As a reminder, here are the core principles of assessment from The University of Melbourne’s  Centre for the Study of Higher Education:

  1. Assessment guides and encourages effective approaches to learning
  2. Assessment validates and reliably measures expected learning outcomes, in particular the higher–order learning that characterises higher education
  3. Assessment and grading defines and protects academic standards.

At this stage of the year, principle 1 should have provided students with tasks that permitted them to test their learning and understanding in their passage towards the achieving the learning outcomes. The best scenario would be student-lecturer negotiated, multiple learning and assessment tasks that were designed to increase in complexity over the semester.

Principle 2 reminds us that we should not be simply testing students’ knowledge, but more their application of skills and knowledge and their ability to independently think though increasingly complex problems associated with their intended discipline. And we must grade each student in a reliable and repeatable manner. More on how we do that later.

Principle 3 helps maintain our standards. Not only do we have to assess if they have achieved the course learning outcomes, but also how well they have achieved it with reference to industry standards and moderation across institutions. Painfully for a teacher, we also must decide if a student has not achieved the learning outcomes.

Using the results from several forms of assessment during the semester, we need a framework to grade effectively. We need to have a considered series of statements that allow us to assign an overall grade to each student. They are like performance indicators. These statements are incremental performance levels of the learning outcomes. The levels are based on professional judgement, industry expectations and the quality standards of the university.

Commonly referred to as rubrics, you should develop a set of guidelines for marking and grading. They are not rules, but a framework to help you and your co-assessors be consistent across the group of students, from year to year and to maintain the academic quality standards expected by your industry/discipline and the students’ future employers. Your institution also relies on your professional judgement to uphold the standard of its awards.

Some argue that rubrics are restrictive, but with a well-developed set of rubrics, time is saved, consistency is improved, standards are upheld, and the course remains constructively aligned — the rubrics being generated from the learning outcomes. Levels of attainment between are  documented and described for the students to see. These can be a little bit fuzzy in their generic form but discussion with students, providing exemplars and using the same rubrics for peer and self-assessment can enhance all participants’ understanding of what a rubric is trying to do. Rubrics can be applied to all types of assessments: essays, drawings, pictures, models, presentations, designs and films.

The links below have some useful reading on rubrics, their purpose, value and how to write them:

http://aadmc.wikispaces.com/file/view/Assessment.pdf

http://www.edutopia.org/assessment-guide-rubrics

http://www.cmu.edu/teaching/designteach/teach/rubrics.html

Unfortunately some students, despite all attempts by us, fail to provide us with the evidence they have achieved the learning outcomes. The result is then a fail. There wouldn’t be a lecturer or educator who does it lightly, but it’s part of upholding the professionalism of our discipline, and the standards of our university. The determined learner who fails will return and do the course again, learning from their mistakes, and hopefully will achieve a better mark the next time around.

So before you start marking this semester try developing your own rubrics. Start by writing your learning outcomes on the left of a tabjboceanle, and then use grade descriptors of what you would expect to see from the students in the boxes. Your learning and teaching advisor can help you create them and your results will be fairer and more consistent. Rubrics are also a great help when marking online or if you have several tutors performing the assessment. What’s more, you can save time!

Well my last student is graded, so I’m off to a BBQ tonight and out into the ocean (Bass Strait) for a sail on the weekend.

I hope you have an enjoyable break.


Find us on Facebook http://www.facebook.com/teachingtomtom and Twitter @teachingtomtom

Games people play Part 2: Let’s pretend

Posted by: Jon Hurford, Senior Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

Metrics

Fitbit Screengrab

Author is comfortably mid-table at 10000 steps a day. Less than Andrea W, more than The Phantom.

Last week was quite a fun post to write and I’m back with the semi-promised ‘Part 2′…

We do often get between 100 and 300 daily hits on a published post at the tomtom. But my post, the 100th post fell a little bit short. But since then there’s been quite a bit of activity and as a number of people pointed out in the comments, my threat to send it to blog-post-Heaven made it seem:

  • like a waste of my effort/their invested time
  • like an arbitrary requirement
  • unfair to demand a group of people to reach an aggregated target
  • churlish for me to take my ball and go home
  • that perhaps at 1400 words I just didn’t know how to end the post?

All true!

What a savvy readership!

And all so relevant to games- Are the rules clear? Are the rules fair? Who thought up these rules? Who’s the umpire? How do I win? How do I quit?

On to what I’d promised, first, extrinsic motivations and measurements.

There’s already a game in place in TAFE and HE that our students play. The game called, for instance, ‘Bachelor of X’ runs for three years, it’s assessed by former players and it will cost you upwards of $30 000 AUD. Insert as many asterisks as you’d like, but essentially you quit by dropping out, you win by graduating, you can cheat in a number of ways and at the end you’re awarded a badge called a degree.

A cynical sketch of the tertiary experience, yes. Shoulder angels should rightly counter with the intangibles and the intrinsic benefits that come with a tertiary experience. University is where students can grapple with ideas, create new knowledge, speak truth to power etcetera.

Click here to visit GEElab. Opens in a new window.

‘Trouble Tower’ app from RMIT’s GEElab.

It would be depressing (or worse) if we held the first view front of mind and didn’t encourage the myriad benefits related to what learning institutions offer. But you’ll find plenty of posts on the tomtom where you can read about graduate attributes, lifelong learning and that sort of thing!

Caveats

My point though is that adding another layer of achievements as instructors is problematic for a system that already has clear thresholds, ladders and badges. Completion of credit points, Competent/Not Yet Competent, Pass to High Distinction: these are the real points of the game. Universities bear the legacy of ranks and hierarchies in their inherited Latin and medieval terms. Just as more recently we have had the AQF imposing a kind of metric system of levels 1-10 on these old terms from guilds, knights and churches.

So an easier conclusion to this post would be for me to write about what could go wrong with adding a game element to your course. That it’s ‘pointsification‘, that it’s infantile, that there’ll be unintended consequences. That it makes university into (even moreso) a token economy. That carrots and sticks are for donkeys. Adding gameful design to your course won’t make up for opaque course outcomes or dated course materials.

But I did promise to explain why an individual instructor might still be interested in adding some sort of a game mechanic to their course.

Click to read the article at news.Discovery.com

A 5000 year old Bronze-Age game: “According to distribution, shape and numbers of the stone pieces, it appears that the game is based on the number 4.” Haluk Sağlamtimur, Ege University İzmir, Turkey. Click above to read the full article.

Case 1: You enjoy games (boardgames, word games or computer games) and recognise that games can add an element of fun to tasks. You want to allow students to learn, track their learning or to present the results of their learning in a modality that’s closer to one in which they’re spending some of their leisure time.

Case 2: You recognise that there’s an element of your course that’s an ‘eat-your-vegetables’ proposition: it needs to be done and many of your students find it difficult. As a result, it’s often skipped over by students or it becomes a point where their performance dips or where they disengage. It might be something like acquiring the appropriate vocabulary for a unit, or acquiring a set of technological skills that are required that can be applied later in the unit. You think that maybe some healthy competition or a bit of incentivising could do the trick.

To address the latter case, it’s worthwhile noting that psychologists talk about the overjustification effect, where extrinsic rewards reduce intrinsic motivation. As one of the examples in Alfie Kohn’s very quotable ‘Punished by Rewards’ goes:

Asked about the likely results of Pizza Hut’s popular food-for-reading program, educational psychologist John Nicholls replied, only half in jest, that it would probably produce “a lot of fat kids who don’t like to read.” (Kohn, 1999)

Kohn’s book has a bigger target of praise and gold stars in schools and performance bonuses in the workplace but his arguments that this type of vegetables-for-dessert bargaining is essentially coercive (and stacked in favour of society’s dominant power structure) and that the results are either counter-productive or short-lived (they end when the reward ends) are certainly worth keeping in mind. You’d better be careful about incentivising an aspect of the course that part of your cohort actually enjoys already for instance.

In response to the first case, I think it’s important to recognise the range of games and the types of players you’re likely to be teaching. It will be impossible to design a semester-long game that engages all of the players, all of the time. Even a leader in the field, Kevin Werbach (whose videos and articles will point you in the right direction) shies away from using an actual game mechanic in his MOOC: Gamification (run through The University of Pennsylvania on Coursera).

Conclusions

So why, as a teacher or lecturer, would you be looking to introduce game-mechanics to your classroom? The short answer is that I think it gives you another way to experiment with your teaching in a way that brings students along for the journey.

If you begin by asking ‘What is the problem that I’m trying to solve in my class?’ you might end up with a dilemma like:

*Survey results indicate that students don’t feel I’m giving enough feedback to them.
OR
*I’m not getting the quality of answers/creative output that I’d expect from this level of students.

What I think game design opens up here is the possibility for you not to simply answer ‘I’ll work harder.’ Making a game of it means you will work with students and you will help them to work with each other to solve problems.  For you, this problem of practice may or may not be openly shared with them- you’ll find a way to link it appropriately to their real success in their course or program.

I think that’s one of the powerful things about games. The ‘let’s pretend’ aspect of them imagines a world where things are simpler and clearer. Where things work. Where there are bright lines, winners and losers but also camaraderie among the players (and the umpires). It’s where achieving 10000 steps with a pedometer or staying ahead of an opponent in a ladder can be the askew goal that keeps you on track for the ‘real’ goal.

I’ll post my ‘feedback game’ ideas in the comments but to bring it back to Skinner (the Principal from The Simpsons, not the behaviorist) his error was not in the silliness of the game (all games are silly) but in assigning a game that he didn’t play himself, that he didn’t play alongside his student. And that’s the great opportunity of games in higher education — more time playing alongside instead of umpiring. Let’s continue this particular game in the comments section…

Share your thoughts about games, gameful design and gamification in the comments!

Follow

Get every new post delivered to your Inbox.

Join 1,180 other followers

%d bloggers like this: