Towards sustainable assessment: some thoughts

Dr Peter Rushbrook, Deputy Head, Learning and Teaching at RMIT University’s School of Education shares some thoughts and references about sustainable assessment. 

image depicting a session in a workplace training centre at a major Singapore supermarket chain.

This is an image depicting a session in a workplace training centre at a major Singapore supermarket chain. The trainees are undergoing assessment of their checkout skills, including managing cash and customer service. This is a simulation but the skills learned are put into practice quickly in the demonstration store next door.

Thinking of assessment for, as and of learning (Earle, 2006), not just of learning, is no longer new. What is new are approaches to assessment that build on this basic premise to support Twenty-first century learning. Paying attention to assessment is critically important for adult learning as David Boud (2010) points out,

Assessment is a central feature of teaching and the curriculum. It powerfully frames how students learn and what students achieve.

Assessment is a process, not just an end result.

Assessment for longer term learning focuses on higher order thinking and skills, such as exercising judgment in context. It develops independent, confident practitioners ready to transition to the next phase, ready to work independently and with others to make informed judgments (Boud 2010). We need to focus not just learning how to learn, but on the subset of learning – how to assess (Boud 2000). We need to move away from multiple choice questions and away from assessors looking for answers from an answer sheet. Such practices do not reflect the increasing cognitive (Darling-Hammond, 2014) and complex psycho-social demands required of our workforce. They fail to contribute to the application of learning to contexts outside a classroom or test environment. Essentially, we need to move from narrow assessment to assessment for deeper sustained learning.

Assessment needs to do double duty: both for credentialing and learning purposes. This includes: formative assessment for learning and summative for certification; focusing on the task and developing lifelong learners; and attending to the learning process as well as the content. Assessment of this nature requires the collection of a range of forms of evidence over time to assess the understanding of a learner, as well as equipping the learner with skills to self-assess. This has profound implications for the design of learning. Boud (2010) suggests there are a number of principles of assessment which we can drawn on and adapted to our context and purpose:

  • assessment is used to engage learners in learning that is productive.
  • feedback is used to actively improve learners’ learning
  • learners and educators become responsible partners in learning and assessment
  • learners are inducted into the assessment practices and cultures of performance for work
  • assessment for learning is placed at the centre of curriculum design.
  • assessment provides inclusive and trustworthy representation of learner achievement, providing reliable evidence of performance
  • when assessment is a focus for those involved in curriculum assessment of student achievements is judged against consistent national and international standards that are subject to continuing dialogue, review and justification within professional communities

A new emphasis and direction towards workplace and other forms of blended learning and assessment (classroom, workplace and/or e-environments) in countries such as Singapore (where I am currently working on some assessment research in its Continuing Education and Training sector) signals a move away from a heavy reliance on classroom assessment, and a reimagining of the possibilities of teaching and learning practice. Outcomes through devolving assessment responsibility to learners and ‘outsourcing’ further aspects to employers and industry have the potential to increase learner employability and improve the learner experience – through carrying over into work the skills required to self-monitor developing skill sets and domain knowledge, as well as self-direct integration within new workplace contexts and communities of practice. RMIT is well on the way to exploring these possibilities within Work Integrated Learning (WIL) and similar programs, but may benefit from uniting current thinking within an overall philosophy of sustainable assessment (and learning).

References

Boud, D. (2010). Assessment 2010. Australian Learning and teaching Council, UTS.

Darling-Hammond, L. (Ed). (2014). Next generation assessment. Moving beyond the bubble test to support 21st century learning. Jossey-Bass.

Earle, L. (2013) Assessment as learning. Sage Publications

Earle, L. (2006). Rethinking Classroom Assessment with Purpose in Mind. Assessment for learning, assessment as learning, assessment of learning. Manitoba Education, Citizenship and Youth.

School of Art: Feedback notes for Students

This week Associate Professor Peter Ellis, Deputy Head of School, Learning and Teaching, School of Art writes about his school’s guidelines on how to explain feedback to students in the creative disciplines.

School of Art: Feedback notes for Students
Work Integrated Learning (WIL) Group Tutorials
Individual Tutorial Guidelines

Year 1 Student Chloe Caday in feedback session with Dr. Robin KingstonThis week’s Teaching TomTom post seeks to provide staff in the college of Design and Social Context (DSC) with some guidelines on how to explain feedback to students in the creative disciplines. The notes have been designed for students within the School of Art, but may be of interest for other schools too.

Attached to this post are Notes on feedback for students designed to inform students on, what feedback is, the types of feedback, how it is given and by whom.

The main idea behind this document is to provide new introductory students and staff with some useful notes on the importance of feedback and how it can be adapted for individual tutorials, Work Integrated Learning Group Tutorials, and Formative and Summative assessment.

The key points being that feedback is a continuous activity, not just at assessment, that it is the way students learn and that it is designed to:

  • Inform them on how well they are meeting the criteria for assessment
  • Inform them on how well they are meeting the learning objectives of courses or projects within courses.

Feedback is designed to:

  • Be supportive, clear, and honest
  • Assist in moving forward with their work in a confident, positive and manageable way
  • Be delivered in a way that clearly indicates what they should do to improve their work and how to move forward to the next level of their learning

Feedback should focus on the successful things your students are doing well, as well as things that need more attention, in order to improve and make their work stronger. Feedback is inclusive, individual and supportive. It is important that all feedback is given in a collegial, positive and supportive learning environment, where there is respect for individual opinion, gender and cultural diversity.

It includes strategies for conducting tutorials including the use of WIL feedback forms that are designed for students to record and reflect upon feedback provided to them by peers and lecturers during WIL group tutorials. The WIL forms that the student present for assessment clearly enables staff to ascertain if the student has understood the feedback that was offered to them.

The WIL form allows students to upload an image of the art work discussed, six keywords that exemplify the work, a description of the artist’s intentions for the work, a section to record the peer and lecturer feedback, and a section on how they will progress with the work after reflection on the feedback. WIL feedback forms also have a section for students to record suggestions from peers and staff about artists they should research, both historic and contemporary, bibliographic ideas, writers, films, critical theory, websites, magazines, YouTube etc. that may be useful for the their progress.

In an environment where the Course Experience Survey (CES) is an important tool for measuring student responses to the feedback we provide, it is crucial that both students and staff are aware of the importance of explaining and understanding what feedback is, that it is continuous in studios every day and is provided in a positive and supportive way.

The feedback we provide must be informed and supportive to encourage, inspire and provide strategies for continual improvement.

Please find some time to look at the attached Notes on feedback for students and provide advice.

I acknowledge Sally Mannall’s assistance in the preparation of the attached notes for students.

 


Share your thoughts, comments or start a discussion on ‘Explaining Feedback to Students’ by leaving a reply in the comments section!

Find us on:

Facebook: www.facebook.com/teachingtomtom
Twitter: @teachingtomtom
WordPress: teachingtomtom.com

Capstone Feedback

This week, Ruth Moeller, Senior Advisor, Learning and Teaching, Design and Social Context College,  shares with us her capstone feedback model.


Picture of Capstones

Capstones Photo credit Ruth Moeller

RMIT is very interested in the employment outcomes of its students, and a lot of research is currently underway to explore factors that effect employability. One of the factors that I have been surprised by is the fact that employers are telling us that graduates have difficulty articulating what they have learnt and how it could be translated into the work environment. (RMIT Graduate Employment Study Draft Final Report, insightrix, 2015)

I find it surprising because isn’t it obvious? You set an assessment task to address a brief, have students work in teams to produce an outcome. Aren’t these activities rich with transferable learnings and experiences; working with diverse others, understanding and meeting client needs, creating a product using the knowledge and skills of your discipline, meeting deadlines, the list goes on. Apparently I get it, but the students don’t, or at least can’t make the connections. So what is needed are ways to help students make the connections between what they do in our learning environments and how that can be communicated to potential employers.

There are a range of different strategies that can be incorporated into your curriculum to help address/support this, but what I would like to offer here is a simple, double edged strategy that I will be trialling at the end of this semester. It involves incorporating an end of course evaluation exercise I commonly do with my students, with reflection and articulation of student learning specifically in relation to workplace and employment contexts.

The feedback model I have used with my classes in the past is based on the premise that, for course feedback to be valuable, it needs to be clear, practical and implementable (whether you choose to action or not). A way of achieving this is to encourage students to reflect on their own experience of the course, but also clarify and moderate it with their peers. Using this model to encourage reflection reduces the likelihood of unsubstantiated and unhelpful comments such as: “it was OK or Things could be explained better”.

What I am planning to do in future classes, is to link this feedback exercise with an the opportunity to analyse and discuss the skills and knowledge they have developed or enhanced, and how what they have learnt can be linked to current or potential employment. (I will link a detailed “How to” to this post but as a start will give you an overview.)

In the last class of semester, I am planning to run an activity where the goals are to:

  • Get feedback from students on their key learnings and their perspectives of the course, its content and delivery, and suggestions on how it could be improved
  • Help students to identify and articulate the knowledge and skills they have developed in this course
  • Link students’ development and learning to their future employment.

My plan is that this activity will be done in two stages. In the first part I will encourage students to reflect and answer the focus questions on their own. Working on their own is an effective way for students to reflect initially on their own experience. In the second part, they form groups to discuss their responses and produce a ‘group’ response to the questions. Working as a group provides an opportunity for the individual responses to be clarified, moderated and validated.

The focus questions that I will provide are:

  1. What I have learnt (formally/informally)? Or had reinforced?
  2. What skills have I developed or improved?
  3. How can this knowledge and skills be used for my future career (does everyone want a career) job, profession, employment?

Now, think about the content and delivery of the course:

  1. What worked well? What should we do again next time?
  2. What suggestions do you have to improveme the course the next time it is run? What changes should we make?
  3. Anything else you would like to add about the course?

Once the students have discussed their responses in small groups, I will open up the discussion so we can explore their learnings and how these can be applicable to life (and work) beyond this course.

I have used this strategy to collect feedback before and it has been highly successful, as it provides tangible and validated feedback. It will be interesting to see how adding the second employability aspect to the activity will go, will it give the students the opportunity to reflect and make connections about the learning and its transferability and in doing so model the communication that employers are looking for?

I have attached the “How to” instructions to run this activity. If you try it too with your classes I would be very interested in your experience/feedback.

Click here to download the “how to” Capstone feedback handout


Share your thoughts and questions on capstone feedback and Ruth’s model in the comments section!

Find us on:

Facebook: www.facebook.com/teachingtomtom
Twitter: @teachingtomtom
and WordPress: www.theteachingtomtom.wordpress.com/

We are listening. Strategies to increase survey responses rates on teaching.

Picture credit iStock.com

Picture credit iStock.com

Posted by: Meredith Seaman, Senior Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

The problem…

We talk quite a lot about student feedback here on the blog, but the specific aspect I want to consider here is appropriate, fair and reliable ways to administer institutional surveys on good teaching to students. In particular I want to examine what could be done to increase survey responses so the results are more meaningful. The challenge in getting students to participate is more significant since surveys moved online, despite the obvious benefits of ‘anywhere/anytime’ for students to complete the survey instruments, and associated efficiencies for the Institution.

With low response rates it can be just a waste of everyone’s time as we need a certain sample size to viable and useful.

The research….

Investigations from the period of transition when many surveys moved online in the United States emphasise the following:

  • … “an important factor in response rates is students’ belief that rating results are used for important decisions about courses and faculty.” (Ballantyne, 1999)
  • Institutions should “encourage instructors to show a personal interest in students completing the forms (e.g., instructors could mention the forms in class, let students know that they pay attention to student responses, or send personal emails to students reminding them to complete the forms)” (Johnson, 2002)

While I’d definitely think twice about sending a personal email to your students, research is consistent that it is important for teachers to set up the context for students to complete the survey. For response rates to increase strategies are required at a range of levels. There is however, consensus that teachers have a role.

Given that the challenge is bigger than an institution sending out disembodied emails or offering iPads. How might we make it a meaningful process, a better experience for our students, and get more reliable data?

Strategies…

Pulling together different ideas from Schools across our College, to do it well, we need to show we are listening. The in-class process might go something like this:

Sowing the seed

Before survey time sow the seed early on about the importance of their ‘feedback’. You could highlight earlier in the course that such surveys inform a range of activities and decisions from university management down to the classroom. You could highlight a couple of specific changes that you have made in response to past survey responses and Student Consultative Committee discussions and so on.

Spending time with your students unpacking the notion of ‘feedback’ more generally is another idea. You might want to emphasise the different types of feedback that students get in your course, such as: from peers, on learning activities, on assessments tasks etc. You could also build on work you are doing with students on their skills to provide constructive feedback, such as giving peer review and feedback on learning activities and assessment tasks etc.

Summarise the Course

At survey time you could summarise the course to date.

e.g. In week one we…  in week 3 we found such and such a concept difficult…, in week 6 we … and finally, reiterate where you’re heading in the remainder of the semester. Helen’s recent post on ‘Going with the flow’ provides a model for this kind of activity. You might want to highlight how have you listened to your students (current and previous) and adjusted your learning and teaching plan.

Set aside time in class: (or in an online space)

Delivery in class time might be tricky, but in terms of getting response rates up it is well worth the time investment. It will show generosity if you allow class time, and that in itself emphasises that you take the process seriously, and are listening. It has been reported that 84% of Australians now have a smart phone, and 71% of those also have a tablet (Horizon Report 2014), and surveys can be completed on these devices online. If these stats seem inflated for your cohort, you could allow time for students to go to the library or build into an existing lab class.

Leave the Room:

It’s important that after all that, you leave the room. It highlights that the process is fair, provides thinking time, and creates a space for their comments to come to the fore.

________________________

At RMIT City Campus the Course Experience Survey (CES) survey is open to students from the 20th of September.

More on the CES at RMIT

Ballantyne, C. (1999). Improving university teaching: Responding to feedback from students. In Zepke, N., Knight, M., L&ach, L. and Viskovic, A. (Eds), Adult Learning Cultures: Challenges and Choices in times of change,  WP Press, Wellington, pp.55-165.

Johnson, T. (2002). ‘Online Student Ratings: Will Students Respond?’ Paper presented at the annual conference of the American Educational Research Association, New Orleans, 2002.

NMC Technology Outlook for Australian Tertiary Education – A Horizon Project Regional Report 2014

Share your thoughts on strategies to increase survey responses rates in the comments section!

_________
Find us on:

Sailing through Peer Review: Five lessons learnt at the coalface

Dr Ehsan Gharaie, Lecturer, School of Property, Construction and Project Management (PCPM)
&
Dallas Wingrove, Senior Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

Having a peer reviewer review your teaching is for many staff an unfamiliar risk taking experience that can be anxiety provoking. Ehsan Gharaie, a lecturer in the school of PCPM at RMIT University, recently underwent the process of peer review. As Ehsan embarked upon this journey he approached me as Senior Advisor L & T to support him through the process which included observing Ehsan’s teaching and providing feedback in response to the Peer Review criteria. What unfolded was highly useful professional learning for us both. In this post we share our experience of peer review and the lessons learnt.

Similar to many Australian and international universities, RMIT has now implemented a process of peer review of teaching. At RMIT, peer review is now mandated for teaching staff who seek an individual teaching award, and in 2015 is also to be introduced for staff seeking academic promotion.

In tertiary education, beyond teaching practice such as team teaching, and Peer Partnerships programs, there are limited opportunities for staff to share their practice with a peer, and receive feedback. The often ‘siloed’ nature of teaching presents many challenges for educators and opening your class room up to someone else for the purpose of peer review can be extremely daunting.

So what does this process mean for teachers? And how can they best prepare to have a positive experience of peer review?

Here are five lessons learnt through our experience of the peer review process:

  1. Understand and engage in the process

Before getting involved in the process it is vital that you understand the peer review process and its purpose. Attend your university’s workshops and information sessions. Familiarise yourself with your university’s guidelines and importantly engage with the teaching dimensions/criteria against which you will be reviewed. Remember, these dimensions/criteria align with recognised principles of good teaching practice. Reflect on how these criteria relate to your own practice and list and discuss with a peer examples which provide evidence of how you contribute to and demonstrate these dimensions in your practice. Contact staff implementing the Peer Review process, ask them questions and share any concerns you may have. At RMIT the process of Peer Review is implemented through the university’s Learning and Teaching Unit Stills of Ehsan Teachingwhich runs induction/information workshops, and provides advice for participants.

  1. Seek support and advice

There are many processes in academia that are competitive, but remember, this is NOT one of them. Your teaching practice will be reviewed against established dimensions/criteria. You are not competing with your peers so if you feel confident enough, share your experiences along the way, and seek and provide support to your peers. Do not hold back. Talk to people who can support you. Your colleagues, peers, program manager, and your university’s Learning and Teaching Advisors/Academic Developers can help you through the process. You may need them to simply listen to you to your concerns and anxieties. Having a colleague to talk to can really help ease your anxiety; this is not a journey that you have to go through alone.

  1. Engage with your peer reviewers

Whilst the formal peer review takes place in your class, there is also important activity which occurs prior to and following the peer review. Similar to other universities, at RMIT it is mandatory to meet with your peer reviewers at least once prior to the review. Remember, any meetings and discussions with your peer reviewers help to build the context for your review. Peer reviewers are experienced educators and learning and teaching experts and your dialogue with them will help to ease your concerns and/or fears. In doing so, demonstrate your knowledge and command of the discipline field and discuss your teaching approach. Initiate further contact with your peer reviewers as needed including if you have questions or require further clarification and advice. Importantly, provide the context for your teaching prior to the review. Identify: the aim of your session, how your class relates to the course and the wider program, the expectations of your students, the class dynamic, the nature of your particular cohort, your teaching and learning goals for the particular session, and provide any other information that you believe would assist your reviewers to understand your teaching and the class to be reviewed.

  1. Seek feedback on your teaching prior to your peer review:

Have the confidence to ask one of your peers or your Learning and Teaching Advisor to observe your teaching practice and provide confidential feedback. Provide the peer review dimensions/criteria and seek feedback about your teaching. It will be very helpful to see your teaching through someone else’s eyes. You also get used to having someone other than your students sitting in your class. In this way, you can dip your toe in the water, and ease yourself more gently into the process of observation, review and feedback.

Access other programs which support peer feedback. Participate in a Peer Partnerships program for example where you partner with another teacher to observe each other’s practice and provide feedback to support continuous improvement. At RMIT you can take up the opportunity to participate in RMIT Peer Partnerships. RMIT Peer Partnerships is a voluntary, confidential program involving peer observation of teaching. RMIT Peer Partnerships facilitates highly useful relevant professional development learning and can assist you to become more comfortable and at ease with sharing your teaching practice, and support critical reflection on practice through giving and receiving feedback.

  1. Believe in yourself: don’t panic, this is just another day in the class.

The prospect of peer review can seem very daunting for many staff. Most if not all educators experience some level of discomfort when having their teaching reviewed or evaluated, these are normal human reactions. However, if you have done your preparation, you understand the process, and you seek feedback beforehand, you will be well placed to feel more comfortable about the process. You just need to resist the nerves in the first five minutes of the class and as soon as you relax you will forget the reviewers are even sitting there. Remember, reviewers are experienced teachers and they can tell if you pretend. Just be yourself. After all this is just another day in the class.

The next steps…

As you contemplate whether you are ready to embark upon the Peer Review journey remember to access all supports and enlist the support of a peer AND remind yourself that the process is one which endeavours to strengthen the teaching culture of your university and to also value and recognise your good teaching practice.

Share your thoughts and questions in the comments section!
_________
Find us on:

Going with their flow…

Posted by: Helen McLean, Senior Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

(cc) Rafters on the North Johnstone River, in Queensland, Australia. Flickr User, Didrik Johnck.

Rafters on the North Johnstone River, Queensland. (cc) Flickr User: Didrik Johnck.

Next week is the mid-semester break.

By now your students will be in the flow of their study with you. They are likely to have completed one piece of assessment and received some feedback to guide them on to the next stage of their learning or specified what they could have done differently in that assignment. Ideally this feedback will have created an opportunity for them to talk with you and/or their peers about what they have learned and encourage them to confidently tackle the next tasks in the course.

It is possibly tempting to let the flow of the assessment tasks keep control of how your carefully prepared study schedule continues for the remainder of the semester. You have set them on their way but do you really know they are on course, their course?

Let’s assume they have been stimulated and excited by the outcome of their assessment task and there are some intriguing points they’d like to explore more deeply or revisit. Perhaps they have come across some new material that they’d like to incorporate in the learning schedule.

Why not provide a touchpoint and check in to see where your students are at and establish what they might need or want next from the course?

This could happen when you all return from the semester break, refreshed and ready for the final stretch. Have a conversation with your students about their learning in the course so far. Find out where they are at and how they are progressing so that you are all on the ‘same page’ for the remainder of the time you have together.

How could you do this?

Set aside some time and ask them to:

  • outline what they have learnt so far in the course
  • reveal what they would like to know more about
  • identify what they are not clear about or on what they need further clarification

They could work in groups, individually, face to face or online to uncover and share what they know or want to know. Be creative, use technology, role play, or a game to find out what they know or need.

Once you have their feedback, take some time to reflect and diagnose. You may need to slow down or even prepare to change direction.

As a facilitator of their learning, challenge yourself to provide them with the opportunities to fill the gaps they have revealed. Be stunned and amazed by the leads they provide for further exploration. They are adult learners who have individual motivations and personal preferences of their learning requirements. Getting them to acknowledge those needs and identify their own areas of interest will help them to develop as self-regulated learners.

They will also feel valued when you address their feedback. Regrouping like this can bring together loose ends or point them in independent directions for their learning before commencing the final stage with you this semester.

Be partners and learn together.

Enjoy the rest of the semester!

Share your thoughts on coming back from the break in the comments section!
_________
Find us on:

RMIT’s 2014 Learning and Teaching Expo

Posted by: Meaghan Botterill,  Senior Coordinator, Educational Technology Integration, e-Learning Strategy and Innovation Group, RMIT University.

Click on the image to register for the event.

Click on the image to register for the event.

RMIT’s annual Learning and Teaching Expo is on 2-3 September, 2014. This is a great opportunity to catch up on what is happening both nationally and locally in learning and teaching. Last year the Expo was a great success, so come and join colleagues from across the university to discuss and explore innovative practices that enhance student learning outcomes.

This year’s theme, Designing Teaching, Creating Learning, explores how good teaching design and pedagogical practices create and enhance student learning opportunities and outcomes. There will be an extensive range of speakers, presentations and workshops from across RMIT and the program features the following guests:

  • Professor James Arvanitakis from the University of Western Sydney who was the 2012 Prime Minister’s Teacher of the Year award winner. James’ passion and enthusiasm for teaching is apparent to any of you who have ever seen him present before. He is continually looking for ways to make connections with his students and to make learning relevant, accessible and exciting.
  • Professor Ruth Wallace is the Director of the Northern Institute, at Charles Darwin University. Her particular interests are related to undertaking engaged research that improves outcomes for stakeholders in regional and remote Australia. Ruth has extensive experience in innovative delivery of compulsory, post-school and VE programs in regional and remote areas across Northern Australia.
  • Associate Professor Nicolette Lee is from Victoria University and she is a 2013 OLT National Senior Teaching Fellow. Her project, Capstone curriculum across disciplines, synthesises theory, practice and policy to provide practical tools for curriculum design. It builds on previous and current work in the sector to identify capstone innovations and models-in-use, how standards might be demonstrated through a range of approaches, and providing publicly available and comprehensive practical tools for staff.
  • Associate Professor John Munro is from the University of Melbourne. John’s research, teaching and publications are in the fields of literacy and mathematics learning, and learning difficulties, learning internationally, gifted learning, professional learning and school improvement. His focus on neurology and the brain form the basis of designing explicit teaching strategies to create learning in diverse student cohorts.
This is a great opportunity to learn more about learning and teaching and what we as educators can do to design teaching to create learning and thus enhance student learning outcomes. Registration is essential. The full program and registration form are available here.

Learning and Teaching Expo 

Date: Tuesday 2 and Wednesday 3 September
Time: 9am to 4.30pm
Venue: Storey Hall, Building 16, City campus
Cost: Free

Registration: Essential
Registrations close Wednesday, 27 August 2014.
Register here now.

_________
Find us on: