Towards sustainable assessment: some thoughts

Dr Peter Rushbrook, Deputy Head, Learning and Teaching at RMIT University’s School of Education shares some thoughts and references about sustainable assessment. 

image depicting a session in a workplace training centre at a major Singapore supermarket chain.

This is an image depicting a session in a workplace training centre at a major Singapore supermarket chain. The trainees are undergoing assessment of their checkout skills, including managing cash and customer service. This is a simulation but the skills learned are put into practice quickly in the demonstration store next door.

Thinking of assessment for, as and of learning (Earle, 2006), not just of learning, is no longer new. What is new are approaches to assessment that build on this basic premise to support Twenty-first century learning. Paying attention to assessment is critically important for adult learning as David Boud (2010) points out,

Assessment is a central feature of teaching and the curriculum. It powerfully frames how students learn and what students achieve.

Assessment is a process, not just an end result.

Assessment for longer term learning focuses on higher order thinking and skills, such as exercising judgment in context. It develops independent, confident practitioners ready to transition to the next phase, ready to work independently and with others to make informed judgments (Boud 2010). We need to focus not just learning how to learn, but on the subset of learning – how to assess (Boud 2000). We need to move away from multiple choice questions and away from assessors looking for answers from an answer sheet. Such practices do not reflect the increasing cognitive (Darling-Hammond, 2014) and complex psycho-social demands required of our workforce. They fail to contribute to the application of learning to contexts outside a classroom or test environment. Essentially, we need to move from narrow assessment to assessment for deeper sustained learning.

Assessment needs to do double duty: both for credentialing and learning purposes. This includes: formative assessment for learning and summative for certification; focusing on the task and developing lifelong learners; and attending to the learning process as well as the content. Assessment of this nature requires the collection of a range of forms of evidence over time to assess the understanding of a learner, as well as equipping the learner with skills to self-assess. This has profound implications for the design of learning. Boud (2010) suggests there are a number of principles of assessment which we can drawn on and adapted to our context and purpose:

  • assessment is used to engage learners in learning that is productive.
  • feedback is used to actively improve learners’ learning
  • learners and educators become responsible partners in learning and assessment
  • learners are inducted into the assessment practices and cultures of performance for work
  • assessment for learning is placed at the centre of curriculum design.
  • assessment provides inclusive and trustworthy representation of learner achievement, providing reliable evidence of performance
  • when assessment is a focus for those involved in curriculum assessment of student achievements is judged against consistent national and international standards that are subject to continuing dialogue, review and justification within professional communities

A new emphasis and direction towards workplace and other forms of blended learning and assessment (classroom, workplace and/or e-environments) in countries such as Singapore (where I am currently working on some assessment research in its Continuing Education and Training sector) signals a move away from a heavy reliance on classroom assessment, and a reimagining of the possibilities of teaching and learning practice. Outcomes through devolving assessment responsibility to learners and ‘outsourcing’ further aspects to employers and industry have the potential to increase learner employability and improve the learner experience – through carrying over into work the skills required to self-monitor developing skill sets and domain knowledge, as well as self-direct integration within new workplace contexts and communities of practice. RMIT is well on the way to exploring these possibilities within Work Integrated Learning (WIL) and similar programs, but may benefit from uniting current thinking within an overall philosophy of sustainable assessment (and learning).

References

Boud, D. (2010). Assessment 2010. Australian Learning and teaching Council, UTS.

Darling-Hammond, L. (Ed). (2014). Next generation assessment. Moving beyond the bubble test to support 21st century learning. Jossey-Bass.

Earle, L. (2013) Assessment as learning. Sage Publications

Earle, L. (2006). Rethinking Classroom Assessment with Purpose in Mind. Assessment for learning, assessment as learning, assessment of learning. Manitoba Education, Citizenship and Youth.

School of Art: Feedback notes for Students

This week Associate Professor Peter Ellis, Deputy Head of School, Learning and Teaching, School of Art writes about his school’s guidelines on how to explain feedback to students in the creative disciplines.

School of Art: Feedback notes for Students
Work Integrated Learning (WIL) Group Tutorials
Individual Tutorial Guidelines

Year 1 Student Chloe Caday in feedback session with Dr. Robin KingstonThis week’s Teaching TomTom post seeks to provide staff in the college of Design and Social Context (DSC) with some guidelines on how to explain feedback to students in the creative disciplines. The notes have been designed for students within the School of Art, but may be of interest for other schools too.

Attached to this post are Notes on feedback for students designed to inform students on, what feedback is, the types of feedback, how it is given and by whom.

The main idea behind this document is to provide new introductory students and staff with some useful notes on the importance of feedback and how it can be adapted for individual tutorials, Work Integrated Learning Group Tutorials, and Formative and Summative assessment.

The key points being that feedback is a continuous activity, not just at assessment, that it is the way students learn and that it is designed to:

  • Inform them on how well they are meeting the criteria for assessment
  • Inform them on how well they are meeting the learning objectives of courses or projects within courses.

Feedback is designed to:

  • Be supportive, clear, and honest
  • Assist in moving forward with their work in a confident, positive and manageable way
  • Be delivered in a way that clearly indicates what they should do to improve their work and how to move forward to the next level of their learning

Feedback should focus on the successful things your students are doing well, as well as things that need more attention, in order to improve and make their work stronger. Feedback is inclusive, individual and supportive. It is important that all feedback is given in a collegial, positive and supportive learning environment, where there is respect for individual opinion, gender and cultural diversity.

It includes strategies for conducting tutorials including the use of WIL feedback forms that are designed for students to record and reflect upon feedback provided to them by peers and lecturers during WIL group tutorials. The WIL forms that the student present for assessment clearly enables staff to ascertain if the student has understood the feedback that was offered to them.

The WIL form allows students to upload an image of the art work discussed, six keywords that exemplify the work, a description of the artist’s intentions for the work, a section to record the peer and lecturer feedback, and a section on how they will progress with the work after reflection on the feedback. WIL feedback forms also have a section for students to record suggestions from peers and staff about artists they should research, both historic and contemporary, bibliographic ideas, writers, films, critical theory, websites, magazines, YouTube etc. that may be useful for the their progress.

In an environment where the Course Experience Survey (CES) is an important tool for measuring student responses to the feedback we provide, it is crucial that both students and staff are aware of the importance of explaining and understanding what feedback is, that it is continuous in studios every day and is provided in a positive and supportive way.

The feedback we provide must be informed and supportive to encourage, inspire and provide strategies for continual improvement.

Please find some time to look at the attached Notes on feedback for students and provide advice.

I acknowledge Sally Mannall’s assistance in the preparation of the attached notes for students.

 


Share your thoughts, comments or start a discussion on ‘Explaining Feedback to Students’ by leaving a reply in the comments section!

Find us on:

Facebook: www.facebook.com/teachingtomtom
Twitter: @teachingtomtom
WordPress: teachingtomtom.com

Capstone Feedback

This week, Ruth Moeller, Senior Advisor, Learning and Teaching, Design and Social Context College,  shares with us her capstone feedback model.


Picture of Capstones

Capstones Photo credit Ruth Moeller

RMIT is very interested in the employment outcomes of its students, and a lot of research is currently underway to explore factors that effect employability. One of the factors that I have been surprised by is the fact that employers are telling us that graduates have difficulty articulating what they have learnt and how it could be translated into the work environment. (RMIT Graduate Employment Study Draft Final Report, insightrix, 2015)

I find it surprising because isn’t it obvious? You set an assessment task to address a brief, have students work in teams to produce an outcome. Aren’t these activities rich with transferable learnings and experiences; working with diverse others, understanding and meeting client needs, creating a product using the knowledge and skills of your discipline, meeting deadlines, the list goes on. Apparently I get it, but the students don’t, or at least can’t make the connections. So what is needed are ways to help students make the connections between what they do in our learning environments and how that can be communicated to potential employers.

There are a range of different strategies that can be incorporated into your curriculum to help address/support this, but what I would like to offer here is a simple, double edged strategy that I will be trialling at the end of this semester. It involves incorporating an end of course evaluation exercise I commonly do with my students, with reflection and articulation of student learning specifically in relation to workplace and employment contexts.

The feedback model I have used with my classes in the past is based on the premise that, for course feedback to be valuable, it needs to be clear, practical and implementable (whether you choose to action or not). A way of achieving this is to encourage students to reflect on their own experience of the course, but also clarify and moderate it with their peers. Using this model to encourage reflection reduces the likelihood of unsubstantiated and unhelpful comments such as: “it was OK or Things could be explained better”.

What I am planning to do in future classes, is to link this feedback exercise with an the opportunity to analyse and discuss the skills and knowledge they have developed or enhanced, and how what they have learnt can be linked to current or potential employment. (I will link a detailed “How to” to this post but as a start will give you an overview.)

In the last class of semester, I am planning to run an activity where the goals are to:

  • Get feedback from students on their key learnings and their perspectives of the course, its content and delivery, and suggestions on how it could be improved
  • Help students to identify and articulate the knowledge and skills they have developed in this course
  • Link students’ development and learning to their future employment.

My plan is that this activity will be done in two stages. In the first part I will encourage students to reflect and answer the focus questions on their own. Working on their own is an effective way for students to reflect initially on their own experience. In the second part, they form groups to discuss their responses and produce a ‘group’ response to the questions. Working as a group provides an opportunity for the individual responses to be clarified, moderated and validated.

The focus questions that I will provide are:

  1. What I have learnt (formally/informally)? Or had reinforced?
  2. What skills have I developed or improved?
  3. How can this knowledge and skills be used for my future career (does everyone want a career) job, profession, employment?

Now, think about the content and delivery of the course:

  1. What worked well? What should we do again next time?
  2. What suggestions do you have to improveme the course the next time it is run? What changes should we make?
  3. Anything else you would like to add about the course?

Once the students have discussed their responses in small groups, I will open up the discussion so we can explore their learnings and how these can be applicable to life (and work) beyond this course.

I have used this strategy to collect feedback before and it has been highly successful, as it provides tangible and validated feedback. It will be interesting to see how adding the second employability aspect to the activity will go, will it give the students the opportunity to reflect and make connections about the learning and its transferability and in doing so model the communication that employers are looking for?

I have attached the “How to” instructions to run this activity. If you try it too with your classes I would be very interested in your experience/feedback.

Click here to download the “how to” Capstone feedback handout


Share your thoughts and questions on capstone feedback and Ruth’s model in the comments section!

Find us on:

Facebook: www.facebook.com/teachingtomtom
Twitter: @teachingtomtom
and WordPress: www.theteachingtomtom.wordpress.com/

Sailing through Peer Review: Five lessons learnt at the coalface

Dr Ehsan Gharaie, Lecturer, School of Property, Construction and Project Management (PCPM)
&
Dallas Wingrove, Senior Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

Having a peer reviewer review your teaching is for many staff an unfamiliar risk taking experience that can be anxiety provoking. Ehsan Gharaie, a lecturer in the school of PCPM at RMIT University, recently underwent the process of peer review. As Ehsan embarked upon this journey he approached me as Senior Advisor L & T to support him through the process which included observing Ehsan’s teaching and providing feedback in response to the Peer Review criteria. What unfolded was highly useful professional learning for us both. In this post we share our experience of peer review and the lessons learnt.

Similar to many Australian and international universities, RMIT has now implemented a process of peer review of teaching. At RMIT, peer review is now mandated for teaching staff who seek an individual teaching award, and in 2015 is also to be introduced for staff seeking academic promotion.

In tertiary education, beyond teaching practice such as team teaching, and Peer Partnerships programs, there are limited opportunities for staff to share their practice with a peer, and receive feedback. The often ‘siloed’ nature of teaching presents many challenges for educators and opening your class room up to someone else for the purpose of peer review can be extremely daunting.

So what does this process mean for teachers? And how can they best prepare to have a positive experience of peer review?

Here are five lessons learnt through our experience of the peer review process:

  1. Understand and engage in the process

Before getting involved in the process it is vital that you understand the peer review process and its purpose. Attend your university’s workshops and information sessions. Familiarise yourself with your university’s guidelines and importantly engage with the teaching dimensions/criteria against which you will be reviewed. Remember, these dimensions/criteria align with recognised principles of good teaching practice. Reflect on how these criteria relate to your own practice and list and discuss with a peer examples which provide evidence of how you contribute to and demonstrate these dimensions in your practice. Contact staff implementing the Peer Review process, ask them questions and share any concerns you may have. At RMIT the process of Peer Review is implemented through the university’s Learning and Teaching Unit Stills of Ehsan Teachingwhich runs induction/information workshops, and provides advice for participants.

  1. Seek support and advice

There are many processes in academia that are competitive, but remember, this is NOT one of them. Your teaching practice will be reviewed against established dimensions/criteria. You are not competing with your peers so if you feel confident enough, share your experiences along the way, and seek and provide support to your peers. Do not hold back. Talk to people who can support you. Your colleagues, peers, program manager, and your university’s Learning and Teaching Advisors/Academic Developers can help you through the process. You may need them to simply listen to you to your concerns and anxieties. Having a colleague to talk to can really help ease your anxiety; this is not a journey that you have to go through alone.

  1. Engage with your peer reviewers

Whilst the formal peer review takes place in your class, there is also important activity which occurs prior to and following the peer review. Similar to other universities, at RMIT it is mandatory to meet with your peer reviewers at least once prior to the review. Remember, any meetings and discussions with your peer reviewers help to build the context for your review. Peer reviewers are experienced educators and learning and teaching experts and your dialogue with them will help to ease your concerns and/or fears. In doing so, demonstrate your knowledge and command of the discipline field and discuss your teaching approach. Initiate further contact with your peer reviewers as needed including if you have questions or require further clarification and advice. Importantly, provide the context for your teaching prior to the review. Identify: the aim of your session, how your class relates to the course and the wider program, the expectations of your students, the class dynamic, the nature of your particular cohort, your teaching and learning goals for the particular session, and provide any other information that you believe would assist your reviewers to understand your teaching and the class to be reviewed.

  1. Seek feedback on your teaching prior to your peer review:

Have the confidence to ask one of your peers or your Learning and Teaching Advisor to observe your teaching practice and provide confidential feedback. Provide the peer review dimensions/criteria and seek feedback about your teaching. It will be very helpful to see your teaching through someone else’s eyes. You also get used to having someone other than your students sitting in your class. In this way, you can dip your toe in the water, and ease yourself more gently into the process of observation, review and feedback.

Access other programs which support peer feedback. Participate in a Peer Partnerships program for example where you partner with another teacher to observe each other’s practice and provide feedback to support continuous improvement. At RMIT you can take up the opportunity to participate in RMIT Peer Partnerships. RMIT Peer Partnerships is a voluntary, confidential program involving peer observation of teaching. RMIT Peer Partnerships facilitates highly useful relevant professional development learning and can assist you to become more comfortable and at ease with sharing your teaching practice, and support critical reflection on practice through giving and receiving feedback.

  1. Believe in yourself: don’t panic, this is just another day in the class.

The prospect of peer review can seem very daunting for many staff. Most if not all educators experience some level of discomfort when having their teaching reviewed or evaluated, these are normal human reactions. However, if you have done your preparation, you understand the process, and you seek feedback beforehand, you will be well placed to feel more comfortable about the process. You just need to resist the nerves in the first five minutes of the class and as soon as you relax you will forget the reviewers are even sitting there. Remember, reviewers are experienced teachers and they can tell if you pretend. Just be yourself. After all this is just another day in the class.

The next steps…

As you contemplate whether you are ready to embark upon the Peer Review journey remember to access all supports and enlist the support of a peer AND remind yourself that the process is one which endeavours to strengthen the teaching culture of your university and to also value and recognise your good teaching practice.

Share your thoughts and questions in the comments section!
_________
Find us on:

RMIT’s 2014 Learning and Teaching Expo

Posted by: Meaghan Botterill,  Senior Coordinator, Educational Technology Integration, e-Learning Strategy and Innovation Group, RMIT University.

Click on the image to register for the event.

Click on the image to register for the event.

RMIT’s annual Learning and Teaching Expo is on 2-3 September, 2014. This is a great opportunity to catch up on what is happening both nationally and locally in learning and teaching. Last year the Expo was a great success, so come and join colleagues from across the university to discuss and explore innovative practices that enhance student learning outcomes.

This year’s theme, Designing Teaching, Creating Learning, explores how good teaching design and pedagogical practices create and enhance student learning opportunities and outcomes. There will be an extensive range of speakers, presentations and workshops from across RMIT and the program features the following guests:

  • Professor James Arvanitakis from the University of Western Sydney who was the 2012 Prime Minister’s Teacher of the Year award winner. James’ passion and enthusiasm for teaching is apparent to any of you who have ever seen him present before. He is continually looking for ways to make connections with his students and to make learning relevant, accessible and exciting.
  • Professor Ruth Wallace is the Director of the Northern Institute, at Charles Darwin University. Her particular interests are related to undertaking engaged research that improves outcomes for stakeholders in regional and remote Australia. Ruth has extensive experience in innovative delivery of compulsory, post-school and VE programs in regional and remote areas across Northern Australia.
  • Associate Professor Nicolette Lee is from Victoria University and she is a 2013 OLT National Senior Teaching Fellow. Her project, Capstone curriculum across disciplines, synthesises theory, practice and policy to provide practical tools for curriculum design. It builds on previous and current work in the sector to identify capstone innovations and models-in-use, how standards might be demonstrated through a range of approaches, and providing publicly available and comprehensive practical tools for staff.
  • Associate Professor John Munro is from the University of Melbourne. John’s research, teaching and publications are in the fields of literacy and mathematics learning, and learning difficulties, learning internationally, gifted learning, professional learning and school improvement. His focus on neurology and the brain form the basis of designing explicit teaching strategies to create learning in diverse student cohorts.
This is a great opportunity to learn more about learning and teaching and what we as educators can do to design teaching to create learning and thus enhance student learning outcomes. Registration is essential. The full program and registration form are available here.

Learning and Teaching Expo 

Date: Tuesday 2 and Wednesday 3 September
Time: 9am to 4.30pm
Venue: Storey Hall, Building 16, City campus
Cost: Free

Registration: Essential
Registrations close Wednesday, 27 August 2014.
Register here now.

_________
Find us on:

Let them know!

Posted by: John Benwell, Principal Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

Now the second semester is up and running, classroom timetables sorted, and new-comers settled, students are already wondering how they are doing.

By week 4 in the semester, hopefully you have given the students an early and simple assessment to make sure their break is behind them (for those starting midyear it could be their first assessment in their program) and they are on the road to successfully completing your course. With both large classes and small, after the assessment, it is time to give them feedback and publish their results.

file000531133054

Nothing grabs a student’s attention like “Marks released for Assignment One!”

Just as we expect to be able to check our bank account or phone or utility bills, students are keen to get their results of submissions quickly and online is the first place they will go looking. Do you use online submissions for your assignments? Do you publish your results online? Both are of great benefit to students and can save you a lot of time.

Getting students to submit online is a great way to keep up with a student’s performance – making sure they have submitted on time and are participating. A quick look at the submissions will show students who have failed to submit, and may be already falling behind.

Once marked, grades will automatically be presented to the students or can be released on your schedule.

If it isn’t possible to submit the assessment online (a project, an artwork for instance) you can still create a column and publish marks or grades online for the students’ benefit. In these cases you might consider having students photograph their work and submit a reflective piece. This can be a good way of keeping things fair if you are dealing with work that is installed in a gallery space for instance, or for when a students might be presenting throughout the week.

Blackboard has a valuable facility built in called Grade Centre. It resembles a spreadsheet, and is automatically populated with the students’ names and student numbers. As a bonus, their last login is listed in the third column; a quick way to see if they are participating online. Columns will be added to Grade Centre when you create Blackboard assignments (quizzes, Turnitin assignments) or you can easily add columns for assignments that cannot be uploaded. Furthermore, Grade Centre has the ability to add calculated columns where you can add mathematical formulae to calculate marks with weightings.

Grade Centre has the facility to download its data in Microsoft Excel format to your desktop/laptop where you can take it away and fill out the results. When you are finished marking, and back on the internet, you can upload the spreadsheet, and your results will be published to you class. Once in Grade Centre, the marks are stored and backed up by IT. Columns can also be hidden from students, or published on a particular date.

Grade Centre also supports groups and multiple markers, so part-time and sessional tutors can group their students and mark their assignments from anywhere on the internet at any time.

So if you’re excited about these possibilities to keep an eye on your students and keep them informed whilst saving yourself time, here are some links to the technologies above in our university’s context:

DevelopMe sessions are also available on: Grade Centre and Blackboard Assessment.

And in the DSC, don’t forget your Learning and Teaching advisors and Educational Developers who can also help you.

Give your students what they want and let them know their results as soon as you can!

Share your thoughts in the comments section!
_________
Find us on:

The unbundling of higher education: Breaking down the whole.

(cc) Flickr user: Mike Linksvayer

(cc) Flickr user: Mike Linksvayer

Posted by: Erika Beljaars-Harris, Educational Developer, College of Design and Social Context, RMIT University.

There’s another movement unfolding in the background of fee deregulation that we need to be aware of: the unbundling of higher education. As Professor Jim Barber (former Vice Chancellor, University of New England) explains, “The concept of ‘unbundled’ education refers to the emergent practice of allowing students to pay for those services, and only those services, that they actually require.” Similar thoughts are being raised in the UK, as this Times Higher Education article points to a report that recommends government funding follow the student and not the institution.

Think of it as the difference between a set menu (preselected courses served at a fixed time and price) compared to free choice from the menu and dishes from any other restaurant. In the higher education arena, this might mean choosing a course from a university, but not paying for the facilities and services offered. The facilities students may choose to not use include the cafeteria and other academic and support services. What’s being called a ‘pick-and-mix’ approach means that students pay for certain facilities on a fee-for-service basis. Which leads to student choosing which parts they want to use and therefore pay for. Choice has always been seen as something students value in a program of study (look at electives, streams, majors, study-abroad and cross-institutional studies for instance) but this movement might see multi-institution degrees become a path that more students select.

According to Harvard Business School Professor Clayton Christensen, the unbundling of higher education is a form of ‘disruptive innovation‘. Christensen explains it as “a process by which a product or service takes root initially in simple applications at the bottom of the market and then relentlessly moves upmarket, eventually displacing established competitors.”  An example of this already exists in the form of consumers (students) having the ability to receive credentials via RPL (Recognition of Prior Learning) through previous work experience or a MOOC. For universities, the unbundling of higher education is a form of disruptive innovation. It is enabling the consumer (student) with the ability to choose subjects and courses from a university that can be delivered on campus, online or both, without the added fees for services and facilities that they may not need nor use. As a consumer (student), this unbundling provides the ability to secure services the individual does want, and not pay for what they don’t want. This hopefully translates to cheaper, but just as, or more effective degrees and experiences selected from a wider pool of providers.

The movement towards unbundling has started. Georgia Institute of Technology is admitting students into a low-fee postgraduate degree. Students are taking courses from the University Without Walls, a university fully supported by the University of Massachusetts, that enables students to design their program of study.

The goals of unbundling of higher education are to increase the quality of lectures, enable more individualised instruction, offer an increase in choice to students and most importantly, provide it all at a lower cost. What it might mean for academics and universities is to take stock of what they deliver well online, in blended environments and on-campus: student expectations aren’t going to do anything except rise.

To be honest, I Iike this movement, I like the goals that this movement professes to be aligning towards. I will be watching those universities to see who gets it right (and wrong) in this evolution of higher education.

Share your thoughts on unbundling in the comments…

_________
Find us on: