Monthly Archives: February 2013

Peer and self-assessment

Students in discussion at RMIT.

Copyright © RMIT University. Photographer: Margund Sallowsky.

Posted by: Dr Alex Radloff, Higher Education Consultant.

Peer and self-assessment use has been growing in Higher Education at undergraduate and postgraduate levels, as has the use of technology to support these forms of assessment. Peer-assessment refers to the process of assessing the quality of the products or outcomes of the learning of peers. Self-assessment refers to the process of assessing the quality of the products or outcomes of learning, or the act of learning, by the learner. Both kinds of assessment can be used as part of formative and summative assessment, either as ‘stand alone’, or in conjunction with teacher generated assessment. Academic staff who have used peer and self assessment report that:

  • The skills are a requirement of many professions/jobs and are valued by potential employers.
  • Using peer and/or self assessment skills demystifies the assessment process and makes it more accessible to learners.
  • Students are provided with more frequent and detailed/richer feedback from more sources.
  • Students develop analytical and critical skills needed to identify and use criteria and standards relevant to work in their discipline/profession. Learners engage more deeply /thoughtfully in learning and assessment tasks.
  • The skills help students to increase their metacognitive awareness and control of learning including planning, monitoring and evaluating learning.

Academic staff who have used peer and self assessment also report:

  • Resistance by students. Resistance is generally based on a lack of trust in the validity (does the assessment assess the stated or intended outcome?) and fairness of peer or self-assessment; a view that assessment is the responsibility of teachers and should only be undertaken by teachers, not learners; concerns about the capacity of learners to assess accurately; and concerns about possible accreditation requirements.
  • Quality issues related to the reliability of the assessment (how consistent assessment outcomes are over time) when based on the judgments of learners and their ability to interpret and apply criteria and standards appropriately.
  • Over-reliance on peer and/or self-assessment, especially for summative assessment purposes, to the exclusion of other forms of assessment can be an issue.
  • Learners need training/support to understand and use peer and self-assessment effectively.
  • The implementation of peer and/or self-assessment especially for large groups of learners, may require access to and the management of, specific technology and software.

Careful design of peer and self-assessment can address the problems and issues identified above. The steps in designing peer and self-assessment follow the typical assessment cycle, namely Purpose of assessment; Selection of assessment tasks; Setting criteria; Administering assessment; Scoring the assessment; Grading the assessment; and Feedback. To increase the effectiveness and efficiency of peer and self-assessment:

  • Make clear the rationale, purpose and expectations of the planned approach with students and colleagues. Address common concerns concerning validity, reliability, fairness and trust.
  • Involve students in developing the assessment criteria. Consider involving students in the design of the assessment activities as well, if appropriate.
  • Make clear how peer and/or self-assessment will be used in conjunction with teacher-assessment, if it is to contribute to a final grade.
  • Provide systematic training and practice for students in using the assessment criteria and standards with examples of products representing different levels of performance.
  • Give students clear, written instructions and guidelines on the assessment process including timelines, deadlines, and any consequences (rewards and/or penalties) associated with the process.
  • If using technology for assessment, ensure that it works and that students know how to access and use it and what to do if they need help.
  • Check how the assessment process is working and intervene if needed to provide feedback and coaching.
  • Keep records of assessment outcomes and monitor how peer and self-assessment compares to teacher assessment over time.
  • Review the outcomes in terms of learning, performance and satisfaction from both the students’ and the teacher’s perspectives, and revise design and implementation if needed.
  • Collaborate with colleagues to discuss different strategies and to share experiences.

Want to know more?

Past posts on peer assessment and peer learning can be accessed by clicking here or on the tags to the right. The following is a short survey of the academic literature relevant to the topic:

Bell, A., Mladenovic, R., & Price, M. (2012). Students’ perceptions of the usefulness of marking guides, grade descriptors and annotated exemplars. Assessment & Evaluation in Higher Education, DOI:10.1080/02602938.2012.714738

Examines students’ views of the usefulness of exemplars, grade descriptors and marking criteria for reflection and learning, or for understanding the assessment task.

O’Donovan, B., Price, M., & Rust, C. (2008). Developing student understanding of assessment standards: A nested hierarchy of approaches. Teaching in Higher Education, 13, 205–217.

Discusses the importance of involving students in the assessment process and describes different ways to help students understand assessment requirements.

Higher Education Academy. Self and peer assessment. Post Graduate Certificate in Professional Development.http://www.glyndwr.ac.uk/cpd/pgcpd/assessment_and_giving_feedback/self_and_peer_assessment/assessment_issues.html

What are your views on peer assessment and peer learning? Share them in the comments section below!

Designing collaborative learning is worth the effort

Posted by: Thembi Mason, Senior Advisor, Learning and Teaching, College of Design and Social Context, RMIT University.

What is an NGLS?

Panorama shot of new learning space at RMIT.

Copyright © RMIT University. Photographer: Margund Sallowsky.

Technology enabled active learning spaces or ‘new generation learning spaces’ are becoming more widespread in universities.  They are designed to support a more student-centred approach to teaching and learning, leading to active learning and higher engagement in students. While new learning spaces vary in their exact characteristics, they typically are:

  • carefully planned to facilitate interactions between students
  • designed to allow for flexible use and arrangement of furniture
  • constructed to enable the teacher to both teach and facilitate the class from anywhere in the room and
  • technology-enabled to encourage active learning both within and outside of the classroom.

How do they help learners and teachers?

So how does a teacher use the space so that students can build their own learning rather than relying on the ‘telling’ expertise of the teacher?

Associate Professor Nick Blismas from the School of Property, Construction and Project Management kindly agreed to let me sit in on one of his classes to see how he teaches in these new learning spaces. It was a great class. There were six students on each table. Nick had to monitor the numbers as students would try to pile into big groups but eventually they were evenly distributed around the room.

Students were learning about procurement methods – that is what procurement method would be chosen before a building was built to ensure that the time, cost and quality were optimally met. The right procurement method was critical to the eventual outcome of the project and Nick had designed a procurement game to build discipline knowledge so students could make more informed choices and decisions.

Hang on, what’s a ‘procurement game’?

students gathered around tables

Copyright © RMIT University. Photographer: Thembi Mason.

He split the student groups evenly into ‘developers’ and ‘clients’. Then he gave the clients information about the type of construction they were to build and asked them to embellish on the basic information and criteria for development he had given. For example, one group was to build a supermarket but they needed to factor in underground car parking. Meanwhile, the developers reviewed the different types of procurement systems. Students could use the wireless network to tap into the internet if they needed to find additional information.

Fifteen minutes later, clients met with developers and outlined their building project. Developers asked questions to clarify some of the criteria. Then the developers had to select the appropriate building approach for their client. The client could then respond as to whether they thought it was the right approach for them and why. All the groups presented their work at the end of the class and all the students voted on whether they thought it was the right approach for each building project.

It was a fantastic class to observe; the students really got into the role-playing. There was heated debate between clients and developers over ideas and you could see that the students were really learning discipline knowledge from each other. As students discussed the issues, Nick facilitated the class by walking around to the different groups and offering advice if he was asked or pushing the thinking when he thought a group was stuck. He was also formatively assessing them as he went.

‘Playing the whole game’

It was a fabulous way to facilitate collaboration and it was made possible because of the learning space – this type of activity would not have worked in a lecture theatre.  Designing activities and class work as Nick did does take some time, however, the students were engaged, they loved it, they learnt from each other and I am sure they will remember that class and what they learnt in that class when they are working in the field.

Nick had designed a lesson that David Perkins would say ‘played the whole game’ of their future professional lives in a practice session. Procurement was seen in context and seen as relevant by the students. They had to problem solve and deal with arguments about their selections. Clients had to listen to developers as they argued for the method they wanted to use. The process allowed students to practice their negotiating skills and improve their interactive skills for dealing with future clients. It showed them that often there is no certainty about any particular procurement method but taught them what each might offer them depending on the context for the development. For the students this was a taster of their future careers as project managers.

Have you got a story about using new learning spaces? Please comment if you do and let us know what worked for you and your class.

The librarian, the academic, the student…

Posted by: June Frost, Liaison Librarian, University Library, Bundoora West Campus, RMIT University.

I recently came across this description of my library colleagues in an Oxford University student’s column:

When it is not attacking other creatures librarianus spectacalus spends most of its time catching the unsuspecting rectangular creature bookius bookius in the strange firm linear webs with which they line their mountain caves.  But librarianus does not eat bookius bookius; instead they catch it for the strange effect it produces when they stare at its underbelly.

rowsofbooksAcademic librarians are a strange breed of people. Of course, there are variations within the breed: some like to catalogue and organise information, some like to present information by using the latest gadgets, some like to verbally impart information (sometimes endlessly it seems), some (just a few) like to keep information secret, but almost all of us love to share information in one way or another. It’s not difficult to distinguish the key word here – INFORMATION! We love the opportunity to share information with our colleagues, teachers, lecturers and researchers and with our families (much to their dismay). Here is a typical exchange:

- How was your day, Mum?
– Oh, really good today. Did you see my post on Facebook about RMIT Library’s new LibrarySearch function? It’s just like doing a Google search except you find all the Library resources on one topic including e-books, e-articles and streaming video?
– Oh, great Mum – hope you didn’t make it public!

But most of all, we love sharing information with students who at certain times of the year, are our biggest fans.  It might be that StudentonmobilephoneatRMITthey’ve never used a certain database or that they hadn’t realised they can access a resource from their iPad or maybe they’ve hit a tricky concept in one of their courses. They might not know it, but they’re usually looking to fill a gap in their information skill-set.

Information skills

Information /ɪnfəˈmeɪʃ(ə)n/ (International Phonetic Alphabet)

  • facts provided or learned about something or someone: a vital piece of information.

(from Oxford Dictionaries Online.)

Traditionally, information skills sessions takes place in the first few weeks of a semester, when students are reeling from information overload.  It doesn’t matter whether they are starting a TAFE certificate or beginning research for a PhD, there’s a lot to take in.  The Library homepage contains a plethora of – you guessed it—INFORMATION, and students need to learn the skills to navigate (we also love the word ‘navigate’) their way around and through this information, until ‘Bingo!’ they find what they are looking for.  To get to the ‘Bingo!’ moment, it’s quite understandable that most students will need some help in: firstly, recognising they need information; secondly, selecting the right method to find the information; thirdly, finding ways to locate, disseminate and store the information; fourthly, synthesising and evaluating the information; and lastly, deciding on the methods to present the information.

Time Pressures

A study by Kent state University Researchers which collected data from higher education institutions across 17 states in the USA found that the biggest barrier to including information skills (or IL: information literacy) in teacher education programs was time:

It makes sense that barriers remained consistent whether educators were trying to integrate IL skills or IL standards. Since most courses consist of well-established content, it is not surprising that lack of time and lack of their own expertise in IL were identified as major hurdles. These responses highlight another possible benefit of collaboration; a librarian, looking at a course from a different perspective, may be able to suggest ways that existing content and assignments can be slightly modified to include important IL skills and knowledge. Kovalik, et al (2010) p.62

I suspect the same might be said of RMIT or indeed across the nation.  It does take time for course coordinators and lecturers to firstly talk to or LIAISE (another of our favourite words) with librarians, schedule in an information skills session and then find a time to incorporate it into the busy course schedule, but it will be worth the effort.

The Solution

StudentreachingforbookatRMITOne solution to this may be to rethink the timing of library skills sessions in the academic year.  How about scheduling a session mid-semester when the student’s first major assessment piece is being delivered?  If the librarian has access to the assignment question and themes, the skills session can then be tailored to the question and the students can walk away with not only skills but some actual resources to set them on their way. For flexibility, we could also ensure this information is available online for students who prefer to learn from their bedroom floor…speaking as someone with teenage children.

For myself, I will happily impart information to students at any time of the year, but by trying to strategically place these research sessions at the right point of the calendar, it may produce better outcomes.

Maybe it could further cement our libraries as AWESOME, SICK and KOOL (yes that’s how they spell it now!) places on campus.  Or to use the IPA/ˈɔːs(ə)m//sɪk/ and /kuːl/.

Share your thoughts about campus libraries and information skills in the comments below!

Reference:

Kovalik, C. L., Jensen, M. L., Schloman, B., & Tipton, M. (2010). Information literacy, collaboration, and teacher education. Communications in Information Literacy, 4(2), 145-169. Retrieved from http://search.proquest.com/docview/865649314?accountid=13552. 1 February 2013

Grounding graduate attributes

Posted by: Margaret Blackburn, Senior Advisor, Strategic Learning and Teaching Initiatives, Office of the Dean, Learning and Teaching, Academic Portfolio, RMIT University.

A commercial aeroplane with landing gear deployed.

(cc) MorgueFile.com

As publisher and presidential advisor C. D. Jackson said, ‘Great ideas need landing gear as well as wings’. The notion of graduate attributes is a pretty abstract one, so the challenge is to provide some landing gear and make them mean something to students and teachers. Otherwise, the risk is that they’ll stay in orbit somewhere in the educational stratosphere and make little impact on the ground.

At RMIT, we have a set of six graduate attributes. They are skills or qualities that we expect all of our graduates, whatever their specific program, will have had the chance to acquire to a suitable level. We want them to be work ready, to be active and lifelong learners and so on. But what does this mean for the curriculum and for how it is delivered, learned and assessed?

Before we go further, let’s agree that there’s confusion over the range of terms used to describe these qualities. Aren’t we talking about learning outcomes here?  Well, yes, but we are now being more precise with outcomes at different levels. At the course level, we have outcomes that are assessed to and gained by successful completion of a course. But outcomes also exist at the program level (what we used to call ‘program capabilities’) and above and informing these are RMIT’s graduate attributes.

View above clouds from an aeroplane window.

(cc) MorgueFile.com

As a tertiary environment at RMIT, we also use the TAFE sector’s labels of competencies, elements, performance criteria and employability skills.

In the new program guide matrix developed to meet AQF requirements, program capabilities are now referred to as program learning outcomes. The matrix enables teachers to align program learning outcomes with the overarching graduate attributes and in the other direction, down to the courses of the program. Work is currently underway to complete this.

TEQSA now requires institutions to demonstrate that all of their graduate attributes, including English language competency, have been attained. Why this new focus on outcomes? The tertiary context has changed radically in recent years. There can be a haphazard quality to the bundle of skills, knowledge and attributes students graduate with. Degrees from different institutions vary in terms of minimum standards as well as content. The kaleidoscope of higher participation rates, funding pressures, more varied models and modes of learning, have all led to a new focus on outcomes and how to measure them.[1] Research also points to a need to re-examine the role of graduate attributes when it comes to standards.

To breathe life into graduate attributes in curricula, in teaching and learning and in assessment, the key is context. To use the word ‘generic’ to describe graduate attributes suggests that we can ‘unplug’ graduate attributes entirely from a specific discipline or teaching area.  But Anna Jones’ research[2] indicates that they are not ‘generic’ or ‘super skills’ that exist beyond disciplinary contexts or professional and vocational fields. Graduate attributes don’t exist in a vacuum. Rather, they start with the content and culture of particular disciplines or fields. A key question is: what is the essence of this discipline? Jones found that the ways that graduate attributes are taught and learned depend on the conceptual frameworks, language, assessment practices, technologies and even physical settings that form the heart of particular disciplines, professions and vocations. As de-contextualised statements, they don’t work. This makes sense to the classroom teacher or lecturer who ideally is also a practitioner or has a deep knowledge of their industry counterparts.

Although the terminology is the same for different disciplines and fields, for example ‘work ready’, graduate attributes have different meanings and are weighted differently in every field or profession. For example, a ‘work ready’ engineering graduate will prioritise in-depth technical competence in at least one engineering discipline. In media and communications, work readiness is primarily about creative practice and critical reflection. ‘Innovative’ in fashion and design disciplines may spotlight imaginative and creative endeavours whereas in business disciplines, innovation is about designing new rules and processes that improve traditional business models. In a business degree, ‘cultural and social awareness’ should include an understanding of how enterprise and business activities affect groups and individuals. In social sciences, however, to understand social justice issues in professional settings may be an essential aspect of this attribute.

Where do you start to bring graduate attributes down to earth?  One approach is, as per the King of Hearts, ‘begin at the beginning’. Consider what gives your discipline or field its identity, its own distinguishing stamp. What are the essential skills or qualities that you want your graduates to have? For example, in economics, one central skill is to be able to apply economic tools to problems. Another might be to analyse macroeconomic data to make predictions. A third might be to be able to develop further economics expertise by being an independent and active learner. But does your economics degree, as you’ve sketched it out, prepare students for the cultural and social implications of their profession? If you use the graduate attributes as a screen while listing those essential skills and qualities, you may find that elements you thought peripheral have a place in ensuring that every RMIT program has the best chance to develop a well-rounded graduate.

As you make your list then, use the graduate attributes as a reference point to help you frame the skills and knowledge of your discipline or profession. Then you are ready to express the skills and qualities it contains in a set of five or six broad program learning outcomes that take into account AQF levels.

How do those program learning outcomes shape the curriculum at course level? They are the starting point for drafting detailed course level learning outcomes that spell out in detail what students will learn and be assessed on in each course. Course learning outcomes must fit with both program learning outcomes and the overarching graduate attributes. Mapping all three across the entire program is helpful. The point is that graduate attributes don’t mean much on their own. They’ll gain their real meaning and impact from the detailed context provided by the learning outcomes at both program and course level.

(cc) MorgueFile.com

(cc) MorgueFile.com

Finally, and critically, structure your students’ learning activities to help them actively engage with the course learning outcomes. Ask yourself: how will this activity, exercise, problem, online discussion exercise, help my students get to grips with a specific learning outcome? And how will the elements of my assessment program enable students to show that they have met the learning outcomes at a particular level? By taking a holistic approach to all three elements, learning outcomes, learning activities and assessment, you’ll ensure that those elusive graduate attributes come back to earth.

Share your comments on graduate attributes in the comments below!

References:

[1] Royce Sadler, (2012) Assessing and assuring graduate attributes, keynote address to AAGLO Conference, July 19.

[2] Anna Jones, (2012) There is nothing generic about graduate attributes: unpacking the scope of context, Journal of Further Education, DOI:10.1080/0309877X.2011.645466

Follow

Get every new post delivered to your Inbox.

Join 1,365 other followers

%d bloggers like this: