Thursday, 11 October 2012

GCTLT: Assessing and Evaluating: What is your experience of moderation in your teaching context? – Discussion 3
for Ass 3

Before this year my experience has mainly been with post-
moderation. Firstly it has been an ‘internal’ process where my colleagues and I
will moderate each other’s work to look for consistency between our two sites
(Hamilton and Dunedin) and also to check our feedback comments in terms of
quality of statements. After this is completed the assessment committee also
provides feedback and then marks are released to students.

The other moderation I have had experience in is via an
external moderator who assesses the course based on set criteria involving,
assessments, student feedback, and a report that I write as course coordinator.
We have just debated this process as some staff felt it isn’t effective as the
majority of time the moderator ‘concurs’ with the course coordinator’s reports.
However, I have found the experience valuable in receiving confirmation for
issues that I’ve raised, (as it has meant that there was ‘backup’ to address
these in the following year), and also to sustain and grow areas that are
working well.

This year I developed a Rubric marking template which meant
I had to complete a pre-moderation process. This involved using previous year’s
scripts to test the new Rubric. A colleague and I blind marked 10% of old
scripts to test for consistency, and to check we complied with being fair and
transparent.The Rubric also went to the
assessment committee for approval which generated some general comments
including; clarifying some terms used in the template, and noting that if a
student doesn’t quite ‘fit’ in one box- you can ‘highlight’ comments across a
number of boxes to give more specific feedback (this was helpful advice when it
came to actual marking in Semester 2). IT was only after this had been ‘tested’
that it was released to students.

To support this we also developed a teaching strategy that
had a number of components to support this assignment. These included a face to
face workshop about the assessment and Rubric marking sheet.Students also had access to online material
they could refer to in their own time which included web-links, a word document
that outlined general comments from me in regard to the assignment, and an open
Moodle forum where they could post any questions or comments. Both myself and
my colleague from Hamilton posted comments on this forum. We also posted
reflective questions students could explore themselves in their own online
forums (such as via the students’ facebook page).

• Why is moderation
important?

Internal moderation is important for consistency within a
course (especially when you have multiple markers), to ensure consistency
between courses and to give assurance to students that their mark is ‘reliable’
as per OP’s assessment policy (2009). ACU National (2008) also identified these
aims in the moderation process: comparability of assessment; quality of
assessment; and adherence to academic
standards

Internal moderation can also help trouble shoot any issues
(ie if one marker is unsure then they can utilise other staff to give an
opinion and/or the assessment committee).

External moderation is also important to give consistency
across the programme. I have found this helpful for checking if assessments are at the correct year level across the
programme as a whole (ie we are not asking first years to complete an assessment task that should be at 3rd year level and vice versa).

• What are the
differences between pre and post-assessment moderation?

Pre assessment is a process we engage in where we check information/processes
in regard to new assessment strategies before
they are given to students. This ensures students only receive assessment strategies
which have been ‘trialled and tested’.

Post assessment happens during and after an assessment
occurs which I engage in regularly with my colleagues as detailed above.

I thought the comment in ACU National about the ‘spirit’ of
moderation was interesting…

·“Moderation
of assessment by self-review, School processes and, where applicable, internal moderators
are integral to quality assessment practices each time a unit is offered.

·Moderation
of assessment by an independent moderator/s at regular intervals provides
opportunities for independent feedback.

·Moderation
will be most effective when conducted in a spirit of professional learning and
quality improvement.” (ACU National, 2008; 433)

GCTLT- Assessing and
Evaluating:- What feedback models or processes do you prefer? Discussion 2 for
Ass 3.

Personally, I don’t have a preference regarding feedback – I
have had benefit from both receiving verbal and written feedback and also
delivering these both to students.

I do however have a preference for timing of the feedback-
if I’m working on a large project, I like on-going feedback that is structured-
i.e. less of the ‘you’re doing a great job’ to being more specific as to what’s
working well, what needs more attention and where to next. I also enjoy the opportunity
to have multiple attempts before ‘submitting’ or the ability to have another
go. I have experienced this through my learning te reo Maori through Te Wananga
o Aotearoa and I offer this option to my students when they have a go at their mihi at noho
marae.

• Why is feedback
important?

Feedback is important to identify areas of learning that
students need to improve on or to enhance areas they may be quite strong in. I
have gained a lot from reading Race, Brown and Smith (2005), Fostaty Young (2005)
and Spiller (2009). In particular their ideas that the assessment process
should contain no surprises, that feedback should contain statements of evidence
that are not confined to ‘what’s missing’ and to think about offering Feed –forward
comments so students can clearly know
what is expected of them in their next assessments have been useful to my own
area of teaching.

I also think encouraging feedback from students is also
important- and from other colleagues.

• What are the
challenges or issues with assessment feedback?

Time is always a factor. The 1 hour per student marking time
is a policy that seems more to do with budget constraints rather than evidence
based practice concerning delivering quality feedback.

I also think it’s a challenge to make sure students have the
necessary tools to complete the assessments- but that we don’t scaffold too
much that they don’t end up learning to climb for themselves. The course I
teach in is a bachelor’s course, so this expectation of competency would be
adjusted say if I was working with foundation learners.

I have also put more effort in helping students understand
expectations prior to the assessment but also in using the feedback given.
After they receive feedback from one assessment- we use this information to plan
for their second assessment.

• How do you balance
how much feedback to give?

I’ve developed Rubric templates for their written
assessments. The first page is the “Feedback” grades and comments, and we go
through this extensively in class before students submit their work. I then
have another box where I write a short paragraph with ‘feed-forward’ comments- such
as what they should focus on in their next assignment. I also use this
feed-forward box when I mark their next assignment so I can assess if they have
addressed any concerns raised (e.g. such as referencing), or followed through
with extending their learning. Students are also invited for 1:1 feedback that
is via face to face or email. At noho marae- students receive verbal feedback and there are
a number of staff that help assess this in a collaborative way.

10 August 2012 - What
does cultural diversity in relation to assessments mean to you? Discussion 1
for Ass 3.

Kia ora koutou

As most of the class has already mentioned, cultural diversity
is a complex yet rich term that as facilitators we can use to our better our professional
practice. I like Monaghan,J. & Just, P.(2000) description of cultural anthropology
whereby they explain that the discipline studies ‘social relations’ and ‘cultural
logic’ of people. I think these phrases can also be applied to tertiary facilitators/teachers
where we are not only working out the relationships between groups of students/colleagues/wider
community etc, but also the cultural logic that drive the individuals within
these groups. When I think of the cohort of students I teach- it’s working out
how I can encourage the best learning without falling into the trap of trying
to be everything to everyone which means I have to think of ways of getting to know their
learner needs and aspirations .

I also agree with many of the postings that culture can
encompass ethnicity but I also understand that the two terms are significantly
different (as are ethnicity and nationality). Some students may be quite
comfortable identifying with their ethnicity/ies- whereas others are more
comfortable identifying through their occupations/ nationality etc.

In relation to cultural diversity and assessments- this
means I accept diversity- I don’t ignore or generalise diversity. What has helped
me in meeting this challenge is being flexible in offering multiple pathways
for how students can access information and/or support to engage with the assessments
early on in the paper I teach and having clear expectations and explaining/demonstrating
these expectations to students in a
variety of ways.

• What factors need
to be considered?

“Cultures make sense
of the world through values, norms and behaviour they acquire and assessments
need to resonate with their beliefs. By not recognising factors of diversity in
the assessment we as educators run the risk of failing to meet educational
needs and therefore increase social disadvantages in communities”
(Rameka,2007).

I was interested in Rameka’s (2007) comment above as on one
hand I fully understand what Rameka is saying as you only have to look at the
latest Maori post-secondary qualifications to recognise the systemic failings
in our educational system. I think to meet this challenge however, is that
tertiary facilitators need to practice and gain feedback about ’ how’ this happens because without this I
find that teachers either continue to do what they do (which is ignore
diversity and do nothing) as we don’t want to make a ‘mistake’. Liz Ditzel’s
posting (24/0/12) is great in giving practical tips.

I challenge my students when they ‘see’ me about what
assumptions are made about how I might teach? Because I’m Maori, Tongan and
Pakeha- am I going to communicate a certain way, am I going to deliver course
material and assess in a different way say compared with my Pakeha colleagues? Do
I want to teach in groups or at an individual level, do they think I prefer
face to face or is online going to be OK? Do they have higher or lower expectations
from me?

They all tell me that’s
rubbish and they will ‘judge’ my skills and abilities as they get to know me.
But we all make judgements about each other – that part is normal- it’s how
these effect our professional practice that we all need to work out. There is now extensive research that shows how
teachers’ beliefs and values do impact on students’ learning (both
positively and negatively) – (see Russell Bishop’s work in this area in the compulsory
sector: http://www.waikato.ac.nz/php/research.php?mode=show&author=rbishop).
So for me a factor I have to consider
then; is to what extent do my judgements impact on my teaching?

When I look at this statement in OP’s assessment policy: “Assessments improve student learning. In the
design of the assessment the values need to be acceptable including fairness,
relevance and respect to all the students”, then I can’t help but challenge
how the terms fair, relevant and respect are defined? And how these may differ
between me/students/ other colleagues?

I can explain why to
my students we don’t eat in class, sit on tables, and give koha when we have
guest speakers in, but that’s not only because of my own ethnicity, but because
I understand (and tell students) the institution I work for has a MOU with our
local runaka and in this agreement it states that as a whole institution we will
abide by our runaka’s tikaka (cultural) practices as per the Treaty of Waitangi.
I find once students have context and logic presented to them for cultural
practices- then the communication (i.e. social relations) flows a lot better as
they can then participate in a meaningful way (sharing in cultural logic).

• How can
consideration of diversity benefit the learner?

It actually benefits us all by teaching patience and
tolerance- (but only through maintaining high standards). By acknowledging diversity
amongst learners means I don’t expect my students to be the ‘same’ so I have to
have different ways of communicating and presenting information so they can
access learning.

It also allows the culture of learning (ako) and
facilitation to be challenged in different ways. As part of an in-class
activity this year I submitted a short
piece of work to students that I wrote and I then asked them to award a grade
with a short rationale as to why. Apart from the ‘joke’ that they were now assessing
me, I encouraged them to use my marking criteria as a close guide and to help
them come to terms with ‘assessment speak’. They then had to present back as to
what mark they would give me and why. I then marked my own work in front of the
class, and explained why. I was at least a grade or two below the students’
grades and although they said I should be nicer to myself- it was a good
activity to clearly show where the expectation was. We then worked as groups to
change, edit and develop the work I had done to gain a higher quality of work. This
exercise took about half an hour but I think it was valuable as it offered an alternative
method to explaining about assessment, by instead demonstrating marking in real time to students and through the
discussion we had.

• What are the challenges?

One of the big challenges is breaking down the myth that Culture
diversity is about being ‘different’ from the ‘norm’. I find that once students
can articulate clearly about their own cultural context (which we do in one of
their assessments) this helps ‘tease’ the term culture out. Culture is a
concept that; applies to groups, is learned, includes beliefs, values, ideas , material
artifacts and behaviours and changes over times (although it ‘roots’ are in the
past), (Peter Morris, 2000).

If we accept Morris’ ideas about culture, then we are all
cultural begins! The other challenge of course is trying to be continually ‘savvy’
as to how you use your time with students- to get the most ‘benefit’ out of the
learning context….

Ka mihi Gina

References:

Monaghan,J. & Just, P.(2000). Social and cultural
anthropology: A very short introduction. Oxford: Oxford University Press.

About Me

Kia ora, I've just realised I have no photos of me, so here's one with my boy last year at his Birthday.
This is my first blog and I'm not entirely comfortable with this medium...
As this whakatauki expresses,
"Kaore te kumara e whaakii ana tana reka"
"The kumara (sweet potatoe) does not say how sweet he is".
So, to share a little about 'me'- I have a dual role here at Polytech (both within operations as well as academic) and work with the Kaitohutohu Team for Professor Khyla Russell. I also have interests outside of Polytechnic and run my own company project managing various research contracts (qualitative and quantitative) and for the past two years I've been festival coordinator for Puaka - Matariki Festival (Maori New Year) here in Dunedin. I sit on too many community groups, and I'm trying to retire from being 'busy' but as my whanau keep telling me- 'whateva'... Most of the time I have a sense of humour but I must admit it fails me around 5:30am.
Looking forward to becoming more familiar with 'blogging' - and to see how this could be used as a learning tool.
Na, Gina