In 2018 most students in most schools will move to an online environment for NAPLAN. This means that students will complete all test sections on a computer or tablet. Test data that is entirely digital can be turned around more rapidly so that results will be available for schools, systems and families much faster.

The implication is that the results can be put to use to assist students with their learning, and teachers with their planning. While this appears to address one of the persistent criticisms of NAPLAN – the lag between testing and results – other questions still need to be asked about NAPLAN. Continuing concerns include high stakes contexts and perverse effects (Lingard, Thompson & Sellar, 2016), the marketization of schooling (Ragusa & Bousfueld, 2017), the hijacking of curriculum (Polesel, Rice & Dulfer, 2014) and the questionable value of NAPLAN for deep learning (beyond test performance).

Almost ten years after its introduction, NAPLAN has been normalised in Australian schooling. Despite some tweaking around the edges, the original assessment architecture remains intact. However, the move to online delivery and automated marking represents a seismic shift that demands urgent attention.

Most student responses in NAPLAN are closed questions. In the new online format these include multiple choice, checkbox, drag and drop, reordering of lists, hot text, lines that can be drawn with a cursor and short answer text boxes. These types of answers are easily scored by optical recognition software, and have been since NAPLAN was introduced.

However the NAPLAN writing task, requiring students to produce an extended original essay in response to an unseen prompt, has always been marked by trained human markers. Markers apply a detailed 10 point rubric addressing: audience, text structure, ideas, persuasive devices, vocabulary, cohesion, paragraphing, sentence structure, punctuation and spelling. In years when narrative writing is allocated, the first four criteria differ however the remaining six remain the same. Scores are allocated for each criterion, using an analytic marking approach which assumes that writing can be effectively evaluated in terms of its separate components.

It is important to stress that online marking by trained and highly experienced teachers is already a feature of high stakes assessment in Australia. In NSW, for example, HSC exams are marked by teachers via an online secure portal according to HSC rubrics. The professional learning that teachers experience through their involvement in such processes is highly valued, with the capacity to enhance their teaching of HSC writing in their own schools.

Moving to online marking (called AES or Automated Essay Scoring by ACARA, also called machine-marking, computer marking or robo-marking) as NAPLAN proposes is completely different from online marking by teachers. While the rubric will remain the same, judgement of all these criteria will be determined by algorithms, pre-programmed into software developed by Pearson, the vendor who was granted the contract. Algorithms cannot “read” for sense, style, context or overall effectiveness in the ways that human experts can. All they can do is count, match patterns, and apply proxy measures to estimate writing complexity.

ACARA’s in-house research (ACARA NASOP Research Team, 2015) insists on the validity and reliability of the software. However, a recent external evaluation of ACARA’s Report is scathing. The evaluation (Perelman, 2017), commissioned by the NSW Teachers’ Federation from a prominent US expert, argues that ACARA’s research is poorly designed and executed. ACARA would not supply the data or software to Perelman for independent examination. However it is clear that AES cannot assess key aspects of writing including audience, ideas and logic. It is least effective for analytic marking (the NAPLAN approach). It may be biased against some linguistic groups. It can easily be distorted by rewarding “verbose high scoring gibberish” (Perelman, 2017, 6). The quality of data available to teachers is unlikely to improve and may lead to perverse effects as students learn to write for robots. The risk of ‘gaming’ the test is likely to be higher than ever, and ‘teaching to the test’ will take on a whole new dimension.

Human input has been used in ACARA’s testing of AES in order to train and calibrate the software and in the future will be limited to reviewing scripts that are ‘red-flagged’ by the software. In 2018 ACARA plans to use both human and auto-marking, and to eliminate humans almost entirely from the marking process by 2019. In effect, this means that evaluation of writing quality will be hidden in a ‘black box’ which is poorly understood and kept at a distance from educational stakeholders.

The major commercial beneficiary, Pearson, is the largest edu-business in the world. Educational assessment in the UK, US and now Australia is central to its core business. Details of the contract and increased profits that will flow from the Australian government to Pearson from the automated marking of writing are not publicly available. Pearson has already been involved in NAPLAN, as several states contracted Pearson to recruit and train NAPLAN markers. Pearson have been described as a “vector of privatisation” (Hogan, 2016, 96) in Australian education, an example of the blurring of social good and private profit, and the shifting of expertise from educators and researchers to corporations.

Writing is one of the most complex areas of learning in schools. NAPLAN results show that it is the most difficult domain for schools to improve. Despite the data that schools already have, writing results have flatlined through the NAPLAN decade. Negative effects and equity gaps have worsened in the secondary years. The pattern of “negative accelerating change” (Wyatt-Smith & Jackson, 2016, 233) in NAPLAN writing requires a sharper focus on writer standards and greater support for teacher professional learning. What will not be beneficial will be furthering narrowing the scope of what can be recognised as effective writing, artfully designed and shaped for real audiences and purposes in the real world.

NAPLAN writing criteria have been criticised as overly prescriptive, so that student narratives demonstrating creativity and originality (Caldwell & White, 2016) )are penalised, and English classrooms are awash with formulaic repetitions (Spina, 2016) of persuasive writing NAPLAN-style. Automated marking may generate data faster, but the quality and usefulness of the data cannot be assumed. Sustained teacher professional learning and capacity building in the teaching of writing – beyond NAPLAN – will be a better investment in the long term. Until then, the major beneficiaries of online marking may be the commercial interests invested in its delivery.

The main purpose of the publication was really to advocate for a new way of analyzing the data to show what they consider a better means of measuring student achievement, i.e. years of progress. It reports that high achieving and low achieving students are not improving their results. Despite this, their starting point in this report for considering a change is still based on certain assumptions about NAPLAN.

Now that NAPLAN testing has concluded for 2016, it is worth examining the assumptions about NAPLAN that pervade this document, as well as the general public and media discourse around NAPLAN. Included in these assumptions are that:

NAPLAN tests actually are a good indicator of overall achievement and success at school, in learning and seeing education as a positive. [But that is not conclusive.]

the NAPLAN test results are a good predictor of a student achieving or not achieving their potential. [Again not conclusive for all students across the range of abilities]

the same or similar test environment occurs across the years and the same or very similar items are included across the years in the tests have been administered.

data and results from the test should be used as a major input upon which to base educational decisions by policy makers.

I think it is timely to draw attention to the limitations of NAPLAN and to remind parents, and students, that it provides information about a child’s achievement based on that one day, at that time, completed under stressful test conditions.

NAPLAN does not take account of the development of students’ interests in learning, their passions, or engagement in learning. NAPLAN outcomes should be considered in the context of all the other measures teachers use to assess student achievement of learning outcomes, especially in the other key learning areas of Creative Arts, Health and Physical Education and so on. NAPLAN does not take into account the other value-added dispositions and community involvement provided by schools that are not measurable in a test. Interventions and pedagogical changes in classrooms at a school take time to demonstrate results, and again, many of these may not be measurable by the NAPLAN test.

Data from NAPLAN is still limited, no matter what approach to data analysis and reporting is undertaken – whether data are compared against benchmarks or measured by years of a student’s progress. It is not the results and reporting that is questionable, it is the basis upon which these data are used for system evaluation of schools, a school’s progress, and to drive policy.

Perhaps governments need to look at employment policies and other support mechanisms, not just school education, for students from low socio-economic backgrounds living in poverty. A multi-pronged approach is needed to improve the outcomes for students to break the cycle of inter-generational disadvantage.

Imagine you had promised your friend or partner that you would go with them to watch a movie each week for a full year.

Imagine then, having made this commitment, that your friend or partner did not consult with you about which movies to watch – they simply selected the movie each week, irrespective of your own preferences or tastes, and expected you to come along.

You might put up with it for awhile. You would likely develop resentment about the situation. Eventually, as an adult, you might confront your friend and explain your feelings and try to change things. If they didn’t change, in all likelihood, you would simply stop going to the movies with them.

For children and young people in schools, the school curriculum is like being forced to go to the movies, to see things they often don’t like or can’t see the point of, but where they do not have the adult prerogative, legally at least, of simply not going to school. Trapped in schools with an unresponsive curriculum, feelings amongst young people towards school can and frequently do include resentment, apathy and disengagement. Every teacher commonly experiences these feelings amongst their children, and not just amongst the children who are the lowest academic achievers.

We should not underestimate the power of an unexplained and unresponsive curriculum as a factor in child and youth disengagement from school. Nor should it be underestimated as an explanation for any perceived decline in international education standards among western nations where, in most facets of life, young people influence and exercise considerable choice in most other areas of their lives except in school.

In writing this piece I am assuming curriculum as a broad entity, ranging from the documents comprising the Australian curriculum and the range of state-based adaptations to it, through to the formal and informal learning experiences of children in classrooms and schools, structured and developed under the auspices of each school.

Curriculum is the key. A cynic might say that curriculum is what education systems DO to learners in schools. A greater cycnic might say that what is done to learners is also being done to teachers. If our curriculum is not carefully thought through and structured, it can act as a straitjacket on teachers and learners, undermining their capacity to explore and engage through education. If the curriculum is over burdened in content areas, over prescribed with mandated teaching points, over tested, over regulated, then it robs learners and teachers of the potential to engage in education with imagination, personal investment, and joy. Learning becomes a chore, for learners and teachers alike. And, often, they disengage as a result. They simply stop trying.

In my many years as a teacher and teacher educator, I have always believed that teaching is among the most creative of professions. There is nothing more satisfying for a teacher than to develop learning experiences that enable children to understand concepts, develop skills and values, develop confidence, and enjoy their learning. The act of conceiving of and creating these learning experiences, ones that you know will bring out the best in your learners, then seeing your creative, intellectual efforts work in the classroom, and seeing children grow and want to keep learning as a result, is the key reward for the teacher.

To achieve this, curriculum needs to be freed up, becoming a crucible for fostering creative imagination rather than a straitjacket encouraging disengagement. We need a flexible curriculum, far less prescriptive than we generally have now, which encourages teachers to engage with and be responsive to the personalities of their students, and which enables young people to become involved with and take responsibility for their learning.

How to do this? We have plenty of evidence that current curricula are generally overcrowded and too prescriptive, so a good first step would be to identify a set of genuinely necessary core competencies, skills, values and content, which are limited and restrained, and which are essential for the social and economic wellbeing of individuals (and through them, the nation). The remainder and bulk of the curriculum should take the form of flexible guidelines which teachers can respond to with imagination and creativity, thereby inspiring their children to become involved and to strive to excel. This is a strength of the current curriculum in Finland, which has been considered the global ‘gold standard’ over the last decade.

We used to have in Australia, in the 1970s and 80s, strong and successful state-based cultures around school-based curriculum development – ones which enabled schools and their teachers to craft engaging and relevant curriculum developed from a clear but limited systemic curriculum framework.

These cultures (like the culture currently emphasised in Finland) had strong expectations of teachers as highly responsible, creative and professional individuals, based on high levels of trust of teachers. Unfortunately, later neo-liberal political ideologies and governance (from both sides of state and federal politics) gradually eroded these cultures. Examining and re-valuing the strengths of these previous curriculum cultures in Australia might be a good place to begin in conceiving how a less centralised, less crowded and more responsive curriculum would work for learners and their teachers.

Secondly, we have plenty of examples of thinking about curriculum, learner motivation and pedagogical approaches which respect the role of learners in learning, and teach us how to be inclusive of the tastes, preferences, talents and humanity that learners bring to their learning and their schools. People who have provided conceptual and practical clarity in their related writings include John Ainley, James Beane, Garth Boomer, John Dewey, Jacquelynne Eccles, Michael Fullan, William Glasser, Susan Groundwater-Smith, Roger Holdsworth, Stephen Kemmis, Tony Knight, Carl Rogers and R.E. Young amongst many others.

These contributions assist us in conceiving of more responsive, dynamic, shared and inclusive learning environments and communities, and of how to create effective and positive relationships between teachers and learners. They show us how these approaches can benefit and stimulate ALL learners – not just the most academically capable.

This, the ‘how’ of curriculum, is just as important as the content it contains. The ‘how’ of curriculum, the way we enable young people to engage in learning, must encourage young learners to make an intellectual and emotional investment in their learning by having input into how it is designed and conducted. That is the real beginning point to their engagement – enabling their committed buy-in to the process of formal learning.

Thirdly, in our teacher professional learning and development opportunities, in both the pre-service and in-service career stages, we need to continually emphasise the role of teachers as professional, imaginative and creative transactors and facilitators of learning. My own suspicion is that too many of our teachers may have come to regard teaching as having become de-professionalised – a profession in which they are simply expected to teach to the dot points the syllabus or school program contains, and to teach to the test.

Those teachers who do feel this way are being quite realistic – an over-crowded, over-mandated, over-tested (and often politically driven and destabilised) curriculum is de-professionalising. We need to give back to our teachers the opportunities and curriculum development skills to create curriculum and learning experiences that capture the hearts and imaginations of our children and young people.

Clearly, some of the above solutions to curriculum may require agitation by the profession and community, leading to macro, politically-endorsed reforms. In the absence of these, there are still very positive things that can be created by schools and classroom teachers from an over-prescriptive curriculum. Many formal curriculum and syllabus documents are not, on a closer reading, necessarily as prescriptive and confining as they first appear. Many mandated themes, topics or teaching points can be interpreted and adapted by the teacher, who can choose what to emphasise within particular topics, how much time should be allotted, what teaching approaches, activities or approaches to assessment might be used, and what opportunities there are to provide students with learning choices. With imagination and creativity, flexibility, personalisation of learning and responsiveness can often be crafted from curriculum documents which may initially seem too prescriptive and unforgiving.

Teachers who do manage to find this flexibility have the opportunity to create spaces in the curriculum into which they can invite their young learners to discuss, craft and conduct learning activities and the content they focus on. These teachers often feel great personal and professional fulfilment when they do engage with their students around their personal learning preferences, and achieve great learning motivation and improved academic outcomes with their learners – even on tests like the NAPLAN (without them having to emphasise the practising of the test).

Let’s return to my opening movie analogy. Imagine instead a classroom in which children and young people are continually participating by suggesting things to learn, and ways to learn, activities to do, ways to assess their learning, and in which they help their teachers to drive learning and learning outcomes. Imagine the creative energy that might drive the group, and the outcomes that might be achieved. Unlike the movies you are forced, unwillingly, to see, this is learning where you see the point, and want to engage, because it is in some ways your curriculum – as a learner (or a movie goer), you help to own the choices. Our curriculum design must be smart enough to enable learning to be personalised, flexible and responsive. Anything less risks more teachers feeling de-professionalised, and more learners in our schools choosing to disengage.

One year after an important Equity in Education symposium at UWS raised this question, a new scholarly book has been published with an international publisher. Contemporary Issues of Equity in Education (Cambridge Scholars Press, Oct. 2014) includes research from scholars in Queensland, NSW and Victoria encompassing primary, secondary and tertiary sectors of education.

According to the editors – Associate Professor Susanne Gannon and Professor Wayne Sawyer of the Centre for Educational Research at UWS – education is one of the “great social justice projects of modern democracy”. They suggest that high quality education must be underpinned by “equity principles” including “access to good schools, challenging and engaging curriculum, committed teaching and engaged learning and appropriate resourcing”. However, they argue, these principles are “currently under assault” from a wide range of directions. The chapters in the book demonstrate some of the ways that researchers are exploring these dilemmas.

Chapters address a wide range of issues including international league tables and testing regimes and their impacts; everyday language and literacy practices of multilingual children; sexuality education in PDHPE curriculum; the assets, resources and experiences of refugee students in schools and universities; high demand pedagogies in low SES schools across diverse curriculum areas; the use of technology for community building; young people’s aspirations and anxieties about their futures; the impacts of increased school leaving ages on young people in single sex, ethnically diverse schools; innovative teacher education for high poverty contexts; beginning teacher experiences in the profession and suggestions for the redesign of secondary schools. 1

Participatory and collaborative research that has been co-designed and conducted with teachers, schools and education systems features throughout the book, with one of the chapters directly addressing the complexities and rewards of ‘teacher-as-researcher’. Throughout the book teachers are understood as intellectual workers, who have much more to offer educational debates than the low-level compliance required by externally designed and imposed assessment and curriculum regimes.

As well as reporting on current research into educational equity, the symposium and book also identified areas that require prioritising in future research. For example, at the upper end of secondary schooling we need to know much more about issues pertaining to school retention and transitions to post-school contexts including further study, training and work. How are these experienced by young people, what are their particular needs and how well are these understood by educational and employment sectors in volatile labour markets? How are senior schooling curriculum and opportunities serving the needs of young people and building and extending on the ‘funds of knowledge’ that they bring with them to school and to the work place? In teacher education, we need to understand how tertiary courses can balance disciplinary expertise with social justice imperatives that ensure beginning teachers are willing and able to support quality outcomes for students from diverse backgrounds and experiences. How might we develop beginning teachers’ capacities to understand and undertake research in collaboration with experienced colleagues and mentors?

More broadly, research agendas need to focus on unpacking the complex factors that contribute to the persistence of educational inequality in Australia and to better understand how schools and teachers can intervene to improve educational engagement and success for young people, their families and their communities. Future networks and activities will respond to these areas of concern. The first of these is the upcoming “Equity! Now more than Ever!” conference run by the Centre for Professional Learning/ NSW Teachers Federation, and the University of Western Sydney, on Nov 7th.

The book Contemporary Issues in Equity in Education can be ordered from the publisher online where you can also access an excerpt comprising the Editorial and the first chapter ‘Equity in Australian Schooling: The absent Presence of Socioeconomic Factors’ by Professor Bob Lingard and Dr Sam Sellar of the University of Queensland.

The provocation in this post is the title of Ken Robinson’s latest TED talk. When teachers at Coonamble High School in rural NSW viewed this talk recently it was greeted with thunderous applause as it concluded. According to executive principal of the school, Margaret Mulcahy: “enthusiasm for what Robinson describes in the TED talk launched what turned out to be a highly successful school development day that firmly focused on 21st Century pedagogy and the Australian National Curriculum”.

What is really interesting here is that alongside pedagogy for classrooms in more technology-rich contexts, some schools are also seeking to re-examine their focus on standardized measures of students’ performances as key arbiters of what students know in 21st Century contexts. There is disquiet in the teaching ranks and Robinson’s work, among other education scholars (Craft, 2012; Ito et al, 2013; Jukes et al, 2012; Martinez & Stager, 2013; McWilliam & Taylor, 2012), calls on governments, schools and teachers to rethink the focus of learning.

This new TED talk now forms a quartet of my favourite Ken Robinson clips, the others being What is in the school of your dreams ; Creativity, learning and the curriculum and Ken Robinson says schools kill creativity. Leaving the other clips to one side for the purposes of this post, Robinson’s most recent conversation uses the metaphor of Death Valley – the driest place on earth – to draw our attention to a new metaphor for education. Robinson explains that the possibility seeds are all in place … surviving … just below the surface and they are waiting for the right conditions to germinate and to burst into flower. He says: “Current education policies in many countries are mechanistic. The push for better data from standardised tests to give us better information to fine tune schools …. is just not true … it won’t … and it never did”. His argument is based on the fact that education is a human system and the compulsion of more tests, more conformity, less breadth in curriculum and more emphasis on teaching as task, will not facilitate good learning for students. He describes how levels of disengagement among students in schools in many countries are evidence of the failure and slippage of current ‘testing’ policies in education.

Like Robinson, I agree ‘tests and testing’ have a place in schools and in education more generally, but they are not the whole story. When governments look to standardised tests as barometers by which to gauge, success or failure, or good or poor teaching in schools, it is ‘thin education measurement’. Rich measurements of schools, teachers and students’ performances comes through using data, for example, to diagnose how students are learning, see how curious they are, understand how well they engage with the whole curriculum, and how often they are given opportunities to produce and express learning in creative ways. This type of measurement and this kind of learning takes time and time means students may cover less content but what they know is deeper, plentiful, imaginative and motivating. And, in such classrooms students are given, as Csíkszentmihályi (1996) found: “Time to get into flow in their learning” (p.22).

In new research (Hunter, 2013) exemplary teachers in Stage 1-5 classrooms in some NSW schools spoke about NAPLAN in the context of how they conceptualized their knowledge of technology integration. While the study teachers were not completely critical of NAPLAN they called for it to be re-imagined. For example, one teacher said: “NAPLAN should be telling us about our students’ progress and how I can improve my teaching” and another described its effect as meaning: “The hijacking of learning in schools right now”*. In examination of more than 500 separate references to support research in the doctoral study (Hunter, 2013), a strong case is made in the work of Zhao (2009, 2012) that shows following East Asian models of schooling (characterized by high levels of testing with recall and reproduction seen as important) in countries like China, South Korea, and Singapore will not develop young people’s creativity and entrepreneurship, nor allow opportunities for exploration, experimentation and expression of meaningful learning. Zhao (2013) cites statistics that show less than 1% of the world’s patents (as useful measures of original thought and innovation) are produced annually by China, whereas in countries like the US, patent production for the same period is greater than 34%. Other education literature (Chen, 2010; Pellegrino & Hilton, 2012; Richardson, 2012) details how ‘innovation and individuality’ are being driven out of public schools globally as a direct consequence of ‘testing regimes’. Such evidence aligns with what Robinson repeats in this recent conversation.

And finally, to return to the conclusion of the TED talk, Robinson invites jurisdictions, schools and teachers to shift the ‘testing focus’ and to think about education in a climate of possibility using his Death Valley metaphor. He reminds us of the famous Benjamin Franklin quote about three classes of people in the world: “The immovable, the movable and … those that move”. Perhaps, now is the time for educators to move?

*Submissions to the Senate Education, Employment and Workplace Relations Committee’s Inquiry and Report on “The effectiveness of the National Assessment Program – Literacy and Numeracy (NAPLAN)” were received until the 7th June 2013. The list of submissions can be viewed here.