A.P.P.

July 15, 2009

A spectre is haunting education — the spectre of APP. All the powers of the educational bureaucracy have entered into a holy alliance to impose this spectre: the DCSF, OFSTED, senior managers and Local Authority consultants.

APP is “Assessing Pupil Progress”. Or possibly it is “Assessing Pupils’ Progress”[1].

It is a method of formative assessment[2]. Or possibly it is a way to calculate a grade[3].

It is a way to assess an entire class[4]. Or possibly it is based around monitoring a sample of students and then guessing for everybody else[5].

It is a new package of assessment materials[6]. Or possibly it is meant to be based on normal class work[7].

It is meant to be used alongside other forms of testing[8]. Or possibly it is meant to replace other assessments[9].

It is meant to be occasional[10]. Or possibly it is something that happens all the time[11].

It will be expected in all schools[12]. Or possibly it will be completely voluntary[13].

It is based on collecting evidence[14]. Or possibly on the opinions of teachers[15].

It has been thoroughly researched by pilot schools[16]. Or possibly the pilots only ever tried using it on a small sample of pupils[17].

It all depends who you ask. The idea has never been consistent or clear. There are only two things anyone seems to be sure of.

1) APP involves using the National Curriculum to identify objectives known as “assessment focuses”[18] (or is it “curricular targets”?[19]) and then ticking students off on a grid when they reach them.

2) This ball-achingly, pointless piece of paperwork is the responsibility of classroom teachers who will have to be trained, patronised and scrutinised.

God help us all.

Of course, there are a number of reasons why a long list of objectives is never a good way to assess students.:

There is too much paperwork, particularly for secondary teachers who may teach hundreds of students. It takes too long to do and when done often creates more data than can ever be used productively.

It is seeking to measure something which is largely a matter of opinion, i.e. whether a student’s grasp of a particular skill or piece of knowledge is firm.

It is measuring something that is complex and constantly changing. Children will learn new skills and forget old ones faster than the paperwork can be updated.

It encourages vast bureaucracies which spend time creating objectives to be met.

Because of the sheer quantity of knowledge and skills that might usefully be taught in schools, attempting to list them will result in either woolly objectives that cover many different things, or content will have to be reduced in order to fit a limited number of objectives.

Because it is time-consuming, useless and subjective teachers will just fake it anyway, and spend the time doing something that will actually benefit the kids instead.

You might think I am being negative for the sake of it. You might think I am resisting an idea simply because it is new.

But if so you have forgotten one of the wisest aphorisms in teaching: There is no such thing as a new idea.

Of course, this crap has been tried before:

1) It was used briefly back in the late 1980s when the National Curriculum was first introduced, but soon abandoned as a waste of time.

2) This was the approach used for assessment for NVQs. These were one of the biggest of the many disasters in vocational educational. Despite a fortune in government money and attempts to convince employers that they were valuable qualifications, most NVQs turned out to be pointless exercises in box-ticking that nobody wanted to do. Hundreds of NVQ qualifications were created at great expense that nobody at all ever did[20].

3) This approach has been tried in the US where the objectives were known as “standards”. This resulted in reduced rigour and eventually the political tide moved in favour of more conventional pen and paper testing[21].

So when we look at APP we are talking about an idea that has repeatedly failed in the past, for which there is every reason to believe it can never be effective, and for which every utterance from the authorities has been contradictory and confused.

Good schools are ignoring it, or playing with it half-heartedly, confident that it will either fall apart, change into something less ridiculous or that somebody will come up with an effort-free way to fake it. Bad schools are declaring that OFSTED will require it to be “embedded” and have wasted time, money and good will trying to impose it on their teaching staff before anybody even knew what it was. Worse, there is a real danger that it is resulting in more reliable forms of assessment (including genuine formative assessment) being squeezed out.

[7]http://nationalstrategies.standards.dcsf.gov.uk/node/20718?uc=force_uj says “It does not require special assessment activities but involves recognising significant evidence from the opportunities generated by planned teaching and learning. It reduces the need to use tests and specific assessment tasks to make assessment judgements by taking into account a far wider range of evidence.”

[17] Both http://www.qca.org.uk/qca_23863.aspx says the pilot “required participating teachers to submit termly data from a small sample of pupils in their classes” and http://www.qca.org.uk/qca_23864.aspx concluded by admitting that the pilot “required participating teachers to submit termly data from a sample of between 6 and 12 pupils in their classes”. There is no report for using APP at Key Stage 3.

Like this:

Related

62 comments

The good news is, of course, that no one will ever check these records in most cases, nor, if they do check to see they have been done, will they have any way of knowing whether they are correct. So – if I am forced to do this – I’ll just guess and award resoanable sounding results showing distinct progress. OfSTED will just want to know it’s being done, so do it. It’s just another ’emperor’s new clothes’….!

My favourite thing about APP so far is that two of our teachers were cascading (yes, it’s on the list of words I hate) their training down to us, and explaining how they’d been told we wouldn’t need to come up with an overall subject level, just separate ones for teach AF (I’m using the acronyms already). While they were telling us this I was skimming through the booklets they gave us and found the page describing how to come up with an overall subject level. They’re going to seek clarification. Good luck to them.

I have sat rigid with boredom and frustration through many hours of APP training. The sad thing is, that my colleague who is responsible for delivering this nonsense, is actually excited by it claiming it to be a wonderful ‘diagnostic tool’. Her biggest problem, however, is how we are going to make it meaningful to the pupils. Unfortunately she is not being ironic.

ICT already has APP i have been using it for a while now.
It is summative rather than formative, although the APP descriptions are a useful tool in planning for pupils.
APP is not compulsory but we are using it as it is a very useful framework and helps when making professional judgements about pupil levels.
I like it and can’t condemn it just becaue it is yet another initiative.

‘You might think I am being negative for the sake of it. You might think I am resisting an idea simply because it is new.’

I did’nt ignore your post. I am wading through it to try and determine whether the above comment is justified. I wanted to do it justice. You have clearly given the matter a good deal of thought and taken the time to write the blog entry.

As an extremely complex issue I do not intend to make a snap judgement. I am currently working my way through my copy of “Does Education Matter?” to determine whether Alison Wolf’s views on the impact of Vocational Qualifications are relevant to the issue of the efficacy of APP in schools.

For a little while I have been looking at collecting data on a continuous basis in my lessons and using this data in addition to that gathered via other instruments such as formal written tests in order to allow me to make better informed judgements about my pupils. I have a suspicion that APP will assist me in this process.

Once I feel that I actually understand what you are saying and can see the strengths and weaknesses in what you are saying I may post again.

To help me out, could you tell me what you mean here.

‘Of course, there are a number of reasons why a long list of objectives is never a good way to assess students.:’

Clearly objectives canot assess pupils therefore I now this is not your meaning. Are you saying that objectives (what is to be learned) should not be the focus of an assessment instrument?
Is it the “long list” you oject to?
Is it the theory behind the approach or the implementation you object to?

I am not asking this to provoke an aggressive response, It would be useful to me in my analysis to know the answer. Any time hat you spend answering the question will be appreciated.

Would you not agree that just because a sound idea has failed in the past due to poor implementation, doesn’t mean it cannot work in the future with better implementation.

The theory being that you specify what it is that the person needs to learn. You then look for evidence that they have learned it from whatever source. You then use the results of the process at any time to develop an appreciation of whether the pupil has met the objective and therefore learned what they were suppsosed to have learned. Indeed one might also identify things that the person has learned that were not within the scope of the orginal objectives.

If you are doing it already then surely you can just keep on ding what you are doing. The powers that be at your place may not have the incisive grasp of this stuff that you do and they may try to get you to use the ‘proper’ forms but surely it is what you are doing in the classroom and out that matters.

i thought the idea was to roll it out across the board. i looked at the app website- its a v complex site so maybe it is just ks3- i could be wrong there…

so for a whole year and say, 4 ks3 classes for an average teacher.

4 classes x 25 kids x 12topics x 20 objectives =

24 000 boxes per year providing we assume its one box per objective.

obviously some teachers teach mostly ks3 so that number is maybe too low.

it seems like beuarcracy to me?

i use formative and summative assessment to inform my planning and reflect on my practice and was trained that way. However I wouldnt record hw or cw detail in the minutae way the website materials suggest.

You certainly couldnt hold them in your head to meaningfully inform planning- thats Rain Man Territory!

As far as I know the NASUWT does not have a reputation for misrepresenting it’s members but I suppose anything is possible. It is a balanced article which looks at benefits and examples of poor practice but it may be a work of fiction.
Just bringing it to the attention of readers.

Unions, particularly those people with union positions who rarely step foot in a classroom, are often inclined to cheerlead the latest bits of fashionable nonsense. (Remember the union speaker who suggested referring to “failure” as “deferred success”?) If they weren’t then the fashionable nonsense would have a hard time making it to the classroom. They also, regretably, have a long term agenda of replacing statutory assessment with teacher assessment and APP may well help that.

That said, I am glad to say my own union has been a bit brighter than the NASUWT on this issue.

Some views of teachers here. First thread is hort and illustrates some of the possible issues. Second is longer but contains a fair number who are positive about AAP. Shows perhaps that benefits depend upon the person implementing and the way it is implemented in individual schools.

The first thread only shows one positive comment (which I believe is from you) and the other thread is predominantly full of teachers who are confused about what they are supposed to be doing and asking for help. Most comments actually expressing an opinion are negative.

I would have to agree with oldandrew here.
There was 1 page in the 1st forum and 5 pages in the 2nd. All were overwhelmingly negative or at least confused. In the 1st link a chap called mr jones was pro APP but the other 8 posts were not.

Julia C Walker 18/04/2009 at 15:17 wrote:

“As assessment co-ordinator I am leading the phasing in of APP and a group of us are currently trialling it – you are right it is a lot of work!! I’ve too been told not to change the statements and I am loasthed to sit and type out a whole new set of sheets etc as I can see it being a waste of time. I have made a simple leaflet for teachers which I used at an inset. I am happy to share if anyone is interested??”

She echoes what I heard at my inset on APP.

A lot of people wanted her leaflet by the way!

I have found, and I admit I have only taught maths, science & PSHE at secondary, is that short tests during the year followed, by an end of year exam is the most reliable, fair and efficient way of assessing students.

I have quoted a number below which were clearly not overwhemingly negative or confused. There were a large number finding it hard going and wanted a copy of the leaflet. There were one or two that had been to an inset session of some sort and come out with the idea that it was not a good initiative.

There were several who said that they found it useful but they could see that if not implmented correctly it could be hard work.

A number were using it successfully and they actually explained how and why. It clearly isn’t going to be the answer for all.

Rob, have you ever read the book by Derek Rowntree: Students how will we know them or something very similar.

Actually I have not approved the post that seemed to be a collection of the least negative comments on that thread as I think it fairer to leave it to readers to decide for themselves what they think the balance of that thread is.

I counted the negative posts and some of these were from people who (like yourelf I fear) had not even tried APP and there were more positive posts than negative.

I quoted the positive posts and you simply cut my post.

You posted a statement that was incorrect and I corrected it posting the evidence to support this.

All you have achieved is to provide a one sided view that supports your view. You did not cut other quotes which supported your view in or to let people make their minds up, one of the privileges of editing a blog I guess.

Some people won’t have time to read the whole of the TES post and will have to rely on your incorrect description, not for the first time.

The point is that my “one sided view” took two sentences and consisted of my own words.

Your view consisted of several paragraphs of cut and pasted material from the TES website.

As you know, the comments facility is for discussion, not for repetition of existing material. Anyone who wishes to decide who is correct about that TES thread can follow the link. If they have trouble finding positive material (I certainly did) then that just confirms the balance of opinion on that thread.

Besides which, at no point in my post did I ever say that “nobody can be found who supports APP”. There is no shortage of willing idiots to impose crap initiatives on us and APP is well funded and heavily promoted. (The same idiots will be telling us that in a couple of years time that it was a good idea in principle but was implemented badly.) So I simply cannot see the point of your desperate effort to suggest that you can find people who will support APP. If anything, what is noticeable is how weak the support is even in the places you are suggesting we look.

i have been back to both links again, i’m afraid i have the same conclusions as I did before so we will have to disagree on that one.

my worry is that even if it doesnt become compulsory, middle leaders will be forced by SLT to do it because it will make their school look bad if they ‘dont join the party’.

so schools will ether replace rigorous assessments with unreliable ones.

or, they do both types requiring excessive workload.

its a lose/lose situation as far as I can see.

again, I have only looked in detail at the science APP stuff on the website- very poor.

during the inset i did this year, I kept my mouth shut, but watched as the course leader, who was charming and capable failed to answer any of the audiences misgivings:

eg. how do we know when this exemplar was done?
Is this evidence from group work or individual work?
Do we level each seperate objective statement?
Do pop quizes count?
Does HW count?
Does copying from a book count?
Does doing questions count?
Can project work count?
What if we think a parent helped with the project?
Do I have to walk around with a clip board whilst the students do the work and record their answers?
Do I manually record stuff from Q&A sessions?
Is this exemplar material really supposed to be robust?
Whats the difference between this and the notorious orange level book from 1993?

These questions remained unanswered yet at least 3 people at the inset said their school was forcing everyone to do it across the board.

it has a real feeling of ‘curriculum 2000’ about it- anyone remember that- everyone happy they paid their taxes for those 2 years?

And that simply is not true, there are more positive comments than negative, from people who have actually implemeted APP.

I actually took the time to paste the positive comments and the blog owner simply cut the post. It seems he only wants quotes that agree with his view.

I am not a science teacher rob, but after you earlier in the blog explained that as a result of your inset your workload would be unworkable. I had to tell you that it isn’t yet introduced in KS4. You clearly have a poor understanding of CPP an maybe this is why you have so many worries.

You have to assess learning. How you do this is up to you as long as you can defend you methods.

Constructive criticism is one thing but this sort of prejudice is what gets teaching a bad reputation.

the info on APP is available on the APP site as pdf files. Anyone can access them. They are ks3 as far as i can tell.

on my inset however we used exemplar material for years 7-10. any confusion i may have had would stem from the fact that the course leader couldnt provide straightforward responses to our questions. judging by the links im not alone.

i have also seen and heard it is being rolled out for ks4 as well as a formative assessment tool in some schools.

my misgivings were shared by those on the INSET I attended and according to old andrews stats above, I seem to be in the majority.

However being in the majority doesnt mean all that much. I am very much my own person, for example, I am (marginally) pro SATS which probably puts me in a group of 1 amongst teachers!

My concerns stem from the questions my peers asked during the inset which were not answered then, nor on the website at a later date.

having been a victim of half baked, expensive initiatives in the past it does get rather tiresome to see the familiar cycle of events at the expense of taxpayer, student and teacher.

im happy to be proved wrong however- i once predicted iphones would be a flop- so there you go. guess what im typing these comments on!

I actually copied and pasted the positive comments so that people could read them and you cut the post. Having cut the copied and pasted threads you have now posted your own self indulgent interpretation which in some way is supposed to prove you correct oldandrew.

I copied every post to support what I was saying.

By the way, there are threads on TES in both the ICT forum and the have your say forum which include posts from people who use APP and use it effectively.

I am sure that there will be enough insightful teachers in the profession who will continue to use and develop APP to the benefit of those whom they educate. Let us just hope that the “it’s all been done and failed before” teachers are not too proud to eat their words when APP succeeds despite their best efforts.

You just carry on what you are doing oldandrew, as I am sure you have been doing for a great many years. And look where it has led you, to this blog.

I do love this idea that because you haven’t been able to post long selective quotations from a TES thread then you are somehow hard done by.

We can all look at the thread. It is overwhelmingly negative. Even if it hadn’t been, the existence of people who are willing to declare any bats-arsed initiative to be a success, doesn’t in any way give us a reason to doubt the main points of the blog entry, i.e. APP is ill-defined, poorly tested, impractical and based on an idea that has been repeatedly tried and failed in the past.

I am quite happy to wait out this initiative, just as teachers have had to wait out the previous efforts to do the same thing.

For what it’s worth, my view of APP is generally positive. It is a tool that provides both formative and summative assessment. It can be based on a collection of work produced by a student over a period time, and it can include specific APP “tests”. The latter are, in my opinion, weak.
My biggest concern about APP is the amount of physical paperwork it involves, and would like to see a proforma designed around 3 assessment periods, say, in the year.
I would also like to add that my department have also been very positive about APP. They feel that it has helped them to identify specific areas of need and helped them focus their teaching.

Is it anything new? Well, assessment has, of course, always been at the heart of teaching. But I think APP provides a useful framework.

And, quite frankly, I’d rather have APP than SATs.

The posts here clearly show a lot of muddled information and confusion surrounding APP, and this is a shame given the potential of the system.

As for the objectives not being pupil friendly – they are for use by teachers.

The complaint was not that it wasn’t new, but that when it has been tried in the past it hasn’t worked.

Also, could you elaborate on how it has helped staff identify specific areas of need? I mean, I can see how an APP tick-list of curricular targets could identify topics or skills that students were struggling with, however, the only way to fill in the tick-list is by seeing students struggling with the topic in the first place. Why not cut out the middle man?

I am not aware of any previous versions of this strategy; I probably haven’t been in teaching long enough. I see APP as a tool to aid what were are already doing, to fine tune our assessment – both informative and summative.

In my subject area, one looks at a body of student work produced over a given period of time – say a term. The AFs are focused on skills, rather than knowledge, so it is possible to identify that a student, a number of students, or a whole class, need to develop a particular skill in order to progress.

And, frankly, you seem to be doing your dismissive thing. You have a hostile way of engaging (or rather, not engaging) with others.

I said that I am not aware of any previous versions of the strategy, meaning that I have not seen any myself during my time in teaching. This was in response to you saying “The complaint was not that it wasn’t new, but that when it has been tried in the past it hasn’t worked.”

My English Dept is asking its teachers to do all 4 optional tests a year (Reading + Writing) with each year group in KS3. Each individual test must be marked for each individual AF. They are also asking us to devise a new and exciting KS3 curriculum with which to energise and enthuse our students. Around these tests.
I always thought I was Briggs – turns out I’m Mother Hen Kay.

I’m so sorry to hear that. What a pity.
Are they asking for every child to be assessed using this format or merely a sample group of children instead?
If you thinks its unfair workload you could ask your union to get involved?
good luck anyway.