Some thoughts on KS2 Progress

Caveats first: these conclusions, such as they are, are drawn from a small sample of a little over 50 schools. That sample of schools isn’t representative: indeed, it has slightly higher attainment than the national picture, both in terms of KS2 outcomes, and in KS1 starting points. However, with over 2000 pupils’ data, it shows some interesting initial patterns – particularly when comparing the three subject areas.

Firstly, on Maths – the least controversial of the three subjects. It seems that – in this sample – pupils who achieved Level 2c at KS1 had an approximately 40% chance of reaching the new expected standard (i.e. a scaled score of 100+). That leaps to around 66% for those achieving L2b at KS1 (i.e. just short of the national average)

The orange bar shows the average of this sample, which is slightly higher than the national average of 70%

It’s important to note, though, that progress measures will not be based on subject levels, but on the combined APS score at Key Stage 1. The graph for these comparisons follows a similar pattern, as you’d expect:

Where fewer than 10 pupils’ data was available for any given APS score, these have been omitted.

There is an interesting step here between pupils in this sample on APS of 13 (or less) who have a chance of 40% or less of reaching the expected standard, while those scoring 13.5 or more have a greater than 50% chance of achieving the standard. (The dip at 12.5 APS points relates to pupils who scored Level 2s in Maths and one English subject, but a level 1 in the other, highlighting the importance of good literacy for achievement in KS2 Maths)

For Reading, the graphs look broadly similar in shape

Blue bar shows average of this sample at 67%, which is slightly higher than national average of 66%

Interestingly here the level 2c children scorers still have only 40% chance of meeting the expected standard, but those achieving 2b have a lower chance than in maths of reaching the expected standard (58% compared to 66% for Maths).

When looking at the APS starting points, there is something of a plateau at the right-hand end of the graph. The numbers of pupils involved here are relatively few here (as few as 31 pupils in some columns). Interestingly, the dip at 18.5 APS points represents the smallest sample group shown, made up of pupils who scored 2a/2b in the two English subjects, but a Level 3 in Maths at Ks1. This will be of comfort to teachers who have been concerned about the negative effect of such patterns on progress measures: it seems likely that we will still be comparing like with like in this respect.

It is in Writing that the differences become more notable – perhaps an artefact of the unusual use of Teacher Assessment to measure progress. Compared to just 40% of pupils attaining L2c in Reading or Maths achieving the new expected standard, some 50% of those in Writing managed to make the conversion, and this against a backdrop of teachers concerned that the expected standard was too high in English. Similarly, over 3/4 of those achieving Level 2b managed to reach the standard (cf 58% Reading, 66% Maths)

In contrast to the other subjects, attainment in this sample appears notably lower in Writing than the national average (at 70% compared to 74% nationally)

With the APS comparisons, there are again slight dips at certain APS points, including 18.5 and 19.5 points. In the latter case, this reflects the groups of pupils who achieved Level 3s in both Reading and Maths, but only a 2b in Writing at KS1, suggesting again that the progress measure does a good job of separating out different abilities, even using combined APS scores.

Of course, this is all of interest (if you’re interested in such things), but the real progress measures will be based on the average score of each pupil with each KS1 APS score. I’d really like to collect some more data to try to get a more reliable estimate of those figures, so if you would be willing to contribute your school’s KS1 and KS2 data, please see my previous blog here.

Spread of data

Following a request in the comments, below, I’ve also attached a table showing the proportions of pupils achieving each scaled score for the two tests. This is now based on around 2800-2900 pupils, and again it’s important to note that this is not a representative sample.

Hi Michael, for secondaries looking to get an indication (and I know it would only be an indication) of the spread of results, would it be possible to post a histogram showing percentages of students that achieved across the range from 80-120 for reading and maths please?

New….but not so new new ‘interim’ frameworks out for 2016 – 17 as they remain unchanged ……so here we go again for yet another year. No sign of long awaited Rochford report on P levels and in the meantime we are left with interim pre Key Stage standards that don’t help sufficiently support SEN attainment and progress in being able to bridge the gap between P levels and NC expected standards.

Thank you for these, really useful. I have looked at the last table and come up with some rough deciles. A child getting a mark of 98 in Maths would be in the lowest 20-30% of children taking the test. A child with a score of 108 in English would be around the 80th percentile – scoring higher than about 80% of children taking the test.

Median progress is no longer the issue. For floor standards you need to have scores which are above -5 for Reading/Maths and above -7 for Writing. For the coasting standards, we don’t yet know the thresholds.
Overall, the national average score is 0 for each, as that is what it’s based on (so your progress figures look very healthy!)

Does that mean that 0 is the average? Are you aware of any data that tells us what percentage of schools achieve progress scores in each band? I presume a lot will fall between -3 and +3.
I guess this data is currently not available but it must exist.

Having ‘explored’ the data in the spreadsheet from the tables checking exercise for our (small) cohort, there are some worrying anomolies created by relying on the KS1 APS as an indicator of general ability in a given subject….

Problem 1: The decimal in the target score is disingenuous for a start – there are no decimal scaled scores and there are some pupils who will have missed out on their target by an unattainable decimal margin.

Problem 2: The margin of difference in the scaled scores looks so small… there’s ‘only four points in it’ you might say…. but the difference between those two target scores in real terms was HUGE.

A mathematics scaled score of 104 (to exceed Child 2’s decimal target) equates to a minimum of 78 points in the test whilst a score of 108 (to exceed Child 1’s decimal target) required 93 points. So child A required 15 marks more than Child B from the outset to ‘make progress’.

Similarly, a rough estimate makes that scaled score of 104 equivalent to an old 4A, whilst 107 would have been a 5B. Their actual score of 110, by my reckoning, is a 5A in old money.

Problem 3: Despite these two pupils having the same ‘starting point’ in KS1 and subsequently achieving the same end score in KS2 Child 1 is deemed to have made ‘more progress’. One was targeted at 6 sub-levels of progress under the old system, the other at 8 sub-levels.

In reality, they both effectively went from a 2A to a 5A, and each made the equivalent of 9 sub-levels of progress.

In real terms, they both did equally brilliantly. The progress data suggests that one didn’t do quite so well.

To compound this issue, another child who got a 2A in maths at KS1 wasn’t quite as capable at reading and writing. As a result his APS was lower and his maths target was just 101.5 (equivalent to a minimum of 70 marks)

He attained 105, giving him a progress measure of 3.5. Effectively, despite ‘only’ attaining the equivalent to a 4A/5C borderline, he is deemed to have made more progress than a child who went from 2A to 5A!

To recap, here’s the implied ‘target conversation’:

“Well children, you all did a little bit better than average at KS1 in maths this year. We’re going to set you progress targets for Year 6 based on your levels of 2A. Child 1 – you have to score at least 78 marks. Child 2 – you need 93. Child 3 – you’re good at maths but can’t read and write very well – you just try to get 70 marks in maths in Year 6, good lad…’

Although all these things look like nonsense, I do think that the new measure is better than it first appears. Instinctively we want to compare children who had the same level in a given subject at KS1 (as your 2a maths children), but actually the DfE stats show that reading & writing ability at KS1 *do* have an impact on attainment at KS2, and that as a result, the estimated outcomes should be affected by them.
They’ve presumably crunched the numbers to conclude that the combined figures are fairer overall. It will never be perfect, but just because the system *looks* more unfair, doesn’t necessarily mean that it is.

Once the current year 3’s reach year six and are being compared against the same numerical scoring system, it’ll probably work ‘better’. But the current variation in targets from the ‘same’ starting point isn’t ….

It’s hard to see just how wide that variation is with just 11 kids, but comparing those in the same ‘KS1 Attainment band’ which is subsequently converted to a ‘prior attainment band for progress measure’ shows that this is primarily allocated according to the APS score.

As such it disregards the actual levels attained in each subject, as various configurations will generate the same APS.

Example:

We have two children in ‘KS 1 band 2’ which becomes ‘progress group 14’.

Their KS1 results were:

Child A – R-2A; W-2C; M: 2A
Child B – R – 2B; W -2B ; M-2A

Both had a KS1 APS of 16. As far as I can tell, it is this alone that dictates which band they go into and which generates their targets.

Despite the differences in their starting levels, their KS2 targets are identical for each subject.

There could easily have been a child who had R-3 W-2B M-2C which would also have been an APS of 16 and who would have had the same KS2 targets.

In maths that would have meant converting their 2C to an equivalent of 4A… against a backdrop of increasingly difficult written calculations as well as ‘fractions, decimals and percentages’.

That conversion isn’t entirely beyond the realms of possibility, but it’s fairly challenging: whilst for child A and B, it would be relatively simple.

Actually, a child with Reading L3, Writing L2b and Maths L2c would have had an APS of 15.5 so been in a lower PA group.
As I say, it’s not perfect, but I’m not persuaded that it’s any worse than the old system.

Matt4 September 2016 at 7:11 pm

I’m not sure it will work better when current year 3s get into year 6. Our current year sevens could have received a 3, 2A, 2B, 2C or a 1 at Ks2. (Mainly) but his years year two children were predominantly split on GDS EXS OR WTS. (Again mainly) so there may be a huge ability difference between child A who achieved EXS and child B who achieved the same grade. Although schools generated a scaled scores on year 2 Sats these were not reported.
I wonder whether this will mean a push toward reporting test scores as scaled scores at KS1 rather than teacher assessment.

what about children working below the level of the tests who come from P Scales. it appears that those coming from PAG 2 cannot achieve a positive progress score unless they get a minimum score of 84 points on the tests. the Pre key stage always gives them a negative score