Brexit stands out as a disaster of the UK First Past The Post (FPTP) system and the illusion that one can use referenda to repair disproportionalities caused by FPTP. This information is missing in the otherwise high quality overview at the BBC.

In the earlier Puzzle on the YouGov poll I estimated Brexit preference orderings from a summary statistic published by YouGov. The next step is to use these orderings for the various voting philosophies. I will be using the website of Rob LeGrand since this makes for easy communication. See his description of the voting philosophies. Robert Loring has a website that referred to LeGrand, and Loring is critical about FPTP too. However, I will use the general framework of my book “Voting theory for democracy” (VTFD), because there are some general principles that many people tend to overlook.

Input format

See the Puzzle weblog text for the problem and the excel sheet with the estimate of the preferences and their weights. LeGrand’s website now requires us to present the data in a particular format. It seems best to transform the percentages into per-millions, since that website seems to require integers and we want some accuracy even though the estimate is tentative. We can also drop the preference rankings with zero weights. Thus we get 14 nonzero weighted options. We enter those and then click on the various schemes. See the YouGov factsheet for the definition of the Brexit options, but for short we have R = Remain, S = Soft / Single Market, T = Tariffs, H = Hard / WTO. Observe that the Remain options are missing, though these are important too.

Philosophy 1. Pareto optimality

The basic situation in voting has a Status Quo. The issue on the table is that we consider alternatives to the Status Quo. Only those options are relevant that are Pareto Improving, i.e. that some advance while none lose. Commonly there are more Pareto options, whence there is a deadlock that Pareto itself cannot resolve, and then majority voting might be used to break the deadlock. Many people tend to forget that majority voting is mainly a deadlock breaking rule. For it would not be acceptable when a majority would plunder a minority. The Pareto condition thus gives the minority veto rights against being plundered. (When voting for a new Parliament then it is generally considered no option to leave the seats empty, whence there would be no status quo. A situation without a status quo tends to be rather exceptional.)

In this case the status quo is that the UK is a member of the EU. The voters for R block a change. The options S, T and H do not compensate the R. Thus the outcome remains R.

This is the fundamental result. The philosophies in the following neglect the status quo and thus should not really be considered.

PM 1. Potentially though, the S, T and H options must be read such that the R will be compensated for their loss.

PM 2. Potentially though, Leavers might reason that the status quo concerns national sovereignty, that the EU breaches upon. The BBC documentary “Europe: ‘Them’ or ‘Us’” remarkably explains that it was Margaret Thatcher who helped abolish the UK veto rights and who accepted EU majority rule, and who ran this through UK Parliament without proper discussion. There seems to be good reason to return to unanimity rule in the EU, yet it is not necessarily a proper method to neglect the rights of R. (And it was Thatcher who encouraged the neoliberal economic policies that many UK voters complain about as if these would come from the EU.)

Philosophy 2. Plurality

On LeGrand’s site we get Plurality as the first step in the Hare method. R gets 35% while the other options are divided with each less than 35%. Thus the outcome is R.

(The Brexit referendum question in 2016 was flawed in design e.g. since it hid the underlying disagreements, and collected all dissent into a single Leave, also sandwiching R between various options for Leave.)

Philosophy 3. Hare, or Instant Run-off, a form of Single Transferable Vote (STV)

When we continue with Hare, then R remains strong and it collects votes when S and H drop off (as it is curiously sandwiched between options for Leave). Eventually R gets 44.4% and T gets 55.6%. Observe that this poll was on June 12-13 2017, and that some 25% of the voters “respect” the 2016 referendum outcome that however was flawed in design. I haven’t found information about preference orderings at the time of the referendum.

Philosophy 4. Borda

Borda generates the collective ranking S > T > R > H. This is Case 9 in the original list (including zero weights), and fortunately this is single-peaked.

Philosophy 5. Condorcet (Copeland)

Using Copeland, we find that S is also the Condorcet winner, i.e. wins from each other option in pairwise contests. This means that S is also the Borda Fixed Point winner.

Conclusions

The major point of this discussion is that the status quo consists of the UK membership of the EU. Part of the status quo is that the UK may leave by invoking article 50. However, the internal process that caused the invoking of article 50 leaves much to be desired. Potentially many voters got the suggestion as if they might vote about membership afresh without the need to compensate those who benefit from Remain.

Jonathan Portes suggested in 2016 that the Brexit referendum question was flawed in design because there might be a hidden Condorcet cycle. The YouGov poll didn’t contain questions that allowed to check this, also because much has happened in 2016-2017, including the misplaced “respect” for the outcome of a flawed referendum. A key point is that options for Remain are not included, even though they would be relevant. My impression is that the break-up of the UK would be a serious issue, even though, curiously, many Scots apparently rather prefer the certainty of the closeness to a larger economy of the UK rather than the uncertainties of continued membership of the EU when the UK is Leaving.

It would make sense for the EU to encourage a reconsideration within the UK about what people really want. The Large Hadron Collider is expensive, but comparatively it might be less expensive when the UK switches to PR, splits up its confused parties, and has a new vote for the House of Commons. The UK already has experience with PR namely for the EU Parliament, and it should not be too complex to use this approach also for the nation. Such a change might make it also more acceptable for other EU member states if the UK would Breget. Nigel Farage much benefited from Proportional Representation (PR) in the EU Parliament, and it would be welcome if he would lobby for PR in the UK too.

Nevertheless, given the observable tendency in the UK to prefer a soft Brexit, the EU would likely be advised to agree with such an outcome, or face a future with a UK that rightly or wrongly feels quite maltreated. As confused as the British have been on Brexit, they might also be sensitive to a “stab-in-the-back myth”.

The French general elections for the Legislative were held on June 11 and 18 2017. The results provided by the French government are presented more accessible in wikipedia (a portal and no source), and have been used in this 2017-France-Lorenz-Gini excel sheet to determine the Lorenz curve and Gini coefficient.

The earlier discussion on Lorenz curve and Gini was about the Dutch and UK general elections.

Both UK and France have district representation (DR) with a First Past the Post rule. In the UK this causes strategic voting, in which a voter may not vote for the candidate of first choice, but tries to block a candidate who might win but would be worst. France has elections in two rounds so that there is less need for such a strategy. The second round is between the two top candidates in the district, and thus one might try to get at least one good candidate in that position.

Proportional representation (PR) may allow a larger (but fairer) share of the seats for the more extreme parties, like the party of Geert Wilders in Holland, yet PR also allows more stability for the center. Thus PR tends to avoid the swings between extremes that might happen in systems of district representation (DR).

Two rounds mean two sets of data

The French system seems to make it more difficult to determine the Lorenz curve and Gini coefficient. There are two rounds, and thus there is the question what data to take. However, the following choice suggests itself:

The data of the first round provide the first preferences, and thus provide the votes.

The data of both rounds provide the seats.

This choice finds support in the data. The first round has a turnout of 48.7% and 0.5 million invalid or blank votes. In the second round, more people remain at home, with a turnout of 42.6%, while those who vote produce almost 2 million invalid or blank votes, who apparently disprove of the available candidates or the system itself. Thus the higher turnout and lower blanks in the first round suggest that these indeed present the first preferences (with some limited level of strategy).

The Lorenz curve and Gini

The Lorenz curve shows a rather surprising level of inequality, with a Gini of 41.6%. Compare the value of Holland with a Gini of 3.6%. If the blue line would cover the pink diagonal then there would be full proportionality.

Data on turnout

The following table gives the data on turnout for the first round. The votes for “Elected in the House” is for parties that eventually got elected in the Legislative. The votes for “Not in the House” is for a radical leftist party that got votes in the first round but got no seat in none of the rounds.

The wasted vote consists of the invalid and blank votes and the latter “Not in the House”, to a total of almost 3%. A standard majority would be 289 seats of a House of 577 seats. If one would keep account of the wasted vote, then one might leave seats empty, or use a qualified majority of 298 seats, thus 9 more than usual.

When we divide the electorate by the number of votes per seat, then the Legislative would require 1222 rather than 577 seats. A majority would require 611 seats, which is more than the actual number of seats used. If one would want to keep account of the voters who did not turn out, then 51.3% or 296 of the 577 seats would be empty, or one would use the 611 seats as a qualified majority.

An example of the inequality

The new French President Emmanuel Macron had the highest score of 24% of the vote in the first round of the Presidential elections of 2017, with runner-up Marine Le Pen with 21.3%. Macron then won the second round with 66.1% (20.7 million) against Marine Le Pen with 33.9% (10.6 million) of the vote.

For the Legislative, Macron’s party REM got 27.6% while the Front National (FN) got 12.9% in the first round. For the Legislative Le Pen managed to get only 3 million votes, compared to the potential of 10.6 million at the presidential elections. With both rounds REM got 308 seats and FN got 8 seats.

These ratios would turn, if Le Pen would manage to motivate the voters of the presidential race to also support her for the Legislative. If the other parties would have a divided vote then Le Pen would benefit from First Past The Post.

Conclusions

For the UK in 2017 we calculated a Gini of 15.6% but this was a very tentative number since we had no estimate about the amount of strategic voting involved. For France we have an indication of the first preferences, namely from the first round.

France appears to have a surprisingly high Gini of 41.6%, which can be compared to the system of proportional representation (PR) in Holland that generates 3.6%.

This political inequality doesn’t bode well for the feelings amongst the French electorate about whether they are represented. The low turnout seems to reflect dissatisfaction rather than satisfaction. Such dissatisfaction might also translate into a protest vote over 4 years, especially when Macron doesn’t deliver.

Many observers in Europe seem to be happy with the election of Macron and his party REM, but the outcome is quite disproportional. If this disproportionality can happen for one party then it might also happen for another party – that one doesn’t like as much.

In the former weblog entry, I reported on a rather important YouGov poll. The UK general election was on June 8 and the poll was taken on June 12-13 so that we may assume that persons polled had still vivid recollections. The poll generated data about confusions in the British electorate. It is useful to belabour the point, for Brexit is a key event and would have quite some impact for the coming decennia. I would respect the UK decision to leave the EU but have my doubts when it is not based upon Proportional Representation (PR). A referendum gives proportions but referenda tend to be silly and dangerous, as they are an instrument of populism rather than of representative democracy. Indeed, it appears that the Brexit referendum question was flawed in design. The YouGov poll helps us to observe how confused a major section of the UK electorate is. Let us dig a bit deeper.

The YouGov poll presents the data in a ranking matrix, with the first preferences in the first row, then the second preferences, and so on. For the Brexit referendum outcome of 48% Remain and 52% Leave, for example, we might have the following setup. It is a guess, since the particular ways of Leaving were not included in the referendum question. This example however is the result that you would expect if Remainers and Leavers would have the mentioned consistent orderings.

Observe that each voting weight (take e.g. 48) for a preference order list is put in precisely one place per row and per column, i.e. that it doesn’t occur more times in a single row or column. This explains why the border sums add up to 100.

The YouGov poll of June 12-13 2017

The YouGov data, that I have been referring to, contain the results of a poll of 1651 adults in Great Britain, i.e. the UK excluding Northern Ireland. From page 13-16 we can collect these data for the whole of Great Britain for 2017. YouGov states that the sample has been weighted for social-economic and political indicators. It is not clear to me how the “Don’t know”s are being handled for this particular issue. See also this discussion by Anthony Wells.

We can observe:

These are percentages, and both the row sums and the column sums should be 100, except for rounding errors.

35% has Remain in the first position, 47% has it in the last position, so that 9 + 8 ≈ 17% (a 1% missing due to rounding) has a confused position, in which Remain is sandwiched between some options for Leaving. We would wonder how such people would vote in a referendum when they are presented with only two options R or L. One cannot say that the referendum was only about the first positions in the rankings, for voters would tend to develop an expectation about what would be the likely kind of Brexit and vote accordingly. Some of these 17% might have voted Remain because they disliked the otherwise expected version for Leave. This might indicate that the outcome for Remain was overstated. Yet we have no information on subdivisions of Remain, that might cause an opposite effect. Some might be okay with Remain as it is but vote for Leave because they fear that the UK otherwise might also join up on the Eurozone or some United States of Europe. The reason why the Brexit referendum question was flawed in design is that it left too much to guess here.

Remarkably, the split between R and L now in June 2017 would be 35% versus 65% instead of 48% versus 52% in 2016. In one single year Great Britain switched from fairly divided to a seemingly clear preference for Brexit (though divided upon how) ? I very much doubt this distribution, see the pre-former weblog discussion. The electoral data still suggest more than 50% for Remain. In the former weblog entry it is discussed that some 26% of the electorate say that they voted for Remain but accept the loss at the referendum, so that they “play along” with the winning side, focusing on what would be the best option for Leave. This seems loyal to some notion of democracy, but it would also be a misplaced loyalty to the flawed Brexit referendum question. (One can respect such loyalty, but it still makes sense to discuss it.)

Using techniques of apportionment we can estimate the actual number of people per cell in the poll. My estimate is (and YouGov would have the true numbers):

Possible permutations of rankings

With 4 options there are 4 possibilities for a first place, 3 remaining for the second place, 2 remaining for the third place, and then the final one follows. Thus there are 4 x 3 x 2 x 1 = 24 permutations for possible rankings. We already saw two of these: R > S > T > H and its reverse. Above ranking matrix is actually based upon these 24 possibilities.

Some of these 24 possibilities will be rather curious. It is not clear what to think about R > H> S > T for example (Case 5 below). This would be a Remainer who would rather prefer a Hard Brexit to the EEA or some agreement not to have a trade war on tariffs. A tentative explanation is that this voter has a somewhat binary position, as Remain versus Hard Brexit, while the other options are neglected.

Voting theory may assume voters that are both autonomous and rational, so that any preference would have some logic. This gives rise to the theory of Single Peakedness. Potentially each voter has his or her own criteria so that the best is on top, and all other follow in proper order. However, for the topics of R, S, T and H there is a logical scale from left to right. Voters with multiple peaks in their preferences have more to explain than voters with a single peak. An example of single-peakedness is Case 7 below, with a ranking S > R > T > H. See the graph below. The 1st rank gets utility level 4, the 2nd rank gets utility level 3, the 3rd rank gets utility level 2, and the 4th rank gets utility level 1. The utility levels are just the reversed of the ranks, but then the case must be reordered to the logical order.

Presentation of preferences via preference orderings

The following are estimates for the preferences orderings that would underlie above YouGov results. The estimate minimises the sum of squared error on that ranking matrix, with a weight of 10 for the error on the first preferences. See the excel sheet in the Appendix. This table shows only the percentages and not the numbers of people in the poll (that add up to above table), since the percentages are the main estimation result. Single dots are zero’s. Some have been caused by explicitly setting the possibility of such a preference ordering to zero, see the “comment” keyword for the reason. (A technical reason are also the degrees of freedom.) The ConR / L and LabR / L subdivisions concern the voters in the poll who voted R or L in the 2016 Brexit referendum and who voted Con or Lab in 2017. They form only a part of the sample, so their sum doesn’t add up to the total on the left. The percentages have a decimal to allow easier identification, not for claimed accuracy.

Discussion on GB

Some observations are:

The YouGov summary ranking matrix already showed a rather even split on S, T and H, but the estimate generates a landscape with even more diversity in opinions.

Only 26.2% has the preference R > S > T > H and only 16.2% its reverse, so that 57.7% (addition effect) have some mixture.

Above results for GB can be split up in on the peaks and sandwich. The combinations give the following percentages:

The mentioned 57.7% split up again in 34.6% who are single peaked, and 23.1% who have multiple peaks.

The sandwich of 17.1% splits up into 10.4% with a single peak and 6.7% with multiple peaks.

Of the 23.1% with multiple peaks there are 9.1% who can join the Remainers with a first preference and there are 7.3% who can join the leavers with Remain in the last position (but unclear how to Leave).

The 6.7% would be a relevant section of the vote. They all voted Leave, but divided on S, T and H. Potentially the outcome of the 2016 Brexit referendum has been decided by the 6.7% GB voters who have Remain neither in the first or last position, and who do not follow the standard logical order on the options.

Discussion on ConR / L and LabR / L

The division of ConR / L and LabR / L is losing its relevance because it are dwingling groups, they are changing loyalties, and their 2016 votes are becoming history while there are new issues. Yet, the 2016 referendum question was flawed, and it is relevant to see how sizeable parts of the UK electorate deal with the logical conundrum that they took part in.

The 17% of confused votes on the first preference can be found in the subdivisions in similar proportions.

33.5% of ConR voters and 61.1% of LabR voters are united on the preference R > S > T > H. Presumably this was also the case in 2016, or there must be factors that increased or reduced consistency or confusion.

28.9% of ConL and 19.4% of LabL are united on the preference H > T > S > R. Presumably this was also the case in 2016, or there must be factors that increased or reduced consistency or confusion.

One might expect that ConR / L and LabR / L voters of 2016 would have the benefit of a party preference and thus show more consistency, yet the distribution of views is quite as much, and the sandwich with multiple peaks is quite present.

The 2016 Conservative Remainers are loyal for 45.5% to the old point of view, but still vote for a Conservative party that is set on Leave. Part will be the misplaced loyalty for the flawed referendum. Alternatively, they voted for a minority in this party that still tries to bring balance ? (A good poll requires a focus group.) (And there is more in the world than just Brexit.)

The 2016 Labour Remainers are 76.1% loyal to the old point of view. Yet Labour leader Corbyn also prefers a Brexit. It might be the pecularities of the British system of District Representation (DR) that caused these voters not to switch to LibDem. (But the LibDem also have a liberal policy that many voters for Labour dislike. The system of DR doesn’t favour the entry of new political competitors.)

The 2016 Leavers have a high loyalty to the old view, ConL 86.8% and LabL 74.4%. Yet this doesn’t diminish the diversity of opinion about how to Leave (though T gets more votes than H).

Comment on uncertainty in this estimate

For n = 4, there are n! = 24 variables, (n-1)^2 = 9 independent equations within the matrix, and there is the addition constraint 1, so that the degrees of freedom are 14. Yet we cannot randomly set weights to zero. If there would be nonzero weights for single-peaked preferences only, then the YouGov ranking matrix would show zeros, which it doesn’t. Thus it takes some arbitration which weights to exclude. There are quite a lot of possibilities, and I can only hope that my choice was wisest. As said, the percentages provided by YouGov have been scaled up to the table given above, and this allows us to determine the error in the estimate. Due to degrees of freedom the calculated error is quite low. The use of an error measure is limited to comparing estimates and not something that is useful to mention here. As said, YouGov have the proper data, and it must be hoped that they will look into this.

Conclusions

The ranking matrix is a fine way to summarize results, yet the preference ordering are more accurate on the underlying and relevant orders. This is merely a matter of presentation by the statistical reporter. A person in a poll who can answer on a ranking matrix in fact gives the personal preference ordering. The statistician can compound these data while not losing information on the permutations. From the permutations it is always possible to create a ranking matrix, yet the reverse requires estimation techniques which generate needless uncertainty.

Asking for voter preference orderings in a poll is a useful exercise. It is not intended to propose this for general elections. For general elections it suffices that voters exercise a single vote for a party of choice. The condition however is Proportional Representation, otherwise there are serious distortions, see the earlier discussion on this weblog.

The information on the rankings and implied preference orderings suggest a rather large state of confusion in the electorate of Great Britain. The notion of single-peakedness appears to be quite useful in highlighting the issue of the preference order.

Appendix July 15 2017

I slightly revised the manner of rounding and included case 16 for all columns. The polished up excel workbook is: 2017-07-15-YouGov-Rankings

Before the UK Brexit referendum of June 23 2016 I warned that referenda can be silly and dangerous, see here in April 2016. I clarified that the Brexit referendum question was flawed in design. I did not look deeper into this, since, like so many others, I had been put on the wrong foot by the 2016 poll average that suggested a continuation of the status quo. After the surprise outcome, I advised the youngsters in the UK to focus attention on this design flaw, as this is the clearest issue and proper reason to argue that the outcome should be annulled, see here in June 2016. When Kenneth Arrow passed away in early 2017, this was an occasion to have a summary text published in the RES Newsletter, which was republished on the LSE Brexit blog. Now, with the 2017 UK general election, I have been looking a bit deeper at these UK election issues.

One result has been the use of the Lorenz curve and Gini coefficient to show the disproportionality in the UK between votes and seats. Almost all EU members have Proportional Representation (PR) with clear exceptions of the UK and France that have District Representation (DR). Apparently, this is a main reason for the influence of populism in the latter two countries. DR allows that politicians are elected with a minority of the vote, which causes a gap with the majority. Politicians like David Cameron can use a referendum to introduce an element of proportionality. Yet referendum questions are quickly flawed.

A main confusion

Another surprise for me was the existence of the Re-Leavers who make up some 23% of the electorate, and who are very likely also a major section in the House of Commons that supported the invoking of article 50.

Apparently many British voters are awfully respectful of democracy, and while they voted for Remain, they accept the referendum outcome, and let their voting behaviour now be guided by Leave. In other words: they no longer operate as voters who are supposed to express their first preference, but they operate as politicians who develop policy using such preferences.

Voters are better not confused about the following angles:

It is one thing to accept the Brexit referendum outcome as a fact. Please accept facts.

It is another thing to discuss the consequences of that fact.

There is always the distinction between your first preference and dealing with new developments.

Your first preference can change, but rather only because of arguments, and not just because of a majority view.

For me, it is easy to say this, in a country that is used to PR. In the UK case of DR it may well be that strategic voting requires voters to run with with herd. Nevertheless, the Re-Leavers cause quite a confusion in the voting record. Also for the general elections of 2017 we now can observe that we don’t know what people really want.

The YouGov data of June 12th – 13th 2017

Anthony Wells provided and discussed these data that show the impact of the Re-Leavers. Let me quote the main part, and for this quotation I also moved their copyright sign up.

These early June data are most relevant for judging the June 8 2017 UK general election. Apparently 26% of all adults in Great Britain (UK excl. Northern Ireland), but also 53% of the voters who voted Remain in 2016, reason as follows:

I did not support Britain leaving the EU, but now the British people have voted to leave the government has a duty to carry out their wishes and leave.

I consider this an illogical and rather undemocratic statement.

Logic would require the annulment of the referendum outcome, and not to take it seriously.

In representative government, it is Parliament that determines policy, not the people by some referendum.

Most of the EU has PR and thus the notion of representative government. The 2016 Remain voters want to remain in the EU, but, 53% apparently also reject the EU notion of representative government, and instead they appeal to the populism of referenda.

More on the design flaw of the Brexit referendum question

A few days ago, I rephrased one aspect as follows: With R for Remain, S for Soft (EEA), T for some Tariffs, and N for No Deal (WTO), there are 6 possible strict preferences for a deal, from R > S > T (Theresa May before the referendum) to T > S > R (Theresa May after the referendum). If S and T are collected in L (Leave) then there arises the claimed binary choice between R and L. Voters who are in the categories S > R > T or T > R > S would face a hard question. If they expect that R might win, but also that their own preferred option might not win, should they still go out and vote ? They might decide not to turn out, or develop assumptions about what L actually might become, given what what they think about future developments. Similarly for the versions of R. See the voting theory about single peaked preferences (and these are not single peaked but double peaked). Overall it is a fallacy that there is a binary choice. Lawyers can argue that one either invokes article 50 or doesn’t invoke it, yet the referendum isn’t such a legal case, for it is an issue of policy preferences.

In fact, above YouGov poll provides us a bit more information on this issue. Look at this section on their page 15:

Look at the column of the total (with 1651 people in the weighted sample). 35% are clearly for Remain, in their first rank. 47% are clearly against Remain, in their last rank. Thus the middle 8 + 9 ≈ 18% (rounding error) is rather confused, for they put Remain between one of the Leave options. How would they have to vote at a referendum that only allows R or L ? We find similar percentages for the subgroups on the right hand side.

Conclusion

The discussion in the UK would be served by greater awareness of these distinctions:

The difference between voting for your first preference (setting the target) and trying to second-guess politicians (as if you are in the driver seat).

A valid question and an invalid or flawed one, like the Brexit referendum question.

The crucial differences between Proportional Representation (PR) and District Representation (DR), linked to the distinction between representative democracy (mostly PR) and populism (mostly DR).

There is also something not discussed in the above, but that is the difference between the failing Trias Politica and improved democracy with an Economic Supreme Court.