I agree with that statement, but I found the following source (from evolutionary biology) framed the issues around Brexit in a new way for me. I'm going to simplify my argument somewhat by assuming two groups and characterising them as follows:

Brexiteers: Mainly white british, rural or Northern, down on their economic luck, anti-immigration.

Remainers: Cosmopolitan, urban or from London, not necessarily rich but doing ok, comfortable with the idea of immigration.

I'm ignoring the public school free market liberals who actively want to return to some imagined glory days of swashbuckling empire for my purposes.

With that in mind I'd like to present the following two sources with a huge pinch of salt

Monkeys

RECIPROCAL ALTRUISM

Kin altruism exists because it promotes the survival of one's relatives; but not all altruistic acts help relatives. Monkeys spend a lot of their time grooming each other, removing parasites from those awkward places a monkey cannot itself reach. Monkeys grooming each other are not always related. Here reciprocal altruism offers an explanation: you scratch
my back and I'll scratch yours.
...

Imagine that a species is divided into several isolated groups -- perhaps they are monkeys whose terrain is divided by rivers which, except in rare droughts, are too swift to cross. Now suppose that reciprocal altruism somehow appears
from time to time in each of these groups. Let us say
that one monkey grooms another monkey, searching for disease-carrying parasites; when it has finished it presents its own back to be groomed. If the genes that make this behavior probable are rare mutations, in most cases the altruistic monkey would find its kindness unrewarded; the groomed monkey would simply move away. Grooming strangers would therefore bring no advantage, and since it leads the monkey to spend its time helping strangers instead of looking after itself, in time this behavior would be eliminated. This elimination may not be good for the group as a whole, but as we have seen, within the group it is individual rather than group selection that dominates.
Now suppose that in one of these isolated groups it just
happens that a lot of monkeys have genes leading them to initiate grooming exchanges. (In a small, closely related group, kin altruism might bring this about.) Then, as we have seen, those who reciprocate could be better off than those who do not. They will groom and be groomed, remaining healthy
while other members of the group succumb to the parasites.
Thus in this particular isolated group, possessing the genes for reciprocal grooming will be a distinct advantage. In time, all the group would have them.
There is one final step. The reciprocal grooming group now has an advantage, as a group, over other groups who do not have any way of ridding themselves of parasites. If the parasites get really bad, the other groups may become extinct, and one dry summer the pressure of population growth in the reciprocal grooming group will push some of its members across the rivers into the territories formerly occupied by the other groups. In this way group selection could have a limited role-limited because the required conditions would not often occur-in the spread of reciprocal altruism.
If we are prepared to allow group selection a role in the inception of reciprocal altruism, we can hardly deny that the survival of some groups rather than others can provide an evolutionary explanation for a more general tendency for altruistic behavior toward other members of a group. This is still quite distinct from the popular view of traits evolving because they help the species survive-groups are far smaller units than species, and come in and out of existence much more frequently, so group selection is more likely to be an effective counterweight to individual selection than is species selection. Nevertheless, a group would have to keep itself distinct from other groups for group altruism to work -- otherwise more egoistically inclined outsiders would work their
way into the group, taking advantage of the altruism of members of the group without offering anything in return. They would then outbreed the more altruistic members of the
group and so begin to outnumber them, until the group would cease to be more altruistic than any other group of the same species. Although this would cost it its evolutionary advantage over other groups, there would be no mechanism for stopping this. If the group altruism had been essential to the group's survival, the group would simply die out.
This suggests that group altruism would work best when coupled with a degree of hostility to outsiders, which would protect the altruism within the group from penetration and subversion from outside. Hostility to outsiders is, in fact, a very common phenomenon in social animals. Although there is a popular myth that human beings are the only animals who kill members of their own species, other species can be as unpleasant toward foreigners as we are. Many social animals, from ants through chickens to rats, will attack and often kill outsiders placed in their midst. In a series of experiments conducted on rhesus monkeys, it has been shown that introducing a strange rhesus monkey into an established group aroused much more aggression than either crowding
the monkeys or reducing their food supply. Admittedly, keeping strangers away could just be a means of protecting one's own food supply and that of one's kin; but it could also be that this behavior serves the same role as geographical isolation in protecting the altruism of the group from debasement.

I realise this is a dangerous extract in isolation - the book goes on to argue that we, as humans, are better than monkeys and that we can and should work toward expanding our inner circle so that we can all include each other. However, it did strike me that it might help to explain the mood of the Brexiteers in some small way. A perceived "threat" to our "food source" would obviously increase hostility, if your narrative suggests that you have to share your limited resources with outsiders. The economic downturn, the language of austerity (particularly from George Osborne), the D***y M**l etc. all fed into that mood.

The Metropolis

Athens was not just a city, in our sense of the term, but a polis, or city-state. All of the poleis of ancient Greece — of which there were, according to the best modern tally, 1,035, though not all were coexistent — were independent states, with their own (very active) armies and their own forms of government.
Each polis had a city center, its astu, usually walled and containing an acropolis, or “city on an extremity,” high up and thus defensible, probably the reason that the original settlement had grown around it. The astu was surrounded by extensive territories, the khora, which included farmlands, olive groves, vineyards. The more established cities — Athens, Sparta, Thebes, Corinth, Argos — also had colonial settlements, other cities that they seized and that would pay taxes to the "metropolis", the mother polis, and were required to be allies in war. Most poleis weren't large. The average number of citizens was between 133 and 800. Athens had a citizen population of about 30,000 while its total number of inhabitants was in the vicinity of 100,000, which means that only one out of three residents held the rights of citizenship. Though there were no property qualifications, as there were in Greek oligarchies, citizenship in the Athenian democracy was hard to come by. Women, children and slaves were excluded from citizenship, as in all poleis. So too were foreign residents, the metics, who were often among the richest of those living in Athens.
Athenians prided themselves on the myth that they, alone of all Hellenes, were autochthonous, literally "sprung from the earth", by which they meant that they had always occupied the same soil. Being born of an Athenian father had long been a requirement for citizenship, but in 451 BCE Pericles tightened this law, the pride in autochthony having strengthened following the the Persian Wars and Athenian imperial hegemony. Now citizenship required both father and mother to be Athenian-born, making citizenship even more exclusive and desirable, just as Athens was asserting itself throughout Hellas as the standard for what made all Hellenes great.

This extract made me think about London as the modern day Athens. For the majority of Athenians, they would have lived in smaller rural communities presumably with clearly defined social groups. Athens (the metropolis) was an order of magnitude larger with around two thirds of its inhabitants being full citizens. There are some further calculations around the fact that women and children are included in that two thirds, but it's reasonable to assume that Athenians were familiar with seeing "outsiders" (metics) in their day-to-day lives. It strikes me that this must have been an economic arrangement that worked for everybody, resulting in a feeling of optimism and reduced hostility towards those foreigners.

Anyway, I don't really have a point. I just thought those passages were interesting.