Last week I filled in a survey about what the NZ Labour Party should do with itself. Half-facetiously I wrote something like: “Do it like the right does it. Lie about your true beliefs to appeal to ‘middle New Zealand’, win power, then underhandedly push through a radical left-wing agenda to end poverty and inequality.’

Guys, this is not what I meant. Thanks for listening but, to employ an ancient saying of my culture, UR DOING IT RONG. And thank you Keith Ng , for so forcefully explaining many of the reasons why it is wrong, in the manner of a genteel professor tearing your face off and shoving it down your throat in a white-hot stats rage.

To start with, like Keith I found it hard to separate my stats rage from my race rage.[1] Amid the clusterfuck of Chinese-house-craziness today, American real estate bots have started following me on Twitter. Not kidding.

Phil Twyford and the Herald deserved all they got from the Ng Army. Reporting or communicating statistics is one of the times when it is entirely right to shoot the messenger; to shoot him and stomp on his pitiful corpse. The messenger is the one who does all the harm. But as for the study itself? Let’s get to defending its very existence in a moment, but the methodology and conclusions?

I don’t have any problem with the suggestion that foreign PRC-based investors are buying loads of houses in New Zealand (as are foreign white people but no-one cares about them, because the narrative is about predation from China, not our own stupid lack of regulation).

And even though it’s easy to mock, I also think it’s quite likely that whatever Bayesian analysis the Labour Party used for the ‘Chinese-sounding names’ was a reasonable enough proxy for people of Chinese ethnicity. (Personally, I’d like to get Rob Salmond’s estimate of the probability of the unfortunate ‘Lena Mok’ being Dutch or German.)

I would even go so far as to say that it’s pretty reasonable that when looking at the proportion of probable Chinese names in the Labour Party dataset as compared with resident Chinese population size, and controlling (or weighting) for demographic and income factors alone, that the overrepresentation of Chinese people buying houses suggests that they are not all local residents.

However there is no realistic way of even guessing the magnitude of that overrepresentation, because even with your Census-based weighting, there is no way of controlling for cultural factors among local Chinese when it comes to buying houses. This is because no-one has researched what they might be. Comparing us to Indians is not good enough, sorry.

Full disclosure: My Chinese parents have been living in Auckland for over 40 years, and in that time, have bought eight houses in their names and sold two. They fucking looooove buying houses. I think my parents caused the Auckland property bubble. If they did, it was because they bought into the ‘Kiwi Dream’ big time, like all the other idiots in this town – and because for immigrants, buying property means security and connection not foreignness and exclusion. I don’t say this as some anecdotal evidence of how it might be possible that locally resident Chinese bought all the ‘Chinese name’ houses. I say it to show you that it’s very very hard for Chinese people not to take this personally.

Man, for Chinese immigrants in the West, buying houses is almost on a par culturally with food. It’s like you’re giving us shit for eating.

With Phil Twyford lying in a bloody puddle somewhere, my stats rage is somewhat cooler. But my old friend race rage is creeping back. It doesn’t matter if 90% of those ‘Chinese sounding names’ really are PRC hot-money investors; for the rest of us under suspicion of being foreign, we know Labour was ready to throw us under the bus.

The Labour study is one of those bits of analysis that comes together because no other data is available to answer the question you really want answered; and is convenient simply because – and this is very important – Chinese people areidentifiable.

If your identification strategy for locating the cause of the housing bubble boils down to ‘What minority group can we effectively single out because of their weird names?’, this is NOT SOMETHING YOU SHOULD EVER ADMIT. Not only does it make the Labour Party sound inherently racist and not to be trusted, it makes social science sound inherently racist and not to be trusted. In a perfect world, I’d be able to love them both unconditionally, and stats rage and race rage would not be a problem.

I can’t actually remember the last time that this much effort has been put by any political party into singling out Chinese people (as opposed to ‘Asians’) for their probability of being ‘foreign’.

And if Labour put this much effort into programming an algorithm to identify us, I wonder if it also estimated how many New Zealand Chinese votes this study would cost them.

The real question is not ‘oh, ha ha this doesn’t look like a very reliable method for guessing whether someone is Chinese.’ (It probably is). It is not ‘could local Chinese possibly be buying all of these houses?’ (It is possible, but not very likely).

The real question is what did the Labour Party think it was doing taking this public. If they just fucked up, so far so familiar. If they did this on purpose for well-calculated reasons – and it works – we Chinese-sounding named people are in way more trouble in New Zealand than we ever thought we would be again.

[1] Since I stopped being the shouty Asian girl on Public Address, I moved to London, studied quantitative research methodologies, and became better at maths than Keith. So I have stats rage/race rage confusion on a daily basis.

I don’t have any problem with the suggestion that foreign PRC-based investors are buying loads of houses in New Zealand (as are foreign white people but no-one cares about them, because the narrative is about predation from China, not our own stupid lack of regulation).

It would be surprising if no there was no one trying to speculate in our housing market given given the terrible regulation and until recently trivially-avoided capital gains tax.

On the other hand, there literally aren’t any other countries where the conditions are like those in China. The government’s determination to hold down interest rates has made bank deposits and bonds relatively unattractive and credit so cheap that huge numbers of retail traders borrow money to buy shares in the two stock markets (and, incredibly, the government responded to to the recent slump by pumping in money to make it easier for them to borrow). There are trillions of dollars in capital and credit looking for places to go.

Just to note: I don't think Labour can blame the Herald for the framing of the story. I gather the angle was pretty much how they wanted it, despite the concerns of some wiser heads on the editorial staff.

Personally, I’d guess that all these explanations are true: that Chinese New Zealanders (on average) buy both homes and investment properties more than other New Zealanders, and that there are foreign property investors of Chinese ethnicity. But that’s a guess: these data don’t tell us — as the Herald explicitly points out.

One of the repeated points I make on StatsChat is that you need to distinguish between what you measured and what you wanted to measure. Using ‘Chinese’ as a surrogate for ‘foreign’ will capture many New Zealanders and miss out on many foreigners.

The misclassifications aren’t just unavoidable bad luck, either. If you have a measure of ‘foreign real estate ownership’ that includes my next-door neighbours and excludes James Cameron, you’re doing it wrong, and in a way that has a long and reprehensible political history.

But on top of that, if there is substantial foreign investment and if it is driving up prices, that’s only because of the artificial restrictions on the supply of Auckland houses. If Auckland could get its consent and zoning right, so that more money meant more homes, foreign investment wouldn’t be a problem for people trying to find somewhere to live. That’s a real problem, and it’s one that lies within the power of governments to solve.

I suppose the difference is that we already know what and where James Cameron is buying because the Overseas Investment Office is obliged to record and approve it and publish the details.

On the other hand, there literally aren't any other countries where the conditions are like those in China. The government's determination to hold down interest rates has made bank deposits and bonds relatively unattractive and credit so cheap that huge numbers of retail traders borrow money to buy shares in the two stock markets. There are trillions of dollars in capital and credit looking for places to go.

Actually, shadow banking in China is the only option for many when it comes to borrowing money. The scale is immense and its practices are oppressive. As for Chinese citizens being able to rely on saving by conservative instruments, it must be soul destroying watching others "get ahead." There are obvious similarities to what's happening in the West, including NZ, as we stagger from one asset bubble to the next.

I would even go so far as to say that it’s pretty reasonable that when looking at the proportion of probable Chinese names in the Labour Party dataset as compared with resident Chinese population size, and controlling (or weighting) for demographic and income factors alone, that the overrepresentation of Chinese people buying houses suggests that they are not all local residents.

Bayesian analysis is versatile and can be applied in different ways. Not sure why they could start with explaining the methodology and the assumptions used. Even the reference to 126,000 is annoying as it's obvious that the potential buyers / investors are a small sub-group of this population. Qualitatively, it would also be interesting to understand if this is part of a mass capital flight movement out of China, which eventually could pose far more grave problems for China and the global economy.

What a massive debate to kick off, can't believe it slipped under my radar. I obviously saw it on the Herald, and pretty much immediately had a number of thoughts:

1. The specific choice of Chinese sounding names would be likely to provoke outrage for its racial profiling approach2. Notwithstanding that, the actual numbers could be valid, and the issue of foreign investment driving up NZ prices is important enough to begin the debate, as one of the many prongs of our insane house price history, at least in Auckland.3. Notwithstanding that, the analysis can and should have been extended to all foreign ownership, but that analysis is probably more difficult, since the bulk of the NZ population have "European sounding" names already. It could still be done, though.4. That this wasn't also done made me think that Labour was probably quite deliberately looking to play a racism dogwhistle card.5. This card play might actually work, much as it did for Don Brash.

Point 3 is vital to me. The issue of hot foreign money isn't solely a Chinese one. That's so far from the whole story that it's not funny. FFS this is a mostly white nation bought and stolen from the natives in the first place. The prices in NZ are only as high as they are because of all that Euro money and immigration that literally made it what it is, whether the Maori wanted it or not. So far as I can tell this is still happening.

As a nation that is meant to be non-racist, the issue should be approached in a racially neutral way. That doesn't mean that profiling shouldn't be used in this analysis, it means that it should have been done to more groups than just the Chinese. There is still a chance to rectify this. It will tell us a lot about just what tactic Labour is playing here, if they do actually try to rectify it or not. I sure hope they do, but would not be surprised if they leave it lying. But I'm not across the magnitude of the analysis I'm suggesting here. Maybe the "Asian sounding" example was particularly amenable to this analysis. Half of our problem here is the sheer paucity of our public record keeping on the true dynamics of our property market. How is it that we don't keep records of the origin of the money used in our property market? It's pretty damned important. Without real data we're struggling in the dark and forced to use data-mining techniques like this, which have nowhere near the statistical strength of properly designed surveying. But they are, at least, something more than people shouting.

I hope Rob Salmond might be able to shed some light on what he was actually tasked to do here, although I can imagine that might be very difficult, on account of possible confidentiality issues with his client.

Hell yes. It's quite literally transformed statistics in the last 20 years. When I first studied stats in the early 90s it was a promising research direction. When I just finished my undergrad degree in stats early this year it was clear to me that's it's more than come of age, it's likely to take over completely in our lifetimes. It is extremely powerful at answering the questions we want answered, unlike frequentist methods of the past which were very good at answering questions that we could hardly even understand. The number of times you have to explain what a p-value is, even to people with considerable mathematical competence, just shows how poor it is for what we really want stats to do.

It panned out that way mostly because decent Bayesian analysis required computational power that just didn't exist in the early 20th century, when frequentism basically took over the mainstream. The tractability of the frequentist algorithms was (and still is) a big part of their appeal. They'll stay there for a while, I'm sure, but the power of Bayesian stats to generate answers in a form that even laypeople can understand is a great power indeed.

The statistical explanation is the rejection of the null hypothesis - that nothing unusual is going on. The alternative hypothesis, foreign investment, is not proven but the correlation of local Indians to local Chinese ethnicities makes the foreign investment explanation more likely.

I imagine they used Chinese names because they were more strongly distinguished in their model. That's fine. I agree that the political decision to link that strongly with 'foreign investment' is another thing altogether.

The number of times you have to explain what a p-value is, even to people with considerable mathematical competence, just shows how poor it is for what we really want stats to do.

I would argue that the p-values and significance should be understood by pretty much anyone involved in research. I've been involved in commercial research, and as a non-natural mathematician, it requires dedication and patience. I constantly review what most learnt in applied maths at high school, but never retained in their adult lives.

All I can see the researcher do is apply a probabilistic inference from population, Whether or not that qualifies as "Bayesian analysis" is beside the point. It wasn't very sophisticated. The readers of the Herald are likely to be much of what they read at face value so "it must be true."

When the Hong Kong government moved to implement disincentives for foreign buyers, they had identified their problem as predominantly increased demand from mainland China;

CLSA analyst Nicole Wong said the latest moves will discourage mainland Chinese property buyers in the city, "hence removing a very meaningful portion of demand." The brokerage firm estimates that such buyers accounted for about 37% of the total value of newly built apartments sold in Hong Kong in the second quarter.

I imagine they used Chinese names because they were more strongly distinguished in their model.

Perhaps. Or more likely because it was the phenomenon that was under investigation: What is the level of non-local-resident Asian investment in NZ? It's something you'd want to know. I'd be surprised if the data didn't also give the euro figures, but the question of local vs foreign still remains unknown.

Also, it could be because there is underlying truth in what is being investigated, that there is unusually high investment coming from that direction. That tallies with endless anecdotal evidence, and with the fact that China is a very populous superpower with hugely increasing wealth over the last couple of decades, whereas European sources have not really changed their prosperity levels much.

It's not really a surprising finding at all, that foreign Asian investment is running high. I do think it's good to at least attempt to get real data about it. We should actually be collecting official data on it. But it's only an interesting feature of general foreign investment in NZ. There's something that whiffs strongly of racial dogwhistling in taking such exception to the sources being from Asia. My question isn't "Is this the case?", but more "So what if this is the case?". Why is it any more cause for concern than, say, Brits coming down here and buying up property? Or white South Africans?

Sure, the level of public ignorance about statistics is much deeper than what scientists have. But I'm suggesting that they're even more ignorant about the older stuff because it is actually much harder to understand.

People think they're getting the probability of the null hypothesis, given the data, something they can understand, sort of. Instead, they get the probability of the data, given the null hypothesis (and a bunch of other assumptions that have to be checked). That's not the same thing, but it's frequently used in a similar way because that is the question asked.

The other way round is quite a strange way of answering the question. Frequentists basically refuse to answer the actual question people want answered, the probability of the null hypothesis given the data (and any other priors). They do so for reasons that kind of make some sense, so long as you buy into the understanding of what science is from about the turn of the 20th century.

At that time the idea of the underlying prior was not well understood and the thinking was that there should be no such thing, that to make prior assumptions is unscientific. It took the rise of various cognitive sciences in the 20th century to really show just how untenable it is for us to say we don't make prior assumptions. Practically all of our growth has come from overturning them, when we realize what they even are. So it's much better to have a framework in which the prior is not seen as a horror to avoid, but an inevitability to be modelled.

Why is it any more cause for concern than, say, Brits coming down here and buying up property? Or white South Africans?

It isn’t. Whether the majority of the foreign funding is of mainland China origin isn’t any more of a concern to me – all QE/ZIRP sourced funding that invests disproportionately in residential housing thereby locking out NZers based on an income to average/market price ratio is of equal threat to NZ – as NZers (of all ethnicity!) borrow more locally (and at higher interest rates) to compete;

People think they're getting the probability of the null hypothesis, given the data, something they can understand, sort of. Instead, they get the probability of the data, given the null hypothesis (and a bunch of other assumptions that have to be checked).

Rejecting a null hypothesis following a standard research design is all very well, but this is not rejecting or supporting a hypothesis given that its directional at best and not statistically significant.

Rejecting a null hypothesis following a standard research design is all very well, but this is not rejecting or supporting a hypothesis given that its directional at best and not statistically significant

Just to note: I don't think Labour can blame the Herald for the framing of the story.

No they fucking can't -- if the shocked pearl-clutching I've seen from certain quarters over The Herald doing what The Herald does with monotonous regularity is at all sincere, well congratulations. Whoever is doing Labour's comms strategy nowadays is just incompetent rather than a malignant race troll. Come to your own conclusions on whether that's a good thing.