The problem with personalization

The challenge of personalizing the news is the heterogeneity of our personal interests and the weakness of the signal we expose to the recommendation engine.

As an example, below are four stories from around the country I was interested in today, for reasons ranging from ‘possibly obvious’ to ‘unknowable’ by an algorithm. And by ‘unknowable’ I mean how likely would a machine be to affirmatively pick any of these four at a very low signal to noise ratio.

The dot-Boston domain is now openWhy am I interested? I was working at the Globe when we originally bid on and won the rights to sell this TLD. (I proposed renaming Boston.com to com.Boston. Just because.)Could a machine have affirmatively predicted my interest? No.

NBC moves 130 Premier League games to streaming serviceWhy am I interested? I am a big fan of the Premier League and watch games on NBC – but found this by random chance at our paper in Sacramento. Could a machine have affirmatively predicted my interest? Yes, I probably leave a wide paper trail on this topic.

So what’s the ‘problem’ here? Personalization depends mostly on observed web behaviors. Much of our interest in the news is based on real life experiences and events. So to provide me a list of recommended stories you need to know not what I clicked on last week, but where I lived in 1998 and my level of interest in urban planning issues.

So even though Facebook knows ostensibly everything about me, and Twitter is packed with people I know/trust and rely on for news – neither of those platforms or any other app I am aware of is going to identify those four stories on a given day and surface them in a unified newsfeed.

In fact I will will be suitably impressed if a machine will ever be able to perform at that level of serendipity. But, if you invent it, I would pay for that convenience as a service.