Newsletter 61 – Deep Learning Appraisal

Deep Learning Appraisal

Gary Marcus has long been a Deep Learning skeptic. His recently published paper ‘Deep Learning – A Critical Appraisal‘ is essential reading. In the paper he notes the remarkable progress of Deep Learning approaches over the last few years but also suggests that we are approaching the domain of diminishing returns for current methods which rely on access to huge quantities of training data, are not transparent to understand, are difficult to engineer and do not transfer very well outside of narrow context of use. He goes on to present ‘ten concerns for deep learning, and suggest that deep learning must be supplemented by other techniques if we are to reach artificial general intelligence‘

A similar sentiment is conveyed by Paul Ford in this recent Bloomberg article in which he outlined his attempts to get an AI to produce something that resembled his own writing.

I set out to learn enough about machine learning that I could feed a neural network everything I’ve ever written and have it write an article, or even just a paragraph, that sounded like me

It’s an end goal I have often thought about and Ford’s experience suggests we have a way to go yet before an AI could write this blog without any human legwork although glimpses of what it might look like in the future are already here notably in the form of lobe.ai previewed here a few weeks ago:

Machine learning is important, but it’s not ready for civilians (although check out lobe.ai to see how things might look in the future). As with all software, machine-learning tools still need people to come along to make them look good and teach them how to behave. At least for now, computers need people as much as we need them.

Ford was the celebrate ‘man in the taupe blazer’ in an epic 2015 Bloomberg publication called simply What is Code? also covered in this blog three years ago.

Artificial Intelligence

there’s a massive untapped market waiting to be unlocked with the right technology. We need something that works on cheap microcontrollers, that uses very little energy, that relies on compute not radio, and that can turn all our wasted sensor data into something useful. This is the gap that machine learning, and specifically deep learning, fills.

Management consulting has already been beset on all sides by competitors and new technologies. They have maintained status and growth through prestige, branding, and long-time client relationships, but they’re ultimately no more immune to the forces of disruption than any other industry. … The increasing pace of technological change means that more and more, consultants’ recommendations are out of date nearly as soon as they’re made.

Microsoft and GitHub

Rumours about Microsoft acquiring GitHub turned out to be substantiated in early June with a deal valuing the popular developer code repository at $7.5bn. The deal further cements Microsoft’s growing open source developer credentials building off the hugely successful release of VS Code:

Microsoft now has more than 1,000 employees actively pushing code to GitHub repositories. Its popularity among developers could see Microsoft earn some much-needed trust and respect from developers. In bigger enterprises and slower moving businesses, the fact Microsoft has acquired GitHub will make it more trusted to use for projects and source control, simply because Microsoft is already trusted across many software and services by these companies. … The Microsoft old isn’t the Microsoft of new, and this GitHub acquisition is a chance for Microsoft to prove that even further.

HBR underscored that the apparently inflated price of the acquisition was all about securing ‘strategic value’:

Microsoft is not paying $7.5 billion for GitHub for its ability to make money (its financial value). It’s paying for the access it gets to the legions of developers who use GitHub’s code repository products on a daily basis (the company’s strategic value) — so they can be guided into the Microsoft developer environment, where the real money is made.

Not everyone was so enamoured. The integration of Clippy in this imagined view of a future GitHub frontend rewrite to align with Microsoft branding was particularly droll:

Here’s hoping that next year, one of the world’s most celebrated CEOs can describe how he and his company are reinventing normal for the hundreds of thousands of workers who toil in its warehouses. That would be a letter to shareholders the whole world would be eager to read.

for Amazon and its allies to reverse an already watered-down tax—the original plan was $500 per head—in an ostensibly progressive metropolis shows just how much weight big businesses have, and how easily they can throw it around anywhere they want.

Venture capitalists, such as Albert Wenger of Union Square Ventures, who was an early investor in Twitter, now talk of a “kill-zone” around the giants. Once a young firm enters, it can be extremely difficult to survive. Tech giants try to squash startups by copying them, or they pay to scoop them up early to eliminate a threat.

Engineering Management

Today successful companies start with the customer. They recognize that customers spend their time across many channels, and wherever those customers are, that’s where they should be meeting their customers’ needs. And the more information you can learn about the customer, the better you can serve their needs, and the more valuable the relationship becomes. That’s digital transformation: from linear transactional channels to a circular, dynamic relationship with your subscriber.

In total, I spent 143 hours over the course of 6 weeks on this process, and this whole endeavour was quite something.

On expenses:

Google discovered that giving every employee a credit card and trusting them to follow policy on expenses, even with some bad actors, was cheaper than the enforcement and structures most companies use.

How much do you waste due to low trust? Can you do the math for your company?

Science

Bitcoin energy consumption to support its “proof of work” algorithm is unsustainable. This resource provides evidence for why:

The machines performing the “work” are consuming huge amounts of energy while doing so. The Bitcoin Energy Consumption Index was created to provide insight into this amount, and raise awareness on the unsustainability of the proof-of-work algorithm.

For Aristotle insisted that happiness is constituted by something greater from and different to an accumulation of agreeable experiences. To be happy, we need to sustain constructive activities that we believe are goal-directed. This requires conscious analysis of our goals and conduct, and practising ‘virtue ethics’, by ‘living well’. It requires being nurtured effectively to develop your intellectual and physical capacities, and identify your potential (Aristotle had strong views on education), and also training yourself to be the best possible version of yourself until you do the right thing habitually, on autopilot. If you deliberately respond in a friendly way to everyone you encounter, you will begin to do so unconsciously, making yourself and others happier.

The phenomenon reportedly begins when we hit the age of 30 and six months, which means that it’s a very real fear for anyone at the tail end of their twenties.

When quizzed on the reason for stopping in the pursuit of new music, survey participants reportedly offered a multitude of excuses – including having kids and quite simply being a bit swamped by the amount of new music on offer.

But it gets even worse, with 60 percent of people admitting that they would only usually listen to artists they already know.

NYMag’s exposé of Vice Media and the gonzo antics of its founder Shane Smith is a must-read. Neither come out of it well:

A senior manager once joked that the company’s hiring strategy had a “22 Rule”: “Hire 22-year-olds, pay them $22,000, and work them 22 hours a day.”

As a foreign correspondent I covered collapsed societies, including the former Yugoslavia. It is impossible for any doomed population to grasp how fragile the decayed financial, social and political system is on the eve of implosion. All the harbingers of collapse are visible: crumbling infrastructure; chronic underemployment and unemployment; the indiscriminate use of lethal force by police; political paralysis and stagnation; an economy built on the scaffolding of debt; nihilistic mass shootings in schools, universities, workplaces, malls, concert venues and movie theaters; opioid overdoses that kill some 64,000 people a year; an epidemic of suicides; unsustainable military expansion; gambling as a desperate tool of economic development and government revenue; the capture of power by a tiny, corrupt clique; censorship; the physical diminishing of public institutions ranging from schools and libraries to courts and medical facilities; the incessant bombardment by electronic hallucinations to divert us from the depressing sight that has become America and keep us trapped in illusions. We suffer the usual pathologies of impending death. I would be happy to be wrong. But I have seen this before. I know the warning signs. All I can say is get ready.