How AI Will Invade Every Corner of Wall Street

The artificial intelligence was unleashed by Winton, the London hedge fund, to test an old principle of the Berkshire Hathaway Inc. chairman with a view to trading on it: that major acquisitions usually hurt the buyers’ shareholders. Researchers collected and analyzed data on almost 9,000 U.S. deals back to the 1960s.

“It prevented us from trading on a false signal and potentially losing money,” said Daniel Mitchell, who runs a team of data scientists at the $30 billion hedge fund. Buffett didn’t respond to a request for comment sent to an assistant.

Bit by bit, AI is laying a claim to the future of investing after many false dawns going back decades. Giant money managers like Two Sigma and Goldman Sachs Group Inc. and smaller players like Schonfeld Strategic Advisors have adopted it as a cornerstone strategy or research tool.

From this foothold, how far will AI go?

Man Group Plc’s Luke Ellis sees a slow takeover coming. The $103.5 billion firm in London already devotes about $13 billion to several hedge funds using machine learning. In 10 years, it will play a role in everything Man does, from executing trades to helping pick securities at the firm’s discretionary unit, Ellis, the chief executive officer, said in an interview.

“If computing power and data generation keep growing at the current rate, then machine learning could be involved in 99 percent of investment management in 25 years,” Ellis said. “It will become ubiquitous in our lives. I don’t think that machine learning is the answer to everything we do. It just can make us better at a lot of things that we do.”

The human toll could be severe: 90,000 jobs in asset management, including fund managers, analysts and back-office staff, out of 300,000 worldwide will go poof by 2025 because of AI, according to estimates by consultancy Opimas from a survey of financial firms.

Quant pioneers like Man Group and Winton have a head start in their AI revamp. The obstacles are daunting for almost everyone else.

There’s a paucity of scientists who can create profitable strategies. The wizardry is hard for investors to grasp, keeping some on the sidelines. And the high costs of the technology and data are a burden to firms already suffering fee pressure from the flow of assets to passive funds.

But machine learning’s prowess in finding investing opportunities beyond the reach of humans makes the technology too alluring to ignore. Firms now use AI to prep reams of messy social media and smartphone data, forecast company earnings and sales faster than analysts, decipher the sentiment of executives from documents and create entire strategies.

“Machines will be doing more of the grunt work of discovering opportunities,” said Vasant Dhar, who 20 years ago founded one of the first machine-learning hedge funds, the $350 million Adaptive Quant Trading program at SCT Capital Management. “They can generate hypotheses, test them, and then tell humans, ‘This is interesting, go dig deeper.’ As machines add more value, it changes the nature of work humans do.”

AI strategies also have to wrestle with the assault from passive investing as BlackRock Inc. and Vanguard Group gobble up assets on their way to potentially managing $20 trillion. Index and smart-beta funds threaten to arbitrage away AI’s edge in picking value or growth stocks. But machine learning is showing it can get ahead of the passive wave and exploit patterns in markets that haven’t been discovered, almost becoming a superior version of smart beta.

Investors, fed up with years of lackluster performance by discretionary firms, are buying in. Assets in quant funds, many of which use AI, have surged by 86 percent to $940 billion since 2010. Last year, when fundamental hedge funds suffered $83 billion in outflows, quants took in $13 billion, according to Hedge Fund Research. The trend continued this year through September.

For all of AI’s power with data, its limitations are just as profound. AI lacks imagination, or the human ability to anticipate events — from political to macroeconomic — if such occurrences haven’t happened in the same way many times before. While hedge fund manager John Paulson saw the subprime mortgage meltdown coming, AI would have had no clue, because it wouldn’t have had enough relevant historical data to make comparisons and form an opinion.

“A machine would have no basis for predicting a crisis since each one is unique,” said Dhar, who’s also a professor of data science and business at NYU. “Humans are good at reasoning about things like a crisis and can sometimes predict it, but we are often wrong. Look at the predictions about interest rates over the last few years.”

Right or wrong, fund managers and their market views will play a major role in the era of AI. Fundamental analysts face a bigger threat.

Firms are sometimes paying almost $1 million in annual compensation for experienced machine learning specialists who can exploit big data. That leaves less money for analysts who research company fundamentals. They may have to learn to code to save their jobs.

“As active managers are forced to spend more money on engineers as their revenues fall, they are going to be forced to slash spending on human equity analysts to protect margins,” said Martin Taylor, who shut down his discretionary hedge fund Nevsky Capital last year in the face of competition from quants. “It’s very depressing for humans.”

Quant firm Acadian Asset Management, where assets soared 79 percent to $93 billion in the past five years, offers a clue to how roles may change in the future.

Managers’ intuition about economic trends are the foundation of Acadian’s long-short and other strategies. Quants then deploy machine learning to refine and improve the 20 most influential factors, from cash flow to unusual events like fraud, that fuel those economic themes to make better predictions. The factors are then plugged into an automated system that takes positions on about 10,000 different stocks across several months or quarters.

Acadian managers and analysts are polymaths: They all have a sophisticated understanding of statistics, and almost everyone writes code and has market experience, said Ryan Stever, director of quantitative global macro research.

The Boston-based firm is investing in AI and big data to better forecast metrics, such as sales, that are key to a company’s performance. If Acadian could wager on sales data before it’s publicly released, the firm would gain an edge.

“You could use machine learning to get the metric earlier, faster and more accurately,” said Wes Chan, director of stock selection research. “If it works, that’s pretty significant.”

An even bigger ambition for some firms is mastery of deep learning, a smarter AI that powers Google’s search and Tesla Inc.’s self-driving cars. Deep learning machines, which loosely mimic activity in the multiple layers of neurons in our brains, require fewer instructions from humans. They make discoveries without being told what to find.

“You will see neural networks become better predictors and better tools for all kinds of trades,” said Juergen Schmidhuber, who helped lay the groundwork for modern AI systems and is a consultant to hedge funds. “Many trades will be executed by self-learning algorithms, with a few high-level guys occasionally injecting human decisions. That’s near-term future.”

Ultimately, the future of AI will depend on its ability to make money. Today’s small group of fully automated AI strategies are off to a middling start. Their performance beats the broader hedge fund industry but not the stock market. Thirteen AI funds gained an average of 10.6 percent annually in six years through 2016, and rose 8.5 percent through October, according to an Eurekahedge index.

The same is true for old-school stock pickers, who will always have a job as long as they produce healthy returns for investors.

AI may have toppled one of Buffett’s pillars. But with Berkshire returning 12.5 percent annually from 2011 through 2016, machines have yet to beat the legendary investor.