Rosy Outlook for AI Chip Startups

Article By : Rick Merritt

SAN JOSE, Calif. — Graphcore showed a system that it has co-developed with Dell, and rival Habana snagged $75 million in a funding round led by Intel Capital. The deals reinforce a report earlier this year that the first AI chip startups are now in production with silicon that looks promising.

At a Dell event last week, Graphcore showed a system that the companies co-designed sporting 16 of its Colossus chips across eight PCI Express Gen 4 cards. As an investor in Graphcore, Dell was among the companies to receive cards from the startup’s first production run, which it said in a blog post is sold out until early next year.

Graphcore claimed that the system delivers more than two petaflops performance spread across more than 100,000 programs running in parallel. The chips use a processor-in-memory architecture, packing 1,216 tiles made up of a proprietary core and a slice of the chip’s total 300-Mbytes SRAM. By contrast, Nvidia’s V100 uses expensive external HBM memory stacks.

For its part, Habana said that its $75 million round brings its total funding to $120 million. The round will fund its work on 7-nm training and inference chips.

In September, Habana announced that it was shipping PCIe cards with its 16-nm Goya inference processor, claiming that it beat Nvidia’s V100 by processing 15,000 ResNet-50 images/second with 1.3-ms latency at a batch size of 10 while running at 100 W. It plans to ship before June a 16-nm Gaudi training processor.

Graphcore gave a glimpse of a system that it co-designed with Dell at a Dell event in Chicago. (Source: Graphcore)

More than 50 silicon startups are building accelerators for deep learning.

“It’s not clear who will be the winners or losers, but the industry won’t accommodate that many new ASIC suppliers, so some will get acquired or go away,” said Robert Hormuth, a chief technologist for Dell’s server group, in an interview with EE Times earlier this week.

Although current systems plug accelerators into PCI Express slots, future designs aim to put the chips on a processor bus, sharing main memory, he said, adding that while the Gen-Z protocol could enable such designs, it will probably take more than two years to finish work on the silicon and systems.

Nvidia’s disappointing revenue forecast led one Wall Street analyst to conclude that the company may be seeing more challenges than a fall-off in crypto-mining demand for its midrange Pascal chips. “A revenue miss of that magnitude can bury a lot of things,” wrote Romit Shah of Nomura in a research report.