Memory integration remains key to AI

TORONTO — Big data applications have already driven the need for architectures that put memory closer to compute resources, but artificial intelligence (AI) and machine learning are further demonstrating how hardware and hardware architectures play a critical role in successful deployments. A key question, however, is where the memory is going to reside.

Research commissioned by Micron Technology found that 89% of respondents say it is important or critical that compute and memory are architecturally close together. The survey, carried out by Forrester Research, also found that memory and storage are the most commonly cited concerns regarding hardware constraints limiting AI and machine learning today. More than 75% of respondents recognize a need to upgrade or re-architect their memory and storage to limit architectural constraints.

AI compounds the challenges already unearthed by big data and analytics requirements because machine learning does multiple accumulation operations on a vast matrix of data over neural networks. These operations are repeated over and over as more results come in to produce an algorithm that is the best path and the best choice each time — it learns from working on the data.

Because there’s so much data, said Colm Lysaght, Micron’s vice president of corporate strategy, a common solution for getting the necessary working memory is simply to add more DRAM. This is shifting the performance bottleneck from raw compute to where the data is. “Memory and storage is where the data is living,” he said. “We have to get it to a CPU and then back again, over and over again, as these vast data sets are being worked on.”

Finding ways to bring compute and memory closer together means saving power because data isn’t being shuttled around as much, Lysaght said. “It’s increasing performance because more things can happen right where they need to happen.”

Micron sees existing memory and storage technologies such as DRAM and 3D NAND SSDs providing the hardware for AI architectures but is also researching newer technologies, such as processor-on-memory architectures, while supporting pioneering startups.

There is a number of different approaches to creating better architectures, Lysaght said. One example is neuromorphic processors that use a neural network internally and break up the internal number of cores into a larger number of smaller cores. “Because there is a large matrix of data that’s being worked on, having many more cores doing relatively simple operations over and over again is a better solution,”Lysaght said.