Cerebras Systems said today that the company has achieved the computational equivalent of the human brain, or the equivalent of 100 trillion synapses.
Cerebras manufactures what it calls the Wafer Scale Engine-1 and -2, a massive 46,225 sq. mm chip more like the size of a computer keyboard than the CPU that goes inside of your PC. The company essentially mounts that chip inside of a standalone CS-2 system, about the size of a dorm refrigerator. Now, the company says that it’s been able to surround the CS-2 with several different technologies to enable the brain-scale computational power to reach 120 trillion synapse equivalents, also called parameters.
Cerebras isn’t alone in trying to model machine learning at the chip level, in an effort to duplicate how the human brain works. In 2020, Intel clustered 768 of its Loihi “brain chips” together to form a ”Pohoiki Springs” cluster with the approximate power of a mole rat. In 2019, the goal for Loihi was to enable 100 billion synapses, about the computational power of a mouse brain.
The Cerebras Wafer Scale Engine Gen 2 chip is several orders of magnitude above that, as is the CS-2 system in which it is housed. While the way in which Cerebras said it achieved this level of complexity involves a number of different technologies, it essentially “disaggregates” or separates the storage and memory from the chip’s computation engine, storing the models off-chip and streaming them in. Cerebras also designed a way to provide up to 2.4 petabytes of off-chip memory for calculations, as well.
One of the ways in which the Cerebras technology will be used is in Natural Language Processing (NLP), the way in which machines interpret human speech, act upon it, and communicate the results. Essentially, what the CS2 system does is to dramatically increase the number of parameters an AI model has access to, making the AI’s understanding even more sophisticated.
“Cerebras’ inventions, which will provide a 100x increase in parameter capacity, may have the potential to transform the industry,” said Rick Stevens, associate director of the Argonne National Laboratory, in a statement. “For the first time we will be able to explore brain-sized models, opening up vast new avenues of research and insight.”