The World’s Biggest Computer Chip
Marcus Woo · Engineering · 2019
When it comes to computer chips, size matters.Smaller has always meant better.According to Moore's law, the number of transistors that can squeeze onto a silicon chip doubles every two years, enabling sleeker and more powerful devices.Now, a California-based company is upending that notion.In August 2019, the Silicon Valley, Los Altos-headquartered Cerebras introduced the largest chip ever produced.The chip, named the Wafer Scale Engine (WSE), is made out of an entire silicon wafer.Roughly the size of an Apple iPad, it measures 46 225 mm 2 , an area about 56 times bigger than the next largest chip, NVIDIA'S GV100 Volta graphics processing unit (GPU), which spans 815 mm 2 (Fig. 1).While the NVIDIA GPU contains 21.1 billion transistors, the WSE holds 1.2 trillion [1,2].''The chip is clearly an integration marvel," said Rakesh Kumar, associate professor of electrical and computer engineering at the University of Illinois at Urbana Champaign.''It is a big deal to have shown it possible to put together a working chip this large."Such a mammoth chip, according to Cerebras, is needed to meet the growing demands of artificial intelligence (AI).AI algorithms learn to do a task by first training on a huge amount of data.In particular, deep learning algorithms, which use neural networks that roughly mimic how the brain works, require enormous computing power, with training runs that can take hours or even days.According to a recent analysis from OpenAI, a San Francisco-based, AIfocused company backed by Microsoft, the computing power demanded by AI training has, from 2012 to 2018, increased by a factor of 300 000, with a doubling time of 3.5 months.That is 25 000 times faster than Moore's law at its peak [3].Handling all that computation requires more cores than are on a single standard chip.So multiple chips must work together in concert.But that also means data must be shuttled between chips-a process tens of thousands of times slower than possible within a single chip [1].Dozens or even hundreds of small chips can be fabricated out of a silicon wafer.But by producing a single integrated chip from one entire wafer, Cerebras has built a processor packed with cores that does not rely on the off-chip communication that can bog down conventional systems.The WSE also holds memory cores close to the compute cores, so the former can constantly feed data to the latter, reducing the idling time of the compute cores.Boasting an architecture designed for machine learning, the chip is optimized for AI training, according to Cerebras.It has 400 000 programmable cores, 18 GB of static random access memory (SRAM), and a