Generative AI Computing Platform

1010 109 108 Training Compute (petaFLOPs) 107 106 Modern Al is a Data Center Scale Computing Workload Data centers are becoming Al factories: Data as input, intelligence as output Before Transformers = 8x/2yrs Transformers = 215x/2yrs 105 104 103 ⚫ AlexNet MT NLG 530B .. GPT-3⚫ Microsoft T-NLG GPT-2⚫ Megatron-NLG⚫ PaLM • Chinchilla XLNet⚫ Wav2Vec 2.0 MoCo ResNet50 Xception BERT Large Inception V3 ⚫ Seq2Seq Resnet VGG-19 • GPT-1 Transformer • ResNeXt • ELMO DenseNet201 ● • BLOOM 102 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 Al Training Computational Requirements 9898. 88988 9898. NVIDIA nVIDI nVIDIA Large Language Models, based on the Transformer architecture, are one of today's most important advanced Al technologies, involving up to trillions of parameters that learn from text. 8898. 8898. 8898. 8898 8898. Developing them is an expensive, time-consuming process that demands deep technical expertise, distributed data center-scale infrastructure, and a full-stack accelerated computing approach. NVIDI 100 -ηνικ Grvou Greens NVIDIA Fueling Giant-Scale Al Infrastructure NVIDIA compute & networking GPU | DPU | CPU 1004 12 NVIDIA.

View entire presentation