#1NVIDIA SCALING AI AND USING IT Bryan Catanzaro; September 9, 2021#2Forward-looking statements Except for the historical information contained herein, certain matters in this presentation including, but not limited to, statements as to: language model complexity doubling every two months; expected timing for 100T parameter single model; benefits, cost and impact of language models and few-shot learning; Megatron scaling on DGX SuperPoD; reinventing NVIDIA DLSS with deep learning; Al growing at a super-exponential rate; NVIDIA building the software and systems to create the most important Al; NVIDIA building advanced Al to transform our core business; and other predictions and estimates are forward- looking statements within the meaning of the Private Securities Litigation Reform Act of 1995. These forward-looking statements and any other forward- looking statements that go beyond historical facts that are made in this presentation are subject to risks and uncertainties that may cause actual results to differ materially. Important factors that could cause actual results to differ materially include: global economic conditions; our reliance on third parties to manufacture, assemble, package and test our products; the impact of technological development and competition; development of new products and technologies or enhancements to our existing product and technologies; market acceptance of our products or our partners' products; design, manufacturing or software defects; changes in consumer preferences and demands; changes in industry standards and interfaces; unexpected loss of performance of our products or technologies when integrated into systems and other factors. NVIDIA has based these forward-looking statements largely on its current expectations and projections about future events and trends that it believes may affect its financial condition, results of operations, business strategy, short- term and long-term business operations and objectives, and financial needs. These forward-looking statements are subject to a number of risks and uncertainties, and you should not rely upon the forward-looking statements as predictions of future events. The future events and trends discussed in this presentation may not occur and actual results could differ materially and adversely from those anticipated or implied in the forward-looking statements. Although NVIDIA believes that the expectations reflected in the forward-looking statements are reasonable, the company cannot guarantee that future results, levels of activity, performance, achievements or events and circumstances reflected in the forward-looking statements will occur. Except as required by law, NVIDIA disclaims any obligation to update these forward-looking statements to reflect future events or circumstances. For a complete discussion of factors that could materially affect our financial results and operations, please refer to the reports we file from time to time with the SEC, including our Annual Report on Form 10-K and quarterly reports on Form 10-Q. Copies of reports we file with the SEC are posted on our website and are available from NVIDIA without charge. NVIDIA#3PetaFLOPS - Days @ctnzr 1.E+04 1.E+03 1.E+02 1.E+01 1.E+00 1.E-01 1.E-02 1.E-03 AlexNet 2012 2013 Exploding Model Complexity Doubling Every 2 Months 2014 ResNet LARGE LANGUAGE MODELS 2016 Megatron-BERT Megatron-GPT2 GPT-2 2017 BERT 2018 GPT-3 Turing NLG 2020 Compute required to train 175B OpenAI GPT-3 314 ZettaFLOP for training (3640 PFLOP/s* day) P KA NVIDIA 100T parameter single model by 2023 Language Models Constrained by Economics NVIDIA#4TOWARDS THE $1B MODEL The value of few-shot learning ► These models are expensive (GPT-3 cost $12M to train) ► But their value is great ►Generalized intelligence with few-shot learning What would it look like to build a $1B model? ► It would need to reinvent an entire company English: I live in California. Spanish: Yo vivo en California. English: I work at NVIDIA. Spanish: Yo trabajo en NVIDIA. English: I believe in science. Spanish: Yo creo en la ciencia. Example: Few-shot translation This model was trained on general text from the internet - yet give it a few examples and ask it to do translation, and it can translate. Language describes all human activity. NVIDIA.#5MEGATRON SCALING ON DGX SUPERPOD ► All of NVIDIA's HW and SW working together ► CUDA, CUDA-X AI, NVSwitch, DGX SuperPod, NCCL, CUBLAS, CUDNN ► Trained with PyTorch Language models useful for NVIDIA's own products 502 petaFLOP/s sustained at 3072 GPUs 52% of tensor-core peak 163 Tflops/GPU Case 1T 530B 145B 39B 7.5B PetaFLOP/s 600 500 400 300 200 100 0 Hidden Size 25600 20480 12288 8192 4096 500 1000 Number of Layers 128 105 80 48 36 1500 2000 GPU Count Model Parallel Size 512 280 64 16 4 2500 ■ Sustained performance ----- Linear scaling 3000 Number of GPUs 3072 2520 1536 512 128 3500 NVIDIA.#6@ctnzr Ray Tracing NVIDIA DLSS Reinventing a Core NVIDIA Business with Deep Learning 4MPixel (1440P) A W Deep Learning Super Sampling 8MPixel (4K) VS Supercomputer Rendered 128MPixel (16K) Ground Truth NVIDIA#7THE TAMPERED THE TV NATIVE 4K 108 TIF TAMPERED DIF TA DLSS 4K 141#8TIE TAKPERED TIF TAMPERED DO NOT TAMPER IF TA NATIVE 4K 108 IF TAMPERE IF TAMPERED SEA O REMOVE LABEL PED 602 4558 DO NOT TAMPER IF TA DLSS 4K DIF TA 141#9DLSS is impressive to the point where I believe you'd be nuts not to use it.” Digital Foundry "The upscaling power of this new Al driven algorithm is extremely impressive... it's basically a free performance button.' - Hardware Unboxed#10CREATING A SELF-DRIVING CAR PLATFORM, INFRASTRUCTURE, AND SERVICE Data Data Factory Data Model Data HD Map Map Generation Model Perception Al Training Model Driving Al Training DRIVE AV Map Model Data AV Synthetic Data Drive Sim 10 NVIDIA#11SCALING AI AND USING IT Al is growing at a super-exponential rate NVIDIA is building the software and systems to create the most important Al We are applying advanced Al to transform and grow our core business We offer the tools for other companies to do so as well 11 NVIDIA#12NVIDIA