Hyperscale AI

• Library & Framework

Library & Framework

• DeepSpeed makes distributed training and inference easy, efficient, and effective. - Microsoft
• FairScale is a PyTorch extension library for high performance and large scale training. - Meta AI
• Megatron is a PyTorch-based framework for training giant language models based on the transformer architecture. - NVIDIA
  • NeMo is a toolkit for conversational AI - NVIDIA
• Transformers provides thousands of pretrained models to perform tasks on different modalities - HuggingFace

Last Modified: 2022/06/13 16:22:47