Six Times Bigger than GPT-3: Inside Google’s TRILLION Parameter Switch Transformer Model

Google’s Switch Transformer model could be the next breakthrough in this area of deep learning.