PaLM 2 is an updated version of the Pathway Language Model (PaLM) that scaled both the training data and model size at a roughly $$1:1$$ ratio. This scaling approach led to improved capabilities in multilingual tasks and reasoning.

Claude

PaLM (Pathway Language Model) is a $$540$$-billion-parameter Transformer decoder pretrained on $$780$$ billion tokens. With its modified designs, it successfully outperformed average human performance on the BIG-Bench benchmark, continuing the trend of scaling language models.

PaLM (Pathway Language Model)

Dive into Deep Learning

PaLM 2

Minerva is a large language model created by further training a generalist model, specifically PaLM. It has demonstrated promising capabilities in solving quantitative and scientific reasoning problems.

Learn Before

Related