Scaling model capacity with language-specific parameters is a technique where certain layer parameters are split by language or language group based on vocabulary similarity. Because each translation direction only accesses a subset of these parameters, model capacity can scale without significantly increasing training and inference times.