A multilingual model is pre-trained on a large corpus of English and Spanish text using a single, unified vocabulary. The model processes the word 'pie', which means 'foot' in Spanish and refers to a baked dish in English. How will this word most likely be represented within the model's vocabulary structure?

Trade-offs of a Unified Vocabulary in Multilingual Models

In a multilingual model pre-trained on English and German, the shared vocabulary is structured into two distinct sections, one for English tokens and one for German tokens, to prevent interference between the languages.

Language-Independent Token Representations

Example of Code-Switching between Chinese and English