A research team trains two different models: one with an encoder-only structure and another with an encoder-decoder structure. Both models are trained using the exact same objective: predicting randomly masked words within a text. A colleague argues that because the model structures are different, they should be classified under separate pre-training categories. Why is this argument fundamentally flawed from a conceptual standpoint?

Classifying a Novel Pre-training Method

Justification for Pre-training Task Classification