A team of engineers is designing a new large-scale language model, aiming for state-of-the-art performance and training efficiency, similar to other successful modern architectures. They are debating whether to use a standard Rectified Linear Unit (ReLU) or a Swish-based Gated Linear Unit for the activation function within the model's feed-forward network blocks. Analyze the primary reasons why the team might choose the Swish-based Gated Linear Unit over ReLU, considering the potential impact on the model's learning capabilities and overall performance.