While deep learning (DL) models achieve promising performance on challenging benchmarks, they largely lack interpretability. It remains difficult to explain why a model excels on one dataset but underperforms on another, what exactly the model has learned, or what minimal neural network architecture is needed to achieve a certain accuracy. Although mechanisms like attention provide some insight, a detailed theoretical understanding of the underlying behavior and dynamics of these models is still lacking. Resolving this gap is crucial for developing more reliable models curated for various applications, such as text analysis.