Multi-fidelity optimization is a strategy that uses approximate proxies to optimize an objective efficiently. In the context of neural network design, this involves assessing whether an architecture is effective based on intermediate results—such as accuracy achieved after only a few training passes through the dataset—rather than incurring the significant computational cost of training every sampled network to full convergence.