Optimizing a massive network design space cheaply relies on four key assumptions. First, general design principles exist, meaning identifying a distribution over networks is a sensible strategy. Second, intermediate training results act as reliable proxies for final accuracy without training to convergence. Third, results obtained at a smaller scale structurally generalize to larger networks. Fourth, aspects of the architectural design can be approximately factorized, allowing their effects on quality to be inferred somewhat independently.