Bayesian optimization (described by Shahriari, et al.) trains the model with different hyperparameter values, and observes the function generated for the model by each set of parameter values repeatedly. Each time selecting hyperparameter values that are slightly different and can help plot the next relevant segment of the problem space. Like sampling methods in statistics, the algorithm ends up with a list of possible hyperparameter value sets and model functions, from which it predicts the optimal function across the entire problem set.