- **Pros**: Quite simple and effective with skilled operators
- **Cons**: Not scientific, unknown if you have fully optimized hyperparameters

University of Michigan - Ann Arbor

Traditionally, hyperparameters were tuned manually by trial and error. This is still commonly done, and experienced operators can “guess” parameter values that will achieve remarkably high accuracy for deep learning models. However, there is a constant search for better, faster, and more automatic methods to optimize hyperparameters. 

Manual Hyperparameter Tuning for Hyperparameter Tuning Methods in Deep Learning

A helpful website that introduces neural networks:
https://missinglink.ai/guides/neural-network-concepts/

Neural Network Reference

When manually picking hyperparameters it is important to use a good scale (especially if you are randomly picking values along an interval). This is especially relevant when tuning the learning rate alpha or any other hyperparameter where minor changes in values at the end of your interval can drastically improve performance. Using the log scale can help you distribute your resources more efficiently and ensure you sample a suitable number of values at the extreme lower/higher side of your chosen interval. 

Learn Before

Related