- Use probability distributions to logically process information 
- Involves the frequentist approach – imaginary resampling of data means that if we were to take the same measurements several times, we would observe a pattern
- Note: only measurements have probability distributions (parameters and models do not)

Bayesian data analysis

- Hierarchical, random effects, varying effects, or mixed effects model 
- Models with lots of levels of uncertainty 
- Help with overfitting through partial pooling 
- Should be the primary regression model used

Multilevel models

- Wind and branches example: it’s logical to conclude that the branches are swaying because of the wind, but that is not necessarily the causal relationship 
- Need to dig further to find causation because correlation does not imply causation 
- Models that are not accurate causally could still be better predictions than models that are casually correct because making correct predictions does not necessarily rely on having identified the specific cause 
- Directed acyclic graph (DAG) for making causal inferences; an example of a graphical causal model that can be used for causal hypotheses

Graphical causal models

- Compare models through cross validation and information criteria to determine the predictive accuracy and tendency for overfitting, in addition to highlighting any observations that are exerting significant influence to the model 
- Overfitting: simple models may lead to more accurate predictions than more complicated models

Model comparison and prediction

- Bayesian data analysis
- Model comparison and prediction
- Multilevel models
- Graphical causal models

University of Michigan - Ann Arbor

- Scientific models are designed for making calculations, but they cannot determine causality on their own. 
- To determine the most appropriate statistical model to use, researchers ask questions about the type of data they have and what they wish to measure. 
- Frequentist approach is based on re-sampling methods, and it is a special case of Bayesian analysis
- We have models that are defined to do certain tests, but as researchers, you need to get behind the methods and determine if results are practically significant 

Statistical Golems 

Figure of Statistical Procedures

1. Frequentist consider a single estimator $\theta$ to make all predictions. Bayesian statisticians use all possible values of $\theta$ to make predictions.
2. Frequentist view $\theta$ as fixed and dataset as variable. Bayesian view dataset as fixed and $\theta$ as variable.

Difference between Frequentist and Bayesian statistics

Tools for Golem Engineering


- Process models indicate causality through their structure
- Statistical models do not indicate causality
- Statistical models can correspond to more than one process model and more than one hypothesis
- Hypotheses can also correspond to more than one process model 

Hypotheses are not models

- Need to account for overfitting 
- Falsification is consensual - it’s not about how logical your arguments are, but how well you are able to prove that to the scientific community in a way that is understandable

Learn Before

Related

Learn After