Accuracy:

• Heuristic approach to the concept, we can have a high probability of making correct predictions and predict wrong most of the time yielding poor accuracy.

• Log scoring Rule for accuracy measure inherently uses the Bayes Theorem with joint probabilities (likelihood).

Entropy:

• The uncertainty contained in a probability distribution.

• Average log-prob of the event

• Is closely related to its scientific meaning, the measure of disorder in a system, now in terms of probability.