Entropy can help distinguish Wolfram's four classes (i.e., equilibrium, cyclic (periodic), random, and complex). To do so, the information entropy must be calculate for different sub-sequences of the time-series data.

University of Michigan - Ann Arbor

A measure of uncertainty based on the number of flips of a fair coin. Low entropy means little information is being revealed (i.e., sun rising in east) and high entropy means the outcomes are very uncertain but when actualized reveal a lot of information (i.e., a lottery ticket and reveal). For example, we would need to ask N questions to learn the sexes of N children. N questions would distinguish $2^N$ potential outcome sequences (N binary random events: N flips of a fair coin). Each outcome sequence has a probability of $$1\frac{1}{2^N}$$, which is converted to N = - $\log_2 (\frac{1}{2^N}) ^2$. If an outcome arises with probability p, it will have an uncertainty of $\log_2 (p) $ (an approximation of the # of yes/no questions needed to identify the outcome sequence).

Information Entropy

An outcome with no uncertainty, entropy = 0. A pencil sitting stationary on a desk is considered an equilibrium outcome.

Equilibrium ( classes of outcomes )

Not completely certain or uncertain (somewhere in-between), intermediate entropy. Neuronal firings are a complex outcome (they fire neither randomly nor in a fixed pattern).

Complex ( classes of outcomes )

Perfectly random, maximal entropy. A sequence of fair coin flips or the stock prices (approximately) are random outcomes.

Random ( classes of outcomes ) 

Somewhat predictable, low entropy. A planet orbiting a star is a cyclic/periodic outcome.

Cyclic / Periodic ( classes of outcomes )

The Model Thinker

Claude Shann used four conditions to find a valid measure of entropy. Any measure of entropy would need to meet all four of these conditions.

Learn Before

Related