Shannon–Khinchin axioms
E7559
The Shannon–Khinchin axioms are a set of fundamental conditions that uniquely characterize Shannon entropy as the standard measure of information and uncertainty in probability theory and information theory.
Statements (46)
| Predicate | Object |
|---|---|
| instanceOf |
axiomatic system
ⓘ
information theory concept ⓘ |
| appliesTo | discrete probability distributions ⓘ |
| characterizes |
Boltzmann–Gibbs entropy in statistical mechanics
ⓘ
Shannon entropy ⓘ |
| contrastWith | generalized entropy axioms ⓘ |
| defines | properties of an entropy function ⓘ |
| field |
information theory
ⓘ
probability theory ⓘ |
| hasAxiom |
Shannon entropy
ⓘ
surface form:
Shannon additivity axiom
additivity axiom ⓘ continuity axiom ⓘ continuity in probabilities ⓘ expansibility axiom ⓘ expansibility with zero-probability events ⓘ maximal entropy for the uniform distribution ⓘ maximality axiom ⓘ recursivity axiom ⓘ recursivity for compound experiments ⓘ symmetry axiom ⓘ |
| hasConsequence |
additivity for independent systems
ⓘ
logarithmic form of entropy ⓘ uniqueness of Shannon entropy as information measure ⓘ |
| implies |
adding an event with probability zero does not change entropy
ⓘ
entropy is Schur-concave ⓘ entropy is continuous in its arguments ⓘ entropy is maximal for the uniform distribution ⓘ entropy is nonnegative ⓘ entropy is symmetric in its arguments ⓘ entropy is uniquely determined up to a positive multiplicative constant ⓘ entropy of a compound experiment is sum of entropies plus conditional entropy ⓘ |
| language | mathematical formulation ⓘ |
| namedAfter |
Aleksandr Khinchin
ⓘ
Claude Shannon ⓘ |
| purpose | to uniquely determine Shannon entropy up to a positive multiplicative constant ⓘ |
| relatedTo |
Boltzmann–Gibbs entropy in statistical mechanics
ⓘ
surface form:
Boltzmann entropy
Faddeev’s axioms ⓘ Khinchin’s characterization of entropy ⓘ Rényi entropy ⓘ Shannon entropy ⓘ Tsallis entropy ⓘ |
| timePeriod | mid 20th century ⓘ |
| usedIn |
coding theory
ⓘ
foundations of information theory ⓘ information geometry ⓘ statistical mechanics ⓘ |
Referenced by (4)
Full triples — surface form annotated when it differs from this entity's canonical label.