Rényi entropy
E6393
Rényi entropy is a generalized measure of information and uncertainty that extends Shannon entropy by introducing a tunable order parameter to emphasize different aspects of a probability distribution.
Observed surface forms (1)
| Surface form | Occurrences |
|---|---|
| Rényi α-entropy | 1 |
Statements (47)
| Predicate | Object |
|---|---|
| instanceOf |
entropy measure
ⓘ
generalization of Shannon entropy ⓘ information-theoretic measure ⓘ |
| alsoKnownAs |
Rényi entropy
ⓘ
surface form:
Rényi α-entropy
|
| appliesTo |
continuous probability distributions
ⓘ
discrete probability distributions ⓘ |
| belongsTo | generalized entropy family ⓘ |
| characterizes | tail behavior of distributions for different α ⓘ |
| dependsOn | probability distribution ⓘ |
| emphasizes | different aspects of a distribution depending on α ⓘ |
| field |
information theory
ⓘ
probability theory ⓘ statistical mechanics ⓘ statistics ⓘ |
| generalizes | Shannon entropy ⓘ |
| hasMathematicalForm | H_α(P) = 1/(1-α) log(∑_i p_i^α) for α ≠ 1 ⓘ |
| hasOrderParameter | α ⓘ |
| hasSpecialCase |
collision entropy at α = 2
ⓘ
max-entropy as α → 0 ⓘ min-entropy as α → ∞ ⓘ |
| introducedBy | Alfréd Rényi ⓘ |
| introducedIn | 1960s ⓘ |
| logarithmBase | depends on chosen units (e.g. base 2 for bits) ⓘ |
| monotoneIn | order parameter α for fixed distribution ⓘ |
| namedAfter | Alfréd Rényi ⓘ |
| parameterDomain | α ≥ 0, α ≠ 1 ⓘ |
| reducesTo | Shannon entropy when α → 1 ⓘ |
| relatedTo |
Rényi divergence
ⓘ
Shannon entropy ⓘ Tsallis entropy ⓘ collision entropy ⓘ min-entropy ⓘ |
| satisfies | data-processing inequality for appropriate α ⓘ |
| unit |
bits when logarithm base 2 is used
ⓘ
nats when natural logarithm is used ⓘ |
| usedFor |
defining Rényi divergence
ⓘ
measuring concentration of probability mass ⓘ measuring diversity ⓘ measuring information content ⓘ measuring uncertainty ⓘ |
| usedIn |
cryptography
ⓘ
fractal analysis ⓘ hypothesis testing ⓘ information-theoretic security ⓘ machine learning ⓘ multifractal analysis ⓘ statistical physics ⓘ |
Referenced by (6)
Full triples — surface form annotated when it differs from this entity's canonical label.
this entity surface form:
Rényi α-entropy
subject surface form:
Boltzmann–Gibbs entropy