minimum description length principle
E700157
The minimum description length principle is a formal method in statistics and machine learning that selects the best explanation for data as the one that yields the shortest overall description of both the model and the data it encodes.
All labels observed (1)
| Label | Occurrences |
|---|---|
| minimum description length principle canonical | 1 |
How this entity was disambiguated
This entity first appeared as the object of triple T7906520 — resolving that mention is where its identity was fixed. The disambiguator weighed these candidate entities and picked the highlighted one (or “None”, minting a new entity). This is how homonymy is resolved: the same surface form can point to different entities.
NED1
Entity disambiguation (via context triple)
gpt-5-mini-2025-08-07
Target entity: minimum description length principle Context triple: [Kolmogorov complexity, relatedTo, minimum description length principle]
-
A.
Kolmogorov complexity
Kolmogorov complexity is a measure of the amount of information in an object, defined as the length of the shortest computer program that can produce it.
-
B.
Bayesian Occam factor
The Bayesian Occam factor is a term in Bayesian model comparison that automatically penalizes overly complex models by integrating over their larger parameter spaces, thereby implementing Occam’s razor in probabilistic inference.
-
C.
Kullback–Leibler divergence
Kullback–Leibler divergence is a fundamental information-theoretic measure that quantifies how one probability distribution differs from a reference distribution.
-
D.
Fano inequality
Fano inequality is a fundamental result in information theory that provides a lower bound on the probability of classification or decoding error in terms of conditional entropy.
-
E.
Shannon–Khinchin axioms
The Shannon–Khinchin axioms are a set of fundamental conditions that uniquely characterize Shannon entropy as the standard measure of information and uncertainty in probability theory and information theory.
- F. None of above. chosen
- G. Unsure - the case is ambiguous/there is not enough information to decide.
NED2
Entity disambiguation (via description)
gpt-5-mini-2025-08-07
Target entity: minimum description length principle Target entity description: The minimum description length principle is a formal method in statistics and machine learning that selects the best explanation for data as the one that yields the shortest overall description of both the model and the data it encodes.
-
A.
Kolmogorov complexity
Kolmogorov complexity is a measure of the amount of information in an object, defined as the length of the shortest computer program that can produce it.
-
B.
Bayesian Occam factor
The Bayesian Occam factor is a term in Bayesian model comparison that automatically penalizes overly complex models by integrating over their larger parameter spaces, thereby implementing Occam’s razor in probabilistic inference.
-
C.
Kullback–Leibler divergence
Kullback–Leibler divergence is a fundamental information-theoretic measure that quantifies how one probability distribution differs from a reference distribution.
-
D.
Fano inequality
Fano inequality is a fundamental result in information theory that provides a lower bound on the probability of classification or decoding error in terms of conditional entropy.
-
E.
Shannon–Khinchin axioms
The Shannon–Khinchin axioms are a set of fundamental conditions that uniquely characterize Shannon entropy as the standard measure of information and uncertainty in probability theory and information theory.
- F. None of above. chosen
Statements (49)
| Predicate | Object |
|---|---|
| instanceOf |
information-theoretic principle
ⓘ
model selection principle ⓘ statistical learning principle ⓘ |
| appliedIn |
classification
ⓘ
clustering ⓘ feature selection ⓘ hypothesis testing ⓘ model selection for graphical models ⓘ pattern recognition ⓘ regression ⓘ time series modeling ⓘ |
| assumes |
data are encoded with an optimal code relative to the model
ⓘ
shorter codes correspond to more probable regularities ⓘ |
| basedOn |
Kolmogorov complexity
NERFINISHED
ⓘ
algorithmic information theory NERFINISHED ⓘ information theory NERFINISHED ⓘ |
| comparedTo |
Bayesian marginal likelihood
NERFINISHED
ⓘ
cross-validation ⓘ |
| contrastsWith | maximum likelihood principle ⓘ |
| coreIdea |
prefer the model that gives the shortest total description of model and data
ⓘ
trade off model complexity and goodness of fit ⓘ |
| criterionType | universal coding-based criterion ⓘ |
| field |
machine learning
ⓘ
model selection ⓘ statistical inference ⓘ statistics ⓘ |
| formalizes | Occam's razor NERFINISHED ⓘ |
| hasVariant |
normalized maximum likelihood MDL
ⓘ
refined MDL ⓘ two-part MDL ⓘ |
| implies |
penalty for model complexity
ⓘ
preference for simpler models that fit data well ⓘ |
| introducedBy | Jorma Rissanen NERFINISHED ⓘ |
| objective |
achieve good generalization
ⓘ
avoid overfitting ⓘ minimize total code length of model and data ⓘ |
| relatedTo |
Akaike information criterion
NERFINISHED
ⓘ
Bayesian information criterion NERFINISHED ⓘ Bayesian model selection NERFINISHED ⓘ minimum message length principle ⓘ |
| usedFor |
choosing model class
ⓘ
choosing model order ⓘ regularization design ⓘ |
| usesConcept |
code length
ⓘ
description length ⓘ lossless data compression ⓘ stochastic complexity ⓘ two-part code ⓘ universal coding ⓘ |
How these facts were elicited
The pipeline generated the facts above by prompting gpt-5.1 with this entity's name + description and the instruction below.
Instruction
You are a knowledge base construction expert. Given a subject entity and a description of it, return factual statements that you know for the subject as a JSON list of dictionaries(triples), where keys must be "subject", "predicate" and "object". The number of facts may be very high, between 25 to 50 or more, for very popular subjects. For less popular subjects, the number of facts can be very low, like 5 or 10. # Requirements - If you don't know the subject at all, return an empty list. - If the subject is not a named entity, return an empty list. - Include at least one triple where predicate is "instanceOf". - Do not get too wordy. - Separate several objects into multiple triples with one object.
Input
Subject: minimum description length principle Description of subject: The minimum description length principle is a formal method in statistics and machine learning that selects the best explanation for data as the one that yields the shortest overall description of both the model and the data it encodes.
Referenced by (1)
Full triples — surface form annotated when it differs from this entity's canonical label.