Contrastive Predictive Coding

E755722

representation learning technique self-supervised learning method

Contrastive Predictive Coding is a self-supervised learning method that learns useful data representations by predicting future inputs in a latent space using a contrastive objective.

Jump to: Surface forms Statements Referenced by

Observed surface forms (1)

Surface form	Occurrences
Representation Learning with Contrastive Predictive Coding	1

Statements (47)

Predicate	Object
instanceOf	representation learning technique ⓘ self-supervised learning method ⓘ
abbreviation	CPC NERFINISHED ⓘ
appliedTo	audio representation learning ⓘ image representation learning ⓘ reinforcement learning state representation ⓘ speech representation learning ⓘ video representation learning ⓘ
architectureAgnostic	true ⓘ
canUseBackbone	convolutional neural networks ⓘ recurrent neural networks ⓘ transformers NERFINISHED ⓘ
codeAvailability	open-source implementations exist ⓘ
coreIdea	predict future inputs in latent space ⓘ
describedInPaper	Representation Learning with Contrastive Predictive Coding NERFINISHED ⓘ
distinguishesBetween	positive samples and negative samples ⓘ
doesNotRequire	manual labels ⓘ
domain	machine learning ⓘ
evaluationMethod	linear probe on learned representations ⓘ
influenced	CPC v2 NERFINISHED ⓘ MoCo NERFINISHED ⓘ SimCLR NERFINISHED ⓘ
inspired	subsequent contrastive self-supervised methods ⓘ
introducedBy	Aaron van den Oord NERFINISHED ⓘ Oriol Vinyals NERFINISHED ⓘ Yazhe Li NERFINISHED ⓘ
learningParadigm	self-supervised learning ⓘ
maximizes	mutual information between context and future latent representations ⓘ
negativeSamplesSource	other positions in batch or sequence ⓘ
objectiveType	contrastive objective ⓘ
operatesIn	latent representation space ⓘ
optimizationMethod	stochastic gradient descent ⓘ
predictionTarget	future latent representations ⓘ
primaryGoal	learn useful data representations ⓘ
publicationYear	2018 ⓘ
relatedToConcept	contrastive learning ⓘ mutual information maximization ⓘ predictive coding ⓘ
representationProperty	captures high-level structure in data ⓘ useful for downstream tasks ⓘ
subfield	unsupervised representation learning ⓘ
trainingSignalSource	data itself ⓘ
trainingStrategy	maximize agreement between true future and predicted future in latent space ⓘ
usesLossFunction	InfoNCE loss NERFINISHED ⓘ
usesMechanism	autoregressive model in latent space ⓘ context encoder ⓘ negative sampling ⓘ

Referenced by (2)

Full triples — surface form annotated when it differs from this entity's canonical label.

Aaron van den Oord → developed → Contrastive Predictive Coding ⓘ

Aaron van den Oord → notableWork → Contrastive Predictive Coding ⓘ

this entity surface form: Representation Learning with Contrastive Predictive Coding