Adam: A Method for Stochastic Optimization

E182822

"Adam: A Method for Stochastic Optimization" is a highly influential machine learning paper that introduces the Adam optimizer, a widely used adaptive gradient-based optimization algorithm for training deep neural networks.

All labels observed (3)

How this entity was disambiguated

Statements (49)

Predicate Object
instanceOf machine learning paper
scientific paper
abbreviation ICLR
application large-scale machine learning problems
training deep neural networks
author Diederik P. Kingma
Jimmy Ba
basedOn stochastic gradient descent
comparesWith AdaGrad
RMSProp
SGD with momentum
defaultSetting beta1 = 0.9
beta2 = 0.999
epsilon = 1e-8
field deep learning
machine learning
optimization
hyperparameter beta1
beta2
epsilon
learning rate
impact widely used optimizer in deep learning frameworks
influenced Adam optimizer
surface form: AdamW optimizer

variants of adaptive gradient methods
introduces Adam optimizer
introducesConcept bias-corrected first moment estimate
bias-corrected second moment estimate
licenseOfImplementation open-source implementations available
propertyClaimed computationally efficient
invariant to diagonal rescaling of gradients
little memory requirement
well suited for problems with large data
well suited for problems with large parameters
proposes Adam: A Method for Stochastic Optimization self-linksurface differs
surface form: adaptive moment estimation
shortTitle Adam
status highly cited
title Adam: A Method for Stochastic Optimization self-link
topic adaptive learning rate methods
gradient-based optimization
stochastic optimization
usedIn Keras
PyTorch
TensorFlow
uses exponentially decaying averages of past gradients
exponentially decaying averages of squared gradients
first moment estimates of gradients
second moment estimates of gradients
venue ICLR
surface form: International Conference on Learning Representations
yearProposed 2014

How these facts were elicited

Referenced by (7)

Full triples — surface form annotated when it differs from this entity's canonical label.

Jimmy Ba coAuthorOf Adam: A Method for Stochastic Optimization
Jimmy Ba hasNotableWork Adam: A Method for Stochastic Optimization
Adam optimizer fullName Adam: A Method for Stochastic Optimization
this entity surface form: Adaptive Moment Estimation
Adam optimizer introducedInPaper Adam: A Method for Stochastic Optimization
Adam: A Method for Stochastic Optimization title Adam: A Method for Stochastic Optimization self-link
Adam: A Method for Stochastic Optimization proposes Adam: A Method for Stochastic Optimization self-linksurface differs
this entity surface form: adaptive moment estimation
Diederik P. Kingma coAuthorOf Adam: A Method for Stochastic Optimization