DARTS

E899016

differentiable architecture search algorithm gradient-based optimization method for architectures neural architecture search method

DARTS is a widely used differentiable neural architecture search method that optimizes network structures in a continuous space using gradient-based techniques.

Jump to: Statements Referenced by

Statements (48)

Predicate	Object
instanceOf	differentiable architecture search algorithm ⓘ gradient-based optimization method for architectures ⓘ neural architecture search method ⓘ
advantage	enables end-to-end differentiable architecture optimization ⓘ significantly reduces search cost compared to RL-based NAS ⓘ
appliedTo	CIFAR-10 NERFINISHED ⓘ ImageNet NERFINISHED ⓘ image classification ⓘ
category	weight-sharing NAS method ⓘ
citationTitle	DARTS: Differentiable Architecture Search NERFINISHED ⓘ
comparedTo	evolutionary algorithm based NAS methods ⓘ reinforcement learning based NAS methods ⓘ
field	deep learning ⓘ machine learning ⓘ neural architecture search ⓘ
fullName	Differentiable Architecture Search NERFINISHED ⓘ
implementation	commonly implemented in PyTorch ⓘ
influenced	many subsequent differentiable NAS methods ⓘ
inspired	FairDARTS NERFINISHED ⓘ PC-DARTS NERFINISHED ⓘ ProxylessNAS NERFINISHED ⓘ RobustDARTS NERFINISHED ⓘ
introduces	architecture parameters (alphas) ⓘ
limitation	can overfit to validation set ⓘ prone to performance collapse in some settings ⓘ tends to prefer parameter-heavy operations ⓘ
objective	validation performance ⓘ
optimizationType	bilevel optimization problem ⓘ
optimizes	architecture parameters jointly with network weights ⓘ cell-based network structures ⓘ neural network architectures ⓘ
output	discrete neural network architecture ⓘ
proposedBy	Hanxiao Liu NERFINISHED ⓘ Karen Simonyan NERFINISHED ⓘ Yiming Yang NERFINISHED ⓘ
publicationYear	2019 ⓘ
publishedIn	International Conference on Learning Representations NERFINISHED ⓘ
represents	discrete architectures in a continuous search space ⓘ
searchSpace	cell-based directed acyclic graphs ⓘ operations on edges between nodes ⓘ
searchStrategy	alternating optimization of weights and architecture parameters ⓘ gradient-based search in continuous space ⓘ
selects	operations with highest architecture weights ⓘ
trainingObjective	minimize training loss with respect to network weights ⓘ
uses	bilevel optimization ⓘ continuous relaxation of architecture choices ⓘ gradient-based optimization ⓘ
validationObjective	minimize validation loss with respect to architecture parameters ⓘ

Referenced by (1)

Full triples — surface form annotated when it differs from this entity's canonical label.

Neural Architecture Search → notableMethod → DARTS ⓘ