Instance Normalization
E701501
Instance Normalization is a neural network normalization technique that normalizes each individual sample and channel independently, commonly used in tasks like style transfer to stabilize training and control feature statistics.
Statements (47)
| Predicate | Object |
|---|---|
| instanceOf |
neural network normalization technique
ⓘ
normalization layer ⓘ |
| advantage |
better control of style in style transfer
ⓘ
more stable behavior for small batch sizes ⓘ |
| alsoKnownAs | InstanceNorm ⓘ |
| appliedAfter | convolution layers ⓘ |
| applies | affine transformation ⓘ |
| category | feature-wise normalization ⓘ |
| commonlyUsedFor |
image generation tasks
ⓘ
image-to-image translation ⓘ neural style transfer ⓘ |
| computes |
per-instance mean
ⓘ
per-instance variance ⓘ |
| computesStatisticsPer |
channel
ⓘ
sample ⓘ |
| differsFrom |
Batch Normalization
NERFINISHED
ⓘ
Group Normalization NERFINISHED ⓘ Layer Normalization NERFINISHED ⓘ |
| doesNotDependOn | batch size ⓘ |
| doesNotUse | batch statistics at inference ⓘ |
| domain |
computer vision
ⓘ
generative modeling ⓘ |
| epsilonRole | numerical stability in variance normalization ⓘ |
| followedBy | learnable affine transform y = gamma * x_hat + beta ⓘ |
| goal |
control feature statistics
ⓘ
reduce style variance across spatial locations ⓘ stabilize training ⓘ |
| hasParameter |
scale parameter gamma
ⓘ
shift parameter beta ⓘ |
| implementedIn |
PyTorch as torch.nn.InstanceNorm1d
NERFINISHED
ⓘ
PyTorch as torch.nn.InstanceNorm2d NERFINISHED ⓘ PyTorch as torch.nn.InstanceNorm3d ⓘ TensorFlow Addons as tfa.layers.InstanceNormalization NERFINISHED ⓘ |
| inspired | use in fast neural style transfer networks ⓘ |
| introducedBy | Dmitry Ulyanov NERFINISHED ⓘ |
| introducedInPaper | Instance Normalization: The Missing Ingredient for Fast Stylization NERFINISHED ⓘ |
| introducedInYear | 2016 ⓘ |
| invariantTo | global contrast of each instance ⓘ |
| mathematicalOperation | x_hat = (x - mu_{n,c}) / sqrt(sigma_{n,c}^2 + epsilon) ⓘ |
| normalizes | feature activations ⓘ |
| normalizesAcross | spatial dimensions ⓘ |
| oftenReplaces | Batch Normalization in style transfer networks ⓘ |
| operatesOn |
individual channels
ⓘ
individual samples ⓘ |
| relatedTo | style normalization ⓘ |
| usedIn |
convolutional neural networks
ⓘ
deep learning models ⓘ |
Referenced by (1)
Full triples — surface form annotated when it differs from this entity's canonical label.