Reformer architecture
E899032
The Reformer architecture is a neural network model that improves Transformer efficiency by using locality-sensitive hashing attention and reversible layers to greatly reduce memory and computational costs.
All labels observed (1)
| Label | Occurrences |
|---|---|
| Reformer architecture canonical | 1 |
How this entity was disambiguated
This entity first appeared as the object of triple T11003393 — resolving that mention is where its identity was fixed. The disambiguator weighed these candidate entities and picked the highlighted one (or “None”, minting a new entity). This is how homonymy is resolved: the same surface form can point to different entities.
NED1
Entity disambiguation (via context triple)
gpt-5-mini-2025-08-07
Target entity: Reformer architecture Context triple: [Łukasz Kaiser, knownFor, Reformer architecture]
-
A.
Inception architecture
The Inception architecture is a deep convolutional neural network design that introduced parallel multi-scale processing modules to achieve state-of-the-art image recognition performance with improved computational efficiency.
-
B.
Learning Transferable Architectures for Scalable Image Recognition
"Learning Transferable Architectures for Scalable Image Recognition" is a research paper that introduced NASNet, a neural architecture search–designed convolutional network that achieved state-of-the-art performance on large-scale image recognition tasks.
-
C.
Swin Transformer
Swin Transformer is a hierarchical vision transformer architecture that uses shifted windows for efficient and scalable image recognition and related computer vision tasks.
-
D.
ResNet
ResNet is a deep convolutional neural network architecture known for its use of residual connections to enable very deep models and achieve state-of-the-art performance in image recognition tasks.
-
E.
Adam: A Method for Stochastic Optimization
"Adam: A Method for Stochastic Optimization" is a highly influential machine learning paper that introduces the Adam optimizer, a widely used adaptive gradient-based optimization algorithm for training deep neural networks.
- F. None of above. chosen
- G. Unsure - the case is ambiguous/there is not enough information to decide.
NED2
Entity disambiguation (via description)
gpt-5-mini-2025-08-07
Target entity: Reformer architecture Target entity description: The Reformer architecture is a neural network model that improves Transformer efficiency by using locality-sensitive hashing attention and reversible layers to greatly reduce memory and computational costs.
-
A.
Inception architecture
The Inception architecture is a deep convolutional neural network design that introduced parallel multi-scale processing modules to achieve state-of-the-art image recognition performance with improved computational efficiency.
-
B.
Learning Transferable Architectures for Scalable Image Recognition
"Learning Transferable Architectures for Scalable Image Recognition" is a research paper that introduced NASNet, a neural architecture search–designed convolutional network that achieved state-of-the-art performance on large-scale image recognition tasks.
-
C.
Swin Transformer
Swin Transformer is a hierarchical vision transformer architecture that uses shifted windows for efficient and scalable image recognition and related computer vision tasks.
-
D.
ResNet
ResNet is a deep convolutional neural network architecture known for its use of residual connections to enable very deep models and achieve state-of-the-art performance in image recognition tasks.
-
E.
Adam: A Method for Stochastic Optimization
"Adam: A Method for Stochastic Optimization" is a highly influential machine learning paper that introduces the Adam optimizer, a widely used adaptive gradient-based optimization algorithm for training deep neural networks.
- F. None of above. chosen
Statements (48)
| Predicate | Object |
|---|---|
| instanceOf |
Transformer-based model
ⓘ
neural network architecture ⓘ |
| aimsTo |
improve Transformer efficiency
ⓘ
reduce computational cost ⓘ reduce memory usage ⓘ |
| attentionRestriction | within hash buckets ⓘ |
| basedOn | Transformer architecture ⓘ |
| belongsTo | efficient Transformer family ⓘ |
| comparedTo | standard Transformer ⓘ |
| competesWith |
Linformer
NERFINISHED
ⓘ
Longformer NERFINISHED ⓘ Sparse Transformer NERFINISHED ⓘ |
| designedFor |
large-context tasks
ⓘ
long sequence modeling ⓘ memory-efficient training ⓘ |
| evaluationDomain | language modeling benchmarks ⓘ |
| groupsTokensBy | hash codes ⓘ |
| hasComponent |
LSH-based self-attention layer
ⓘ
position-wise feed-forward network ⓘ reversible residual block ⓘ |
| hasKeyFeature |
chunked feed-forward layers
ⓘ
locality-sensitive hashing attention ⓘ reversible residual layers ⓘ shared query-key projection ⓘ |
| implementedIn |
JAX
NERFINISHED
ⓘ
PyTorch NERFINISHED ⓘ TensorFlow NERFINISHED ⓘ |
| inspiredBy | locality-sensitive hashing methods ⓘ |
| introducedInPaper | Reformer: The Efficient Transformer NERFINISHED ⓘ |
| memoryOptimizationTechnique |
activation recomputation from outputs
ⓘ
reversible residual computation ⓘ |
| optimizationGoal | scalability to very long sequences ⓘ |
| proposedBy |
Anselm Levskaya
NERFINISHED
ⓘ
Nikita Kitaev NERFINISHED ⓘ Łukasz Kaiser NERFINISHED ⓘ |
| publicationYear | 2020 ⓘ |
| publishedBy | Google researchers NERFINISHED ⓘ |
| reduces |
activation memory footprint
ⓘ
attention computation cost ⓘ |
| reducesComplexityOf | self-attention ⓘ |
| standardTransformerComplexity | O(L^2) ⓘ |
| supports |
autoregressive language modeling
ⓘ
sequence-to-sequence tasks ⓘ |
| targetComplexity | O(L log L) ⓘ |
| uses |
LSH attention
ⓘ
locality-sensitive hashing ⓘ reversible layers ⓘ |
| usesSorting | hash buckets for attention ⓘ |
How these facts were elicited
The pipeline generated the facts above by prompting gpt-5.1 with this entity's name + description and the instruction below.
Instruction
You are a knowledge base construction expert. Given a subject entity and a description of it, return factual statements that you know for the subject as a JSON list of dictionaries(triples), where keys must be "subject", "predicate" and "object". The number of facts may be very high, between 25 to 50 or more, for very popular subjects. For less popular subjects, the number of facts can be very low, like 5 or 10. # Requirements - If you don't know the subject at all, return an empty list. - If the subject is not a named entity, return an empty list. - Include at least one triple where predicate is "instanceOf". - Do not get too wordy. - Separate several objects into multiple triples with one object.
Input
Subject: Reformer architecture Description of subject: The Reformer architecture is a neural network model that improves Transformer efficiency by using locality-sensitive hashing attention and reversible layers to greatly reduce memory and computational costs.
Referenced by (1)
Full triples — surface form annotated when it differs from this entity's canonical label.
subject surface form:
Łukasz Kaiser