BERT
E457858
BERT is a widely used transformer-based language model developed by Google that learns deep bidirectional representations of text for tasks like question answering and text classification.
All labels observed (1)
| Label | Occurrences |
|---|---|
| BERT canonical | 3 |
How this entity was disambiguated
This entity first appeared as the object of triple T4651120 — resolving that mention is where its identity was fixed. The disambiguator weighed these candidate entities and picked the highlighted one (or “None”, minting a new entity). This is how homonymy is resolved: the same surface form can point to different entities.
NED1
Entity disambiguation (via context triple)
gpt-5-mini-2025-08-07
Target entity: BERT Context triple: [Transformer, foundationFor, BERT]
-
A.
DistilBERT
DistilBERT is a smaller, faster, and lighter-weight distilled version of the BERT language model designed to retain most of its performance while being more efficient for practical NLP applications.
-
B.
RoBERTa
RoBERTa is a robustly optimized transformer-based language model developed by Facebook AI that improves upon BERT through enhanced training strategies and larger-scale data.
-
C.
DeBERTa
DeBERTa is a transformer-based language model developed by Microsoft that improves upon BERT and RoBERTa using disentangled attention and enhanced mask decoder mechanisms for superior natural language understanding.
-
D.
GPT-2
GPT-2 is a large transformer-based language model known for generating coherent, human-like text and sparking widespread discussion about the implications of advanced AI text generation.
-
E.
XLNet
XLNet is a generalized autoregressive pretraining model for natural language processing that improves on BERT by leveraging permutation-based language modeling to better capture bidirectional context.
- F. None of above. chosen
- G. Unsure - the case is ambiguous/there is not enough information to decide.
NED2
Entity disambiguation (via description)
gpt-5-mini-2025-08-07
Target entity: BERT Target entity description: BERT is a widely used transformer-based language model developed by Google that learns deep bidirectional representations of text for tasks like question answering and text classification.
-
A.
DistilBERT
DistilBERT is a smaller, faster, and lighter-weight distilled version of the BERT language model designed to retain most of its performance while being more efficient for practical NLP applications.
-
B.
RoBERTa
RoBERTa is a robustly optimized transformer-based language model developed by Facebook AI that improves upon BERT through enhanced training strategies and larger-scale data.
-
C.
DeBERTa
DeBERTa is a transformer-based language model developed by Microsoft that improves upon BERT and RoBERTa using disentangled attention and enhanced mask decoder mechanisms for superior natural language understanding.
-
D.
GPT-2
GPT-2 is a large transformer-based language model known for generating coherent, human-like text and sparking widespread discussion about the implications of advanced AI text generation.
-
E.
XLNet
XLNet is a generalized autoregressive pretraining model for natural language processing that improves on BERT by leveraging permutation-based language modeling to better capture bidirectional context.
- F. None of above. chosen
Statements (54)
| Predicate | Object |
|---|---|
| instanceOf |
BERT variant
ⓘ
BERT variant ⓘ language model ⓘ neural network model ⓘ transformer-based model ⓘ |
| acronymFor | Bidirectional Encoder Representations from Transformers NERFINISHED ⓘ |
| architecture | Transformer NERFINISHED ⓘ |
| benchmarkPerformance |
state-of-the-art on GLUE at time of publication
ⓘ
state-of-the-art on SQuAD at time of publication ⓘ |
| developer |
Google
ⓘ
Google AI Language NERFINISHED ⓘ |
| fineTuningApproach | task-specific output layer on top of shared encoder ⓘ |
| fullName | Bidirectional Encoder Representations from Transformers NERFINISHED ⓘ |
| hiddenSize |
1024
ⓘ
768 ⓘ |
| implementation | TensorFlow NERFINISHED ⓘ |
| influenced |
ALBERT
NERFINISHED
ⓘ
DistilBERT NERFINISHED ⓘ ELECTRA NERFINISHED ⓘ RoBERTa NERFINISHED ⓘ XLNet NERFINISHED ⓘ |
| inputEmbedding | sum of token, segment, and position embeddings ⓘ |
| inputRepresentation | subword tokens ⓘ |
| introducedInPaper | BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding NERFINISHED ⓘ |
| language | English ⓘ |
| license | Apache License 2.0 ⓘ |
| numAttentionHeads |
12
ⓘ
16 ⓘ |
| numLayers |
12
ⓘ
24 ⓘ |
| openSource | true ⓘ |
| paperAuthors |
Jacob Devlin
NERFINISHED
ⓘ
Kenton Lee NERFINISHED ⓘ Kristina Toutanova NERFINISHED ⓘ Ming-Wei Chang NERFINISHED ⓘ |
| pretrainingObjective |
masked language modeling
ⓘ
next sentence prediction ⓘ |
| publicationYear | 2018 ⓘ |
| representationType | deep bidirectional contextual representations ⓘ |
| supportsTask |
named entity recognition
ⓘ
natural language inference ⓘ paraphrase detection ⓘ question answering ⓘ semantic similarity ⓘ sentiment analysis ⓘ sequence labeling ⓘ text classification ⓘ token classification ⓘ |
| taskType | self-supervised learning ⓘ |
| trainingCorpus |
BooksCorpus
NERFINISHED
ⓘ
English Wikipedia NERFINISHED ⓘ |
| usesTokenization | WordPiece NERFINISHED ⓘ |
| variant |
BERT_BASE
ⓘ
BERT_LARGE NERFINISHED ⓘ |
How these facts were elicited
The pipeline generated the facts above by prompting gpt-5.1 with this entity's name + description and the instruction below.
Instruction
You are a knowledge base construction expert. Given a subject entity and a description of it, return factual statements that you know for the subject as a JSON list of dictionaries(triples), where keys must be "subject", "predicate" and "object". The number of facts may be very high, between 25 to 50 or more, for very popular subjects. For less popular subjects, the number of facts can be very low, like 5 or 10. # Requirements - If you don't know the subject at all, return an empty list. - If the subject is not a named entity, return an empty list. - Include at least one triple where predicate is "instanceOf". - Do not get too wordy. - Separate several objects into multiple triples with one object.
Input
Subject: BERT Description of subject: BERT is a widely used transformer-based language model developed by Google that learns deep bidirectional representations of text for tasks like question answering and text classification.
Referenced by (3)
Full triples — surface form annotated when it differs from this entity's canonical label.