BERT

E457858

BERT variant language model neural network model transformer-based model

BERT is a widely used transformer-based language model developed by Google that learns deep bidirectional representations of text for tasks like question answering and text classification.

Try in SPARQL Jump to: Surface forms Statements Referenced by

Observed surface forms (2)

Surface form	Occurrences
BERT_BASE	0
BERT_LARGE	0

Statements (54)

Predicate	Object
instanceOf	BERT variant ⓘ BERT variant ⓘ language model ⓘ neural network model ⓘ transformer-based model ⓘ
acronymFor	Bidirectional Encoder Representations from Transformers NERFINISHED ⓘ
architecture	Transformer NERFINISHED ⓘ
benchmarkPerformance	state-of-the-art on GLUE at time of publication ⓘ state-of-the-art on SQuAD at time of publication ⓘ
developer	Google ⓘ Google AI Language NERFINISHED ⓘ
fineTuningApproach	task-specific output layer on top of shared encoder ⓘ
fullName	Bidirectional Encoder Representations from Transformers NERFINISHED ⓘ
hiddenSize	1024 ⓘ 768 ⓘ
implementation	TensorFlow NERFINISHED ⓘ
influenced	ALBERT NERFINISHED ⓘ DistilBERT NERFINISHED ⓘ ELECTRA NERFINISHED ⓘ RoBERTa NERFINISHED ⓘ XLNet NERFINISHED ⓘ
inputEmbedding	sum of token, segment, and position embeddings ⓘ
inputRepresentation	subword tokens ⓘ
introducedInPaper	BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding NERFINISHED ⓘ
language	English ⓘ
license	Apache License 2.0 ⓘ
numAttentionHeads	12 ⓘ 16 ⓘ
numLayers	12 ⓘ 24 ⓘ
openSource	true ⓘ
paperAuthors	Jacob Devlin NERFINISHED ⓘ Kenton Lee NERFINISHED ⓘ Kristina Toutanova NERFINISHED ⓘ Ming-Wei Chang NERFINISHED ⓘ
pretrainingObjective	masked language modeling ⓘ next sentence prediction ⓘ
publicationYear	2018 ⓘ
representationType	deep bidirectional contextual representations ⓘ
supportsTask	named entity recognition ⓘ natural language inference ⓘ paraphrase detection ⓘ question answering ⓘ semantic similarity ⓘ sentiment analysis ⓘ sequence labeling ⓘ text classification ⓘ token classification ⓘ
taskType	self-supervised learning ⓘ
trainingCorpus	BooksCorpus NERFINISHED ⓘ English Wikipedia NERFINISHED ⓘ
usesTokenization	WordPiece NERFINISHED ⓘ
variant	BERT_BASE ⓘ BERT_LARGE NERFINISHED ⓘ

Referenced by (3)

Full triples — surface form annotated when it differs from this entity's canonical label.

Transformer → foundationFor → BERT ⓘ

Elmo → influenced → BERT ⓘ

Layer Normalization → usedIn → BERT ⓘ