GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts.

2019年11月5日 · As the final model release of GPT‑2 ⁠ ’s staged release ⁠, we’re releasing the largest version (1.5B parameters) of GPT‑2 along with code and model weights ⁠ to facilitate detection of outputs of GPT‑2 models.

Code and models from the paper "Language Models are Unsupervised Multitask Learners". You can read about GPT-2 and its staged release in our original blog post, 6 month follow-up post, and final post. We have also released a dataset for researchers to study their behaviors.

GPT-2 is a large transformer-based language model with 1.5 billion parameters, trained on a dataset [1] of 8 million web pages. GPT-2 is trained with a simple objective: predict the next word, given all of the previous words within some text.

2019年8月20日 · We’re releasing the 774 million parameter GPT‑2 language model after the release of our small 124M model ⁠ in February, staged release of our medium 355M model ⁠ in May, and subsequent research with partners and the AI community into the model’s potential for misuse and societal benefit.

2019年8月12日 · The GPT2 was, however, a very large, transformer-based language model trained on a massive dataset. In this post, we’ll look at the architecture that enabled the model to produce its results. We will go into the depths of its self-attention layer.

2024年4月30日 · In testing GPT2 has been able to break with learned conventions, create ASCII art, and is particularly good at coding.

2019年2月14日 · GPT‑2 is a large transformer ⁠ -based language model with 1.5 billion parameters, trained on a dataset A of 8 million web pages. GPT‑2 is trained with a simple objective: predict the next word, given all of the previous words within some text.

Inspired by Model Cards for Model Reporting (Mitchell et al.), we’re providing some accompanying information about the GPT-2 family of models we're releasing. Model Details.

2024年9月15日 · This post presents a detailed architectural diagram of GPT-2 that shows how input data transforms as it flows through the model. The diagram is meant to help you trace the steps the model takes...