Language Models are Unsupervised Multitask Learners

Alec RadfordJeffrey WuRewon ChildDavid LuanDario AmodeiIlya Sutskever

   Papers with code   Abstract  PDF

Natural language processing tasks, such as question answering, machine translation, reading comprehension, and summarization, are typically approached with supervised learning on taskspecific datasets. We demonstrate that language models begin to learn these tasks without any explicit supervision when trained on a new dataset of millions of webpages called WebText... (read more)

Benchmarked Models

RANK
MODEL
REPO
CODE RESULT
PAPER RESULT
ε-REPRODUCED
BUILD