$\color{black}\rule{365px}{3px}$

“Improving Language Understanding by Generative Pre-Training” - 2018

Link to Paper:

cdn.openai.com

Table of Contents

1. Introduction

$\color{black}\rule{365px}{3px}$

Motivation

Prior to GPT-1:

• NLP systems often relied heavily on supervised learning, requiring large and high-quality labeled datasets for each specific task.
• Researchers struggled to transfer learned representations across different tasks effectively.

Contributions