Let's build GPT: from scratch, in code, spelled out.

Why it was recommended

the best video I have seen on the topic

Strong recommendationwesammikhail · Original2023-05

What it covers

Andrej Karpathy builds a GPT from scratch in code, following Attention Is All You Need and GPT-2/3, in about two hours using PyTorch. He connects it to ChatGPT and assumes. It is for someone who already knows the basic terms and wants the next step.

Video details

Channel

Andrej Karpathy

Length

1 hr 56 min

Year

2023

More in this topic

• Transformers, the tech behind LLMs | Deep Learning Chapter 5
• Deep Dive into LLMs like ChatGPT
• A Hackers' Guide to Language Models
• [1hr Talk] Intro to Large Language Models
• Sparks of AGI: early experiments with GPT-4
• Let's build the GPT Tokenizer