Recording intelligenceAI-generated brief · check the source for context

Lecture 1 - Reasoning LLMs from Scratch - Series Introduction

49:04 recording · AUTO · 1 speaker

Watch the original

Executive Summary AI
  • Dr. Rajat Dandekar opens his first YouTube course, Reasoning-based LLMs from Scratch, introducing himself as the twin brother of Dr. Raj, a B.Tech and M.Tech from IIT Madras with a PhD from Purdue University, and co-founder of Vijwara with Rajan Sridhar.
  • He frames reasoning through system 1, the fast intuitive thinking that he says accounts for about 95% of our thinking, and system 2, the slow, logical, lazy and indecisive thinking used for questions like choosing a movie.
  • He traces the shift from ChatGPT 3.5, released November 2022, which answered fast but hallucinated and showed no steps, to OpenAI's O1 released on September 20, 2024, and then to DeepSeek R1, which was open source, comparable in accuracy to O1, and achieved reasoning by pure reinforcement learning with its documented aha moment.
  • Using Roger's five tennis balls and two cans of three, he shows a regular LLM spending three tokens on the answer while a thinking model spends 23, which is how test time compute — the computing resources used during inference — scales model accuracy.
  • He closes by listing the four methods the course covers: inference time compute scaling, pure RL, supervised fine tuning with reinforcement learning, and pure supervised fine tuning with distillation, the last two with hands-on builds using GRPO on a 5.4 model and distillation.

Brief overview

This opening lecture explains reasoning in LLMs and the four methods the course will teach from scratch.

  1. Reasoning means showing the intermediate stepsHe defines reasoning narrowly as answering questions that require complex multi-step reasoning with intermediate steps, illustrated by the train travelling 60 miles per hour for 3 hours to cover 180 m
  2. More compute at inference buys better answersJust as humans answer better with more time, allotting more computing resources during inference makes the model generate reasoning, and accuracy scales with test time compute.
  3. Trust is why high-stakes clients want reasoning modelsFor a healthcare, pharma or finance application, clients trust the product far more when they can see the step-by-step process that led the LLM to its answer.

Questions this recording answers

10 questions, each answered where it is said
What is the difference between System 1 and System 2 reasoning?

Reasoning comes in two types. System 1 is fast and immediate, about 95% of our thinking, driven by intuition, past experience and instinct, like answering what is the capital of India. System 2 is slow, logical, lazy and indecisive, like choosing which movie to watch tonight.

Could ChatGPT 3.5 reason, and where did it fall short?

No. AI is not made to think like humans, it is made to answer like them. ChatGPT 3.5, out in November 2022, was crazy good on System 1 tasks needing fast answers, but weak on puzzles, thinking and planning, showed no step-by-step process, and hallucinated random answers.

Why does showing the reasoning process matter for real applications?

Imagine representing a healthcare, pharma or finance company and delivering a high-stakes application that impacts people's lives. Would you give the client a direct answer, or show the step-by-step reasoning that led the LLM there? Clients trust the second case much more. That is the power of reasoning models.

When was the first reasoning model released, and by whom?

Research continued and O1, the first reasoning model, was released by OpenAI on September 20, 2024. Asked the same question, which movie shall I watch tonight, it reasoned for a couple of seconds and showed it was thinking through comedic styles, dramedy, action and classic options.

What made DeepSeek R1 special when O1 already existed?

Three things, mainly. It was completely open source, while OpenAI has never revealed the architecture of any model, so we finally saw a working reasoning architecture. Its accuracy was comparable to O1. And it achieved reasoning by pure reinforcement learning, which was a tremendous breakthrough.

What is test time compute, and what is inference?

Test time compute is the number of computing resources used by the model during inference, and inference is the process that leads the trained model to give you an answer. It is like an exam: preparation is the training stage, actually writing the answers is the inference stage.

How many tokens does making a model think actually add?

On Roger's tennis balls, a regular LLM answers in three tokens: answer, is, 11. Forced to think first, Roger started with five, two cans of three is six, five plus six is eleven, it needs 4 plus 11 plus 5 plus 3, twenty-three tokens, seven to eight times more compute.

What is chain of thought prompting, and who introduced it?

The brain team at Google Research introduced it in 2022. You give the model a worked example, Roger started with 5 balls, 2 cans of 3 is 6, 5 plus 6 is 11. The model then answers new questions, like the cafeteria's 23 apples, the same step-by-step way.

Which four methods does this course cover?

Inference time compute scaling, pure reinforcement learning, supervised fine tuning with reinforcement learning, and pure supervised fine tuning with distillation. The last two come with hands-on work: providing reasoning to a model using pure RL, and providing reasoning to a model using the process of distillation.

How does distillation give a small model reasoning ability?

You take a large model with reasoning capabilities, like DeepSeek R1, and generate input output pairs, say 500 questions and 500 answers carrying the entire reasoning, also called supervised fine-tuned data. Fine-tune a small non-reasoning model such as Qwen or Microsoft's Phi on it, and it gains reasoning.

Key Quote

“I am his twin brother, don't confuse me with him.”

— Dr. Rajat Dandekar, Lecture 1 - Reasoning LLMs from Scratch - Series Introduction, at 0:51▶ watch at 0:51
Key Quote

“System 1 helps us think immediately for an answer.”

— Dr. Rajat Dandekar, Lecture 1 - Reasoning LLMs from Scratch - Series Introduction, at 5:03▶ watch at 5:03
Key Quote

“AI is not made to think like humans. AI is made to answer like humans.”

— Dr. Rajat Dandekar, Lecture 1 - Reasoning LLMs from Scratch - Series Introduction, at 6:58▶ watch at 6:58
Key Quote

“It says wait, wait, wait that's an aha moment I can flag here.”

— Dr. Rajat Dandekar, Lecture 1 - Reasoning LLMs from Scratch - Series Introduction, at 13:34▶ watch at 13:34
Key Quote

“Test time compute is the number of computing resources used by the model during inference.”

— Dr. Rajat Dandekar, Lecture 1 - Reasoning LLMs from Scratch - Series Introduction, at 26:30▶ watch at 26:30
Key Quote

“So the researchers at DeepSeek actually did not expect this at all.”

— Dr. Rajat Dandekar, Lecture 1 - Reasoning LLMs from Scratch - Series Introduction, at 38:01▶ watch at 38:01

Cite this

“Lecture 1 - Reasoning LLMs from Scratch - Series Introduction.” https://www.youtube.com/watch?v=k2QEElQ9eeM. Transcript summary by WhipScribe, https://whipscribe.com/transcript-pages/lecture-1-reasoning-llms-from-scratch-series-introduction.html.

Report this page
What is wrong with this page?