Why this lesson exists
Four architectural ideas explain nearly every quirk you will meet in practice — why it is cheap, why long sessions behave differently from short ones, why it sometimes thinks too long, and why the model can read images at all. You do not need to implement any of this. You need to be able to predict behaviour.
Idea one: mixture of experts
DeepSeek's large models are not one network that fires in full on every token. They are a mixture of experts: the model contains many specialised sub-networks, and a small routing mechanism selects a few of them for each token.
The practical shape of it: the model has an enormous number of parameters, and each token only activates a small fraction of them. The headline parameter count describes the model's capacity. The number that determines your inference bill is the activated fraction.
Why this matters to you: when someone quotes a huge parameter count and you wonder how this can possibly be cheap, the answer is that only a slice of it runs per token. Capacity and cost decouple. This is the single most important economics fact about the model you are using.
It also explains a behavioural trait you will notice. MoE models route each token to specialised experts. That specialisation tends to reward prompts that clearly signal which kind of task this is. A prompt that reads like a clear domain task ("rewrite this SQL query") is easier to route well than a vague one ("help me with this"). You will see this echoed in Lesson 8 as "specificity changes routing, not just comprehension."
Idea two: multi-head latent attention and the KV cache
This is the idea that made long-running agents affordable. It arrived with DeepSeek-V2, and it is worth understanding properly because it is the reason this course can tell you to keep a coding agent running for hours.
Every transformer model, when generating, must remember the keys and values ("KV") for every token that came before. This store is called the KV cache. It grows linearly with context length. Now the part that surprises people:
At sufficient context length, the KV cache outgrows the model weights themselves.
Think about what that means for an agent. The model's weights sit in memory once. But a long agent session keeps appending — tool output, file contents, conversation, thinking traces. Past a certain length, the thing eating your memory budget is not the model. It is the model's notes about the conversation.
Multi-head latent attention compresses the cached representation — storing a latent, lower-dimensional form instead of the full keys and values. The effect is a dramatic reduction in cache memory and the cost of serving long contexts. DeepSeek's own commentary on the V4.1 architecture frames cache management as the central cost problem for teams "running agents at scale."
Two behavioural consequences you should internalise:
- Long context is genuinely usable here, not a marketing number you avoid because of cost. That is rare and it changes what you can attempt.
- Long context is still not free, and its cost is dominated by caching behaviour, not raw length. Lesson 6 turns this into concrete prompt design rules.
Idea three: reasoning learned by reinforcement
The R1 result established that structured reasoning emerges from reinforcement learning against a verifiable reward. The mechanism that got the attention was a group-relative policy optimisation approach — comparing a group of sampled answers to each other rather than against a learned critic.
You do not need the maths. You need three observable facts that follow from it:
- Thinking is a behaviour, not a hidden process. The model emits thinking tokens. You can read them, count them, and reason about them.
- Because it is a trained behaviour, it is steerable. If the model was trained with varying amounts of reinforcement, different effort budgets are available at inference time. That is exactly why this generation exposes
low/high/max. - Reasoning is not universally better. A model that has been trained to deliberate will deliberate even when deliberation is counterproductive. For a formatting task or a straightforward rewrite, extended thinking is wasted tokens — and occasionally produces a worse answer than the model's first instinct.
The overthinking failure
Thinking mode is on by default. On genuinely trivial tasks this means you pay for tokens that add nothing, and more importantly you wait — sometimes several seconds — for output that a non-thinking call would have produced instantly and identically. The failure is not dramatic; it is a steady tax on latency and budget that is easy to never notice.
The fix is a habit, not a setting: before firing a request, ask what shape the task is. Extraction, formatting, classification, short rewrites, and simple lookups do not benefit from deliberation. Lesson 9 gives you the decision table.
Idea four: native multimodal input
V4.1-Flash handles images in the same request path as text — there is no separate vision endpoint, and no separate model name to remember. Images arrive as content blocks inside the message array, alongside text.
Why this is larger than it sounds: it collapses two workflows into one. Screenshot-driven UI review, reading a chart, transcribing a scan, and checking a rendered page against a design brief all become the same kind of call you were already making. For a builder, that means the model can participate in the visual half of the job rather than being handed a description of it.
You will use this in Module 4 for design work and in Module 6 for verification.
Putting the four ideas together
| Idea | What it produces | Where you meet it in this course |
|---|---|---|
| Mixture of experts | Capacity without proportional cost; specialisation that rewards clear task signals | Lessons 8, 12 — prompt specificity |
| Latent attention / KV cache | Long, cheap, genuinely usable context; cache behaviour as the dominant cost lever | Lessons 6, 11 — cost and context engineering |
| RL-learned reasoning | Visible, steerable, sometimes wasteful thinking | Lessons 9, 18 — effort control and the build loop |
| Native vision | One request path for text and images | Lessons 14, 24, 25 — design and verification |
Checkpoint
- Why does a huge parameter count not translate into a proportional inference bill?
- What grows faster than the model weights during a long agent session, and why does that matter?
- Give one task that thinking mode actively makes worse.
- What does "native" add over a separate vision model, practically?
Answers
- Mixture-of-experts routing activates only a small fraction of parameters per token. Capacity and per-token cost are decoupled.
- The KV cache — the model's stored keys and values for prior tokens. At long context it exceeds the weights, so memory and cost become about cache management, not model size.
- Formatting, extraction, classification, trivial rewrites. Deliberation adds cost and latency without improving the answer.
- No second model name, no second endpoint, and no translation layer — images travel in the same message array as text, so visual work is just another call.
Exercise: predict before you test
Write down, right now, your predictions for two requests against deepseek-flash:
- A request that classifies a support ticket into one of five categories.
- A request that debugs a failing function given the code and the error message.
For each, predict: will thinking mode help or hurt? Will the answer be the same with thinking off? Roughly how many output tokens will each produce?
Then actually run both, with thinking on and off, and record the wall-clock time, output token count and answer quality for each. You now have a small, personal version of the decision table from Lesson 9 — built from your own measurements rather than taken on faith.
Next
Lesson 4 takes the architecture you just learned and turns it into an honest capability map — including the benchmarks, read critically.