The Cost Model: Cache Hits, Off-Peak, and Penny Builds

The price table

All figures are per million tokens, for deepseek-flash. Off-peak is exactly half of peak.

Read the ratio column and notice it is not a gentle gradient. A cache hit is fifty times cheaper than a cache miss. Output is two hundred times the price of a hit.

For comparison, deepseek-v4-pro runs roughly four times these input rates with output at $1.98 off-peak / $3.96 peak, and has no vision support. That is the entire cost argument for making Flash your default.

Peak and off-peak: the schedule

Peak hours are 01:00–04:00 and 06:00–10:00 UTC, Monday through Friday. Every other hour is off-peak. Off-peak rates are half of peak.

Two practical observations:

• Peak is short. Seven hours a day, weekdays only. Weekends are entirely off-peak. A large majority of any week is discounted, including everything outside a five-hour window in most Western working hours.
• Batch work naturally lands off-peak. Overnight jobs, scheduled rebuilds, and long document processing are almost always in the cheap window. If you are doing something expensive, it is usually only a question of when you schedule it.

Prefix caching: the idea

Here is the mechanism, in the simplest form that is still accurate.

When you send a request, the provider processes your tokens. For the first tokens — typically a stable prefix such as a long system prompt, a document you always include, or a tool schema — it can store the internal state and, on a later request that begins with the exact same prefix, reuse it instead of recomputing. That reuse is billed as a cache hit, at one-fiftieth the price.

Caching works on the prefix. That word is doing all the work. The token sequence must match from the start, in order, exactly.

What makes a cache hit

• Identical leading token sequence, byte for byte
• Sent within whatever lifetime the provider keeps the cache warm
• Same account, same endpoint

What destroys a cache hit

• Putting anything variable at the top. A timestamp, a session ID, a username, "Today's date is…" — the moment a changing token appears early, everything after it is a miss.
• Reordering. Moving a section changes the sequence. Even a pure improvement in wording invalidates the prefix from the first changed word onward.
• Appending above the stable content. New instructions belong below the stable block, never above it.

The two-layer prompt

[ LAYER 1 — STABLE ]  ← never changes, always cached
  system instructions
  tool / function schemas
  style guide, house rules
  reference documents (large, static)
  few-shot examples

------- the boundary below this line costs full price -------

[ LAYER 2 — VARIABLE ]  ← changes every call
  the actual user request
  today's data
  the file currently being discussed

Done properly, a large recurring instruction block is paid for once and then costs almost nothing for the rest of the session. Done badly — with a timestamp at the top — it costs full price on every single call, forever, invisibly.

The order of magnitude of a hit versus a miss

Concretely, imagine a 200,000-token context you send on every call of a long session, fifty times.

Same tokens. Same model. Same task. A fifty-fold difference created purely by whether the leading content stayed stable.

And output dominates everything: 10,000 output tokens costs $0.006 off-peak, which is more than a million cached input tokens. If you want to spend less, the highest-leverage question is usually not "how do I shrink the input?" but "do I need this much output?"

Three rules to carry forward

1. Stable first, variable last. Never let a changing value sit above stable content in your prompt.
2. Watch the cache split, not the total. Read prompt_cache_hit_tokens versus prompt_cache_miss_tokens. If hits are consistently near zero on a repeated workload, you have a cache bug — and it is silent.
3. Schedule deliberately, cap output deliberately. Off-peak halves the bill; constraining output size cuts the most expensive dimension.

Exercise: make a workload fifty times cheaper

1. Measure the bad version. Build a prompt with a 2,000-token stable instruction block and one variable line, but put a timestamp at the very top. Send it ten times. Record the cache hit and miss token counts from usage each time.
2. Fix the architecture. Move the timestamp to the end, below the variable line. Send it ten more times.
3. Compare and compute. Calculate the actual cost of each run using the off-peak table. Expect the hit tokens to appear from the second call onward in the fixed version — the first call populates the cache.
4. Time it. Note the wall-clock difference. Cache hits are cheaper and faster, which is a latency win as well as a cost one.

You now have a personal, measured demonstration of the single largest cost lever in this course.