Prompt Caching Explained: Faster Repeated Context

Prompt caching stores a reusable prefix of your API request so later calls with the same opening content skip reprocessing that chunk. You pay less for those reused tokens and get a faster first token when the cache hits. The feature helps chatbots with a fixed system prompt, agents with the same tools every turn, and apps that ask many questions against one long document or codebase. Chat UIs may hide the machinery; API builders control the layout.

You will leave knowing what vendors cache, how to structure prompts for hits, when caching wastes money, and mid-2026 notes for Claude Fable 5, Opus 5, and Sonnet 5, GPT-5.6 Sol, Terra, and Luna, plus Gemini 3.5 Flash and Gemini 3.1 Pro.

What prompt caching is (and who it helps)

A normal API call sends the full input every time: system rules, tool schemas, long context, then the new user message. The model (or its serving stack) must process that whole prefix before it can answer. Prompt caching keeps a processed form of the shared prefix on the provider side for a short window. The next request that starts with the same bytes reuses that work instead of rebuilding it from scratch.

Think of a script binder. You print the company handbook once and clip new questions on top. Without caching, you reprint the handbook for every question. With caching, the binder stays open on the desk for a few minutes (or up to an hour on some plans), and you only pay the cheap rate to flip back to it.

Value shows up for:

• Builders who ship the same system prompt and tool list across many turns
• Product teams that run dozens of queries against one uploaded PDF, repo, or knowledge pack
• Agents that keep a large instruction block stable while the user message changes
• Support and ops bots with a thick policy document that stays fixed mid-day

Who should treat it as optional: one-off chat with short prompts, experiments where every request mutates the system text, and workloads below the vendor minimum token count for a cacheable prefix. Caching is an infrastructure win for repeated context. It does not make a weak prompt smart.

How prompt caching works in plain English

Vendors cache a prefix, not a vibe. The bytes from the start of the request through a chosen breakpoint must match a prior request. If you insert a timestamp, shuffle tool order, or tweak one word in the system block, the match breaks and you write a new cache entry (or fall back to full-price input). Place stable content first and variable content last. That single layout rule drives most of the savings.

Providers differ on how you turn the feature on. Anthropic asks you to mark cache control on the request or on content blocks. OpenAI applies caching on eligible models with optional keys and breakpoints for tighter control on the GPT-5.6 family. Google Gemini offers implicit hits when prefixes match, plus an explicit cache object you create, name, and reference. The product names differ; the habit is the same: freeze the shared front, append the fresh question at the end.

A cache hit cuts input cost on the reused tokens and often cuts time-to-first-token because the stack skips recomputing attention state for that prefix. A cache miss processes the full input and may charge a write premium on some vendors. Hits only help if you reuse the prefix before the time-to-live (TTL) expires. Quiet gaps longer than the TTL force a fresh write.

What gets cached vs what stays fresh

Cached material is the shared front of the request: tool definitions, system instructions, and earlier messages up to the breakpoint the vendor supports. Fresh material is whatever you change per call: the latest user question, a new document chunk you only need once, or session fields that differ by customer.

Example shape that hits well:

1. Tools and schemas (stable)
2. System prompt and style rules (stable)
3. Long reference text: handbook, codebase summary, product catalog (stable for the session)
4. User question or ticket text (changes every call)

Example shape that misses often: put the user id, "today's date," or a random request id at the top of the system prompt. That one changing line sits inside the prefix, so every call looks new to the cache. Move volatile fields into the final user message.

Cache hits, misses, writes, and TTL

On a miss with caching enabled, the provider processes the prefix and stores it. Anthropic bills that store as a cache write (higher than base input for 5-minute or 1-hour TTL options). OpenAI's GPT-5.6 family also charges cache writes on that generation; older OpenAI families documented automatic caching without a separate write fee. Google explicit caches bill for creating and keeping the cache for the TTL you choose.

On a hit, you pay the cheap cache-read rate for the matching prefix tokens and normal rates for the new tail. Anthropic refreshes the TTL on use for the default ephemeral cache, so a steady stream of hits can keep a 5-minute window alive across a busy session. Leave the app idle past the TTL and the next call pays for a write again. Check usage fields in the API response (cache_read / cached_tokens style counters) so you measure hits instead of guessing.

A step-by-step workflow to get cache hits

Use this when you call the API from code. Chat products may already reuse context behind the scenes; you still want the same mental model when you design system prompts and libraries. The goal is a stable prefix you can prove in logs, not a hope that "long prompts are cheaper."

Budget an hour once to redesign prompt order. Teams that bolt caching onto a prompt that rebuilds the system string every request see write fees without hits. Measure before and after on the same traffic pattern: cache read tokens, cache write tokens, end-to-end latency, and quality on a fixed eval set. Caching should not change answers if the text is identical; if quality drifts, you changed more than order.

If you are still drafting the reusable system block itself, tighten the wording first. A clear, short instruction set that you freeze is easier to cache than a meandering brief you edit every afternoon. PromptMake /text can help you turn a rough policy into a clean system prompt you then lock in code; the cache cares that the locked text stays byte-stable across calls.

Steps 1-3: Inventory, split, freeze

1. List every string you send today: tools, system, few-shot examples, retrieved docs, user text. Mark each as stable for minutes, stable for days, or unique per request.
2. Reorder the request so stable blocks come first and unique blocks come last. Keep tool JSON in a fixed order. Sort keys if your serializer is nondeterministic.
3. Freeze the stable blocks in version control. Bump a prompt version when you edit them. Deploy the new version in one cut so you do not thrash the cache with partial rollouts that alternate two system texts.

At this point you should hold a prefix that is long enough for your model's minimum (often on the order of hundreds to a few thousand tokens; check the current docs for your exact model) and short enough that you are not caching junk you will edit every hour.

Steps 4-6: Enable, traffic, measure

1. Turn on the vendor mechanism: Anthropic cache_control (automatic top-level or explicit breakpoints), OpenAI eligibility plus prompt_cache_key / explicit breakpoints on GPT-5.6-class models when you need sticky routing, or Gemini implicit reuse / explicit cachedContents for a named corpus.
2. Send a warm-up request with the full prefix, then a burst of follow-ups that only change the final user message. Space them inside the TTL so the second call can hit.
3. Read usage metadata. Aim for high cache-read share on the prefix tokens across the burst. If reads stay at zero, check minimum length, prefix mismatch, model support, and whether idle time exceeded TTL.

A healthy hit rate means you can keep the prefix frozen and revise it only behind a version bump. Re-warm after big prompt edits. For Gemini explicit caches, delete or expire old cache objects when the corpus changes so callers do not reference stale names.

Common prompt caching mistakes

The frequent failure is a moving prefix. People put datetime.now(), a per-request UUID, or personalized "User name: …" lines inside the system block. Every call writes; few calls read. Move those fields to the last user message.

Another failure is caching content that is too short. Vendors enforce minimum prefix sizes. A 200-token system prompt may never enter the cache no matter how many times you send it. Either expand the stable context you meant to reuse (docs, examples, tool schemas) or skip caching until the prefix is large enough.

Teams also over-cache. They mark breakpoints on sections that change every turn, pay write premiums, and confuse themselves with noisy metrics. Cache the layer that is shared across many calls. Leave the volatile tail uncached.

Idle gaps kill hit rates on short TTLs. A human-in-the-loop agent that waits fifteen minutes between steps will miss a five-minute Anthropic default cache unless you choose a longer TTL where available, accept the write cost, or redesign the loop to batch work.

Nondeterministic serialization breaks matches: unstable JSON key order, floating whitespace from templates, or tool lists rebuilt from a set. Serialize once, store the canonical string, reuse it.

People also expect caching to fix quality. Caching only speeds and cheapens identical prefixes; better answers still come from better prompts and evals. Identical prefixes should yield the same model behavior aside from normal sampling variance. Use caching to run a strong frozen prompt at volume once the checklist already passes.

Vendor notes for prompt caching in 2026

As of mid-2026, all three major API stacks support some form of prompt or context caching. Exact multipliers, minimums, and TTL options change; treat numbers below as orientation and confirm on the vendor pricing page before you forecast a bill.

Anthropic Claude (Fable 5, Opus 5, Sonnet 5, and related): you enable caching with cache_control. Automatic mode places a breakpoint for you; explicit mode lets you mark blocks when different sections change at different rates. The cache covers the prompt hierarchy tools → system → messages through the breakpoint. Default ephemeral TTL is five minutes and refreshes on hit; a one-hour TTL is available at a higher write multiplier. Cache reads are billed at a steep discount versus base input (on the order of 0.1×). Minimum cacheable lengths vary by model (examples in current docs include ~512 tokens for some Opus 5 / Fable 5 paths and ~1,024 for Sonnet 5). Check the response usage fields for creation vs read tokens.

OpenAI (GPT-5.6 Sol, Terra, Luna, and other eligible models): prompt caching reuses matching prefixes on the API. On the GPT-5.6 family, docs describe cache writes at a premium to uncached input, implicit breakpoints by default, and optional explicit breakpoints plus TTL controls (docs list a 30-minute window as the supported value). A prompt_cache_key improves the chance that traffic with the same prefix lands where the cache lives. Older model generations documented automatic caching above about 1,024 tokens with discounted cached input and no separate write fee. Read cached_tokens / cache_write_tokens style fields in usage to verify behavior for your model snapshot.

Google Gemini (Gemini 3.5 Flash, Gemini 3.1 Pro, and other current Gemini API models): implicit caching can discount matching prefixes without a manual cache object. Explicit context caching lets you upload a corpus once, set a TTL (often defaulting to about an hour if unset), and reference the cache by name on later generateContent calls. Explicit mode is the path when you need a guaranteed discount on a large fixed context. Minimum token thresholds differ by model family; Gemini 3-class limits are higher than some older Flash tiers. Prefer stable ordering: put the shared corpus in the cache or at the front, append the per-request question at the end.

Cross-vendor habit that still holds: identical prefix, variable suffix, measure hits, and avoid editing the frozen block mid-flight. Model swaps invalidate assumptions; a prefix tuned on Sonnet 5 may need a fresh warm-up on Opus 5 or Sol even if the text is unchanged, because caches are scoped to model and account boundaries.

When to use caching (and where PromptMake fits)

Turn caching on when the same large prefix appears in many calls inside the TTL window: multi-turn agents, doc Q&A, code assistants with a fixed repo summary, and customer bots with a stable policy. Skip it for tiny prompts, one-shot scripts, and prototypes that rewrite the system string every run.

Freeze a reusable prompt before you wire breakpoints. Vague system text that you rewrite each day will thrash any cache. Draft the job, constraints, and output shape until the instruction set is boring and complete. If that draft is the hard part, run the rough brief through PromptMake /text, pick the model you will call in production, and paste the enhanced system prompt into your repo as the cached prefix. Soft sell only: the tool helps you write the stable front; the API vendor caches it once you stop editing it.

Ship with a dashboard for cache read ratio and a runbook for prompt version bumps. Sol, Fable 5, or Gemini 3.1 Pro pricing and TTL options will shift over time; you adjust multipliers in the forecast, not the whole product architecture.

FAQ

What is prompt caching in plain terms?

Prompt caching means the API keeps a processed copy of the shared start of your request for a short time. The next call that begins with the same content reuses that copy, so you pay less for those tokens and often wait less for the first output token. The new user question at the end still processes as normal input. You still send the full text; the provider recognizes the matching prefix.

How is prompt caching different from my chat app "remembering" the thread?

Chat products store conversation history so the model can see prior turns. Prompt caching is a billing and speed optimization on repeated prefix tokens inside API requests. History can be long and unique; caching only helps when large identical chunks repeat across calls. You can have chat memory without cache hits, and you can have strong cache hits on a single-turn API that resends the same handbook every time.

When does prompt caching save the most money?

Savings show up when a long stable prefix is reused many times before the TTL expires: the same tools and system prompt across hundreds of tickets, or one big document asked twenty questions in a row. The break-even point depends on write fees versus read discounts. A single reuse can already pay for a modest write multiplier; long gaps that force repeated writes erase the gain. Measure on your traffic rather than assuming every long prompt is cheaper.

Does prompt caching change the model's answers?

It should not change answers when the prompt text is identical. The feature reuses computation for the same tokens; it is not a second model or a summary. If outputs shift after you "enable caching," look for accidental prompt edits, different models, temperature, or nondeterministic tool payloads. Keep an eval set and compare cached vs uncached runs on the same inputs when you first adopt the feature.

Which models support prompt caching in 2026?

As of mid-2026, plan around Claude Fable 5, Opus 5, and Sonnet 5 with Anthropic's cache_control; GPT-5.6 Sol, Terra, and Luna (and other eligible OpenAI API models) with OpenAI prompt caching; and Gemini 3.5 Flash and Gemini 3.1 Pro with Gemini implicit and explicit context caching. Always confirm the exact model id on the vendor's caching page, because minimum lengths and write pricing differ by snapshot. A chat-only surface may hide caching knobs even when the API for the same model family supports them. Build against the API docs for the id you deploy, not against the consumer chat UI label.

Why are my cache read tokens always zero?

Common causes: the prefix is under the model minimum, something near the front of the prompt changes every call, the idle time exceeded TTL, you are on a model or endpoint that does not support caching, or (on OpenAI) requests with the same prefix are not sticking to the same cache locality without a prompt_cache_key. Log the exact serialized prefix hash in staging. Warm once, then send a second call within the TTL that only changes the final user message. If reads stay at zero after that control test, compare your request shape to the vendor cookbook example for the same model id.

How do I start without overbuilding?

Pick one production path that already resends a large system prompt. Freeze that string in git, move volatile fields to the last message, enable the simplest caching switch your vendor offers, and watch usage for a day. If you need a cleaner system prompt before you freeze it, draft it with PromptMake /text on the free tier, then lock the result and add caching in code. Expand to explicit breakpoints or named Gemini caches only after the simple path shows real hit rates.