Skip to content

Budgeting web data

Prompt caching for fetched pages: one copy, many prompts

Lay out prompts so a fetched page that many questions share is billed at the cached input rate: stable page first, one stored copy, question last.

By Curtis Mccann, Live Web Data team · Published

This article was drafted with AI assistance from the sources listed below and edited and published by the Live Web Data team. Disclosure: the Live Web Data API, which this article mentions, is this site's own product.

Make a shared page cacheable
  1. 1One stored copyRead the page once and reuse the same text, not a new live read per prompt.
  2. 2Stable part firstInstructions, then the page with its URL, country and fetch time.
  3. 3Question lastAnything that changes per request goes after the page.
  4. 4Check the usage fieldscache_read_input_tokens or cached_tokens show whether the prefix was reused.

A support assistant answers questions about one public pricing page. Two hundred users ask about it in an hour. Each prompt carries the same page, so the app pays for the same few thousand input tokens two hundred times. Prompt caching is the provider feature that bills the repeated part at a lower rate, but only if every prompt starts with exactly the same text. Pages read from the live web often do not, and the reason is in how the page gets into the prompt.

How prompt caching decides what to reuse

Anthropic and OpenAI both cache the start of a prompt, not arbitrary pieces of it.

  • Anthropic caches the prefix in the order tools, system, then messages, up to the block you mark with cache_control. A hit needs the segments to be 100 percent identical up to that block. You can set up to 4 breakpoints, or add one top-level cache_control field and let the API place the breakpoint for you.
  • OpenAI turns caching on by default for supported models. Reuse needs the entire rendered prefix to match, and the docs advise putting stable instructions and shared reference material first and dynamic content, such as timestamps or user-specific text, at the end.

The rule that matters for fetched pages is the same in both: one changed character before the cached point, and everything after it is processed as new input.

Why a live read per prompt breaks the cache

The simple design reads the page live for every question and pastes the result into the prompt. That breaks the prefix in three ways.

The page text can differ between reads. Pages can carry parts that change from one read to the next, such as a "last updated" line, a rotating banner or a block of related links. Two reads a minute apart can be two different prefixes.

The fetch time changes. If you pass the fetch time to the model, as the post on fetch timestamps in LLM context recommends, a new read means a new timestamp, and the prefix changes even when the page did not.

The question comes first. A prompt that starts with the user's question and then adds the page has a different prefix for every user.

The fix is a decision you already make for freshness: how old may the copy behind this answer be? Read the page once, store the text with its URL, country and fetch time, and reuse that exact stored copy for every prompt within the max age you allow. The cached prefix then stays identical until you decide to read the page again.

Lay out the prompt: stable first, question last

Order the prompt from what never changes to what changes on every request:

[system]   Instructions for answering from sources
[page]     <source url="https://example.com/pricing" country="us" fetched="2026-11-20T08:00:00Z">
           ...stored page text...
           </source>                       <- cache breakpoint here
[user]     The user's question

On Anthropic's API, the marked page block looks like this; the breakpoint sits on the last block you want cached:

{
  "type": "text",
  "text": "<source url=\"https://example.com/pricing\" country=\"us\" fetched=\"2026-11-20T08:00:00Z\">...stored page text...</source>",
  "cache_control": { "type": "ephemeral" }
}

The fetch time stays inside the cached block. It is the time of the stored copy, so it is the same for every prompt that uses that copy, and the model still knows how old the page is.

When several pages go into one prompt, sort them in a fixed order, for example by URL, so the same set of pages always produces the same prefix.

What the numbers look like

Anthropic's docs give the multipliers against the base input price. A 5-minute cache write costs 1.25 times base input, a 1-hour write 2 times, and a cache read 0.1 times, with lower read multipliers on some newer models.

A worked example with labelled assumptions: one stored page of 10,000 tokens, used in 20 prompts within its cache lifetime, on a model with the 0.1 read rate and the 5-minute write.

Without cachingWith caching
First prompt10,00012,500 (write at 1.25)
Next 19 prompts190,00019,000 (reads at 0.1)
Total, in base-input-token equivalents200,00031,500

The page part of those 20 prompts costs about 16% of the uncached amount. The question and the answer are billed as usual.

OpenAI states that reused tokens are billed at a cached input rate "discounted up to 95%." On GPT-5.6 and later, writes cost 1.25 times the uncached input rate for the default lifetime and reads 0.1 times, or 0.05 times on some models; the docs' model-differences table lists no additional cache-write charge on earlier models. Check the current page for the model you use.

Lifetimes, minimums and timing

Lifetime. Anthropic's cache lasts 5 minutes by default and is refreshed at no extra cost each time it is used; a 1-hour lifetime is an option. OpenAI documents around 5 to 10 minutes of inactivity, up to one hour, for the in-memory cache on earlier models, and a default minimum lifetime of 30 minutes on GPT-5.6 and later. A page asked about a few times a day gains little; a page in steady use gains the most.

Minimum length. OpenAI's minimum is 1,024 tokens for GPT-5.6 and later. Anthropic lists minimums from 512 to 4,096 tokens depending on the model, and a shorter prompt is processed without caching and without an error. A short status page may never be cached.

Parallel requests. Anthropic notes that a cache entry only becomes available after the first response starts. If 50 requests with the same page go out at once, send one first and the rest once it has started answering.

Check that it works

Read the usage fields on each response. On Anthropic's API, cache_creation_input_tokens counts tokens written to the cache, cache_read_input_tokens counts tokens read from it, and if both are 0, the prompt was not cached. On OpenAI's API, cached_tokens reports the reused part. Log these next to the URL of the page in the prompt; a page with steady traffic and no cache reads usually has something changing in its prefix.

For the full cost picture, fetch fees plus input tokens, see the guide on web data budgets for AI apps.

Where Live Web Data API fits

Every request fetches the page live; there is no cache. So the stored copy that keeps your prompt prefix stable is yours to keep: store the text with statuses[].fetchedAt and reuse it within your max age. Failed fetches cost 0 credits. The API reference lists the response fields.

Frequently asked questions

What is prompt caching for web pages?

Model providers can reuse the processed start of a prompt when the next request begins with exactly the same content. If a fetched page sits at the start of many prompts, the repeated part can be billed at a reduced cached input rate instead of the full input price.

Why does my fetched page not hit the prompt cache?

The prefix must match exactly. A new live read can return slightly different text, a fetch time in the prompt changes with every read, and a question placed before the page changes the prefix. Reuse one stored copy and put the question after the page.

How long does a prompt cache last?

It depends on the provider and model. Anthropic's default lifetime is 5 minutes, refreshed at no extra cost each time the cached content is used, with a 1-hour option. OpenAI documents around 5 to 10 minutes of inactivity, up to one hour, for the in-memory cache on earlier models and a default minimum of 30 minutes on GPT-5.6 and later.

Is a short page worth caching?

Often it cannot be cached at all. Each provider has a minimum cacheable length: OpenAI states 1,024 tokens for GPT-5.6 and later, and Anthropic lists minimums from 512 to 4,096 tokens depending on the model. Anthropic processes a shorter prompt without caching and returns no error.

Sources

  1. Prompt caching (Claude Docs) (Documentation)
  2. Prompt caching (OpenAI API docs) (Documentation)