Token cost of web pages: how many tokens one page really is

Every page that enters a context window is paid for twice: once to fetch it and once as input tokens. The web data budget guide calls the second one the token bill and leaves one line for you to fill in: tokens per page. That line is the hardest to guess, because web pages do not have a typical size. They have a median, a mean far above it, and a tail more than 40 times longer than the median page.

What one large sample measured

In December 2025, DEJAN AI published token counts for 44,684 web pages: articles, papers, product pages, documentation and PDFs, counted with Gemini's tokenizer.

Two limits before you reuse these numbers. The write-up does not say how the pages were chosen, and it does not say whether a count covers only a page's main content or everything on it. Treat the table as the shape of the problem, not as a figure for your own pages.

The shape is the useful part. The mean is more than three times the median. A budget built on the mean overstates a typical request; one built on the median misses what the long pages cost.

Format moves the number more than the page does

The same page costs very different amounts depending on what you send. Cloudflare's February 2026 launch post for its Markdown for Agents feature uses itself as the example: 16,180 tokens as HTML and 3,150 tokens as Markdown. That is one page, measured by the vendor that sells the conversion, so read it as an illustration. It shows the mechanism, though: Markdown keeps headings, lists and links and drops tags, attributes and scripts. The Markdown, text or HTML guide covers what each format keeps.

A third variable is the tokenizer. The sample above was counted with Gemini's; other models split the same text into a different number of tokens, so a budget measured on one model is a starting point for the next one, not a result.

Budget the median, cap the tail

A rule that holds up when sizes skew this far:

1. Estimate with the median of your own sample, times page reads per month. That is your typical bill.
2. Cap the tail with a character limit per page type, so one very long page cannot cost as much as 40 typical ones.
3. Check against the mean after the cap. If the capped mean is still far above the median, the cap is too loose for your sources.

A worked example with one labelled assumption: an app reads 10,000 pages a month. At the sample's median, that is 32,010,000 input tokens (10,000 × 3,201). At the sample's mean, it is 104,030,000 (10,000 × 10,403). Same pages, a threefold difference in the forecast, depending only on which number you picked. Multiply both by your model's input price per million tokens, and plan for the higher figure until a cap and your own measurements bring it down.

Turn a token ceiling into a character cap

The budget guide shows how to measure tokens per page on ten of your own URLs. Keep two more numbers from that same sample:

1. The longest page. If it is far above the median, your sources have a tail like the one above, and a cap will matter.
2. Characters per token. Divide each page's character count by its token count and take the median. It differs by language and by how much code or markup a page carries, so measure it rather than borrowing a rule of thumb.

Then pick a token ceiling per page type and multiply it by your characters per token. That product is the character cap to set. In a distribution like the sample's, a ceiling near the 75th percentile (8,207 tokens) leaves about three in four pages untouched and cuts only the long ones. Check after the cut that the answers you need are still on the page.

What this means for your app

• Put tokens per page in the budget as a measured median, not a guess.
• Set a character cap per page type: tight for pricing and status pages, looser for docs.
• Recount when you change models or formats.
• Record when a page was cut, so a truncated copy never passes as a complete one.

How this maps to Live Web Data API

Each result carries the page as markdown and text, and as html when you set html: true, so one fetch gives you every format to count. text.verbosity: "compact", the default, returns the main content only. text.maxCharacters cuts Markdown and text at the limit you set, and results[].textTruncated is true when it did. Failed fetches cost 0 credits. The API reference lists every option and its limits.