· Freshness
Tell the model how old the page is: fetch timestamps in context
A model cannot see when a page was read. Put the fetch time, URL and country next to the content, in one format, and tell the model how to use them.
Live web data notes: headers, timestamps and tokens
One question per post, answered with sources you can check. Each note links up to the guide that covers the whole decision.
RSS feedFreshness and caching
· Freshness
Last-Modified and ETag: what page headers say about freshness
Last-Modified and ETag tell a cache whether a copy changed, not when the content was written. How reliable each one is, and what to store instead.
Budgeting web data
· Budget
Token cost of web pages: how many tokens one page really is
In one 44,684-page sample the median page was 3,201 tokens and the 99th percentile 141,410. How to budget the token bill when page sizes skew that far.
Clean page content
· Updated · Clean content
Final URL or canonical URL: which address an LLM should cite
A fetched page has up to three addresses. Which one to pass to the model, which one to key stored copies by, and what redirects reveal about country and age.