Guide · Clean page content
Markdown vs HTML for LLMs: which page format to pass
Markdown vs HTML for LLM context, plus plain text: what each format keeps, how much of the page to pass, tables, metadata and which URL to cite.
Guide · Updated
This article was drafted with AI assistance from the sources listed below and edited and published by Teneo Labs. Disclosure: this site is run by Teneo Labs, the company behind Live Web Data API, which this article mentions.
| Criterion | Markdown | Plain text | HTML |
|---|---|---|---|
| Keeps headings and lists | Yes | No, words only | Yes, as tags |
| Keeps links | Yes, inline | No | Yes, with attributes |
| Tables | Simple tables survive | Flattened | Fully preserved |
| Token use | Low | Lowest | Highest |
| Best for | Reading, summaries, citations | Classification, short checks | When structure is the data |
A model reads exactly what you give it. Pass a page as raw HTML and it reads class names, scripts and menus along with the article. Pass it as clean Markdown and it reads the headings, paragraphs, lists and links. This guide covers what each format keeps, when the full page matters, and which metadata to send with the content.
The three formats
Markdown keeps the structure a reader uses: headings, lists, emphasis, links, code blocks and simple tables. It drops tags, attributes and scripts. For reading, summarizing and citing, it is the best default.
Plain text keeps only the words, with whitespace normalized. It is the cheapest in tokens (the web data budget guide shows how to put that token bill into numbers) and the right choice when structure does not matter: classification, language detection, keyword checks.
HTML keeps the page as served. It costs the most tokens and carries the most noise, but it is the only format that preserves attributes, forms and complex tables. Use it when the structure is the data.
There is an honest counterargument: some practitioners report that cleaned HTML works well because its closing tags make structure explicit. That is a reasonable choice for structure-heavy pages. The bigger gain, in either format, comes from removing what is not content.
Main content vs full page
The second decision is how much of the page to pass. It matters as much as the format.
- Main content (Live Web Data API's
verbosity: "compact", the default) returns the main content only. Use it for almost everything. - Full page (
verbosity: "full") keeps the whole body, including navigation and footer. Use it when the question is about those parts: a legal notice in the footer, a site's menu structure, or a banner's wording.
Navigation text in the context is not just a cost problem. Menus full of product names and "related articles" lists give the model plausible-looking text to answer from that is not the page's topic.
Tables
Simple tables convert cleanly to GitHub Flavored Markdown: one header row, one value per cell. Conversion struggles with:
- merged cells (
rowspan,colspan), - nested tables,
- tables used for page layout rather than data,
- cells that contain lists or several paragraphs.
If a page's value is in a complex table, for example a detailed feature matrix, request HTML for that page and let the model read the table markup, or parse the table yourself before prompting.
Metadata worth keeping
The content is half the story. Send the model the context that lets it judge and cite the source:
| Field | Why it matters |
|---|---|
| Final URL | What to cite; the address after redirects |
| Title | Lets the model refer to the source by name |
| Published date | Distinguishes an old article from a current one, when the page exposes it |
| Author | Useful for attribution, when the page exposes it |
| Fetch time | How current the content is; record it yourself |
| Country | Which regional version was read |
Live Web Data API returns url, title, author, publishedDate, image and favicon per result, with null when a page does not expose a value. Do not invent missing metadata; pass null or leave the field out.
Cite the final URL
Requests are often redirected: to HTTPS, to a trailing slash, to a localized path or to a page's new home. The address you requested is not always the address a reader should open. Use the final URL (results[].url) when you show sources, and keep the requested URL (results[].id) for your own logs and cache keys.
JavaScript-only pages
Some pages send almost no content in their HTML and build it in the browser. A static fetch cannot turn those into useful Markdown. Live Web Data API flags them with a THIN_CONTENT warning or a CRAWL_NOT_RENDERED error (0 credits); set render: true, or renderFallback: true to render only when needed, and the page is read in a browser before it is converted. Our sister site has a guide on whether you need a headless browser for a given page.
Where Live Web Data API fits
Each successful result includes the page as Markdown and plain text, and as HTML when you set html: true, so you can choose per task without fetching twice. text.verbosity switches between main content and full page, and text.maxCharacters caps the length of Markdown and text. Compare the formats on your own pages in the playground, or see the response fields in the API reference.
Checklist
- Markdown is the default; HTML is an explicit per-page exception.
- Plain text is used only where structure does not matter.
- Main content is the default; full page only when the question is about navigation or footers.
- Pages with complex tables are flagged and handled as HTML.
- Final URL, title, fetch time and country travel with the content.
- Missing metadata stays missing.
Frequently asked questions
Is Markdown better than HTML for LLMs?
For reading, summarizing and citing, usually yes. Main-content Markdown keeps the structure a model uses (headings, lists, links, simple tables) and drops markup, scripts and navigation, so it costs far fewer tokens. Keep HTML when attributes or exact structure are what you need.
When should I use plain text instead of Markdown?
When structure does not matter and tokens do: classification, language detection, keyword checks or short yes-or-no questions. For anything that cites or summarizes sections, the headings in Markdown help.
Do HTML tables survive conversion to Markdown?
Simple tables with one header row usually do. Merged cells, nested tables and tables used for layout often lose structure. If the table is the data, request HTML for that page.
Should I remove navigation and footers before sending a page to an LLM?
Usually. They add tokens and can pull the answer off topic. Keep the full page only when the question is about those parts, such as a footer's legal notice or a site's navigation.
Which URL should the model cite?
The final URL after redirects, not the one you requested. It is the address a reader will land on when they follow the citation.
Sources
- GitHub Flavored Markdown spec (Documentation)
- The /llms.txt file proposal (Documentation)