---
name: processing-html
description: Creates, reads, and modifies local HTML files — extracting text, tables, and links, or editing markup with BeautifulSoup. Use when the user asks to parse, scrape data out of, edit, clean up, or generate an .html or .htm file, extract a table or links from saved HTML, or build a static HTML page. Do not use for fetching live web pages (that is web browsing/scraping) or for XML data files.
---
# Processing HTML
## When to use / when NOT to use
- **Use for:** local `.html`/`.htm` files — extracting text, tables, or links; targeted markup edits; generating static pages from scratch.
- **Do NOT use for:** fetching live web pages (that is web browsing/scraping — get the file first, then this skill applies) or XML data files (use the XML skill — XML parsers are strict, HTML parsers are tolerant).
## Quick reference
**Default:** BeautifulSoup4 with the built-in `html.parser` (`pip install beautifulsoup4`). **Escape hatch:** the `lxml` parser (`pip install lxml`) for large or badly malformed files — faster and more lenient.
```python
from bs4 import BeautifulSoup
soup = BeautifulSoup(open("page.html", encoding="utf-8"), "html.parser")
# Read / extract
title = soup.title.get_text(strip=True) if soup.title else ""
links = [(a.get_text(strip=True), a["href"]) for a in soup.find_all("a", href=True)]
rows = [[c.get_text(strip=True) for c in tr.find_all(["td", "th"])]
for tr in soup.select("table tr")] # lists-of-lists
# Modify — targeted, leave everything else untouched
for img in soup.find_all("img", src=True):
if not img.get("alt"):
img["alt"] = ""
open("page.html", "w", encoding="utf-8").write(str(soup))
```
**Create** — write HTML5 directly, no library:
```html
Page title
…
```
## Rules
- **Never regex-parse HTML.** Use the soup — selectors and tree navigation, always.
- Edit surgically: modify the matched tags only; do not re-indent or `prettify()` an existing file (it rewrites every text node's whitespace).
- New pages are HTML5: doctype, ``, semantic tags (`main`, `nav`, `article`), `lang` attribute.
- Extract tables as lists-of-lists; flag `rowspan`/`colspan` cells instead of silently mis-aligning columns.
## Workflow
1. Identify the operation: extract / modify / create.
2. Parse with `html.parser`; if the file is huge (>5 MB) or the tree looks wrong (missing siblings, truncated body), reparse with `"lxml"`.
3. Perform the operation with the narrowest selector that matches (recipes in references/recipes.md).
4. Write back with the file's original encoding.
5. **Validate:** re-parse the written file and confirm the edit landed (query the changed element) and the element count of untouched regions is unchanged. Fix and repeat until clean.
## Edge cases & failure modes
- **bs4/lxml missing** → `pip install beautifulsoup4` / `pip install lxml`.
- **Malformed HTML** → parsers auto-repair rather than error; if extraction returns nothing, the repaired tree may differ from the source — try the `"lxml"` parser and inspect `soup.prettify()[:2000]` to see the actual tree before concluding data is absent.
- **Encoding** → check `` before assuming UTF-8; on `UnicodeDecodeError`, reopen with that charset or pass raw bytes to BeautifulSoup and let it detect.
- **JavaScript-rendered content** → if the data isn't in the file, it never was: say so — no parser will find DOM built at runtime.
- **Fragments** (no ``/``) → parse and write back as-is; do not let output gain wrapper tags the input lacked (html.parser doesn't add them; lxml does).
## References
Deeper copy-paste recipes (selectors, table→CSV, rewriting links, sanitizing): see [references/recipes.md](references/recipes.md).