<!--
Author: Simon-Pierre Boucher
Contact: contact@spboucher.ai
-->

# XML Recipes

## Contents
- Create a document with namespaces
- Read and extract (XPath)
- Modify: insert, remove, rename, move
- Validate against an XSD
- Streaming huge files
- Convert XML ↔ dict/JSON
- Gotchas

## Create a document with namespaces

```python
import xml.etree.ElementTree as ET

NS = "http://example.com/catalog"
ET.register_namespace("", NS)  # default namespace, no prefix in output

root = ET.Element(f"{{{NS}}}catalog")
book = ET.SubElement(root, f"{{{NS}}}book", {"id": "bk101"})
ET.SubElement(book, f"{{{NS}}}title").text = "XML Developer's Guide"
ET.SubElement(book, f"{{{NS}}}price").text = "44.95"

ET.indent(root)                     # 2-space pretty print (3.9+)
ET.ElementTree(root).write("catalog.xml",
                           xml_declaration=True, encoding="utf-8")
```

## Read and extract (XPath)

ElementTree supports a *subset* of XPath — child paths, `//`, `[@attr]`, `[tag='text']`, positional `[1]`:

```python
tree = ET.parse("catalog.xml")
root = tree.getroot()
ns = {"c": "http://example.com/catalog"}

first = root.find("c:book[1]", ns)                    # first book
cheap = root.findall(".//c:book[c:price='44.95']", ns)
ids = [b.get("id") for b in root.findall(".//c:book", ns)]
text = root.findtext(".//c:book/c:title", default="", namespaces=ns)
```

Full XPath 1.0 (functions, `contains()`, axes) needs lxml:

```python
from lxml import etree
root = etree.parse("catalog.xml").getroot()
titles = root.xpath("//c:book[contains(c:title,'Guide')]/c:title/text()",
                    namespaces={"c": "http://example.com/catalog"})
```

## Modify: insert, remove, rename, move

```python
tree = ET.parse("catalog.xml")
root = tree.getroot()
ns = {"c": "http://example.com/catalog"}

# Insert after an existing child (ElementTree has no insert-after: use index)
books = root.findall("c:book", ns)
new = ET.Element(f"{{http://example.com/catalog}}book", {"id": "bk102"})
root.insert(list(root).index(books[-1]) + 1, new)

# Remove — you must remove from the PARENT
for bad in root.findall("c:book[@id='bk101']", ns):
    root.remove(bad)

# Rename a tag
for el in root.iter(f"{{http://example.com/catalog}}price"):
    el.tag = f"{{http://example.com/catalog}}cost"

tree.write("catalog.xml", xml_declaration=True, encoding="utf-8")
```

To find a parent when you only matched the child (stdlib has no `getparent()`):

```python
parents = {c: p for p in root.iter() for c in p}
parents[child].remove(child)
```

## Validate against an XSD

Requires lxml (`pip install lxml`):

```python
from lxml import etree

schema = etree.XMLSchema(etree.parse("catalog.xsd"))
doc = etree.parse("catalog.xml")
if not schema.validate(doc):
    for err in schema.error_log:
        print(f"line {err.line}, col {err.column}: {err.message}")
```

Report every error with line/column; fix the tree, re-serialize, re-validate.

## Streaming huge files

Load-then-clear keeps memory flat regardless of file size:

```python
import xml.etree.ElementTree as ET

for event, elem in ET.iterparse("huge.xml", events=("end",)):
    if elem.tag.endswith("record"):
        process(elem)
        elem.clear()   # release children; keeps memory bounded
```

## Convert XML ↔ dict/JSON

No stdlib one-liner exists; for flat, repetitive data write it explicitly:

```python
rows = [{"id": b.get("id"), "title": b.findtext("c:title", "", ns)}
        for b in root.findall(".//c:book", ns)]
import json; json.dump(rows, open("books.json", "w"), indent=2)
```

For arbitrary nesting, `pip install xmltodict` and `xmltodict.parse(open("f.xml", "rb"))` — but beware: single children become dicts, repeated children become lists, so downstream code must handle both shapes.

## Gotchas

- **`ns0:` prefix pollution.** ElementTree invents `ns0:` prefixes on write unless you call `ET.register_namespace(prefix, uri)` *before* parsing/writing. Register `""` for a default namespace.
- **Truthiness trap.** An element with no children is falsy: `if elem:` is False even when the element exists. Always use `if elem is not None:`.
- **`find()` vs `findall()` with no namespace map** silently return nothing on namespaced documents — the tag you search must be `{uri}tag` or use the `ns` map. "It finds nothing" almost always means a missing namespace map.
- **No parent pointers** in stdlib ElementTree — removal requires the parent (see recipe above); lxml elements have `.getparent()`.
- **Comments and processing instructions are dropped** by the default ElementTree parser. If they must survive a round-trip, use lxml, which preserves them.
- **`ET.tostring()` omits the XML declaration** unless `xml_declaration=True` — and returns bytes when an encoding is given.
- **Entity attacks (billion laughs, external entities).** Never parse untrusted XML with plain ElementTree/lxml defaults; use defusedxml drop-ins (`defusedxml.ElementTree.parse`).
- **`ET.indent()` mutates text nodes** — don't use it on documents where whitespace inside elements is significant (mixed content).