spb/ultra-sharp-agent-skills Public

1---2name: processing-csv3description: Creates, reads, modifies, and converts CSV and TSV files — filtering rows, adding or renaming columns, changing delimiters, and converting to or from JSON. Use when the user asks to read, write, edit, filter, sort, clean, split, or convert a .csv or .tsv file, mentions comma- or tab-separated data, or asks to turn CSV into JSON or JSON into CSV. Do not use for data-quality profiling or auditing a CSV (a separate profiling-csv-data skill covers that) and not for .xlsx Excel files.4---56<!--7Author: Simon-Pierre Boucher8Contact: contact@spboucher.ai9-->1011# Processing CSV1213## When to use / when NOT to use14- **Use for:** any task where a `.csv` or `.tsv` file is the input or output — creating, editing, filtering, converting.15- **Do NOT use for:** profiling/auditing data quality (use `profiling-csv-data`) or `.xlsx` Excel files (use `processing-xlsx`).1617## Quick reference — one default per operation1819**Read — stdlib `csv`, always `newline=""`:**20```python21import csv22with open("in.csv", newline="", encoding="utf-8") as f:23    rows = list(csv.DictReader(f))        # rows as dicts keyed by header24```2526**Write:**27```python28with open("out.csv", "w", newline="", encoding="utf-8") as f:29    w = csv.DictWriter(f, fieldnames=["id", "name"])30    w.writeheader()31    w.writerows(rows)32```3334**Unknown delimiter — sniff it:**35```python36with open("in.csv", newline="", encoding="utf-8") as f:37    dialect = csv.Sniffer().sniff(f.read(4096))   # 4 KB is plenty to detect the delimiter38    f.seek(0)39    rows = list(csv.DictReader(f, dialect=dialect))40```4142**TSV:** same code with `delimiter="\t"` passed to the reader/writer.4344**Escape hatch — pandas** for large files (>~1 GB) or typed/numeric operations:45```python46import pandas as pd                        # pip install pandas47df = pd.read_csv("in.csv", dtype=str)      # dtype=str avoids silent type coercion48```4950## Rules51- Always open CSV files with `newline=""` — omitting it doubles line breaks on Windows.52- The `csv` module quotes fields containing delimiters/quotes/newlines automatically — never hand-assemble CSV with string joins.53- Keep the header row unless the user explicitly wants it gone.54- Modify via read-all → transform → write-new (or temp file + `os.replace` to edit in place); never patch a CSV with regex.5556## Workflow571. Read the input with `DictReader` (sniff the dialect if the delimiter is unknown).582. Transform in Python (filter/map on the list of dicts).593. Write the result with `DictWriter`, explicit `fieldnames`.604. **Validate:** re-open the output, check the header matches `fieldnames` and the row count equals what the transform should produce. Fix and rewrite if not.615. Report output path, row count in → row count out, and columns changed.6263## Edge cases & failure modes64- **Missing dependency** → only pandas is third-party: `pip install pandas`.65- **Malformed row (wrong field count)** → `DictReader` puts extras under `None`/fills missing with `None`; count such rows, report them, and ask before dropping — don't silently discard data.66- **Non-UTF-8 file** → `UnicodeDecodeError`; retry with `encoding="latin-1"` and tell the user which encoding was used.67- **Huge file** → stream row-by-row (iterate the reader, write as you go) instead of `list(...)`; or use pandas with `chunksize`.68- **Empty file / header-only** → report it plainly; output a header-only file if a transform was requested.69- **Values with leading zeros or big IDs** (postal codes, phone numbers) → keep them as strings; that's why the pandas escape hatch uses `dtype=str`.7071## References72Deeper recipes (filter/sort/dedupe, column ops, csv↔json, delimiter conversion, gotchas): see [references/recipes.md](references/recipes.md).73