data2prompt: CLI Tool Fits Data-Heavy Projects into LLM Context Windows

data2prompt is an open-source CLI tool that converts data-heavy projects—CSVs, Excel workbooks, SQLite databases, and Jupyter notebooks—into a single structured document sized for an LLM's context window. Unlike generic repo-to-prompt tools that treat data files as plain text and produce multi-megabyte, unusable dumps, data2prompt profiles every table on the full dataset (schema, dtypes, missing-value stats) and pairs that with a seeded random sample of real rows.

A --budget flag lets engineers target a specific token count; the tool runs a de-escalation ladder that shrinks samples, drops statistics blocks, and omits heavy files as needed, re-rendering and re-counting tokens at each step rather than estimating. Every reduction, truncation, or redaction is disclosed inline using a consistent notice format, so the model always knows what it isn't seeing.

SQLite files are opened read-only with full CREATE TABLE DDL preserved, and columnar formats (Parquet/Feather/Arrow) are supported via an optional pyarrow extra. For engineers building LLM-driven data analysis or code-generation workflows, the tool addresses a concrete gap: making real datasets fit within context limits without losing statistical fidelity or silently truncating information.