Simon Willison has released datasette-arrows 0.1a0, a plugin that extends Datasette 1.0a43 to support streaming exports in the Apache Arrow IPC format. The core application already streams CSV and TSV files so users can download millions of rows without loading them entirely into memory. This new tool applies that same pattern to Arrow, allowing data to flow in chunks suitable for high-performance tools like DuckDB, pandas, and Polars. Willison followed his usual practice of pairing a feature with an immediate demonstration, noting that Claude Fable 5.2 suggested the format and built the plugin quickly. The export dialog now lists this option alongside existing streams, offering a Download button and a Copy URL link for direct use by analysis software.
The significance lies in efficiency for large datasets, as the format avoids memory bottlenecks common with standard row-by-row processing. It enables analysts to pass data directly into computational environments without intermediate conversion steps. This approach maintains the lightweight nature of the application while expanding its utility for serious data work.
* Supports streaming for DuckDB, pandas, and Polars
* Adds an Arrow option to the export dialog
* Built in response to a suggestion from Claude Fable 5.2

