Repository navigation
Faster ?_stream=1, export dialog groups formats that can stream - #3070
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3070 +/- ##
=====================================
Coverage 0.00% 0.00%
=====================================
Files 80 80
Lines 14282 14285 +3
=====================================
- Misses 14282 14285 +3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Streaming every row previously paginated using the default page size
of 100, so a large export ran ten times more queries than it needed.
It now uses the largest page size unless ?_size= is in the URL.
Microbenchmark: /db/table.csv?_stream=1 for a 200,000 row, four column
table (7MB of CSV), median of five runs against a local server:
100 row pages (before): 5.21s
1,000 row pages (after): 1.09s
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Bvfq9SBhxPegj4hvi5NF4
Plugin output formats that accept row_stream now share the form used by TSV and CSV, so the "Include all rows" and "Expand labels" options apply to them. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Bvfq9SBhxPegj4hvi5NF4
simonw
force-pushed
the
claude/funny-noether-7rc2rv
branch
from
November 18, 2026 15:32
dcc61cf to
defca32
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two follow-ups to the streaming output renderers in #3069, found while building datasette-arrows against main.
?_stream=1fetches pages ofmax_returned_rowsClaude Code
?_stream=1fetches pages ofmax_returned_rowsStreaming every row paginated using the default page size of 100, so a large export ran ten times more queries than it needed. It now uses the largest page size.
?_size=in the URL is still respected.sizeset in table configuration no longer applies to a streaming request.row_stream.pages()therefore yields pages of up tomax_returned_rowsrows when streaming.Microbenchmark:
/db/table.csv?_stream=1for a 200,000 row, four column table (7MB of CSV), median of five runs against a local server:The "before" figure was measured by adding
&_size=100to the URL on this branch.Export dialog groups formats that can stream with TSV and CSV
"Include all rows" only applied to TSV and CSV, so a plugin format that accepts
row_streamcould only be streamed by editing the URL. Those formats now share the TSV/CSV form, so "Include all rows" and "Expand labels" apply to them.is_binaryandis_webpageformats in the group keep their reduced set of actions.?_size=max, as TSV and CSV do, so without the checkbox they return up tomax_returned_rowsrows rather than the rows on the page.Testing
Generated by Claude Code