Fix Full data toggle overlap in AI explorations

[fly1d]

Summary

Fixes #1893.

In Lumen AI explorations, the Full data checkbox appeared above the result table and could overlap the SQL editor. The final layout renders the table first and adds the checkbox in a right-aligned row below it, only when the materialized result reaches its SQLLimit.

The table keeps a fixed 10-row remote page size. Enabling the checkbox removes the query limit; disabling it restores the original limit for that query.

Tabulator hides its page-number group when footer contents overflow. A scoped stylesheet makes the paginator and page group wrap, so long page numbers remain available without competing with Next or Last.

Verification

• pixi run -e test-312 pytest lumen/tests/ai/test_editors.py -q (37 passed)
• pixi run -e test-312 pre-commit run --files lumen/ai/editors.py lumen/tests/ai/test_editors.py (passed)
• git diff --check (passed)
• Real lumen ai serve --provider checks covered the 1,000,025-row DuckDB result, six-digit pagination, narrow result panes, pane resizing, and Full data toggle round-trips without overlap.
• GitHub CI passed on Ubuntu, macOS, and Windows with Python 3.12/3.13.

OpenAI Codex assisted with the implementation and verification; fly1d reviewed the submitted changes.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.57%. Comparing base (4b70a41) to head (2aa274b).
⚠️ Report is 9 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2045      +/-   ##
==========================================
+ Coverage   75.02%   75.57%   +0.55%     
==========================================
  Files         212      214       +2     
  Lines       38883    39727     +844     
==========================================
+ Hits        29171    30025     +854     
+ Misses       9712     9702      -10     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ghostiee-11]

Hey @fly1d, Thanks for taking this up :)

Can you attach a quick video/photo of the fix.

[fly1d]

Thanks! Here is the fix at a 768 x 700 viewport:

The checkbox now stays in normal layout flow above the table, with the SQL editor fully separated from the result controls.

[ghostiee-11]

This checkbox is not looking good here. Can you do something else or align it

[fly1d]

Thanks for the feedback. I tightened the control row and let the checkbox use its natural content width, so Full data stays on one line and aligns with the table's right edge while remaining in normal layout flow.

Updated in 832567fa. I verified the layout at 768 x 700 and 320 x 700, ran lumen/tests/ai/test_editors.py (36 passed), and reran pre-commit on the changed files.

[ghostiee-11]

Thanks @fly1d, Much better can you also upload video with the functionality working properly :)

Also instead of probe can you run the server.

[fly1d]

Thanks! I ran the actual Panel server and recorded the real Full data interaction (not the earlier isolated probe):

Full data functionality recording (MP4)

For a short, visible demonstration, the local server uses four DuckDB rows with SQLLimit(3). Clicking Full data changes the live table from three rows to all four. That three-row threshold is only part of the demo setup; the production limit and PR code are unchanged.

[ghostiee-11]

Just a quick suggestion: can we add this full data checkbox somewhere down here, on the right-hand side?

Also can you take around 150+ rows and then show me the video of this

[fly1d]

Thanks for the suggestion. Updated in 77b72871:

• Full data now sits at the bottom-right beside the pagination controls. The footer reserves space for it, so it does not cover the page buttons.
• At viewports below 430px, it moves to a separate right-aligned row so the controls do not overlap.

I also ran the actual Panel server with a 175-row DuckDB source and SQLLimit(150). This recording starts at 150 loaded rows, then automatically toggles the real Full data checkbox through the same watcher and shows all 175 rows:

175-row Full data recording (MP4)

Verification: lumen/tests/ai/test_editors.py (36 passed), pre-commit passed, and the layout was checked at 768 x 700, 532 x 700, and 320 x 700.

[ghostiee-11]

Can you please run lumen ai serve --provider through that and run a real query and then shoot a video using any dataset and then show me the fix :)

[fly1d]

Thanks! I reran this through the actual Lumen AI server (not the isolated Panel probe) and used a real natural-language query.

Command:

pixi run -e test-312 lumen-ai serve /private/tmp/lumen-full-data-demo.db --provider ollama --model qwen3:4b --port 5010 --address 127.0.0.1 --allow-websocket-origin=127.0.0.1:5010 --log-level info

The DuckDB records table contains 1,000,025 rows (id 0 through 1,000,024). In the recording, the query Show all rows from the records table. generates SELECT * FROM records.

Real-server Full data recording (MP4)

The recording shows:

• the default result capped at 1,000,000 rows, with page 50,000 as the last page;
• enabling the real Full data checkbox through its normal watcher;
• page 50,002 becoming available and the final row 1,000,024 being displayed;
• disabling Full data again to restore the capped result.

While validating the real server, I found one remaining theme-specific layout issue: the earlier rule reserved space on .tabulator-footer, but Tabulator applies the effective padding to .tabulator-footer-contents, and Panel's theme stylesheet loads later and resets that inner padding to 10px. That could still let the checkbox crowd the Last button.

I fixed this narrowly in b1f47ce9 by targeting the actual footer contents and preserving 125px of right-side space across theme load order. The existing narrow-screen rule restores the normal 10px padding below 430px, where the checkbox moves to its own row.

I also reproduced two follow-up issues while checking the recording:

• With page_size=None, the responsive Tabulator frontend recalculated the page size whenever the result pane was resized. That made the page count appear to jump (for example, 50,000 pages at 20 rows/page versus 100,000 pages at 10 rows/page), even though the row data had not changed.
• Disabling Full data always restored a hard-coded 1,000,000-row limit. For queries using another cap (such as 150 rows), that left the table unbounded from the user's point of view.

Both are fixed in 5a02550b: the table now uses a stable 20-row page size, and disabling Full data restores the original SQLLimit value for that query.

Verification after the fix:

• real browser DOM geometry: Last right edge 1145px, Full data left edge 1154px (9px gap, no overlap);
• computed footer-content right padding: 125px;
• pixi run -e test-312 pytest lumen/tests/ai/test_editors.py -q: 36 passed, including full-data on/off restoration and fixed page size;
• pre-commit on lumen/ai/editors.py: all hooks passed.

I kept this PR focused on the current layout. Longer term, would you prefer this kind of control to use an explicit Panel/Tabulator footer slot or extension point instead of being overlaid with reserved footer space? I can investigate that separately if it matches the maintainers' direction.

[fly1d]

Correction to the pagination verification above: the real-server viewport and recording use 10 rows per page, so the expected final page for 1,000,025 rows is ceil(1,000,025 / 10) = 100003 (not 50,002).

I initially fixed the resize instability with a 20-row page size, which made the expected final page 50,002 and did not match the maintainer's requested viewport evidence. I corrected this in 3d681138 by keeping the stable page size at 10. The original-limit restoration fix remains unchanged.

The test suite still passes (36 passed) and pre-commit remains clean. With the latest code, resizing the result pane should leave the page size at 10 and keep the last page at 100003 for the full 1,000,025-row dataset, or 100000 for the capped 1,000,000-row result.

[fly1d]

Thanks for the careful review. I reproduced the pagination behavior on the real Lumen AI server, fixed the remaining issues, and recorded the latest result.

Sorry, GitHub's PR uploader available here only accepted videos up to 10 MB, so I could not upload the full-resolution original recording (10,949,090 bytes). I reduced the frame rate to 8 fps and cropped only the browser chrome; the compressed recording is 1,669,167 bytes and keeps the relevant UI and values readable:

Latest real-server recording: pagination and limit fixes (MP4)

The recording shows the complete fixed sequence:

1. With Full data disabled, the original 1,000,000-row cap is active and the last page is 100000 at 10 rows per page.
2. Enabling Full data loads all 1,000,025 rows; the last page becomes 100003, and its last row is id = 1,000,024.
3. Expanding the adjacent panel changes the available table width, but the page size remains 10 and the last page remains 100003.
4. Disabling Full data restores the original cap and returns the last page to 100000.

The issues and fixes are:

• page_size=None allowed the responsive Tabulator frontend to recalculate the page size after pane resizing, so the displayed total page count changed even though the rows did not. 3d681138 uses a stable 10-row page size.
• Disabling Full data restored a hard-coded 1,000,000-row limit instead of the query's actual prior limit. 5a02550b records and restores each query's original SQLLimit.
• Panel's theme could override the footer padding and let the checkbox crowd the pagination buttons. b1f47ce9 reserves space on the effective .tabulator-footer-contents element while preserving the narrow-screen layout.

Verification on the latest code:

• pixi run -e test-312 pytest lumen/tests/ai/test_editors.py -q: 36 passed
• pre-commit: all hooks passed
• GitHub CI: Ubuntu, macOS, and Windows checks passed on Python 3.12/3.13

The fixes and this verification summary were prepared with OpenAI Codex assistance.

[fly1d]

I made the follow-up layout adjustment in f947a637.

• At the normal table width, the Full data row now has an explicit 30px height, align-items: center, and a 5px bottom offset so its checkbox/label are vertically centered with the pagination buttons instead of sitting on the table's bottom edge.
• The table wrapper is now an inline-size container. When the table itself becomes narrower than 760px (for example, after opening the adjacent navigation panel), Full data moves to a right-aligned row below the Tabulator footer and the footer padding reservation is removed. This prevents it from covering Next, Last, or page-number buttons.
• The existing 430px viewport fallback remains in place for very narrow screens.

I checked the real server DOM in the narrowed layout (table container width 419px): the footer ends before the separate Full data row begins, with no overlap. The focused test suite still passes (36 passed), pre-commit passes, and all GitHub CI checks pass across Ubuntu/macOS/Windows on Python 3.12/3.13.

This change was prepared with OpenAI Codex assistance.

[fly1d]

I refined the footer layout again in 786f42cf.

The Full data checkbox now stays on the same row as the pagination controls. At narrower application viewports, the footer keeps only First, Prev, the active page, Next, and Last; the extra page-number buttons are hidden so the row remains compact and usable. The checkbox is given a fixed compact width and centered vertically with the buttons.

I also moved the responsive rules into the actual Tabulator and Panel shadow-DOM stylesheets, so they apply reliably in the running application. Local DOM verification at a 1280px viewport showed:

• pagination button centers and the checkbox center both at y=693;
• Last button right edge at 1160px and the checkbox starting at 1176.7px (about 17px gap);
• no footer overlap.

Verification: 36 passed, pre-commit passed, and all GitHub CI checks passed on Ubuntu/macOS/Windows with Python 3.12/3.13.

This change was prepared with OpenAI Codex assistance.

[fly1d]

I found a regression in the previous layout adjustment: the narrow-screen rule hid every non-active page button, so a multi-page result could show only the current page. Sorry about that.

I corrected it in 917462f2:

• all rendered page-number buttons remain visible;
• narrower widths use smaller pagination padding/font size and reserve the right-side checkbox space instead of hiding pages;
• Full data now uses the same 36px control height as the footer and is vertically centered with the pagination;
• the original query limit is still restored when the checkbox is turned off.

I verified the running Panel components with five pages at 1280px and fixed table widths of 600px and 430px. In each case the complete page sequence stayed on one row, Last remained before Full data, and no boxes overlapped. The focused suite remains 36 passed and pre-commit remains clean.

This follow-up was prepared with OpenAI Codex assistance.

[fly1d]

I found one more layout case while testing the real 1,000,000-row result: when the paginator reaches six-digit page numbers (99996 through 100000), the page-button group becomes wider and can crowd the Full data label near Last.

Before making another layout change, could you advise which behavior you prefer for this case?

1. Keep Full data on the same footer row and allow more aggressive compact styling for long page numbers; or
2. Move Full data to a right-aligned row below the paginator whenever the available footer width is insufficient.

The second option keeps every page button readable and guarantees no overlap, while the first preserves the single-row appearance. I will follow the project's preference rather than guessing at another breakpoint.

[fly1d]

This screenshot is from the real Lumen AI server with the 1,000,000-row result. It shows the remaining overlap when page numbers reach six digits: the paginator reaches 100000 and the Full data control crowds the Last button. Could you advise whether the project prefers a wrapped second row when the footer is too narrow, or another layout approach?

[fly1d]

Thanks for considering this. I am happy to follow whichever layout pattern the maintainers consider most appropriate for this footer. If the project already has a preferred Panel or Tabulator approach, I would appreciate your guidance and can adapt the patch accordingly.

In particular, what would you recommend for keeping the full pagination readable while making the Full data control reliable at both normal widths and six-digit page counts? I am also open to an approach beyond the two options described above.

[ghostiee-11]

@fly1d, The full data thing is overlapping with next and last have to look any other way then

[fly1d]

Thanks for the feedback. I changed the approach so the checkbox no longer sits on top of the footer: Full data now lives in a right-aligned row in normal document flow directly below the results table.

Because the pagination keeps its full footer width and the checkbox is no longer absolutely positioned, long page numbers can never push it into Next or Last, regardless of viewport size. I verified this in the real browser DOM with 100,000 rows and six-digit page numbers (9996–10000): all page buttons remain visible, and the footer ends at y=772 while the Full data row starts at y=777, so there is no overlap.

Updated in 61be5210. Verification remains: lumen/tests/ai/test_editors.py (36 passed) and pre-commit clean.

This change was prepared with OpenAI Codex assistance.

[fly1d]

Thanks for the screenshot. I reproduced this case with the real 1,000,025-row DuckDB result. The page-number buttons are still generated, but Tabulator hides the .tabulator-pages group when the footer overflows horizontally, which is why only First, Prev, Next, and Last remain visible.

I fixed this in 2aa274bd by allowing the paginator and page-number group to wrap instead of being hidden. Full data remains in its separate right-aligned row below the table, so it cannot overlap the pagination controls. At 389px and 320px table widths, the real six-digit sequence (99999 through 100003) remains visible, the footer has no horizontal overflow, and the buttons remain usable. At wider widths the existing one-row alignment is unchanged.

Verification: pixi run -e test-312 pytest lumen/tests/ai/test_editors.py -q (37 passed), pre-commit passed, and git verify-commit plus GitHub verification both report a valid SSH signature for fly1d <309400591+fly1d@users.noreply.github.com>.

Could you please confirm whether this wrapped paginator layout matches the project's preferred behavior for narrow result panes? This follow-up was prepared with OpenAI Codex assistance and reviewed by fly1d.