feat(app): DQX Studio dqlake integration

.github/workflows/push.yml

@@ -54,7 +54,24 @@ jobs:
 
       - name: Run unit tests
         if: env.IS_FORK_PR != 'true'
-        run: make test
+        env:
+          # Lets the watchdog's SIGABRT print a stack for every thread of every process.
+          PYTHONFAULTHANDLER: "1"
+        run: |
+          # This step has hung *after* the last test completed, inside pytest-xdist's session
+          # teardown. No pytest timeout covers that phase (--timeout only guards a test's
+          # setup/call/teardown), so the job emitted minutes of silence and then died on the
+          # runner's shutdown signal with nothing to debug. Abort before the runner reclaims the
+          # VM instead, so faulthandler shows where each thread is parked. The budget has to
+          # clear a normal run (dependency sync dominates, tests are ~1min) while still landing
+          # before the shutdown signal.
+          ( sleep 480
+            echo "::error::unit tests still running after 8m - dumping thread stacks"
+            pkill -ABRT -f '[p]ython'
+          ) &
+          watchdog=$!
+          trap 'kill "$watchdog" 2>/dev/null || true' EXIT
+          make test
 
       - name: Publish unit test coverage
         # 3.12 only to avoid duplicate uploads.

.gitignore

@@ -192,3 +192,8 @@ docs/dqx/docs/reference/api
 app/bun.lock
 app/package-lock.json
 /uv.toml
+.playwright-mcp/
+
+# Superpowers brainstorm scratch (mockups, session state)
+.superpowers/
+docs/superpowers/

AGENTS.md

@@ -30,21 +30,29 @@ The Studio is a separate uv sub-project with its own pyproject + lockfile under
 ```bash
 make app-install              # yarn install --frozen-lockfile
 make app-build                # OpenAPI dump + orval + Vite build + wheels
-make app-check                # bun tsc -b  +  basedpyright (type-check only, no lint)
+make app-check                # bun tsc -b + basedpyright + bun UI unit tests (app-test-ui)
 make app-test                 # backend pytest
+make app-test-ui              # UI unit tests only (bun test)
 make app-start-dev            # uvicorn (:9002) + Vite (:9001), foreground
 make app-stop-dev             # pkill the two dev servers
 make app-regen-api            # dump OpenAPI + run orval (after backend model changes)
 make app-deploy   PROFILE=<p> TARGET=<t>   # build + bundle deploy (creates resources + native UC grants) + bundle run
-make lock-app-dependencies    # refresh app/uv.lock + app/yarn.lock
+make lock-app-dependencies    # refresh app/uv.lock + app/yarn.lock (+ app/.build-constraints.txt)
 ```
 
-App docs (read these before touching `app/`): [`app/CLAUDE.md`](app/CLAUDE.md), [`app/DEPLOYMENT.md`](app/DEPLOYMENT.md), [`app/DEVELOPMENT.md`](app/DEVELOPMENT.md), [`app/README.md`](app/README.md), and the backend / UI CLAUDEs under `app/src/databricks_labs_dqx_app/`.
+App docs (read these before touching `app/`): [`app/AGENTS.md`](app/AGENTS.md) (agent + architecture context; thin `CLAUDE.md` stubs under `app/` link here), [`app/DEPLOYMENT.md`](app/DEPLOYMENT.md), [`app/DEVELOPMENT.md`](app/DEVELOPMENT.md), [`app/README.md`](app/README.md).
+
+**Contributing to DQX Studio** (separate from DQX Core):
+
+- **Code** lives under [`app/`](app/) (its own `pyproject.toml` / lockfiles). Prefer `make app-*` targets from the repo root.
+- **User docs** live under [`docs/dqx/docs/studio/`](docs/dqx/docs/studio/) (published at `/docs/studio/`). Do not bury Studio how-tos in Core guide pages — link to `/docs/studio/` instead. See [Authoring Documentation](https://databrickslabs.github.io/dqx/docs/dev/docs_authoring/).
+- **Issues / PRs** that touch Studio should use the GitHub label **`DQX App`** (use **`DQX Core`** for the Python library under `src/`). See [Contributing](https://databrickslabs.github.io/dqx/docs/dev/contributing/).
+- **What's new:** major, user-facing Studio features are summarized in the pull request that ships them (for release notes). There is no in-docs "What's new" page.
 
 ### Dependency installs and lock files
 
 - Use **`make dev`** from the repo root to create `.venv` and install Python dependencies. Do **not** run `uv sync`, `uv lock`, or `uv add` for normal setup — that bypasses `UV_FROZEN=1` and may modify `uv.lock` or bake in internal registry URLs.
-- To **update lock files** after intentional dependency changes: `make lock-dependencies` (root `uv.lock` and `.build-constraints.txt`) and/or `make lock-app-dependencies` (`app/uv.lock`) of the app.
+- To **update lock files** after intentional dependency changes: `make lock-dependencies` (root `uv.lock` and `.build-constraints.txt`) and/or `make lock-app-dependencies` (`app/uv.lock`, `app/yarn.lock`, and `app/.build-constraints.txt`).
 
 ---
 

CHANGELOG.md

@@ -61,7 +61,7 @@ BREAKING CHANGES!
 * Allow custom check failure messages ([#1092](https://github.com/databrickslabs/dqx/issues/1092)). `DQRule` now accepts an optional `message_expr` parameter that lets users define custom failure messages as a Spark `Column` or a SQL expression string. The same option is supported for checks defined declaratively in metadata (YAML/JSON), specified as a top-level `message_expr` key on the check definition alongside `criticality` and `check`. When omitted, the default message behavior is preserved; when provided, the custom message replaces the default message for failed rows.
 * Added a Query Results Cookbook and aligned stored check names and fingerprints ([#1193](https://github.com/databrickslabs/dqx/issues/1193)). A new reference page provides "copy-paste" SQL and PySpark recipes for querying DQX result tables (summary metrics, output, quarantine, and checks) to trace errors and warnings across runs, rows, and check definitions. To make the cookbook's fingerprint and name joins reliable, checks saved without an explicit `name` now store the same autogenerated name and name-inclusive `rule_fingerprint` that `apply_checks` writes to `_errors`/`_warnings` (named checks and `for_each_column` rules are byte-identical to before). Requesting summary metrics via `metrics_config` without a configured observer now fails fast with an `InvalidParameterError` instead of silently skipping the metrics table.
 * Added in-app language switching to DQX Studio ([#1172](https://github.com/databrickslabs/dqx/issues/1172)). DQX Studio now ships with four locales (English, Brazilian Portuguese, Italian, and Spanish), selectable from a new Preferences card on the user's Profile page. The choice is persisted per-browser via `localStorage` with no server-side or table changes, and the change is frontend-only. Non-English translations are AI-assisted and not yet reviewed by native speakers.
-* DQX Studio: replaced the apx build framework with first-party build and dev scripts ([#1223](https://github.com/databrickslabs/dqx/issues/1223)). The app no longer depends on the `apx` package. `scripts/build_app.py` generates the FastAPI OpenAPI schema, runs orval, builds the frontend with Vite, and produces the application wheel (with a build-tagged local-version segment so successive deploys at the same commit always reinstall fresh code). `scripts/dev.py` runs uvicorn with reload alongside the Vite dev server, forwarding signals and tearing down both processes together. The bundle and warehouse-grant scripts were updated to support both bundle-managed and external (reuse) SQL warehouse modes. There is no runtime behavior change in the app itself.
+* DQX Studio: replaced the previous third-party build framework with first-party build and dev scripts ([#1223](https://github.com/databrickslabs/dqx/issues/1223)). The app no longer depends on that external build package. `scripts/build_app.py` generates the FastAPI OpenAPI schema, runs orval, builds the frontend with Vite, and produces the application wheel (with a build-tagged local-version segment so successive deploys at the same commit always reinstall fresh code). `scripts/dev.py` runs uvicorn with reload alongside the Vite dev server, forwarding signals and tearing down both processes together. The bundle and warehouse-grant scripts were updated to support both bundle-managed and external (reuse) SQL warehouse modes. There is no runtime behavior change in the app itself.
 * DQX Studio: added Lakebase storage backend to improve app latency with declarative storage and destroy protection ([#1173](https://github.com/databrickslabs/dqx/issues/1173)). Schemas, the wheels volume, and the Lakebase instance and logical database are now declared in the bundle with `prevent_destroy` lifecycle protection, and `make app-bind` adopts pre-existing resources. OLTP tables (rules, settings, RBAC, comments, schedules) move to Postgres via a migration runner, while analytical tables (validation runs, profiling, quarantine, metrics) stay on Delta. Error, warning, and input row counts from the DQX observer are now persisted and surfaced in the UI, label badges and label filtering were added to rule selection and scheduling, and a Spark Connect `Observation.get` mutability bug that overwrote total row counts was fixed.
 * Fixed quarantine-only writes when no output table is configured ([#1183](https://github.com/databrickslabs/dqx/issues/1183)). `apply_checks_and_save_in_table` and `apply_checks_by_metadata_and_save_in_table` previously raised `AttributeError` when called with `output_config=None` and a `quarantine_config`. `output_config` is now optional and skipped when unset, so quarantine-only runs write just the invalid records; passing neither configuration raises a clear `InvalidParameterError`.
 * Allow special characters in catalog and schema names ([#1232](https://github.com/databrickslabs/dqx/issues/1232)). The validation regex for storage locations now accepts catalog and schema names that contain characters such as hyphens, which were previously rejected.

Makefile

@@ -11,7 +11,11 @@ export UV_FROZEN := 1
 export UV_BUILD_CONSTRAINT := .build-constraints.txt
 
 UV_RUN := uv run --exact --all-extras
-UV_TEST := $(UV_RUN) pytest -n 10 --timeout 60 --durations 20
+# xdist worker count. Recursively expanded so a target can override it (see ``test``).
+# Workspace-backed suites (integration/e2e) keep the fixed high count: they are bound by
+# control-plane latency rather than CPU, so oversubscribing the runner is what keeps them fast.
+TEST_WORKERS ?= 10
+UV_TEST = $(UV_RUN) pytest -n $(TEST_WORKERS) --timeout 60 --durations 20
 
 # ``make help`` parses ``##`` annotations next to each target and ``##@``
 # section headers so the listing stays in sync with the Makefile
@@ -53,6 +57,10 @@ fmt: ## Format and auto-fix Python (black, ruff --fix, mypy, pylint, version syn
 
 ##@ Tests (DQX library)
 
+# One worker per core. The unit suite is CPU-bound, so a fixed ``-n 10`` oversubscribes the
+# CI runners and only widens the surface xdist has to tear down at session end. Override with
+# ``make test TEST_WORKERS=10`` to restore the old count.
+test: TEST_WORKERS := auto
 test: ## Run unit tests (writes coverage-unit.xml)
 	$(UV_TEST) --cov --cov-report=xml:coverage-unit.xml tests/unit/
 
@@ -160,11 +168,21 @@ open('.build/openapi.json', 'w').write(json.dumps(app.openapi(), indent=2))"
 # (``-b``) so subsequent runs only re-check changed files; the Python
 # pass runs basedpyright at ``error`` level only (per the existing
 # pyproject configuration excluding tests, see [tool.basedpyright]).
-app-check: ## Type-check app: tsc -b (TypeScript) + basedpyright (Python)
+app-check: ## Type-check app: tsc -b (TypeScript) + basedpyright (Python) + bun UI unit tests
 	@echo "🔍 Checking TypeScript..."
 	cd app && bun run tsc -b --incremental
 	@echo "🔍 Checking Python..."
-	cd app && $(UV_RUN) basedpyright --level error
+	# basedpyright lives in the app ``dev`` dependency group — do not use
+	# root ``UV_RUN`` (``--all-extras``) here; the app has no extras and that
+	# sync would strip the dev tools from app/.venv.
+	cd app && uv run --exact --group dev basedpyright --level error
+	@$(MAKE) app-test-ui
+
+# Front-end unit tests (bun's built-in test runner — runs *.test.ts natively,
+# no extra config). Kept fast + dependency-free so it can run inside app-check.
+app-test-ui: ## Run app UI unit tests (bun test)
+	@echo "🧪 Testing UI (bun test)..."
+	cd app && bun test src/databricks_labs_dqx_app/ui
 
 # Run the app's backend unit-test suite (pytest, no Databricks dependencies).
 # Usage:  make app-test            # run everything
@@ -361,6 +379,10 @@ lock-app-dependencies: ## Regenerate app/uv.lock, app/yarn.lock, app/.build-cons
 	yarn --cwd app install
 	perl -ni -e 'print unless /^  resolved /' app/yarn.lock
 	cd app && uv lock --exclude-newer "7 days"
+	# Normalize the lock so contributors inside Databricks (private proxy) and outside (public PyPI)
+	# produce an identical file. A proxy mirrors PyPI with identical paths, so rewrite the registry
+	# index and every per-package "/packages/..." download URL to the public hosts. Also drop the
+	# "size" field: the private proxy never reports it, so it is the only form both can reproduce.
 	perl -pi -e 's|registry = "https://[^"]*"|registry = "https://pypi.org/simple"|g; s|url = "https://[^/"]+/packages/|url = "https://files.pythonhosted.org/packages/|g; s|, size = \d+||g' app/uv.lock
 	# UV_FROZEN=1 for the helper below: this target sets UV_FROZEN=0 to re-lock app/uv.lock, but the
 	# `uv run` here executes from the repo root and would otherwise re-lock (and proxy-taint) the root
@@ -402,4 +424,4 @@ fork-sync: ## Mirror a fork PR to a branch in the main repo for full CI (PR=<num
 	./.github/scripts/fork-sync-pr.sh $(PR)
 
 .DEFAULT: all
-.PHONY: help all clean dev lint fmt test integration e2e perf anomaly coverage combine-coverage docs-build docs-serve-dev docs-install docs-serve docs-clean app-install app-build app-start-dev app-stop-dev app-regen-api app-check app-test app-check-cli app-deploy fork-sync build lock-dependencies lock-app-dependencies
+.PHONY: help all clean dev lint fmt test integration e2e perf anomaly coverage combine-coverage docs-build docs-serve-dev docs-install docs-serve docs-clean app-install app-build app-start-dev app-stop-dev app-regen-api app-check app-test app-test-ui app-check-cli app-deploy fork-sync build lock-dependencies lock-app-dependencies