Skip to content

feat: add analyst-friendly ClickHouse views - #53

Merged
notwedtm merged 3 commits into
solana-rpc:mainfrom
kevredlabs:feat/analyst-views
Aug 31, 2026
Merged

feat: add analyst-friendly ClickHouse views#53
notwedtm merged 3 commits into
solana-rpc:mainfrom
kevredlabs:feat/analyst-views

Conversation

@Leomrqdd

Copy link
Copy Markdown

Summary

Adds four plain (non-materialized) SQL views over default.transactions for ad-hoc analytics. They encapsulate three pitfalls of the raw schema: binary keys (re-encoded to base58), ALT-loaded accounts (merged so account filters do not miss v0 transactions), and vote transactions (pre-filtered in the analytical views).

View Grain
tx_summary one row per non-vote transaction
sol_transfers one row per (tx, account) with a SOL balance change
token_transfers one row per (tx, token account) with an SPL balance change
transactions_decoded strictly 1:1 with transactions (votes included)

Changes

  • New view_*.sql files following the existing DDL convention: same basenames in ddl/local/, ddl/cluster/, and ddl/replicated/ (clustered variants use ON CLUSTER '{cluster}'; they are otherwise identical since plain views store nothing). The views are optional and only require default.transactions.
  • The Compose clickhouse-ddl service applies them after the base DDL. Tilt/k8s deploy manifests are intentionally left untouched.
  • docs/analyst-views.md documents semantics, consistency checks, performance notes, and 7 example queries; indexed in docs/README.md and referenced from ddl/README.md.
  • Zero ingestion/storage impact: plain views, computed at query time.

Testing

Tested against 100 mainnet slots (115k transactions, ClickHouse 26.1) ingested via the ingest-rpc Compose profile.

@notwedtm

Copy link
Copy Markdown
Contributor

Hey @Leomrqdd, thanks for this PR!

I can see how these can be useful, however I'm hesitant to include them in the base install as they will consume additional resources not strictly required for serving spec data.

Would you be willing to put these in a different directory for additional resources instead?

@Leomrqdd

Leomrqdd commented Jul 31, 2026

Copy link
Copy Markdown
Author

Sure! I was not sure where to put the files to be honest.

I’ll move the views out of the base DDL install into ddl/additional/ (opt-in, not wired into Docker Compose).
Does that location work for you?

Leomrqdd and others added 2 commits August 20, 2026 10:31
Move the plain views out of the base schema sets and stop applying
them in Compose, so the default install only includes objects needed
to serve spec data.
@notwedtm
notwedtm added this pull request to the merge queue Aug 31, 2026
Merged via the queue into solana-rpc:main with commit b879bba Aug 31, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants