Skip to content

feat(uat): create node keys with consensus keys init instead of sample secrets #1283

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
leninmehedy wants to merge 1 commit into from
Draft

feat(uat): create node keys with consensus keys init instead of sample secrets #1283

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions docs/dev/daemon/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,11 @@ on daemon self-upgrade, #500). Keep the upgrade package **config-only** for a cl
task, which sanity-checks the grant with `kubectl auth can-i`.
- **Per-node secrets.** `consensus node install` needs the gossip signing secret
(`node<N>-gossip-keys`) and gRPC TLS secret (`node<N>-grpc-tls-keys`) in the orbit
namespace, plus a pull secret for private images. The `secrets` task creates these
from the sample manifests (reuses `uat:secrets`).
namespace, plus a pull secret for private images. The `secrets` task creates the pull
secret (reuses `uat:secrets`); the `keys` task generates real keys with
`consensus keys init` and creates the dated, labelled node secrets
(`node<N>-gossip-keys-<date>-<fp8>`, …), which `consensus node install` finds by label.
Use `task -d docs/dev/daemon secrets SAMPLE_KEYS=true` to apply the sample secrets instead.
- **A solo-operator checkout at v0.7.0+ on the host** (`SOLO_OPERATOR_DIR`, defaults
to a sibling of solo-weaver) for `beacon` and the deployment zips — the `prep` task
git-clones it there if it is missing (needs `GITHUB_USER`/`GITHUB_ACCESS_TOKEN` in
Expand Down Expand Up @@ -123,11 +126,12 @@ task -d docs/dev/daemon prep # clone solo-operator (if needed) + buil
# substrate — the daemon's real install path (single-node k8s)
task -d docs/dev/daemon rebuild # build CLI + daemon on the VM + self-install (uat:rebuild)
task -d docs/dev/daemon cluster # ghcr login + kube cluster install + orbit ns + hedera user + upgrade dir
task -d docs/dev/daemon secrets # gossip/gRPC + pull secrets (uat:secrets)
task -d docs/dev/daemon secrets # pull secrets in operator + orbit ns (uat:secrets)
task -d docs/dev/daemon keys # consensus keys init + verify --from-cluster (gossip/gRPC/admin secrets)
task -d docs/dev/daemon operator # kube operator install (v0.7.0 operator + CRDs)

# a REAL v0.74.0 consensus network, in DAEMON-DELEGATED mode
task -d docs/dev/daemon consensus # consensus node install --provisioner-daemon + genesis
task -d docs/dev/daemon consensus # consensus node install --provisioner-daemon + genesis

# install + run the host daemon (the execute-phase performer)
task -d docs/dev/daemon daemon:install # daemon service install (daemon-cn RBAC + scoped kubeconfig)
Expand Down Expand Up @@ -164,9 +168,10 @@ Each task wraps a real command; the notable ones:
| `sync` (host) | runs the root `vm:sync` — rsyncs the weaver checkout (incl. staged zips/package/beacon) into `/mnt/solo-weaver` |
| `rebuild` | `uat:rebuild` — `task build:cli` (builds daemon too, stamps its digest) + self-install |
| `cluster` | `uat:registry:login:ghcr` + `kube cluster install --non-interactive` + create orbit ns + `hedera` user/group + upgrade staging dir (owned `hedera:hedera`, mode `0775`, `weaver` in the `hedera` group) |
| `secrets` | `uat:secrets NS=<orbit> NODE=node<id>` — pull secret in operator + orbit ns, plus `node<id>-gossip-keys` / `node<id>-grpc-tls-keys` |
| `secrets` | `uat:secrets NS=<orbit> NODE=node<id> SAMPLE_KEYS=false` — pull secret in operator + orbit ns (`SAMPLE_KEYS=true` also applies the sample node secrets) |
| `keys` | `consensus keys init --experimental --namespace <orbit> --node-id <id> --keys-dir $KEYS_DIR` (default `/tmp/cn-keys/node<id>`), then `keys verify --from-cluster`; creates the labelled `node<id>-gossip-keys-…` / `-grpc-tls-keys-…` / `-admin-pubkey-…` secrets |
| `operator` | `kube operator install --non-interactive`; checks the `networkupgradeexecutes` CRD is registered |
| `network` | `consensus node install --experimental --provisioner-daemon --deployment-package-dir <pkg> …` (creates the Orbit in mainnet/daemon-delegated mode), then `consensus network genesis --experimental` |
| `consensus` | `consensus node install --experimental --provisioner-daemon --deployment-package-dir <pkg> …` (creates the Orbit in mainnet/daemon-delegated mode), then `consensus network genesis --experimental` |
| `daemon:install` | precondition: the local daemon build exists under `/mnt/solo-weaver/bin`; then `daemon service install --components consensus-node --cn-node-id <id> --cn-orbit <ns> --daemon-bin <built>` (`--cn-namespace` is a hidden alias for `--cn-orbit`) + `kubectl auth can-i` checks |
| `config` | writes `/tmp/beacon-solo/config.yaml` + the 0.0.2 payer PEM |
| `upload`/`prepare`/`freeze` | `beacon file upload/info`, `beacon freeze prepare/upgrade` (`{{.BEACON}}` from `/mnt/solo-weaver/bin`) — all pinned to `H` |
Expand Down
40 changes: 32 additions & 8 deletions docs/dev/daemon/Taskfile.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,9 @@ vars:
# The consensus node account id (0.0.3) — the v0.74.0 genesis has node0 account 0.0.3, which is the default payer for beacon's config.yaml.
ACCOUNT_ID: '{{.ACCOUNT_ID | default "0.0.3"}}' # node0 account in the v0.74.0 genesis

# Where `keys init` writes node<N>'s key material.
KEYS_DIR: '{{.KEYS_DIR | default (printf "/tmp/cn-keys/node%s" .NODE_ID)}}'

# Daemon upgrade staging dir — daemon reads the extracted package here; the CN pod
# extracts special file 0.0.150 into the same path (shared on a single host).
UPGRADE_DIR: /opt/hgcapp/services-hedera/HapiApp2.0/data/upgrade/current
Expand Down Expand Up @@ -163,7 +166,8 @@ tasks:
# substrate: the daemon's real install path (single-node k8s)
T1 task -d docs/dev/daemon rebuild # build CLI+daemon on the VM + self-install
T2 task -d docs/dev/daemon cluster # registry login + kube cluster install + orbit ns + hedera user + upgrade dir
T3 task -d docs/dev/daemon secrets # gossip/gRPC + pull secrets (reuses uat:secrets)
T3 task -d docs/dev/daemon secrets # pull secrets (reuses uat:secrets)
T3b task -d docs/dev/daemon keys # consensus keys init + verify (gossip/gRPC secrets + a sample admin public key)
T4 task -d docs/dev/daemon operator # kube operator install (v0.7.1 operator + CRDs)
# ── how to run solo-operator (pick one) ──
# A) released image in-cluster: nothing more to do — `operator` (above) runs it.
Expand Down Expand Up @@ -240,17 +244,37 @@ tasks:
sudo usermod -aG hedera weaver
sudo -u weaver touch {{.UPGRADE_DIR}}/.probe && sudo -u weaver rm {{.UPGRADE_DIR}}/.probe && echo "✓ weaver can write {{.UPGRADE_DIR}}"

# ── 3. secrets: pull secret (operator+orbit ns) + node0 gossip/gRPC ──────────
# ── 3. secrets: pull secret (operator + orbit ns) ────────────────────────────
secrets:
desc: "Inside VM: create the private-registry pull secret (operator + orbit ns) and node0 gossip/gRPC secrets (reuses uat:secrets)"
desc: "Inside VM: create the private-registry pull secret (operator + orbit ns); node keys come from `keys` (reuses uat:secrets)"
cmds:
# uat:secrets rewrites the sample manifests' namespace to NS and applies the
# node0-gossip-keys / node0-grpc-tls-keys secrets `consensus node install` expects.
# SAMPLE_KEYS=false: skip the fixed sample node secrets; `keys` creates real ones.
# Pass SAMPLE_KEYS=true to use the sample secrets instead.
- task: uat:secrets
vars:
NS: "{{.NS}}"
OPERATOR_NS: "{{.OPERATOR_NS}}"
NODE: "node{{.NODE_ID}}"
SAMPLE_KEYS: "{{.SAMPLE_KEYS | default \"false\"}}"

# ── 3b. keys: generate node keys + create their labelled secrets ─────────────
keys:
desc: "Inside VM: generate node{{.NODE_ID}}'s gossip/gRPC keys into KEYS_DIR, add a sample admin public key, and create the dated, labelled secrets (consensus keys init)"
cmds:
- |
set -e
# The admin key comes from an external wallet in real use; here a random
# 32-byte value stands in for its Ed25519 public key (test value only).
sudo mkdir -p {{.KEYS_DIR}}
[ -f {{.KEYS_DIR}}/admin.pub ] || openssl rand -hex 32 | sudo tee {{.KEYS_DIR}}/admin.pub >/dev/null
echo "▶ consensus keys init (node{{.NODE_ID}}, keys in {{.KEYS_DIR}})"
sudo solo-provisioner consensus keys init --experimental \
--namespace {{.NS}} --node-id {{.NODE_ID}} --keys-dir {{.KEYS_DIR}}
echo "▶ consensus keys verify --from-cluster"
sudo solo-provisioner consensus keys verify --experimental \
--namespace {{.NS}} --node-id {{.NODE_ID}} --from-cluster
echo "✓ node{{.NODE_ID}} secrets (install finds them by label):"
sudo kubectl -n {{.NS}} get secret -l solo.hiero.org/node=node{{.NODE_ID}} --show-labels

# ── 4. operator (real CRDs, v0.7.0) ──────────────────────────────────────────
operator:
Expand Down Expand Up @@ -327,9 +351,9 @@ tasks:
--namespace {{.NS}} --node-id {{.NODE_ID}} --account-id {{.ACCOUNT_ID}} --profile local \
--provisioner-daemon \
--default-volume-type hostpath \
--deployment-package-dir {{.PKG}} \
--signing-secret node{{.NODE_ID}}-gossip-keys \
--grpc-tls-secret node{{.NODE_ID}}-grpc-tls-keys
--deployment-package-dir {{.PKG}}
# No --signing-secret/--grpc-tls-secret/--admin-key-secret: install resolves the
# labelled secrets that `keys` created (pass them explicitly to override).

echo "▶ consensus network genesis"
sudo solo-provisioner consensus network genesis --experimental --namespace {{.NS}}
Expand Down
11 changes: 8 additions & 3 deletions taskfiles/uat.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,17 +47,19 @@ tasks:
- task: uat:bootstrap

uat:secrets:
desc: "Inside VM: create ALL secrets for a consensus deployment in one shot — the private-registry image-pull secret in BOTH the operator namespace (for 'kube operator install') and the consensus namespace (threaded onto the pods), plus the sample consensus-node gossip/gRPC secrets. Same private registry serves both, so one credential covers everything. Select the registry host with REGISTRY= (default ghcr.io), the consensus namespace with NS= (default hiero-network-1), the operator namespace with OPERATOR_NS= (default solo-operator), a single node with NODE=node2, and the pull-secret name with SECRET_NAME= (default private-registry-creds). Run AFTER 'kube cluster install'. Requires GITHUB_USER/GITHUB_ACCESS_TOKEN in a root .env file (see .env.example)."
desc: "Inside VM: create ALL secrets for a consensus deployment in one shot — the private-registry image-pull secret in BOTH the operator namespace (for 'kube operator install') and the consensus namespace (threaded onto the pods), plus the sample consensus-node gossip/gRPC secrets. Same private registry serves both, so one credential covers everything. Select the registry host with REGISTRY= (default ghcr.io), the consensus namespace with NS= (default hiero-network-1), the operator namespace with OPERATOR_NS= (default solo-operator), a single node with NODE=node2, skip the sample node secrets with SAMPLE_KEYS=false, and the pull-secret name with SECRET_NAME= (default private-registry-creds). Run AFTER 'kube cluster install'. Requires GITHUB_USER/GITHUB_ACCESS_TOKEN in a root .env file (see .env.example)."
vars:
NS: '{{.NS | default "hiero-network-1"}}'
OPERATOR_NS: '{{.OPERATOR_NS | default "solo-operator"}}'
NODE: '{{.NODE | default ""}}'
SECRET_NAME: '{{.SECRET_NAME | default "private-registry-creds"}}'
REGISTRY: '{{.REGISTRY | default "ghcr.io"}}'
SAMPLE_KEYS: '{{.SAMPLE_KEYS | default "true"}}'
cmds:
- |
set -e
DIR=/mnt/solo-weaver/test/sample-secrets
SAMPLE_KEYS='{{.SAMPLE_KEYS}}'
NS='{{.NS}}'; OPERATOR_NS='{{.OPERATOR_NS}}'; NODE='{{.NODE}}'; SECRET_NAME='{{.SECRET_NAME}}'; REGISTRY='{{.REGISTRY}}'
: "${GITHUB_USER:?Set GITHUB_USER in a root .env file (see .env.example)}"
: "${GITHUB_ACCESS_TOKEN:?Set GITHUB_ACCESS_TOKEN (token with read:packages) in a root .env file (see .env.example)}"
Expand Down Expand Up @@ -94,8 +96,11 @@ tasks:
# Sample consensus-node gossip/gRPC secrets (consensus namespace only). The
# sample manifests carry a fixed namespace; rewrite it to $NS on apply so the
# same files work for any orbit/namespace.
# SAMPLE_KEYS=false skips them (create real keys with `consensus keys init`).
apply_one() { sed "s/namespace: hiero-network-1/namespace: $NS/" "$1" | kubectl apply -f -; }
if [ -n "$NODE" ]; then
if [ "$SAMPLE_KEYS" = "false" ]; then
echo "▶ Skipping sample gossip/gRPC secrets (SAMPLE_KEYS=false)"
elif [ -n "$NODE" ]; then
echo "▶ Applying sample gossip/gRPC secrets for $NODE in namespace $NS ..."
apply_one "$DIR/$NODE.yaml"
else
Expand All @@ -107,7 +112,7 @@ tasks:
echo '✓ Applied:'
kubectl -n "$OPERATOR_NS" get secret "$SECRET_NAME"
kubectl -n "$NS" get secret "$SECRET_NAME"
kubectl -n "$NS" get secret -l app=consensus-node
[ "$SAMPLE_KEYS" = "false" ] || kubectl -n "$NS" get secret -l app=consensus-node

uat:registry:login:ghcr:
desc: "Inside VM: write ghcr.io credentials into the Helm registry config that 'solo-provisioner ... install' reads, so it can pull the PRIVATE solo-operator OCI chart (oci://ghcr.io/hashgraph/solo-operator/...). Because install runs under sudo (root), the auth is written to BOTH root's and the invoking user's config. Requires GITHUB_USER/GITHUB_ACCESS_TOKEN (root .env; see .env.example). Run once per VM before 'kube cluster install --node-type consensus'."
Expand Down