Comprehensive Diátaxis Documentation System with GitBook & GH Pages Compatibility

.github/workflows/docs-ci.yml

@@ -0,0 +1,53 @@
+name: Documentation CI
+
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - "docs/**"
+      - "*.txt"
+      - "*.xml"
+      - "mkdocs.yml"
+      - "scripts/mkdocs_hooks.py"
+      - ".github/workflows/docs-ci.yml"
+  pull_request:
+    branches:
+      - main
+    paths:
+      - "docs/**"
+      - "*.txt"
+      - "*.xml"
+      - "mkdocs.yml"
+      - "scripts/mkdocs_hooks.py"
+      - ".github/workflows/docs-ci.yml"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+jobs:
+  lint-and-validate:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout Repository
+        uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
+        with:
+          persist-credentials: false
+
+      - name: Set up Python
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
+        with:
+          python-version: "3.12"
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install mkdocs-material==9.5.18 pymdown-extensions
+
+      - name: Build documentation site
+        run: |
+          mkdocs build --strict

docs/SUMMARY.md

@@ -0,0 +1,26 @@
+# Summary - Diátaxis Documentation
+
+* [Home](diataxis_index.md)
+
+## Tutorials
+* [Step-by-Step Installation](tutorials/installation.md)
+* [Ansible Quickstart](tutorials/ansible_quickstart.md)
+
+## How-To Guides
+* [System Tuning & Optimization](how-to/system_tuning.md)
+* [Distributed WSL2 Cluster](how-to/wsl2_cluster.md)
+* [Gitea & Semaphore Secure Operations](how-to/secure_gitea_semaphore.md)
+
+## Upgrade Plans
+* [Elastic 9.5.0 Upgrade Plan](ELASTIC_9_UPGRADE_PLAN.md)
+
+## Reference
+* [CLI Scripts Reference](reference/cli_scripts.md)
+* [Ansible Playbooks Spec](reference/playbooks_spec.md)
+* [Podman Compose Configurations](reference/compose_configs.md)
+* [MkDocs Rewriter Hook API](reference/mkdocs_hook_api.md)
+
+## Explanation
+* [Architecture Overview](explanation/architecture_overview.md)
+* [Hardened Wolfi Images & Security Decisions](explanation/security_decisions.md)
+* [Diátaxis Framework Integration](explanation/diataxis.md)

docs/diataxis_index.md

@@ -0,0 +1,51 @@
+---
+title: "Documentation Home"
+description: "Welcome to the Podman-based Elastic Stack and Gitea/Semaphore deployment documentation system."
+nav_order: 1
+---
+
+# Sovereign Infrastructure Documentation
+
+Welcome to our production-ready, structured documentation system. This documentation is organized into four distinct categories according to the **Diátaxis Framework**:
+
+- **[Tutorials](tutorials/installation.md)**: Learning-oriented, step-by-step lessons to help you get started from scratch.
+- **[How-To Guides](how-to/system_tuning.md)**: Task-oriented, practical directions for specific, real-world problems.
+- **[Reference](reference/cli_scripts.md)**: Information-oriented, comprehensive technical descriptions, parameters, and specifications.
+- **[Explanation](explanation/architecture_overview.md)**: Understanding-oriented, architectural maps, concept details, and high-level decisions.
+
+---
+
+## 🗺️ Navigation Map
+
+
+### Tutorials
+
+1. **[Step-by-Step Installation](tutorials/installation.md)**: Build an unprivileged, rootless single-node Elastic Stack on WSL2 or bare-metal Linux.
+2. **[Ansible Quickstart](tutorials/ansible_quickstart.md)**: Get up and running with our Ansible playbooks in less than five minutes.
+
+
+### How-To Guides
+
+1. **[System Tuning & Optimization](how-to/system_tuning.md)**: Apply kernel rules (`vm.max_map_count`, memory limits) on WSL2 or Linux hosts.
+2. **[Distributed WSL2 Cluster](how-to/wsl2_cluster.md)**: Scale up a simulated multi-node high-availability Elastic Cluster.
+3. **[Gitea & Semaphore Secure Operations](how-to/secure_gitea_semaphore.md)**: Set up rootless git servers and secure, trusted CI pipelines.
+
+
+### Upgrade Plans
+
+1. **[Elastic 9.5.0 Upgrade Plan](ELASTIC_9_UPGRADE_PLAN.md)**: Master architectural blueprint and 2-week upgrade roadmap.
+
+
+### Reference
+
+1. **[CLI Scripts Reference](reference/cli_scripts.md)**: Detailed option and interface breakdown for setup and feedback scripts.
+2. **[Ansible Playbooks Spec](reference/playbooks_spec.md)**: Complete map of roles, tasks, variables, and telemetry logs.
+3. **[Podman Compose Configurations](reference/compose_configs.md)**: Core environment attributes, resource bounds, and network topologies.
+4. **[MkDocs Rewriter Hook API](reference/mkdocs_hook_api.md)**: Functional parameters, regex patterns, and normalization mechanics.
+
+
+### Explanation
+
+1. **[Architecture Overview](explanation/architecture_overview.md)**: Core structural topology, unprivileged workflows, and data boundary rules.
+2. **[Hardened Wolfi Images & Security Decisions](explanation/security_decisions.md)**: Snyk audit, Zero-CVE Wolfi bases, TLS trust setups, and telemetry logging bounds.
+3. **[Diátaxis Framework Integration](explanation/diataxis.md)**: Comprehensive explanation of the Diátaxis architecture in this workspace.

docs/explanation/architecture_overview.md

@@ -0,0 +1,39 @@
+---
+title: "Architecture Overview"
+description: "Conceptual explanation of our structural topology, unprivileged workflows, and data boundary rules."
+nav_order: 40
+---
+
+# Architecture Overview
+
+This explanation guide outlines the core design goals, system boundaries, and structural elements of the deployment architecture.
+
+---
+
+## 🏛️ Component Boundaries
+
+The project establishes three segregated operational layers:
+
+```mermaid
+flowchart TD
+    User["Human Operator / CLI"] --> Controller["Ansible Controller"]
+    Controller --> Podman["Podman Engine (Rootless/User Mode)"]
+    subgraph Isolated Stack Net
+        Podman --> ES["Hardened Wolfi Elasticsearch"]
+        Podman --> Kib["Hardened Wolfi Kibana"]
+        Podman --> Git["Sovereign Gitea"]
+        Podman --> Sem["SemaphoreUI Quadlet Stack"]
+    end
+```
+
+---
+
+## 🔒 Unprivileged & Rootless Execution
+
+Standard setups often run container runtimes with root privileges, creating potential privilege-escalation risks.
+
+Our project enforces a **Strict Zero-Privilege Rule**:
+1. All container tasks are managed under standard user permissions via rootless Podman execution contexts.
+2. Port binding ranges are shifted above privileged values (e.g. mapping internal ports securely to host ranges such as `3000` or `5601`).
+3. Services utilize a shared unprivileged user bridge that provides internal container connectivity and isolation from unrelated external host networks, rather than enforcing logical network isolation between the attached services themselves.
+4. Separate from this bridge layer, host-port exposure is bounded: selected services are made accessible externally via explicitly configured interface bindings. Specifically, the setup helper scripts `setup_elasticsearch.sh` and `setup_kibana.sh` accept an optional `BIND_ADDRESS` parameter (defaulting to `127.0.0.1`) which enforces that public-facing container ports (e.g., `9200` and `5601`) bind strictly to the specified host interface.

docs/explanation/diataxis.md

@@ -0,0 +1,55 @@
+---
+title: "Diátaxis Framework Integration"
+description: "Explanation of how the Diátaxis documentation structure is adopted, integrated, and maintained in our project."
+nav_order: 42
+---
+
+# Diátaxis Framework Integration
+
+This page explains why and how our project adopts the **Diátaxis Framework** to organize, govern, and maintain our technical documentation ecosystem.
+
+---
+
+## 🧭 What is Diátaxis?
+
+The [Diátaxis Framework](https://diataxis.fr/) is a systematic approach to technical writing that classifies technical content into four distinct, complementary user needs:
+
+```text
+               |  PRACTICAL STEP  |  THEORETICAL STEP
+---------------+------------------+-------------------
+ACQUISITION    |    Tutorials     |    Explanation
+---------------+------------------+-------------------
+APPLICATION    |   How-To Guides  |     Reference
+```
+
+---
+
+## 🛠️ How we apply Diátaxis in this Project
+
+Our document architecture is separated cleanly inside the `docs/` workspace to solve explicit user situations:
+
+### 1. Tutorials (Learning-Oriented)
+- **Path**: `docs/tutorials/`
+- **Goal**: Guided, step-by-step learning lessons for beginners. Focuses on learning through execution.
+- **Example**: Creating a single-node deployment from scratch without needing complex orchestration options.
+
+### 2. How-To Guides (Problem-Oriented)
+- **Path**: `docs/how-to/`
+- **Goal**: Practical directions to help you solve a specific task or real-world problem.
+- **Example**: Overriding host memory parameters, or setting up TLS trust across local servers.
+
+### 3. Reference (Information-Oriented)
+- **Path**: `docs/reference/`
+- **Goal**: Absolute technical accuracy, CLI variables, APIs, inputs, outputs, and programmatic signatures.
+- **Example**: Sourcing the exact list of options accepted by the `setup_elasticsearch.sh` script.
+
+### 4. Explanation (Understanding-Oriented)
+- **Path**: `docs/explanation/`
+- **Goal**: High-level conceptual clarification, component boundaries, architectural choices, and security decisions.
+- **Example**: Explaining why rootless execution matrices prevent container privilege escalation.
+
+---
+
+## 📈 Long-term Maintenance
+
+To ensure our documentation never goes stale, we enforce automatic link audits and snippet validation using GitHub Actions pipelines (`.github/workflows/docs-ci.yml`). This maintains dual compatibility across GitBook sitemaps and compiled GitHub Pages dashboards.

docs/explanation/security_decisions.md

@@ -0,0 +1,34 @@
+---
+title: "Hardened Wolfi Images & Security Decisions"
+description: "High-level review of security mechanisms, image audits, and telemetry boundaries."
+nav_order: 41
+---
+
+# Hardened Wolfi Images & Security Decisions
+
+This document details the critical security paradigms, base image choices, and operational auditing patterns implemented across the project.
+
+---
+
+## 🛡️ Zero-CVE Hardened Wolfi Images
+
+Standard container deployments often include excess packages, compilation tools, and utilities that expand the service's attack vector.
+
+Our architecture tackles this through **Wolfi-hardened base images**:
+- **Minimal Footprint**: Wolfi containers do not contain diagnostic tools, shell environments (unless explicitly required), or unneeded binaries.
+- **Dynamic vulnerability audits**: The images are continuously audited with `Snyk` to maintain a zero-CVE state.
+
+---
+
+## 🔑 Automated Secrets and Key Scopes
+
+Hardcoded deployment configurations, default database tokens, and pre-baked SSH keys represent significant risks.
+
+We mitigate these vulnerabilities by:
+1. Sourcing high-entropy passwords dynamically using the exact security generators: the playbooks leverage `ansible.builtin.password` to provision secure 24-character passwords and invoke `openssl rand -base64 32` to generate robust API access keys.
+2. Isolating active tokens to untracked files (`*temp_credentials.txt`, `*gitea_credentials.txt`).
+3. Configuring `.gitignore` patterns, which exclude matching untracked files by default. Note that `.gitignore` only applies to untracked files and will not prevent committing files that are already tracked or added with a force option (`git add -f`).
+
+If a secret has already been committed to the repository history, you must immediately:
+- Rotate all compromised credentials/tokens/keys on any active systems.
+- Use history cleaning tools (e.g., `git-filter-repo` or `BFG Repo-Cleaner`) to purge the sensitive files entirely from all repository commits and historical timelines.

docs/how-to/secure_gitea_semaphore.md

@@ -0,0 +1,60 @@
+---
+title: "Gitea & Semaphore Secure Operations"
+description: "How to operate secure, unprivileged Gitea servers and trusted Semaphore pipelines."
+nav_order: 22
+---
+
+# Gitea & Semaphore Secure Operations
+
+This guide provides practical instructions for operating secure, unprivileged code servers and trusted CI pipelines in isolated rootless scopes.
+
+---
+
+## 🔒 Task 1: Generate High-Entropy Git Secrets Dynamically
+
+If not manually set, Gitea playbooks dynamically generate strong passwords.
+
+
+### Step 1: Identify Password Files
+
+Sovereign credentials are automatically created and isolated from Git tracking inside local `.txt` paths. Gitea credentials default to `gitea_credentials.txt` in the deployment directory. Semaphore credentials are saved to the path defined by `semaphore_credentials_file` (which defaults to `~/.config/containers/semaphoreui/secrets/semaphore_credentials.txt` but can be overridden with the `semaphore_credentials_override` variable):
+
+*   **Gitea Secrets**: `gitea_credentials.txt`
+*   **Semaphore Secrets**: Configured via `semaphore_credentials_file`
+
+
+### Step 2: Enforce Strict File Permissions
+
+Ensure secrets are not readable by other unprivileged system accounts. The chmod example should target the resolved configured path rather than assuming a current-directory filename:
+
+```bash
+# Secure the dynamically generated credentials files
+chmod 0600 gitea_credentials.txt
+chmod 0600 "${HOME}/.config/containers/semaphoreui/secrets/semaphore_credentials.txt"
+```
+
+---
+
+## 🤝 Task 2: Configure TLS Trust for GitOps Execution
+
+To enable Semaphore's `go-git` engine to securely clone repositories from local self-signed HTTPS Gitea instances, the self-signed certificate must be registered in the host CA store.
+
+
+### Step 1: Register Certificate
+
+```bash
+sudo cp gitea.crt /usr/local/share/ca-certificates/
+sudo update-ca-certificates
+```
+
+
+### Step 2: Volume Mount Host Bundle
+
+The Semaphore deployment automatically mounts the host CA bundle directly inside the execution containers as a read-only volume:
+
+```yaml
+volumes:
+  - /etc/ssl/certs:/etc/ssl/certs:ro
+```
+
+This read-only CA bundle enables Semaphore execution containers to verify server certificates for outbound HTTPS connections, establishing secure one-way server authentication. Client certificates are managed separately and are required only if mutual TLS (mTLS) is explicitly enforced for bidirectional verification.

docs/how-to/system_tuning.md

@@ -0,0 +1,59 @@
+---
+title: "System Tuning & Optimization"
+description: "Practical task-based instructions to apply host/kernel modifications for unprivileged Podman deployments."
+nav_order: 20
+---
+
+# System Tuning & Optimization Guide
+
+This how-to guide explains how to apply mandatory kernel modifications and resource limits on WSL2 or bare-metal Linux.
+
+---
+
+## 🛠️ Task 1: Check & Apply `vm.max_map_count` Limits
+
+Elasticsearch requires a minimum virtual memory allocation parameter to prevent Out Of Memory crashes.
+
+
+### Step 1: Query the Active Limit
+
+```bash
+sysctl vm.max_map_count
+```
+
+
+### Step 2: Set the Count Permanently
+
+On your host or WSL2 environment, edit `/etc/sysctl.conf` or `/etc/sysctl.d/99-elasticsearch.conf` and set:
+
+```text
+vm.max_map_count=262144
+```
+
+Apply the configuration instantly:
+
+```bash
+sudo sysctl --system
+```
+
+---
+
+## 📁 Task 2: Fix Inotify Limits for Large Stacks
+
+WSL2 and native Linux distributions have default limitations on directory watch monitors, which can cause compose environments to fail to track file events.
+
+
+### Step 1: Set Inotify Limits
+
+Write the updated boundaries to `/etc/sysctl.d/50-inotify.conf`:
+
+```text
+fs.inotify.max_user_watches=524288
+fs.inotify.max_user_instances=512
+```
+
+Reload the runtime kernel boundaries:
+
+```bash
+sudo sysctl -p /etc/sysctl.d/50-inotify.conf
+```