[ZEPPELIN-6432] Modernize the Jekyll documentation build

[jongyoul]

What is this PR for?

Modernize the versioned documentation build and keep its publication output compatible with zeppelin-site.

• Replace github-pages / Jekyll 3.9.5 / Redcarpet with Jekyll 4.4.1 and Kramdown GFM.
• Make Docker the documented environment for dependency updates, preview, and production builds.
• Exclude build-only files from _site and document the Zeppelin-to-zeppelin-site publication boundary.
• Remove obsolete third-party analytics, comment, sharing, and feed integrations.
• Replace two externally hosted graph diagrams with original Apache-licensed SVG diagrams.
• Preserve Apache Zeppelin's approved ASF Matomo site ID 69 only in --safe production builds.
• Add a generated-site check for non-ASF embedded resources and unapproved trackers.

This also prevents the build manifests covered by ZEPPELIN-6431 from being copied into future versioned documentation snapshots. Cleaning historical snapshots remains work in apache/zeppelin-site.

What type of PR is it?

Improvement

Todos

• Upgrade the Jekyll and Ruby toolchain.
• Document and verify the Docker-only build.
• Remove legacy integrations and empty Atom/RSS feeds.
• Replace external documentation images with original local SVG diagrams.
• Preserve approved ASF Matomo tracking for production docs.
• Verify generated output and external-resource policy.

What is the Jira issue?

• https://issues.apache.org/jira/browse/ZEPPELIN-6432
• Related: https://issues.apache.org/jira/browse/ZEPPELIN-6431

How should this be tested?

From docs/:

docker run --rm \
  -v "$PWD:/docs" \
  -w /docs \
  ruby:4.0.6 \
  bash -lc "bundle install && bundle exec jekyll build --safe"

Then:

docker run --rm \
  -v "$PWD:/docs:ro" \
  -w /docs \
  ruby:4.0.6 \
  ruby check_external_resources.rb _site

The final verification generated 94 HTML files with Jekyll 4.4.1, emitted no build warnings, retained ASF Matomo site ID 69 only in the safe build, and produced no Atom/RSS or build-manifest output. It also verified that both original SVG diagrams are present in the generated site without the previous GitHub/S3 resource URLs. The full Apache RAT check passed for all 56 reactor modules.

Screenshots (if appropriate)

The replacement diagrams are committed as SVG files and rendered directly by the documentation page; this PR does not change the documentation CSS or page layout.

Questions:

• Does the license files need to update? No.
• Is there breaking changes for older versions? No runtime behavior changes; this affects the documentation build and generated output.
• Does this needs documentation? docs/README.md and docs/AGENTS.md are updated.

[Copilot]

Pull request overview

This PR modernizes the docs/ Jekyll build to produce versioned documentation output compatible with zeppelin-site, while removing legacy third-party integrations and tightening publication policy checks.

Changes:

• Upgrades the docs toolchain from GitHub Pages/Jekyll 3 + Redcarpet to Jekyll 4.4.1 + Kramdown GFM and documents Docker-based build/preview workflows.
• Removes Atom/RSS feeds, comments, sharing, and legacy third-party analytics integrations; preserves ASF Matomo (site ID 69) only for --safe builds.
• Replaces externally hosted diagram images with local ASF-licensed SVGs and adds a generated-site external resource/tracker check script.

Reviewed changes

Copilot reviewed 23 out of 26 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
docs/usage/display_system/basic.md	Replaces externally hosted graph images with local SVG assets referenced via {{BASE_PATH}}.
docs/rss.xml	Removes obsolete RSS feed template from the docs build.
docs/atom.xml	Removes obsolete Atom feed template from the docs build.
docs/README.md	Updates documentation build instructions to a Docker-only workflow and adds generated-site verification step.
docs/AGENTS.md	Adds scoped contributor/agent guidance for docs/ including Docker build commands and publication boundary.
docs/Gemfile	Pins Jekyll 4.4.1 explicitly and removes GitHub Pages/redcarpet-era gems.
docs/Gemfile.lock	Refreshes the locked Ruby dependencies for the new Jekyll/Kramdown-based toolchain.
docs/_config.yml	Switches markdown engine to Kramdown GFM, expands excludes to keep build-only files out of _site, and adds Matomo site ID config.
docs/check_external_resources.rb	Adds a post-build scan to reject non-ASF embedded resources/unapproved trackers in generated _site.
docs/_includes/themes/zeppelin/default.html	Removes feed link tags and legacy analytics include; conditionally includes Matomo only when site.safe.
docs/_includes/themes/zeppelin/post.html	Removes comments include from post template.
docs/_includes/JB/matomo	Adds ASF Matomo embed snippet (used only during --safe builds).
docs/_includes/JB/sharing	Removes legacy sharing helper include.
docs/_includes/JB/comments	Removes legacy comments helper include.
docs/_includes/JB/comments-providers/livefyre	Removes legacy Livefyre comment provider include.
docs/_includes/JB/comments-providers/intensedebate	Removes legacy IntenseDebate comment provider include.
docs/_includes/JB/comments-providers/facebook	Removes legacy Facebook comments embed include.
docs/_includes/JB/comments-providers/disqus	Removes legacy Disqus comments embed include.
docs/_includes/JB/analytics	Removes legacy analytics helper include.
docs/_includes/JB/analytics-providers/piwik	Removes legacy Piwik analytics include.
docs/_includes/JB/analytics-providers/mixpanel	Removes legacy Mixpanel analytics include.
docs/_includes/JB/analytics-providers/google-universal	Removes legacy Google Analytics (universal) include.
docs/_includes/JB/analytics-providers/google-classic	Removes legacy Google Analytics (classic) include.
docs/_includes/JB/analytics-providers/getclicky	Removes legacy GetClicky analytics include.
docs/assets/themes/zeppelin/img/docs-img/property-graph-example.svg	Adds local, ASF-licensed replacement diagram for the property graph example.
docs/assets/themes/zeppelin/img/docs-img/labeled-property-graph-model.svg	Adds local, ASF-licensed replacement diagram for the labeled property graph model.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[kimyenac]

I compared this branch's generated site against the same docs sources built with Jekyll 3.9.5 and Redcarpet (Gemfile swapped, markdown: redcarpet restored, docs content untouched). Two pages lose headings under Kramdown, both from unclosed <tr> tags in HTML that already existed in the docs. Redcarpet passed the raw HTML through, while Kramdown parses the structure, so the markdown after an open row is emitted unprocessed.

• docs/interpreter/spark.md: lines 135 and 140 are <tr> where </tr> was meant, and the row opened on line 141 is never closed. Everything from line 264 to the end of the page then comes out as raw markdown.
• docs/usage/rest_api/interpreter.md: line 623 is </td> where </tr> was meant, leaving the row opened on line 611 unclosed.

I applied both fixes and rebuilt:

page	build	h2/h3	raw markdown emitted
interpreter/spark.html	Redcarpet	20	none
	Kramdown, current	4	20 headings, 10 links
	Kramdown, fixed	20	none
usage/rest_api/interpreter.html	Redcarpet	13	none
	Kramdown, current	12	1 heading
	Kramdown, fixed	13	none

Diffing the whole generated site before and after the fix, only those two pages change and the other 92 are identical. A scan for unclosed <tr> across docs/ turns up only these two files.

10 literal backticks remain on the Spark page and 210 on the interpreter page after the fix. Those come from a separate pattern, markdown written inside hand-written HTML tables, which I have not looked into yet.

Turning on kramdown: show_warnings: true produces 75 warnings across 22 files, including the mis-nested tr, table and td ones for both files above (6 and 3). With that enabled, and perhaps failing the docs build when the generated site still contains literal backticks or ^#{2,4} sequences, I think this kind of problem could be caught early!