Skip to content
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 32 additions & 2 deletions docs/source/user_guide/parameters.rst
Original file line number Diff line number Diff line change
Expand Up @@ -68,9 +68,11 @@ There are 6 output-specific parameters:
- *(none)*
- Where the post-processing results (``post/`` directory) should go.
* - ``www``
- **Yes**
- *(none)*
- No
Comment thread
tomvothecoder marked this conversation as resolved.
- ``""``
- Where the post-processing visuals should go (to be viewed online).
Leave empty and set ``[simboard] enabled = True`` to have ``zppy``
infer this path from Mache. See :doc:`tasks/simboard` for details.
* - ``campaign``
- No
- ``"none"``
Expand Down Expand Up @@ -404,6 +406,34 @@ These are no longer defined in ``zppy/defaults/default.ini``:
These are still defined in ``zppy/defaults/default.ini``, but have no effect:

.. code-block:: text

ncclimo_cmd
nrows
ncols

SimBoard section parameters
============================

The ``[simboard]`` section controls SimBoard-compatible publishing. It is
a configuration-only hook; see :doc:`tasks/simboard` for full details.

.. list-table::
:header-rows: 1
:widths: 22 10 18 50

* - Parameter
- Required
- Default
- Description
* - ``enabled``
- No
- ``False``
- Set to ``True`` to enable SimBoard-compatible publishing.
When ``True`` and ``[default] www`` is empty, ``zppy`` infers
``www`` from Mache's ``web_portal.base_path``.
* - ``simulation_type``
- No
- ``"production"``
- Archive sub-directory for the run. One of ``"production"``,
``"development"``, or ``"none"``.
Must not be ``"none"`` when ``enabled = True``.
3 changes: 3 additions & 0 deletions docs/source/user_guide/tasks/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ Listed for reference (bundle jobs are submitted after task jobs are generated in
- Description
* - :doc:`bundle`
- Bundle multiple tasks into a single SLURM job
* - :doc:`simboard`
Comment thread
tomvothecoder marked this conversation as resolved.
- Configure SimBoard-compatible diagnostics publishing
* - :doc:`climo`
- Generate climatology files using NCO's ``ncclimo``
* - :doc:`ts`
Expand Down Expand Up @@ -50,6 +52,7 @@ Listed for reference (bundle jobs are submitted after task jobs are generated in
:hidden:

bundle
simboard
climo
ts
e3sm_to_cmip
Expand Down
90 changes: 90 additions & 0 deletions docs/source/user_guide/tasks/simboard.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
.. _task-simboard:
Comment thread
tomvothecoder marked this conversation as resolved.

simboard — SimBoard Publishing Configuration
============================================

The ``simboard`` section is a configuration-only hook that controls
SimBoard-compatible publishing behavior. Like :doc:`bundle`, it does not
launch an HPC job of its own; instead it influences how other tasks are
configured — specifically, it can infer the ``www`` output path from the
machine's Mache configuration.

When ``enabled = True`` and ``www`` is left empty in ``[default]``,
``zppy`` derives ``www`` from the ``web_portal.base_path`` recorded in
Mache for the current machine:

.. code-block:: text

<web_portal_base_path>/diagnostics_archive/<simulation_type>/

This gives SimBoard a single, predictable archive root to scan for
diagnostics.

Expected behavior
-----------------

.. list-table::
:header-rows: 1
:widths: 20 15 65

* - ``simboard.enabled``
- ``www``
- Behavior
* - ``False``
- any
- ``zppy`` does nothing SimBoard-specific.
* - ``True``
- empty
- Infer the SimBoard archive path from Mache's
``web_portal.base_path``.
* - ``True``
- set
- Use the explicit ``www`` path and do not override it.
``simboard.enabled`` still controls SimBoard-specific validation
(e.g., ``simulation_type`` must not be ``"none"``).
* - ``True``
- empty, but path cannot be inferred
- Raise a clear configuration error.

Configuration example
---------------------

.. code-block:: cfg

[default]
case = v3.LR.historical_0051
input = /path/to/input
output = /path/to/output
# Leave www empty to let zppy infer it from Mache when simboard is enabled.
www =

[simboard]
enabled = True
simulation_type = production

Parameters
----------

.. list-table::
:header-rows: 1
:widths: 22 10 18 50

* - Parameter
- Required
- Default
- Description
* - ``enabled``
- No
- ``False``
- Set to ``True`` to enable SimBoard-compatible publishing behavior.
When enabled and ``[default] www`` is empty, ``zppy`` infers
``www`` from Mache's ``web_portal.base_path``.
* - ``simulation_type``
- No
- ``"production"``
- Diagnostic classification for the archive path. One of
``"production"``, ``"development"``, or ``"none"``.
Must not be ``"none"`` when ``enabled = True``.

.. note::
The ``[simboard]`` section does not support subsections.
132 changes: 132 additions & 0 deletions tests/integration/test_simboard_settings.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
import ast
import configparser
import re
from pathlib import Path
from typing import Any, Dict, Optional
from unittest.mock import MagicMock

import pytest
from configobj import ConfigObj
from validate import Validator

from zppy.__main__ import _determine_parameters
from zppy.utils import write_settings_file

_WEB_PORTAL_BASE_PATH = "/global/cfs/cdirs/e3sm/www"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a Perlmutter-specific path, but that's fine because we're just doing dry-run testing here.



def _default_ini_path() -> Path:
return Path(__file__).resolve().parents[2] / "zppy" / "defaults" / "default.ini"


def _write_cfg(tmp_path: Path, *, www: str, simboard_enabled: bool) -> Path:
config_path = tmp_path / "simboard_settings.cfg"
config_path.write_text(
"\n".join(
[
"[default]",
"case = test_case",
"input = /input",
f"output = {tmp_path / 'output'}",
"dry_run = True",
"machine = pm-cpu",
f"www = {www}",
"",
"[simboard]",
f"enabled = {'True' if simboard_enabled else 'False'}",
"simulation_type = production",
"",
]
)
)
return config_path


def _validated_config(config_path: Path) -> ConfigObj:
config = ConfigObj(str(config_path), configspec=str(_default_ini_path()))
validation_result = config.validate(Validator())
assert validation_result is True
return config


def _fake_machine_info(
web_portal_base_path: Optional[str] = _WEB_PORTAL_BASE_PATH,
) -> MagicMock:
machine_config = configparser.ConfigParser()
machine_config["e3sm_unified"] = {"base_path": "/unified"}
machine_config["diagnostics"] = {"base_path": "/diagnostics"}
machine_config["web_portal"] = {"base_url": "https://portal.nersc.gov/cfs/e3sm"}
if web_portal_base_path is not None:
machine_config["web_portal"]["base_path"] = web_portal_base_path

machine_info = MagicMock()
machine_info.machine = "pm-cpu"
machine_info.config = machine_config
machine_info.get_account_defaults.return_value = ("e3sm", "regular", "cpu", None)
return machine_info


def _write_default_settings(tmp_path: Path, config: ConfigObj) -> Path:
settings_path = tmp_path / "resolved_default.settings"
write_settings_file(str(settings_path), dict(config["default"]), (1, 1))
return settings_path


def _read_default_settings(settings_path: Path) -> Dict[str, Any]:
settings_text = settings_path.read_text()
parsed_settings = ast.parse(settings_text, mode="exec")
assert parsed_settings.body
first_expression = parsed_settings.body[0]
assert isinstance(first_expression, ast.Expr)
default_settings = ast.literal_eval(first_expression.value)
assert isinstance(default_settings, dict)
return default_settings


def test_simboard_disabled_preserves_explicit_www(tmp_path: Path) -> None:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checks this condition:

simboard.enabled www Behavior
False any zppy does nothing SimBoard-specific.

config_path = _write_cfg(
tmp_path, www="/some/explicit/path", simboard_enabled=False
)
config = _validated_config(config_path)

updated_config = _determine_parameters(_fake_machine_info(), config)
settings_path = _write_default_settings(tmp_path, updated_config)
default_settings = _read_default_settings(settings_path)

assert default_settings["www"] == "/some/explicit/path"


def test_simboard_enabled_infers_www_from_machine_info(tmp_path: Path) -> None:
Comment thread
tomvothecoder marked this conversation as resolved.
config_path = _write_cfg(tmp_path, www="", simboard_enabled=True)
config = _validated_config(config_path)
expected_www = f"{_WEB_PORTAL_BASE_PATH}/diagnostics_archive/production/"

updated_config = _determine_parameters(_fake_machine_info(), config)
settings_path = _write_default_settings(tmp_path, updated_config)
default_settings = _read_default_settings(settings_path)

assert default_settings["www"] == expected_www


def test_simboard_enabled_preserves_explicit_www(tmp_path: Path) -> None:
Comment thread
tomvothecoder marked this conversation as resolved.
config_path = _write_cfg(tmp_path, www="/custom/path", simboard_enabled=True)
config = _validated_config(config_path)

updated_config = _determine_parameters(_fake_machine_info(), config)
settings_path = _write_default_settings(tmp_path, updated_config)
default_settings = _read_default_settings(settings_path)

assert default_settings["www"] == "/custom/path"


def test_simboard_enabled_empty_www_requires_inferable_path(tmp_path: Path) -> None:
Comment thread
tomvothecoder marked this conversation as resolved.
config_path = _write_cfg(tmp_path, www="", simboard_enabled=True)
config = _validated_config(config_path)
expected_message = (
"www is empty and simboard.enabled is True, but machine 'pm-cpu' "
"has no web_portal.base_path in mache; cannot infer a "
"diagnostics_archive path."
)

with pytest.raises(ValueError, match=re.escape(expected_message)):
_determine_parameters(_fake_machine_info(web_portal_base_path=None), config)
9 changes: 9 additions & 0 deletions tests/test_sections.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,15 @@ def test_sections():
}
compare(actual_default, expected_default)

# simboard
section_name = "simboard"
actual_section = config[section_name]
expected_section = {
"enabled": False,
"simulation_type": "production",
}
compare(actual_section, expected_section)

# ts
section_name = "ts"
actual_section = config[section_name]
Expand Down
Loading
Loading