Skip to content

Dev: corosync: Add corosync health check - #2149

Open
nicholasyang2022 wants to merge 2 commits into
ClusterLabs:masterfrom
nicholasyang2022:feature/corosync-health-check
Open

Dev: corosync: Add corosync health check#2149
nicholasyang2022 wants to merge 2 commits into
ClusterLabs:masterfrom
nicholasyang2022:feature/corosync-health-check

Conversation

@nicholasyang2022

Copy link
Copy Markdown
Collaborator

Introduce the corosync health check under 'crm cluster health corosync'.

  • Add crmsh/corosync_healthcheck.py containing local configuration
    validity, cross-node consistency, quorum status, link status, and Node ID
    mapping validation.
  • Implement ui_cluster.py integration for the 'corosync' health component.
  • Write test/unittests/test_corosync_healthcheck.py and test_ui_cluster.py
    for comprehensive unit test coverage.
  • Update doc/crm.8.adoc with help information and documentation.
  • Add functional behavior tests in test/features/corosync_ui.feature.

@nicholasyang2022

nicholasyang2022 commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator Author

Output examples:

> sudo crm cluster health corosync
[PASS] Validate Corosync Configuration File
[PASS] Validate Corosync Configuration File Consistency
[PASS] Check Quorum Status
[FAIL] Check Corosync Links Status
       Corosync link(s) are not operational:
       Link 1 to node 2 has status 'disconnected'.
       Recommended Action: Check network connectivity and corosync configuration/service on the affected nodes.
[PASS] Check Node ID to Node Name Mapping
> sudo crm cluster health corosync --local --json
{"returncode": 1, "results": [{"check_name": "Validate Corosync Configuration File", "checked_nodes": ["ha-3-1"], "returncode": 0, "result_description": null, "recommended_action": null}, {"check_name": "Check Quorum Status", "checked_nodes": ["ha-3-1"], "returncode": 0, "result_description": null, "recommended_action": null}, {"check_name": "Check Corosync Links Status", "checked_nodes": ["ha-3-1"], "returncode": 1, "result_description": "Corosync link(s) are not operational:\nLink 1 to node 2 has status 'disconnected'.", "recommended_action": "Check network connectivity and corosync configuration/service on the affected nodes."}, {"check_name": "Check Node ID to Node Name Mapping", "checked_nodes": ["ha-3-1"], "returncode": 0, "result_description": null, "recommended_action": null}]}

@nicholasyang2022
nicholasyang2022 force-pushed the feature/corosync-health-check branch 2 times, most recently from 9b8c15e to fdd7e3e Compare July 6, 2026 09:05
@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.60432% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.93%. Comparing base (c11725f) to head (6e90050).

Files with missing lines Patch % Lines
crmsh/corosync_healthcheck.py 95.39% 10 Missing ⚠️
crmsh/ui_cluster.py 91.80% 5 Missing ⚠️
Additional details and impacted files
Flag Coverage Δ
integration 55.55% <75.89%> (+0.19%) ⬆️
unit 53.33% <94.60%> (+0.48%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
crmsh/ui_cluster.py 73.82% <91.80%> (+1.85%) ⬆️
crmsh/corosync_healthcheck.py 95.39% <95.39%> (ø)

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nicholasyang2022
nicholasyang2022 force-pushed the feature/corosync-health-check branch from fdd7e3e to 0432bcb Compare July 6, 2026 11:12
@nicholasyang2022
nicholasyang2022 marked this pull request as ready for review July 6, 2026 11:17
Comment thread crmsh/corosync_healthcheck.py Outdated
prev = x.stdout
else:
result_description = StringIO()
result_description.write("Corosync configuration files are inconsistent across nodes:\n")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[FAIL] Validate Corosync Configuration File Consistency
       Corosync configuration files are inconsistent across nodes:
       2136532295a0985f1c4f595138df113453f9f750926099c5908c296793345da4  /etc/corosync/corosync.conf
       14ace3030271e1b87b51030edbee561863e9cb2d98f567bc9959a3b10478ee2f  /etc/corosync/corosync.conf
       Recommended Action: Run "crm corosync push" to synchronize the configuration across cluster nodes.
  • Include node names in consistency mismatch output, like: node_name hash corosync_path
  • Suggest the user see the difference by running crm corosync diff before running push; Or, here can show the diff output directly

@liangxin1300

Copy link
Copy Markdown
Collaborator

health corosync could give a warning while using non-knet transport

@liangxin1300

Copy link
Copy Markdown
Collaborator

health corosync only check, not fix, right? If it doesn't support fix, the doc should mention that

@nicholasyang2022

nicholasyang2022 commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator Author

health corosync only check, not fix, right? If it doesn't support fix, the doc should mention that

--fix is not listed as an argument under * corosync: check corosync-related configurations and status.. This follows the same convention with existing health checks like sbd.

@liangxin1300

liangxin1300 commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator
  • --json crashes while validating corosync configure file;
  • The output of health corosync should be string, not bytes
sle16-1:~ # crm cluster health corosync 
[FAIL] Validate Corosync Configuration File
       b'Aug 05 14:10:41.949 error   [MAIN  ] parse error in config: This totem parser can only parse version 2 configurations.'
       b'Aug 05 14:10:41.949 error   [MAIN  ] Corosync Cluster Engine exiting with status 8 at main.c:1456.'

sle16-1:~ # crm cluster health corosync --json
ERROR: Object of type bytes is not JSON serializable
{"returncode": 8, "results": [{"check_name": "Validate Corosync Configuration File", "checked_nodes": ["sle16-1"], "returncode": 8, "result_description": Traceback (most recent call last):
  File "/sbin/crm", line 53, in <module>
    rc = main.run()
  File "/usr/lib/python3.13/site-packages/crmsh/main.py", line 385, in run
    return main_input_loop(context, user_args)
  File "/usr/lib/python3.13/site-packages/crmsh/main.py", line 256, in main_input_loop
    rc = handle_noninteractive_use(context, user_args)
  File "/usr/lib/python3.13/site-packages/crmsh/main.py", line 213, in handle_noninteractive_use
    if context.run(' '.join(l)):
       ~~~~~~~~~~~^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/crmsh/ui_context.py", line 93, in run
    rv = self.execute_command() is not False
         ~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/site-packages/crmsh/ui_context.py", line 276, in execute_command
    rv = self.command_info.function(*arglist)
  File "/usr/lib/python3.13/site-packages/crmsh/ui_cluster.py", line 939, in do_health
    json.dump(json_data, sys.stdout, ensure_ascii=False)
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/json/__init__.py", line 179, in dump
    for chunk in iterable:
                 ^^^^^^^^
  File "/usr/lib64/python3.13/json/encoder.py", line 433, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/usr/lib64/python3.13/json/encoder.py", line 407, in _iterencode_dict
    yield from chunks
  File "/usr/lib64/python3.13/json/encoder.py", line 326, in _iterencode_list
    yield from chunks
  File "/usr/lib64/python3.13/json/encoder.py", line 407, in _iterencode_dict
    yield from chunks
  File "/usr/lib64/python3.13/json/encoder.py", line 440, in _iterencode
    o = _default(o)
  File "/usr/lib64/python3.13/json/encoder.py", line 180, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
                    f'is not JSON serializable')
TypeError: Object of type bytes is not JSON serializable```

@nicholasyang2022
nicholasyang2022 force-pushed the feature/corosync-health-check branch from 51d9dfa to 2bb7412 Compare August 12, 2026 05:45
Introduce the corosync health check under 'crm cluster health corosync'.

- Add crmsh/corosync_healthcheck.py containing local configuration
  validity, cross-node consistency, quorum status, link status, and Node ID
  mapping validation.
- Implement ui_cluster.py integration for the 'corosync' health component.
- Write test/unittests/test_corosync_healthcheck.py and test_ui_cluster.py
  for comprehensive unit test coverage.
- Update doc/crm.8.adoc with help information and documentation.
- Add functional behavior tests in test/features/corosync_ui.feature.
Add check_deprecated_transport in health check to give a warning when
corosync uses a deprecated non-knet transport (such as udp or udpu).

Load LinkManager once in ui_cluster.py for corosync health checks and pass
it to check_nodeid_to_nodename_mapping and check_deprecated_transport.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants