-
Notifications
You must be signed in to change notification settings - Fork 15
feat: Quality pack targets #277
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 5 commits
97767df
921722c
7fc7786
716c734
c1f9b39
5f4abbf
8590671
53850b1
f6e9d7e
fd12941
461ff90
775713d
c23da1e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| .. | ||
| # ******************************************************************************* | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This .rst does not fit the S-CORE module template. I suggest move this a into the README.md in a section, next to the existing "Coverage report" one: ## Quality pack
Requirements traceability is produced by the documentation build. Run the tests
first: the docs build reads `bazel-testlogs` to link each requirement to the
tests that verify it.
```sh
bazel test //:unit_tests //:component_tests
bazel run //:docs
```
This generates:
- `_build/index.html` — documentation, showing the source-code and test
links on each requirement
- `_build/needs.json` — the requirements model
- `_build/metrics.json` — traceability metrics
To print the traceability summary on the command line:
```sh
bazel run //:traceability_gate -- --metrics-json "$PWD/_build/metrics.json"
```
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done |
||
| # Copyright (c) 2026 Contributors to the Eclipse Foundation | ||
| # | ||
| # See the NOTICE file(s) distributed with this work for additional | ||
| # information regarding copyright ownership. | ||
| # | ||
| # This program and the accompanying materials are made available under the | ||
| # terms of the Apache License Version 2.0 which is available at | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # ******************************************************************************* | ||
|
|
||
| Quality Pack Targets | ||
| #################### | ||
|
|
||
| The ``score_someip_gateway`` module plugs into the Score docs-as-code | ||
| dashboards and quality gates as described in the upstream how-to: | ||
| https://eclipse-score.github.io/docs-as-code/main/how-to/dashboards_and_quality_gates.html. | ||
|
|
||
| The Bazel targets below are the ones consumed by CI to produce | ||
| dashboard artefacts and to enforce traceability thresholds. | ||
|
|
||
| Unit tests | ||
| ========== | ||
|
|
||
| - **Tag:** ``unit`` (carried by every ``cc_test`` intended as a unit | ||
| test, currently ``//score/serializer:null_serializer_test`` and | ||
| ``//score/socom/test/unit:socom_test``). | ||
| - **Aggregate target:** ``//:unit_tests``. | ||
| - **Command:** ``bazel test //:unit_tests``. | ||
| - **Results:** JUnit XML and stdout log per test target under | ||
| ``bazel-testlogs/<package>/<test>/{test.log,test.xml}``. | ||
|
|
||
| Component tests | ||
| =============== | ||
|
|
||
| Component tests exercise a component together with real collaborators | ||
| (SOCom runtime plus the gateway IPC binding, or the SOCom stress harness). | ||
| They cross more than one translation unit and rely on the real | ||
| ``message_passing`` / shared-memory transports. | ||
|
|
||
| - **Tag:** ``component``. | ||
| - **Aggregate target:** ``//:component_tests``. | ||
| - **Command:** ``bazel test //:component_tests``. | ||
| - **Included tests (existing tests reclassified, not new ones):** | ||
|
|
||
| - ``//score/gateway_ipc_binding/test:gateway_ipc_binding_test`` | ||
| - ``//score/socom/test/stress:socom_stress_test`` | ||
|
|
||
| - **Results:** JUnit XML and stdout log per test target under | ||
| ``bazel-testlogs/<package>/<test>/{test.log,test.xml}``. | ||
|
|
||
| Code coverage | ||
| ============= | ||
|
|
||
| - **Config:** ``coverage.bazelrc`` (LLVM ``llvm-cov`` toolchain). | ||
| - **Command:** ``bazel coverage --config=coverage //:unit_tests //:component_tests``. | ||
| - **Results:** raw ``lcov`` data under | ||
| ``$(bazel info output_path)/_coverage/_coverage_report.dat``. Report | ||
| post-processing (HTML / Cobertura) uses the standard | ||
| ``score_tooling`` coverage flow shared with the rest of S-CORE. | ||
|
|
||
| Requirements traceability (dashboards + gate) | ||
| ============================================= | ||
|
|
||
| Component requirements live alongside each component under | ||
| ``score/<component>/docs/requirements/requirements.rst`` and use the | ||
| Score metamodel ``comp_req::`` directive. The externally-visible | ||
| feature-level requirements (``feat_req__some_ip_gateway__*``) belong to | ||
| the upstream ``eclipse-score/score`` repo (see | ||
| `Open SOME/IP <https://github.com/some-ip-com/open-someip-spec>`_ for | ||
| the protocol these features track). Once ``@score_platform`` / | ||
| ``@score_process`` needs.json can be consumed cleanly from this repo, | ||
| they will be pulled in via the ``external_needs`` attribute of the | ||
| root ``docs()`` macro. | ||
|
|
||
| Source-code and test-code links are consumed by ``score_docs_as_code``. | ||
| See the upstream how-to for the marker syntax, the required GoogleTest | ||
| ``RecordProperty`` fields, and the validation rules that decide whether a | ||
| link is counted: | ||
| https://eclipse-score.github.io/docs-as-code/main/how-to/dashboards_and_quality_gates.html. | ||
|
|
||
| Repo-local wiring: | ||
|
|
||
| - **Source-code markers** live in the C++ implementation files. Per | ||
| component, the marked files are collected into a | ||
| ``requirement_marked_sources`` ``filegroup`` (for example | ||
| ``//score/gatewayd:requirement_marked_sources``) and passed to that | ||
| component's ``docs_bundle`` via ``code_targets``. Grep the tree for | ||
| ``# req-Id:`` to see current examples. | ||
| - **Test-code links** use ``RecordProperty`` inside the test body. Grep | ||
| for ``FullyVerifies`` to see current examples (for instance | ||
| ``//score/socom/test/unit:socom_test``). | ||
| - **Bazel target:** ``//:docs`` — HTML output plus ``_build/needs.json`` | ||
| and ``_build/metrics.json`` (traceability metrics extracted from needs). | ||
| - **Local flow** — order matters, the docs build reads | ||
| ``bazel-testlogs`` for test links: | ||
|
|
||
| .. code-block:: bash | ||
|
|
||
| bazel test //:unit_tests //:component_tests | ||
| bazel run //:docs | ||
|
|
||
| Live coverage numbers are posted as a sticky comment on every pull | ||
| request by ``.github/workflows/quality_pack_comment.yml`` (rendered by | ||
| ``tools/quality_pack/render_pr_comment.py`` from ``_build/metrics.json``); | ||
| see the workflow's uploaded ``quality-pack-metrics`` artifact for the | ||
| raw JSON. The initial gate thresholds — matching the baseline at | ||
| introduction so the gate never regresses without a follow-up ticket — | ||
| are: | ||
|
|
||
| ==================== ===== =============================== | ||
| Threshold Value Meaning | ||
| ==================== ===== =============================== | ||
| min-req-code 67 ``with_code_link_pct`` ≥ 67 | ||
| min-req-test 9 ``with_test_link_pct`` ≥ 9 | ||
| min-req-fully-linked 9 ``fully_linked_pct`` ≥ 9 | ||
| min-tests-linked 1 ≥ 1 test carrying ``FullyVerifies`` | ||
| ==================== ===== =============================== | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # ******************************************************************************* | ||
| # Copyright (c) 2026 Contributors to the Eclipse Foundation | ||
| # | ||
| # See the NOTICE file(s) distributed with this work for additional | ||
| # information regarding copyright ownership. | ||
| # | ||
| # This program and the accompanying materials are made available under the | ||
| # terms of the Apache License Version 2.0 which is available at | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # ******************************************************************************* | ||
|
|
||
| load("@score_docs_as_code//:docs.bzl", "docs_bundle") | ||
|
|
||
| # Docs bundle for this component. Kept in its own package so the parent | ||
| # BUILD does not need to load @score_docs_as_code. | ||
| docs_bundle( | ||
| name = "docs_bundle", | ||
| code_targets = ["//score/config:requirement_marked_sources"], | ||
| source_dir = "", | ||
| visibility = ["//visibility:public"], | ||
| ) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| .. | ||
| # ******************************************************************************* | ||
| # Copyright (c) 2026 Contributors to the Eclipse Foundation | ||
| # | ||
| # See the NOTICE file(s) distributed with this work for additional | ||
| # information regarding copyright ownership. | ||
| # | ||
| # This program and the accompanying materials are made available under the | ||
| # terms of the Apache License Version 2.0 which is available at | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # ******************************************************************************* | ||
|
|
||
| .. _config: | ||
|
|
||
| Config | ||
| ###### | ||
|
|
||
| .. comp:: Config | ||
| :id: comp__config | ||
| :security: NO | ||
| :safety: QM | ||
| :status: valid | ||
| :version: 1 | ||
| :belongs_to: feat__someip_gateway | ||
|
|
||
| Abstract | ||
| ======== | ||
|
|
||
| Provides the FlatBuffers schema (mw_someip_config.fbs) and JSON schema used to define and validate gateway configuration. | ||
|
|
||
| .. toctree:: | ||
| :maxdepth: 1 | ||
|
|
||
| requirements/index |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| .. | ||
| # ******************************************************************************* | ||
| # Copyright (c) 2026 Contributors to the Eclipse Foundation | ||
| # | ||
| # See the NOTICE file(s) distributed with this work for additional | ||
| # information regarding copyright ownership. | ||
| # | ||
| # This program and the accompanying materials are made available under the | ||
| # terms of the Apache License Version 2.0 which is available at | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # ******************************************************************************* | ||
|
|
||
| Requirements | ||
| ############ | ||
|
|
||
| .. toctree:: | ||
|
|
||
| requirements |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| .. | ||
| # ******************************************************************************* | ||
| # Copyright (c) 2026 Contributors to the Eclipse Foundation | ||
| # | ||
| # See the NOTICE file(s) distributed with this work for additional | ||
| # information regarding copyright ownership. | ||
| # | ||
| # This program and the accompanying materials are made available under the | ||
| # terms of the Apache License Version 2.0 which is available at | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # ******************************************************************************* | ||
|
|
||
| Component Config Requirements | ||
| ############################# | ||
|
|
||
| Functional Requirements | ||
| ----------------------- | ||
|
|
||
| .. comp_req:: FlatBuffers gateway configuration schema | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The PR description scopes requirements to the SOME/IP protocol specification. These are derived from the existing implementation instead, so they fall outside that scope — please remove them.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done |
||
| :id: comp_req__config__flatbuffer_schema | ||
| :reqtype: Interface | ||
| :security: NO | ||
| :safety: QM | ||
| :status: valid | ||
| :version: 1 | ||
| :satisfied_by: comp__config | ||
|
|
||
| The gateway configuration shall be defined as a FlatBuffers schema | ||
| (``mw_someip_config.fbs``) together with a JSON schema | ||
| (``mw_someip_config.schema.json``) so both binary and JSON forms of | ||
| the configuration can be validated against a single source of truth. | ||
|
|
||
| .. comp_req:: Service instance identity keyed by SOME/IP IDs | ||
| :id: comp_req__config__service_instance_ids | ||
| :reqtype: Interface | ||
| :security: NO | ||
| :safety: QM | ||
| :status: valid | ||
| :version: 1 | ||
| :satisfied_by: comp__config | ||
|
|
||
| The configuration schema shall represent every service via a 16-bit | ||
| ``service_id``, an 8-bit ``service_version_major`` and a 32-bit | ||
| ``service_version_minor`` (SOME/IP wildcard ``0xFFFFFFFF`` allowed), | ||
| and every service instance via a 16-bit ``instance_id``, so that the | ||
| on-wire identity of every configured service and instance matches | ||
| the SOME/IP header fields defined by the open protocol | ||
| specification. | ||
|
|
||
| .. comp_req:: Separate local and remote service instance sets per service type | ||
| :id: comp_req__config__local_remote_split | ||
| :reqtype: Functional | ||
| :security: NO | ||
| :safety: QM | ||
| :status: valid | ||
| :version: 1 | ||
| :satisfied_by: comp__config | ||
|
|
||
| Each ``ServiceType`` shall carry two disjoint lists, | ||
| ``local_service_instances`` (offered from this host) and | ||
| ``remote_service_instances`` (consumed from remote hosts), so the | ||
| gateway can decide from configuration alone which instances to | ||
| publish and which to subscribe to. | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.