Skip to content
8 changes: 8 additions & 0 deletions BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,14 @@ test_suite(
visibility = ["//visibility:public"],
)

test_suite(
name = "cit_tests_local",
tests = [
"//tests/test_cases:cit_cpp",
],
visibility = ["//visibility:public"],
)

test_suite(
name = "miri_tests",
tags = ["manual"],
Expand Down
129 changes: 81 additions & 48 deletions docs/verification_report/module_verification_report.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,86 +24,119 @@ Verification Report
:realizes: wp__verification_module_ver_report[version==1]
:tags: persistency

.. mod:: Persistency Module
:id: mod__persistency
:version: 1
:status: valid
:safety: ASIL_B
:security: NO
:includes: comp__persistency_kvs[version==1]

.. mod_ver_report:: Persistency Module Verification Summary
:id: mod_vrep__persistency__report
:version: 1
:safety: ASIL_B
:security: NO
:status: valid
:verification_method: test
:requirements_tests_completely_passed_percent: 100
:line_coverage_percent: 0
:branch_coverage_percent: 0
:applies_to_module_version: 1
:belongs_to: mod__persistency
:covers: comp_req__kvs__key_naming[version==1], comp_req__kvs__value_default[version==1], feat_arc_sta__persistency__static[version==1], feat_saf_dfa__persistency__execution_blocking[version==1], feat_saf_fmea__persistency__err_handl[version==1]
:evidence: doc__persistency_verification_report, doc__persistency_safety_analysis_fdr
:realizes: wp__verification_module_ver_report[version==1]

This verification report is based on the verification plan.
It covers all the components of the above stated module.
It covers the persistency module and all linked component/feature verification evidence.

Verification Report contains:

**1. Verification Coverage**

**1.1. on Requirements**
- Lists of component requirements (incl. AoU satisfied by the component) tested by which test case, passed/failed and completeness verdict
(this shall be generated by tools and accompanied by progress charts to be usable also for project steering)
- For external component Assumptions of Use: coverage by platform safety manual or feature/components incl. test case, passed/failed and completeness verdict
- This is split in a list of QM requirements tested and a separate list of tests for ASIL rated requirements.
- List of component requirements (ASIL rated) linked to inspection checklist and verdict (derived from PR export)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here it would be good to have overview pie charts like:

  • req/aou inspected/valid/invalid
  • req without testcases/with testcases linked
  • req tested passed/skipped/failed

.. needtable:: Requirements and AoU Traceability Coverage
:filter: type in ["comp_req", "aou_req"] and docname is not None and "kvs/docs/requirements" in docname and status == "valid"
:style: table
:sort: id
:columns: id as "ID";status as "Status";tags as "Tags";fully_verifies_back as "Fully Verified By Tests";partially_verifies_back as "Partially Verified By Tests"

**1.2. on Architecture**
- List of component architecture tags tested by which test case, passed/failed and completeness verdict
(this shall be generated by tools and accompanied by progress charts to be usable also for project steering)
- This is split in a list of QM components tested and a separate list of tests for ASIL rated components.
- List of component architecture tags (ASIL rated) linked to inspection checklist and verdict (derived from PR export)

.. needtable:: Architecture Verification Coverage
:filter: type in ["feat_arc_sta", "feat_arc_dyn", "comp_arc_sta", "comp_arc_dyn"] and status == "valid" and ("persistency" in id or "kvs" in id)
:style: table
:sort: id
:columns: id as "ID";type_name as "Type";status as "Status";fully_verifies_back as "Fully Verified By Tests";partially_verifies_back as "Partially Verified By Tests"

**1.3. on Detailed Design**
- List of detailed design tags tested by which test case, passed/failed and completeness verdict
(this shall be generated by tools and accompanied by progress charts to be usable also for project steering)
- This is split in a list of QM components tested and a separate list of tests for ASIL rated components.
- List of detailed design tags (ASIL rated) linked to inspection checklist and verdict (derived from PR export)

- The lists may also contain other verification methods like "Analysis" - process tbd
.. needtable:: Detailed Design Verification Coverage
:filter: id == "doc__kvs_detailed_design"
:style: table
:columns: id as "ID";status as "Status";realizes as "Realizes"

**2. DFA Report**
- :need:`doc__kvs_dfa` documents that DFA is performed on feature level :need:`doc__persistency_dfa`
- :need:`doc__persistency_safety_analysis_fdr` performed
The above requirement, architecture, and detailed design tables are generated from needs relations.

.. needtable:: List of performed feature DFA analyses
:filter: "persistency" in id and type == "feat_saf_dfa" and is_external == False
:style: table
:sort: status
:columns: id as "ID";mitigated_by as "Mitigation";sufficient as "Sufficient";status as "Status"
:colwidths: 40,30,15,15
**2. DFA Report**

DFA is performed. All mitigations are sufficient to reduce the risk to an acceptable level. The AoU is documented
in the safety manual.
.. needtable:: Performed Feature DFA Analyses
:filter: "persistency" in id and type == "feat_saf_dfa" and is_external == False
:style: table
:sort: id
:columns: id as "ID";mitigated_by as "Mitigation";sufficient as "Sufficient";status as "Status"
:colwidths: 40,30,15,15

**3. Safety Analysis Report**
- :need:`doc__kvs_fmea` documents that FMEA is performed on feature level :need:`doc__persistency_fmea`
- :need:`doc__persistency_safety_analysis_fdr` performed

.. needtable:: List of performed feature FMEA analyses
:filter: "persistency" in id and type == "feat_saf_fmea" and is_external == False
:style: table
:sort: status
:columns: id as "ID";mitigated_by as "Mitigation";sufficient as "Sufficient";status as "Status"
:colwidths: 40,30,15,15

FMEA is performed. All mitigations are sufficient to reduce the risk to an acceptable level. The AoU's are documented
in the safety manual.
.. needtable:: Performed Feature FMEA Analyses
:filter: "persistency" in id and type == "feat_saf_fmea" and is_external == False
:style: table
:sort: id
:columns: id as "ID";mitigated_by as "Mitigation";sufficient as "Sufficient";status as "Status"
:colwidths: 40,30,15,15

**4. Unit Verification Coverage**

**4.1. Structural Coverage**
- List of the units with C0(line) and C1(branch) coverage absolute and percentage
(this shall be generated by tools and accompanied by progress charts to be usable also for project steering)
- List includes a column for the safety rating of each unit.

Automatic structural/branch coverage needs are currently not available in this repository's needs dataset.
This section remains manual until unit coverage artifacts are exported as needs.

**4.2. Static Code Analysis**
- List of the units with compiler warning numbers and coding rule violation numbers
(this shall be generated by tools and accompanied by progress charts to be usable also for project steering)
- List includes a column for the safety rating of each unit.

**4.3. Manual Code Inspection**
- List of components (ASIL rated) linked to inspection checklist and verdict (derived from PR export)
Automatic static-analysis findings needs are currently not available in this repository's needs dataset.
This section remains manual until static-analysis artifacts are exported as needs.

**5. Software component qualification verification report**
- Contains the needed verification results if for some components a qualification of pre-developed SW is performed.

No separate pre-developed software qualification report is currently linked in this module.

**6. Test results**
- Test result per test case with status passed/failed/not_run

.. needpie:: Test Results
:labels: passed, failed, skipped
:colors: green, red, orange

type == 'testcase' and result == 'passed'
type == 'testcase' and result == 'failed'
type == 'testcase' and result == 'skipped'

.. needtable:: Test Result Per Testcase
:filter: type == "testcase"
:style: table
:sort: id
:columns: id as "Need ID";name as "Testcase";result as "Result";fully_verifies as "Fully Verifies";partially_verifies as "Partially Verifies"

**7. Test logs**
- Test log per test case with status passed/failed/not_run

.. needtable:: Test Logs Per Testcase

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is not clear for me what this table shows
links in the "need ID" do not work for me

:filter: type == "testcase"
:style: table
:sort: id
:columns: id as "Need ID";name as "Testcase";result_text as "Result Text";file as "File";line as "Line"

**Note1:** The verification report is valid for the module version tagged together with the report

Expand Down
Loading