-
Notifications
You must be signed in to change notification settings - Fork 28
adopt verification report #330
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
FScholPer
wants to merge
12
commits into
main
Choose a base branch
from
update_verification_report
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
b9a40cc
adopt verification report
FScholPer 482c283
remove local CIT debug config and docs note
FScholPer 12d9088
pin score_docs_as_code override to docs-as-code PR 612 commit
FScholPer 9940a95
Merge branch 'main' into update_verification_report
FScholPer 0b9bc43
fixed linking
FScholPer 4c9be38
adopt verification report
FScholPer 666b5b1
remove local CIT debug config and docs note
FScholPer b23e8f5
fixed linking
FScholPer eea3d1c
Merge branch 'update_verification_report' of https://github.com/eclip…
FScholPer fa527b2
updated lock
FScholPer 5c7ee04
removed inspections
FScholPer 10eaafd
Merge branch 'main' into update_verification_report
PandaeDo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -24,86 +24,119 @@ Verification Report | |
| :realizes: wp__verification_module_ver_report[version==1] | ||
| :tags: persistency | ||
|
|
||
| .. mod:: Persistency Module | ||
| :id: mod__persistency | ||
| :version: 1 | ||
| :status: valid | ||
| :safety: ASIL_B | ||
| :security: NO | ||
| :includes: comp__persistency_kvs[version==1] | ||
|
|
||
| .. mod_ver_report:: Persistency Module Verification Summary | ||
| :id: mod_vrep__persistency__report | ||
| :version: 1 | ||
| :safety: ASIL_B | ||
| :security: NO | ||
| :status: valid | ||
| :verification_method: test | ||
| :requirements_tests_completely_passed_percent: 100 | ||
| :line_coverage_percent: 0 | ||
| :branch_coverage_percent: 0 | ||
| :applies_to_module_version: 1 | ||
| :belongs_to: mod__persistency | ||
| :covers: comp_req__kvs__key_naming[version==1], comp_req__kvs__value_default[version==1], feat_arc_sta__persistency__static[version==1], feat_saf_dfa__persistency__execution_blocking[version==1], feat_saf_fmea__persistency__err_handl[version==1] | ||
| :evidence: doc__persistency_verification_report, doc__persistency_safety_analysis_fdr | ||
| :realizes: wp__verification_module_ver_report[version==1] | ||
|
|
||
| This verification report is based on the verification plan. | ||
| It covers all the components of the above stated module. | ||
| It covers the persistency module and all linked component/feature verification evidence. | ||
|
|
||
| Verification Report contains: | ||
|
|
||
| **1. Verification Coverage** | ||
|
|
||
| **1.1. on Requirements** | ||
| - Lists of component requirements (incl. AoU satisfied by the component) tested by which test case, passed/failed and completeness verdict | ||
| (this shall be generated by tools and accompanied by progress charts to be usable also for project steering) | ||
| - For external component Assumptions of Use: coverage by platform safety manual or feature/components incl. test case, passed/failed and completeness verdict | ||
| - This is split in a list of QM requirements tested and a separate list of tests for ASIL rated requirements. | ||
| - List of component requirements (ASIL rated) linked to inspection checklist and verdict (derived from PR export) | ||
|
|
||
| .. needtable:: Requirements and AoU Traceability Coverage | ||
| :filter: type in ["comp_req", "aou_req"] and docname is not None and "kvs/docs/requirements" in docname and status == "valid" | ||
| :style: table | ||
| :sort: id | ||
| :columns: id as "ID";status as "Status";tags as "Tags";fully_verifies_back as "Fully Verified By Tests";partially_verifies_back as "Partially Verified By Tests" | ||
|
|
||
| **1.2. on Architecture** | ||
| - List of component architecture tags tested by which test case, passed/failed and completeness verdict | ||
| (this shall be generated by tools and accompanied by progress charts to be usable also for project steering) | ||
| - This is split in a list of QM components tested and a separate list of tests for ASIL rated components. | ||
| - List of component architecture tags (ASIL rated) linked to inspection checklist and verdict (derived from PR export) | ||
|
|
||
| .. needtable:: Architecture Verification Coverage | ||
| :filter: type in ["feat_arc_sta", "feat_arc_dyn", "comp_arc_sta", "comp_arc_dyn"] and status == "valid" and ("persistency" in id or "kvs" in id) | ||
| :style: table | ||
| :sort: id | ||
| :columns: id as "ID";type_name as "Type";status as "Status";fully_verifies_back as "Fully Verified By Tests";partially_verifies_back as "Partially Verified By Tests" | ||
|
|
||
| **1.3. on Detailed Design** | ||
| - List of detailed design tags tested by which test case, passed/failed and completeness verdict | ||
| (this shall be generated by tools and accompanied by progress charts to be usable also for project steering) | ||
| - This is split in a list of QM components tested and a separate list of tests for ASIL rated components. | ||
| - List of detailed design tags (ASIL rated) linked to inspection checklist and verdict (derived from PR export) | ||
|
|
||
| - The lists may also contain other verification methods like "Analysis" - process tbd | ||
| .. needtable:: Detailed Design Verification Coverage | ||
| :filter: id == "doc__kvs_detailed_design" | ||
| :style: table | ||
| :columns: id as "ID";status as "Status";realizes as "Realizes" | ||
|
|
||
| **2. DFA Report** | ||
| - :need:`doc__kvs_dfa` documents that DFA is performed on feature level :need:`doc__persistency_dfa` | ||
| - :need:`doc__persistency_safety_analysis_fdr` performed | ||
| The above requirement, architecture, and detailed design tables are generated from needs relations. | ||
|
|
||
| .. needtable:: List of performed feature DFA analyses | ||
| :filter: "persistency" in id and type == "feat_saf_dfa" and is_external == False | ||
| :style: table | ||
| :sort: status | ||
| :columns: id as "ID";mitigated_by as "Mitigation";sufficient as "Sufficient";status as "Status" | ||
| :colwidths: 40,30,15,15 | ||
| **2. DFA Report** | ||
|
|
||
| DFA is performed. All mitigations are sufficient to reduce the risk to an acceptable level. The AoU is documented | ||
| in the safety manual. | ||
| .. needtable:: Performed Feature DFA Analyses | ||
| :filter: "persistency" in id and type == "feat_saf_dfa" and is_external == False | ||
| :style: table | ||
| :sort: id | ||
| :columns: id as "ID";mitigated_by as "Mitigation";sufficient as "Sufficient";status as "Status" | ||
| :colwidths: 40,30,15,15 | ||
|
|
||
| **3. Safety Analysis Report** | ||
| - :need:`doc__kvs_fmea` documents that FMEA is performed on feature level :need:`doc__persistency_fmea` | ||
| - :need:`doc__persistency_safety_analysis_fdr` performed | ||
|
|
||
| .. needtable:: List of performed feature FMEA analyses | ||
| :filter: "persistency" in id and type == "feat_saf_fmea" and is_external == False | ||
| :style: table | ||
| :sort: status | ||
| :columns: id as "ID";mitigated_by as "Mitigation";sufficient as "Sufficient";status as "Status" | ||
| :colwidths: 40,30,15,15 | ||
|
|
||
| FMEA is performed. All mitigations are sufficient to reduce the risk to an acceptable level. The AoU's are documented | ||
| in the safety manual. | ||
| .. needtable:: Performed Feature FMEA Analyses | ||
| :filter: "persistency" in id and type == "feat_saf_fmea" and is_external == False | ||
| :style: table | ||
| :sort: id | ||
| :columns: id as "ID";mitigated_by as "Mitigation";sufficient as "Sufficient";status as "Status" | ||
| :colwidths: 40,30,15,15 | ||
|
|
||
| **4. Unit Verification Coverage** | ||
|
|
||
| **4.1. Structural Coverage** | ||
| - List of the units with C0(line) and C1(branch) coverage absolute and percentage | ||
| (this shall be generated by tools and accompanied by progress charts to be usable also for project steering) | ||
| - List includes a column for the safety rating of each unit. | ||
|
|
||
| Automatic structural/branch coverage needs are currently not available in this repository's needs dataset. | ||
| This section remains manual until unit coverage artifacts are exported as needs. | ||
|
|
||
| **4.2. Static Code Analysis** | ||
| - List of the units with compiler warning numbers and coding rule violation numbers | ||
| (this shall be generated by tools and accompanied by progress charts to be usable also for project steering) | ||
| - List includes a column for the safety rating of each unit. | ||
|
|
||
| **4.3. Manual Code Inspection** | ||
| - List of components (ASIL rated) linked to inspection checklist and verdict (derived from PR export) | ||
| Automatic static-analysis findings needs are currently not available in this repository's needs dataset. | ||
| This section remains manual until static-analysis artifacts are exported as needs. | ||
|
|
||
| **5. Software component qualification verification report** | ||
| - Contains the needed verification results if for some components a qualification of pre-developed SW is performed. | ||
|
|
||
| No separate pre-developed software qualification report is currently linked in this module. | ||
|
|
||
| **6. Test results** | ||
| - Test result per test case with status passed/failed/not_run | ||
|
|
||
| .. needpie:: Test Results | ||
| :labels: passed, failed, skipped | ||
| :colors: green, red, orange | ||
|
|
||
| type == 'testcase' and result == 'passed' | ||
| type == 'testcase' and result == 'failed' | ||
| type == 'testcase' and result == 'skipped' | ||
|
|
||
| .. needtable:: Test Result Per Testcase | ||
| :filter: type == "testcase" | ||
| :style: table | ||
| :sort: id | ||
| :columns: id as "Need ID";name as "Testcase";result as "Result";fully_verifies as "Fully Verifies";partially_verifies as "Partially Verifies" | ||
|
|
||
| **7. Test logs** | ||
| - Test log per test case with status passed/failed/not_run | ||
|
|
||
| .. needtable:: Test Logs Per Testcase | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. it is not clear for me what this table shows |
||
| :filter: type == "testcase" | ||
| :style: table | ||
| :sort: id | ||
| :columns: id as "Need ID";name as "Testcase";result_text as "Result Text";file as "File";line as "Line" | ||
|
|
||
| **Note1:** The verification report is valid for the module version tagged together with the report | ||
|
|
||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here it would be good to have overview pie charts like: