feat: Add integration test framework by lurtz · Pull Request #126 · eclipse-score/inc_someip_gateway

lurtz · 2026-05-18T20:14:24Z

We need a way to run integration tests and in the best case they are oblivious to the platform they are run on. For that reason the code from communication/quality has been copied, but the sanitizer tests have been excluded.

In contrast to the code from communication, all tests are run using QEMU and Linux and QNX for the simple reason that then network setup is uniform. For supporting Linux a new ITF plugin has been written and the same tests pass using Linux and QNX.

For test stability the decision has been made that per Python test file and bazel target only a single test will be run. Otherwise there might be stale state from previous tests influencing the behavior of the current test.

#19 is similar. It focuses only on QNX and runs two QEMU instances.

github-actions · 2026-05-18T20:17:49Z

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run //:license-check

Status: ⚠️ Needs Review

Click to expand output

[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.6.0) and connecting to it...
INFO: Invocation ID: 74412530-f5a1-4e96-a327-697ce11374e9
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 3 packages loaded
Loading: 3 packages loaded
    currently loading: 
Loading: 3 packages loaded
    currently loading: 
Loading: 3 packages loaded
    currently loading: 
Loading: 3 packages loaded
    currently loading: 
Analyzing: target //:license-check (4 packages loaded, 0 targets configured)
Analyzing: target //:license-check (4 packages loaded, 0 targets configured)

Analyzing: target //:license-check (36 packages loaded, 10 targets configured)

Analyzing: target //:license-check (125 packages loaded, 461 targets configured)

Analyzing: target //:license-check (166 packages loaded, 5459 targets configured)

Analyzing: target //:license-check (174 packages loaded, 9493 targets configured)

Analyzing: target //:license-check (174 packages loaded, 9493 targets configured)

Analyzing: target //:license-check (175 packages loaded, 9616 targets configured)

Analyzing: target //:license-check (179 packages loaded, 11624 targets configured)

Analyzing: target //:license-check (179 packages loaded, 11624 targets configured)

Analyzing: target //:license-check (179 packages loaded, 11624 targets configured)

INFO: Analyzed target //:license-check (180 packages loaded, 13203 targets configured).
[16 / 17] Generating Dash formatted dependency file ...; 0s disk-cache
INFO: From Generating Dash formatted dependency file ...:
INFO: Successfully converted 2 packages from Cargo.lock to bazel-out/k8-fastbuild/bin/formatted.txt
INFO: Found 1 target...
Target //:license.check.license_check up-to-date:
  bazel-bin/license.check.license_check
  bazel-bin/license.check.license_check.jar
INFO: Elapsed time: 166.083s, Critical Path: 0.21s
INFO: 17 processes: 5 disk cache hit, 12 internal.
INFO: Build completed successfully, 17 total actions
INFO: Running command line: bazel-bin/license.check.license_check ./formatted.txt <args omitted>
usage: org.eclipse.dash.licenses.cli.Main [-batch <int>] [-cd <url>]
       [-confidence <int>] [-ef <url>] [-excludeSources <sources>] [-help] [-lic
       <url>] [-project <shortname>] [-repo <url>] [-review] [-summary <file>]
       [-timeout <seconds>] [-token <token>]

github-actions · 2026-05-18T20:19:09Z

The created documentation from the pull request is available at: docu-html

This reverts commit 8f84bd0.

TimofteBogdan1900 · 2026-06-23T14:38:20Z

+        ping_process = bash_process.__enter__()
+        assert ping_process is not None, "Failed to start ping process"
+        assert ping_process.is_running(), bash_process.get_output().decode()
+        assert tcpdump_process.poll() is None, get_output(tcpdump_process)


i can confirm that this test fails in wsl2
bazel test --config=qemu-integration //tests/integration_test:test_tcpdump --nocache_test_results --test_output=errors

assert tcpdump_process.poll() is None, get_output(tcpdump_process)

E AssertionError:

i have no solution :(

TimofteBogdan1900

done

…ergration-tests

This reverts commit 1dbbc55.

lurtz · 2026-06-25T15:53:00Z

Prefer to merge #166 before this PR

Co-authored-by: lurtz <727209+lurtz@users.noreply.github.com> Signed-off-by: lurtz <727209+lurtz@users.noreply.github.com>

This reverts commit d35e4b8.

in CI and one time locally tcpdump failed to capure ICMP messages. Thus use a more defensive approach

…ergration-tests

opajonk

I briefly checked this for what I know. Review by @TimofteBogdan1900 is good for me!

lurtz added 10 commits May 13, 2026 17:49

add hello world itf test

5fa6733

add docker in docker feature for running itf

7520403

stricter capabilities check

8c7b658

copy integration test framework from communication

b4f24d4

add quality to QNX tests

c6ffa97

delete unused files

1342568

restore docs-as-code 3.0.0 to fix build

2cb9000

actually run tests on QNX

350a350

remove tests already present in communication

9775c06

use actual qnx workflow

073b6fe

lurtz added 18 commits May 19, 2026 09:12

add another qnx toolchain

8df4cf1

use_repo score qcc toolchain pkg

3de7ab9

bump QNX version to 8.0.3

0273b7b

bump score bazel cpp toolchains to 0.5.1

057c091

run integration tests in Linux via QEMU

03f49e0

at the moment no ARM64 QNX image is build

ced97e3

add Qemu, ping and genisoimage to workflow

c90fa43

add Qemu, ping and genisoimage to devcontainer

902c043

only build seed.iso if really needed

fca75e4

proper QNX x86_64 constraint

1cd3779

avoid patching ITF and use a plugin instead

ec5322f

remove unused config_setting

96c49f0

remove not needed user-data

8f84bd0

run Linux QEMU tests in CI

d56b344

enable KVM

f25f084

Revert "remove not needed user-data"

1005d20

This reverts commit 8f84bd0.

ensure Ubuntu image is not permanently altered by tests

d48ca0a

only build linux qemu plugin on linux

2d3cc49

TimofteBogdan1900 reviewed Jun 17, 2026

View reviewed changes

Comment thread .bazelrc Outdated

TimofteBogdan1900 reviewed Jun 17, 2026

View reviewed changes

Comment thread quality/integration_testing/integration_testing.bzl

lurtz added 3 commits June 17, 2026 20:49

Fix host qemu workflow

2be6d11

block internet access for integration tests

e2c571c

Only enforce network namespace for integration tests

37bbe54

TimofteBogdan1900 reviewed Jun 23, 2026

View reviewed changes

Comment thread MODULE.bazel Outdated

TimofteBogdan1900 reviewed Jun 23, 2026

View reviewed changes

Comment thread bazel/rules/expand_template.bzl Outdated

TimofteBogdan1900 reviewed Jun 23, 2026

View reviewed changes

TimofteBogdan1900 approved these changes Jun 23, 2026

View reviewed changes

lurtz added 8 commits June 25, 2026 11:58

Merge remote-tracking branch 'origin/main' into add-score-itf-for-int…

49cfb1d

…ergration-tests

delete unused expand_template

af5e3c7

Use Ubuntu image from fixed date

ed8bc51

use s-core setup-bazel-cache

d5554b1

remove test QNX job

1dbbc55

remove unneeded file export

31b3716

Revert "remove test QNX job"

2309d8e

This reverts commit 1dbbc55.

patch score_tooling to remove Android dependency

d35e4b8

lurtz and others added 4 commits June 26, 2026 14:54

Remove build_and_test_qnx from CI job

948e8a5

Co-authored-by: lurtz <727209+lurtz@users.noreply.github.com> Signed-off-by: lurtz <727209+lurtz@users.noreply.github.com>

Revert "patch score_tooling to remove Android dependency"

e7ebd8d

This reverts commit d35e4b8.

resend ping if tcpdump capture failed

a6e3b62

in CI and one time locally tcpdump failed to capure ICMP messages. Thus use a more defensive approach

Merge remote-tracking branch 'origin/main' into add-score-itf-for-int…

3c5bc0b

…ergration-tests

lurtz enabled auto-merge June 30, 2026 12:27

opajonk approved these changes Jun 30, 2026

View reviewed changes

lurtz added this pull request to the merge queue Jun 30, 2026

Merged via the queue into main with commit b4971d1 Jun 30, 2026
16 checks passed

lurtz deleted the add-score-itf-for-intergration-tests branch June 30, 2026 12:52

mariuswbr mentioned this pull request Jul 8, 2026

feat: TC8 SOME/IP conformance test suite for someipd #60

Open

Navigation Menu

Uh oh!

feat: Add integration test framework - #126

feat: Add integration test framework#126
lurtz merged 135 commits into
mainfrom
add-score-itf-for-intergration-tests

lurtz commented May 18, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 18, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 18, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

TimofteBogdan1900 Jun 23, 2026 •

edited

Loading

Uh oh!

Uh oh!

TimofteBogdan1900 left a comment

Uh oh!

lurtz commented Jun 25, 2026 •

edited

Loading

Uh oh!

opajonk left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

lurtz commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

License Check Results

Uh oh!

github-actions Bot commented May 18, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

TimofteBogdan1900 Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

TimofteBogdan1900 left a comment

Choose a reason for hiding this comment

Uh oh!

lurtz commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

opajonk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

lurtz commented May 18, 2026 •

edited

Loading

github-actions Bot commented May 18, 2026 •

edited

Loading

TimofteBogdan1900 Jun 23, 2026 •

edited

Loading

lurtz commented Jun 25, 2026 •

edited

Loading