Repository navigation
Replies: 1 comment 6 replies
|
I hope this is an AI created ticket ;-) ad 1. Looks okay ad 21. ad 22: You are not the only one I know of, that is struggeling with the Testbed for Oman. That's why my best shot is to assume an error in the Testbed |


Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hi Philip,
We are currently implementing and completing the Oman Peppol Testbed qualification for our Access Point using phoss-ap / phase4.
We are now blocked on all four official Oman outbound Testbed scenarios at exactly the same stage: the OM Tax Data Document (TDD).
We are NOT reporting this as a confirmed phase4 bug.
We are asking for your technical guidance because we have performed extensive application-level, XML, SMP, certificate, AS4 wire, correlation and Testbed timeline analysis, but the remaining failure appears to happen inside the remote Testbed test-case processing path.
We already have an OpenPeppol Service Desk case:
PEPPOL-25142
This is currently blocking us from completing the required Oman Peppol Testbed qualification/onboarding/accreditation steps, so we would greatly appreciate any guidance that could help us resolve this as soon as possible.
1. Environment / participants
Our Access Point participant:
0248:POM001192Business-document Testbed receiver:
9922:OPTBCNTRLP1007OM Tax Data Document / TDD receiver:
0242:000006-TESTBED.3001The TDD resolves and is actually transmitted to:
https://c5-phase4-controller.testbed.peppol.org/rcv1TDD Document Type:
urn:peppol:schema:om-taxdata:1.0::TaxData##urn:peppol:taxdata:om-1::1.0TDD Document Type scheme:
busdox-docid-qnsTDD Process:
urn:peppol:taxreportingProcess scheme:
cenbii-procid-ublAS4 transport profile:
peppol-transport-as4-v2_0Our current AP certificate is a Peppol TEST G3 certificate.
SHA-256:
4d74a0f2a889a0d942ac7a4f1e042f8785e93569fbfe1c2529abaafd85edb794Serial:
5b21099a03554307c5b1ec2a870b11c3334a16cbSubject:
CN=POM001192,OU=PEPPOL TEST AP,O=Elsayed Hamdi For Advanced Technology,C=OMIssuer:
CN=PEPPOL ACCESS POINT TEST CA - G3,OU=FOR TEST ONLY,O=OpenPEPPOL AISBL,C=BEValidity:
2026-07-22 -> 2028-07-10The private key has been verified to match this certificate.
2. The problem
The same TDD-stage failure blocks all four official Oman outbound Testbed cases:
The Business Document side succeeds.
The observed sequence is:
This distinction is very important:
The TDD is NOT simply "not sent".
Our phase4 wire evidence proves that the TDD is transmitted to the remote C5 receiver and the remote Testbed returns an AS4 Error approximately one second later.
The 15-minute TDD_RECEPTION timeout shown by the Testbed UI occurs later.
It is a consequence/symptom of the missing TDD MLS, not the first failure event.
3. Representative case — Billing Invoice
Testbed RESTART:
2026-08-06T19:00:24ZBusiness UBL ID:
AUTO-TB-INV-20260806-225554Business UUID:
3e81fc5a-24b9-5360-b16d-d2e3d289dd08Business SBDH InstanceIdentifier:
fe1afeff-62c9-46f5-bebf-334b157fd6d4Business AS4 MessageId:
c4b5e227-0f49-42fa-a8e2-b148f24a11ac@phase4.phoss-apBusiness transmitted:
2026-08-06T19:00:55ZRemote Business response:
AS4 ReceiptTestbed:
BD_RECEPTION = PASSEDat:
2026-08-06T19:00:56ZBusiness MLS InstanceIdentifier:
1fe13cae-8a96-4efa-bdeb-646d5c7fe5c4Business MLS ResponseCode:
ABBusiness MLS description:
The document passed validationBusiness MLS received:
2026-08-06T19:00:57ZCorresponding OM TDD:
TDD UUID:
1c88d2aa-378d-4d69-b20a-845fe218f4e1TDD SBDH InstanceIdentifier:
c48f0ae8-4dbe-4da8-9abb-8f9307602cc0ReporterRole:
01ReportedDocument/ID:
AUTO-TB-INV-20260806-225554ReportedDocument/UUID:
3e81fc5a-24b9-5360-b16d-d2e3d289dd08TransportHeaderID:
Absent in this historical case.
It is optional in the published OM TDD schema.
TDD transmitted:
2026-08-06T19:07:45ZTDD AS4 MessageId:
40335676-6e75-4ba4-9710-78e5d9a2ba7b@phase4.phoss-apRemote AS4 Error received:
2026-08-06T19:07:46ZTestbed Trace:
f56d660b-1ba9-4581-a2c5-ebc874f62146TDD MLS received:
NONETestbed TDD_RECEPTION timeout:
2026-08-06T19:15:57Z4. Exact remote AS4 error
The error received from the remote side contains:
category = ContenterrorCode = EBMS:0004severity = failureshortDescription = OtherDescription = An undefined error occurred.The ErrorDetail contains:
com.helger.phase4.peppol.Phase4PeppolException - TestBed Error Responsewith:
Code=-5001and the Testbed description:
There is an internal error that prevented the execution of the test case. Please contact OpenPeppol service desk with the trace reference.Representative trace:
f56d660b-1ba9-4581-a2c5-ebc874f62146Our local phase4 result is:
AS4_ERROR_MESSAGE_RECEIVEDImportant:
The value
-5001is already present inside the remote Testbed ErrorDetail.It is not a local code that our application or phase4 translated into
-5001.5. All four official failing cases
CASE 1 — Billing Invoice
Business SBDH:
fe1afeff-62c9-46f5-bebf-334b157fd6d4TDD SBDH:
c48f0ae8-4dbe-4da8-9abb-8f9307602cc0TDD attempts:
1Trace:
f56d660b-1ba9-4581-a2c5-ebc874f62146Result:
EBMS:0004 / -5001TDD MLS:
NONECASE 2 — Billing Credit Note
Business SBDH:
f07869dd-ea97-45f3-a225-9c89828331a7TDD SBDH:
bd651fb3-6302-40c4-a95d-bea5521de2bcTDD attempts:
3Attempt 1 trace:
77aa7fe2-302d-4150-9ac6-e982638fe750Attempt 2 trace:
f9e34b2f-41f1-40a9-8a8b-24aaa87ff928Attempt 3 trace:
f32c6210-5597-44dd-8602-314e579522c8All three attempts:
EBMS:0004 / -5001TDD MLS:
NONECASE 3 — Self-Billing Invoice
Business SBDH:
b9f40411-988b-4b1d-a5ef-b683eff3fb77TDD SBDH:
80ef3a79-f384-4374-a95b-793ad05d8cd6TDD attempts:
3Attempt 1 trace:
cc455305-8d95-46bb-9cb5-f177c3ab605bAttempt 2 trace:
9fd64f38-6691-4195-8f92-4f0c3119a321Attempt 3 trace:
77906f1d-5eba-45a3-a3f7-b04abe659966All three attempts:
EBMS:0004 / -5001TDD MLS:
NONECASE 4 — Self-Billing Credit Note
Business SBDH:
79352da6-5c1c-42e7-9f14-f1dcf755e078TDD SBDH:
cf0e9df0-6c7a-44d1-8908-370e06d388b6TDD attempts:
3Attempt 1 trace:
bb6b03d2-a3f5-45fb-8354-c1520f4a6f8aAttempt 2 trace:
aa72380b-61f2-4290-a05e-6dc7940ef5baAttempt 3 trace:
7cceedd9-9abd-4c80-b70b-57f9aabec68cAll three attempts:
EBMS:0004 / -5001TDD MLS:
NONEThe retry behaviour is also interesting:
The Testbed code did not change on retries.
For CASE 2, CASE 3 and CASE 4, every TDD retry returned the same
-5001condition.6. Controlled clean re-test
We performed an additional controlled Billing Invoice/TDD attempt specifically to test correlation assumptions.
Business SBDH InstanceIdentifier:
9a6616dc-e0b1-4a61-b919-2fecc3dc36f1Business AS4 MessageId:
e3b4cf80-b135-4a08-9cff-a79011be2a75@phase4.phoss-apBusiness result:
AS4 Receipt / acceptedThen we created a new TDD.
TDD UUID:
ab38b950-8b38-4a68-8ae8-f80c0058ab58TDD SBDH InstanceIdentifier:
0d1c013a-96b8-41fd-844d-8b1cd0c81a4dFor this controlled test we explicitly included:
TransportHeaderID = 9a6616dc-e0b1-4a61-b919-2fecc3dc36f1which exactly references the successful Business Document SBDH InstanceIdentifier.
Local preflight:
17 / 17 PASSXSD:
PASSSchematron:
PASSSchematron failures:
0The TDD was deliberately transmitted once.
TDD AS4 MessageId:
aba9ce9f-2c36-4d1a-aaa4-887092f65a79@phase4.phoss-apTDD sent:
2026-08-06T20:03:34ZTestbed trace:
c2e3f0c8-425d-4ae1-b442-cfbbe91d9cb0Result:
EBMS:0004 / Testbed Code -5001TDD MLS:
NONESo explicitly supplying the Business SBDH InstanceIdentifier as the optional
TransportHeaderIDdid not resolve the failure.7. Local Oman TDD validation
Our rejected principal TDDs pass our current Oman TDD validation.
For the controlled test above:
Preflight:
17/17 PASSXSD:
PASSSchematron:
PASSThe investigation also checked the current Oman TDD rules including the current rule family through
ibr-tdd-64.We have not found a missing mandatory TDD element that explains the four failures.
We are therefore not assuming that the document is perfect simply because local validation passes, but we have not found a specification validation failure that corresponds to the remote
-5001.8. Normal C5 behaviour observed in other historical controls
We have evidence that the same C5 side can produce normal TDD MLS responses.
We hold seven historical TDD controls where a genuine MLS was received with:
ResponseCode = ABand:
The document passed validationWe also hold two deliberately/content-invalid TDD cases where C5 returned:
ResponseCode = REand:
The document failed validationThe observed outcome families are therefore:
Valid TDD:
TDD -> MLS ABInvalid/content-rejected TDD:
TDD -> MLS REOur four blocked scenarios:
TDD -> AS4 Error EBMS:0004 -> Testbed Code -5001 -> NO MLSThis makes us suspect that the
-5001condition occurs before the normal TDD MLS validation-result path completes.We are NOT claiming to know the exact internal Testbed component or function where it fails.
9. Correlation checks already performed
For the representative Billing Invoice case:
Business UBL
cbc:IDmatches:
TDD
ReportedDocument/cbc:IDBusiness UBL
cbc:UUIDmatches:
TDD
ReportedDocument/cbc:UUIDThe TDD contains the reported Business Document in
SourceDocument.ReportingParty corresponds to our AP participant:
0248:POM001192ReceivingParty corresponds to the TaxData receiver:
0242:000006-TESTBED.3001Expected outbound ReporterRole:
01 = SenderCASE 1, CASE 2 and CASE 3 used ReporterRole
01.Historical CASE 4 did transmit ReporterRole
02.We have recorded that CASE 4 value as an inconsistency on our side.
However, we do not believe it can be the common cause of the four failures because CASE 1, CASE 2 and CASE 3 all had the expected
01value and received the same-5001result.The new implementation prevents the official outbound scenarios from using ReporterRole other than
01.10. TransportHeaderID
We investigated
TransportHeaderIDseparately.The Oman TDD schema describes it as the reported document transport-header/SBDH unique identifier.
It is optional (
minOccurs=0).It is absent from the official examples we inspected and absent from multiple historical TDDs that received MLS AB.
Nevertheless, because correlation was a concern, our controlled test explicitly supplied the successful Business SBDH InstanceIdentifier as
TransportHeaderID.That test still received
-5001.Therefore we only conclude:
TransportHeaderID alone is not sufficient to resolve this condition.We are not claiming that all correlation requirements have been proven correct.
11. Certificate / SMP investigation
We previously had a real certificate-publication issue in the Oman Tax Authority test SMP.
The Tax Authority notified us that our SMP certificate field was invalid.
Our forensic audit reconstructed the problem.
The underlying X.509 certificate identity was correct, but an old SMP publication process appended human-readable OpenSSL output after the base64 certificate, including:
subject=...and:
issuer=...The current AP certificate fingerprint is:
4d74a0f2a889a0d942ac7a4f1e042f8785e93569fbfe1c2529abaafd85edb794Our current test SMP is operated by the Oman Tax Authority:
test-smp.taxoman.gov.omWe currently have eight relevant registrations across our two receiving participant identities.
Six Certificate fields are correctly encoded.
Two legacy
busdox-docid-qnsbilling registrations remain malformed because they still contain 142 characters of appendedsubject=/issuer=text:Important:
The X.509 certificate embedded in those malformed values is still the same correct AP certificate fingerprint above.
This is an encoding/publication defect, not a different certificate identity.
We prepared clean replacement ServiceMetadata payloads and verified that the only semantic difference would be the Certificate field.
However, we have NOT written those repairs yet because the SMP is operated by the Oman Tax Authority and our host does not possess the OTA SMP write credentials or a confirmed authenticated write API schema.
We did not guess a credential or write API against live Tax Authority metadata.
Most importantly for this TDD problem:
The four failing official Testbed scenarios use
peppol-doctype-wildcardBusiness Document registrations.Those four wildcard registrations are clean.
The two malformed
busdox-docid-qnsregistrations were not used by these four failing Business sends.12. Certificate proof from actual AS4 wire messages
We did not rely only on SMP metadata.
We extracted the signing certificate from phase4's stored outbound AS4 envelopes.
Across all fourteen Business/TDD transmission attempts belonging to the four principal cases:
Distinct AP signing certificate identities:
1SHA-256:
4d74a0f2a889a0d942ac7a4f1e042f8785e93569fbfe1c2529abaafd85edb794The Business Documents and TDDs therefore used the same AP signing identity.
For CASE 1 specifically:
Business AS4 signing certificate:
4d74a0f2a889a0d942ac7a4f1e042f8785e93569fbfe1c2529abaafd85edb794TDD AS4 signing certificate:
4d74a0f2a889a0d942ac7a4f1e042f8785e93569fbfe1c2529abaafd85edb794They are identical and match the currently active phase4 certificate.
13. TDD receiver certificate
The current TaxData receiver:
0242:000006-TESTBED.3001resolves to the C5 endpoint:
https://c5-phase4-controller.testbed.peppol.org/rcv1Its certificate is:
Subject:
CN=POP000006SHA-256:
53fee898090fc453028b0ec6a77314784c111f63c0fb44220068d4209850fe58Validity:
2026-03-11 -> 2028-02-28That certificate is valid and matches the peer encryption certificate present in our historical TDD wire envelope.
For comparison, the Business-document receiver uses
POP000005.We understand these are different participants/endpoints and therefore different peer certificates are expected.
14. WS-Security / certificate log analysis
We searched the phase4 logs specifically around all four
-5001failure windows.At the four
-5001timestamps we found:WSSecurityException = 0FAILED_CHECK = 0Cannot find key = 0and no relevant:
There ARE old
WSSecurityException / Cannot find key / FAILED_CHECKentries in our phase4 log, but they are from:2026-07-22which was the certificate installation period, approximately two weeks before these failing Testbed scenarios.
We therefore deliberately do not associate those historical security errors with the current
-5001events.15. Successful inbound Business MLS
The Business MLS is successfully returned to our AP and processed/decrypted.
For example, CASE 1 received its Business MLS successfully with:
ResponseCode = ABThis demonstrates that the Testbed can send a protected AS4 response to our AP which our current key material can process.
We understand that this does NOT prove every possible Testbed-side certificate binding is correct, but it is relevant counter-evidence to a general AP certificate/decryption failure.
16. Why phase4 Discussion #337 caught our attention
We reviewed the earlier phase4 Discussion #337, "Peppol TestBed issues".
The sending error shown there had a very similar outer error shape:
EBMS:0004ContentfailureOtherbut the Testbed code was:
-4204and the Testbed explicitly indicated that the inbound request could not be correlated to a valid running test case.
Our response is different:
Code=-5001with the generic internal test-case execution message.
We understand that this difference does NOT prove that correlation succeeded in our case.
But it seems important that the Testbed has at least one explicit correlation error code (
-4204), whereas our four scenarios consistently receive-5001.In your answer in #337 you also explained that, at that time, Testbed transmission tests did not use an SMP lookup for the AP certificate but instead used the client certificate supplied during Testbed onboarding.
That discussion was from 2025, so we do NOT want to assume the same behaviour is still used today.
This leads to one of our most important questions below.
17. Testbed onboarding certificate — the remaining certificate unknown
Our local certificate investigation is now mostly closed.
The one certificate-related value we cannot determine locally is:
Which AP certificate fingerprint is currently stored/bound to our participant inside the OpenPeppol Testbed onboarding system?
For:
0248:POM001192the certificate we expect is:
SHA-256:
4d74a0f2a889a0d942ac7a4f1e042f8785e93569fbfe1c2529abaafd85edb794Serial:
5b21099a03554307c5b1ec2a870b11c3334a16cbSubject:
CN=POM001192,OU=PEPPOL TEST AP,O=Elsayed Hamdi For Advanced Technology,C=OMIssuer:
PEPPOL ACCESS POINT TEST CA - G3Could you please tell us whether the Testbed still uses an onboarding-bound AP certificate for these transmission scenarios?
And if it does, could OpenPeppol verify that the fingerprint bound to our onboarding record is the fingerprint above?
18. Additional error-signal forensics
We have also analysed the AS4 Error Signal population beyond the four primary cases.
The outer AS4 error representation is highly generic.
For both remote Testbed Code
-1and Code-5001populations, the AS4 errors use the same overall structure:EBMS:0004ContentfailureOtherAn undefined error occurred.The actual numeric Testbed Code is present inside the remote ErrorDetail and is not generated or translated locally by our application.
This is one reason we are trying to understand the Testbed-specific meaning of
-5001, rather than treatingEBMS:0004itself as the root cause.19. What we currently believe is proven
We believe the following facts are supported by the stored wire evidence:
20. What we are NOT claiming
We are NOT claiming:
Our current root-cause status remains:
ROOT_CAUSE_PROVEN = false21. Questions for you / OpenPeppol
Philip, your opinion on any of these would be extremely valuable to us.
Question 1
Do you know what OpenPeppol Testbed Code:
-5001means internally?
Does it indicate failure in the Testbed test-case execution engine before the normal document-validation/MLS response is produced?
Question 2
Can
-5001be caused by Testbed correlation/session/state problems even when the explicit-4204correlation code is not returned?We do not want to incorrectly exclude correlation just because
-4204is absent.Question 3
For an Oman outbound Business Document followed by an OM Tax Data Document, what exact values does the Testbed use to correlate the TDD to the previously accepted Business Document?
For example, does it use:
ReportedDocument/IDReportedDocument/UUIDInstanceIdentifierTransportHeaderIDMessageIdQuestion 4
Is there a Testbed-only state/correlation requirement between the first SBD and the TaxData SBD that an AP implementation must preserve even though that state is not represented as a mandatory field in the published OM TDD XML?
Question 5
Does the CURRENT Testbed still use the AP/client certificate supplied during onboarding for transmission tests, as you described in Discussion #337?
Or does it now resolve/use certificate information differently?
If onboarding binding is still used, could someone verify our onboarding fingerprint:
4d74a0f2a889a0d942ac7a4f1e042f8785e93569fbfe1c2529abaafd85edb794Question 6
From a phase4/phoss-ap point of view, does the following sequence suggest anything specific to you?
TDD UserMessage transmitted successfullythen approximately one second later:
AS4_ERROR_MESSAGE_RECEIVEDwith:
EBMS:0004 / Content / failure / Otherand remote:
Testbed Code=-5001with no subsequent MLS.
Question 7
Do you know of any Oman-specific Testbed configuration or TaxData sequence requirement that is implemented/expected by the Testbed but is not obvious from the published OM TDD specification?
Question 8
Would any specific AS4 artefact be most useful for you to inspect?
We can provide sanitized copies of:
22. Why we are asking urgently
We are trying to complete the official Oman Peppol Testbed qualification and the remaining certification/accreditation requirements.
At the moment all four required outbound business scenarios are blocked at the same TDD stage.
We have spent considerable time reducing this problem from a generic Testbed timeout to exact transmitted bytes, hashes, SBDH identifiers, AS4 MessageIds, timestamps and remote Testbed trace references.
The main point we need help with now is:
Why does the Testbed accept the Business Document and return a successful Business MLS, but then return Code
-5001immediately when the corresponding OM TDD is transmitted instead of producing a TDD MLS?Any guidance from you about the phase4/Testbed interaction, the meaning of
-5001, correlation behaviour, or the current onboarding-certificate mechanism would help us enormously.We would be very grateful for any direction that helps us complete these tests as soon as possible.
Thank you very much for your time, and thank you for maintaining phase4 and the related Peppol libraries.
Best regards,
Elsayed Hamdi
Data Wave AI
Invoice and Tax
Oman Peppol Access Point
@phax
All reactions