Is your feature request related to a problem? Please describe.
We are seeing OTel fail to instrument applications when running the OTel Java agent against a Java application deployed on a Liberty server with an OpenJ9-based JVM, such as IBM Semeru.
Specifically, the failures occur for instrumentation classes that implement classLoaderOptimization(), which performs name-based matching with ClassLoaderHasClassesNamedMatcher. On some server restarts, instrumentation works as expected; on others, it is silently skipped with no clear pattern.
Our best understanding of the root cause is that the caching mechanism used in ClassLoaderHasClassesNamedMatcher is not sufficient for the way Liberty's classloader operates with the OpenJ9-based JVM's Shared Class Cache functionality.
Disabling the Shared Class Cache in Semeru improves some scenarios, but not consistently.
When we disabled (modified the code to not use cache results when classLoaderOptimization() is invoked) the classloader matcher cache—mirroring what the existing ClassLoaderMatcher.disableMatcherCache() already does during build-time muzzle checks—the instrumentation became consistent across all restarts. We observed that disabling the cache consistently resolves the symptom in our setup.
If disabling the matcher cache makes the instrumentation work consistently, then the same result should be expected when the cache is enabled, given that caching is intended only to improve performance and should not change the outcome.
While this difference in behavior could be considered an issue in itself, providing an option to disable this type of caching—particularly when different JVMs and servers may use different classloading techniques—would offer flexibility and a viable workaround for diverse environments.
Describe the solution you'd like
Expose a JVM system property, otel.javaagent.classloader-matcher.cache.disabled or something similar, which defaults to false and preserves the existing behavior. This would allow operators to disable the cache at runtime as a workaround:
-Dotel.javaagent.classloader-matcher.cache.disabled=true
The implementation would be a one-line change to the field initializer in ClassLoaderHasClassesNamedMatcher:
private static boolean useCache =
!Boolean.parseBoolean(
System.getProperty("otel.javaagent.classloader-matcher.cache.disabled", "false"));
- Default value (
false) → useCache = true → existing cached behavior; no change for current users.
-Dotel.javaagent.classloader-matcher.cache.disabled=true → useCache = false → the cache is bypassed, and getResource() is called afresh for every check.
Describe alternatives you've considered
Removing classLoaderOptimization() from the affected instrumentation module—in our case, JMS—works, but it removes the optimization for all environments.
Additional context
This issue was discovered with JMS instrumentation (jms-1.1) on a Liberty server running IBM Semeru on RHEL.
The disableMatcherCache() method already exists in ClassLoaderMatcher for the build-time muzzle path. This request is simply asking for the same control to be made available at runtime.
Tip
React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.
Is your feature request related to a problem? Please describe.
We are seeing OTel fail to instrument applications when running the OTel Java agent against a Java application deployed on a Liberty server with an OpenJ9-based JVM, such as IBM Semeru.
Specifically, the failures occur for instrumentation classes that implement
classLoaderOptimization(), which performs name-based matching withClassLoaderHasClassesNamedMatcher. On some server restarts, instrumentation works as expected; on others, it is silently skipped with no clear pattern.Our best understanding of the root cause is that the caching mechanism used in
ClassLoaderHasClassesNamedMatcheris not sufficient for the way Liberty's classloader operates with the OpenJ9-based JVM's Shared Class Cache functionality.Disabling the Shared Class Cache in Semeru improves some scenarios, but not consistently.
When we disabled (modified the code to not use cache results when
classLoaderOptimization()is invoked) the classloader matcher cache—mirroring what the existingClassLoaderMatcher.disableMatcherCache()already does during build-time muzzle checks—the instrumentation became consistent across all restarts. We observed that disabling the cache consistently resolves the symptom in our setup.If disabling the matcher cache makes the instrumentation work consistently, then the same result should be expected when the cache is enabled, given that caching is intended only to improve performance and should not change the outcome.
While this difference in behavior could be considered an issue in itself, providing an option to disable this type of caching—particularly when different JVMs and servers may use different classloading techniques—would offer flexibility and a viable workaround for diverse environments.
Describe the solution you'd like
Expose a JVM system property,
otel.javaagent.classloader-matcher.cache.disabledor something similar, which defaults tofalseand preserves the existing behavior. This would allow operators to disable the cache at runtime as a workaround:The implementation would be a one-line change to the field initializer in
ClassLoaderHasClassesNamedMatcher:false) →useCache = true→ existing cached behavior; no change for current users.-Dotel.javaagent.classloader-matcher.cache.disabled=true→useCache = false→ the cache is bypassed, andgetResource()is called afresh for every check.Describe alternatives you've considered
Removing
classLoaderOptimization()from the affected instrumentation module—in our case, JMS—works, but it removes the optimization for all environments.Additional context
This issue was discovered with JMS instrumentation (
jms-1.1) on a Liberty server running IBM Semeru on RHEL.The
disableMatcherCache()method already exists inClassLoaderMatcherfor the build-time muzzle path. This request is simply asking for the same control to be made available at runtime.Tip
React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding
+1orme too, to help us triage it. Learn more here.