Skip to content

[seekdb][allocator] Support Sanity with jemalloc - #1313

Open
hnwyllmm wants to merge 1 commit into
masterfrom
task/2026081700118304202
Open

[seekdb][allocator] Support Sanity with jemalloc#1313
hnwyllmm wants to merge 1 commit into
masterfrom
task/2026081700118304202

Conversation

@hnwyllmm

Copy link
Copy Markdown
Member

Task Description

Integrate the Sanity memory debugging feature with the bundled jemalloc allocator. The previous Sanity integration is incompatible with the current jemalloc implementation. Since jemalloc manages the underlying memory metadata, and higher-level allocators (PageArena, MemoryContext) sub-allocate from larger blocks, poisoning only at the base allocator level leaves the boundaries of these arenas unchecked.

Solution Description

  • Implement a jemalloc-aware Sanity adapter using a dedicated arena, custom extent hooks, shadow bytes for tracking poison state, and allocation redzones.
  • Route jemalloc allocation APIs through this adapter in Sanity-enabled builds. Use link-time wrappers for libc functions to prevent recursive dlsym calls during allocation.
  • Add explicit poison/unpoison logic for all relevant PageArena operations: allocation, alignment adjustments, reallocation, memory reuse, partial frees, and tracer rollback paths. Extend this support to MemoryContext.
  • Integrate the adapter into the Bazel build system for Sanity builds and add targeted unit tests for the allocator and PageArena components.

Passed Regressions

  • ./bazel.py build --//bazel:enable_sanity=true //src/observer:seekdb
  • Building both Sanity and non-Sanity variants of _ob_malloc_impl and _oblib_foundation_impl
  • build_bazel/bin/src/observer/seekdb --help
  • 20 focused allocator/PageArena test cases: 4 valid cases passed, 16 overflow/use-after-free cases triggered the expected memory_sanity_abort.
  • Verification of ELF symbols for the Sanity wrapper functions and jemalloc entry points.
  • git diff --check

Upgrade Compatibility

No changes to storage formats or network protocols. The new Sanity behavior is only active in builds compiled with the Sanity flag (--//bazel:enable_sanity=true). The standard (non-Sanity) build path continues to use the unmodified jemalloc allocator.

Other Information

  • This Merge Request contains a single commit.
  • Memory areas not covered by this adapter include: direct-mmap coroutine/thread stacks and SQL-operator datum-specific memory checks.
  • Squash-on-merge will be enabled after MR creation.
  • Related internal link: DIMA-2026081700118304202

Release Note

Feature: Memory Sanity debugging is now supported in builds using the bundled jemalloc allocator. Enable with the Bazel flag --//bazel:enable_sanity=true.

@hnwyllmm

Copy link
Copy Markdown
Member Author

The mapping Dima issue is related to sanity integration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant