Make the GC work harder on low-end hardware instead of calling Gc.compact

[kit-ty-kate]

Fixes #6521

Tested using time (ulimit -v <num>000 ; ./opam show --debug --raw dune.3.24.2 > /dev/null) on OCaml 4.14:

• Before this PR (120% overhead): opam needed 512MB and takes 0.403s
• After this PR (20% overhead): opam needs 336MB and takes 0.414s
• If we set the overhead to 1%: opam needs 303MB and takes 0.484s

However testing this code with OCaml 5.x (tested on the latest trunk (ocaml/ocaml#d03f0e4c7), 5.5, 5.4, 5.3, 5.2, 5.1 and 5.0) nothing happens. opam always takes 564MB regardless of space_overhead. I also tried disabling our use Gc.ramp_up just in case this interfere with this but this doesn't do anything either.

[kit-ty-kate]

Does the problem described above ring a bell? @Octachron @gasche

[gasche]

I can definitely see a performance impact of setting space-overhead through OCAMLRUNPARAM on my own 5.4 or 5.5 switches (below are 5.4 numbers, the results are similar for 5.5):

$ hyperfine -L overhead 20,60,100,140,180,220 "OCAMLRUNPARAM=o={overhead} opam show --raw dune"
[...]
  OCAMLRUNPARAM=o=220 opam show --raw dune ran
    1.05 ± 0.05 times faster than OCAMLRUNPARAM=o=180 opam show --raw dune
    1.11 ± 0.06 times faster than OCAMLRUNPARAM=o=140 opam show --raw dune
    1.37 ± 0.10 times faster than OCAMLRUNPARAM=o=100 opam show --raw dune
    1.83 ± 0.12 times faster than OCAMLRUNPARAM=o=60 opam show --raw dune
    2.54 ± 1.00 times faster than OCAMLRUNPARAM=o=20 opam show --raw dune

[kit-ty-kate]

Maybe more work is done but the space is still allocated?

[gasche]

Indeed, 5.x do not include automatic compaction for now, and in particular they will not release memory to the OS unless compaction is requested explictly. In workloads that suffer from fragmentation this can lead to a noticeable increase of peak memory consumption. (Otherwise this should not change peak memory consumption that much.)

It is possible under 4.x that the small overhead setting you put forces automatic compaction much more often, with the result of effectively fighting fragmentation and reducing peak memory usage. I am not familiar with the pacing logic for automatic compaction (most of what I learned about the GC is from 5.x times).

[kit-ty-kate]

It is possible under 4.x that the small overhead setting you put forces automatic compaction much more often, with the result of effectively fighting fragmentation and reducing peak memory usage. I am not familiar with the pacing logic for automatic compaction (most of what I learned about the GC is from 5.x times).

I'm not sure this is what's happening. I've tried to force manual compaction with trunk with the following trick for good measure and nothing happened either.

diff --git a/src/client/opamCliMain.ml b/src/client/opamCliMain.ml
index f921761f6..ea9e6c32e 100644
--- a/src/client/opamCliMain.ml
+++ b/src/client/opamCliMain.ml
@@ -444,6 +444,19 @@ let set_gc_params total_ram =
   if total_ram > 0L && (total_ram : int64) < two_GB then begin
     log "Low end machine detected. Setting the GC to work harder";
     Gc.set {(Gc.get ()) with space_overhead = 20};
+    let _ : Gc.alarm =
+      Gc.create_alarm @@ fun () ->
+      log "Compacting...";
+      Gc.compact ();
+    in
+    let _ : Thread.t =
+      Thread.create (fun () ->
+          while true do
+            Gc.full_major ();
+          done)
+        ()
+    in
+    ()
   end
 
 let run () =

It feels to me like memory allocation (as in from the system) is completely disconnected from how much the GC is collecting or compacting

[kit-ty-kate]

I've also tried using ocaml/ocaml#14796 just in case but i get the same behaviour (although the full_major/compact loop is 4x slower than 5.5 for some reason, but it's not finished nor merged so we can ignore that for now)

I've also noticed that the full_major/compact loop trick actually uses more system memory than not doing anything (about 250MB more), which seems counter-intuitive

[kit-ty-kate]

While reading ocaml/ocaml#12193 description, i got to wonder about this:

TLDR: PR reintroduces compaction for the pools in the GC's major heap (i.e small blocks < 128 words)

Does that mean that larger blocks are not compacted?

I also debugged this further (see 5420779) and it looks like Marshal.from_channel is the culprit.

The logs show:

00:00.606  CACHE(repository)               aaaaaaaa 3 MB
Fatal error:
Out of memory
( ulimit -v 500000; ./opam show --debug --raw dune.3.24.2 > /dev/null; )  18.27s user 0.04s system 99% cpu 18.469 total

So Marshal.from_channel alone goes from 3MB of heap_words to more than 500MB of virtual memory. The file it's trying to marshal from is only 44MB

[kit-ty-kate]

Here's a short reproduction case that doesn't use opam lib (but uses its marshalled data):

(* ocamlfind ocamlopt -linkpkg -linkall -package unix,threads -thread test.ml *)
(* To test using: cd /tmp && cp ~/.opam/repo/*.cache . && (ulimit -v 400000 && CAMLRUNPARAM=o=20 ./a.out) *)

let get_heap () =
  let {Gc.heap_words; _} = Gc.quick_stat () in
  heap_words * Sys.word_size / 8 / 1024 / 1024

let () =
  Printf.printf "heap1: %d MB\n%!" (get_heap ());
  let th =
    Thread.create (fun () ->
        while true do
          Gc.full_major ();
          Gc.compact ();
        done)
      ()
  in
  Printf.printf "heap2: %d MB\n%!" (get_heap ());
  let ic = open_in_bin "state-22ED6214.cache" in
  Printf.printf "heap3: %d MB\n%!" (get_heap ());
  let _ = Marshal.from_channel ic in
  Printf.printf "heap4: %d MB\n%!" (get_heap ());
  close_in ic;
  Printf.printf "heap5: %d MB\n%!" (get_heap ());
  Printf.printf "please press Ctrl-C\n%!";
  Thread.join th

On 4.14 this uses less than 400MB and on OCaml 5.5 this shoots up to less than 700MB.

I'll open a ticket in ocaml/ocaml in a bit.