mirror of
https://codeberg.org/redict/redict.git
synced 2025-01-22 16:18:28 -05:00
d4e7ffb38c
Background: Following the upgrade to jemalloc 5.2, there was a test that used to be flaky and started failing consistently (on 32bit), so we disabled it (see #9645). This is a test that i introduced in #7289 when i attempted to solve a rare stagnation problem, and it later turned out i failed to solve it, ans what's more i added a test that caused it to be not so rare, and as i mentioned, now in jemalloc 5.2 it became consistent on 32bit. Stagnation can happen when all the slabs of the bin are equally utilized, so the decision to move an allocation from a relatively empty slab to a relatively full one, will never happen, and in that test all the slabs are at 50% utilization, so the defragger could just keep scanning the keyspace and not move anything. What this PR changes: * First, finally in jemalloc 5.2 we have the count of non-full slabs, so when we compare the utilization of the current slab, we can compare it to the average utilization of the non-full slabs in our bin, instead of the total average of our bin. this takes the full slabs out of the game, since they're not candidates for migration (neither source nor target). * Secondly, We add some 12% (100/8) to the decision to defrag an allocation, this is the part that aims to avoid stagnation, and it's especially important since the above mentioned change can get us closer to stagnation. * Thirdly, since jemalloc 5.2 adds sharded bins, we take into account all shards (something that's missing from the original PR that merged it), this isn't expected to make any difference since anyway there should be just one shard. How this was benchmarked. What i did was run the memefficiency test unit with `--verbose` and compare the defragger hits and misses the tests reported. At first, when i took into consideration only the non-full slabs, it got a lot worse (i got into stagnation, or just got a lot of misses and a lot of hits), but when i added the 10% i got back to results that were slightly better than the ones of the jemalloc 5.1 branch. i.e. full defragmentation was achieved with fewer hits (relocations), and fewer misses (keyspace scans). |
||
|---|---|---|
| .. | ||
| moduleapi | ||
| type | ||
| acl.tcl | ||
| aofrw.tcl | ||
| auth.tcl | ||
| bitfield.tcl | ||
| bitops.tcl | ||
| client-eviction.tcl | ||
| cluster.tcl | ||
| dump.tcl | ||
| expire.tcl | ||
| geo.tcl | ||
| hyperloglog.tcl | ||
| info.tcl | ||
| introspection-2.tcl | ||
| introspection.tcl | ||
| keyspace.tcl | ||
| latency-monitor.tcl | ||
| lazyfree.tcl | ||
| limits.tcl | ||
| maxmemory.tcl | ||
| memefficiency.tcl | ||
| multi.tcl | ||
| networking.tcl | ||
| obuf-limits.tcl | ||
| oom-score-adj.tcl | ||
| other.tcl | ||
| pause.tcl | ||
| pendingquerybuf.tcl | ||
| printver.tcl | ||
| protocol.tcl | ||
| pubsub.tcl | ||
| querybuf.tcl | ||
| quit.tcl | ||
| scan.tcl | ||
| scripting.tcl | ||
| shutdown.tcl | ||
| slowlog.tcl | ||
| sort.tcl | ||
| tls.tcl | ||
| tracking.tcl | ||
| violations.tcl | ||
| wait.tcl | ||