caleb-brown/redict
1
0
Fork 0
mirror of https://codeberg.org/redict/redict.git synced 2025-01-22 08:08:53 -05:00
Code Issues Actions Packages Projects Releases Wiki Activity

Optimize the efficiency of active expiration when databases exceeds 16. (#12738)

Browse Source 
Currently, when the number of databases exceeds 16,
the efficiency of cleaning expired keys is relatively low.

The reason is that by default only 16 databases are scanned when
attempting to clean expired keys (CRON_DBS_PER_CALL is 16). But users
may set databases higher than 16, such as 256, but it does not
necessarily mean that all 256 databases have expiration time set. If
only one database has expiration time set, this database needs 16
activeExpireCycle rounds in order to be scanned once, and 15 of those
rounds are meaningless.

To optimize the efficiency of expiration in such scenarios, we use dbs_per_call
to control the number of databases with expired keys being scanned.

Additionally, add a condition to limit the maximum number of rounds
to server.dbnum to prevent excessive spinning. This ensures that even if
only one database has expired keys, it can be triggered within a single cron job.
This commit is contained in:
zhaozhao.zz 2023-11-27 12:12:12 +08:00 committed by GitHub
parent 56ec1ff1ce
commit 1bd0b54957
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 9 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
src/expire.c
View File

@ -174,6 +174,7 @@ void activeExpireCycle(int type) {
int j, iteration = 0;
int dbs_per_call = CRON_DBS_PER_CALL;
int dbs_performed = 0;
long long start = ustime(), timelimit, elapsed;
/* If 'expire' action is paused, for whatever reason, then don't expire any key.
@ -226,7 +227,12 @@ void activeExpireCycle(int type) {
/* Try to smoke-out bugs (server.also_propagate should be empty here) */
serverAssert(server.also_propagate.numops == 0);
for (j = 0; j < dbs_per_call && timelimit_exit == 0; j++) {
/* Stop iteration when one of the following conditions is met:
*
* 1) We have checked a sufficient number of databases with expiration time.
* 2) The time limit has been exceeded.
* 3) All databases have been traversed. */
for (j = 0; dbs_performed < dbs_per_call && timelimit_exit == 0 && j < server.dbnum; j++) {
/* Scan callback data including expired and checked count per iteration. */
expireScanData data;
@ -238,6 +244,8 @@ void activeExpireCycle(int type) {
* distribute the time evenly across DBs. */
current_db++;
if (dbSize(db, DB_EXPIRES)) dbs_performed++;
/* Continue to expire if at the end of the cycle there are still
* a big percentage of keys to expire, compared to the number of keys
* we scanned. The percentage, stored in config_cycle_acceptable_stale