2012-11-08 12:25:23 -05:00
/* Configuration file parsing and CONFIG GET/SET commands implementation.
*
* Copyright ( c ) 2009 - 2012 , Salvatore Sanfilippo < antirez at gmail dot com >
* All rights reserved .
*
* Redistribution and use in source and binary forms , with or without
* modification , are permitted provided that the following conditions are met :
*
* * Redistributions of source code must retain the above copyright notice ,
* this list of conditions and the following disclaimer .
* * Redistributions in binary form must reproduce the above copyright
* notice , this list of conditions and the following disclaimer in the
* documentation and / or other materials provided with the distribution .
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission .
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS " AS IS "
* AND ANY EXPRESS OR IMPLIED WARRANTIES , INCLUDING , BUT NOT LIMITED TO , THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED . IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT , INDIRECT , INCIDENTAL , SPECIAL , EXEMPLARY , OR
* CONSEQUENTIAL DAMAGES ( INCLUDING , BUT NOT LIMITED TO , PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES ; LOSS OF USE , DATA , OR PROFITS ; OR BUSINESS
* INTERRUPTION ) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY , WHETHER IN
* CONTRACT , STRICT LIABILITY , OR TORT ( INCLUDING NEGLIGENCE OR OTHERWISE )
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE , EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE .
*/
2015-07-26 09:14:57 -04:00
# include "server.h"
2013-10-09 09:37:20 -04:00
# include "cluster.h"
2010-06-21 18:07:48 -04:00
2013-05-14 06:32:25 -04:00
# include <fcntl.h>
# include <sys/stat.h>
2021-06-16 14:59:38 -04:00
# include <glob.h>
# include <string.h>
2013-05-14 06:32:25 -04:00
2015-03-11 11:59:56 -04:00
/*-----------------------------------------------------------------------------
* Config file name - value maps .
* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */
typedef struct configEnum {
const char * name ;
const int val ;
} configEnum ;
configEnum maxmemory_policy_enum [ ] = {
2015-07-27 03:41:48 -04:00
{ " volatile-lru " , MAXMEMORY_VOLATILE_LRU } ,
2016-07-15 06:12:52 -04:00
{ " volatile-lfu " , MAXMEMORY_VOLATILE_LFU } ,
2015-07-27 03:41:48 -04:00
{ " volatile-random " , MAXMEMORY_VOLATILE_RANDOM } ,
{ " volatile-ttl " , MAXMEMORY_VOLATILE_TTL } ,
{ " allkeys-lru " , MAXMEMORY_ALLKEYS_LRU } ,
2016-07-15 06:12:52 -04:00
{ " allkeys-lfu " , MAXMEMORY_ALLKEYS_LFU } ,
2015-07-27 03:41:48 -04:00
{ " allkeys-random " , MAXMEMORY_ALLKEYS_RANDOM } ,
{ " noeviction " , MAXMEMORY_NO_EVICTION } ,
2015-03-11 11:59:56 -04:00
{ NULL , 0 }
} ;
configEnum syslog_facility_enum [ ] = {
2013-05-13 05:11:35 -04:00
{ " user " , LOG_USER } ,
{ " local0 " , LOG_LOCAL0 } ,
{ " local1 " , LOG_LOCAL1 } ,
{ " local2 " , LOG_LOCAL2 } ,
{ " local3 " , LOG_LOCAL3 } ,
{ " local4 " , LOG_LOCAL4 } ,
{ " local5 " , LOG_LOCAL5 } ,
{ " local6 " , LOG_LOCAL6 } ,
{ " local7 " , LOG_LOCAL7 } ,
{ NULL , 0 }
} ;
2015-03-11 18:20:57 -04:00
configEnum loglevel_enum [ ] = {
2015-07-27 03:41:48 -04:00
{ " debug " , LL_DEBUG } ,
{ " verbose " , LL_VERBOSE } ,
{ " notice " , LL_NOTICE } ,
{ " warning " , LL_WARNING } ,
2015-03-11 18:20:57 -04:00
{ NULL , 0 }
} ;
configEnum supervised_mode_enum [ ] = {
2015-07-27 03:41:48 -04:00
{ " upstart " , SUPERVISED_UPSTART } ,
{ " systemd " , SUPERVISED_SYSTEMD } ,
{ " auto " , SUPERVISED_AUTODETECT } ,
{ " no " , SUPERVISED_NONE } ,
2015-03-11 18:20:57 -04:00
{ NULL , 0 }
} ;
configEnum aof_fsync_enum [ ] = {
{ " everysec " , AOF_FSYNC_EVERYSEC } ,
{ " always " , AOF_FSYNC_ALWAYS } ,
{ " no " , AOF_FSYNC_NO } ,
{ NULL , 0 }
} ;
2019-07-01 08:22:29 -04:00
configEnum repl_diskless_load_enum [ ] = {
{ " disabled " , REPL_DISKLESS_LOAD_DISABLED } ,
{ " on-empty-db " , REPL_DISKLESS_LOAD_WHEN_DB_EMPTY } ,
{ " swapdb " , REPL_DISKLESS_LOAD_SWAPDB } ,
{ NULL , 0 }
} ;
2020-07-28 03:45:21 -04:00
configEnum tls_auth_clients_enum [ ] = {
{ " no " , TLS_CLIENT_AUTH_NO } ,
{ " yes " , TLS_CLIENT_AUTH_YES } ,
{ " optional " , TLS_CLIENT_AUTH_OPTIONAL } ,
{ NULL , 0 }
} ;
2020-11-22 06:57:56 -05:00
configEnum oom_score_adj_enum [ ] = {
{ " no " , OOM_SCORE_ADJ_NO } ,
{ " yes " , OOM_SCORE_RELATIVE } ,
{ " relative " , OOM_SCORE_RELATIVE } ,
{ " absolute " , OOM_SCORE_ADJ_ABSOLUTE } ,
{ NULL , 0 }
} ;
Adds pub/sub channel patterns to ACL (#7993)
Fixes #7923.
This PR appropriates the special `&` symbol (because `@` and `*` are taken),
followed by a literal value or pattern for describing the Pub/Sub patterns that
an ACL user can interact with. It is similar to the existing key patterns
mechanism in function (additive) and implementation (copy-pasta). It also adds
the allchannels and resetchannels ACL keywords, naturally.
The default user is given allchannels permissions, whereas new users get
whatever is defined by the acl-pubsub-default configuration directive. For
backward compatibility in 6.2, the default of this directive is allchannels but
this is likely to be changed to resetchannels in the next major version for
stronger default security settings.
Unless allchannels is set for the user, channel access permissions are checked
as follows :
* Calls to both PUBLISH and SUBSCRIBE will fail unless a pattern matching the
argumentative channel name(s) exists for the user.
* Calls to PSUBSCRIBE will fail unless the pattern(s) provided as an argument
literally exist(s) in the user's list.
Such failures are logged to the ACL log.
Runtime changes to channel permissions for a user with existing subscribing
clients cause said clients to disconnect unless the new permissions permit the
connections to continue. Note, however, that PSUBSCRIBErs' patterns are matched
literally, so given the change bar:* -> b*, pattern subscribers to bar:* will be
disconnected.
Notes/questions:
* UNSUBSCRIBE, PUNSUBSCRIBE and PUBSUB remain unprotected due to lack of reasons
for touching them.
2020-12-01 07:21:39 -05:00
configEnum acl_pubsub_default_enum [ ] = {
{ " allchannels " , USER_FLAG_ALLCHANNELS } ,
{ " resetchannels " , 0 } ,
{ NULL , 0 }
} ;
Sanitize dump payload: ziplist, listpack, zipmap, intset, stream
When loading an encoded payload we will at least do a shallow validation to
check that the size that's encoded in the payload matches the size of the
allocation.
This let's us later use this encoded size to make sure the various offsets
inside encoded payload don't reach outside the allocation, if they do, we'll
assert/panic, but at least we won't segfault or smear memory.
We can also do 'deep' validation which runs on all the records of the encoded
payload and validates that they don't contain invalid offsets. This lets us
detect corruptions early and reject a RESTORE command rather than accepting
it and asserting (crashing) later when accessing that payload via some command.
configuration:
- adding ACL flag skip-sanitize-payload
- adding config sanitize-dump-payload [yes/no/clients]
For now, we don't have a good way to ensure MIGRATE in cluster resharding isn't
being slowed down by these sanitation, so i'm setting the default value to `no`,
but later on it should be set to `clients` by default.
changes:
- changing rdbReportError not to `exit` in RESTORE command
- adding a new stat to be able to later check if cluster MIGRATE isn't being
slowed down by sanitation.
2020-08-13 09:41:05 -04:00
configEnum sanitize_dump_payload_enum [ ] = {
{ " no " , SANITIZE_DUMP_NO } ,
{ " yes " , SANITIZE_DUMP_YES } ,
{ " clients " , SANITIZE_DUMP_CLIENTS } ,
{ NULL , 0 }
} ;
2015-03-11 11:59:56 -04:00
/* Output buffer limits presets. */
2015-07-28 10:58:04 -04:00
clientBufferLimitsConfig clientBufferLimitsDefaults [ CLIENT_TYPE_OBUF_COUNT ] = {
2013-05-13 12:34:18 -04:00
{ 0 , 0 , 0 } , /* normal */
{ 1024 * 1024 * 256 , 1024 * 1024 * 64 , 60 } , /* slave */
{ 1024 * 1024 * 32 , 1024 * 1024 * 8 , 60 } /* pubsub */
} ;
2020-08-12 10:58:56 -04:00
/* OOM Score defaults */
int configOOMScoreAdjValuesDefaults [ CONFIG_OOM_COUNT ] = { 0 , 200 , 800 } ;
2019-11-28 04:11:07 -05:00
/* Generic config infrastructure function pointers
* int is_valid_fn ( val , err )
* Return 1 when val is valid , and 0 when invalid .
2020-01-14 03:10:39 -05:00
* Optionally set err to a static error string .
2019-11-28 04:11:07 -05:00
*/
2019-11-22 11:54:22 -05:00
/* Configuration values that require no special handling to set, get, load or
2019-05-31 15:05:18 -04:00
* rewrite . */
2019-07-19 13:15:35 -04:00
typedef struct boolConfigData {
int * config ; /* The pointer to the server config this value is stored in */
const int default_value ; /* The default value of the config on rewrite */
2021-01-21 04:56:08 -05:00
int ( * is_valid_fn ) ( int val , const char * * err ) ; /* Optional function to check validity of new value (generic doc above) */
2019-07-19 13:15:35 -04:00
} boolConfigData ;
typedef struct stringConfigData {
2019-11-22 11:54:22 -05:00
char * * config ; /* Pointer to the server config this value is stored in. */
const char * default_value ; /* Default value of the config on rewrite. */
2021-01-21 04:56:08 -05:00
int ( * is_valid_fn ) ( char * val , const char * * err ) ; /* Optional function to check validity of new value (generic doc above) */
2019-11-22 11:54:22 -05:00
int convert_empty_to_null ; /* Boolean indicating if empty strings should
2019-11-22 00:31:53 -05:00
be stored as a NULL value . */
2019-07-19 13:15:35 -04:00
} stringConfigData ;
2020-12-17 12:26:33 -05:00
typedef struct sdsConfigData {
sds * config ; /* Pointer to the server config this value is stored in. */
const char * default_value ; /* Default value of the config on rewrite. */
2021-01-21 04:56:08 -05:00
int ( * is_valid_fn ) ( sds val , const char * * err ) ; /* Optional function to check validity of new value (generic doc above) */
2020-12-17 12:26:33 -05:00
int convert_empty_to_null ; /* Boolean indicating if empty SDS strings should
be stored as a NULL value . */
} sdsConfigData ;
2019-07-19 13:15:35 -04:00
typedef struct enumConfigData {
int * config ; /* The pointer to the server config this value is stored in */
configEnum * enum_value ; /* The underlying enum type this data represents */
const int default_value ; /* The default value of the config on rewrite */
2021-01-21 04:56:08 -05:00
int ( * is_valid_fn ) ( int val , const char * * err ) ; /* Optional function to check validity of new value (generic doc above) */
2019-07-19 13:15:35 -04:00
} enumConfigData ;
typedef enum numericType {
2019-11-22 11:54:22 -05:00
NUMERIC_TYPE_INT ,
2019-11-28 04:11:07 -05:00
NUMERIC_TYPE_UINT ,
NUMERIC_TYPE_LONG ,
NUMERIC_TYPE_ULONG ,
2019-11-22 11:54:22 -05:00
NUMERIC_TYPE_LONG_LONG ,
2019-11-28 04:11:07 -05:00
NUMERIC_TYPE_ULONG_LONG ,
NUMERIC_TYPE_SIZE_T ,
NUMERIC_TYPE_SSIZE_T ,
NUMERIC_TYPE_OFF_T ,
NUMERIC_TYPE_TIME_T ,
2019-07-19 13:15:35 -04:00
} numericType ;
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
# define INTEGER_CONFIG 0 /* No flags means a simple integer configuration */
# define MEMORY_CONFIG (1<<0) /* Indicates if this value can be loaded as a memory value */
# define PERCENT_CONFIG (1<<1) /* Indicates if this value can be loaded as a percent (and stored as a negative int) */
2021-10-19 02:58:52 -04:00
# define OCTAL_CONFIG (1<<2) /* This value uses octal representation */
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
2019-07-19 13:15:35 -04:00
typedef struct numericConfigData {
union {
int * i ;
2019-11-28 04:11:07 -05:00
unsigned int * ui ;
long * l ;
2019-11-22 11:54:22 -05:00
unsigned long * ul ;
2019-11-28 04:11:07 -05:00
long long * ll ;
unsigned long long * ull ;
2019-07-19 13:15:35 -04:00
size_t * st ;
2019-11-28 04:11:07 -05:00
ssize_t * sst ;
off_t * ot ;
time_t * tt ;
2019-07-19 13:15:35 -04:00
} config ; /* The pointer to the numeric config this value is stored in */
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
unsigned int flags ;
2019-07-19 13:15:35 -04:00
numericType numeric_type ; /* An enum indicating the type of this value */
long long lower_bound ; /* The lower bound of this numeric value */
long long upper_bound ; /* The upper bound of this numeric value */
const long long default_value ; /* The default value of the config on rewrite */
2021-01-21 04:56:08 -05:00
int ( * is_valid_fn ) ( long long val , const char * * err ) ; /* Optional function to check validity of new value (generic doc above) */
2019-07-19 13:15:35 -04:00
} numericConfigData ;
typedef union typeData {
2019-11-22 00:49:52 -05:00
boolConfigData yesno ;
2019-07-19 13:15:35 -04:00
stringConfigData string ;
2020-12-17 12:26:33 -05:00
sdsConfigData sds ;
2019-07-19 13:15:35 -04:00
enumConfigData enumd ;
numericConfigData numeric ;
} typeData ;
2021-12-01 03:15:11 -05:00
typedef int ( * apply_fn ) ( const char * * err ) ;
2019-07-19 13:15:35 -04:00
typedef struct typeInterface {
2019-11-26 09:52:28 -05:00
/* Called on server start, to init the server with default value */
void ( * init ) ( typeData data ) ;
2021-12-01 03:15:11 -05:00
/* Called on server startup and CONFIG SET, returns 1 on success,
* 2 meaning no actual change done , 0 on error and can set a verbose err
* string */
int ( * set ) ( typeData data , sds * argv , int argc , const char * * err ) ;
/* Optional: called after `set()` to apply the config change. Used only in
* the context of CONFIG SET . Returns 1 on success , 0 on failure .
* Optionally set err to a static error string . */
apply_fn apply ;
/* Called on CONFIG GET, returns sds to be used in reply */
sds ( * get ) ( typeData data ) ;
2019-07-19 13:15:35 -04:00
/* Called on CONFIG REWRITE, required to rewrite the config state */
void ( * rewrite ) ( typeData data , const char * name , struct rewriteConfigState * state ) ;
} typeInterface ;
typedef struct standardConfig {
2019-05-31 15:05:18 -04:00
const char * name ; /* The user visible name of this config */
const char * alias ; /* An alias that can also be used for this config */
2021-03-16 01:00:29 -04:00
const unsigned int flags ; /* Flags for this specific config */
2019-11-22 00:31:53 -05:00
typeInterface interface ; /* The function pointers that define the type interface */
2019-07-19 13:15:35 -04:00
typeData data ; /* The type specific data exposed used by the interface */
} standardConfig ;
2021-03-16 01:00:29 -04:00
# define MODIFIABLE_CONFIG 0 / * This is the implied default for a standard
* config , which is mutable . */
# define IMMUTABLE_CONFIG (1ULL<<0) /* Can this value only be set at startup? */
# define SENSITIVE_CONFIG (1ULL<<1) /* Does this value contain sensitive information */
2021-08-05 14:59:12 -04:00
# define DEBUG_CONFIG (1ULL<<2) /* Values that are useful for debugging. */
2021-11-07 06:40:08 -05:00
# define MULTI_ARG_CONFIG (1ULL<<3) /* This config receives multiple arguments. */
2021-12-08 05:44:10 -05:00
# define HIDDEN_CONFIG (1ULL<<4) /* This config is hidden in `config get <pattern>` (used for tests/debugging) */
2021-03-16 01:00:29 -04:00
2019-07-19 13:15:35 -04:00
standardConfig configs [ ] ;
2019-05-31 15:05:18 -04:00
2010-06-21 18:07:48 -04:00
/*-----------------------------------------------------------------------------
2015-03-11 11:59:56 -04:00
* Enum access functions
2010-06-21 18:07:48 -04:00
* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */
2015-03-11 11:59:56 -04:00
/* Get enum value from name. If there is no match INT_MIN is returned. */
int configEnumGetValue ( configEnum * ce , char * name ) {
while ( ce - > name ! = NULL ) {
if ( ! strcasecmp ( ce - > name , name ) ) return ce - > val ;
ce + + ;
}
return INT_MIN ;
}
/* Get enum name from value. If no match is found NULL is returned. */
const char * configEnumGetName ( configEnum * ce , int val ) {
while ( ce - > name ! = NULL ) {
if ( ce - > val = = val ) return ce - > name ;
ce + + ;
}
return NULL ;
}
2018-10-20 22:37:36 -04:00
/* Wrapper for configEnumGetName() returning "unknown" instead of NULL if
2015-03-11 11:59:56 -04:00
* there is no match . */
const char * configEnumGetNameOrUnknown ( configEnum * ce , int val ) {
const char * name = configEnumGetName ( ce , val ) ;
return name ? name : " unknown " ;
}
/* Used for INFO generation. */
2016-04-22 11:43:48 -04:00
const char * evictPolicyToString ( void ) {
return configEnumGetNameOrUnknown ( maxmemory_policy_enum , server . maxmemory_policy ) ;
2015-03-11 11:59:56 -04:00
}
2010-06-21 18:07:48 -04:00
/*-----------------------------------------------------------------------------
* Config file parsing
* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */
2015-01-08 15:22:33 -05:00
2010-06-21 18:07:48 -04:00
int yesnotoi ( char * s ) {
if ( ! strcasecmp ( s , " yes " ) ) return 1 ;
else if ( ! strcasecmp ( s , " no " ) ) return 0 ;
else return - 1 ;
}
void appendServerSaveParams ( time_t seconds , int changes ) {
server . saveparams = zrealloc ( server . saveparams , sizeof ( struct saveparam ) * ( server . saveparamslen + 1 ) ) ;
server . saveparams [ server . saveparamslen ] . seconds = seconds ;
server . saveparams [ server . saveparamslen ] . changes = changes ;
server . saveparamslen + + ;
}
2014-03-15 19:51:54 -04:00
void resetServerSaveParams ( void ) {
2010-06-21 18:07:48 -04:00
zfree ( server . saveparams ) ;
server . saveparams = NULL ;
server . saveparamslen = 0 ;
}
2016-06-13 03:39:44 -04:00
void queueLoadModule ( sds path , sds * argv , int argc ) {
2016-06-05 03:03:34 -04:00
int i ;
2016-06-13 03:51:06 -04:00
struct moduleLoadQueueEntry * loadmod ;
2016-06-05 03:03:34 -04:00
2016-06-13 03:51:06 -04:00
loadmod = zmalloc ( sizeof ( struct moduleLoadQueueEntry ) ) ;
2018-04-20 09:01:21 -04:00
loadmod - > argv = argc ? zmalloc ( sizeof ( robj * ) * argc ) : NULL ;
2016-06-05 03:03:34 -04:00
loadmod - > path = sdsnew ( path ) ;
loadmod - > argc = argc ;
for ( i = 0 ; i < argc ; i + + ) {
2016-06-13 03:40:28 -04:00
loadmod - > argv [ i ] = createRawStringObject ( argv [ i ] , sdslen ( argv [ i ] ) ) ;
2016-06-05 03:03:34 -04:00
}
listAddNodeTail ( server . loadmodule_queue , loadmod ) ;
}
2021-08-29 08:03:05 -04:00
/* Parse an array of `arg_len` sds strings, validate and populate
* server . client_obuf_limits if valid .
* Used in CONFIG SET and configuration file parsing . */
static int updateClientOutputBufferLimit ( sds * args , int arg_len , const char * * err ) {
int j ;
int class ;
unsigned long long hard , soft ;
int hard_err , soft_err ;
int soft_seconds ;
char * soft_seconds_eptr ;
clientBufferLimitsConfig values [ CLIENT_TYPE_OBUF_COUNT ] ;
int classes [ CLIENT_TYPE_OBUF_COUNT ] = { 0 } ;
/* We need a multiple of 4: <class> <hard> <soft> <soft_seconds> */
if ( arg_len % 4 ) {
if ( err ) * err = " Wrong number of arguments in "
" buffer limit configuration. " ;
2021-11-07 06:40:08 -05:00
return 0 ;
2021-08-29 08:03:05 -04:00
}
/* Sanity check of single arguments, so that we either refuse the
* whole configuration string or accept it all , even if a single
* error in a single client class is present . */
for ( j = 0 ; j < arg_len ; j + = 4 ) {
class = getClientTypeByName ( args [ j ] ) ;
if ( class = = - 1 | | class = = CLIENT_TYPE_MASTER ) {
if ( err ) * err = " Invalid client class specified in "
" buffer limit configuration. " ;
2021-11-07 06:40:08 -05:00
return 0 ;
2021-08-29 08:03:05 -04:00
}
hard = memtoull ( args [ j + 1 ] , & hard_err ) ;
soft = memtoull ( args [ j + 2 ] , & soft_err ) ;
soft_seconds = strtoll ( args [ j + 3 ] , & soft_seconds_eptr , 10 ) ;
if ( hard_err | | soft_err | |
soft_seconds < 0 | | * soft_seconds_eptr ! = ' \0 ' )
{
if ( err ) * err = " Error in hard, soft or soft_seconds setting in "
" buffer limit configuration. " ;
2021-11-07 06:40:08 -05:00
return 0 ;
2021-08-29 08:03:05 -04:00
}
values [ class ] . hard_limit_bytes = hard ;
values [ class ] . soft_limit_bytes = soft ;
values [ class ] . soft_limit_seconds = soft_seconds ;
classes [ class ] = 1 ;
}
/* Finally set the new config. */
for ( j = 0 ; j < CLIENT_TYPE_OBUF_COUNT ; j + + ) {
if ( classes [ j ] ) server . client_obuf_limits [ j ] = values [ j ] ;
}
2021-11-07 06:40:08 -05:00
return 1 ;
2021-08-29 08:03:05 -04:00
}
2019-11-26 09:52:28 -05:00
void initConfigValues ( ) {
for ( standardConfig * config = configs ; config - > name ! = NULL ; config + + ) {
2021-11-07 06:40:08 -05:00
if ( config - > interface . init ) config - > interface . init ( config - > data ) ;
2019-11-26 09:52:28 -05:00
}
}
2021-12-01 03:15:11 -05:00
/* Note this is here to support detecting we're running a config set from
* within conf file parsing . This is only needed to support the deprecated
* abnormal aggregate ` save T C ` functionality . Remove in the future . */
static int reading_config_file ;
2011-12-01 07:44:53 -05:00
void loadServerConfigFromString ( char * config ) {
2021-11-11 06:51:33 -05:00
char buf [ 1024 ] ;
2021-01-21 04:56:08 -05:00
const char * err = NULL ;
2011-12-01 07:44:53 -05:00
int linenum = 0 , totlines , i ;
sds * lines ;
2010-06-21 18:07:48 -04:00
2021-12-01 03:15:11 -05:00
reading_config_file = 1 ;
2011-12-01 07:44:53 -05:00
lines = sdssplitlen ( config , strlen ( config ) , " \n " , 1 , & totlines ) ;
2010-06-21 18:07:48 -04:00
2011-12-01 07:44:53 -05:00
for ( i = 0 ; i < totlines ; i + + ) {
2010-06-21 18:07:48 -04:00
sds * argv ;
2011-12-01 07:44:53 -05:00
int argc ;
2010-06-21 18:07:48 -04:00
2011-12-01 07:44:53 -05:00
linenum = i + 1 ;
lines [ i ] = sdstrim ( lines [ i ] , " \t \r \n " ) ;
2010-06-21 18:07:48 -04:00
2013-03-06 06:40:48 -05:00
/* Skip comments and blank lines */
2011-12-01 07:44:53 -05:00
if ( lines [ i ] [ 0 ] = = ' # ' | | lines [ i ] [ 0 ] = = ' \0 ' ) continue ;
2010-06-21 18:07:48 -04:00
/* Split into arguments */
2011-12-01 07:44:53 -05:00
argv = sdssplitargs ( lines [ i ] , & argc ) ;
2013-02-06 00:47:14 -05:00
if ( argv = = NULL ) {
2013-03-06 06:24:12 -05:00
err = " Unbalanced quotes in configuration line " ;
2013-02-06 00:47:14 -05:00
goto loaderr ;
}
2013-03-06 06:40:48 -05:00
/* Skip this line if the resulting command vector is empty. */
if ( argc = = 0 ) {
sdsfreesplitres ( argv , argc ) ;
2013-05-18 10:21:52 -04:00
continue ;
2013-03-06 06:40:48 -05:00
}
2010-06-21 18:07:48 -04:00
sdstolower ( argv [ 0 ] ) ;
2019-05-31 15:05:18 -04:00
/* Iterate the configs that are standard */
int match = 0 ;
2019-07-19 13:15:35 -04:00
for ( standardConfig * config = configs ; config - > name ! = NULL ; config + + ) {
2019-05-31 15:05:18 -04:00
if ( ( ! strcasecmp ( argv [ 0 ] , config - > name ) | |
2019-11-22 11:54:22 -05:00
( config - > alias & & ! strcasecmp ( argv [ 0 ] , config - > alias ) ) ) )
2019-05-31 15:05:18 -04:00
{
2021-11-07 06:40:08 -05:00
/* For normal single arg configs enforce we have a single argument.
* Note that MULTI_ARG_CONFIGs need to validate arg count on their own */
if ( ! ( config - > flags & MULTI_ARG_CONFIG ) & & argc ! = 2 ) {
2020-02-05 04:41:24 -05:00
err = " wrong number of arguments " ;
goto loaderr ;
}
2021-11-07 06:40:08 -05:00
/* Set config using all arguments that follows */
2021-12-01 03:15:11 -05:00
if ( ! config - > interface . set ( config - > data , & argv [ 1 ] , argc - 1 , & err ) ) {
2019-07-19 13:15:35 -04:00
goto loaderr ;
2019-05-31 15:05:18 -04:00
}
2019-07-19 13:15:35 -04:00
2019-05-31 15:05:18 -04:00
match = 1 ;
break ;
}
}
if ( match ) {
sdsfreesplitres ( argv , argc ) ;
continue ;
}
2010-06-21 18:07:48 -04:00
/* Execute config directives */
2021-11-07 06:40:08 -05:00
if ( ! strcasecmp ( argv [ 0 ] , " include " ) & & argc = = 2 ) {
2020-10-11 06:43:23 -04:00
loadServerConfig ( argv [ 1 ] , 0 , NULL ) ;
2010-06-21 18:07:48 -04:00
} else if ( ! strcasecmp ( argv [ 0 ] , " list-max-ziplist-entries " ) & & argc = = 2 ) {
2014-12-16 00:49:14 -05:00
/* DEAD OPTION */
2010-11-03 07:14:36 -04:00
} else if ( ! strcasecmp ( argv [ 0 ] , " list-max-ziplist-value " ) & & argc = = 2 ) {
2014-12-16 00:49:14 -05:00
/* DEAD OPTION */
2010-11-03 07:14:36 -04:00
} else if ( ! strcasecmp ( argv [ 0 ] , " rename-command " ) & & argc = = 3 ) {
Treat subcommands as commands (#9504)
## Intro
The purpose is to allow having different flags/ACL categories for
subcommands (Example: CONFIG GET is ok-loading but CONFIG SET isn't)
We create a small command table for every command that has subcommands
and each subcommand has its own flags, etc. (same as a "regular" command)
This commit also unites the Redis and the Sentinel command tables
## Affected commands
CONFIG
Used to have "admin ok-loading ok-stale no-script"
Changes:
1. Dropped "ok-loading" in all except GET (this doesn't change behavior since
there were checks in the code doing that)
XINFO
Used to have "read-only random"
Changes:
1. Dropped "random" in all except CONSUMERS
XGROUP
Used to have "write use-memory"
Changes:
1. Dropped "use-memory" in all except CREATE and CREATECONSUMER
COMMAND
No changes.
MEMORY
Used to have "random read-only"
Changes:
1. Dropped "random" in PURGE and USAGE
ACL
Used to have "admin no-script ok-loading ok-stale"
Changes:
1. Dropped "admin" in WHOAMI, GENPASS, and CAT
LATENCY
No changes.
MODULE
No changes.
SLOWLOG
Used to have "admin random ok-loading ok-stale"
Changes:
1. Dropped "random" in RESET
OBJECT
Used to have "read-only random"
Changes:
1. Dropped "random" in ENCODING and REFCOUNT
SCRIPT
Used to have "may-replicate no-script"
Changes:
1. Dropped "may-replicate" in all except FLUSH and LOAD
CLIENT
Used to have "admin no-script random ok-loading ok-stale"
Changes:
1. Dropped "random" in all except INFO and LIST
2. Dropped "admin" in ID, TRACKING, CACHING, GETREDIR, INFO, SETNAME, GETNAME, and REPLY
STRALGO
No changes.
PUBSUB
No changes.
CLUSTER
Changes:
1. Dropped "admin in countkeysinslots, getkeysinslot, info, nodes, keyslot, myid, and slots
SENTINEL
No changes.
(note that DEBUG also fits, but we decided not to convert it since it's for
debugging and anyway undocumented)
## New sub-command
This commit adds another element to the per-command output of COMMAND,
describing the list of subcommands, if any (in the same structure as "regular" commands)
Also, it adds a new subcommand:
```
COMMAND LIST [FILTERBY (MODULE <module-name>|ACLCAT <cat>|PATTERN <pattern>)]
```
which returns a set of all commands (unless filters), but excluding subcommands.
## Module API
A new module API, RM_CreateSubcommand, was added, in order to allow
module writer to define subcommands
## ACL changes:
1. Now, that each subcommand is actually a command, each has its own ACL id.
2. The old mechanism of allowed_subcommands is redundant
(blocking/allowing a subcommand is the same as blocking/allowing a regular command),
but we had to keep it, to support the widespread usage of allowed_subcommands
to block commands with certain args, that aren't subcommands (e.g. "-select +select|0").
3. I have renamed allowed_subcommands to allowed_firstargs to emphasize the difference.
4. Because subcommands are commands in ACL too, you can now use "-" to block subcommands
(e.g. "+client -client|kill"), which wasn't possible in the past.
5. It is also possible to use the allowed_firstargs mechanism with subcommand.
For example: `+config -config|set +config|set|loglevel` will block all CONFIG SET except
for setting the log level.
6. All of the ACL changes above required some amount of refactoring.
## Misc
1. There are two approaches: Either each subcommand has its own function or all
subcommands use the same function, determining what to do according to argv[0].
For now, I took the former approaches only with CONFIG and COMMAND,
while other commands use the latter approach (for smaller blamelog diff).
2. Deleted memoryGetKeys: It is no longer needed because MEMORY USAGE now uses the "range" key spec.
4. Bugfix: GETNAME was missing from CLIENT's help message.
5. Sentinel and Redis now use the same table, with the same function pointer.
Some commands have a different implementation in Sentinel, so we redirect
them (these are ROLE, PUBLISH, and INFO).
6. Command stats now show the stats per subcommand (e.g. instead of stats just
for "config" you will have stats for "config|set", "config|get", etc.)
7. It is now possible to use COMMAND directly on subcommands:
COMMAND INFO CONFIG|GET (The pipeline syntax was inspired from ACL, and
can be used in functions lookupCommandBySds and lookupCommandByCString)
8. STRALGO is now a container command (has "help")
## Breaking changes:
1. Command stats now show the stats per subcommand (see (5) above)
2021-10-20 04:52:57 -04:00
struct redisCommand * cmd = lookupCommandBySds ( argv [ 1 ] ) ;
2010-11-03 07:14:36 -04:00
int retval ;
if ( ! cmd ) {
err = " No such command in rename-command " ;
goto loaderr ;
}
2013-01-16 12:00:20 -05:00
/* If the target command name is the empty string we just
2010-11-03 07:14:36 -04:00
* remove it from the command table . */
retval = dictDelete ( server . commands , argv [ 1 ] ) ;
2015-07-26 09:29:53 -04:00
serverAssert ( retval = = DICT_OK ) ;
2010-11-03 07:14:36 -04:00
/* Otherwise we re-add the command under a different name. */
if ( sdslen ( argv [ 2 ] ) ! = 0 ) {
sds copy = sdsdup ( argv [ 2 ] ) ;
retval = dictAdd ( server . commands , copy , cmd ) ;
if ( retval ! = DICT_OK ) {
sdsfree ( copy ) ;
err = " Target command name already exists " ; goto loaderr ;
}
}
2019-02-04 06:55:26 -05:00
} else if ( ! strcasecmp ( argv [ 0 ] , " user " ) & & argc > = 2 ) {
2019-02-04 07:00:38 -05:00
int argc_err ;
if ( ACLAppendUserForLoading ( argv , argc , & argc_err ) = = C_ERR ) {
2021-01-21 04:56:08 -05:00
const char * errmsg = ACLSetUserStringError ( ) ;
2019-02-04 07:04:35 -05:00
snprintf ( buf , sizeof ( buf ) , " Error in user declaration '%s': %s " ,
argv [ argc_err ] , errmsg ) ;
err = buf ;
2019-02-04 06:55:26 -05:00
goto loaderr ;
}
2016-06-05 03:03:34 -04:00
} else if ( ! strcasecmp ( argv [ 0 ] , " loadmodule " ) & & argc > = 2 ) {
queueLoadModule ( argv [ 1 ] , & argv [ 2 ] , argc - 2 ) ;
2012-07-23 06:54:52 -04:00
} else if ( ! strcasecmp ( argv [ 0 ] , " sentinel " ) ) {
/* argc == 1 is handled by main() as we need to enter the sentinel
* mode ASAP . */
if ( argc ! = 1 ) {
if ( ! server . sentinel_mode ) {
err = " sentinel directive while not in sentinel mode " ;
goto loaderr ;
}
Sentinel: Fix Config Dependency and Rewrite Sequence (#8271)
This commit fixes a well known and an annoying issue in Sentinel mode.
Cause of this issue:
Currently, Redis rewrite process works well in server mode, however in sentinel mode,
the sentinel config has variant semantics for different configurations, in example configuration
https://github.com/redis/redis/blob/unstable/sentinel.conf, we put comments on these.
However the rewrite process only treat the sentinel config as a single option. During rewrite
process, it will mess up with the lines and comments.
Approaches:
In order to solve this issue, we need to differentiate different subconfig options in sentinel separately,
for example, sentinel monitor <master-name> <ip> <redis-port> <quorum>
we can treat it as sentinel monitor option, instead of the sentinel option.
This commit also fixes the dependency issue when putting configurations in sentinel.conf.
For example before this commit,we must put
`sentinel monitor <master-name> <ip> <redis-port> <quorum>` before
`sentinel auth-pass <master-name> <password>` for a single master,
otherwise the server cannot start and will return error. This commit fixes this issue, as long as
the monitoring master was configured, no matter the sequence is, the sentinel can start and run properly.
2021-01-26 02:31:54 -05:00
queueSentinelConfig ( argv + 1 , argc - 1 , linenum , lines [ i ] ) ;
2012-07-23 06:54:52 -04:00
}
2010-06-21 18:07:48 -04:00
} else {
err = " Bad directive or wrong number of arguments " ; goto loaderr ;
}
2011-12-01 07:44:53 -05:00
sdsfreesplitres ( argv , argc ) ;
2010-06-21 18:07:48 -04:00
}
2013-03-05 06:58:22 -05:00
2021-06-20 04:34:04 -04:00
if ( server . logfile [ 0 ] ! = ' \0 ' ) {
FILE * logfp ;
/* Test if we are able to open the file. The server will not
* be able to abort just for this problem later . . . */
logfp = fopen ( server . logfile , " a " ) ;
if ( logfp = = NULL ) {
err = sdscatprintf ( sdsempty ( ) ,
" Can't open the log file: %s " , strerror ( errno ) ) ;
goto loaderr ;
}
fclose ( logfp ) ;
}
2013-03-05 06:58:22 -05:00
/* Sanity checks. */
if ( server . cluster_enabled & & server . masterhost ) {
2018-09-10 06:22:20 -04:00
err = " replicaof directive not allowed in cluster mode " ;
2013-03-05 06:58:22 -05:00
goto loaderr ;
}
2014-01-20 05:08:14 -05:00
2021-03-25 21:59:52 -04:00
/* To ensure backward compatibility and work while hz is out of range */
if ( server . config_hz < CONFIG_MIN_HZ ) server . config_hz = CONFIG_MIN_HZ ;
if ( server . config_hz > CONFIG_MAX_HZ ) server . config_hz = CONFIG_MAX_HZ ;
2014-01-20 05:08:14 -05:00
sdsfreesplitres ( lines , totlines ) ;
2021-12-01 03:15:11 -05:00
reading_config_file = 0 ;
2010-06-21 18:07:48 -04:00
return ;
loaderr :
2020-02-27 12:21:12 -05:00
fprintf ( stderr , " \n *** FATAL CONFIG FILE ERROR (Redis %s) *** \n " ,
REDIS_VERSION ) ;
2021-11-07 06:40:08 -05:00
if ( i < totlines ) {
fprintf ( stderr , " Reading the configuration file, at line %d \n " , linenum ) ;
fprintf ( stderr , " >>> '%s' \n " , lines [ i ] ) ;
}
2010-06-21 18:07:48 -04:00
fprintf ( stderr , " %s \n " , err ) ;
exit ( 1 ) ;
}
2011-12-01 07:44:53 -05:00
/* Load the server configuration from the specified filename.
* The function appends the additional configuration directives stored
* in the ' options ' string to the config file before loading .
*
* Both filename and options can be NULL , in such a case are considered
2013-01-16 12:00:20 -05:00
* empty . This way loadServerConfig can be used to just load a file or
2011-12-01 07:44:53 -05:00
* just load a string . */
2020-10-11 06:43:23 -04:00
void loadServerConfig ( char * filename , char config_from_stdin , char * options ) {
2011-12-01 07:44:53 -05:00
sds config = sdsempty ( ) ;
2015-07-27 03:41:48 -04:00
char buf [ CONFIG_MAX_LINE + 1 ] ;
2020-10-11 06:43:23 -04:00
FILE * fp ;
2021-06-16 14:59:38 -04:00
glob_t globbuf ;
2011-12-01 07:44:53 -05:00
/* Load the file content */
if ( filename ) {
2021-06-16 14:59:38 -04:00
/* The logic for handling wildcards has slightly different behavior in cases where
* there is a failure to locate the included file .
* Whether or not a wildcard is specified , we should ALWAYS log errors when attempting
* to open included config files .
*
* However , we desire a behavioral difference between instances where a wildcard was
* specified and those where it hasn ' t :
* no wildcards : attempt to open the specified file and fail with a logged error
* if the file cannot be found and opened .
* with wildcards : attempt to glob the specified pattern ; if no files match the
* pattern , then gracefully continue on to the next entry in the
* config file , as if the current entry was never encountered .
* This will allow for empty conf . d directories to be included . */
if ( strchr ( filename , ' * ' ) | | strchr ( filename , ' ? ' ) | | strchr ( filename , ' [ ' ) ) {
/* A wildcard character detected in filename, so let us use glob */
if ( glob ( filename , 0 , NULL , & globbuf ) = = 0 ) {
for ( size_t i = 0 ; i < globbuf . gl_pathc ; i + + ) {
if ( ( fp = fopen ( globbuf . gl_pathv [ i ] , " r " ) ) = = NULL ) {
serverLog ( LL_WARNING ,
" Fatal error, can't open config file '%s': %s " ,
globbuf . gl_pathv [ i ] , strerror ( errno ) ) ;
exit ( 1 ) ;
}
while ( fgets ( buf , CONFIG_MAX_LINE + 1 , fp ) ! = NULL )
config = sdscat ( config , buf ) ;
fclose ( fp ) ;
}
globfree ( & globbuf ) ;
}
} else {
/* No wildcard in filename means we can use the original logic to read and
* potentially fail traditionally */
if ( ( fp = fopen ( filename , " r " ) ) = = NULL ) {
serverLog ( LL_WARNING ,
" Fatal error, can't open config file '%s': %s " ,
filename , strerror ( errno ) ) ;
exit ( 1 ) ;
}
while ( fgets ( buf , CONFIG_MAX_LINE + 1 , fp ) ! = NULL )
config = sdscat ( config , buf ) ;
fclose ( fp ) ;
2011-12-01 07:44:53 -05:00
}
}
2021-06-16 14:59:38 -04:00
2020-10-11 06:43:23 -04:00
/* Append content from stdin */
if ( config_from_stdin ) {
serverLog ( LL_WARNING , " Reading config from stdin " ) ;
fp = stdin ;
while ( fgets ( buf , CONFIG_MAX_LINE + 1 , fp ) ! = NULL )
config = sdscat ( config , buf ) ;
}
2011-12-01 07:44:53 -05:00
/* Append the additional options */
if ( options ) {
config = sdscat ( config , " \n " ) ;
config = sdscat ( config , options ) ;
}
loadServerConfigFromString ( config ) ;
sdsfree ( config ) ;
}
2021-12-01 03:15:11 -05:00
static int performInterfaceSet ( standardConfig * config , sds value , const char * * errstr ) {
sds * argv ;
int argc , res ;
if ( config - > flags & MULTI_ARG_CONFIG ) {
argv = sdssplitlen ( value , sdslen ( value ) , " " , 1 , & argc ) ;
} else {
argv = ( char * * ) & value ;
argc = 1 ;
}
/* Set the config */
res = config - > interface . set ( config - > data , argv , argc , errstr ) ;
if ( config - > flags & MULTI_ARG_CONFIG ) sdsfreesplitres ( argv , argc ) ;
return res ;
}
static void restoreBackupConfig ( standardConfig * * set_configs , sds * old_values , int count , apply_fn * apply_fns ) {
int i ;
const char * errstr = " unknown error " ;
/* Set all backup values */
for ( i = 0 ; i < count ; i + + ) {
if ( ! performInterfaceSet ( set_configs [ i ] , old_values [ i ] , & errstr ) )
serverLog ( LL_WARNING , " Failed restoring failed CONFIG SET command. Error setting %s to '%s': %s " ,
set_configs [ i ] - > name , old_values [ i ] , errstr ) ;
}
/* Apply backup */
if ( apply_fns ) {
for ( i = 0 ; i < count & & apply_fns [ i ] ! = NULL ; i + + ) {
if ( ! apply_fns [ i ] ( & errstr ) )
serverLog ( LL_WARNING , " Failed applying restored failed CONFIG SET command: %s " , errstr ) ;
}
}
}
2010-06-21 18:07:48 -04:00
/*-----------------------------------------------------------------------------
2013-05-09 18:15:18 -04:00
* CONFIG SET implementation
2010-06-21 18:07:48 -04:00
* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */
2021-12-01 03:15:11 -05:00
2015-07-26 09:20:46 -04:00
void configSetCommand ( client * c ) {
2021-01-21 04:56:08 -05:00
const char * errstr = NULL ;
2021-12-15 02:46:32 -05:00
const char * invalid_arg_name = NULL ;
const char * err_arg_name = NULL ;
2021-12-01 03:15:11 -05:00
standardConfig * * set_configs ; /* TODO: make this a dict for better performance */
sds * new_values ;
sds * old_values = NULL ;
apply_fn * apply_fns ; /* TODO: make this a set for better performance */
int config_count , i , j ;
int invalid_args = 0 ;
2021-12-09 09:37:05 -05:00
int * config_map_fns ;
2021-12-01 03:15:11 -05:00
/* Make sure we have an even number of arguments: conf-val pairs */
if ( c - > argc & 1 ) {
addReplyErrorObject ( c , shared . syntaxerr ) ;
return ;
}
config_count = ( c - > argc - 2 ) / 2 ;
2010-06-21 18:07:48 -04:00
2021-12-01 03:15:11 -05:00
set_configs = zcalloc ( sizeof ( standardConfig * ) * config_count ) ;
new_values = zmalloc ( sizeof ( sds * ) * config_count ) ;
old_values = zcalloc ( sizeof ( sds * ) * config_count ) ;
apply_fns = zcalloc ( sizeof ( apply_fn ) * config_count ) ;
2021-12-09 09:37:05 -05:00
config_map_fns = zmalloc ( sizeof ( int ) * config_count ) ;
2019-05-31 15:05:18 -04:00
2021-12-01 03:15:11 -05:00
/* Find all relevant configs */
for ( i = 0 ; i < config_count ; i + + ) {
for ( standardConfig * config = configs ; config - > name ! = NULL ; config + + ) {
if ( ( ! strcasecmp ( c - > argv [ 2 + i * 2 ] - > ptr , config - > name ) | |
( config - > alias & & ! strcasecmp ( c - > argv [ 2 ] - > ptr , config - > alias ) ) ) ) {
/* Note: it's important we run over ALL passed configs and check if we need to call `redactClientCommandArgument()`.
* This is in order to avoid anyone using this command for a log / slowlog / monitor / etc . displaying sensitive info .
* So even if we encounter an error we still continue running over the remaining arguments . */
if ( config - > flags & SENSITIVE_CONFIG ) {
redactClientCommandArgument ( c , 2 + i * 2 + 1 ) ;
2021-11-07 06:40:08 -05:00
}
2021-03-01 09:04:44 -05:00
2021-12-01 03:15:11 -05:00
if ( ! invalid_args ) {
if ( config - > flags & IMMUTABLE_CONFIG ) {
/* Note: we don't abort the loop since we still want to handle redacting sensitive configs (above) */
errstr = " can't set immutable config " ;
2021-12-15 02:46:32 -05:00
err_arg_name = c - > argv [ 2 + i * 2 ] - > ptr ;
2021-12-01 03:15:11 -05:00
invalid_args = 1 ;
}
/* If this config appears twice then fail */
for ( j = 0 ; j < i ; j + + ) {
if ( set_configs [ j ] = = config ) {
/* Note: we don't abort the loop since we still want to handle redacting sensitive configs (above) */
errstr = " duplicate parameter " ;
2021-12-15 02:46:32 -05:00
err_arg_name = c - > argv [ 2 + i * 2 ] - > ptr ;
2021-12-01 03:15:11 -05:00
invalid_args = 1 ;
break ;
}
}
set_configs [ i ] = config ;
new_values [ i ] = c - > argv [ 2 + i * 2 + 1 ] - > ptr ;
}
break ;
}
}
/* Fail if we couldn't find this config */
/* Note: we don't abort the loop since we still want to handle redacting sensitive configs (above) */
if ( ! invalid_args & & ! set_configs [ i ] ) {
2021-12-15 02:46:32 -05:00
invalid_arg_name = c - > argv [ 2 + i * 2 ] - > ptr ;
2021-12-01 03:15:11 -05:00
invalid_args = 1 ;
}
}
if ( invalid_args ) goto err ;
/* Backup old values before setting new ones */
for ( i = 0 ; i < config_count ; i + + )
old_values [ i ] = set_configs [ i ] - > interface . get ( set_configs [ i ] - > data ) ;
/* Set all new values (don't apply yet) */
for ( i = 0 ; i < config_count ; i + + ) {
int res = performInterfaceSet ( set_configs [ i ] , new_values [ i ] , & errstr ) ;
if ( ! res ) {
restoreBackupConfig ( set_configs , old_values , i + 1 , NULL ) ;
2021-12-15 02:46:32 -05:00
err_arg_name = set_configs [ i ] - > name ;
2021-12-01 03:15:11 -05:00
goto err ;
} else if ( res = = 1 ) {
/* A new value was set, if this config has an apply function then store it for execution later */
if ( set_configs [ i ] - > interface . apply ) {
/* Check if this apply function is already stored */
int exists = 0 ;
for ( j = 0 ; apply_fns [ j ] ! = NULL & & j < = i ; j + + ) {
if ( apply_fns [ j ] = = set_configs [ i ] - > interface . apply ) {
exists = 1 ;
break ;
}
}
/* Apply function not stored, store it */
2021-12-09 09:37:05 -05:00
if ( ! exists ) {
2021-12-01 03:15:11 -05:00
apply_fns [ j ] = set_configs [ i ] - > interface . apply ;
2021-12-09 09:37:05 -05:00
config_map_fns [ j ] = i ;
}
2010-06-21 18:07:48 -04:00
}
2011-02-13 20:51:27 -05:00
}
2010-06-21 18:07:48 -04:00
}
2015-03-11 18:20:57 -04:00
2021-12-01 03:15:11 -05:00
/* Apply all configs after being set */
for ( i = 0 ; i < config_count & & apply_fns [ i ] ! = NULL ; i + + ) {
if ( ! apply_fns [ i ] ( & errstr ) ) {
2021-12-09 09:37:05 -05:00
serverLog ( LL_WARNING , " Failed applying new configuration. Possibly related to new %s setting. Restoring previous settings. " , set_configs [ config_map_fns [ i ] ] - > name ) ;
2021-12-01 03:15:11 -05:00
restoreBackupConfig ( set_configs , old_values , config_count , apply_fns ) ;
2021-12-15 02:46:32 -05:00
err_arg_name = set_configs [ config_map_fns [ i ] ] - > name ;
2021-12-01 03:15:11 -05:00
goto err ;
}
}
addReply ( c , shared . ok ) ;
goto end ;
2010-06-21 18:07:48 -04:00
2021-12-01 03:15:11 -05:00
err :
2021-12-15 02:46:32 -05:00
if ( invalid_arg_name ) {
addReplyErrorFormat ( c , " Unknown option or number of arguments for CONFIG SET - '%s' " , invalid_arg_name ) ;
} else if ( errstr ) {
addReplyErrorFormat ( c , " CONFIG SET failed (possibly related to argument '%s') - %s " , err_arg_name , errstr ) ;
2019-11-26 09:52:28 -05:00
} else {
2021-12-15 02:46:32 -05:00
addReplyErrorFormat ( c , " CONFIG SET failed (possibly related to argument '%s') " , err_arg_name ) ;
2019-11-26 09:52:28 -05:00
}
2021-12-01 03:15:11 -05:00
end :
zfree ( set_configs ) ;
zfree ( new_values ) ;
for ( i = 0 ; i < config_count ; i + + )
sdsfree ( old_values [ i ] ) ;
zfree ( old_values ) ;
zfree ( apply_fns ) ;
2021-12-09 09:37:05 -05:00
zfree ( config_map_fns ) ;
2010-06-21 18:07:48 -04:00
}
2013-05-09 18:15:18 -04:00
/*-----------------------------------------------------------------------------
* CONFIG GET implementation
* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */
2015-07-26 09:20:46 -04:00
void configGetCommand ( client * c ) {
2010-10-17 12:09:23 -04:00
robj * o = c - > argv [ 2 ] ;
2018-11-08 11:54:50 -05:00
void * replylen = addReplyDeferredLen ( c ) ;
2010-06-21 18:07:48 -04:00
char * pattern = o - > ptr ;
int matches = 0 ;
2015-07-26 09:29:53 -04:00
serverAssertWithInfo ( c , o , sdsEncodedObject ( o ) ) ;
2010-06-21 18:07:48 -04:00
2019-07-19 13:15:35 -04:00
for ( standardConfig * config = configs ; config - > name ! = NULL ; config + + ) {
2021-12-08 05:44:10 -05:00
/* Hidden configs require an exact match (not a pattern) */
if ( config - > flags & HIDDEN_CONFIG ) {
if ( ! strcasecmp ( pattern , config - > name ) ) {
addReplyBulkCString ( c , config - > name ) ;
addReplyBulkSds ( c , config - > interface . get ( config - > data ) ) ;
matches + + ;
break ;
} else if ( config - > alias & & ! strcasecmp ( pattern , config - > alias ) ) {
addReplyBulkCString ( c , config - > alias ) ;
addReplyBulkSds ( c , config - > interface . get ( config - > data ) ) ;
matches + + ;
break ;
}
continue ;
}
2019-07-19 13:15:35 -04:00
if ( stringmatch ( pattern , config - > name , 1 ) ) {
addReplyBulkCString ( c , config - > name ) ;
2021-12-01 03:15:11 -05:00
addReplyBulkSds ( c , config - > interface . get ( config - > data ) ) ;
2019-07-19 13:15:35 -04:00
matches + + ;
}
if ( config - > alias & & stringmatch ( pattern , config - > alias , 1 ) ) {
addReplyBulkCString ( c , config - > alias ) ;
2021-12-01 03:15:11 -05:00
addReplyBulkSds ( c , config - > interface . get ( config - > data ) ) ;
2019-07-19 13:15:35 -04:00
matches + + ;
}
}
2018-11-08 11:54:50 -05:00
setDeferredMapLen ( c , replylen , matches ) ;
2010-06-21 18:07:48 -04:00
}
2013-05-09 18:15:18 -04:00
/*-----------------------------------------------------------------------------
* CONFIG REWRITE implementation
* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */
2013-11-19 11:58:11 -05:00
# define REDIS_CONFIG_REWRITE_SIGNATURE "# Generated by CONFIG REWRITE"
2013-05-09 18:15:18 -04:00
/* We use the following dictionary type to store where a configuration
* option is mentioned in the old configuration file , so it ' s
* like " maxmemory " - > list of line numbers ( first line is zero ) . */
2021-08-05 01:25:58 -04:00
void dictListDestructor ( dict * d , void * val ) ;
2013-05-09 18:15:18 -04:00
2013-11-19 03:48:12 -05:00
/* Sentinel config rewriting is implemented inside sentinel.c by
* rewriteConfigSentinelOption ( ) . */
void rewriteConfigSentinelOption ( struct rewriteConfigState * state ) ;
2013-05-09 18:15:18 -04:00
dictType optionToLineDictType = {
2013-12-19 09:25:45 -05:00
dictSdsCaseHash , /* hash function */
2013-05-09 18:15:18 -04:00
NULL , /* key dup */
NULL , /* val dup */
2013-12-19 09:25:45 -05:00
dictSdsKeyCaseCompare , /* key compare */
2013-05-09 18:15:18 -04:00
dictSdsDestructor , /* key destructor */
Limit the main db and expires dictionaries to expand (#7954)
As we know, redis may reject user's requests or evict some keys if
used memory is over maxmemory. Dictionaries expanding may make
things worse, some big dictionaries, such as main db and expires dict,
may eat huge memory at once for allocating a new big hash table and be
far more than maxmemory after expanding.
There are related issues: #4213 #4583
More details, when expand dict in redis, we will allocate a new big
ht[1] that generally is double of ht[0], The size of ht[1] will be
very big if ht[0] already is big. For db dict, if we have more than
64 million keys, we need to cost 1GB for ht[1] when dict expands.
If the sum of used memory and new hash table of dict needed exceeds
maxmemory, we shouldn't allow the dict to expand. Because, if we
enable keys eviction, we still couldn't add much more keys after
eviction and rehashing, what's worse, redis will keep less keys when
redis only remains a little memory for storing new hash table instead
of users' data. Moreover users can't write data in redis if disable
keys eviction.
What this commit changed ?
Add a new member function expandAllowed for dict type, it provide a way
for caller to allow expand or not. We expose two parameters for this
function: more memory needed for expanding and dict current load factor,
users can implement a function to make a decision by them.
For main db dict and expires dict type, these dictionaries may be very
big and cost huge memory for expanding, so we implement a judgement
function: we can stop dict to expand provisionally if used memory will
be over maxmemory after dict expands, but to guarantee the performance
of redis, we still allow dict to expand if dict load factor exceeds the
safe load factor.
Add test cases to verify we don't allow main db to expand when left
memory is not enough, so that avoid keys eviction.
Other changes:
For new hash table size when expand. Before this commit, the size is
that double used of dict and later _dictNextPower. Actually we aim to
control a dict load factor between 0.5 and 1.0. Now we replace *2 with
+1, since the first check is that used >= size, the outcome of before
will usually be the same as _dictNextPower(used+1). The only case where
it'll differ is when dict_can_resize is false during fork, so that later
the _dictNextPower(used*2) will cause the dict to jump to *4 (i.e.
_dictNextPower(1025*2) will return 4096).
Fix rehash test cases due to changing algorithm of new hash table size
when expand.
2020-12-06 04:53:04 -05:00
dictListDestructor , /* val destructor */
NULL /* allow to expand */
2013-05-09 18:15:18 -04:00
} ;
2013-12-19 09:25:45 -05:00
dictType optionSetDictType = {
dictSdsCaseHash , /* hash function */
NULL , /* key dup */
NULL , /* val dup */
dictSdsKeyCaseCompare , /* key compare */
dictSdsDestructor , /* key destructor */
Limit the main db and expires dictionaries to expand (#7954)
As we know, redis may reject user's requests or evict some keys if
used memory is over maxmemory. Dictionaries expanding may make
things worse, some big dictionaries, such as main db and expires dict,
may eat huge memory at once for allocating a new big hash table and be
far more than maxmemory after expanding.
There are related issues: #4213 #4583
More details, when expand dict in redis, we will allocate a new big
ht[1] that generally is double of ht[0], The size of ht[1] will be
very big if ht[0] already is big. For db dict, if we have more than
64 million keys, we need to cost 1GB for ht[1] when dict expands.
If the sum of used memory and new hash table of dict needed exceeds
maxmemory, we shouldn't allow the dict to expand. Because, if we
enable keys eviction, we still couldn't add much more keys after
eviction and rehashing, what's worse, redis will keep less keys when
redis only remains a little memory for storing new hash table instead
of users' data. Moreover users can't write data in redis if disable
keys eviction.
What this commit changed ?
Add a new member function expandAllowed for dict type, it provide a way
for caller to allow expand or not. We expose two parameters for this
function: more memory needed for expanding and dict current load factor,
users can implement a function to make a decision by them.
For main db dict and expires dict type, these dictionaries may be very
big and cost huge memory for expanding, so we implement a judgement
function: we can stop dict to expand provisionally if used memory will
be over maxmemory after dict expands, but to guarantee the performance
of redis, we still allow dict to expand if dict load factor exceeds the
safe load factor.
Add test cases to verify we don't allow main db to expand when left
memory is not enough, so that avoid keys eviction.
Other changes:
For new hash table size when expand. Before this commit, the size is
that double used of dict and later _dictNextPower. Actually we aim to
control a dict load factor between 0.5 and 1.0. Now we replace *2 with
+1, since the first check is that used >= size, the outcome of before
will usually be the same as _dictNextPower(used+1). The only case where
it'll differ is when dict_can_resize is false during fork, so that later
the _dictNextPower(used*2) will cause the dict to jump to *4 (i.e.
_dictNextPower(1025*2) will return 4096).
Fix rehash test cases due to changing algorithm of new hash table size
when expand.
2020-12-06 04:53:04 -05:00
NULL , /* val destructor */
NULL /* allow to expand */
2013-12-19 09:25:45 -05:00
} ;
2013-05-09 18:15:18 -04:00
/* The config rewrite state. */
struct rewriteConfigState {
dict * option_to_line ; /* Option -> list of config file lines map */
2013-12-19 09:25:45 -05:00
dict * rewritten ; /* Dictionary of already processed options */
2013-05-09 18:15:18 -04:00
int numlines ; /* Number of lines in current config */
sds * lines ; /* Current lines as an array of sds strings */
2021-08-05 14:59:12 -04:00
int needs_signature ; /* True if we need to append the rewrite
signature . */
int force_write ; /* True if we want all keywords to be force
written . Currently only used for testing
and debug information . */
2013-05-09 18:15:18 -04:00
} ;
2021-08-05 14:59:12 -04:00
/* Free the configuration rewrite state. */
void rewriteConfigReleaseState ( struct rewriteConfigState * state ) {
sdsfreesplitres ( state - > lines , state - > numlines ) ;
dictRelease ( state - > option_to_line ) ;
dictRelease ( state - > rewritten ) ;
zfree ( state ) ;
}
/* Create the configuration rewrite state */
struct rewriteConfigState * rewriteConfigCreateState ( ) {
struct rewriteConfigState * state = zmalloc ( sizeof ( * state ) ) ;
state - > option_to_line = dictCreate ( & optionToLineDictType ) ;
state - > rewritten = dictCreate ( & optionSetDictType ) ;
state - > numlines = 0 ;
state - > lines = NULL ;
state - > needs_signature = 1 ;
state - > force_write = 0 ;
return state ;
}
2013-05-09 18:15:18 -04:00
/* Append the new line to the current configuration state. */
void rewriteConfigAppendLine ( struct rewriteConfigState * state , sds line ) {
state - > lines = zrealloc ( state - > lines , sizeof ( char * ) * ( state - > numlines + 1 ) ) ;
state - > lines [ state - > numlines + + ] = line ;
}
/* Populate the option -> list of line numbers map. */
void rewriteConfigAddLineNumberToOption ( struct rewriteConfigState * state , sds option , int linenum ) {
list * l = dictFetchValue ( state - > option_to_line , option ) ;
if ( l = = NULL ) {
l = listCreate ( ) ;
dictAdd ( state - > option_to_line , sdsdup ( option ) , l ) ;
}
listAddNodeTail ( l , ( void * ) ( long ) linenum ) ;
}
2013-12-19 09:25:45 -05:00
/* Add the specified option to the set of processed options.
* This is useful as only unused lines of processed options will be blanked
* in the config file , while options the rewrite process does not understand
* remain untouched . */
2015-03-11 11:59:56 -04:00
void rewriteConfigMarkAsProcessed ( struct rewriteConfigState * state , const char * option ) {
2013-12-19 09:25:45 -05:00
sds opt = sdsnew ( option ) ;
2013-12-19 09:30:06 -05:00
if ( dictAdd ( state - > rewritten , opt , NULL ) ! = DICT_OK ) sdsfree ( opt ) ;
2013-12-19 09:25:45 -05:00
}
2013-05-09 18:15:18 -04:00
/* Read the old file, split it into lines to populate a newly created
* config rewrite state , and return it to the caller .
*
* If it is impossible to read the old file , NULL is returned .
* If the old file does not exist at all , an empty state is returned . */
struct rewriteConfigState * rewriteConfigReadOldFile ( char * path ) {
FILE * fp = fopen ( path , " r " ) ;
if ( fp = = NULL & & errno ! = ENOENT ) return NULL ;
2019-05-07 03:59:16 -04:00
char buf [ CONFIG_MAX_LINE + 1 ] ;
int linenum = - 1 ;
2021-08-05 14:59:12 -04:00
struct rewriteConfigState * state = rewriteConfigCreateState ( ) ;
2013-05-09 18:15:18 -04:00
if ( fp = = NULL ) return state ;
/* Read the old file line by line, populate the state. */
2015-07-27 03:41:48 -04:00
while ( fgets ( buf , CONFIG_MAX_LINE + 1 , fp ) ! = NULL ) {
2013-05-09 18:15:18 -04:00
int argc ;
sds * argv ;
sds line = sdstrim ( sdsnew ( buf ) , " \r \n \t " ) ;
linenum + + ; /* Zero based, so we init at -1 */
/* Handle comments and empty lines. */
if ( line [ 0 ] = = ' # ' | | line [ 0 ] = = ' \0 ' ) {
2021-08-05 14:59:12 -04:00
if ( state - > needs_signature & & ! strcmp ( line , REDIS_CONFIG_REWRITE_SIGNATURE ) )
state - > needs_signature = 0 ;
2013-05-09 18:15:18 -04:00
rewriteConfigAppendLine ( state , line ) ;
continue ;
}
/* Not a comment, split into arguments. */
argv = sdssplitargs ( line , & argc ) ;
if ( argv = = NULL ) {
/* Apparently the line is unparsable for some reason, for
* instance it may have unbalanced quotes . Load it as a
* comment . */
sds aux = sdsnew ( " # ??? " ) ;
aux = sdscatsds ( aux , line ) ;
sdsfree ( line ) ;
rewriteConfigAppendLine ( state , aux ) ;
continue ;
}
sdstolower ( argv [ 0 ] ) ; /* We only want lowercase config directives. */
/* Now we populate the state according to the content of this line.
* Append the line and populate the option - > line numbers map . */
rewriteConfigAppendLine ( state , line ) ;
2018-09-10 06:22:20 -04:00
/* Translate options using the word "slave" to the corresponding name
* " replica " , before adding such option to the config name - > lines
* mapping . */
char * p = strstr ( argv [ 0 ] , " slave " ) ;
if ( p ) {
sds alt = sdsempty ( ) ;
2020-08-02 06:59:51 -04:00
alt = sdscatlen ( alt , argv [ 0 ] , p - argv [ 0 ] ) ;
2018-09-10 06:22:20 -04:00
alt = sdscatlen ( alt , " replica " , 7 ) ;
alt = sdscatlen ( alt , p + 5 , strlen ( p + 5 ) ) ;
sdsfree ( argv [ 0 ] ) ;
argv [ 0 ] = alt ;
}
Sentinel: Fix Config Dependency and Rewrite Sequence (#8271)
This commit fixes a well known and an annoying issue in Sentinel mode.
Cause of this issue:
Currently, Redis rewrite process works well in server mode, however in sentinel mode,
the sentinel config has variant semantics for different configurations, in example configuration
https://github.com/redis/redis/blob/unstable/sentinel.conf, we put comments on these.
However the rewrite process only treat the sentinel config as a single option. During rewrite
process, it will mess up with the lines and comments.
Approaches:
In order to solve this issue, we need to differentiate different subconfig options in sentinel separately,
for example, sentinel monitor <master-name> <ip> <redis-port> <quorum>
we can treat it as sentinel monitor option, instead of the sentinel option.
This commit also fixes the dependency issue when putting configurations in sentinel.conf.
For example before this commit,we must put
`sentinel monitor <master-name> <ip> <redis-port> <quorum>` before
`sentinel auth-pass <master-name> <password>` for a single master,
otherwise the server cannot start and will return error. This commit fixes this issue, as long as
the monitoring master was configured, no matter the sequence is, the sentinel can start and run properly.
2021-01-26 02:31:54 -05:00
/* If this is sentinel config, we use sentinel "sentinel <config>" as option
to avoid messing up the sequence . */
if ( server . sentinel_mode & & argc > 1 & & ! strcasecmp ( argv [ 0 ] , " sentinel " ) ) {
sds sentinelOption = sdsempty ( ) ;
sentinelOption = sdscatfmt ( sentinelOption , " %S %S " , argv [ 0 ] , argv [ 1 ] ) ;
rewriteConfigAddLineNumberToOption ( state , sentinelOption , linenum ) ;
sdsfree ( sentinelOption ) ;
} else {
rewriteConfigAddLineNumberToOption ( state , argv [ 0 ] , linenum ) ;
}
2013-05-09 18:15:18 -04:00
sdsfreesplitres ( argv , argc ) ;
}
fclose ( fp ) ;
return state ;
}
/* Rewrite the specified configuration option with the new "line".
* It progressively uses lines of the file that were already used for the same
2013-11-18 12:18:04 -05:00
* configuration option in the old version of the file , removing that line from
2013-05-09 18:15:18 -04:00
* the map of options - > line numbers .
*
* If there are lines associated with a given configuration option and
* " force " is non - zero , the line is appended to the configuration file .
* Usually " force " is true when an option has not its default value , so it
* must be rewritten even if not present previously .
2014-06-26 12:48:40 -04:00
*
2013-05-09 18:15:18 -04:00
* The first time a line is appended into a configuration file , a comment
* is added to show that starting from that point the config file was generated
* by CONFIG REWRITE .
*
* " line " is either used , or freed , so the caller does not need to free it
* in any way . */
2015-03-11 11:59:56 -04:00
void rewriteConfigRewriteLine ( struct rewriteConfigState * state , const char * option , sds line , int force ) {
2013-05-09 18:15:18 -04:00
sds o = sdsnew ( option ) ;
list * l = dictFetchValue ( state - > option_to_line , o ) ;
2013-12-19 09:25:45 -05:00
rewriteConfigMarkAsProcessed ( state , option ) ;
2021-08-05 14:59:12 -04:00
if ( ! l & & ! force & & ! state - > force_write ) {
2013-05-09 18:15:18 -04:00
/* Option not used previously, and we are not forced to use it. */
sdsfree ( line ) ;
sdsfree ( o ) ;
return ;
}
if ( l ) {
listNode * ln = listFirst ( l ) ;
int linenum = ( long ) ln - > value ;
/* There are still lines in the old configuration file we can reuse
* for this option . Replace the line with the new one . */
listDelNode ( l , ln ) ;
if ( listLength ( l ) = = 0 ) dictDelete ( state - > option_to_line , o ) ;
sdsfree ( state - > lines [ linenum ] ) ;
state - > lines [ linenum ] = line ;
} else {
/* Append a new line. */
2021-08-05 14:59:12 -04:00
if ( state - > needs_signature ) {
2013-05-09 18:15:18 -04:00
rewriteConfigAppendLine ( state ,
2013-11-19 11:58:11 -05:00
sdsnew ( REDIS_CONFIG_REWRITE_SIGNATURE ) ) ;
2021-08-05 14:59:12 -04:00
state - > needs_signature = 0 ;
2013-05-09 18:15:18 -04:00
}
rewriteConfigAppendLine ( state , line ) ;
}
sdsfree ( o ) ;
}
2013-05-15 05:33:02 -04:00
/* Write the long long 'bytes' value as a string in a way that is parsable
* inside redis . conf . If possible uses the GB , MB , KB notation . */
int rewriteConfigFormatMemory ( char * buf , size_t len , long long bytes ) {
int gb = 1024 * 1024 * 1024 ;
int mb = 1024 * 1024 ;
int kb = 1024 ;
if ( bytes & & ( bytes % gb ) = = 0 ) {
return snprintf ( buf , len , " %lldgb " , bytes / gb ) ;
} else if ( bytes & & ( bytes % mb ) = = 0 ) {
return snprintf ( buf , len , " %lldmb " , bytes / mb ) ;
} else if ( bytes & & ( bytes % kb ) = = 0 ) {
return snprintf ( buf , len , " %lldkb " , bytes / kb ) ;
} else {
return snprintf ( buf , len , " %lld " , bytes ) ;
}
}
2013-05-09 18:15:18 -04:00
/* Rewrite a simple "option-name <bytes>" configuration option. */
2019-07-19 13:15:35 -04:00
void rewriteConfigBytesOption ( struct rewriteConfigState * state , const char * option , long long value , long long defvalue ) {
2013-05-15 05:33:02 -04:00
char buf [ 64 ] ;
2013-05-09 18:15:18 -04:00
int force = value ! = defvalue ;
2013-05-15 05:33:02 -04:00
sds line ;
2013-05-09 18:15:18 -04:00
2013-05-15 05:33:02 -04:00
rewriteConfigFormatMemory ( buf , sizeof ( buf ) , value ) ;
line = sdscatprintf ( sdsempty ( ) , " %s %s " , option , buf ) ;
2013-05-09 18:15:18 -04:00
rewriteConfigRewriteLine ( state , option , line , force ) ;
}
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
/* Rewrite a simple "option-name n%" configuration option. */
void rewriteConfigPercentOption ( struct rewriteConfigState * state , const char * option , long long value , long long defvalue ) {
int force = value ! = defvalue ;
sds line = sdscatprintf ( sdsempty ( ) , " %s %lld%% " , option , value ) ;
rewriteConfigRewriteLine ( state , option , line , force ) ;
}
2013-05-14 05:17:18 -04:00
/* Rewrite a yes/no option. */
2019-05-31 15:05:18 -04:00
void rewriteConfigYesNoOption ( struct rewriteConfigState * state , const char * option , int value , int defvalue ) {
2013-05-09 18:15:18 -04:00
int force = value ! = defvalue ;
sds line = sdscatprintf ( sdsempty ( ) , " %s %s " , option ,
value ? " yes " : " no " ) ;
rewriteConfigRewriteLine ( state , option , line , force ) ;
}
2013-05-14 05:17:18 -04:00
/* Rewrite a string option. */
2019-07-19 13:15:35 -04:00
void rewriteConfigStringOption ( struct rewriteConfigState * state , const char * option , char * value , const char * defvalue ) {
2013-05-09 18:15:18 -04:00
int force = 1 ;
sds line ;
/* String options set to NULL need to be not present at all in the
* configuration file to be set to NULL again at the next reboot . */
2013-12-19 09:25:45 -05:00
if ( value = = NULL ) {
rewriteConfigMarkAsProcessed ( state , option ) ;
return ;
}
2013-05-09 18:15:18 -04:00
2014-09-04 05:23:19 -04:00
/* Set force to zero if the value is set to its default. */
2013-05-09 18:15:18 -04:00
if ( defvalue & & strcmp ( value , defvalue ) = = 0 ) force = 0 ;
line = sdsnew ( option ) ;
line = sdscatlen ( line , " " , 1 ) ;
line = sdscatrepr ( line , value , strlen ( value ) ) ;
rewriteConfigRewriteLine ( state , option , line , force ) ;
}
2020-12-17 12:26:33 -05:00
/* Rewrite a SDS string option. */
2021-11-09 13:35:22 -05:00
void rewriteConfigSdsOption ( struct rewriteConfigState * state , const char * option , sds value , const char * defvalue ) {
2020-12-17 12:26:33 -05:00
int force = 1 ;
sds line ;
/* If there is no value set, we don't want the SDS option
* to be present in the configuration at all . */
if ( value = = NULL ) {
rewriteConfigMarkAsProcessed ( state , option ) ;
return ;
}
/* Set force to zero if the value is set to its default. */
2021-11-09 13:35:22 -05:00
if ( defvalue & & strcmp ( value , defvalue ) = = 0 ) force = 0 ;
2020-12-17 12:26:33 -05:00
line = sdsnew ( option ) ;
line = sdscatlen ( line , " " , 1 ) ;
line = sdscatrepr ( line , value , sdslen ( value ) ) ;
rewriteConfigRewriteLine ( state , option , line , force ) ;
}
2013-05-14 05:17:18 -04:00
/* Rewrite a numerical (long long range) option. */
2019-07-19 13:15:35 -04:00
void rewriteConfigNumericalOption ( struct rewriteConfigState * state , const char * option , long long value , long long defvalue ) {
2013-05-09 18:15:18 -04:00
int force = value ! = defvalue ;
sds line = sdscatprintf ( sdsempty ( ) , " %s %lld " , option , value ) ;
rewriteConfigRewriteLine ( state , option , line , force ) ;
}
Squash merging 125 typo/grammar/comment/doc PRs (#7773)
List of squashed commits or PRs
===============================
commit 66801ea
Author: hwware <wen.hui.ware@gmail.com>
Date: Mon Jan 13 00:54:31 2020 -0500
typo fix in acl.c
commit 46f55db
Author: Itamar Haber <itamar@redislabs.com>
Date: Sun Sep 6 18:24:11 2020 +0300
Updates a couple of comments
Specifically:
* RM_AutoMemory completed instead of pointing to docs
* Updated link to custom type doc
commit 61a2aa0
Author: xindoo <xindoo@qq.com>
Date: Tue Sep 1 19:24:59 2020 +0800
Correct errors in code comments
commit a5871d1
Author: yz1509 <pro-756@qq.com>
Date: Tue Sep 1 18:36:06 2020 +0800
fix typos in module.c
commit 41eede7
Author: bookug <bookug@qq.com>
Date: Sat Aug 15 01:11:33 2020 +0800
docs: fix typos in comments
commit c303c84
Author: lazy-snail <ws.niu@outlook.com>
Date: Fri Aug 7 11:15:44 2020 +0800
fix spelling in redis.conf
commit 1eb76bf
Author: zhujian <zhujianxyz@gmail.com>
Date: Thu Aug 6 15:22:10 2020 +0800
add a missing 'n' in comment
commit 1530ec2
Author: Daniel Dai <764122422@qq.com>
Date: Mon Jul 27 00:46:35 2020 -0400
fix spelling in tracking.c
commit e517b31
Author: Hunter-Chen <huntcool001@gmail.com>
Date: Fri Jul 17 22:33:32 2020 +0800
Update redis.conf
Co-authored-by: Itamar Haber <itamar@redislabs.com>
commit c300eff
Author: Hunter-Chen <huntcool001@gmail.com>
Date: Fri Jul 17 22:33:23 2020 +0800
Update redis.conf
Co-authored-by: Itamar Haber <itamar@redislabs.com>
commit 4c058a8
Author: 陈浩鹏 <chenhaopeng@heytea.com>
Date: Thu Jun 25 19:00:56 2020 +0800
Grammar fix and clarification
commit 5fcaa81
Author: bodong.ybd <bodong.ybd@alibaba-inc.com>
Date: Fri Jun 19 10:09:00 2020 +0800
Fix typos
commit 4caca9a
Author: Pruthvi P <pruthvi@ixigo.com>
Date: Fri May 22 00:33:22 2020 +0530
Fix typo eviciton => eviction
commit b2a25f6
Author: Brad Dunbar <dunbarb2@gmail.com>
Date: Sun May 17 12:39:59 2020 -0400
Fix a typo.
commit 12842ae
Author: hwware <wen.hui.ware@gmail.com>
Date: Sun May 3 17:16:59 2020 -0400
fix spelling in redis conf
commit ddba07c
Author: Chris Lamb <chris@chris-lamb.co.uk>
Date: Sat May 2 23:25:34 2020 +0100
Correct a "conflicts" spelling error.
commit 8fc7bf2
Author: Nao YONASHIRO <yonashiro@r.recruit.co.jp>
Date: Thu Apr 30 10:25:27 2020 +0900
docs: fix EXPIRE_FAST_CYCLE_DURATION to ACTIVE_EXPIRE_CYCLE_FAST_DURATION
commit 9b2b67a
Author: Brad Dunbar <dunbarb2@gmail.com>
Date: Fri Apr 24 11:46:22 2020 -0400
Fix a typo.
commit 0746f10
Author: devilinrust <63737265+devilinrust@users.noreply.github.com>
Date: Thu Apr 16 00:17:53 2020 +0200
Fix typos in server.c
commit 92b588d
Author: benjessop12 <56115861+benjessop12@users.noreply.github.com>
Date: Mon Apr 13 13:43:55 2020 +0100
Fix spelling mistake in lazyfree.c
commit 1da37aa
Merge: 2d4ba28 af347a8
Author: hwware <wen.hui.ware@gmail.com>
Date: Thu Mar 5 22:41:31 2020 -0500
Merge remote-tracking branch 'upstream/unstable' into expiretypofix
commit 2d4ba28
Author: hwware <wen.hui.ware@gmail.com>
Date: Mon Mar 2 00:09:40 2020 -0500
fix typo in expire.c
commit 1a746f7
Author: SennoYuki <minakami1yuki@gmail.com>
Date: Thu Feb 27 16:54:32 2020 +0800
fix typo
commit 8599b1a
Author: dongheejeong <donghee950403@gmail.com>
Date: Sun Feb 16 20:31:43 2020 +0000
Fix typo in server.c
commit f38d4e8
Author: hwware <wen.hui.ware@gmail.com>
Date: Sun Feb 2 22:58:38 2020 -0500
fix typo in evict.c
commit fe143fc
Author: Leo Murillo <leonardo.murillo@gmail.com>
Date: Sun Feb 2 01:57:22 2020 -0600
Fix a few typos in redis.conf
commit 1ab4d21
Author: viraja1 <anchan.viraj@gmail.com>
Date: Fri Dec 27 17:15:58 2019 +0530
Fix typo in Latency API docstring
commit ca1f70e
Author: gosth <danxuedexing@qq.com>
Date: Wed Dec 18 15:18:02 2019 +0800
fix typo in sort.c
commit a57c06b
Author: ZYunH <zyunhjob@163.com>
Date: Mon Dec 16 22:28:46 2019 +0800
fix-zset-typo
commit b8c92b5
Author: git-hulk <hulk.website@gmail.com>
Date: Mon Dec 16 15:51:42 2019 +0800
FIX: typo in cluster.c, onformation->information
commit 9dd981c
Author: wujm2007 <jim.wujm@gmail.com>
Date: Mon Dec 16 09:37:52 2019 +0800
Fix typo
commit e132d7a
Author: Sebastien Williams-Wynn <s.williamswynn.mail@gmail.com>
Date: Fri Nov 15 00:14:07 2019 +0000
Minor typo change
commit 47f44d5
Author: happynote3966 <01ssrmikururudevice01@gmail.com>
Date: Mon Nov 11 22:08:48 2019 +0900
fix comment typo in redis-cli.c
commit b8bdb0d
Author: fulei <fulei@kuaishou.com>
Date: Wed Oct 16 18:00:17 2019 +0800
Fix a spelling mistake of comments in defragDictBucketCallback
commit 0def46a
Author: fulei <fulei@kuaishou.com>
Date: Wed Oct 16 13:09:27 2019 +0800
fix some spelling mistakes of comments in defrag.c
commit f3596fd
Author: Phil Rajchgot <tophil@outlook.com>
Date: Sun Oct 13 02:02:32 2019 -0400
Typo and grammar fixes
Redis and its documentation are great -- just wanted to submit a few corrections in the spirit of Hacktoberfest. Thanks for all your work on this project. I use it all the time and it works beautifully.
commit 2b928cd
Author: KangZhiDong <worldkzd@gmail.com>
Date: Sun Sep 1 07:03:11 2019 +0800
fix typos
commit 33aea14
Author: Axlgrep <axlgrep@gmail.com>
Date: Tue Aug 27 11:02:18 2019 +0800
Fixed eviction spelling issues
commit e282a80
Author: Simen Flatby <simen@oms.no>
Date: Tue Aug 20 15:25:51 2019 +0200
Update comments to reflect prop name
In the comments the prop is referenced as replica-validity-factor,
but it is really named cluster-replica-validity-factor.
commit 74d1f9a
Author: Jim Green <jimgreen2013@qq.com>
Date: Tue Aug 20 20:00:31 2019 +0800
fix comment error, the code is ok
commit eea1407
Author: Liao Tonglang <liaotonglang@gmail.com>
Date: Fri May 31 10:16:18 2019 +0800
typo fix
fix cna't to can't
commit 0da553c
Author: KAWACHI Takashi <tkawachi@gmail.com>
Date: Wed Jul 17 00:38:16 2019 +0900
Fix typo
commit 7fc8fb6
Author: Michael Prokop <mika@grml.org>
Date: Tue May 28 17:58:42 2019 +0200
Typo fixes
s/familar/familiar/
s/compatiblity/compatibility/
s/ ot / to /
s/itsef/itself/
commit 5f46c9d
Author: zhumoing <34539422+zhumoing@users.noreply.github.com>
Date: Tue May 21 21:16:50 2019 +0800
typo-fixes
typo-fixes
commit 321dfe1
Author: wxisme <850885154@qq.com>
Date: Sat Mar 16 15:10:55 2019 +0800
typo fix
commit b4fb131
Merge: 267e0e6 3df1eb8
Author: Nikitas Bastas <nikitasbst@gmail.com>
Date: Fri Feb 8 22:55:45 2019 +0200
Merge branch 'unstable' of antirez/redis into unstable
commit 267e0e6
Author: Nikitas Bastas <nikitasbst@gmail.com>
Date: Wed Jan 30 21:26:04 2019 +0200
Minor typo fix
commit 30544e7
Author: inshal96 <39904558+inshal96@users.noreply.github.com>
Date: Fri Jan 4 16:54:50 2019 +0500
remove an extra 'a' in the comments
commit 337969d
Author: BrotherGao <yangdongheng11@gmail.com>
Date: Sat Dec 29 12:37:29 2018 +0800
fix typo in redis.conf
commit 9f4b121
Merge: 423a030 e504583
Author: BrotherGao <yangdongheng@xiaomi.com>
Date: Sat Dec 29 11:41:12 2018 +0800
Merge branch 'unstable' of antirez/redis into unstable
commit 423a030
Merge: 42b02b7 46a51cd
Author: 杨东衡 <yangdongheng@xiaomi.com>
Date: Tue Dec 4 23:56:11 2018 +0800
Merge branch 'unstable' of antirez/redis into unstable
commit 42b02b7
Merge: 68c0e6e b8febe6
Author: Dongheng Yang <yangdongheng11@gmail.com>
Date: Sun Oct 28 15:54:23 2018 +0800
Merge pull request #1 from antirez/unstable
update local data
commit 714b589
Author: Christian <crifei93@gmail.com>
Date: Fri Dec 28 01:17:26 2018 +0100
fix typo "resulution"
commit e23259d
Author: garenchan <1412950785@qq.com>
Date: Wed Dec 26 09:58:35 2018 +0800
fix typo: segfauls -> segfault
commit a9359f8
Author: xjp <jianping_xie@aliyun.com>
Date: Tue Dec 18 17:31:44 2018 +0800
Fixed REDISMODULE_H spell bug
commit a12c3e4
Author: jdiaz <jrd.palacios@gmail.com>
Date: Sat Dec 15 23:39:52 2018 -0600
Fixes hyperloglog hash function comment block description
commit 770eb11
Author: 林上耀 <1210tom@163.com>
Date: Sun Nov 25 17:16:10 2018 +0800
fix typo
commit fd97fbb
Author: Chris Lamb <chris@chris-lamb.co.uk>
Date: Fri Nov 23 17:14:01 2018 +0100
Correct "unsupported" typo.
commit a85522d
Author: Jungnam Lee <jungnam.lee@oracle.com>
Date: Thu Nov 8 23:01:29 2018 +0900
fix typo in test comments
commit ade8007
Author: Arun Kumar <palerdot@users.noreply.github.com>
Date: Tue Oct 23 16:56:35 2018 +0530
Fixed grammatical typo
Fixed typo for word 'dictionary'
commit 869ee39
Author: Hamid Alaei <hamid.a85@gmail.com>
Date: Sun Aug 12 16:40:02 2018 +0430
fix documentations: (ThreadSafeContextStart/Stop -> ThreadSafeContextLock/Unlock), minor typo
commit f89d158
Author: Mayank Jain <mayankjain255@gmail.com>
Date: Tue Jul 31 23:01:21 2018 +0530
Updated README.md with some spelling corrections.
Made correction in spelling of some misspelled words.
commit 892198e
Author: dsomeshwar <someshwar.dhayalan@gmail.com>
Date: Sat Jul 21 23:23:04 2018 +0530
typo fix
commit 8a4d780
Author: Itamar Haber <itamar@redislabs.com>
Date: Mon Apr 30 02:06:52 2018 +0300
Fixes some typos
commit e3acef6
Author: Noah Rosamilia <ivoahivoah@gmail.com>
Date: Sat Mar 3 23:41:21 2018 -0500
Fix typo in /deps/README.md
commit 04442fb
Author: WuYunlong <xzsyeb@126.com>
Date: Sat Mar 3 10:32:42 2018 +0800
Fix typo in readSyncBulkPayload() comment.
commit 9f36880
Author: WuYunlong <xzsyeb@126.com>
Date: Sat Mar 3 10:20:37 2018 +0800
replication.c comment: run_id -> replid.
commit f866b4a
Author: Francesco 'makevoid' Canessa <makevoid@gmail.com>
Date: Thu Feb 22 22:01:56 2018 +0000
fix comment typo in server.c
commit 0ebc69b
Author: 줍 <jubee0124@gmail.com>
Date: Mon Feb 12 16:38:48 2018 +0900
Fix typo in redis.conf
Fix `five behaviors` to `eight behaviors` in [this sentence ](antirez/redis@unstable/redis.conf#L564)
commit b50a620
Author: martinbroadhurst <martinbroadhurst@users.noreply.github.com>
Date: Thu Dec 28 12:07:30 2017 +0000
Fix typo in valgrind.sup
commit 7d8f349
Author: Peter Boughton <peter@sorcerersisle.com>
Date: Mon Nov 27 19:52:19 2017 +0000
Update CONTRIBUTING; refer doc updates to redis-doc repo.
commit 02dec7e
Author: Klauswk <klauswk1@hotmail.com>
Date: Tue Oct 24 16:18:38 2017 -0200
Fix typo in comment
commit e1efbc8
Author: chenshi <baiwfg2@gmail.com>
Date: Tue Oct 3 18:26:30 2017 +0800
Correct two spelling errors of comments
commit 93327d8
Author: spacewander <spacewanderlzx@gmail.com>
Date: Wed Sep 13 16:47:24 2017 +0800
Update the comment for OBJ_ENCODING_EMBSTR_SIZE_LIMIT's value
The value of OBJ_ENCODING_EMBSTR_SIZE_LIMIT is 44 now instead of 39.
commit 63d361f
Author: spacewander <spacewanderlzx@gmail.com>
Date: Tue Sep 12 15:06:42 2017 +0800
Fix <prevlen> related doc in ziplist.c
According to the definition of ZIP_BIG_PREVLEN and other related code,
the guard of single byte <prevlen> should be 254 instead of 255.
commit ebe228d
Author: hanael80 <hanael80@gmail.com>
Date: Tue Aug 15 09:09:40 2017 +0900
Fix typo
commit 6b696e6
Author: Matt Robenolt <matt@ydekproductions.com>
Date: Mon Aug 14 14:50:47 2017 -0700
Fix typo in LATENCY DOCTOR output
commit a2ec6ae
Author: caosiyang <caosiyang@qiyi.com>
Date: Tue Aug 15 14:15:16 2017 +0800
Fix a typo: form => from
commit 3ab7699
Author: caosiyang <caosiyang@qiyi.com>
Date: Thu Aug 10 18:40:33 2017 +0800
Fix a typo: replicationFeedSlavesFromMaster() => replicationFeedSlavesFromMasterStream()
commit 72d43ef
Author: caosiyang <caosiyang@qiyi.com>
Date: Tue Aug 8 15:57:25 2017 +0800
fix a typo: servewr => server
commit 707c958
Author: Bo Cai <charpty@gmail.com>
Date: Wed Jul 26 21:49:42 2017 +0800
redis-cli.c typo: conut -> count.
Signed-off-by: Bo Cai <charpty@gmail.com>
commit b9385b2
Author: JackDrogon <jack.xsuperman@gmail.com>
Date: Fri Jun 30 14:22:31 2017 +0800
Fix some spell problems
commit 20d9230
Author: akosel <aaronjkosel@gmail.com>
Date: Sun Jun 4 19:35:13 2017 -0500
Fix typo
commit b167bfc
Author: Krzysiek Witkowicz <krzysiekwitkowicz@gmail.com>
Date: Mon May 22 21:32:27 2017 +0100
Fix #4008 small typo in comment
commit 2b78ac8
Author: Jake Clarkson <jacobwclarkson@gmail.com>
Date: Wed Apr 26 15:49:50 2017 +0100
Correct typo in tests/unit/hyperloglog.tcl
commit b0f1cdb
Author: Qi Luo <qiluo-msft@users.noreply.github.com>
Date: Wed Apr 19 14:25:18 2017 -0700
Fix typo
commit a90b0f9
Author: charsyam <charsyam@naver.com>
Date: Thu Mar 16 18:19:53 2017 +0900
fix typos
fix typos
fix typos
commit 8430a79
Author: Richard Hart <richardhart92@gmail.com>
Date: Mon Mar 13 22:17:41 2017 -0400
Fixed log message typo in listenToPort.
commit 481a1c2
Author: Vinod Kumar <kumar003vinod@gmail.com>
Date: Sun Jan 15 23:04:51 2017 +0530
src/db.c: Correct "save" -> "safe" typo
commit 586b4d3
Author: wangshaonan <wshn13@gmail.com>
Date: Wed Dec 21 20:28:27 2016 +0800
Fix typo they->the in helloworld.c
commit c1c4b5e
Author: Jenner <hypxm@qq.com>
Date: Mon Dec 19 16:39:46 2016 +0800
typo error
commit 1ee1a3f
Author: tielei <43289893@qq.com>
Date: Mon Jul 18 13:52:25 2016 +0800
fix some comments
commit 11a41fb
Author: Otto Kekäläinen <otto@seravo.fi>
Date: Sun Jul 3 10:23:55 2016 +0100
Fix spelling in documentation and comments
commit 5fb5d82
Author: francischan <f1ancis621@gmail.com>
Date: Tue Jun 28 00:19:33 2016 +0800
Fix outdated comments about redis.c file.
It should now refer to server.c file.
commit 6b254bc
Author: lmatt-bit <lmatt123n@gmail.com>
Date: Thu Apr 21 21:45:58 2016 +0800
Refine the comment of dictRehashMilliseconds func
SLAVECONF->REPLCONF in comment - by andyli029
commit ee9869f
Author: clark.kang <charsyam@naver.com>
Date: Tue Mar 22 11:09:51 2016 +0900
fix typos
commit f7b3b11
Author: Harisankar H <harisankarh@gmail.com>
Date: Wed Mar 9 11:49:42 2016 +0530
Typo correction: "faield" --> "failed"
Typo correction: "faield" --> "failed"
commit 3fd40fc
Author: Itamar Haber <itamar@redislabs.com>
Date: Thu Feb 25 10:31:51 2016 +0200
Fixes a typo in comments
commit 621c160
Author: Prayag Verma <prayag.verma@gmail.com>
Date: Mon Feb 1 12:36:20 2016 +0530
Fix typo in Readme.md
Spelling mistakes -
`eviciton` > `eviction`
`familar` > `familiar`
commit d7d07d6
Author: WonCheol Lee <toctoc21c@gmail.com>
Date: Wed Dec 30 15:11:34 2015 +0900
Typo fixed
commit a4dade7
Author: Felix Bünemann <buenemann@louis.info>
Date: Mon Dec 28 11:02:55 2015 +0100
[ci skip] Improve supervised upstart config docs
This mentions that "expect stop" is required for supervised upstart
to work correctly. See http://upstart.ubuntu.com/cookbook/#expect-stop
for an explanation.
commit d9caba9
Author: daurnimator <quae@daurnimator.com>
Date: Mon Dec 21 18:30:03 2015 +1100
README: Remove trailing whitespace
commit 72d42e5
Author: daurnimator <quae@daurnimator.com>
Date: Mon Dec 21 18:29:32 2015 +1100
README: Fix typo. th => the
commit dd6e957
Author: daurnimator <quae@daurnimator.com>
Date: Mon Dec 21 18:29:20 2015 +1100
README: Fix typo. familar => familiar
commit 3a12b23
Author: daurnimator <quae@daurnimator.com>
Date: Mon Dec 21 18:28:54 2015 +1100
README: Fix typo. eviciton => eviction
commit 2d1d03b
Author: daurnimator <quae@daurnimator.com>
Date: Mon Dec 21 18:21:45 2015 +1100
README: Fix typo. sever => server
commit 3973b06
Author: Itamar Haber <itamar@garantiadata.com>
Date: Sat Dec 19 17:01:20 2015 +0200
Typo fix
commit 4f2e460
Author: Steve Gao <fu@2token.com>
Date: Fri Dec 4 10:22:05 2015 +0800
Update README - fix typos
commit b21667c
Author: binyan <binbin.yan@nokia.com>
Date: Wed Dec 2 22:48:37 2015 +0800
delete redundancy color judge in sdscatcolor
commit 88894c7
Author: binyan <binbin.yan@nokia.com>
Date: Wed Dec 2 22:14:42 2015 +0800
the example output shoule be HelloWorld
commit 2763470
Author: binyan <binbin.yan@nokia.com>
Date: Wed Dec 2 17:41:39 2015 +0800
modify error word keyevente
Signed-off-by: binyan <binbin.yan@nokia.com>
commit 0847b3d
Author: Bruno Martins <bscmartins@gmail.com>
Date: Wed Nov 4 11:37:01 2015 +0000
typo
commit bbb9e9e
Author: dawedawe <dawedawe@gmx.de>
Date: Fri Mar 27 00:46:41 2015 +0100
typo: zimap -> zipmap
commit 5ed297e
Author: Axel Advento <badwolf.bloodseeker.rev@gmail.com>
Date: Tue Mar 3 15:58:29 2015 +0800
Fix 'salve' typos to 'slave'
commit edec9d6
Author: LudwikJaniuk <ludvig.janiuk@gmail.com>
Date: Wed Jun 12 14:12:47 2019 +0200
Update README.md
Co-Authored-By: Qix <Qix-@users.noreply.github.com>
commit 692a7af
Author: LudwikJaniuk <ludvig.janiuk@gmail.com>
Date: Tue May 28 14:32:04 2019 +0200
grammar
commit d962b0a
Author: Nick Frost <nickfrostatx@gmail.com>
Date: Wed Jul 20 15:17:12 2016 -0700
Minor grammar fix
commit 24fff01aaccaf5956973ada8c50ceb1462e211c6 (typos)
Author: Chad Miller <chadm@squareup.com>
Date: Tue Sep 8 13:46:11 2020 -0400
Fix faulty comment about operation of unlink()
commit 3cd5c1f3326c52aa552ada7ec797c6bb16452355
Author: Kevin <kevin.xgr@gmail.com>
Date: Wed Nov 20 00:13:50 2019 +0800
Fix typo in server.c.
From a83af59 Mon Sep 17 00:00:00 2001
From: wuwo <wuwo@wacai.com>
Date: Fri, 17 Mar 2017 20:37:45 +0800
Subject: [PATCH] falure to failure
From c961896 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E5=B7=A6=E6=87=B6?= <veficos@gmail.com>
Date: Sat, 27 May 2017 15:33:04 +0800
Subject: [PATCH] fix typo
From e600ef2 Mon Sep 17 00:00:00 2001
From: "rui.zou" <rui.zou@yunify.com>
Date: Sat, 30 Sep 2017 12:38:15 +0800
Subject: [PATCH] fix a typo
From c7d07fa Mon Sep 17 00:00:00 2001
From: Alexandre Perrin <alex@kaworu.ch>
Date: Thu, 16 Aug 2018 10:35:31 +0200
Subject: [PATCH] deps README.md typo
From b25cb67 Mon Sep 17 00:00:00 2001
From: Guy Korland <gkorland@gmail.com>
Date: Wed, 26 Sep 2018 10:55:37 +0300
Subject: [PATCH 1/2] fix typos in header
From ad28ca6 Mon Sep 17 00:00:00 2001
From: Guy Korland <gkorland@gmail.com>
Date: Wed, 26 Sep 2018 11:02:36 +0300
Subject: [PATCH 2/2] fix typos
commit 34924cdedd8552466fc22c1168d49236cb7ee915
Author: Adrian Lynch <adi_ady_ade@hotmail.com>
Date: Sat Apr 4 21:59:15 2015 +0100
Typos fixed
commit fd2a1e7
Author: Jan <jsteemann@users.noreply.github.com>
Date: Sat Oct 27 19:13:01 2018 +0200
Fix typos
Fix typos
commit e14e47c1a234b53b0e103c5f6a1c61481cbcbb02
Author: Andy Lester <andy@petdance.com>
Date: Fri Aug 2 22:30:07 2019 -0500
Fix multiple misspellings of "following"
commit 79b948ce2dac6b453fe80995abbcaac04c213d5a
Author: Andy Lester <andy@petdance.com>
Date: Fri Aug 2 22:24:28 2019 -0500
Fix misspelling of create-cluster
commit 1fffde52666dc99ab35efbd31071a4c008cb5a71
Author: Andy Lester <andy@petdance.com>
Date: Wed Jul 31 17:57:56 2019 -0500
Fix typos
commit 204c9ba9651e9e05fd73936b452b9a30be456cfe
Author: Xiaobo Zhu <xiaobo.zhu@shopee.com>
Date: Tue Aug 13 22:19:25 2019 +0800
fix typos
Squashed commit of the following:
commit 1d9aaf8
Author: danmedani <danmedani@gmail.com>
Date: Sun Aug 2 11:40:26 2015 -0700
README typo fix.
Squashed commit of the following:
commit 32bfa7c
Author: Erik Dubbelboer <erik@dubbelboer.com>
Date: Mon Jul 6 21:15:08 2015 +0200
Fixed grammer
Squashed commit of the following:
commit b24f69c
Author: Sisir Koppaka <sisir.koppaka@gmail.com>
Date: Mon Mar 2 22:38:45 2015 -0500
utils/hashtable/rehashing.c: Fix typos
Squashed commit of the following:
commit 4e04082
Author: Erik Dubbelboer <erik@dubbelboer.com>
Date: Mon Mar 23 08:22:21 2015 +0000
Small config file documentation improvements
Squashed commit of the following:
commit acb8773
Author: ctd1500 <ctd1500@gmail.com>
Date: Fri May 8 01:52:48 2015 -0700
Typo and grammar fixes in readme
commit 2eb75b6
Author: ctd1500 <ctd1500@gmail.com>
Date: Fri May 8 01:36:18 2015 -0700
fixed redis.conf comment
Squashed commit of the following:
commit a8249a2
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Date: Fri Dec 11 11:39:52 2015 +0530
Revise correction of typos.
Squashed commit of the following:
commit 3c02028
Author: zhaojun11 <zhaojun11@jd.com>
Date: Wed Jan 17 19:05:28 2018 +0800
Fix typos include two code typos in cluster.c and latency.c
Squashed commit of the following:
commit 9dba47c
Author: q191201771 <191201771@qq.com>
Date: Sat Jan 4 11:31:04 2020 +0800
fix function listCreate comment in adlist.c
Update src/server.c
commit 2c7c2cb536e78dd211b1ac6f7bda00f0f54faaeb
Author: charpty <charpty@gmail.com>
Date: Tue May 1 23:16:59 2018 +0800
server.c typo: modules system dictionary type comment
Signed-off-by: charpty <charpty@gmail.com>
commit a8395323fb63cb59cb3591cb0f0c8edb7c29a680
Author: Itamar Haber <itamar@redislabs.com>
Date: Sun May 6 00:25:18 2018 +0300
Updates test_helper.tcl's help with undocumented options
Specifically:
* Host
* Port
* Client
commit bde6f9ced15755cd6407b4af7d601b030f36d60b
Author: wxisme <850885154@qq.com>
Date: Wed Aug 8 15:19:19 2018 +0800
fix comments in deps files
commit 3172474ba991532ab799ee1873439f3402412331
Author: wxisme <850885154@qq.com>
Date: Wed Aug 8 14:33:49 2018 +0800
fix some comments
commit 01b6f2b6858b5cf2ce4ad5092d2c746e755f53f0
Author: Thor Juhasz <thor@juhasz.pro>
Date: Sun Nov 18 14:37:41 2018 +0100
Minor fixes to comments
Found some parts a little unclear on a first read, which prompted me to have a better look at the file and fix some minor things I noticed.
Fixing minor typos and grammar. There are no changes to configuration options.
These changes are only meant to help the user better understand the explanations to the various configuration options
2020-09-10 06:43:38 -04:00
/* Rewrite an octal option. */
2021-10-19 02:58:52 -04:00
void rewriteConfigOctalOption ( struct rewriteConfigState * state , const char * option , long long value , long long defvalue ) {
2013-05-09 18:15:18 -04:00
int force = value ! = defvalue ;
2021-10-19 02:58:52 -04:00
sds line = sdscatprintf ( sdsempty ( ) , " %s %llo " , option , value ) ;
2013-05-09 18:15:18 -04:00
rewriteConfigRewriteLine ( state , option , line , force ) ;
}
2015-03-11 18:20:57 -04:00
/* Rewrite an enumeration option. It takes as usually state and option name,
* and in addition the enumeration array and the default value for the
* option . */
2019-07-19 13:15:35 -04:00
void rewriteConfigEnumOption ( struct rewriteConfigState * state , const char * option , int value , configEnum * ce , int defval ) {
2013-05-09 18:15:18 -04:00
sds line ;
2015-03-11 18:20:57 -04:00
const char * name = configEnumGetNameOrUnknown ( ce , value ) ;
int force = value ! = defval ;
2014-06-26 12:48:40 -04:00
2015-03-11 18:20:57 -04:00
line = sdscatprintf ( sdsempty ( ) , " %s %s " , option , name ) ;
2013-05-09 18:15:18 -04:00
rewriteConfigRewriteLine ( state , option , line , force ) ;
}
2013-05-14 05:17:18 -04:00
/* Rewrite the save option. */
2021-11-07 06:40:08 -05:00
void rewriteConfigSaveOption ( typeData data , const char * name , struct rewriteConfigState * state ) {
UNUSED ( data ) ;
2013-05-13 05:11:35 -04:00
int j ;
sds line ;
2020-10-22 12:47:32 -04:00
/* In Sentinel mode we don't need to rewrite the save parameters */
if ( server . sentinel_mode ) {
2021-11-07 06:40:08 -05:00
rewriteConfigMarkAsProcessed ( state , name ) ;
2020-10-22 12:47:32 -04:00
return ;
}
2021-03-29 11:53:20 -04:00
/* Rewrite save parameters, or an empty 'save ""' line to avoid the
* defaults from being used .
*/
if ( ! server . saveparamslen ) {
2021-11-07 06:40:08 -05:00
rewriteConfigRewriteLine ( state , name , sdsnew ( " save \" \" " ) , 1 ) ;
2021-03-29 11:53:20 -04:00
} else {
for ( j = 0 ; j < server . saveparamslen ; j + + ) {
line = sdscatprintf ( sdsempty ( ) , " save %ld %d " ,
( long ) server . saveparams [ j ] . seconds , server . saveparams [ j ] . changes ) ;
2021-11-07 06:40:08 -05:00
rewriteConfigRewriteLine ( state , name , line , 1 ) ;
2021-03-29 11:53:20 -04:00
}
2013-05-13 05:11:35 -04:00
}
2021-03-29 11:53:20 -04:00
2013-12-23 06:48:39 -05:00
/* Mark "save" as processed in case server.saveparamslen is zero. */
2021-11-07 06:40:08 -05:00
rewriteConfigMarkAsProcessed ( state , name ) ;
2013-05-09 18:15:18 -04:00
}
2019-02-05 04:48:17 -05:00
/* Rewrite the user option. */
void rewriteConfigUserOption ( struct rewriteConfigState * state ) {
/* If there is a user file defined we just mark this configuration
* directive as processed , so that all the lines containing users
* inside the config file gets discarded . */
if ( server . acl_filename [ 0 ] ! = ' \0 ' ) {
rewriteConfigMarkAsProcessed ( state , " user " ) ;
return ;
}
/* Otherwise scan the list of users and rewrite every line. Note that
* in case the list here is empty , the effect will just be to comment
* all the users directive inside the config file . */
raxIterator ri ;
raxStart ( & ri , Users ) ;
raxSeek ( & ri , " ^ " , NULL , 0 ) ;
while ( raxNext ( & ri ) ) {
user * u = ri . data ;
sds line = sdsnew ( " user " ) ;
line = sdscatsds ( line , u - > name ) ;
line = sdscatlen ( line , " " , 1 ) ;
sds descr = ACLDescribeUser ( u ) ;
line = sdscatsds ( line , descr ) ;
sdsfree ( descr ) ;
rewriteConfigRewriteLine ( state , " user " , line , 1 ) ;
}
raxStop ( & ri ) ;
/* Mark "user" as processed in case there are no defined users. */
rewriteConfigMarkAsProcessed ( state , " user " ) ;
}
2013-05-14 05:17:18 -04:00
/* Rewrite the dir option, always using absolute paths.*/
2021-11-07 06:40:08 -05:00
void rewriteConfigDirOption ( typeData data , const char * name , struct rewriteConfigState * state ) {
UNUSED ( data ) ;
2013-05-13 05:26:43 -04:00
char cwd [ 1024 ] ;
2013-12-19 09:25:45 -05:00
if ( getcwd ( cwd , sizeof ( cwd ) ) = = NULL ) {
2021-11-07 06:40:08 -05:00
rewriteConfigMarkAsProcessed ( state , name ) ;
2013-12-19 09:25:45 -05:00
return ; /* no rewrite on error. */
}
2021-11-07 06:40:08 -05:00
rewriteConfigStringOption ( state , name , cwd , NULL ) ;
2013-05-09 18:15:18 -04:00
}
2013-05-14 05:17:18 -04:00
/* Rewrite the slaveof option. */
2021-11-07 06:40:08 -05:00
void rewriteConfigReplicaOfOption ( typeData data , const char * name , struct rewriteConfigState * state ) {
UNUSED ( data ) ;
2013-05-13 05:26:43 -04:00
sds line ;
/* If this is a master, we want all the slaveof config options
2014-01-20 05:10:42 -05:00
* in the file to be removed . Note that if this is a cluster instance
* we don ' t want a slaveof directive inside redis . conf . */
if ( server . cluster_enabled | | server . masterhost = = NULL ) {
2021-11-07 06:40:08 -05:00
rewriteConfigMarkAsProcessed ( state , name ) ;
2013-12-19 09:25:45 -05:00
return ;
}
2021-11-07 06:40:08 -05:00
line = sdscatprintf ( sdsempty ( ) , " %s %s %d " , name ,
2013-05-13 05:26:43 -04:00
server . masterhost , server . masterport ) ;
2021-11-07 06:40:08 -05:00
rewriteConfigRewriteLine ( state , name , line , 1 ) ;
2013-05-09 18:15:18 -04:00
}
2013-05-14 05:17:18 -04:00
/* Rewrite the notify-keyspace-events option. */
2021-11-07 06:40:08 -05:00
void rewriteConfigNotifyKeyspaceEventsOption ( typeData data , const char * name , struct rewriteConfigState * state ) {
UNUSED ( data ) ;
2013-05-13 12:34:18 -04:00
int force = server . notify_keyspace_events ! = 0 ;
sds line , flags ;
flags = keyspaceEventsFlagsToString ( server . notify_keyspace_events ) ;
2021-11-07 06:40:08 -05:00
line = sdsnew ( name ) ;
2013-05-15 05:15:18 -04:00
line = sdscatlen ( line , " " , 1 ) ;
line = sdscatrepr ( line , flags , sdslen ( flags ) ) ;
2013-05-13 12:34:18 -04:00
sdsfree ( flags ) ;
2021-11-07 06:40:08 -05:00
rewriteConfigRewriteLine ( state , name , line , force ) ;
2013-05-09 18:15:18 -04:00
}
2013-05-14 05:17:18 -04:00
/* Rewrite the client-output-buffer-limit option. */
2021-11-07 06:40:08 -05:00
void rewriteConfigClientOutputBufferLimitOption ( typeData data , const char * name , struct rewriteConfigState * state ) {
UNUSED ( data ) ;
2013-05-13 12:34:18 -04:00
int j ;
2015-07-28 10:58:04 -04:00
for ( j = 0 ; j < CLIENT_TYPE_OBUF_COUNT ; j + + ) {
2013-05-13 12:34:18 -04:00
int force = ( server . client_obuf_limits [ j ] . hard_limit_bytes ! =
clientBufferLimitsDefaults [ j ] . hard_limit_bytes ) | |
( server . client_obuf_limits [ j ] . soft_limit_bytes ! =
clientBufferLimitsDefaults [ j ] . soft_limit_bytes ) | |
( server . client_obuf_limits [ j ] . soft_limit_seconds ! =
clientBufferLimitsDefaults [ j ] . soft_limit_seconds ) ;
sds line ;
2013-05-15 05:33:02 -04:00
char hard [ 64 ] , soft [ 64 ] ;
rewriteConfigFormatMemory ( hard , sizeof ( hard ) ,
server . client_obuf_limits [ j ] . hard_limit_bytes ) ;
rewriteConfigFormatMemory ( soft , sizeof ( soft ) ,
server . client_obuf_limits [ j ] . soft_limit_bytes ) ;
2013-05-13 12:34:18 -04:00
2018-09-10 06:27:37 -04:00
char * typename = getClientTypeName ( j ) ;
if ( ! strcmp ( typename , " slave " ) ) typename = " replica " ;
2013-05-15 05:33:02 -04:00
line = sdscatprintf ( sdsempty ( ) , " %s %s %s %s %ld " ,
2021-11-07 06:40:08 -05:00
name , typename , hard , soft ,
2013-05-13 12:34:18 -04:00
( long ) server . client_obuf_limits [ j ] . soft_limit_seconds ) ;
2021-11-07 06:40:08 -05:00
rewriteConfigRewriteLine ( state , name , line , force ) ;
2013-05-13 12:34:18 -04:00
}
2013-05-09 18:15:18 -04:00
}
2020-08-12 10:58:56 -04:00
/* Rewrite the oom-score-adj-values option. */
2021-11-07 06:40:08 -05:00
void rewriteConfigOOMScoreAdjValuesOption ( typeData data , const char * name , struct rewriteConfigState * state ) {
UNUSED ( data ) ;
2020-08-12 10:58:56 -04:00
int force = 0 ;
int j ;
sds line ;
2021-11-07 06:40:08 -05:00
line = sdsnew ( name ) ;
2020-09-08 09:00:20 -04:00
line = sdscatlen ( line , " " , 1 ) ;
2020-08-12 10:58:56 -04:00
for ( j = 0 ; j < CONFIG_OOM_COUNT ; j + + ) {
if ( server . oom_score_adj_values [ j ] ! = configOOMScoreAdjValuesDefaults [ j ] )
force = 1 ;
line = sdscatprintf ( line , " %d " , server . oom_score_adj_values [ j ] ) ;
if ( j + 1 ! = CONFIG_OOM_COUNT )
line = sdscatlen ( line , " " , 1 ) ;
}
2021-11-07 06:40:08 -05:00
rewriteConfigRewriteLine ( state , name , line , force ) ;
2020-08-12 10:58:56 -04:00
}
2013-07-04 12:50:15 -04:00
/* Rewrite the bind option. */
2021-11-07 06:40:08 -05:00
void rewriteConfigBindOption ( typeData data , const char * name , struct rewriteConfigState * state ) {
UNUSED ( data ) ;
2013-07-04 12:50:15 -04:00
int force = 1 ;
sds line , addresses ;
2021-06-22 05:50:17 -04:00
int is_default = 0 ;
/* Compare server.bindaddr with CONFIG_DEFAULT_BINDADDR */
if ( server . bindaddr_count = = CONFIG_DEFAULT_BINDADDR_COUNT ) {
is_default = 1 ;
char * default_bindaddr [ CONFIG_DEFAULT_BINDADDR_COUNT ] = CONFIG_DEFAULT_BINDADDR ;
for ( int j = 0 ; j < CONFIG_DEFAULT_BINDADDR_COUNT ; j + + ) {
if ( strcmp ( server . bindaddr [ j ] , default_bindaddr [ j ] ) ! = 0 ) {
is_default = 0 ;
break ;
}
}
}
2013-07-04 12:50:15 -04:00
2021-06-22 05:50:17 -04:00
if ( is_default ) {
2021-11-07 06:40:08 -05:00
rewriteConfigMarkAsProcessed ( state , name ) ;
2013-12-19 09:25:45 -05:00
return ;
}
2013-07-04 12:50:15 -04:00
/* Rewrite as bind <addr1> <addr2> ... <addrN> */
2021-06-22 05:50:17 -04:00
if ( server . bindaddr_count > 0 )
addresses = sdsjoin ( server . bindaddr , server . bindaddr_count , " " ) ;
else
addresses = sdsnew ( " \" \" " ) ;
2021-11-07 06:40:08 -05:00
line = sdsnew ( name ) ;
2013-07-04 12:50:15 -04:00
line = sdscatlen ( line , " " , 1 ) ;
line = sdscatsds ( line , addresses ) ;
sdsfree ( addresses ) ;
2021-11-07 06:40:08 -05:00
rewriteConfigRewriteLine ( state , name , line , force ) ;
2013-07-04 12:50:15 -04:00
}
2018-04-20 09:01:21 -04:00
/* Rewrite the loadmodule option. */
void rewriteConfigLoadmoduleOption ( struct rewriteConfigState * state ) {
sds line ;
dictIterator * di = dictGetIterator ( modules ) ;
dictEntry * de ;
while ( ( de = dictNext ( di ) ) ! = NULL ) {
struct RedisModule * module = dictGetVal ( de ) ;
line = sdsnew ( " loadmodule " ) ;
line = sdscatsds ( line , module - > loadmod - > path ) ;
for ( int i = 0 ; i < module - > loadmod - > argc ; i + + ) {
line = sdscatlen ( line , " " , 1 ) ;
line = sdscatsds ( line , module - > loadmod - > argv [ i ] - > ptr ) ;
}
rewriteConfigRewriteLine ( state , " loadmodule " , line , 1 ) ;
}
dictReleaseIterator ( di ) ;
/* Mark "loadmodule" as processed in case modules is empty. */
rewriteConfigMarkAsProcessed ( state , " loadmodule " ) ;
}
2013-05-14 04:22:55 -04:00
/* Glue together the configuration lines in the current configuration
* rewrite state into a single string , stripping multiple empty lines . */
2013-05-09 18:15:18 -04:00
sds rewriteConfigGetContentFromState ( struct rewriteConfigState * state ) {
sds content = sdsempty ( ) ;
2013-05-14 04:22:55 -04:00
int j , was_empty = 0 ;
2013-05-09 18:15:18 -04:00
for ( j = 0 ; j < state - > numlines ; j + + ) {
2013-05-14 04:22:55 -04:00
/* Every cluster of empty lines is turned into a single empty line. */
if ( sdslen ( state - > lines [ j ] ) = = 0 ) {
if ( was_empty ) continue ;
was_empty = 1 ;
} else {
was_empty = 0 ;
}
2013-05-09 18:15:18 -04:00
content = sdscatsds ( content , state - > lines [ j ] ) ;
content = sdscatlen ( content , " \n " , 1 ) ;
}
return content ;
}
2013-05-14 05:17:18 -04:00
/* At the end of the rewrite process the state contains the remaining
* map between " option name " = > " lines in the original config file " .
* Lines used by the rewrite process were removed by the function
* rewriteConfigRewriteLine ( ) , all the other lines are " orphaned " and
* should be replaced by empty lines .
*
* This function does just this , iterating all the option names and
2013-12-19 09:55:25 -05:00
* blanking all the lines still associated . */
2013-05-14 05:17:18 -04:00
void rewriteConfigRemoveOrphaned ( struct rewriteConfigState * state ) {
dictIterator * di = dictGetIterator ( state - > option_to_line ) ;
dictEntry * de ;
while ( ( de = dictNext ( di ) ) ! = NULL ) {
list * l = dictGetVal ( de ) ;
2013-12-19 09:25:45 -05:00
sds option = dictGetKey ( de ) ;
/* Don't blank lines about options the rewrite process
* don ' t understand . */
2013-12-23 06:48:39 -05:00
if ( dictFind ( state - > rewritten , option ) = = NULL ) {
2015-07-27 03:41:48 -04:00
serverLog ( LL_DEBUG , " Not rewritten option: %s " , option ) ;
2013-12-19 09:25:45 -05:00
continue ;
}
2013-05-14 05:17:18 -04:00
while ( listLength ( l ) ) {
listNode * ln = listFirst ( l ) ;
int linenum = ( long ) ln - > value ;
sdsfree ( state - > lines [ linenum ] ) ;
state - > lines [ linenum ] = sdsempty ( ) ;
listDelNode ( l , ln ) ;
}
}
dictReleaseIterator ( di ) ;
}
2021-08-05 14:59:12 -04:00
/* This function returns a string representation of all the config options
* marked with DEBUG_CONFIG , which can be used to help with debugging . */
sds getConfigDebugInfo ( ) {
struct rewriteConfigState * state = rewriteConfigCreateState ( ) ;
state - > force_write = 1 ; /* Force the output */
state - > needs_signature = 0 ; /* Omit the rewrite signature */
/* Iterate the configs and "rewrite" the ones that have
* the debug flag . */
for ( standardConfig * config = configs ; config - > name ! = NULL ; config + + ) {
if ( ! ( config - > flags & DEBUG_CONFIG ) ) continue ;
config - > interface . rewrite ( config - > data , config - > name , state ) ;
}
sds info = rewriteConfigGetContentFromState ( state ) ;
rewriteConfigReleaseState ( state ) ;
return info ;
}
2020-09-25 05:55:45 -04:00
/* This function replaces the old configuration file with the new content
* in an atomic manner .
2013-05-14 06:32:25 -04:00
*
* The function returns 0 on success , otherwise - 1 is returned and errno
2020-09-25 05:55:45 -04:00
* is set accordingly . */
2013-05-14 06:32:25 -04:00
int rewriteConfigOverwriteFile ( char * configfile , sds content ) {
2020-09-25 05:55:45 -04:00
int fd = - 1 ;
int retval = - 1 ;
char tmp_conffile [ PATH_MAX ] ;
const char * tmp_suffix = " .XXXXXX " ;
size_t offset = 0 ;
ssize_t written_bytes = 0 ;
int tmp_path_len = snprintf ( tmp_conffile , sizeof ( tmp_conffile ) , " %s%s " , configfile , tmp_suffix ) ;
if ( tmp_path_len < = 0 | | ( unsigned int ) tmp_path_len > = sizeof ( tmp_conffile ) ) {
serverLog ( LL_WARNING , " Config file full path is too long " ) ;
errno = ENAMETOOLONG ;
return retval ;
2013-05-14 06:32:25 -04:00
}
2020-09-25 05:55:45 -04:00
# ifdef _GNU_SOURCE
fd = mkostemp ( tmp_conffile , O_CLOEXEC ) ;
# else
/* There's a theoretical chance here to leak the FD if a module thread forks & execv in the middle */
fd = mkstemp ( tmp_conffile ) ;
# endif
if ( fd = = - 1 ) {
serverLog ( LL_WARNING , " Could not create tmp config file (%s) " , strerror ( errno ) ) ;
return retval ;
2013-05-14 06:32:25 -04:00
}
2020-09-25 05:55:45 -04:00
while ( offset < sdslen ( content ) ) {
written_bytes = write ( fd , content + offset , sdslen ( content ) - offset ) ;
if ( written_bytes < = 0 ) {
if ( errno = = EINTR ) continue ; /* FD is blocking, no other retryable errors */
2020-10-20 02:12:24 -04:00
serverLog ( LL_WARNING , " Failed after writing (%zd) bytes to tmp config file (%s) " , offset , strerror ( errno ) ) ;
2020-09-25 05:55:45 -04:00
goto cleanup ;
}
offset + = written_bytes ;
2013-05-14 06:32:25 -04:00
}
2020-09-25 05:55:45 -04:00
if ( fsync ( fd ) )
serverLog ( LL_WARNING , " Could not sync tmp config file to disk (%s) " , strerror ( errno ) ) ;
2021-01-20 14:57:24 -05:00
else if ( fchmod ( fd , 0644 & ~ server . umask ) = = - 1 )
2020-09-25 05:55:45 -04:00
serverLog ( LL_WARNING , " Could not chmod config file (%s) " , strerror ( errno ) ) ;
else if ( rename ( tmp_conffile , configfile ) = = - 1 )
serverLog ( LL_WARNING , " Could not rename tmp config file (%s) " , strerror ( errno ) ) ;
else {
retval = 0 ;
serverLog ( LL_DEBUG , " Rewritten config file (%s) successfully " , configfile ) ;
2013-05-14 06:32:25 -04:00
}
cleanup :
close ( fd ) ;
2020-09-25 05:55:45 -04:00
if ( retval ) unlink ( tmp_conffile ) ;
2013-05-14 06:32:25 -04:00
return retval ;
}
2013-05-09 18:15:18 -04:00
/* Rewrite the configuration file at "path".
* If the configuration file already exists , we try at best to retain comments
* and overall structure .
*
* Configuration parameters that are at their default value , unless already
* explicitly included in the old configuration file , are not rewritten .
2021-08-05 14:59:12 -04:00
* The force_write flag overrides this behavior and forces everything to be
2020-09-09 08:43:11 -04:00
* written . This is currently only used for testing purposes .
2013-05-09 18:15:18 -04:00
*
* On error - 1 is returned and errno is set accordingly , otherwise 0. */
2021-08-05 14:59:12 -04:00
int rewriteConfig ( char * path , int force_write ) {
2013-05-09 18:15:18 -04:00
struct rewriteConfigState * state ;
sds newcontent ;
2013-05-14 06:32:25 -04:00
int retval ;
2013-05-09 18:15:18 -04:00
/* Step 1: read the old config into our rewrite state. */
if ( ( state = rewriteConfigReadOldFile ( path ) ) = = NULL ) return - 1 ;
2021-08-05 14:59:12 -04:00
if ( force_write ) state - > force_write = 1 ;
2013-05-09 18:15:18 -04:00
/* Step 2: rewrite every single option, replacing or appending it inside
* the rewrite state . */
2019-05-31 15:05:18 -04:00
/* Iterate the configs that are standard */
2019-07-19 13:15:35 -04:00
for ( standardConfig * config = configs ; config - > name ! = NULL ; config + + ) {
2021-11-07 06:40:08 -05:00
if ( config - > interface . rewrite ) config - > interface . rewrite ( config - > data , config - > name , state ) ;
2019-05-31 15:05:18 -04:00
}
2019-02-05 04:48:17 -05:00
rewriteConfigUserOption ( state ) ;
2018-04-20 09:01:21 -04:00
rewriteConfigLoadmoduleOption ( state ) ;
2015-03-11 18:20:57 -04:00
/* Rewrite Sentinel config if in Sentinel mode. */
2013-11-19 03:48:12 -05:00
if ( server . sentinel_mode ) rewriteConfigSentinelOption ( state ) ;
2013-05-09 18:15:18 -04:00
/* Step 3: remove all the orphaned lines in the old file, that is, lines
* that were used by a config option and are no longer used , like in case
* of multiple " save " options or duplicated options . */
2013-05-14 05:17:18 -04:00
rewriteConfigRemoveOrphaned ( state ) ;
2013-05-09 18:15:18 -04:00
/* Step 4: generate a new configuration file from the modified state
* and write it into the original file . */
newcontent = rewriteConfigGetContentFromState ( state ) ;
2013-05-14 06:32:25 -04:00
retval = rewriteConfigOverwriteFile ( server . configfile , newcontent ) ;
2013-05-09 18:15:18 -04:00
sdsfree ( newcontent ) ;
rewriteConfigReleaseState ( state ) ;
2013-05-14 06:32:25 -04:00
return retval ;
2013-05-09 18:15:18 -04:00
}
2019-07-19 13:15:35 -04:00
/*-----------------------------------------------------------------------------
* Configs that fit one of the major types and require no special handling
* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */
# define LOADBUF_SIZE 256
static char loadbuf [ LOADBUF_SIZE ] ;
2021-03-16 01:00:29 -04:00
# define embedCommonConfig(config_name, config_alias, config_flags) \
2019-07-19 13:15:35 -04:00
. name = ( config_name ) , \
. alias = ( config_alias ) , \
2021-03-16 01:00:29 -04:00
. flags = ( config_flags ) ,
2019-07-19 13:15:35 -04:00
2021-12-01 03:15:11 -05:00
# define embedConfigInterface(initfn, setfn, getfn, rewritefn, applyfn) .interface = { \
2019-11-26 09:52:28 -05:00
. init = ( initfn ) , \
2019-07-19 13:15:35 -04:00
. set = ( setfn ) , \
. get = ( getfn ) , \
2021-12-01 03:15:11 -05:00
. rewrite = ( rewritefn ) , \
. apply = ( applyfn ) \
2019-07-19 13:15:35 -04:00
} ,
2019-11-22 11:54:22 -05:00
/* What follows is the generic config types that are supported. To add a new
2019-07-19 13:15:35 -04:00
* config with one of these types , add it to the standardConfig table with
* the creation macro for each type .
2019-11-22 11:54:22 -05:00
*
2019-07-19 13:15:35 -04:00
* Each type contains the following :
* * A function defining how to load this type on startup .
* * A function defining how to update this type on CONFIG SET .
* * A function defining how to serialize this type on CONFIG SET .
* * A function defining how to rewrite this type on CONFIG REWRITE .
2019-11-22 11:54:22 -05:00
* * A Macro defining how to create this type .
2019-07-19 13:15:35 -04:00
*/
2019-11-22 11:54:22 -05:00
2019-07-19 13:15:35 -04:00
/* Bool Configs */
2019-11-26 09:52:28 -05:00
static void boolConfigInit ( typeData data ) {
* data . yesno . config = data . yesno . default_value ;
}
2021-12-01 03:15:11 -05:00
static int boolConfigSet ( typeData data , sds * argv , int argc , const char * * err ) {
2021-11-07 06:40:08 -05:00
UNUSED ( argc ) ;
int yn = yesnotoi ( argv [ 0 ] ) ;
2020-02-05 04:41:24 -05:00
if ( yn = = - 1 ) {
2019-07-19 13:15:35 -04:00
* err = " argument must be 'yes' or 'no' " ;
return 0 ;
}
2019-11-26 09:52:28 -05:00
if ( data . yesno . is_valid_fn & & ! data . yesno . is_valid_fn ( yn , err ) )
return 0 ;
2019-11-28 04:11:07 -05:00
int prev = * ( data . yesno . config ) ;
2021-12-01 03:15:11 -05:00
if ( prev ! = yn ) {
* ( data . yesno . config ) = yn ;
return 1 ;
2019-11-28 04:11:07 -05:00
}
2021-12-01 03:15:11 -05:00
return 2 ;
2019-07-19 13:15:35 -04:00
}
2021-12-01 03:15:11 -05:00
static sds boolConfigGet ( typeData data ) {
return sdsnew ( * data . yesno . config ? " yes " : " no " ) ;
2019-07-19 13:15:35 -04:00
}
static void boolConfigRewrite ( typeData data , const char * name , struct rewriteConfigState * state ) {
2019-11-22 00:49:52 -05:00
rewriteConfigYesNoOption ( state , name , * ( data . yesno . config ) , data . yesno . default_value ) ;
2019-07-19 13:15:35 -04:00
}
2021-12-01 03:15:11 -05:00
# define createBoolConfig(name, alias, flags, config_addr, default, is_valid, apply) { \
2021-03-16 01:00:29 -04:00
embedCommonConfig ( name , alias , flags ) \
2021-12-01 03:15:11 -05:00
embedConfigInterface ( boolConfigInit , boolConfigSet , boolConfigGet , boolConfigRewrite , apply ) \
2019-11-22 00:49:52 -05:00
. data . yesno = { \
2019-07-19 13:15:35 -04:00
. config = & ( config_addr ) , \
2019-11-26 09:52:28 -05:00
. default_value = ( default ) , \
. is_valid_fn = ( is_valid ) , \
2019-07-19 13:15:35 -04:00
} \
}
/* String Configs */
2019-11-26 09:52:28 -05:00
static void stringConfigInit ( typeData data ) {
2020-12-17 12:26:33 -05:00
* data . string . config = ( data . string . convert_empty_to_null & & ! data . string . default_value ) ? NULL : zstrdup ( data . string . default_value ) ;
2019-11-26 09:52:28 -05:00
}
2021-12-01 03:15:11 -05:00
static int stringConfigSet ( typeData data , sds * argv , int argc , const char * * err ) {
2021-11-07 06:40:08 -05:00
UNUSED ( argc ) ;
if ( data . string . is_valid_fn & & ! data . string . is_valid_fn ( argv [ 0 ] , err ) )
2019-11-26 09:52:28 -05:00
return 0 ;
2019-11-28 04:11:07 -05:00
char * prev = * data . string . config ;
2021-12-01 03:15:11 -05:00
char * new = ( data . string . convert_empty_to_null & & ! argv [ 0 ] [ 0 ] ) ? NULL : argv [ 0 ] ;
if ( new ! = prev & & ( new = = NULL | | prev = = NULL | | strcmp ( prev , new ) ) ) {
* data . string . config = new ! = NULL ? zstrdup ( new ) : NULL ;
zfree ( prev ) ;
return 1 ;
2019-11-28 04:11:07 -05:00
}
2021-12-01 03:15:11 -05:00
return 2 ;
2019-07-19 13:15:35 -04:00
}
2021-12-01 03:15:11 -05:00
static sds stringConfigGet ( typeData data ) {
return sdsnew ( * data . string . config ? * data . string . config : " " ) ;
2019-07-19 13:15:35 -04:00
}
static void stringConfigRewrite ( typeData data , const char * name , struct rewriteConfigState * state ) {
rewriteConfigStringOption ( state , name , * ( data . string . config ) , data . string . default_value ) ;
}
2020-12-17 12:26:33 -05:00
/* SDS Configs */
static void sdsConfigInit ( typeData data ) {
* data . sds . config = ( data . sds . convert_empty_to_null & & ! data . sds . default_value ) ? NULL : sdsnew ( data . sds . default_value ) ;
}
2021-12-01 03:15:11 -05:00
static int sdsConfigSet ( typeData data , sds * argv , int argc , const char * * err ) {
2021-11-07 06:40:08 -05:00
UNUSED ( argc ) ;
if ( data . sds . is_valid_fn & & ! data . sds . is_valid_fn ( argv [ 0 ] , err ) )
2020-12-17 12:26:33 -05:00
return 0 ;
sds prev = * data . sds . config ;
2021-12-01 03:15:11 -05:00
sds new = ( data . string . convert_empty_to_null & & ( sdslen ( argv [ 0 ] ) = = 0 ) ) ? NULL : argv [ 0 ] ;
if ( new ! = prev & & ( new = = NULL | | prev = = NULL | | sdscmp ( prev , new ) ) ) {
* data . sds . config = new ! = NULL ? sdsdup ( new ) : NULL ;
sdsfree ( prev ) ;
return 1 ;
2020-12-17 12:26:33 -05:00
}
2021-12-01 03:15:11 -05:00
return 2 ;
2020-12-17 12:26:33 -05:00
}
2021-12-01 03:15:11 -05:00
static sds sdsConfigGet ( typeData data ) {
2020-12-17 12:26:33 -05:00
if ( * data . sds . config ) {
2021-12-01 03:15:11 -05:00
return sdsdup ( * data . sds . config ) ;
2020-12-17 12:26:33 -05:00
} else {
2021-12-01 03:15:11 -05:00
return sdsnew ( " " ) ;
2020-12-17 12:26:33 -05:00
}
}
static void sdsConfigRewrite ( typeData data , const char * name , struct rewriteConfigState * state ) {
2021-11-09 13:35:22 -05:00
rewriteConfigSdsOption ( state , name , * ( data . sds . config ) , data . sds . default_value ) ;
2020-12-17 12:26:33 -05:00
}
2019-11-22 00:31:53 -05:00
# define ALLOW_EMPTY_STRING 0
# define EMPTY_STRING_IS_NULL 1
2021-12-01 03:15:11 -05:00
# define createStringConfig(name, alias, flags, empty_to_null, config_addr, default, is_valid, apply) { \
2021-03-16 01:00:29 -04:00
embedCommonConfig ( name , alias , flags ) \
2021-12-01 03:15:11 -05:00
embedConfigInterface ( stringConfigInit , stringConfigSet , stringConfigGet , stringConfigRewrite , apply ) \
2019-07-19 13:15:35 -04:00
. data . string = { \
. config = & ( config_addr ) , \
. default_value = ( default ) , \
2019-11-26 09:52:28 -05:00
. is_valid_fn = ( is_valid ) , \
. convert_empty_to_null = ( empty_to_null ) , \
2019-07-19 13:15:35 -04:00
} \
}
2021-12-01 03:15:11 -05:00
# define createSDSConfig(name, alias, flags, empty_to_null, config_addr, default, is_valid, apply) { \
2021-03-16 01:00:29 -04:00
embedCommonConfig ( name , alias , flags ) \
2021-12-01 03:15:11 -05:00
embedConfigInterface ( sdsConfigInit , sdsConfigSet , sdsConfigGet , sdsConfigRewrite , apply ) \
2020-12-17 12:26:33 -05:00
. data . sds = { \
. config = & ( config_addr ) , \
. default_value = ( default ) , \
. is_valid_fn = ( is_valid ) , \
. convert_empty_to_null = ( empty_to_null ) , \
} \
}
2019-07-19 13:15:35 -04:00
/* Enum configs */
2020-02-05 04:41:24 -05:00
static void enumConfigInit ( typeData data ) {
2019-11-26 09:52:28 -05:00
* data . enumd . config = data . enumd . default_value ;
}
2021-12-01 03:15:11 -05:00
static int enumConfigSet ( typeData data , sds * argv , int argc , const char * * err ) {
2021-11-07 06:40:08 -05:00
UNUSED ( argc ) ;
int enumval = configEnumGetValue ( data . enumd . enum_value , argv [ 0 ] ) ;
2019-07-19 13:15:35 -04:00
if ( enumval = = INT_MIN ) {
sds enumerr = sdsnew ( " argument must be one of the following: " ) ;
configEnum * enumNode = data . enumd . enum_value ;
while ( enumNode - > name ! = NULL ) {
2020-02-27 11:47:50 -05:00
enumerr = sdscatlen ( enumerr , enumNode - > name ,
strlen ( enumNode - > name ) ) ;
2019-11-22 01:28:50 -05:00
enumerr = sdscatlen ( enumerr , " , " , 2 ) ;
2019-07-19 13:15:35 -04:00
enumNode + + ;
}
2020-02-27 11:47:50 -05:00
sdsrange ( enumerr , 0 , - 3 ) ; /* Remove final ", ". */
2019-07-19 13:15:35 -04:00
strncpy ( loadbuf , enumerr , LOADBUF_SIZE ) ;
2020-01-14 03:10:39 -05:00
loadbuf [ LOADBUF_SIZE - 1 ] = ' \0 ' ;
2019-07-19 13:15:35 -04:00
sdsfree ( enumerr ) ;
* err = loadbuf ;
return 0 ;
}
2019-11-26 09:52:28 -05:00
if ( data . enumd . is_valid_fn & & ! data . enumd . is_valid_fn ( enumval , err ) )
return 0 ;
2019-11-28 04:11:07 -05:00
int prev = * ( data . enumd . config ) ;
2021-12-01 03:15:11 -05:00
if ( prev ! = enumval ) {
* ( data . enumd . config ) = enumval ;
return 1 ;
2019-11-28 04:11:07 -05:00
}
2021-12-01 03:15:11 -05:00
return 2 ;
2019-07-19 13:15:35 -04:00
}
2021-12-01 03:15:11 -05:00
static sds enumConfigGet ( typeData data ) {
return sdsnew ( configEnumGetNameOrUnknown ( data . enumd . enum_value , * data . enumd . config ) ) ;
2019-07-19 13:15:35 -04:00
}
2020-02-05 04:41:24 -05:00
static void enumConfigRewrite ( typeData data , const char * name , struct rewriteConfigState * state ) {
2019-07-19 13:15:35 -04:00
rewriteConfigEnumOption ( state , name , * ( data . enumd . config ) , data . enumd . enum_value , data . enumd . default_value ) ;
}
2021-12-01 03:15:11 -05:00
# define createEnumConfig(name, alias, flags, enum, config_addr, default, is_valid, apply) { \
2021-03-16 01:00:29 -04:00
embedCommonConfig ( name , alias , flags ) \
2021-12-01 03:15:11 -05:00
embedConfigInterface ( enumConfigInit , enumConfigSet , enumConfigGet , enumConfigRewrite , apply ) \
2019-07-19 13:15:35 -04:00
. data . enumd = { \
. config = & ( config_addr ) , \
. default_value = ( default ) , \
2019-11-26 09:52:28 -05:00
. is_valid_fn = ( is_valid ) , \
. enum_value = ( enum ) , \
2019-07-19 13:15:35 -04:00
} \
}
2019-11-28 04:11:07 -05:00
/* Gets a 'long long val' and sets it into the union, using a macro to get
* compile time type check . */
# define SET_NUMERIC_TYPE(val) \
if ( data . numeric . numeric_type = = NUMERIC_TYPE_INT ) { \
* ( data . numeric . config . i ) = ( int ) val ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_UINT ) { \
* ( data . numeric . config . ui ) = ( unsigned int ) val ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_LONG ) { \
* ( data . numeric . config . l ) = ( long ) val ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_ULONG ) { \
* ( data . numeric . config . ul ) = ( unsigned long ) val ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_LONG_LONG ) { \
* ( data . numeric . config . ll ) = ( long long ) val ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_ULONG_LONG ) { \
* ( data . numeric . config . ull ) = ( unsigned long long ) val ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_SIZE_T ) { \
* ( data . numeric . config . st ) = ( size_t ) val ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_SSIZE_T ) { \
* ( data . numeric . config . sst ) = ( ssize_t ) val ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_OFF_T ) { \
* ( data . numeric . config . ot ) = ( off_t ) val ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_TIME_T ) { \
* ( data . numeric . config . tt ) = ( time_t ) val ; \
}
/* Gets a 'long long val' and sets it with the value from the union, using a
* macro to get compile time type check . */
# define GET_NUMERIC_TYPE(val) \
if ( data . numeric . numeric_type = = NUMERIC_TYPE_INT ) { \
val = * ( data . numeric . config . i ) ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_UINT ) { \
val = * ( data . numeric . config . ui ) ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_LONG ) { \
val = * ( data . numeric . config . l ) ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_ULONG ) { \
val = * ( data . numeric . config . ul ) ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_LONG_LONG ) { \
val = * ( data . numeric . config . ll ) ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_ULONG_LONG ) { \
val = * ( data . numeric . config . ull ) ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_SIZE_T ) { \
val = * ( data . numeric . config . st ) ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_SSIZE_T ) { \
val = * ( data . numeric . config . sst ) ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_OFF_T ) { \
val = * ( data . numeric . config . ot ) ; \
} else if ( data . numeric . numeric_type = = NUMERIC_TYPE_TIME_T ) { \
val = * ( data . numeric . config . tt ) ; \
}
2019-07-19 13:15:35 -04:00
/* Numeric configs */
2019-11-26 09:52:28 -05:00
static void numericConfigInit ( typeData data ) {
2019-11-28 04:11:07 -05:00
SET_NUMERIC_TYPE ( data . numeric . default_value )
2019-11-26 09:52:28 -05:00
}
2021-01-21 04:56:08 -05:00
static int numericBoundaryCheck ( typeData data , long long ll , const char * * err ) {
2019-11-28 04:11:07 -05:00
if ( data . numeric . numeric_type = = NUMERIC_TYPE_ULONG_LONG | |
data . numeric . numeric_type = = NUMERIC_TYPE_UINT | |
data . numeric . numeric_type = = NUMERIC_TYPE_SIZE_T ) {
/* Boundary check for unsigned types */
unsigned long long ull = ll ;
unsigned long long upper_bound = data . numeric . upper_bound ;
unsigned long long lower_bound = data . numeric . lower_bound ;
if ( ull > upper_bound | | ull < lower_bound ) {
2021-10-19 02:58:52 -04:00
if ( data . numeric . flags & OCTAL_CONFIG ) {
snprintf ( loadbuf , LOADBUF_SIZE ,
" argument must be between %llo and %llo inclusive " ,
lower_bound ,
upper_bound ) ;
} else {
snprintf ( loadbuf , LOADBUF_SIZE ,
" argument must be between %llu and %llu inclusive " ,
lower_bound ,
upper_bound ) ;
}
2019-11-28 04:11:07 -05:00
* err = loadbuf ;
return 0 ;
}
} else {
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
/* Boundary check for percentages */
if ( data . numeric . flags & PERCENT_CONFIG & & ll < 0 ) {
if ( ll < data . numeric . lower_bound ) {
snprintf ( loadbuf , LOADBUF_SIZE ,
" percentage argument must be less or equal to %lld " ,
- data . numeric . lower_bound ) ;
* err = loadbuf ;
return 0 ;
}
}
2019-11-28 04:11:07 -05:00
/* Boundary check for signed types */
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
else if ( ll > data . numeric . upper_bound | | ll < data . numeric . lower_bound ) {
2019-11-28 04:11:07 -05:00
snprintf ( loadbuf , LOADBUF_SIZE ,
" argument must be between %lld and %lld inclusive " ,
data . numeric . lower_bound ,
data . numeric . upper_bound ) ;
* err = loadbuf ;
return 0 ;
}
2019-07-19 13:15:35 -04:00
}
2019-12-01 01:19:25 -05:00
return 1 ;
}
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
static int numericParseString ( typeData data , sds value , const char * * err , long long * res ) {
/* First try to parse as memory */
if ( data . numeric . flags & MEMORY_CONFIG ) {
int memerr ;
* res = memtoull ( value , & memerr ) ;
if ( ! memerr )
return 1 ;
}
/* Attempt to parse as percent */
if ( data . numeric . flags & PERCENT_CONFIG & &
sdslen ( value ) > 1 & & value [ sdslen ( value ) - 1 ] = = ' % ' & &
string2ll ( value , sdslen ( value ) - 1 , res ) & &
* res > = 0 ) {
/* We store percentage as negative value */
* res = - * res ;
return 1 ;
}
2021-10-19 02:58:52 -04:00
/* Attempt to parse as an octal number */
if ( data . numeric . flags & OCTAL_CONFIG ) {
char * endptr ;
errno = 0 ;
* res = strtoll ( value , & endptr , 8 ) ;
if ( errno = = 0 & & * endptr = = ' \0 ' )
return 1 ; /* No overflow or invalid characters */
}
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
/* Attempt a simple number (no special flags set) */
if ( ! data . numeric . flags & & string2ll ( value , sdslen ( value ) , res ) )
return 1 ;
/* Select appropriate error string */
if ( data . numeric . flags & MEMORY_CONFIG & &
data . numeric . flags & PERCENT_CONFIG )
* err = " argument must be a memory or percent value " ;
else if ( data . numeric . flags & MEMORY_CONFIG )
* err = " argument must be a memory value " ;
2021-10-19 02:58:52 -04:00
else if ( data . numeric . flags & OCTAL_CONFIG )
* err = " argument couldn't be parsed as an octal number " ;
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
else
* err = " argument couldn't be parsed into an integer " ;
return 0 ;
}
2021-08-23 14:00:40 -04:00
2021-12-01 03:15:11 -05:00
static int numericConfigSet ( typeData data , sds * argv , int argc , const char * * err ) {
2021-11-07 06:40:08 -05:00
UNUSED ( argc ) ;
2020-02-05 04:41:24 -05:00
long long ll , prev = 0 ;
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
2021-11-07 06:40:08 -05:00
if ( ! numericParseString ( data , argv [ 0 ] , err , & ll ) )
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
return 0 ;
2019-12-01 01:19:25 -05:00
if ( ! numericBoundaryCheck ( data , ll , err ) )
return 0 ;
2019-11-22 00:31:53 -05:00
2019-11-26 09:52:28 -05:00
if ( data . numeric . is_valid_fn & & ! data . numeric . is_valid_fn ( ll , err ) )
return 0 ;
2019-11-28 04:11:07 -05:00
GET_NUMERIC_TYPE ( prev )
2021-12-01 03:15:11 -05:00
if ( prev ! = ll ) {
SET_NUMERIC_TYPE ( ll )
return 1 ;
2019-11-28 04:11:07 -05:00
}
2021-12-01 03:15:11 -05:00
return 2 ;
2019-07-19 13:15:35 -04:00
}
2021-12-01 03:15:11 -05:00
static sds numericConfigGet ( typeData data ) {
2019-07-19 13:15:35 -04:00
char buf [ 128 ] ;
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
long long value = 0 ;
GET_NUMERIC_TYPE ( value )
2019-07-19 13:15:35 -04:00
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
if ( data . numeric . flags & PERCENT_CONFIG & & value < 0 ) {
int len = ll2string ( buf , sizeof ( buf ) , - value ) ;
buf [ len ] = ' % ' ;
buf [ len + 1 ] = ' \0 ' ;
}
else if ( data . numeric . flags & MEMORY_CONFIG ) {
ull2string ( buf , sizeof ( buf ) , value ) ;
2021-10-19 02:58:52 -04:00
} else if ( data . numeric . flags & OCTAL_CONFIG ) {
snprintf ( buf , sizeof ( buf ) , " %llo " , value ) ;
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
} else {
2021-08-23 14:00:40 -04:00
ll2string ( buf , sizeof ( buf ) , value ) ;
}
2021-12-01 03:15:11 -05:00
return sdsnew ( buf ) ;
2019-07-19 13:15:35 -04:00
}
static void numericConfigRewrite ( typeData data , const char * name , struct rewriteConfigState * state ) {
2019-11-22 00:49:52 -05:00
long long value = 0 ;
2019-11-28 04:11:07 -05:00
GET_NUMERIC_TYPE ( value )
2019-07-19 13:15:35 -04:00
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
if ( data . numeric . flags & PERCENT_CONFIG & & value < 0 ) {
rewriteConfigPercentOption ( state , name , - value , data . numeric . default_value ) ;
} else if ( data . numeric . flags & MEMORY_CONFIG ) {
2019-07-19 13:15:35 -04:00
rewriteConfigBytesOption ( state , name , value , data . numeric . default_value ) ;
2021-10-19 02:58:52 -04:00
} else if ( data . numeric . flags & OCTAL_CONFIG ) {
rewriteConfigOctalOption ( state , name , value , data . numeric . default_value ) ;
2019-07-19 13:15:35 -04:00
} else {
rewriteConfigNumericalOption ( state , name , value , data . numeric . default_value ) ;
}
}
2021-12-01 03:15:11 -05:00
# define embedCommonNumericalConfig(name, alias, _flags, lower, upper, config_addr, default, num_conf_flags, is_valid, apply) { \
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
embedCommonConfig ( name , alias , _flags ) \
2021-12-01 03:15:11 -05:00
embedConfigInterface ( numericConfigInit , numericConfigSet , numericConfigGet , numericConfigRewrite , apply ) \
2019-07-19 13:15:35 -04:00
. data . numeric = { \
. lower_bound = ( lower ) , \
. upper_bound = ( upper ) , \
. default_value = ( default ) , \
2019-11-26 09:52:28 -05:00
. is_valid_fn = ( is_valid ) , \
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
. flags = ( num_conf_flags ) ,
2019-07-19 13:15:35 -04:00
2021-12-01 03:15:11 -05:00
# define createIntConfig(name, alias, flags, lower, upper, config_addr, default, num_conf_flags, is_valid, apply) \
embedCommonNumericalConfig ( name , alias , flags , lower , upper , config_addr , default , num_conf_flags , is_valid , apply ) \
2019-07-19 13:15:35 -04:00
. numeric_type = NUMERIC_TYPE_INT , \
. config . i = & ( config_addr ) \
} \
}
2021-12-01 03:15:11 -05:00
# define createUIntConfig(name, alias, flags, lower, upper, config_addr, default, num_conf_flags, is_valid, apply) \
embedCommonNumericalConfig ( name , alias , flags , lower , upper , config_addr , default , num_conf_flags , is_valid , apply ) \
2019-11-28 04:11:07 -05:00
. numeric_type = NUMERIC_TYPE_UINT , \
. config . ui = & ( config_addr ) \
} \
}
2021-12-01 03:15:11 -05:00
# define createLongConfig(name, alias, flags, lower, upper, config_addr, default, num_conf_flags, is_valid, apply) \
embedCommonNumericalConfig ( name , alias , flags , lower , upper , config_addr , default , num_conf_flags , is_valid , apply ) \
2019-11-28 04:11:07 -05:00
. numeric_type = NUMERIC_TYPE_LONG , \
. config . l = & ( config_addr ) \
} \
}
2021-12-01 03:15:11 -05:00
# define createULongConfig(name, alias, flags, lower, upper, config_addr, default, num_conf_flags, is_valid, apply) \
embedCommonNumericalConfig ( name , alias , flags , lower , upper , config_addr , default , num_conf_flags , is_valid , apply ) \
2019-11-28 04:11:07 -05:00
. numeric_type = NUMERIC_TYPE_ULONG , \
2019-07-19 13:15:35 -04:00
. config . ul = & ( config_addr ) \
} \
}
2021-12-01 03:15:11 -05:00
# define createLongLongConfig(name, alias, flags, lower, upper, config_addr, default, num_conf_flags, is_valid, apply) \
embedCommonNumericalConfig ( name , alias , flags , lower , upper , config_addr , default , num_conf_flags , is_valid , apply ) \
2019-07-19 13:15:35 -04:00
. numeric_type = NUMERIC_TYPE_LONG_LONG , \
. config . ll = & ( config_addr ) \
} \
}
2021-12-01 03:15:11 -05:00
# define createULongLongConfig(name, alias, flags, lower, upper, config_addr, default, num_conf_flags, is_valid, apply) \
embedCommonNumericalConfig ( name , alias , flags , lower , upper , config_addr , default , num_conf_flags , is_valid , apply ) \
2019-11-28 04:11:07 -05:00
. numeric_type = NUMERIC_TYPE_ULONG_LONG , \
. config . ull = & ( config_addr ) \
} \
}
2021-12-01 03:15:11 -05:00
# define createSizeTConfig(name, alias, flags, lower, upper, config_addr, default, num_conf_flags, is_valid, apply) \
embedCommonNumericalConfig ( name , alias , flags , lower , upper , config_addr , default , num_conf_flags , is_valid , apply ) \
2019-07-19 13:15:35 -04:00
. numeric_type = NUMERIC_TYPE_SIZE_T , \
. config . st = & ( config_addr ) \
} \
}
2021-12-01 03:15:11 -05:00
# define createSSizeTConfig(name, alias, flags, lower, upper, config_addr, default, num_conf_flags, is_valid, apply) \
embedCommonNumericalConfig ( name , alias , flags , lower , upper , config_addr , default , num_conf_flags , is_valid , apply ) \
2019-11-28 04:11:07 -05:00
. numeric_type = NUMERIC_TYPE_SSIZE_T , \
. config . sst = & ( config_addr ) \
} \
}
2021-12-01 03:15:11 -05:00
# define createTimeTConfig(name, alias, flags, lower, upper, config_addr, default, num_conf_flags, is_valid, apply) \
embedCommonNumericalConfig ( name , alias , flags , lower , upper , config_addr , default , num_conf_flags , is_valid , apply ) \
2019-11-28 04:11:07 -05:00
. numeric_type = NUMERIC_TYPE_TIME_T , \
. config . tt = & ( config_addr ) \
} \
}
2021-12-01 03:15:11 -05:00
# define createOffTConfig(name, alias, flags, lower, upper, config_addr, default, num_conf_flags, is_valid, apply) \
embedCommonNumericalConfig ( name , alias , flags , lower , upper , config_addr , default , num_conf_flags , is_valid , apply ) \
2019-11-28 04:11:07 -05:00
. numeric_type = NUMERIC_TYPE_OFF_T , \
. config . ot = & ( config_addr ) \
} \
}
2021-12-01 03:15:11 -05:00
# define createSpecialConfig(name, alias, modifiable, setfn, getfn, rewritefn, applyfn) { \
2021-11-07 06:40:08 -05:00
embedCommonConfig ( name , alias , modifiable ) \
2021-12-01 03:15:11 -05:00
embedConfigInterface ( NULL , setfn , getfn , rewritefn , applyfn ) \
2021-11-07 06:40:08 -05:00
}
2021-01-21 04:56:08 -05:00
static int isValidActiveDefrag ( int val , const char * * err ) {
2019-11-26 09:52:28 -05:00
# ifndef HAVE_DEFRAG
if ( val ) {
* err = " Active defragmentation cannot be enabled: it "
" requires a Redis server compiled with a modified Jemalloc "
" like the one shipped by default with the Redis source "
" distribution " ;
return 0 ;
}
# else
UNUSED ( val ) ;
UNUSED ( err ) ;
# endif
return 1 ;
}
2021-01-21 04:56:08 -05:00
static int isValidDBfilename ( char * val , const char * * err ) {
2019-11-28 04:24:57 -05:00
if ( ! pathIsBaseName ( val ) ) {
* err = " dbfilename can't be a path, just a filename " ;
return 0 ;
}
return 1 ;
}
2021-01-21 04:56:08 -05:00
static int isValidAOFfilename ( char * val , const char * * err ) {
2019-11-28 04:24:57 -05:00
if ( ! pathIsBaseName ( val ) ) {
* err = " appendfilename can't be a path, just a filename " ;
return 0 ;
}
return 1 ;
}
2021-01-28 11:17:39 -05:00
/* Validate specified string is a valid proc-title-template */
static int isValidProcTitleTemplate ( char * val , const char * * err ) {
if ( ! validateProcTitleTemplate ( val ) ) {
* err = " template format is invalid or contains unknown variables " ;
return 0 ;
}
return 1 ;
}
2021-12-01 03:15:11 -05:00
static int updateProcTitleTemplate ( const char * * err ) {
2021-01-28 11:17:39 -05:00
if ( redisSetProcTitle ( NULL ) = = C_ERR ) {
* err = " failed to set process title " ;
return 0 ;
}
return 1 ;
}
2021-12-01 03:15:11 -05:00
static int updateHZ ( const char * * err ) {
2019-11-28 04:24:57 -05:00
UNUSED ( err ) ;
Squash merging 125 typo/grammar/comment/doc PRs (#7773)
List of squashed commits or PRs
===============================
commit 66801ea
Author: hwware <wen.hui.ware@gmail.com>
Date: Mon Jan 13 00:54:31 2020 -0500
typo fix in acl.c
commit 46f55db
Author: Itamar Haber <itamar@redislabs.com>
Date: Sun Sep 6 18:24:11 2020 +0300
Updates a couple of comments
Specifically:
* RM_AutoMemory completed instead of pointing to docs
* Updated link to custom type doc
commit 61a2aa0
Author: xindoo <xindoo@qq.com>
Date: Tue Sep 1 19:24:59 2020 +0800
Correct errors in code comments
commit a5871d1
Author: yz1509 <pro-756@qq.com>
Date: Tue Sep 1 18:36:06 2020 +0800
fix typos in module.c
commit 41eede7
Author: bookug <bookug@qq.com>
Date: Sat Aug 15 01:11:33 2020 +0800
docs: fix typos in comments
commit c303c84
Author: lazy-snail <ws.niu@outlook.com>
Date: Fri Aug 7 11:15:44 2020 +0800
fix spelling in redis.conf
commit 1eb76bf
Author: zhujian <zhujianxyz@gmail.com>
Date: Thu Aug 6 15:22:10 2020 +0800
add a missing 'n' in comment
commit 1530ec2
Author: Daniel Dai <764122422@qq.com>
Date: Mon Jul 27 00:46:35 2020 -0400
fix spelling in tracking.c
commit e517b31
Author: Hunter-Chen <huntcool001@gmail.com>
Date: Fri Jul 17 22:33:32 2020 +0800
Update redis.conf
Co-authored-by: Itamar Haber <itamar@redislabs.com>
commit c300eff
Author: Hunter-Chen <huntcool001@gmail.com>
Date: Fri Jul 17 22:33:23 2020 +0800
Update redis.conf
Co-authored-by: Itamar Haber <itamar@redislabs.com>
commit 4c058a8
Author: 陈浩鹏 <chenhaopeng@heytea.com>
Date: Thu Jun 25 19:00:56 2020 +0800
Grammar fix and clarification
commit 5fcaa81
Author: bodong.ybd <bodong.ybd@alibaba-inc.com>
Date: Fri Jun 19 10:09:00 2020 +0800
Fix typos
commit 4caca9a
Author: Pruthvi P <pruthvi@ixigo.com>
Date: Fri May 22 00:33:22 2020 +0530
Fix typo eviciton => eviction
commit b2a25f6
Author: Brad Dunbar <dunbarb2@gmail.com>
Date: Sun May 17 12:39:59 2020 -0400
Fix a typo.
commit 12842ae
Author: hwware <wen.hui.ware@gmail.com>
Date: Sun May 3 17:16:59 2020 -0400
fix spelling in redis conf
commit ddba07c
Author: Chris Lamb <chris@chris-lamb.co.uk>
Date: Sat May 2 23:25:34 2020 +0100
Correct a "conflicts" spelling error.
commit 8fc7bf2
Author: Nao YONASHIRO <yonashiro@r.recruit.co.jp>
Date: Thu Apr 30 10:25:27 2020 +0900
docs: fix EXPIRE_FAST_CYCLE_DURATION to ACTIVE_EXPIRE_CYCLE_FAST_DURATION
commit 9b2b67a
Author: Brad Dunbar <dunbarb2@gmail.com>
Date: Fri Apr 24 11:46:22 2020 -0400
Fix a typo.
commit 0746f10
Author: devilinrust <63737265+devilinrust@users.noreply.github.com>
Date: Thu Apr 16 00:17:53 2020 +0200
Fix typos in server.c
commit 92b588d
Author: benjessop12 <56115861+benjessop12@users.noreply.github.com>
Date: Mon Apr 13 13:43:55 2020 +0100
Fix spelling mistake in lazyfree.c
commit 1da37aa
Merge: 2d4ba28 af347a8
Author: hwware <wen.hui.ware@gmail.com>
Date: Thu Mar 5 22:41:31 2020 -0500
Merge remote-tracking branch 'upstream/unstable' into expiretypofix
commit 2d4ba28
Author: hwware <wen.hui.ware@gmail.com>
Date: Mon Mar 2 00:09:40 2020 -0500
fix typo in expire.c
commit 1a746f7
Author: SennoYuki <minakami1yuki@gmail.com>
Date: Thu Feb 27 16:54:32 2020 +0800
fix typo
commit 8599b1a
Author: dongheejeong <donghee950403@gmail.com>
Date: Sun Feb 16 20:31:43 2020 +0000
Fix typo in server.c
commit f38d4e8
Author: hwware <wen.hui.ware@gmail.com>
Date: Sun Feb 2 22:58:38 2020 -0500
fix typo in evict.c
commit fe143fc
Author: Leo Murillo <leonardo.murillo@gmail.com>
Date: Sun Feb 2 01:57:22 2020 -0600
Fix a few typos in redis.conf
commit 1ab4d21
Author: viraja1 <anchan.viraj@gmail.com>
Date: Fri Dec 27 17:15:58 2019 +0530
Fix typo in Latency API docstring
commit ca1f70e
Author: gosth <danxuedexing@qq.com>
Date: Wed Dec 18 15:18:02 2019 +0800
fix typo in sort.c
commit a57c06b
Author: ZYunH <zyunhjob@163.com>
Date: Mon Dec 16 22:28:46 2019 +0800
fix-zset-typo
commit b8c92b5
Author: git-hulk <hulk.website@gmail.com>
Date: Mon Dec 16 15:51:42 2019 +0800
FIX: typo in cluster.c, onformation->information
commit 9dd981c
Author: wujm2007 <jim.wujm@gmail.com>
Date: Mon Dec 16 09:37:52 2019 +0800
Fix typo
commit e132d7a
Author: Sebastien Williams-Wynn <s.williamswynn.mail@gmail.com>
Date: Fri Nov 15 00:14:07 2019 +0000
Minor typo change
commit 47f44d5
Author: happynote3966 <01ssrmikururudevice01@gmail.com>
Date: Mon Nov 11 22:08:48 2019 +0900
fix comment typo in redis-cli.c
commit b8bdb0d
Author: fulei <fulei@kuaishou.com>
Date: Wed Oct 16 18:00:17 2019 +0800
Fix a spelling mistake of comments in defragDictBucketCallback
commit 0def46a
Author: fulei <fulei@kuaishou.com>
Date: Wed Oct 16 13:09:27 2019 +0800
fix some spelling mistakes of comments in defrag.c
commit f3596fd
Author: Phil Rajchgot <tophil@outlook.com>
Date: Sun Oct 13 02:02:32 2019 -0400
Typo and grammar fixes
Redis and its documentation are great -- just wanted to submit a few corrections in the spirit of Hacktoberfest. Thanks for all your work on this project. I use it all the time and it works beautifully.
commit 2b928cd
Author: KangZhiDong <worldkzd@gmail.com>
Date: Sun Sep 1 07:03:11 2019 +0800
fix typos
commit 33aea14
Author: Axlgrep <axlgrep@gmail.com>
Date: Tue Aug 27 11:02:18 2019 +0800
Fixed eviction spelling issues
commit e282a80
Author: Simen Flatby <simen@oms.no>
Date: Tue Aug 20 15:25:51 2019 +0200
Update comments to reflect prop name
In the comments the prop is referenced as replica-validity-factor,
but it is really named cluster-replica-validity-factor.
commit 74d1f9a
Author: Jim Green <jimgreen2013@qq.com>
Date: Tue Aug 20 20:00:31 2019 +0800
fix comment error, the code is ok
commit eea1407
Author: Liao Tonglang <liaotonglang@gmail.com>
Date: Fri May 31 10:16:18 2019 +0800
typo fix
fix cna't to can't
commit 0da553c
Author: KAWACHI Takashi <tkawachi@gmail.com>
Date: Wed Jul 17 00:38:16 2019 +0900
Fix typo
commit 7fc8fb6
Author: Michael Prokop <mika@grml.org>
Date: Tue May 28 17:58:42 2019 +0200
Typo fixes
s/familar/familiar/
s/compatiblity/compatibility/
s/ ot / to /
s/itsef/itself/
commit 5f46c9d
Author: zhumoing <34539422+zhumoing@users.noreply.github.com>
Date: Tue May 21 21:16:50 2019 +0800
typo-fixes
typo-fixes
commit 321dfe1
Author: wxisme <850885154@qq.com>
Date: Sat Mar 16 15:10:55 2019 +0800
typo fix
commit b4fb131
Merge: 267e0e6 3df1eb8
Author: Nikitas Bastas <nikitasbst@gmail.com>
Date: Fri Feb 8 22:55:45 2019 +0200
Merge branch 'unstable' of antirez/redis into unstable
commit 267e0e6
Author: Nikitas Bastas <nikitasbst@gmail.com>
Date: Wed Jan 30 21:26:04 2019 +0200
Minor typo fix
commit 30544e7
Author: inshal96 <39904558+inshal96@users.noreply.github.com>
Date: Fri Jan 4 16:54:50 2019 +0500
remove an extra 'a' in the comments
commit 337969d
Author: BrotherGao <yangdongheng11@gmail.com>
Date: Sat Dec 29 12:37:29 2018 +0800
fix typo in redis.conf
commit 9f4b121
Merge: 423a030 e504583
Author: BrotherGao <yangdongheng@xiaomi.com>
Date: Sat Dec 29 11:41:12 2018 +0800
Merge branch 'unstable' of antirez/redis into unstable
commit 423a030
Merge: 42b02b7 46a51cd
Author: 杨东衡 <yangdongheng@xiaomi.com>
Date: Tue Dec 4 23:56:11 2018 +0800
Merge branch 'unstable' of antirez/redis into unstable
commit 42b02b7
Merge: 68c0e6e b8febe6
Author: Dongheng Yang <yangdongheng11@gmail.com>
Date: Sun Oct 28 15:54:23 2018 +0800
Merge pull request #1 from antirez/unstable
update local data
commit 714b589
Author: Christian <crifei93@gmail.com>
Date: Fri Dec 28 01:17:26 2018 +0100
fix typo "resulution"
commit e23259d
Author: garenchan <1412950785@qq.com>
Date: Wed Dec 26 09:58:35 2018 +0800
fix typo: segfauls -> segfault
commit a9359f8
Author: xjp <jianping_xie@aliyun.com>
Date: Tue Dec 18 17:31:44 2018 +0800
Fixed REDISMODULE_H spell bug
commit a12c3e4
Author: jdiaz <jrd.palacios@gmail.com>
Date: Sat Dec 15 23:39:52 2018 -0600
Fixes hyperloglog hash function comment block description
commit 770eb11
Author: 林上耀 <1210tom@163.com>
Date: Sun Nov 25 17:16:10 2018 +0800
fix typo
commit fd97fbb
Author: Chris Lamb <chris@chris-lamb.co.uk>
Date: Fri Nov 23 17:14:01 2018 +0100
Correct "unsupported" typo.
commit a85522d
Author: Jungnam Lee <jungnam.lee@oracle.com>
Date: Thu Nov 8 23:01:29 2018 +0900
fix typo in test comments
commit ade8007
Author: Arun Kumar <palerdot@users.noreply.github.com>
Date: Tue Oct 23 16:56:35 2018 +0530
Fixed grammatical typo
Fixed typo for word 'dictionary'
commit 869ee39
Author: Hamid Alaei <hamid.a85@gmail.com>
Date: Sun Aug 12 16:40:02 2018 +0430
fix documentations: (ThreadSafeContextStart/Stop -> ThreadSafeContextLock/Unlock), minor typo
commit f89d158
Author: Mayank Jain <mayankjain255@gmail.com>
Date: Tue Jul 31 23:01:21 2018 +0530
Updated README.md with some spelling corrections.
Made correction in spelling of some misspelled words.
commit 892198e
Author: dsomeshwar <someshwar.dhayalan@gmail.com>
Date: Sat Jul 21 23:23:04 2018 +0530
typo fix
commit 8a4d780
Author: Itamar Haber <itamar@redislabs.com>
Date: Mon Apr 30 02:06:52 2018 +0300
Fixes some typos
commit e3acef6
Author: Noah Rosamilia <ivoahivoah@gmail.com>
Date: Sat Mar 3 23:41:21 2018 -0500
Fix typo in /deps/README.md
commit 04442fb
Author: WuYunlong <xzsyeb@126.com>
Date: Sat Mar 3 10:32:42 2018 +0800
Fix typo in readSyncBulkPayload() comment.
commit 9f36880
Author: WuYunlong <xzsyeb@126.com>
Date: Sat Mar 3 10:20:37 2018 +0800
replication.c comment: run_id -> replid.
commit f866b4a
Author: Francesco 'makevoid' Canessa <makevoid@gmail.com>
Date: Thu Feb 22 22:01:56 2018 +0000
fix comment typo in server.c
commit 0ebc69b
Author: 줍 <jubee0124@gmail.com>
Date: Mon Feb 12 16:38:48 2018 +0900
Fix typo in redis.conf
Fix `five behaviors` to `eight behaviors` in [this sentence ](antirez/redis@unstable/redis.conf#L564)
commit b50a620
Author: martinbroadhurst <martinbroadhurst@users.noreply.github.com>
Date: Thu Dec 28 12:07:30 2017 +0000
Fix typo in valgrind.sup
commit 7d8f349
Author: Peter Boughton <peter@sorcerersisle.com>
Date: Mon Nov 27 19:52:19 2017 +0000
Update CONTRIBUTING; refer doc updates to redis-doc repo.
commit 02dec7e
Author: Klauswk <klauswk1@hotmail.com>
Date: Tue Oct 24 16:18:38 2017 -0200
Fix typo in comment
commit e1efbc8
Author: chenshi <baiwfg2@gmail.com>
Date: Tue Oct 3 18:26:30 2017 +0800
Correct two spelling errors of comments
commit 93327d8
Author: spacewander <spacewanderlzx@gmail.com>
Date: Wed Sep 13 16:47:24 2017 +0800
Update the comment for OBJ_ENCODING_EMBSTR_SIZE_LIMIT's value
The value of OBJ_ENCODING_EMBSTR_SIZE_LIMIT is 44 now instead of 39.
commit 63d361f
Author: spacewander <spacewanderlzx@gmail.com>
Date: Tue Sep 12 15:06:42 2017 +0800
Fix <prevlen> related doc in ziplist.c
According to the definition of ZIP_BIG_PREVLEN and other related code,
the guard of single byte <prevlen> should be 254 instead of 255.
commit ebe228d
Author: hanael80 <hanael80@gmail.com>
Date: Tue Aug 15 09:09:40 2017 +0900
Fix typo
commit 6b696e6
Author: Matt Robenolt <matt@ydekproductions.com>
Date: Mon Aug 14 14:50:47 2017 -0700
Fix typo in LATENCY DOCTOR output
commit a2ec6ae
Author: caosiyang <caosiyang@qiyi.com>
Date: Tue Aug 15 14:15:16 2017 +0800
Fix a typo: form => from
commit 3ab7699
Author: caosiyang <caosiyang@qiyi.com>
Date: Thu Aug 10 18:40:33 2017 +0800
Fix a typo: replicationFeedSlavesFromMaster() => replicationFeedSlavesFromMasterStream()
commit 72d43ef
Author: caosiyang <caosiyang@qiyi.com>
Date: Tue Aug 8 15:57:25 2017 +0800
fix a typo: servewr => server
commit 707c958
Author: Bo Cai <charpty@gmail.com>
Date: Wed Jul 26 21:49:42 2017 +0800
redis-cli.c typo: conut -> count.
Signed-off-by: Bo Cai <charpty@gmail.com>
commit b9385b2
Author: JackDrogon <jack.xsuperman@gmail.com>
Date: Fri Jun 30 14:22:31 2017 +0800
Fix some spell problems
commit 20d9230
Author: akosel <aaronjkosel@gmail.com>
Date: Sun Jun 4 19:35:13 2017 -0500
Fix typo
commit b167bfc
Author: Krzysiek Witkowicz <krzysiekwitkowicz@gmail.com>
Date: Mon May 22 21:32:27 2017 +0100
Fix #4008 small typo in comment
commit 2b78ac8
Author: Jake Clarkson <jacobwclarkson@gmail.com>
Date: Wed Apr 26 15:49:50 2017 +0100
Correct typo in tests/unit/hyperloglog.tcl
commit b0f1cdb
Author: Qi Luo <qiluo-msft@users.noreply.github.com>
Date: Wed Apr 19 14:25:18 2017 -0700
Fix typo
commit a90b0f9
Author: charsyam <charsyam@naver.com>
Date: Thu Mar 16 18:19:53 2017 +0900
fix typos
fix typos
fix typos
commit 8430a79
Author: Richard Hart <richardhart92@gmail.com>
Date: Mon Mar 13 22:17:41 2017 -0400
Fixed log message typo in listenToPort.
commit 481a1c2
Author: Vinod Kumar <kumar003vinod@gmail.com>
Date: Sun Jan 15 23:04:51 2017 +0530
src/db.c: Correct "save" -> "safe" typo
commit 586b4d3
Author: wangshaonan <wshn13@gmail.com>
Date: Wed Dec 21 20:28:27 2016 +0800
Fix typo they->the in helloworld.c
commit c1c4b5e
Author: Jenner <hypxm@qq.com>
Date: Mon Dec 19 16:39:46 2016 +0800
typo error
commit 1ee1a3f
Author: tielei <43289893@qq.com>
Date: Mon Jul 18 13:52:25 2016 +0800
fix some comments
commit 11a41fb
Author: Otto Kekäläinen <otto@seravo.fi>
Date: Sun Jul 3 10:23:55 2016 +0100
Fix spelling in documentation and comments
commit 5fb5d82
Author: francischan <f1ancis621@gmail.com>
Date: Tue Jun 28 00:19:33 2016 +0800
Fix outdated comments about redis.c file.
It should now refer to server.c file.
commit 6b254bc
Author: lmatt-bit <lmatt123n@gmail.com>
Date: Thu Apr 21 21:45:58 2016 +0800
Refine the comment of dictRehashMilliseconds func
SLAVECONF->REPLCONF in comment - by andyli029
commit ee9869f
Author: clark.kang <charsyam@naver.com>
Date: Tue Mar 22 11:09:51 2016 +0900
fix typos
commit f7b3b11
Author: Harisankar H <harisankarh@gmail.com>
Date: Wed Mar 9 11:49:42 2016 +0530
Typo correction: "faield" --> "failed"
Typo correction: "faield" --> "failed"
commit 3fd40fc
Author: Itamar Haber <itamar@redislabs.com>
Date: Thu Feb 25 10:31:51 2016 +0200
Fixes a typo in comments
commit 621c160
Author: Prayag Verma <prayag.verma@gmail.com>
Date: Mon Feb 1 12:36:20 2016 +0530
Fix typo in Readme.md
Spelling mistakes -
`eviciton` > `eviction`
`familar` > `familiar`
commit d7d07d6
Author: WonCheol Lee <toctoc21c@gmail.com>
Date: Wed Dec 30 15:11:34 2015 +0900
Typo fixed
commit a4dade7
Author: Felix Bünemann <buenemann@louis.info>
Date: Mon Dec 28 11:02:55 2015 +0100
[ci skip] Improve supervised upstart config docs
This mentions that "expect stop" is required for supervised upstart
to work correctly. See http://upstart.ubuntu.com/cookbook/#expect-stop
for an explanation.
commit d9caba9
Author: daurnimator <quae@daurnimator.com>
Date: Mon Dec 21 18:30:03 2015 +1100
README: Remove trailing whitespace
commit 72d42e5
Author: daurnimator <quae@daurnimator.com>
Date: Mon Dec 21 18:29:32 2015 +1100
README: Fix typo. th => the
commit dd6e957
Author: daurnimator <quae@daurnimator.com>
Date: Mon Dec 21 18:29:20 2015 +1100
README: Fix typo. familar => familiar
commit 3a12b23
Author: daurnimator <quae@daurnimator.com>
Date: Mon Dec 21 18:28:54 2015 +1100
README: Fix typo. eviciton => eviction
commit 2d1d03b
Author: daurnimator <quae@daurnimator.com>
Date: Mon Dec 21 18:21:45 2015 +1100
README: Fix typo. sever => server
commit 3973b06
Author: Itamar Haber <itamar@garantiadata.com>
Date: Sat Dec 19 17:01:20 2015 +0200
Typo fix
commit 4f2e460
Author: Steve Gao <fu@2token.com>
Date: Fri Dec 4 10:22:05 2015 +0800
Update README - fix typos
commit b21667c
Author: binyan <binbin.yan@nokia.com>
Date: Wed Dec 2 22:48:37 2015 +0800
delete redundancy color judge in sdscatcolor
commit 88894c7
Author: binyan <binbin.yan@nokia.com>
Date: Wed Dec 2 22:14:42 2015 +0800
the example output shoule be HelloWorld
commit 2763470
Author: binyan <binbin.yan@nokia.com>
Date: Wed Dec 2 17:41:39 2015 +0800
modify error word keyevente
Signed-off-by: binyan <binbin.yan@nokia.com>
commit 0847b3d
Author: Bruno Martins <bscmartins@gmail.com>
Date: Wed Nov 4 11:37:01 2015 +0000
typo
commit bbb9e9e
Author: dawedawe <dawedawe@gmx.de>
Date: Fri Mar 27 00:46:41 2015 +0100
typo: zimap -> zipmap
commit 5ed297e
Author: Axel Advento <badwolf.bloodseeker.rev@gmail.com>
Date: Tue Mar 3 15:58:29 2015 +0800
Fix 'salve' typos to 'slave'
commit edec9d6
Author: LudwikJaniuk <ludvig.janiuk@gmail.com>
Date: Wed Jun 12 14:12:47 2019 +0200
Update README.md
Co-Authored-By: Qix <Qix-@users.noreply.github.com>
commit 692a7af
Author: LudwikJaniuk <ludvig.janiuk@gmail.com>
Date: Tue May 28 14:32:04 2019 +0200
grammar
commit d962b0a
Author: Nick Frost <nickfrostatx@gmail.com>
Date: Wed Jul 20 15:17:12 2016 -0700
Minor grammar fix
commit 24fff01aaccaf5956973ada8c50ceb1462e211c6 (typos)
Author: Chad Miller <chadm@squareup.com>
Date: Tue Sep 8 13:46:11 2020 -0400
Fix faulty comment about operation of unlink()
commit 3cd5c1f3326c52aa552ada7ec797c6bb16452355
Author: Kevin <kevin.xgr@gmail.com>
Date: Wed Nov 20 00:13:50 2019 +0800
Fix typo in server.c.
From a83af59 Mon Sep 17 00:00:00 2001
From: wuwo <wuwo@wacai.com>
Date: Fri, 17 Mar 2017 20:37:45 +0800
Subject: [PATCH] falure to failure
From c961896 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E5=B7=A6=E6=87=B6?= <veficos@gmail.com>
Date: Sat, 27 May 2017 15:33:04 +0800
Subject: [PATCH] fix typo
From e600ef2 Mon Sep 17 00:00:00 2001
From: "rui.zou" <rui.zou@yunify.com>
Date: Sat, 30 Sep 2017 12:38:15 +0800
Subject: [PATCH] fix a typo
From c7d07fa Mon Sep 17 00:00:00 2001
From: Alexandre Perrin <alex@kaworu.ch>
Date: Thu, 16 Aug 2018 10:35:31 +0200
Subject: [PATCH] deps README.md typo
From b25cb67 Mon Sep 17 00:00:00 2001
From: Guy Korland <gkorland@gmail.com>
Date: Wed, 26 Sep 2018 10:55:37 +0300
Subject: [PATCH 1/2] fix typos in header
From ad28ca6 Mon Sep 17 00:00:00 2001
From: Guy Korland <gkorland@gmail.com>
Date: Wed, 26 Sep 2018 11:02:36 +0300
Subject: [PATCH 2/2] fix typos
commit 34924cdedd8552466fc22c1168d49236cb7ee915
Author: Adrian Lynch <adi_ady_ade@hotmail.com>
Date: Sat Apr 4 21:59:15 2015 +0100
Typos fixed
commit fd2a1e7
Author: Jan <jsteemann@users.noreply.github.com>
Date: Sat Oct 27 19:13:01 2018 +0200
Fix typos
Fix typos
commit e14e47c1a234b53b0e103c5f6a1c61481cbcbb02
Author: Andy Lester <andy@petdance.com>
Date: Fri Aug 2 22:30:07 2019 -0500
Fix multiple misspellings of "following"
commit 79b948ce2dac6b453fe80995abbcaac04c213d5a
Author: Andy Lester <andy@petdance.com>
Date: Fri Aug 2 22:24:28 2019 -0500
Fix misspelling of create-cluster
commit 1fffde52666dc99ab35efbd31071a4c008cb5a71
Author: Andy Lester <andy@petdance.com>
Date: Wed Jul 31 17:57:56 2019 -0500
Fix typos
commit 204c9ba9651e9e05fd73936b452b9a30be456cfe
Author: Xiaobo Zhu <xiaobo.zhu@shopee.com>
Date: Tue Aug 13 22:19:25 2019 +0800
fix typos
Squashed commit of the following:
commit 1d9aaf8
Author: danmedani <danmedani@gmail.com>
Date: Sun Aug 2 11:40:26 2015 -0700
README typo fix.
Squashed commit of the following:
commit 32bfa7c
Author: Erik Dubbelboer <erik@dubbelboer.com>
Date: Mon Jul 6 21:15:08 2015 +0200
Fixed grammer
Squashed commit of the following:
commit b24f69c
Author: Sisir Koppaka <sisir.koppaka@gmail.com>
Date: Mon Mar 2 22:38:45 2015 -0500
utils/hashtable/rehashing.c: Fix typos
Squashed commit of the following:
commit 4e04082
Author: Erik Dubbelboer <erik@dubbelboer.com>
Date: Mon Mar 23 08:22:21 2015 +0000
Small config file documentation improvements
Squashed commit of the following:
commit acb8773
Author: ctd1500 <ctd1500@gmail.com>
Date: Fri May 8 01:52:48 2015 -0700
Typo and grammar fixes in readme
commit 2eb75b6
Author: ctd1500 <ctd1500@gmail.com>
Date: Fri May 8 01:36:18 2015 -0700
fixed redis.conf comment
Squashed commit of the following:
commit a8249a2
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Date: Fri Dec 11 11:39:52 2015 +0530
Revise correction of typos.
Squashed commit of the following:
commit 3c02028
Author: zhaojun11 <zhaojun11@jd.com>
Date: Wed Jan 17 19:05:28 2018 +0800
Fix typos include two code typos in cluster.c and latency.c
Squashed commit of the following:
commit 9dba47c
Author: q191201771 <191201771@qq.com>
Date: Sat Jan 4 11:31:04 2020 +0800
fix function listCreate comment in adlist.c
Update src/server.c
commit 2c7c2cb536e78dd211b1ac6f7bda00f0f54faaeb
Author: charpty <charpty@gmail.com>
Date: Tue May 1 23:16:59 2018 +0800
server.c typo: modules system dictionary type comment
Signed-off-by: charpty <charpty@gmail.com>
commit a8395323fb63cb59cb3591cb0f0c8edb7c29a680
Author: Itamar Haber <itamar@redislabs.com>
Date: Sun May 6 00:25:18 2018 +0300
Updates test_helper.tcl's help with undocumented options
Specifically:
* Host
* Port
* Client
commit bde6f9ced15755cd6407b4af7d601b030f36d60b
Author: wxisme <850885154@qq.com>
Date: Wed Aug 8 15:19:19 2018 +0800
fix comments in deps files
commit 3172474ba991532ab799ee1873439f3402412331
Author: wxisme <850885154@qq.com>
Date: Wed Aug 8 14:33:49 2018 +0800
fix some comments
commit 01b6f2b6858b5cf2ce4ad5092d2c746e755f53f0
Author: Thor Juhasz <thor@juhasz.pro>
Date: Sun Nov 18 14:37:41 2018 +0100
Minor fixes to comments
Found some parts a little unclear on a first read, which prompted me to have a better look at the file and fix some minor things I noticed.
Fixing minor typos and grammar. There are no changes to configuration options.
These changes are only meant to help the user better understand the explanations to the various configuration options
2020-09-10 06:43:38 -04:00
/* Hz is more a hint from the user, so we accept values out of range
2019-11-28 04:24:57 -05:00
* but cap them to reasonable values . */
if ( server . config_hz < CONFIG_MIN_HZ ) server . config_hz = CONFIG_MIN_HZ ;
if ( server . config_hz > CONFIG_MAX_HZ ) server . config_hz = CONFIG_MAX_HZ ;
server . hz = server . config_hz ;
return 1 ;
}
2021-12-01 03:15:11 -05:00
static int updatePort ( const char * * err ) {
if ( changeListenPort ( server . port , & server . ipfd , acceptTcpHandler ) = = C_ERR ) {
2021-03-01 09:04:44 -05:00
* err = " Unable to listen on this port. Check server logs. " ;
return 0 ;
}
return 1 ;
}
2021-12-01 03:15:11 -05:00
static int updateJemallocBgThread ( const char * * err ) {
2019-11-28 04:11:07 -05:00
UNUSED ( err ) ;
2021-12-01 03:15:11 -05:00
set_jemalloc_bg_thread ( server . jemalloc_bg_thread ) ;
2019-11-28 04:11:07 -05:00
return 1 ;
}
2019-11-26 09:52:28 -05:00
2021-12-01 03:15:11 -05:00
static int updateReplBacklogSize ( const char * * err ) {
2019-11-28 04:24:57 -05:00
UNUSED ( err ) ;
2021-12-01 03:15:11 -05:00
resizeReplicationBacklog ( ) ;
2019-11-28 04:24:57 -05:00
return 1 ;
}
2021-12-01 03:15:11 -05:00
static int updateMaxmemory ( const char * * err ) {
2019-11-28 04:24:57 -05:00
UNUSED ( err ) ;
2021-12-01 03:15:11 -05:00
if ( server . maxmemory ) {
2020-02-06 02:23:22 -05:00
size_t used = zmalloc_used_memory ( ) - freeMemoryGetNotCountedMemory ( ) ;
2021-12-01 03:15:11 -05:00
if ( server . maxmemory < used ) {
2020-02-06 02:23:22 -05:00
serverLog ( LL_WARNING , " WARNING: the new maxmemory value set via CONFIG SET (%llu) is smaller than the current memory usage (%zu). This will result in key eviction and/or the inability to accept new write commands depending on the maxmemory-policy. " , server . maxmemory , used ) ;
2019-11-28 04:24:57 -05:00
}
2020-09-16 02:16:01 -04:00
performEvictions ( ) ;
2019-11-28 04:24:57 -05:00
}
return 1 ;
}
2021-12-01 03:15:11 -05:00
static int updateGoodSlaves ( const char * * err ) {
2019-11-28 04:24:57 -05:00
UNUSED ( err ) ;
refreshGoodSlavesCount ( ) ;
return 1 ;
}
2021-12-01 03:15:11 -05:00
static int updateWatchdogPeriod ( const char * * err ) {
2021-11-07 06:40:08 -05:00
UNUSED ( err ) ;
applyWatchdogPeriod ( ) ;
return 1 ;
}
2021-12-01 03:15:11 -05:00
static int updateAppendonly ( const char * * err ) {
if ( ! server . aof_enabled & & server . aof_state ! = AOF_OFF ) {
2019-11-28 04:24:57 -05:00
stopAppendOnly ( ) ;
2021-12-01 03:15:11 -05:00
} else if ( server . aof_enabled & & server . aof_state = = AOF_OFF ) {
2019-11-28 04:24:57 -05:00
if ( startAppendOnly ( ) = = C_ERR ) {
* err = " Unable to turn on AOF. Check server logs. " ;
return 0 ;
}
}
return 1 ;
}
2021-12-01 03:15:11 -05:00
static int updateSighandlerEnabled ( const char * * err ) {
2020-07-29 10:05:14 -04:00
UNUSED ( err ) ;
2021-12-01 03:15:11 -05:00
if ( server . crashlog_enabled )
2020-07-29 10:05:14 -04:00
setupSignalHandlers ( ) ;
else
removeSignalHandlers ( ) ;
return 1 ;
}
2021-12-01 03:15:11 -05:00
static int updateMaxclients ( const char * * err ) {
unsigned int new_maxclients = server . maxclients ;
adjustOpenFilesLimit ( ) ;
if ( server . maxclients ! = new_maxclients ) {
static char msg [ 128 ] ;
sprintf ( msg , " The operating system is not able to handle the specified number of clients, try with %d " , server . maxclients ) ;
* err = msg ;
return 0 ;
}
if ( ( unsigned int ) aeGetSetSize ( server . el ) <
server . maxclients + CONFIG_FDSET_INCR )
{
if ( aeResizeSetSize ( server . el ,
server . maxclients + CONFIG_FDSET_INCR ) = = AE_ERR )
2019-11-28 04:24:57 -05:00
{
2021-12-01 03:15:11 -05:00
* err = " The event loop API used by Redis is not able to handle the specified number of clients " ;
return 0 ;
2019-11-28 04:24:57 -05:00
}
}
return 1 ;
}
2021-12-01 03:15:11 -05:00
static int updateOOMScoreAdj ( const char * * err ) {
if ( setOOMScoreAdj ( - 1 ) = = C_ERR ) {
* err = " Failed to set current oom_score_adj. Check server logs. " ;
return 0 ;
2020-08-12 10:58:56 -04:00
}
return 1 ;
}
2021-12-01 03:15:11 -05:00
int updateRequirePass ( const char * * err ) {
2021-02-26 00:00:27 -05:00
UNUSED ( err ) ;
/* The old "requirepass" directive just translates to setting
* a password to the default user . The only thing we do
* additionally is to remember the cleartext password in this
* case , for backward compatibility with Redis < = 5. */
2021-12-01 03:15:11 -05:00
ACLUpdateDefaultUserPassword ( server . requirepass ) ;
2021-02-26 00:00:27 -05:00
return 1 ;
}
2021-12-01 03:15:11 -05:00
static int applyBind ( const char * * err ) {
if ( changeBindAddr ( ) = = C_ERR ) {
* err = " Failed to bind to specified addresses. " ;
return 0 ;
}
2021-11-07 21:56:03 -05:00
2021-12-01 03:15:11 -05:00
return 1 ;
}
int updateClusterFlags ( const char * * err ) {
2021-11-07 21:56:03 -05:00
UNUSED ( err ) ;
clusterUpdateMyselfFlags ( ) ;
return 1 ;
}
2021-12-01 03:15:11 -05:00
static int updateClusterIp ( const char * * err ) {
2021-11-07 21:56:03 -05:00
UNUSED ( err ) ;
clusterUpdateMyselfIp ( ) ;
return 1 ;
}
2019-11-28 04:24:57 -05:00
# ifdef USE_OPENSSL
2021-12-01 03:15:11 -05:00
static int applyTlsCfg ( const char * * err ) {
2019-11-28 04:24:57 -05:00
UNUSED ( err ) ;
2020-08-17 10:36:50 -04:00
/* If TLS is enabled, try to configure OpenSSL. */
if ( ( server . tls_port | | server . tls_replication | | server . tls_cluster )
& & tlsConfigure ( & server . tls_ctx_config ) = = C_ERR ) {
2020-07-10 04:33:47 -04:00
* err = " Unable to update TLS configuration. Check server logs. " ;
2019-11-28 04:24:57 -05:00
return 0 ;
}
return 1 ;
}
2021-03-01 09:04:44 -05:00
2021-12-01 03:15:11 -05:00
static int applyTLSPort ( const char * * err ) {
/* Configure TLS in case it wasn't enabled */
if ( ! isTlsConfigured ( ) & & tlsConfigure ( & server . tls_ctx_config ) = = C_ERR ) {
2021-03-01 09:04:44 -05:00
* err = " Unable to update TLS configuration. Check server logs. " ;
return 0 ;
}
2021-12-01 03:15:11 -05:00
if ( changeListenPort ( server . tls_port , & server . tlsfd , acceptTLSHandler ) = = C_ERR ) {
2021-03-01 09:04:44 -05:00
* err = " Unable to listen on this port. Check server logs. " ;
return 0 ;
}
return 1 ;
}
2019-11-28 04:24:57 -05:00
# endif /* USE_OPENSSL */
2021-12-01 03:15:11 -05:00
static int setConfigDirOption ( typeData data , sds * argv , int argc , const char * * err ) {
2021-11-07 06:40:08 -05:00
UNUSED ( data ) ;
if ( argc ! = 1 ) {
* err = " wrong number of arguments " ;
return 0 ;
}
if ( chdir ( argv [ 0 ] ) = = - 1 ) {
* err = strerror ( errno ) ;
return 0 ;
}
return 1 ;
}
2021-12-01 03:15:11 -05:00
static sds getConfigDirOption ( typeData data ) {
2021-11-07 06:40:08 -05:00
UNUSED ( data ) ;
char buf [ 1024 ] ;
if ( getcwd ( buf , sizeof ( buf ) ) = = NULL )
buf [ 0 ] = ' \0 ' ;
2021-12-01 03:15:11 -05:00
return sdsnew ( buf ) ;
2021-11-07 06:40:08 -05:00
}
2021-12-01 03:15:11 -05:00
static int setConfigSaveOption ( typeData data , sds * argv , int argc , const char * * err ) {
2021-11-07 06:40:08 -05:00
UNUSED ( data ) ;
int j ;
/* Special case: treat single arg "" as zero args indicating empty save configuration */
if ( argc = = 1 & & ! strcasecmp ( argv [ 0 ] , " " ) )
argc = 0 ;
/* Perform sanity check before setting the new config:
* - Even number of args
* - Seconds > = 1 , changes > = 0 */
if ( argc & 1 ) {
* err = " Invalid save parameters " ;
return 0 ;
}
for ( j = 0 ; j < argc ; j + + ) {
char * eptr ;
long val ;
val = strtoll ( argv [ j ] , & eptr , 10 ) ;
if ( eptr [ 0 ] ! = ' \0 ' | |
( ( j & 1 ) = = 0 & & val < 1 ) | |
( ( j & 1 ) = = 1 & & val < 0 ) ) {
* err = " Invalid save parameters " ;
return 0 ;
}
}
/* Finally set the new config */
2021-12-01 03:15:11 -05:00
if ( ! reading_config_file ) {
2021-11-07 06:40:08 -05:00
resetServerSaveParams ( ) ;
} else {
/* We don't reset save params before loading, because if they're not part
* of the file the defaults should be used .
*/
static int save_loaded = 0 ;
if ( ! save_loaded ) {
save_loaded = 1 ;
resetServerSaveParams ( ) ;
}
}
for ( j = 0 ; j < argc ; j + = 2 ) {
time_t seconds ;
int changes ;
seconds = strtoll ( argv [ j ] , NULL , 10 ) ;
changes = strtoll ( argv [ j + 1 ] , NULL , 10 ) ;
appendServerSaveParams ( seconds , changes ) ;
}
return 1 ;
}
2021-12-01 03:15:11 -05:00
static sds getConfigSaveOption ( typeData data ) {
2021-11-07 06:40:08 -05:00
UNUSED ( data ) ;
sds buf = sdsempty ( ) ;
int j ;
for ( j = 0 ; j < server . saveparamslen ; j + + ) {
buf = sdscatprintf ( buf , " %jd %d " ,
( intmax_t ) server . saveparams [ j ] . seconds ,
server . saveparams [ j ] . changes ) ;
if ( j ! = server . saveparamslen - 1 )
buf = sdscatlen ( buf , " " , 1 ) ;
}
2021-12-01 03:15:11 -05:00
return buf ;
2021-11-07 06:40:08 -05:00
}
2021-12-01 03:15:11 -05:00
static int setConfigClientOutputBufferLimitOption ( typeData data , sds * argv , int argc , const char * * err ) {
2021-11-07 06:40:08 -05:00
UNUSED ( data ) ;
return updateClientOutputBufferLimit ( argv , argc , err ) ;
}
2021-12-01 03:15:11 -05:00
static sds getConfigClientOutputBufferLimitOption ( typeData data ) {
2021-11-07 06:40:08 -05:00
UNUSED ( data ) ;
sds buf = sdsempty ( ) ;
int j ;
for ( j = 0 ; j < CLIENT_TYPE_OBUF_COUNT ; j + + ) {
buf = sdscatprintf ( buf , " %s %llu %llu %ld " ,
getClientTypeName ( j ) ,
server . client_obuf_limits [ j ] . hard_limit_bytes ,
server . client_obuf_limits [ j ] . soft_limit_bytes ,
( long ) server . client_obuf_limits [ j ] . soft_limit_seconds ) ;
if ( j ! = CLIENT_TYPE_OBUF_COUNT - 1 )
buf = sdscatlen ( buf , " " , 1 ) ;
}
2021-12-01 03:15:11 -05:00
return buf ;
2021-11-07 06:40:08 -05:00
}
2021-12-01 03:15:11 -05:00
/* Parse an array of CONFIG_OOM_COUNT sds strings, validate and populate
* server . oom_score_adj_values if valid .
*/
static int setConfigOOMScoreAdjValuesOption ( typeData data , sds * argv , int argc , const char * * err ) {
int i ;
int values [ CONFIG_OOM_COUNT ] ;
2021-12-07 09:05:51 -05:00
int change = 0 ;
2021-11-07 06:40:08 -05:00
UNUSED ( data ) ;
2021-12-01 03:15:11 -05:00
2021-11-07 06:40:08 -05:00
if ( argc ! = CONFIG_OOM_COUNT ) {
* err = " wrong number of arguments " ;
return 0 ;
}
2021-12-01 03:15:11 -05:00
for ( i = 0 ; i < CONFIG_OOM_COUNT ; i + + ) {
char * eptr ;
long long val = strtoll ( argv [ i ] , & eptr , 10 ) ;
if ( * eptr ! = ' \0 ' | | val < - 2000 | | val > 2000 ) {
if ( err ) * err = " Invalid oom-score-adj-values, elements must be between -2000 and 2000. " ;
return - 1 ;
}
values [ i ] = val ;
}
/* Verify that the values make sense. If they don't omit a warning but
* keep the configuration , which may still be valid for privileged processes .
*/
if ( values [ CONFIG_OOM_REPLICA ] < values [ CONFIG_OOM_MASTER ] | |
values [ CONFIG_OOM_BGCHILD ] < values [ CONFIG_OOM_REPLICA ] )
{
serverLog ( LL_WARNING ,
" The oom-score-adj-values configuration may not work for non-privileged processes! "
" Please consult the documentation. " ) ;
}
for ( i = 0 ; i < CONFIG_OOM_COUNT ; i + + ) {
2021-12-07 09:05:51 -05:00
if ( server . oom_score_adj_values [ i ] ! = values [ i ] ) {
server . oom_score_adj_values [ i ] = values [ i ] ;
change = 1 ;
}
2021-12-01 03:15:11 -05:00
}
2021-12-07 09:05:51 -05:00
return change ? 1 : 2 ;
2021-11-07 06:40:08 -05:00
}
2021-12-01 03:15:11 -05:00
static sds getConfigOOMScoreAdjValuesOption ( typeData data ) {
2021-11-07 06:40:08 -05:00
UNUSED ( data ) ;
sds buf = sdsempty ( ) ;
int j ;
for ( j = 0 ; j < CONFIG_OOM_COUNT ; j + + ) {
buf = sdscatprintf ( buf , " %d " , server . oom_score_adj_values [ j ] ) ;
if ( j ! = CONFIG_OOM_COUNT - 1 )
buf = sdscatlen ( buf , " " , 1 ) ;
}
2021-12-01 03:15:11 -05:00
return buf ;
2021-11-07 06:40:08 -05:00
}
2021-12-01 03:15:11 -05:00
static int setConfigNotifyKeyspaceEventsOption ( typeData data , sds * argv , int argc , const char * * err ) {
2021-11-07 06:40:08 -05:00
UNUSED ( data ) ;
if ( argc ! = 1 ) {
* err = " wrong number of arguments " ;
return 0 ;
}
int flags = keyspaceEventsStringToFlags ( argv [ 0 ] ) ;
if ( flags = = - 1 ) {
* err = " Invalid event class character. Use 'Ag$lshzxeKEtmd'. " ;
return 0 ;
}
server . notify_keyspace_events = flags ;
return 1 ;
}
2021-12-01 03:15:11 -05:00
static sds getConfigNotifyKeyspaceEventsOption ( typeData data ) {
2021-11-07 06:40:08 -05:00
UNUSED ( data ) ;
2021-12-01 03:15:11 -05:00
return keyspaceEventsFlagsToString ( server . notify_keyspace_events ) ;
2021-11-07 06:40:08 -05:00
}
2021-12-01 03:15:11 -05:00
static int setConfigBindOption ( typeData data , sds * argv , int argc , const char * * err ) {
2021-11-07 06:40:08 -05:00
UNUSED ( data ) ;
2021-12-01 03:15:11 -05:00
int j ;
2021-11-07 06:40:08 -05:00
if ( argc > CONFIG_BINDADDR_MAX ) {
* err = " Too many bind addresses specified. " ;
return 0 ;
}
2021-12-01 03:15:11 -05:00
/* A single empty argument is treated as a zero bindaddr count */
if ( argc = = 1 & & sdslen ( argv [ 0 ] ) = = 0 ) argc = 0 ;
2021-11-07 06:40:08 -05:00
2021-12-01 03:15:11 -05:00
/* Free old bind addresses */
for ( j = 0 ; j < server . bindaddr_count ; j + + ) {
zfree ( server . bindaddr [ j ] ) ;
2021-11-07 06:40:08 -05:00
}
2021-12-01 03:15:11 -05:00
for ( j = 0 ; j < argc ; j + + )
server . bindaddr [ j ] = zstrdup ( argv [ j ] ) ;
server . bindaddr_count = argc ;
2021-11-07 06:40:08 -05:00
return 1 ;
}
2021-12-01 03:15:11 -05:00
static int setConfigReplicaOfOption ( typeData data , sds * argv , int argc , const char * * err ) {
2021-11-07 06:40:08 -05:00
UNUSED ( data ) ;
if ( argc ! = 2 ) {
* err = " wrong number of arguments " ;
return 0 ;
}
sdsfree ( server . masterhost ) ;
server . masterhost = NULL ;
if ( ! strcasecmp ( argv [ 0 ] , " no " ) & & ! strcasecmp ( argv [ 1 ] , " one " ) ) {
return 1 ;
}
char * ptr ;
server . masterport = strtol ( argv [ 1 ] , & ptr , 10 ) ;
if ( server . masterport < 0 | | server . masterport > 65535 | | * ptr ! = ' \0 ' ) {
* err = " Invalid master port " ;
return 0 ;
}
server . masterhost = sdsnew ( argv [ 0 ] ) ;
server . repl_state = REPL_STATE_CONNECT ;
return 1 ;
}
2021-12-01 03:15:11 -05:00
static sds getConfigBindOption ( typeData data ) {
2021-11-07 06:40:08 -05:00
UNUSED ( data ) ;
2021-12-01 03:15:11 -05:00
return sdsjoin ( server . bindaddr , server . bindaddr_count , " " ) ;
2021-11-07 06:40:08 -05:00
}
2021-12-01 03:15:11 -05:00
static sds getConfigReplicaOfOption ( typeData data ) {
2021-11-07 06:40:08 -05:00
UNUSED ( data ) ;
char buf [ 256 ] ;
if ( server . masterhost )
snprintf ( buf , sizeof ( buf ) , " %s %d " ,
server . masterhost , server . masterport ) ;
else
buf [ 0 ] = ' \0 ' ;
2021-12-01 03:15:11 -05:00
return sdsnew ( buf ) ;
2021-11-07 06:40:08 -05:00
}
2019-07-19 13:15:35 -04:00
standardConfig configs [ ] = {
/* Bool configs */
2019-11-28 04:24:57 -05:00
createBoolConfig ( " rdbchecksum " , NULL , IMMUTABLE_CONFIG , server . rdb_checksum , 1 , NULL , NULL ) ,
2019-12-01 01:19:25 -05:00
createBoolConfig ( " daemonize " , NULL , IMMUTABLE_CONFIG , server . daemonize , 0 , NULL , NULL ) ,
2021-08-05 14:59:12 -04:00
createBoolConfig ( " io-threads-do-reads " , NULL , DEBUG_CONFIG | IMMUTABLE_CONFIG , server . io_threads_do_reads , 0 , NULL , NULL ) , /* Read + parse from threads? */
createBoolConfig ( " lua-replicate-commands " , NULL , DEBUG_CONFIG | MODIFIABLE_CONFIG , server . lua_always_replicate_commands , 1 , NULL , NULL ) ,
2019-12-01 01:19:25 -05:00
createBoolConfig ( " always-show-logo " , NULL , IMMUTABLE_CONFIG , server . always_show_logo , 0 , NULL , NULL ) ,
createBoolConfig ( " protected-mode " , NULL , MODIFIABLE_CONFIG , server . protected_mode , 1 , NULL , NULL ) ,
createBoolConfig ( " rdbcompression " , NULL , MODIFIABLE_CONFIG , server . rdb_compression , 1 , NULL , NULL ) ,
2020-03-04 11:44:21 -05:00
createBoolConfig ( " rdb-del-sync-files " , NULL , MODIFIABLE_CONFIG , server . rdb_del_sync_files , 0 , NULL , NULL ) ,
2019-12-01 01:19:25 -05:00
createBoolConfig ( " activerehashing " , NULL , MODIFIABLE_CONFIG , server . activerehashing , 1 , NULL , NULL ) ,
createBoolConfig ( " stop-writes-on-bgsave-error " , NULL , MODIFIABLE_CONFIG , server . stop_writes_on_bgsave_err , 1 , NULL , NULL ) ,
Add 'set-proc-title' config so that this mechanism can be disabled (#3623)
if option `set-proc-title' is no, then do nothing for proc title.
The reason has been explained long ago, see following:
We update redis to 2.8.8, then found there are some side effect when
redis always change the process title.
We run several slave instance on one computer, and all these salves
listen on unix socket only, then ps will show:
1 S redis 18036 1 0 80 0 - 56130 ep_pol 14:02 ? 00:00:31 /usr/sbin/redis-server *:0
1 S redis 23949 1 0 80 0 - 11074 ep_pol 15:41 ? 00:00:00 /usr/sbin/redis-server *:0
for redis 2.6 the output of ps is like following:
1 S redis 18036 1 0 80 0 - 56130 ep_pol 14:02 ? 00:00:31 /usr/sbin/redis-server /etc/redis/a.conf
1 S redis 23949 1 0 80 0 - 11074 ep_pol 15:41 ? 00:00:00 /usr/sbin/redis-server /etc/redis/b.conf
Later is more informational in our case. The situation
is worse when we manage the config and process running
state by salt. Salt check the process by running "ps |
grep SIG" (for Gentoo System) to check the running
state, where SIG is the string to search for when
looking for the service process with ps. Previously, we
define sig as "/usr/sbin/redis-server
/etc/redis/a.conf". Since the ps output is identical for
our case, so we have no way to check the state of
specified redis instance.
So, for our case, we prefer the old behavior, i.e, do
not change the process title for the main redis process.
Or add an option such as "set-proc-title [yes|no]" to
control this behavior.
Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-01-28 04:12:39 -05:00
createBoolConfig ( " set-proc-title " , NULL , IMMUTABLE_CONFIG , server . set_proc_title , 1 , NULL , NULL ) , /* Should setproctitle be used? */
2019-12-01 01:19:25 -05:00
createBoolConfig ( " dynamic-hz " , NULL , MODIFIABLE_CONFIG , server . dynamic_hz , 1 , NULL , NULL ) , /* Adapt hz to # of clients.*/
2021-08-05 14:59:12 -04:00
createBoolConfig ( " lazyfree-lazy-eviction " , NULL , DEBUG_CONFIG | MODIFIABLE_CONFIG , server . lazyfree_lazy_eviction , 0 , NULL , NULL ) ,
createBoolConfig ( " lazyfree-lazy-expire " , NULL , DEBUG_CONFIG | MODIFIABLE_CONFIG , server . lazyfree_lazy_expire , 0 , NULL , NULL ) ,
createBoolConfig ( " lazyfree-lazy-server-del " , NULL , DEBUG_CONFIG | MODIFIABLE_CONFIG , server . lazyfree_lazy_server_del , 0 , NULL , NULL ) ,
createBoolConfig ( " lazyfree-lazy-user-del " , NULL , DEBUG_CONFIG | MODIFIABLE_CONFIG , server . lazyfree_lazy_user_del , 0 , NULL , NULL ) ,
createBoolConfig ( " lazyfree-lazy-user-flush " , NULL , DEBUG_CONFIG | MODIFIABLE_CONFIG , server . lazyfree_lazy_user_flush , 0 , NULL , NULL ) ,
2019-12-01 01:19:25 -05:00
createBoolConfig ( " repl-disable-tcp-nodelay " , NULL , MODIFIABLE_CONFIG , server . repl_disable_tcp_nodelay , 0 , NULL , NULL ) ,
2021-08-05 14:59:12 -04:00
createBoolConfig ( " repl-diskless-sync " , NULL , DEBUG_CONFIG | MODIFIABLE_CONFIG , server . repl_diskless_sync , 0 , NULL , NULL ) ,
2019-12-01 01:19:25 -05:00
createBoolConfig ( " aof-rewrite-incremental-fsync " , NULL , MODIFIABLE_CONFIG , server . aof_rewrite_incremental_fsync , 1 , NULL , NULL ) ,
createBoolConfig ( " no-appendfsync-on-rewrite " , NULL , MODIFIABLE_CONFIG , server . aof_no_fsync_on_rewrite , 0 , NULL , NULL ) ,
createBoolConfig ( " cluster-require-full-coverage " , NULL , MODIFIABLE_CONFIG , server . cluster_require_full_coverage , 1 , NULL , NULL ) ,
createBoolConfig ( " rdb-save-incremental-fsync " , NULL , MODIFIABLE_CONFIG , server . rdb_save_incremental_fsync , 1 , NULL , NULL ) ,
createBoolConfig ( " aof-load-truncated " , NULL , MODIFIABLE_CONFIG , server . aof_load_truncated , 1 , NULL , NULL ) ,
createBoolConfig ( " aof-use-rdb-preamble " , NULL , MODIFIABLE_CONFIG , server . aof_use_rdb_preamble , 1 , NULL , NULL ) ,
2021-10-25 06:08:34 -04:00
createBoolConfig ( " aof-timestamp-enabled " , NULL , MODIFIABLE_CONFIG , server . aof_timestamp_enabled , 0 , NULL , NULL ) ,
2021-11-07 21:56:03 -05:00
createBoolConfig ( " cluster-replica-no-failover " , " cluster-slave-no-failover " , MODIFIABLE_CONFIG , server . cluster_slave_no_failover , 0 , NULL , updateClusterFlags ) , /* Failover by default. */
2019-12-01 01:19:25 -05:00
createBoolConfig ( " replica-lazy-flush " , " slave-lazy-flush " , MODIFIABLE_CONFIG , server . repl_slave_lazy_flush , 0 , NULL , NULL ) ,
createBoolConfig ( " replica-serve-stale-data " , " slave-serve-stale-data " , MODIFIABLE_CONFIG , server . repl_serve_stale_data , 1 , NULL , NULL ) ,
2021-08-05 14:59:12 -04:00
createBoolConfig ( " replica-read-only " , " slave-read-only " , DEBUG_CONFIG | MODIFIABLE_CONFIG , server . repl_slave_ro , 1 , NULL , NULL ) ,
2019-12-01 01:19:25 -05:00
createBoolConfig ( " replica-ignore-maxmemory " , " slave-ignore-maxmemory " , MODIFIABLE_CONFIG , server . repl_slave_ignore_maxmemory , 1 , NULL , NULL ) ,
2019-11-28 04:11:07 -05:00
createBoolConfig ( " jemalloc-bg-thread " , NULL , MODIFIABLE_CONFIG , server . jemalloc_bg_thread , 1 , NULL , updateJemallocBgThread ) ,
2021-08-05 14:59:12 -04:00
createBoolConfig ( " activedefrag " , NULL , DEBUG_CONFIG | MODIFIABLE_CONFIG , server . active_defrag_enabled , 0 , isValidActiveDefrag , NULL ) ,
2019-11-28 04:24:57 -05:00
createBoolConfig ( " syslog-enabled " , NULL , IMMUTABLE_CONFIG , server . syslog_enabled , 0 , NULL , NULL ) ,
createBoolConfig ( " cluster-enabled " , NULL , IMMUTABLE_CONFIG , server . cluster_enabled , 0 , NULL , NULL ) ,
createBoolConfig ( " appendonly " , NULL , MODIFIABLE_CONFIG , server . aof_enabled , 0 , NULL , updateAppendonly ) ,
2019-12-17 02:40:19 -05:00
createBoolConfig ( " cluster-allow-reads-when-down " , NULL , MODIFIABLE_CONFIG , server . cluster_allow_reads_when_down , 0 , NULL , NULL ) ,
2020-07-29 10:05:14 -04:00
createBoolConfig ( " crash-log-enabled " , NULL , MODIFIABLE_CONFIG , server . crashlog_enabled , 1 , NULL , updateSighandlerEnabled ) ,
createBoolConfig ( " crash-memcheck-enabled " , NULL , MODIFIABLE_CONFIG , server . memcheck_enabled , 1 , NULL , NULL ) ,
2021-12-08 05:44:10 -05:00
createBoolConfig ( " use-exit-on-panic " , NULL , MODIFIABLE_CONFIG | HIDDEN_CONFIG , server . use_exit_on_panic , 0 , NULL , NULL ) ,
2020-10-27 09:04:18 -04:00
createBoolConfig ( " disable-thp " , NULL , MODIFIABLE_CONFIG , server . disable_thp , 1 , NULL , NULL ) ,
2021-04-04 02:43:24 -04:00
createBoolConfig ( " cluster-allow-replica-migration " , NULL , MODIFIABLE_CONFIG , server . cluster_allow_replica_migration , 1 , NULL , NULL ) ,
2021-03-30 16:40:22 -04:00
createBoolConfig ( " replica-announced " , NULL , MODIFIABLE_CONFIG , server . replica_announced , 1 , NULL , NULL ) ,
2019-07-19 13:15:35 -04:00
/* String Configs */
2019-12-01 01:19:25 -05:00
createStringConfig ( " aclfile " , NULL , IMMUTABLE_CONFIG , ALLOW_EMPTY_STRING , server . acl_filename , " " , NULL , NULL ) ,
createStringConfig ( " unixsocket " , NULL , IMMUTABLE_CONFIG , EMPTY_STRING_IS_NULL , server . unixsocket , NULL , NULL , NULL ) ,
createStringConfig ( " pidfile " , NULL , IMMUTABLE_CONFIG , EMPTY_STRING_IS_NULL , server . pidfile , NULL , NULL , NULL ) ,
createStringConfig ( " replica-announce-ip " , " slave-announce-ip " , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . slave_announce_ip , NULL , NULL , NULL ) ,
2021-03-16 01:00:29 -04:00
createStringConfig ( " masteruser " , NULL , MODIFIABLE_CONFIG | SENSITIVE_CONFIG , EMPTY_STRING_IS_NULL , server . masteruser , NULL , NULL , NULL ) ,
2021-11-07 21:56:03 -05:00
createStringConfig ( " cluster-announce-ip " , NULL , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . cluster_announce_ip , NULL , NULL , updateClusterIp ) ,
2021-10-08 01:32:40 -04:00
createStringConfig ( " cluster-config-file " , NULL , IMMUTABLE_CONFIG , ALLOW_EMPTY_STRING , server . cluster_configfile , " nodes.conf " , NULL , NULL ) ,
2019-11-28 04:24:57 -05:00
createStringConfig ( " syslog-ident " , NULL , IMMUTABLE_CONFIG , ALLOW_EMPTY_STRING , server . syslog_ident , " redis " , NULL , NULL ) ,
createStringConfig ( " dbfilename " , NULL , MODIFIABLE_CONFIG , ALLOW_EMPTY_STRING , server . rdb_filename , " dump.rdb " , isValidDBfilename , NULL ) ,
createStringConfig ( " appendfilename " , NULL , IMMUTABLE_CONFIG , ALLOW_EMPTY_STRING , server . aof_filename , " appendonly.aof " , isValidAOFfilename , NULL ) ,
Support setcpuaffinity on linux/bsd
Currently, there are several types of threads/child processes of a
redis server. Sometimes we need deeply optimise the performance of
redis, so we would like to isolate threads/processes.
There were some discussion about cpu affinity cases in the issue:
https://github.com/antirez/redis/issues/2863
So implement cpu affinity setting by redis.conf in this patch, then
we can config server_cpulist/bio_cpulist/aof_rewrite_cpulist/
bgsave_cpulist by cpu list.
Examples of cpulist in redis.conf:
server_cpulist 0-7:2 means cpu affinity 0,2,4,6
bio_cpulist 1,3 means cpu affinity 1,3
aof_rewrite_cpulist 8-11 means cpu affinity 8,9,10,11
bgsave_cpulist 1,10-11 means cpu affinity 1,10,11
Test on linux/freebsd, both work fine.
Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
2020-05-02 08:05:39 -04:00
createStringConfig ( " server_cpulist " , NULL , IMMUTABLE_CONFIG , EMPTY_STRING_IS_NULL , server . server_cpulist , NULL , NULL , NULL ) ,
createStringConfig ( " bio_cpulist " , NULL , IMMUTABLE_CONFIG , EMPTY_STRING_IS_NULL , server . bio_cpulist , NULL , NULL , NULL ) ,
createStringConfig ( " aof_rewrite_cpulist " , NULL , IMMUTABLE_CONFIG , EMPTY_STRING_IS_NULL , server . aof_rewrite_cpulist , NULL , NULL , NULL ) ,
createStringConfig ( " bgsave_cpulist " , NULL , IMMUTABLE_CONFIG , EMPTY_STRING_IS_NULL , server . bgsave_cpulist , NULL , NULL , NULL ) ,
2021-01-07 10:06:05 -05:00
createStringConfig ( " ignore-warnings " , NULL , MODIFIABLE_CONFIG , ALLOW_EMPTY_STRING , server . ignore_warnings , " " , NULL , NULL ) ,
2021-01-28 11:17:39 -05:00
createStringConfig ( " proc-title-template " , NULL , MODIFIABLE_CONFIG , ALLOW_EMPTY_STRING , server . proc_title_template , CONFIG_DEFAULT_PROC_TITLE_TEMPLATE , isValidProcTitleTemplate , updateProcTitleTemplate ) ,
2021-06-24 12:48:18 -04:00
createStringConfig ( " bind-source-addr " , NULL , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . bind_source_addr , NULL , NULL , NULL ) ,
2021-10-08 01:33:08 -04:00
createStringConfig ( " logfile " , NULL , IMMUTABLE_CONFIG , ALLOW_EMPTY_STRING , server . logfile , " " , NULL , NULL ) ,
2019-07-19 13:15:35 -04:00
2020-12-17 12:26:33 -05:00
/* SDS Configs */
2021-03-16 01:00:29 -04:00
createSDSConfig ( " masterauth " , NULL , MODIFIABLE_CONFIG | SENSITIVE_CONFIG , EMPTY_STRING_IS_NULL , server . masterauth , NULL , NULL , NULL ) ,
createSDSConfig ( " requirepass " , NULL , MODIFIABLE_CONFIG | SENSITIVE_CONFIG , EMPTY_STRING_IS_NULL , server . requirepass , NULL , NULL , updateRequirePass ) ,
2020-12-17 12:26:33 -05:00
2019-07-19 13:15:35 -04:00
/* Enum Configs */
2019-12-01 01:19:25 -05:00
createEnumConfig ( " supervised " , NULL , IMMUTABLE_CONFIG , supervised_mode_enum , server . supervised_mode , SUPERVISED_NONE , NULL , NULL ) ,
createEnumConfig ( " syslog-facility " , NULL , IMMUTABLE_CONFIG , syslog_facility_enum , server . syslog_facility , LOG_LOCAL0 , NULL , NULL ) ,
2021-08-05 14:59:12 -04:00
createEnumConfig ( " repl-diskless-load " , NULL , DEBUG_CONFIG | MODIFIABLE_CONFIG , repl_diskless_load_enum , server . repl_diskless_load , REPL_DISKLESS_LOAD_DISABLED , NULL , NULL ) ,
2019-12-01 01:19:25 -05:00
createEnumConfig ( " loglevel " , NULL , MODIFIABLE_CONFIG , loglevel_enum , server . verbosity , LL_NOTICE , NULL , NULL ) ,
createEnumConfig ( " maxmemory-policy " , NULL , MODIFIABLE_CONFIG , maxmemory_policy_enum , server . maxmemory_policy , MAXMEMORY_NO_EVICTION , NULL , NULL ) ,
createEnumConfig ( " appendfsync " , NULL , MODIFIABLE_CONFIG , aof_fsync_enum , server . aof_fsync , AOF_FSYNC_EVERYSEC , NULL , NULL ) ,
2020-11-22 06:57:56 -05:00
createEnumConfig ( " oom-score-adj " , NULL , MODIFIABLE_CONFIG , oom_score_adj_enum , server . oom_score_adj , OOM_SCORE_ADJ_NO , NULL , updateOOMScoreAdj ) ,
Fix "default" and overwritten / reset users will not have pubsub channels permissions by default. (#8723)
Background:
Redis 6.2 added ACL control for pubsub channels (#7993), which were supposed
to be permissive by default to retain compatibility with redis 6.0 ACL.
But due to a bug, only newly created users got this `acl-pubsub-default` applied,
while overwritten (updated) users got reset to `resetchannels` (denied).
Since the "default" user exists before loading the config file,
any ACL change to it, results in an update / overwrite.
So when a "default" user is loaded from config file or include ACL
file with no channels related rules, the user will not have any
permissions to any channels. But other users will have default
permissions to any channels.
When upgraded from 6.0 with config rewrite, this will lead to
"default" user channels permissions lost.
When users are loaded from include file, then call "acl load", users
will also lost channels permissions.
Similarly, the `reset` ACL rule, would have reset the user to be denied
access to any channels, ignoring `acl-pubsub-default` and breaking
compatibility with redis 6.0.
The implication of this fix is that it regains compatibility with redis 6.0,
but breaks compatibility with redis 6.2.0 and 2.0.1. e.g. after the upgrade,
the default user will regain access to pubsub channels.
Other changes:
Additionally this commit rename server.acl_pubusub_default to
server.acl_pubsub_default and fix typo in acl tests.
2021-04-05 16:13:20 -04:00
createEnumConfig ( " acl-pubsub-default " , NULL , MODIFIABLE_CONFIG , acl_pubsub_default_enum , server . acl_pubsub_default , USER_FLAG_ALLCHANNELS , NULL , NULL ) ,
2021-08-05 14:59:12 -04:00
createEnumConfig ( " sanitize-dump-payload " , NULL , DEBUG_CONFIG | MODIFIABLE_CONFIG , sanitize_dump_payload_enum , server . sanitize_dump_payload , SANITIZE_DUMP_NO , NULL , NULL ) ,
2019-07-19 13:15:35 -04:00
/* Integer configs */
2019-12-01 01:19:25 -05:00
createIntConfig ( " databases " , NULL , IMMUTABLE_CONFIG , 1 , INT_MAX , server . dbnum , 16 , INTEGER_CONFIG , NULL , NULL ) ,
2021-03-01 09:04:44 -05:00
createIntConfig ( " port " , NULL , MODIFIABLE_CONFIG , 0 , 65535 , server . port , 6379 , INTEGER_CONFIG , NULL , updatePort ) , /* TCP port. */
2021-08-05 14:59:12 -04:00
createIntConfig ( " io-threads " , NULL , DEBUG_CONFIG | IMMUTABLE_CONFIG , 1 , 128 , server . io_threads_num , 1 , INTEGER_CONFIG , NULL , NULL ) , /* Single threaded by default */
2019-12-01 01:19:25 -05:00
createIntConfig ( " auto-aof-rewrite-percentage " , NULL , MODIFIABLE_CONFIG , 0 , INT_MAX , server . aof_rewrite_perc , 100 , INTEGER_CONFIG , NULL , NULL ) ,
createIntConfig ( " cluster-replica-validity-factor " , " cluster-slave-validity-factor " , MODIFIABLE_CONFIG , 0 , INT_MAX , server . cluster_slave_validity_factor , 10 , INTEGER_CONFIG , NULL , NULL ) , /* Slave max data age factor. */
2021-11-24 06:34:13 -05:00
createIntConfig ( " list-max-listpack-size " , " list-max-ziplist-size " , MODIFIABLE_CONFIG , INT_MIN , INT_MAX , server . list_max_listpack_size , - 2 , INTEGER_CONFIG , NULL , NULL ) ,
2019-12-01 01:19:25 -05:00
createIntConfig ( " tcp-keepalive " , NULL , MODIFIABLE_CONFIG , 0 , INT_MAX , server . tcpkeepalive , 300 , INTEGER_CONFIG , NULL , NULL ) ,
createIntConfig ( " cluster-migration-barrier " , NULL , MODIFIABLE_CONFIG , 0 , INT_MAX , server . cluster_migration_barrier , 1 , INTEGER_CONFIG , NULL , NULL ) ,
createIntConfig ( " active-defrag-cycle-min " , NULL , MODIFIABLE_CONFIG , 1 , 99 , server . active_defrag_cycle_min , 1 , INTEGER_CONFIG , NULL , NULL ) , /* Default: 1% CPU min (at lower threshold) */
createIntConfig ( " active-defrag-cycle-max " , NULL , MODIFIABLE_CONFIG , 1 , 99 , server . active_defrag_cycle_max , 25 , INTEGER_CONFIG , NULL , NULL ) , /* Default: 25% CPU max (at upper threshold) */
createIntConfig ( " active-defrag-threshold-lower " , NULL , MODIFIABLE_CONFIG , 0 , 1000 , server . active_defrag_threshold_lower , 10 , INTEGER_CONFIG , NULL , NULL ) , /* Default: don't defrag when fragmentation is below 10% */
createIntConfig ( " active-defrag-threshold-upper " , NULL , MODIFIABLE_CONFIG , 0 , 1000 , server . active_defrag_threshold_upper , 100 , INTEGER_CONFIG , NULL , NULL ) , /* Default: maximum defrag force at 100% fragmentation */
createIntConfig ( " lfu-log-factor " , NULL , MODIFIABLE_CONFIG , 0 , INT_MAX , server . lfu_log_factor , 10 , INTEGER_CONFIG , NULL , NULL ) ,
createIntConfig ( " lfu-decay-time " , NULL , MODIFIABLE_CONFIG , 0 , INT_MAX , server . lfu_decay_time , 1 , INTEGER_CONFIG , NULL , NULL ) ,
createIntConfig ( " replica-priority " , " slave-priority " , MODIFIABLE_CONFIG , 0 , INT_MAX , server . slave_priority , 100 , INTEGER_CONFIG , NULL , NULL ) ,
createIntConfig ( " repl-diskless-sync-delay " , NULL , MODIFIABLE_CONFIG , 0 , INT_MAX , server . repl_diskless_sync_delay , 5 , INTEGER_CONFIG , NULL , NULL ) ,
createIntConfig ( " maxmemory-samples " , NULL , MODIFIABLE_CONFIG , 1 , INT_MAX , server . maxmemory_samples , 5 , INTEGER_CONFIG , NULL , NULL ) ,
2020-09-16 02:16:01 -04:00
createIntConfig ( " maxmemory-eviction-tenacity " , NULL , MODIFIABLE_CONFIG , 0 , 100 , server . maxmemory_eviction_tenacity , 10 , INTEGER_CONFIG , NULL , NULL ) ,
2019-12-01 01:19:25 -05:00
createIntConfig ( " timeout " , NULL , MODIFIABLE_CONFIG , 0 , INT_MAX , server . maxidletime , 0 , INTEGER_CONFIG , NULL , NULL ) , /* Default client timeout: infinite */
createIntConfig ( " replica-announce-port " , " slave-announce-port " , MODIFIABLE_CONFIG , 0 , 65535 , server . slave_announce_port , 0 , INTEGER_CONFIG , NULL , NULL ) ,
2019-12-26 06:59:58 -05:00
createIntConfig ( " tcp-backlog " , NULL , IMMUTABLE_CONFIG , 0 , INT_MAX , server . tcp_backlog , 511 , INTEGER_CONFIG , NULL , NULL ) , /* TCP listen backlog. */
2021-10-19 01:28:27 -04:00
createIntConfig ( " cluster-port " , NULL , IMMUTABLE_CONFIG , 0 , 65535 , server . cluster_port , 0 , INTEGER_CONFIG , NULL , NULL ) ,
2019-12-01 01:19:25 -05:00
createIntConfig ( " cluster-announce-bus-port " , NULL , MODIFIABLE_CONFIG , 0 , 65535 , server . cluster_announce_bus_port , 0 , INTEGER_CONFIG , NULL , NULL ) , /* Default: Use +10000 offset. */
createIntConfig ( " cluster-announce-port " , NULL , MODIFIABLE_CONFIG , 0 , 65535 , server . cluster_announce_port , 0 , INTEGER_CONFIG , NULL , NULL ) , /* Use server.port */
2021-03-30 16:11:32 -04:00
createIntConfig ( " cluster-announce-tls-port " , NULL , MODIFIABLE_CONFIG , 0 , 65535 , server . cluster_announce_tls_port , 0 , INTEGER_CONFIG , NULL , NULL ) , /* Use server.tls_port */
2019-12-01 01:19:25 -05:00
createIntConfig ( " repl-timeout " , NULL , MODIFIABLE_CONFIG , 1 , INT_MAX , server . repl_timeout , 60 , INTEGER_CONFIG , NULL , NULL ) ,
createIntConfig ( " repl-ping-replica-period " , " repl-ping-slave-period " , MODIFIABLE_CONFIG , 1 , INT_MAX , server . repl_ping_slave_period , 10 , INTEGER_CONFIG , NULL , NULL ) ,
2021-08-05 14:59:12 -04:00
createIntConfig ( " list-compress-depth " , NULL , DEBUG_CONFIG | MODIFIABLE_CONFIG , 0 , INT_MAX , server . list_compress_depth , 0 , INTEGER_CONFIG , NULL , NULL ) ,
2021-12-08 05:44:10 -05:00
createIntConfig ( " rdb-key-save-delay " , NULL , MODIFIABLE_CONFIG | HIDDEN_CONFIG , INT_MIN , INT_MAX , server . rdb_key_save_delay , 0 , INTEGER_CONFIG , NULL , NULL ) ,
createIntConfig ( " key-load-delay " , NULL , MODIFIABLE_CONFIG | HIDDEN_CONFIG , INT_MIN , INT_MAX , server . key_load_delay , 0 , INTEGER_CONFIG , NULL , NULL ) ,
2019-12-01 01:19:25 -05:00
createIntConfig ( " active-expire-effort " , NULL , MODIFIABLE_CONFIG , 1 , 10 , server . active_expire_effort , 1 , INTEGER_CONFIG , NULL , NULL ) , /* From 1 to 10. */
createIntConfig ( " hz " , NULL , MODIFIABLE_CONFIG , 0 , INT_MAX , server . config_hz , CONFIG_DEFAULT_HZ , INTEGER_CONFIG , NULL , updateHZ ) ,
createIntConfig ( " min-replicas-to-write " , " min-slaves-to-write " , MODIFIABLE_CONFIG , 0 , INT_MAX , server . repl_min_slaves_to_write , 0 , INTEGER_CONFIG , NULL , updateGoodSlaves ) ,
createIntConfig ( " min-replicas-max-lag " , " min-slaves-max-lag " , MODIFIABLE_CONFIG , 0 , INT_MAX , server . repl_min_slaves_max_lag , 10 , INTEGER_CONFIG , NULL , updateGoodSlaves ) ,
2021-12-08 05:44:10 -05:00
createIntConfig ( " watchdog-period " , NULL , MODIFIABLE_CONFIG | HIDDEN_CONFIG , 0 , INT_MAX , server . watchdog_period , 0 , INTEGER_CONFIG , NULL , updateWatchdogPeriod ) ,
2019-11-28 04:24:57 -05:00
/* Unsigned int configs */
createUIntConfig ( " maxclients " , NULL , MODIFIABLE_CONFIG , 1 , UINT_MAX , server . maxclients , 10000 , INTEGER_CONFIG , NULL , updateMaxclients ) ,
2021-10-19 02:58:52 -04:00
createUIntConfig ( " unixsocketperm " , NULL , IMMUTABLE_CONFIG , 0 , 0777 , server . unixsocketperm , 0 , OCTAL_CONFIG , NULL , NULL ) ,
2019-07-19 13:15:35 -04:00
/* Unsigned Long configs */
2019-12-01 01:19:25 -05:00
createULongConfig ( " active-defrag-max-scan-fields " , NULL , MODIFIABLE_CONFIG , 1 , LONG_MAX , server . active_defrag_max_scan_fields , 1000 , INTEGER_CONFIG , NULL , NULL ) , /* Default: keys with more than 1000 fields will be processed separately */
createULongConfig ( " slowlog-max-len " , NULL , MODIFIABLE_CONFIG , 0 , LONG_MAX , server . slowlog_max_len , 128 , INTEGER_CONFIG , NULL , NULL ) ,
2020-02-04 07:19:40 -05:00
createULongConfig ( " acllog-max-len " , NULL , MODIFIABLE_CONFIG , 0 , LONG_MAX , server . acllog_max_len , 128 , INTEGER_CONFIG , NULL , NULL ) ,
2019-07-19 13:15:35 -04:00
/* Long Long configs */
2021-10-05 10:03:12 -04:00
createLongLongConfig ( " script-time-limit " , " lua-time-limit " , MODIFIABLE_CONFIG , 0 , LONG_MAX , server . script_time_limit , 5000 , INTEGER_CONFIG , NULL , NULL ) , /* milliseconds */
2019-12-01 01:19:25 -05:00
createLongLongConfig ( " cluster-node-timeout " , NULL , MODIFIABLE_CONFIG , 0 , LLONG_MAX , server . cluster_node_timeout , 15000 , INTEGER_CONFIG , NULL , NULL ) ,
createLongLongConfig ( " slowlog-log-slower-than " , NULL , MODIFIABLE_CONFIG , - 1 , LLONG_MAX , server . slowlog_log_slower_than , 10000 , INTEGER_CONFIG , NULL , NULL ) ,
createLongLongConfig ( " latency-monitor-threshold " , NULL , MODIFIABLE_CONFIG , 0 , LLONG_MAX , server . latency_monitor_threshold , 0 , INTEGER_CONFIG , NULL , NULL ) ,
2021-08-05 14:59:12 -04:00
createLongLongConfig ( " proto-max-bulk-len " , NULL , DEBUG_CONFIG | MODIFIABLE_CONFIG , 1024 * 1024 , LONG_MAX , server . proto_max_bulk_len , 512ll * 1024 * 1024 , MEMORY_CONFIG , NULL , NULL ) , /* Bulk request max size */
2019-12-26 06:59:58 -05:00
createLongLongConfig ( " stream-node-max-entries " , NULL , MODIFIABLE_CONFIG , 0 , LLONG_MAX , server . stream_node_max_entries , 100 , INTEGER_CONFIG , NULL , NULL ) ,
Replication backlog and replicas use one global shared replication buffer (#9166)
## Background
For redis master, one replica uses one copy of replication buffer, that is a big waste of memory,
more replicas more waste, and allocate/free memory for every reply list also cost much.
If we set client-output-buffer-limit small and write traffic is heavy, master may disconnect with
replicas and can't finish synchronization with replica. If we set client-output-buffer-limit big,
master may be OOM when there are many replicas that separately keep much memory.
Because replication buffers of different replica client are the same, one simple idea is that
all replicas only use one replication buffer, that will effectively save memory.
Since replication backlog content is the same as replicas' output buffer, now we
can discard replication backlog memory and use global shared replication buffer
to implement replication backlog mechanism.
## Implementation
I create one global "replication buffer" which contains content of replication stream.
The structure of "replication buffer" is similar to the reply list that exists in every client.
But the node of list is `replBufBlock`, which has `id, repl_offset, refcount` fields.
```c
/* Replication buffer blocks is the list of replBufBlock.
*
* +--------------+ +--------------+ +--------------+
* | refcount = 1 | ... | refcount = 0 | ... | refcount = 2 |
* +--------------+ +--------------+ +--------------+
* | / \
* | / \
* | / \
* Repl Backlog Replia_A Replia_B
*
* Each replica or replication backlog increments only the refcount of the
* 'ref_repl_buf_node' which it points to. So when replica walks to the next
* node, it should first increase the next node's refcount, and when we trim
* the replication buffer nodes, we remove node always from the head node which
* refcount is 0. If the refcount of the head node is not 0, we must stop
* trimming and never iterate the next node. */
/* Similar with 'clientReplyBlock', it is used for shared buffers between
* all replica clients and replication backlog. */
typedef struct replBufBlock {
int refcount; /* Number of replicas or repl backlog using. */
long long id; /* The unique incremental number. */
long long repl_offset; /* Start replication offset of the block. */
size_t size, used;
char buf[];
} replBufBlock;
```
So now when we feed replication stream into replication backlog and all replicas, we only need
to feed stream into replication buffer `feedReplicationBuffer`. In this function, we set some fields of
replication backlog and replicas to references of the global replication buffer blocks. And we also
need to check replicas' output buffer limit to free if exceeding `client-output-buffer-limit`, and trim
replication backlog if exceeding `repl-backlog-size`.
When sending reply to replicas, we also need to iterate replication buffer blocks and send its
content, when totally sending one block for replica, we decrease current node count and
increase the next current node count, and then free the block which reference is 0 from the
head of replication buffer blocks.
Since now we use linked list to manage replication backlog, it may cost much time for iterating
all linked list nodes to find corresponding replication buffer node. So we create a rax tree to
store some nodes for index, but to avoid rax tree occupying too much memory, i record
one per 64 nodes for index.
Currently, to make partial resynchronization as possible as much, we always let replication
backlog as the last reference of replication buffer blocks, backlog size may exceeds our setting
if slow replicas that reference vast replication buffer blocks, and this method doesn't increase
memory usage since they share replication buffer. To avoid freezing server for freeing unreferenced
replication buffer blocks when we need to trim backlog for exceeding backlog size setting,
we trim backlog incrementally (free 64 blocks per call now), and make it faster in
`beforeSleep` (free 640 blocks).
### Other changes
- `mem_total_replication_buffers`: we add this field in INFO command, it means the total
memory of replication buffers used.
- `mem_clients_slaves`: now even replica is slow to replicate, and its output buffer memory
is not 0, but it still may be 0, since replication backlog and replicas share one global replication
buffer, only if replication buffer memory is more than the repl backlog setting size, we consider
the excess as replicas' memory. Otherwise, we think replication buffer memory is the consumption
of repl backlog.
- Key eviction
Since all replicas and replication backlog share global replication buffer, we think only the
part of exceeding backlog size the extra separate consumption of replicas.
Because we trim backlog incrementally in the background, backlog size may exceeds our
setting if slow replicas that reference vast replication buffer blocks disconnect.
To avoid massive eviction loop, we don't count the delayed freed replication backlog into
used memory even if there are no replicas, i.e. we also regard this memory as replicas's memory.
- `client-output-buffer-limit` check for replica clients
It doesn't make sense to set the replica clients output buffer limit lower than the repl-backlog-size
config (partial sync will succeed and then replica will get disconnected). Such a configuration is
ignored (the size of repl-backlog-size will be used). This doesn't have memory consumption
implications since the replica client will share the backlog buffers memory.
- Drop replication backlog after loading data if needed
We always create replication backlog if server is a master, we need it because we put DELs in
it when loading expired keys in RDB, but if RDB doesn't have replication info or there is no rdb,
it is not possible to support partial resynchronization, to avoid extra memory of replication backlog,
we drop it.
- Multi IO threads
Since all replicas and replication backlog use global replication buffer, if I/O threads are enabled,
to guarantee data accessing thread safe, we must let main thread handle sending the output buffer
to all replicas. But before, other IO threads could handle sending output buffer of all replicas.
## Other optimizations
This solution resolve some other problem:
- When replicas disconnect with master since of out of output buffer limit, releasing the output
buffer of replicas may freeze server if we set big `client-output-buffer-limit` for replicas, but now,
it doesn't cause freezing.
- This implementation may mitigate reply list copy cost time(also freezes server) when one replication
has huge reply buffer and another replica can copy buffer for full synchronization. now, we just copy
reference info, it is very light.
- If we set replication backlog size big, it also may cost much time to copy replication backlog into
replica's output buffer. But this commit eliminates this problem.
- Resizing replication backlog size doesn't empty current replication backlog content.
2021-10-25 02:24:31 -04:00
createLongLongConfig ( " repl-backlog-size " , NULL , MODIFIABLE_CONFIG , 1 , LLONG_MAX , server . repl_backlog_size , 1024 * 1024 , MEMORY_CONFIG , NULL , updateReplBacklogSize ) , /* Default: 1mb */
2019-11-28 04:24:57 -05:00
/* Unsigned Long Long configs */
2019-12-01 01:19:25 -05:00
createULongLongConfig ( " maxmemory " , NULL , MODIFIABLE_CONFIG , 0 , ULLONG_MAX , server . maxmemory , 0 , MEMORY_CONFIG , NULL , updateMaxmemory ) ,
2019-07-19 13:15:35 -04:00
/* Size_t configs */
2021-08-10 02:18:49 -04:00
createSizeTConfig ( " hash-max-listpack-entries " , " hash-max-ziplist-entries " , MODIFIABLE_CONFIG , 0 , LONG_MAX , server . hash_max_listpack_entries , 512 , INTEGER_CONFIG , NULL , NULL ) ,
2019-12-01 01:19:25 -05:00
createSizeTConfig ( " set-max-intset-entries " , NULL , MODIFIABLE_CONFIG , 0 , LONG_MAX , server . set_max_intset_entries , 512 , INTEGER_CONFIG , NULL , NULL ) ,
2021-09-09 11:18:53 -04:00
createSizeTConfig ( " zset-max-listpack-entries " , " zset-max-ziplist-entries " , MODIFIABLE_CONFIG , 0 , LONG_MAX , server . zset_max_listpack_entries , 128 , INTEGER_CONFIG , NULL , NULL ) ,
2019-12-26 06:59:58 -05:00
createSizeTConfig ( " active-defrag-ignore-bytes " , NULL , MODIFIABLE_CONFIG , 1 , LLONG_MAX , server . active_defrag_ignore_bytes , 100 < < 20 , MEMORY_CONFIG , NULL , NULL ) , /* Default: don't defrag if frag overhead is below 100mb */
2021-08-10 02:18:49 -04:00
createSizeTConfig ( " hash-max-listpack-value " , " hash-max-ziplist-value " , MODIFIABLE_CONFIG , 0 , LONG_MAX , server . hash_max_listpack_value , 64 , MEMORY_CONFIG , NULL , NULL ) ,
2019-12-01 01:19:25 -05:00
createSizeTConfig ( " stream-node-max-bytes " , NULL , MODIFIABLE_CONFIG , 0 , LONG_MAX , server . stream_node_max_bytes , 4096 , MEMORY_CONFIG , NULL , NULL ) ,
2021-09-09 11:18:53 -04:00
createSizeTConfig ( " zset-max-listpack-value " , " zset-max-ziplist-value " , MODIFIABLE_CONFIG , 0 , LONG_MAX , server . zset_max_listpack_value , 64 , MEMORY_CONFIG , NULL , NULL ) ,
2019-12-01 01:19:25 -05:00
createSizeTConfig ( " hll-sparse-max-bytes " , NULL , MODIFIABLE_CONFIG , 0 , LONG_MAX , server . hll_sparse_max_bytes , 3000 , MEMORY_CONFIG , NULL , NULL ) ,
2020-02-19 13:00:29 -05:00
createSizeTConfig ( " tracking-table-max-keys " , NULL , MODIFIABLE_CONFIG , 0 , LONG_MAX , server . tracking_table_max_keys , 1000000 , INTEGER_CONFIG , NULL , NULL ) , /* Default: 1 million keys max. */
2021-08-05 14:59:12 -04:00
createSizeTConfig ( " client-query-buffer-limit " , NULL , DEBUG_CONFIG | MODIFIABLE_CONFIG , 1024 * 1024 , LONG_MAX , server . client_max_querybuf_len , 1024 * 1024 * 1024 , MEMORY_CONFIG , NULL , NULL ) , /* Default: 1GB max query buffer. */
Client eviction (#8687)
### Description
A mechanism for disconnecting clients when the sum of all connected clients is above a
configured limit. This prevents eviction or OOM caused by accumulated used memory
between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
mechanism which takes into account not only a single client and not only output buffers
but rather all memory used by all clients.
#### Design
The general design is as following:
* We track memory usage of each client, taking into account all memory used by the
client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
after reading from the socket, after processing commands and after writing to the socket.
* Based on the used memory we sort all clients into buckets. Each bucket contains all
clients using up up to x2 memory of the clients in the bucket below it. For example up
to 1m clients, up to 2m clients, up to 4m clients, ...
* Before processing a command and before sleep we check if we're over the configured
limit. If we are we start disconnecting clients from larger buckets downwards until we're
under the limit.
#### Config
`maxmemory-clients` max memory all clients are allowed to consume, above this threshold
we disconnect clients.
This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
would mean 10% of `maxmemory`).
#### Important code changes
* During the development I encountered yet more situations where our io-threads access
global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
memory buckets (which are global) while their memory usage changes in the io-thread.
To achieve this I decided to simplify how we check if we're in an io-thread and make it
much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
if the client is in an io-thread (it wasn't used for anything else) and just used the global
`io_threads_op` variable the same way to check during writes.
* I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
We now store a pointer in the `client` struct to this list so we don't need to search in it
(`pending_read_list_node`).
* Added `evicted_clients` stat to `INFO` command.
* Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
client eviction mechanism. Added corrosponding 'e' flag in the client info string.
* Added `multi-mem` field in the client info string to show how much memory is used up
by buffered multi commands.
* Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
channels (partially), tracking prefixes (partially).
* CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
clients will be disconnected between processing different clients and not only before sleep.
This new function can be used in the future for work we want to do outside the command
processing loop but don't want to wait for all clients to be processed before we get to it.
Specifically I wanted to handle output-buffer-limit related closing before we process client
eviction in case the two race with each other.
* Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
buckets.
* Each client now holds a pointer to the client eviction memory usage bucket it belongs to
and listNode to itself in that bucket for quick removal.
* Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
indicating no io-threading is currently being executed.
* In order to track memory used by each clients in real-time we can't rely on updating
these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
(used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
writing data to pubsub clients, after writing the output buffer and after reading from the
socket (and maybe other places too). The function is written to be fast.
* Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
processing a command (before performing oom-checks and key-eviction).
* All clients memory usage buckets are grouped as follows:
* All clients using less than 64k.
* 64K..128K
* 128K..256K
* ...
* 2G..4G
* All clients using 4g and up.
* Added client-eviction.tcl with a bunch of tests for the new mechanism.
* Extended maxmemory.tcl to test the interaction between maxmemory and
maxmemory-clients settings.
* Added an option to flag a numeric configuration variable as a "percent", this means that
if we encounter a '%' after the number in the config file (or config set command) we
consider it as valid. Such a number is store internally as a negative value. This way an
integer value can be interpreted as either a percent (negative) or absolute value (positive).
This is useful for example if some numeric configuration can optionally be set to a percentage
of something else.
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-09-23 07:02:16 -04:00
createSSizeTConfig ( " maxmemory-clients " , NULL , MODIFIABLE_CONFIG , - 100 , SSIZE_MAX , server . maxmemory_clients , 0 , MEMORY_CONFIG | PERCENT_CONFIG , NULL , NULL ) ,
2019-11-28 04:24:57 -05:00
/* Other configs */
2019-12-01 01:19:25 -05:00
createTimeTConfig ( " repl-backlog-ttl " , NULL , MODIFIABLE_CONFIG , 0 , LONG_MAX , server . repl_backlog_time_limit , 60 * 60 , INTEGER_CONFIG , NULL , NULL ) , /* Default: 1 hour */
createOffTConfig ( " auto-aof-rewrite-min-size " , NULL , MODIFIABLE_CONFIG , 0 , LLONG_MAX , server . aof_rewrite_min_size , 64 * 1024 * 1024 , MEMORY_CONFIG , NULL , NULL ) ,
2021-12-08 05:44:10 -05:00
createOffTConfig ( " loading-process-events-interval-bytes " , NULL , MODIFIABLE_CONFIG | HIDDEN_CONFIG , 1024 , INT_MAX , server . loading_process_events_interval_bytes , 1024 * 1024 * 2 , INTEGER_CONFIG , NULL , NULL ) ,
2019-11-28 04:24:57 -05:00
# ifdef USE_OPENSSL
2021-12-01 03:15:11 -05:00
createIntConfig ( " tls-port " , NULL , MODIFIABLE_CONFIG , 0 , 65535 , server . tls_port , 0 , INTEGER_CONFIG , NULL , applyTLSPort ) , /* TCP port. */
createIntConfig ( " tls-session-cache-size " , NULL , MODIFIABLE_CONFIG , 0 , INT_MAX , server . tls_ctx_config . session_cache_size , 20 * 1024 , INTEGER_CONFIG , NULL , applyTlsCfg ) ,
createIntConfig ( " tls-session-cache-timeout " , NULL , MODIFIABLE_CONFIG , 0 , INT_MAX , server . tls_ctx_config . session_cache_timeout , 300 , INTEGER_CONFIG , NULL , applyTlsCfg ) ,
createBoolConfig ( " tls-cluster " , NULL , MODIFIABLE_CONFIG , server . tls_cluster , 0 , NULL , applyTlsCfg ) ,
createBoolConfig ( " tls-replication " , NULL , MODIFIABLE_CONFIG , server . tls_replication , 0 , NULL , applyTlsCfg ) ,
2020-07-28 03:45:21 -04:00
createEnumConfig ( " tls-auth-clients " , NULL , MODIFIABLE_CONFIG , tls_auth_clients_enum , server . tls_auth_clients , TLS_CLIENT_AUTH_YES , NULL , NULL ) ,
2021-12-01 03:15:11 -05:00
createBoolConfig ( " tls-prefer-server-ciphers " , NULL , MODIFIABLE_CONFIG , server . tls_ctx_config . prefer_server_ciphers , 0 , NULL , applyTlsCfg ) ,
createBoolConfig ( " tls-session-caching " , NULL , MODIFIABLE_CONFIG , server . tls_ctx_config . session_caching , 1 , NULL , applyTlsCfg ) ,
createStringConfig ( " tls-cert-file " , NULL , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . tls_ctx_config . cert_file , NULL , NULL , applyTlsCfg ) ,
createStringConfig ( " tls-key-file " , NULL , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . tls_ctx_config . key_file , NULL , NULL , applyTlsCfg ) ,
createStringConfig ( " tls-key-file-pass " , NULL , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . tls_ctx_config . key_file_pass , NULL , NULL , applyTlsCfg ) ,
createStringConfig ( " tls-client-cert-file " , NULL , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . tls_ctx_config . client_cert_file , NULL , NULL , applyTlsCfg ) ,
createStringConfig ( " tls-client-key-file " , NULL , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . tls_ctx_config . client_key_file , NULL , NULL , applyTlsCfg ) ,
createStringConfig ( " tls-client-key-file-pass " , NULL , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . tls_ctx_config . client_key_file_pass , NULL , NULL , applyTlsCfg ) ,
createStringConfig ( " tls-dh-params-file " , NULL , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . tls_ctx_config . dh_params_file , NULL , NULL , applyTlsCfg ) ,
createStringConfig ( " tls-ca-cert-file " , NULL , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . tls_ctx_config . ca_cert_file , NULL , NULL , applyTlsCfg ) ,
createStringConfig ( " tls-ca-cert-dir " , NULL , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . tls_ctx_config . ca_cert_dir , NULL , NULL , applyTlsCfg ) ,
createStringConfig ( " tls-protocols " , NULL , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . tls_ctx_config . protocols , NULL , NULL , applyTlsCfg ) ,
createStringConfig ( " tls-ciphers " , NULL , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . tls_ctx_config . ciphers , NULL , NULL , applyTlsCfg ) ,
createStringConfig ( " tls-ciphersuites " , NULL , MODIFIABLE_CONFIG , EMPTY_STRING_IS_NULL , server . tls_ctx_config . ciphersuites , NULL , NULL , applyTlsCfg ) ,
2019-11-28 04:24:57 -05:00
# endif
2019-07-19 13:15:35 -04:00
2021-11-07 06:40:08 -05:00
/* Special configs */
2021-12-01 03:15:11 -05:00
createSpecialConfig ( " dir " , NULL , MODIFIABLE_CONFIG , setConfigDirOption , getConfigDirOption , rewriteConfigDirOption , NULL ) ,
createSpecialConfig ( " save " , NULL , MODIFIABLE_CONFIG | MULTI_ARG_CONFIG , setConfigSaveOption , getConfigSaveOption , rewriteConfigSaveOption , NULL ) ,
createSpecialConfig ( " client-output-buffer-limit " , NULL , MODIFIABLE_CONFIG | MULTI_ARG_CONFIG , setConfigClientOutputBufferLimitOption , getConfigClientOutputBufferLimitOption , rewriteConfigClientOutputBufferLimitOption , NULL ) ,
createSpecialConfig ( " oom-score-adj-values " , NULL , MODIFIABLE_CONFIG | MULTI_ARG_CONFIG , setConfigOOMScoreAdjValuesOption , getConfigOOMScoreAdjValuesOption , rewriteConfigOOMScoreAdjValuesOption , updateOOMScoreAdj ) ,
createSpecialConfig ( " notify-keyspace-events " , NULL , MODIFIABLE_CONFIG , setConfigNotifyKeyspaceEventsOption , getConfigNotifyKeyspaceEventsOption , rewriteConfigNotifyKeyspaceEventsOption , NULL ) ,
createSpecialConfig ( " bind " , NULL , MODIFIABLE_CONFIG | MULTI_ARG_CONFIG , setConfigBindOption , getConfigBindOption , rewriteConfigBindOption , applyBind ) ,
createSpecialConfig ( " replicaof " , " slaveof " , IMMUTABLE_CONFIG | MULTI_ARG_CONFIG , setConfigReplicaOfOption , getConfigReplicaOfOption , rewriteConfigReplicaOfOption , NULL ) ,
2021-11-07 06:40:08 -05:00
2019-07-19 13:15:35 -04:00
/* NULL Terminator */
{ NULL }
} ;
2013-05-09 18:15:18 -04:00
/*-----------------------------------------------------------------------------
Treat subcommands as commands (#9504)
## Intro
The purpose is to allow having different flags/ACL categories for
subcommands (Example: CONFIG GET is ok-loading but CONFIG SET isn't)
We create a small command table for every command that has subcommands
and each subcommand has its own flags, etc. (same as a "regular" command)
This commit also unites the Redis and the Sentinel command tables
## Affected commands
CONFIG
Used to have "admin ok-loading ok-stale no-script"
Changes:
1. Dropped "ok-loading" in all except GET (this doesn't change behavior since
there were checks in the code doing that)
XINFO
Used to have "read-only random"
Changes:
1. Dropped "random" in all except CONSUMERS
XGROUP
Used to have "write use-memory"
Changes:
1. Dropped "use-memory" in all except CREATE and CREATECONSUMER
COMMAND
No changes.
MEMORY
Used to have "random read-only"
Changes:
1. Dropped "random" in PURGE and USAGE
ACL
Used to have "admin no-script ok-loading ok-stale"
Changes:
1. Dropped "admin" in WHOAMI, GENPASS, and CAT
LATENCY
No changes.
MODULE
No changes.
SLOWLOG
Used to have "admin random ok-loading ok-stale"
Changes:
1. Dropped "random" in RESET
OBJECT
Used to have "read-only random"
Changes:
1. Dropped "random" in ENCODING and REFCOUNT
SCRIPT
Used to have "may-replicate no-script"
Changes:
1. Dropped "may-replicate" in all except FLUSH and LOAD
CLIENT
Used to have "admin no-script random ok-loading ok-stale"
Changes:
1. Dropped "random" in all except INFO and LIST
2. Dropped "admin" in ID, TRACKING, CACHING, GETREDIR, INFO, SETNAME, GETNAME, and REPLY
STRALGO
No changes.
PUBSUB
No changes.
CLUSTER
Changes:
1. Dropped "admin in countkeysinslots, getkeysinslot, info, nodes, keyslot, myid, and slots
SENTINEL
No changes.
(note that DEBUG also fits, but we decided not to convert it since it's for
debugging and anyway undocumented)
## New sub-command
This commit adds another element to the per-command output of COMMAND,
describing the list of subcommands, if any (in the same structure as "regular" commands)
Also, it adds a new subcommand:
```
COMMAND LIST [FILTERBY (MODULE <module-name>|ACLCAT <cat>|PATTERN <pattern>)]
```
which returns a set of all commands (unless filters), but excluding subcommands.
## Module API
A new module API, RM_CreateSubcommand, was added, in order to allow
module writer to define subcommands
## ACL changes:
1. Now, that each subcommand is actually a command, each has its own ACL id.
2. The old mechanism of allowed_subcommands is redundant
(blocking/allowing a subcommand is the same as blocking/allowing a regular command),
but we had to keep it, to support the widespread usage of allowed_subcommands
to block commands with certain args, that aren't subcommands (e.g. "-select +select|0").
3. I have renamed allowed_subcommands to allowed_firstargs to emphasize the difference.
4. Because subcommands are commands in ACL too, you can now use "-" to block subcommands
(e.g. "+client -client|kill"), which wasn't possible in the past.
5. It is also possible to use the allowed_firstargs mechanism with subcommand.
For example: `+config -config|set +config|set|loglevel` will block all CONFIG SET except
for setting the log level.
6. All of the ACL changes above required some amount of refactoring.
## Misc
1. There are two approaches: Either each subcommand has its own function or all
subcommands use the same function, determining what to do according to argv[0].
For now, I took the former approaches only with CONFIG and COMMAND,
while other commands use the latter approach (for smaller blamelog diff).
2. Deleted memoryGetKeys: It is no longer needed because MEMORY USAGE now uses the "range" key spec.
4. Bugfix: GETNAME was missing from CLIENT's help message.
5. Sentinel and Redis now use the same table, with the same function pointer.
Some commands have a different implementation in Sentinel, so we redirect
them (these are ROLE, PUBLISH, and INFO).
6. Command stats now show the stats per subcommand (e.g. instead of stats just
for "config" you will have stats for "config|set", "config|get", etc.)
7. It is now possible to use COMMAND directly on subcommands:
COMMAND INFO CONFIG|GET (The pipeline syntax was inspired from ACL, and
can be used in functions lookupCommandBySds and lookupCommandByCString)
8. STRALGO is now a container command (has "help")
## Breaking changes:
1. Command stats now show the stats per subcommand (see (5) above)
2021-10-20 04:52:57 -04:00
* CONFIG HELP
2013-05-09 18:15:18 -04:00
* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */
Treat subcommands as commands (#9504)
## Intro
The purpose is to allow having different flags/ACL categories for
subcommands (Example: CONFIG GET is ok-loading but CONFIG SET isn't)
We create a small command table for every command that has subcommands
and each subcommand has its own flags, etc. (same as a "regular" command)
This commit also unites the Redis and the Sentinel command tables
## Affected commands
CONFIG
Used to have "admin ok-loading ok-stale no-script"
Changes:
1. Dropped "ok-loading" in all except GET (this doesn't change behavior since
there were checks in the code doing that)
XINFO
Used to have "read-only random"
Changes:
1. Dropped "random" in all except CONSUMERS
XGROUP
Used to have "write use-memory"
Changes:
1. Dropped "use-memory" in all except CREATE and CREATECONSUMER
COMMAND
No changes.
MEMORY
Used to have "random read-only"
Changes:
1. Dropped "random" in PURGE and USAGE
ACL
Used to have "admin no-script ok-loading ok-stale"
Changes:
1. Dropped "admin" in WHOAMI, GENPASS, and CAT
LATENCY
No changes.
MODULE
No changes.
SLOWLOG
Used to have "admin random ok-loading ok-stale"
Changes:
1. Dropped "random" in RESET
OBJECT
Used to have "read-only random"
Changes:
1. Dropped "random" in ENCODING and REFCOUNT
SCRIPT
Used to have "may-replicate no-script"
Changes:
1. Dropped "may-replicate" in all except FLUSH and LOAD
CLIENT
Used to have "admin no-script random ok-loading ok-stale"
Changes:
1. Dropped "random" in all except INFO and LIST
2. Dropped "admin" in ID, TRACKING, CACHING, GETREDIR, INFO, SETNAME, GETNAME, and REPLY
STRALGO
No changes.
PUBSUB
No changes.
CLUSTER
Changes:
1. Dropped "admin in countkeysinslots, getkeysinslot, info, nodes, keyslot, myid, and slots
SENTINEL
No changes.
(note that DEBUG also fits, but we decided not to convert it since it's for
debugging and anyway undocumented)
## New sub-command
This commit adds another element to the per-command output of COMMAND,
describing the list of subcommands, if any (in the same structure as "regular" commands)
Also, it adds a new subcommand:
```
COMMAND LIST [FILTERBY (MODULE <module-name>|ACLCAT <cat>|PATTERN <pattern>)]
```
which returns a set of all commands (unless filters), but excluding subcommands.
## Module API
A new module API, RM_CreateSubcommand, was added, in order to allow
module writer to define subcommands
## ACL changes:
1. Now, that each subcommand is actually a command, each has its own ACL id.
2. The old mechanism of allowed_subcommands is redundant
(blocking/allowing a subcommand is the same as blocking/allowing a regular command),
but we had to keep it, to support the widespread usage of allowed_subcommands
to block commands with certain args, that aren't subcommands (e.g. "-select +select|0").
3. I have renamed allowed_subcommands to allowed_firstargs to emphasize the difference.
4. Because subcommands are commands in ACL too, you can now use "-" to block subcommands
(e.g. "+client -client|kill"), which wasn't possible in the past.
5. It is also possible to use the allowed_firstargs mechanism with subcommand.
For example: `+config -config|set +config|set|loglevel` will block all CONFIG SET except
for setting the log level.
6. All of the ACL changes above required some amount of refactoring.
## Misc
1. There are two approaches: Either each subcommand has its own function or all
subcommands use the same function, determining what to do according to argv[0].
For now, I took the former approaches only with CONFIG and COMMAND,
while other commands use the latter approach (for smaller blamelog diff).
2. Deleted memoryGetKeys: It is no longer needed because MEMORY USAGE now uses the "range" key spec.
4. Bugfix: GETNAME was missing from CLIENT's help message.
5. Sentinel and Redis now use the same table, with the same function pointer.
Some commands have a different implementation in Sentinel, so we redirect
them (these are ROLE, PUBLISH, and INFO).
6. Command stats now show the stats per subcommand (e.g. instead of stats just
for "config" you will have stats for "config|set", "config|get", etc.)
7. It is now possible to use COMMAND directly on subcommands:
COMMAND INFO CONFIG|GET (The pipeline syntax was inspired from ACL, and
can be used in functions lookupCommandBySds and lookupCommandByCString)
8. STRALGO is now a container command (has "help")
## Breaking changes:
1. Command stats now show the stats per subcommand (see (5) above)
2021-10-20 04:52:57 -04:00
void configHelpCommand ( client * c ) {
const char * help [ ] = {
2021-01-04 10:02:57 -05:00
" GET <pattern> " ,
" Return parameters matching the glob-like <pattern> and their values. " ,
" SET <directive> <value> " ,
" Set the configuration <directive> to <value>. " ,
" RESETSTAT " ,
" Reset statistics reported by the INFO command. " ,
" REWRITE " ,
" Rewrite the configuration file. " ,
2017-12-06 06:05:11 -05:00
NULL
Treat subcommands as commands (#9504)
## Intro
The purpose is to allow having different flags/ACL categories for
subcommands (Example: CONFIG GET is ok-loading but CONFIG SET isn't)
We create a small command table for every command that has subcommands
and each subcommand has its own flags, etc. (same as a "regular" command)
This commit also unites the Redis and the Sentinel command tables
## Affected commands
CONFIG
Used to have "admin ok-loading ok-stale no-script"
Changes:
1. Dropped "ok-loading" in all except GET (this doesn't change behavior since
there were checks in the code doing that)
XINFO
Used to have "read-only random"
Changes:
1. Dropped "random" in all except CONSUMERS
XGROUP
Used to have "write use-memory"
Changes:
1. Dropped "use-memory" in all except CREATE and CREATECONSUMER
COMMAND
No changes.
MEMORY
Used to have "random read-only"
Changes:
1. Dropped "random" in PURGE and USAGE
ACL
Used to have "admin no-script ok-loading ok-stale"
Changes:
1. Dropped "admin" in WHOAMI, GENPASS, and CAT
LATENCY
No changes.
MODULE
No changes.
SLOWLOG
Used to have "admin random ok-loading ok-stale"
Changes:
1. Dropped "random" in RESET
OBJECT
Used to have "read-only random"
Changes:
1. Dropped "random" in ENCODING and REFCOUNT
SCRIPT
Used to have "may-replicate no-script"
Changes:
1. Dropped "may-replicate" in all except FLUSH and LOAD
CLIENT
Used to have "admin no-script random ok-loading ok-stale"
Changes:
1. Dropped "random" in all except INFO and LIST
2. Dropped "admin" in ID, TRACKING, CACHING, GETREDIR, INFO, SETNAME, GETNAME, and REPLY
STRALGO
No changes.
PUBSUB
No changes.
CLUSTER
Changes:
1. Dropped "admin in countkeysinslots, getkeysinslot, info, nodes, keyslot, myid, and slots
SENTINEL
No changes.
(note that DEBUG also fits, but we decided not to convert it since it's for
debugging and anyway undocumented)
## New sub-command
This commit adds another element to the per-command output of COMMAND,
describing the list of subcommands, if any (in the same structure as "regular" commands)
Also, it adds a new subcommand:
```
COMMAND LIST [FILTERBY (MODULE <module-name>|ACLCAT <cat>|PATTERN <pattern>)]
```
which returns a set of all commands (unless filters), but excluding subcommands.
## Module API
A new module API, RM_CreateSubcommand, was added, in order to allow
module writer to define subcommands
## ACL changes:
1. Now, that each subcommand is actually a command, each has its own ACL id.
2. The old mechanism of allowed_subcommands is redundant
(blocking/allowing a subcommand is the same as blocking/allowing a regular command),
but we had to keep it, to support the widespread usage of allowed_subcommands
to block commands with certain args, that aren't subcommands (e.g. "-select +select|0").
3. I have renamed allowed_subcommands to allowed_firstargs to emphasize the difference.
4. Because subcommands are commands in ACL too, you can now use "-" to block subcommands
(e.g. "+client -client|kill"), which wasn't possible in the past.
5. It is also possible to use the allowed_firstargs mechanism with subcommand.
For example: `+config -config|set +config|set|loglevel` will block all CONFIG SET except
for setting the log level.
6. All of the ACL changes above required some amount of refactoring.
## Misc
1. There are two approaches: Either each subcommand has its own function or all
subcommands use the same function, determining what to do according to argv[0].
For now, I took the former approaches only with CONFIG and COMMAND,
while other commands use the latter approach (for smaller blamelog diff).
2. Deleted memoryGetKeys: It is no longer needed because MEMORY USAGE now uses the "range" key spec.
4. Bugfix: GETNAME was missing from CLIENT's help message.
5. Sentinel and Redis now use the same table, with the same function pointer.
Some commands have a different implementation in Sentinel, so we redirect
them (these are ROLE, PUBLISH, and INFO).
6. Command stats now show the stats per subcommand (e.g. instead of stats just
for "config" you will have stats for "config|set", "config|get", etc.)
7. It is now possible to use COMMAND directly on subcommands:
COMMAND INFO CONFIG|GET (The pipeline syntax was inspired from ACL, and
can be used in functions lookupCommandBySds and lookupCommandByCString)
8. STRALGO is now a container command (has "help")
## Breaking changes:
1. Command stats now show the stats per subcommand (see (5) above)
2021-10-20 04:52:57 -04:00
} ;
addReplyHelp ( c , help ) ;
}
/*-----------------------------------------------------------------------------
* CONFIG RESETSTAT
* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */
void configResetStatCommand ( client * c ) {
resetServerStats ( ) ;
resetCommandTableStats ( server . commands ) ;
resetErrorTableStats ( ) ;
addReply ( c , shared . ok ) ;
}
/*-----------------------------------------------------------------------------
* CONFIG REWRITE
* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */
void configRewriteCommand ( client * c ) {
if ( server . configfile = = NULL ) {
addReplyError ( c , " The server is running without a config file " ) ;
2017-12-03 12:34:31 -05:00
return ;
2010-06-21 18:07:48 -04:00
}
Treat subcommands as commands (#9504)
## Intro
The purpose is to allow having different flags/ACL categories for
subcommands (Example: CONFIG GET is ok-loading but CONFIG SET isn't)
We create a small command table for every command that has subcommands
and each subcommand has its own flags, etc. (same as a "regular" command)
This commit also unites the Redis and the Sentinel command tables
## Affected commands
CONFIG
Used to have "admin ok-loading ok-stale no-script"
Changes:
1. Dropped "ok-loading" in all except GET (this doesn't change behavior since
there were checks in the code doing that)
XINFO
Used to have "read-only random"
Changes:
1. Dropped "random" in all except CONSUMERS
XGROUP
Used to have "write use-memory"
Changes:
1. Dropped "use-memory" in all except CREATE and CREATECONSUMER
COMMAND
No changes.
MEMORY
Used to have "random read-only"
Changes:
1. Dropped "random" in PURGE and USAGE
ACL
Used to have "admin no-script ok-loading ok-stale"
Changes:
1. Dropped "admin" in WHOAMI, GENPASS, and CAT
LATENCY
No changes.
MODULE
No changes.
SLOWLOG
Used to have "admin random ok-loading ok-stale"
Changes:
1. Dropped "random" in RESET
OBJECT
Used to have "read-only random"
Changes:
1. Dropped "random" in ENCODING and REFCOUNT
SCRIPT
Used to have "may-replicate no-script"
Changes:
1. Dropped "may-replicate" in all except FLUSH and LOAD
CLIENT
Used to have "admin no-script random ok-loading ok-stale"
Changes:
1. Dropped "random" in all except INFO and LIST
2. Dropped "admin" in ID, TRACKING, CACHING, GETREDIR, INFO, SETNAME, GETNAME, and REPLY
STRALGO
No changes.
PUBSUB
No changes.
CLUSTER
Changes:
1. Dropped "admin in countkeysinslots, getkeysinslot, info, nodes, keyslot, myid, and slots
SENTINEL
No changes.
(note that DEBUG also fits, but we decided not to convert it since it's for
debugging and anyway undocumented)
## New sub-command
This commit adds another element to the per-command output of COMMAND,
describing the list of subcommands, if any (in the same structure as "regular" commands)
Also, it adds a new subcommand:
```
COMMAND LIST [FILTERBY (MODULE <module-name>|ACLCAT <cat>|PATTERN <pattern>)]
```
which returns a set of all commands (unless filters), but excluding subcommands.
## Module API
A new module API, RM_CreateSubcommand, was added, in order to allow
module writer to define subcommands
## ACL changes:
1. Now, that each subcommand is actually a command, each has its own ACL id.
2. The old mechanism of allowed_subcommands is redundant
(blocking/allowing a subcommand is the same as blocking/allowing a regular command),
but we had to keep it, to support the widespread usage of allowed_subcommands
to block commands with certain args, that aren't subcommands (e.g. "-select +select|0").
3. I have renamed allowed_subcommands to allowed_firstargs to emphasize the difference.
4. Because subcommands are commands in ACL too, you can now use "-" to block subcommands
(e.g. "+client -client|kill"), which wasn't possible in the past.
5. It is also possible to use the allowed_firstargs mechanism with subcommand.
For example: `+config -config|set +config|set|loglevel` will block all CONFIG SET except
for setting the log level.
6. All of the ACL changes above required some amount of refactoring.
## Misc
1. There are two approaches: Either each subcommand has its own function or all
subcommands use the same function, determining what to do according to argv[0].
For now, I took the former approaches only with CONFIG and COMMAND,
while other commands use the latter approach (for smaller blamelog diff).
2. Deleted memoryGetKeys: It is no longer needed because MEMORY USAGE now uses the "range" key spec.
4. Bugfix: GETNAME was missing from CLIENT's help message.
5. Sentinel and Redis now use the same table, with the same function pointer.
Some commands have a different implementation in Sentinel, so we redirect
them (these are ROLE, PUBLISH, and INFO).
6. Command stats now show the stats per subcommand (e.g. instead of stats just
for "config" you will have stats for "config|set", "config|get", etc.)
7. It is now possible to use COMMAND directly on subcommands:
COMMAND INFO CONFIG|GET (The pipeline syntax was inspired from ACL, and
can be used in functions lookupCommandBySds and lookupCommandByCString)
8. STRALGO is now a container command (has "help")
## Breaking changes:
1. Command stats now show the stats per subcommand (see (5) above)
2021-10-20 04:52:57 -04:00
if ( rewriteConfig ( server . configfile , 0 ) = = - 1 ) {
serverLog ( LL_WARNING , " CONFIG REWRITE failed: %s " , strerror ( errno ) ) ;
addReplyErrorFormat ( c , " Rewriting config file: %s " , strerror ( errno ) ) ;
} else {
serverLog ( LL_WARNING , " CONFIG REWRITE executed with success. " ) ;
addReply ( c , shared . ok ) ;
}
2010-06-21 18:07:48 -04:00
}