Commit Graph

6573 Commits

Author SHA1 Message Date
Salvatore Sanfilippo
7e64aa80d8
Merge pull request #6690 from yangbodong22011/fix-spop-return-nil
Fix spop return nil #4709
2020-02-27 17:58:49 +01:00
antirez
9abdc089f8 Fix SDS misuse in enumConfigSet(). Related to #6778. 2020-02-27 17:51:30 +01:00
antirez
43f39f2e37 Remove useless comment from enumConfigSet(). 2020-02-27 17:45:56 +01:00
Salvatore Sanfilippo
ec73881a4d
Merge pull request #6778 from pponnuvel/fix_possible_overflow
Fix a potential overflow with strncpy
2020-02-27 17:45:02 +01:00
antirez
d8ab7e8012 Improve aeDeleteEventLoop() top comment grammar. 2020-02-27 17:41:48 +01:00
Salvatore Sanfilippo
e77db59660
Merge pull request #6189 from ShooterIT/del-event-loop
Del event loop
2020-02-27 17:40:51 +01:00
Salvatore Sanfilippo
652c532292
Merge pull request #6785 from patpatbear/issue_#6696_fix_aof_child_whitelist_SIGUSR1
fix impl of aof-child whitelist SIGUSR1 feature.
2020-02-27 17:39:33 +01:00
Salvatore Sanfilippo
ca825701dc
Merge pull request #6876 from MeirShpilraien/changed_log_level_of_module_fork_api
Changed log level for module fork api from 'notice' to 'verbos'.
2020-02-27 10:26:55 +01:00
Salvatore Sanfilippo
f7e05dc755
Merge pull request #6783 from hwware/memoryleakfix
Fix Memory Leaks in debug.c
2020-02-27 10:12:07 +01:00
Salvatore Sanfilippo
4f29753726
Merge pull request #6931 from t00350320/unstable
modify the read buf size according to the write buf size PROTO_IOBUF_LEN
2020-02-27 10:02:24 +01:00
Salvatore Sanfilippo
37612118c8
Merge pull request #6919 from ashtul/Fix-ThreadSafeContext-functions-names
fix ThreadSafeContext lock/unlock function names
2020-02-27 10:01:48 +01:00
Salvatore Sanfilippo
049969971e
Merge pull request #6921 from guybe7/fix_xclaim_propagate
XREADGROUP should propagate XCALIM/SETID in MULTI/EXEC
2020-02-27 09:59:51 +01:00
Salvatore Sanfilippo
79de8f3c25
Merge pull request #6922 from guybe7/refix_blocked_module_memleak
Modules: Do not auto-unblock clients if not blocked on keys
2020-02-27 09:53:52 +01:00
Hengjian Tang
1ab5508324 modify the read buf size according to the write buf size PROTO_IOBUF_LEN defined before 2020-02-25 15:55:28 +08:00
antirez
b6378edcd6 Tracking: optin/out implemented. 2020-02-24 19:09:45 +01:00
Guy Benoish
376a806bfb Modules: Do not auto-unblock clients if not blocked on keys 2020-02-23 19:36:46 +05:30
Guy Benoish
a6c7e61e61 XREADGROUP should propagate XCALIM/SETID in MULTI/EXEC
Use built-in alsoPropagate mechanism that wraps commands
in MULTI/EXEC before sending them to replica/AOF
2020-02-23 19:13:15 +05:30
Ariel
28bde89172 fix ThreadSafeContext lock/unlock function names 2020-02-22 23:49:23 +02:00
hwware
743cfc0ad6 add missing file marco 2020-02-22 11:38:51 -05:00
chendianqiang
9947956d7b use correct list for moduleUnregisterUsedAPI 2020-02-22 15:03:01 +08:00
Salvatore Sanfilippo
ab8c383082
Merge pull request #6901 from hwware/debughelp
add missing subcommand description for DEBUG OOM in HELP
2020-02-20 17:42:24 +01:00
Salvatore Sanfilippo
155526a44e
Merge pull request #6894 from madolson/csc-fixes
Client side caching fixes
2020-02-20 13:03:31 +01:00
Salvatore Sanfilippo
e741b0c257
Merge pull request #6903 from oranagra/defrag_lists
Defrag big lists in portions to avoid latency and freeze
2020-02-20 13:00:39 +01:00
Salvatore Sanfilippo
9fe02ce1cd
Merge pull request #6905 from guybe7/xgroup_destroy_unblock
XGROUP DESTROY should unblock XREADGROUP with -NOGROUP
2020-02-20 12:54:14 +01:00
Salvatore Sanfilippo
42c4d6aec7
Merge pull request #6911 from hayashier/fix-typo-fss
fix typo from fss to rss
2020-02-20 12:02:05 +01:00
antirez
c8c78bd4b9 Tracking: fix max-keys configuration directive. 2020-02-19 19:00:29 +01:00
Guy Benoish
770cb0ba97 XGROUP DESTROY should unblock XREADGROUP with -NOGROUP 2020-02-19 08:25:31 +05:30
Oran Agra
485425cec7 Defrag big lists in portions to avoid latency and freeze
When active defrag kicks in and finds a big list, it will create a bookmark to
a node so that it is able to resume iteration from that node later.

The quicklist manages that bookmark, and updates it in case that node is deleted.

This will increase memory usage only on lists of over 1000 (see
active-defrag-max-scan-fields) quicklist nodes (1000 ziplists, not 1000 items)
by 16 bytes.

In 32 bit build, this change reduces the maximum effective config of
list-compress-depth and list-max-ziplist-size (from 32767 to 8191)
2020-02-18 17:22:32 +02:00
hwware
7d4ebe114f add missing subcommand description for debug oom 2020-02-17 23:40:24 -05:00
Oran Agra
6fff2cf9b6 module api docs for aux_save and aux_load 2020-02-16 15:43:19 +02:00
Madelyn Olson
c1b5220ee7 Give an error message if you specify redirect twice 2020-02-16 05:41:39 -08:00
Madelyn Olson
d63a435362 Minor CSC fixes and fixed documentation 2020-02-16 05:28:29 -08:00
antirez
df45fed050 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2020-02-14 18:23:13 +01:00
antirez
20eeddfb8a Signal key as modified when expired on-access.
This fixes WATCH and client side caching with keys expiring because of
a synchronous access and not because of background expiring.
2020-02-14 18:22:25 +01:00
Itamar Haber
8a44b2cc7e Fixes segfault on calling trackingGetTotalKeys
... with CSC disabled
2020-02-14 17:13:58 +02:00
antirez
090bc0c1a3 Merge branch 'csc2' into unstable 2020-02-14 14:29:18 +01:00
antirez
47177c9edc Tracking: fix operators precedence error in bcast check. 2020-02-14 14:27:45 +01:00
antirez
f6e32a832f Tracking: fix behavior when switchinig from normal to BCAST. 2020-02-14 14:27:43 +01:00
Salvatore Sanfilippo
c21c23bbba
Merge pull request #6863 from oranagra/module_commands_no_slowlog
add no-slowlog option to RM_CreateCommand
2020-02-13 17:02:19 +01:00
antirez
6922ccc0b9 Tracking: fix sending messages bug + tracking off bug. 2020-02-13 16:58:07 +01:00
antirez
40194a2a68 Tracking: BCAST: basic feature now works. 2020-02-12 19:22:04 +01:00
Salvatore Sanfilippo
8aa0b19d83
Merge pull request #6691 from kraj/fno-common
Mark extern definition of SDS_NOINIT in sds.h
2020-02-12 14:04:53 +01:00
lifubang
dc8f947d7c correct help info for --user and --pass
Signed-off-by: lifubang <lifubang@acmcoder.com>
2020-02-12 16:34:22 +08:00
antirez
71f3f3f1af Tracking: BCAST: broadcasting of keys in prefixes implemented. 2020-02-11 18:11:59 +01:00
antirez
3f7ba86255 Tracking: BCAST: registration in the prefix table. 2020-02-11 17:26:29 +01:00
antirez
dfe126f3e9 Tracking: BCAST: parsing of the options + skeleton. 2020-02-10 17:18:11 +01:00
antirez
f53cc00c09 Tracking: always reply with an array of keys. 2020-02-10 13:42:18 +01:00
meir@redislabs.com
67dbc5f365 Changed log level for module fork api from 'notice' to 'verbos'. 2020-02-10 12:10:32 +02:00
antirez
85e4777d5c Tracking: minor change of names and new INFO field. 2020-02-07 18:12:45 +01:00
antirez
1ea6672430 Rax.c: populate data field after random walk. 2020-02-07 18:12:10 +01:00
antirez
d933d6f2a4 Tracking: rename INFO field with total items. 2020-02-07 17:19:17 +01:00
antirez
92357b2d61 Tracking: first conversion from hashing to key names. 2020-02-07 14:03:43 +01:00
Oran Agra
46216b0e83 add no-slowlog option to RM_CreateCommand 2020-02-06 15:06:33 +02:00
Oran Agra
c8c3281f06 add no_auth to COMMAND INFO 2020-02-06 14:53:54 +02:00
Salvatore Sanfilippo
9c00bdd86e
Merge pull request #6822 from guybe7/diskless_load_module_hook_fix
Diskless-load emptyDb-related fixes
2020-02-06 13:10:00 +01:00
Guy Benoish
92dc5e1fa4 Diskless-load emptyDb-related fixes
1. Call emptyDb even in case of diskless-load: We want modules
   to get the same FLUSHDB event as disk-based replication.
2. Do not fire any module events when flushing the backups array.
3. Delete redundant call to signalFlushedDb (Called from emptyDb).
2020-02-06 16:48:02 +05:30
antirez
50d4326e3b Merge branch 'unstable' of github.com:/antirez/redis into unstable 2020-02-06 11:24:22 +01:00
antirez
d5c6a833c8 Merge branch 'acl-log' into unstable 2020-02-06 11:24:16 +01:00
Salvatore Sanfilippo
1012514e0f
Merge pull request #6826 from lifubang/opensslcli
fix ssl args check for redis-cli
2020-02-06 11:22:20 +01:00
Oran Agra
28ef18a894 RM_Scan disable dict rehashing
The callback approach we took is very efficient, the module can do any
filtering of keys without building any list and cloning strings, it can
also read data from the key's value. but if the user tries to re-open
the key, or any other key, this can cause dict re-hashing (dictFind does
that), and that's very bad to do from inside dictScan.

this commit protects the dict from doing any rehashing during scan, but
also warns the user not to attempt any writes or command calls from
within the callback, for fear of unexpected side effects and crashes.
2020-02-06 11:48:12 +02:00
Salvatore Sanfilippo
5bfba06a3b
Merge pull request #6821 from guybe7/key_miss_notify
Exclude "keymiss" notification from NOTIFY_ALL
2020-02-06 10:43:05 +01:00
Salvatore Sanfilippo
2e1dd00c80
Merge pull request #6837 from oranagra/signal_modified_key_doc
update RM_SignalModifiedKey doc comment
2020-02-06 10:38:36 +01:00
Salvatore Sanfilippo
730cacf672
Merge pull request #6840 from oranagra/short_read_moduleid
Add handling of short read of module id in rdb
2020-02-06 10:35:42 +01:00
Salvatore Sanfilippo
33f613bf87
Merge pull request #6843 from oranagra/command_flags
A few non-data commands that should be allowed while loading or stale
2020-02-06 10:34:26 +01:00
Salvatore Sanfilippo
4aafdb185a
Merge pull request #6844 from oranagra/bind_config_leak
Memory leak when bind config is provided twice
2020-02-06 10:33:40 +01:00
Salvatore Sanfilippo
be520829b8
Merge pull request #6845 from oranagra/maxmemory_warning
fix maxmemory config warning
2020-02-06 10:33:15 +01:00
Salvatore Sanfilippo
13741fb99d
Merge pull request #6846 from oranagra/module_client_flags
Fix client flags to be int64 in module.c
2020-02-06 10:32:35 +01:00
Salvatore Sanfilippo
08b218bfa5
Merge pull request #6847 from oranagra/module_read_err_panic
moduleRDBLoadError, add key name, and use panic rather than exit
2020-02-06 10:31:29 +01:00
Salvatore Sanfilippo
5558c0e4cf
Merge pull request #6848 from oranagra/opt_use_diskless_load_calls
reduce repeated calls to use_diskless_load
2020-02-06 10:30:39 +01:00
Salvatore Sanfilippo
7cf53252ee
Merge pull request #6849 from oranagra/free_client_mutex
freeClientAsync don't lock mutex if there's just one thread
2020-02-06 10:29:46 +01:00
Salvatore Sanfilippo
44266d6d92
Merge pull request #6850 from oranagra/restart_aof_unset_master
move restartAOFAfterSYNC from replicaofCommand to replicationUnsetMaster
2020-02-06 10:27:05 +01:00
Salvatore Sanfilippo
58a3a3fa68
Merge pull request #6851 from oranagra/aof_rewrite_scheduled
stopAppendOnly resets aof_rewrite_scheduled
2020-02-06 10:23:11 +01:00
Salvatore Sanfilippo
711b13801e
Merge pull request #6852 from oranagra/acl_help
add SAVE subcommand to ACL HELP and top comment
2020-02-06 10:22:16 +01:00
Salvatore Sanfilippo
f25962337d
Merge pull request #6853 from oranagra/debug_help
DEBUG HELP - add PROTOCOL
2020-02-06 10:21:55 +01:00
Guy Benoish
91c41b6dde Some refactroing using getClientType instead of CLIENT_SLAVE 2020-02-06 14:12:08 +05:30
Oran Agra
31ffbf1133 DEBUG HELP - add PROTOCOL 2020-02-06 10:40:29 +02:00
Guy Benoish
e33fffbde1 Fix small bugs related to replica and monitor ambiguity
1. server.repl_no_slaves_since can be set when a MONITOR client disconnects
2. c->repl_ack_time can be set by a newline from a MONITOR client
3. Improved comments
2020-02-06 14:10:02 +05:30
Oran Agra
6d29c34da7 add SAVE subcommand to ACL HELP and top comment 2020-02-06 10:31:43 +02:00
Oran Agra
69e8ea7143 stopAppendOnly resets aof_rewrite_scheduled
althouh in theory, users can do BGREWRITEAOF even if aof is disabled, i
suppose it is more common that the scheduled flag is set by either
startAppendOnly, of a failed initial AOFRW fork (AOF_WAIT_REWRITE)
2020-02-06 10:17:34 +02:00
Oran Agra
aac6a4cf13 move restartAOFAfterSYNC from replicaofCommand to replicationUnsetMaster
replicationUnsetMaster can be called from other places, not just
replicaofCOmmand, and all of these need to restart AOF
2020-02-06 10:14:32 +02:00
Oran Agra
86e302f5f3 freeClientAsync don't lock mutex if there's just one thread 2020-02-06 10:07:17 +02:00
Oran Agra
485d5d4a18 reduce repeated calls to use_diskless_load
this function possibly iterates on the module list
2020-02-06 09:41:45 +02:00
Oran Agra
85cc696f50 moduleRDBLoadError, add key name, and use panic rather than exit
using panic rather than exit means you get s stack trace of the code
path that experianced the error, and possibly other info.
2020-02-06 09:37:04 +02:00
Oran Agra
d454d5a4f5 Fix client flags to be int64 in module.c
currently there's no bug since the flags these functions handle are
always lower than 32bit, but still better fix the type to prevent future
bugs.
2020-02-06 09:33:20 +02:00
Oran Agra
a17bddf2a1 fix maxmemory config warning
the warning condition was if usage > limit (saying it'll cause eviction
or oom), but in fact the eviction and oom depends on used minus slave
buffers.

other than fixing the condition, i add info about the current usage and
limit, which may be useful when looking at the log.
2020-02-06 09:23:22 +02:00
Oran Agra
7e1d5954e5 Memory leak when bind config is provided twice 2020-02-06 09:17:39 +02:00
Oran Agra
ac2c96f5b1 A few non-data commands that should be allowed while loading or stale
SELECT, and HELLO are commands that may be executed by the client
as soon as it connects, there's no reason to block them, preventing the
client from doing the rest of his sequence (which might just be INFO or
CONFIG, etc).

MONITOR, DEBUG, SLOWLOG, TIME, LASTSAVE are all non-data accessing
commands, which there's no reason to block.
2020-02-06 08:53:23 +02:00
Oran Agra
fe7e8dc955 Add handling of short read of module id in rdb 2020-02-05 19:47:09 +02:00
Oran Agra
3795aaf42a update RM_SignalModifiedKey doc comment 2020-02-05 18:15:38 +02:00
Oran Agra
774d8cd721 Optimize temporary memory allocations for getKeysFromCommand mechanism
now that we may use it more often (ACL), these excessive calls to malloc
and free can become an overhead.
2020-02-05 18:06:33 +02:00
Salvatore Sanfilippo
44ac202fbf
Merge pull request #6831 from oranagra/config_c_refactor_3
config.c verbose error replies for CONFIG SET, like config file parsing
2020-02-05 13:41:15 +01:00
Oran Agra
ffdd0620d0 config.c verbose error replies for CONFIG SET, like config file parsing
We noticed that the error replies for the generic mechanism for enums
are very verbose for config file parsing, but not for config set
command.

instead of replicating this code, i did a small refactoring to share
code between CONFIG SET and config file parsing.

and also renamed the enum group functions to be consistent with the
naming of other types.
2020-02-05 11:41:24 +02:00
Oran Agra
9ac6cb9ce4 memoryGetKeys helper function so that ACL can limit access to keys for MEMORY command 2020-02-05 09:42:49 +02:00
Guy Benoish
1c7a2269d6 Add RM_CreateStringFromDouble 2020-02-04 19:28:09 +05:30
antirez
90fae58b49 ACL LOG: make max log entries configurable. 2020-02-04 13:19:40 +01:00
antirez
0c1a4b5576 ACL LOG: log failed auth attempts. 2020-02-04 12:55:26 +01:00
lifubang
540b917a26 fix ssl flag check for redis-cli
Signed-off-by: lifubang <lifubang@acmcoder.com>
2020-02-04 17:32:30 +08:00
Salvatore Sanfilippo
cfe39b7859
Merge pull request #6818 from leomurillo/zskiplistLevels
Set ZSKIPLIST_MAXLEVEL to 32
2020-02-04 10:20:56 +01:00
WuYunlong
eb2196f5ce Fix lua related memory leak. 2020-02-04 16:38:46 +08:00
Salvatore Sanfilippo
2400f5782e
Merge pull request #6808 from yossigo/fix/redis-cli-openssl-1.0.x
TLS: Fix missing initialization in redis-cli.
2020-02-03 16:43:56 +01:00
Oran Agra
138ed120dd fix uninitialized info_cb var in module.c 2020-02-03 16:35:39 +02:00
Guy Benoish
2fda5f5c98 Exclude "keymiss" notification from NOTIFY_ALL
Because "keymiss" is "special" compared to the rest of
the notifications (Trying not to break existing apps
using the 'A' format for notifications)

Also updated redis.conf and module.c docs
2020-02-03 15:43:44 +05:30
Leo Murillo
560e1e6c4b Set ZSKIPLIST_MAXLEVEL to optimal value given 2^64 elements and p=0.25 2020-02-02 02:48:00 -06:00
Salvatore Sanfilippo
53ac8c7df2
Merge pull request #6812 from guybe7/str_convert_fix
ld2string should fail if string contains \0 in the middle
2020-01-30 17:51:04 +01:00
Guy Benoish
d72b7ed4fb DEBUG OBJECT should pass keyname to module when loading 2020-01-30 19:15:12 +05:30
Guy Benoish
2deb55512f ld2string should fail if string contains \0 in the middle
This bug affected RM_StringToLongDouble and HINCRBYFLOAT.
I added tests for both cases.

Main changes:
1. Fixed string2ld to fail if string contains \0 in the middle
2. Use string2ld in getLongDoubleFromObject - No point of
   having duplicated code here

The two changes above broke RM_SaveLongDouble/RM_LoadLongDouble
because the long double string was saved with length+1 (An innocent
mistake, but it's actually a bug - The length passed to
RM_SaveLongDouble should not include the last \0).
2020-01-30 18:15:17 +05:30
antirez
396161765b ACL LOG: also log ACL errors in the scripting/MULTI ctx. 2020-01-30 10:50:32 +01:00
Yossi Gottlieb
93edb3ff3a TLS: Fix missing initialization in redis-cli. 2020-01-29 21:40:02 +02:00
antirez
30a466ba38 ACL LOG: implement LOG RESET. 2020-01-29 18:51:04 +01:00
antirez
6671032faf ACL LOG: group similar entries in a given time delta. 2020-01-29 18:40:32 +01:00
antirez
17ff3173d6 Add more info in the unblockClientFromModule() function. 2020-01-29 12:47:50 +01:00
Salvatore Sanfilippo
bf53f9280a
Merge pull request #6699 from guybe7/module_blocked_on_key_timeout_memleak
Modules: Fix blocked-client-related memory leak
2020-01-29 12:06:14 +01:00
antirez
61dffd8669 ACL LOG: actually emit entries. 2020-01-28 18:04:20 +01:00
antirez
e8d0057710 ACL LOG: implement ACL LOG subcommadn skeleton. 2020-01-28 17:30:50 +01:00
antirez
3e9e27e98f ACL LOG: data structures and initial functions. 2020-01-27 18:37:52 +01:00
qetu3790
bd60c11bd8
Fix not used constant in lru_test_mode.
LRU_CYCLE_PERIOD is defined,but not used.
2020-01-23 17:18:07 +08:00
srzhao
38f6207f88 Check OOM at script start to get stable lua OOM state.
Checking OOM by `getMaxMemoryState` inside script might get different result
with `freeMemoryIfNeededAndSafe` at script start, because lua stack and
arguments also consume memory.

This leads to memory `borderline` when memory grows near server.maxmemory:

- `freeMemoryIfNeededAndSafe` at script start detects no OOM, no memory freed
- `getMaxMemoryState` inside script detects OOM, script aborted

We solve this 'borderline' issue by saving OOM state at script start to get
stable lua OOM state.

related to issue #6565 and #5250.
2020-01-20 21:34:05 +08:00
srzhao
dd05b7f67f fix impl of aof-child whitelist SIGUSR1 feature. 2020-01-17 11:50:51 +08:00
hwware
67ee87522a format fix 2020-01-16 17:35:26 -05:00
hwware
3f6d00d387 fix potentical memory leaks 2020-01-16 17:33:23 -05:00
antirez
7ef2270ee7 Change error message for #6775. 2020-01-15 17:55:24 +01:00
Salvatore Sanfilippo
456bbc02c8
Merge pull request #6775 from SqREL/make-error-of-invalid-command-withing-context-explicit
Make error when submitting command in incorrect context more explicit
2020-01-15 17:53:51 +01:00
Ponnuvel Palaniyappan
1d4ea00d12 Fix a potential overflow with strncpy 2020-01-14 08:10:39 +00:00
antirez
7d5fb5df3f Setting N I/O threads should mean N-1 additional + 1 main thread. 2020-01-13 18:53:36 +01:00
antirez
28d146bed6 Simplify #6379 changes. 2020-01-13 13:25:37 +01:00
Salvatore Sanfilippo
9506c860f6
Merge pull request #6739 from trevor211/fixMemoryLeak
Fix potential memory leak of rioWriteBulkStreamID().
2020-01-13 13:23:49 +01:00
antirez
baa88a1c14 Jump to right label on AOF parsing error.
Related to #6054.
2020-01-13 13:16:15 +01:00
antirez
d28948b143 Port PR #6110 to new connection object code. 2020-01-13 12:54:39 +01:00
antirez
a8c912ead6 A few comments about main thread serving I/O as well.
Related to #6110.
2020-01-13 12:50:26 +01:00
Salvatore Sanfilippo
24896427fc
Merge pull request #6110 from soloestoy/enhance-io-threaded
Enhance IO Threaded: use main thread to handle read/write work
2020-01-13 12:46:40 +01:00
Vasyl Melnychuk
b5f52bf01c Make error when submitting command in incorrect context more explicit
So error message `ERR only (P)SUBSCRIBE / (P)UNSUBSCRIBE / PING / QUIT allowed in this context` will become
`ERR 'get' command submitted, but only (P)SUBSCRIBE / (P)UNSUBSCRIBE / PING / QUIT allowed in this context`
2020-01-10 23:34:15 +02:00
antirez
8105f91a02 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2020-01-10 13:15:36 +01:00
Salvatore Sanfilippo
b8b8dd5c15
Merge pull request #6114 from ShooterIT/async-rename-rdb
Rename rdb when replica finish receiving rdb asynchronously
2020-01-10 13:06:39 +01:00
antirez
9466dae4e5 Free fakeclient argv on AOF error.
We exit later, so no bug fixed, but it is more correct.

See #6054, thanks to @ShooterIT for finding the issue.
2020-01-10 13:02:45 +01:00
Salvatore Sanfilippo
51a01ca8a6
Merge pull request #6746 from leomurillo/unstable
Fix bug on KEYS command where pattern starts with * followed by \x00
2020-01-10 12:18:19 +01:00
Salvatore Sanfilippo
bb93686754
Merge pull request #6703 from guybe7/blocking_xread_empty_reply
Blocking XREAD[GROUP] should always reply with valid data (or timeout)
2020-01-09 17:32:14 +01:00
Salvatore Sanfilippo
64ef0ad71a
Merge pull request #6724 from JohnSully/incremental
Add support for incremental build with header files
2020-01-09 17:10:02 +01:00
Salvatore Sanfilippo
0f3c353dd3
Merge pull request #6715 from trevor211/fixPotentialClusterLinkError
Fix petential cluster link error.
2020-01-09 17:05:54 +01:00
Salvatore Sanfilippo
f616ecf361
Merge pull request #6698 from yossigo/ctx-flags-multi-dirty
Add REDISMODULE_CTX_FLAGS_MULTI_DIRTY.
2020-01-08 13:09:48 +01:00
Salvatore Sanfilippo
2ee4f9f34e
Merge pull request #6748 from hwware/typofix
typo fix in acl.c
2020-01-08 13:06:06 +01:00
Salvatore Sanfilippo
c89ec5af1e
Merge pull request #6735 from itamarhaber/io-threads
Adjusts 'io_threads_num' max in config.c
2020-01-08 12:50:13 +01:00
Salvatore Sanfilippo
0e38c5f39c
Merge pull request #6743 from yz1509/unstable
Sentinel: avoid sentinel changes promoted_slave to be its own replica.
2020-01-08 12:30:49 +01:00
antirez
7080a28efc XCLAIM: Create the consumer only on successful claims.
Fixes #6744.
2020-01-08 10:10:11 +01:00
hwware
fda2bf9c92 typo fix in acl.c 2020-01-07 21:09:44 -05:00
Leo Murillo
63e46e5f8d Fix bug on KEYS command where pattern starts with * followed by \x00 (null char). 2020-01-07 13:55:26 -06:00
WuYunlong
d6a13f4440 Free allocated sds in pfdebugCommand() to avoid memory leak. 2020-01-07 11:17:52 +08:00
yz1509
18c2676084 avoid sentinel changes promoted_slave to be its own replica. 2020-01-07 10:29:54 +08:00
WuYunlong
9387f7333e Fix potential memory leak of clusterLoadConfig(). 2020-01-07 10:28:36 +08:00
WuYunlong
2f8134a7ff Fix potential memory leak of rioWriteBulkStreamID(). 2020-01-06 19:58:13 +08:00
Itamar Haber
408e8e9f44 Adjusts 'io_threads_num' max to 128
Instead of 512, use the defined max from networking.c
2020-01-04 18:33:24 +02:00
antirez
0af467d18f Fix active expire division by zero.
Likely fix #6723.

This is what happens AFAIK: we enter the main loop where we expire stuff
until a given percentage of keys is still found to be logically expired.
There are however other potential exit conditions.

However the "sampled" variable is not always incremented inside the
loop, because we may found no valid slot as we scan the hash table, but
just NULLs ad dict entries. So when the do/while loop condition is
triggered at the end, we do (expired*100/sampled), dividing by zero if
we sampled 0 keys.
2020-01-01 18:13:13 +01:00
John Sully
e5565a793e Add support for incremental build with header files 2020-01-01 10:33:02 -05:00
ShooterIT
2bc8db9ca5 Rename rdb asynchronously 2019-12-31 21:45:32 +08:00
wangyuan21
3848849013 free time event when delete eventloop 2019-12-31 19:57:02 +08:00
WuYunlong
0992ada2fe Fix petential cluster link error.
Funcion adjustOpenFilesLimit() has an implicit parameter, which is server.maxclients.
This function aims to ajust maximum file descriptor number according to server.maxclients
by best effort, which is "bestlimit" could be lower than "maxfiles" but greater than "oldlimit".
When we try to increase "maxclients" using CONFIG SET command, we could increase maximum
file descriptor number to a bigger value without calling aeResizeSetSize the same time.
When later more and more clients connect to server, the allocated fd could be bigger and bigger,
and eventually exceeds events size of aeEventLoop.events. When new nodes joins the cluster,
new link is created, together with new fd, but when calling aeCreateFileEvent, we did not
check the return value. In this case, we have a non-null "link" but the associated fd is not
registered.

So when we dynamically set "maxclients" we could reach an inconsistency between maximum file
descriptor number of the process and server.maxclients. And later could cause cluster link and link
fd inconsistency.

While setting "maxclients" dynamically, we consider it as failed when resulting "maxclients" is not
the same as expected. We try to restore back the maximum file descriptor number when we failed to set
"maxclients" to the specified value, so that server.maxclients could act as a guard as before.
2019-12-31 18:16:30 +08:00
hayashier
a3b0d8631f fix typo from fss to rss 2019-12-31 17:46:48 +09:00
Guy Benoish
d7d13721d3 Modules: Fix blocked-client-related memory leak
If a blocked module client times-out (or disconnects, unblocked
by CLIENT command, etc.) we need to call moduleUnblockClient
in order to free memory allocated by the module sub-system
and blocked-client private data

Other changes:
Made blockedonkeys.tcl tests a bit more aggressive in order
to smoke-out potential memory leaks
2019-12-30 10:10:59 +05:30
Guy Benoish
a351e74fe9 Blocking XREAD[GROUP] should always reply with valid data (or timeout)
This commit solves the following bug:
127.0.0.1:6379> XGROUP CREATE x grp $ MKSTREAM
OK
127.0.0.1:6379> XADD x 666 f v
"666-0"
127.0.0.1:6379> XREADGROUP GROUP grp Alice BLOCK 0 STREAMS x >
1) 1) "x"
   2) 1) 1) "666-0"
         2) 1) "f"
            2) "v"
127.0.0.1:6379> XADD x 667 f v
"667-0"
127.0.0.1:6379> XDEL x 667
(integer) 1
127.0.0.1:6379> XREADGROUP GROUP grp Alice BLOCK 0 STREAMS x >
1) 1) "x"
   2) (empty array)

The root cause is that we use s->last_id in streamCompareID
while we should use the last *valid* ID
2019-12-30 10:06:01 +05:30
antirez
6e4f70b817 Fix duplicated CLIENT SETNAME reply.
Happened when we set the name to "" to cancel the name.
Was introduced during the RESP3 refactoring.

See #6036.
2019-12-29 15:44:59 +01:00
antirez
faea46710c Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-12-29 15:42:08 +01:00
antirez
5521910de7 Inline protocol: handle empty strings well.
This bug is from the first version of Redis. Probably the problem here
is that before we used an SDS split function that created empty strings
for additional spaces, like in "SET    foo          bar".
AFAIK later we replaced it with the curretn sdssplitarg() API that has
no such a problem. As a result, we introduced a bug, where it is no
longer possible to do something like:

    SET foo ""

Using the inline protocol. Now it is fixed.
2019-12-29 15:40:40 +01:00
Salvatore Sanfilippo
3ff95d9074
Merge pull request #6706 from guybe7/stream_id_edge_cases
Stream: Handle streamID-related edge cases
2019-12-29 14:53:06 +01:00
Oran Agra
0c3fe52ef7 config.c adjust config limits and mutable
- make lua-replicate-commands mutable (it never was, but i don't see why)
- make tcp-backlog immutable (fix a recent refactory mistake)
- increase the max limit of a few configs to match what they were before
the recent refactory
2019-12-26 15:16:15 +02:00
Guy Benoish
1f75ce30df Stream: Handle streamID-related edge cases
This commit solves several edge cases that are related to
exhausting the streamID limits: We should correctly calculate
the succeeding streamID instead of blindly incrementing 'seq'
This affects both XREAD and XADD.

Other (unrelated) changes:
Reply with a better error message when trying to add an entry
to a stream that has exhausted last_id
2019-12-26 15:31:37 +05:30
Yossi Gottlieb
0c898bdb07 Add REDISMODULE_CTX_FLAGS_MULTI_DIRTY. 2019-12-23 19:43:51 +02:00
Oran Agra
660645c742 modules don't signalModifiedKey in setKey() since that's done (optionally) in RM_CloseKey 2019-12-23 10:15:52 +02:00
Khem Raj
af02478ba0 Mark extern definition of SDS_NOINIT in sds.h
This helps in avoiding multiple definition of this variable, its also
defined globally in sds.c

Signed-off-by: Khem Raj <raj.khem@gmail.com>
2019-12-21 11:47:31 -08:00
bodong.ybd
337dcde9fa Fix spop return nil #4709 2019-12-21 21:27:38 +08:00
antirez
324e22accf Fix ip and missing mode in RM_GetClusterNodeInfo(). 2019-12-20 12:29:02 +01:00
Johannes Truschnigg
dfb598cf33 Signal systemd readiness atfer Partial Resync
"Partial Resynchronization" is a special variant of replication success
that we have to tell systemd about if it is managing redis-server via a
Type=Notify service unit.
2019-12-19 21:47:24 +01:00
Salvatore Sanfilippo
d3a9dff6b9
Merge pull request #6615 from soloestoy/wrap-also-propagate-as-multi
Wrap also propagate as multi
2019-12-19 09:24:52 +01:00
Salvatore Sanfilippo
9a7b6a9f51
Merge pull request #5780 from soloestoy/lua-multi-more-clear
scripting: flag lua_client as CLIENT_MULTI after redis.replicate_command() immediately
2019-12-19 09:06:28 +01:00
Salvatore Sanfilippo
f4b8197060
Merge pull request #6052 from jtru/better-systemd-integration-v2
Better systemd integration v2
2019-12-19 08:54:22 +01:00
antirez
c5bc1c14c0 Modules: rewrite top function doc of AvoidReplicaTraffic(). 2019-12-18 17:11:03 +01:00
Salvatore Sanfilippo
ef0b45ece8
Merge pull request #6497 from oranagra/avoid_replica_traffic
Add config and module API for AvoidReplicaTraffic
2019-12-18 17:06:05 +01:00
antirez
5a72c5058c Fix GEOHASH negative shifting in a more compatible way. 2019-12-18 12:57:32 +01:00
antirez
e6e58e455c Revert "Geo: output 10 chars of geohash, not 11."
This reverts commit 009862ab7e.
2019-12-18 12:54:46 +01:00
antirez
5b9d3ac6c6 Avoid changing setKey() API after #6679 fix. 2019-12-18 11:58:02 +01:00
zhaozhao.zz
3c0ed0309a lazyfree: add a new configuration lazyfree-lazy-user-del
Delete keys in async way when executing DEL command, if
lazyfree-lazy-user-del is yes.
2019-12-18 16:54:49 +08:00
zhaozhao.zz
58554396d6 incrbyfloat: fix issue #5256 ttl lost after propagate 2019-12-18 15:44:51 +08:00
zhaozhao.zz
24044f3356 add a new SET option KEEPTTL that doesn't remove expire time 2019-12-18 15:20:36 +08:00
Guy Benoish
5e0fe4c77c streamReplyWithRangeFromConsumerPEL: Redundant streamDecodeID 2019-12-18 12:27:37 +05:30
Salvatore Sanfilippo
b7c78b7651
Merge pull request #5916 from madolson/dev-unstable-acl-module-pr
Add module APIs for custom authentication
2019-12-17 09:58:26 +01:00
antirez
4348d25fc4 Add some comment to PR #6144 changes. 2019-12-17 09:19:47 +01:00
Madelyn Olson
7f04a15311 Removed old auth context reference 2019-12-17 07:51:19 +00:00
Madelyn Olson
a511a37bb7 Fixed some documentation 2019-12-17 07:49:21 +00:00
Madelyn Olson
7b3e3d6a13 Resolved merge miss 2019-12-16 23:40:19 -08:00
Madelyn Olson
12caffee61 Added a missed space in lua errors 2019-12-16 23:38:31 -08:00
Madelyn Olson
576a08908b Split error message so dependandent callers give a useful result 2019-12-16 23:34:37 -08:00
Madelyn Olson
44aa22c635 Added better exception handling around scripting and module 2019-12-16 23:33:53 -08:00
Madelyn Olson
c95a582a74 Add configuration option for allowing reads on cluster down 2019-12-16 23:33:16 -08:00
Madelyn Olson
f891b64a83 Tweaking the documentation 2019-12-17 07:28:55 +00:00
Madelyn Olson
67aa527b22 Added some documentation and fixed a test 2019-12-17 07:15:04 +00:00
Madelyn Olson
034dcf185c Add module APIs for custom authentication 2019-12-17 06:59:59 +00:00
antirez
901b7469b9 Clarify change in #6662. 2019-12-16 11:24:19 +01:00
antirez
aa3f13761f Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-12-16 11:18:20 +01:00
Salvatore Sanfilippo
276a093753
Merge pull request #6662 from oranagra/fix_hz_div0
init server.hz early to avoid div by 0 during config file loading
2019-12-16 11:18:08 +01:00
antirez
096592506e Modules: allow to execute RM_Call() from the module init function. 2019-12-13 17:34:37 +01:00
Yossi Gottlieb
0283db5883 Improve RM_ModuleTypeReplaceValue() API.
With the previous API, a NULL return value was ambiguous and could
represent either an old value of NULL or an error condition. The new API
returns a status code and allows the old value to be returned
by-reference.

This commit also includes test coverage based on
tests/modules/datatype.c which did not exist at the time of the original
commit.
2019-12-12 18:50:11 +02:00
Oran Agra
ef6e263fde init server.hz early to avoid div by 0 during config file loading
since the refactory of config.c, it was initialized from config_hz in initServer
but apparently that's too late since the config file loading creates objects
which call LRU_CLOCK
2019-12-12 17:56:02 +02:00
Salvatore Sanfilippo
118db9eeae
Merge pull request #6516 from IAmATeaPot418/patch-2
Add no-slowlog to acl command to prevent passwords
2019-12-12 09:41:19 +01:00
antirez
d7a87d0ea4 redis-cli: some refactoring about not logging AUTH & co. 2019-12-12 09:40:36 +01:00
Salvatore Sanfilippo
81dc180323
Merge pull request #6524 from itamarhaber/acl-cli
Omits  `ACL SETUSER` from redis-cli's history
2019-12-12 09:33:48 +01:00
antirez
476333d192 Remove useless space from moduleLoad(). 2019-12-12 09:29:10 +01:00
Salvatore Sanfilippo
2968d8e3cb
Merge branch 'unstable' into ModuleSecurity 2019-12-12 09:27:14 +01:00
Oran Agra
b5f3247ca5 Add module API for AvoidReplicaTraffic
This is useful to tell redis and modules to try to avoid doing things that may
increment the replication offset, and should be used when draining a master
and waiting for replicas to be in perfect sync before a failover.
2019-12-12 08:38:09 +02:00
Salvatore Sanfilippo
c6fb9d0963
Merge pull request #6656 from oranagra/leak_rm_load_from_str
fix leak in RM_LoadDataTypeFromString, and save
2019-12-11 11:39:27 +01:00
Oran Agra
a37cca3644 fix leak in RM_LoadDataTypeFromString, and save 2019-12-11 12:35:00 +02:00
antirez
25b36f8d65 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-12-11 10:18:15 +01:00
antirez
441cd9713c Clients connected and bytes used is too spammy for verbose.
This message is there for ten years, but is hardly useful.
Moreover it is likely that it will fill an entire disk if log ratation
is not configured, for no good reasons.
2019-12-11 10:17:23 +01:00
Oran Agra
173cca5413 fix possible warning on incomplete struct init 2019-12-10 11:16:13 +02:00
antirez
e4b6070c0f Modules: more clarification about disconnection callback. 2019-12-09 12:15:38 +01:00
Oran Agra
5941730c37 Add ULL suffix to CLIENT_TRACKING flag to prevent sign extension
the code in:
        c->flags &= ~(CLIENT_TRACKING|CLIENT_TRACKING_BROKEN_REDIR);
will do sign extension and turn on all the high 31 bits
no damage so far since we don't have any yet
2019-12-09 10:03:23 +02:00
Salvatore Sanfilippo
7e24e219fd
Merge pull request #6645 from eliblight/fix-memcpy
Fix memcpy of id from key to be slightly safer.
2019-12-05 16:51:42 +01:00
Eran Liberty
08c3fe8063 - memcpy(&id,ri.key,ri.key_len);
+        memcpy(&id,ri.key,sizeof(id));

The memcpy from the key to the id reliease on the fact that this key
*should* be 8 bytes long as it was entered as such a few lines up the
code.

BUT if someone will change the code to the point this is no longer true,
current code can trash the stack which makes debugging very hard
while this fix will result in some garbage id, or even page fault.
Both are preferable to stack mangaling.
2019-12-05 13:37:11 +00:00
antirez
2271cf03f5 Modules: clarify when the disconnection callback is called. 2019-12-05 10:38:18 +01:00
antirez
7b5f4b175b Modules: create timers in contexts without a client. 2019-12-03 16:21:23 +01:00
antirez
4092a75d85 Avoid collision with MacOS LIST_HEAD macro after #6384. 2019-12-02 09:13:29 +01:00
Salvatore Sanfilippo
e5b5f9a2f6
Merge pull request #6384 from devnexen/apple_smaps_impl
Getting region date per process in Darwin
2019-12-02 09:02:08 +01:00
Salvatore Sanfilippo
ce7ec725e3
Merge pull request #6624 from oranagra/config_c_step_3
Additional config.c refractory and bugfixes
2019-12-02 08:59:36 +01:00
Salvatore Sanfilippo
ff14945e22
Merge pull request #5360 from youjh90/unstable
update leap year comment when div by 400
2019-12-02 08:56:08 +01:00
Andy Guan
1ed5f84a79 Added the missed macro definition in slowlog.h 2019-12-02 00:28:41 -05:00
Oran Agra
6b4366b85b config.c post refactory cleanup 2019-12-01 08:19:25 +02:00
antirez
f5d48537f1 Fix Pi build needing -latomic. Issue #6275. 2019-11-29 17:35:59 +01:00
Oran Agra
18e72c5cc7 Converting more configs to use generic infra, and moving defaults to config.c
Changes in behavior:
- Change server.stream_node_max_entries from int64_t to long long, so that it can be used by the generic infra
- standard error reply instead of "repl-backlog-size must be 1 or greater" and such
- tls-port and a few TLS booleans were readable (config get) even when USE_OPENSSL was off (now they aren't)
- syslog-enabled, syslog-ident, cluster-enabled, appendfilename, and supervised didn't have a get (now they do)
- pidfile was initialized to NULL in InitServerConfig but had CONFIG_DEFAULT_PID_FILE in rewriteConfig (so the real default was "", but rewrite would cause it to be set), fixed the rewrite.
- TLS config in server.h was uninitialized (if no tls config args were provided)

Adding test for sanity and coverage
2019-11-28 11:24:57 +02:00
Oran Agra
28beb05aa3 More improvements and fixes to generic config infra
- Adding is_valid_fn and update_fn, both return 1 for success and 0 for failure with an optional error message.
- Bugfix in handling boundary check of unsigned numeric types (was boundaries as signed)
- Adding more numeric types to generic mechanism: uint, ulonglong, long, time_t, off_t
- More verbose error replies ("argument must be between" in out of range CONFIG SET (like config file parsing)
2019-11-28 11:11:07 +02:00
Oran Agra
e0cc3c99d2 Additional config.c refactory and bugfixes
- add capability for each config to have a callback to check if value is valid and return error string
  will enable converting many of the remaining custom configs into generic ones (reducing the x4 repetition for set,get,config,rewrite)
- add capability for each config to  to run some update code after config is changed (only for CONFIG SET)
  will also enable converting many of the remaining custom configs into generic ones
- add capability to move default values from server.h and server.c to config.c
  will reduce many excess lines in server.h and server.c (plus, no need to rebuild the entire code base when a default change 8-))

other behavior changes:
- fix bug in bool config get (always returning 'yes')
- fix a bug in modifying jemalloc-bg-thread at runtime (didn't call set_jemalloc_bg_thread, due to bad merge conflict resolution (my fault))
- side effect when a failed attempt to enable activedefrag at runtime, we now respond with -ERR and not with -DISABLED
2019-11-26 16:52:28 +02:00
Salvatore Sanfilippo
a40669893e
Merge pull request #6609 from daidaotong/macrofix
Add macro in bio.h
2019-11-25 10:04:28 +01:00
Salvatore Sanfilippo
6203ed0474
Merge pull request #6593 from daidaotong/movecommandfix
fix move command: WATCH/MULTI/EXEC mechanism breaks
2019-11-25 10:02:43 +01:00
Salvatore Sanfilippo
d39afb687d
Merge pull request #6563 from devnexen/fbsd_build_fix2
Add missing header
2019-11-25 09:45:29 +01:00
antirez
ef5186d920 config.c: remove trailing spaces, trim comments to 80 cols. 2019-11-22 17:54:22 +01:00
Salvatore Sanfilippo
a5e4c95cfd
Merge pull request #6607 from madolson/dev-unstable-new-refactor
Refactored configs into types that can be generic
2019-11-22 17:42:42 +01:00
zhaozhao.zz
6b056d29f3 block: propagate BRPOPLPUSH as RPOPLPUSH when unblock 2019-11-22 16:38:49 +08:00
zhaozhao.zz
2c970532dc Propagation: flag module client as CLIENT_MULTI if needed
in case of nested MULTI/EXEC
2019-11-22 16:20:30 +08:00
zhaozhao.zz
c73d70fb46 Propagation: propagate EXEC directly in lua script 2019-11-22 15:45:21 +08:00
zhaozhao.zz
37a10cef02 Propagation: wrap commands in also_propagate array with MULIT/EXEC
Random command like SPOP with count is replicated as
some SREM operations, and store them in also_propagate
array to propagate after the call, but this would break
atomicity.

To keep the command's atomicity, wrap also_propagate
array with MULTI/EXEC.
2019-11-22 15:42:49 +08:00
Madelyn Olson
e8f4010e27 Changed a tab to a space 2019-11-22 06:44:12 +00:00
Madelyn Olson
d50ae36e17 Fixed a bug with enum log printing 2019-11-22 06:28:50 +00:00
Madelyn Olson
818a4e0f70 Fixed some linux warnings 2019-11-22 05:49:52 +00:00
Madelyn Olson
519822bcf9 Added a flag for strings that are stored as NULL 2019-11-21 21:31:53 -08:00
zhaozhao.zz
73841e8c49 scripting: a little refactor about EVAL and redis.call()
Change server.lua_client's flag in a more explicit way.
2019-11-22 11:58:03 +08:00
zhaozhao.zz
097c4a6536 scripting: flag lua_client as CLIENT_MULTI after redis.replicate_commands() immediately
To avoid nested MULTI/EXEC, we check the lua_caller's flag,
if we are in the MULTI context we flag the lua_client as
CLIENT_MULTI, but it's not enough we shoud flag lua_client
as CLIENT_MULTI after redis.replicate_commands() immediately
or the first write command after redis.replicate_commands()
cannot know it's in an transaction, I know the missing CLIENT_MULTI
doesn't have any effect now, but it's a real bug and we should fix
it, in case someday we allow some dangerous command like BLPOP.
2019-11-22 11:51:18 +08:00
Salvatore Sanfilippo
64c2508ee3
Merge branch 'unstable' into rm_get_server_info 2019-11-21 10:06:15 +01:00
Salvatore Sanfilippo
f1f259de5b
Merge pull request #6547 from guybe7/module_api_streams
Support streams in general module API functions
2019-11-21 10:03:50 +01:00
antirez
da47d52c79 Recomment PR #6346. 2019-11-21 10:01:49 +01:00
Salvatore Sanfilippo
70469b76d6
Merge branch 'unstable' into expose_zmalloc_capabilities 2019-11-21 09:57:19 +01:00
Daniel Dai
bfd8c1fdd2 macro fix
macro fix
2019-11-20 20:39:04 -05:00
Maxim Ivanov
c7b68d10ea
Prune leftover TODO comment
Is it sufficient... ? -- Yes it is. In standalone mode, we say READY=1
at the comment point; however in replicated mode, we delay sending
READY=1 until the replication sync completes.
2019-11-20 19:45:25 +02:00
Madelyn Olson
803d765d49 Refactored renaming types in config 2019-11-20 09:22:38 -08:00
Salvatore Sanfilippo
c697edf4f4
Merge pull request #6559 from oranagra/active_defrag_tunables
Adjustments for active defrag defaults
2019-11-20 10:08:08 +01:00