Commit Graph

10425 Commits

Author SHA1 Message Date
Huang Zhw
1895e134a7
Fix missing separator in module info line (usedby and using lists) (#9241)
Fix module info genModulesInfoStringRenderModulesList lack separator when there's more than one module in the list.

Co-authored-by: Oran Agra <oran@redislabs.com>
2021-07-19 11:10:25 +03:00
Huang Zhw
d54c9086c2
Remove testmodule in src/modules/Makefile. (#9250)
src/modules make failed. As in #3718 testmodule.c was removed. But the makefile was not updated
2021-07-19 08:23:25 +03:00
Binbin
e7ccdeebc9
Fix rank overflow in zslInsert with more than 2B entries. (#9249)
If there are more than 2B entries in a zset.
The calculated span will overflow.
2021-07-18 16:47:33 +03:00
Erik Dubbelboer
ee4bdf10ee
Fix format strings serverLogObjectDebugInfo to use unsigned (#2927)
This doesn't have any real impact, just a cleanup.
2021-07-18 15:27:42 +03:00
Binbin
c9ca3b7065
Add 'ist' to wordlist. (#9248)
fix spell checker CI after recently merged old PR
2021-07-18 14:59:00 +03:00
Paul Kulchenko
8a73cd1f9a Update Lua debugging help message commands for consistency. (#2946) 2021-07-18 14:11:09 +03:00
Lawrence Chou
71c0466c04
Fix typo in redis.conf comment (#9245) 2021-07-17 21:03:28 +03:00
Binbin
11dc4e59b3
SMOVE only notify dstset when the addition is successful. (#9244)
in case dest key already contains the member, the dest key isn't modified, so the command shouldn't invalidate watch.
2021-07-17 09:54:06 +03:00
qetu3790
f03af47a34
Set TCP keepalive on inbound clusterbus connections (#9230)
Set TCP keepalive on inbound clusterbus connections to prevent memory leak
2021-07-15 20:40:25 -07:00
ZEEKLING
334a9789f1
fix timeout spell error (#9150) 2021-07-15 16:01:11 -07:00
Oran Agra
6a5bac309e
Test infra, handle RESP3 attributes and big-numbers and bools (#9235)
- promote the code in DEBUG PROTOCOL to addReplyBigNum
- DEBUG PROTOCOL ATTRIB skips the attribute when client is RESP2
- networking.c addReply for push and attributes generate assertion when
  called on a RESP2 client, anything else would produce a broken
  protocol that clients can't handle.
2021-07-14 19:14:31 +03:00
Yossi Gottlieb
277e4dc203
Fix compatibility with OpenSSL 1.1.0. (#9233) 2021-07-14 12:32:13 +03:00
gourav
7ac42ab7b3
Return -1 when connTLSWrite fails to write due to socket being closed (#9222) 2021-07-14 10:46:33 +03:00
Binbin
d0f36eeda0
Add missing comma in memory/xgroup command help message. (#9210)
This would have resulted in missing newline in the help message
2021-07-13 18:16:05 +03:00
perryitay
ac8b1df885
Fail EXEC command in case a watched key is expired (#9194)
There are two issues fixed in this commit: 
1. we want to fail the EXEC command in case there is a watched key that's logically
   expired but not yet deleted by active expire or lazy expire.
2. we saw that currently cache time is update in every `call()` (including nested calls),
   this time is being also being use for the isKeyExpired comparison, we want to update
   the cache time only in the first call (execCommand)

Co-authored-by: Oran Agra <oran@redislabs.com>
2021-07-11 13:17:23 +03:00
Huang Zhw
cb961d8c8e
Do not install a file event to send data to rewrite child when parent stop sending diff to child in aof rewrite. (#8767)
In aof rewrite, when parent stop sending data to child, if there is
new rewrite data, aofChildWriteDiffData write event will be installed.
Then this event is issued and deletes the file event without do anyting.
This will happen over and over again until aof rewrite finish.

This bug used to waste a few system calls per excessive wake-up
(epoll_ctl and epoll_wait) per cycle, each cycle triggered by receiving
a write command from a client.
2021-07-11 13:00:17 +03:00
Binbin
fe5d325107
Add test for ziplist (nextdiff == -4 && reqlen < 4) (#9218)
The if judgement `nextdiff == -4 && reqlen < 4` in __ziplistInsert.
It's strange, but it's useful. Without it there will be problems during
chain update.
Till now these lines didn't have coverage in the tests, and there was
a question if they are at all needed (#7170)
2021-07-11 11:26:31 +03:00
Yossi Gottlieb
92e8004705
Pre-test bind-source-addr before running test. (#9214)
This attempts to catch any non-standard configuration where the test may
fail and produce a false positive.
2021-07-11 09:54:07 +03:00
Shogo Hayashi
350c6306b4
fix explanation of sha256 (#9220) 2021-07-10 10:04:54 -05:00
Huang Zhw
9a22457e8f
Start redis-check-aof/redis-check-rdb only if executable name contains (#9215)
redis-check-aof/redis-check-rdb.

Related to #9176. Before this commit, redis-server starts as
redis-check-aof/redis-check-rdb if the directory it is started from
contains the string redis-check-aof/redis-check-rdb. We check the
executable name instead of directory.
2021-07-09 21:45:58 +03:00
Mikhail Fesenko
1eb4baa5b8
Direct redis-cli repl prints to stderr, because --rdb can print to stdout. fflush stdout after responses (#9136)
1. redis-cli can output --rdb data to stdout
   but redis-cli also write some messages to stdout which will mess up the rdb.

2. Make redis-cli flush stdout when printing a reply
  This was needed in order to fix a hung in redis-cli test that uses
  --replica.
   Note that printf does flush when there's a newline, but fwrite does not.

3. fix the redis-cli --replica test which used to pass previously
   because it didn't really care what it read, and because redis-cli
   used printf to print these other things to stdout.

4. improve redis-cli --replica test to run with both diskless and disk-based.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Viktor Söderqvist <viktor@zuiderkwast.se>
2021-07-07 08:26:26 +03:00
Binbin
3a26f61fb3
Add range check for master port in replicaof. (#9201)
So that we can avoid commands that are obviously wrong.

Also unified with loadServerConfigFromString
because we also checked the range.
2021-07-06 10:46:10 +03:00
zhaozhao.zz
5ea3c3db15
a little optimization in touchAllWatchedKeysInDb (#9200)
touch keys only if the key is in one of the dbs,
in case rewind the list when the key doesn't exist.
2021-07-06 15:01:37 +08:00
Omer Shadmi
f06d782f5a
Avoid exiting to allow diskless loading to recover from RDB short read on module AUX data (#9199)
Currently a replica is able to recover from a short read (when diskless loading
is enabled) and avoid crashing/exiting, replying to the master and then the rdb
could be sent again by the master for another load attempt by the replica.
There were a few scenarios that were not behaving similarly, such as when
there is no end-of-file marker, or when module aux data failed to load, which
should be allowed to occur due to a short read.
2021-07-06 08:21:17 +03:00
Oran Agra
f8da33ae59
update issue template, replace IRC with discussions (#9198) 2021-07-05 18:09:48 +03:00
uriyage
67732bacb1
redis-cli: reset dbnum and tx prompt state after RESET (#9096) 2021-07-05 10:55:55 +03:00
Binbin
a418a2d3fc
hrandfield and zrandmember with count should return emptyarray when key does not exist. (#9178)
due to a copy-paste bug, it used to reply with null response rather than empty array.
this commit includes new tests that are looking at the RESP response directly in
order to be able to tell the difference between them.

Co-authored-by: Oran Agra <oran@redislabs.com>
2021-07-05 10:41:57 +03:00
Oran Agra
5a3de81925
Use accept4 on linux instead of fcntl to make a client socket non-blocking (#9177)
This reduces system calls on linux when a new connection is made / accepted.

Changes:
* Add the SOCK_CLOEXEC option to the accept4() call
  This  ensure that a fork/exec call does not leak a file descriptor.
* Move anetCloexec and connNonBlock info anetGenericAccept
* Moving connNonBlock from accept handlers to anetGenericAccept

Moving connNonBlock from createClient, is safe because createClient is
used in the following ways:
1. without a connection (fake client)
2. on an accepted connection (see above)
3. creating the master client by using connConnect (see below)

The third case, can either use anetTcpNonBlockConnect, or connTLSConnect
which is by default non-blocking.

Co-authored-by: Rajiv Kurian <geetasen@gmail.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Yoav Steinberg <yoav@redislabs.com>
2021-07-05 10:34:20 +03:00
Oran Agra
ec582cc7ad Query buffer shrinking improvements (#5013)
when tracking the peak, don't reset the peak to 0, reset it to the
maximum of the current used, and the planned to be used by the current
arg.

when shrining, split the two separate conditions.
the idle time shrinking will remove all free space.
but the peak based shrinking will keep room for the current arg.

when we resize due to a peak (rahter than idle time), don't trim all
unused space, let the qbuf keep a size that's sufficient for the
currently process bulklen, and the current peak.

Co-authored-by: sundb <sundbcn@gmail.com>
Co-authored-by: yoav-steinberg <yoav@monfort.co.il>
2021-07-05 09:30:16 +03:00
zhaozhao.zz
2248eaacac resize query buffer more accurately
1. querybuf_peak has not been updated correctly in readQueryFromClient.
2. qbuf shrinking uses sdsalloc instead of sdsAllocSize

see more details in issue #4983
2021-07-05 09:30:16 +03:00
Oran Agra
9894ce5fc4
add missing skips to daily cycle (#9196)
minor cleanups.
2021-07-05 08:46:19 +03:00
Oran Agra
7103367ad4
Tests: add a way to read raw RESP protocol reponses (#9193)
This makes it possible to distinguish between null response and an empty
array (currently the tests infra translates both to an empty string/list)
2021-07-04 19:43:58 +03:00
Oran Agra
a8518cce95
fix valgrind issues with recently added test in modules/blockonbackground (#9192)
fixes test issue introduced in #9167

1. invalid reads due to accessing non-retained string (passed as unblock context).
2. leaking module blocked client context, see #6922 for info.
2021-07-04 14:21:53 +03:00
Madelyn Olson
8f59f131e5
Update incrDecrCommand to use addReplyLongLong (#9188)
Update incrDecrCommand to use addReplyLongLong
2021-07-03 10:51:53 -05:00
Yossi Gottlieb
aa139e2f02
Fix CLIENT UNBLOCK crashing modules. (#9167)
Modules that use background threads with thread safe contexts are likely
to use RM_BlockClient() without a timeout function, because they do not
set up a timeout.

Before this commit, `CLIENT UNBLOCK` would result with a crash as the
`NULL` timeout callback is called. Beyond just crashing, this is also
logically wrong as it may throw the module into an unexpected client
state.

This commits makes `CLIENT UNBLOCK` on such clients behave the same as
any other client that is not in a blocked state and therefore cannot be
unblocked.
2021-07-01 17:11:27 +03:00
Oran Agra
de9bae21ef
Fix bug in sdscatfmt when % is the last format char (#9173)
For the sdscatfmt function in sds.c, when the parameter fmt ended up with '%',
the behavior is undefined. This commit fix this bug.

Co-authored-by: stafuc <stafuc@gmail.com>
2021-07-01 08:19:04 +03:00
Wang Yuan
16e04ed944
Don't start in sentinel mode if only the folder name contains redis-sentinel (#9176)
Before this commit, redis-server starts in sentinel mode if the first startup
argument has the string redis-sentinel, so redis also starts in sentinel mode
if the directory it was started from contains the string redis-sentinel.
Now we check the executable name instead of directory.

Some examples:
1. Execute ./redis-sentinel/redis/src/redis-sentinel, starts in sentinel mode.
2. Execute ./redis-sentinel/redis/src/redis-server, starts in server mode,
   but before, redis will start in sentinel mode.
3. Execute ./redis-sentinel/redis/src/redis-server --sentinel, of course, like
   before, starts in sentinel mode.
2021-07-01 08:05:18 +03:00
ZhaolongLi
6476c8e856
Fix range issues in default value of LIMIT argument to XADD and XTRIM (#9147)
This seems to be an unimportant bug that was accidentally generated. If the user does not specify limit in streamParseAddOrTrimArgsOrReply, the initial value of args->limit is 100 * server.stream_node_max_entries, which may lead to out of bounds, and then the default function of limit in xadd becomes invalid (this failure occurs in streamTrim).

Additionally, provide sane default for args->limit in case stream_node_max_entries is set to 0.

Co-authored-by: lizhaolong.lzl <lizhaolong.lzl@B-54MPMD6R-0221.local>
Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: guybe7 <guy.benoish@redislabs.com>
2021-06-30 16:55:09 +03:00
Mikhail Fesenko
74fe15b360
redis-cli --rdb: fix broken fsync/ftruncate for stdout (#9135)
A change in redis 6.2 caused redis-cli --rdb that's directed to stdout to fail because fsync fails.
This commit avoids doing ftruncate (fails with a warning) and fsync (fails with an error) when the
output file is `-`, and adds the missing documentation that `-` means stdout.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Wang Yuan <wangyuancode@163.com>
2021-06-30 16:49:54 +03:00
Rob Snyder
eaa52719a3
Fix ziplist length updates on bigendian platforms (#2080)
Adds call to intrev16ifbe to ensure ZIPLIST_LENGTH is compared correctly
2021-06-30 16:46:06 +03:00
Huang Zhw
70dac4b436
Make redis-cli --help/-h output to stdout and exit with 0. (#9124)
If user executes redis-cli --help, the content should be output to stdout and exits with 0.
2021-06-30 14:56:44 +03:00
Binbin
5dddf496ce
Add missing pause tcl test to test_helper.tcl (#9158)
* Add keyname tags to avoid CROSSSLOT errors in external server CI
* Use new wait_for_blocked_clients_count in pause.tcl
2021-06-30 13:32:51 +03:00
luvine
4278c45c91
HSETNX can lead coding type change even when skipped due to NX (#4615)
1. Add one key-value pair to myhash, which the length of key and value both less than hash-max-ziplist-value, for example:
>hset myhash key value
2. Then execute the following command
>hsetnx myhash key value1 (the length greater than hash-max-ziplist-value)
3. This will add nothing, but the code type of "myhash" changed from ziplist to dict even there are only one key-value pair in "myhash", and both of them less than hash-max-ziplist-value.
2021-06-30 12:14:11 +03:00
Binbin
1d5aa37d68
Fix timing issue in psync2 test. (#9159)
*** [err]: PSYNC2: total sum of full synchronizations is exactly 4 intests/integration/psync2.tcl
Expected 5 == 4 (context: type eval line 8 cmd {assert {$sum == 4}} proc::test)

Sometime the test got an unexpected full sync since a replica switch to master,
before the new master change propagated the new replid to all replicas,
a replica attempted to sync with it using a wrong replid and triggered a full resync.

Consider this scenario:
    1 slaveof 4 full resync
    0 slaveof 4 full resync
    2 slaveof 0 full resync
    3 slaveof 1 full resync

    1 slaveof no one, replid changed
    3 reconnect 1, did a partial resyn and got the new replid

    Before 2 inherits the new replid.
    3 slaveof 2
    3 try to do a partial resyn with 2.
    But their replication ids are inconsistent, so a full resync happens.

:) A special thank you for oran and helping me in this test case.

Co-authored-by: Oran Agra <oran@redislabs.com>
2021-06-30 09:18:10 +03:00
ZhaolongLi
cf1d49adbf
Simplify logic in raxSeek, eliminate it->key reassembly on gt and lt (#9115)
In the original version, the operation of traversing the stack only seems to
reconstruct the key that does not contain the current node.

But in fact We have got the matched length and splitpos in the key in the
raxlowwalk, so I think we can simplify the logic of this part.

Co-authored-by: lizhaolong.lzl <lizhaolong.lzl@B-54MPMD6R-0221.local>
2021-06-30 08:21:19 +03:00
Yossi Gottlieb
5d8ea4b326
Add missing needs:repl tag. (#9169) 2021-06-29 16:48:52 +03:00
Leibale Eidelman
95274f1f8a
fix ZRANGESTORE - should return 0 when src points to an empty key (#9089)
mistakenly it used to return an empty array rather than 0.

Co-authored-by: Oran Agra <oran@redislabs.com>
2021-06-29 16:38:10 +03:00
Tsonglew
7913d34d7c
Improve doc comment about AE_DONT_WAIT (#9165)
Co-authored-by: yoav-steinberg <yoav@monfort.co.il>
2021-06-29 14:37:02 +03:00
guybe7
4434cebbb3
Include sizeof(struct stream) in objectComputeSize (#9164)
Affects MEMORY USAGE
2021-06-29 14:34:18 +03:00
Binbin
4bc5a8324d
ZRANDMEMBER WITHSCORES with negative COUNT may return bad score (#9162)
Return a bad score when used with negative count (or count of 1), and non-ziplist encoded zset.
Also add test to validate the return value and cover the issue.
2021-06-29 10:14:28 +03:00