caleb-brown/redict
1
0
Fork 0
mirror of https://codeberg.org/redict/redict.git synced 2025-01-23 16:48:27 -05:00
Code Issues Actions Packages Projects Releases Wiki Activity
7,339 Commits 4 Branches 5 Tags 115 MiB
9849cf2df5
Commit Graph

5013 Commits

Author SHA1 Message Date
antirez
3e7349fdaf Make pending buffer processing safe for CLIENT_MASTER client. 
Related to #5305.
 2018-09-03 18:17:31 +02:00
zhaozhao.zz
247d2a734b networking: optimize parsing large bulk greater than 32k 
If we are going to read a large object from network
try to make it likely that it will start at c->querybuf
boundary so that we can optimize object creation
avoiding a large copy of data.

But only when the data we have not parsed is less than
or equal to ll+2. If the data length is greater than
ll+2, trimming querybuf is just a waste of time, because
at this time the querybuf contains not only our bulk.

It's easy to reproduce the that:

Time1: call `client pause 10000` on slave.

Time2: redis-benchmark -t set -r 10000 -d 33000 -n 10000.

Then slave hung after 10 seconds.
 2018-09-04 00:02:25 +08:00
zhaozhao.zz
2290c4bee1 if master is already unblocked, do not unblock it twice 2018-09-03 14:36:48 +08:00
zhaozhao.zz
e3dfd8c811 fix multiple unblock for clientsArePaused() 2018-09-03 14:26:14 +08:00
Amin Mesbah
a036c64c01 Use geohash limit defines in constraint check 
Slight robustness improvement, especially if the limit values are
changed, as was suggested in antires/redis#4291 [1].

[1] https://github.com/antirez/redis/pull/4291
 2018-09-02 00:06:20 -07:00
antirez
7fa493912e After slave Lua script leaves busy state, re-process the master buffer. 
Technically speaking we don't really need to put the master client in
the clients that need to be processed, since in practice the PING
commands from the master will take care, however it is conceptually more
sane to do so.
 2018-08-31 16:45:02 +02:00
antirez
9ab91b8c6c While the slave is busy, just accumulate master input. 
Processing command from the master while the slave is in busy state is
not correct, however we cannot, also, just reply -BUSY to the
replication stream commands from the master. The correct solution is to
stop processing data from the master, but just accumulate the stream
into the buffers and resume the processing later.

Related to #5297.
 2018-08-31 16:45:02 +02:00
antirez
83af8ef1fd Allow scripts to timeout even if from the master instance. 
However the master scripts will be impossible to kill.

Related to #5297.
 2018-08-31 16:45:02 +02:00
antirez
f5b29c6444 Allow scripts to timeout on slaves as well. 
See reasoning in #5297.
 2018-08-31 16:45:01 +02:00
zhaozhao.zz
dce7cefb7c networking: fix unexpected negative or zero readlen 
To avoid copying buffers to create a large Redis Object which
exceeding PROTO_IOBUF_LEN 32KB, we just read the remaining data
we need, which may less than PROTO_IOBUF_LEN. But the remaining
len may be zero, if the bulklen+2 equals sdslen(c->querybuf),
in client pause context.

For example:

Time1:

python
>>> import os, socket
>>> server="127.0.0.1"
>>> port=6379
>>> data1="*3\r\n$3\r\nset\r\n$1\r\na\r\n$33000\r\n"
>>> data2="".join("x" for _ in range(33000)) + "\r\n"
>>> data3="\n\n"
>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> s.settimeout(10)
>>> s.connect((server, port))
>>> s.send(data1)
28

Time2:

redis-cli client pause 10000

Time3:

>>> s.send(data2)
33002
>>> s.send(data3)
2
>>> s.send(data3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
socket.error: [Errno 104] Connection reset by peer

To fix that, we should check if remaining is greater than zero.
 2018-08-31 20:02:09 +08:00
Salvatore Sanfilippo
d05f5c8f64
Merge pull request #5268 from 0xtonyxia/fix-latency-cmd-comments2 
Revise the comments of latency command.
 2018-08-29 16:19:13 +02:00
Salvatore Sanfilippo
476eea95da
Merge pull request #4216 from lamby/did-not-received-typos 
Correct "did not received" -> "did not receive" typos/grammar.
 2018-08-29 16:18:11 +02:00
zhaozhao.zz
7d39c149c4 Supplement to PR #4835, just take info/memory/command as random commands 2018-08-29 18:23:05 +08:00
Salvatore Sanfilippo
0e21efdb1c
Merge pull request #4835 from soloestoy/command-script-flag 
some commands' flags should be set correctly, issue #4834
 2018-08-29 12:13:50 +02:00
zhaozhao.zz
32844178ac some commands' flags should be set correctly, issue #4834 2018-08-29 18:07:01 +08:00
antirez
e245a2046a Make slave-ignore-maxmemory configurable. 2018-08-27 12:27:17 +02:00
antirez
067647a783 Introduce repl_slave_ignore_maxmemory flag internally. 
Note: this breaks backward compatibility with Redis 4, since now slaves
by default are exact copies of masters and do not try to evict keys
independently.
 2018-08-27 12:20:27 +02:00
antirez
abf52c7cf4 Better variable meaning in processCommand(). 2018-08-27 12:17:34 +02:00
antirez
66b5afdaa4 Re-apply rebased #2358. 2018-08-27 12:17:14 +02:00
antirez
c241f51607 Fix build errors caused by #2358. 2018-08-27 12:15:55 +02:00
Salvatore Sanfilippo
19880ab851
Merge pull request #5248 from soloestoy/rewrite-brpoplpush 
rewrite BRPOPLPUSH as RPOPLPUSH to propagate
 2018-08-26 16:31:24 +02:00
Salvatore Sanfilippo
80e1695652
Merge pull request #5244 from soloestoy/optimize-pipeline 
pipeline: do not sdsrange querybuf unless all commands processed
 2018-08-26 16:30:49 +02:00
Chris Lamb
132be8aed5 Correct "did not received" -> "did not receive" typos/grammar. 2018-08-26 14:45:39 +02:00
Salvatore Sanfilippo
46d89a9abb
Merge pull request #2992 from lamby/source-date-epoch 
Use SOURCE_DATE_EPOCH over unreproducible uname + date calls.
 2018-08-26 11:25:44 +02:00
Chris Lamb
f63e81c202
Merge branch 'unstable' into config-set-maxmemory-grammar 2018-08-25 21:49:29 +02:00
zhaozhao.zz
f2ad89a314 networking: make setProtocolError simple and clear 
Function setProtocolError just records proctocol error
details in server log, set client as CLIENT_CLOSE_AFTER_REPLY.
It doesn't care about querybuf sdsrange, because we
will do it after procotol parsing.
 2018-08-23 12:21:28 +08:00
dejun.xdj
1ab64d405e Revise the comments of latency command. 2018-08-22 18:07:02 +08:00
zhaozhao.zz
9a65f9cd3e block: format code 2018-08-14 20:59:32 +08:00
zhaozhao.zz
8a1219d93b block: rewrite BRPOPLPUSH as RPOPLPUSH to propagate 2018-08-14 20:58:58 +08:00
zhaozhao.zz
ef2a95c461 networking: just move qb_pos instead of sdsrange in processInlineBuffer 2018-08-14 14:50:37 +08:00
zhaozhao.zz
e623bd22ba networking: just return C_OK if multibulk processing saw a <= 0 length. 2018-08-14 13:55:30 +08:00
zhaozhao.zz
14c4ddb5a6 pipeline: do not sdsrange querybuf unless all commands processed 
This is an optimization for processing pipeline, we discussed a
problem in issue #5229: clients may be paused if we apply `CLIENT
PAUSE` command, and then querybuf may grow too large, the cost of
memmove in sdsrange after parsing a completed command will be
horrible. The optimization is that parsing all commands in queyrbuf
, after that we can just call sdsrange only once.
 2018-08-14 00:43:42 +08:00
Jeffrey Lovitz
bd01334da1 CLI Help text loop verifies arg count 2018-08-12 12:47:01 -04:00
Salvatore Sanfilippo
39c70e728b
Merge pull request #5146 from 0xtonyxia/fix-xclaim-id-parse 
Streams: ID of xclaim command should start from the sixth argument.
 2018-08-03 13:45:27 +02:00
Salvatore Sanfilippo
7b5e7f3ed0
Merge pull request #5151 from shenlongxing/fix-stream 
Fix stream command paras
 2018-08-03 13:39:32 +02:00
shenlongxing
fe56c67405 Fix stream command paras 2018-08-03 19:01:15 +08:00
antirez
0ce8323c0d Fix AOF comment to report the current behavior. 
Realted to #5201.
 2018-08-03 12:46:06 +02:00
antirez
2f282aee0b Fix zslUpdateScore() edge case. 
When the element new score is the same of prev/next node, the
lexicographical order kicks in, so we can safely update the node in
place only when the new score is strictly between the adjacent nodes
but never equal to one of them.

Technically speaking we could do extra checks to make sure that even if the
score is the same as one of the adjacent nodes, we can still update on
place, but this rarely happens, so probably not a good deal to make it
more complex.

Related to #5179.
 2018-08-01 19:04:53 +02:00
antirez
29226a3919 More commenting of zslUpdateScore(). 2018-08-01 18:54:15 +02:00
antirez
12ff0c0d79 Explain what's the point of zslUpdateScore() in top comment. 2018-08-01 18:53:09 +02:00
antirez
e3e94ec35c Remove old commented zslUpdateScore() from source. 2018-08-01 18:50:49 +02:00
antirez
0b800c4332 Optimize zslUpdateScore() as asked in #5179. 2018-08-01 18:50:31 +02:00
antirez
6dd0d6f7dd zsetAdd() refactored adding zslUpdateScore(). 2018-08-01 18:27:56 +02:00
antirez
2f2987ffc5 Streams IDs parsing refactoring. 
Related to #5184.
 2018-07-31 18:08:52 +02:00
Salvatore Sanfilippo
187fa78637
Merge pull request #5184 from 0xtonyxia/streams-id-opt 
Streams: rearrange the usage of '-' and '+' IDs in stream commands.
 2018-07-31 17:58:13 +02:00
Salvatore Sanfilippo
7b41701eb8
Merge pull request #5187 from oranagra/testsuite_improvements 
Testsuite improvements
 2018-07-31 17:20:09 +02:00
antirez
e245ed9a44 Cluster cron announce IP minor refactoring. 2018-07-31 17:13:03 +02:00
Salvatore Sanfilippo
093adb7ef9
Merge pull request #5191 from shenlongxing/fix-memory-leak 
Fix cluster-announce-ip memory leak
 2018-07-31 17:11:16 +02:00
antirez
b0392e75ec Tranfer -> transfer typo fixed. 2018-07-31 16:41:33 +02:00
Salvatore Sanfilippo
92b39a0abf
Merge pull request #5189 from soloestoy/refactor-dbOverwrite 
refactor dbOverwrite to make lazyfree work
 2018-07-31 16:40:35 +02:00
antirez
db693be00d Refactoring: replace low-level checks with writeCommandsDeniedByDiskError(). 2018-07-31 13:16:43 +02:00
antirez
11dd3f4b03 Fix writeCommandsDeniedByDiskError() inverted return value. 2018-07-31 13:14:24 +02:00
antirez
0e49938b68 Better top comment for writeCommandsDeniedByDiskError(). 2018-07-31 13:10:37 +02:00
antirez
5401fe7fb9 Introduce writeCommandsDeniedByDiskError(). 2018-07-31 13:09:38 +02:00
shenlongxing
35ca670060 Fix cluster-announce-ip memory leak 2018-07-31 16:01:44 +08:00
zhaozhao.zz
fddeeae724 refactor dbOverwrite to make lazyfree work 2018-07-31 12:07:57 +08:00
Salvatore Sanfilippo
ac3c012a7f
Merge pull request #5153 from trevor211/fixLuaScript 
Consider aof write error as well as rdb in lua script.
 2018-07-30 18:10:06 +02:00
Pavel Rochnyack
1e394b73b7 INFO CPU: higher precision of reported values 
Closes: #5033
 2018-07-30 18:06:22 +02:00
Salvatore Sanfilippo
1465755e38
Merge pull request #5183 from gkorland/patch-1 
Few typo fixes
 2018-07-30 17:59:12 +02:00
antirez
3c19ae941d Add year in log. 
User: "is there a reason why redis server logs  are missing the year in
the "date time"?"

Me: "I guess I did not imagine it would be stable enough to run for
several years".
 2018-07-30 17:42:30 +02:00
dejun.xdj
6491717c88 Streams: rearrange the usage of '-' and '+' IDs in stream commands. 2018-07-30 21:33:01 +08:00
dejun.xdj
d6f5ec6f09 Streams: add mmid_supp argument in streamParseIDOrReply(). 
If 'mmid_supp' is set to 0, "-" and "+" will be
treated as an invalid ID.
 2018-07-30 21:32:07 +08:00
Guy Korland
acaa18f1d1
Few typo fixes 2018-07-30 16:18:56 +03:00
antirez
be28050ac0 Make dynamic hz actually configurable. 2018-07-30 13:44:52 +02:00
antirez
c426d85c4c Control dynamic HZ via server configuration. 2018-07-30 13:37:30 +02:00
antirez
4e9c30a6ca Merge branch 'dynamic-hz' into unstable 2018-07-30 13:31:23 +02:00
Oran Agra
4a30adde31 add DEBUG LOG, to to assist test suite debugging 2018-07-30 12:23:51 +03:00
antirez
fd174cca23 Remove useless conditional from emptyDb(). 
Related to #4852.
 2018-07-25 16:34:57 +02:00
antirez
1c2352f0dd Make emptyDb() change introduced in #4852 simpler to read. 2018-07-25 16:32:52 +02:00
zhaozhao.zz
dd071ff6c9 optimize flushdb, avoid useless loops 2018-07-25 18:13:34 +08:00
Salvatore Sanfilippo
73cbc33a0a
Merge pull request #5133 from soloestoy/fix-xdel-memory-leak 
Streams: fix xdel memory leak
 2018-07-24 18:57:05 +02:00
zhaozhao.zz
eb0e5fe4e3 Streams: fix xdel memory leak 2018-07-25 00:06:27 +08:00
antirez
0bdeb861f4 Example the magic +1 in migrateCommand(). 
Related to #5154.
 2018-07-24 17:31:43 +02:00
antirez
53d46fa712 Make changes of PR #5154 hopefully simpler. 2018-07-24 17:27:43 +02:00
Salvatore Sanfilippo
e52a036ac6
Merge pull request #5154 from trevor211/optimizeMigrate 
Do not migrate already expired keys.
 2018-07-24 17:24:37 +02:00
antirez
8513f31be0 Streams: refactoring of next entry seek in the iterator. 
After #5161 the code could be made a bit more obvious for newcomers.
 2018-07-24 11:07:27 +02:00
Salvatore Sanfilippo
41a886605a
Merge pull request #5161 from soloestoy/optimize-stream-iterator 
Streams: skip master fileds only when we are going forward in streamIteratorGetID
 2018-07-24 10:59:55 +02:00
antirez
f4ac796c34 string2ll(): better commenting. 2018-07-24 10:27:20 +02:00
antirez
b9a8e56439 Merge branch 'removeRedundantCheck' of https://github.com/dsomeshwar/redis into unstable 2018-07-24 10:24:24 +02:00
antirez
b28cbe90e1 Restore string2ll() to original version. 
See PR #5157.
 2018-07-24 10:22:12 +02:00
antirez
1a8d7cd9c2 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2018-07-24 00:21:49 +02:00
antirez
e8f5d21940 string2ll(): remove duplicated check for special case. 
Related to #5157. The PR author correctly indentified that the check was
duplicated, but removing the second one introduces a bug that was fixed
in the past (hence the duplication). Instead we can remove the first
instance of the check without issues.
 2018-07-24 00:20:20 +02:00
antirez
1daee8c748 string2ll(): test for NULL pointer in all the cases. 2018-07-24 00:18:39 +02:00
Salvatore Sanfilippo
ab1d5fe11a
Merge pull request #5160 from oranagra/fix_zmalloc_usable 
fix recursion typo in zmalloc_usable
 2018-07-24 00:07:59 +02:00
antirez
9a375e5408 Change 42 to 1000 as warning level for cached scripts. 
Related to #4883.
 2018-07-23 18:44:58 +02:00
Salvatore Sanfilippo
445a2a2b1b
Merge pull request #4883 from itamarhaber/lua_scripts-in-info-memory 
Adds memory information about the scripts' cache to INFO
 2018-07-23 18:43:05 +02:00
antirez
b65ddfb16a Dynamic HZ: adapt cron frequency to number of clients. 2018-07-23 14:21:04 +02:00
antirez
e6ea603ad3 Dynamic HZ: separate hz from the configured hz. 
This way we can remember what the user configured HZ is, but change the
actual HZ dynamically if needed in the dynamic HZ feature
implementation.
 2018-07-23 14:13:58 +02:00
Itamar Haber
993716c351 Adds Lua overheads to MEMORY STATS, smartens the MEMORY DOCTOR 2018-07-22 21:16:00 +03:00
zhaozhao.zz
9674ede85a Streams: skip master fileds only when we are going forward in streamIteratorGetID 2018-07-22 17:52:09 +08:00
Oran Agra
780815dd6e fix recursion typo in zmalloc_usable 2018-07-22 10:17:35 +03:00
dsomeshwar
bb5b8b3a6f removing redundant check 2018-07-21 23:42:08 +05:30
WuYunlong
4017a11144 Do not migrate already expired keys. 2018-07-21 10:00:32 +08:00
WuYunlong
dd760bd5e6 Consider aof write error as well as rdb in lua script. 2018-07-21 08:48:51 +08:00
antirez
4ff47a0b9b Top comment clientsCron(). 2018-07-20 09:46:18 +02:00
antirez
aba6855282 Clarify that clientsCronTrackExpansiveClients() indexes may jump ahead. 2018-07-20 09:36:48 +02:00
antirez
be88c0b16a Rename INFO CLIENT max buffers field names for correctness. 
They are actually delayed a few seconds, so let's call them "recent".
 2018-07-19 17:38:20 +02:00
antirez
0cf3794e6e Fix wrong array index variable in getExpansiveClientsInfo(). 2018-07-19 17:34:15 +02:00
antirez
ea3a20c5d0 Change INFO CLIENTS sections to report pre-computed max/min client buffers. 2018-07-19 17:16:19 +02:00
dejun.xdj
de5ca516ae Streams: ID of xclaim command starts from the sixth argument. 2018-07-19 20:48:08 +08:00