Commit Graph

498 Commits

Author SHA1 Message Date
Salvatore Sanfilippo
8d6dc136e1
Merge pull request #7041 from hwware/trackingfix
CLIENT TRACKING Command Fix: Add More Checking for OPTIN/OPTOUT mode
2020-04-02 11:22:30 +02:00
antirez
1d13ff0b39 Minor changes to #7037. 2020-03-31 17:10:09 +02:00
Guy Benoish
d6eb3afd13 Modules: Test MULTI/EXEC replication of RM_Replicate
Makse sure call() doesn't wrap replicated commands with
a redundant MULTI/EXEC

Other, unrelated changes:
1. Formatting compiler warning in INFO CLIENTS
2. Use CLIENT_ID_AOF instead of UINT64_MAX
2020-03-31 13:55:51 +03:00
hwware
86c76cad74 add check for not switching between optin optout mode directly 2020-03-29 23:20:54 -04:00
hwware
1ef44f8243 add check for not providing both optin optout flag 2020-03-29 23:06:50 -04:00
Yossi Gottlieb
2dab5015b7 Fix crashes related to failed/rejected accepts. 2020-03-22 14:47:44 +02:00
antirez
f9c56dbb09 ACL: default user off should not allow automatic authentication.
This fixes issue #7011.
2020-03-20 12:45:48 +01:00
Salvatore Sanfilippo
15338ab694
Merge pull request #6991 from soloestoy/io-threads-bugfix
Threaded IO: bugfix client kill may crash redis
2020-03-16 10:53:54 +01:00
zhaozhao.zz
a6a0e05a1a Threaded IO: bugfix client kill may crash redis 2020-03-15 23:30:25 +08:00
antirez
573c4673ee Aesthetic changes in PR #6989. 2020-03-15 16:10:37 +01:00
zhaozhao.zz
606a01df70 Threaded IO: bugfix #6988 process events while blocked 2020-03-15 22:07:36 +08:00
antirez
b6378edcd6 Tracking: optin/out implemented. 2020-02-24 19:09:45 +01:00
Madelyn Olson
c1b5220ee7 Give an error message if you specify redirect twice 2020-02-16 05:41:39 -08:00
Madelyn Olson
d63a435362 Minor CSC fixes and fixed documentation 2020-02-16 05:28:29 -08:00
antirez
47177c9edc Tracking: fix operators precedence error in bcast check. 2020-02-14 14:27:45 +01:00
antirez
6922ccc0b9 Tracking: fix sending messages bug + tracking off bug. 2020-02-13 16:58:07 +01:00
antirez
40194a2a68 Tracking: BCAST: basic feature now works. 2020-02-12 19:22:04 +01:00
antirez
3f7ba86255 Tracking: BCAST: registration in the prefix table. 2020-02-11 17:26:29 +01:00
antirez
dfe126f3e9 Tracking: BCAST: parsing of the options + skeleton. 2020-02-10 17:18:11 +01:00
Salvatore Sanfilippo
7cf53252ee
Merge pull request #6849 from oranagra/free_client_mutex
freeClientAsync don't lock mutex if there's just one thread
2020-02-06 10:29:46 +01:00
Guy Benoish
91c41b6dde Some refactroing using getClientType instead of CLIENT_SLAVE 2020-02-06 14:12:08 +05:30
Guy Benoish
e33fffbde1 Fix small bugs related to replica and monitor ambiguity
1. server.repl_no_slaves_since can be set when a MONITOR client disconnects
2. c->repl_ack_time can be set by a newline from a MONITOR client
3. Improved comments
2020-02-06 14:10:02 +05:30
Oran Agra
86e302f5f3 freeClientAsync don't lock mutex if there's just one thread 2020-02-06 10:07:17 +02:00
antirez
7d5fb5df3f Setting N I/O threads should mean N-1 additional + 1 main thread. 2020-01-13 18:53:36 +01:00
antirez
d28948b143 Port PR #6110 to new connection object code. 2020-01-13 12:54:39 +01:00
antirez
a8c912ead6 A few comments about main thread serving I/O as well.
Related to #6110.
2020-01-13 12:50:26 +01:00
Salvatore Sanfilippo
24896427fc
Merge pull request #6110 from soloestoy/enhance-io-threaded
Enhance IO Threaded: use main thread to handle read/write work
2020-01-13 12:46:40 +01:00
antirez
6e4f70b817 Fix duplicated CLIENT SETNAME reply.
Happened when we set the name to "" to cancel the name.
Was introduced during the RESP3 refactoring.

See #6036.
2019-12-29 15:44:59 +01:00
antirez
5521910de7 Inline protocol: handle empty strings well.
This bug is from the first version of Redis. Probably the problem here
is that before we used an SDS split function that created empty strings
for additional spaces, like in "SET    foo          bar".
AFAIK later we replaced it with the curretn sdssplitarg() API that has
no such a problem. As a result, we introduced a bug, where it is no
longer possible to do something like:

    SET foo ""

Using the inline protocol. Now it is fixed.
2019-12-29 15:40:40 +01:00
Madelyn Olson
034dcf185c Add module APIs for custom authentication 2019-12-17 06:59:59 +00:00
Salvatore Sanfilippo
bbdbfabf9e
Merge pull request #6531 from oranagra/rm_save_long_double
Module API for loading and saving long double
2019-11-04 17:33:35 +01:00
Oran Agra
779aebc91c Module API for loading and saving long double
looks like each platform implements long double differently (different bit count)
so we can't save them as binary, and we also want to avoid creating a new RDB
format version, so we save these are hex strings using "%La".

This commit includes a change in the arguments of ld2string to support this.
as well as tests for coverage and short reads.

coded by @guybe7
2019-11-03 16:42:31 +02:00
Oran Agra
51c3ff8d75 Modules hooks: complete missing hooks for the initial set of hooks
* replication hooks: role change, master link status, replica online/offline
* persistence hooks: saving, loading, loading progress
* misc hooks: cron loop, shutdown, module loaded/unloaded
* change the way hooks test work, and add tests for all of the above

startLoading() now gets flag indicating what is loaded.
stopLoading() now gets an indication of success or failure.
adding startSaving() and stopSaving() with similar args and role.
2019-10-29 17:59:09 +02:00
antirez
8cccc6b468 Modules hooks: don't call the hook for fake clients. 2019-10-23 18:39:53 +02:00
antirez
94c50314f1 Modules hooks: fix sub event in calls from client change. 2019-10-23 18:39:53 +02:00
antirez
019ac37e9c Modules hooks: fix define / linker issues. Implement one test event. 2019-10-23 18:39:53 +02:00
Yossi Gottlieb
0db3b0a0ff Merge remote-tracking branch 'upstream/unstable' into tls 2019-10-16 17:08:07 +03:00
Oran Agra
6b6294807c TLS: Implement support for write barrier. 2019-10-07 21:06:30 +03:00
Oran Agra
5a47794606 diskless replication rdb transfer uses pipe, and writes to sockets form the parent process.
misc:
- handle SSL_has_pending by iterating though these in beforeSleep, and setting timeout of 0 to aeProcessEvents
- fix issue with epoll signaling EPOLLHUP and EPOLLERR only to the write handlers. (needed to detect the rdb pipe was closed)
- add key-load-delay config for testing
- trim connShutdown which is no longer needed
- rioFdsetWrite -> rioFdWrite - simplified since there's no longer need to write to multiple FDs
- don't detect rdb child exited (don't call wait3) until we detect the pipe is closed
- Cleanup bad optimization from rio.c, add another one
2019-10-07 21:06:30 +03:00
Yossi Gottlieb
b087dd1db6 TLS: Connections refactoring and TLS support.
* Introduce a connection abstraction layer for all socket operations and
integrate it across the code base.
* Provide an optional TLS connections implementation based on OpenSSL.
* Pull a newer version of hiredis with TLS support.
* Tests, redis-cli updates for TLS support.
2019-10-07 21:06:13 +03:00
antirez
cb384127d1 RESP3: Use verbatim in CLIENT LIST. 2019-09-18 18:52:13 +02:00
antirez
e57067b677 More strict checks and better comments in flushSlaveOutputBuffers().
Related to #6296.
2019-08-31 14:46:24 +02:00
antirez
2303ba1441 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-08-31 14:41:28 +02:00
antirez
ca5ccc58c5 Improve comment in flushSlavesOutputBuffers(). 2019-08-31 14:40:09 +02:00
zhaozhao.zz
d3f7eb0c87 networking: flushSlavesOutputBuffers bugfix 2019-08-02 17:17:19 +08:00
Salvatore Sanfilippo
3f1c84751a
Merge pull request #6196 from AngusP/resp3
[RESP3, Minor] in networking.c double representation for -infiinity leaves out comma
2019-07-12 13:08:46 +02:00
antirez
c7aaf8db4d Client side caching: implement CLIENT GETREDIR.
This subcommand may simplify the writing of Redis client libraries
using the tracking feature and/or improve observability and debugging
capabilities.
2019-07-10 18:17:10 +02:00
antirez
46edb55de9 Client side caching: implement trackingInvalidateKey(). 2019-07-03 19:16:20 +02:00
antirez
db16a861a1 Client side caching: CLIENT TRACKING subcommand. 2019-07-03 11:58:20 +02:00
Angus Pearson
593f6656c1 RESP3 double representation for -infinity is ,-inf\r\n, not -inf\r\n 2019-07-02 14:28:48 +01:00
antirez
45d64f229e Client side caching: fields and flags for tracking mode. 2019-06-29 20:08:41 -04:00
zhaozhao.zz
1398fac3f1 Threaded IO: use main thread to handle read work 2019-05-21 11:42:10 +08:00
zhaozhao.zz
8b33975944 Threaded IO: use main thread to handle write work 2019-05-21 11:37:13 +08:00
antirez
9724ca4e22 Make comment in getClientOutputBufferMemoryUsage() describing the present. 2019-05-13 17:30:06 +02:00
antirez
3d053dbb6d Threaded IO: handleClientsWithPendingReadsUsingThreads top comment. 2019-05-06 18:02:51 +02:00
antirez
1c0c436757 Threaded IO: ability to disable reads from threaded path. 2019-05-06 18:02:51 +02:00
antirez
90d720e7a5 Threaded IO: put fflush() inside tio_debug conditional. 2019-05-06 18:02:51 +02:00
antirez
6ab6a97fe6 Threaded IO: parsing WIP 2: refactoring to parse from thread. 2019-05-06 18:02:51 +02:00
antirez
647a66ebba Threaded IO: parsing WIP 1: set current_client in a better scoped way. 2019-05-06 18:02:51 +02:00
antirez
463ccf8664 Threaded IO: logging should be safe in I/O threads.
Potentially it is possible that we get interleaved writes, even if
serverLog() makes sure to write into a buffer and then use printf(), so
even this should be ok. However in general POSIX guarantees that writing
to the same file pointer object from multiple threads is safe. Anyway
currently we *reopen* the file at each call, but for the standard output
logging.

The logging functions actually also access global configuration while
performing the log (for instance in order to check the log level, the
log filename and so forth), however dunring the I/O threads execution
we cannot alter such shared state in any way.
2019-05-06 18:02:51 +02:00
antirez
8d7d2be24f Threaded IO: process read queue before stopping threads. 2019-05-06 18:02:51 +02:00
antirez
63a0ffd36a Threaded IO: read side WIP 3. 2019-05-06 18:02:51 +02:00
antirez
a2245f8ff1 Threaded IO: read side WIP 2. 2019-05-06 18:02:51 +02:00
antirez
dd5b105c73 Threaded IO: read side WIP. 2019-05-06 18:02:51 +02:00
antirez
74591fb5bd Threaded IO: hide more debugging printfs under conditional. 2019-05-06 18:02:51 +02:00
antirez
9814b2a5f3 Threaded IO: make num of I/O threads configurable. 2019-05-06 18:02:51 +02:00
antirez
30091dc29f Threaded IO: use main thread if num of threads is 1. 2019-05-06 18:02:51 +02:00
Ubuntu
9bf7f302a7 Threaded IO: stop threads when no longer needed + C11 in Makefile.
Now threads are stopped even when the connections drop immediately to
zero, not allowing the networking code to detect the condition and stop
the threads. serverCron() will handle that.
2019-05-06 18:02:51 +02:00
antirez
ea35a81c42 Threaded IO: 3rd version: use the mutex only to stop the thread. 2019-05-06 18:02:51 +02:00
antirez
6f4f36c0fb Threaded IO: second attempt without signaling conditions. 2019-05-06 18:02:51 +02:00
antirez
a2dbd9bd97 Threaded IO: allow to disable debug printf. 2019-05-06 18:02:51 +02:00
antirez
f468e653b5 Threaded IO: implement handleClientsWithPendingWritesUsingThreads().
This is just an experiment for now, there are a couple of race
conditions, mostly harmless for the performance gain experiment that
this commit represents so far.

The general idea here is to take Redis single threaded and instead
fan-out on expansive kernel calls: write(2) in this case, but the same
concept could be easily implemented for read(2) and protcol parsing.

However just threading writes like in this commit, is enough to evaluate
if the approach is sounding.
2019-05-06 18:02:51 +02:00
Salvatore Sanfilippo
843de8b786
Merge pull request #5971 from devnexen/unstable
build fix
2019-04-26 17:32:21 +02:00
David Carlier
4de88828d9 build fix 2019-03-28 06:38:16 +00:00
Oran Agra
acba2fc9b4 slave corrupts replication stream when module blocked client uses large reply (or POSTPONED_ARRAY)
when redis appends the blocked client reply list to the real client, it didn't
bother to check if it is in fact the master client. so a slave executing that
module command will send replies to the master, causing the master to send the
slave error responses, which will mess up the replication offset
(slave will advance it's replication offset, and the master does not)
2019-03-24 14:17:37 +02:00
Oran Agra
29b0a57695 diskless fork kept streaming RDB to a disconnected slave 2019-03-21 20:24:52 +02:00
antirez
68c75f248e Gopher: reply in gopher mode only if argv[0] starts with slash.
As documented but never implemented.
2019-02-27 22:20:31 +01:00
antirez
a7780f716e Merge branch 'gopher' into unstable 2019-02-25 18:16:58 +01:00
antirez
21f92e9e34 RESP3: SETNAME option for HELLO. 2019-02-25 16:56:58 +01:00
antirez
d4d15315a8 RESP3: AUTH option for HELLO. 2019-02-25 16:55:16 +01:00
antirez
5748439770 RESP3: refactoring of CLIENT SETNAME to implement SETNAME in HELLO. 2019-02-25 16:51:49 +01:00
antirez
3b420034bb RESP3: allow HELLO to be used with version = 2. 2019-02-25 16:41:00 +01:00
antirez
87594a7470 ACL: move AUTH implementation in acl.c. 2019-02-25 16:33:38 +01:00
antirez
e00b22e090 Gopher: initial request handling. 2019-02-21 23:13:08 +01:00
Madelyn Olson
9131fc56d6 Refactored manual computation of object length 2019-02-21 21:35:00 +00:00
antirez
d5e4a7f439 ACL: when client->user is NULL the client is a superuser.
Related to #5832.
2019-02-12 09:44:30 +01:00
zhaozhao.zz
0f42447a0e ACL: show client's user 2019-02-12 16:03:58 +08:00
antirez
c8391388c2 ACL: remove server.requirepass + some refactoring. 2019-01-18 11:49:30 +01:00
antirez
35fe59935e ACL: automatically authenticate the nopass default user. 2019-01-15 17:57:49 +01:00
antirez
aced0328e3 ACL: avoid a radix tree lookup for the default user. 2019-01-11 11:32:41 +01:00
antirez
4278104acc ACL: add a reference to the user in each client. 2019-01-10 16:34:13 +01:00
antirez
f5d918b2bb ACL: HELLO should stop if the user is not authenticated. 2019-01-09 17:00:30 +01:00
antirez
709a6612eb RESP3: addReplyString() -> addReplyProto().
The function naming was totally nuts. Let's fix it as we break PRs
anyway with RESP3 refactoring and changes.
2019-01-09 17:00:30 +01:00
antirez
e291170385 RESP3: verbatim reply API + DEBUG PROTOCOL support. 2019-01-09 17:00:30 +01:00
antirez
8042afb246 RESP3: Fix addReplyBool() RESP2/3 output. 2019-01-09 17:00:30 +01:00
antirez
809e3a44a7 RESP3: addReplyBool() implemented. 2019-01-09 17:00:29 +01:00
antirez
4f0860cbfd RESP3: initial implementation of the HELLO command. 2019-01-09 17:00:29 +01:00
antirez
1a17cdfadf RESP3: addReplyNullArray() added for better RESP2 compat. 2019-01-09 17:00:29 +01:00
antirez
317f8b9d38 RESP3: most null replies converted. 2019-01-09 17:00:29 +01:00
antirez
1b7298e66a RESP3: addReplyNull() added. 2019-01-09 17:00:29 +01:00