Commit Graph

337 Commits

Author SHA1 Message Date
Oran Agra
6b6294807c TLS: Implement support for write barrier. 2019-10-07 21:06:30 +03:00
Oran Agra
5a47794606 diskless replication rdb transfer uses pipe, and writes to sockets form the parent process.
misc:
- handle SSL_has_pending by iterating though these in beforeSleep, and setting timeout of 0 to aeProcessEvents
- fix issue with epoll signaling EPOLLHUP and EPOLLERR only to the write handlers. (needed to detect the rdb pipe was closed)
- add key-load-delay config for testing
- trim connShutdown which is no longer needed
- rioFdsetWrite -> rioFdWrite - simplified since there's no longer need to write to multiple FDs
- don't detect rdb child exited (don't call wait3) until we detect the pipe is closed
- Cleanup bad optimization from rio.c, add another one
2019-10-07 21:06:30 +03:00
Yossi Gottlieb
b087dd1db6 TLS: Connections refactoring and TLS support.
* Introduce a connection abstraction layer for all socket operations and
integrate it across the code base.
* Provide an optional TLS connections implementation based on OpenSSL.
* Pull a newer version of hiredis with TLS support.
* Tests, redis-cli updates for TLS support.
2019-10-07 21:06:13 +03:00
antirez
06d490342f RESP3: fix cases of NULL reported instead of empty aggregate. 2019-09-02 12:50:47 +02:00
antirez
0e9b5adbd3 Replication: clarify why repl_put_online_on_ack exists at all. 2019-08-05 17:38:15 +02:00
Salvatore Sanfilippo
fc5c2052b5
Merge pull request #6266 from madolson/dev-unstable-hide-auth-and-hello
Hide HELLO and AUTH from slowlog and monitor
2019-07-31 11:12:46 +02:00
Salvatore Sanfilippo
f42846e8c7
Merge pull request #6264 from oranagra/modules_api_aux_rdb
Implement module api for aux data in rdb
2019-07-30 11:50:44 +02:00
antirez
32efd9adf8 Client side caching: config option for table fill rate. 2019-07-24 11:35:01 +02:00
antirez
c98e7717bb Client side caching: show tracking slots usage in INFO. 2019-07-23 11:02:14 +02:00
antirez
9268493e8d Client side caching: implement full slot limit function. 2019-07-23 10:57:22 +02:00
Madelyn Olson
7d21754710 Hide HELLO and AUTH from slowlog and monitor 2019-07-22 22:53:15 -07:00
Oran Agra
3b6aeea44c Implement module api for aux data in rdb
Other changes:
* fix memory leak in error handling of rdb loading of type OBJ_MODULE
2019-07-22 21:15:33 +03:00
antirez
c41f94d2a3 Client side caching: split invalidation into key / slot. 2019-07-22 18:59:53 +02:00
zhaozhao.zz
6191ea90a1 Client side caching: implement trackingInvalidateKeysOnFlush() 2019-07-17 20:33:52 +08:00
Oran Agra
2de544cfcc diskless replication on slave side (don't store rdb to file), plus some other related fixes
The implementation of the diskless replication was currently diskless only on the master side.
The slave side was still storing the received rdb file to the disk before loading it back in and parsing it.

This commit adds two modes to load rdb directly from socket:
1) when-empty
2) using "swapdb"
the third mode of using diskless slave by flushdb is risky and currently not included.

other changes:
--------------
distinguish between aof configuration and state so that we can re-enable aof only when sync eventually
succeeds (and not when exiting from readSyncBulkPayload after a failed attempt)
also a CONFIG GET and INFO during rdb loading would have lied

When loading rdb from the network, don't kill the server on short read (that can be a network error)

Fix rdb check when performed on preamble AOF

tests:
run replication tests for diskless slave too
make replication test a bit more aggressive
Add test for diskless load swapdb
2019-07-08 15:37:48 +03:00
Salvatore Sanfilippo
722446510f
Merge pull request #6116 from AngusP/scan-types
SCAN: New Feature `SCAN cursor [TYPE type]` modifier suggested in issue #6107
2019-07-08 12:53:34 +02:00
Angus Pearson
6eb52e200c Change typeNameCanonicalize -> getObjectTypeName, and other style changes 2019-07-08 11:04:37 +01:00
antirez
6b29f2d83d Client side caching: RESP2 support. 2019-07-05 12:24:28 +02:00
antirez
46edb55de9 Client side caching: implement trackingInvalidateKey(). 2019-07-03 19:16:20 +02:00
antirez
506764b3f8 Client side caching: hook inside call() for tracking. 2019-07-03 12:42:16 +02:00
antirez
c29f3bcf2e Client side caching: enable tracking mode. 2019-06-30 06:19:08 -04:00
antirez
45d64f229e Client side caching: fields and flags for tracking mode. 2019-06-29 20:08:41 -04:00
Angus Pearson
38cd5fd9f6 Spelling cannonical -> canonical 2019-06-13 17:49:33 +01:00
Angus Pearson
e2adea2188 Add char* typeNameCanonicalize(robj*) to remove duplicate code between SCAN and TYPE commands,
and to keep OBJ_* enum to string canonicalization in one place.
2019-06-10 17:41:44 +01:00
Salvatore Sanfilippo
e633254ccf
Merge pull request #6053 from soloestoy/enhance-aof-fsync-everysec
aof: enhance AOF_FSYNC_EVERYSEC, more details in #5985
2019-05-10 18:06:40 +02:00
antirez
1c0c436757 Threaded IO: ability to disable reads from threaded path. 2019-05-06 18:02:51 +02:00
antirez
6ab6a97fe6 Threaded IO: parsing WIP 2: refactoring to parse from thread. 2019-05-06 18:02:51 +02:00
antirez
63a0ffd36a Threaded IO: read side WIP 3. 2019-05-06 18:02:51 +02:00
antirez
dd5b105c73 Threaded IO: read side WIP. 2019-05-06 18:02:51 +02:00
antirez
9814b2a5f3 Threaded IO: make num of I/O threads configurable. 2019-05-06 18:02:51 +02:00
Ubuntu
9bf7f302a7 Threaded IO: stop threads when no longer needed + C11 in Makefile.
Now threads are stopped even when the connections drop immediately to
zero, not allowing the networking code to detect the condition and stop
the threads. serverCron() will handle that.
2019-05-06 18:02:51 +02:00
antirez
f468e653b5 Threaded IO: implement handleClientsWithPendingWritesUsingThreads().
This is just an experiment for now, there are a couple of race
conditions, mostly harmless for the performance gain experiment that
this commit represents so far.

The general idea here is to take Redis single threaded and instead
fan-out on expansive kernel calls: write(2) in this case, but the same
concept could be easily implemented for read(2) and protcol parsing.

However just threading writes like in this commit, is enough to evaluate
if the approach is sounding.
2019-05-06 18:02:51 +02:00
zhaozhao.zz
bcac165fab aof: enhance AOF_FSYNC_EVERYSEC, more details in #5985 2019-04-29 14:38:28 +08:00
Oran Agra
acba2fc9b4 slave corrupts replication stream when module blocked client uses large reply (or POSTPONED_ARRAY)
when redis appends the blocked client reply list to the real client, it didn't
bother to check if it is in fact the master client. so a slave executing that
module command will send replies to the master, causing the master to send the
slave error responses, which will mess up the replication offset
(slave will advance it's replication offset, and the master does not)
2019-03-24 14:17:37 +02:00
Salvatore Sanfilippo
5e8caca036
Merge pull request #5944 from yossigo/command-filtering
Command Filtering API
2019-03-22 17:43:49 +01:00
Salvatore Sanfilippo
122f42844a
Merge pull request #5945 from dvirsky/miss_notification
Added keyspace miss notifications support
2019-03-22 17:41:00 +01:00
Dvir Volk
bc269c85e1 remove extra linebreak 2019-03-21 12:48:37 +02:00
Dvir Volk
99c2fe0bcf added special flag for keyspace miss notifications 2019-03-21 11:47:14 +02:00
Yossi Gottlieb
c3e187190b Initial command filter experiment. 2019-03-18 13:50:34 +02:00
Yossi Gottlieb
a88264d934 Add RedisModule_GetKeyNameFromIO(). 2019-03-15 10:23:27 +02:00
Salvatore Sanfilippo
0cce98f2f9
Merge pull request #5834 from guybe7/trim_module_sds
Trim SDS free space of retained module strings
2019-03-14 12:41:31 +01:00
antirez
6fd4af1f12 Merge branch 'sharedapi' into unstable 2019-03-14 11:24:48 +01:00
Yuan Zhou
feb4ebff45 server.h: remove dead code
hashTypeTryObjectEncoding() is not used now
2019-03-07 18:38:27 +08:00
Salvatore Sanfilippo
88b720672c
Merge pull request #5877 from vattezhang/unstable_sentinel_cmd
fix: fix sentinel command table and new flags format
2019-02-27 15:45:03 +01:00
vattezhang
9d632230b6 fix: fix sentinel command table and new flags format 2019-02-27 21:35:58 +08:00
antirez
a7780f716e Merge branch 'gopher' into unstable 2019-02-25 18:16:58 +01:00
antirez
3b420034bb RESP3: allow HELLO to be used with version = 2. 2019-02-25 16:41:00 +01:00
antirez
e00b22e090 Gopher: initial request handling. 2019-02-21 23:13:08 +01:00
antirez
3de9ccf190 Gopher: config setting to turn support on/off. 2019-02-21 17:28:53 +01:00
zhaozhao.zz
14507457a0 ACL: show categories in COMMAND reply
Adding another new filed categories at the end of
command reply, it's easy to read and distinguish
flags and categories, also compatible with old format.
2019-02-14 00:13:01 +08:00