Commit Graph

8452 Commits

Author SHA1 Message Date
antirez
32efd9adf8 Client side caching: config option for table fill rate. 2019-07-24 11:35:01 +02:00
antirez
a67d0411e7 Remove experimental warning from diskless replication. 2019-07-24 11:33:53 +02:00
wubostc
48f4cfb601
Reduce the calling stack 2019-07-24 16:22:26 +08:00
Salvatore Sanfilippo
dbee7ca007
Merge pull request #6265 from madolson/dev-unstable-remove-create-object
Removed unnecessary creation of Redis objects
2019-07-23 17:05:10 +02:00
antirez
c98e7717bb Client side caching: show tracking slots usage in INFO. 2019-07-23 11:02:14 +02:00
antirez
9268493e8d Client side caching: implement full slot limit function. 2019-07-23 10:57:22 +02:00
Salvatore Sanfilippo
47ce1cebeb
Merge pull request #6267 from soloestoy/opt-claim-trackingtable
Client side caching: do not reclaim tracking table if it's empty
2019-07-23 09:43:41 +02:00
zhaozhao.zz
9012e587ca Client side caching: do not reclaim tracking table if it's empty 2019-07-23 15:25:00 +08:00
Madelyn Olson
7d21754710 Hide HELLO and AUTH from slowlog and monitor 2019-07-22 22:53:15 -07:00
Madelyn Olson
8fe173452a Removed unecessary creation of Redis objects 2019-07-22 22:52:16 -07:00
Oran Agra
3b6aeea44c Implement module api for aux data in rdb
Other changes:
* fix memory leak in error handling of rdb loading of type OBJ_MODULE
2019-07-22 21:15:33 +03:00
antirez
c41f94d2a3 Client side caching: split invalidation into key / slot. 2019-07-22 18:59:53 +02:00
antirez
09c06698e9 Client side caching: redis-cli ability to enable tracking.
This is extremely useful in order to simulate an high load of requests
about different keys, and force Redis to track a lot of informations
about several clients, to simulate real world workloads.
2019-07-22 18:45:47 +02:00
antirez
436d02dd7b Move signalFlushedDb() into a better place.
Now that the call also invalidates client side caching slots, it is
important that after an internal flush operation we both send the
notifications to the clients and, at the same time, are able to reclaim
the memory of the tracking table. This may even fix a few edge cases
related to MULTI/EXEC + WATCH during resync, not sure, but in general
looks more correct.
2019-07-22 12:37:47 +02:00
antirez
7d3992a80a Client side caching: reclaim the tracking table on FLUSHALL. 2019-07-22 12:33:49 +02:00
antirez
842b44dc46 Client side caching: call the invalidation functions always.
Otherwise what happens is that the tracking table will never get garbage
collected if there are no longer clients with tracking enabled.
Now the invalidation function immediately checks if there is any table
allocated, otherwise it returns ASAP, so the overhead when the feature
is not used should be near zero.
2019-07-22 12:29:54 +02:00
antirez
f850201c64 Client side caching: don't hash the key if not needed. 2019-07-22 12:10:51 +02:00
antirez
e00442eb13 Clinet side caching: take count of used caching slots. 2019-07-22 11:47:44 +02:00
chendianqiang
004c26ec28 make memory usage consistent of robj with OBJ_ENCODING_INT 2019-07-22 17:45:30 +08:00
Oran Agra
d7d028a7a7 Allow modules to handle RDB loading errors.
This is especially needed in diskless loading, were a short read could have
caused redis to exit. now the module can handle the error and return to the
caller gracefully.

this fixes #5326
2019-07-21 18:19:32 +03:00
antirez
bc5cb168f5 RDB: fix MODULE_AUX loading by continuing to next opcode.
Thanks to @JohnSully for noticing this problem.
2019-07-19 11:12:46 +02:00
antirez
5f450e4928 RDB: make sure to abort on LZF encoding error. 2019-07-18 18:59:38 +02:00
antirez
bd0f06c18c RDB: handle encoding errors with rdbExitReportCorruptRDB().
Without such change, the diskless replicas, when loading RDB files from
the socket will not abort when a broken RDB file gets loaded. This is
potentially unsafe, because right now Redis is not able to guarantee
that encoding errors are safe from the POV of memory corruptions (for
instance the LZF library may not be safe against untrusted data?) so
better to abort when the RDB file we are going to load is corrupted.

Instead I/O errors are still returned to the caller without aborting,
so that in case of short read the diskless replica can try again.
2019-07-18 18:51:45 +02:00
Salvatore Sanfilippo
41ed85bd26
Merge pull request #6252 from soloestoy/tracking-flushdb
Tracking flushdb
2019-07-18 16:41:37 +02:00
antirez
947319cadd RDB: update rdbLoadRio comment about EOF condition. 2019-07-18 12:37:55 +02:00
zhaozhao.zz
f59119aad6 Client side caching: filter clients untracking 2019-07-18 16:46:19 +08:00
antirez
42b6305964 RDB: try to make error handling code more readable. 2019-07-17 17:30:02 +02:00
antirez
6b72b04a37 Rio: when in error condition avoid doing the operation. 2019-07-17 16:46:22 +02:00
antirez
5189db3d81 RDB: modify rdbReportError() var name for clarity. 2019-07-17 16:46:22 +02:00
antirez
48d91cf4cc Rio: remember read/write error conditions. 2019-07-17 16:46:22 +02:00
Oran Agra
c56b4ddc6f prevent diskless replica from terminating on short read
now that replica can read rdb directly from the socket, it should avoid exiting
on short read and instead try to re-sync.

this commit tries to have minimal effects on non-diskless rdb reading.
and includes a test that tries to trigger this scenario on various read cases.
2019-07-17 16:46:22 +02:00
Oran Agra
56258c6b7d Module API for Forking
* create module API for forking child processes.
* refactor duplicate code around creating and tracking forks by AOF and RDB.
* child processes listen to SIGUSR1 and dies exitFromChild in order to
  eliminate a valgrind warning of unhandled signal.
* note that BGSAVE error reply has changed.

valgrind error is:
  Process terminating with default action of signal 10 (SIGUSR1)
2019-07-17 16:40:24 +03:00
zhaozhao.zz
6191ea90a1 Client side caching: implement trackingInvalidateKeysOnFlush() 2019-07-17 20:33:52 +08:00
Oran Agra
6e07fac405 RM_Log - add support for logging without a context or context without module
for instance detached thread safe contexts, or various callbacks that don't
provide a context.
2019-07-17 15:08:18 +03:00
antirez
241d18d954 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-17 12:26:38 +02:00
Salvatore Sanfilippo
5827b69861
Merge pull request #6185 from AlexRenCN/unstable
fix readme.md,Redis data types should add `t_stream.c`.
2019-07-16 18:33:55 +02:00
Salvatore Sanfilippo
4b2579a064
Merge pull request #5779 from madolson/dev-unstable-geo-bugfix
Fixed a rounding bug in geo.tcl
2019-07-15 23:41:57 +02:00
Salvatore Sanfilippo
3f1c84751a
Merge pull request #6196 from AngusP/resp3
[RESP3, Minor] in networking.c double representation for -infiinity leaves out comma
2019-07-12 13:08:46 +02:00
Salvatore Sanfilippo
6100a82a3a
Merge pull request #5738 from bmerry/fix-swapdb-int-overflow
Make dbSwapDatabases take args as long
2019-07-12 12:36:33 +02:00
antirez
54c4e7f86c Vertically compact code in aofWrite(). 2019-07-12 12:18:33 +02:00
antirez
2a99f199bc Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-10 19:48:44 +02:00
antirez
dad0e916ff Client side caching: filter clients no longer there. 2019-07-10 19:46:46 +02:00
Salvatore Sanfilippo
7567323071
Merge pull request #6198 from oranagra/skiplist-mem-usage
missing per-skiplist overheads in MEMORY USAGE
2019-07-10 19:43:24 +02:00
antirez
c7aaf8db4d Client side caching: implement CLIENT GETREDIR.
This subcommand may simplify the writing of Redis client libraries
using the tracking feature and/or improve observability and debugging
capabilities.
2019-07-10 18:17:10 +02:00
antirez
923e4fb312 Client side caching: add tracking clients in INFO. 2019-07-10 18:08:31 +02:00
antirez
7c9f6ebc8f Diskless replica: fix disklessLoadRestoreBackups() bug. 2019-07-10 12:36:26 +02:00
antirez
3bbb9a1413 Diskless replica: refactoring of DBs backups. 2019-07-10 11:42:26 +02:00
antirez
3300e98ff0 Diskless replica: fix mispelled var name. 2019-07-10 09:34:21 +02:00
antirez
dfcbeaf115 Diskless replica: a few aesthetic changes to rio.c 2019-07-08 18:39:59 +02:00
antirez
81b18fa3a0 Diskless replica: a few aesthetic changes to replication.c. 2019-07-08 18:32:47 +02:00