Commit Graph

175 Commits

Author SHA1 Message Date
antirez
0b58ad301e CG: Replication WIP 1: XREADGROUP and XCLAIM propagated as XCLAIM. 2018-03-19 18:02:19 +01:00
antirez
0cf6b1e3ae CG: XINFO CONSUMERS implemented. 2018-03-15 12:54:10 +01:00
antirez
b26f03bd69 CG: XCLAIM now updates the idle time of the message. 2018-03-15 12:54:10 +01:00
antirez
1bc31666da CG: XPENDING without start/stop variant implemented. 2018-03-15 12:54:10 +01:00
antirez
388c69fe4e CG: XACK implementation. 2018-03-15 12:54:10 +01:00
antirez
bd1c11dc35 CG: add XREADGROUP in the command table. 2018-03-15 12:54:10 +01:00
antirez
58f0c000a5 CG: data structures design + XGROUP CREATE implementation. 2018-03-15 12:54:10 +01:00
antirez
432bf4770e Cluster: ability to prevent slaves from failing over their masters.
This commit, in some parts derived from PR #3041 which is no longer
possible to merge (because the user deleted the original branch),
implements the ability of slaves to have a special configuration
preventing that they try to start a failover when the master is failing.

There are multiple reasons for wanting this, and the feautre was
requested in issue #3021 time ago.

The differences between this patch and the original PR are the
following:

1. The flag is saved/loaded on the nodes configuration.
2. The 'myself' node is now flag-aware, the flag is updated as needed
   when the configuration is changed via CONFIG SET.
3. The flag name uses NOFAILOVER instead of NO_FAILOVER to be consistent
   with existing NOADDR.
4. The redis.conf documentation was rewritten.

Thanks to @deep011 for the original patch.
2018-03-14 14:01:38 +01:00
antirez
ffde73c57d Track number of logically expired keys still in memory.
This commit adds two new fields in the INFO output, stats section:

expired_stale_perc:0.34
expired_time_cap_reached_count:58

The first field is an estimate of the number of keys that are yet in
memory but are already logically expired. They reason why those keys are
yet not reclaimed is because the active expire cycle can't spend more
time on the process of reclaiming the keys, and at the same time nobody
is accessing such keys. However as the active expire cycle runs, while
it will eventually have to return to the caller, because of time limit
or because there are less than 25% of keys logically expired in each
given database, it collects the stats in order to populate this INFO
field.

Note that expired_stale_perc is a running average, where the current
sample accounts for 5% and the history for 95%, so you'll see it
changing smoothly over time.

The other field, expired_time_cap_reached_count, counts the number
of times the expire cycle had to stop, even if still it was finding a
sizeable number of keys yet to expire, because of the time limit.
This allows people handling operations to understand if the Redis
server, during mass-expiration events, is able to collect keys fast
enough usually. It is normal for this field to increment during mass
expires, but normally it should very rarely increment. When instead it
constantly increments, it means that the current workloads is using
a very important percentage of CPU time to expire keys.

This feature was created thanks to the hints of Rashmi Ramesh and
Bart Robinson from Twitter. In private email exchanges, they noted how
it was important to improve the observability of this parameter in the
Redis server. Actually in big deployments, the amount of keys that are
yet to expire in each server, even if they are logically expired, may
account for a very big amount of wasted memory.
2018-02-19 11:12:49 +01:00
antirez
e1e0bbe04d Remove useless comment from serverCron().
The behavior is well specified by the code itself.
2018-01-17 11:23:41 +01:00
Salvatore Sanfilippo
a18e4c964e
Merge pull request #4546 from hqin6/unstable
fixbug for #4545 dead loop aof rewrite
2018-01-17 11:21:55 +01:00
heqin
3d3faa0a19 fixbug for #4545 dead loop aof rewrite 2018-01-17 18:08:30 +08:00
antirez
8075572207 New config options about protocol prefixed with "proto".
Related to #4568.
2018-01-11 11:27:41 +01:00
Salvatore Sanfilippo
c2fa4b8a60
Merge pull request #4568 from oranagra/restore_size_limit
fix RESTORE command size limits
2018-01-11 11:20:02 +01:00
Salvatore Sanfilippo
e954742fbd
Merge pull request #3356 from yusaku/fix-module-firstkey
Fix the firstkey, lastkey, and keystep of moduleCommand
2018-01-09 18:58:31 +01:00
Oran Agra
b509a14c3e Add config options for max-bulk-len and max-querybuf-len mainly to support RESTORE of large keys 2017-12-29 12:43:48 +02:00
heqin
365e6c45ea fixbug for #4545 dead loop aof rewrite 2017-12-18 17:59:03 +08:00
antirez
522760fac7 Change indentation and other minor details of PR #4489.
The main change introduced by this commit is pretending that help
arrays are more text than code, thus indenting them at level 0. This
improves readability, and is an old practice when defining arrays of
C strings describing text.

Additionally a few useless return statements are removed, and the HELP
subcommand capitalized when printed to the user.
2017-12-06 12:05:14 +01:00
Itamar Haber
8b51121998 Merge remote-tracking branch 'upstream/unstable' into help_subcommands 2017-12-05 18:14:59 +02:00
Salvatore Sanfilippo
c56fbb246c
Merge pull request #4508 from trevor211/fixNotes
fix some notes
2017-12-05 15:47:05 +01:00
Salvatore Sanfilippo
6830fa25e3
Merge pull request #4488 from itamarhaber/debug_arity
Standardizes arity handling of DEBUG
2017-12-05 15:29:42 +01:00
WuYunlong
3f232ebfb8 fix some notes 2017-12-05 14:41:16 +08:00
antirez
9bb18e5438 Streams: XRANGE REV option -> XREVRANGE command. 2017-12-01 10:24:25 +01:00
antirez
19b06935d5 Streams: fix XADD API and keyspace notifications.
XADD was suboptimal in the first incarnation of the command, not being
able to accept an ID (very useufl for replication), nor options for
having capped streams.

The keyspace notification for streams was not implemented.
2017-12-01 10:24:24 +01:00
antirez
6468cb2e82 Streams: fix XREAD ready-key signaling.
With lists we need to signal only on key creation, but streams can
provide data to clients listening at every new item added.
To make this slightly more efficient we now track different classes of
blocked clients to avoid signaling keys when there is nobody listening.
A typical case is when the stream is used as a time series DB and
accessed only by range with XRANGE.
2017-12-01 10:24:24 +01:00
antirez
110041825c Streams: XREAD get-keys method. 2017-12-01 10:24:24 +01:00
antirez
4a377cecd8 Streams: initial work to use blocking lists logic for streams XREAD. 2017-12-01 10:24:24 +01:00
antirez
ec9bbe96bf Streams: XLEN command. 2017-12-01 10:24:24 +01:00
antirez
79866a6361 Streams: 12 commits squashed into the initial Streams implementation. 2017-12-01 10:24:24 +01:00
antirez
3b9be93fda Prevent corruption of server.executable after DEBUG RESTART.
Doing the following ended with a broken server.executable:

1. Start Redis with src/redis-server
2. Send CONFIG SET DIR /tmp/
3. Send DEBUG RESTART

At this point we called execve with an argv[0] that is no longer related
to the new path. So after the restart the absolute path of the
executable is recomputed in the wrong way. With this fix we pass the
absolute path already computed as argv[0].
2017-11-30 18:30:06 +01:00
antirez
d8f8701032 Be more verbose when DEBUG RESTART fails. 2017-11-30 18:08:21 +01:00
Itamar Haber
59d52f7fab Standardizes the 'help' subcommand
This adds a new `addReplyHelp` helper that's used by commands
when returning a help text. The following commands have been
touched: DEBUG, OBJECT, COMMAND, PUBSUB, SCRIPT and SLOWLOG.

WIP

Fix entry command table entry for OBJECT for HELP option.

After #4472 the command may have just 2 arguments.

Improve OBJECT HELP descriptions.

See #4472.

WIP 2

WIP 3
2017-11-28 21:15:45 +02:00
Itamar Haber
8c7f90e91e Standardizes arity handling of DEBUG 2017-11-28 18:18:45 +02:00
antirez
b412c544fd Fix entry command table entry for OBJECT for HELP option.
After #4472 the command may have just 2 arguments.
2017-11-27 13:16:07 +01:00
Salvatore Sanfilippo
9d86ae4597
Merge pull request #4412 from soloestoy/bugfix-psync2
PSYNC2: safe free backlog when reach the time limit and others
2017-11-24 10:56:18 +01:00
antirez
de914ede93 Modules: fix for scripting replication of modules commands.
See issue #4466 / #4467.
2017-11-23 15:14:17 +01:00
zhaozhao.zz
57bd8feb8d rehash: handle one db until finished 2017-11-21 09:49:42 +01:00
zhaozhao.zz
b8579c225c PSYNC2: clarify the scenario when repl_stream_db can be -1 2017-11-02 10:45:33 +08:00
antirez
c1c99e9f4e PSYNC2: Fix the way replication info is saved/loaded from RDB.
This commit attempts to fix a number of bugs reported in #4316.
They are related to the way replication info like replication ID,
offsets, and currently selected DB in the master client, are stored
and loaded by Redis. In order to avoid inconsistencies the changes in
this commit try to enforce that:

1. Replication information are only stored when the RDB file is
generated by a slave that has a valid 'master' client, so that we can
always extract the currently selected DB.
2. When replication informations are persisted in the RDB file, all the
info for a successful PSYNC or nothing is persisted.
3. The RDB replication informations are only loaded if the instance is
configured as a slave, otherwise a master can start with IDs that relate
to a different history of the data set, and stil retain such IDs in the
future while receiving unrelated writes.
2017-09-19 23:03:39 +02:00
Oran Agra
b122cadc66 Flush append only buffers before existing.
when SHUTDOWN command is recived it is possible that some of the recent
command were not yet flushed from the AOF buffer, and the server
experiences data loss at shutdown.
2017-09-17 07:22:16 +03:00
Salvatore Sanfilippo
34a79c353f Merge pull request #3935 from itamarhaber/module-cmdstats
Changes command stats iteration to being dict-based
2017-08-02 12:51:26 +02:00
antirez
fc7ecd8d35 AOF check utility: ability to check files with RDB preamble. 2017-07-10 13:38:23 +02:00
antirez
f9fac7f777 Avoid closing invalid FDs to make Valgrind happier. 2017-07-05 15:40:25 +02:00
antirez
f8547e53f0 Added GEORADIUS(BYMEMBER)_RO variants for read-only operations.
Issue #4084 shows how for a design error, GEORADIUS is a write command
because of the STORE option. Because of this it does not work
on readonly slaves, gets redirected to masters in Redis Cluster even
when the connection is in READONLY mode and so forth.

To break backward compatibility at this stage, with Redis 4.0 to be in
advanced RC state, is problematic for the user base. The API can be
fixed into the unstable branch soon if we'll decide to do so in order to
be more consistent, and reease Redis 5.0 with this incompatibility in
the future. This is still unclear.

However, the ability to scale GEO queries in slaves easily is too
important so this commit adds two read-only variants to the GEORADIUS
and GEORADIUSBYMEMBER command: GEORADIUS_RO and GEORADIUSBYMEMBER_RO.
The commands are exactly as the original commands, but they do not
accept the STORE and STOREDIST options.
2017-06-30 10:03:37 +02:00
antirez
01a4b9892d HMSET and MSET implementations unified. HSET now variadic.
This is the first step towards getting rid of HMSET which is a command
that does not make much sense once HSET is variadic, and has a saner
return value.
2017-06-29 17:38:46 +02:00
Suraj Narkhede
d303bca587 Fix brpop command table entry and redirect blocked clients. 2017-06-22 23:52:00 -07:00
xuzhou
530fcf8687 Fix set with ex/px option when propagated to aof 2017-06-16 17:51:38 +08:00
antirez
53cb27b1d7 SLOWLOG: log offending client address and name. 2017-06-15 12:57:54 +02:00
Qu Chen
4740424049 Implement getKeys procedure for georadius and georadiusbymember
commands.
2017-06-14 18:15:48 +02:00
antirez
e91b81c612 More informative -MISCONF error message. 2017-05-19 12:03:30 +02:00