Commit Graph

6613 Commits

Author SHA1 Message Date
antirez
13ff7bc3ef CG: fix RDB saving when there are no consumer groups. 2018-03-15 12:54:10 +01:00
antirez
267f7f2c97 Streams: fix error description for XADD when specified ID is small. 2018-03-15 12:54:10 +01:00
antirez
0a6780e560 CG: XCLAIM initial draft. 2018-03-15 12:54:10 +01:00
antirez
00a29b1a81 Make addReplyError...() family functions able to get error codes.
Now you can use:

    addReplyError("-MYERRORCODE some message");

If the error code is omitted, the behavior is like in the past,
the generic -ERR will be used.
2018-03-15 12:54:10 +01:00
antirez
54b9a0e612 CG: test XACK remaining features. 2018-03-15 12:54:10 +01:00
antirez
5f8c57f28a CG: test XACK ability to remove items from the PELs. 2018-03-15 12:54:10 +01:00
antirez
f5799e728a CG: test XPENDING ability to return pending items. 2018-03-15 12:54:10 +01:00
antirez
c2ecac4746 CG: test XGROUPREAD abilities. 2018-03-15 12:54:10 +01:00
antirez
18ab0e31f3 CG: test group creation. 2018-03-15 12:54:10 +01:00
antirez
c9d86c2b16 CG: More specific duplicated group error. 2018-03-15 12:54:10 +01:00
antirez
9f60a6bcee CG: RDB loading, fix inverted conditional. 2018-03-15 12:54:10 +01:00
antirez
f4e1a4de25 CG: RDB loading first implementation. 2018-03-15 12:54:10 +01:00
antirez
db7a5f23b4 CG: RDB saving part 2, consumers. 2018-03-15 12:54:10 +01:00
antirez
8fb6048ed0 CG: RDB saving part 1, metadata and PEL. 2018-03-15 12:54:10 +01:00
antirez
e76fb4ab25 CG: XPENDING should not create consumers and obey to count. 2018-03-15 12:54:10 +01:00
antirez
f3708af7f9 CG: XPENDING with start/stop/count variant implemented. 2018-03-15 12:54:10 +01:00
antirez
1bc31666da CG: XPENDING without start/stop variant implemented. 2018-03-15 12:54:10 +01:00
antirez
b65fe09bb8 CG: Now XREADGROUP + blocking operations work. 2018-03-15 12:54:10 +01:00
antirez
5ad29325fe CG: XACK should return zero when nothing is processed. 2018-03-15 12:54:10 +01:00
antirez
388c69fe4e CG: XACK implementation. 2018-03-15 12:54:10 +01:00
antirez
5bbd117c29 CG: XREADGROUP can fetch data from the consumer PEL. 2018-03-15 12:54:10 +01:00
antirez
aa808394f6 CG: first draft of streamReplyWithRangeFromConsumerPEL(). 2018-03-15 12:54:10 +01:00
antirez
bbec4569a5 CG: Fix order of calls in streamReplyWithRange().
We need to check if we are going to serve the request via the PEL before
inserting a deferred array len in the client output buffer.
2018-03-15 12:54:10 +01:00
antirez
41809fd969 CG: creation of NACK entries in PELs. 2018-03-15 12:54:10 +01:00
antirez
1ffb6723f5 CG: fix XREADGROUP ">" special ID parsing due to missing "continue". 2018-03-15 12:54:10 +01:00
antirez
6c0af37b6e CG: streamCompareID() + group last_id updating. 2018-03-15 12:54:10 +01:00
antirez
86fe8fde20 CG: consumer lookup + initial streamReplyWithRange() work to supprot CG. 2018-03-15 12:54:10 +01:00
antirez
ccdae09046 CG: add & populate group+consumer in the blocking state. 2018-03-15 12:54:10 +01:00
antirez
b8e5232161 CG: fix parsing in XREADGROUP and streamLookupCG() NULL check. 2018-03-15 12:54:10 +01:00
antirez
bd1c11dc35 CG: add XREADGROUP in the command table. 2018-03-15 12:54:10 +01:00
antirez
2bbb2bf427 CG: XGROUPREAD group option parsing and groups lookup. 2018-03-15 12:54:10 +01:00
antirez
1fafe7def1 CG: fix raxFind() retval check in streamCreateCG(). 2018-03-15 12:54:10 +01:00
antirez
58f0c000a5 CG: data structures design + XGROUP CREATE implementation. 2018-03-15 12:54:10 +01:00
antirez
d8207d09ee Cluster: add test for the nofailover flag. 2018-03-14 16:30:32 +01:00
antirez
432bf4770e Cluster: ability to prevent slaves from failing over their masters.
This commit, in some parts derived from PR #3041 which is no longer
possible to merge (because the user deleted the original branch),
implements the ability of slaves to have a special configuration
preventing that they try to start a failover when the master is failing.

There are multiple reasons for wanting this, and the feautre was
requested in issue #3021 time ago.

The differences between this patch and the original PR are the
following:

1. The flag is saved/loaded on the nodes configuration.
2. The 'myself' node is now flag-aware, the flag is updated as needed
   when the configuration is changed via CONFIG SET.
3. The flag name uses NOFAILOVER instead of NO_FAILOVER to be consistent
   with existing NOADDR.
4. The redis.conf documentation was rewritten.

Thanks to @deep011 for the original patch.
2018-03-14 14:01:38 +01:00
antirez
84b281209a Stream: update the listpack pointer in streamTrimByLength(). 2018-03-01 17:26:02 +01:00
antirez
efcbc01fbd Remove warning from lpGet snprintf(). 2018-03-01 15:26:27 +01:00
antirez
d63caaa820 redis-cli: fix missed unit in array. Change define name. 2018-03-01 15:06:41 +01:00
Salvatore Sanfilippo
83b5b5a476
Merge pull request #4714 from charsyam/feature/fix-out-of-index-range
[BugFix] Fix out of array index range for findBigKeys in redis-cli
2018-03-01 03:39:15 -08:00
antirez
3a5bf75ede Actually use ae_flags to add AE_BARRIER if needed.
Many thanks to @Plasma that spotted this problem reviewing the code.
2018-02-28 18:03:51 +01:00
Salvatore Sanfilippo
7a73db7512
Merge pull request #4715 from charsyam/feature/refactoring-make-condition-clear-for-rdb
[BugFix] fix calculation length in rdbSaveAuxField
2018-02-27 10:15:27 -08:00
antirez
92696e49d2 expireIfNeeded() needed a top comment documenting the behavior. 2018-02-27 16:44:43 +01:00
antirez
b00c4ffab5 expireIfNeeded() comment: claim -> pretend. 2018-02-27 16:37:37 +01:00
charsyam
76386c48b8 refactoring-make-condition-clear-for-rdb 2018-02-27 21:55:20 +09:00
charsyam
6168d5a1a6 fix-out-of-index-range-for-redis-cli-findbigkey 2018-02-27 21:46:19 +09:00
antirez
956350ef89 ae.c: insetad of not firing, on AE_BARRIER invert the sequence.
AE_BARRIER was implemented like:

    - Fire the readable event.
    - Do not fire the writabel event if the readable fired.

However this may lead to the writable event to never be called if the
readable event is always fired. There is an alterantive, we can just
invert the sequence of the calls in case AE_BARRIER is set. This commit
does that.
2018-02-27 13:06:42 +01:00
antirez
75987431f0 AOF: fix a bug that may prevent proper fsyncing when fsync=always.
In case the write handler is already installed, it could happen that we
serve the reply of a query in the same event loop cycle we received it,
preventing beforeSleep() from guaranteeing that we do the AOF fsync
before sending the reply to the client.

The AE_BARRIER mechanism, introduced in a previous commit, prevents this
problem. This commit makes actual use of this new feature to fix the
bug.
2018-02-27 13:06:42 +01:00
antirez
533d0e0375 Cluster: improve crash-recovery safety after failover auth vote.
Add AE_BARRIER to the writable event loop so that slaves requesting
votes can't be served before we re-enter the event loop in the next
iteration, so clusterBeforeSleep() will fsync to disk in time.
Also add the call to explicitly fsync, given that we modified the last
vote epoch variable.
2018-02-27 13:06:42 +01:00
antirez
548e478e40 ae.c: introduce the concept of read->write barrier.
AOF fsync=always, and certain Redis Cluster bus operations, require to
fsync data on disk before replying with an acknowledge.
In such case, in order to implement Group Commits, we want to be sure
that queries that are read in a given cycle of the event loop, are never
served to clients in the same event loop iteration. This way, by using
the event loop "before sleep" callback, we can fsync the information
just one time before returning into the event loop for the next cycle.
This is much more efficient compared to calling fsync() multiple times.

Unfortunately because of a bug, this was not always guaranteed: the
actual way the events are installed was the sole thing that could
control. Normally this problem is hard to trigger when AOF is enabled
with fsync=always, because we try to flush the output buffers to the
socekt directly in the beforeSleep() function of Redis. However if the
output buffers are full, we actually install a write event, and in such
a case, this bug could happen.

This change to ae.c modifies the event loop implementation to make this
concept explicit. Write events that are registered with:

    AE_WRITABLE|AE_BARRIER

Are guaranteed to never fire after the readable event was fired for the
same file descriptor. In this way we are sure that data is persisted to
disk before the client performing the operation receives an
acknowledged.

However note that this semantics does not provide all the guarantees
that one may believe are automatically provided. Take the example of the
blocking list operations in Redis.

With AOF and fsync=always we could have:

    Client A doing: BLPOP myqueue 0
    Client B doing: RPUSH myqueue a b c

In this scenario, Client A will get the "a" elements immediately after
the Client B RPUSH will be executed, even before the operation is persisted.
However when Client B will get the acknowledge, it can be sure that
"b,c" are already safe on disk inside the list.

What to note here is that it cannot be assumed that Client A receiving
the element is a guaranteed that the operation succeeded from the point
of view of Client B.

This is due to the fact that the barrier exists within the same socket,
and not between different sockets. However in the case above, the
element "a" was not going to be persisted regardless, so it is a pretty
synthetic argument.
2018-02-27 13:06:42 +01:00
Salvatore Sanfilippo
d8830200b4
Merge pull request #3828 from oranagra/sdsnewlen_pr
add SDS_NOINIT option to sdsnewlen to avoid unnecessary memsets.
2018-02-27 04:04:32 -08:00