Commit Graph

247 Commits

Author SHA1 Message Date
Itamar Haber
efe92ee546
Removes dead code (#7642)
Appears to be handled by server.stream_node_max_bytes in reality.
2020-08-11 08:11:47 +03:00
Wen Hui
f33acb3f02
Add SignalModifiedKey hook in XGROUP CREATE with MKSTREAM option (#7562) 2020-07-29 08:22:54 +03:00
WuYunlong
86fed3fe09
Refactor streamAppendItem() by deleting redundancy condition. (#7487)
It will never happen that "lp != NULL && lp_bytes >= server.stream_node_max_bytes".
Assume that "lp != NULL && lp_bytes >= server.stream_node_max_bytes",
we got the following conditions:
a. lp != NULL
b. lp_bytes >= server.stream_node_max_bytes

If server.stream_node_max_bytes is 0, given condition a, condition b is always satisfied
If server.stream_node_max_bytes is not 0, given condition a and condition b, the codes just a
	few lines above set lp to NULL, a controdiction with condition a

So that condition b is recundant. We could delete it safely.
2020-07-20 13:14:27 +03:00
杨博东
8596d483bc
Stream avoid duplicate parse id (#7450) 2020-07-16 08:57:27 +03:00
WuYunlong
93bdbf5aa4
Fix command help for unexpected options (#7476) 2020-07-15 12:38:22 +03:00
Guy Benoish
bce3d08c66 XPENDING should not update consumer's seen-time
Same goes for XGROUP DELCONSUMER (But in this case, it doesn't
have any visible effect)
2020-05-04 11:40:42 +03:00
Guy Benoish
057865a6a5 XINFO STREAM FULL should have a default COUNT of 10 2020-04-28 18:09:03 +03:00
Salvatore Sanfilippo
e0de7c0852
Merge pull request #7134 from guybe7/xstate_command
Extend XINFO STREAM output
2020-04-28 16:31:00 +02:00
Guy Benoish
1e2aee3919 Extend XINFO STREAM output
Introducing XINFO STREAM <key> FULL
2020-04-28 13:03:43 +03:00
antirez
57a0c9c98d Also use propagate() in streamPropagateGroupID(). 2020-04-24 10:13:20 +02:00
antirez
12cd67b7ca Minor aesthetic changes to #7135. 2020-04-23 16:13:45 +02:00
Salvatore Sanfilippo
2a3d933cc6
Merge pull request #7135 from valentinogeron/optimize-xreadgroup-noack
XREADGROUP with NOACK should propagate only one XGROUP SETID command
2020-04-23 16:12:08 +02:00
Valentino Geron
6fd2d7cfee XREADGROUP with NOACK should propagate only one XGROUP SETID command 2020-04-22 17:03:51 +03:00
antirez
94f2e7f9f9 Tracking: NOLOOP internals implementation. 2020-04-21 10:51:46 +02:00
antirez
c479eace45 Fix XCLAIM propagation in AOF/replicas for blocking XREADGROUP.
See issue #7105.
2020-04-17 12:38:15 +02:00
antirez
839bbcb039 RESP3: change streams items from maps to arrays.
Streams items are similar to dictionaries, however they preserve both
the order, and allow for duplicated field names. So a map is not a
semantically sounding way to deal with this.

https://twitter.com/antirez/status/1248261087553880069
2020-04-10 10:12:26 +02:00
Salvatore Sanfilippo
cbf212f981
Merge pull request #7030 from valentinogeron/xread-in-lua
XREAD and XREADGROUP should not be allowed from scripts when BLOCK op…
2020-04-03 11:14:13 +02:00
Salvatore Sanfilippo
dfef407499
Merge pull request #7029 from valentinogeron/fix-xack
XACK should be executed in a "all or nothing" fashion.
2020-04-02 11:23:23 +02:00
Guy Benoish
12d7479144 streamReplyWithRange: Redundant XSETIDs to replica
propagate_last_id is declared outside of the loop but used
only from within the loop. Once it's '1' it will never go
back to '0' and will replicate XSETID even for IDs that
don't actually change the last_id.
While not a serious bug (XSETID always used group->last_id
so there's no risk), it does causes redundant traffic
between master and its replicas
2020-03-30 10:53:03 +03:00
Valentino Geron
9a1843ef2d XREAD and XREADGROUP should not be allowed from scripts when BLOCK option is being used 2020-03-26 15:46:31 +02:00
Valentino Geron
1547d72cf3 XACK should be executed in a "all or nothing" fashion.
First, we must parse the IDs, so that we abort ASAP.
The return value of this command cannot be an error if
the client successfully acknowledged some messages,
so it should be executed in a "all or nothing" fashion.
2020-03-26 15:40:23 +02:00
Salvatore Sanfilippo
cf31602bce
Merge pull request #6677 from guybe7/deocde_id_once
streamReplyWithRangeFromConsumerPEL: Redundant streamDecodeID
2020-02-27 18:09:48 +01:00
Salvatore Sanfilippo
049969971e
Merge pull request #6921 from guybe7/fix_xclaim_propagate
XREADGROUP should propagate XCALIM/SETID in MULTI/EXEC
2020-02-27 09:59:51 +01:00
Guy Benoish
a6c7e61e61 XREADGROUP should propagate XCALIM/SETID in MULTI/EXEC
Use built-in alsoPropagate mechanism that wraps commands
in MULTI/EXEC before sending them to replica/AOF
2020-02-23 19:13:15 +05:30
Guy Benoish
770cb0ba97 XGROUP DESTROY should unblock XREADGROUP with -NOGROUP 2020-02-19 08:25:31 +05:30
Salvatore Sanfilippo
bb93686754
Merge pull request #6703 from guybe7/blocking_xread_empty_reply
Blocking XREAD[GROUP] should always reply with valid data (or timeout)
2020-01-09 17:32:14 +01:00
antirez
7080a28efc XCLAIM: Create the consumer only on successful claims.
Fixes #6744.
2020-01-08 10:10:11 +01:00
Guy Benoish
a351e74fe9 Blocking XREAD[GROUP] should always reply with valid data (or timeout)
This commit solves the following bug:
127.0.0.1:6379> XGROUP CREATE x grp $ MKSTREAM
OK
127.0.0.1:6379> XADD x 666 f v
"666-0"
127.0.0.1:6379> XREADGROUP GROUP grp Alice BLOCK 0 STREAMS x >
1) 1) "x"
   2) 1) 1) "666-0"
         2) 1) "f"
            2) "v"
127.0.0.1:6379> XADD x 667 f v
"667-0"
127.0.0.1:6379> XDEL x 667
(integer) 1
127.0.0.1:6379> XREADGROUP GROUP grp Alice BLOCK 0 STREAMS x >
1) 1) "x"
   2) (empty array)

The root cause is that we use s->last_id in streamCompareID
while we should use the last *valid* ID
2019-12-30 10:06:01 +05:30
Guy Benoish
1f75ce30df Stream: Handle streamID-related edge cases
This commit solves several edge cases that are related to
exhausting the streamID limits: We should correctly calculate
the succeeding streamID instead of blindly incrementing 'seq'
This affects both XREAD and XADD.

Other (unrelated) changes:
Reply with a better error message when trying to add an entry
to a stream that has exhausted last_id
2019-12-26 15:31:37 +05:30
Guy Benoish
5e0fe4c77c streamReplyWithRangeFromConsumerPEL: Redundant streamDecodeID 2019-12-18 12:27:37 +05:30
Salvatore Sanfilippo
64c2508ee3
Merge branch 'unstable' into rm_get_server_info 2019-11-21 10:06:15 +01:00
Salvatore Sanfilippo
f1f259de5b
Merge pull request #6547 from guybe7/module_api_streams
Support streams in general module API functions
2019-11-21 10:03:50 +01:00
antirez
5b80a41cad Remove additional space from comment. 2019-11-19 11:05:55 +01:00
Guy Benoish
4a12047c61 XADD with ID 0-0 stores an empty key
Calling XADD with 0-0 or 0 would result in creating an
empty key and storing it in the database.
Even worse, because XADD will reply with error the action
will not be replicated, creating a master-replica
inconsistency
2019-11-13 16:47:30 +05:30
Guy Benoish
1833d008b3 Support streams in general module API functions
Fixes GitHub issue #6492
Added stream support in RM_KeyType and RM_ValueLength.
Also moduleDelKeyIfEmpty was updated, even though it has
no effect now (It will be relevant when stream type direct
API will be coded - i.e. RM_StreamAdd)
2019-11-06 15:52:19 +05:30
Loris Cro
b12d2f65d6 fix unreported overflow in autogerenared stream IDs 2019-11-04 16:36:06 +01:00
Oran Agra
0423309768 Add RM_ServerInfoGetFieldUnsigned
rename RM_ServerInfoGetFieldNumerical RM_ServerInfoGetFieldSigned
move string2ull to util.c
fix leak in RM_GetServerInfo when duplicate info fields exist
2019-11-04 08:50:29 +02:00
Guy Benoish
2fae0192e8 Fix usage of server.stream_node_max_* 2019-10-10 09:47:48 +02:00
James Rouzier
3ccdcbc088 Fix start and end key initialize 2019-04-11 12:19:02 -04:00
Salvatore Sanfilippo
2537b2140f
Merge pull request #5787 from soloestoy/bugfix-xgroup-create-with-mkstream
Streams: checkType before XGROUP CREATE
2019-03-13 12:34:29 +01:00
Steve Webster
dfcb227b50 Only increment delivery count if JUSTID option is omitted 2019-03-12 20:27:53 +00:00
Steve Webster
f1e7df4b7c Increment delivery counter on XCLAIM unless RETRYCOUNT specified
The XCLAIM docs state the XCLAIM increments the delivery counter for
messages. This PR makes the code match the documentation - which seems
like the desired behaviour - whilst still allowing RETRYCOUNT to be
specified manually.

My understanding of the way streamPropagateXCLAIM() works is that this
change will safely propagate to replicas since retry count is pulled
directly from the streamNACK struct.

Fixes #5194
2019-03-08 17:09:11 +00:00
zhaozhao.zz
645d44d545 Streams: checkType before XGROUP CREATE
Fix issue #5785, in case create group on a key is not stream.
2019-01-16 19:19:14 +08:00
antirez
8a0391fbc9 RESP3: t_stream.c updated. 2019-01-09 17:00:29 +01:00
antirez
2bd6802fa1 Stream: fix XREADGROUP history reading of deleted messages.
This commit fixes #5570. It is a similar bug to one fixed a few weeks
ago and is due to the range API to be called with NULL as "end ID"
parameter instead of repeating again the start ID, to be sure that we
selectively issue the entry with a given ID, or we get zero returned
(and we know we should emit a NULL reply).
2018-11-19 17:00:34 +01:00
antirez
29251f58e2 Streams: fix XREADGROUP history reading when CG last_id is low.
This fixes the issue reported in #5570.
This was fixed the hard way, that is, propagating more information to
the lower level API about this being a request to read just the history,
so that the code is simpler and less likely to regress.
2018-11-19 16:41:27 +01:00
antirez
3830ef2483 t_stream.c comment resized to 80 cols. 2018-11-19 16:26:02 +01:00
antirez
6ba50784b5 Fix XCLAIM missing entry bug.
This bug had a double effect:

1. Sometimes entries may not be emitted, producing broken protocol where
the array length was greater than the emitted entires, blocking the
client waiting for more data.

2. Some other time the right entry was claimed, but a wrong entry was
returned to the client.

This fix should correct both the instances.
2018-11-05 13:17:32 +01:00
antirez
514bbdd670 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2018-11-05 13:07:14 +01:00
antirez
e7c579e1fe Improve streamReplyWithRange() top comment. 2018-11-05 13:06:01 +01:00