Commit Graph

5013 Commits

Author SHA1 Message Date
artix
be94e89031 Cluster Manager: fixed expected slots calculation (rebalance)
Cluster Manager: fixed argument parsing after --cluster-weight
2018-04-23 16:28:48 +02:00
artix
2f31545beb Cluster Manager: fixed bug when parsing CLUSTER NODES reply (clusterManagerNodeLoadInfo) 2018-04-23 16:28:48 +02:00
artix
5bc2c98789 Cluster Manager: code improvements and more comments added. 2018-04-23 16:28:48 +02:00
artix
5f358dae33 Cluster Manager: set-timeout command 2018-04-23 16:28:48 +02:00
artix
52f17f6f8e - Cluster Manager: del-node command.
- Cluster Manager: fixed bug in clusterManagerNodeWithLeastReplicas
2018-04-23 16:28:48 +02:00
artix
aeaf6ee1c3 Cluster Manager: add-node command. 2018-04-23 16:28:48 +02:00
artix
efa51f1617 Cluster Manager: added clusterManagerCheckCluster to import command 2018-04-23 16:28:48 +02:00
artix
1f548359cb Cluster Manager: import command 2018-04-23 16:28:48 +02:00
artix
3f8a4adb49 Cluster Manager: fix command. 2018-04-23 16:28:48 +02:00
artix
6d1a7cec23 Cluster Manager: rebalance command 2018-04-23 16:28:48 +02:00
artix
adebee8adc clusterManagerAddSlots: changed the way ADDSLOTS command is built 2018-04-23 16:28:48 +02:00
artix
a4a1c7bb52 ClusterManager: fixed --cluster-from 'all' parsing 2018-04-23 16:28:48 +02:00
Artix
5b3d73b3d7 Cluster Manager: fixed some memory error 2018-04-23 16:28:48 +02:00
artix
220375b424 Fixed memory write error in clusterManagerGetConfigSignature 2018-04-23 16:28:48 +02:00
artix
99da9c9508 Cluster Manager: reshard command, fixed slots
parsing bug and other minor bugs.
2018-04-23 16:28:48 +02:00
artix
7d609ff952 Cluster Manager:
- Almost all Cluster Manager related code moved to
  the same section.
- Many macroes converted to functions
- Added various comments
- Little code restyling
2018-04-23 16:28:48 +02:00
artix
4e0c2f9c3c - Fixed bug in clusterManagerGetAntiAffinityScore
- Code improvements
2018-04-23 16:28:48 +02:00
artix
18910013cd Cluster Manager: colorized output 2018-04-23 16:28:48 +02:00
artix
307d995f75 Cluster Manager: improved cleanup/error handling in various functions 2018-04-23 16:28:48 +02:00
artix
2d677e2bf3 Cluster Manager: 'call' command. 2018-04-23 16:28:48 +02:00
artix
1ad1f00163 Cluster Manager: CLUSTER_MANAGER_NODE_CONNECT macro 2018-04-23 16:28:48 +02:00
artix
2f48d62423 ClusterManager: added replicas count to clusterManagerNode 2018-04-23 16:28:48 +02:00
artix
c002b95d89 Cluster Manager: cluster is considered consistent if only one node has been found 2018-04-23 16:28:48 +02:00
artix
4cc8de1a37 Cluster Manager: reply error catch for MEET command 2018-04-23 16:28:48 +02:00
artix
65d37960e7 Cluster Manager: slots coverage check. 2018-04-23 16:28:48 +02:00
artix
b3e0ca3412 - Cluster Manager: fixed various memory leaks
- Cluster Manager: fixed flags assignment in
  clusterManagerNodeLoadInfo
2018-04-23 16:28:48 +02:00
artix
8c7ad80f9f Added check for open slots (clusterManagerCheckCluster) 2018-04-23 16:28:47 +02:00
artix
486c7af7b8 Cluster Manager: 'create', 'info' and 'check' commands 2018-04-23 16:28:47 +02:00
artix
ad911a338a Cluster Manager mode 2018-04-23 16:28:47 +02:00
zhaozhao.zz
24036b4d32 RDB: expand dict if needed when rdb load object 2018-04-22 22:30:44 +08:00
antirez
e6b0e8d9ec Streams: XTRIM command added. 2018-04-19 16:25:29 +02:00
antirez
19ae809458 Streams: XADD, handle signaling of write in different ways. 2018-04-19 16:11:43 +02:00
antirez
aba76320d5 Streams: XDEL command. 2018-04-18 13:12:09 +02:00
antirez
9c149bf1f1 Streams: higher level "delete by ID" API for streams. 2018-04-17 17:55:56 +02:00
antirez
24ac2b4c74 Streams: iterator entry deletion abilities. 2018-04-17 17:18:00 +02:00
zhaozhao.zz
775adf544c Modules Timer API: fix wrong raxInsert() usage 2018-04-14 01:20:02 +08:00
antirez
e07af6a2b7 Modules API: Add call to get the blocked client handle from the context.
This is useful in the reply and timeout callback, if the module wants to
do some cleanup of the blocked client handle that may be stored around
in the module-private data structures.
2018-04-13 13:48:11 +02:00
antirez
da0e192277 Modules API: moduleGetReplyClient() refactoring. 2018-04-13 13:42:49 +02:00
antirez
404160a271 Modules API: blocked client disconnection callback. 2018-04-12 13:21:48 +02:00
antirez
005c932f22 Modules API: fix timer example. 2018-04-12 13:00:18 +02:00
antirez
4e53624052 Modules API: fix OOM_WARNING flag implementation. 2018-04-11 16:25:54 +02:00
antirez
0afac6939a Modules API: OOM_WARNING flags.
In some modules it may be useful to have an idea about being near to
OOM. Anyway additionally an explicit call to get the fill ratio will be
added in the future.
2018-04-11 16:22:52 +02:00
antirez
de7de53e64 getMaxmemoryState() fixed and improved. 2018-04-11 12:48:26 +02:00
dejun.xdj
b6b00f35ca Return more accurate message when specified ID is invalid 2018-04-10 17:11:10 +08:00
dejun.xdj
3c20b3fc24 Bugfix: xadd command ID parse
strictly check the string to be converted,
strtoull() in libc may not set errno to EINVAL when the string contains
invalid digits.
2018-04-10 17:05:48 +08:00
dejun.xdj
07125b8d22 Bugfix: xadd comand <field,value> arity check
missing parenthesis causes wrong arithmetic priority.
2018-04-10 16:36:05 +08:00
antirez
f97efe0cac Modules: context flags now include OOM flag.
Plus freeMemoryIfNeeded() refactoring to improve legibility.
Please review this commit for sanity.
2018-04-09 17:44:30 +02:00
antirez
6888c1a10d freeMemoryIfNeeded() lacked a top comment. 2018-04-09 17:20:36 +02:00
antirez
9a0dbbb594 Modules: remove trailing empty spaces. 2018-04-09 17:16:55 +02:00
antirez
eaafea4828 Modules API: experimental APIs version.
This way it is possible to use conditional compilation to be compatible
with a larger amount of Redis versions, however note that this breaks
binary compatibiltiy, so the module must be compiled with the
corresponding redismodule.h file depending on the version of Redis
targeted.
2018-04-09 12:17:19 +02:00
antirez
49e098234a Modules API: blocked client free callback modified to get a context.
Note that this was an experimental API that can only be enabled with
REIDSMODULE_EXPERIMENTAL_API, so it is subject to change until its
promoted to stable API. Sorry for the breakage, it is trivial to
resolve btw. This change will not be back ported to Redis 4.0.
2018-04-09 11:54:44 +02:00
charpty
0fd2b25c8d Remove unnecessary return statements
Signed-off-by: charpty <charpty@gmail.com>
2018-04-06 18:46:24 +08:00
antirez
b2868c7b9c Modules API: RM_GetRandomBytes() / GetRandomHexChars(). 2018-04-05 13:24:22 +02:00
antirez
c75582889a Modules Cluster API: GetClusterSize() added. 2018-04-01 16:36:32 +02:00
antirez
27f9c8108c Modules Cluster API: GetMyClusterID() added. 2018-04-01 16:20:57 +02:00
antirez
19c42c901b Modules Timer API: add example of API. 2018-03-31 10:12:34 +02:00
antirez
ee982f4031 Modules Timer API: Wait at least 1 ms per iteration. Convert to ms. 2018-03-31 10:11:30 +02:00
antirez
4c11bc6cf0 Modules Timer API: fix wrong raxRemove() key argument. 2018-03-31 09:58:43 +02:00
antirez
2f7da0fd1a Modules Timer API: fix infinite loop and export API. 2018-03-31 00:44:46 +02:00
antirez
b85a465c25 Modules Timer API: timer handling implemented. 2018-03-30 22:50:21 +02:00
antirez
561039c125 Modules Timer API: initial implementation. 2018-03-30 20:40:35 +02:00
antirez
192361b562 Modules Cluster API: node API exported, example improved. 2018-03-30 17:00:45 +02:00
antirez
16178b692e Modules Cluster API: nodes list and info API. 2018-03-30 16:16:47 +02:00
antirez
83ec35770e Modules Cluster API: node information struct and flags. 2018-03-30 13:16:55 +02:00
antirez
a97df1a6e1 Modules Cluster API: make node IDs pointers constant. 2018-03-30 13:16:07 +02:00
antirez
061f03d730 Modules Cluster API: add a simple example module. 2018-03-30 12:49:45 +02:00
antirez
82004f9dbe Modules Cluster API: fix new API calls exporting. 2018-03-30 12:49:16 +02:00
antirez
b4dc782e4e Modules Cluster API: sending / receiving API first implementation. 2018-03-30 11:06:08 +02:00
antirez
0701cad3de Modules Cluster API: message bus implementation. 2018-03-29 15:13:31 +02:00
zhaozhao.zz
83cf0e3668 adjust position of _dictNextPower in dictExpand 2018-03-29 17:36:15 +08:00
antirez
8ac7af1c5d Fix ae.c when a timer finalizerProc adds an event.
While this feature is not used by Redis, ae.c implements the ability for
a timer to call a finalizer callback when an timer event is deleted.
This feature was bugged since the start, and because it was never used
we never noticed a problem. However Anthony LaTorre was using the same
library in order to implement a different system: he found a bug that he
describes as follows, and which he fixed with the patch in this commit,
sent me by private email:

    --- Anthony email ---

've found one bug in the current implementation of the timed events.
It's possible to lose track of a timed event if an event is added in
the finalizerProc of another event.

For example, suppose you start off with three timed events 1, 2, and
3. Then the linked list looks like:

3 -> 2 -> 1

Then, you run processTimeEvents and events 2 and 3 finish, so now the
list looks like:

-1 -> -1 -> 2

Now, on the next iteration of processTimeEvents it starts by deleting
the first event, and suppose this finalizerProc creates a new event,
so that the list looks like this:

4 -> -1 -> 2

On the next iteration of the while loop, when it gets to the second
event, the variable prev is still set to NULL, so that the head of the
event loop after the next event will be set to 2, i.e. after deleting
the next event the event loop will look like:

2

and the event with id 4 will be lost.

I've attached an example program to illustrate the issue. If you run
it you will see that it prints:

```
foo id = 0
spam!
```

But if you uncomment line 29 and run it again it won't print "spam!".

    --- End of email ---

Test.c source code is as follows:

    #include "ae.h"
    #include <stdio.h>

    aeEventLoop *el;

    int foo(struct aeEventLoop *el, long long id, void *data)
    {
	printf("foo id = %lld\n", id);

	return AE_NOMORE;
    }

    int spam(struct aeEventLoop *el, long long id, void *data)
    {
	printf("spam!\n");

	return AE_NOMORE;
    }

    void bar(struct aeEventLoop *el, void *data)
    {
	aeCreateTimeEvent(el, 0, spam, NULL, NULL);
    }

    int main(int argc, char **argv)
    {
	el = aeCreateEventLoop(100);

	//aeCreateTimeEvent(el, 0, foo, NULL, NULL);
	aeCreateTimeEvent(el, 0, foo, NULL, bar);

	aeMain(el);

	return 0;
    }

Anthony fixed the problem by using a linked list for the list of timers, and
sent me back this patch after he tested the code in production for some time.
The code looks sane to me, so committing it to Redis.
2018-03-28 14:11:04 +02:00
antirez
28d28ef3cf AOF: enable RDB-preamble rewriting by default.
There are too many advantages in doing this, RDB is faster to persist,
more compact, much faster to load back. The main issues here are that
the code is less tested because this was not the old default (so we are
enabling it for the new 5.0 release), and that the AOF is no longer a
trivially parsable format from now on. However the non-preamble mode
will be supported in the future as well, if new data types will be
added.
2018-03-25 11:43:30 +02:00
antirez
8b20112860 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2018-03-23 17:21:46 +01:00
antirez
1392c83fb8 CG: AOF rewriting implemented. 2018-03-23 17:21:31 +01:00
Salvatore Sanfilippo
e08d6bb37e
Merge pull request #4703 from gechunlin/unstable
Update object.c
2018-03-22 16:32:45 +01:00
Salvatore Sanfilippo
bad679137e
Merge pull request #4679 from youjiali1995/fix-lru
Some commands will udpate lfu-counter many times in one call.
2018-03-22 16:26:01 +01:00
Salvatore Sanfilippo
44f2cfa631
Merge pull request #4722 from charsyam/feature/refactoring-call-aeDeleteFileEvent-twice-in-freeClusterLink
Refactoring to call aeDeleteFileEvent twice as once
2018-03-22 16:23:40 +01:00
Salvatore Sanfilippo
b38f52169b
Merge pull request #4777 from soloestoy/avoid-double-close
anet: avoid double close
2018-03-22 16:22:32 +01:00
Salvatore Sanfilippo
3163c9cb63
Merge pull request #4781 from guybe7/block_list_notify
Make blocking list commands send keyspace notifications
2018-03-22 16:21:19 +01:00
antirez
6c4cb1670a Add top comments in two addReply*() functions. 2018-03-22 11:45:04 +01:00
antirez
b86c26b2fd Massivily simplify addReply*() functions in networking.c 2018-03-22 11:42:50 +01:00
Guy Benoish
fa00e20b16 Make blocking list commands send keyspace notifications 2018-03-22 17:22:26 +07:00
Salvatore Sanfilippo
5d8c89163c
Merge pull request #4757 from charsyam/feature/fix-invalid-listpack-type-in-makefile
fix listpack.c to listpack.o in Makefile
2018-03-22 09:20:17 +01:00
Salvatore Sanfilippo
7acf0e0a18
Merge pull request #3826 from charsyam/feature/fix_dlopen_leak
fix missed dlclose leak in module.c
2018-03-22 09:19:26 +01:00
Salvatore Sanfilippo
da621783f0
Merge pull request #4691 from oranagra/active_defrag_v2
Active defrag v2
2018-03-22 09:16:32 +01:00
charsyam
71c7477fd4 fix memory leak for streamPropagateXCLAIM 2018-03-21 17:41:59 +09:00
zhaozhao.zz
46f767aa19 fix missed call on freeaddrinfo 2018-03-21 15:34:13 +08:00
zhaozhao.zz
11baa8487c anet: avoid double close 2018-03-21 15:13:51 +08:00
antirez
233da29f4d CG: propagate XACK by incrementing server.dirty.
This should be more than enough, even if in case of partial IDs that are
not found, we send all the IDs to the slave/AOF, but this is definitely
a corner case without bad effects if not some wasted space.
2018-03-20 17:53:45 +01:00
antirez
0f2c6b6a18 Streams: improve MEMORY USAGE computation, include CGs. 2018-03-20 17:50:40 +01:00
antirez
5577130451 CG: Make XINFO Great Again (and more Redis-ish).
With XINFO out of the blue I invented a new syntax for commands never
used in Redis in the past... Let's fix it and make it

        Great Again!!11one (TM)
2018-03-20 11:52:42 +01:00
antirez
2ba9a57c9a CG: Add JUSTID to XCLAIM propagation to reduce CPU usage. 2018-03-20 11:25:49 +01:00
antirez
ad716e29c0 CG: Fix propagate() arg count in streamPropagateXCLAIM(). 2018-03-20 11:13:41 +01:00
antirez
3cae5f0321 CG: XGROUP CREATE/DELCONUSMER replication. 2018-03-20 11:07:01 +01:00
antirez
0b58ad301e CG: Replication WIP 1: XREADGROUP and XCLAIM propagated as XCLAIM. 2018-03-19 18:02:19 +01:00
antirez
36b78e8dfe Aesthetic changes to PR #4749. 2018-03-16 16:57:53 +01:00
antirez
7e3c583532 Merge branch 'hyperloglog-improvement' of https://github.com/oertl/redis into oertl-hyperloglog-improvement 2018-03-16 16:27:38 +01:00
antirez
38bc8ea531 RDB version 9. 2018-03-16 13:48:44 +01:00
antirez
8b0cfb1e66 RDB: Implement future-proof module AUX data loading. 2018-03-16 13:47:10 +01:00
zhaozhao.zz
54cae05ea7 rdb: incremental fsync when redis saves rdb 2018-03-16 00:44:50 +08:00
antirez
b1aae86fc6 RDB: make RDB check aware of LFU/LRU opcodes. 2018-03-15 16:39:33 +01:00
antirez
8176a2ee76 RDB: LRU/LFU branches missed continue. 2018-03-15 16:33:18 +01:00
antirez
1ce50a7adf RDB: Ability to load LFU/LRU info. 2018-03-15 16:24:53 +01:00
antirez
d7a5c0eb71 RDB: Ability to save LFU/LRU info.
This is a big win for caching use cases, since on reloading Redis will
still have some idea about what is worth to evict and what not.
However this only solves part of the problem because the information is
only partially propagated to slaves (on write operations). Reads will
not affect slaves LFU and LRU counters, so after a failover the eviction
decisions are kinda random until keys start to collect some aging/freq info.

However since new slaves are initially populated via RDB file transfer,
this means that if we spin up a new slave from a master, and perform an
immediate manual failover (for instance in order to upgrade the master),
the slave will have eviction informations to use for some time.

The LFU/LRU info is persisted only if the maxmemory policy is set to one
of the relevant type, even if no actual "maxmemory"  memory limit is
set.
2018-03-15 13:15:55 +01:00
antirez
6614361615 CG: XINFO STREAM. 2018-03-15 12:54:10 +01:00
antirez
d7d8cd0b2f CG: XINFO GROUPS + output format changes.
XINFO is mainly an observability command that will be used more by
humans than computers, and even when used by computers it will be a very
low traffic command. For this reason the format was changed in order to
have field names. They'll consume some bandwidth and CPU cycles, but in
this context this is much better than having to understand what the
numbers in the output array are.
2018-03-15 12:54:10 +01:00
antirez
0cf6b1e3ae CG: XINFO CONSUMERS implemented. 2018-03-15 12:54:10 +01:00
antirez
67eeeb0b10 Streams: fix XREAD missing check for NULL object. 2018-03-15 12:54:10 +01:00
antirez
8d8755c7b5 CG: throw an error if XREADGROUP is used without GROUP. 2018-03-15 12:54:10 +01:00
antirez
d4f81ebdba CG: XGROUP DELCONSUMER implemented. 2018-03-15 12:54:10 +01:00
antirez
9b423ae237 CG: remove stale comment. XREADGROUP is now implemented. 2018-03-15 12:54:10 +01:00
antirez
e6d434c167 CG: implement RETRYCOUNT in XCLAIM. 2018-03-15 12:54:10 +01:00
antirez
f3d9520ccb CG: fix CG RDB loading not found conditional. 2018-03-15 12:54:10 +01:00
antirez
34614a1508 CG: NACK created by XCLAIM FORCE must be set in global PEL. 2018-03-15 12:54:10 +01:00
antirez
596264aee9 CG: implement XCLAIM FORCE option. 2018-03-15 12:54:10 +01:00
antirez
b26f03bd69 CG: XCLAIM now updates the idle time of the message. 2018-03-15 12:54:10 +01:00
antirez
f7d4c3acdf Streams: trap more errors in stream loading + RDB check type name. 2018-03-15 12:54:10 +01:00
antirez
8727b4845b CG: XCLAIM, use minidle and fix array len. 2018-03-15 12:54:10 +01:00
antirez
09e3b3b975 CG: remove unused argument from streamReplyWithRangeFromConsumerPEL(). 2018-03-15 12:54:10 +01:00
antirez
13ff7bc3ef CG: fix RDB saving when there are no consumer groups. 2018-03-15 12:54:10 +01:00
antirez
267f7f2c97 Streams: fix error description for XADD when specified ID is small. 2018-03-15 12:54:10 +01:00
antirez
0a6780e560 CG: XCLAIM initial draft. 2018-03-15 12:54:10 +01:00
antirez
00a29b1a81 Make addReplyError...() family functions able to get error codes.
Now you can use:

    addReplyError("-MYERRORCODE some message");

If the error code is omitted, the behavior is like in the past,
the generic -ERR will be used.
2018-03-15 12:54:10 +01:00
antirez
c9d86c2b16 CG: More specific duplicated group error. 2018-03-15 12:54:10 +01:00
antirez
9f60a6bcee CG: RDB loading, fix inverted conditional. 2018-03-15 12:54:10 +01:00
antirez
f4e1a4de25 CG: RDB loading first implementation. 2018-03-15 12:54:10 +01:00
antirez
db7a5f23b4 CG: RDB saving part 2, consumers. 2018-03-15 12:54:10 +01:00
antirez
8fb6048ed0 CG: RDB saving part 1, metadata and PEL. 2018-03-15 12:54:10 +01:00
antirez
e76fb4ab25 CG: XPENDING should not create consumers and obey to count. 2018-03-15 12:54:10 +01:00
antirez
f3708af7f9 CG: XPENDING with start/stop/count variant implemented. 2018-03-15 12:54:10 +01:00
antirez
1bc31666da CG: XPENDING without start/stop variant implemented. 2018-03-15 12:54:10 +01:00
antirez
b65fe09bb8 CG: Now XREADGROUP + blocking operations work. 2018-03-15 12:54:10 +01:00
antirez
5ad29325fe CG: XACK should return zero when nothing is processed. 2018-03-15 12:54:10 +01:00
antirez
388c69fe4e CG: XACK implementation. 2018-03-15 12:54:10 +01:00
antirez
5bbd117c29 CG: XREADGROUP can fetch data from the consumer PEL. 2018-03-15 12:54:10 +01:00
antirez
aa808394f6 CG: first draft of streamReplyWithRangeFromConsumerPEL(). 2018-03-15 12:54:10 +01:00
antirez
bbec4569a5 CG: Fix order of calls in streamReplyWithRange().
We need to check if we are going to serve the request via the PEL before
inserting a deferred array len in the client output buffer.
2018-03-15 12:54:10 +01:00
antirez
41809fd969 CG: creation of NACK entries in PELs. 2018-03-15 12:54:10 +01:00
antirez
1ffb6723f5 CG: fix XREADGROUP ">" special ID parsing due to missing "continue". 2018-03-15 12:54:10 +01:00
antirez
6c0af37b6e CG: streamCompareID() + group last_id updating. 2018-03-15 12:54:10 +01:00
antirez
86fe8fde20 CG: consumer lookup + initial streamReplyWithRange() work to supprot CG. 2018-03-15 12:54:10 +01:00
antirez
ccdae09046 CG: add & populate group+consumer in the blocking state. 2018-03-15 12:54:10 +01:00
antirez
b8e5232161 CG: fix parsing in XREADGROUP and streamLookupCG() NULL check. 2018-03-15 12:54:10 +01:00
antirez
bd1c11dc35 CG: add XREADGROUP in the command table. 2018-03-15 12:54:10 +01:00
antirez
2bbb2bf427 CG: XGROUPREAD group option parsing and groups lookup. 2018-03-15 12:54:10 +01:00
antirez
1fafe7def1 CG: fix raxFind() retval check in streamCreateCG(). 2018-03-15 12:54:10 +01:00
antirez
58f0c000a5 CG: data structures design + XGROUP CREATE implementation. 2018-03-15 12:54:10 +01:00
charsyam
c76f890209 fix listpack.c to listpack.o in Makefile 2018-03-15 20:32:08 +09:00
Otmar Ertl
15d7e61701 fixed compilation error when using clang as reported by michael-grunder 2018-03-14 21:00:06 +01:00
antirez
432bf4770e Cluster: ability to prevent slaves from failing over their masters.
This commit, in some parts derived from PR #3041 which is no longer
possible to merge (because the user deleted the original branch),
implements the ability of slaves to have a special configuration
preventing that they try to start a failover when the master is failing.

There are multiple reasons for wanting this, and the feautre was
requested in issue #3021 time ago.

The differences between this patch and the original PR are the
following:

1. The flag is saved/loaded on the nodes configuration.
2. The 'myself' node is now flag-aware, the flag is updated as needed
   when the configuration is changed via CONFIG SET.
3. The flag name uses NOFAILOVER instead of NO_FAILOVER to be consistent
   with existing NOADDR.
4. The redis.conf documentation was rewritten.

Thanks to @deep011 for the original patch.
2018-03-14 14:01:38 +01:00
Oran Agra
806736cdf9 Adding real allocator fragmentation to INFO and MEMORY command + active defrag test
other fixes / improvements:
- LUA script memory isn't taken from zmalloc (taken from libc malloc)
  so it can cause high fragmentation ratio to be displayed (which is false)
- there was a problem with "fragmentation" info being calculated from
  RSS and used_memory sampled at different times (now sampling them together)

other details:
- adding a few more allocator info fields to INFO and MEMORY commands
- improve defrag test to measure defrag latency of big keys
- increasing the accuracy of the defrag test (by looking at real grag info)
  this way we can use an even lower threshold and still avoid false positives
- keep the old (total) "fragmentation" field unchanged, but add new ones for spcific things
- add these the MEMORY DOCTOR command
- deduct LUA memory from the rss in case of non jemalloc allocator (one for which we don't "allocator active/used")
- reduce sampling rate of the rss and allocator info
2018-03-12 15:08:52 +02:00
Oran Agra
be1b4aa9aa active defrag v2
- big keys are not defragged in one go from within the dict scan
  instead they are scanned in parts after the main dict hash bucket is done.
- add latency monitor sample for defrag
- change default active-defrag-cycle-min to induce lower latency
- make active defrag start a new scan right away if needed, so it's easier
  (for the test suite) to detect when it's done
- make active defrag quick the current cycle after each db / big key
- defrag  some non key long term global allocations
- some refactoring for smaller functions and more reusable code
- during dict rehashing, one scan iteration of the dict, can end up scanning
  one bucket in the smaller dict and many many buckets in the larger dict.
  so waiting for 16 scan iterations before checking the time, may be much too long.
2018-03-12 15:07:43 +02:00
Otmar Ertl
97bde9f623 use all 64 bits of the hash value instead of 63 2018-03-11 09:18:00 +01:00
Otmar Ertl
44698f45e7 made constant static 2018-03-10 20:44:20 +01:00
Otmar Ertl
633983d479 improved definition of HLL_Q 2018-03-10 20:22:42 +01:00
Otmar Ertl
1e9a774871 improved HyperLogLog cardinality estimation
based on method described in https://arxiv.org/abs/1702.01284
that does not rely on any magic constants
2018-03-10 20:13:21 +01:00
Otmar Ertl
6470b21f59 replaced tab by spaces 2018-03-10 20:09:41 +01:00
pan.liangp
f4eb64cd35 move get clients max buffer calculate into info clients command 2018-03-02 17:16:00 +08:00
antirez
84b281209a Stream: update the listpack pointer in streamTrimByLength(). 2018-03-01 17:26:02 +01:00
antirez
efcbc01fbd Remove warning from lpGet snprintf(). 2018-03-01 15:26:27 +01:00
antirez
d63caaa820 redis-cli: fix missed unit in array. Change define name. 2018-03-01 15:06:41 +01:00
charsyam
da7f5700cf refactoring-call-aeDeleteFileEvent-twice-in-freeClusterLink 2018-03-01 22:30:39 +09:00
charsyam
51a03f6356 fix dlopen leak 2018-03-01 21:22:42 +09:00
Salvatore Sanfilippo
83b5b5a476
Merge pull request #4714 from charsyam/feature/fix-out-of-index-range
[BugFix] Fix out of array index range for findBigKeys in redis-cli
2018-03-01 03:39:15 -08:00
antirez
3a5bf75ede Actually use ae_flags to add AE_BARRIER if needed.
Many thanks to @Plasma that spotted this problem reviewing the code.
2018-02-28 18:03:51 +01:00
Salvatore Sanfilippo
7a73db7512
Merge pull request #4715 from charsyam/feature/refactoring-make-condition-clear-for-rdb
[BugFix] fix calculation length in rdbSaveAuxField
2018-02-27 10:15:27 -08:00
antirez
92696e49d2 expireIfNeeded() needed a top comment documenting the behavior. 2018-02-27 16:44:43 +01:00
antirez
b00c4ffab5 expireIfNeeded() comment: claim -> pretend. 2018-02-27 16:37:37 +01:00
charsyam
76386c48b8 refactoring-make-condition-clear-for-rdb 2018-02-27 21:55:20 +09:00
charsyam
6168d5a1a6 fix-out-of-index-range-for-redis-cli-findbigkey 2018-02-27 21:46:19 +09:00
antirez
956350ef89 ae.c: insetad of not firing, on AE_BARRIER invert the sequence.
AE_BARRIER was implemented like:

    - Fire the readable event.
    - Do not fire the writabel event if the readable fired.

However this may lead to the writable event to never be called if the
readable event is always fired. There is an alterantive, we can just
invert the sequence of the calls in case AE_BARRIER is set. This commit
does that.
2018-02-27 13:06:42 +01:00
antirez
75987431f0 AOF: fix a bug that may prevent proper fsyncing when fsync=always.
In case the write handler is already installed, it could happen that we
serve the reply of a query in the same event loop cycle we received it,
preventing beforeSleep() from guaranteeing that we do the AOF fsync
before sending the reply to the client.

The AE_BARRIER mechanism, introduced in a previous commit, prevents this
problem. This commit makes actual use of this new feature to fix the
bug.
2018-02-27 13:06:42 +01:00
antirez
533d0e0375 Cluster: improve crash-recovery safety after failover auth vote.
Add AE_BARRIER to the writable event loop so that slaves requesting
votes can't be served before we re-enter the event loop in the next
iteration, so clusterBeforeSleep() will fsync to disk in time.
Also add the call to explicitly fsync, given that we modified the last
vote epoch variable.
2018-02-27 13:06:42 +01:00
antirez
548e478e40 ae.c: introduce the concept of read->write barrier.
AOF fsync=always, and certain Redis Cluster bus operations, require to
fsync data on disk before replying with an acknowledge.
In such case, in order to implement Group Commits, we want to be sure
that queries that are read in a given cycle of the event loop, are never
served to clients in the same event loop iteration. This way, by using
the event loop "before sleep" callback, we can fsync the information
just one time before returning into the event loop for the next cycle.
This is much more efficient compared to calling fsync() multiple times.

Unfortunately because of a bug, this was not always guaranteed: the
actual way the events are installed was the sole thing that could
control. Normally this problem is hard to trigger when AOF is enabled
with fsync=always, because we try to flush the output buffers to the
socekt directly in the beforeSleep() function of Redis. However if the
output buffers are full, we actually install a write event, and in such
a case, this bug could happen.

This change to ae.c modifies the event loop implementation to make this
concept explicit. Write events that are registered with:

    AE_WRITABLE|AE_BARRIER

Are guaranteed to never fire after the readable event was fired for the
same file descriptor. In this way we are sure that data is persisted to
disk before the client performing the operation receives an
acknowledged.

However note that this semantics does not provide all the guarantees
that one may believe are automatically provided. Take the example of the
blocking list operations in Redis.

With AOF and fsync=always we could have:

    Client A doing: BLPOP myqueue 0
    Client B doing: RPUSH myqueue a b c

In this scenario, Client A will get the "a" elements immediately after
the Client B RPUSH will be executed, even before the operation is persisted.
However when Client B will get the acknowledge, it can be sure that
"b,c" are already safe on disk inside the list.

What to note here is that it cannot be assumed that Client A receiving
the element is a guaranteed that the operation succeeded from the point
of view of Client B.

This is due to the fact that the barrier exists within the same socket,
and not between different sockets. However in the case above, the
element "a" was not going to be persisted regardless, so it is a pretty
synthetic argument.
2018-02-27 13:06:42 +01:00
Salvatore Sanfilippo
d8830200b4
Merge pull request #3828 from oranagra/sdsnewlen_pr
add SDS_NOINIT option to sdsnewlen to avoid unnecessary memsets.
2018-02-27 04:04:32 -08:00
antirez
813960dbdd Fix ziplist prevlen encoding description. See #4705. 2018-02-23 12:19:35 +01:00
gechunlin
d4e6d1086f
Update object.c 2018-02-22 20:57:54 -06:00
antirez
ffde73c57d Track number of logically expired keys still in memory.
This commit adds two new fields in the INFO output, stats section:

expired_stale_perc:0.34
expired_time_cap_reached_count:58

The first field is an estimate of the number of keys that are yet in
memory but are already logically expired. They reason why those keys are
yet not reclaimed is because the active expire cycle can't spend more
time on the process of reclaiming the keys, and at the same time nobody
is accessing such keys. However as the active expire cycle runs, while
it will eventually have to return to the caller, because of time limit
or because there are less than 25% of keys logically expired in each
given database, it collects the stats in order to populate this INFO
field.

Note that expired_stale_perc is a running average, where the current
sample accounts for 5% and the history for 95%, so you'll see it
changing smoothly over time.

The other field, expired_time_cap_reached_count, counts the number
of times the expire cycle had to stop, even if still it was finding a
sizeable number of keys yet to expire, because of the time limit.
This allows people handling operations to understand if the Redis
server, during mass-expiration events, is able to collect keys fast
enough usually. It is normal for this field to increment during mass
expires, but normally it should very rarely increment. When instead it
constantly increments, it means that the current workloads is using
a very important percentage of CPU time to expire keys.

This feature was created thanks to the hints of Rashmi Ramesh and
Bart Robinson from Twitter. In private email exchanges, they noted how
it was important to improve the observability of this parameter in the
Redis server. Actually in big deployments, the amount of keys that are
yet to expire in each server, even if they are logically expired, may
account for a very big amount of wasted memory.
2018-02-19 11:12:49 +01:00
antirez
aa57481d8c Remove non semantical spaces from module.c. 2018-02-15 21:41:03 +01:00
Salvatore Sanfilippo
7830f8492f
Merge pull request #4479 from dvirsky/notify
Keyspace notifications API for modules
2018-02-15 21:36:32 +01:00
antirez
f4dc736cca Fix typo in notifyKeyspaceEvent() comment. 2018-02-15 21:33:06 +01:00
Dvir Volk
0a36196ce4 Add doc comment about notification flags 2018-02-14 21:54:00 +02:00
Dvir Volk
10efdf307b Add REDISMODULE_NOTIFY_STREAM flag to support stream notifications 2018-02-14 21:50:42 +02:00
Dvir Volk
613831f820 Fix indentation and comment style in testmodule 2018-02-14 21:43:06 +02:00
Dvir Volk
f27a64232e Use one static client for all keyspace notification callbacks 2018-02-14 21:40:10 +02:00
Dvir Volk
3aab12414f Remove the NOTIFY_MODULE flag and simplify the module notification flow if there aren't subscribers 2018-02-14 21:40:10 +02:00
Dvir Volk
a8e2e99a88 Document flags for notifications 2018-02-14 21:38:58 +02:00
Dvir Volk
d4d753dae4 removed some trailing whitespaces 2018-02-14 21:38:58 +02:00
Dvir Volk
5b7b12e38f removed hellonotify.c 2018-02-14 21:38:58 +02:00
Dvir Volk
896db12b41 fixed test 2018-02-14 21:38:58 +02:00
Dvir Volk
2136035e47 finished implementation of notifications. Tests unfinished 2018-02-14 21:38:58 +02:00
charsyam
9d41436115 getting rid of duplicated code 2018-02-14 00:12:13 +09:00
antirez
ae29bcd8e2 More verbose logging when slave sends errors to master.
See #3832.
2018-02-13 16:01:31 +01:00
Salvatore Sanfilippo
756df19134
Merge pull request #3832 from oranagra/slave_reply_to_master_pr
when a slave responds with an error on commands that come from master, log it
2018-02-13 15:55:26 +01:00
Salvatore Sanfilippo
f9e6c2046f
Merge pull request #3745 from guybe7/unstable
enlarged buffer given to ld2string
2018-02-13 15:50:21 +01:00
antirez
c14ba46e3a Make it explicit with a comment why we kill the old AOF rewrite.
See #3858.
2018-02-13 15:43:34 +01:00
Guy Benoish
f782006782 rewriteAppendOnlyFileBackground() failure fix
It is possible to do BGREWRITEAOF even if appendonly=no. This is by design.
stopAppendonly() didn't turn off aof_rewrite_scheduled (it can be turned on
again by BGREWRITEAOF even while appendonly is off anyway).
After configuring `appendonly yes` it will see that the state is AOF_OFF,
there's no RDB fork, so it will do rewriteAppendOnlyFileBackground() which
will fail since the aof_child_pid is set (was scheduled and started by cron).

Solution:
stopAppendonly() will turn off the schedule flag (regardless of who asked for it).
startAppendonly() will terminate any existing fork and start a new one (so it is the most recent).
2018-02-13 15:41:06 +01:00
Oran Agra
8e8d957ff8 fix to latency monitor reporting wrong max latency
in some cases LATENCY HISTORY reported latency that was
higher than the max latency reported by LATENCY LATEST / DOCTOR
2018-02-13 15:58:40 +02:00
赵磊
aacecbc997 Remove updateLFU() in dbOverwrite(). 2018-02-11 21:02:07 +08:00
antirez
32ac4c64ba Rax updated to latest antirez/rax commit. 2018-02-02 11:10:18 +01:00
zhaozhao.zz
968cb26693 config: handle special configuration "" for auth 2018-01-26 22:49:39 +08:00
Salvatore Sanfilippo
4aa2ecd98b
Merge pull request #4269 from jianqingdu/unstable
fix not call va_end() when syncWrite() failed
2018-01-24 10:55:25 +01:00
Mark Nunberg
062bd733da
redismodule.h: Check ModuleNameBusy before calling it
Older versions might not have this function.
2018-01-23 10:49:18 -05:00
antirez
727dd43614 Fix migrateCommand() access of not initialized byte. 2018-01-18 12:41:05 +01:00
Guy Benoish
fd8efb7c36 Replication buffer fills up on high rate traffic.
When feeding the master with a high rate traffic the the slave's feed is much slower.
This causes the replication buffer to grow (indefinitely) which leads to slave disconnection.
The problem is that writeToClient() decides to stop writing after NET_MAX_WRITES_PER_EVENT
writes (In order to be fair to clients).
We should ignore this when the client is a slave.
It's better if clients wait longer, the alternative is that the slave has no chance to stay in
sync in this situation.
2018-01-18 12:10:48 +01:00
antirez
1673a3f32c Cluster: improve anti-affinity algo in redis-trib.rb.
See #3462 and related PRs.

We use a simple algorithm to calculate the level of affinity violation,
and then an optimizer that performs random swaps until things improve.
2018-01-18 11:44:19 +01:00
antirez
e1e0bbe04d Remove useless comment from serverCron().
The behavior is well specified by the code itself.
2018-01-17 11:23:41 +01:00
Salvatore Sanfilippo
a18e4c964e
Merge pull request #4546 from hqin6/unstable
fixbug for #4545 dead loop aof rewrite
2018-01-17 11:21:55 +01:00
heqin
3d3faa0a19 fixbug for #4545 dead loop aof rewrite 2018-01-17 18:08:30 +08:00
Salvatore Sanfilippo
81401878de
Merge pull request #4609 from Qinch/unstable
fix assert problem in ZIP_DECODE_PREVLENSIZE macro
2018-01-17 10:45:11 +01:00
antirez
b23927b240 Hopefully more clear comment to explain the change in #4607. 2018-01-16 15:52:13 +01:00
qinchao
1e0e168570 fix assert problem in ZIP_DECODE_PREVLENSIZE
, see issue: https://github.com/antirez/redis/issues/4587
2018-01-16 22:43:06 +08:00
Oran Agra
689b64c3ad PSYNC2 fix - promoted slave should hold on to it's backlog
after a slave is promoted (assuming it has no slaves
and it booted over an hour ago), it will lose it's replication
backlog at the next replication cron, rather than waiting for slaves
to connect to it.
so on a simple master/slave faiover, if the new slave doesn't connect
immediately, it may be too later and PSYNC2 will fail.
2018-01-16 10:10:42 +02:00
zhaozhao.zz
1b8eec3e53 aof: format code and comment 2018-01-15 13:01:03 +01:00
antirez
c45366be0a Put more details in the comment introduced by #4601. 2018-01-15 12:50:08 +01:00
Salvatore Sanfilippo
1ed5ac7ce5
Merge pull request #4601 from soloestoy/fix-memoryleak-for-lazy-server-del
lazyfree: fix memory leak for lazyfree-lazy-server-del
2018-01-15 12:43:55 +01:00
zhaozhao.zz
0517ab8397 lazyfree: fix memory leak for lazyfree-lazy-server-del 2018-01-15 00:45:37 +08:00
Salvatore Sanfilippo
aeeb747796
Merge pull request #4575 from soloestoy/bugfix-benchmark
redis-benchmark: bugfix - handle zero liveclients in right way
2018-01-12 17:43:01 +01:00
Salvatore Sanfilippo
72187fa8a5
Merge pull request #4581 from dvirsky/module_unlink
Added RM_UnlinkKey - a low level analog to UNLINK command
2018-01-12 17:41:09 +01:00
antirez
a5b6bc2bd7 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2018-01-12 17:16:38 +01:00
Salvatore Sanfilippo
71914387ba
Merge pull request #4586 from gnuhpc/fix-crashlog-typo
Fix a typo(maybe instruction?) in crash log
2018-01-12 17:16:12 +01:00
antirez
2f8476df91 Fix getKeysUsingCommandTable() in the case of nagative arity.
This fixes a crash with Redis Cluster when OBJECT is mis-used, because
getKeysUsingCommandTable() will call serverPanic() detecting we are
accessing an invalid argument in the case "OBJECT foo" is called.

This bug was introduced when OBJECT HELP was introduced, because the key
argument is set fixed at index 2 in the command table, however now
OBJECT may be called with an insufficient number of arguments to extract
the key.

The "Right Thing" would be to have a specific function to extract keys
from the OBJECT command, however this is kinda of an overkill, so I
preferred to make getKeysUsingCommandTable() more robust and just return
no keys when it's not possible to honor the command table, because new
commands are often added and also there are a number with an HELP
subcommand violating the normal form, and crashing for this trivial
reason or having many command-specific key extraction functions is not
great.
2018-01-12 11:26:29 +01:00
antirez
13650446ac proto-max-querybuf-len -> client-query-buffer-limit.
We already had client buffer limits exported as configuration options.
Stick with the naming scheme already used.

See #4568.
2018-01-11 11:36:26 +01:00
antirez
8075572207 New config options about protocol prefixed with "proto".
Related to #4568.
2018-01-11 11:27:41 +01:00
Salvatore Sanfilippo
c2fa4b8a60
Merge pull request #4568 from oranagra/restore_size_limit
fix RESTORE command size limits
2018-01-11 11:20:02 +01:00
huijing.whj
f0e09bf803 fix int overflow problem in freeMemoryIfNeeded 2018-01-10 23:47:02 +08:00
Salvatore Sanfilippo
e954742fbd
Merge pull request #3356 from yusaku/fix-module-firstkey
Fix the firstkey, lastkey, and keystep of moduleCommand
2018-01-09 18:58:31 +01:00
antirez
3ce1c28d47 Rewrite MIGRATE AUTH option.
See PR #2507. This is a reimplementation of the fix that contained
different problems.
2018-01-09 18:48:26 +01:00
gnuhpc
36e71b88b5 Fix a typo(maybe instruction?) in crash log 2018-01-09 16:13:02 +08:00
Dvir Volk
e76dfc90a7 Added RM_UnlinkKey - a low level analog to UNLINK command 2018-01-07 16:41:43 +02:00
gnuhpc
5113a53df6 Fix memory usage list bug 2018-01-05 12:16:24 +08:00
zhaozhao.zz
c0f611f12a redis-benchmark: bugfix - handle zero liveclients in right way 2018-01-03 12:47:25 +08:00
Oran Agra
b509a14c3e Add config options for max-bulk-len and max-querybuf-len mainly to support RESTORE of large keys 2017-12-29 12:43:48 +02:00
Oran Agra
60a4f12f8b fix processing of large bulks (above 2GB)
- protocol parsing (processMultibulkBuffer) was limitted to 32big positions in the buffer
  readQueryFromClient potential overflow
- rioWriteBulkCount used int, although rioWriteBulkString gave it size_t
- several places in sds.c that used int for string length or index.
- bugfix in RM_SaveAuxField (return was 1 or -1 and not length)
- RM_SaveStringBuffer was limitted to 32bit length
2017-12-29 12:24:19 +02:00
antirez
0b561883b4 Hyperloglog: refresh hdr variable correctly.
This is a fix for the #3819 improvements. The o->ptr may change because
of hllSparseSet() calls, so 'hdr' must be correctly re-fetched.
2017-12-22 11:26:31 +01:00
antirez
27b81f3fbc Hyperloglog: Support for PFMERGE sparse encoding as target.
This is a fix for #3819.
2017-12-22 11:01:27 +01:00
antirez
cbe9d725a7 Hyperloglog: refactoring of sparse/dense add function.
The commit splits the add functions into a set() and add() set of
functions, so that it's possible to set registers in an independent way
just having the index and count.

Related to #3819, otherwise a fix is not possible.
2017-12-22 11:00:22 +01:00
heqin
365e6c45ea fixbug for #4545 dead loop aof rewrite 2017-12-18 17:59:03 +08:00
Salvatore Sanfilippo
238c9bd086
Merge pull request #4540 from hqin6/unstable
fixbug for #4538 Error opening /setting AOF rewrite IPC pipes: No suc…
2017-12-15 12:41:02 +01:00
heqin
ee724cac49 fixbug for #4538 Error opening /setting AOF rewrite IPC pipes: No such file or directory 2017-12-15 11:06:58 +08:00
Salvatore Sanfilippo
d745e8dfb3
Merge pull request #4536 from tporadowski/antirez_unstable-cli-history
Always enable command history in redis-cli run on a TTY
2017-12-14 22:14:45 +01:00
Salvatore Sanfilippo
398b2084af
Merge pull request #4528 from nashe/patch-oob-read-4527
Prevent off-by-one read in stringmatchlen()
2017-12-14 12:21:32 +01:00
antirez
12e65a424c safe_write -> aofWrite. Function commented.
Related to #4498.
2017-12-14 12:19:36 +01:00
Salvatore Sanfilippo
5182dcd645
Merge pull request #4498 from soloestoy/aof-safe-write
Aof safe write -- fix the short write
2017-12-14 12:15:46 +01:00
Tomasz Poradowski
77653ebe8a always enable command history in redis-cli
- when redis-cli is running in a TTY - always enable command history
buffering, regardless if history file path can be successfully
determined
2017-12-13 23:49:41 +01:00
antirez
de276b6a43 Cluster: allow read-only EVAL/EVALSHA in slaves.
Fix #3665.
2017-12-13 13:36:01 +01:00
nashe
f43eb5adcf Prevent off-by-one read in stringmatchlen() (fixes #4527) 2017-12-12 01:25:03 +01:00
zhaozhao.zz
109ee497be zset: change the span of zskiplistNode to unsigned long 2017-12-08 16:09:27 +08:00
zhaozhao.zz
e8901b2fe4 zset: fix the int problem 2017-12-08 15:37:08 +08:00
antirez
522760fac7 Change indentation and other minor details of PR #4489.
The main change introduced by this commit is pretending that help
arrays are more text than code, thus indenting them at level 0. This
improves readability, and is an old practice when defining arrays of
C strings describing text.

Additionally a few useless return statements are removed, and the HELP
subcommand capitalized when printed to the user.
2017-12-06 12:05:14 +01:00
Itamar Haber
482d678e95 C style 2017-12-05 19:09:19 +02:00
Itamar Haber
b23c8babed Uses an offset in addReplyHelp 2017-12-05 18:17:14 +02:00
Itamar Haber
8b51121998 Merge remote-tracking branch 'upstream/unstable' into help_subcommands 2017-12-05 18:14:59 +02:00
antirez
62a4b817c6 add linkClient(): adds the client and caches the list node.
We have this operation in two places: when caching the master and
when linking a new client after the client creation. By having an API
for this we avoid incurring in errors when modifying one of the two
places forgetting the other. The function is also a good place where to
document why we cache the linked list node.

Related to #4497 and #4210.
2017-12-05 16:02:03 +01:00
Salvatore Sanfilippo
03cfc8bf3a
Merge pull request #4497 from soloestoy/optimize-unlink-client
networking: optimize unlinkClient() in freeClient()
2017-12-05 15:51:15 +01:00
Salvatore Sanfilippo
c56fbb246c
Merge pull request #4508 from trevor211/fixNotes
fix some notes
2017-12-05 15:47:05 +01:00
antirez
479ea57797 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2017-12-05 15:38:21 +01:00
zhaozhao.zz
7c6ddbc37d dict: fix the int problem for defrag 2017-12-05 15:38:03 +01:00
zhaozhao.zz
d1176b582c dict: fix the int problem 2017-12-05 15:37:59 +01:00
Salvatore Sanfilippo
6830fa25e3
Merge pull request #4488 from itamarhaber/debug_arity
Standardizes arity handling of DEBUG
2017-12-05 15:29:42 +01:00
zhaozhao.zz
42387d6c1a set: fix the int problem for qsort 2017-12-05 17:42:19 +08:00
zhaozhao.zz
de809666f8 set: fix the int problem for SPOP & SRANDMEMBER 2017-12-05 17:19:19 +08:00
WuYunlong
3f232ebfb8 fix some notes 2017-12-05 14:41:16 +08:00
zhaozhao.zz
b9491b65d9 quicklist: change the len of quicklist to unsigned long 2017-12-04 19:49:40 +08:00
zhaozhao.zz
664bbfe760 quicklist: fix the return value of quicklistCount 2017-12-04 19:49:23 +08:00
antirez
2869284e44 Streams: fix a few type mismatches in t_stream.c. 2017-12-04 11:59:15 +01:00
antirez
60d26acfc8 Refactoring: improve luaCreateFunction() API.
The function in its initial form, and after the fixes for the PSYNC2
bugs, required code duplication in multiple spots. This commit modifies
it in order to always compute the script name independently, and to
return the SDS of the SHA of the body: this way it can be used in all
the places, including for SCRIPT LOAD, without duplicating the code to
create the Lua function name. Note that this requires to re-compute the
body SHA1 in the case of EVAL seeing a script for the first time, but
this should not change scripting performance in any way because new
scripts definition is a rare event happening the first time a script is
seen, and the SHA1 computation is anyway not a very slow process against
the typical Redis script and compared to the actua Lua byte compiling of
the body.

Note that the function used to assert() if a duplicated script was
loaded, however actually now two times over three, we want the function
to handle duplicated scripts just fine: this happens in SCRIPT LOAD and
in RDB AUX "lua" loading. Moreover the assert was not defending against
some obvious failure mode, so now the function always tests against
already defined functions at start.
2017-12-04 11:25:20 +01:00
antirez
c6eca690ee Remove useless variable check from luaCreateFunction().
The block is already inside if (allow_dup).
2017-12-04 10:55:54 +01:00
antirez
68681f2bcf Fix issue #4505, Lua RDB AUX field loading of existing scripts.
Unfortunately, as outlined by @soloestoy in #4505, "lua" AUX RDB field
loading in case of duplicated script was still broken. This commit fixes
this problem and also a memory leak introduced by the past commit.

Note that now we have a regression test able to duplicate the issue, so
this commit was actually tested against the regression. The original PR
also had a valid fix, but I prefer to hide the details of scripting.c
outside scripting.c, and later "SCRIPT LOAD" should also be able to use
the function luaCreateFunction() instead of redoing the work.
2017-12-04 10:33:04 +01:00
antirez
6a1bf07a46 DEBUG change-repl-id implemented.
With PSYNC2 to force a full SYNC in tests is hard. With this new DEBUG
subcommand we just need to call it and then CLIENT KILL TYPE master in
the slave.
2017-12-04 10:24:52 +01:00
Itamar Haber
51eb6cb395 Adds help to CONFIG 2017-12-03 19:34:31 +02:00
Itamar Haber
bd5af03dbd Adds help to CLUSTER command 2017-12-03 19:05:10 +02:00
Itamar Haber
ee3884e63c Improve slowlog help 2017-12-03 17:39:52 +02:00
Itamar Haber
d884ba4bc9 Helps CLIENT 2017-12-03 16:49:29 +02:00
antirez
65a9740fa8 Fix loading of RDB files lua AUX fields when the script is defined.
In the case of slaves loading the RDB from master, or in other similar
cases, the script is already defined, and the function registering the
script should not fail in the assert() call.
2017-12-01 16:01:10 +01:00
antirez
8ac76be5f2 Streams: DEBUG DIGEST support. 2017-12-01 15:04:05 +01:00
antirez
f42df6f43a Streams: add code to compute the stream memory usage.
It's a bit of black magic without actually tracking it inside rax.c,
however Redis usage of the radix tree for the stream data structure is
quite consistent, so a few magic constants apparently are producing
results that make sense.
2017-12-01 12:50:27 +01:00
antirez
115d076d65 Streams: fix lp-count field for non-same-fields entries. 2017-12-01 10:24:25 +01:00
antirez
9bb18e5438 Streams: XRANGE REV option -> XREVRANGE command. 2017-12-01 10:24:25 +01:00
antirez
9dc79c039a Streams: fix reverse iterator discarding of items out of range. 2017-12-01 10:24:25 +01:00
antirez
6919280cc5 Streams: fix reverse iteration next node jumping. 2017-12-01 10:24:25 +01:00
antirez
ee3490ec48 Streams: state machine for reverse iteration WIP 1. 2017-12-01 10:24:25 +01:00
antirez
3c5d773f82 Streams: augment stream entries to allow backward scanning. 2017-12-01 10:24:25 +01:00
antirez
0381931b4c Streams: Update listpack to fix 32bit strings encoding error.
Note that streams produced by XADD in previous broken versions having
elements with 4096 bytes or more will be permanently broken and must be
created again from scratch.

Fix #4428
Fix #4349
2017-12-01 10:24:24 +01:00
antirez
020fe26bd6 Streams: fix COUNT parsing, issue #4433. 2017-12-01 10:24:24 +01:00
antirez
abab0b7817 Streams: fix redis-cli to understand the stream type. 2017-12-01 10:24:24 +01:00
antirez
671b1f6a9d Streams: fix TYPE for stream type. 2017-12-01 10:24:24 +01:00
antirez
5082ec6419 Streams: move ID ms/seq separator from '.' to '-'
After checking with the community via Twitter (here:
https://twitter.com/antirez/status/915130876861788161) the verdict was to
use ":". However I later realized, after users lamented the fact that
it's hard to copy IDs just with double click, that this was the reason
why I moved to "." in the first instance. Fortunately "-", that was the
other option with most votes, also gets selected with double click on
most terminal applications on Linux and MacOS.

So my reasoning was:

1) We can't retain "." because it's actually confusing to newcomers, it
looks like a floating number, people may be tricked into thinking they
can order IDs numerically as floats.

2) Moving to a double-click-to-select format is much better. People will
work with such IDs for long time when coding / debugging. Why making now
a choice that will impact this for the next years?

The only other viable option was "-", and that's what I did. Thanks.
2017-12-01 10:24:24 +01:00
antirez
50595a5889 Streams: fix XADD + MAXLEN propagation due to var shadowing.
Clang should be more prone to return warnings by default when there is
same-var-name shadowing. GCC does this and can avoid bugs like that.
2017-12-01 10:24:24 +01:00
antirez
a4e6aae6b8 Streams: fix memory leak in streamTrimByLength(). 2017-12-01 10:24:24 +01:00
antirez
0248a6b125 Streams: fix streamTrimByLength() standalone items skipping. 2017-12-01 10:24:24 +01:00
antirez
0540803288 Streams: XADD MAXLEN implementation.
The core of this change is the implementation of stream trimming, and
the resulting MAXLEN option of XADD as a trivial result of having
trimming functionalities. MAXLEN already works but in order to be more
efficient listpack GC should be implemented, currently marked as a TODO
item inside the comments.
2017-12-01 10:24:24 +01:00
antirez
0c00fd7834 Streams: reduce listpack max size to 2k to speedup range queries.
Listpack max size is a tradeoff between space and time. A 2k max entry
puts the memory usage approximately at a similar order of magnitude (5
million entries went from 96 to 120 MB), but the range queries speed
doubled (because there are half entries to scan in the average case).

Lower values could be considered, or maybe this parameter should be
made tunable.
2017-12-01 10:24:24 +01:00
antirez
f24d3a7de0 Streams: delta encode IDs based on key. Add count + deleted fields.
We used to have the master ID stored at the start of the listpack,
however using the key directly makes more sense in order to create a
space efficient representation: anyway the key at the radix tree is very
unlikely to change because of how the stream is implemented. Moreover on
nodes merging, to rewrite the merged listpacks is anyway the most
sensible operation, and we can use the iterator and the append-to-stream
function in order to avoid re-implementing the code needed for merging.

This commit also adds two items at the start of the listpack: the
number of valid items inside the listpack, and the number of items
marked as deleted. This means that there is no need to scan a listpack
in order to understand if it's a good candidate for garbage collection,
if the ration between valid/deleted items triggers the GC.
2017-12-01 10:24:24 +01:00
antirez
cea421a021 Streams: specify better how the master enty works. 2017-12-01 10:24:24 +01:00
antirez
3f2d7e277e Streams: items compression implemented.
The approach used is to set a fixed header at the start of every
listpack blob (that contains many entries). The header contains a
"master" ID and fields, that are initially just obtained from the first
entry inserted in the listpack, so that the first enty is always well
compressed. Later every new entry is checked against these fields, and
if it matches, the SAMEFIELD flag is set in the entry so that we know to
just use the master entry flags. The IDs are always delta-encoded
against the first entry. This approach avoids cascading effects in which
entries are encoded depending on the previous entries, in order to avoid
complexity and rewritings of the data when data is removed in the middle
(which is a planned feature).
2017-12-01 10:24:24 +01:00
antirez
8f00cf85a7 Streams: fixed memory leaks when blocking again for same stream.
blockForKeys() was not freeing the allocation holding the ID when the
key was already found busy. Fortunately the unit test checked explicitly
for blocking multiple times for the same key (copying a regression in
the blocking lists tests), so the bug was detected by the Redis test leak
checker.
2017-12-01 10:24:24 +01:00
antirez
26d4f8e3ec Streams: AOF rewriting + minor iterator improvements. 2017-12-01 10:24:24 +01:00
antirez
01ea018c40 Streams: export iteration API. 2017-12-01 10:24:24 +01:00
antirez
9ed40f0fc3 Streams: implement streamReplyWithRange() in terms of the iterator. 2017-12-01 10:24:24 +01:00
antirez
a58733cacf Streams: stream iteration refactoring, WIP 2. 2017-12-01 10:24:24 +01:00
antirez
b1ec333633 Streams: stream iteration refactoring, WIP 1. 2017-12-01 10:24:24 +01:00
antirez
1a603e1a87 Streams: fix bug in XREAD last received ID processing. 2017-12-01 10:24:24 +01:00
antirez
94af55c5ea Streams: fix memory leak in freeStream(). 2017-12-01 10:24:24 +01:00
antirez
3a0b78bc52 Streams: rewrite XADD ID argument for AOF/slaves. 2017-12-01 10:24:24 +01:00
antirez
19b06935d5 Streams: fix XADD API and keyspace notifications.
XADD was suboptimal in the first incarnation of the command, not being
able to accept an ID (very useufl for replication), nor options for
having capped streams.

The keyspace notification for streams was not implemented.
2017-12-01 10:24:24 +01:00
antirez
db89f7474d Streams: When XREAD blocks without COUNT, set a default one.
A client may lose a lot of time between invocations of blocking XREAD,
for example because it is processing the messages or for any other
cause. When it returns back, it may provide a low enough message ID that
the server will block to send an unreasonable number of messages in a
single call. For this reason we set a COUNT when the client is blocked
with XREAD calls, even if no COUNT is given. This is arbitrarily set to
1000 because it's enough to avoid slowing down the reception of many
messages, but low enough to avoid to block.
2017-12-01 10:24:24 +01:00
antirez
c128190026 Streams: fix handleClientsBlockedOnKeys() access to invalid ID. 2017-12-01 10:24:24 +01:00
antirez
6468cb2e82 Streams: fix XREAD ready-key signaling.
With lists we need to signal only on key creation, but streams can
provide data to clients listening at every new item added.
To make this slightly more efficient we now track different classes of
blocked clients to avoid signaling keys when there is nobody listening.
A typical case is when the stream is used as a time series DB and
accessed only by range with XRANGE.
2017-12-01 10:24:24 +01:00
antirez
b5be5093fe Streams: fix XREAD timeout handling, zero is valid. 2017-12-01 10:24:24 +01:00
antirez
2cacdcd6f8 Streams: XREAD related code to serve blocked clients. 2017-12-01 10:24:24 +01:00
antirez
0adb43b68f Streams: XREAD ability to block fixed. 2017-12-01 10:24:24 +01:00
antirez
6a1c92d52d Streams: synchronous xread fixes and improvements. 2017-12-01 10:24:24 +01:00
antirez
a7d898334a Streams: XREAD get-key method fixed. 2017-12-01 10:24:24 +01:00
antirez
110041825c Streams: XREAD get-keys method. 2017-12-01 10:24:24 +01:00
antirez
fa61720d30 Streams: XREAD, first draft. Handling of blocked clients still missing. 2017-12-01 10:24:24 +01:00
antirez
e65b4825f0 Streams: XREAD arguments parsing. 2017-12-01 10:24:24 +01:00
antirez
4086dff477 Streams: augment client.bpop with XREAD specific fields. 2017-12-01 10:24:24 +01:00
antirez
f80dfbf464 Streams: more internal preparation for blocking XREAD. 2017-12-01 10:24:24 +01:00
antirez
4a377cecd8 Streams: initial work to use blocking lists logic for streams XREAD. 2017-12-01 10:24:24 +01:00
antirez
439120c620 Streams: implement stream object release. 2017-12-01 10:24:24 +01:00
antirez
ec9bbe96bf Streams: XLEN command. 2017-12-01 10:24:24 +01:00
antirez
98d184db12 Streams: Save stream->length in RDB. 2017-12-01 10:24:24 +01:00
antirez
cd18f06e9c Streams: change listpack allocator to zmalloc. 2017-12-01 10:24:24 +01:00
antirez
edd70c1993 Streams: RDB loading. RDB saving modified.
After a few attempts it looked quite saner to just add the last item ID
at the end of the serialized listpacks, instead of scanning the last
listpack loaded from head to tail just to fetch it. It's a disk space VS
CPU-and-simplicity tradeoff basically.
2017-12-01 10:24:24 +01:00
antirez
485014cc74 Streams: RDB saving. 2017-12-01 10:24:24 +01:00
antirez
100d43c1ac Streams: assign value of 6 to OBJ_STREAM + some refactoring. 2017-12-01 10:24:24 +01:00
antirez
79866a6361 Streams: 12 commits squashed into the initial Streams implementation. 2017-12-01 10:24:24 +01:00
antirez
045d65c3af PSYNC2: Fix off by one buffer size in luaCreateFunction(). 2017-11-30 18:38:29 +01:00
antirez
452ad2e928 PSYNC2: just store script bodies into RDB.
Related to #4483. As suggested by @soloestoy, we can retrieve the SHA1
from the body. Given that in the new implementation using AUX fields we
ended copying around a lot to create new objects and strings, extremize
such concept and trade CPU for space inside the RDB file.
2017-11-30 18:38:26 +01:00
antirez
28dfdca733 PSYNC2: luaCreateFunction() should handle NULL client parameter.
See #4483. This is needed because luaCreateFunction() is now called
from RDB loading code outside a client context.
2017-11-30 18:37:52 +01:00
antirez
f11a7585a8 PSYNC2: Save Lua scripts state into RDB file.
This is currently needed in order to fix #4483, but this can be
useful in other contexts, so maybe later we may want to remove the
conditionals and always save/load scripts.

Note that we are using the "lua" AUX field here, in order to guarantee
backward compatibility of the RDB file. The unknown AUX fields must be
discarded by past versions of Redis.
2017-11-30 18:37:52 +01:00
antirez
3b9be93fda Prevent corruption of server.executable after DEBUG RESTART.
Doing the following ended with a broken server.executable:

1. Start Redis with src/redis-server
2. Send CONFIG SET DIR /tmp/
3. Send DEBUG RESTART

At this point we called execve with an argv[0] that is no longer related
to the new path. So after the restart the absolute path of the
executable is recomputed in the wrong way. With this fix we pass the
absolute path already computed as argv[0].
2017-11-30 18:30:06 +01:00
antirez
d8f8701032 Be more verbose when DEBUG RESTART fails. 2017-11-30 18:08:21 +01:00
zhaozhao.zz
43be967690 networking: optimize unlinkClient() in freeClient() 2017-11-30 18:11:05 +08:00
zhaozhao.zz
1b5f56d042 aof: cast sdslen to ssize_t 2017-11-30 10:27:12 +08:00
zhaozhao.zz
2d73cf2367 aof: fix the short write 2017-11-30 10:22:12 +08:00
Itamar Haber
0752a834f9 Check arity in SLOWLOG before accessing arg 2017-11-30 00:30:30 +02:00
antirez
2785d6caa0 Merge branch 'lfu-fixes' into unstable 2017-11-29 17:16:13 +01:00
Itamar Haber
59d52f7fab Standardizes the 'help' subcommand
This adds a new `addReplyHelp` helper that's used by commands
when returning a help text. The following commands have been
touched: DEBUG, OBJECT, COMMAND, PUBSUB, SCRIPT and SLOWLOG.

WIP

Fix entry command table entry for OBJECT for HELP option.

After #4472 the command may have just 2 arguments.

Improve OBJECT HELP descriptions.

See #4472.

WIP 2

WIP 3
2017-11-28 21:15:45 +02:00
Salvatore Sanfilippo
565e139a56
Merge pull request #4200 from jeesyn/fix_typo
fix a typo
2017-11-28 18:44:11 +01:00
Salvatore Sanfilippo
923502a70b
Merge pull request #4166 from charpty/wip-redisclic-typo
redis-cli.c typo: helpe -> helper.
2017-11-28 18:41:51 +01:00
Salvatore Sanfilippo
26826329f5
Merge pull request #4167 from charpty/wip-redisclic-typo2
redis-cli.c typo: Requets -> Requests.
2017-11-28 18:41:28 +01:00
Salvatore Sanfilippo
3508b9c440
Merge pull request #4170 from TehWebby/patch-2
Fix typo
2017-11-28 18:40:43 +01:00
antirez
851e9fc48b t_hash.c: clarify calling two times the same function. 2017-11-28 18:39:00 +01:00
antirez
c44732ac58 adlist: fix listJoin() in the case the second list is empty.
See #4192, the original PR removed lines of code that are actually
needed, so thanks to @chunqiulfq for reporting the problem, but merging
solution from @jeesyn after checking, together with @artix75, that the
logic covers all the cases.
2017-11-28 18:25:14 +01:00
Salvatore Sanfilippo
a13106e001
Merge pull request #4374 from rouzier/patch-1
Fix file descriptor leak and error handling
2017-11-28 17:33:23 +01:00
Salvatore Sanfilippo
bf71b120f1
Merge pull request #4451 from devnexen/minor_build_fixes
Fix undefined behavior constant defined.
2017-11-28 17:23:48 +01:00
Itamar Haber
8c7f90e91e Standardizes arity handling of DEBUG 2017-11-28 18:18:45 +02:00
antirez
06ca9d6839 LFU: Fix LFUDecrAndReturn() to just decrement.
Splitting the popularity in half actually just needs decrementing the
counter because the counter is logarithmic.
2017-11-28 12:18:30 +01:00
zhaozhao.zz
9f131c9a89 LFU: add hotkeys option to redis-cli 2017-11-27 18:39:29 +01:00
zhaozhao.zz
583c314725 LFU: do some changes about LFU to find hotkeys
Firstly, use access time to replace the decreas time of LFU.
For function LFUDecrAndReturn,
it should only try to get decremented counter,
not update LFU fields, we will update it in an explicit way.
And we will times halve the counter according to the times of
elapsed time than server.lfu_decay_time.
Everytime a key is accessed, we should update the LFU
including update access time, and increment the counter after
call function LFUDecrAndReturn.
If a key is overwritten, the LFU should be also updated.
Then we can use `OBJECT freq` command to get a key's frequence,
and LFUDecrAndReturn should be called in `OBJECT freq` command
in case of the key has not been accessed for a long time,
because we update the access time only when the key is read or
overwritten.
2017-11-27 18:39:22 +01:00
zhaozhao.zz
53cea97204 LFU: change lfu* parameters to int 2017-11-27 18:38:55 +01:00
zhaozhao.zz
dfc42ec447 LFU: fix the missing of config get and rewrite 2017-11-27 18:38:33 +01:00
antirez
75fa7879e6 Improve OBJECT HELP descriptions.
See #4472.
2017-11-27 18:09:08 +01:00
antirez
b412c544fd Fix entry command table entry for OBJECT for HELP option.
After #4472 the command may have just 2 arguments.
2017-11-27 13:16:07 +01:00
Salvatore Sanfilippo
29252391c4
Merge pull request #4472 from itamarhaber/object_patch
A minor fix and `help` subcommand for `OBJECT`
2017-11-27 12:41:02 +01:00
Itamar Haber
1c08220022 Adds -u <uri> option to redis-cli. 2017-11-27 11:34:11 +01:00
Itamar Haber
02d38f6b51 Adds OBJECT help 2017-11-24 19:59:05 +02:00
Itamar Haber
b28fb3d753 Prevents OBJECT freq with noeviction
When maxmemory is set to noeviction, idletime is implicitly kept. This renders access frequency nonsensical.
2017-11-24 19:58:37 +02:00
Salvatore Sanfilippo
c508cb6793
Merge pull request #4452 from soloestoy/expire-latency
expire & latency: fix the missing latency records generated by expire
2017-11-24 18:21:35 +01:00
antirez
7229fa8d6d Modules: fix memory leak in RM_IsModuleNameBusy(). 2017-11-24 13:29:54 +01:00
antirez
4d063bb6ba PSYNC2: reorganize comments related to recent fixes.
Related to PR #4412 and issue #4407.
2017-11-24 11:08:29 +01:00
Salvatore Sanfilippo
9d86ae4597
Merge pull request #4412 from soloestoy/bugfix-psync2
PSYNC2: safe free backlog when reach the time limit and others
2017-11-24 10:56:18 +01:00
Salvatore Sanfilippo
f739c27229
Merge pull request #4344 from soloestoy/fix-module-name-conflict
Fix module name conflict
2017-11-24 09:37:06 +01:00
Oran Agra
adf2701cc9 fix string to double conversion, stopped parsing on \0 even if the string has more data.
getLongLongFromObject calls string2ll which has this line:
/* Return if not all bytes were used. */
so if you pass an sds with 3 characters "1\01" it will fail.

but getLongDoubleFromObject calls strtold, and considers it ok if eptr[0]==`\0`
i.e. if the end of the string found by strtold ends with null terminator

127.0.0.1:6379> set a 1
OK
127.0.0.1:6379> setrange a 2 2
(integer) 3
127.0.0.1:6379> get a
"1\x002"
127.0.0.1:6379> incrbyfloat a 2
"3"
127.0.0.1:6379> get a
"3"
2017-11-23 17:15:27 +02:00
antirez
de914ede93 Modules: fix for scripting replication of modules commands.
See issue #4466 / #4467.
2017-11-23 15:14:17 +01:00
Yossi Gottlieb
2c70d28295 Nested MULTI/EXEC may replicate in different cases.
For example:
1. A module command called within a MULTI section.
2. A Lua script with replicate_commands() called within a MULTI section.
3. A module command called from a Lua script in the above context.
2017-11-22 22:02:51 +02:00
zhaozhao.zz
ea2e51c630 PSYNC2: persist cached_master's dbid inside the RDB 2017-11-22 12:11:26 +08:00
zhaozhao.zz
93037f7642 PSYNC2: make repl_stream_db never be -1
it means that after this change all the replication
info in RDB is valid, and it can distinguish us from
the older version.
2017-11-22 12:05:34 +08:00
zhaozhao.zz
7a808fd8a7 expire & latency: fix the missing latency records generated by expire 2017-11-21 23:35:30 +08:00
zhaozhao.zz
57bd8feb8d rehash: handle one db until finished 2017-11-21 09:49:42 +01:00
David Carlier
62689ef0cf Fix undefined behavior constant defined. 2017-11-19 16:23:42 +00:00
Salvatore Sanfilippo
cf9a3f7048
Merge pull request #2741 from kmiku7/unstable
fix boundary case for _dictNextPower
2017-11-08 17:06:09 +01:00
Itamar Haber
2564963dc8
Fixes an off-by-one in argument handling of MEMORY USAGE
Fixes #4430
2017-11-08 16:08:29 +02:00
antirez
a1944c3e4d Fix saving of zero-length lists.
Normally in modern Redis you can't create zero-len lists, however it's
possible to load them from old RDB files generated, for instance, using
Redis 2.8 (see issue #4409). The "Right Thing" would be not loading such
lists at all, but this requires to hook in rdb.c random places in a not
great way, for a problem that is at this point, at best, minor.

Here in this commit instead I just fix the fact that zero length lists,
materialized as quicklists with the first node set to NULL, were
iterated in the wrong way while they are saved, leading to a crash.

The other parts of the list implementation are apparently able to deal
with empty lists correctly, even if they are no longer a thing.
2017-11-06 12:37:03 +01:00
antirez
34d5804d4c SDS: improve sdsRemoveFreeSpace() to avoid useless data copy.
Since SDS v2, we no longer have a single header, so the function to
rewrite the SDS in terms of the minimum space required, instead of just
using realloc() and let the underlying allocator decide what to do,
was doing an allocation + copy every time the minimum possible header
needed to represent the string was different than the current one.
This could be often a bit wasteful, because if we go, for instance, from
the 32 bit fields header to the 16 bit fields header, the overhead of
the header is normally very small. With this commit we call realloc
instead, unless the change in header size is very significant in relation
to the string length.
2017-11-03 10:19:27 +01:00
zhaozhao.zz
b8579c225c PSYNC2: clarify the scenario when repl_stream_db can be -1 2017-11-02 10:45:33 +08:00
zhaozhao.zz
885c4f856e PSYNC2 & RDB: fix the missing rdbSaveInfo for BGSAVE 2017-11-01 17:52:43 +08:00
zhaozhao.zz
6ddf0ea293 PSYNC2: safe free backlog when reach the time limit
When we free the backlog, we should use a new
replication ID and clear the ID2. Since without
backlog we can not increment master_repl_offset
even do write commands, that may lead to inconsistency
when we try to connect a "slave-before" master
(if this master is our slave before, our replid
equals the master's replid2). As the master have our
history, so we can match the master's replid2 and
second_replid_offset, that make partial sync work,
but the data is inconsistent.
2017-11-01 17:32:27 +08:00
antirez
ffcf7d5ab1 Fix buffer overflows occurring reading redis.conf.
There was not enough sanity checking in the code loading the slots of
Redis Cluster from the nodes.conf file, this resulted into the
attacker's ability to write data at random addresses in the process
memory, by manipulating the index of the array. The bug seems
exploitable using the following techique: the config file may be altered so
that one of the nodes gets, as node ID (which is the first field inside the
structure) some data that is actually executable: then by writing this
address in selected places, this node ID part can be executed after a
jump. So it is mostly just a matter of effort in order to exploit the
bug. In practice however the issue is not very critical because the
bug requires an unprivileged user to be able to modify the Redis cluster
nodes configuration, and at the same time this should result in some
gain. However Redis normally is unprivileged as well. Yet much better to
have this fixed indeed.

Fix #4278.
2017-10-31 09:41:22 +01:00
antirez
de474186bd More robust object -> double conversion.
Certain checks were useless, at the same time certain malformed inputs
were accepted without problems (emtpy strings parsed as zero).
Cases where strtod() returns ERANGE but we still want to parse the input
where ok in getDoubleFromObject() but not in the long variant.

As a side effect of these fixes, this commit fixes #4391.
2017-10-30 13:39:58 +01:00
rouzier
6eb996540c Fix file descriptor leak and error handling 2017-10-13 13:20:45 -04:00
antirez
2bf8c2c130 Limit statement in RM_BlockClient() to 80 cols. 2017-09-28 23:15:34 +02:00
zhaozhao.zz
6dffc1b7a3 Modules: handle the busy module name 2017-09-28 17:38:40 +08:00
zhaozhao.zz
cb9dde3280 Modules: handle the conflict of registering commands 2017-09-28 16:21:21 +08:00
Dvir Volk
7393fd814e Added safety net preventing redis from crashing if a module decide to block in MULTI 2017-09-27 15:17:53 +03:00
Dvir Volk
b246635d6d Renamed GetCtxFlags to GetContextFlags 2017-09-27 11:58:16 +03:00
Dvir Volk
616c546b01 Added support for module context flags with RM_GetCtxFlags 2017-09-27 11:58:07 +03:00
antirez
474adba9fa Clarify comment in change fixing #4323. 2017-09-21 12:35:04 +02:00
zhaozhao.zz
269760edbb Lazyfree: avoid memory leak when free slowlog entry 2017-09-21 14:19:21 +08:00
antirez
bb3b5ddd19 PSYNC2: More refinements related to #4316. 2017-09-20 11:28:13 +02:00
zhaozhao.zz
b541ccef25 PSYNC2: make persisiting replication info more solid
This commit is a reinforcement of commit c1c99e9.

1. Replication information can be stored when the RDB file is
generated by a mater using server.slaveseldb when server.repl_backlog
is not NULL, or set repl_stream_db be -1. That's safe, because
NULL server.repl_backlog will trigger full synchronization,
then master will send SELECT command to replicaiton stream.
2. Only do rdbSave* when rsiptr is not NULL,
if we do rdbSave* without rdbSaveInfo, slave will miss repl-stream-db.
3. Save the replication informations also in the case of
SAVE command, FLUSHALL command and DEBUG reload.
2017-09-20 11:18:10 +02:00
antirez
c1c99e9f4e PSYNC2: Fix the way replication info is saved/loaded from RDB.
This commit attempts to fix a number of bugs reported in #4316.
They are related to the way replication info like replication ID,
offsets, and currently selected DB in the master client, are stored
and loaded by Redis. In order to avoid inconsistencies the changes in
this commit try to enforce that:

1. Replication information are only stored when the RDB file is
generated by a slave that has a valid 'master' client, so that we can
always extract the currently selected DB.
2. When replication informations are persisted in the RDB file, all the
info for a successful PSYNC or nothing is persisted.
3. The RDB replication informations are only loaded if the instance is
configured as a slave, otherwise a master can start with IDs that relate
to a different history of the data set, and stil retain such IDs in the
future while receiving unrelated writes.
2017-09-19 23:03:39 +02:00
antirez
a4152119c6 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2017-09-19 10:35:49 +02:00
antirez
b75ae0bbea PSYNC2: Create backlog on slave partial sync as well.
A slave may be started with an RDB file able to provide enough slave to
perform a successful partial SYNC with its master. However in such a
case, how outlined in issue #4268, the slave backlog will not be
started, since it was only initialized on full syncs attempts. This
creates different problems with successive PSYNC attempts that will
always result in full synchronizations.

Thanks to @fdingiit for discovering the issue.
2017-09-19 10:33:14 +02:00
Salvatore Sanfilippo
2d13bf4c59 Merge pull request #3785 from GitHubMota/unstable
redis-benchmark: default value size usage update.
2017-09-18 12:18:57 +02:00
Salvatore Sanfilippo
9b4cb4addc Merge pull request #3554 from jybaek/Delete_duplicate
Remove Duplicate Processing
2017-09-18 12:18:15 +02:00
Oran Agra
b122cadc66 Flush append only buffers before existing.
when SHUTDOWN command is recived it is possible that some of the recent
command were not yet flushed from the AOF buffer, and the server
experiences data loss at shutdown.
2017-09-17 07:22:16 +03:00
jianqingdu
498f65ffb7 fix not call va_end when syncWrite() failed
fix not call va_end when syncWrite() failed in sendSynchronousCommand()
2017-08-30 21:20:14 -05:00
Richard Li
2ef8c2f6c2 Update the comment 2017-08-18 11:27:45 +08:00
jeesyn.liu
447b373fc9 fix a typo 2017-08-08 17:45:51 +08:00
jybaek
a8c08b9b76 Add missing fclose() 2017-08-03 17:28:04 +09:00
Salvatore Sanfilippo
34a79c353f Merge pull request #3935 from itamarhaber/module-cmdstats
Changes command stats iteration to being dict-based
2017-08-02 12:51:26 +02:00
antirez
bc64df9a66 Add MEMORY DOCTOR to MEMORY HELP. 2017-07-28 17:47:54 +02:00
Shaun Webb
2e6f285009 Fix typo 2017-07-27 09:37:37 +09:00
Bo Cai
00954f4d48 redis-cli.c typo: Requets -> Requests.
Signed-off-by: Bo Cai <charpty@gmail.com>
2017-07-26 21:33:29 +08:00
Bo Cai
005d9fa861 redis-cli.c typo: helpe -> helper.
Signed-off-by: Bo Cai <charpty@gmail.com>
2017-07-26 21:24:28 +08:00
Mota
81fe7a4733 redis-benchmark: default value size usage update.
default size of SET/GET value in usage should be 3 bytes as in main code.
2017-07-25 23:43:46 +08:00
Salvatore Sanfilippo
6b64cc47a0 Merge pull request #2259 from badboy/fix-2258
Check that the whole first argument is a number
2017-07-24 15:19:53 +02:00
Salvatore Sanfilippo
964224b77f Merge pull request #4124 from lamby/proceding-proceeding-typo
Correct proceding -> proceeding typo.
2017-07-24 15:19:21 +02:00
Salvatore Sanfilippo
ae40e5f362 Merge pull request #4125 from trevor211/fixAutoAofRewirteMinSize
fix rewrite config: auto-aof-rewrite-min-size
2017-07-24 15:18:56 +02:00
Salvatore Sanfilippo
25c231c4c1 Merge pull request #1998 from grobe0ba/unstable
Fix missing '-' in redis-benchmark help output (Issue #1996)
2017-07-24 15:18:08 +02:00
Salvatore Sanfilippo
d9565379da Merge pull request #4128 from leonchen83/unstable
fix mismatch argument and return wrong value of clusterDelNodeSlots
2017-07-24 14:18:28 +02:00
liangsijian
ffbbe5a720 Fix lua ldb command log 2017-07-24 19:24:06 +08:00
antirez
314043552b Modules: don't crash when Lua calls a module blocking command.
Lua scripting does not support calling blocking commands, however all
the native Redis commands are flagged as "s" (no scripting flag), so
this is not possible at all. With modules there is no such mechanism in
order to flag a command as non callable by the Lua scripting engine,
moreover we cannot trust the modules users from complying all the times:
it is likely that modules will be released to have blocking commands
without such commands being flagged correctly, even if we provide a way to
signal this fact.

This commit attempts to address the problem in a short term way, by
detecting that a module is trying to block in the context of the Lua
scripting engine client, and preventing to do this. The module will
actually believe to block as usually, but what happens is that the Lua
script receives an error immediately, and the background call is ignored
by the Redis engine (if not for the cleanup callbacks, once it
unblocks).

Long term, the more likely solution, is to introduce a new call called
RedisModule_GetClientFlags(), so that a command can detect if the caller
is a Lua script, and return an error, or avoid blocking at all.

Being the blocking API experimental right now, more work is needed in
this regard in order to reach a level well blocking module commands and
all the other Redis subsystems interact peacefully.

Now the effect is like the following:

    127.0.0.1:6379> eval "redis.call('hello.block',1,5000)" 0
    (error) ERR Error running script (call to
    f_b5ba35ff97bc1ef23debc4d6e9fd802da187ed53): @user_script:1: ERR
    Blocking module command called from Lua script

This commit fixes issue #4127 in the short term.
2017-07-23 12:55:37 +02:00
antirez
5bfdfbe174 Fix typo in unblockClientFromModule() top comment. 2017-07-23 12:41:26 +02:00
antirez
a3778f3b0f Make representClusterNodeFlags() more robust.
This function failed when an internal-only flag was set as an only flag
in a node: the string was trimmed expecting a final comma before
exiting the function, causing a crash. See issue #4142.
Moreover generation of flags representation only needed at DEBUG log
level was always performed: a waste of CPU time. This is fixed as well
by this commit.
2017-07-20 15:17:35 +02:00
antirez
b1c2e1a19c Fix two bugs in moduleTypeLookupModuleByID().
The function cache was not working at all, and the function returned
wrong values if there where two or more modules exporting native data
types.

See issue #4131 for more details.
2017-07-20 14:59:42 +02:00
Leon Chen
9e7a8c0207 fix return wrong value of clusterDelNodeSlots 2017-07-20 17:24:38 +08:00
Leon Chen
2cdf4cc656 fix mismatch argument 2017-07-18 02:28:24 -05:00
WuYunlong
c32c690de6 fix rewrite config: auto-aof-rewrite-min-size 2017-07-15 10:20:56 +08:00
Chris Lamb
7560d347da Correct proceding -> proceeding typo. 2017-07-14 22:53:14 +01:00
antirez
bd1782fa0a Modules: fix thread safe context DB selection.
Before this fix the DB currenty selected by the client blocked was not
respected and operations were always performed on DB 0.
2017-07-14 13:02:15 +02:00
antirez
8eefc9323d Allow certain modules APIs only defining REDISMODULE_EXPERIMENTAL_API.
Those calls may be subject to changes in the future, so the user should
acknowledge it is using non stable API.
2017-07-14 12:07:52 +02:00
antirez
f03947a676 Modules documentation removed from source.
Moving to redis-doc repository to publish via Redis.io.
2017-07-14 11:33:59 +02:00
antirez
43aaf96163 Markdown generation of Redis Modules API reference improved. 2017-07-14 11:29:31 +02:00
antirez
e74f0aa6d1 Fix replication of SLAVEOF inside transaction.
In Redis 4.0 replication, with the introduction of PSYNC2, masters and
slaves replicate commands to cascading slaves and to the replication
backlog itself in a different way compared to the past.

Masters actually replicate the effects of client commands.
Slaves just propagate what they receive from masters.

This mechanism can cause problems when the configuration of an instance
is changed from master to slave inside a transaction. For instance
we could send to a master instance the following sequence:

    MULTI
    SLAVEOF 127.0.0.1 0
    EXEC
    SLAVEOF NO ONE

Before the fixes in this commit, the MULTI command used to be propagated
into the replication backlog, however after the SLAVEOF command the
instance is a slave, so the EXEC implementation failed to also propagate
the EXEC command. When the slaves of the above instance reconnected,
they were incrementally synchronized just sending a "MULTI". This put
the master client (in the slaves) into MULTI state, breaking the
replication.

Notably even Redis Sentinel uses the above approach in order to guarantee
that configuration changes are always performed together with rewrites
of the configuration and with clients disconnection. Sentiel does:

    MULTI
    SLAVEOF ...
    CONFIG REWRITE
    CLIENT KILL TYPE normal
    EXEC

So this was a really problematic issue. However even with the fix in
this commit, that will add the final EXEC to the replication stream in
case the instance was switched from master to slave during the
transaction, the result would be to increment the slave replication
offset, so a successive reconnection with the new master, will not
permit a successful partial resynchronization: no way the new master can
provide us with the backlog needed, we incremented our offset to a value
that the new master cannot have.

However the EXEC implementation waits to emit the MULTI, so that if the
commands inside the transaction actually do not need to be replicated,
no commands propagation happens at all. From multi.c:

    if (!must_propagate && !(c->cmd->flags & (CMD_READONLY|CMD_ADMIN))) {
	execCommandPropagateMulti(c);
	must_propagate = 1;
    }

The above code is already modified by this commit you are reading.
Now also ADMIN commands do not trigger the emission of MULTI. It is actually
not clear why we do not just check for CMD_WRITE... Probably I wrote it this
way in order to make the code more reliable: better to over-emit MULTI
than not emitting it in time.

So this commit should indeed fix issue #3836 (verified), however it looks
like some reconsideration of this code path is needed in the long term.

BONUS POINT: The reverse bug.

Even in a read only slave "B", in a replication setup like:

	A -> B -> C

There are commands without the READONLY nor the ADMIN flag, that are also
not flagged as WRITE commands. An example is just the PING command.

So if we send B the following sequence:

    MULTI
    PING
    SLAVEOF NO ONE
    EXEC

The result will be the reverse bug, where only EXEC is emitted, but not the
previous MULTI. However this apparently does not create problems in practice
but it is yet another acknowledge of the fact some work is needed here
in order to make this code path less surprising.

Note that there are many different approaches we could follow. For instance
MULTI/EXEC blocks containing administrative commands may be allowed ONLY
if all the commands are administrative ones, otherwise they could be
denined. When allowed, the commands could simply never be replicated at all.
2017-07-12 11:07:28 +02:00
antirez
e1b8b4b6da CLUSTER GETKEYSINSLOT: avoid overallocating.
Close #3911.
2017-07-11 15:49:09 +02:00
antirez
5bd46d33db Fix isHLLObjectOrReply() to handle integer encoded strings.
Close #3766.
2017-07-11 12:44:59 +02:00
antirez
e203a46cf3 Clients blocked in modules: free argv/argc later.
See issue #3844 for more information.
2017-07-11 12:33:01 +02:00
antirez
14c32c3569 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2017-07-11 09:46:58 +02:00
antirez
54e4bbeabd Event loop: call after sleep() only from top level.
In general we do not want before/after sleep() callbacks to be called
when we re-enter the event loop, since those calls are only designed in
order to perform operations every main iteration of the event loop, and
re-entering is often just a way to incrementally serve clietns with
error messages or other auxiliary operations. However, if we call the
callbacks, we are then forced to think at before/after sleep callbacks
as re-entrant, which is much harder without any good need.

However here there was also a clear bug: beforeSleep() was actually
never called when re-entering the event loop. But the new afterSleep()
callback was. This is broken and in this instance re-entering
afterSleep() caused a modules GIL dead lock.
2017-07-11 00:13:52 +02:00
Salvatore Sanfilippo
58104d8327 Merge pull request #4113 from guybe7/module_io_bytes
Modules: Fix io->bytes calculation in RDB save
2017-07-10 19:14:34 +02:00
antirez
11182a1a58 redis-check-aof: tell users there is a --fix option. 2017-07-10 16:41:25 +02:00
Guy Benoish
dfb68cd235 Modules: Fix io->bytes calculation in RDB save 2017-07-10 14:41:57 +03:00
antirez
fc7ecd8d35 AOF check utility: ability to check files with RDB preamble. 2017-07-10 13:38:23 +02:00
Salvatore Sanfilippo
6b0670daad Merge pull request #3853 from itamarhaber/issue-3851
Sets up fake client to select current db in RM_Call()
2017-07-06 15:02:11 +02:00
Salvatore Sanfilippo
38dd30af42 Merge pull request #4105 from spinlock/unstable-networking
Optimize addReplyBulkSds for better performance
2017-07-06 14:31:08 +02:00
Salvatore Sanfilippo
2d5aa00959 Merge pull request #4106 from petersunbag/unstable
minor fix in listJoin().
2017-07-06 14:29:37 +02:00
sunweinan
87f771bff1 minor fix in listJoin(). 2017-07-06 19:47:21 +08:00
antirez
2b36950e9b Free IO context if any in RDB loading code.
Thanks to @oranagra for spotting this bug.
2017-07-06 11:20:49 +02:00
antirez
51ffd062d3 Modules: DEBUG DIGEST interface. 2017-07-06 11:04:46 +02:00
spinlock
10db81af71 update Makefile for test-sds 2017-07-05 14:32:09 +00:00
spinlock
ea31a4eae3 Optimize addReplyBulkSds for better performance 2017-07-05 14:25:05 +00:00
antirez
f9fac7f777 Avoid closing invalid FDs to make Valgrind happier. 2017-07-05 15:40:25 +02:00
antirez
413c2bc180 Modules: no MULTI/EXEC for commands replicated from async contexts.
They are technically like commands executed from external clients one
after the other, and do not constitute a single atomic entity.
2017-07-05 10:10:20 +02:00
Salvatore Sanfilippo
09dd7b5ff0 Merge pull request #4101 from dvirsky/fix_modules_reply_len
Proposed fix to #4100
2017-07-04 12:01:51 +02:00
antirez
eddd8d34c4 Add symmetrical assertion to track c->reply_buffer infinite growth.
Redis clients need to have an instantaneous idea of the amount of memory
they are consuming (if the number is not exact should at least be
proportional to the actual memory usage). We do that adding and
subtracting the SDS length when pushing / popping from the client->reply
list. However it is quite simple to add bugs in such a setup, by not
taking the objects in the list and the count in sync. For such reason,
Redis has an assertion to track counts near 2^64: those are always the
result of the counter wrapping around because we subtract more than we
add. This commit adds the symmetrical assertion: when the list is empty
since we sent everything, the reply_bytes count should be zero. Thanks
to the new assertion it should be simple to also detect the other
problem, where the count slowly increases because of over-counting.
The assertion adds a conditional in the code that sends the buffer to
the socket but should not create any measurable performance slowdown,
listLength() just accesses a structure field, and this code path is
totally dominated by write(2).

Related to #4100.
2017-07-04 11:55:05 +02:00
Dvir Volk
86e564e9ff fixed #4100 2017-07-04 00:02:19 +03:00
antirez
b2cd9fcab6 Fix GEORADIUS edge case with huge radius.
This commit closes issue #3698, at least for now, since the root cause
was not fixed: the bounding box function, for huge radiuses, does not
return a correct bounding box, there are points still within the radius
that are left outside.

So when using GEORADIUS queries with radiuses in the order of 5000 km or
more, it was possible to see, at the edge of the area, certain points
not correctly reported.

Because the bounding box for now was used just as an optimization, and
such huge radiuses are not common, for now the optimization is just
switched off when the radius is near such magnitude.

Three test cases found by the Continuous Integration test were added, so
that we can easily trigger the bug again, both for regression testing
and in order to properly fix it as some point in the future.
2017-07-03 19:38:31 +02:00
Jun He
c8ca71d40b Fixed stack trace generation on aarch64
Change-Id: I9801239c98cb7362ed07e8b9ec2ba7e45749dba7
Signed-off-by: Jun He <jun.he@arm.com>
2017-07-03 08:47:55 +00:00
antirez
26e638a8e9 redis-cli --latency: ability to run non interactively.
This feature was proposed by @rosmo in PR #2643 and later redesigned
in order to fit better with the other options for non-interactive modes
of redis-cli. The idea is basically to allow to collect latency
information in scripts, cron jobs or whateever, just running for a
limited time and then producing a single output.
2017-06-30 15:41:58 +02:00
antirez
7bad78bd2f Fix abort typo in Lua debugger help screen. 2017-06-30 12:12:00 +02:00
antirez
f8547e53f0 Added GEORADIUS(BYMEMBER)_RO variants for read-only operations.
Issue #4084 shows how for a design error, GEORADIUS is a write command
because of the STORE option. Because of this it does not work
on readonly slaves, gets redirected to masters in Redis Cluster even
when the connection is in READONLY mode and so forth.

To break backward compatibility at this stage, with Redis 4.0 to be in
advanced RC state, is problematic for the user base. The API can be
fixed into the unstable branch soon if we'll decide to do so in order to
be more consistent, and reease Redis 5.0 with this incompatibility in
the future. This is still unclear.

However, the ability to scale GEO queries in slaves easily is too
important so this commit adds two read-only variants to the GEORADIUS
and GEORADIUSBYMEMBER command: GEORADIUS_RO and GEORADIUSBYMEMBER_RO.
The commands are exactly as the original commands, but they do not
accept the STORE and STOREDIST options.
2017-06-30 10:03:37 +02:00
antirez
01a4b9892d HMSET and MSET implementations unified. HSET now variadic.
This is the first step towards getting rid of HMSET which is a command
that does not make much sense once HSET is variadic, and has a saner
return value.
2017-06-29 17:38:46 +02:00
Salvatore Sanfilippo
634c64dd18 Merge pull request #4075 from sgn1/brpop_keys
Fix Issues in blocking commands in cluster mode.
2017-06-27 17:51:19 +02:00
antirez
365dd037dc RDB modules values serialization format version 2.
The original RDB serialization format was not parsable without the
module loaded, becuase the structure was managed only by the module
itself. Moreover RDB is a streaming protocol in the sense that it is
both produce di an append-only fashion, and is also sometimes directly
sent to the socket (in the case of diskless replication).

The fact that modules values cannot be parsed without the relevant
module loaded is a problem in many ways: RDB checking tools must have
loaded modules even for doing things not involving the value at all,
like splitting an RDB into N RDBs by key or alike, or just checking the
RDB for sanity.

In theory module values could be just a blob of data with a prefixed
length in order for us to be able to skip it. However prefixing the values
with a length would mean one of the following:

1. To be able to write some data at a previous offset. This breaks
stremaing.
2. To bufferize values before outputting them. This breaks performances.
3. To have some chunked RDB output format. This breaks simplicity.

Moreover, the above solution, still makes module values a totally opaque
matter, with the fowllowing problems:

1. The RDB check tool can just skip the value without being able to at
least check the general structure. For datasets composed mostly of
modules values this means to just check the outer level of the RDB not
actually doing any checko on most of the data itself.
2. It is not possible to do any recovering or processing of data for which a
module no longer exists in the future, or is unknown.

So this commit implements a different solution. The modules RDB
serialization API is composed if well defined calls to store integers,
floats, doubles or strings. After this commit, the parts generated by
the module API have a one-byte prefix for each of the above emitted
parts, and there is a final EOF byte as well. So even if we don't know
exactly how to interpret a module value, we can always parse it at an
high level, check the overall structure, understand the types used to
store the information, and easily skip the whole value.

The change is backward compatible: older RDB files can be still loaded
since the new encoding has a new RDB type: MODULE_2 (of value 7).
The commit also implements the ability to check RDB files for sanity
taking advantage of the new feature.
2017-06-27 13:19:16 +02:00
antirez
c3998728a2 ARM: Fix stack trace generation on crash. 2017-06-26 10:36:16 +02:00
antirez
c9097393bf Issue #4027: unify comment and modify return value in freeMemoryIfNeeded().
It looks safer to return C_OK from freeMemoryIfNeeded() when clients are
paused because returning C_ERR may prevent success of writes. It is
possible that there is no difference in practice since clients cannot
execute writes while clients are paused, but it looks more correct this
way, at least conceptually.

Related to PR #4028.
2017-06-23 11:42:25 +02:00
Salvatore Sanfilippo
936ade80b2 Merge pull request #4028 from zintrepid/prevent_expirations_while_paused
Prevent expirations and evictions while paused
2017-06-23 11:39:02 +02:00
Suraj Narkhede
f85f36f50d Fix following issues in blocking commands:
1. brpop last key index, thus checking all keys for slots.
2. Memory leak in clusterRedirectBlockedClientIfNeeded.
3. Remove while loop in clusterRedirectBlockedClientIfNeeded.
2017-06-23 00:30:21 -07:00
Suraj Narkhede
d303bca587 Fix brpop command table entry and redirect blocked clients. 2017-06-22 23:52:00 -07:00
antirez
8b768e8ea4 Aesthetic changes to #4068 PR to conform to Redis coding standard.
1. Inline if ... statement if short.
2. No lines over 80 columns.
2017-06-22 11:00:34 +02:00
Salvatore Sanfilippo
6476f1a979 Merge pull request #4068 from FreedomU007/unstable
Fix set with ex/px option when propagated to aof
2017-06-22 10:46:58 +02:00
xuzhou
86e9f48a0c Optimize set command with ex/px when updating aof. 2017-06-22 11:06:40 +08:00
Salvatore Sanfilippo
ef446bf16d Merge pull request #3802 from flowly/bugfix-calc-stat-net-output-bytes
Bugfix calc stat net output bytes
2017-06-20 17:01:16 +02:00
Salvatore Sanfilippo
1d857a99d5 Merge pull request #4056 from season89/unstable
Fixed comments of slowlog duration
2017-06-20 16:55:29 +02:00
Salvatore Sanfilippo
0a03187ac4 Merge pull request #3659 from cbgbt/cli-elapsed
cli: Only print elapsed time on OUTPUT_STANDARD.
2017-06-20 16:53:56 +02:00
antirez
2a84927f35 redis-benchmark: add -t hset target. 2017-06-19 09:41:11 +02:00
xuzhou
530fcf8687 Fix set with ex/px option when propagated to aof 2017-06-16 17:51:38 +08:00
antirez
53cb27b1d7 SLOWLOG: log offending client address and name. 2017-06-15 12:57:54 +02:00
antirez
ab9d398835 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2017-06-14 18:29:53 +02:00
Qu Chen
4740424049 Implement getKeys procedure for georadius and georadiusbymember
commands.
2017-06-14 18:15:48 +02:00
xuchengxuan
3fc4bf07cc Fixed comments of slowlog duration 2017-06-14 16:42:21 +08:00
Salvatore Sanfilippo
d3b32ca48d Merge pull request #4034 from amallia/patch-1
Fixed comment in clusterMsg version field
2017-06-13 06:28:23 -07:00
Salvatore Sanfilippo
33035cad04 Merge pull request #4035 from amallia/patch-2
Removed duplicate 'sys/socket.h'  include
2017-06-13 06:27:31 -07:00
antirez
5877c02c51 Fix PERSIST expired key resuscitation issue #4048. 2017-06-13 10:35:51 +02:00
Antonio Mallia
2d1d57eb47 Removed duplicate 'sys/socket.h' include 2017-06-04 15:26:53 +01:00
Antonio Mallia
591dba8055 Fixed comment in clusterMsg version field 2017-06-04 15:09:05 +01:00
Zachary Marquez
a3e53cf9bc Prevent expirations and evictions while paused
Proposed fix to https://github.com/antirez/redis/issues/4027
2017-06-01 16:28:40 -05:00
antirez
e91b81c612 More informative -MISCONF error message. 2017-05-19 12:03:30 +02:00
antirez
e498d9ee3e Collect fork() timing info only if fork succeeded. 2017-05-19 11:10:36 +02:00
antirez
78211aaaaf redis-cli --bigkeys: show error when TYPE fails.
Close #3993.
2017-05-15 11:22:28 +02:00
antirez
1f598fc2bb Modules TSC: use atomic var for server.unixtime.
This avoids Helgrind complaining, but we are actually not using
atomicGet() to get the unixtime value for now: too many places where it
is used and given tha time_t is word-sized it should be safe in all the
archs we support as it is.

On the other hand, Helgrind, when Redis is compiled with "make helgrind"
in order to force the __sync macros, will detect the write in
updateCachedTime() as a read (because atomic functions are used) and
will not complain about races.

This commit also includes minor refactoring of mutex initializations and
a "helgrind" target in the Makefile.
2017-05-10 10:04:16 +02:00
antirez
de786186a5 atomicvar.h: show used API in INFO. Add macro to force __sync builtin.
The __sync builtin can be correctly detected by Helgrind so to force it
is useful for testing. The API in the INFO output can be useful for
debugging after problems are reported.
2017-05-10 09:33:49 +02:00
Guy Benoish
89a9e5a9a2 Merge branch 'unstable' of https://github.com/antirez/redis into unstable 2017-05-09 18:42:32 +03:00
antirez
6eb51bf1ec zmalloc.c: remove thread safe mode, it's the default way. 2017-05-09 16:59:51 +02:00
antirez
9390c384b8 Modules TSC: Add mutex for server.lruclock.
Only useful for when no atomic builtins are available.
2017-05-09 16:32:49 +02:00
antirez
ece658713b Modules TSC: Improve inter-thread synchronization.
More work to do with server.unixtime and similar. Need to write Helgrind
suppression file in order to suppress the valse positives.
2017-05-09 11:57:09 +02:00
antirez
2a51bac44e Simplify atomicvar.h usage by having the mutex name implicit. 2017-05-04 17:01:00 +02:00
antirez
52bc74f221 Lazyfree: fix lazyfreeGetPendingObjectsCount() race reading counter. 2017-05-04 10:35:40 +02:00
antirez
7d9326b1f3 Modules TSC: HELLO.KEYS reply format fixed. 2017-05-03 23:43:49 +02:00
antirez
9b01b64430 Modules TSC: put the client in the pending write list. 2017-05-03 14:54:48 +02:00
antirez
e67fb915eb adlist: fix final list count in listJoin(). 2017-05-03 14:54:14 +02:00
antirez
79226cb9fa adlist: fix listJoin() to handle empty lists. 2017-05-03 14:15:25 +02:00
antirez
6798736909 Modules: remove unused var in example module. 2017-05-03 14:10:21 +02:00
antirez
1ed2ff5570 Modules TSC: HELLO.KEYS example draft finished. 2017-05-03 14:08:12 +02:00
antirez
7127f15ebe Module: fix RedisModule_Call() "l" specifier to create a raw string. 2017-05-03 14:07:10 +02:00
antirez
3fcf959e60 Modules TSC: Release the GIL for all the time we are blocked.
Instead of giving the module background operations just a small time to
run in the beforeSleep() function, we can have the lock released for all
the time we are blocked in the multiplexing syscall.
2017-05-03 11:26:21 +02:00