caleb-brown/redict
1
0
Fork 0
mirror of https://codeberg.org/redict/redict.git synced 2025-01-24 00:59:02 -05:00
Code Issues Actions Packages Projects Releases Wiki Activity
3,942 Commits 4 Branches 5 Tags 115 MiB
a89c8bb87c
Commit Graph

337 Commits

Author SHA1 Message Date
antirez
d728ec6dee Cluster: send a ping to every node we never contacted in timeout/2 seconds. 
Usually we try to send just 1 ping every second, however when we detect
we are going to have unreliable failure detection because we can't ping
some node in time, send an additional ping.

This should only happen with very large clusters or when the the node
timeout is set to a very low value.
 2013-03-05 12:16:02 +01:00
antirez
e7628be2a7 Cluster: set node->slaveof correctly when a node state is updated. 2013-03-05 11:50:11 +01:00
antirez
d6457577d4 Cluster: don't perform startup slots sanity check for slaves. 
If we are a cluster node the DB content will not match our configured
slots. Don't do the check at all.
 2013-03-04 19:47:00 +01:00
antirez
d334897e80 Cluster: fix maximum line length when loading config. 
There are pathological cases where the line can be even longer a single
node may contain all the slots in importing/migrating state.
 2013-03-04 19:45:36 +01:00
antirez
b8a28bf442 Cluster: actually setup replication in CLUSTER REPLICATE. 2013-03-04 15:27:58 +01:00
antirez
0c01088b51 Cluster: REPLICATE subcommand and stub for clusterSetMaster(). 2013-03-04 13:15:09 +01:00
charsyam
bc84c399f8 adding check error code 
adding check error code
 2013-03-04 11:20:11 +01:00
antirez
caf9b24a7d Cluster: don't set the slot as unassigned because of PONG info. 
As stated in the comment this is usually due to a resharding in progress
so the client should be still redirected to the old node that will
handle the redirection elsewhere.
 2013-02-28 15:54:29 +01:00
antirez
0d77440b26 Cluster: better handling of slots changes in PONG packets. 
The new code makes sure that the node slots bitmap is always consistent
with the cluster->slots array.
 2013-02-28 15:41:54 +01:00
antirez
5f8fd27ace Cluster: refactoring of clusterNode*Bit to use helper bitmap functions. 2013-02-28 15:23:09 +01:00
antirez
d21d6b666f Cluster: use node->numslots instead of popcount() where possible. 2013-02-28 15:13:32 +01:00
antirez
4521115b17 Cluster: new field in cluster node structure, "numslots". 
Before a relatively slow popcount() operation was needed every time we
needed to get the number of slots served by a given cluster node.
Now we just need to check an integer that is taken in sync with the
bitmap.
 2013-02-28 15:11:05 +01:00
antirez
a2566d6618 Cluster: don't gossip about nodes that are not useful to the cluster. 2013-02-28 15:00:09 +01:00
antirez
d45d184118 Cluster: CLUSTER FORGET implemented. 2013-02-27 17:55:59 +01:00
antirez
d2b8281b3f Cluster: added a missing return on CLUSTER SETSLOT. 2013-02-27 17:53:48 +01:00
antirez
d20dea3eb7 Cluster: blank node address when flagging it as NOADDR. 2013-02-27 17:09:33 +01:00
antirez
2dcb5ab72b Cluster: add comments in sub-sections of CLUSTER command. 2013-02-27 16:12:59 +01:00
antirez
f9b5ca29fd Use GCC printf format attribute for redisLog(). 
This commit also fixes redisLog() statements producing warnings.
 2013-02-27 12:27:15 +01:00
antirez
d0992d6e8b Cluster: a few random fixes to the new failure detection. 2013-02-26 15:15:44 +01:00
antirez
f288b07563 Cluster: log the event when we clear the FAIL flag. 2013-02-26 15:03:38 +01:00
antirez
97ffcd351b Cluster: use the failure report API to reimplement failure detection. 
The new system detects a failure only when there is quorum from masters.
 2013-02-26 14:58:39 +01:00
antirez
1b1b3f6c06 Cluster: invert two functions declarations in more natural order. 2013-02-26 11:19:48 +01:00
antirez
d5e8b0a47f Cluster: cleanup idle failure reports every time we remove one. 
This is not very important as anyway when the function counting the
number of reports is called the cleanup is performed. However with this
change if only part of the nodes that reported the failure will report
the node is back ok, we'll cleanup the older entries ASAP. In complex
split net split scenarios, and when we are dealing with clusters having
nodes in the order of ~ 1000, this can save some CPU.
 2013-02-26 11:15:18 +01:00
antirez
9cb578ced0 Cluster: new function clusterNodeDelFailureReport() for failure reports. 
This is the missing part of the API that will be used to reimplement
failure detection of Cluster nodes.
 2013-02-25 19:13:22 +01:00
antirez
18f537083a Cluster: no limits for the count parameter of CLUSTER GETKEYSINSLOT. 
Not sure why I set a limit to 1 million keys, there is no reason for
this artificial limit, and anyway this is s a stupid limit because it is
already high enough to create latency issues. So let's the users shoot
on their feet because maybe they just actually know what they are doing.
 2013-02-25 12:41:13 +01:00
antirez
544bbe5387 Cluster: validate slot number in CLUSTER COUNTKEYSINSLOT. 2013-02-25 12:40:32 +01:00
antirez
d4fa40655d Cluster: new sub-command CLUSTER COUNTKEYSINSLOT. 
The new sub-command uses the new countKeysInSlot() API and allows a
cluster client to get the number of keys for a given hashslot.
 2013-02-25 12:04:31 +01:00
antirez
a517c89321 Cluster: verifyClusterConfigWithData() implemented. 2013-02-25 11:43:49 +01:00
antirez
d2154254be Cluster: fix case for getKeysInSlot() and countKeysInSlot(). 
Redis functions start in low case. A few functions about cluster were
capitalized the wrong way.
 2013-02-25 11:25:40 +01:00
antirez
c2eb4a606f Cluster: use CountKeysInSlot() when we just need the count. 2013-02-25 11:23:04 +01:00
antirez
ad3bca1fdf Cluster: added stub for verifyClusterConfigWithData(). 
See the top-comment for the function in this commit for details about
what the function is supposed to do.
 2013-02-25 11:20:17 +01:00
antirez
825e07f2fd Cluster: if no previous config exists, create the myself node as master. 2013-02-22 19:24:01 +01:00
antirez
f4093753e4 Cluster: add cluster_size field in CLUSTER INFO output. 2013-02-22 19:20:38 +01:00
antirez
d218a4e244 Cluster: new state information, cluster size. 
The definition of cluster size is: the number of known nodes in the
cluster that are masters and serving at least an hash slot.
 2013-02-22 19:18:30 +01:00
antirez
5c55ed9388 Cluster: remove warning adding clusterNodeSetSlotBit() prototype. 2013-02-22 17:45:49 +01:00
antirez
974929770b Cluster: introduced a failure reports system. 
A §Redis Cluster node used to mark a node as failing when itself
detected a failure for that node, and a single acknowledge was received
about the possible failure state.

The new API will be used in order to possible to require that N other
nodes have a PFAIL or FAIL state for a given node for a node to set it
as failing.
 2013-02-22 17:43:35 +01:00
antirez
07b6322735 Cluster: more correct update of slots state when PONG is received. 2013-02-21 16:52:06 +01:00
antirez
c6da9d9fac Call clusterUpdateState() after CLUSTER SETSLOT too. 2013-02-21 16:31:22 +01:00
antirez
3a99d1228a Aesthetic change to make a line more 80-cols friendly. 2013-02-21 16:24:48 +01:00
antirez
dc4af60628 Cluster: clusterAddSlot() was not doing what stated in the comment. 2013-02-21 11:51:17 +01:00
antirez
fdb57233e2 Cluster: always use cluster(Add|Del)Slot to modify the cluster slots table. 2013-02-21 11:44:58 +01:00
antirez
786b8d6c87 Use RESTORE-ASKING for MIGRATE when in cluster mode. 2013-02-20 17:36:54 +01:00
antirez
ea7fc82a4a Cluster: new command flag forcing implicit ASKING. 
Also using this new flag the RESTORE-ASKING command was implemented that
will be used by MIGRATE.
 2013-02-20 17:28:35 +01:00
antirez
9a04e12cc0 Cluster: I/O errors are now logged at DEBUG level. 2013-02-20 13:18:51 +01:00
antirez
02796ba7a7 Cluster: sanity checks on the cluster bus message length. 2013-02-15 16:44:39 +01:00
antirez
6b9c661838 Cluster: make valgrind happy initializing all the bytes of the node IP. 2013-02-15 12:58:35 +01:00
antirez
7371d5e248 Remove wrong decrRefCount() from getNodeByQuery(). 
This fixes issue #607.
 2013-02-15 11:57:53 +01:00
antirez
20f52b5b78 Top comment for getNodeByQuery() improved. 2013-02-15 11:50:54 +01:00
antirez
e0e15bd06d Cluster: with 16384 slots we need bigger buffers. 2013-02-14 15:36:33 +01:00
antirez
1649e509c3 Cluster: the cluster state structure is now heap allocated. 2013-02-14 13:20:56 +01:00
antirez
9dfd11c3da Cluster: Initialize ip and port in createClusterNode(). 2013-02-14 13:01:28 +01:00
antirez
ebd666db47 Cluster: from 4096 to 16384 hash slots. 2013-02-14 12:49:16 +01:00
antirez
b70b459b0e TCP_NODELAY after SYNC: changes to the implementation. 2013-02-05 12:04:30 +01:00
guiquanz
9d09ce3981 Fixed many typos. 2013-01-19 10:59:44 +01:00
antirez
2feef47aa1 MIGRATE: retry one time on I/O error. 
Now that we cache connections, a retry attempt makes sure that the
operation don't fail just because there is an existing connection error
on the socket, like the other end closing the connection.

Unfortunately this condition is not detectable using
getsockopt(SO_ERROR), so the only option left is to retry.

We don't retry on timeouts.
 2012-11-14 11:30:24 +01:00
antirez
05705bc8bb MIGRATE: fix default timeout to 1000 milliseconds. 
When a timeout <= 0 is provided we set a default timeout of 1 second.
It was set to 1 millisecond for an error resulting from a recent change.
 2012-11-12 18:54:35 +01:00
antirez
149b527a74 MIGRATE timeout should be in milliseconds. 
While it is documented that the MIGRATE timeout is in milliseconds, it
was in seconds instead. This commit fixes the problem.
 2012-11-12 14:01:02 +01:00
antirez
e23d281e48 MIGRATE TCP connections caching. 
By caching TCP connections used by MIGRATE to chat with other Redis
instances a 5x performance improvement was measured with
redis-benchmark against small keys.

This can dramatically speedup cluster resharding and other processes
where an high load of MIGRATE commands are used.
 2012-11-12 00:47:24 +01:00
antirez
4365e5b2d3 BSD license added to every C source and header file. 2012-11-08 18:31:32 +01:00
antirez
1237d71c4e COPY and REPLACE options for MIGRATE. 
With COPY now MIGRATE does not remove the key from the source instance.
With REPLACE it uses RESTORE REPLACE on the target host so that even if
the key already eixsts in the target instance it will be overwritten.

The options can be used together.
 2012-11-07 15:32:27 +01:00
antirez
e5b5763f56 REPLACE option for RESTORE. 
The REPLACE option deletes an existing key with the same name (if any)
and materializes the new one. The default behavior without RESTORE is to
return an error if a key already exists.
 2012-11-07 10:57:23 +01:00
antirez
6fdc635447 Better Out of Memory handling. 
The previous implementation of zmalloc.c was not able to handle out of
memory in an application-specific way. It just logged an error on
standard error, and aborted.

The result was that in the case of an actual out of memory in Redis
where malloc returned NULL (In Linux this actually happens under
specific overcommit policy settings and/or with no or little swap
configured) the error was not properly logged in the Redis log.

This commit fixes this problem, fixing issue #509.
Now the out of memory is properly reported in the Redis log and a stack
trace is generated.

The approach used is to provide a configurable out of memory handler
to zmalloc (otherwise the default one logging the event on the
standard output is used).
 2012-08-24 12:55:37 +02:00
antirez
21661d7acc Fixed a bug in propagation of PUBLISH via the cluster bus. 
This bug was spotted by clang on FreeBSD.
 2012-04-24 11:28:10 +02:00
antirez
e54fe9a79f A few compiler warnings suppressed. 2012-04-24 11:11:55 +02:00
antirez
a3fb7fd4f6 Minor MIGRATE implementation simplification about ttl handling. 2012-04-10 16:46:29 +02:00
antirez
46738646d4 dump/restore fixed to use the new crc64 API. 2012-04-09 12:33:57 +02:00
antirez
12e91892a0 Another fix for MIGRATE. 2012-04-03 15:10:42 +02:00
antirez
84e5684bca Two fixed for MIGRATE: fix TTL propagation and fix transferring of data bigger than 64k. 2012-04-03 12:17:40 +02:00
antirez
31f2ecf436 MIGRATE now let the client distinguish I/O errors and timeouts from other erros. 2012-04-02 16:38:24 +02:00
antirez
f8ea19e539 DUMP/RESTORE now use CRC64 instead of truncated SHA1. 2012-04-02 13:10:39 +02:00
antirez
70d848e1fa RESTORE ability to set a TTL fixed, bug introduced with millisecond expires. 2012-04-02 11:14:47 +02:00
antirez
a149ce6875 Prettify source code of create/verify DUMP payload. 2012-04-02 10:52:39 +02:00
antirez
bd04465931 DUMP / RESTORE: store RDB version in little endian. 2012-04-02 10:46:24 +02:00
antirez
4de6c9a055 New DUMP format includes RDB version and truncated SHA1 checksum. 2012-04-01 12:51:40 +02:00
antirez
44f508f1a8 clusterGetRandomName() generalized into getRandomHexChars() so that we can use it for the run_id field as well. 2012-03-08 10:08:44 +01:00
antirez
d329031fad Fixed another possible bug in cluster.c found by clang --analyze. 2012-01-25 16:59:36 +01:00
antirez
6710ff24d0 Fixed a non critical bug signaled by clang static analyzer thanks to Mukund Sivaraman for reporting it: there was a not initialized field populating the cluster message header, but it is always fixed at later time before sending the packet. 2012-01-25 16:46:35 +01:00
antirez
c0ba9ebe13 dict.c API names modified to be more coincise and consistent. 2011-11-08 17:07:55 +01:00
antirez
f013f40003 Fixed a few warnings compiling on Linux. 2011-10-23 10:57:01 +02:00
antirez
73fac227a0 use signalModifiedKey to set the key as dirty in the context of WATCH for both MIGRATE and RESTORE. 2011-10-20 11:17:30 +02:00
antirez
6856c7b4d6 First implementation of the ASKING command. Semantics still to verify. 2011-10-17 17:35:23 +02:00
antirez
e0aab1fc79 MIGRATE +NOKEY error was missing the final CRLF causing a protocol error. 2011-10-17 16:44:08 +02:00
antirez
bfbc16ae83 Fixed a typo causing segfault on MIGRATE 2011-10-17 16:39:05 +02:00
antirez
2b9ce0192e small comment added 2011-10-13 14:51:29 +02:00
antirez
d38ef52085 Redis Cluster: process node to node CLUSTERMSG_TYPE_PUBLISH messages and send it to the local clients. 2011-10-07 16:34:16 +02:00
antirez
c563ce463b propagate PUBLISH messages using the redis cluster nodes bus. Still need to process the incoming packets of that type. Work in progress. 2011-10-07 15:37:34 +02:00
antirez
9465d83efd A node can populate a slot if a message from a trusted slot is received claiming ownership of this slot, that is currently empty or served by a node in FAIL state. However this feature was broken since calling clusterAddSlot() was not enough as the slot bit is already set in the node that pong us. We need to directly alter the table. This commit fixes the issue. 2011-10-05 17:40:02 +02:00
antirez
ad7a86fbe0 just minor aesthetic changes to cluster.c 2011-10-05 16:02:45 +02:00
antirez
eab0e26e03 replaced redisAssert() with redisAssertWithInfo() in a shitload of places. 2011-10-04 18:43:03 +02:00
antirez
37d650032a Rewrite MIGRATE as DEL for AOF/replication. Also increment the dirty counter to both force replication and persistence. 2011-10-03 15:51:10 +02:00
antirez
2a95c944ff increment server.dirty in restore command to make sure it is replicated and increments the dirty count for persistence concerns. 2011-10-03 15:45:14 +02:00
antirez
66c146592a when processing gossip packets make sure to also update the node bitmap when associating slots to nodes. Fixed simply using the appropriate helper function to add a slot. 2011-10-01 15:11:07 +02:00
antirez
d38d2fdfdc Fixed two bugs in the CLUSTER SETSLOT ... NODE subcommand 2011-09-30 22:17:24 +02:00
antirez
c5954c1966 Clear the importing status from the slot if a CLUSTER SETSLOT command permanently assigns the slot to the importing node. 2011-09-30 19:20:56 +02:00
antirez
3b5289a04c added comment about SETSLOT NODE form. 2011-09-29 15:26:01 +02:00
antirez
1ef8b0a96d show node with myself flag always as connected in CLUSTER NODES output. 2011-09-28 18:28:36 +02:00
antirez
f96a8a8054 rioInitWithFile nad rioInitWithBuffer functions now take a rio structure pointer to avoid copying a structure to return value to the caller. 2011-09-22 16:00:40 +02:00
antirez
f9c6f39b2b merge conflicts resolved 2011-09-22 15:15:26 +02:00
antirez
f85cd526c1 DB API refactoring. The changes were designed together with Pieter Noordhuis. 2011-06-20 16:42:16 +02:00
Pieter Noordhuis
f1d8e4968e Make RDB types/opcodes explicit; load/save object type 2011-05-13 22:14:39 +02:00
Pieter Noordhuis
2e4b0e7727 Abstract file/buffer I/O to support in-memory serialization 2011-05-13 17:31:00 +02:00
antirez
0caa750748 clear importing/exporing state when appropriate on SETSLOT or ADDSLOTS cluster commands 2011-05-06 16:08:10 +02:00
antirez
f9cbdcb1a6 CLUSTER SETSLOT command and some refactoring of the cluster command 2011-05-06 15:44:09 +02:00
antirez
f384df8302 CLUSTER DELSLOTS 2011-05-06 13:38:27 +02:00
antirez
a7b058dae6 Fixed semantics of CLUSTER SETSLOT, SELECT now only denied in cluster mode if selected DB is not 0 so that MIGRATE still works well. 2011-05-05 18:10:02 +02:00
antirez
46834808fe CLUSTER SETSLOT STABLE fixed 2011-05-05 17:56:12 +02:00
antirez
4763ecc9ad missing return caused protocol desync in CLUSTER SETSLOT 2011-05-05 17:52:19 +02:00
antirez
0276e5545c Fixed problem in cluster redirection due to a stupid typo 2011-05-05 17:50:14 +02:00
antirez
eda827f8b7 cluster import/export of hash slots implemented in the query redirection engine 2011-05-05 11:13:21 +02:00
antirez
a5dce40726 if /dev/urandom is not available use rand() to get a random node name 2011-05-04 10:30:22 +02:00
antirez
0ba2932298 CLUSTER SETSLOT implemented 2011-05-04 09:31:37 +02:00
antirez
66f2517feb render migrating and importing slots in cluster nodes info 2011-05-02 19:04:33 +02:00
antirez
2f52dac9ef CLUSTER subcommands to set slots in migrating or importing state. Still a work in progress... 2011-04-29 17:34:03 +02:00
antirez
484354ff95 CLUSTER GETKEYSINSLOT implemented 2011-04-29 16:17:58 +02:00
antirez
1eb713a4c1 CLUSTER KEYSLOT command 2011-04-29 14:31:18 +02:00
antirez
c772d9c6e7 take a hashslot -> keys index, will be used for cluster rehasing 2011-04-28 19:00:33 +02:00
antirez
9d89e99c99 no longer useful debugging printf removed 2011-04-13 11:42:05 +02:00
antirez
d415f9ff41 Merge branch 'unstable' of github.com:antirez/redis into unstable 2011-04-12 15:36:10 +02:00
Salvatore Sanfilippo
8c4c50906c added known nodes info in CLUSTER INFO 2011-04-11 17:40:35 +02:00
antirez
8d727af8c4 update state when FAIL is cleared from a node 2011-04-08 14:19:52 +02:00
antirez
fd7a584f7a do not process node failure messages about yourself 2011-04-08 10:17:41 +02:00
antirez
a55c7868c1 fixed lame error in slot assignment 2011-04-07 23:33:18 +02:00
antirez
5a547b27f7 evaluate cluster state after config load. Still bugs in the slot allocation after nodes config load to fix. 2011-04-07 23:23:27 +02:00
antirez
93666e583c master node without slaves rejoin fixed 2011-04-07 23:10:32 +02:00
antirez
152d937b8c when loading config set the ping/pong timestaps to the current value. Even a node in FAIL state can be accepted again if there are no slaves for this node so election was not possible 2011-04-07 23:06:01 +02:00
antirez
1793752d97 node cluster configuration file configurable via redis.conf 2011-04-07 21:34:41 +02:00
antirez
2bc52b2c02 useless newline removed from log message 2011-04-07 19:22:24 +02:00
antirez
d01a6bb3f9 fixes to configuration loading and saving. However there is to still fix the logic for reconnection/behavior of nodes after a restart. 2011-04-07 19:04:16 +02:00
antirez
92690d29fe first version of cluster config loading code 2011-04-07 17:46:28 +02:00
antirez
726a39c15f initial cluster config load code 2011-04-07 12:55:02 +02:00
antirez
f797c7dc17 bad data on RESTORE can no longer crash the server but create a memory leak with some input string 2011-04-05 13:57:28 +02:00
antirez
626f6b2d64 DUMP implemented, RESTORE and MIGRATE fixed. Use zcalloc() instead of zmalloc()+memset() in sds.c when a new string is created with NULL initialization pointer. 2011-04-01 18:59:28 +02:00
antirez
6c390c0b23 minor change with no actual effects til the loading node config code is in place 2011-03-30 18:12:51 +02:00
antirez
ef21ab960e centralized cluster config file name. Assigned slots in CLUSTER NODES output and in cluster config file. 2011-03-30 17:41:13 +02:00
antirez
4b72c5617f fix to configuration saving on first start 2011-03-30 16:51:28 +02:00
antirez
c7c7cfbddc cluster configuration saving 2011-03-30 14:58:19 +02:00
antirez
ecc9109434 Cluster branch merged to unstable. 2011-03-29 17:51:15 +02:00