diff --git a/TODO b/TODO
index 8d4b711ed..59c0bb142 100644
--- a/TODO
+++ b/TODO
@@ -5,8 +5,7 @@ VERSION 1.2 TODO (Zsets, Integer encoding, Append only journal)
Most of the features already implemented for this release. The following is a list of the missing things in order to release the first beta tar.gz:
-* When Redis runs as slave make sure to set the fake client it uses to talk to the master as c->authenticated = 1 to avoid problems with slaves with requirepassword set.
-* Document BGREWRITEAOF.
+* Continue adding tests accordingly to gcov output.
VERSION 1.4 TODO (Hash type)
============================
diff --git a/doc/BgrewriteaofCommand.html b/doc/BgrewriteaofCommand.html
new file mode 100644
index 000000000..61e51c309
--- /dev/null
+++ b/doc/BgrewriteaofCommand.html
@@ -0,0 +1,41 @@
+
+
+
+
+
+
+
+
BGREWRITEAOF rewrites the Append Only File in background when it gets toobig. The Redis Append Only File is a Journal, so every operation modifyingthe dataset is logged in the Append Only File (and replayed at startup).This means that the Append Only File always grows. In order to rebuildits content the BGREWRITEAOF creates a new version of the append only filestarting directly form the dataset in memory in order to guarantee thegeneration of the minimal number of commands needed to rebuild the database.
ZADDkeyscorememberAdd the specified member to the Set value at key or update the score if it already exist
ZREMkeymemberRemove the specified member from the Set value at key
ZRANGEkeystartendReturn a range of elements from the sorted set at key
ZREVRANGEkeystartendReturn a range of elements from the sorted set at key, exactly like ZRANGE, but the sorted set is ordered in traversed in reverse order, from the greatest to the smallest score
ZRANGEBYSCOREkeyminmaxReturn all the elements with score >= min and score <= max (a range query) from the sorted set
ZCARDkeyReturn the cardinality (number of elements) of the sorted set at key
ZSCOREkeyelementReturn the score associated with the specified element of the sorted set at key
ZREMRANGEBYSCOREkeyminmaxRemove all the elements with score >= min and score <= max from the sorted set
Redis Lists are lists of Redis Strings, sorted by insertion order. It's possible to add elements to a Redis List pushing new elements on the head (on the left) or on the tail (on the right) of the list.
The LPUSH command inserts a new elmenet on head, while RPUSH inserts a new element on tail. A new list is created when one of this operations is performed against an empty key.
For instance if perform the following operations:
+ #sidebar ListCommandsSidebar
Redis Lists are lists of Redis Strings, sorted by insertion order. It's possible to add elements to a Redis List pushing new elements on the head (on the left) or on the tail (on the right) of the list.
The LPUSH command inserts a new elmenet on head, while RPUSH inserts a new element on tail. A new list is created when one of this operations is performed against an empty key.
For instance if perform the following operations:
LPUSH mylist a # now the list is "a"
LPUSH mylist b # now the list is "b","a"
RPUSH mylist c # now the list is "b","a","c" (RPUSH was used this time)
The resulting list stored at mylist will contain the elements "b","a","c".
The max length of a list is 232-1 elements (4294967295, more than 4 billion of elements per list).
Redis Lists are implemented as doubly liked lists. A few commands benefit from the fact the lists are doubly linked in order to reach the needed element starting from the nearest extreme (head or tail). LRANGE and LINDEX are examples of such commands.
The use of linked lists also guarantees that regardless of the length of the list pushing and popping are O(1) operations.
Redis Lists cache length information so LLEN is O(1) as well.
-
Set the the rispective keys to the rispective values. MSET will replace oldvalues with new values, while MSETNX will not perform any operation at alleven if just a single key already exists.
+Time complexity: O(1) to set every key
Set the the respective keys to the respective values. MSET will replace oldvalues with new values, while MSETNX will not perform any operation at alleven if just a single key already exists.
Because of this semantic MSETNX can be used in order to set different keysrepresenting different fields of an unique logic object in a way thatensures that either all the fields or none at all are set.
Both MSET and MSETNX are atomic operations. This means that for instanceif the keys A and B are modified, another client talking to Redis can eithersee the changes to both A and B at once, or no modification at all.
Redis is a database. To be specific, Redis is a database implementing a dictionary, where every key is associated with a value. For example I can set the key "surname_1992" to the string "Smith".
-What makes Redis different from many other key-value stores, is that every single value has a type. The following types are supported:
String
List
Set
Sorted Set (since version 1.1)
+What makes Redis different from many other key-value stores, is that every single value has a type. The following types are supported:
The type of a value determines what operations (called commands) are available for the value itself.
For example you can append elements to a list stored at the key "mylist" using the LPUSH or RPUSH command in O(1). Later you'll be able to get a range of elements with LRANGE or trim the list with LTRIM. Sets are very flexible too, it is possible to add and remove elements from Sets (unsorted collections of strings), and then ask for server-side intersection, union, difference of Sets. Each command is performed through server-side atomic operations.
Please refer to the Command Reference to see the full list of operations associated to these data types.
In other words, you can look at Redis as a data structures server. A Redis user is virtually provided with an interface to Abstract Data Types, saving her from the responsibility to implement concrete data structures and algorithms. Indeed both algorithms and data structures in Redis are properly choosed in order to obtain the best performance.
Redis loads and mantains the whole dataset into memory, but the dataset is persistent, since at the same time it is saved on disk, so that when the server is restarted data can be loaded back in memory.
There are two kind of persistence supported: the first one is called snapshotting. In this mode Redis, from time to time, writes a dump on disk asynchronously. The dataset is loaded from the dump every time the server is (re)started.
Redis can be configured to save the dataset when a certain number of changes is reached and after a given number of seconds elapses. For example, you can configure Redis to save after 1000 changes and at most 60 seconds since the last save. You can specify any combination for these numbers.
Because data is written asynchronously, when a system crash occurs, the last few queries can get lost (that is acceptable in many applications but not in all). In order to make this a non issue Redis supports another, safer persistence mode, called Append Only File, where every command received altering the dataset (so not a read-only command, but a write command) is written on an append only file ASAP. This commands are replayed when the server is restarted in order to rebuild the dataset in memory.
Redis Append Only File supports a very handy feature: the server is able to safely rebuild the append only file in background in a non-blocking fashion when it gets too long. You can find more details in the Append Only File HOWTO.
Whatever will be the persistence mode you'll use Redis supports master-slave replications if you want to stay really safe or if you need to scale to huge amounts of reads.
Redis Replication is trivial to setup. So trivial that all you need to do in order to configure a Redis server to be a slave of another one, with automatic synchronization if the link will go down and so forth, is the following config line: slaveof 192.168.1.100 6379. We provide a Replication Howto if you want to know more about this feature.
Redis can be used as a memcached on steroids because is as fast as memcached but with a number of features more. Like memcached, Redis also supports setting timeouts to keys so that this key will be automatically removed when a given amount of time passes.
All these features allow to use Redis as the sole DB for your scalable application without the need of any relational database. We wrote a simple Twitter clone in PHP + Redis to show a real world example, the link points to an article explaining the design and internals in very simple words.
Redis supports multiple databases with commands to atomically move keys from one database to the other. By default DB 0 is selected for every new connection, but using the SELECT command it is possible to select a different database. The MOVE operation can move an item from one DB to another atomically. This can be used as a base for locking free algorithms together with the 'RANDOMKEY' commands.
(note, you can skip this section if you are only interested in "formal" doc.)
Later in this document you can find detailed information about Redis commands,
@@ -79,7 +79,6 @@ exist, and ':1' for 'foo', a key that actually exists. Replies starting with the
learn all the commands supported by Redis and the PROTOCOL SPECIFICATION
section for more details about the protocol used if you plan to implement one
for a language missing a decent client implementation.
Add the specified member to the set value stored at key. If memberis already a member of the set no operation is performed. If keydoes not exist a new set with the specified member as sole member iscrated. If the key exists but does not hold a set value an error isreturned.
+Time complexity O(1)
Add the specified member to the set value stored at key. If memberis already a member of the set no operation is performed. If keydoes not exist a new set with the specified member as sole member iscreated. If the key exists but does not hold a set value an error isreturned.
C1 and C2 read lock.foo to check the timestamp, because SETNX returned 0 to both C1 and C2, as the lock is still hold by C3 that crashed after holding the lock.
C1 sends DEL lock.foo
C1 sends SETNX => success!
C2 sends DEL lock.foo
C2 sends SETNX => success!
ERROR: both C1 and C2 acquired the lock because of the race condition.
Fortunately it's possible to avoid this issue using the following algorithm.Let's see how C4, our sane client, uses the good algorithm:
C4 sends SETNX lock.foo in order to acquire the lock
The crashed C3 client still holds it, so Redis will reply with 0 to C4.
C4 GET lock.foo to check if the lock expired. If not it will sleep one second (for instance) and retry from the start.
If instead the lock is expired because the UNIX time at lock.foo is older than the current UNIX time, C4 tries to perform GETSET lock.foo <current unix timestamp + lock timeout + 1>
Thanks to the GETSET command semantic C4 can check if the old value stored at key is still an expired timestamp. If so we acquired the lock!
Otherwise if another client, for instance C5, was faster than C4 and acquired the lock with the GETSET operation, C4 GETSET operation will return a non expired timestamp. C4 will simply restart from the first step. Note that even if C4 set the key a bit a few seconds in the future this is not a problem.
+IMPORTANT NOTE: In order to make this locking algorithm more robust, a client holding a lock should always check the timeout didn't expired before to unlock the key with DEL because client failures can be complex, not just crashing but also blocking a lot of time against some operation and trying to issue DEL after a lot of time (when the LOCK is already hold by some other client).
diff --git a/doc/Sets.html b/doc/Sets.html
index db7e7f9dd..19fe7f806 100644
--- a/doc/Sets.html
+++ b/doc/Sets.html
@@ -26,8 +26,7 @@
Redis Sets are unordered collections of Redis Strings. It's possible to add, remove, and test for existence of members in O(1).
Redis Sets have the desirable property of not allowing repeated members. Adding the same element multiple times will result in a set having a single copy of this element. Practically speaking this means that adding an members does not require a "check if exists then add" operation.
Commands operating on sets try to make a good use of the return value in order to signal the application about previous existence of members. For instance the SADD command will return 1 if the element added was not already a member of the set, otherwise will return 0.
The max number of members in a set is 232-1 (4294967295, more than 4 billion of members per set).
Redis Sets support a wide range of operations, like union, intersection, difference. Intersection is optimized in order to perform the smallest number of lookups. For instance if you try to intersect a 10000 members set with a 2 members set Redis will iterate the 2 members set testing for members existence in the other set, performing 2 lookups instead of 10000.
Redis Sets are implemented using hash tables, so adding, removing and testing for members is O(1) in the average. The hash table will automatically resize when new elements are added or removed into a Set.
The hash table resizing is a blocking operation performed synchronously so working with huge sets (consisting of many millions of elements) care should be taken when mass-inserting a very big amount of elements in a Set while other clients are querying Redis at high speed.
It is possible that in the near future Redis will switch to skip lists (already used in sorted sets) in order to avoid such a problem.
-
+ #sidebar SetCommandsSidebar
Redis Sets are unordered collections of Redis Strings. It's possible to add, remove, and test for existence of members in O(1).
Redis Sets have the desirable property of not allowing repeated members. Adding the same element multiple times will result in a set having a single copy of this element. Practically speaking this means that adding an members does not require a "check if exists then add" operation.
Commands operating on sets try to make a good use of the return value in order to signal the application about previous existence of members. For instance the SADD command will return 1 if the element added was not already a member of the set, otherwise will return 0.
The max number of members in a set is 232-1 (4294967295, more than 4 billion of members per set).
Redis Sets support a wide range of operations, like union, intersection, difference. Intersection is optimized in order to perform the smallest number of lookups. For instance if you try to intersect a 10000 members set with a 2 members set Redis will iterate the 2 members set testing for members existence in the other set, performing 2 lookups instead of 10000.
Redis Sets are implemented using hash tables, so adding, removing and testing for members is O(1) in the average. The hash table will automatically resize when new elements are added or removed into a Set.
The hash table resizing is a blocking operation performed synchronously so working with huge sets (consisting of many millions of elements) care should be taken when mass-inserting a very big amount of elements in a Set while other clients are querying Redis at high speed.
It is possible that in the near future Redis will switch to skip lists (already used in sorted sets) in order to avoid such a problem.
Redis Sorted Sets are, similarly to Sets, collections of Redis Strings. The difference is that every member of a Sorted Set hash an associated score that is used in order to take this member in order.
The ZaddCommand command is used to add a new member to a Sorted Set, specifying the score of the element. Calling ZADD against a member already present in the sorted set but using a different score will update the score for the element, moving it to the right position in order to preserve ordering.
It's possible to get ranges of elements from Sorted Sets in a very similar way to what happens with Lists and the LRANGE command using the Sorted Sets ZRANGE command.
It's also possible to get or remove ranges of elements by score using the ZRANGEBYSCORE and ZREMRANGEBYSCORE commands.
The max number of members in a sorted set is 232-1 (4294967295, more than 4 billion of members per set).
Note that while Sorted Sets are already ordered, it is still possible to use the SORT command against sorted sets to get the elements in a different order.
Redis Sets are implemented using a dual-ported data structure containing a skip list and an hash table. When an element is added a map between the element and the score is added to the hash table (so that given the element we get the score in O(1)), and a map between the score and the element is added in the skip list so that elements are taken in order.
Redis uses a special skip list implementation that is doubly linked so that it's possible to traverse the sorted set from tail to head if needed (Check the ZREVRANGE command).
When ZADD is used in order to update the score of an element, Redis retrieve the score of the element using the hash table, so that it's fast to access the element inside the skip list (that's indexed by score) in order to update the position.
Like it happens for Sets the hash table resizing is a blocking operation performed synchronously so working with huge sorted sets (consisting of many millions of elements) care should be taken when mass-inserting a very big amount of elements in a Set while other clients are querying Redis at high speed.
It is possible that in the near future Redis will switch to skip lists even for the element => score map, so every Sorted Set will have two skip lists, one indexed by element and one indexed by score.
-
+ #sidebar SortedSetCommandsSidebar
Redis Sorted Sets are, similarly to Sets, collections of Redis Strings. The difference is that every member of a Sorted Set hash an associated score that is used in order to take this member in order.
The ZaddCommand command is used to add a new member to a Sorted Set, specifying the score of the element. Calling ZADD against a member already present in the sorted set but using a different score will update the score for the element, moving it to the right position in order to preserve ordering.
It's possible to get ranges of elements from Sorted Sets in a very similar way to what happens with Lists and the LRANGE command using the Sorted Sets ZRANGE command.
It's also possible to get or remove ranges of elements by score using the ZRANGEBYSCORE and ZREMRANGEBYSCORE commands.
The max number of members in a sorted set is 232-1 (4294967295, more than 4 billion of members per set).
Note that while Sorted Sets are already ordered, it is still possible to use the SORT command against sorted sets to get the elements in a different order.
Redis Sets are implemented using a dual-ported data structure containing a skip list and an hash table. When an element is added a map between the element and the score is added to the hash table (so that given the element we get the score in O(1)), and a map between the score and the element is added in the skip list so that elements are taken in order.
Redis uses a special skip list implementation that is doubly linked so that it's possible to traverse the sorted set from tail to head if needed (Check the ZREVRANGE command).
When ZADD is used in order to update the score of an element, Redis retrieve the score of the element using the hash table, so that it's fast to access the element inside the skip list (that's indexed by score) in order to update the position.
Like it happens for Sets the hash table resizing is a blocking operation performed synchronously so working with huge sorted sets (consisting of many millions of elements) care should be taken when mass-inserting a very big amount of elements in a Set while other clients are querying Redis at high speed.
It is possible that in the near future Redis will switch to skip lists even for the element => score map, so every Sorted Set will have two skip lists, one indexed by element and one indexed by score.
I'm accepting sponsorships for Redis development, the idea is that a company using Redis and willing to donate some money can receive something back: visibility in the Redis site, and prioritization of features planned in the TODO list that are somewhat more important for this company compared to other features.
In the last year I spent 50% of my time working on Redis. At the same time Redis is released under the very liberal BSD license, this is very important for users as it prevents that in the future the project will be killed, but at the same time it's not possible to build a business model selling licenses like it happens for MySQL. The alternative is to run a consultancy company, but this means to use more time to work with customers than working to the Redis code base itself, or sponsorship, that I think is the best option currently to ensure fast development of the project.
So, if you are considering a donation, thank you! This is a set of simple rules I developed in order to make sure I'm fair with everybody willing to help the project:
1. Every company can donate any amount of money, even 10$, in order to support Redis development.
2. Every company donating an amount equal or greater than 1000$ will be featured in the home page for at least 6 months, and anyway for all the time the sponsored feature takes to reach a stable release of Redis.
3. Every company donating at least 100$ will anyway be featured in the "Sponsors" page forever, this page is linked near to the logos of the current sponsors in the front page (the logos about point 2 of this list).
4. A sponsoring company can donate for sponsorship of a feature already in the TODO list. If a feature not planned is needed we should first get in touch, discuss if this is a good idea, put it in the TODO list, and then the sponsorship can start, but I've to be genuinely convinced this feature will be good and of general interest ;)
5. Not really a sponsorship/donation, but in rare case of a vertical, self-contained feature, I could develop it as a patch for the current stable Redis distribution for a "donation" proportional to the work needed to develop the feature, but in order to have the patch for the next release of Redis there will be to donate again for the porting work and so forth.
6. Features for which I receive a good sponsorship (proportionally to the work required to implement the sponsored feature) are prioritized and will get developed faster than other features, possibly changing the development roadmap.
7. To sponsor a specific feature is not a must, a company can just donate to the project as a whole.
+If you want to get in touch with me about this issues please drop me an email to my gmail account (username is antirez) or direct-message me @antirez on Twitter. Thanks in advance for the help!
If you just feel like donating a small amount to Redis the simplest way is to use paypal, my paypal address is antirez@invece.org. Please specify in the donation if you don't like to have your name / company name published in the donations history (the amount will not be published anyway).
+
-Time complexity O(N) where N is the total number of elements in all the provided sets
This commnad works exactly like SUNION but instead of being returned the resulting set is sotred as dstkey.
+Time complexity O(N) where N is the total number of elements in all the provided sets
This command works exactly like SUNION but instead of being returned the resulting set is stored as dstkey. Any existing value in dstkey will be over-written.
Hello! The followings are pointers to different parts of the Redis Documentation.
The README is the best starting point to know more about the project.
This short Quick Start provides a five minutes step-by-step istructions on how to download, compile, run and test the basic workings of a Redis server.
The command reference is a description of all the Redis commands with links to command specific pages.
Hello! The followings are pointers to different parts of the Redis Documentation.
The README is the best starting point to know more about the project.
This short Quick Start provides a five minutes step-by-step istructions on how to download, compile, run and test the basic workings of a Redis server.
The command reference is a description of all the Redis commands with links to command specific pages.
The Redis Replication HOWTO is what you need to read in order to understand how Redis master <-> slave replication works.
The Append Only File HOWTO explains how the alternative Redis durability mode works. AOF is an alternative to snapshotting on disk from time to time (the default).
The Protocol Specification is all you need in order to implement a Redis client library for a missing language. PHP, Python, Ruby and Erlang are already supported.
+
+ 
