caleb-brown/redict
1
0
Fork 0
mirror of https://codeberg.org/redict/redict.git synced 2025-01-23 08:38:27 -05:00
Code Issues Actions Packages Projects Releases Wiki Activity
8,627 Commits 4 Branches 5 Tags 115 MiB
6203ed0474
Commit Graph

6124 Commits

Author SHA1 Message Date
Salvatore Sanfilippo
2478b47b89
Merge pull request #6304 from dbenders/fix-issue-6141 
Fix issue #6141: cluster rebalance does not support binary key
 2019-09-30 17:43:58 +02:00
antirez
6a2831c24a Fix comments aesthetics. 2019-09-30 17:17:36 +02:00
antirez
23f5cb4233 Change a bit the style of #6385. 2019-09-30 10:58:15 +02:00
Salvatore Sanfilippo
ad45d7e407
Merge pull request #6385 from filipecosta90/perf-reply-ss-error 
Improve performance of RM_ReplyWithSimpleString and RM_ReplyWi…
 2019-09-30 10:57:05 +02:00
antirez
916c4e5d86 moduleRDBLoadError(): io->ctx may be NULL. 
The correct way to access the module about a given IO context is to
deference io->type->module, since io->ctx is only populated if the user
requests an explicit context from an IO object.
 2019-09-27 18:42:38 +02:00
antirez
8b5848a4f5 Fix memory leak in moduleLoadFromQueue(). 2019-09-27 18:33:21 +02:00
antirez
b394817754 Fix memory leak in RM_UnregisterCommandFilter(). 2019-09-27 18:33:05 +02:00
antirez
0a07f8ffee TerminateModuleForkChild(): fix function prototype. 2019-09-27 12:23:07 +02:00
antirez
721d3c9e0c TerminateModuleForkChild(): move safety checks there. 
We don't want that the API could be used directly in an unsafe way,
without checking if there is an active child. Now the safety checks are
moved directly in the function performing the operations.
 2019-09-27 12:17:47 +02:00
antirez
de1f82aa33 Function renamed hasForkChild() -> hasActiveChildProcess(). 2019-09-27 12:03:09 +02:00
antirez
82845f8d04 TerminateModuleForkChild(): use wait4 for safety. 
In theory currently there is only one active child, but the API may
change or for bugs in the implementation we may have several (it was
like that for years because of a bug). Better to wait for a specific
pid and avoid consuing other pending children information.
 2019-09-27 11:59:58 +02:00
antirez
beb1356b5b Improve error message in BGSAVE. 2019-09-27 11:59:37 +02:00
antirez
ae3ef964c1 Modules fork: improve SIGUSR1 handling, fix include. 
We can't expect SIGUSR1 to have any specific value range, so let's
define an exit code that we can handle in a special way.
This also fixes an #include <wait.h> that is not standard.
 2019-09-27 11:39:45 +02:00
Salvatore Sanfilippo
6129758558
Merge branch 'unstable' into modules_fork 2019-09-27 11:24:06 +02:00
antirez
fddc4757c8 BGREWRITEAOF: improve the generic error message. 2019-09-26 16:14:21 +02:00
Oran Agra
83e87bac76 Fix lastbgsave_status, when new child signal handler get intended kill 
And add a test for that.
 2019-09-26 15:16:34 +03:00
antirez
b7b23bdfb8 INFO: more info about loaded modules. 
Related to #6024.
 2019-09-26 12:18:55 +02:00
Salvatore Sanfilippo
c1ea6175c5
Merge pull request #6024 from itamarhaber/info_modules 
Adds a "Modules" section to `INFO`
 2019-09-26 11:58:52 +02:00
Salvatore Sanfilippo
959fb5cf68
Merge pull request #6235 from oranagra/module_rdb_load_errors 
Allow modules to handle RDB loading errors.
 2019-09-26 11:52:42 +02:00
Salvatore Sanfilippo
b0a90d8fa8
Merge pull request #4980 from yossigo/rm-assert 
Add RedisModule_Assert() API call.
 2019-09-26 11:39:29 +02:00
filipecosta90
af15b285fa [fix] fixed the un-refactor bug. 2019-09-25 17:28:42 +01:00
Salvatore Sanfilippo
388bc55344
Merge pull request #6354 from jaredzhu/aarch64-devel 
Add aarch64 uc_mcontext debug dump info
 2019-09-25 18:14:24 +02:00
antirez
b3d6cb268a Modify #6401 changes to fit 80 cols. 2019-09-25 18:08:11 +02:00
Salvatore Sanfilippo
f3aaf2b4d8
Merge pull request #6401 from valentinogeron/fix-discard-during-oom 
DISCARD should not fail during OOM
 2019-09-25 18:07:25 +02:00
Salvatore Sanfilippo
f6cf08e582
Merge pull request #6402 from mieko/unstable 
Seed SipHash with 128-bit key
 2019-09-25 18:03:39 +02:00
antirez
8a531cedb0 ACL: fix ##6408, default user state affecting all the connections. 2019-09-25 17:45:05 +02:00
Mike A. Owens
0a4d2bbd9c Seed SipHash with 128-bit key 
SipHash expects a 128-bit key, and we were indeed generating 128-bits,
but restricting them to hex characters 0-9a-f, effectively giving us
only 4 bits-per-byte of key material, and 64 bits overall.

Now, we skip the hex conversion and supply 128 bits of unfiltered
random data.
 2019-09-23 19:24:09 -04:00
filipecosta90
733280d9cb [fix] un-refactor the code. [perf] replyWithStatus now makes usage of addReplyProto 2019-09-23 23:45:31 +01:00
antirez
eda703ab28 redis-cli: support for ACL style user/pass AUTH. 2019-09-23 19:57:13 +02:00
antirez
b21dd082c3 redis-cli: CSV and RAW target for more RESP3 types. 2019-09-23 19:36:06 +02:00
antirez
cc108057ff redis-cli: ability to start a session in RESP3 mode. 2019-09-23 17:55:05 +02:00
antirez
8ea185ea30 redis-cli: AUTH can now have 3 arguments as well. 2019-09-23 17:47:36 +02:00
antirez
bb7546c913 RESP3: varbatim handling for other redis-cli outputs. 2019-09-23 17:42:57 +02:00
antirez
264708427b RESP3: implementation of verbatim output with TTY target. 2019-09-23 17:39:42 +02:00
valentino
7a73b7f168 DISCARD should not fail during OOM 
discard command should not fail during OOM, otherwise client MULTI state
will not be cleared.
 2019-09-22 09:22:53 +03:00
antirez
2e4fa7bb48 Make config.c always_replicate_commands more uniform. 
Better if it resembles the other similar code paths.
 2019-09-20 11:46:35 +02:00
antirez
1dd36347ed Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-09-20 11:45:02 +02:00
antirez
630638dcde Remove redundant statement in config.c. 
Thanks to @guybe7 for spotting the error in the original PR I merged.
 2019-09-20 11:44:32 +02:00
Salvatore Sanfilippo
f135d6295b
Merge pull request #6367 from Dolondro/unstable 
Spelling: Typo fixes for API documentation
 2019-09-20 11:33:58 +02:00
Salvatore Sanfilippo
3976f4c88e
Merge pull request #6368 from suntw2015/unstable 
fix rdb function rdbLoadIntegerObject comment
 2019-09-20 11:33:27 +02:00
Salvatore Sanfilippo
d0050db236
Merge pull request #6389 from ocadaruma/patch-1 
Fix typos in hyperloglog.c
 2019-09-20 11:24:07 +02:00
Salvatore Sanfilippo
975eb73145
Merge pull request #6400 from mors741/unstable 
Fix compilation error
 2019-09-20 11:19:56 +02:00
antirez
0a146a8be4 Add useless break for uniformity / future protection. 2019-09-20 11:19:07 +02:00
Salvatore Sanfilippo
bd90c99f83
Merge pull request #6398 from trevor211/fixRESP3ToLuaBool 
RESP3: Fix function redisProtocolToLuaType about RESP3->Lua bools.
 2019-09-20 11:15:20 +02:00
Evgeny
a1a65d238c
Fix compilation error 2019-09-20 01:11:20 -07:00
WuYunlong
30a3644e64 RESP3: Fix function redisProtocolToLuaType about RESP3->Lua bools. 2019-09-20 08:37:23 +08:00
WuYunlong
74e3a622a2 Fix bad handling of unexpected option while loading config "lua-replicate-commands". 2019-09-20 08:14:36 +08:00
antirez
474a923160 RESP3: Use verbatim in LATENCY subcommands. 2019-09-18 18:53:22 +02:00
antirez
cb384127d1 RESP3: Use verbatim in CLIENT LIST. 2019-09-18 18:52:13 +02:00
antirez
dd2f695d7e RESP3: Use verbatim in CLUSTER subcommands. 2019-09-18 18:51:15 +02:00
antirez
1b3cb3b0de RESP3: Use verbatim in MEMORY subcommands. 2019-09-18 18:48:14 +02:00
antirez
ff9a5d231b RESP3: Use verbatim in DEBUG HTSTATS / HTSTATS-KEY. 2019-09-18 18:46:11 +02:00
antirez
aca5482fbf RESP3: Use verbatim in INFO output. 2019-09-18 18:33:13 +02:00
antirez
68519b7034 RESP3: Verbatim conversion test in the LOLWUT command. 
redis-cli is currently not able to handle it after going in RESP3 mode,
because of hiredis limitations.
 2019-09-18 18:28:51 +02:00
antirez
8ea4bdd91d RESP3: Lua double -> RESP3 conversion. 2019-09-17 19:26:46 +02:00
antirez
89f929b12a RESP3: RESP3 double -> Lua conversion. 2019-09-17 19:20:30 +02:00
antirez
19aba4ac78 RESP3: double -> human readable conversion. 2019-09-17 19:08:33 +02:00
antirez
e8e30bc402 RESP3: bool and null values in RESP -> human readable conversion. 2019-09-17 18:57:24 +02:00
Okada Haruki
c1cc5ca767
Fix typo 2019-09-17 06:18:01 +09:00
antirez
f01f0c02d1 RESP3: convert RESP3 null as Lua nil. Implement RESP3->Lua bools. 2019-09-16 18:36:16 +02:00
antirez
6931004969 RESP3: change behavior of Lua returning true/false for RESP3. 
Here we introduce a change in the way we convert values from Lua to
Redis when RESP3 is selected: this is possible without breaking the fact
we can return directly what a command returned, because there is no
Redis command in RESP2 that returns true or false to Lua, so the
conversion in the case of RESP2 is totally arbitrary. When a script is
written selecting RESP3 from Lua, it totally makes sense to change such
behavior and return RESP3 true/false when Lua true/false is returned.
 2019-09-16 18:18:17 +02:00
antirez
2cc4d0286c RESP3: implement new NULL representation parsing in Lua. 2019-09-16 17:49:47 +02:00
antirez
ca81d49006 RESP3: handle set Lua -> Redis conversion. 2019-09-16 12:19:19 +02:00
antirez
c792504fab RESP3: handle map Lua -> Redis conversion. 2019-09-16 12:15:39 +02:00
antirez
dad38c19c8 RESP3: report set/map as nested tables to Lua. 2019-09-16 11:49:42 +02:00
filipecosta90
4a30a26f8f [add] improved performance of RM_ReplyWithSimpleString and RM_ReplyWithError by making usage addReplyProto instead of addReplySds 2019-09-15 21:16:30 +01:00
antirez
888efc1b36 RESP3: Lua parsing should depend on lua client, not lua caller. 
We want all the scripts to run in RESP2 mode by default. It's up to the
caller to switch to V3 using redis.setresp() if it is really needed.
This way most scripts written for past Redis versions will continue to
work with Redis >= 6 even if the client is in RESP3 mode.
 2019-09-13 19:38:39 +02:00
antirez
26b6c697d3 RESP3: Lua debugger support for printing sets and maps. 2019-09-13 19:19:10 +02:00
antirez
93c52ff5ff RESP3: implement lua.setresp(). 2019-09-13 19:01:39 +02:00
antirez
b170a01fa7 ACL: store hashed passwords in memory. 
Note that this breaks API compatibility with Redis < 6:

    CONFIG GET requirepass

Will no longer return a cleartext password as well, but the SHA256 hash
of the password set.
 2019-09-12 12:54:57 +02:00
antirez
ae5054b476 ACL: SHA256 based password hashing function implemented. 2019-09-12 12:33:22 +02:00
antirez
9d2ecf6be3 ACL: add slightly modified version of sha256.c for password hashing. 
memory.h include removed, types substituted with stdint types.
 2019-09-12 12:21:37 +02:00
antirez
f4d37173fe ACL: protect MULTI/EXEC transactions after rules change. 2019-09-11 19:42:10 +02:00
antirez
ae8cab3296 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-09-06 12:24:40 +02:00
antirez
a092f20d87 handleClientsBlockedOnKeys() refactoring. 2019-09-06 12:24:26 +02:00
suntiawnen
781f3fd8be fix rdb function rdbLoadIntegerObject comment 2019-09-06 12:01:44 +08:00
Doug Nelson
cc8f06ece9 Typo fixes in API documentation 2019-09-05 16:25:06 +01:00
Salvatore Sanfilippo
86aade9a02
Merge pull request #6364 from oranagra/fix_module_aux_when 
Fix to module aux data rdb format for backwards compatibility with old check-rdb
 2019-09-05 13:30:26 +02:00
Oran Agra
3bfcae247a Fix to module aux data rdb format for backwards compatibility with old check-rdb 
When implementing the code that saves and loads these aux fields we used rdb
format that was added for that in redis 5.0, but then we added the 'when' field
which meant that the old redis-check-rdb won't be able to skip these.
this fix adds an opcode as if that 'when' is part of the module data.
 2019-09-05 14:11:37 +03:00
antirez
89ad0ca566 Fix handleClientsBlockedOnKeys() names in comments. 2019-09-05 13:05:57 +02:00
antirez
12c56a8e75 redis-cli: always report server errors on read errors. 
Before this commit we may have not consumer buffers when a read error is
encountered. Such buffers may contain errors that are important clues
for the user: for instance a protocol error in the payload we send in
pipe mode will cause the server to abort the connection. If the user
does not get the protocol error, debugging what is happening can be a
nightmare.

This commit fixes issue #3756.
 2019-09-04 17:55:46 +02:00
antirez
f7090f43ad AOF: be future-proof and close the file pointer. 
Currently useless but we release the fake client, so better to do a full
cleanup. Thanks to @TomMD reporting this in #6353.
 2019-09-04 17:20:37 +02:00
zhudacai 00228490
66a5c1cd7e The aarch64 architecture is support normal memory unaligned accesses, 
so add the UNALIGNED_LE_CPU to the aarch64 .
 2019-09-04 12:14:25 +00:00
antirez
b2e10131c0 Rio: fix flag name, function is never used btw. 
Thanks to @tnclong for reporting the problem.
 2019-09-04 13:01:07 +02:00
antirez
06d490342f RESP3: fix cases of NULL reported instead of empty aggregate. 2019-09-02 12:50:47 +02:00
antirez
b12ccc840e Cluster: abort loading nodes data if vars arguments are unbalanced. 
See for reference PR #6337. Thanks to @git-hulk for spotting this.
 2019-09-02 11:41:20 +02:00
antirez
e57067b677 More strict checks and better comments in flushSlaveOutputBuffers(). 
Related to #6296.
 2019-08-31 14:46:24 +02:00
antirez
2303ba1441 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-08-31 14:41:28 +02:00
antirez
ca5ccc58c5 Improve comment in flushSlavesOutputBuffers(). 2019-08-31 14:40:09 +02:00
Salvatore Sanfilippo
16435e02bf
Merge pull request #6296 from soloestoy/flushSlavesOutputBuffers-bugfix 
networking: flushSlavesOutputBuffers bugfix
 2019-08-31 14:39:33 +02:00
zhudacai 00228490
d7c25806c0 src/debug.c do not support aarch64 dump utcontext, so add the context 
of aarch64.
The content comes from the definition of the sigcontext and tested on
my aarch64 server.

sigcontext defined at the linux kernel code:
arch/arm64/include/uapi/asm/sigcontext.h
 2019-08-31 07:47:11 +00:00
Oran Agra
78bbb9b58d Modlue fork is killed when the parent exists 2019-08-25 10:11:48 +03:00
Oran Agra
61853ad8de Module INFO, support default section for simple modules 2019-08-18 10:01:57 +03:00
Oran Agra
1d6e5dc4dc Module INFO, add support for dict fields, rename API to have common prefix 2019-08-18 09:41:45 +03:00
Oran Agra
1026d2caf8 fix error handling on config parsing of repl-diskless-load 2019-08-08 14:53:36 +03:00
Oran Agra
0a97149dec RM_ReplyWithCString was missing registration 2019-08-07 13:06:11 +03:00
Diego Bendersky
b27f388344 pass len to alloc in clusterManagerMigrateKeysInReply 2019-08-05 17:35:50 -03:00
antirez
0e9b5adbd3 Replication: clarify why repl_put_online_on_ack exists at all. 2019-08-05 17:38:15 +02:00
zhaozhao.zz
d3f7eb0c87 networking: flushSlavesOutputBuffers bugfix 2019-08-02 17:17:19 +08:00
antirez
583933e2d6 Fix regression causing EXEC to appear in the slow log. 
This was recently introduced with PR #6266.
 2019-07-31 19:05:20 +02:00
antirez
2a8a63af44 Make EMBSTR case of #6261 more obvious. 2019-07-31 12:03:10 +02:00
Salvatore Sanfilippo
f899c31fbc
Merge pull request #6261 from chendq8/robj 
make memory usage consistent of robj with OBJ_ENCODING_INT
 2019-07-31 11:54:11 +02:00
Salvatore Sanfilippo
fc5c2052b5
Merge pull request #6266 from madolson/dev-unstable-hide-auth-and-hello 
Hide HELLO and AUTH from slowlog and monitor
 2019-07-31 11:12:46 +02:00
antirez
e1b13b4d39 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-31 10:37:17 +02:00
antirez
b2079cb508 HyperLogLog: fix the fix of a corruption bug. 2019-07-31 10:36:23 +02:00
Salvatore Sanfilippo
f0c10d4360
Merge pull request #6277 from devnexen/fbsd_get_rss 
Updating resident memory request impl on FreeBSD.
 2019-07-30 18:24:38 +02:00
Oran Agra
40c4183196 Log message when modules prevent diskless-load 2019-07-30 16:32:58 +03:00
Oran Agra
e5187ad2ae Merge remote-tracking branch 'oss/unstable' into module_rdb_load_errors 2019-07-30 15:14:08 +03:00
Oran Agra
4339706e07 Avoid diskelss-load if modules did not declare they handle read errors 2019-07-30 15:11:57 +03:00
Salvatore Sanfilippo
f42846e8c7
Merge pull request #6264 from oranagra/modules_api_aux_rdb 
Implement module api for aux data in rdb
 2019-07-30 11:50:44 +02:00
Salvatore Sanfilippo
ce72cac0f8
Merge pull request #6269 from wubostc/patch-1 
Reduce the calling stack
 2019-07-30 11:41:48 +02:00
antirez
e4b3c8bbc3 emptyDbGeneric(): call signalFlushDb() before deleting the keys. 
This was broken since a refactoring performed recently by myself.
 2019-07-30 11:20:54 +02:00
John Sully
d659654f53 Fix HLL corruption bug 2019-07-29 18:11:52 -04:00
David Carlier
f1c6c658ac Updating resident memory request impl on FreeBSD. 2019-07-28 14:33:57 +01:00
Salvatore Sanfilippo
5591a14c0a
Merge pull request #6248 from oranagra/module-log 
RM_Log - add support for logging without a context or context without module
 2019-07-25 18:48:46 +02:00
Oran Agra
e91d9a6fff Extend modules API to allow modules report to redis INFO 
this implements #6012
 2019-07-24 12:58:15 +03:00
antirez
fde5e737dc Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-24 11:45:19 +02:00
antirez
32efd9adf8 Client side caching: config option for table fill rate. 2019-07-24 11:35:01 +02:00
wubostc
48f4cfb601
Reduce the calling stack 2019-07-24 16:22:26 +08:00
Salvatore Sanfilippo
dbee7ca007
Merge pull request #6265 from madolson/dev-unstable-remove-create-object 
Removed unnecessary creation of Redis objects
 2019-07-23 17:05:10 +02:00
antirez
c98e7717bb Client side caching: show tracking slots usage in INFO. 2019-07-23 11:02:14 +02:00
antirez
9268493e8d Client side caching: implement full slot limit function. 2019-07-23 10:57:22 +02:00
zhaozhao.zz
9012e587ca Client side caching: do not reclaim tracking table if it's empty 2019-07-23 15:25:00 +08:00
Madelyn Olson
7d21754710 Hide HELLO and AUTH from slowlog and monitor 2019-07-22 22:53:15 -07:00
Madelyn Olson
8fe173452a Removed unecessary creation of Redis objects 2019-07-22 22:52:16 -07:00
Oran Agra
3b6aeea44c Implement module api for aux data in rdb 
Other changes:
* fix memory leak in error handling of rdb loading of type OBJ_MODULE
 2019-07-22 21:15:33 +03:00
antirez
c41f94d2a3 Client side caching: split invalidation into key / slot. 2019-07-22 18:59:53 +02:00
antirez
09c06698e9 Client side caching: redis-cli ability to enable tracking. 
This is extremely useful in order to simulate an high load of requests
about different keys, and force Redis to track a lot of informations
about several clients, to simulate real world workloads.
 2019-07-22 18:45:47 +02:00
antirez
436d02dd7b Move signalFlushedDb() into a better place. 
Now that the call also invalidates client side caching slots, it is
important that after an internal flush operation we both send the
notifications to the clients and, at the same time, are able to reclaim
the memory of the tracking table. This may even fix a few edge cases
related to MULTI/EXEC + WATCH during resync, not sure, but in general
looks more correct.
 2019-07-22 12:37:47 +02:00
antirez
7d3992a80a Client side caching: reclaim the tracking table on FLUSHALL. 2019-07-22 12:33:49 +02:00
antirez
842b44dc46 Client side caching: call the invalidation functions always. 
Otherwise what happens is that the tracking table will never get garbage
collected if there are no longer clients with tracking enabled.
Now the invalidation function immediately checks if there is any table
allocated, otherwise it returns ASAP, so the overhead when the feature
is not used should be near zero.
 2019-07-22 12:29:54 +02:00
antirez
f850201c64 Client side caching: don't hash the key if not needed. 2019-07-22 12:10:51 +02:00
antirez
e00442eb13 Clinet side caching: take count of used caching slots. 2019-07-22 11:47:44 +02:00
chendianqiang
004c26ec28 make memory usage consistent of robj with OBJ_ENCODING_INT 2019-07-22 17:45:30 +08:00
Oran Agra
d7d028a7a7 Allow modules to handle RDB loading errors. 
This is especially needed in diskless loading, were a short read could have
caused redis to exit. now the module can handle the error and return to the
caller gracefully.

this fixes #5326
 2019-07-21 18:19:32 +03:00
antirez
bc5cb168f5 RDB: fix MODULE_AUX loading by continuing to next opcode. 
Thanks to @JohnSully for noticing this problem.
 2019-07-19 11:12:46 +02:00
antirez
5f450e4928 RDB: make sure to abort on LZF encoding error. 2019-07-18 18:59:38 +02:00
antirez
bd0f06c18c RDB: handle encoding errors with rdbExitReportCorruptRDB(). 
Without such change, the diskless replicas, when loading RDB files from
the socket will not abort when a broken RDB file gets loaded. This is
potentially unsafe, because right now Redis is not able to guarantee
that encoding errors are safe from the POV of memory corruptions (for
instance the LZF library may not be safe against untrusted data?) so
better to abort when the RDB file we are going to load is corrupted.

Instead I/O errors are still returned to the caller without aborting,
so that in case of short read the diskless replica can try again.
 2019-07-18 18:51:45 +02:00
Salvatore Sanfilippo
41ed85bd26
Merge pull request #6252 from soloestoy/tracking-flushdb 
Tracking flushdb
 2019-07-18 16:41:37 +02:00
antirez
947319cadd RDB: update rdbLoadRio comment about EOF condition. 2019-07-18 12:37:55 +02:00
zhaozhao.zz
f59119aad6 Client side caching: filter clients untracking 2019-07-18 16:46:19 +08:00
antirez
42b6305964 RDB: try to make error handling code more readable. 2019-07-17 17:30:02 +02:00
antirez
6b72b04a37 Rio: when in error condition avoid doing the operation. 2019-07-17 16:46:22 +02:00
antirez
5189db3d81 RDB: modify rdbReportError() var name for clarity. 2019-07-17 16:46:22 +02:00
antirez
48d91cf4cc Rio: remember read/write error conditions. 2019-07-17 16:46:22 +02:00
Oran Agra
c56b4ddc6f prevent diskless replica from terminating on short read 
now that replica can read rdb directly from the socket, it should avoid exiting
on short read and instead try to re-sync.

this commit tries to have minimal effects on non-diskless rdb reading.
and includes a test that tries to trigger this scenario on various read cases.
 2019-07-17 16:46:22 +02:00
Oran Agra
56258c6b7d Module API for Forking 
* create module API for forking child processes.
* refactor duplicate code around creating and tracking forks by AOF and RDB.
* child processes listen to SIGUSR1 and dies exitFromChild in order to
  eliminate a valgrind warning of unhandled signal.
* note that BGSAVE error reply has changed.

valgrind error is:
  Process terminating with default action of signal 10 (SIGUSR1)
 2019-07-17 16:40:24 +03:00
zhaozhao.zz
6191ea90a1 Client side caching: implement trackingInvalidateKeysOnFlush() 2019-07-17 20:33:52 +08:00
Oran Agra
6e07fac405 RM_Log - add support for logging without a context or context without module 
for instance detached thread safe contexts, or various callbacks that don't
provide a context.
 2019-07-17 15:08:18 +03:00
antirez
241d18d954 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-17 12:26:38 +02:00
Salvatore Sanfilippo
3f1c84751a
Merge pull request #6196 from AngusP/resp3 
[RESP3, Minor] in networking.c double representation for -infiinity leaves out comma
 2019-07-12 13:08:46 +02:00
Salvatore Sanfilippo
6100a82a3a
Merge pull request #5738 from bmerry/fix-swapdb-int-overflow 
Make dbSwapDatabases take args as long
 2019-07-12 12:36:33 +02:00
antirez
54c4e7f86c Vertically compact code in aofWrite(). 2019-07-12 12:18:33 +02:00
antirez
2a99f199bc Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-10 19:48:44 +02:00
antirez
dad0e916ff Client side caching: filter clients no longer there. 2019-07-10 19:46:46 +02:00
Salvatore Sanfilippo
7567323071
Merge pull request #6198 from oranagra/skiplist-mem-usage 
missing per-skiplist overheads in MEMORY USAGE
 2019-07-10 19:43:24 +02:00
antirez
c7aaf8db4d Client side caching: implement CLIENT GETREDIR. 
This subcommand may simplify the writing of Redis client libraries
using the tracking feature and/or improve observability and debugging
capabilities.
 2019-07-10 18:17:10 +02:00
antirez
923e4fb312 Client side caching: add tracking clients in INFO. 2019-07-10 18:08:31 +02:00
antirez
7c9f6ebc8f Diskless replica: fix disklessLoadRestoreBackups() bug. 2019-07-10 12:36:26 +02:00
antirez
3bbb9a1413 Diskless replica: refactoring of DBs backups. 2019-07-10 11:42:26 +02:00
antirez
3300e98ff0 Diskless replica: fix mispelled var name. 2019-07-10 09:34:21 +02:00
antirez
dfcbeaf115 Diskless replica: a few aesthetic changes to rio.c 2019-07-08 18:39:59 +02:00
antirez
81b18fa3a0 Diskless replica: a few aesthetic changes to replication.c. 2019-07-08 18:32:47 +02:00
Oran Agra
2de544cfcc diskless replication on slave side (don't store rdb to file), plus some other related fixes 
The implementation of the diskless replication was currently diskless only on the master side.
The slave side was still storing the received rdb file to the disk before loading it back in and parsing it.

This commit adds two modes to load rdb directly from socket:
1) when-empty
2) using "swapdb"
the third mode of using diskless slave by flushdb is risky and currently not included.

other changes:
--------------
distinguish between aof configuration and state so that we can re-enable aof only when sync eventually
succeeds (and not when exiting from readSyncBulkPayload after a failed attempt)
also a CONFIG GET and INFO during rdb loading would have lied

When loading rdb from the network, don't kill the server on short read (that can be a network error)

Fix rdb check when performed on preamble AOF

tests:
run replication tests for diskless slave too
make replication test a bit more aggressive
Add test for diskless load swapdb
 2019-07-08 15:37:48 +03:00
Salvatore Sanfilippo
722446510f
Merge pull request #6116 from AngusP/scan-types 
SCAN: New Feature `SCAN cursor [TYPE type]` modifier suggested in issue #6107
 2019-07-08 12:53:34 +02:00
Angus Pearson
6eb52e200c Change typeNameCanonicalize -> getObjectTypeName, and other style changes 2019-07-08 11:04:37 +01:00
Guy Korland
2d07883cab
fix build tracking.c should be tracking.o 
thanks to @rafie
 2019-07-07 18:28:15 +03:00
Salvatore Sanfilippo
e8ae07aabf
Merge pull request #6162 from madolson/dev-unstable-config-refactor 
Refactored yesno configs so there is less duplication
 2019-07-07 17:13:11 +02:00
Salvatore Sanfilippo
677e95e2f5
Merge pull request #6161 from swilly22/redismodule-loading-ctx-flag 
Extend REDISMODULE_CTX_FLAGS to indicate if redis is  loading
 2019-07-07 17:01:29 +02:00
Salvatore Sanfilippo
a714004738
Merge pull request #6022 from itamarhaber/RedisModule_ReplyWithCString 
Adds RedisModule_ReplyWithCString
 2019-07-05 20:25:18 +02:00
antirez
6b29f2d83d Client side caching: RESP2 support. 2019-07-05 12:24:28 +02:00
antirez
f099def733 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-04 18:34:53 +02:00
Oran Agra
41c5ebcab6 missing per-skiplist overheads in MEMORY USAGE 
these had severe impact for small zsets, for instance ones with just one
element that is longer than 64 (causing it not to be ziplist encoded)
 2019-07-04 15:20:49 +03:00
antirez
3e43aed1ac Client side caching: fix invalidate message len and content. 2019-07-04 14:07:23 +02:00
antirez
46edb55de9 Client side caching: implement trackingInvalidateKey(). 2019-07-03 19:16:20 +02:00
antirez
506764b3f8 Client side caching: hook inside call() for tracking. 2019-07-03 12:42:16 +02:00
antirez
db16a861a1 Client side caching: CLIENT TRACKING subcommand. 2019-07-03 11:58:20 +02:00
Angus Pearson
593f6656c1 RESP3 double representation for -infinity is ,-inf\r\n, not -inf\r\n 2019-07-02 14:28:48 +01:00
antirez
c29f3bcf2e Client side caching: enable tracking mode. 2019-06-30 06:19:08 -04:00
antirez
45d64f229e Client side caching: fields and flags for tracking mode. 2019-06-29 20:08:41 -04:00
antirez
a28d7918d7 Client side caching: add new file and description. 2019-06-29 09:09:38 -04:00
Madelyn Olson
dd51fc5a4a Refactored yesno configs so there was less duplication 2019-06-13 12:18:25 -07:00
Angus Pearson
38cd5fd9f6 Spelling cannonical -> canonical 2019-06-13 17:49:33 +01:00
swilly22
f54d95e803 Extend REDISMODULE_CTX_FLAGS to indicate if redis is currently loading from either RDB or AOF 2019-06-12 15:37:19 +03:00
Angus Pearson
e2adea2188 Add char* typeNameCanonicalize(robj*) to remove duplicate code between SCAN and TYPE commands, 
and to keep OBJ_* enum to string canonicalization in one place.
 2019-06-10 17:41:44 +01:00
Salvatore Sanfilippo
0cabe0cfa7
Merge pull request #6149 from madolson/acl-spelling 
Fixed some spelling issues in ACL codepath including a user facing error
 2019-06-07 22:54:52 +02:00
Madelyn Olson
dd5f437871 Fixed some spelling issues in ACL codepath including user facing error 2019-06-07 13:25:22 -07:00
artix
d3ed53f11b Redis Benchmark: prevent CONFIG failure from exiting program 2019-06-05 16:34:55 +02:00
Oran Agra
09f99c2a92 make redis purge jemalloc after flush, and enable background purging thread 
jemalloc 5 doesn't immediately release memory back to the OS, instead there's a decaying
mechanism, which doesn't work when there's no traffic (no allocations).
this is most evident if there's no traffic after flushdb, the RSS will remain high.

1) enable jemalloc background purging
2) explicitly purge in flushdb
 2019-06-02 15:33:14 +03:00
Yuan Zhou
412eb67d21 aof: fix assignment for aof_fsync_offset 
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
 2019-05-29 14:48:46 +08:00
Angus Pearson
bf963253ec Implement SCAN cursor [TYPE type] modifier suggested in issue #6107. 
Add tests to check basic functionality of this optional keyword, and also tested with
a module (redisgraph). Checked quickly with valgrind, no issues.

Copies name the type name canonicalisation code from `typeCommand`, perhaps this would
be better factored out to prevent the two diverging and both needing to be edited to
add new `OBJ_*` types, but this is a little fiddly with C strings.

The [redis-doc](https://github.com/antirez/redis-doc/blob/master/commands.json) repo
will need to be updated with this new arg if accepted.

A quirk to be aware of here is that the GEO commands are backed by zsets not their own
type, so they're not distinguishable from other zsets.

Additionally, for sparse types this has the same behaviour as `MATCH` in that it may
return many empty results before giving something, even for large `COUNT`s.
 2019-05-22 16:39:04 +01:00
antirez
074d24df1e Narrow the effects of PR #6029 to the exact state. 
CLIENT PAUSE may be used, in other contexts, for a long time making all
the slaves time out. Better for now to be more specific about what
should disable senidng PINGs.

An alternative to that would be to virtually refresh the slave
interactions when clients are paused, however for now I went for this
more conservative solution.
 2019-05-15 12:16:43 +02:00
Salvatore Sanfilippo
caf74e507e
Merge pull request #6029 from chendq8/clientpause 
fix cluster failover time out
 2019-05-15 12:03:19 +02:00
antirez
9724ca4e22 Make comment in getClientOutputBufferMemoryUsage() describing the present. 2019-05-13 17:30:06 +02:00
Salvatore Sanfilippo
e633254ccf
Merge pull request #6053 from soloestoy/enhance-aof-fsync-everysec 
aof: enhance AOF_FSYNC_EVERYSEC, more details in #5985
 2019-05-10 18:06:40 +02:00
Salvatore Sanfilippo
34f6d94701
Merge pull request #6074 from trevor211/fixActiveExpire 
Do not active expire keys in the background when the switch is off.
 2019-05-10 12:39:35 +02:00
Salvatore Sanfilippo
1e9b8d853d
Merge pull request #6080 from soloestoy/fix-makefile 
Makefile: 1TD -> STD
 2019-05-10 12:37:07 +02:00
Salvatore Sanfilippo
2903c56bda
Merge pull request #6073 from yossigo/fix/blocked-client-id 
Preserve client->id for blocked clients.
 2019-05-10 12:32:47 +02:00
Salvatore Sanfilippo
6b490e33c4
Merge pull request #6081 from Happy4Code/mod_comments 
change the comments there may have a mis type
 2019-05-10 12:28:12 +02:00
Salvatore Sanfilippo
891f4a38f6
Merge pull request #6077 from zltl/patch-7 
delete sdsTest() from REDIS_TEST in server.c to fix build failed
 2019-05-10 12:27:47 +02:00
Salvatore Sanfilippo
433f763814
Merge pull request #6083 from tangquanawj/fix-inappropriate-add-operation-code 
fix-inappropriate-add-operation-code
 2019-05-10 12:25:18 +02:00
Salvatore Sanfilippo
402b0d0bd1
Merge pull request #6086 from yongman/fix-uint64-in-defrag 
Fix uint64_t hash value in active defrag
 2019-05-10 12:23:59 +02:00
Angus Pearson
a1fb0be1d7 Enlarge error buffer in redis-check-aof.c to remove compiler warning of output truncation through snprintf format string 2019-05-08 12:13:45 +01:00
yongman
fb4ee7f0c5 Fix uint64_t hash value in active defrag 2019-05-08 16:29:16 +08:00
唐权
842dd85b26
Update ziplist.c 
Hi, @antirez

In the code, to get the size of ziplist, "unsigned int bytes = ZIPLIST_HEADER_SIZE+1;" is correct, 
but why not make it more readable and easy to understand
 2019-05-08 12:53:56 +08:00
zhaozhao.zz
48d591a010 fix memory leak when rewrite config file 2019-05-07 15:59:16 +08:00
stan011
d9d3d3065b change the comments there may have a mis type 2019-05-07 14:22:40 +08:00
zhaozhao.zz
340a723b87 Makefile: 1TD -> STD 2019-05-07 13:35:27 +08:00
antirez
3d053dbb6d Threaded IO: handleClientsWithPendingReadsUsingThreads top comment. 2019-05-06 18:02:51 +02:00
antirez
5baeb14cf3 Threaded IO: configuration directive for turning on/off reads. 2019-05-06 18:02:51 +02:00
antirez
1c0c436757 Threaded IO: ability to disable reads from threaded path. 2019-05-06 18:02:51 +02:00
antirez
90d720e7a5 Threaded IO: put fflush() inside tio_debug conditional. 2019-05-06 18:02:51 +02:00
antirez
6ab6a97fe6 Threaded IO: parsing WIP 2: refactoring to parse from thread. 2019-05-06 18:02:51 +02:00
antirez
647a66ebba Threaded IO: parsing WIP 1: set current_client in a better scoped way. 2019-05-06 18:02:51 +02:00
antirez
463ccf8664 Threaded IO: logging should be safe in I/O threads. 
Potentially it is possible that we get interleaved writes, even if
serverLog() makes sure to write into a buffer and then use printf(), so
even this should be ok. However in general POSIX guarantees that writing
to the same file pointer object from multiple threads is safe. Anyway
currently we *reopen* the file at each call, but for the standard output
logging.

The logging functions actually also access global configuration while
performing the log (for instance in order to check the log level, the
log filename and so forth), however dunring the I/O threads execution
we cannot alter such shared state in any way.
 2019-05-06 18:02:51 +02:00
antirez
8d7d2be24f Threaded IO: process read queue before stopping threads. 2019-05-06 18:02:51 +02:00
antirez
63a0ffd36a Threaded IO: read side WIP 3. 2019-05-06 18:02:51 +02:00
antirez
a2245f8ff1 Threaded IO: read side WIP 2. 2019-05-06 18:02:51 +02:00
antirez
dd5b105c73 Threaded IO: read side WIP. 2019-05-06 18:02:51 +02:00
antirez
74591fb5bd Threaded IO: hide more debugging printfs under conditional. 2019-05-06 18:02:51 +02:00
antirez
9814b2a5f3 Threaded IO: make num of I/O threads configurable. 2019-05-06 18:02:51 +02:00
antirez
30091dc29f Threaded IO: use main thread if num of threads is 1. 2019-05-06 18:02:51 +02:00
Ubuntu
9bf7f302a7 Threaded IO: stop threads when no longer needed + C11 in Makefile. 
Now threads are stopped even when the connections drop immediately to
zero, not allowing the networking code to detect the condition and stop
the threads. serverCron() will handle that.
 2019-05-06 18:02:51 +02:00
antirez
ea35a81c42 Threaded IO: 3rd version: use the mutex only to stop the thread. 2019-05-06 18:02:51 +02:00
antirez
6f4f36c0fb Threaded IO: second attempt without signaling conditions. 2019-05-06 18:02:51 +02:00
antirez
a2dbd9bd97 Threaded IO: allow to disable debug printf. 2019-05-06 18:02:51 +02:00
antirez
f468e653b5 Threaded IO: implement handleClientsWithPendingWritesUsingThreads(). 
This is just an experiment for now, there are a couple of race
conditions, mostly harmless for the performance gain experiment that
this commit represents so far.

The general idea here is to take Redis single threaded and instead
fan-out on expansive kernel calls: write(2) in this case, but the same
concept could be easily implemented for read(2) and protcol parsing.

However just threading writes like in this commit, is enough to evaluate
if the approach is sounding.
 2019-05-06 18:02:51 +02:00
liaotonglang
b1c7e3393d delete sdsTest() from REDIS_TEST 
sdsTest() defined in sds.c dit not match the call in server.c.
remove it from REDIS_TEST, since test-sds defined in Makefile.
 2019-05-06 19:24:00 +08:00
WuYunlong
1c0913dc4e Do not active expire keys in the background when the switch is off. 2019-05-06 11:46:07 +08:00
Yossi Gottlieb
9f3679880a Preserve client->id for blocked clients. 2019-05-05 20:32:53 +03:00
Salvatore Sanfilippo
0a6090bfd8
Merge pull request #6051 from vattezhang/unstable 
fix: benchmark auth fails when server have requirepass
 2019-05-02 09:48:02 +02:00
zhaozhao.zz
bcac165fab aof: enhance AOF_FSYNC_EVERYSEC, more details in #5985 2019-04-29 14:38:28 +08:00
Salvatore Sanfilippo
843de8b786
Merge pull request #5971 from devnexen/unstable 
build fix
 2019-04-26 17:32:21 +02:00
Salvatore Sanfilippo
5ba908fa5f
Merge pull request #6002 from yongman/fix-memleak-in-bitfield 
Fix memleak in bitfieldCommand
 2019-04-26 17:31:42 +02:00
Salvatore Sanfilippo
5c5197fe4f
Merge pull request #6013 from rouzier/fix/stream_iterator_init 
Fix stream interator start and end key initialize
 2019-04-26 17:25:20 +02:00
vattezhang
4e38ced488 fix: benchmark auth fails when server have requirepass 2019-04-26 19:47:07 +08:00
vattezhang
bc36404c79 update 2019-04-26 18:50:51 +08:00
vattezhang
d3d96b041a Merge github.com:antirez/redis into unstable 2019-04-26 18:47:11 +08:00
git-hulk
fa97ef34ca FIX: core dump in redis-benchmark when the -r is the last arg 2019-04-24 20:05:35 +08:00
chendianqiang
11f2c6b115 stop ping when client pause 2019-04-17 21:20:10 +08:00
Itamar Haber
52686f4866 Adds a "Modules" section to INFO 
Fixes #6012.

As long as "INFO is broken", this should be adequate IMO. Once we rework
`INFO`, perhaps into RESP3, this implementation should be revisited.
 2019-04-16 22:16:12 +03:00
Itamar Haber
c184b32d4c Uses addReplyBulkCString 
Signed-off-by: Itamar Haber <itamar@redislabs.com>
 2019-04-16 17:38:33 +03:00
Itamar Haber
26d9d4ec2c Adds RedisModule_ReplyWithCString 
Signed-off-by: Itamar Haber <itamar@redislabs.com>
 2019-04-16 17:15:23 +03:00
James Rouzier
3ccdcbc088 Fix start and end key initialize 2019-04-11 12:19:02 -04:00
Salvatore Sanfilippo
d044e33c20
Merge pull request #5961 from yossigo/modules-tests 
Modules tests
 2019-04-11 07:56:23 -07:00
antirez
9e67691ffb Aesthetic change to #5962 to conform to Redis style. 2019-04-10 18:53:35 +02:00
Salvatore Sanfilippo
6de5d25062
Merge pull request #5962 from oranagra/module_blocked_reply 
slave corrupts replication stream when module blocked client uses large reply (or POSTPONED_ARRAY)
 2019-04-10 09:41:36 -07:00
yongman
d490752d58 Fix memleak in bitfieldCommand 2019-04-09 09:24:22 +08:00
antirez
c24e32041b ACL: Fix memory leak in ACLResetSubcommandsForCommand(). 
This commit fixes bug reported at #5998. Thanks to @tomcat1102.
 2019-04-08 18:08:37 +02:00
antirez
75648f99a5 Fix assert comparison in fetchClusterSlotsConfiguration(). 2019-04-08 17:39:26 +02:00