caleb-brown/redict
1
0
Fork 0
mirror of https://codeberg.org/redict/redict.git synced 2025-01-23 08:38:27 -05:00
Code Issues Actions Packages Projects Releases Wiki Activity
8,984 Commits 4 Branches 5 Tags 115 MiB
bce9a68b39
Commit Graph

6408 Commits

Author SHA1 Message Date
Oran Agra
deebed23e1 Adding RM_ServerInfoGetFieldC 2019-11-04 07:57:52 +02:00
Oran Agra
87332ce524 Module API for PUBLISH, FLUSHALL, RANDOMKEY, DBSIZE 2019-11-04 07:40:52 +02:00
Oran Agra
779aebc91c Module API for loading and saving long double 
looks like each platform implements long double differently (different bit count)
so we can't save them as binary, and we also want to avoid creating a new RDB
format version, so we save these are hex strings using "%La".

This commit includes a change in the arguments of ld2string to support this.
as well as tests for coverage and short reads.

coded by @guybe7
 2019-11-03 16:42:31 +02:00
Oran Agra
4d580438b0 Add module api for looking into INFO fields 
- Add RM_GetServerInfo and friends
- Add auto memory for new opaque struct
- Add tests for new APIs

other minor fixes:
- add const in various char pointers
- requested_section in modulesCollectInfo was actually not sds but char*
- extract new string2d out of getDoubleFromObject for code reuse

Add module API for
 2019-11-03 15:02:25 +02:00
antirez
fdaea2a7a7 Modules: fix thread safe context creation crash. 
See #6525, this likely creates a NULL deference if the client was
terminated by Redis between the creation of the blocked client and the
creation of the thread safe context.
 2019-10-31 18:07:33 +01:00
antirez
1ee195d255 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-10-31 17:55:18 +01:00
antirez
dd5feec5e8 Modules: block on keys: fix stale comment. 2019-10-31 17:45:07 +01:00
antirez
825adcf3f5 Modules: block on keys: finish implementing RM_UnblockClient(). 2019-10-31 17:39:05 +01:00
Itamar Haber
9f32d0eed2 Ommits from redis-cli's history 2019-10-31 15:45:52 +02:00
antirez
37bf3e18cb Modules: block ok keys: improve example. 2019-10-31 12:31:22 +01:00
antirez
66f55bc5c1 Modules: block on keys: fix bugs in processing order. 2019-10-31 12:23:55 +01:00
antirez
629081f839 Modules: block on keys: fix the top comments. 2019-10-31 11:43:45 +01:00
antirez
91f4bdc9f9 Modules: block on keys: use a better interface. 
Using the is_key_ready() callback plus the reply callback later, creates
different issues AFAIK:

1. More complex API.
2. We need to call the reply callback() ASAP if the is_key_ready()
interface returned success, however the internals do not work in that
way, so when the reply callback is called the setup could be different.
To fix that, there is to break the current design that handles the
unblocked clients asyncrhonously, and run the list ASAP.
 2019-10-31 11:35:07 +01:00
antirez
4534960b29 Modules: remove spurious call from moduleHandleBlockedClients(). 
Now we handle propagation when we free the context.
 2019-10-31 10:32:59 +01:00
antirez
228bc89ecb Modules: block on keys: example on hellotype.c. 2019-10-31 10:30:54 +01:00
antirez
215b72c0ba Modules: block on keys: implement the internals. 2019-10-30 10:57:44 +01:00
antirez
fb6110ac20 Modules: block on keys: export APIs. 2019-10-30 10:20:28 +01:00
antirez
3649568ff2 Modules: block on keys functions layout and mechanism. 2019-10-30 10:11:58 +01:00
Jamie Scott
5eb2290f68
Add no-slowlog to acl command to prevent passwords 
Adding no-slowlog to acl command to prevent acl passwords from showing in slowlog
 2019-10-29 15:10:07 -07:00
Oran Agra
51c3ff8d75 Modules hooks: complete missing hooks for the initial set of hooks 
* replication hooks: role change, master link status, replica online/offline
* persistence hooks: saving, loading, loading progress
* misc hooks: cron loop, shutdown, module loaded/unloaded
* change the way hooks test work, and add tests for all of the above

startLoading() now gets flag indicating what is loaded.
stopLoading() now gets an indication of success or failure.
adding startSaving() and stopSaving() with similar args and role.
 2019-10-29 17:59:09 +02:00
Oran Agra
e978bdf9ef Module API for controlling LRU and LFU, and OpenKey without TOUCH 
Some commands would want to open a key without touching it's LRU/LFU
similarly to the OBJECT or DEBUG command do.

Other commands may want to implement logic similar to what RESTORE
does (and in the future MIGRATE) and get/set the LRU or LFU.
 2019-10-29 15:03:05 +02:00
Salvatore Sanfilippo
764b420f5f
Merge pull request #6487 from oranagra/module_modified_key 
Module API for explicit SignalModifiedKey instead of implicit one.
 2019-10-29 10:40:22 +01:00
Salvatore Sanfilippo
94973fd0eb
Merge pull request #6508 from guybe7/fix_module_notify_comp_error 
Fix compilation error introduced by cee6dead3
 2019-10-29 10:07:45 +01:00
Salvatore Sanfilippo
90169cdf16
Merge pull request #6495 from oranagra/rm_call_argv 
Module api RM_CallArgv and test
 2019-10-29 10:07:05 +01:00
Guy Benoish
43e2a398f9 Fix compilation error introduced by cee6dead3 
Need to add calls to REDISMODULE_API_FUNC...
 2019-10-28 17:58:07 +05:30
Oran Agra
0399b5a27e Module api tests for RM_Call 
Adding a test for coverage for RM_Call in a new "misc" unit
to be used for various short simple tests

also solves compilation warnings in redismodule.h and fork.c
 2019-10-28 12:39:57 +02:00
Guy Benoish
cee6dead3c Modules: Allow notifying custom keyspace events 
Also, add an API for getting server.notify_keyspace_events

Other (unrelated) changes:
Add RM_GetKeynameFromModuleKey
 2019-10-28 15:53:35 +05:30
antirez
cfcb475435 LOLWUT 6: always pick a default color sequence. 2019-10-28 11:10:58 +01:00
Oran Agra
a12f07792f Module API for explicit SignalModifiedKey instead of implicit one. 
This commit also fixes an uninitialized module struct member (that luckily never got released)
 2019-10-28 12:09:25 +02:00
antirez
8337e083b6 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-10-28 11:09:09 +01:00
Salvatore Sanfilippo
c328c807e7
Merge pull request #6496 from oranagra/module_tests_valgrind 
Make module tests pass with valgrind, and fix a leak in diskless load
 2019-10-28 10:05:52 +01:00
Salvatore Sanfilippo
5aab012676
Merge pull request #6498 from oranagra/rm_latency_add_sample 
Module API for LatencyAddSample
 2019-10-28 09:59:26 +01:00
Salvatore Sanfilippo
e097c99681
Merge pull request #4761 from WJWH/minor-typos 
Fix some minor typos in comments
 2019-10-28 09:58:38 +01:00
Salvatore Sanfilippo
6f2fc34228
Merge pull request #6502 from happynote3966/typo-fix 
fix comment typo in redis-cli.c
 2019-10-28 09:57:43 +01:00
Salvatore Sanfilippo
d070221707
Merge pull request #6505 from swilly22/replaywith-module-API-additions 
Introduce ReplyWithVerbatimString, ReplyWithEmptyArray, ReplyWithNull…
 2019-10-28 09:56:56 +01:00
swilly22
56a7c45521 Introduce ReplyWithVerbatimString, ReplyWithEmptyArray, ReplyWithNullArray and ReplyWithEmptyString to redis module API 2019-10-28 08:50:25 +02:00
zhaozhao.zz
c74398e130 Modules: make unloading module more safe 
As we know if a module exports module-side data types,
unload it is not allowed. This rule is the same with
blocked clients in module, because we use background
threads to implement module blocked clients, and it's
not safe to unload a module if there are background
threads running. So it's necessary to check if any
blocked clients running in this module when unload it.

Moreover, after that we can ensure that if no modules,
then no module blocked clients even module unloaded.
So, we can call moduleHandleBlockedClients only when
we have installed modules.
 2019-10-28 11:31:58 +08:00
happynote3966
cd02c5944f fix comment typo in redis-cli.c 2019-10-25 22:31:55 +09:00
Wander Hillen
fb1f4f4e7e
Merge branch 'unstable' into minor-typos 2019-10-25 10:18:26 +02:00
Oran Agra
e3b1d6d3ad Module API for LatencyAddSample 2019-10-24 14:24:55 +03:00
antirez
5dd1130272 Remove trailing space from server.c. 2019-10-24 11:20:15 +02:00
antirez
6e98214f74 Modules hooks: test flush event. 2019-10-24 10:51:03 +02:00
Oran Agra
be352633a0 Make module tests pass with valgrind, and fix a leak in diskless load 2019-10-24 09:45:25 +03:00
antirez
7cefbeaad5 Modules hooks: select DB zero by default. 2019-10-23 18:51:44 +02:00
antirez
dddfde85ef Modules hooks: fix memory leak in example module. 2019-10-23 18:39:53 +02:00
antirez
61ac2aef2b Modules hooks: for nested calls, create new fake clients. 
We can't use the same client at the same time when re-entering the hook.
 2019-10-23 18:39:53 +02:00
antirez
8cccc6b468 Modules hooks: don't call the hook for fake clients. 2019-10-23 18:39:53 +02:00
antirez
83de18805c Modules hooks: do not re-enter in hooks. 
Calling a module hook callback may result in callback operations in turn
triggering other events the module is subscribed too. We don't want to
trigger those, it's unsafe and quite confusing, and to do it correcly we
would need to maintain an event list: quite a more complex
implementation.
 2019-10-23 18:39:53 +02:00
antirez
89af7e4c85 Modules hooks: select the right DB for FLUSHDB events. 2019-10-23 18:39:53 +02:00
antirez
bc1ef48e56 Modules hooks: FLUSHDB event example. 2019-10-23 18:39:53 +02:00
antirez
ed833c9f99 Modules hooks: implement the FLUSHDB event. 2019-10-23 18:39:53 +02:00
antirez
8ec2fc3969 Modules hooks: unify structures definitions. 2019-10-23 18:39:53 +02:00
antirez
db8c9a8520 Modules hooks: document what yet to implement WIP 2. 2019-10-23 18:39:53 +02:00
antirez
18d1607f5e Modules hooks: document what yet to implement WIP 1. 2019-10-23 18:39:53 +02:00
antirez
c379fdbae2 Modules hooks: do more in example client callback. 2019-10-23 18:39:53 +02:00
antirez
cf52e0ad44 Module hooks: fixes and an example module. 2019-10-23 18:39:53 +02:00
antirez
94c50314f1 Modules hooks: fix sub event in calls from client change. 2019-10-23 18:39:53 +02:00
antirez
019ac37e9c Modules hooks: fix define / linker issues. Implement one test event. 2019-10-23 18:39:53 +02:00
antirez
d54652682d Modules hooks: handle module unloading + API export. 2019-10-23 18:39:53 +02:00
antirez
1e78681df8 Modules hooks: fix a leak and a few more issues. 2019-10-23 18:39:53 +02:00
antirez
b9af7e24e4 Modules hooks: event firing logic. 2019-10-23 18:39:53 +02:00
antirez
6e56f513b4 Modules: refactoring of RM_GetClientInfoById(). 2019-10-23 18:39:53 +02:00
antirez
21d8544128 Modules: fix RedisModule_GetClientInfoById() example. 2019-10-23 18:39:53 +02:00
antirez
e9b5484ddd Modules: export GetClientInfoById(). 2019-10-23 18:39:53 +02:00
antirez
6ea8641ae0 Modules: add the client ID to the client info structure. 2019-10-23 18:39:53 +02:00
antirez
d2d74ca0f3 Modules: RM_GetClientInfoById() draft. 2019-10-23 18:39:53 +02:00
antirez
bebb2e1994 Modules hooks: a first version of events and some API. 2019-10-23 18:39:53 +02:00
antirez
3170f63330 Modules hooks: initial design of data structures. 2019-10-23 18:39:53 +02:00
antirez
ed2ba31e4c Modules: fix moduleCreateArgvFromUserFormat() casting bug. 
In 32 bit systems casting to "long" will cut the result to 32 bit.
 2019-10-23 18:39:14 +02:00
antirez
673c9d7029 module: fix propagation API bug. 2019-10-17 18:28:34 +02:00
antirez
97f6e314c1 Modules: allow to check for AOF loading client. 2019-10-17 18:10:50 +02:00
antirez
8651e6a10d Modules: add new flags to context, replica state + more. 2019-10-17 17:49:33 +02:00
Yossi Gottlieb
8e29b0b22b Fix Makefile merge issue. 2019-10-16 17:31:02 +03:00
Yossi Gottlieb
0db3b0a0ff Merge remote-tracking branch 'upstream/unstable' into tls 2019-10-16 17:08:07 +03:00
Salvatore Sanfilippo
b8e02f2b40
Merge pull request #5926 from JimB123/unstable 
Addition of RedisModule_OnUnload hook
 2019-10-16 11:13:12 +02:00
Yossi Gottlieb
c469f6ad9e Code review minor changes (names, comments). 2019-10-15 17:21:51 +03:00
Yossi Gottlieb
71f10de4de Conns: write() 0 retval should not trigger error. 2019-10-15 17:20:58 +03:00
Yossi Gottlieb
0132189007 Fix compile warnings when BUILD_TLS=no. 2019-10-15 15:24:32 +03:00
Salvatore Sanfilippo
8e3d910d4e
Merge pull request #6429 from charsyam/feature/typo-slave 
[trivial] fix typos salves to slaves in replication.c
 2019-10-10 14:56:43 +02:00
Salvatore Sanfilippo
363604a27d
Merge pull request #6437 from OMG-By/redis-omg_by 
there should is AUTH && HELLO non authenticated state.
 2019-10-10 14:55:03 +02:00
Salvatore Sanfilippo
1677f4223c
Merge pull request #6443 from guybe7/fix_stream_rax_node_limits 
Fix usage of server.stream_node_max_*
 2019-10-10 14:49:35 +02:00
Salvatore Sanfilippo
14a9da0613
Merge pull request #6145 from oranagra/jemalloc_purge_bg 
purge jemalloc after flush, and enable background purging thread
 2019-10-10 14:48:59 +02:00
antirez
509fe9cba1 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-10-10 10:31:43 +02:00
antirez
747be463d2 Cluster: fix memory leak of cached master. 
This is what happened:

1. Instance starts, is a slave in the cluster configuration, but
actually server.masterhost is not set, so technically the instance
is acting like a master.

2. loadDataFromDisk() calls replicationCacheMasterUsingMyself() even if
the instance is a master, in the case it is logically a slave and the
cluster is enabled. So now we have a cached master even if the instance
is practically configured as a master (from the POV of
server.masterhost value and so forth).

3. clusterCron() sees that the instance requires to replicate from its
master, because logically it is a slave, so it calls
replicationSetMaster() that will in turn call
replicationCacheMasterUsingMyself(): before this commit, this call would
overwrite the old cached master, creating a memory leak.
 2019-10-10 10:23:34 +02:00
Guy Benoish
2fae0192e8 Fix usage of server.stream_node_max_* 2019-10-10 09:47:48 +02:00
omg-by
1a292e0634 there should is AUTH && HELLO non authenticated state. 2019-10-10 00:52:26 +08:00
Salvatore Sanfilippo
a92921da13
Merge pull request #6430 from jaamison/jj/unstable 
stream.h: fix typo
 2019-10-08 17:40:47 +02:00
Salvatore Sanfilippo
17bf0b25c1
Merge pull request #6427 from oranagra/static-analyzer 
fix issues found by a static analyzer
 2019-10-08 17:10:57 +02:00
antirez
009862ab7e Geo: output 10 chars of geohash, not 11. 
This does not limit the actual precision, because the last digit bits were
garbage, and the shift value became even negative in the last iteration.
 2019-10-08 17:09:37 +02:00
Yossi Gottlieb
d7f2681a0c TLS: Improve CA certifiate configuration options. 
This adds support for explicit configuration of a CA certs directory (in
addition to the previously supported bundle file).  For redis-cli, if no
explicit CA configuration is supplied the system-wide default
configuration will be adopted.
 2019-10-08 17:58:50 +03:00
Yossi Gottlieb
61733ded14 TLS: Configuration options. 
Add configuration options for TLS protocol versions, ciphers/cipher
suites selection, etc.
 2019-10-07 21:07:27 +03:00
Oran Agra
6b6294807c TLS: Implement support for write barrier. 2019-10-07 21:06:30 +03:00
Oran Agra
5a47794606 diskless replication rdb transfer uses pipe, and writes to sockets form the parent process. 
misc:
- handle SSL_has_pending by iterating though these in beforeSleep, and setting timeout of 0 to aeProcessEvents
- fix issue with epoll signaling EPOLLHUP and EPOLLERR only to the write handlers. (needed to detect the rdb pipe was closed)
- add key-load-delay config for testing
- trim connShutdown which is no longer needed
- rioFdsetWrite -> rioFdWrite - simplified since there's no longer need to write to multiple FDs
- don't detect rdb child exited (don't call wait3) until we detect the pipe is closed
- Cleanup bad optimization from rio.c, add another one
 2019-10-07 21:06:30 +03:00
Yossi Gottlieb
b087dd1db6 TLS: Connections refactoring and TLS support. 
* Introduce a connection abstraction layer for all socket operations and
integrate it across the code base.
* Provide an optional TLS connections implementation based on OpenSSL.
* Pull a newer version of hiredis with TLS support.
* Tests, redis-cli updates for TLS support.
 2019-10-07 21:06:13 +03:00
Jamison Judge
3ef6b79c80 stream.h: fix typo 2019-10-07 11:01:01 -07:00
antirez
c65347ab17 LOLWUT: version 6: change text message & credits. 2019-10-07 19:19:38 +02:00
antirez
c448f6fe55 LOLWUT: version 6: give credits. 2019-10-07 18:57:25 +02:00
antirez
58cdc054e3 LOLWUT: version 6: improve rithm of the image. 2019-10-07 18:41:29 +02:00
antirez
c10889150e LOLWUT: version 6 initial output. May change a bit. 2019-10-07 18:24:34 +02:00
charsyam
bea0384f5d fix type salves to slaves 2019-10-07 23:48:11 +09:00
antirez
f019f28e7b LOLWUT: version 6 initial concept. 2019-10-07 12:35:17 +02:00
Oran Agra
d1a005ab39 fix issues found by a static analyzer 
cluster.c - stack buffer memory alignment
    The pointer 'buf' is cast to a more strictly aligned pointer type
evict.c - lazyfree_lazy_eviction, lazyfree_lazy_eviction always called
defrag.c - bug in dead code
server.c - casting was missing parenthesis
rax.c - indentation / newline suggested an 'else if' was intended
 2019-10-07 09:09:32 +03:00
antirez
bd60447066 LOLWUT: ability to specify VERSION <ver> option. 2019-10-04 19:19:54 +02:00
antirez
9073d56eec LOLWUT: refactoring + skeleton of LOLWUT 6. 2019-10-04 19:19:48 +02:00
Oran Agra
0f1969f16f trim the double implementation of jemalloc purge 2019-10-04 14:22:13 +03:00
Oran Agra
bf759cc9c3 Merge remote-tracking branch 'antirez/unstable' into jemalloc_purge_bg 2019-10-04 13:53:40 +03:00
antirez
ee1cef189f Minor aesthetic changes to #6419. 2019-10-04 12:00:41 +02:00
Salvatore Sanfilippo
264778af0a
Merge pull request #6419 from oranagra/lua_timeout_sha 
On LUA script timeout, print the script SHA to the log
 2019-10-04 11:58:05 +02:00
Salvatore Sanfilippo
c326f28833
Merge pull request #6308 from oranagra/repl-diskless-load-config 
fix error handling on config parsing of repl-diskless-load
 2019-10-04 11:56:08 +02:00
antirez
3eaff2941e Modules: RM_Call(): give pointer to documentation. 2019-10-04 11:46:53 +02:00
antirez
2a81e49dde Modules: RM_Call/Replicate() ability to exclude AOF/replicas. 2019-10-04 11:44:53 +02:00
antirez
1bca62c4b7 Modules: RM_Replicate() in thread safe contexts. 2019-10-03 11:03:46 +02:00
antirez
e938bbc543 Modules: implement RM_Replicate() from async callbacks. 2019-10-03 10:56:37 +02:00
antirez
c549513acd Modules: handle propagation when ctx is freed. Flag modules commands ctx. 2019-10-02 18:33:40 +02:00
Oran Agra
2e19b94113 RED-31295 - redis: avoid race between dlopen and thread creation 
It seeems that since I added the creation of the jemalloc thread redis
sometimes fails to start with the following error:

Inconsistency detected by ld.so: dl-tls.c: 493: _dl_allocate_tls_init: Assertion `listp->slotinfo[cnt].gen <= GL(dl_tls_generation)' failed!

This seems to be due to a race bug in ld.so, in which TLS creation on the
thread, collide with dlopen.

Move the creation of BIO and jemalloc threads to after modules are loaded.

plus small bugfix when trying to disable the jemalloc thread at runtime
 2019-10-02 15:39:44 +03:00
antirez
758b39be99 Speedup INFO server section. 2019-10-02 11:30:20 +02:00
antirez
f49f0a6f72 SDS: make sdscatfmt() faster by pre-allocating a bit. 2019-10-02 11:30:20 +02:00
Oran Agra
98426e9886 On LUA script timeout, print the script SHA to the log 
since the slowlog and other means that can help you detect the bad script
are only exposed after the script is done. it might be a good idea to at least
print the script name (sha) to the log when it timeouts.
 2019-10-02 08:40:35 +03:00
antirez
40acb4412d GEORADIUS reply: fix of the previous fix about #6417. 2019-10-01 19:18:55 +02:00
Salvatore Sanfilippo
2e2fe98f9c
Merge pull request #6270 from oranagra/modules_info 
Extend modules API to allow modules report to redis INFO
 2019-10-01 18:02:33 +02:00
antirez
3281ebb495 Fix GEORADIUS replies broken after RESP3 introduction. 
This commit fixes #6417.
 2019-10-01 10:38:56 +02:00
antirez
beebb3c013 Merge branch 'hashed-pass-opcode' into unstable 2019-10-01 10:33:04 +02:00
Oran Agra
1b4f888109 Use sdscatfmt instead of sdscatprintf in module info 
sdscatfmt is faster
 2019-09-30 21:13:13 +03:00
Oran Agra
d5c14c70b7 Merge remote-tracking branch 'antirez/unstable' into modules_info 2019-09-30 20:47:35 +03:00
antirez
a9628142fd Clarify a comment about memory total_frag field. 2019-09-30 18:37:59 +02:00
antirez
09041b9359 ACLs: change hashed passwords opcode to also remove them. 
Related to PR #6405
 2019-09-30 18:28:45 +02:00
Madelyn Olson
ea7c3fe7fd Allowed passing in of password hash and fixed config rewrite 2019-09-30 17:57:49 +02:00
Salvatore Sanfilippo
2478b47b89
Merge pull request #6304 from dbenders/fix-issue-6141 
Fix issue #6141: cluster rebalance does not support binary key
 2019-09-30 17:43:58 +02:00
antirez
6a2831c24a Fix comments aesthetics. 2019-09-30 17:17:36 +02:00
antirez
23f5cb4233 Change a bit the style of #6385. 2019-09-30 10:58:15 +02:00
Salvatore Sanfilippo
ad45d7e407
Merge pull request #6385 from filipecosta90/perf-reply-ss-error 
Improve performance of RM_ReplyWithSimpleString and RM_ReplyWi…
 2019-09-30 10:57:05 +02:00
antirez
916c4e5d86 moduleRDBLoadError(): io->ctx may be NULL. 
The correct way to access the module about a given IO context is to
deference io->type->module, since io->ctx is only populated if the user
requests an explicit context from an IO object.
 2019-09-27 18:42:38 +02:00
antirez
8b5848a4f5 Fix memory leak in moduleLoadFromQueue(). 2019-09-27 18:33:21 +02:00
antirez
b394817754 Fix memory leak in RM_UnregisterCommandFilter(). 2019-09-27 18:33:05 +02:00
antirez
0a07f8ffee TerminateModuleForkChild(): fix function prototype. 2019-09-27 12:23:07 +02:00
antirez
721d3c9e0c TerminateModuleForkChild(): move safety checks there. 
We don't want that the API could be used directly in an unsafe way,
without checking if there is an active child. Now the safety checks are
moved directly in the function performing the operations.
 2019-09-27 12:17:47 +02:00
antirez
de1f82aa33 Function renamed hasForkChild() -> hasActiveChildProcess(). 2019-09-27 12:03:09 +02:00
antirez
82845f8d04 TerminateModuleForkChild(): use wait4 for safety. 
In theory currently there is only one active child, but the API may
change or for bugs in the implementation we may have several (it was
like that for years because of a bug). Better to wait for a specific
pid and avoid consuing other pending children information.
 2019-09-27 11:59:58 +02:00
antirez
beb1356b5b Improve error message in BGSAVE. 2019-09-27 11:59:37 +02:00
antirez
ae3ef964c1 Modules fork: improve SIGUSR1 handling, fix include. 
We can't expect SIGUSR1 to have any specific value range, so let's
define an exit code that we can handle in a special way.
This also fixes an #include <wait.h> that is not standard.
 2019-09-27 11:39:45 +02:00
Salvatore Sanfilippo
6129758558
Merge branch 'unstable' into modules_fork 2019-09-27 11:24:06 +02:00
antirez
fddc4757c8 BGREWRITEAOF: improve the generic error message. 2019-09-26 16:14:21 +02:00
Oran Agra
83e87bac76 Fix lastbgsave_status, when new child signal handler get intended kill 
And add a test for that.
 2019-09-26 15:16:34 +03:00
antirez
b7b23bdfb8 INFO: more info about loaded modules. 
Related to #6024.
 2019-09-26 12:18:55 +02:00
Salvatore Sanfilippo
c1ea6175c5
Merge pull request #6024 from itamarhaber/info_modules 
Adds a "Modules" section to `INFO`
 2019-09-26 11:58:52 +02:00
Salvatore Sanfilippo
959fb5cf68
Merge pull request #6235 from oranagra/module_rdb_load_errors 
Allow modules to handle RDB loading errors.
 2019-09-26 11:52:42 +02:00
Salvatore Sanfilippo
b0a90d8fa8
Merge pull request #4980 from yossigo/rm-assert 
Add RedisModule_Assert() API call.
 2019-09-26 11:39:29 +02:00
filipecosta90
af15b285fa [fix] fixed the un-refactor bug. 2019-09-25 17:28:42 +01:00
Salvatore Sanfilippo
388bc55344
Merge pull request #6354 from jaredzhu/aarch64-devel 
Add aarch64 uc_mcontext debug dump info
 2019-09-25 18:14:24 +02:00
antirez
b3d6cb268a Modify #6401 changes to fit 80 cols. 2019-09-25 18:08:11 +02:00
Salvatore Sanfilippo
f3aaf2b4d8
Merge pull request #6401 from valentinogeron/fix-discard-during-oom 
DISCARD should not fail during OOM
 2019-09-25 18:07:25 +02:00
Salvatore Sanfilippo
f6cf08e582
Merge pull request #6402 from mieko/unstable 
Seed SipHash with 128-bit key
 2019-09-25 18:03:39 +02:00
antirez
8a531cedb0 ACL: fix ##6408, default user state affecting all the connections. 2019-09-25 17:45:05 +02:00
Mike A. Owens
0a4d2bbd9c Seed SipHash with 128-bit key 
SipHash expects a 128-bit key, and we were indeed generating 128-bits,
but restricting them to hex characters 0-9a-f, effectively giving us
only 4 bits-per-byte of key material, and 64 bits overall.

Now, we skip the hex conversion and supply 128 bits of unfiltered
random data.
 2019-09-23 19:24:09 -04:00
filipecosta90
733280d9cb [fix] un-refactor the code. [perf] replyWithStatus now makes usage of addReplyProto 2019-09-23 23:45:31 +01:00
antirez
eda703ab28 redis-cli: support for ACL style user/pass AUTH. 2019-09-23 19:57:13 +02:00
antirez
b21dd082c3 redis-cli: CSV and RAW target for more RESP3 types. 2019-09-23 19:36:06 +02:00
antirez
cc108057ff redis-cli: ability to start a session in RESP3 mode. 2019-09-23 17:55:05 +02:00
antirez
8ea185ea30 redis-cli: AUTH can now have 3 arguments as well. 2019-09-23 17:47:36 +02:00
antirez
bb7546c913 RESP3: varbatim handling for other redis-cli outputs. 2019-09-23 17:42:57 +02:00
antirez
264708427b RESP3: implementation of verbatim output with TTY target. 2019-09-23 17:39:42 +02:00
valentino
7a73b7f168 DISCARD should not fail during OOM 
discard command should not fail during OOM, otherwise client MULTI state
will not be cleared.
 2019-09-22 09:22:53 +03:00
David Carlier
5a8a005026 Adding AnonHugePages case + comments 2019-09-20 11:01:36 +01:00
antirez
2e4fa7bb48 Make config.c always_replicate_commands more uniform. 
Better if it resembles the other similar code paths.
 2019-09-20 11:46:35 +02:00
antirez
1dd36347ed Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-09-20 11:45:02 +02:00
antirez
630638dcde Remove redundant statement in config.c. 
Thanks to @guybe7 for spotting the error in the original PR I merged.
 2019-09-20 11:44:32 +02:00
Salvatore Sanfilippo
f135d6295b
Merge pull request #6367 from Dolondro/unstable 
Spelling: Typo fixes for API documentation
 2019-09-20 11:33:58 +02:00
Salvatore Sanfilippo
3976f4c88e
Merge pull request #6368 from suntw2015/unstable 
fix rdb function rdbLoadIntegerObject comment
 2019-09-20 11:33:27 +02:00
Salvatore Sanfilippo
d0050db236
Merge pull request #6389 from ocadaruma/patch-1 
Fix typos in hyperloglog.c
 2019-09-20 11:24:07 +02:00
Salvatore Sanfilippo
975eb73145
Merge pull request #6400 from mors741/unstable 
Fix compilation error
 2019-09-20 11:19:56 +02:00
antirez
0a146a8be4 Add useless break for uniformity / future protection. 2019-09-20 11:19:07 +02:00
Salvatore Sanfilippo
bd90c99f83
Merge pull request #6398 from trevor211/fixRESP3ToLuaBool 
RESP3: Fix function redisProtocolToLuaType about RESP3->Lua bools.
 2019-09-20 11:15:20 +02:00
Evgeny
a1a65d238c
Fix compilation error 2019-09-20 01:11:20 -07:00
WuYunlong
30a3644e64 RESP3: Fix function redisProtocolToLuaType about RESP3->Lua bools. 2019-09-20 08:37:23 +08:00
WuYunlong
74e3a622a2 Fix bad handling of unexpected option while loading config "lua-replicate-commands". 2019-09-20 08:14:36 +08:00
antirez
474a923160 RESP3: Use verbatim in LATENCY subcommands. 2019-09-18 18:53:22 +02:00
antirez
cb384127d1 RESP3: Use verbatim in CLIENT LIST. 2019-09-18 18:52:13 +02:00
antirez
dd2f695d7e RESP3: Use verbatim in CLUSTER subcommands. 2019-09-18 18:51:15 +02:00
antirez
1b3cb3b0de RESP3: Use verbatim in MEMORY subcommands. 2019-09-18 18:48:14 +02:00
antirez
ff9a5d231b RESP3: Use verbatim in DEBUG HTSTATS / HTSTATS-KEY. 2019-09-18 18:46:11 +02:00
antirez
aca5482fbf RESP3: Use verbatim in INFO output. 2019-09-18 18:33:13 +02:00
antirez
68519b7034 RESP3: Verbatim conversion test in the LOLWUT command. 
redis-cli is currently not able to handle it after going in RESP3 mode,
because of hiredis limitations.
 2019-09-18 18:28:51 +02:00
antirez
8ea4bdd91d RESP3: Lua double -> RESP3 conversion. 2019-09-17 19:26:46 +02:00
antirez
89f929b12a RESP3: RESP3 double -> Lua conversion. 2019-09-17 19:20:30 +02:00
antirez
19aba4ac78 RESP3: double -> human readable conversion. 2019-09-17 19:08:33 +02:00
antirez
e8e30bc402 RESP3: bool and null values in RESP -> human readable conversion. 2019-09-17 18:57:24 +02:00
Okada Haruki
c1cc5ca767
Fix typo 2019-09-17 06:18:01 +09:00
antirez
f01f0c02d1 RESP3: convert RESP3 null as Lua nil. Implement RESP3->Lua bools. 2019-09-16 18:36:16 +02:00
antirez
6931004969 RESP3: change behavior of Lua returning true/false for RESP3. 
Here we introduce a change in the way we convert values from Lua to
Redis when RESP3 is selected: this is possible without breaking the fact
we can return directly what a command returned, because there is no
Redis command in RESP2 that returns true or false to Lua, so the
conversion in the case of RESP2 is totally arbitrary. When a script is
written selecting RESP3 from Lua, it totally makes sense to change such
behavior and return RESP3 true/false when Lua true/false is returned.
 2019-09-16 18:18:17 +02:00
antirez
2cc4d0286c RESP3: implement new NULL representation parsing in Lua. 2019-09-16 17:49:47 +02:00
antirez
ca81d49006 RESP3: handle set Lua -> Redis conversion. 2019-09-16 12:19:19 +02:00
antirez
c792504fab RESP3: handle map Lua -> Redis conversion. 2019-09-16 12:15:39 +02:00
antirez
dad38c19c8 RESP3: report set/map as nested tables to Lua. 2019-09-16 11:49:42 +02:00
filipecosta90
4a30a26f8f [add] improved performance of RM_ReplyWithSimpleString and RM_ReplyWithError by making usage addReplyProto instead of addReplySds 2019-09-15 21:16:30 +01:00
David Carlier
819a661be5 Getting region date per process in Darwin 2019-09-15 14:05:00 +01:00
antirez
888efc1b36 RESP3: Lua parsing should depend on lua client, not lua caller. 
We want all the scripts to run in RESP2 mode by default. It's up to the
caller to switch to V3 using redis.setresp() if it is really needed.
This way most scripts written for past Redis versions will continue to
work with Redis >= 6 even if the client is in RESP3 mode.
 2019-09-13 19:38:39 +02:00
antirez
26b6c697d3 RESP3: Lua debugger support for printing sets and maps. 2019-09-13 19:19:10 +02:00
antirez
93c52ff5ff RESP3: implement lua.setresp(). 2019-09-13 19:01:39 +02:00
antirez
b170a01fa7 ACL: store hashed passwords in memory. 
Note that this breaks API compatibility with Redis < 6:

    CONFIG GET requirepass

Will no longer return a cleartext password as well, but the SHA256 hash
of the password set.
 2019-09-12 12:54:57 +02:00
antirez
ae5054b476 ACL: SHA256 based password hashing function implemented. 2019-09-12 12:33:22 +02:00
antirez
9d2ecf6be3 ACL: add slightly modified version of sha256.c for password hashing. 
memory.h include removed, types substituted with stdint types.
 2019-09-12 12:21:37 +02:00
antirez
f4d37173fe ACL: protect MULTI/EXEC transactions after rules change. 2019-09-11 19:42:10 +02:00
antirez
ae8cab3296 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-09-06 12:24:40 +02:00
antirez
a092f20d87 handleClientsBlockedOnKeys() refactoring. 2019-09-06 12:24:26 +02:00
suntiawnen
781f3fd8be fix rdb function rdbLoadIntegerObject comment 2019-09-06 12:01:44 +08:00
Doug Nelson
cc8f06ece9 Typo fixes in API documentation 2019-09-05 16:25:06 +01:00
Salvatore Sanfilippo
86aade9a02
Merge pull request #6364 from oranagra/fix_module_aux_when 
Fix to module aux data rdb format for backwards compatibility with old check-rdb
 2019-09-05 13:30:26 +02:00
Oran Agra
3bfcae247a Fix to module aux data rdb format for backwards compatibility with old check-rdb 
When implementing the code that saves and loads these aux fields we used rdb
format that was added for that in redis 5.0, but then we added the 'when' field
which meant that the old redis-check-rdb won't be able to skip these.
this fix adds an opcode as if that 'when' is part of the module data.
 2019-09-05 14:11:37 +03:00
antirez
89ad0ca566 Fix handleClientsBlockedOnKeys() names in comments. 2019-09-05 13:05:57 +02:00
antirez
12c56a8e75 redis-cli: always report server errors on read errors. 
Before this commit we may have not consumer buffers when a read error is
encountered. Such buffers may contain errors that are important clues
for the user: for instance a protocol error in the payload we send in
pipe mode will cause the server to abort the connection. If the user
does not get the protocol error, debugging what is happening can be a
nightmare.

This commit fixes issue #3756.
 2019-09-04 17:55:46 +02:00
antirez
f7090f43ad AOF: be future-proof and close the file pointer. 
Currently useless but we release the fake client, so better to do a full
cleanup. Thanks to @TomMD reporting this in #6353.
 2019-09-04 17:20:37 +02:00
zhudacai 00228490
66a5c1cd7e The aarch64 architecture is support normal memory unaligned accesses, 
so add the UNALIGNED_LE_CPU to the aarch64 .
 2019-09-04 12:14:25 +00:00
antirez
b2e10131c0 Rio: fix flag name, function is never used btw. 
Thanks to @tnclong for reporting the problem.
 2019-09-04 13:01:07 +02:00
antirez
06d490342f RESP3: fix cases of NULL reported instead of empty aggregate. 2019-09-02 12:50:47 +02:00
antirez
b12ccc840e Cluster: abort loading nodes data if vars arguments are unbalanced. 
See for reference PR #6337. Thanks to @git-hulk for spotting this.
 2019-09-02 11:41:20 +02:00
antirez
e57067b677 More strict checks and better comments in flushSlaveOutputBuffers(). 
Related to #6296.
 2019-08-31 14:46:24 +02:00
antirez
2303ba1441 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-08-31 14:41:28 +02:00
antirez
ca5ccc58c5 Improve comment in flushSlavesOutputBuffers(). 2019-08-31 14:40:09 +02:00
Salvatore Sanfilippo
16435e02bf
Merge pull request #6296 from soloestoy/flushSlavesOutputBuffers-bugfix 
networking: flushSlavesOutputBuffers bugfix
 2019-08-31 14:39:33 +02:00
zhudacai 00228490
d7c25806c0 src/debug.c do not support aarch64 dump utcontext, so add the context 
of aarch64.
The content comes from the definition of the sigcontext and tested on
my aarch64 server.

sigcontext defined at the linux kernel code:
arch/arm64/include/uapi/asm/sigcontext.h
 2019-08-31 07:47:11 +00:00
Oran Agra
78bbb9b58d Modlue fork is killed when the parent exists 2019-08-25 10:11:48 +03:00
Oran Agra
61853ad8de Module INFO, support default section for simple modules 2019-08-18 10:01:57 +03:00
Oran Agra
1d6e5dc4dc Module INFO, add support for dict fields, rename API to have common prefix 2019-08-18 09:41:45 +03:00
Oran Agra
1026d2caf8 fix error handling on config parsing of repl-diskless-load 2019-08-08 14:53:36 +03:00
Oran Agra
0a97149dec RM_ReplyWithCString was missing registration 2019-08-07 13:06:11 +03:00
Diego Bendersky
b27f388344 pass len to alloc in clusterManagerMigrateKeysInReply 2019-08-05 17:35:50 -03:00
antirez
0e9b5adbd3 Replication: clarify why repl_put_online_on_ack exists at all. 2019-08-05 17:38:15 +02:00
zhaozhao.zz
d3f7eb0c87 networking: flushSlavesOutputBuffers bugfix 2019-08-02 17:17:19 +08:00
antirez
583933e2d6 Fix regression causing EXEC to appear in the slow log. 
This was recently introduced with PR #6266.
 2019-07-31 19:05:20 +02:00
antirez
2a8a63af44 Make EMBSTR case of #6261 more obvious. 2019-07-31 12:03:10 +02:00
Salvatore Sanfilippo
f899c31fbc
Merge pull request #6261 from chendq8/robj 
make memory usage consistent of robj with OBJ_ENCODING_INT
 2019-07-31 11:54:11 +02:00
Salvatore Sanfilippo
fc5c2052b5
Merge pull request #6266 from madolson/dev-unstable-hide-auth-and-hello 
Hide HELLO and AUTH from slowlog and monitor
 2019-07-31 11:12:46 +02:00
antirez
e1b13b4d39 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-31 10:37:17 +02:00
antirez
b2079cb508 HyperLogLog: fix the fix of a corruption bug. 2019-07-31 10:36:23 +02:00
Salvatore Sanfilippo
f0c10d4360
Merge pull request #6277 from devnexen/fbsd_get_rss 
Updating resident memory request impl on FreeBSD.
 2019-07-30 18:24:38 +02:00
Oran Agra
40c4183196 Log message when modules prevent diskless-load 2019-07-30 16:32:58 +03:00
Oran Agra
e5187ad2ae Merge remote-tracking branch 'oss/unstable' into module_rdb_load_errors 2019-07-30 15:14:08 +03:00
Oran Agra
4339706e07 Avoid diskelss-load if modules did not declare they handle read errors 2019-07-30 15:11:57 +03:00
Salvatore Sanfilippo
f42846e8c7
Merge pull request #6264 from oranagra/modules_api_aux_rdb 
Implement module api for aux data in rdb
 2019-07-30 11:50:44 +02:00
Salvatore Sanfilippo
ce72cac0f8
Merge pull request #6269 from wubostc/patch-1 
Reduce the calling stack
 2019-07-30 11:41:48 +02:00
antirez
e4b3c8bbc3 emptyDbGeneric(): call signalFlushDb() before deleting the keys. 
This was broken since a refactoring performed recently by myself.
 2019-07-30 11:20:54 +02:00
John Sully
d659654f53 Fix HLL corruption bug 2019-07-29 18:11:52 -04:00
David Carlier
f1c6c658ac Updating resident memory request impl on FreeBSD. 2019-07-28 14:33:57 +01:00
Salvatore Sanfilippo
5591a14c0a
Merge pull request #6248 from oranagra/module-log 
RM_Log - add support for logging without a context or context without module
 2019-07-25 18:48:46 +02:00
Oran Agra
e91d9a6fff Extend modules API to allow modules report to redis INFO 
this implements #6012
 2019-07-24 12:58:15 +03:00
antirez
fde5e737dc Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-24 11:45:19 +02:00
antirez
32efd9adf8 Client side caching: config option for table fill rate. 2019-07-24 11:35:01 +02:00
wubostc
48f4cfb601
Reduce the calling stack 2019-07-24 16:22:26 +08:00
Salvatore Sanfilippo
dbee7ca007
Merge pull request #6265 from madolson/dev-unstable-remove-create-object 
Removed unnecessary creation of Redis objects
 2019-07-23 17:05:10 +02:00
antirez
c98e7717bb Client side caching: show tracking slots usage in INFO. 2019-07-23 11:02:14 +02:00
antirez
9268493e8d Client side caching: implement full slot limit function. 2019-07-23 10:57:22 +02:00
zhaozhao.zz
9012e587ca Client side caching: do not reclaim tracking table if it's empty 2019-07-23 15:25:00 +08:00
Madelyn Olson
7d21754710 Hide HELLO and AUTH from slowlog and monitor 2019-07-22 22:53:15 -07:00
Madelyn Olson
8fe173452a Removed unecessary creation of Redis objects 2019-07-22 22:52:16 -07:00
Oran Agra
3b6aeea44c Implement module api for aux data in rdb 
Other changes:
* fix memory leak in error handling of rdb loading of type OBJ_MODULE
 2019-07-22 21:15:33 +03:00
antirez
c41f94d2a3 Client side caching: split invalidation into key / slot. 2019-07-22 18:59:53 +02:00
antirez
09c06698e9 Client side caching: redis-cli ability to enable tracking. 
This is extremely useful in order to simulate an high load of requests
about different keys, and force Redis to track a lot of informations
about several clients, to simulate real world workloads.
 2019-07-22 18:45:47 +02:00
antirez
436d02dd7b Move signalFlushedDb() into a better place. 
Now that the call also invalidates client side caching slots, it is
important that after an internal flush operation we both send the
notifications to the clients and, at the same time, are able to reclaim
the memory of the tracking table. This may even fix a few edge cases
related to MULTI/EXEC + WATCH during resync, not sure, but in general
looks more correct.
 2019-07-22 12:37:47 +02:00
antirez
7d3992a80a Client side caching: reclaim the tracking table on FLUSHALL. 2019-07-22 12:33:49 +02:00
antirez
842b44dc46 Client side caching: call the invalidation functions always. 
Otherwise what happens is that the tracking table will never get garbage
collected if there are no longer clients with tracking enabled.
Now the invalidation function immediately checks if there is any table
allocated, otherwise it returns ASAP, so the overhead when the feature
is not used should be near zero.
 2019-07-22 12:29:54 +02:00
antirez
f850201c64 Client side caching: don't hash the key if not needed. 2019-07-22 12:10:51 +02:00
antirez
e00442eb13 Clinet side caching: take count of used caching slots. 2019-07-22 11:47:44 +02:00
chendianqiang
004c26ec28 make memory usage consistent of robj with OBJ_ENCODING_INT 2019-07-22 17:45:30 +08:00
Oran Agra
d7d028a7a7 Allow modules to handle RDB loading errors. 
This is especially needed in diskless loading, were a short read could have
caused redis to exit. now the module can handle the error and return to the
caller gracefully.

this fixes #5326
 2019-07-21 18:19:32 +03:00
John Sully
ff682d7902 Modules must have execute permissions to load 2019-07-19 15:28:31 -04:00
antirez
bc5cb168f5 RDB: fix MODULE_AUX loading by continuing to next opcode. 
Thanks to @JohnSully for noticing this problem.
 2019-07-19 11:12:46 +02:00
John Sully
70534846d1 Modules must have execute permissions to load 2019-07-19 01:37:34 -04:00
antirez
5f450e4928 RDB: make sure to abort on LZF encoding error. 2019-07-18 18:59:38 +02:00
antirez
bd0f06c18c RDB: handle encoding errors with rdbExitReportCorruptRDB(). 
Without such change, the diskless replicas, when loading RDB files from
the socket will not abort when a broken RDB file gets loaded. This is
potentially unsafe, because right now Redis is not able to guarantee
that encoding errors are safe from the POV of memory corruptions (for
instance the LZF library may not be safe against untrusted data?) so
better to abort when the RDB file we are going to load is corrupted.

Instead I/O errors are still returned to the caller without aborting,
so that in case of short read the diskless replica can try again.
 2019-07-18 18:51:45 +02:00
Salvatore Sanfilippo
41ed85bd26
Merge pull request #6252 from soloestoy/tracking-flushdb 
Tracking flushdb
 2019-07-18 16:41:37 +02:00
antirez
947319cadd RDB: update rdbLoadRio comment about EOF condition. 2019-07-18 12:37:55 +02:00
zhaozhao.zz
f59119aad6 Client side caching: filter clients untracking 2019-07-18 16:46:19 +08:00
antirez
42b6305964 RDB: try to make error handling code more readable. 2019-07-17 17:30:02 +02:00
antirez
6b72b04a37 Rio: when in error condition avoid doing the operation. 2019-07-17 16:46:22 +02:00
antirez
5189db3d81 RDB: modify rdbReportError() var name for clarity. 2019-07-17 16:46:22 +02:00
antirez
48d91cf4cc Rio: remember read/write error conditions. 2019-07-17 16:46:22 +02:00
Oran Agra
c56b4ddc6f prevent diskless replica from terminating on short read 
now that replica can read rdb directly from the socket, it should avoid exiting
on short read and instead try to re-sync.

this commit tries to have minimal effects on non-diskless rdb reading.
and includes a test that tries to trigger this scenario on various read cases.
 2019-07-17 16:46:22 +02:00
Oran Agra
56258c6b7d Module API for Forking 
* create module API for forking child processes.
* refactor duplicate code around creating and tracking forks by AOF and RDB.
* child processes listen to SIGUSR1 and dies exitFromChild in order to
  eliminate a valgrind warning of unhandled signal.
* note that BGSAVE error reply has changed.

valgrind error is:
  Process terminating with default action of signal 10 (SIGUSR1)
 2019-07-17 16:40:24 +03:00
zhaozhao.zz
6191ea90a1 Client side caching: implement trackingInvalidateKeysOnFlush() 2019-07-17 20:33:52 +08:00
Oran Agra
6e07fac405 RM_Log - add support for logging without a context or context without module 
for instance detached thread safe contexts, or various callbacks that don't
provide a context.
 2019-07-17 15:08:18 +03:00
antirez
241d18d954 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-17 12:26:38 +02:00
Salvatore Sanfilippo
3f1c84751a
Merge pull request #6196 from AngusP/resp3 
[RESP3, Minor] in networking.c double representation for -infiinity leaves out comma
 2019-07-12 13:08:46 +02:00
Salvatore Sanfilippo
6100a82a3a
Merge pull request #5738 from bmerry/fix-swapdb-int-overflow 
Make dbSwapDatabases take args as long
 2019-07-12 12:36:33 +02:00
antirez
54c4e7f86c Vertically compact code in aofWrite(). 2019-07-12 12:18:33 +02:00
antirez
2a99f199bc Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-10 19:48:44 +02:00
antirez
dad0e916ff Client side caching: filter clients no longer there. 2019-07-10 19:46:46 +02:00
Salvatore Sanfilippo
7567323071
Merge pull request #6198 from oranagra/skiplist-mem-usage 
missing per-skiplist overheads in MEMORY USAGE
 2019-07-10 19:43:24 +02:00
antirez
c7aaf8db4d Client side caching: implement CLIENT GETREDIR. 
This subcommand may simplify the writing of Redis client libraries
using the tracking feature and/or improve observability and debugging
capabilities.
 2019-07-10 18:17:10 +02:00
antirez
923e4fb312 Client side caching: add tracking clients in INFO. 2019-07-10 18:08:31 +02:00
antirez
7c9f6ebc8f Diskless replica: fix disklessLoadRestoreBackups() bug. 2019-07-10 12:36:26 +02:00
antirez
3bbb9a1413 Diskless replica: refactoring of DBs backups. 2019-07-10 11:42:26 +02:00
antirez
3300e98ff0 Diskless replica: fix mispelled var name. 2019-07-10 09:34:21 +02:00
antirez
dfcbeaf115 Diskless replica: a few aesthetic changes to rio.c 2019-07-08 18:39:59 +02:00
antirez
81b18fa3a0 Diskless replica: a few aesthetic changes to replication.c. 2019-07-08 18:32:47 +02:00
Oran Agra
2de544cfcc diskless replication on slave side (don't store rdb to file), plus some other related fixes 
The implementation of the diskless replication was currently diskless only on the master side.
The slave side was still storing the received rdb file to the disk before loading it back in and parsing it.

This commit adds two modes to load rdb directly from socket:
1) when-empty
2) using "swapdb"
the third mode of using diskless slave by flushdb is risky and currently not included.

other changes:
--------------
distinguish between aof configuration and state so that we can re-enable aof only when sync eventually
succeeds (and not when exiting from readSyncBulkPayload after a failed attempt)
also a CONFIG GET and INFO during rdb loading would have lied

When loading rdb from the network, don't kill the server on short read (that can be a network error)

Fix rdb check when performed on preamble AOF

tests:
run replication tests for diskless slave too
make replication test a bit more aggressive
Add test for diskless load swapdb
 2019-07-08 15:37:48 +03:00
Salvatore Sanfilippo
722446510f
Merge pull request #6116 from AngusP/scan-types 
SCAN: New Feature `SCAN cursor [TYPE type]` modifier suggested in issue #6107
 2019-07-08 12:53:34 +02:00
Angus Pearson
6eb52e200c Change typeNameCanonicalize -> getObjectTypeName, and other style changes 2019-07-08 11:04:37 +01:00
Guy Korland
2d07883cab
fix build tracking.c should be tracking.o 
thanks to @rafie
 2019-07-07 18:28:15 +03:00
Salvatore Sanfilippo
e8ae07aabf
Merge pull request #6162 from madolson/dev-unstable-config-refactor 
Refactored yesno configs so there is less duplication
 2019-07-07 17:13:11 +02:00
Salvatore Sanfilippo
677e95e2f5
Merge pull request #6161 from swilly22/redismodule-loading-ctx-flag 
Extend REDISMODULE_CTX_FLAGS to indicate if redis is  loading
 2019-07-07 17:01:29 +02:00
Salvatore Sanfilippo
a714004738
Merge pull request #6022 from itamarhaber/RedisModule_ReplyWithCString 
Adds RedisModule_ReplyWithCString
 2019-07-05 20:25:18 +02:00
antirez
6b29f2d83d Client side caching: RESP2 support. 2019-07-05 12:24:28 +02:00
antirez
f099def733 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-04 18:34:53 +02:00
Oran Agra
41c5ebcab6 missing per-skiplist overheads in MEMORY USAGE 
these had severe impact for small zsets, for instance ones with just one
element that is longer than 64 (causing it not to be ziplist encoded)
 2019-07-04 15:20:49 +03:00
antirez
3e43aed1ac Client side caching: fix invalidate message len and content. 2019-07-04 14:07:23 +02:00
antirez
46edb55de9 Client side caching: implement trackingInvalidateKey(). 2019-07-03 19:16:20 +02:00
antirez
506764b3f8 Client side caching: hook inside call() for tracking. 2019-07-03 12:42:16 +02:00
antirez
db16a861a1 Client side caching: CLIENT TRACKING subcommand. 2019-07-03 11:58:20 +02:00
Angus Pearson
593f6656c1 RESP3 double representation for -infinity is ,-inf\r\n, not -inf\r\n 2019-07-02 14:28:48 +01:00
antirez
c29f3bcf2e Client side caching: enable tracking mode. 2019-06-30 06:19:08 -04:00
antirez
45d64f229e Client side caching: fields and flags for tracking mode. 2019-06-29 20:08:41 -04:00
antirez
a28d7918d7 Client side caching: add new file and description. 2019-06-29 09:09:38 -04:00
Madelyn Olson
dd51fc5a4a Refactored yesno configs so there was less duplication 2019-06-13 12:18:25 -07:00
Angus Pearson
38cd5fd9f6 Spelling cannonical -> canonical 2019-06-13 17:49:33 +01:00
swilly22
f54d95e803 Extend REDISMODULE_CTX_FLAGS to indicate if redis is currently loading from either RDB or AOF 2019-06-12 15:37:19 +03:00
Angus Pearson
e2adea2188 Add char* typeNameCanonicalize(robj*) to remove duplicate code between SCAN and TYPE commands, 
and to keep OBJ_* enum to string canonicalization in one place.
 2019-06-10 17:41:44 +01:00
Salvatore Sanfilippo
0cabe0cfa7
Merge pull request #6149 from madolson/acl-spelling 
Fixed some spelling issues in ACL codepath including a user facing error
 2019-06-07 22:54:52 +02:00
Madelyn Olson
dd5f437871 Fixed some spelling issues in ACL codepath including user facing error 2019-06-07 13:25:22 -07:00
artix
d3ed53f11b Redis Benchmark: prevent CONFIG failure from exiting program 2019-06-05 16:34:55 +02:00
Oran Agra
09f99c2a92 make redis purge jemalloc after flush, and enable background purging thread 
jemalloc 5 doesn't immediately release memory back to the OS, instead there's a decaying
mechanism, which doesn't work when there's no traffic (no allocations).
this is most evident if there's no traffic after flushdb, the RSS will remain high.

1) enable jemalloc background purging
2) explicitly purge in flushdb
 2019-06-02 15:33:14 +03:00
Yuan Zhou
412eb67d21 aof: fix assignment for aof_fsync_offset 
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
 2019-05-29 14:48:46 +08:00
Angus Pearson
bf963253ec Implement SCAN cursor [TYPE type] modifier suggested in issue #6107. 
Add tests to check basic functionality of this optional keyword, and also tested with
a module (redisgraph). Checked quickly with valgrind, no issues.

Copies name the type name canonicalisation code from `typeCommand`, perhaps this would
be better factored out to prevent the two diverging and both needing to be edited to
add new `OBJ_*` types, but this is a little fiddly with C strings.

The [redis-doc](https://github.com/antirez/redis-doc/blob/master/commands.json) repo
will need to be updated with this new arg if accepted.

A quirk to be aware of here is that the GEO commands are backed by zsets not their own
type, so they're not distinguishable from other zsets.

Additionally, for sparse types this has the same behaviour as `MATCH` in that it may
return many empty results before giving something, even for large `COUNT`s.
 2019-05-22 16:39:04 +01:00
zhaozhao.zz
1398fac3f1 Threaded IO: use main thread to handle read work 2019-05-21 11:42:10 +08:00
zhaozhao.zz
8b33975944 Threaded IO: use main thread to handle write work 2019-05-21 11:37:13 +08:00
antirez
074d24df1e Narrow the effects of PR #6029 to the exact state. 
CLIENT PAUSE may be used, in other contexts, for a long time making all
the slaves time out. Better for now to be more specific about what
should disable senidng PINGs.

An alternative to that would be to virtually refresh the slave
interactions when clients are paused, however for now I went for this
more conservative solution.
 2019-05-15 12:16:43 +02:00
Salvatore Sanfilippo
caf74e507e
Merge pull request #6029 from chendq8/clientpause 
fix cluster failover time out
 2019-05-15 12:03:19 +02:00
antirez
9724ca4e22 Make comment in getClientOutputBufferMemoryUsage() describing the present. 2019-05-13 17:30:06 +02:00
Salvatore Sanfilippo
e633254ccf
Merge pull request #6053 from soloestoy/enhance-aof-fsync-everysec 
aof: enhance AOF_FSYNC_EVERYSEC, more details in #5985
 2019-05-10 18:06:40 +02:00
Salvatore Sanfilippo
34f6d94701
Merge pull request #6074 from trevor211/fixActiveExpire 
Do not active expire keys in the background when the switch is off.
 2019-05-10 12:39:35 +02:00
Salvatore Sanfilippo
1e9b8d853d
Merge pull request #6080 from soloestoy/fix-makefile 
Makefile: 1TD -> STD
 2019-05-10 12:37:07 +02:00
Salvatore Sanfilippo
2903c56bda
Merge pull request #6073 from yossigo/fix/blocked-client-id 
Preserve client->id for blocked clients.
 2019-05-10 12:32:47 +02:00
Salvatore Sanfilippo
6b490e33c4
Merge pull request #6081 from Happy4Code/mod_comments 
change the comments there may have a mis type
 2019-05-10 12:28:12 +02:00
Salvatore Sanfilippo
891f4a38f6
Merge pull request #6077 from zltl/patch-7 
delete sdsTest() from REDIS_TEST in server.c to fix build failed
 2019-05-10 12:27:47 +02:00
Salvatore Sanfilippo
433f763814
Merge pull request #6083 from tangquanawj/fix-inappropriate-add-operation-code 
fix-inappropriate-add-operation-code
 2019-05-10 12:25:18 +02:00
Salvatore Sanfilippo
402b0d0bd1
Merge pull request #6086 from yongman/fix-uint64-in-defrag 
Fix uint64_t hash value in active defrag
 2019-05-10 12:23:59 +02:00
Angus Pearson
a1fb0be1d7 Enlarge error buffer in redis-check-aof.c to remove compiler warning of output truncation through snprintf format string 2019-05-08 12:13:45 +01:00
yongman
fb4ee7f0c5 Fix uint64_t hash value in active defrag 2019-05-08 16:29:16 +08:00
唐权
842dd85b26
Update ziplist.c 
Hi, @antirez

In the code, to get the size of ziplist, "unsigned int bytes = ZIPLIST_HEADER_SIZE+1;" is correct, 
but why not make it more readable and easy to understand
 2019-05-08 12:53:56 +08:00
zhaozhao.zz
48d591a010 fix memory leak when rewrite config file 2019-05-07 15:59:16 +08:00
stan011
d9d3d3065b change the comments there may have a mis type 2019-05-07 14:22:40 +08:00
zhaozhao.zz
340a723b87 Makefile: 1TD -> STD 2019-05-07 13:35:27 +08:00
antirez
3d053dbb6d Threaded IO: handleClientsWithPendingReadsUsingThreads top comment. 2019-05-06 18:02:51 +02:00
antirez
5baeb14cf3 Threaded IO: configuration directive for turning on/off reads. 2019-05-06 18:02:51 +02:00
antirez
1c0c436757 Threaded IO: ability to disable reads from threaded path. 2019-05-06 18:02:51 +02:00
antirez
90d720e7a5 Threaded IO: put fflush() inside tio_debug conditional. 2019-05-06 18:02:51 +02:00
antirez
6ab6a97fe6 Threaded IO: parsing WIP 2: refactoring to parse from thread. 2019-05-06 18:02:51 +02:00
antirez
647a66ebba Threaded IO: parsing WIP 1: set current_client in a better scoped way. 2019-05-06 18:02:51 +02:00
antirez
463ccf8664 Threaded IO: logging should be safe in I/O threads. 
Potentially it is possible that we get interleaved writes, even if
serverLog() makes sure to write into a buffer and then use printf(), so
even this should be ok. However in general POSIX guarantees that writing
to the same file pointer object from multiple threads is safe. Anyway
currently we *reopen* the file at each call, but for the standard output
logging.

The logging functions actually also access global configuration while
performing the log (for instance in order to check the log level, the
log filename and so forth), however dunring the I/O threads execution
we cannot alter such shared state in any way.
 2019-05-06 18:02:51 +02:00
antirez
8d7d2be24f Threaded IO: process read queue before stopping threads. 2019-05-06 18:02:51 +02:00
antirez
63a0ffd36a Threaded IO: read side WIP 3. 2019-05-06 18:02:51 +02:00
antirez
a2245f8ff1 Threaded IO: read side WIP 2. 2019-05-06 18:02:51 +02:00
antirez
dd5b105c73 Threaded IO: read side WIP. 2019-05-06 18:02:51 +02:00
antirez
74591fb5bd Threaded IO: hide more debugging printfs under conditional. 2019-05-06 18:02:51 +02:00
antirez
9814b2a5f3 Threaded IO: make num of I/O threads configurable. 2019-05-06 18:02:51 +02:00
antirez
30091dc29f Threaded IO: use main thread if num of threads is 1. 2019-05-06 18:02:51 +02:00
Ubuntu
9bf7f302a7 Threaded IO: stop threads when no longer needed + C11 in Makefile. 
Now threads are stopped even when the connections drop immediately to
zero, not allowing the networking code to detect the condition and stop
the threads. serverCron() will handle that.
 2019-05-06 18:02:51 +02:00
antirez
ea35a81c42 Threaded IO: 3rd version: use the mutex only to stop the thread. 2019-05-06 18:02:51 +02:00
antirez
6f4f36c0fb Threaded IO: second attempt without signaling conditions. 2019-05-06 18:02:51 +02:00
antirez
a2dbd9bd97 Threaded IO: allow to disable debug printf. 2019-05-06 18:02:51 +02:00
antirez
f468e653b5 Threaded IO: implement handleClientsWithPendingWritesUsingThreads(). 
This is just an experiment for now, there are a couple of race
conditions, mostly harmless for the performance gain experiment that
this commit represents so far.

The general idea here is to take Redis single threaded and instead
fan-out on expansive kernel calls: write(2) in this case, but the same
concept could be easily implemented for read(2) and protcol parsing.

However just threading writes like in this commit, is enough to evaluate
if the approach is sounding.
 2019-05-06 18:02:51 +02:00
liaotonglang
b1c7e3393d delete sdsTest() from REDIS_TEST 
sdsTest() defined in sds.c dit not match the call in server.c.
remove it from REDIS_TEST, since test-sds defined in Makefile.
 2019-05-06 19:24:00 +08:00
WuYunlong
1c0913dc4e Do not active expire keys in the background when the switch is off. 2019-05-06 11:46:07 +08:00
Yossi Gottlieb
9f3679880a Preserve client->id for blocked clients. 2019-05-05 20:32:53 +03:00
Salvatore Sanfilippo
0a6090bfd8
Merge pull request #6051 from vattezhang/unstable 
fix: benchmark auth fails when server have requirepass
 2019-05-02 09:48:02 +02:00
zhaozhao.zz
bcac165fab aof: enhance AOF_FSYNC_EVERYSEC, more details in #5985 2019-04-29 14:38:28 +08:00
Salvatore Sanfilippo
843de8b786
Merge pull request #5971 from devnexen/unstable 
build fix
 2019-04-26 17:32:21 +02:00
Salvatore Sanfilippo
5ba908fa5f
Merge pull request #6002 from yongman/fix-memleak-in-bitfield 
Fix memleak in bitfieldCommand
 2019-04-26 17:31:42 +02:00
Salvatore Sanfilippo
5c5197fe4f
Merge pull request #6013 from rouzier/fix/stream_iterator_init 
Fix stream interator start and end key initialize
 2019-04-26 17:25:20 +02:00
vattezhang
4e38ced488 fix: benchmark auth fails when server have requirepass 2019-04-26 19:47:07 +08:00
vattezhang
bc36404c79 update 2019-04-26 18:50:51 +08:00
vattezhang
d3d96b041a Merge github.com:antirez/redis into unstable 2019-04-26 18:47:11 +08:00
git-hulk
fa97ef34ca FIX: core dump in redis-benchmark when the -r is the last arg 2019-04-24 20:05:35 +08:00
chendianqiang
11f2c6b115 stop ping when client pause 2019-04-17 21:20:10 +08:00
Itamar Haber
52686f4866 Adds a "Modules" section to INFO 
Fixes #6012.

As long as "INFO is broken", this should be adequate IMO. Once we rework
`INFO`, perhaps into RESP3, this implementation should be revisited.
 2019-04-16 22:16:12 +03:00
Itamar Haber
c184b32d4c Uses addReplyBulkCString 
Signed-off-by: Itamar Haber <itamar@redislabs.com>
 2019-04-16 17:38:33 +03:00
Itamar Haber
26d9d4ec2c Adds RedisModule_ReplyWithCString 
Signed-off-by: Itamar Haber <itamar@redislabs.com>
 2019-04-16 17:15:23 +03:00
James Rouzier
3ccdcbc088 Fix start and end key initialize 2019-04-11 12:19:02 -04:00
Salvatore Sanfilippo
d044e33c20
Merge pull request #5961 from yossigo/modules-tests 
Modules tests
 2019-04-11 07:56:23 -07:00
antirez
9e67691ffb Aesthetic change to #5962 to conform to Redis style. 2019-04-10 18:53:35 +02:00
Salvatore Sanfilippo
6de5d25062
Merge pull request #5962 from oranagra/module_blocked_reply 
slave corrupts replication stream when module blocked client uses large reply (or POSTPONED_ARRAY)
 2019-04-10 09:41:36 -07:00
yongman
d490752d58 Fix memleak in bitfieldCommand 2019-04-09 09:24:22 +08:00
antirez
c24e32041b ACL: Fix memory leak in ACLResetSubcommandsForCommand(). 
This commit fixes bug reported at #5998. Thanks to @tomcat1102.
 2019-04-08 18:08:37 +02:00
antirez
75648f99a5 Fix assert comparison in fetchClusterSlotsConfiguration(). 2019-04-08 17:39:26 +02:00
David Carlier
4de88828d9 build fix 2019-03-28 06:38:16 +00:00
Oran Agra
acba2fc9b4 slave corrupts replication stream when module blocked client uses large reply (or POSTPONED_ARRAY) 
when redis appends the blocked client reply list to the real client, it didn't
bother to check if it is in fact the master client. so a slave executing that
module command will send replies to the master, causing the master to send the
slave error responses, which will mess up the replication offset
(slave will advance it's replication offset, and the master does not)
 2019-03-24 14:17:37 +02:00
Yossi Gottlieb
ec0b6bd2c3 Add runtest-moduleapi with commandfilter coverage. 2019-03-24 12:03:03 +02:00
Yossi Gottlieb
822a992f91 fix: missing initialization. 2019-03-24 12:00:33 +02:00
Salvatore Sanfilippo
5e8caca036
Merge pull request #5944 from yossigo/command-filtering 
Command Filtering API
 2019-03-22 17:43:49 +01:00
Salvatore Sanfilippo
fcac342955
Merge pull request #3830 from oranagra/diskless_capa_pr 
several bugfixes to diskless replication
 2019-03-22 17:41:40 +01:00
Salvatore Sanfilippo
122f42844a
Merge pull request #5945 from dvirsky/miss_notification 
Added keyspace miss notifications support
 2019-03-22 17:41:00 +01:00
Dvir Volk
040e52c77f Renamed event name from "miss" to "keymiss" 2019-03-21 20:33:11 +02:00
Oran Agra
29b0a57695 diskless fork kept streaming RDB to a disconnected slave 2019-03-21 20:24:52 +02:00
Yossi Gottlieb
6c0a5fde3d CommandFilter API: REDISMODULE_CMDFILTER_NOSELF. 
Add a flag to automatically protect filters from being called
recursively by their own module.
 2019-03-21 19:53:12 +02:00
Yossi Gottlieb
4ea3ed896b CommandFilter API: fix UnregisterCommandFilter. 2019-03-21 19:45:41 +02:00
antirez
b3408e9a9b More sensible name for function: restartAOFAfterSYNC(). 
Related to #3829.
 2019-03-21 17:21:29 +01:00
antirez
9588fd52ac Mostly aesthetic changes to restartAOF(). 
See #3829.
 2019-03-21 17:18:24 +01:00
Salvatore Sanfilippo
544373ea66
Merge pull request #3829 from oranagra/restart_aof_pr 
bugfix to restartAOF, exit will never happen
 2019-03-21 17:15:55 +01:00
Yossi Gottlieb
e2626f69ec CommandFilter API: Add unregister option. 
A filter handle is returned and can be used to unregister a filter.  In
the future it can also be used to further configure or manipulate the
filter.

Filters are now automatically unregistered when a module unloads.
 2019-03-21 14:44:49 +02:00
Salvatore Sanfilippo
d75c36ee48
Merge pull request #5390 from oranagra/lookup_key_write 
change SORT and SPOP to use lookupKeyWrite rather than lookupKeyRead
 2019-03-21 13:39:38 +01:00
antirez
9dabbd1ab0 Alter coding style in #4696 to conform to Redis code base. 2019-03-21 12:18:55 +01:00