Commit Graph

6095 Commits

Author SHA1 Message Date
antirez
fdaea2a7a7 Modules: fix thread safe context creation crash.
See #6525, this likely creates a NULL deference if the client was
terminated by Redis between the creation of the blocked client and the
creation of the thread safe context.
2019-10-31 18:07:33 +01:00
antirez
1ee195d255 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-10-31 17:55:18 +01:00
antirez
dd5feec5e8 Modules: block on keys: fix stale comment. 2019-10-31 17:45:07 +01:00
antirez
825adcf3f5 Modules: block on keys: finish implementing RM_UnblockClient(). 2019-10-31 17:39:05 +01:00
antirez
37bf3e18cb Modules: block ok keys: improve example. 2019-10-31 12:31:22 +01:00
antirez
66f55bc5c1 Modules: block on keys: fix bugs in processing order. 2019-10-31 12:23:55 +01:00
antirez
629081f839 Modules: block on keys: fix the top comments. 2019-10-31 11:43:45 +01:00
antirez
91f4bdc9f9 Modules: block on keys: use a better interface.
Using the is_key_ready() callback plus the reply callback later, creates
different issues AFAIK:

1. More complex API.
2. We need to call the reply callback() ASAP if the is_key_ready()
interface returned success, however the internals do not work in that
way, so when the reply callback is called the setup could be different.
To fix that, there is to break the current design that handles the
unblocked clients asyncrhonously, and run the list ASAP.
2019-10-31 11:35:07 +01:00
antirez
4534960b29 Modules: remove spurious call from moduleHandleBlockedClients().
Now we handle propagation when we free the context.
2019-10-31 10:32:59 +01:00
antirez
228bc89ecb Modules: block on keys: example on hellotype.c. 2019-10-31 10:30:54 +01:00
antirez
215b72c0ba Modules: block on keys: implement the internals. 2019-10-30 10:57:44 +01:00
antirez
fb6110ac20 Modules: block on keys: export APIs. 2019-10-30 10:20:28 +01:00
antirez
3649568ff2 Modules: block on keys functions layout and mechanism. 2019-10-30 10:11:58 +01:00
Oran Agra
51c3ff8d75 Modules hooks: complete missing hooks for the initial set of hooks
* replication hooks: role change, master link status, replica online/offline
* persistence hooks: saving, loading, loading progress
* misc hooks: cron loop, shutdown, module loaded/unloaded
* change the way hooks test work, and add tests for all of the above

startLoading() now gets flag indicating what is loaded.
stopLoading() now gets an indication of success or failure.
adding startSaving() and stopSaving() with similar args and role.
2019-10-29 17:59:09 +02:00
Oran Agra
e978bdf9ef Module API for controlling LRU and LFU, and OpenKey without TOUCH
Some commands would want to open a key without touching it's LRU/LFU
similarly to the OBJECT or DEBUG command do.

Other commands may want to implement logic similar to what RESTORE
does (and in the future MIGRATE) and get/set the LRU or LFU.
2019-10-29 15:03:05 +02:00
Salvatore Sanfilippo
764b420f5f
Merge pull request #6487 from oranagra/module_modified_key
Module API for explicit SignalModifiedKey instead of implicit one.
2019-10-29 10:40:22 +01:00
Salvatore Sanfilippo
94973fd0eb
Merge pull request #6508 from guybe7/fix_module_notify_comp_error
Fix compilation error introduced by cee6dead3
2019-10-29 10:07:45 +01:00
Salvatore Sanfilippo
90169cdf16
Merge pull request #6495 from oranagra/rm_call_argv
Module api RM_CallArgv and test
2019-10-29 10:07:05 +01:00
Guy Benoish
43e2a398f9 Fix compilation error introduced by cee6dead3
Need to add calls to REDISMODULE_API_FUNC...
2019-10-28 17:58:07 +05:30
Oran Agra
0399b5a27e Module api tests for RM_Call
Adding a test for coverage for RM_Call in a new "misc" unit
to be used for various short simple tests

also solves compilation warnings in redismodule.h and fork.c
2019-10-28 12:39:57 +02:00
Guy Benoish
cee6dead3c Modules: Allow notifying custom keyspace events
Also, add an API for getting server.notify_keyspace_events

Other (unrelated) changes:
Add RM_GetKeynameFromModuleKey
2019-10-28 15:53:35 +05:30
antirez
cfcb475435 LOLWUT 6: always pick a default color sequence. 2019-10-28 11:10:58 +01:00
Oran Agra
a12f07792f Module API for explicit SignalModifiedKey instead of implicit one.
This commit also fixes an uninitialized module struct member (that luckily never got released)
2019-10-28 12:09:25 +02:00
antirez
8337e083b6 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-10-28 11:09:09 +01:00
Salvatore Sanfilippo
c328c807e7
Merge pull request #6496 from oranagra/module_tests_valgrind
Make module tests pass with valgrind, and fix a leak in diskless load
2019-10-28 10:05:52 +01:00
Salvatore Sanfilippo
5aab012676
Merge pull request #6498 from oranagra/rm_latency_add_sample
Module API for LatencyAddSample
2019-10-28 09:59:26 +01:00
Salvatore Sanfilippo
e097c99681
Merge pull request #4761 from WJWH/minor-typos
Fix some minor typos in comments
2019-10-28 09:58:38 +01:00
Salvatore Sanfilippo
6f2fc34228
Merge pull request #6502 from happynote3966/typo-fix
fix comment typo in redis-cli.c
2019-10-28 09:57:43 +01:00
Salvatore Sanfilippo
d070221707
Merge pull request #6505 from swilly22/replaywith-module-API-additions
Introduce ReplyWithVerbatimString, ReplyWithEmptyArray, ReplyWithNull…
2019-10-28 09:56:56 +01:00
swilly22
56a7c45521 Introduce ReplyWithVerbatimString, ReplyWithEmptyArray, ReplyWithNullArray and ReplyWithEmptyString to redis module API 2019-10-28 08:50:25 +02:00
zhaozhao.zz
c74398e130 Modules: make unloading module more safe
As we know if a module exports module-side data types,
unload it is not allowed. This rule is the same with
blocked clients in module, because we use background
threads to implement module blocked clients, and it's
not safe to unload a module if there are background
threads running. So it's necessary to check if any
blocked clients running in this module when unload it.

Moreover, after that we can ensure that if no modules,
then no module blocked clients even module unloaded.
So, we can call moduleHandleBlockedClients only when
we have installed modules.
2019-10-28 11:31:58 +08:00
happynote3966
cd02c5944f fix comment typo in redis-cli.c 2019-10-25 22:31:55 +09:00
Wander Hillen
fb1f4f4e7e
Merge branch 'unstable' into minor-typos 2019-10-25 10:18:26 +02:00
Oran Agra
e3b1d6d3ad Module API for LatencyAddSample 2019-10-24 14:24:55 +03:00
antirez
5dd1130272 Remove trailing space from server.c. 2019-10-24 11:20:15 +02:00
antirez
6e98214f74 Modules hooks: test flush event. 2019-10-24 10:51:03 +02:00
Oran Agra
be352633a0 Make module tests pass with valgrind, and fix a leak in diskless load 2019-10-24 09:45:25 +03:00
antirez
7cefbeaad5 Modules hooks: select DB zero by default. 2019-10-23 18:51:44 +02:00
antirez
dddfde85ef Modules hooks: fix memory leak in example module. 2019-10-23 18:39:53 +02:00
antirez
61ac2aef2b Modules hooks: for nested calls, create new fake clients.
We can't use the same client at the same time when re-entering the hook.
2019-10-23 18:39:53 +02:00
antirez
8cccc6b468 Modules hooks: don't call the hook for fake clients. 2019-10-23 18:39:53 +02:00
antirez
83de18805c Modules hooks: do not re-enter in hooks.
Calling a module hook callback may result in callback operations in turn
triggering other events the module is subscribed too. We don't want to
trigger those, it's unsafe and quite confusing, and to do it correcly we
would need to maintain an event list: quite a more complex
implementation.
2019-10-23 18:39:53 +02:00
antirez
89af7e4c85 Modules hooks: select the right DB for FLUSHDB events. 2019-10-23 18:39:53 +02:00
antirez
bc1ef48e56 Modules hooks: FLUSHDB event example. 2019-10-23 18:39:53 +02:00
antirez
ed833c9f99 Modules hooks: implement the FLUSHDB event. 2019-10-23 18:39:53 +02:00
antirez
8ec2fc3969 Modules hooks: unify structures definitions. 2019-10-23 18:39:53 +02:00
antirez
db8c9a8520 Modules hooks: document what yet to implement WIP 2. 2019-10-23 18:39:53 +02:00
antirez
18d1607f5e Modules hooks: document what yet to implement WIP 1. 2019-10-23 18:39:53 +02:00
antirez
c379fdbae2 Modules hooks: do more in example client callback. 2019-10-23 18:39:53 +02:00
antirez
cf52e0ad44 Module hooks: fixes and an example module. 2019-10-23 18:39:53 +02:00
antirez
94c50314f1 Modules hooks: fix sub event in calls from client change. 2019-10-23 18:39:53 +02:00
antirez
019ac37e9c Modules hooks: fix define / linker issues. Implement one test event. 2019-10-23 18:39:53 +02:00
antirez
d54652682d Modules hooks: handle module unloading + API export. 2019-10-23 18:39:53 +02:00
antirez
1e78681df8 Modules hooks: fix a leak and a few more issues. 2019-10-23 18:39:53 +02:00
antirez
b9af7e24e4 Modules hooks: event firing logic. 2019-10-23 18:39:53 +02:00
antirez
6e56f513b4 Modules: refactoring of RM_GetClientInfoById(). 2019-10-23 18:39:53 +02:00
antirez
21d8544128 Modules: fix RedisModule_GetClientInfoById() example. 2019-10-23 18:39:53 +02:00
antirez
e9b5484ddd Modules: export GetClientInfoById(). 2019-10-23 18:39:53 +02:00
antirez
6ea8641ae0 Modules: add the client ID to the client info structure. 2019-10-23 18:39:53 +02:00
antirez
d2d74ca0f3 Modules: RM_GetClientInfoById() draft. 2019-10-23 18:39:53 +02:00
antirez
bebb2e1994 Modules hooks: a first version of events and some API. 2019-10-23 18:39:53 +02:00
antirez
3170f63330 Modules hooks: initial design of data structures. 2019-10-23 18:39:53 +02:00
antirez
ed2ba31e4c Modules: fix moduleCreateArgvFromUserFormat() casting bug.
In 32 bit systems casting to "long" will cut the result to 32 bit.
2019-10-23 18:39:14 +02:00
antirez
673c9d7029 module: fix propagation API bug. 2019-10-17 18:28:34 +02:00
antirez
97f6e314c1 Modules: allow to check for AOF loading client. 2019-10-17 18:10:50 +02:00
antirez
8651e6a10d Modules: add new flags to context, replica state + more. 2019-10-17 17:49:33 +02:00
Yossi Gottlieb
8e29b0b22b Fix Makefile merge issue. 2019-10-16 17:31:02 +03:00
Yossi Gottlieb
0db3b0a0ff Merge remote-tracking branch 'upstream/unstable' into tls 2019-10-16 17:08:07 +03:00
Salvatore Sanfilippo
b8e02f2b40
Merge pull request #5926 from JimB123/unstable
Addition of RedisModule_OnUnload hook
2019-10-16 11:13:12 +02:00
Yossi Gottlieb
c469f6ad9e Code review minor changes (names, comments). 2019-10-15 17:21:51 +03:00
Yossi Gottlieb
71f10de4de Conns: write() 0 retval should not trigger error. 2019-10-15 17:20:58 +03:00
Yossi Gottlieb
0132189007 Fix compile warnings when BUILD_TLS=no. 2019-10-15 15:24:32 +03:00
Salvatore Sanfilippo
8e3d910d4e
Merge pull request #6429 from charsyam/feature/typo-slave
[trivial] fix typos salves to slaves in replication.c
2019-10-10 14:56:43 +02:00
Salvatore Sanfilippo
363604a27d
Merge pull request #6437 from OMG-By/redis-omg_by
there should is AUTH && HELLO non authenticated state.
2019-10-10 14:55:03 +02:00
Salvatore Sanfilippo
1677f4223c
Merge pull request #6443 from guybe7/fix_stream_rax_node_limits
Fix usage of server.stream_node_max_*
2019-10-10 14:49:35 +02:00
Salvatore Sanfilippo
14a9da0613
Merge pull request #6145 from oranagra/jemalloc_purge_bg
purge jemalloc after flush, and enable background purging thread
2019-10-10 14:48:59 +02:00
antirez
509fe9cba1 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-10-10 10:31:43 +02:00
antirez
747be463d2 Cluster: fix memory leak of cached master.
This is what happened:

1. Instance starts, is a slave in the cluster configuration, but
actually server.masterhost is not set, so technically the instance
is acting like a master.

2. loadDataFromDisk() calls replicationCacheMasterUsingMyself() even if
the instance is a master, in the case it is logically a slave and the
cluster is enabled. So now we have a cached master even if the instance
is practically configured as a master (from the POV of
server.masterhost value and so forth).

3. clusterCron() sees that the instance requires to replicate from its
master, because logically it is a slave, so it calls
replicationSetMaster() that will in turn call
replicationCacheMasterUsingMyself(): before this commit, this call would
overwrite the old cached master, creating a memory leak.
2019-10-10 10:23:34 +02:00
Guy Benoish
2fae0192e8 Fix usage of server.stream_node_max_* 2019-10-10 09:47:48 +02:00
omg-by
1a292e0634 there should is AUTH && HELLO non authenticated state. 2019-10-10 00:52:26 +08:00
Salvatore Sanfilippo
a92921da13
Merge pull request #6430 from jaamison/jj/unstable
stream.h: fix typo
2019-10-08 17:40:47 +02:00
Salvatore Sanfilippo
17bf0b25c1
Merge pull request #6427 from oranagra/static-analyzer
fix issues found by a static analyzer
2019-10-08 17:10:57 +02:00
antirez
009862ab7e Geo: output 10 chars of geohash, not 11.
This does not limit the actual precision, because the last digit bits were
garbage, and the shift value became even negative in the last iteration.
2019-10-08 17:09:37 +02:00
Yossi Gottlieb
d7f2681a0c TLS: Improve CA certifiate configuration options.
This adds support for explicit configuration of a CA certs directory (in
addition to the previously supported bundle file).  For redis-cli, if no
explicit CA configuration is supplied the system-wide default
configuration will be adopted.
2019-10-08 17:58:50 +03:00
Yossi Gottlieb
61733ded14 TLS: Configuration options.
Add configuration options for TLS protocol versions, ciphers/cipher
suites selection, etc.
2019-10-07 21:07:27 +03:00
Oran Agra
6b6294807c TLS: Implement support for write barrier. 2019-10-07 21:06:30 +03:00
Oran Agra
5a47794606 diskless replication rdb transfer uses pipe, and writes to sockets form the parent process.
misc:
- handle SSL_has_pending by iterating though these in beforeSleep, and setting timeout of 0 to aeProcessEvents
- fix issue with epoll signaling EPOLLHUP and EPOLLERR only to the write handlers. (needed to detect the rdb pipe was closed)
- add key-load-delay config for testing
- trim connShutdown which is no longer needed
- rioFdsetWrite -> rioFdWrite - simplified since there's no longer need to write to multiple FDs
- don't detect rdb child exited (don't call wait3) until we detect the pipe is closed
- Cleanup bad optimization from rio.c, add another one
2019-10-07 21:06:30 +03:00
Yossi Gottlieb
b087dd1db6 TLS: Connections refactoring and TLS support.
* Introduce a connection abstraction layer for all socket operations and
integrate it across the code base.
* Provide an optional TLS connections implementation based on OpenSSL.
* Pull a newer version of hiredis with TLS support.
* Tests, redis-cli updates for TLS support.
2019-10-07 21:06:13 +03:00
Jamison Judge
3ef6b79c80 stream.h: fix typo 2019-10-07 11:01:01 -07:00
antirez
c65347ab17 LOLWUT: version 6: change text message & credits. 2019-10-07 19:19:38 +02:00
antirez
c448f6fe55 LOLWUT: version 6: give credits. 2019-10-07 18:57:25 +02:00
antirez
58cdc054e3 LOLWUT: version 6: improve rithm of the image. 2019-10-07 18:41:29 +02:00
antirez
c10889150e LOLWUT: version 6 initial output. May change a bit. 2019-10-07 18:24:34 +02:00
charsyam
bea0384f5d fix type salves to slaves 2019-10-07 23:48:11 +09:00
antirez
f019f28e7b LOLWUT: version 6 initial concept. 2019-10-07 12:35:17 +02:00
Oran Agra
d1a005ab39 fix issues found by a static analyzer
cluster.c - stack buffer memory alignment
    The pointer 'buf' is cast to a more strictly aligned pointer type
evict.c - lazyfree_lazy_eviction, lazyfree_lazy_eviction always called
defrag.c - bug in dead code
server.c - casting was missing parenthesis
rax.c - indentation / newline suggested an 'else if' was intended
2019-10-07 09:09:32 +03:00
antirez
bd60447066 LOLWUT: ability to specify VERSION <ver> option. 2019-10-04 19:19:54 +02:00
antirez
9073d56eec LOLWUT: refactoring + skeleton of LOLWUT 6. 2019-10-04 19:19:48 +02:00
Oran Agra
0f1969f16f trim the double implementation of jemalloc purge 2019-10-04 14:22:13 +03:00
Oran Agra
bf759cc9c3 Merge remote-tracking branch 'antirez/unstable' into jemalloc_purge_bg 2019-10-04 13:53:40 +03:00