redict/tests/unit/moduleapi
judeng 07ed0eafa9
improve performance for scan command when matching pattern or data type (#12209)
Optimized the performance of the SCAN command in a few ways:
1. Move the key filtering (by MATCH pattern) in the scan callback,
  so as to avoid collecting them for later filtering.
2. Reduce a many memory allocations and copying (use a reference
  to the original sds, instead of creating an robj, an excessive 2 mallocs
  and one string duplication)
3. Compare TYPE filter directly (as integers), instead of inefficient string
  compare per key.
4. fixed a small bug: when scan zset and hash types, maxiterations uses
  a more accurate number to avoid wrong double maxiterations.

Changes **postponed** for a later version (8.0):
1. Prepare to move the TYPE filtering to the scan callback as well. this was
  put on hold since it has side effects that can be considered a breaking
  change, which is that we will not attempt to do lazy expire (delete) a key
  that was filtered by not matching the TYPE (changing it would mean TYPE filter
  starts behaving the same as MATCH filter already does in that respect). 
2. when the specified key TYPE filter is an unknown type, server will reply a error
  immediately instead of doing a full scan that comes back empty handed. 

Benchmark result:
For different scenarios, we obtained about 30% or more performance improvement.

Co-authored-by: Oran Agra <oran@redislabs.com>
2023-06-27 16:43:46 +03:00
..
aclcheck.tcl Module commands to have ACL categories. (#11708) 2023-03-21 10:07:11 -07:00
async_rm_call.tcl Fix use after free on blocking RM_Call. (#12342) 2023-06-25 14:12:27 +03:00
auth.tcl Fix module redact test for valgrind (#10432) 2022-03-16 08:53:57 +02:00
basics.tcl Add reply_schema to command json files (internal for now) (#10273) 2023-03-11 10:14:16 +02:00
blockedclient.tcl Modules: Unblock from within a timer coverage (#12337) 2023-06-22 23:15:16 +03:00
blockonbackground.tcl Changed latency histogram output to omit trailing 0s and periods (#10075) 2022-01-09 17:04:18 -08:00
blockonkeys.tcl Modules: Unblock from within a timer coverage (#12337) 2023-06-22 23:15:16 +03:00
cluster.tcl Attempt to solve MacOS CI issues in GH Actions (#12013) 2023-04-12 09:19:21 +03:00
cmdintrospection.tcl Add reply_schema to command json files (internal for now) (#10273) 2023-03-11 10:14:16 +02:00
commandfilter.tcl Add ability for modules to know which client is being cmd filtered (#12219) 2023-06-20 09:03:52 +03:00
datatype2.tcl Tests: don't rely on the response of MEMORY USAGE when mem_allocator is not jemalloc (#10010) 2021-12-27 21:37:21 +02:00
datatype.tcl improve performance for scan command when matching pattern or data type (#12209) 2023-06-27 16:43:46 +03:00
defrag.tcl Tests: fix new defrag test to be skipped when not supported (#8185) 2020-12-14 11:13:46 +02:00
eventloop.tcl sub-command support for ACL CAT and COMMAND LIST. redisCommand always stores fullname (#10127) 2022-01-23 10:05:06 +02:00
fork.tcl Fix race in module fork kill test (#10717) 2022-05-12 20:10:38 +03:00
getchannels.tcl Implemented module getchannels api and renamed channel keyspec (#10299) 2022-02-22 11:00:03 +02:00
getkeys.tcl Unify ACL failure error messaging. (#11160) 2022-10-16 09:01:37 +03:00
hash.tcl sub-command support for ACL CAT and COMMAND LIST. redisCommand always stores fullname (#10127) 2022-01-23 10:05:06 +02:00
hooks.tcl Add a special notification unlink available only for modules (#9406) 2022-11-30 11:56:36 +02:00
infotest.tcl Fix missing sections for INFO ALL with module (#11291) 2022-09-21 08:10:03 +03:00
infra.tcl Build TLS as a loadable module 2022-08-23 12:37:56 +03:00
keyspace_events.tcl Module API to allow writes after key space notification hooks (#11199) 2022-11-24 19:00:04 +02:00
keyspecs.tcl Unify ACL failure error messaging. (#11160) 2022-10-16 09:01:37 +03:00
list.tcl Add listpack encoding for list (#11303) 2022-11-16 20:29:46 +02:00
mallocsize.tcl Add RM_MallocSizeString, RM_MallocSizeDict (#10542) 2022-04-17 08:31:57 +03:00
misc.tcl Fix broken protocol when PUBLISH emits local push inside MULTI (#12326) 2023-06-20 20:41:41 +03:00
moduleauth.tcl Custom authentication for Modules (#11659) 2023-03-15 15:18:42 -07:00
moduleconfigs.tcl Module commands to have ACL categories. (#11708) 2023-03-21 10:07:11 -07:00
postnotifications.tcl Module API to allow writes after key space notification hooks (#11199) 2022-11-24 19:00:04 +02:00
propagate.tcl reprocess command when client is unblocked on keys (#11012) 2023-01-01 23:35:42 +02:00
publish.tcl Fix broken protocol when PUBLISH emits local push inside MULTI (#12326) 2023-06-20 20:41:41 +03:00
rdbloadsave.tcl Add RM_RdbLoad and RM_RdbSave module API functions (#11852) 2023-04-09 12:07:32 +03:00
reply.tcl Align RM_ReplyWithErrorFormat with RM_ReplyWithError (#12321) 2023-06-20 20:44:43 +03:00
scan.tcl Fix zuiFind crash / RM_ScanKey hang on SET object listpack encoding (#11581) 2022-12-09 17:08:01 +02:00
stream.tcl Delete empty key if fails after moduleCreateEmptyKey() in module (#12129) 2023-05-07 10:13:19 +03:00
subcommands.tcl Fix regression not aborting transaction on error, and re-edit some error responses (#10612) 2022-04-25 13:08:13 +03:00
test_lazyfree.tcl Sort out mess around propagation and MULTI/EXEC (#9890) 2021-12-23 00:03:48 +02:00
testrdb.tcl Tests: Do not save an RDB by default and add a SIGTERM default AOFRW test (#12064) 2023-04-18 16:14:26 +03:00
timer.tcl forbid module to unload when it holds ongoing timer (#10187) 2022-02-01 14:54:11 +02:00
usercall.tcl Cleanup around script_caller, fix tracking of scripts and ACL logging for RM_Call (#11770) 2023-02-16 08:07:35 +02:00
zset.tcl Delete empty key if fails after moduleCreateEmptyKey() in module (#12129) 2023-05-07 10:13:19 +03:00