redict/tests/unit/moduleapi/blockonkeys.tcl

387 lines
12 KiB
Tcl
Raw Normal View History

set testmodule [file normalize tests/modules/blockonkeys.so]
start_server {tags {"modules"}} {
r module load $testmodule
test "Module client blocked on keys: Circular BPOPPUSH" {
set rd1 [redis_deferring_client]
set rd2 [redis_deferring_client]
r del src dst
$rd1 fsl.bpoppush src dst 0
$rd2 fsl.bpoppush dst src 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {2}
} else {
fail "Clients are not blocked"
}
r fsl.push src 42
assert_equal {42} [r fsl.getall src]
assert_equal {} [r fsl.getall dst]
}
test "Module client blocked on keys: Self-referential BPOPPUSH" {
set rd1 [redis_deferring_client]
r del src
$rd1 fsl.bpoppush src src 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
r fsl.push src 42
assert_equal {42} [r fsl.getall src]
}
test {Module client blocked on keys (no metadata): No block} {
r del k
r fsl.push k 33
r fsl.push k 34
r fsl.bpop k 0
} {34}
test {Module client blocked on keys (no metadata): Timeout} {
r del k
set rd [redis_deferring_client]
$rd fsl.bpop k 1
assert_equal {Request timedout} [$rd read]
}
test {Module client blocked on keys (no metadata): Blocked} {
r del k
set rd [redis_deferring_client]
$rd fsl.bpop k 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
r fsl.push k 34
assert_equal {34} [$rd read]
}
test {Module client blocked on keys (with metadata): No block} {
r del k
r fsl.push k 34
r fsl.bpopgt k 30 0
} {34}
test {Module client blocked on keys (with metadata): Timeout} {
r del k
set rd [redis_deferring_client]
$rd client id
set cid [$rd read]
r fsl.push k 33
$rd fsl.bpopgt k 35 1
assert_equal {Request timedout} [$rd read]
r client kill id $cid ;# try to smoke-out client-related memory leak
}
test {Module client blocked on keys (with metadata): Blocked, case 1} {
r del k
set rd [redis_deferring_client]
$rd client id
set cid [$rd read]
r fsl.push k 33
$rd fsl.bpopgt k 33 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
r fsl.push k 34
assert_equal {34} [$rd read]
r client kill id $cid ;# try to smoke-out client-related memory leak
}
test {Module client blocked on keys (with metadata): Blocked, case 2} {
r del k
Blocked module clients should be aware when a key is deleted (#11310) The use case is a module that wants to implement a blocking command on a key that necessarily exists and wants to unblock the client in case the key is deleted (much like what we implemented for XREADGROUP in #10306) New module API: * RedisModule_BlockClientOnKeysWithFlags Flags: * REDISMODULE_BLOCK_UNBLOCK_NONE * REDISMODULE_BLOCK_UNBLOCK_DELETED ### Detailed description of code changes blocked.c: 1. Both module and stream functions are called whether the key exists or not, regardless of its type. We do that in order to allow modules/stream to unblock the client in case the key is no longer present or has changed type (the behavior for streams didn't change, just code that moved into serveClientsBlockedOnStreamKey) 2. Make sure afterCommand is called in serveClientsBlockedOnKeyByModule, in order to propagate actions from moduleTryServeClientBlockedOnKey. 3. handleClientsBlockedOnKeys: call propagatePendingCommands directly after lookupKeyReadWithFlags to prevent a possible lazy-expire DEL from being mixed with any command propagated by the preceding functions. 4. blockForKeys: Caller can specifiy that it wants to be awakened if key is deleted. Minor optimizations (use dictAddRaw). 5. signalKeyAsReady became signalKeyAsReadyLogic which can take a boolean in case the key is deleted. It will only signal if there's at least one client that awaits key deletion (to save calls to handleClientsBlockedOnKeys). Minor optimizations (use dictAddRaw) db.c: 1. scanDatabaseForDeletedStreams is now scanDatabaseForDeletedKeys and will signalKeyAsReady for any key that was removed from the database or changed type. It is the responsibility of the code in blocked.c to ignore or act on deleted/type-changed keys. 2. Use the new signalDeletedKeyAsReady where needed blockedonkey.c + tcl: 1. Added test of new capabilities (FSL.BPOPGT now requires the key to exist in order to work)
2022-10-18 12:50:02 -04:00
r fsl.push k 32
set rd [redis_deferring_client]
$rd fsl.bpopgt k 35 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
r fsl.push k 33
r fsl.push k 34
r fsl.push k 35
r fsl.push k 36
assert_equal {36} [$rd read]
}
Blocked module clients should be aware when a key is deleted (#11310) The use case is a module that wants to implement a blocking command on a key that necessarily exists and wants to unblock the client in case the key is deleted (much like what we implemented for XREADGROUP in #10306) New module API: * RedisModule_BlockClientOnKeysWithFlags Flags: * REDISMODULE_BLOCK_UNBLOCK_NONE * REDISMODULE_BLOCK_UNBLOCK_DELETED ### Detailed description of code changes blocked.c: 1. Both module and stream functions are called whether the key exists or not, regardless of its type. We do that in order to allow modules/stream to unblock the client in case the key is no longer present or has changed type (the behavior for streams didn't change, just code that moved into serveClientsBlockedOnStreamKey) 2. Make sure afterCommand is called in serveClientsBlockedOnKeyByModule, in order to propagate actions from moduleTryServeClientBlockedOnKey. 3. handleClientsBlockedOnKeys: call propagatePendingCommands directly after lookupKeyReadWithFlags to prevent a possible lazy-expire DEL from being mixed with any command propagated by the preceding functions. 4. blockForKeys: Caller can specifiy that it wants to be awakened if key is deleted. Minor optimizations (use dictAddRaw). 5. signalKeyAsReady became signalKeyAsReadyLogic which can take a boolean in case the key is deleted. It will only signal if there's at least one client that awaits key deletion (to save calls to handleClientsBlockedOnKeys). Minor optimizations (use dictAddRaw) db.c: 1. scanDatabaseForDeletedStreams is now scanDatabaseForDeletedKeys and will signalKeyAsReady for any key that was removed from the database or changed type. It is the responsibility of the code in blocked.c to ignore or act on deleted/type-changed keys. 2. Use the new signalDeletedKeyAsReady where needed blockedonkey.c + tcl: 1. Added test of new capabilities (FSL.BPOPGT now requires the key to exist in order to work)
2022-10-18 12:50:02 -04:00
test {Module client blocked on keys (with metadata): Blocked, DEL} {
r del k
r fsl.push k 32
set rd [redis_deferring_client]
$rd fsl.bpopgt k 35 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
r del k
assert_error {*UNBLOCKED key no longer exists*} {$rd read}
}
test {Module client blocked on keys (with metadata): Blocked, FLUSHALL} {
r del k
r fsl.push k 32
set rd [redis_deferring_client]
$rd fsl.bpopgt k 35 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
r flushall
assert_error {*UNBLOCKED key no longer exists*} {$rd read}
}
test {Module client blocked on keys (with metadata): Blocked, SWAPDB, no key} {
r select 9
r del k
r fsl.push k 32
set rd [redis_deferring_client]
$rd fsl.bpopgt k 35 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
r swapdb 0 9
assert_error {*UNBLOCKED key no longer exists*} {$rd read}
}
test {Module client blocked on keys (with metadata): Blocked, SWAPDB, key exists, case 1} {
;# Key exists on other db, but wrong type
r flushall
r select 9
r fsl.push k 32
r select 0
r lpush k 38
r select 9
set rd [redis_deferring_client]
$rd fsl.bpopgt k 35 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
r swapdb 0 9
assert_error {*UNBLOCKED key no longer exists*} {$rd read}
r select 9
}
test {Module client blocked on keys (with metadata): Blocked, SWAPDB, key exists, case 2} {
;# Key exists on other db, with the right type, but the value doesn't allow to unblock
r flushall
r select 9
r fsl.push k 32
r select 0
r fsl.push k 34
r select 9
set rd [redis_deferring_client]
$rd fsl.bpopgt k 35 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
r swapdb 0 9
assert_equal {1} [s 0 blocked_clients]
r fsl.push k 38
assert_equal {38} [$rd read]
r select 9
}
test {Module client blocked on keys (with metadata): Blocked, SWAPDB, key exists, case 3} {
;# Key exists on other db, with the right type, the value allows to unblock
r flushall
r select 9
r fsl.push k 32
r select 0
r fsl.push k 38
r select 9
set rd [redis_deferring_client]
$rd fsl.bpopgt k 35 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
r swapdb 0 9
assert_equal {38} [$rd read]
r select 9
}
test {Module client blocked on keys (with metadata): Blocked, CLIENT KILL} {
r del k
Blocked module clients should be aware when a key is deleted (#11310) The use case is a module that wants to implement a blocking command on a key that necessarily exists and wants to unblock the client in case the key is deleted (much like what we implemented for XREADGROUP in #10306) New module API: * RedisModule_BlockClientOnKeysWithFlags Flags: * REDISMODULE_BLOCK_UNBLOCK_NONE * REDISMODULE_BLOCK_UNBLOCK_DELETED ### Detailed description of code changes blocked.c: 1. Both module and stream functions are called whether the key exists or not, regardless of its type. We do that in order to allow modules/stream to unblock the client in case the key is no longer present or has changed type (the behavior for streams didn't change, just code that moved into serveClientsBlockedOnStreamKey) 2. Make sure afterCommand is called in serveClientsBlockedOnKeyByModule, in order to propagate actions from moduleTryServeClientBlockedOnKey. 3. handleClientsBlockedOnKeys: call propagatePendingCommands directly after lookupKeyReadWithFlags to prevent a possible lazy-expire DEL from being mixed with any command propagated by the preceding functions. 4. blockForKeys: Caller can specifiy that it wants to be awakened if key is deleted. Minor optimizations (use dictAddRaw). 5. signalKeyAsReady became signalKeyAsReadyLogic which can take a boolean in case the key is deleted. It will only signal if there's at least one client that awaits key deletion (to save calls to handleClientsBlockedOnKeys). Minor optimizations (use dictAddRaw) db.c: 1. scanDatabaseForDeletedStreams is now scanDatabaseForDeletedKeys and will signalKeyAsReady for any key that was removed from the database or changed type. It is the responsibility of the code in blocked.c to ignore or act on deleted/type-changed keys. 2. Use the new signalDeletedKeyAsReady where needed blockedonkey.c + tcl: 1. Added test of new capabilities (FSL.BPOPGT now requires the key to exist in order to work)
2022-10-18 12:50:02 -04:00
r fsl.push k 32
set rd [redis_deferring_client]
$rd client id
set cid [$rd read]
$rd fsl.bpopgt k 35 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
r client kill id $cid ;# try to smoke-out client-related memory leak
}
test {Module client blocked on keys (with metadata): Blocked, CLIENT UNBLOCK TIMEOUT} {
r del k
Blocked module clients should be aware when a key is deleted (#11310) The use case is a module that wants to implement a blocking command on a key that necessarily exists and wants to unblock the client in case the key is deleted (much like what we implemented for XREADGROUP in #10306) New module API: * RedisModule_BlockClientOnKeysWithFlags Flags: * REDISMODULE_BLOCK_UNBLOCK_NONE * REDISMODULE_BLOCK_UNBLOCK_DELETED ### Detailed description of code changes blocked.c: 1. Both module and stream functions are called whether the key exists or not, regardless of its type. We do that in order to allow modules/stream to unblock the client in case the key is no longer present or has changed type (the behavior for streams didn't change, just code that moved into serveClientsBlockedOnStreamKey) 2. Make sure afterCommand is called in serveClientsBlockedOnKeyByModule, in order to propagate actions from moduleTryServeClientBlockedOnKey. 3. handleClientsBlockedOnKeys: call propagatePendingCommands directly after lookupKeyReadWithFlags to prevent a possible lazy-expire DEL from being mixed with any command propagated by the preceding functions. 4. blockForKeys: Caller can specifiy that it wants to be awakened if key is deleted. Minor optimizations (use dictAddRaw). 5. signalKeyAsReady became signalKeyAsReadyLogic which can take a boolean in case the key is deleted. It will only signal if there's at least one client that awaits key deletion (to save calls to handleClientsBlockedOnKeys). Minor optimizations (use dictAddRaw) db.c: 1. scanDatabaseForDeletedStreams is now scanDatabaseForDeletedKeys and will signalKeyAsReady for any key that was removed from the database or changed type. It is the responsibility of the code in blocked.c to ignore or act on deleted/type-changed keys. 2. Use the new signalDeletedKeyAsReady where needed blockedonkey.c + tcl: 1. Added test of new capabilities (FSL.BPOPGT now requires the key to exist in order to work)
2022-10-18 12:50:02 -04:00
r fsl.push k 32
set rd [redis_deferring_client]
$rd client id
set cid [$rd read]
$rd fsl.bpopgt k 35 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
r client unblock $cid timeout ;# try to smoke-out client-related memory leak
assert_equal {Request timedout} [$rd read]
}
test {Module client blocked on keys (with metadata): Blocked, CLIENT UNBLOCK ERROR} {
r del k
Blocked module clients should be aware when a key is deleted (#11310) The use case is a module that wants to implement a blocking command on a key that necessarily exists and wants to unblock the client in case the key is deleted (much like what we implemented for XREADGROUP in #10306) New module API: * RedisModule_BlockClientOnKeysWithFlags Flags: * REDISMODULE_BLOCK_UNBLOCK_NONE * REDISMODULE_BLOCK_UNBLOCK_DELETED ### Detailed description of code changes blocked.c: 1. Both module and stream functions are called whether the key exists or not, regardless of its type. We do that in order to allow modules/stream to unblock the client in case the key is no longer present or has changed type (the behavior for streams didn't change, just code that moved into serveClientsBlockedOnStreamKey) 2. Make sure afterCommand is called in serveClientsBlockedOnKeyByModule, in order to propagate actions from moduleTryServeClientBlockedOnKey. 3. handleClientsBlockedOnKeys: call propagatePendingCommands directly after lookupKeyReadWithFlags to prevent a possible lazy-expire DEL from being mixed with any command propagated by the preceding functions. 4. blockForKeys: Caller can specifiy that it wants to be awakened if key is deleted. Minor optimizations (use dictAddRaw). 5. signalKeyAsReady became signalKeyAsReadyLogic which can take a boolean in case the key is deleted. It will only signal if there's at least one client that awaits key deletion (to save calls to handleClientsBlockedOnKeys). Minor optimizations (use dictAddRaw) db.c: 1. scanDatabaseForDeletedStreams is now scanDatabaseForDeletedKeys and will signalKeyAsReady for any key that was removed from the database or changed type. It is the responsibility of the code in blocked.c to ignore or act on deleted/type-changed keys. 2. Use the new signalDeletedKeyAsReady where needed blockedonkey.c + tcl: 1. Added test of new capabilities (FSL.BPOPGT now requires the key to exist in order to work)
2022-10-18 12:50:02 -04:00
r fsl.push k 32
set rd [redis_deferring_client]
$rd client id
set cid [$rd read]
$rd fsl.bpopgt k 35 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
r client unblock $cid error ;# try to smoke-out client-related memory leak
assert_error "*unblocked*" {$rd read}
}
test {Module client blocked on keys, no timeout CB, CLIENT UNBLOCK TIMEOUT} {
r del k
set rd [redis_deferring_client]
$rd client id
set cid [$rd read]
$rd fsl.bpop k 0 NO_TO_CB
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
assert_equal [r client unblock $cid timeout] {0}
$rd close
}
test {Module client blocked on keys, no timeout CB, CLIENT UNBLOCK ERROR} {
r del k
set rd [redis_deferring_client]
$rd client id
set cid [$rd read]
$rd fsl.bpop k 0 NO_TO_CB
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
assert_equal [r client unblock $cid error] {0}
$rd close
}
test {Module client re-blocked on keys after woke up on wrong type} {
r del k
set rd [redis_deferring_client]
$rd fsl.bpop k 0
;# wait until clients are actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Clients are not blocked"
}
r lpush k 12
r lpush k 13
r lpush k 14
r del k
r fsl.push k 34
assert_equal {34} [$rd read]
assert_equal {1} [r get fsl_wrong_type] ;# first lpush caused one wrong-type wake-up
}
test {Module client blocked on keys woken up by LPUSH} {
r del k
set rd [redis_deferring_client]
$rd blockonkeys.popall k
# wait until client is actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Client is not blocked"
}
r lpush k 42 squirrel banana
assert_equal {banana squirrel 42} [$rd read]
$rd close
}
test {Module client unblocks BLPOP} {
r del k
set rd [redis_deferring_client]
$rd blpop k 3
# wait until client is actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Client is not blocked"
}
r blockonkeys.lpush k 42
assert_equal {k 42} [$rd read]
$rd close
}
test {Module unblocks module blocked on non-empty list} {
r del k
r lpush k aa
# Module client blocks to pop 5 elements from list
set rd [redis_deferring_client]
$rd blockonkeys.blpopn k 5
# Wait until client is actually blocked
wait_for_condition 50 100 {
[s 0 blocked_clients] eq {1}
} else {
fail "Client is not blocked"
}
# Check that RM_SignalKeyAsReady() can wake up BLPOPN
r blockonkeys.lpush_unblock k bb cc ;# Not enough elements for BLPOPN
r lpush k dd ee ff ;# Doesn't unblock module
r blockonkeys.lpush_unblock k gg ;# Unblocks module
assert_equal {gg ff ee dd cc} [$rd read]
$rd close
}
}