Commit Graph

17 Commits

Author SHA1 Message Date
Oran Agra
60a4f12f8b fix processing of large bulks (above 2GB)
- protocol parsing (processMultibulkBuffer) was limitted to 32big positions in the buffer
  readQueryFromClient potential overflow
- rioWriteBulkCount used int, although rioWriteBulkString gave it size_t
- several places in sds.c that used int for string length or index.
- bugfix in RM_SaveAuxField (return was 1 or -1 and not length)
- RM_SaveStringBuffer was limitted to 32bit length
2017-12-29 12:24:19 +02:00
antirez
8ec28002be Modules: support for modules native data types. 2016-06-03 18:14:04 +02:00
Oran Agra
5e3880a492 various cleanups and minor fixes 2016-04-25 16:49:57 +03:00
antirez
10aafdad56 Diskless replication: rio fdset target new supports buffering.
To perform a socket write() for each RDB rio API write call was
extremely unefficient, so now rio has minimal buffering capabilities.
Writes are accumulated into a buffer and only when a given limit is
reacehd are actually wrote to the N slaves FDs.

Trivia: rio lacked support for buffering since our targets were:

1) Memory buffers.
2) C standard I/O.

Both were buffered already.
2014-10-17 11:36:12 +02:00
antirez
2a436aaeab rio.c fdset target: tolerate (and report) a subset of FDs in error.
Fdset target is used when we want to write an RDB file directly to
slave's sockets. In this setup as long as there is a single slave that
is still receiving our payload, we want to continue sennding instead of
aborting. However rio calls should abort of no FD is ok.

Also we want the errors reported so that we can signal the parent who is
ok and who is broken, so there is a new set integers with the state of
each fd. Zero is ok, non-zero is the errno of the failure, if avaialble,
or a generic EIO.
2014-10-14 17:19:42 +02:00
antirez
850ea57c37 rio.c: draft implementation of fdset target implemented. 2014-10-10 17:44:06 +02:00
antirez
f590dd82ce Fixed typo in rio.h, simgle -> single. 2013-07-16 15:43:36 +02:00
yoav
63d15dfc87 Chunked loading of RDB to prevent redis from stalling reading very large keys. 2013-07-16 15:41:24 +02:00
antirez
91f4213ddf rio.c: added ability to fdatasync() from time to time while writing. 2013-04-24 10:26:30 +02:00
antirez
8419397665 Make rio.c comment 80-columns friendly. 2013-04-03 12:41:14 +02:00
antirez
4365e5b2d3 BSD license added to every C source and header file. 2012-11-08 18:31:32 +01:00
antirez
c44ab51da1 Make inline functions rioRead/Write/Tell static. This fixes issue #447. 2012-04-11 11:58:32 +02:00
antirez
8491f1d9fd Fixed compilation of new rio.c changes (typos and so forth.) 2012-04-09 12:36:44 +02:00
antirez
736b7c3f04 Add checksum computation to rio.c 2012-04-09 12:33:09 +02:00
antirez
f96a8a8054 rioInitWithFile nad rioInitWithBuffer functions now take a rio structure pointer to avoid copying a structure to return value to the caller. 2011-09-22 16:00:40 +02:00
antirez
4c0462972e comment on top of the _rio structure modified for correctness as actually fwrite/fread semantics is different in general, but was 0/1 in our old usage before rio.c as we always used 1 as number items, and the actual number of bytes to read as item length. 2011-09-22 15:47:48 +02:00
Pieter Noordhuis
2e4b0e7727 Abstract file/buffer I/O to support in-memory serialization 2011-05-13 17:31:00 +02:00