Commit Graph

5225 Commits

Author SHA1 Message Date
antirez
d308cadc8a Geo: added my copyright notice in modified files. 2015-06-29 16:34:02 +02:00
antirez
69c5b27273 Geo: support units only in abbreviated form.
I'm not a strong believer in multiple syntax for the same stuff, so
now units can be specified only as m, km, ft, mi.
2015-06-29 16:02:33 +02:00
antirez
083acbebc8 Geo: remove static declarations.
Stack traces produced by Redis on crash are the most useful tool we
have to fix non easily reproducible crashes, or even easily reproducible
ones where the user just posts a bug report and does not collaborate
furhter.

By declaring functions "static" they no longer show up in the stack
trace.
2015-06-29 15:57:17 +02:00
antirez
f108c687ad Geo: GEODIST and tests. 2015-06-29 12:44:34 +02:00
antirez
a12192f5ff Geo: command function names converted to lowercase, as elsewhere.
In Redis MULTIWORDCOMMANDNAME are mapped to functions where the command
name is all lowercase: multiwordcommandnameCommand().
2015-06-29 12:07:18 +02:00
antirez
aae0a1f9cc Geo: GEOPOS command and tests. 2015-06-29 10:47:07 +02:00
antirez
f6edd0cb93 Geo: GEORADIUS COUNT tests. 2015-06-29 09:52:23 +02:00
antirez
7cd2a4e196 Geo: GEOENCODE test fixed for new return value. 2015-06-29 09:46:58 +02:00
antirez
ddc7b85c5f Geo: GEOENCODE: fix command arity check. 2015-06-29 09:39:34 +02:00
antirez
6a8e108e2d Geo: GEOENCODE now returns score ranges.
If GEOENCODE must be our door to enter the Geocoding implementation
details and do fancy things client side, than return the scores as well
so that we can query the sorted sets directly if we wish to do the same
search multiple times, or want to compute the boxes in the client side
to refine our search needs.
2015-06-29 09:34:05 +02:00
antirez
1884bff12d Geo: fix comment indentation. 2015-06-29 09:24:22 +02:00
antirez
db3df44184 Geo: debugging printf calls removed. 2015-06-29 09:21:31 +02:00
antirez
6d21027a23 Geo: GEOADD form using radius removed.
Can't immagine how this is useful in the context of the API exported by
Redis, and we are always in time to add more bloat if needed, later.
2015-06-29 09:20:07 +02:00
antirez
7d59e0a8c3 Geo: commands top comment as in other Redis code. 2015-06-29 09:16:27 +02:00
antirez
a3b07b1718 Geo: COUNT option for GEORADIUS. 2015-06-27 10:23:58 +02:00
antirez
cd91beea1c Geo: only one way to specify any given option. 2015-06-27 09:43:47 +02:00
antirez
710c05ac2a Geo: remove useless variable. geoRadiusGeneric() top comment improved. 2015-06-27 09:38:56 +02:00
antirez
fa9d62d34f Geo: from lat,lon API to lon,lat API according to GIS standard
The GIS standard and all the major DBs implementing GIS related
functions take coordinates as x,y that is longitude,latitude.
It was a bad start for Redis to do things differently, so even if this
means that existing users of the Geo module will be required to change
their code, Redis now conforms to the standard.

Usually Redis is very backward compatible, but this is not an exception
to this rule, since this is the first Geo implementation entering the
official Redis source code. It is not wise to try to be backward
compatible with code forks... :-)

Close #2637.
2015-06-26 10:58:27 +02:00
antirez
03ce189628 Geo: explain increment magic in membersOfGeoHashBox(). 2015-06-24 17:37:20 +02:00
antirez
5fd756bf13 Geo: GEOHASH command test. 2015-06-24 16:34:20 +02:00
antirez
87521f4455 Geo: GEOHASH command added, returning standard geohash strings. 2015-06-24 16:34:07 +02:00
antirez
55c4a365d7 Geo: Fix geohashEstimateStepsByRadius() step underestimation.
The returned step was in some case not enough towards normal
coordinates (for example when our search position was was already near the
margin of the central area, and we had to match, using the east or west
neighbor, a very far point). Example:

    geoadd points 67.575457940146066 -62.001317572780565 far
    geoadd points 66.685439060295664 -58.925040587282297 center
    georadius points 66.685439060295664 -58.925040587282297 200 km

In the above case the code failed to find a match (happens at smaller
latitudes too) even if far and center are at less than 200km.

Another fix introduced by this commit is a progressively larger area
towards the poles, since meridians are a lot less far away, so we need
to compensate for this.

The current implementation works comparably to the Tcl brute-force
stress tester implemented in the fuzzy test in the geo.tcl unit for
latitudes between -70 and 70, and is pretty accurate over +/-80 too,
with sporadic false negatives.

A more mathematically clean implementation is possible by computing the
meridian distance at the specified latitude and computing the step
according to it.
2015-06-24 10:42:16 +02:00
antirez
cf89a19f16 Geo: GEORADIUS fuzzy testing by reimplementing it in Tcl.
We set random points in the world, pick a random position, and check if
the returned points by Redis match the ones computed by Tcl by brute
forcing all the points using the distance between two points formula.

This approach is sounding since immediately resulted in finding a bug in
the original implementation.
2015-06-24 10:38:46 +02:00
antirez
8d5ad19d15 Geo: return REDIS_* where appropriate, improve commenting 2015-06-23 10:27:48 +02:00
antirez
0425c60381 Geo: test GEOADD with wrong input coordinates 2015-06-23 10:20:15 +02:00
antirez
bb3284563c Geo: GEOADD implementation improved, replication fixed
1. We no longer use a fake client but just rewriting.
2. We group all the inserts into a single ZADD dispatch (big speed win).
3. As a side effect of the correct implementation, replication works.
4. The return value of the command is now correct.
2015-06-23 10:20:14 +02:00
antirez
ae5fd11563 Geo: more x,y renamed lat,lon 2015-06-23 09:35:43 +02:00
antirez
a3018a215f Geo: rename x,y to lat,lon for clarity 2015-06-23 09:30:14 +02:00
antirez
51b4a4724b Geo: use the high level API to decode in geoAppendIfWithinRadius() 2015-06-23 09:03:56 +02:00
antirez
0b93139048 Geo: big refactoring of geo.c, zset.[ch] removed.
This commit simplifies the implementation in a few ways:

1. zsetScore implementation improved a bit and moved into t_zset.c where
   is now also used to implement the ZSCORE command.

2. Range extraction from the sorted set remains a separated
   implementation from the one in t_zset.c, but was hyper-specialized in
   order to avoid accumulating results into a list and remove the ones
   outside the radius.

3. A new type is introduced: geoArray, which can accumulate geoPoint
   structures in a vector with power of two expansion policy. This is
   useful since we have to call qsort() against it before returning the
   result to the user.

4. As a result of 1, 2, 3, the two files zset.c and zset.h are now
   removed, including the function to merge two lists (now handled with
   functions that can add elements to existing geoArray arrays) and
   the machinery used in order to pass zset results.

5. geoPoint structure simplified because of the general code structure
   simplification, so we no longer need to take references to objects.

6. Not counting the JSON removal the refactoring removes 200 lines of
   code for the same functionalities, with a simpler to read
   implementation.

7. GEORADIUS is now 2.5 times faster testing with 10k elements and a
   radius resulting in 124 elements returned. However this is mostly a
   side effect of the refactoring and simplification. More speed gains
   can be achieved by trying to optimize the code.
2015-06-23 08:42:57 +02:00
antirez
3d9031eda4 Geo: compile again with optimizations
For some reason the Geo PR included disabling the fact that Redis is
compiled with optimizations. Apparently it was just @mattsta attempt to
speedup the modify-compile-test iteration and there are no other
reasons.
2015-06-22 17:28:48 +02:00
antirez
9fc47ddf0b Geo: zsetScore refactoring
Now used both in geo.c and t_zset to provide ZSCORE.
2015-06-22 17:26:36 +02:00
antirez
575e247a0e Geo: fix tests after distance precision change 2015-06-22 15:00:37 +02:00
antirez
2f66550729 Geo: Pub/Sub feature removed
This feature apparently is not going to be very useful, to send a
GEOADD+PUBLISH combo is exactly the same. One that would make a ton of
difference is the ability to subscribe to a position and a radius, and
get the updates in terms of objects entering/exiting the area.
2015-06-22 14:18:18 +02:00
antirez
fc03d08ee0 Geo: addReplyDoubleDistance() precision set to 4 digits
Also:
1. The function was renamed.
2. An useless initialization of a buffer was removed.
2015-06-22 13:08:52 +02:00
antirez
b18c68aa7f Geo: JSON features removed
The command can only return data in the normal Redis protocol. It is up
to the caller to translate to JSON if needed.
2015-06-22 12:03:44 +02:00
antirez
f193b3caa8 Geo: removed bool usage from Geo code inside Redis 2015-06-22 11:24:58 +02:00
antirez
73134f6a0b Geo: removed JSON failing test (false positive)
Server output is matched to a pre-computed output. The last digits
differ because of rouding errors.
2015-06-22 11:16:36 +02:00
antirez
5e46e8a58a Geo: removed useless functions, Marcatore coordinates, bool usage 2015-06-22 11:16:15 +02:00
Matt Stancliff
7f4ac3d19c [In-Progress] Add Geo Commands
Current todo:
  - replace functions in zset.{c,h} with a new unified Redis
    zset access API.

Once we get the zset interface fixed, we can squash
relevant commits in this branch and have one nice commit
to merge into unstable.

This commit adds:
  - Geo commands
  - Tests; runnable with: ./runtest --single unit/geo
  - Geo helpers in deps/geohash-int/
  - src/geo.{c,h} and src/geojson.{c,h} implementing geo commands
  - Updated build configurations to get everything working
  - TEMPORARY: src/zset.{c,h} implementing zset score and zset
    range reading without writing to client output buffers.
  - Modified linkage of one t_zset.c function for use in zset.c

Conflicts:
	src/Makefile
	src/redis.c
2015-06-22 09:07:13 +02:00
antirez
821a986643 Sentinel: fix bug in config rewriting during failover
We have a check to rewrite the config properly when a failover is in
progress, in order to add the current (already failed over) master as
slave, and don't include in the slave list the promoted slave itself.

However there was an issue, the variable with the right address was
computed but never used when the code was modified, and no tests are
available for this feature for two reasons:

1. The Sentinel unit test currently does not test Sentinel ability to
persist its state at all.
2. It is a very hard to trigger state since it lasts for little time in
the context of the testing framework.

However this feature should be covered in the test in some way.

The bug was found by @badboy using the clang static analyzer.

Effects of the bug on safety of Sentinel
===

This bug results in severe issues in the following case:

1. A Sentinel is elected leader.
2. During the failover, it persists a wrong config with a known-slave
entry listing the master address.
3. The Sentinel crashes and restarts, reading invalid configuration from
disk.
4. It sees that the slave now does not obey the logical configuration
(should replicate from the current master), so it sends a SLAVEOF
command to the master (since the slave master is the same) creating a
replication loop (attempt to replicate from itself) which Redis is
currently unable to detect.
5. This means that the master is no longer available because of the bug.

However the lack of availability should be only transient (at least
in my tests, but other states could be possible where the problem
is not recovered automatically) because:

6. Sentinels treat masters reporting to be slaves as failing.
7. A new failover is triggered, and a slave is promoted to master.

Bug lifetime
===

The bug is there forever. Commit 16237d78 actually tried to fix the bug
but in the wrong way (the computed variable was never used! My fault).
So this bug is there basically since the start of Sentinel.

Since the bug is hard to trigger, I remember little reports matching
this condition, but I remember at least a few. Also in automated tests
where instances were stopped and restarted multiple times automatically
I remember hitting this issue, however I was not able to reproduce nor
to determine with the information I had at the time what was causing the
issue.
2015-06-12 18:36:17 +02:00
Salvatore Sanfilippo
4b5a0f0376 Merge pull request #2614 from linfangrong/patch-1
Update t_zset.c
2015-06-11 15:15:22 +02:00
antirez
8366907bed Use best effort address binding to connect to the master
We usually want to reach the master using the address of the interface
Redis is bound to (via the "bind" config option). That's useful since
the master will get (and publish) the slave address getting the peer
name of the incoming socket connection from the slave.

However, when this is not possible, for example because the slave is
bound to the loopback interface but repliaces from a master accessed via
an external interface, we want to still connect with the master even
from a different interface: in this case it is not really important that
the master will provide any other address, while it is vital to be able
to replicate correctly.

Related to issues #2609 and #2612.
2015-06-11 14:34:38 +02:00
antirez
a017b7ec0e anet.c: new API anetTcpNonBlockBestEffortBindConnect()
This performs a best effort source address binding attempt. If it is
possible to bind the local address and still have a successful
connect(), then this socket is returned. Otherwise the call is retried
without source address binding attempt.

Related to issues #2609 and #2612.
2015-06-11 14:34:38 +02:00
antirez
8fa8b251a9 anetTcpGenericConnect(), jump to error not end on error
Two code paths jumped to the "ok, return the socket to the user" code
path to handle error conditions.

Related to issues #2609 and #2612.
2015-06-11 14:34:38 +02:00
antirez
a401a84eb2 Don't try to bind the source address for MIGRATE
Related to issues #2609 and #2612.
2015-06-11 14:34:38 +02:00
Ben Murphy
ffd6637e90 hide access to debug table 2015-06-03 13:33:28 +02:00
Ben Murphy
fdf9d45509 disable loading lua bytecode 2015-06-03 13:33:28 +02:00
antirez
357a40c4fc Scripting: Lua cmsgpack lib updated to include str8 support 2015-06-03 08:44:43 +02:00
linfangrong
0dc6a5d497 Update t_zset.c 2015-06-02 18:12:57 +08:00