Made geo into its own part, and removed percolation

310_Geolocation/10_Intro.asciidoc

@@ -1,6 +1,8 @@
 [[geoloc]]
-== Geolocation
+= Geolocation
 
+[partintro]
+--
 Gone are the days when we wander around a city with paper maps. Thanks to
 smartphones, we now know exactly where we are all of the time, and we expect
 websites to use that information.  I'm not interested in restaurants in
@@ -29,5 +31,6 @@ other hand, are used purely for filtering.  They can be used to decide whether
 two shapes overlap or not, or whether one shape completely contains other
 shapes.
 
+--
 
 

310_Geolocation.asciidoc

@@ -1,56 +0,0 @@
-:ref:  http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/
-
-include::310_Geolocation/10_Intro.asciidoc[]
-
-include::310_Geolocation/20_Geopoints.asciidoc[]
-
-include::310_Geolocation/30_Filter_by_geopoint.asciidoc[]
-
-include::310_Geolocation/32_Bounding_box.asciidoc[]
-
-include::310_Geolocation/34_Geo_distance.asciidoc[]
-
-include::310_Geolocation/36_Caching_geofilters.asciidoc[]
-
-include::310_Geolocation/38_Reducing_memory.asciidoc[]
-
-include::310_Geolocation/40_Geohashes.asciidoc[]
-
-include::310_Geolocation/50_Sorting_by_distance.asciidoc[]
-
-include::310_Geolocation/60_Geo_aggs.asciidoc[]
-
-include::310_Geolocation/62_Geo_distance_agg.asciidoc[]
-
-include::310_Geolocation/64_Geohash_grid_agg.asciidoc[]
-
-include::310_Geolocation/66_Geo_bounds_agg.asciidoc[]
-
-include::310_Geolocation/70_Geoshapes.asciidoc[]
-
-include::310_Geolocation/72_Mapping_geo_shapes.asciidoc[]
-
-include::310_Geolocation/74_Indexing_geo_shapes.asciidoc[]
-
-include::310_Geolocation/76_Querying_geo_shapes.asciidoc[]
-
-include::310_Geolocation/78_Indexed_geo_shapes.asciidoc[]
-
-include::310_Geolocation/80_Caching_geo_shapes.asciidoc[]
-
-
-////////
-
-
-
-geo_shape:
-    mapping
-        tree
-        precision
-        type of shapes
-    indexing
-        indexed shapes
-    filters
-        geoshape
-
-////////

310_Geolocation/40_Geohashes.asciidoc

@@ -1,177 +0,0 @@
-[[geohashes]]
-=== Geohashes
-
-http://en.wikipedia.org/wiki/Geohash[Geohashes] are a way of encoding
-`lat/lon` points as strings.  The original intention was to have a
-URL-friendly way of specifying geolocations, but geohashes have turned out to
-be a useful way of indexing geo-points and geo- shapes in databases.
-
-Geohashes divide the world up into a grid of 32 cells -- 4 rows and 8 columns
-Greenland, all of Iceland and most of Great Britian. Each cell can be further
-divided into anokther 32 cells, which can be divided into another 32 cells,
-and so on.  The `gc`, cell covers Ireland and England, `gcp` covers most of
-London and part of Southern England, and `gcpuuz94k` is the entrance to
-Buckingham Palace, accurate to about 5 metres.
-
-In other words, the longer the geohash string, the more accurate it is.  If
-two geohashes share a prefix -- `gcpuux` and `gcpuuz` -- then it implies that
-they are near to each other.  The longer the shared prefix, the closer they
-are.
-
-That said, two locations that are right next to each other may have completely
-different geohashes. For instance, the
-http://en.wikipedia.org/wiki/Millennium_Dome[Millenium Dome] in London has
-geohash `u10hbp`, because it falls into the `u` cell, the next top-level cell
-to the east of the `g` cell.
-
-Geo-points can index their associated geohashes automatically, but more
-importantly, they can also index all geohash *prefixes*. Indexing the location
-of the entrance to Buckingham Palace -- latitude `51.501568` and longitude
-`-0.141257` -- would index all of the geohashes listed in the table below,
-along with  the approximate dimensions of each geohash cell:
-
-[cols="1m,1m,3d",options="header"]
-|=============================================
-|Geohash        |Level| Dimensions
-|g              |1    | ~ 5,004km x 5,004km
-|gc             |2    | ~ 1,251km x 625km
-|gcp            |3    | ~ 156km x 156km
-|gcpu           |4    | ~ 39km x 19.5km
-|gcpuu          |5    | ~ 4.9km x 4.9km
-|gcpuuz         |6    | ~ 1.2km x 0.61km
-|gcpuuz9        |7    | ~ 152.8m x 152.8m
-|gcpuuz94       |8    | ~ 38.2m x 19.1m
-|gcpuuz94k      |9    | ~ 4.78m x 4.78m
-|gcpuuz94kk     |10   | ~ 1.19m x 0.60m
-|gcpuuz94kkp    |11   | ~ 14.9cm x 14.9cm
-|gcpuuz94kkp5   |12   | ~ 3.7cm x 1.8cm
-|=============================================
-
-The {ref}query-dsl-geohash-cell-filter.html[`geohash_cell` filter] can use
-these geohash prefixes to find locations near a specified `lat/lon` point.
-
-[[geohash-mapping]]
-==== Mapping geohashes
-
-The first step is to decide just how much precision you need.  While you could
-index all geo-points with the default full 12 levels of precision, do you
-really need to be accurate to within a few centimeters? You can save yourself
-a lot of space in the index by reducing your precision requirements to
-something more realistic, such as `1km`.
-
-[source,json]
-----------------------------
-PUT /attractions
-{
-  "mappings": {
-    "restaurant": {
-      "properties": {
-        "name": {
-          "type": "string"
-        },
-        "location": {
-          "type":               "geo_point",
-          "geohash_prefix":     true, <1>
-          "geohash_precision":  "1km" <2>
-        }
-      }
-    }
-  }
-}
-----------------------------
-<1> Setting `geohash_prefix` to `true` tells Elasticsearch to index
-    all geohash prefixes, up to the specified precision.
-<2> The precision can be specified as an absolute number, representing the
-    length of the geohash, or as a distance. A precision of `1km` corresponds
-    to a geohash of length `7`.
-
-With this mapping in place, geohash prefixes of lengths 1 to 7 will be indexed,
-providing geohashes accuracate to about 150 meters.
-
-[[geohash-cell-filter]]
-==== `geohash_cell` filter
-
-The `geohash_cell` filter simply translates a `lat/lon` location into a
-geohash with the specified precision and finds all locations which contain
-that geohash -- a very efficient filter indeed.
-
-[source,json]
-----------------------------
-GET /attractions/restaurant/_search
-{
-  "query": {
-    "filtered": {
-      "filter": {
-        "geohash_cell": {
-          "location": {
-            "lat":  40.718,
-            "lon": -73.983
-          },
-          "precision": "2km" <1>
-        }
-      }
-    }
-  }
-}
-----------------------------
-<1> The `precision` cannot be more precise than that specified in the
-    `geohash_precision` mapping.
-
-This filter translates the `lat/lon` point into a geohash of the appropriate
-length -- in this example `dr5rsk` -- and looks for all locations that contain
-that exact term.
-
-However, the filter as written above may not return all restaurants within 5km
-of the specified point.  Remember that a geohash is just a rectangle, and the
-point may fall anywhere within that rectangle.  If the point happens to fall
-near the edge of a geohash cell, then the filter may well exclude any
-restaurants in the adjacent cell.
-
-To fix that, we can tell the filter to include the neigbouring cells, by
-setting `neighbors` to `true`:
-
-[source,json]
-----------------------------
-GET /attractions/restaurant/_search
-{
-  "query": {
-    "filtered": {
-      "filter": {
-        "geohash_cell": {
-          "location": {
-            "lat":  40.718,
-            "lon": -73.983
-          },
-          "neighbors": true, <1>
-          "precision": "2km"
-        }
-      }
-    }
-  }
-}
-----------------------------
-
-<1> This filter will look for the resolved geohash and all of the surrounding
-    geohashes.
-
-Clearly, looking for a geohash with precision `2km` plus all the neighbouring
-cells, results in quite a large search area.  This filter is not built for
-accuracy, but it is very efficient and can be used as a pre-filtering step
-before applying a more accurate geo-filter.
-
-TIP: Specifying the `precision` as a distance can be misleading. A `precision`
-of `2km` is converted to a geohash of length 6, which actually has dimensions
-of about 1.2km x 0.6km.  You may find it more understandable to specify an
-actual length like `5` or `6`.
-
-The other advantage that this filter has over a `geo_bounding_box` filter is
-that it supports multiple locations per field.  The `lat_lon` option that we
-discussed in <<optimize-bounding-box>> is very efficient, but only when there
-is a single `lat/lon` point per field.
-
-
-
-
-
-

310_Geopoints.asciidoc

@@ -0,0 +1,14 @@
+include::310_Geopoints/20_Geopoints.asciidoc[]
+
+include::310_Geopoints/30_Filter_by_geopoint.asciidoc[]
+
+include::310_Geopoints/32_Bounding_box.asciidoc[]
+
+include::310_Geopoints/34_Geo_distance.asciidoc[]
+
+include::310_Geopoints/36_Caching_geofilters.asciidoc[]
+
+include::310_Geopoints/38_Reducing_memory.asciidoc[]
+
+include::310_Geopoints/50_Sorting_by_distance.asciidoc[]
+

310_Geolocation/20_Geopoints.asciidoc

@@ -1,8 +1,12 @@
-[[indexing-geopoints]]
-=== Indexing geo-points
+[[geopoints]]
+== Geo-points
+
+A geo-point is a single latitude-longitude point on the Earth's surface. They
+can be used to calculate distance from a point, to determine whether a point
+falls within a bounding box, or in aggregations.
 
 Geo-points cannot be automatically detected with
-<<dynamic-mapping,dynamic mapping>>. Instead, geo-points fields should be
+<<dynamic-mapping,dynamic mapping>>. Instead, `geo_point` fields should be
 mapped explicitly:
 
 [source,json]
@@ -25,7 +29,8 @@ PUT /attractions
 -----------------------
 
 [[lat-lon-formats]]
-==== Lat/Lon formats
+[float]
+=== Lat/Lon formats
 
 With the `location` field defined as a `geo_point`, we can proceed to index
 documents containing latitude/longitude pairs, which can be formatted as

320_Geohashes.asciidoc

@@ -0,0 +1,6 @@
+include::320_Geohashes/40_Geohashes.asciidoc[]
+
+include::320_Geohashes/50_Geohash_mapping.asciidoc[]
+
+include::320_Geohashes/60_Geohash_cell_filter.asciidoc[]
+

320_Geohashes/40_Geohashes.asciidoc

@@ -0,0 +1,53 @@
+[[geohashes]]
+== Geohashes
+
+http://en.wikipedia.org/wiki/Geohash[Geohashes] are a way of encoding
+`lat/lon` points as strings.  The original intention was to have a
+URL-friendly way of specifying geolocations, but geohashes have turned out to
+be a useful way of indexing geo-points and geo- shapes in databases.
+
+Geohashes divide the world up into a grid of 32 cells -- 4 rows and 8 columns
+-- each represented by a letter or number.  The `g` cell covers half of
+Greenland, all of Iceland and most of Great Britian. Each cell can be further
+divided into anokther 32 cells, which can be divided into another 32 cells,
+and so on.  The `gc`, cell covers Ireland and England, `gcp` covers most of
+London and part of Southern England, and `gcpuuz94k` is the entrance to
+Buckingham Palace, accurate to about 5 metres.
+
+In other words, the longer the geohash string, the more accurate it is.  If
+two geohashes share a prefix -- `gcpuux` and `gcpuuz` -- then it implies that
+they are near to each other.  The longer the shared prefix, the closer they
+are.
+
+That said, two locations that are right next to each other may have completely
+different geohashes. For instance, the
+http://en.wikipedia.org/wiki/Millennium_Dome[Millenium Dome] in London has
+geohash `u10hbp`, because it falls into the `u` cell, the next top-level cell
+to the east of the `g` cell.
+
+Geo-points can index their associated geohashes automatically, but more
+importantly, they can also index all geohash *prefixes*. Indexing the location
+of the entrance to Buckingham Palace -- latitude `51.501568` and longitude
+`-0.141257` -- would index all of the geohashes listed in the table below,
+along with  the approximate dimensions of each geohash cell:
+
+[cols="1m,1m,3d",options="header"]
+|=============================================
+|Geohash        |Level| Dimensions
+|g              |1    | ~ 5,004km x 5,004km
+|gc             |2    | ~ 1,251km x 625km
+|gcp            |3    | ~ 156km x 156km
+|gcpu           |4    | ~ 39km x 19.5km
+|gcpuu          |5    | ~ 4.9km x 4.9km
+|gcpuuz         |6    | ~ 1.2km x 0.61km
+|gcpuuz9        |7    | ~ 152.8m x 152.8m
+|gcpuuz94       |8    | ~ 38.2m x 19.1m
+|gcpuuz94k      |9    | ~ 4.78m x 4.78m
+|gcpuuz94kk     |10   | ~ 1.19m x 0.60m
+|gcpuuz94kkp    |11   | ~ 14.9cm x 14.9cm
+|gcpuuz94kkp5   |12   | ~ 3.7cm x 1.8cm
+|=============================================
+
+The {ref}query-dsl-geohash-cell-filter.html[`geohash_cell` filter] can use
+these geohash prefixes to find locations near a specified `lat/lon` point.
+

320_Geohashes/50_Geohash_mapping.asciidoc

@@ -0,0 +1,38 @@
+[[geohash-mapping]]
+=== Mapping geohashes
+
+The first step is to decide just how much precision you need.  While you could
+index all geo-points with the default full 12 levels of precision, do you
+really need to be accurate to within a few centimeters? You can save yourself
+a lot of space in the index by reducing your precision requirements to
+something more realistic, such as `1km`.
+
+[source,json]
+----------------------------
+PUT /attractions
+{
+  "mappings": {
+    "restaurant": {
+      "properties": {
+        "name": {
+          "type": "string"
+        },
+        "location": {
+          "type":               "geo_point",
+          "geohash_prefix":     true, <1>
+          "geohash_precision":  "1km" <2>
+        }
+      }
+    }
+  }
+}
+----------------------------
+<1> Setting `geohash_prefix` to `true` tells Elasticsearch to index
+    all geohash prefixes, up to the specified precision.
+<2> The precision can be specified as an absolute number, representing the
+    length of the geohash, or as a distance. A precision of `1km` corresponds
+    to a geohash of length `7`.
+
+With this mapping in place, geohash prefixes of lengths 1 to 7 will be indexed,
+providing geohashes accuracate to about 150 meters.
+

320_Geohashes/60_Geohash_cell_filter.asciidoc

@@ -0,0 +1,82 @@
+[[geohash-cell-filter]]
+=== `geohash_cell` filter
+
+The `geohash_cell` filter simply translates a `lat/lon` location into a
+geohash with the specified precision and finds all locations which contain
+that geohash -- a very efficient filter indeed.
+
+[source,json]
+----------------------------
+GET /attractions/restaurant/_search
+{
+  "query": {
+    "filtered": {
+      "filter": {
+        "geohash_cell": {
+          "location": {
+            "lat":  40.718,
+            "lon": -73.983
+          },
+          "precision": "2km" <1>
+        }
+      }
+    }
+  }
+}
+----------------------------
+<1> The `precision` cannot be more precise than that specified in the
+    `geohash_precision` mapping.
+
+This filter translates the `lat/lon` point into a geohash of the appropriate
+length -- in this example `dr5rsk` -- and looks for all locations that contain
+that exact term.
+
+However, the filter as written above may not return all restaurants within 5km
+of the specified point.  Remember that a geohash is just a rectangle, and the
+point may fall anywhere within that rectangle.  If the point happens to fall
+near the edge of a geohash cell, then the filter may well exclude any
+restaurants in the adjacent cell.
+
+To fix that, we can tell the filter to include the neigbouring cells, by
+setting `neighbors` to `true`:
+
+[source,json]
+----------------------------
+GET /attractions/restaurant/_search
+{
+  "query": {
+    "filtered": {
+      "filter": {
+        "geohash_cell": {
+          "location": {
+            "lat":  40.718,
+            "lon": -73.983
+          },
+          "neighbors": true, <1>
+          "precision": "2km"
+        }
+      }
+    }
+  }
+}
+----------------------------
+
+<1> This filter will look for the resolved geohash and all of the surrounding
+    geohashes.
+
+Clearly, looking for a geohash with precision `2km` plus all the neighbouring
+cells, results in quite a large search area.  This filter is not built for
+accuracy, but it is very efficient and can be used as a pre-filtering step
+before applying a more accurate geo-filter.
+
+TIP: Specifying the `precision` as a distance can be misleading. A `precision`
+of `2km` is converted to a geohash of length 6, which actually has dimensions
+of about 1.2km x 0.6km.  You may find it more understandable to specify an
+actual length like `5` or `6`.
+
+The other advantage that this filter has over a `geo_bounding_box` filter is
+that it supports multiple locations per field.  The `lat_lon` option that we
+discussed in <<optimize-bounding-box>> is very efficient, but only when there
+is a single `lat/lon` point per field.
+
+

320_Percolation.asciidoc

@@ -1,27 +0,0 @@
-[[percolation]]
-== Percolation - Turning search upside down (TODO)
-
-Instead of asking _What documents match this query?_, sometimes we want to know
-if a particular document matches any of a list of predefined queries.  For
-instance, users may ask to be informed whenever a new article is published
-which mentions their company name.
-
-This chapter explains how to use percolation.
-
-=== What is percolation
-.
-
-
-=== Creating percolators
-.
-
-
-=== Percolate API
-.
-
-
-=== Live percolation
-.
-
-
-

330_Geo_aggs.asciidoc

@@ -0,0 +1,9 @@
+
+include::330_Geo_aggs/60_Geo_aggs.asciidoc[]
+
+include::330_Geo_aggs/62_Geo_distance_agg.asciidoc[]
+
+include::330_Geo_aggs/64_Geohash_grid_agg.asciidoc[]
+
+include::330_Geo_aggs/66_Geo_bounds_agg.asciidoc[]
+

310_Geolocation/60_Geo_aggs.asciidoc

@@ -1,5 +1,5 @@
 [[geo-aggs]]
-=== Geo-aggregations
+== Geo-aggregations
 
 While filtering or scoring results by geolocation is useful, it is often more
 useful to be able to present information to the user on a map. A search may

340_Geoshapes.asciidoc

@@ -0,0 +1,13 @@
+include::340_Geoshapes/70_Geoshapes.asciidoc[]
+
+include::340_Geoshapes/72_Mapping_geo_shapes.asciidoc[]
+
+include::340_Geoshapes/74_Indexing_geo_shapes.asciidoc[]
+
+include::340_Geoshapes/76_Querying_geo_shapes.asciidoc[]
+
+include::340_Geoshapes/78_Indexed_geo_shapes.asciidoc[]
+
+include::340_Geoshapes/80_Caching_geo_shapes.asciidoc[]
+
+

310_Geolocation/70_Geoshapes.asciidoc

@@ -1,5 +1,5 @@
 [[geo-shapes]]
-=== Geo-shapes
+== Geo-shapes
 
 Geo-shapes use a completely different approach to geo-points. A circle on a
 computer screen does not consist of a perfect continuous line. Instead it is

book.asciidoc

@@ -83,13 +83,17 @@ include::305_Significant_Terms.asciidoc[]
 
 include::306_Practical_Considerations.asciidoc[]
 
+
 // Part 5
-[[more-than-search]]
-= More than search (TODO)
+include::04_Geolocation.asciidoc[]
+
+include::310_Geopoints.asciidoc[]
+
+include::320_Geohashes.asciidoc[]
 
-include::310_Geolocation.asciidoc[]
+include::330_Geo_aggs.asciidoc[]
 
-include::320_Percolation.asciidoc[]
+include::340_Geoshapes.asciidoc[]
 
 // Part 6