Skip to content

Commit

Permalink
Merge branch 'master' into retention-leases-version
Browse files Browse the repository at this point in the history
* master: (36 commits)
  Ensure joda compatibility in custom date formats (elastic#38171)
  Do not compute cardinality if the `terms` execution mode does not use `global_ordinals` (elastic#38169)
  Do not set timeout for IndexRequests in GatewayIndexStateIT (elastic#38147)
  Zen2ify testMasterFailoverDuringIndexingWithMappingChanges (elastic#38178)
  SQL: [Docs] Add limitation for aggregate functions on scalars (elastic#38186)
  Add elasticsearch-node detach-cluster command (elastic#37979)
  Add tests for fractional epoch parsing (elastic#38162)
  Enable bw tests for elastic#37871 and elastic#38032. (elastic#38167)
  Clear send behavior rule in CloseWhileRelocatingShardsIT (elastic#38159)
  Fix testCorruptedIndex (elastic#38161)
  Add finalReduce flag to SearchRequest (elastic#38104)
  Forbid negative field boosts in analyzed queries (elastic#37930)
  Remove AtomiFieldData#getLegacyFieldValues (elastic#38087)
  Universal cluster bootstrap method for tests with autoMinMasterNodes=false (elastic#38038)
  Fix FullClusterRestartIT.testHistoryUUIDIsAdded (elastic#38098)
  Replace joda time in ingest-common module (elastic#38088)
  Fix eclipse config for ssl-config (elastic#38096)
  Don't load global ordinals with the `map` execution_hint (elastic#37833)
  Relax fault detector in some disruption tests (elastic#38101)
  Fix java time epoch date formatters (elastic#37829)
  ...
  • Loading branch information
jasontedor committed Feb 1, 2019
2 parents 73fb8f5 + 35ed137 commit 7cf145c
Show file tree
Hide file tree
Showing 125 changed files with 4,298 additions and 1,817 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,8 @@
import org.elasticsearch.search.aggregations.bucket.filter.ParsedFilters;
import org.elasticsearch.search.aggregations.bucket.geogrid.GeoHashGridAggregationBuilder;
import org.elasticsearch.search.aggregations.bucket.geogrid.ParsedGeoHashGrid;
import org.elasticsearch.search.aggregations.bucket.geogrid.ParsedGeoTileGrid;
import org.elasticsearch.search.aggregations.bucket.geogrid.GeoTileGridAggregationBuilder;
import org.elasticsearch.search.aggregations.bucket.global.GlobalAggregationBuilder;
import org.elasticsearch.search.aggregations.bucket.global.ParsedGlobal;
import org.elasticsearch.search.aggregations.bucket.histogram.AutoDateHistogramAggregationBuilder;
Expand Down Expand Up @@ -1760,6 +1762,7 @@ static List<NamedXContentRegistry.Entry> getDefaultNamedXContents() {
map.put(FilterAggregationBuilder.NAME, (p, c) -> ParsedFilter.fromXContent(p, (String) c));
map.put(InternalSampler.PARSER_NAME, (p, c) -> ParsedSampler.fromXContent(p, (String) c));
map.put(GeoHashGridAggregationBuilder.NAME, (p, c) -> ParsedGeoHashGrid.fromXContent(p, (String) c));
map.put(GeoTileGridAggregationBuilder.NAME, (p, c) -> ParsedGeoTileGrid.fromXContent(p, (String) c));
map.put(RangeAggregationBuilder.NAME, (p, c) -> ParsedRange.fromXContent(p, (String) c));
map.put(DateRangeAggregationBuilder.NAME, (p, c) -> ParsedDateRange.fromXContent(p, (String) c));
map.put(GeoDistanceAggregationBuilder.NAME, (p, c) -> ParsedGeoDistance.fromXContent(p, (String) c));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,7 @@ public void testClusterHealthYellowClusterLevel() throws IOException {
assertThat(response.getIndices().size(), equalTo(0));
}

@AwaitsFix(bugUrl="https://github.com/elastic/elasticsearch/issues/35450")
public void testClusterHealthYellowIndicesLevel() throws IOException {
createIndex("index", Settings.EMPTY);
createIndex("index2", Settings.EMPTY);
Expand Down
185 changes: 185 additions & 0 deletions docs/reference/aggregations/bucket/geotilegrid-aggregation.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
[[search-aggregations-bucket-geotilegrid-aggregation]]
=== GeoTile Grid Aggregation

A multi-bucket aggregation that works on `geo_point` fields and groups points into
buckets that represent cells in a grid. The resulting grid can be sparse and only
contains cells that have matching data. Each cell corresponds to a
https://en.wikipedia.org/wiki/Tiled_web_map[map tile] as used by many online map
sites. Each cell is labeled using a "{zoom}/{x}/{y}" format, where zoom is equal
to the user-specified precision.

* High precision keys have a larger range for x and y, and represent tiles that
cover only a small area.
* Low precision keys have a smaller range for x and y, and represent tiles that
each cover a large area.

See https://wiki.openstreetmap.org/wiki/Zoom_levels[Zoom level documentation]
on how precision (zoom) correlates to size on the ground. Precision for this
aggregation can be between 0 and 29, inclusive.

WARNING: The highest-precision geotile of length 29 produces cells that cover
less than a 10cm by 10cm of land and so high-precision requests can be very
costly in terms of RAM and result sizes. Please see the example below on how
to first filter the aggregation to a smaller geographic area before requesting
high-levels of detail.

The specified field must be of type `geo_point` (which can only be set
explicitly in the mappings) and it can also hold an array of `geo_point`
fields, in which case all points will be taken into account during aggregation.


==== Simple low-precision request

[source,js]
--------------------------------------------------
PUT /museums
{
"mappings": {
"properties": {
"location": {
"type": "geo_point"
}
}
}
}
POST /museums/_bulk?refresh
{"index":{"_id":1}}
{"location": "52.374081,4.912350", "name": "NEMO Science Museum"}
{"index":{"_id":2}}
{"location": "52.369219,4.901618", "name": "Museum Het Rembrandthuis"}
{"index":{"_id":3}}
{"location": "52.371667,4.914722", "name": "Nederlands Scheepvaartmuseum"}
{"index":{"_id":4}}
{"location": "51.222900,4.405200", "name": "Letterenhuis"}
{"index":{"_id":5}}
{"location": "48.861111,2.336389", "name": "Musée du Louvre"}
{"index":{"_id":6}}
{"location": "48.860000,2.327000", "name": "Musée d'Orsay"}
POST /museums/_search?size=0
{
"aggregations" : {
"large-grid" : {
"geotile_grid" : {
"field" : "location",
"precision" : 8
}
}
}
}
--------------------------------------------------
// CONSOLE

Response:

[source,js]
--------------------------------------------------
{
...
"aggregations": {
"large-grid": {
"buckets": [
{
"key" : "8/131/84",
"doc_count" : 3
},
{
"key" : "8/129/88",
"doc_count" : 2
},
{
"key" : "8/131/85",
"doc_count" : 1
}
]
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\./"took": $body.took,"_shards": $body._shards,"hits":$body.hits,"timed_out":false,/]

==== High-precision requests

When requesting detailed buckets (typically for displaying a "zoomed in" map)
a filter like <<query-dsl-geo-bounding-box-query,geo_bounding_box>> should be
applied to narrow the subject area otherwise potentially millions of buckets
will be created and returned.

[source,js]
--------------------------------------------------
POST /museums/_search?size=0
{
"aggregations" : {
"zoomed-in" : {
"filter" : {
"geo_bounding_box" : {
"location" : {
"top_left" : "52.4, 4.9",
"bottom_right" : "52.3, 5.0"
}
}
},
"aggregations":{
"zoom1":{
"geotile_grid" : {
"field": "location",
"precision": 22
}
}
}
}
}
}
--------------------------------------------------
// CONSOLE
// TEST[continued]

[source,js]
--------------------------------------------------
{
...
"aggregations" : {
"zoomed-in" : {
"doc_count" : 3,
"zoom1" : {
"buckets" : [
{
"key" : "22/2154412/1378379",
"doc_count" : 1
},
{
"key" : "22/2154385/1378332",
"doc_count" : 1
},
{
"key" : "22/2154259/1378425",
"doc_count" : 1
}
]
}
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\./"took": $body.took,"_shards": $body._shards,"hits":$body.hits,"timed_out":false,/]


==== Options

[horizontal]
field:: Mandatory. The name of the field indexed with GeoPoints.

precision:: Optional. The integer zoom of the key used to define
cells/buckets in the results. Defaults to 7.
Values outside of [0,29] will be rejected.

size:: Optional. The maximum number of geohash buckets to return
(defaults to 10,000). When results are trimmed, buckets are
prioritised based on the volumes of documents they contain.

shard_size:: Optional. To allow for more accurate counting of the top cells
returned in the final result the aggregation defaults to
returning `max(10,(size x number-of-shards))` buckets from each
shard. If this heuristic is undesirable, the number considered
from each shard can be over-ridden using this parameter.
4 changes: 3 additions & 1 deletion docs/reference/ingest/processors/user-agent.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,9 @@ Which returns
"version": "10.10.5",
"full": "Mac OS X 10.10.5"
},
"device": "Other"
"device" : {
"name" : "Other"
},
}
}
}
Expand Down
Loading

0 comments on commit 7cf145c

Please sign in to comment.