Percentiles aggregation on histogram-type field of an object fails

[axw]

Elasticsearch version (bin/elasticsearch --version): 7.6.0 BC3

Version: 7.6.0, Build: default/docker/d90e0399161efda5d2386d9a1376ce1a11c070c6/2020-01-29T17:39:50.161706Z, JVM: 13.0.2

Plugins installed: None

JVM version (java -version):

openjdk version "13.0.2" 2020-01-14
OpenJDK Runtime Environment AdoptOpenJDK (build 13.0.2+8)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 13.0.2+8, mixed mode, sharing)

OS version (uname -a if on a Unix-like system):

Linux f693bd1687ce 4.15.0-72-generic #81-Ubuntu SMP Tue Nov 26 12:20:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

(Running in Docker, FWIW.)

Description of the problem including expected versus actual behavior:

Using the new histogram field type, there seems to be differing behaviour when the field exists at the top level vs. as a field of another object. In particular, a percentiles aggregation works as expected on a top-level histogram field, but returns null for a histogram field within another object.

Steps to reproduce:

1. Create index mapping like so:

PUT /example
{
    "mappings": {
        "properties": {
            "foo": {
              "properties": {
                "bar": {
                  "type": "histogram"
                },
                "baz": {
                  "type": "double"
                }
              }
            },
            "qux": {
              "type": "histogram"
            }
        }
    }
}

2. Index some data:

POST /example/_doc
{
    "foo" : {
      "bar": {
        "values": [1, 2, 3, 4, 5],
        "counts": [1, 2, 3, 4, 5]
      },
      "baz": [1, 2, 2, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 5]
    },
    "qux": {
      "values": [1, 2, 3, 4, 5],
      "counts": [1, 2, 3, 4, 5]
    }
}

3. Run percentiles aggregation on each of foo.bar, foo.baz, and qux

GET /example/_search
{
  "size": 0,
  "aggs": {
    "percentiles_object_field_histogram": {
      "percentiles": {
        "field": "foo.bar",
        "percents": [1, 5, 25, 50, 75, 95, 99]
      }
    },
    "percentiles_individual_data": {
      "percentiles": {
        "field": "foo.baz",
        "percents": [1, 5, 25, 50, 75, 95, 99]
      }
    },
    "percentiles_top_level_histogram": {
      "percentiles": {
        "field": "qux",
        "percents": [1, 5, 25, 50, 75, 95, 99]
      }
    }
  }
}

Output:

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "percentiles_individual_data" : {
      "values" : {
        "1.0" : 1.0,
        "5.0" : 1.25,
        "25.0" : 3.0,
        "50.0" : 4.0,
        "75.0" : 5.0,
        "95.0" : 5.0,
        "99.0" : 5.0
      }
    },
    "percentiles_top_level_histogram" : {
      "values" : {
        "1.0" : 1.0,
        "5.0" : 1.1666666666666667,
        "25.0" : 2.7,
        "50.0" : 3.857142857142857,
        "75.0" : 4.722222222222222,
        "95.0" : 5.0,
        "99.0" : 5.0
      }
    },
    "percentiles_object_field_histogram" : {
      "values" : {
        "1.0" : null,
        "5.0" : null,
        "25.0" : null,
        "50.0" : null,
        "75.0" : null,
        "95.0" : null,
        "99.0" : null
      }
    }
  }
}

I expect them all to succeed.

[elasticmachine]

Pinging @elastic/es-analytics-geo (:Analytics/Aggregations)

[polyfractal]

/cc @iverase

@axw Were there any error messages / stack traces from the server? Or just the null values in the response?

[axw]

@polyfractal nope, nothing - just the null values.

[iverase]

Great catch! The issue is that we are generating the doc values using the simple name instead of the fully qualified name.