Pie chart doesn't correctly calculate sub-sub-aggregation counts

[ejustis]

Kibana version:
7.2.0

Elasticsearch version:
7.2.0

Server OS version:
CentOS 7.6.1810

Browser version:
Chrome 78.0.3904.97 64-bit

Browser OS version:
MacOS 10.14.6

Original install method (e.g. download page, yum, from source, etc.):
yum

Describe the bug:
When creating pie charts that are sliced multiple times it appears the counts and percentages no longer get calculated correctly. When slicing a chart twice, as seen below, everything works as intended. The counts in the outer rings all add up to the inner ring. Once you add another slice operation to the chart, however, the sums are thrown off.

Steps to reproduce:

1. Create new Pie Chart Visualization
2. Add a slice parameter on Term count
3. Add another slice parameter on Term count
4. Add a third slice parameter on Term count

Expected behavior:
Should show the counts correctly per sub-aggregate that is added. The first slice would be the totals with the second slice showing the sub-counts of the previous slice. So if the first ring had 4000 in one slice and the second ring has 4 sections with each count of 1000. The final ring might show 2 slices with count 500 per each of the 1000 count slices in the previous ring.

Screenshots (if relevant):
2-slice Version:

4-slice Version:

Errors in browser console (if relevant):
N/A

Provide logs and/or server output (if relevant):
N/A

Any additional context:

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[timroes]

Hi,

could you please share what sizes you configured for the terms aggregation on each level?

The percentage will be calculated of the values present in the chart. This doesn’t necessarily always need to add up to 100% of the value on one level higher, since it might be that e.g. the top 3 terms are not all terms that were part of level 1. So if level 1 would have a value of 1000, and you select show top 3 in the next level, and each of them is 200, you will still see 33% for each of those outer slices even though they are not summing up to 1000, but they are each 33% of the values available in that layer.

Cheers,
Tim

[ejustis]

Hello,

In my test I updated the terms aggregation to 20 on each level. With the data that I was using this would have accounted for all possibilities. I also tried checking the add Other data and Missed data to make sure everything was accounted for. I am not sure if there is a hard limit on how many slices can occur per level, but it still didn't add up correctly.

Thanks for the help,
Elijah

[timroes]

Hi,

could you perhaps go to "Inspect" (on a dashboard in the panel menu, in visualize editor in the top menu) and copy the request and response that you'll find in the "Requests" view?

Cheers,
Tim

[ejustis]

Doesn't looks great in the "code block" let me know if you would like it in another form.

Request

{
  "aggs": {
    "3": {
      "terms": {
        "field": "error_type.keyword",
        "order": {
          "_count": "desc"
        },
        "size": 5
      },
      "aggs": {
        "2": {
          "terms": {
            "field": "location.segment.keyword",
            "order": {
              "_count": "desc"
            },
            "size": 5
          },
          "aggs": {
            "4": {
              "terms": {
                "field": "location.field",
                "order": {
                  "_count": "desc"
                },
                "size": 5
              },
              "aggs": {
                "6": {
                  "terms": {
                    "field": "location.component",
                    "order": {
                      "_count": "desc"
                    },
                    "size": 5
                  },
                  "aggs": {
                    "5": {
                      "terms": {
                        "field": "location.description.keyword",
                        "order": {
                          "_count": "desc"
                        },
                        "size": 5
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  "size": 0,
  "_source": {
    "excludes": []
  },
  "stored_fields": [
    "*"
  ],
  "script_fields": {},
  "docvalue_fields": [
    {
      "field": "@timestamp",
      "format": "date_time"
    }
  ],
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "@timestamp": {
              "format": "strict_date_optional_time",
              "gte": "2019-11-25T08:00:00.000Z",
              "lte": "2019-12-03T18:18:06.836Z"
            }
          }
        }
      ],
      "filter": [
        {
          "match_all": {}
        },
        {
          "match_all": {}
        }
      ],
      "should": [],
      "must_not": []
    }
  }
}

Response

{
  "took": 608,
  "timed_out": false,
  "_shards": {
    "total": 151,
    "successful": 151,
    "skipped": 137,
    "failed": 0
  },
  "hits": {
    "total": 132216,
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "3": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 522,
      "buckets": [
        {
          "2": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 1088,
            "buckets": [
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 5953,
                  "buckets": [
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Surname",
                                  "doc_count": 3113
                                }
                              ]
                            },
                            "key": 2,
                            "doc_count": 3113
                          },
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 3113
                                }
                              ]
                            },
                            "key": 19,
                            "doc_count": 3113
                          },
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 3113
                                }
                              ]
                            },
                            "key": 20,
                            "doc_count": 3113
                          }
                        ]
                      },
                      "key": 19,
                      "doc_count": 12452
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Identifier",
                                  "doc_count": 3113
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 3113
                          }
                        ]
                      },
                      "key": 30,
                      "doc_count": 6226
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 3113
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 3113
                          }
                        ]
                      },
                      "key": 15,
                      "doc_count": 5134
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": []
                      },
                      "key": 25,
                      "doc_count": 3113
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": []
                      },
                      "key": 29,
                      "doc_count": 3113
                    }
                  ]
                },
                "key": "ORC",
                "doc_count": 35991
              },
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 3425,
                  "buckets": [
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Surname",
                                  "doc_count": 1535
                                }
                              ]
                            },
                            "key": 2,
                            "doc_count": 1535
                          },
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 1535
                                }
                              ]
                            },
                            "key": 19,
                            "doc_count": 1535
                          },
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 1535
                                }
                              ]
                            },
                            "key": 20,
                            "doc_count": 1535
                          }
                        ]
                      },
                      "key": 9,
                      "doc_count": 6140
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Surname",
                                  "doc_count": 1535
                                }
                              ]
                            },
                            "key": 2,
                            "doc_count": 1535
                          },
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 1535
                                }
                              ]
                            },
                            "key": 19,
                            "doc_count": 1535
                          },
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 1535
                                }
                              ]
                            },
                            "key": 20,
                            "doc_count": 1535
                          }
                        ]
                      },
                      "key": 17,
                      "doc_count": 6140
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "ID Number",
                                  "doc_count": 3113
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 3113
                          }
                        ]
                      },
                      "key": 19,
                      "doc_count": 4648
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 3104
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 3104
                          }
                        ]
                      },
                      "key": 45,
                      "doc_count": 4639
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 1760
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 1760
                          }
                        ]
                      },
                      "key": 44,
                      "doc_count": 3295
                    }
                  ]
                },
                "key": "PV1",
                "doc_count": 28287
              },
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": [
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 3113
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 3113
                          }
                        ]
                      },
                      "key": 29,
                      "doc_count": 3113
                    }
                  ]
                },
                "key": "PID",
                "doc_count": 3113
              },
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": []
                },
                "key": "RXC",
                "doc_count": 1228
              },
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": [
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 1087
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 1087
                          }
                        ]
                      },
                      "key": 4,
                      "doc_count": 1087
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 6
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 6
                          }
                        ]
                      },
                      "key": 3,
                      "doc_count": 6
                    }
                  ]
                },
                "key": "RXA",
                "doc_count": 1093
              }
            ]
          },
          "key": "entity_received_fewer_times_than_expected",
          "doc_count": 70800
        },
        {
          "2": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 1,
            "buckets": [
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 3760,
                  "buckets": [
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Identifier",
                                  "doc_count": 3113
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 3113
                          }
                        ]
                      },
                      "key": 30,
                      "doc_count": 6226
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 3113
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 3113
                          }
                        ]
                      },
                      "key": 15,
                      "doc_count": 5134
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": []
                      },
                      "key": 19,
                      "doc_count": 3113
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": []
                      },
                      "key": 25,
                      "doc_count": 3113
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": []
                      },
                      "key": 29,
                      "doc_count": 3113
                    }
                  ]
                },
                "key": "ORC",
                "doc_count": 24459
              },
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 1805,
                  "buckets": [
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "ID Number",
                                  "doc_count": 3113
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 3113
                          }
                        ]
                      },
                      "key": 19,
                      "doc_count": 4648
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 3104
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 3104
                          }
                        ]
                      },
                      "key": 45,
                      "doc_count": 4639
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 1760
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 1760
                          }
                        ]
                      },
                      "key": 44,
                      "doc_count": 3295
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": []
                      },
                      "key": 9,
                      "doc_count": 1535
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": []
                      },
                      "key": 17,
                      "doc_count": 1535
                    }
                  ]
                },
                "key": "PV1",
                "doc_count": 17457
              },
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": []
                },
                "key": "RXC",
                "doc_count": 1228
              },
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": []
                },
                "key": "RXR",
                "doc_count": 1087
              },
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": [
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Time",
                                  "doc_count": 6
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 6
                          }
                        ]
                      },
                      "key": 3,
                      "doc_count": 6
                    }
                  ]
                },
                "key": "RXA",
                "doc_count": 6
              }
            ]
          },
          "key": "required_entity_not_received",
          "doc_count": 44238
        },
        {
          "2": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": [
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": []
                      },
                      "key": 39,
                      "doc_count": 1578
                    }
                  ]
                },
                "key": "PV1",
                "doc_count": 1578
              },
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": [
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": []
                      },
                      "key": 2,
                      "doc_count": 790
                    }
                  ]
                },
                "key": "RXA",
                "doc_count": 790
              },
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": [
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Repeat Pattern",
                                  "doc_count": 543
                                }
                              ]
                            },
                            "key": 2,
                            "doc_count": 543
                          }
                        ]
                      },
                      "key": 7,
                      "doc_count": 543
                    },
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Identifier",
                                  "doc_count": 1
                                }
                              ]
                            },
                            "key": 1,
                            "doc_count": 1
                          }
                        ]
                      },
                      "key": 16,
                      "doc_count": 1
                    }
                  ]
                },
                "key": "ORC",
                "doc_count": 544
              },
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": [
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Name of Coding System",
                                  "doc_count": 1
                                }
                              ]
                            },
                            "key": 3,
                            "doc_count": 1
                          }
                        ]
                      },
                      "key": 4,
                      "doc_count": 1
                    }
                  ]
                },
                "key": "OBR",
                "doc_count": 1
              }
            ]
          },
          "key": "max_length_exceeded",
          "doc_count": 2913
        },
        {
          "2": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": [
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Message Event Type",
                                  "doc_count": 1535
                                }
                              ]
                            },
                            "key": 2,
                            "doc_count": 1535
                          }
                        ]
                      },
                      "key": 8,
                      "doc_count": 1535
                    }
                  ]
                },
                "key": "MSH",
                "doc_count": 1535
              }
            ]
          },
          "key": "invalid_entity_value",
          "doc_count": 1535
        },
        {
          "2": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": []
                },
                "key": "OBX",
                "doc_count": 490
              },
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": []
                },
                "key": "RXC",
                "doc_count": 44
              },
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": [
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                          {
                            "5": {
                              "doc_count_error_upper_bound": 0,
                              "sum_other_doc_count": 0,
                              "buckets": [
                                {
                                  "key": "Interval",
                                  "doc_count": 21
                                }
                              ]
                            },
                            "key": 2,
                            "doc_count": 21
                          }
                        ]
                      },
                      "key": 7,
                      "doc_count": 21
                    }
                  ]
                },
                "key": "ORC",
                "doc_count": 21
              },
              {
                "4": {
                  "doc_count_error_upper_bound": 0,
                  "sum_other_doc_count": 0,
                  "buckets": [
                    {
                      "6": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": []
                      },
                      "key": 9,
                      "doc_count": 10
                    }
                  ]
                },
                "key": "PV1",
                "doc_count": 10
              }
            ]
          },
          "key": "entity_received_too_many_times",
          "doc_count": 565
        }
      ]
    }
  },
  "status": 200
}