Analitics page crash. Elasticsearch has no templates, mappings or aliases

[bercab]

Seems api-umbrella does not initialize properly elasticsearch templates, index mappings or aliases.

When accessing to analytics dashboard (/admin/#/stats/drilldown), web-puma shows this log:

2016-01-26T17:14:32.57037 Processing by Api::V1::AnalyticsController#drilldown as JSON
2016-01-26T17:14:32.57077   Parameters: {"tz"=>"Europe/Berlin", "search"=>"", "start_at"=>"2015-12-28", "end_at"=>"2016-01-26", "query"=>"{\"condition\":\"AND\",\"rules\":[{\"field\":\"gatekeeper_denied_code\",\"
id\":\"gatekeeper_denied_code\",\"input\":\"select\",\"operator\":\"is_null\",\"type\":\"string\",\"value\":null}]}", "interval"=>"day", "prefix"=>"0/"}
2016-01-26T17:14:32.61434 GET http://localhost:14002/api-umbrella-logs-2015-12,api-umbrella-logs-2016-01/_search?allow_no_indices=true&ignore_unavailable=missing&size=0 [status:200, request:0.027s, query:0.002s]
2016-01-26T17:14:32.61437 Completed 500 Internal Server Error in 42.5ms
2016-01-26T17:14:32.61589 
2016-01-26T17:14:32.61591 NoMethodError (undefined method `[]' for nil:NilClass):

The error seems because there are no results (should not crash this way)
Maybe there is no results because there is no alias from index api-umbrella-logs-2016-01 to: api-umbrella-logs-v1-2016-01

I think the problem is that elasticsearch is not initialized propertly.
I can see proxy/jobs/elasticsearch_setup.lua, which seems to initialize some elasticsearch templates and aliases.

But first process to write to elasticsearch is heka (umbrella receives requests), and seems to auto-geneate a mapping for the index, so there is no previously index template creation:

2016-01-26T17:12:45.84590 [2016-01-26 12:12:45,845][INFO ][cluster.metadata         ] [Aminedi] [api-umbrella-logs-v1-2016-01] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings [log]
2016-01-26T17:12:47.99495 [2016-01-26 12:12:47,994][INFO ][cluster.metadata         ] [Aminedi] [api-umbrella-logs-v1-2016-01] update_mapping [log] (dynamic)

If I create elasticsearch alias, it finds results, but crashes because of NumberFormatException on field required_at, and it seems like elastisearch needs a correct mapping for this field.

Api-umbrella version is 0.11.0, from deb packages.
I have tried embeeded elasticsearch and also standalone elasticsearch cluster v2.1.1.

Anyone have v0.11.0 working with elasticsearch analytics?

[bercab]

Finally got elasticsearch index template initialized:

It seems that if elastisearch host setting is a dns name (not an IP), then api-umbrella does not perform the elasticsearch inititalization, and does not create template indexes with mappings, making web crash.

I had this setting:
So moving from:

elasticsearch:
  hosts:
    - http://localhost:14002

to:

elasticsearch:
  hosts:
    - http://127.0.0.1:14002

Makes it work. note that with dns name, all insertions work, but i'ts not initialized.

[GUI]

Odd, but thanks for the sleuthing and pinpointing the problem to DNS--that should make this much easier to resolve. Sorry for the trouble, but we'll look to get this fixed in the next release.

[GUI]

I'm still not sure what would case this, but we've switched all the default values in v0.14.0 to use 127.0.0.1 instead of localhost, so hopefully this should eliminate this as a potential issue.

[sraghunathan]

But still I'm getting the error on /admin/#/stats/drilldown
An unexpected error occurred while trying to load the page. Please try again or go back.