Enable the slow log in Elastic Search

Elasticsearch is pretty cool, you can just fire of HTTP commands to it to change (most of) its settings on the fly, without restarting the service. Here’s how you can enable the slowlog to lo queries that exceed a certain time treshold.

These are enabled per index you have, so you can be selective about it.

Get all indexes in your Elastic Search

To start, get a list of all your Elasticsearch indexes. I’m using jq here for the JSON formatting (get jq here).

$ curl -s -XGET 'http://127.0.0.1:9200/_cat/indices'
green open index1 BV8NLebPuHr6wh2qUnp7XpTLBT 2 0 425739 251734  1.3gb  1.3gb
green open index2 3hfdy8Ldw7imoq1KDGg2FMyHAe 2 0 425374 185515  1.2gb  1.2gb
green open index3 ldKod8LPUOphh7BKCWevYp3xTd 2 0 425674 274984  1.5gb  1.5gb

This shows you have 3 indexes, called index1, index2 and index3.

Enable slow log per index

Make a PUT HTTP call to change the settings of a particular index. In this case, index index3 will be changed.

$ curl -XPUT -d '{"index.search.slowlog.threshold.query.warn" : "50ms","index.search.slowlog.threshold.fetch.warn": "50ms","index.indexing.slowlog.threshold.index.warn": "50ms"}' http://127.0.0.1:9200/index3/_settings | jq

If you pretty-print the JSON payload, it looks like this:

{
  "index.search.slowlog.threshold.query.warn" : "50ms",
  "index.search.slowlog.threshold.fetch.warn": "50ms",
  "index.indexing.slowlog.threshold.index.warn": "50ms"
}

Which essentially means: log all queries, fetches and index rebuilds that exceed 50ms with a severity of “warning”.

Enable global warning logging

To make sure those warning logs get written to your logs, make sure you enable that logging in your cluster.

$ curl -XPUT -d '{"transient" : {"logger.index.search.slowlog" : "WARN", "logger.index.indexing.slowlog" : "WARN" }}' http://127.0.0.1:9200/_cluster/settings | jq

Again, pretty-printed payload:

{
  "transient" : {
    "logger.index.search.slowlog" : "WARN",
    "logger.index.indexing.slowlog" : "WARN"
  }
}

These settings aren’t persisted on restart, they are only written to memory and active for the currently running elasticsearch instance.