Elasticsearch fails to start: access denied “/etc/elasticsearch/${instance}/scripts”

Oh Dear! monitors your entire site, not just the homepage. We crawl and search for broken pages and mixed content, send alerts when your site is down and notify you on expiring SSL certificates.

Start your free 10 day trial! »

Image of Mattias Geniar

Mattias Geniar, September 11, 2018

Follow me on Twitter as @mattiasgeniar

Ran into a not-so-obvious issue with Elasticsearch today. After an update from 6.2.x to 6.4.x, an elasticsearch instance failed to start with following error in its logs.

$ tail -f instance.log
...
Caused by: java.security.AccessControlException:
  access denied ("java.io.FilePermission" "/etc/elasticsearch/production/scripts" "read")
   at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
   at java.security.AccessController.checkPermission(AccessController.java:884)
   at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
...

The logs indicated it couldn’t read from /etc/elasticsearch/production/scripts, but all permissions were OK. Apparently, Elasticsearch doesn’t follow symlinks for the scripts directory, even if it has all the required read permissions.

In this case, that scripts directory was indeed a symlink to a shared resource.

$ ls -alh /etc/elasticsearch/production/scripts 
/etc/elasticsearch/production/scripts -> /usr/share/elasticsearch/scripts

The fix was simple, as the shared directory didn’t contain any scripts to begin with.

$ rm -f /etc/elasticsearch/production/scripts
$ mkdir /etc/elasticsearch/production/scripts
$ chown elasticsearch:elasticsearch /etc/elasticsearch/production/scripts

Turn that symlink into a normal directory & restart your instance.

If you run multiple instances, you’ll have to repeat these steps for every instance that fails to start.

Will you help me share this post?

It would mean a lot to me if you could help share this post on social media. 🤗