Elasticsearch fails to start: access denied “/etc/elasticsearch/${instance}/scripts”

Oh Dear monitors your entire site, not just the homepage. We crawl and search for broken pages and mixed content, send alerts when your site is down and notify you on expiring SSL certificates.

Start your free 10 day trial! »

Profile image of Mattias Geniar

Mattias Geniar, September 11, 2018

Follow me on Twitter as @mattiasgeniar

Ran into a not-so-obvious issue with Elasticsearch today. After an update from 6.2.x to 6.4.x, an elasticsearch instance failed to start with following error in its logs.

$ tail -f instance.log
Caused by: java.security.AccessControlException:
  access denied ("java.io.FilePermission" "/etc/elasticsearch/production/scripts" "read")
   at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
   at java.security.AccessController.checkPermission(AccessController.java:884)
   at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)

The logs indicated it couldn’t read from /etc/elasticsearch/production/scripts, but all permissions were OK. Apparently, Elasticsearch doesn’t follow symlinks for the scripts directory, even if it has all the required read permissions.

In this case, that scripts directory was indeed a symlink to a shared resource.

$ ls -alh /etc/elasticsearch/production/scripts 
/etc/elasticsearch/production/scripts -> /usr/share/elasticsearch/scripts

The fix was simple, as the shared directory didn’t contain any scripts to begin with.

$ rm -f /etc/elasticsearch/production/scripts
$ mkdir /etc/elasticsearch/production/scripts
$ chown elasticsearch:elasticsearch /etc/elasticsearch/production/scripts

Turn that symlink into a normal directory & restart your instance.

If you run multiple instances, you’ll have to repeat these steps for every instance that fails to start.

Want to subscribe to the cron.weekly newsletter?

I write a weekly-ish newsletter on Linux, open source & webdevelopment called cron.weekly.

It features the latest news, guides & tutorials and new open source projects. You can sign up via email below.

No spam. Just some good, practical Linux & open source content.