A better way to run PHP-FPM

Oh Dear! monitors your entire site, not just the homepage. We crawl and search for broken pages and mixed content, send alerts when your site is down and notify you on expiring SSL certificates.

Start your free 10 day trial! »

Image of Mattias Geniar

Mattias Geniar, April 09, 2014

Follow me on Twitter as @mattiasgeniar

If you search the web for PHP-FPM configurations, you’ll find many of the same configurations popping up. They nearly all use the ‘dynamic’ process manager and all assume you will have one master process for running PHP-FPM configurations. While there’s nothing technically wrong with that, there is a better way to run PHP-FPM.

In this blogpost I’ll detail;

  1. Why ‘dynamic’ should not be your default process manager
  2. Why it’s better to have multiple PHP-FPM masters

Why you should prefer the ‘ondemand’ Process Manager instead of ‘dynamic’

Most of the copy/paste work on the internet for PHP-FPM have configurations such as the one below.

[pool_name]
...
pm = dynamic
pm.max_children = 5
pm.start_servers = 3
pm.min_spare_servers = 2
pm.max_spare_servers = 4
pm.max_requests = 200

Most “guides” advocate the use of the ‘dynamic’ Process Manager (‘pm’ option in the config), which allows you to choose how many minimum and maximum (spare) processes you have per pool. Many guides however make blind assumptions on what your server specs are and will cause, like in the example above, a minimum of 3 PHP processes running per pool (_pm.startservers = 3). If you’re on a low-traffic site, that could very well be overkill. For your server, it looks like this in your processlist.

root   3986  4704 ?   Ss   19:04   php-fpm: master process (/etc/php-fpm.conf)
user   3987  4504 ?   S    19:04   \_ php-fpm: pool pool_name
user   3987  4504 ?   S    19:04   \_ php-fpm: pool pool_name
user   3987  4504 ?        19:04   \_ php-fpm: pool pool_name

Those 3 processes will always be running, whether they’re needed or not.

Ondemand Process Manager

A far better way to run PHP-FPM pools, but badly documented, would be the ‘ondemand’ Process Manager. As the name suggests, it does not leave processes lingering, but spawns them as they are needed. The configuration is similar to the above, but simplified.

[pool_name]
...
pm = ondemand
pm.max_children = 5
pm.process_idle_timeout = 10s
pm.max_requests = 200

The ‘ondemand’ process manager was added since PHP 5.3.8. The config above causes your default processlist to look like this.

root   3986  4704 ?   Ss   19:04  php-fpm: master process (/etc/php-fpm.conf)

Only the master process is spawned, there are no pre-forked PHP-FPM processes. Only when processes are needed will they be started, to a maximum of 5 (with the config above, which is defined by pm.max_children). So if there are 2 simultaneous PHP requests going on, the processlist would be:

root   3986  4704 ?   Ss   19:04   php-fpm: master process (/etc/php-fpm.conf)
user   3987  4504 ?   S    19:04   \_ php-fpm: pool pool_name
user   3987  4504 ?   S    19:04   \_ php-fpm: pool pool_name

After the configured timeout in “pm.process_idle_timeout”, the process will be stopped again. This does not impact PHP’s max_execution_time, because the process manager only considers a process “idle” when it’s not serving any request.

If you’re working on a high performance PHP setup, the ‘ondemand’ PM may not be for you. In that case, it’s wise to pre-fork your PHP-FPM processes up to the maximum your server can handle. That way, all your processes are ready to serve your requests without needing to be spawned first. However, for 90% of the sites out there, the ondemand PHP-FPM configuration is better than either static or dynamic.

A shared APC or OPcache: why multiple PHP-FPM masters are better

You may not be aware that the APC or OPcache is actually held by the master process in PHP. Any configuration for APC needs to come from the .INI configurations and cannot be overwritten later on via _iniset() or php_admin_value. That’s because the spawned PHP-FPM processes have no influence on the size or config of the APC cache, as it is initiated and managed by the master process.

That inherently means that the APC/OPcache cache is shared between all PHP-FPM pools. If you only have a single site to serve, that’s no issue. If you have a few dozen sites on the same server via PHP-FPM, you should be aware that they all share the same APC/OPcache cache. The APC or OPcache size should then be big enough to hold the opcode cache of all your sites combined.

To avoid this, each PHP-FPM pool can also be started separately and have it’s own master process. That means each site can have its own APC or OPcache and can be started/stopped independently from all the other PHP-FPM pools. A change in one pool’s config does not cause all the other FPM pools to be reloaded when the new config needs to be activated, which is the default behaviour of “/etc/init.d/php-fpm reload” (it would reload all pools).

What’s needed to achieve this then;

  1. Each PHP-FPM pool needs its own init.d start/stop script
  2. Each PHP-FPM pool needs its own php-fpm.conf file to have a unique PID

If you manage your environment via a CMS such as Puppet/Chef/Salt/Ansible, the above is not difficult to set up. If you do things manually, it can become a burden and difficult to manage.

Looking at the PHP-FPM configuration

Here’s what an abbreviated configuration can look like. You would now have a single .conf file that contains the configuration of your master process (PID etc.) as well as the definition of 1 PHP-FPM pool.

$ cat /etc/php-fpm.d/pool1.conf
[global]
pid = /var/run/php-fpm/pool1.pid
log_level = notice
emergency_restart_threshold = 0
emergency_restart_interval = 0
process_control_timeout = 0
daemonize = yes

[pool1]
listen = /var/run/php-fpm/pool1.sock
listen.owner = pool1
listen.group = pool1
listen.mode = 0666

user = pool1
group = pool1

pm = ondemand
pm.max_children = 5
pm.process_idle_timeout = 10s
pm.max_requests = 500

The above contains the most important bits; the main config determines that it can be daemonized and where the PID-file should be located. The Pool-configuration has the basic information of where to listen to and the type of Process Manager.

The init.d file is a simple copy/paste from the default /etc/init.d/php-fpm with a few modifications.

$ cat /etc/init.d/php-fpm-pool1
#! /bin/sh
#
# chkconfig: - 84 16
# description:  PHP FastCGI Process Manager for pool 'pool1'
# processname: php-fpm-pool1
# config: /etc/php-fpm.d/pool1.conf
# pidfile: /var/run/php-fpm/pool1.pid

# Standard LSB functions
#. /lib/lsb/init-functions

# Source function library.
. /etc/init.d/functions

# Check that networking is up.
. /etc/sysconfig/network

if [ "$NETWORKING" = "no" ]
then
    exit 0
fi

RETVAL=0
prog="php-fpm-pool1"
pidfile=/var/run/php-fpm/pool1.pid
lockfile=/var/lock/subsys/php-fpm-pool1
fpmconfig=/etc/php-fpm.d/pool1

start () {
    echo -n $"Starting $prog: "
    daemon --pidfile ${pidfile} php-fpm --fpm-config=${fpmconfig} --daemonize
    RETVAL=$?
    echo
    [ $RETVAL -eq 0 ] && touch ${lockfile}
}
stop () {
    echo -n $"Stopping $prog: "
    killproc -p ${pidfile} php-fpm
    RETVAL=$?
    echo
    if [ $RETVAL -eq 0 ] ; then
        rm -f ${lockfile} ${pidfile}
    fi
}

restart () {
        stop
        start
}

reload () {
    echo -n $"Reloading $prog: "
    killproc -p ${pidfile} php-fpm -USR2
    RETVAL=$?
    echo
}


# See how we were called.
case "$1" in
  start)
    start
    ;;
  stop)
    stop
    ;;
  status)
    status -p ${pidfile} php-fpm
    RETVAL=$?
    ;;
  restart)
    restart
    ;;
  reload|force-reload)
    reload
    ;;
  condrestart|try-restart)
    [ -f ${lockfile} ] && restart || :
    ;;
  *)
    echo $"Usage: $0 {start|stop|status|restart|reload|force-reload|condrestart|try-restart}"
    RETVAL=2
        ;;
esac

exit $RETVAL

The only pieces we changed to that init.d script are at the top; a new process name has been defined (this needs to be unique) and the PID-file has been changed to point to our custom PID-file for this pool, as it’s defined in the pool1.conf file above.

You can now start/stop this pool separately from all the others. It’s configuration can be changed without impacting others. If you have multiple pools configured, your process list would look like this.

root      5963  4704 ?        Ss   19:23   0:00 php-fpm: master process (/etc/php-fpm.d/pool1.conf)
root      6036  4744 ?        Ss   19:23   0:00 php-fpm: master process (/etc/php-fpm.d/pool2.conf)

Multiple master processes are running as root and are listening to a socket defined in the pool configuration. As soon as PHP requests are made, they spawn children to handle them and stop them again after 10s of idling. The master process also shows which configuration file it loaded, making it easy to pinpoint the configuration of that particular pool.

Better separation

As soon as PHP requests are made, the processlist looks like this.

root  5963  4704  Ss  19:23  php-fpm: master process (/etc/php-fpm.d/p1.conf)
user  3987  4504  S   19:23   \_ php-fpm: pool pool1
user  3987  4504  S   19:23   \_ php-fpm: pool pool1
root  6036  4744  Ss  19:23  php-fpm: master process (/etc/php-fpm.d/p2.conf)
user  3987  4504  S   19:23   \_ php-fpm: pool pool2

To summarise, the above has 2 main advantages:

  1. a separate APC/OPcache bytecode cache per PHP-FPM pool
  2. the ability to start/stop/reconfigure PHP-FPM pools without impacting the other defined pools

For anyone struggling with APC/OPcache/realpath/stat cache issues on PHP deploys, this configuration could be a solution by allowing (sudo) access to restart or reload the master PHP-FPM process of your particular pool in order to clear all caches.

Things to keep in mind when doing this:

  • Logrotate should be modified if you use error logging in each FPM pool, to use the correct pool’s PID to reload the master process;
  • Make sure all your FPM pools start on system boot, as they each have a new init.d script (check via chkconfig –list);

Feedback appreciated!

Will you help me share this post?

It would mean a lot to me if you could help share this post on social media. 🤗