HHVM versus PHP-FPM 5.4 vs PHP-FPM 5.5: performance comparison

If you haven’t heard of HHVM in the last 2 years, I recommend checking out the website. I won’t go into details here. I’ve even mentioned it briefly on this blog in 2010.

But I was curious: given a stock PHP-FPM 5.4 with APC or 5.5 with OpCache enabled, how much faster would HHVM 3.1.0 be? I decided to load a working Drupal site into a Vagrant VM and do some benchmarks. It’s a stock Drupal with Memcached configured, but Drupal’s page cache disabled. After all, I want to test PHP’s performance (PHP parsing of modules & execution etc.), not serving cached pages. I want it load and execute every module, on every pageload.

HHVM was installed using the Github Wiki guide, using CentOS 6.5 and the yum repository by “hop5”. PHP 5.4.30 was installed IUS repository, with a default set of extensions (gd, intl, xml, pecl/memcache, bcmath). The same goes for PHP 5.5.15, with the opcache enabled, same extensions.

I’m running my benchmark using Apache’s Bench (ab) like this.

$ ab -c 1 -n 100 http://mattias.drupal:81/

I’ll make 100 requests, one at a time, to a local VM running Nginx + PHP-FPM and Nginx + HHVM. I’m doing a concurrency of one, to test how fast each config can handle the single requests.

It’s a stock Nginx 1.6, using sockets to talk to both PHP-FPM and HHVM. In PHP 5.4 APC’s apc.stat is on. For 5.5 it’s the default OpCache settings.

Benchmarking PHP-FPM 5.4

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   163.102 seconds

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   160.255 seconds

Two consecutive runs, the first taking 163 seconds, the second taking 160 seconds. Let’s average that to 161.5 seconds for 100 requests. That’s 1.615 seconds per page request on PHP-FPM.

Benchmarking PHP-FPM 5.5

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   82.343 seconds

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   81.985 seconds

That’s an easy average of 82 seconds. So we’ve got 0.73 seconds per page request.

Benchmarking HHVM 3.1

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   61.431 seconds

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   55.624 seconds

Again two consecutive runs. Let’s average them out at 58.5 seconds for 100 requests. That’s 0.58 seconds per request. Holy cow, that’s 2.7x times faster than stock PHP 5.4!

Conclusions

The numbers speak for themselves. This is how long it takes for 100 requests of a standard Drupal site to load. Less seconds is obviously better.

• PHP-FPM 5.4: 161.5 seconds for 100 requests
  • PHP-FPM 5.5: 82 seconds for 100 requests

    • HHVM 3.1: 58.5 seconds for 100 requests

    

    Running the PHP-FPM benchmark, I could only see a single core of my 2-core machine being used. This makes sense, as PHP is single-threaded and I was only starting the benchmark one request at a time. However, when starting the benchmarking on HHVM, I could see both cores being equally loaded. Probably because running “top” refreshes once a second, but since requests finish before a second, there will be overlap in the processes being visible.

    I’m happy to notice a few things;

    • Getting HHVM to work with Drupal was a piece of cake, took 2 minutes (modify configs, set socket, done).
    • Getting HHVM running on CentOS was a piece of cake, thanks to the hop5 repository. Dependencies were all working.
    • Even the difference between PHP 5.4 and 5.5 is very noticeable (2x performance).
    • Difference between PHP 5.4 and HHVM is huge, between PHP 5.5 and HHVM it’s starting to get closer

    The future is bright for PHP! When I have a chance, I’ll try to give PHP-NG a go, as it’s claimed to be even faster than HHVM.