HHVM versus PHP-FPM 5.4 vs PHP-FPM 5.5: performance comparison

Mattias Geniar, Thursday, August 7, 2014 - last modified: Thursday, December 18, 2014

If you haven't heard of HHVM in the last 2 years, I recommend checking out the website. I won't go into details here. I've even mentioned it briefly on this blog in 2010.

But I was curious: given a stock PHP-FPM 5.4 with APC or 5.5 with OpCache enabled, how much faster would HHVM 3.1.0 be? I decided to load a working Drupal site into a Vagrant VM and do some benchmarks. It's a stock Drupal with Memcached configured, but Drupal's page cache disabled. After all, I want to test PHP's performance (PHP parsing of modules & execution etc.), not serving cached pages. I want it load and execute every module, on every pageload.

HHVM was installed using the Github Wiki guide, using CentOS 6.5 and the yum repository by "hop5". PHP 5.4.30 was installed IUS repository, with a default set of extensions (gd, intl, xml, pecl/memcache, bcmath). The same goes for PHP 5.5.15, with the opcache enabled, same extensions.

I'm running my benchmark using Apache's Bench (ab) like this.

$ ab -c 1 -n 100 http://mattias.drupal:81/

I'll make 100 requests, one at a time, to a local VM running Nginx + PHP-FPM and Nginx + HHVM. I'm doing a concurrency of one, to test how fast each config can handle the single requests.

It's a stock Nginx 1.6, using sockets to talk to both PHP-FPM and HHVM. In PHP 5.4 APC's apc.stat is on. For 5.5 it's the default OpCache settings.

Benchmarking PHP-FPM 5.4

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   163.102 seconds

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   160.255 seconds

Two consecutive runs, the first taking 163 seconds, the second taking 160 seconds. Let's average that to 161.5 seconds for 100 requests. That's 1.615 seconds per page request on PHP-FPM.

Benchmarking PHP-FPM 5.5

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   82.343 seconds

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   81.985 seconds

That's an easy average of 82 seconds. So we've got 0.73 seconds per page request.

Benchmarking HHVM 3.1

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   61.431 seconds

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   55.624 seconds

Again two consecutive runs. Let's average them out at 58.5 seconds for 100 requests. That's 0.58 seconds per request. Holy cow, that's 2.7x times faster than stock PHP 5.4!

Conclusions

The numbers speak for themselves. This is how long it takes for 100 requests of a standard Drupal site to load. Less seconds is obviously better.

  • PHP-FPM 5.4: 161.5 seconds for 100 requests
  • PHP-FPM 5.5: 82 seconds for 100 requests
  • HHVM 3.1: 58.5 seconds for 100 requests

PHP-FPM vs HHVM

Running the PHP-FPM benchmark, I could only see a single core of my 2-core machine being used. This makes sense, as PHP is single-threaded and I was only starting the benchmark one request at a time. However, when starting the benchmarking on HHVM, I could see both cores being equally loaded. Probably because running "top" refreshes once a second, but since requests finish before a second, there will be overlap in the processes being visible.

I'm happy to notice a few things;

  • Getting HHVM to work with Drupal was a piece of cake, took 2 minutes (modify configs, set socket, done).
  • Getting HHVM running on CentOS was a piece of cake, thanks to the hop5 repository. Dependencies were all working.
  • Even the difference between PHP 5.4 and 5.5 is very noticeable (2x performance).
  • Difference between PHP 5.4 and HHVM is huge, between PHP 5.5 and HHVM it's starting to get closer

The future is bright for PHP! When I have a chance, I'll try to give PHP-NG a go, as it's claimed to be even faster than HHVM.



Hi! My name is Mattias Geniar. I'm a Support Manager at Nucleus Hosting in Belgium, a general web geek & public speaker. Currently working on DNS Spy & Oh Dear!. Follow me on Twitter as @mattiasgeniar.


Share this post

Did you like this post? Will you help me share it on social media? Thanks!

Comments

Ali Asif Friday, August 8, 2014 at 14:01 - Reply

Hi,
Really nice comparison , I was looking for something similar from many days. By the way do you have any plan to do performance comparison of php application with node.js application.


    Mattias Geniar Friday, August 8, 2014 at 14:42 - Reply

    Since there’s no way to correctly compare PHP vs NodeJS (you can’t load a Drupal app that works in PHP into NodeJS), I don’t see a point in comparing both languages.

    I do plan to test PHP 5.6 (or “php-ng”) in the future.


sam Thursday, August 28, 2014 at 14:49 - Reply

Is fph-fpm with opcache?


Amol Rajoba Friday, October 3, 2014 at 17:24 - Reply

During test in PHP 5.4 APC’s apc.stat should have been off for performace gain


windy Saturday, November 8, 2014 at 15:26 - Reply

Is php5.5’s opcache per php-fpm master process or per php-fpm pool? If you have different pools for different users but one master process is that a security risk because of the shared opcache?


Janio Sarmento Wednesday, April 29, 2015 at 17:35 - Reply

Hello,

Would you allow me to use your graph to illustrate a section in my site, with a link for this post?


Luca Gervasi Monday, October 5, 2015 at 11:31 - Reply

Good points!
I think you should create a real world scenario using real logs. I think there is no point using a single page to benchmark as you are really stressing APC or other OPCache methods.
I suggest you to use jmeter to replay this wordpress installation navigation. This way you could have enough variability both in requests and rate to have a real hhvm/fpm stress.
Moreover, you should also consider including mod_php + apache, as it is very good and not so “legacy” to be ignored.

Good work, keep up!


Leave a Reply

Your email address will not be published. Required fields are marked *

Inbound links