HHVM versus PHP-FPM 5.4 vs PHP-FPM 5.5: performance comparison

Mattias Geniar, Thursday, August 7, 2014 - last modified: Thursday, December 18, 2014

If you haven't heard of HHVM in the last 2 years, I recommend checking out the website. I won't go into details here. I've even mentioned it briefly on this blog in 2010.

But I was curious: given a stock PHP-FPM 5.4 with APC or 5.5 with OpCache enabled, how much faster would HHVM 3.1.0 be? I decided to load a working Drupal site into a Vagrant VM and do some benchmarks. It's a stock Drupal with Memcached configured, but Drupal's page cache disabled. After all, I want to test PHP's performance (PHP parsing of modules & execution etc.), not serving cached pages. I want it load and execute every module, on every pageload.

HHVM was installed using the Github Wiki guide, using CentOS 6.5 and the yum repository by "hop5". PHP 5.4.30 was installed IUS repository, with a default set of extensions (gd, intl, xml, pecl/memcache, bcmath). The same goes for PHP 5.5.15, with the opcache enabled, same extensions.

I'm running my benchmark using Apache's Bench (ab) like this.

$ ab -c 1 -n 100 http://mattias.drupal:81/

I'll make 100 requests, one at a time, to a local VM running Nginx + PHP-FPM and Nginx + HHVM. I'm doing a concurrency of one, to test how fast each config can handle the single requests.

It's a stock Nginx 1.6, using sockets to talk to both PHP-FPM and HHVM. In PHP 5.4 APC's apc.stat is on. For 5.5 it's the default OpCache settings.

Benchmarking PHP-FPM 5.4

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   163.102 seconds

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   160.255 seconds

Two consecutive runs, the first taking 163 seconds, the second taking 160 seconds. Let's average that to 161.5 seconds for 100 requests. That's 1.615 seconds per page request on PHP-FPM.

Benchmarking PHP-FPM 5.5

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   82.343 seconds

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   81.985 seconds

That's an easy average of 82 seconds. So we've got 0.73 seconds per page request.

Benchmarking HHVM 3.1

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   61.431 seconds

$ ab -c 1 -n 100 http://mattias.drupal:81/
...
Time taken for tests:   55.624 seconds

Again two consecutive runs. Let's average them out at 58.5 seconds for 100 requests. That's 0.58 seconds per request. Holy cow, that's 2.7x times faster than stock PHP 5.4!

Conclusions

The numbers speak for themselves. This is how long it takes for 100 requests of a standard Drupal site to load. Less seconds is obviously better.

  • PHP-FPM 5.4: 161.5 seconds for 100 requests
  • PHP-FPM 5.5: 82 seconds for 100 requests
  • HHVM 3.1: 58.5 seconds for 100 requests

PHP-FPM vs HHVM

Running the PHP-FPM benchmark, I could only see a single core of my 2-core machine being used. This makes sense, as PHP is single-threaded and I was only starting the benchmark one request at a time. However, when starting the benchmarking on HHVM, I could see both cores being equally loaded. Probably because running "top" refreshes once a second, but since requests finish before a second, there will be overlap in the processes being visible.

I'm happy to notice a few things;

  • Getting HHVM to work with Drupal was a piece of cake, took 2 minutes (modify configs, set socket, done).
  • Getting HHVM running on CentOS was a piece of cake, thanks to the hop5 repository. Dependencies were all working.
  • Even the difference between PHP 5.4 and 5.5 is very noticeable (2x performance).
  • Difference between PHP 5.4 and HHVM is huge, between PHP 5.5 and HHVM it's starting to get closer

The future is bright for PHP! When I have a chance, I'll try to give PHP-NG a go, as it's claimed to be even faster than HHVM.



Hi! My name is Mattias Geniar. I'm a Support Manager at Nucleus Hosting in Belgium, a general web geek, public speaker and podcaster. Currently working on DNS Spy. Follow me on Twitter as @mattiasgeniar.

I respect your privacy and you won't get spam. Ever.
Just a weekly newsletter about Linux and open source.

SysCast podcast

In the SysCast podcast I talk about Linux & open source projects, interview sysadmins or developers and discuss web-related technologies. A show by and for geeks!

cron.weekly newsletter

A weekly newsletter - delivered every Sunday - for Linux sysadmins and open source users. It helps keeps you informed about open source projects, Linux guides & tutorials and the latest news.

Share this post

Did you like this post? Will you help me share it on social media? Thanks!

Comments

Ali Asif Friday, August 8, 2014 at 14:01 (permalink)

Hi,
Really nice comparison , I was looking for something similar from many days. By the way do you have any plan to do performance comparison of php application with node.js application.

Reply


    Mattias Geniar Friday, August 8, 2014 at 14:42 (permalink)

    Since there’s no way to correctly compare PHP vs NodeJS (you can’t load a Drupal app that works in PHP into NodeJS), I don’t see a point in comparing both languages.

    I do plan to test PHP 5.6 (or “php-ng”) in the future.

    Reply


sam Thursday, August 28, 2014 at 14:49 (permalink)

Is fph-fpm with opcache?

Reply


Amol Rajoba Friday, October 3, 2014 at 17:24 (permalink)

During test in PHP 5.4 APC’s apc.stat should have been off for performace gain

Reply


windy Saturday, November 8, 2014 at 15:26 (permalink)

Is php5.5’s opcache per php-fpm master process or per php-fpm pool? If you have different pools for different users but one master process is that a security risk because of the shared opcache?

Reply


    Mattias Geniar Thursday, April 30, 2015 at 13:07 (permalink)

    The opcache is per fpm master, not per pool.

    So yes, having multiple pools use the same master process is potentially a security risk. Mostly, it’s a performance hit, though.

    I’ve described an alternative approach to that problem in my A Better Way to Run PHP-FPM post.

    Reply


Janio Sarmento Wednesday, April 29, 2015 at 17:35 (permalink)

Hello,

Would you allow me to use your graph to illustrate a section in my site, with a link for this post?

Reply


Luca Gervasi Monday, October 5, 2015 at 11:31 (permalink)

Good points!
I think you should create a real world scenario using real logs. I think there is no point using a single page to benchmark as you are really stressing APC or other OPCache methods.
I suggest you to use jmeter to replay this wordpress installation navigation. This way you could have enough variability both in requests and rate to have a real hhvm/fpm stress.
Moreover, you should also consider including mod_php + apache, as it is very good and not so “legacy” to be ignored.

Good work, keep up!

Reply


Leave a Reply

Your email address will not be published. Required fields are marked *

Inbound links