Hi, I'm Mattias


I'm a developer, Sysadmin, Blogger and Podcaster. I live and breath online while combining offline time with my lovely wife and 2 little kids.

Using Oh Dear! to keep your Varnish cache warm

If we're already crawling your site, we might as well update your cached pages in the meanwhile!

The idea is as follows: if you've enabled our broken links or mixed content checks for any of your sites, we'll crawl your sites to find any broken pages.

On top of that, we have the ability to set custom HTTP headers per website that get added to both the uptime checks and our crawler.

Combining our crawler and the custom HTTP headers allows you to authorize our crawler in your Varnish configs to let it update the cache.

Source: Using Oh Dear! to keep your Varnish cache warm -- Oh Dear! blog


Chalk Talk #3: how to optimise your content for Varnish

The previous 2 Chalk Talk videos were about introducing Varnish and explaining how it works.

In this 3rd video, we deep dive into the caching engine of Varnish to explain how you as a developer/webmaster can optimise your content for Varnish, to achieve the best cache hit rate.

There's just one video left in this series, which will be about common mistakes made in Varnish and how to troubleshoot your Varnish caching.


Chalk Talk #2: how does Varnish work?

In the first Chalk Talk video, we looked at what Varnish can do. In this second video, I explain how Varnish does this.

As usual, if you like a Dutch written version, have a look at the company blog.

Next videos will focus more on the technical internals, like how the hashing works, how to optimize your content & site and how to debug Varnish.


Varnish: same hash, different results? Check the Vary header!

I'll admit I get bitten by the Vary header once every few months. It's something a lot of CMS's randomly add, and it has a serious impact on how Varnish handles and treats requests.

For instance, here's a request I was troubleshooting that had these varnishlog hash() data:

-   VCL_call       HASH
-   Hash           "/images/path/to/file.jpg%00"
-   Hash           "http%00"
-   Hash           "www.yoursite.tld%00"
-   Hash           "/images/path/to/file.jpg.jpg%00"
-   Hash           "www.yoursite.tld%00"
-   VCL_return     lookup
-   VCL_call       MISS

A new request, giving the exact same hashing data, would return a different page from the cache/backend. So why does a request with the same hash return different data?

Let me introduce the Vary header.

In this case, the page I was requesting added the following header:

Vary: Accept-Encoding,User-Agent

This instructs Varnish to keep a separate version each page for every value of Accept-Encoding and User-Agent it finds.

The Accept-Encoding would make sense, but Varnish already handles that internally. A gziped/plain version will return different data, that makes sense. There's no real point in adding that header for Varnish, but other proxies in between might still benefit from it.

The User-Agent is plain nonsense, why would you serve a different version of a page per browser? If you consider a typical User-Agent string to contain text like Mozilla/5.0 (Macintosh; Intel Mac OS X...) AppleWebKit/537.xx (KHTML, like Gecko) Chrome/65.x.y.z Safari/xxx, that's practically unique per visitor you have.

So, quick hack in this case, I remove the Vary header altogether.

sub vcl_backend_response {
  unset beresp.http.Vary;
  ...
}

No more variations of the cache based on what a random CMS does or says.


Varnishlog: show the hash() data in output

Since Varnish 4.x and later, the default varnishlog output doesn't show the vcl_hash() data anymore.

Your varnishlog will look like this:

-   VCL_call       HASH
-   VCL_return     lookup
-   Hit            103841868
-   VCL_call       HIT
-   VCL_return     deliver

If you're debugging, this isn't very useful.

To include the "old" behaviour of Varnish and add all the hash elements as well, start Varnish with the following extra parameter:

 -p vsl_mask=+Hash

After a Varnish restart, this will procedure varnishlog output like this:

-   VCL_call       HASH
-   Hash           "/images/path/to/file.jpg%00"
-   Hash           "http%00"
-   Hash           "www.yoursite.tld%00"
-   Hash           "/images/path/to/file.jpg%00"
-   Hash           "www.yoursite.tld%00"
-   VCL_return     lookup
-   VCL_call       MISS

In this example, the hash included:

hash_data(req.url);
hash_data(req.http.X-Forwarded-Proto);
hash_data(req.http.host);
hash_data(req.http.Cookie);

The %00 are null characters to prevent cache-conflicts in case you have hash('123'); hash('456); vs. hash('1'); hash('23456');.


Chalk Talk #1: An introduction to Varnish

At work we started a new video series called Chalk Talk, where we (try to) explain complicated technologies with a single chalkboard as guidance.

Our first series is on Varnish and will be hosted by yours truly.

Here's the first episode, giving an introduction to Varnish.

If you like a Dutch written version of the video, it's available at the Nucleus blog.

There will be 5 videos in the Varnish series, each will be posted weekly and every episode will dive deeper into the Varnish internals and offer practical tips on getting started. They'll appear on both the Nucleus blog & this one.


Varnish 5.x VCL configuration templates

Varnish 5 has been released a few weeks ago, and the good news for configuration maintainers like me is: backwards compatible configs!

So updating the repository with support for Varnish 5.x was relatively easy, you can find updated configurations here: github.com/mattiasgeniar/varnish-5.0-configuration-templates.

Many of the boilerplate configurations are just exactly the same as the 4.x configurations. However, creating a new repository for this allows me to include 5.x-only features like PROXY, HTTP/2 etc.

If you just want to start with Varnish 5, have a look at that repo. It doesn't do many varnish-5-only features yet, but I plan to add that very soon!

Link: Varnish 5 Configuration Templates


Varnish Explained

A few months ago, I gave a presentation at LaraconEU in Amsterdam titled "Varnish for PHP developers". The generic title of that presentation is actually Varnish Explained and this is a write-up of that presentation, the video and the slides.

The simplest way to get a grip of the basics of Varnish would be to watch the video recording of that presentation.

Further down you'll find the slides and a transcript of each slide, with accompanying links.

Varnish Explained: presentation video

This is the recording of the talk at LaraconEU on August 2016.

Varnish Explained

Here's a write-up per slide, with the most important topics.

(Alternatively, if you prefer to just skim through the slides, they are up on Speakerdeck too.)

varnish_explained_002

In order to fully understand Varnish, it's vital that you understand the HTTP protocol (at least the basics) and how its HTTP headers are formed and which purpose they serve.

Later on, I'll cover the Varnish internals, cool acronyms like ESI, Grace, TTL & more.


varnish_explained_003

A quick introduction: I'm Mattias Geniar, I'm the Support/Operations manager at Nucleus Hosting in Belgium.

I run the cron.weekly linux & open source newsletter and, when I find the time, host the SysCast podcast.


Part One: What's Varnish?

varnish_explained_004

Let's go.


varnish_explained_005

Varnish can do a lot of things, but it's mostly known as a reverse HTTP proxy. It brands itself as an HTTP accelerator, making HTTP requests faster by caching them.

It can also serve as a TCP load balancer (using pipe in the VCL code, more on that later), an HTTP load balancer or a generic reverse proxy.

It uses the Varnish Configuration Language (VCL) to allow you as a developer or sysadmin to modify the behaviour of Varnish and to play with the HTTP protocol to shape it so it behaves the way you want.

Varnish is also extendable with VMODs, modules that can be loaded by varnish and provide additional functions/methods and functionality you can call upon in your VCL configurations. Think of these like PHP extensions; binary files that are loaded by Varnish that extend the userspace in which you can create VCL configurations.


varnish_explained_006

A couple of important versions: Varnish 3.x is probably the most widely used, but is now EOL (so: no more security updates). Varnish 4.1 is still supported and Varnish 5 came out a few weeks ago (so the slides are slightly outdated).

If you're upgrading from Varnish 3.x to later: beware, the configurations changed drastically. You'll need to update your syntax/methods in Varnish, or Varnish won't start.


varnish_explained_007

Using Varnish in your infrastructure gives you a couple of advantages: performance, scalability, control & security.

Varnish is usually associated with performance, but it greatly increases your options to scale your infrastructure (load balancing, failover backends etc) and adds a security layer right out of the box: you can easily let Varnish protect you from the httpoxy vulnerability or slowloris type attacks.


Part Deux: HTTP Headers

varnish_explained_008

Let's talk HTTP headers.


varnish_explained_009

A quick summary of HTTP headers: there are request and response headers (basically what your User Agent (=browser) requests and what the Server (Apache/Nginx) responds) and Varnish listens to these by default to determine if requests can or should be cached.


varnish_explained_010

Whenever your browser goes to a certain website, it'll a do a few low-level things: it'll resolve the DNS of the hostname you want to browse to, open a TCP connection to that IP and start sending the HTTP request headers.


varnish_explained_011

They are basically a new-line separated key/value pair. The concepts are the same in HTTP/2, except it's binary and more structured.

The browser describes the kind of response it can receive (plain text, compressed), what site it wants to load (because remember, the browser just connected to the IP, it now needs to tell the webserver which website at that IP address it wants to load).


varnish_explained_012

If all goes well, the server will respond with a similar set of headers: the response headers. It will confirm the HTTP protocol (or down/up-grade the protocol), give a status code and describe the response (plain text / compressed).

Good, that was the HTTP reminder -- now let's get to Varnish.

Part Three: How does Varnish work?

varnish_explained_013

Go!


varnish_explained_014

Varnish is a reverse proxy, so it sits between the User Agent (the browser) and the webserver. It makes decisions to either deliver a cached version of the page, or send the request to the backend webserver for processing.

By default, it'll do so by listening to the HTTP headers the client sends as well as the ones the server responds with. But, those usually suck -- creating the need for proper Varnish templates to avoid boilerplate code being re-created all the time.


varnish_explained_015

In a normal scenario, the browser probably connects directly to the webserver, which in turn will let PHP execute the request. The simplest possible setup.


varnish_explained_016

When Varnish is introduced, a few things happen: the webserver is hidden to the user, as that user now only connects to Varnish. Varnish has bound itself on port :80 and the webserver is either running on a different server or on an alternative port. It's Varnish that will make the connection to the webserver if needed, the user has no idea.


varnish_explained_017

If you follow the official Varnish book, the "internals of Varnish" are described as this. While it's 100% correct, if you're new to Varnish, that image does you no good: it's scary, confusing and it teaches you nothing.

Instead of such a flowchart, I prefer to visualise Varnish slightly different.


varnish_explained_018

Let's start with a basic example: a user tries to connect to a server running Varnish. It'll make a connection to the port where Varnish is running and Varnish can start to process the request.


varnish_explained_019

It'll trigger a routine in Varnish called vcl_recv().

It's a routine where you can write custom VCL code to manipulate requests, determine backends, redirect users, ... it gives you full control over the HTTP request of the user.


varnish_explained_020

After that routine, Varnish can do a cache lookup: the page the user is requesting, does it already exist in the cache?


varnish_explained_021

If it does, it can serve that page to the user. Before it does that, it triggers a new internal routine called vcl_deliver(). It's another place in Varnish where you can manipulate the request, change the HTTP headers etc.


varnish_explained_022

Once that request is finished, Varnish sends the response to the user so he can render it on screen.


varnish_explained_023

Of course, not everything is a cache hit: if Varnish does a cache lookup but finds it doesn't have that object/request in the cache, it has to go and fetch that content from its own backend.

To do so, it'll trigger another internal routine called vcl_backend_fetcht().

You guessed it, it's yet another place where you can further manipulate the request before it gets sent to the backend webserver(s).


varnish_explained_024

That routine will eventually call out to another webserver and have it process the request the user made.


varnish_explained_025

Once the response was created on the backend webserver, Varnish will receive it and fire another routine: vcl_backend_response().

Hey, yet another place to play with the HTTP protocol and change the headers etc. before Varnish will process the request internally.


varnish_explained_026

Varnish can then store the item in its cache (if it's cacheable) and deliver it to the client, via the same vcl_deliver() routine.

Pfew, what a ride!


Part Four: The Varnish Configuration Language

varnish_explained_027

Now that you have a visual of the flows within Varnish, let's see what the configurations look like.


varnish_explained_028

The Varnish Configuration Language is a C-like language, which should be familiar to those writing PHP or JavaScript code.

It looks like the screenshot above, where routines are described using the sub keyword. For a complete example, have a look at my varnish 4.x configuration templates.


varnish_explained_029

Each of those routines I described earlier allow you to customise the HTTP request with that VCL language.

To do so, it offers you a few useful objects: req, req.http, ... Each of those objects represent a part of the HTTP protocol. There's req.method to indicate a GET, POST, PUT, PURGE, ...

There's req.http which contains all the raw HTTP headers sent by the client, so there's req.http.host for the Host-header, req.http.User-Agent for the User-Agent (Chrome/Firefox) of the client, ...

Just like the HTTP protocol itself, the HTTP headers are case insensitive. So req.http.HOST is the same as req.http.host. But do yourself a favour, pick a coding/style standard you like and stick to it.

In the example above we tell Varnish, in the vcl_recv() routine (the very first routine to be called by Varnish) to only deal with GET or HEAD requests. Anything with a different method (a POST, PUT, ...) will be sent to the backend webserver(s) via the return (pass); command, indicating that the request should be passed on to the backend.

Further down, we manipulate the cookies the client has set. This is probably the part where Varnish has thought me the most about the HTTP protocol! As a user, every cookie that is set on a particular domain gets sent to the server for every request. It's a single string, semi-colon delimited. You can change the value of cookies in Varnish, remove or add cookies, etc.


varnish_explained_030

You can also use Varnish to be a "dumb load balancer": it can look at the Host header sent by the user and redirect requests to different backends.

This would allow you to set up a single Varnish server which can send requests to multiple backend servers, depending on the site the user is requesting.

The set req.backend = xxx; code instructs varnish to use a particular backend for that HTTP request.


varnish_explained_031

A common task in Varnish is dealing with Cookies (more tips on that later by the way) and removing cookies for objects or HTTP requests that don't need them.

A typical example is content like CSS, JavaScript, images, ... whether cookies are set or not, chances are that the image won't change. So we remove cookies altogether to determine the cacheability of such an object.


varnish_explained_032

Alternatively, we could also modify the response the webserver has sent us, in case it was a cache miss.

If a webserver wants to set a new cookie, it'll send along one or multiple Set-Cookie headers. In Varnish, we can remove those headers by removing the bereq.http.Set-Cookie header, making it look like the webserver never even sent those requests in the first place!


varnish_explained_033

Another interesting trick is to tell Varnish to never cache a response when it's an server error, something with a status code in the HTTP/500 range. By telling it to reply with return (abandon); it'll bail out that routine and exit the flow.


varnish_explained_034

In the vcl_deliver(); routine you have a final place, right before the HTTP response is sent to your user, to add or remove HTTP headers.

I commonly use this to clean up the request: remove sensitive headers (like Apache or PHP version numbers), add some debug headers to indicate if it was a cache miss or hit, ...


varnish_explained_035

Having so many places to tinker with the request and response can make it hard to visualise where changes were happening and what the outcome would/could be. It can also make you question what's the best place to even begin to manipulate such a request.

To be honest: there's no right or wrong answer here. It depends on your setup. But, there are a couple of tricks to help work with Varnish.


varnish_explained_036

For example, let's run through a request in Varnish and see where/how we can play with it.


varnish_explained_037

In this example, a user wants to connect to our server and load the laravel.com homepage. It sends a request to load the / page (homepage) indicating that it's the Chrome User-Agent.

When Varnish receives that request, it'll start its vcl_recv() routine.


varnish_explained_038

In that vcl_recv routine, we can manipulate the request.

For instance, we can change the request so that it no longer wants to load laravel.com but forge.laravel.com, by setting a new value in the req.http.host object.

Important to know here: the user/client has no idea we're doing this. This is purely internally in Varnish. For the user, it still thinks it's going to receive a response for laravel.com!

Varnish will use that new information (the modified req.http.host and req.http.User-Agent) to check its cache if it has a response it can serve to the client. In this example, we'll assume it was a cache miss.


varnish_explained_039

So now Varnish has to fetch the content from its own backend webserver. But because we modified the request in vcl_recv(), Varnish will ask the backend for the website behind forge.laravel.com, claiming to be an IE5 User-Agent.


varnish_explained_040

The webserver does its thing, and if all goes well it can generate the homeapge for forge.laravel.com.

It'll respond with an HTTP/200 status code, indicating everything was OK. It will also set a PHP session ID, tell that it was generated with the Apache Webserver and indicate that this HTTP response should not be cached, by setting a Cache-Control: No-Cache header.


varnish_explained_041

Varnish receives that response and triggers its routine vcl_backend_response(), where we can modify what the webserver sent us.

Even though the webserver said we should not cache the page (remember: Cache-Control: No-Cache), and Varnish would by default listen to that header and not cache the page, we can overwrite it by setting the beresp.ttl object. The TTL -- or Time To Live -- determines how long we can cache that object.

Additionally, we will overwrite the status code that the webserver sent us (HTTP/200) with one that we invented (HTTP/123).

And finally, we'll remove that Laravel session ID that was generated server side by removing the beresp.http.Set-Cookie header.

After all that fiddling with the HTTP response, Varnish will store the object in its cache using our modified parameters. So it'll store that object as an HTTP/123 response without cookies, even though the server generated an HTTP/200, with cookies and clearly indicated the page shouldn't be cached.


varnish_explained_042

So what eventually gets delivered to the client is a page for forge.laravel.com (our client requested laravel.com) with an HTTP status code of HTTP/123 (the server actually said it was an HTTP/200), without cookies being set.

Such Varnish power. Much amaze. So many places to fuck things up.


varnish_explained_043

There's another important Varnish routine though, it's called vcl_hash(). This is where Varnish decides which parameters it will use to determine its cache/hash key. If the key can match an object already in the cache, it will deliver that object.

By default, Varnish looks at 3 key parameters: the Host-header, the URL being requested and the cookies sent by the client.

Behind the scenes, it will take care of the Content-Encoding that clients can request: it'll store a compressed version of every page in memory, but if a client requests a plain text -- non-compressed -- version, it'll decompress it on the way out and deliver a plain text version to the client.

If any of the parameters inside the vcl_hash() routine changes, Varnish will consider it a new HTTP request and the cache lookup will fail.


varnish_explained_044

If you were thinking ahead, you can see a problem with that kind of hash-key determination.

For instance, the URL /?page=1&languange=nl will show the same content as /?language=nl&page=1, but since the order of the parameters is different, the hash will be different. To resolve this, we normalise the incoming requests as much as possible.

Varnish offers a good tool for this called std.querysort(), which will sort the query parameters alphabetically.

Additionally, you might want to strip query parameters that are irrelevant to the backend, like Google Analytics campaign trackers.


varnish_explained_045

Once you normalised the URL, cookies are the Next Big Thing.

Because they are by default used in the hash-key lookup, any change in the cookies client-side will result in a different hash and a new object in the cache.

If every one of your users has a unique PHP session ID, every Cookie set will be different and thus every hash-key will be unique. Just setting a unique PHP session ID for all your users would effectively destroy caching for your site, as every request is unique.

There are a couple of fixes for this, but the biggest one is to only initialise a PHP session when you need it (ie: on the login page), and not for every page. This is called lazy session initialisation.

Alternatively, some XSRF implementations rely on cookies and should be unique per user too.


varnish_explained_046

Varnish offers an interesting VMOD (extension to Varnish) to help manipulate Cookies.

The cookie.parse object allows to you do interesting things like whitelisting cookies in Varnish, without crazy mind-breaking regular expressions.


varnish_explained_047

In addition, the Host header is also a key in your hashing. So a request for laravel.com and www.laravel.com, even though they're probably the same site, would result in 2 different cached objects in Varnish.

To help Varnish, it's best to redirect users to a single domain and not keep multiple possible domains that would result in the same content being generated.

There are plenty of SEO & technical reasons this is a good move too.

The code example above detects a page without the www-subdomain and triggers a synthetic response in Varnish, via the new vcl_synth() routine.

In that routine, you can detect the resp.status object and create logic handling that differently. In this case, we set a custom error code of 720 in order to replace the Location block in the HTTP response and force a 301 redirect.


varnish_explained_048

That same vcl_hash() can be expended to take any HTTP header into account when determining the cache key. It can add Authorization headers for basic HTTP authentication, custom headers for your API, ...

We also use it at Nucleus to separate cache responses for HTTP vs. HTTPS content, since we often use an Nginx TLS proxy in front of Varnish.


Part Five: Flushing the Cache

varnish_explained_049

There are actually more than 2 hard things in computer science: naming things, cache invalidation, off-by-one errors, DNS, Docker, ...


varnish_explained_050

There are basically 3 ways to flush a cache: the old PURGE HTTP method (which is basically obsoleted), the newer "ban" method (suggested) or the absolute lazy way: restart your varnish service.


varnish_explained_051

The HTTP PURGE method is a "special" HTTP request where you can set the method to PURGE. That in and of itself doesn't actually do anything, but it allows you to catch that special method in varnish's vcl_recv() routine (the very first routine Varnish will call when it receives a request) and trigger an internal cache flush via the return (purge); call.


varnish_explained_052

The preferred way is actually to use "bans": this uses the Varnish Administration CLI tool called varnishadm to connect to a running Varnish daemon on its administrative port and send it control commands.

This syntax gives you more flexibility to flush content based on regex's, either on the host, the URL or arbitrary headers.


Part Six: ESI, TTL's, Grace & Saint

varnish_explained_053

More acronyms! More!


varnish_explained_054

Edge Side Includes are a very powerful but tricky feature of Varnish. By default, Varnish will either cache an entire page, or not cache an entire page.

With ESI, Varnish can cache parts of a page. Funky, right?

In a way, this works very similar to PHP's include or require method or the old Server Side Includes in webservers.


varnish_explained_055

Take for instance a default layout for a news website: it'll contain a personal greeting at the top, some ads, some navigation and the main body.

These different "fragments" that usually build a page are what Varnish needs to be able to do partial caching.


varnish_explained_056

Instead of generating the content for each partial directly in the HTML/DOM of the page, you generate special esi:include tags instead.


varnish_explained_057

Eventually, your server-side generated HTML looks like this. It has several esi:include blocks, where it would normally have either include/require statements from PHP. But instead of having PHP do the combining of the page, those esi:include tags are going to be interpreted by Varnish.


varnish_explained_058

When Varnish processes a vcl_backend_response(), it can look inside the body and detect those esi:include tags.

For each of those ESI tags, it will -- on its own -- send a new request to the backend and request those URLs. The backend will in turn process them, reply them back to Varnish, and Varnish will build the page by glueing all the fragments together.

The biggest benefit is that each fragment can have its own caching policy: in the best case scenario, all those esi:include calls were already in the Varnish cache and it could assemble the page entirely from memory.

The magic of ESI is that the user just requested a single page, it had no idea that it would in turn fire of multiple small requests to the backend to assemble that page before getting it delivered.


varnish_explained_059

Now, by default, Varnish will look at the Cache-Control headers (or ETAG, Expires, ...) to determine how long it can cache an object. In the absolute best case scenario, the HTTP response headers determine the lifetime of each HTTP response.

But you can artificially overwrite that response by setting a new value in the beresp.ttl object.


varnish_explained_060

The TTL or cache lifetime can be modified even further by introducing the concept of Grace Mode.

You can let Varnish keep objects in its cache that have actually surpassed their lifetime. This is useful in case your backends are down; it's better to serve old content, than no content.

It's also of value when there are HTTP requests coming in for expired objects: Varnish can reply with a slightly outdated response, while asynchronously fetching a new version from the backend. Your user never needs to wait for that response (it got a slightly outdated response), while your webserver can generate the response.

This is absolutely a lifesaver in cache stampedes scenario's, where Varnish will only send a single request to the backend, even though 100+ users are requesting the same page.


Bonus: CLI tools

varnish_explained_061

Quick demo: some CLI tools.


varnish_explained_062

varnishhist shows a histogram of the requests currently being served by Varnish: the more to the left they are shown, the faster they have been served. These are generally more cache hits, as they're served the fastest.

The more items to the right, the slower each request is.

A pipe sign | is a cache hit, a hashtag # is a cache miss (can also be a cache miss on purpose, if the VCL code says return (pass);).

varnishhist is a good quick glance to see if Varnish is caching anything and if so, what the overal ratio is: lots of cache hits or misses. But beyond that, the tool probably won't tell you that much.


varnish_explained_063

varnishlog on the other hand is the real gem: it will show in great detail the request headers of a user and the response headers returned by Varnish. If you really need to debug Varnish, this'll be the tool that tells you the most.

There are filters you can apply to only log requests from a certain IP (ie: your source IP), only log requests matching a certain URL pattern, you can see which cookies are being stripped in the VCL and which aren't, etc.

The only downside: varnishlog can be a bit daunting at first with lots of output, it takes a while to grok the format.


varnish_explained_064

Another interesting one, especially when just getting started with Varnish, is varnishncsa: it takes the complicated varnishlog and shows the output just like the Apache or Nginx access logs.

It won't tell you many details (like which cookies get stripped, which VCL routines got triggered etc.), but it will tell you in a more readable format (or at least: a format you're more used to) which requests Varnish is serving.

You can also run varnishncsa as a daemon/service to log all requests made by Varnish in this format, a bit like the default behaviour in Apache or Nginx. For debugging purposes, this could be useful.


Part Seven: getting started with Varnish

varnish_explained_065

Let's get started.


varnish_explained_066

This is the really short version of getting started with Varnish;

  • Download my Varnish templates from Github
  • Install varnish, by default it'll run on a custom port (check /etc/sysconfig/varnish)
  • Configure your backend so that it points to your existing Apache/Nginx
  • Test your site on the custom port: domain.tld:8080
  • If everything still works, swap the ports: set your Apache/Nginx on port :8080, move Varnish to port :80 and change the backends in Varnish so that it sends its requests to port :8080.

After that, you're done: Varnish now serves all requests on port :80 and will proxy everything it can't serve to port :8080.

Chances are, it'll take longer to implement Varnish than just those couple of steps, but that's basically the gist of it. Run Varnish on an alternative port, so that you can test it safely without interfering with your site, and once it's tested and approved, swap out the ports and let it run on port :80.

A couple of tips, that might help you get started quicker;

As with everything: testing, testing & testing is what will make Varnish deployment a success!


varnish_explained_067

The end!

For any questions, leave a comment on this blog, poke me on Twitter, send me an e-mail, ...


Varnish Cache 5.0 Released

This just got posted to the varnish-announce mailing list.

We have just released Varnish Cache 5.0:

http://varnish-cache.org/docs/5.0/whats-new/relnote-5.0.html

This release comes almost exactly 10 years after Varnish Cache 1.0,
and also marks 10 years without any significant security incidents.

Next major release (either 5.1 or 6.0) will happen on March 15 2017.

Enjoy!

PS: Please help fund my Varnish habit: http://phk.freebsd.dk/VML/

---
Poul-Henning Kamp

varnish-announce mailing list

Lots of changes (HTTP/2, Shard director, ban lurker improvements, ...) and info on upgrading to Varnish 5!

I'll be working on updated configs for Varnish 5 (as I did for Varnish 4) as soon as I find some time for it.


Mark a varnish backend as healthy, sick or automatic via CLI

This is a useful little command for when you want to perform maintenance on a Varnish installation and want to dynamically mark backends as healthy or sick via the command line, without restarting or reloading varnish.

See varnish backend health status

To see all backends, there are 2 methods: a debug output and a normalized output.

$ varnishadm -S /etc/varnish/secret -T localhost:6082 backend.list
Backend name                   Refs   Admin      Probe
backend1(127.0.0.1,,80)        1      probe      Sick 0/4
fallback(172.16.80.5,,80)      12     probe      Healthy (no probe)

$ varnishadm -S /etc/varnish/secret -T localhost:6082 debug.health
Backend backend1 is Sick
Current states  good:  0 threshold:  2 window:  4
Average responsetime of good probes: 0.000000
Oldest                                                    Newest
================================================================
---------------------------------------------------------------- Happy

The backend.list shows all backends, even those without a probe (= healtcheck) configured.

The debug.health command will show in-depth statistics on the varnish probes that are being executed, including the IPv4 connect state, whether a send/receive has worked and if the response code was HTTP/200.

For instance, a healthy backend will be shown like this, with each state of the check (IPv4, send, receive & HTTP response code) on a seperate line.

$ varnishadm -S /etc/varnish/secret -T localhost:6082 debug.health
Backend backend1 is Healthy
Current states  good:  5 threshold:  4 window:  5
Average responsetime of good probes: 0.014626
Oldest                                                    Newest
================================================================
4444444444444444444444444444444444444444444444444444444444444444 Good IPv4
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Good Xmit
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR Good Recv
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH Happy

Now, to change backend statuses.

Mark a varnish backend as healthy or sick

In order to mark a particular backend as sick or healthy, thus overriding the probe, you can do so like this.

$ varnishadm -S /etc/varnish/secret -T localhost:6082 backend.set_health backend1 healthy

The above command will mark the backend named backend1 as healthy. Likewise, you can mark a backend as sick to prevent it from getting traffic.

$ varnishadm -S /etc/varnish/secret -T localhost:6082 backend.set_health backend1 sick

If you have multiple Varnish backends and they're configured in a director to load balance traffic, all traffic should gracefully be sent to the other backend(s). (see the examples in mattiasgeniar/varnish-4.0-configuration-templates)

If you mark a backend explicitly as sick, the backend.list output changes and the admin column removes the 'probe' and marks it as 'sick' explicitly, indicating it was changed via CLI.

$ varnishadm -S /etc/varnish/secret -T localhost:6082 backend.list
Backend name                   Refs   Admin      Probe
backend1(127.0.0.1,,80)        1      sick       Sick 0/4
fallback(172.16.80.5,,80)      12     probe      Healthy (no probe)

You can also change it back to let Varnish decide the backend health.

Mark the backend as 'varnish managed', let probes decide the health

To let Varnish decide the health itself, by using it probes, mark the backend to be auto again:

$ varnishadm -S /etc/varnish/secret -T localhost:6082 backend.set_health backend1 auto

So to summarise: the backend.set_healthy command in varnishadm allows you to manipulate the backend health state of varnish backends, overriding the result of a probe.

Useful when you're trying to gracefully update several backend servers, by marking backends as sick one by one without waiting for the probes to discover that backends are sick. This method allows you to do things gracefully before the update.


Varnish Agent: an HTML frontend to manage & monitor your varnish installation

I've been using Varnish for several years, but I only just recently learned of the Varnish Agent. It's a small daemon that can connect to a running Varnish instance to help manipulate it: load new VCLs, see statistics, watch the varnishlog, flush caches, ...

If you're new to Varnish, this is an easier way of getting started than by learning all the CLI tools.

Installing Varnish Agent

The installation is pretty straight forward, assuming you're already using the Varnish repositories.

$ yum install varnish-agent

If you don't have the package available in your repositories, clone the source from varnish/vagent2 on Github and compile it yourself.

After that, start the service and it will bind on port :6085 by default.

$ systemctl start varnish-agent

By default, the web interface is protected by a simple HTTP authentication requiring username + password. Those get randomly generated during the installation and you can find them in /etc/varnish/agent_secret.

$ cat /etc/varnish/agent_secret
varnish:yourpass

After that, browse to $IP:6085, log in and behold Varnish Agent.

What does Varnish Agent look like?

To give you an idea, here's a screenshot of the Varnish agent running on this server.

(As you can see, it's powered by the Bootstrap CSS framework that I also used on this site.)

varnish_agent_demo

A couple of features are worth diving into even further.

Cache invalidation via Varnish Agent

One of the very useful features is that the Varnish Agent offers you a simple form to purge the cache for certain URLs. In Varnish terminology, this is called "banning".

varnish_agent_cache_invalidation

There are limits though: you pass the URL parameters, but you can't (yet?) pass the host. So if you want to ban the URL /index.html, you'll purge it for all the sites on that Varnish instance.

See cache misses

Another useful one is the parsing of varnishtop right in the web frontend.

varnish_agent_varnishtop_cache_misses

It instantly shows you which URLs are being fetched from the backend and are thus cache misses. These are probably the URLs or HTTP calls to focus on and see where cacheability can be improved.

Inline VCL editor

I consider this a very dangerous feature but a lifesaver at the same time: the web frontend allows you to edit the VCL of Varnish and instantly load it in the running Varnish instance (without losing the cache). If you're hit by a sudden traffic spike or need to quickly manipulate the HTTP requests, having the ability directly modify the Varnish VCL is pretty convenient.

Important to know is that the VCL configs aren't persisted on disk: they are passed to the running Varnish instance directly, but restarting the server (or the Varnish service) will cause the default .vcl file to be loaded again.

Varnishstat: statistics, graphs & numbers

The CLI tool varnishstat shows you number of hits/misses, connections per second, ... from the command line. But it isn't very useful to see historical data. That's usually handled by your monitoring system which fetches those datapoints and shows them in a timeline.

The Varnish Agent can parse those numbers and show you a (limited) timeline about how they evolved. It looks like this.

varnish_agent_graphs

The use case is limited, but it helps for a quick glance of the state of your Varnish instance.

Conclusion

While I personally still prefer the command line, I see the benefits of a simple web interface to quickly assess the state of your Varnish instance.

Having a built-in form to perform cache invalidation is useful and prevents having to create your own varnish URL purger.

If you're going to run Varnish Agent, make sure to look into firewalling the Varnish Agent port so only you are allowed access.


Enable Keepalive connections in Nginx Upstream proxy configurations

A very common setup to see nowadays is to have an Nginx SSL proxy in front of a Varnish configuration, that handles all the SSL configurations while Varnish still maintains the caching abilities.

It's one of my "2015 server stack predictions" that held up pretty accurately so far. Although, honestly, this wasn't a hard one to predict.

An Nginx configuration however, by default, only talks HTTP/1 to the upstream (in this exampple a Varnish). Keepalive connections are only supported as of HTTP/1.1, an upgrade to the HTTP protocol. We have to explicitly enable this setting in Nginx so it does keepalive connections to the upstream it's connecting to.

This can greatly reduce the number of new TCP connections in an Nginx SSL setup, as Nginx can now reuse its existing connections (keepalive) per upstream.

To enable Keepalive in Nginx upstream configurations, add the following to your configs. First, modify your upstream definition and add the keepalive parameter.

upstream your_upstream {
  # The keepalive parameter sets the maximum number of idle keepalive connections
  # to upstream servers that are preserved in the cache of each worker process. When
  # this number is exceeded, the least recently used connections are closed.
  keepalive 100;

  server 10.5.1.5:80;
  server 10.5.1.6:80;
  ...
}

Next, enable the HTTP/1.1 protocol in all upstream requests.


location / {
  ...
  proxy_pass             http://your_upstream;
  proxy_read_timeout     300;
  proxy_connect_timeout  300;
  ...

  # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
  proxy_http_version 1.1;

  # Remove the Connection header if the client sends it,
  # it could be "close" to close a keepalive connection
  proxy_set_header Connection "";
}

If your upstream server supports keepalive in its config, Nginx will now reuse existing TCP connections without creating new ones. This can greatly reduce the number of TIME_WAIT TCP connections on a busy SSL server.


Varnish VCL Syntax Highlighting in Sublime Text

I love the Sublime Text editor on Mac. It's super simple & lightweight, it's got plenty of shortcuts and once you get the hang of it -- it's a real power horse.

Since I do quite a bit work in Varnish, writing VCLs and custom rules, syntax highlighting is essential. Here's a plugin that handles that.

$ cd ~/Library/Application\ Support/Sublime\ Text\ 2/Packages/User 
$ wget --no-check-certificate https://raw.githubusercontent.com/mattiasgeniar/Varnish.tmbundle/master/Syntaxes/VCL.tmLanguage

It's a fork from the original work by zephirworks/Varnish.tmbundle, many thanks for that hard work!

varnish_syntax_highlighting

Much better than the plain-text alternative.


Varnish VCL: Case Insensitive Regex

By default, a regex match in Varnish happens case sensitive. If you want to use a case insensitive check, you can use the (?i) flag.

This is a normal, case sensitive regex check:

if (req.http.host ~ "^domain.(be|nl|com)$") {
  ...
}

To make the same check case insensitive, add the (?i) modifier at the beginning.

if (req.http.host ~ "(?i)^domain.(be|nl|com)$") {
  ...
}

Odd as it may be, some people type in their domain names with caps, and the browser does not convert those to lowercase on submit. Though in practice, every Host-header is treated as a lowercase value in all webserver configs.


Varnish Cache 3.0 Is End Of Life

A year after the release of Varnish 4, version 3.0 has been declared end-of-life. (more…)


Running Varnish 4.x on systemd

If you're thinking about running Varnish 4.x on a systemd system, you may be surprised that many of your "older" configs no longer work. (more…)


Debug Varnish 4.x on systemd That Fails to Start

So you're stuck in systemctl start varnish, now what? (more…)


Explicitly Approving (Whitelisting) Cookies in Varnish With Libvmod-Cookie

In all my previous Varnish 3.x configs, I've always used blacklisting as the way of handling Cookies. You explicitly tell which cookies you want to remove in vcl_recv, all others remain. But just as security measures, whitelisting is always better than blacklisting. (more…)


Varnish FetchError http first read error: -1 11 (Resource temporarily unavailable)

Just like the Straight insufficient bytes error, this is another error you can see in your varnishlogs. (more…)


Reload Varnish VCL without losing cache data

You can reload the Varnish VCL configuration without actually restarting Varnish. A restart would stop the varnishd process and start it anew, clearing all the cache it has built up in the meantime. But you can also reload the varnish configurations, to load your new VCL without losing the cache.

Beware though, there are times when you want to clear your cache on VCL changes: for instance, healthchecks on backend definitions can get pretty funky when a reload of the VCL would modify their IPs, and there are situations where when you change the vcl_hash routine, a restart is advised since the data in memory would never be used again (because of a hash-change). Having said that, there are plenty of reasons to reload a Varnish VCL cache without losing the data in memory. (more…)


Combine Apache’s HTTP authentication with X-Forwarded-For IP whitelisting in Varnish

Such a long title for a post. If you want to protect a page or an entire website with HTTP authentication, but also want to whitelist a few fixed IPs (for instance: office or VPN IPs), you can combine both authentication mechanismes in Apache via .htaccess files. (more…)


Varnish tip: see which cookies are being stripped in your VCL

Most Varnish configs contain a lot of logic to strip cookies from the client, to avoid them being sent to the server. This is needed, because cookies are often part of the hash-calculation of a request (if they are included in the vcl_hash routine), and a random cookie would make for a unique hash each time and that can destroy your caching strategy. Here's a tip on how you can see what cookies your client is sending, and what cookies your Varnish VCL is removing and sending on to the backend. (more…)


Varnish FetchError: straight insufficient bytes

In your varnishlog, you may see the following error appearing. (more…)


Varnish FetchError: Gunzip+ESI Failed at the very end

This error can occur in a Varnish setup in multiple forms, but they commonly include the Gunzip + ESI error line. (more…)


Varnish 4.0.0 released together with configuration templates

Good news! Today, Varnish 4.0.0 has been released!. Among the most important features are;

* Full support for streaming objects through from the backend on a cache miss.
Bytes will be sent to 1..n requesting clients as they come in from the backend
server.
* Background (re)fetch of expired objects. On a cache miss where a stale copy
is available, serve the client the stale copy while fetching an updated copy
from the backend in the background.
* New varnishlog query language, allowing automatic grouping of requests when
debugging ESI or a failed backend request. (among much more)
* Comprehensive request timestamp and byte counters.
* Security improvements, including disabling of run-time changes to security
sensitive parameters.

Together with this release, I'm making public the first draft of Varnish 4.0 configuration templates on Github. Since the syntax and internals have changed quite a bit, this is a Work-in-Progress.

My config templates for Varnish 3.x have received a great deal of feedback and attention, I'm hoping to accomplish the same with these 4.x configs. They're still in draft and need finetuning, so I appreciate any feedback!


  • Enable the RPC JSON API with password authentication in Bitcoin Core
  • How to upgrade to the latest Bitcoin Core version
  • I forgot how to manage a server
  • Retrieving the Genesis block in Bitcoin with bitcoin-cli
  • Requesting certificates with Let’s Encrypt’s official certbot client
  • Lazily load below-the-fold images and iframes
  • Using Oh Dear! to keep your Varnish cache warm
  • Podcast: Exploring Bitcoin with Edd Mann
  • The end of Extended Validation certificates
  • Archiving the bitcoin-dev mailing lists
  • Initial impressions on running a Bitcoin Core full node
  • MySQL 8 & Laravel: The server requested authentication method unknown to the client
  • Showing the DNS score in your dashboard & an updated layout
  • Run a Bitcoin Lightning Node on CentOS 7
  • Limit the disk space consumed by Bitcoin Core nodes on Linux
  • Run a Bitcoin Core full node on CentOS 7
  • Remote Code Execution in apt/apt-get
  • Our Gitlab CI pipeline for Laravel applications – Oh Dear! blog
  • PHP 7.3.0 Release Announcement
  • Deploying laravel-websockets with Nginx reverse proxy and supervisord
  • Benchmarking websocket server performance with Artillery
  • Increase the number of open files for jobs managed by supervisord
  • My Laracon EU talk: Minimum Viable Linux
  • Why your NS records matter
  • Tracking SQL queries
  • A big update to DNS Spy – DNS Spy Blog
  • HTTP-over-QUIC will officially become HTTP/3
  • Automatic monitoring of Laravel Forge managed sites with Oh Dear!
  • How to size & scale your Laravel Queues
  • Laravel Telescope: Data too long for column ‘content’
  • The convincing Bitcoin scam e-mail extorting you
  • MySQL 8 removes shorthand for creating user + permissions
  • Find the public key belonging to a private key
  • ssh error: unable to negotiate with IP: no matching cipher found
  • HHVM is Ending PHP Support
  • Indie Hackers interview with Oh Dear!
  • Elasticsearch fails to start: access denied “/etc/elasticsearch/${instance}/scripts”
  • MySQL: Calculate the free space in IBD files
  • Chrome stops showing “www”-prefix in URI
  • Remote Desktop error: CredSSP encryption oracle remediation
  • DNS Spy now checks for the “Null MX”
  • Certificate Transparency logging now mandatory
  • Chalk Talk #3: how to optimise your content for Varnish
  • Upcoming presentation at LOADays: Varnish Internals – Speeding up a site x100
  • Chalk Talk #2: how does Varnish work?
  • Varnish: same hash, different results? Check the Vary header!
  • Varnishlog: show the hash() data in output
  • Drupal core – Highly critical – Remote Code Execution – SA-CORE-2018-002
  • It’s about what broke, not who broke it
  • Why I usually run ‘w’ first when troubleshooting unknown machines
  • Chalk Talk #1: An introduction to Varnish
  • Upcoming Drupal 7 and 8 core highly critical security release: March 28th
  • The security footgun in etcd
  • Enable the slow log in Elastic Search
  • Varnish 6.0 released
  • Laravel & MySQL auto-adding “on update current_timestamp()” to timestamp fields
  • Clear systemd journal
  • Announcing “Oh Dear!”: a new innovative tool to monitor your websites
  • Update a docker container to the latest version
  • Show IDN punycode in Firefox to avoid phishing URLs
  • Nginx adds support for HTTP/2 Server-Side Push
  • Chrome will mark all HTTP sites as “not secure”
  • Red Hat reverts microcode update to mitigate Spectre, refers to hardware vendors for fix
  • Mozilla Firefox to require HTTPS in order to use latest features
  • I’m taking a break from cron.weekly
  • Intel’s CPU vulnerabilities: Meltdown and Spectre
  • “du –max-depth” alternative on Mac OSX
  • Root login without password allowed by default on Mac OSX High Sierra
  • Fall cleaning: shutting down some of my projects
  • Staat der Nederlanden CA might be revoked from Mozilla Policy?
  • Compile PHP from source: error: utf8_mime2text() has new signature
  • KRACK Attacks: Breaking WPA2
  • Get shell in running Docker container
  • Antwerp WordPress User Group offering public speaking course
  • Laravel Forge + Envoyer + Managed Hosting = Nucleus
  • Due to CAA records, unable to issue TLS certs for names in private.cam.ac.uk
  • Why we’ve cancelled our free tier – DNS Spy Blog
  • DNS Research: using SPF to query internal DNS resolvers
  • A proposal for cryptocurrency addresses in DNS
  • Chrome & Firefox now force .dev domains to HTTPS via preloaded HSTS
  • Linux man-pages: man-pages-4.13 is released
  • Cloudflare now serves F-Root instance
  • Laravel Horizon: requires ext-posix, missing from CentOS
  • Presentation: Code Obfuscation, PHP shells & more
  • cron.weekly issue #97: kernel 4.13, TLS, LLVM, Yarn, Vagrant, AWX, Nginx & more
  • Coming soon: Oh Dear! – Monitoring for the encrypted web
  • CAA record checking now mandatory for Certificate Authorities
  • 2 interesting things happened in last cron.weekly’s newsletter
  • Release Notes Driven Development (RNDD)
  • Restore one-click certificate viewing in Chrome
  • Removing root-owned files as a non-root user
  • MariaDB: JSON datatype supported as of 10.2
  • RHEL & CentOS 7.4 restores HTTP/2 functionality on Nginx
  • Podcast: The Ceremony, the birth of Zcash
  • Choose source IP with ping to force ARP refreshes
  • Apache httpd 2.2.15-60: underscores in hostnames are now blocked
  • mysqldump without table locks (MyISAM and InnoDB)
  • Unix time 1.500.000.000
  • Launching the cron.weekly forum
  • Some more nuances to the systemd debacle
  • Podcast: Caching using Varnish
  • Giving perspective on systemd’s “usernames that start with digit get root privileges”-bug
  • MariaDB MaxScale 2.1 defaulting to IPv6
  • HTTP/2 push is tougher than I thought
  • Samba CVE-2017-7494: Remote Code Execution in Samba 3.5.0 and upwards
  • CentOS 7.4 to ship with TLS 1.2 + ALPN
  • (Dutch) Tech45 podcast #341: Technologica & WannaCry ransomware
  • WordPress starts Bug Bounty program on HackerOne
  • Ways in which the WannaCry ransomware could have been much worse
  • Why the Next 10 Days Are Critical to the Internet’s Future
  • Chrome to restore one-click certificate viewing in browser
  • NIST recommendation: remove periodic password change requirements
  • How to enable TLS 1.3 on Nginx
  • Remote security exploit in all 2008+ Intel platforms
  • Are homogenic nameserver names a single point of failure?
  • Top 5 security checks for secure, unhackable web applications
  • cron.weekly issue #77: OpenStack, Moby, Caddy, Devuan, Linuxkit, Tmux, Jenkins & more
  • Canada Just Ruled to Uphold Net Neutrality
  • Follow-up: MIT no longer owns their /8
  • Interview on Laravel Spark & DNS Spy
  • MIT no longer owns 18.0.0.0/8
  • DNS Spy has launched!
  • Nginx might have 33% market share, Apache isn’t falling below 50%
  • CAA checking becomes mandatory for SSL/TLS certificates
  • Intent to Deprecate and Remove: Trust in existing Symantec-issued Certificates
  • cron.weekly issue #72: FreeBSD, lkml, llvm, dnscontrol, buck, ReOpenLDAP, Postfix, Bash, Xargs & more
  • Finding the biggest data (storage) consumers in Zabbix
  • Drupal 7.x Services module unserialize() to Remote Code Execution
  • WordPress on PHP 7.1
  • CVE-2017-2636: Linux local privilege escalation flaw in ‘n_hdlc’
  • cron.weekly issue #70: GitHub, Kompose, Usql, tmux-cssh, HAProxy, icdiff, GlusterFS & more
  • Log all queries in a Laravel CLI command
  • Podcast: Config Management Camp: Kubernetes, Sysdig & Mgmt
  • DNS Spy enters public beta
  • Mitigating PHP’s long standing issue with OPCache leaking sensitive data
  • Cloudbleed: Cloudflare Reverse Proxies have Dumped Uninitialized Memory
  • Announcing the first SHA1 collision
  • Kernel patching with kexec: updating a CentOS 7 kernel without a full reboot
  • Linux kernel: CVE-2017-6074 – local privilege escalation in DCCP
  • Security is Hard: Where Do I Start?
  • HTML & CSS Is Hard: A friendly web development tutorial
  • PHP 7.2 to get modern cryptography into its standard library
  • IETF Draft: Hypertext Transfer Protocol (HTTP) over multicast QUIC
  • Introducing Docker Secrets Management
  • Server-side timings in the Chrome Devtools
  • Review: Ubiquiti’s Amplifi HD, mesh WiFi networking done right?
  • Brotli compression saves 1.5 petabytes per day at Google’s Play Store
  • Flame graphs for file systems
  • cron.weekly issue #66: Git Filesystem, Security, JVM, Fission, Habitat, TLS 1.3
  • Microsoft announces GVFS: Git Virtual File System
  • Stop Disabling SELinux: A Real-World guide
  • Standardising the “URL”
  • htop Explained Visually
  • Implementing “Save For Offline” with Service Workers in the Browser
  • A change of RSS feeds
  • Look before you paste from a website to terminal
  • Chrome 56 Will Aggressively Throttle Background Tabs
  • Return of the Unauthenticated, Unfirewalled protocols
  • Create a SOCKS proxy on a Linux server with SSH to bypass content filters
  • Starting with sponsorships for this blog
  • Despite revoked CA’s, StartCom and WoSign continue to sell certificates
  • Google Infrastructure Security Design Overview
  • WordPress to get secure, cryptographic updates
  • Staying Safe Online – A short guide for non-technical people
  • A collection of Drupal Drush one liners and commands
  • Show the environment variables of a running process in Linux
  • In 2017, I’m going to stop watching the news
  • Mac OSX keeps prompting SSH key passphrase, does not use KeyChain
  • In virtual environments, less (CPU’s) is more
  • Laravel: The only supported ciphers are AES-128-CBC and AES-256-CBC with the correct key lengths
  • Apache HTTP authentication in .htaccess
  • Run Nginx proxy in Docker container for HTTP/2
  • Varnish 5.x VCL configuration templates
  • Pi-Hole: A DNS-based blacklist for ads and tracking for Raspberry Pi
  • Varnish Explained
  • macOS: Convert PDF to PNG files for each page
  • From MailChimp to Sendy: how I saved 600$ a year
  • Heads up: CentOS 5 goes end-of-life in 6 months
  • Varnish Cache 5.0 Released
  • Podcast: Application Security, Cryptography & PHP
  • TCP vulnerability in Linux kernels pre 4.7: CVE-2016-5696
  • youtube-dl: download audio-only files from YouTube on Mac
  • Mark a varnish backend as healthy, sick or automatic via CLI
  • zsh: slow startup for new terminals
  • Docker Cheat Sheet
  • Awk trick: show lines longer than X characters
  • Podcast: Ansible config management & deploying code with James Cammarata
  • Postfix mail queue: deliver e-mail to an alternate address
  • Chrome 52: return old backspace behaviour
  • Google’s QUIC protocol: moving the web from TCP to UDP
  • Enable QUIC protocol in Google Chrome
  • Varnish Agent: an HTML frontend to manage & monitor your varnish installation
  • Why do we automate?
  • A new website layout, focussed on speed and simplicity
  • vsftpd on linux: 500 OOPS: vsftpd: refusing to run with writable root inside chroot()
  • Highly Critical Remote Code Execution patch for Drupal (PSA-2016-001)
  • How To Get Pokémon Go on iPhone Outside US
  • The Bash For Loop, The First Step in Automation on Linux
  • Podcast: curl, libcurl and the future of the web
  • I started a podcast for sysadmins and developers: SysCast
  • Limit the runtime of a cronjob or script
  • The async Puppet pattern
  • Redis: OOM command not allowed when used memory > ‘maxmemory’
  • The day Google Chrome disables HTTP/2 for nearly everyone: May 31st, 2016
  • Podcast: Devops, SSL & HTTP2 op HTTP Café
  • Apple prepares iOS to move to IPv6-only networks
  • Security week: 2x High Severity OpenSSL vulnerability & critical ImageMagick flaw
  • Yum Update: DB_RUNRECOVERY Fatal error, run database recovery
  • Nginx 1.10 brings HTTP/2 support to the stable releases
  • Staying up-to-date on open source announcements & security issues via Twitter
  • Bash on Windows: a hidden bitcoin goldmine?
  • What happens when you run “rm -fr /” on a Linux machine?
  • Video: HTTP/2 for PHP Developers at FOSDEM 2016
  • ssh fatal: Access denied for user by PAM account configuration [preauth]
  • Running Bash on Ubuntu on Windows: Demo
  • Ruby gem search: show all remote versions
  • nginx-1.9.13: no longer retry non-idempotent upstream requests by default
  • certdiff
  • Postfix: cannot update mailbox /var/mail for user, File too large
  • Optimize the size of .PNG images automatically on your webserver with optipng
  • PHP Warning: Zend OPcache can’t be temporary enabled
  • Mac OSX: mtr: unable to get raw sockets
  • Example: set an advanced parameter value in VMware via vSphere PowerCLI
  • Pre-compress content for Apache to avoid compressing on every request
  • Set Access-Control-Allow-Origin (CORS) headers in Apache vhost or htaccess
  • The “.well-known” directory on webservers (aka: RFC 5785)
  • Remote Code Execution in all git versions (client + server) < 2.7.4: CVE-2016-2324, CVE-2016‑2315
  • Drupal’s itok parameters explained and a Varnish workaround for different backends
  • Nginx Proxy: upstream sent too big header while reading response header from upstream
  • A technical guide to SEO
  • OpenSSL vulnerabilities: DROWN attack and CacheBleed
  • MongoDB performance tuning with ‘dex’
  • Critical glibc buffer overflow vulnerability in getaddrinfo() on Linux (CVE-2015-7547 & CVE-2015-5229)
  • Slides: HTTP/2 for PHP developers
  • A clean mailing list browser, focussing on readability
  • Coming soon: SysCast
  • Delete files older than X days in Linux
  • Sort ‘ps’ by memory usage in Linux
  • Get the file or directory owner in Bash for use in scripts on Linux
  • Apache 2.4 AH01762 & AH01760: failed to initialize shm (Shared Memory Segment)
  • Introducing cron.weekly, a new weekly newsletter for Linux sysadmins
  • Review: Google Chromecast 2
  • The best tech question to ask in a job interview
  • Grep: show lines before and after the match in Linux
  • apachectl on Linux: get the entire Apache vhost configuration in one output
  • PHP-FPM: failed to ptrace(PEEKDATA) pid 123: Input/output error (5)
  • Podcast: Delving into HTTP/2
  • Use awk to filter log files on values ‘greater than’ or ‘less than’ a threshold in Linux
  • PHP pear.php.net is using a unsupported protocol – This should never happen.
  • PHP Composer installation fails: file could not be downloaded: allow_url_fopen must be enabled
  • Drush PEAR channel: Error getting channel info from pear.drush.org
  • Linux Date Format: change the date output for scripts or commands
  • Very simple ‘git push’ workflow to deploy code on your own server
  • Identify a physical NIC from the Linux CLI with ‘ethtool’
  • PHP Session Locking: How To Prevent Sessions Blocking in PHP requests
  • MySQL: Convert a table from MyISAM to InnoDB or vica versa
  • How to set iTerm2 as the default ssh:// handler on Mac OSX
  • What is a ‘unikernel’?
  • Querying JSON data at the command line with ‘jq’
  • PHP’s Memcached sessions: Failed to write session data (memcached) for Magento
  • Create a password-protected ZIP file on Mac OSX
  • Chrome drops NPN support for HTTP/2, ALPN only
  • Simulate low-bandwidth conditions with Chrome’s network throttling
  • Auto resize screen size for Windows 10 guest VMs in Virtualbox
  • Nginx: Cannot assign requested address for upstream
  • Linux increase ip_local_port_range TCP port range
  • Enable Keepalive connections in Nginx Upstream proxy configurations
  • The Lie We Live: The Gigabyte That Isn’t
  • Varnish VCL Syntax Highlighting in Sublime Text
  • How To Generate a /etc/passwd password hash via the Command Line on Linux
  • Firefox Nightly starts marking login-forms in HTTP as insecure
  • A Distributed Web
  • 25% of all websites are about to get a REST API
  • The Ping That Makes a Sound
  • The Otto Project: Meet the Successor to Vagrant
  • Boot in single user mode on CentOS 7 / RHEL 7
  • Terminal escape sequences – the new XSS for Linux sysadmins
  • Enable HTTP/2 in Nginx
  • New presentation: HTTP/2: The Next Version of the Internet
  • CentOS 7 NetworkManager Keeps Overwriting /etc/resolv.conf
  • Pretty git log in one line
  • MySQL Back-up: Take a mysqldump with each database in its own SQL File
  • Install Go 1.5 On CentOS 6 and 7
  • Foreman 1.9: ERROR: column hosts.last_freshcheck does not exist
  • Apache 2.4: Unknown Authz provider: ip
  • iPhone’s “Field Test” debug screen: Dial *3001#12345#* for the real signal strength
  • Nginx SSL Certificate Errors: PEM_read_bio_X509_AUX, PEM_read_bio_X509, SSL_CTX_use_PrivateKey_file
  • The Hidden “Refresh” Menu in Chrome when Developer Tools Are Opened
  • Apple’s DYLD_PRINT_TO_FILE vulnerability: from zero to root in 2 seconds
  • Bind/Named Crash: REQUIRE(*name == ((void *)0)) failed, CVE-2015-5477
  • How To Read The SSL Certificate Info From the CLI
  • What happens to a new URL the first 10 minutes on the internet?
  • Block User-Agent in htaccess for Apache Webserver
  • Effectively Using and Detecting The Slowloris HTTP DoS Tool
  • Start or Stop a Service on CentOS 7
  • Enable or Disable Service At Boot on CentOS 7
  • Monitor All HTTP Requests (like TCPdump) On a Linux Server with httpry
  • How To Use A Jumphost in your SSH Client Configurations
  • How To Create A Self-Signed SSL Certificate With OpenSSL
  • supervisor job: spawnerr: can’t find command ‘something’
  • How To Clear PHP’s Opcache
  • How To Take a Screenshot on Your Apple Watch
  • Rsyslog Configuration with Dynamic Log File Destination Based On Program Name
  • Logrotate On RHEL/CentOS 7 Complains About Insecure Permissions on Parent Directory, World Writable
  • How To Increase Amount of Disk inodes in Linux
  • How To Add Secondary IP / Alias On Network Interface in RHEL / CentOS 7
  • Increase/Expand an XFS Filesystem in RHEL 7 / CentOS 7
  • Apache 2.4: ProxyPass (For PHP) Taking Precedence Over Files/FilesMatch In Htaccess
  • Why We’re Still Seeing PHP 5.3 In The Wild (Or: PHP Versions, A History)
  • This American Life: The DevOps Episode
  • Chrome 44 Sending HTTPs Header By Mistake, Breaking (Some) Web Applications
  • Critical Cross-Site Scripting Vulnerability in WordPress < 4.2.2
  • The Worst Possible DevOps Advice
  • My Apple Watch Review
  • Apple Favours IPv6, Gives IPv4 a 25ms Penalty
  • Enable click-to-play for Flash & Java in Chrome, Firefox & Safari
  • Rethinking Security Advisory Severities
  • OpenSSL CVE-2015-1793: Man-in-the-Middle Attack
  • Why Internet Explorer Won’t Allow Cookies On (sub)domains With Underscores
  • Rewriting software from scratch, a lesson learned from Mac OSX 10.10.4’s discoveryd
  • The Broken State of Trust In Root Certificates
  • RFC 7568: SSL 3.0 Is Now Officially Deprecated
  • Increasing Nginx Performance With Thread Pooling
  • How SSH In Windows Server Completely Changes The Game
  • Kickstarter Now Available In Belgium
  • uBlock Origin Now Blocking Access To SourceForge
  • HHVM Performance Lockdown: Is HHVM Reaching Its Limit?
  • Google Jump: VR For The Masses
  • iOS9 Major Feature: Fixing The Shift Key
  • Custom Fonts On Kindle Paperwhite First Generation
  • rtop: Remote System Monitoring Via SSH
  • Bitrot
  • Stunt Hacking: The Sad State of Our Security Industry
  • The Scary Revelation: The Advertising Options On Facebook & Twitter
  • It’s Cheaper To Upgrade An Illegal Windows Version Than Buy A New One
  • Understanding the /bin, /sbin, /usr/bin and /usr/sbin Split
  • Hypertext Transfer Protocol Version 2 (HTTP/2): RFC7540
  • Microsoft Remotely Locking Xbox One’s
  • Linux futex_wait bug
  • Who Came First: The Source Code Or The Compiler?
  • Scan Your WordPress For Security Vulnerabilities With WPScan
  • Under The Hood: Facebook’s Cold Storage System
  • In Defence Of WordPress
  • The (Lack Of?) Durability in SSDs
  • uBlock Origin
  • French Law Forces Backdoors On ISPs
  • Security In Medical Equipment
  • What’s The Value Of Owning A Browser?
  • Using Webserver Access Logs As A Database Storage System
  • 248 days
  • Increase open-files-limit in MariaDB on CentOS 7 with systemd
  • Drupal EngineHack Detection Website
  • Varnish VCL: Case Insensitive Regex
  • Using Zabbix To Notify If Your Site/Domain Makes It To The HackerNews Frontpage
  • Blogging Tip: Send Yourself Blogpost Anniversary Reminders
  • Using JavaScript To Read L3 CPU Cache
  • Magento eCommerce PHP Remote Code Execution
  • Nginx Open Sources TCP Load Balancing
  • Double-clicking On The Web
  • HTTP/1 vs HTTP/2 Page Loading
  • Open Source Puppet 4 Released
  • Taking Netflix’s Vector (Performance Monitoring Tool) For A Spin
  • Nginx HTTP/2 Support Coming Late 2015
  • Nginx Getting JavaScript Scripting Engine
  • Varnish Cache 3.0 Is End Of Life
  • Linux Kernel 4.0
  • When The New TLD .SUCKS
  • When Mailing Lists Turn Into Online Forums
  • Microsoft’s Nano Server & Hyper-V Containers
  • Chrome Version 42 Starts Marking SHA-1 SSL Certificates As Insecure
  • HHVM’s Threading Difference – Not The Same as PHP-FPM
  • The Irony of Random Passwords For Each Service
  • Hosting Superheroes
  • Mozilla Blocking CCNIC’s CA Too
  • Google’s War on China
  • µPuppet & Ensible
  • Obsessive Efficiency Disorder
  • Belgium Leader in IPv6 Adoption
  • When Private Browsing Isn’t Private On iOS: HTML5 And AirPlay
  • Life Without Ops
  • Silly Little IP Tricks
  • OpenSSL CVE-2015-0291 and CVE-2015-0286
  • Forthcoming OpenSSL releases
  • Running Varnish 4.x on systemd
  • Debug Varnish 4.x on systemd That Fails to Start
  • Virtual Reality As The Next Step After Reponsive Webdesign?
  • Drupal engine_ssid_ And engine_ssl_ cookies: You’ve Been Hacked
  • Bundling Software Installs With Adware
  • Up And Close With PHP 7’s New RFCs
  • Firefox 36 Fully Supports HTTP/2 Standard
  • Tearing Down Lenovo’s Superfish Statement
  • HTTP/2 Specification Is Final
  • Async MySQL Calls in HHVM 3.6
  • Giving HTTPie a Chance
  • Three Tiers of Package Managers
  • Service Side Push in HTTP/2 With nghttp2
  • Puppet: clearing all or a particular node from PuppetDB (exported resources)
  • FCC: Net Neutrality in the United States
  • Check if Value is Present in Array in Puppet
  • Deep Insights: The Kernel Boot Process
  • Fosdem 2015 Notes
  • Ntimed: an NTPD replacement
  • What’s New in systemd, 2015 Edition
  • Live Migrations for Containers with CRIU
  • Docker Storage Performance Tests
  • Running a Tor relay: lessons learned
  • My Fosdem 2015 Schedule
  • Quick tests for GHOST gethostbyname () vulnerability (CVE-2015-0235)
  • Modeling Package Manager Dependencies In Config Management
  • GHOST: critical glibc update (CVE-2015-0235) in gethostbyname() calls
  • The PHP Paradox
  • PHP Benelux 2015 Unconference Schedule Saturday
  • PHP Benelux 2015: Day One
  • Meet Microsoft’s Project Spartan, The New IE6
  • Security Panel Lands In Firefox 37
  • Nginx sets HTTP 200 OK on PHP-FPM parse errors
  • The Frontpage of Hacker News: Stats, Graphs & Some Analysis
  • PHP 6: The Missing Version Number
  • A good week for this blog!
  • PHP7 To Remove Deprecated Functionality
  • Learning systemd
  • Does SDPY (or HTTP/2) Actually Help?
  • Enable HTTP/2 support in Chrome
  • Audi’s Cruise Control “>=” Bug
  • View the HTTP/SPDY/HTTP2 Protocol in Google Chrome
  • Recent OpenSSL Security Advisories Are a Good Thing
  • Architecting Websites For The HTTP/2 Era
  • Flush all the content from Memcached via the CLI
  • Making Standard Resources Overwritable In Your Own Puppet Modules
  • PHP’s CVE vulnerabilities are irrelevant
  • The (unexpected?) workload associated with migrating to HHVM
  • The 2014 Blog in Numbers
  • Compiling from source faster: multi-threading in `make`
  • The Infinite File Downloader
  • Undoing Years of Optimisations With HTTP/2
  • Testing Puppet Resources at the CLI
  • The Surprising Mixed Content Handling on SSL/HTTPS Enabled Websites
  • MongoDB startup: /usr/bin/dirname: extra operand ‘2>&1.pid’
  • Replacing Software Stacks Is Never The Solution
  • Explicitly Approving (Whitelisting) Cookies in Varnish With Libvmod-Cookie
  • List The Files In A Yum/RPM Package
  • Setting HTTPS $_SERVER variables in PHP-FPM with Nginx
  • Azure Cloud Outage Root Cause Analysis
  • Nginx & SPDY error: net::ERR_SPDY_PROTOCOL_ERROR
  • PHP’s OPcache and Symlink-based Deploys
  • Force Redirect From HTTP to HTTPs On A Custom Port in Nginx
  • Enable SPDY in Nginx on CentOS 6
  • Presentation: Managing Zabbix Hosts with Puppet’s Config Management
  • Varnish FetchError http first read error: -1 11 (Resource temporarily unavailable)
  • Reload Varnish VCL without losing cache data
  • Combine Apache’s HTTP authentication with X-Forwarded-For IP whitelisting in Varnish
  • Chrome To Explicitly Mark HTTP Connections As Non-Secure
  • Varnish tip: see which cookies are being stripped in your VCL
  • Generate PHP core dumps on segfaults in PHP-FPM
  • The Real Cost of the “S” in HTTPS
  • Varnish FetchError: straight insufficient bytes
  • PHP 5.5 Opcache Settings
  • Apache’s mod_fastcgi and mod_deflate troubles
  • A plea for backwards compatibility breaks in PHP7
  • Asynchronous PHP
  • Puppet: could not prefetch yumrepo provider ‘inifile’: Section is already defined, cannot redefine
  • Automating the Unknown
  • On Timing Attacks in PHP
  • The Next-Generation PHP Server Stack For 2015
  • Devuan, the Debian Fork
  • Drupal 7.33 performance penalty
  • Rebuilding the yum RPM database
  • Search yum for the content of a package
  • StartDBSyncers in Zabbix
  • Test Gzip Compression of Site via Curl
  • Varnish FetchError: Gunzip+ESI Failed at the very end
  • Debugging Performance Problems With Zabbix Internal Items
  • Remove Orphaned Data From Zabbix’s MySQL Tables
  • Remote Code Execution via ‘less’ on Linux Boxes
  • Presentation: DNSSEC, The Good, The Bad & The Secure
  • Presentation: Mobile Zabbix, Why Mobile Matters (MoZBX)
  • CPU Flame Graphs
  • Enable MySQL’s slow query log without a restart
  • The PHP circle: from Apache to Nginx and back
  • Yet Another Microsoft Windows CVE: Local Privilege Escalation MS14-068
  • Make HTTPerf use a proxy for connections
  • A Certificate Authority to Encrypt the Entire Web
  • Follow-up: 3 years of automation with Puppet
  • REST API best practices and versioning
  • The Chocolatey Kickstarter: Making Windows More Like Linux
  • Remove a single iptables rule
  • 3 Years of Puppet Config Management: lessons learned
  • Clear APC cache in PHP
  • Running Kali Linux as a Vagrant Box (virtual machine)
  • Microsofts Open Source Strategy
  • Benchmarking the performance of ‘Wordfence’, a WordPress plugin
  • Slow login/DNS via Active Directory on OSX Mavericks and Yosemite (.local domains)
  • Microsoft SSL/TLS vulnerability MS14-066
  • Firefox Developer Edition released
  • A collection of PHP exploit scripts
  • Behind the scenes at World of Warcraft: the datacenter tech
  • Patch your webservers for the SSLv3 POODLE vulnerability (CVE­-2014­-3566)
  • DNSSEC: NSEC3 iterations too big for weakest DNSKEY strength
  • You Need Passion
  • Compress a PDF file on Linux via the CLI
  • Sysdig CLI examples
  • HHVM versus PHP-FPM 5.4 vs PHP-FPM 5.5: performance comparison
  • HTTPd: Cannot load mod_status.so into server: undefined symbol: ap_copy_scoreboard_worker
  • PHP-FPM environment variables are limited to 1024 chars
  • CVE-2014-0185: PHP-FPM sockets unavailable after updating PHP
  • OpenSSL: validate that certificate matches / signs the private key
  • Whois at the CLI: get all IP ranges from an AS number
  • Debugging HTTP requests to PHP via the CLI
  • Follow-up: use ondemand PHP-FPM masters using systemd
  • Varnish 4.0.0 released together with configuration templates
  • Scan your network for Heartbleed vulnerabilities with Nmap
  • A better way to run PHP-FPM
  • Patch against the heartbleed OpenSSL bug (CVE-2014-0160)
  • Presentation: Code Obfuscation, PHP shells & more: what hackers do once they get passed your code
  • What if you are the network admin tasked with blocking IPs in Turkey?
  • Puppet performance troubleshooting: using the built-in profiler in standalone puppet apply
  • Puppet: Error 400 on SERVER ArgumentError: malformed format string – %S at …
  • Puppet: Error: Could not retrieve catalog from remote server: Error 400 on SERVER: stack level too deep on node something.pp
  • Taking a MySQL-dump with a wildcard on table names
  • Read all keys & values from a Memcached instance
  • Setting custom puppet facts from within your Vagrantfile
  • Nginx: nginx: [warn] load balancing method redefined
  • CentOS / RHEL: configure: error: C++ compiler g++ does not work or no compiler found
  • RVM: Installing Ruby 1.8.7 on Mac OSX 10.8
  • Gearman Queue error: mysql_stmt_execute failed: Unknown prepared statement handler (1)
  • Stop. Caching. Static. Files. In. Varnish.
  • Public jPackage Repository Mirror
  • Disconnecting
  • Using telnet from the VMware 5.x ESXi shell
  • Now stop using dsbl.org blacklisting: it’s offline
  • e2fsck: Error allocating directory block array: Memory allocation failed
  • Facter/Puppet: Could not retrieve selinux: Invalid argument – /proc/self/attr/current
  • Prevent cronjobs from overlapping in Linux
  • Reinstall the Linux Kernel on CentOS or RHEL
  • My thoughts on mobile webdesign
  • CentOS 6: /usr/sbin/qpidd eating up all CPU (at 100%)
  • A fix for the Java Leap Second bug
  • Running php-cgi scripts via the CLI as a webserver would (by faking them)
  • How to install and use the EPEL repository for CentOS 5 and CentOS 6
  • Puppet + Razor: pxe boot failing to load with ‘Operating system not found’
  • MySQL: SHOW FUNCTION STATUS WHERE Db = ‘name’: Cannot load from mysql.proc. The table is probably corrupted
  • Install VMware Tools via the Yum Package Manager on RHEL and CentOS
  • Dependency errors for git installs on CentOS with EPEL repo
  • The insanity of URL shorteners and tracking
  • Dependency fail on php-api when installing php-mcrypt or php-mhash from EPEL on CentOS 6
  • How to change the reserved blocks on EXT3 or EXT4 filesystem in Linux
  • Apache + PHP on Ubuntu: apache2filter does not support gzip
  • Loading the 3ware/LSI 9650SE drivers into VMware ESXi
  • Re-enabling IPv6 support on CentOS kernels after update
  • Autoloading MIB files in client SNMP configuration on Linux
  • Upgrade your Ubuntu Desktop 10.04 LTS to 12.04 LTS
  • MySQL error: table is marked as crashed and should be repaired
  • Zabbix: debugging ‘Lock wait timeout exceeded; try restarting transaction’ on the ‘ids’ table
  • Importing the Puppet Learning VM into VirtualBox: unknown resource type in hardware item
  • Bad ORM is infinitely worse than bad SQL
  • Nginx and php-fpm: upstream timed out / failed (110: Connection timed out or reset by peer) while reading
  • PHP execution via passthru(), exec(), shell_exec(), …: sh: /php: No such file or directory
  • Rescan disk devices on a Linux VMware virtual machine without a reboot
  • What are the ‘encrypted_search_terms’ hits on your website/wordpress blog?
  • OpenVZ: vzctl enter gives “Unable to open pty: No such file or directory”
  • yum update on OpenVZ container: error: cannot open Packages database in /%{_dbpath}
  • Enable IPv6 connectivity for Nginx
  • Remove a single mail from the postfix queue on Linux
  • Xapian Bindings 1.2.7/8 in PHP: undefined symbol: zend_error_noreturn in Unknown on line
  • SSH logins or rsync’s without using a password prompt
  • Using ssh-copy-id on an alternative SSH destination port
  • Incoming bandwidth limitation on OpenVZ with CentOS 6.x running on VMware ESX(i)
  • Installing VMware Tools on a Debian 6 Linux Virtual Machine
  • Checking the version of VMware Tools inside a Linux Virtual Machine on ESX(i)
  • Removing a package without its dependencies in CentOS or RHEL
  • On removing users with %postun in RPM SPEC files
  • Nginx: password protect a directory
  • The hidden images within PHP
  • Changing interface back to eth0 from eth1 in Linux (CentOS/RHEL)
  • Setting up a catch-all e-mail account on Linux using Postfix
  • Hide PHP errors and log them to a file or syslog (php-fpm or mod_php)
  • A vim configuration for the PHP developer
  • PHP: php_value vs php_admin_value and the use of php_flag explained
  • Letting memcached only listen on localhost on CentOS/RHEL
  • Enable syntax highlighting in VIM for PHP-FPM configurations
  • The bugged sysinfo() call in OpenVZ hosts running Kernel 2.6.32+
  • Blocking Googlebot/MSNbot/Bingbot in Apache configuration from accessing your server/site
  • Add ‘Options FollowSymLinks’ and avoid ‘Options SymLinksIfOwnerMatch’ in Apache to save disk I/O
  • Avoid ‘AllowOverride All’ in Apache to limit disk I/O access
  • RHEL 6/ CentOS 6: Slow SSH logins with DNS timeouts
  • The infamous /w00tw00t.at.ISC.SANS.DFind GET requests in your access logs
  • Linux application/script debugging with ‘strace’
  • WordPress plugin: undo the default WordPress formatting (single/double quotes/dashes)
  • Plesk: Postfix & authenticated SMTP on an alternative port – login failed, generic failure
  • WordPress 3.3: Stop replacing double dashes and single quotes
  • 2011: An overview of projects & popular blogposts
  • Ubuntu 9.10 with VMware Convertor: GrubInstaller failed – /vmware-updateGrub.sh: 38: grub: not found
  • The results after 1 week Duck Duck Go: I’m switching back to Google.
  • Plesk & DrWeb: “read error” on e-mails being scanned
  • Porting standard Apache’s mod_rewrite rules to Nginx
  • Enabling and disabling PHP’s APC cache on a per Virtual Host level in Apache
  • Apache Access Log: don’t log static content
  • Nginx Access Log: log the real user’s IP instead of the proxy
  • Plesk: passing the authorization headers in PHP when running with FastCGI/mod_fcgid
  • Zabbix: monitor a TCP port with the Zabbix Agent
  • Ground rules for when compiling applications from source
  • Varnish running in foreground but fails to run as service/daemon
  • Varnish: filter by Source IP using Varnishlog (in Varnish 2.x, 3.x and 4.x)
  • Varnish: varnishhist/varnishtop: error while loading shared libraries: libvarnishapi.so.1: cannot open shared object file
  • Nginx returning blank white page on PHP parsed with FastCGI or PHP-FPM
  • RHEL6 and CentOS 6: missing libmcrypt and libmhash in default repository
  • Fixing MySQL master-slave replication upon query error
  • MySQL purge the binary logs from replication (mysql-bin.xxxx files)
  • Plesk: show all FTP accounts and passwords
  • Plesk: listing all the database account info (dbname/username/password)
  • Plesk: listing all the mailbox account info (username/password)
  • VMWare ESXi 4.1 Update 2: warning: syslog not configured
  • Changing the time and timezone settings on CentOS or RHEL
  • Akismet letting you down? Try a simple math question.
  • Zabbix: Can’t recreate shared memory for collector. [too many attempts]
  • Zabbix: zabbix_agentd: Can’t recreate Zabbix semaphores for IPC key 0x123456 Semaphore ID 123456. Operation not permitted.
  • This is what Open Source is about: Mobile Zabbix used on Hospital Ship
  • Post-it wars at the office: Mario & Princess Peach
  • 4 days without an iPhone
  • Plesk: Unable to validate password policy: (1753) There are no more endpoints available from the endpoint mapper.
  • Pukkelpop: overzicht van de chaos
  • Find in Bash scripting: paths must precede expression
  • Postfix in Plesk: Out: 451 4.3.0 Error: queue file write error
  • Latest tool: QuickDiff – visually showing the differences between text
  • MySQL: table is read-only
  • Windows Virtuozzo: unable to copy registry key on container start
  • Installing the PECL/PEAR mailparse module for PHP on CentOS
  • Finished a draft Varnish 3.0 VCL for Fork CMS
  • Checking for MySQL optimizations via scripts
  • Posting my Varnish 3.0 configs to Github
  • Useful Varnish 3.0 Commands: one-liners with varnishtop and varnishlog
  • Add an existing user to an existing group in Linux
  • VMware ESX(i) & Windows Server 2008: Updated driver for sluggish console access
  • Bandwidth limitations in Belgium: oh sigh.
  • Plesk Password retrieval via the command line (Linux / Windows)
  • Fixing Listen IPs In IIS (The network location cannot be reached)
  • Starting a photo blog
  • The Kindle (e-Reader): your best travel companion
  • Getting verbose debug info on nss_ldap in CentOS (LDAP)
  • Amazon EC² downtime: a collection of horror stories
  • Guide: running NginX 1.0 with PHP-FPM 5.3 on CentOS 5.x
  • Using Perl regex for faster grep
  • WikiLeaks Mirror Hosting Page now offline
  • Linux: preserving color output on match when piping grep to more/less
  • How is facebook tracking our (outgoing) URLs?
  • Rebuilding your RPM package database: fixing RPM/yum problems
  • My “Real Security in a Virtual Environment” Presentation
  • Recovering from a DRBD split-brain scenario in heartbeat
  • The “SHOW PROCESSLIST;” equivalent in Microsoft SQL-Server
  • VMWare Converter: vmodl.MethodFault (CloneVolume Error)
  • Firefox extension: the ‘Shortlink Revealer’ (bit.ly exposer – Proof of Concept)
  • Getting started with Git: linkpost
  • CentOS, Git & http-push unavailable (git not compiled with USE_CURL_MULTI)
  • Profiling: Learning from Engadget’s infrastructure on traffic spike handling (ie: iPad2 launch)
  • CentOS, Apache & mod_fcgid: IPCCommTimeout not working as expected
  • There’s more to the php.net site than meets the eye
  • Building your own kernel based on CentOS, ‘switchroot: mount failed. Kernel Panic.’
  • Resizing a DRBD Resource Built On A LVM
  • Compiling PHP on a 64bit CentOS installation
  • Silly Yum Tricks: whatprovides, groups & repolist
  • CentOS, Heartbeat, DRBD & NFS: umount failed, device is busy
  • Plesk throws “A mandatory extension policy in the request is not accepted by the server for this resource.”
  • PHPBenelux 2011: Conference Schedule
  • Running phpMyAdmin with APC cache enabled, fatal PHP errors
  • Net Neutrality: The real internet backbone
  • 2010 in numbers: statistics, statistics & statistics
  • MySQL – myisamchk: error: myisam_sort_buffer_size is too small
  • Compile: /usr/bin/ld: cannot find -lltdl, collect2: ld returned 1 exit status
  • WikiLeaks’ mirrors: where are they hosted?
  • Tracking root cause of Apache spawned processes: shell scripts / daemons
  • MySQL: “Table ‘mysql.plugin’ doesn’t exist” after MySQL Upgrade
  • Df command in Linux not updating actual diskspace, wrong data
  • Introducing the ‘Decoder, Encoder & Obfuscator’
  • vCenter Converter: Shrink Disk Size of EXT4 System Not Supported
  • Plesk 10 Update: Update/Autoinstaller Port Change & Plesk Panel Port Redirection on Virtuozzo
  • Introducing the Extended Mail Header Parser
  • A custom init.d start-up script for HAProxy: start, stop, restart, reload, checkconfig
  • Enabling json_encode And json_decode in PHP < 5.2.0 (JSON)
  • Advanced Monitoring Of HAProxy With Zabbix Agent
  • Let Plesk Create Zend Directories Whenever A Physical Hosting (domain) Is Added
  • Maximizing PHP’s Mail Throughput: To Fork Or Not To Fork?
  • MoZBX: The Mobile Zabbix Client
  • My BarCamp Antwerp Presentation: “High Availability in IT = AAARGH”
  • Compile HAProxy With TPROXY Support
  • Compile a (CentOS) Kernel And IPTables With TPROXY Support
  • Increase A VMware Disk Size (VMDK) Formatted As Linux LVM without rebooting
  • Compiling, Running and Cursing on Chromium OS – Getting It To Work
  • Exclude Local Networks Via Juniper NetScreen-Remote VPN
  • WordPress v3.1, v3.2: Don’t Replace (Double & Single) Dashes & (Double & Single) Quotes!
  • Chromium OS: error: RPC failed; result=22, HTTP code = 502
  • VMware: internal error – vmodl.fault.HostCommunication
  • Facebook’s Techtalks Worth Watching
  • MySQL Upgrade To 5.1: Database Name Prefix #mysql50#
  • Deploying Undersea Cables: Laying The Internet’s Backbone
  • System Administrator Appreciation Day 2010 @ Nucleus
  • phpsh: Running An Interactive PHP Shell (Python)
  • Implementing & Maintaining DNSSEC On Bind9 Nameservers
  • How, What and Why – On Scalability, Availability & Manageability (Facebook)
  • Selling A Classic Mini British Open (Special Edition)
  • TCP Traffic Redirection on Windows
  • How IPv6 Headers Are Formed, Compared To IPv4
  • IPv6 And Security: What You Probably Don’t Know
  • Stateless Autoconfiguration To Replace DHCP For Some Systems
  • Don’t Upgrade OpenSSL If You’re Using Plesk (= Broken Controlpanel)
  • Neighbor Discovery (ND) To Replace ARP In IPv6
  • IPv6 Address Space & Reservations
  • Address Types: Unicast, Multicast & Anycast
  • Address Notations In IPv6
  • Recycling older IPv6-related posts
  • “Show Full Processlist;” Equivalent Of MySQL For PostgreSQL
  • Top 25 Most Dangerous Programming Errors
  • BackTrack 4 Final Release
  • NetBIOS exploiting
  • TEDx Brussels: My Box Is On Fire
  • TEDx Brussels: The Groundrules
  • MetaData: I’ll Bet You Thought That Was Private?
  • Using rsync over cp: don’t we just love progressbars?
  • Define Happiness: The Act Of Making Someone Else Happy
  • System Administrator Appreciation Day @ Nucleus = Win
  • Why We Deserve Our System Administration Appreciation Day
  • How To Change A Server’s Main IP Address (including DNS records) In Plesk
  • “See Pricing”, Your Marketing Tip Of The Week!
  • “Schools Kill Creativity”, by Ken Robinson
  • Get It While It’s Hot: FireFox 3.5
  • Slowloris HTTP DoS: Be Afraid, Be Very Afraid
  • ‘We’ll Solve Clickjacking By 2017’
  • “Are we in control of our own decisions?”
  • Running IE6, IE7, IE8, FF 2, FF 3, Chrome, Opera & Safari
  • Windows 7 Release Candidate Available For Download
  • Webcast: Database Sharding at Netlog with MySQL and PHP
  • Creating Benjamin Button’s Face Through CGI
  • What The Pirate Bay’s Guilty Verdict Really Means
  • Case Insensitive Table And Column Names In MySQL
  • Compiling PHP With “–with-jpeg-dir” Not Working?
  • The Importance Of The URL In SEO
  • Starting The IP:v6::guide – IPv6 Knowledge
  • On The Subject Of Naming Variables
  • Connect SQL Server Management Studio Express To Alternate TCP Port
  • ‘php_admin_value’ Bug When Overwriting PHP Settings
  • Reverse Shell Implementation in PHP5
  • PowerShell: Windows Scripting Made Fun Again!
  • FTP & SELinux: 500 OOPS: cannot change directory
  • Update A Specific Package With apt-get
  • Input Validation: Using filter_var() Over Regular Expressions
  • There’s No Such Thing As Unlimited
  • I’m Attending MIT, Stanford & Harvard
  • PEAR:: Classes Worthy Of Attention
  • Install New PEAR Packages From The Linux CLI
  • Error Handling In PHP’s PDO
  • Method Chaining In PHP – Fluent Interface
  • Exception Handling In PHP5 – Good Cop, Bad Cop
  • Using Plesk’s SMTP Server: DNS Blacklist Prevents Sending
  • Save Bandwidth – Enable GZip Compression In PHP
  • Get Rid Of The ‘WWW’ Prefix – It’s Long Overdue
  • Permanent Failure – Probe failed: Illegal To: address (invalid domain name)
  • Google’s Chrome Won’t Hurt IE – Only FireFox
  • Plesk & MailEnable – missing measp.dll component: Error ‘ASP 0177 : 800401f3’
  • Plesk: Unable to create system user: Empty error message from utility.
  • Named/Bind: Could not listen on UDP socket : permission denied
  • Code Quality & Code Requirements
  • URL file-access is disabled in the server configuration
  • System Calls In Apache (Linux) vs IIS (Windows)
  • Plesk Site Preview Not Working
  • How Do I Reset My Admin Password Of Plesk (Linux CLI)
  • Securing The Internet – Google’s Obfuscated TCP
  • You No Take Brains!
  • Limiting Resources: nice & ionice
  • MySQL: #1054 – Unknown column ‘table.columnname’ in ‘on clause’ Even Though Column Name Exists
  • How To Mount A USB Hard Disk Through Command Line (Linux)
  • Widgets, Gadgets, … You Can Keep’m!
  • Domains In Plesk Showing No Disk Usage Or Bandwidth
  • How To Reset A (Administrator) Password On A Windows Server 2003
  • Web Seminars – The Good And The Bad
  • Corrupted dBase Bug (DBF) When Using DATE-Field Type
  • Importance Of Hosting & Location In SEO
  • Reading A .FLV File From Flash – Not Working Because Of Missing MIME Type
  • Fennec Alpha – Just Like The Other Fox
  • Hardware Testing: Randomly Play “Fur Elise”
  • Australia To Become The New China – Internet Censorship
  • Making The Switch To Linux – Keep In Mind … (10 Ubuntu Tips)
  • Very Handy Linux CLI Cheat Sheet
  • Explaining Port Forwarding & Routing To A Novice
  • We Don’t Have To Talk In Riddles
  • Plesk Issues After Upgrade (Horde Webmail Showing Default Plesk Page)
  • How The Internet Works – Peering & Transit
  • Using Google As Your Dictionary?
  • How To Find Out Your Current Linux Distribution
  • How To Allow Empty Passwords In Remote Desktop (RDP – WinXP/2000/2003)
  • How To Bypass Windows 98’s Security System
  • Restyling – For Better Or For Worse
  • Prevent Thumbs.db Files From Being Created And Stored
  • Comics Related To IT/Programming
  • When A RegEx Goes Bad – For Cisco
  • Little Coding Habits
  • ISP: It’s Impossible For Us to Stop Illegal P2P
  • Top Tools For Penetration Testing (Security Analysis/Hacking)
  • “The service could not bind instance 1. The data is the error code.”
  • “If It Ain’t Broken, Don’t Fix It”
  • Install GD Library For PHP5 On CentOS
  • Demystifying the “duplicate content penalty” – SEO Update By Google
  • Enable DNS Blackhole Lists & SPF Checks In Plesk’s Spamassassin
  • How To Compile And Install PHP Extensions From Source
  • Projects Page (Finally) Updated
  • We Live To Die Another Day
  • Memory Leak In Google Analytics – Blame Canada?
  • A Recipe For Disaster: XSS, Google-Analytics.com And DNS Cache Poisoning
  • Simple Apache Security Trick – ServerTokens & ServerSignature
  • It’s The End Of The World As We Know It
  • Google’s Chrome EULA Altered – Less Evilness
  • Easiest Way To Create A MySQL Database Copy
  • Google Chrome And Its End-User License Agreement (EULA)
  • It’s Bling-Bling Time – Let’s Talk Chrome
  • Making Full Use Of ‘Dig’ In Linux
  • How To Monitor Your Bandwidth Usage (Upstream/Downstream) In Windows
  • How To Prevent Server Access By Hostname/Ip/Mac/…. (Windows)
  • How DNS.BE punishes the end-user
  • Microsoft .NET Framework 3.0 Update (kb928416) On x64 Server
  • How To View E-Mail Headers In Outlook 2007
  • Take Over Your Mobile Phone From Your Desktop PC
  • There Are HTTP Headers, And Then There Are HTTP Headers
  • Multiple Headers With Same Name Causing Errors In IIS
  • You’re Not A Hygiene Manager …
  • Why Is Mail Being Blocked By A Spamfilter?
  • It’s Linux Tutorial Time!
  • “_gat is not defined” Google Analytics Error
  • Enable showing errors in PHP5
  • The Right SQL User, For The Right Job
  • 6 Hints To Pimp Your Webforms’ Style
  • New Boyscouts Website Launched
  • Setting Maximum Width For Image In IE6, IE7 & Firefox
  • All Your Base Are Belong To Us
  • How To Enable PEAR Packages In Plesk
  • Why Programming Is Less Rewarding Than Designing
  • Get Upgraded, Or Get Lost (For IE6 Support)
  • Using Conditional CSS Stylesheets For Easier Maintenance
  • 404 Week At Google’s Blog, Time To Fix Your Headers!
  • How To Identify Hidden Processes In Windows (Rootkits)
  • Is Your GMail Notifier Suddenly Broken?
  • Common Security Flaws In PHP Applications
  • Using GoogleBot’s User Agent In Your Advantage
  • How To Identify The Bad Processes On A Hacked Linux Box
  • DNS Poisoning Attack, How Safe Am I?
  • Remote Code Execution Through Intel CPU Bugs
  • Using Proper Header Redirects In PHP
  • Plesk Creation Of Mailbox Failed Because Alias Once Existed
  • Converting Windows 2008 Server To A Workstation
  • Generic Syntax Highlighting For PHP
  • NsLookup Is More Powerful Than You Think
  • Controlling the San Fransisco Computer Network – The Powers Of A SysAdmin
  • How We Save The Day, But Receive No Glory For It
  • Installing SQL 2000 and SQL 2005 on the same server
  • Domain theft at a new height
  • Telenet raises monthly traffic limits, but still fails
  • Stress test your windows server
  • The iPhone in Belgium and the debate of tying sales
  • Keeping control of airline passengers
  • How not to sue a BitTorrent tracker (TPB)
  • IPv4 and IPv6 network charts
  • Why to avoid URLS ending in .0 for Google search-results
  • Changing network settings through the windows command line
  • Update: seo analyzer v0.3
  • Ajax’s death struggle?
  • Some must-have cheat papers for designers & programmers
  • “Net neutrality” at a glance
  • Beta-launch: seo analyzer V0.2
  • Why do movie release dates suck?
  • Learn from the best at Google Code University
  • Quick math to show how terrible Belgian bandwidth providers are
  • A technical look at Myspace’s growth
  • Overview of Windows User Accounts (including for IIS)
  • A look at the future of cars, by BMW
  • Female sourcecode better than male?
  • Some terrible (truely terrible) captcha’s
  • Grandfather builds webbrowser for autistic boy
  • Alternative movie titles/screens
  • A look at the Olympic Datacenter
  • Repairing qmail mailboxes (with Plesk 8.x)
  • It doesn’t take words to be cruel
  • Germany to invade your home (computer)
  • Creating excel files through PHP
  • Manage the Qmail mail queue
  • Screen? A must for SSH
  • ThePlanet datacenter nog steeds offline
  • Google opens up
  • Pimp my S710
  • And all the UPS’ in the world …
  • Yarr!