This is the Ops way to do it.
A few days ago, an article on Twilio made it to the HackerNews frontpage. Its goal was to get a notification whenever their site was promoted to the HN frontpage, so they could prepare/react on the incoming traffic spike.
Cool blog post. But I’ve got a much easier solution. Here’s the Ops way to deal with it.
Zabbix + Webchecks = <3
I’m an avid Zabbix user. It’s an advanced monitoring tool we use to keep our servers in check.
One of the most powerful features of Zabbix are their webchecks. It allows you to configure a URL, an update interval, an expected HTTP status code and string in the source code.
If either the HTTP status code is wrong or the checkstring is missing, Zabbix considers the webcheck as failed and can trigger an action (like sending an SMS, a push notification or a plain old email).
I’m using the reverse logic to monitor the HN frontpage: I get an alert whenever a checkstring is found on the frontpage.
Zabbix Config
Assuming you’ve already got Zabbix running, adding this kind of monitoring is dead simple.
First, add a webcheck to retrieve the HN frontpage every 5 minutes. You start by making a Scenario.
Once you’ve got a Scenario, add a single step in that scenario: retrieve the frontpage.
This is where you configure the expected HTTP status code and the string that should be present in the source code. Zabbix retrieves the raw HTML, it does no parsing of that HTML. That means you can include HTML characters as well.
If your goal is to be alerted when a traffic spike is possible, I think it’s more valuable to monitor the New Links section, where all new submissions are sent. This is where your post either gets upvoted or where it goes to die.
So I’ll add monitoring on the New Links page as well.
Once the scenario is created, I’ve added the single step to retrieve the New Links page.
Getting data in Zabbix
Since about 99,99% of the time, your website or domain won’t be on the Hacker News frontpage, Zabbix will retrieve the URLs and find that the expected pattern doesn’t match. Your webchecks will look like this.
And this makes sense, since the string https://ma.ttias.be indeed isn’t on the HN frontpage.
Triggers and Actions
Now to make sure I receive an alert whenever this blog is added, I create 2 triggers.
Zabbix has a concept of failed and successful webchecks. A webcheck is failed when one of its steps doesn’t match the expectation: either the HTTP status code or the required pattern is missing.
My triggers look different from most: I’m triggering for when the webcheck is succeeding instead of failing, since a succeeded webcheck would mean the domain name is present.
Here’s my alert for when the domain name is found on the frontpage.
The trigger looks like this:
{ma.ttias.be:web.test.fail[Check if domain ma.ttias.be is on the HN homepage].last(0)}=0
The web.test.fail item will have the value 1 when the test failed, and when the test contained no errors. Hence, I'm alerting when the value reaches.
The same check is added for the webcheck that monitors the New Links page.
Which looks like this:
{ma.ttias.be:web.test.fail[Check if domain ma.ttias.be is on the HN New Links page].last(0)}=0
What It Looks Like
As soon as the pattern is found on either the homepage or the New Links section, a trigger fires in the dashboard.
You’ll notice the webcheck returns OK, because both the HTTP status code and the expected string were found.
It hasn’t made it to the frontpage. Yet.
Because I’ve got actions configured to e-mail me any alert, I got the following alerted by mail as well.
Caveats
While it’s a simple trick to (ab)use Zabbix like this, if someone uses a bit.ly shortener or an alternative domain name to point to my site, the check won’t work.
The only reason this works on HackerNews, is because they are kind enough to include the actual destination URL in the source code. They could’ve changed outgoing links to URLs like out.php?id=1234, to track the amount of clicks and hide the destination URL.
This Zabbix webcheck monitoring technique will also work on Reddit, as the actual destination URLs are in the source code.
If your site doesn’t have a unique domain name, you may get some false positive alerts.
The Ops Way
Granted, Twilio’s solution is sexier: they’ve got NodeJS, API’s, hackers, …
My solution uses old and boring technology. But it works like a charm.
Monitoring my domain on Hacker News took me less than 5 minutes. Writing this blogpost took far longer.
I like doing things The Ops Way.