Welcome to cron.weekly issue #20 for Sunday March 20th, 2016.
I consider 20 consecutive issues to a pretty impressive milestone, so if you don’t mind I’ll go ahead and celebrate just a little bit.
You may notice a shift in this issue, as I’m moving more ‘news’ to the tools & project section. New releases for already established projects will now be listed there, no longer in the news part – where they previously got posted.
In addition, the online archives for cron.weekly now have a search function. If you know you’ve read about a particular topic but can’t find it again, try searching for it online. I tweaked the search results page a bit so I hope everyone can quickly find what they’re after.
News
Google shares software network load balancer design powering GCP networking
This post by the Google Cloud Platform team describes the workings of Maglev, the “Fast and Reliable Software Network Load Balancer”. It uses Equal-Cost Multi-Path routing (ECMP) with a consistent hashing technique, making it possible to run your load balancers in active/active setups.
The One Million Container Challenge
A couple of years ago we had the C10k problem/challenge, where the challenge was to hold 10.000 sockets/connections. Now Nomad, the container scheduler of Hashicorp, has another challenge (which they completed successfully): scheduling 1.000.000 containers in less than 5 minutes.
Remote Code Execution in all git versions (client + server) < 2.7.4
A vulnerability that got prematurely disclosed, by accident. All versions of git, both client-side and server-side, are vulnerable for a remote code execution vulnerability. Patches are available and if you run a publicly available git server, you need to act ASAP.
How Badoo saved one million dollars switching to PHP7
We already know PHP 7 is fast. In most cases, up to 2-3x as fast as PHP 5.6. So if you have a lot of servers, upgrading to PHP 7 can save a lot of CPU cycles and memory!
MongoDB: The Frankenstein Monster of NoSQL Databases
Opinionated, but a clever read nonetheless: the writer complains about MongoDB’s data access layer, the query framework, its MapReduce history, … Lots of food for thought.
Linux Kernel 4.5 released
Linus released version 4.5 of the Linux Kernel. There’s a human-readable write-up over at KernelNewbies.org. The main features: (file) copy offloading with new copy_file_range(2) system call, btrfs improvements, better epoll multithreading, performance improvements to UDP sockets & lots more.
Kernel 4.6: full ASLR randomization for 32-bit programs
This topic deserves special highlighting: the 4.6 kernel also introduces Address Space Layout Randomization (ASLR) for 32-bit programs.
The Real Reason to Not Use SIGKILL
A comic that’s worth your read: SIGKILL leaves children without parents. Seriously, it’s bad, m’kay.
Tools & Projects
PuppetDB 4.0 released
It was quite the Puppet announce week for new releases: PuppetDB, the backend for Puppet’s exported resources, reached version 4.0. It’s a major release that improves performance and contains many new query features, including a new easier to use query language. It also drops support for Ruby 1.8.7, Puppet 3.x.
Puppet 4.4.0 released
Next to PuppetDB, the Puppetlabs crew also released Puppet 4.4. The headline feature here is
Static Catalogs: file resources with ‘puppet:///’ source attributes will now include the checksum of the file inside the catalog, rather than requiring additional http requests to the master as the catalog is being applied. A full changelog is available on the site.
Nmap 7.10 released
A new version of Nmap, the open source network scanner, was released: 12 new NSE scripts, hundreds of new
OS/version fingerprints, and dozens if smaller improvements and bug fixes.
surge.sh: Static web publishing for Front-End Developers
Simple, single-command web publishing. Publish HTML, CSS, and JS for free, without leaving the command line. This builds upon the popularity of static site generators, massively simplifying the hosting setup required to run the site.
Puppet’s mgmtgraph module
I told you, it’s a heavy Puppet week: this Puppet module allows you to compile simple Puppet manifests into a data structure that is suitable for mgmt (that other config management tool) to consume. It’s basically a convert-puppet-to-mgmt module if you ever want to switch over.
nplaym: Space Invadors for npm packages
A wrapper for npm so you can play a Space-Invaders-a-like game while installing. This is a terminal-based game, so you can even play it over SSH.
csvkit
csvkit is a suite of command-line tools for converting to and working with CSV, the king of tabular file formats.
git2consul
git2consul takes one or many git repositories and mirrors them into Consul (a tool for Service Discovery & configuration). The goal is for organizations of any size to use git as the backing store, audit trail, and access control mechanism for configuration changes and Consul as the delivery mechanism.
nerd fonts
If you’re into typography, like me, you’ll find this interesting: a collection of over 20 patched fonts for Powerline, Font Awesome, Octicons, Devicons, and Vim.
bindiff
Ever tried to run ‘diff’ on 2 binary files? Doesn’t quite work the way you intended. BinDiff is a comparison tool for binary files that helps to quickly find differences and similarities in disassembled code.
PPM: a new PHP process manager
There are already a couple of ways to run your PHP code (mod_php, fastcgi, FPM, …), but here’s another – relatively new – one: PPM. PPM is a process manager, supercharger and load balancer for PHP applications. It’s based on ReactPHP and works best with applications that use request-response frameworks like Symfony’s HTTPKernel.
Guides & Tutorials
Server Security: Indicators of Compromised Behavior with OSSEC
This is a nice write-up of using OSSEC, an open source intrusion detection system (IDS). Lots of practical content on implementing it on your own.
tcpdump is amazing
This is a practical guide on using tcpdump to capture the network packets on a particular interface and investigate them using Wireshark (an easier GUI than plain tcpdump output).
How does perf work? (in which we read the Linux kernel source)
Just like the previous post about tcpdump, this post is written by Julia Evians – a blog I highly recommend following. This post covers the ‘perf‘ tool for analysis and takes a different approach at explaining it: by reading (and explaining, thankfully) the Linux source code.
Satellite: Monitoring Kubernetes in Production
Monitoring the state of a Kubernetes cluster is not straightforward using traditional monitoring tools. This post covers an alternative called satellite, a simple and extensible monitoring agent / library for Kubernetes.
Using Nmap + Logstash to Gain Insight Into Your Network
The short summary of this guide is: import nmap scan results directly into Elasticsearch and easily query them for analysis. Super cool idea.
Practical SELinux and Containers
Straight from the SELinux project lead: Dan Walsh talks about how to go about enabling SELinux to controlling access between standard Docker containers.
Installing Fedora on the Raspberry Pi 3
Step-by-step instructions on getting Fedora to run on your brand new Raspberry Pi 3.
NSA-proof your e-mail in 2 hours
Slightly older, but still very relevant: this guide covers all the steps involved in running your own mailserver, encrypting all your mails, adding anti-spam services, proper push support, …
Linux Insides: Timers and time management
This Github repository contains 5 very excellent articles: this is a 5-step description on Linux kernel timers, how the ‘clocksource’ and ‘tick broadcast’ framework works, … Very low-level if you’re into that kind of things.
Ditching tmux
What if … you wanted a life without tmux. A couple of pointers of reducing the switching between panes and tabs.
LXD 2.0: Introduction to LXD
LXD is a daemon which provides a REST API to drive LXC containers. LXD containers are ‘heavier’ than Docker containers, they contain a full Linux system, exactly as it would be when run on metal or in a VM. Those containers will typically be long running and based on a clean distribution image. This post explores, in great detail, why you’d want to use it and how.
Lessons Learned From A Year Of Elasticsearch In Production
Very good pointers if you’re just starting out with Elasticsearch: tips on heap sizes, using doc values, storage-backend advice, slow query logging, … I wish I had known these earlier.
The Puppet 4 Lookup Function
This post covers a couple of ‘lookup patterns’, where you tell Puppet to retrieve data for its modules from an external location (be it hiera or something else).
Maintain Separate GitHub accounts
This is a neat trick that doesn’t only apply to GitHub. It describes using 2 separate ~/.ssh/config blocks for “personal” and “work” accounts, each pointing to a different SSH keyfile.
Use the Unofficial Bash Strict Mode (Unless You Looove Debugging)
Some good points on a couple of “set” commands in Bash for more sane defaults: fail on exit straight away, better undefined variable handling & when piping commands, let any failed command be the exit code of that set of commands.
Scaling to 12 Million Concurrent Connections
The Linux kernel can handle _a lot _of connections, but not out of the box. This guide explains some of the tweaks needed to be able to handle 12 million concurrent connections. A good reminder on ulimit & TCP port limitations.