I want to show you how I use a tool called
goaccess to do some quick analysis of access logs on webservers. Now that I’m doing more & more consulting work, this has become my favorite tool to quickly get a birds-eye view on the site I’m working on.
Installing goaccess on Mac
If you have Brew installed, it’s as simple as:
$ brew install goaccess
Once installed, you’ll have the
goaccess binary at your disposal.
Generating HTML reports of access logs
There’s a terminal & web output option for
goaccess, I prefer to see the stats in HTML. The graphs are clearer and it’s easier to navigate.
First, I download the access log(s) that I want to analyze locally, to my machine.
Then, it’s time to launch
goaccess to parse them.
$ goaccess *.log -o report.html --log-format=COMMON --real-time-html
Open the corresponding
report.html file in your browser and be amazed!
Access logs formats
There are a lot of access log formats out there. Luckily, some of them are standardized. 😉
goaccess can read several log formats, their presets are as follows:
COMBINED (NCSA Combined Log Format) %h %^[%d:%t %^] "%r" %s %b "%R" "%u" VCOMBINED (NCSA Combined Log Format with Virtual Host) %v:%^ %h %^[%d:%t %^] "%r" %s %b "%R" "%u" COMMON (Common Log Format (CLF)) %h %^[%d:%t %^] "%r" %s %b VCOMMON (Common Log Format (CLF) with Virtual Host) %v:%^ %h %^[%d:%t %^] "%r" %s %b W3C %d %t %^ %m %U %q %^ %^ %h %u %R %s %^ %^ %L SQUID (Squid Native Format) %^ %^ %^ %v %^: %x.%^ %~%L %h %^/%s %b %m %U
When calling the
goaccess binary, you can pass the format using one of the presets above
$ goaccess access.log --log-format=COMMON ... $ goaccess access.log --log-format=VCOMBINED ...
To pass the format manually, see below.
Custom access formats
If you’re dealing with non-standardized log formats, you need to be very explicit about the format in which to parse them. Here’s an example.
$ goaccess access.log \ --log-format='%h %^[%d:%t %^] "%r" %s %b' \ --date-format="%d/%b/%Y" \ --time-format="%H:%M:%S" \ --real-time-html \ -o report.html
The many specifiers can be found in the manpage.