Analysing webservers logs locally with goaccess

I want to show you how I use a tool called goaccess to do some quick analysis of access logs on webservers. Now that I’m doing more & more consulting work, this has become my favorite tool to quickly get a birds-eye view on the site I’m working on.

Installing goaccess on Mac

If you have Brew installed, it’s as simple as:

$ brew install goaccess

Once installed, you’ll have the goaccess binary at your disposal.

Generating HTML reports of access logs

There’s a terminal & web output option for goaccess, I prefer to see the stats in HTML. The graphs are clearer and it’s easier to navigate.

First, I download the access log(s) that I want to analyze locally, to my machine.

Then, it’s time to launch goaccess to parse them.

$ goaccess *.log -o report.html --log-format=COMMON --real-time-html

Open the corresponding report.html file in your browser and be amazed!

Access logs formats

There are a lot of access log formats out there. Luckily, some of them are standardized. 😉

goaccess can read several log formats, their presets are as follows:

COMBINED (NCSA Combined Log Format)
  %h %^[%d:%t %^] "%r" %s %b "%R" "%u"

VCOMBINED (NCSA Combined Log Format with Virtual Host)
  %v:%^ %h %^[%d:%t %^] "%r" %s %b "%R" "%u"

COMMON (Common Log Format (CLF))
  %h %^[%d:%t %^] "%r" %s %b

VCOMMON (Common Log Format (CLF) with Virtual Host)
  %v:%^ %h %^[%d:%t %^] "%r" %s %b

W3C
  %d %t %^ %m %U %q %^ %^ %h %u %R %s %^ %^ %L

SQUID (Squid Native Format)
  %^ %^ %^ %v %^: %x.%^ %~%L %h %^/%s %b %m %U

When calling the goaccess binary, you can pass the format using one of the presets above

$ goaccess access.log --log-format=COMMON ...
$ goaccess access.log --log-format=VCOMBINED ...

To pass the format manually, see below.

Custom access formats

If you’re dealing with non-standardized log formats, you need to be very explicit about the format in which to parse them. Here’s an example.

$ goaccess access.log \
    --log-format='%h %^[%d:%t %^] "%r" %s %b' \
    --date-format="%d/%b/%Y" \
    --time-format="%H:%M:%S" \
    --real-time-html \
    -o report.html

The many specifiers can be found in the manpage.