Analysing webservers logs locally with goaccess

Oh Dear! monitors your entire site, not just the homepage. We crawl and search for broken pages and mixed content, send alerts when your site is down and notify you on expiring SSL certificates.

Start your free 10 day trial! »

Image of Mattias Geniar

Mattias Geniar, November 15, 2019

Follow me on Twitter as @mattiasgeniar

I want to show you how I use a tool called goaccess to do some quick analysis of access logs on webservers. Now that I’m doing more & more consulting work, this has become my favorite tool to quickly get a birds-eye view on the site I’m working on.

Installing goaccess on Mac

If you have Brew installed, it’s as simple as:

$ brew install goaccess

Once installed, you’ll have the goaccess binary at your disposal.

Generating HTML reports of access logs

There’s a terminal & web output option for goaccess, I prefer to see the stats in HTML. The graphs are clearer and it’s easier to navigate.

Go access example

First, I download the access log(s) that I want to analyze locally, to my machine.

Then, it’s time to launch goaccess to parse them.

$ goaccess *.log -o report.html --log-format=COMMON --real-time-html

Open the corresponding report.html file in your browser and be amazed!

Access logs formats

There are a lot of access log formats out there. Luckily, some of them are standardized. 😉

goaccess can read several log formats, their presets are as follows:

COMBINED (NCSA Combined Log Format)
  %h %^[%d:%t %^] "%r" %s %b "%R" "%u"

VCOMBINED (NCSA Combined Log Format with Virtual Host)
  %v:%^ %h %^[%d:%t %^] "%r" %s %b "%R" "%u"

COMMON (Common Log Format (CLF))
  %h %^[%d:%t %^] "%r" %s %b

VCOMMON (Common Log Format (CLF) with Virtual Host)
  %v:%^ %h %^[%d:%t %^] "%r" %s %b

W3C
  %d %t %^ %m %U %q %^ %^ %h %u %R %s %^ %^ %L

SQUID (Squid Native Format)
  %^ %^ %^ %v %^: %x.%^ %~%L %h %^/%s %b %m %U

When calling the goaccess binary, you can pass the format using one of the presets above

$ goaccess access.log --log-format=COMMON ...
$ goaccess access.log --log-format=VCOMBINED ...

To pass the format manually, see below.

Custom access formats

If you’re dealing with non-standardized log formats, you need to be very explicit about the format in which to parse them. Here’s an example.

$ goaccess access.log \
    --log-format='%h %^[%d:%t %^] "%r" %s %b' \
    --date-format="%d/%b/%Y" \
    --time-format="%H:%M:%S" \
    --real-time-html \
    -o report.html

The many specifiers can be found in the manpage.



Want to subscribe to the cron.weekly newsletter?

I write a weekly-ish newsletter on Linux, open source & webdevelopment called cron.weekly.

It features the latest news, guides & tutorials and new open source projects. You can sign up via email below.

No spam. Just some good, practical Linux & open source content.