Analyze HTTP request logs with GoAccess

Analyzing HTTP request logs can provide crucial insight into how an application is performing and how an audience is interacting with content. GoAccess is an open source tool that can be used to visualize request log data to can gain deeper insights into an application.

Install GoAccess

To install GoAccess, go to their download page and find specific instructions for the local machine’s operating system.

For macOS, using homebrew is recommended: brew install goaccess

For Windows, there is a choice between a specific Windows implementation called CowAxess which provides an installer file, or using the Windows Subsystem for Linux and following the Linux installation steps.

For additional information about GoAccess configuration and troubleshooting not covered here, refer to the GoAccess manual.

Configure GoAccess

Once GoAccess is installed, configure it to parse the downloaded VIP logs.

The GoAccess configuration file will be either ~/.goaccessrc or /%sysdir%/goaccess.conf where %sysdir% is one of /usr/etc/, /etc/, or /usr/local/etc.

Open that file, and add the following formatting options:

# Set time format
time-format %T

# Set date format
date-format %d/%b/%Y

# Set log format
log-format { "body_bytes_sent": "%b", "content_type": "%M", "http_host": "%v", "http_referer": "%R", "http_user_agent": "%u", "http_version": "%H", "remote_addr": "%h", "remote_user": "%e", "request_time": "%T", "request_type": "%m", "request_url": "%U", "sent_x_cache": "%C", "status": "%s", "timestamp": "%d:%t %^", "tls_version": "%K", "wplogin": "%e" }

Preparing compressed log files for GoAccess reporting

VIP’s log shipping delivers a series of gzipped (compressed) JSON log files. Because GoAccess can only parse uncompressed log files, those files must be uncompressed first. A popular command-line option is to use the gunzip command.

To uncompress a gzip (.gz) archive, use the following command:

gunzip 17_50_0.gz

The command above will remove the 17_50_0.gz file and create a file named 17_50_0 with the uncompressed content.

Create a report with GoAccess

To view a report in your terminal, use the following command:

goaccess 17_50_0 17_55_0 18_00_0 18_05_0 18_10_0

The command above is simply calling goaccess and then passing it a number of access logs, like 17_50_0, separating them by a space.

To create an HTML report, run the following command in the terminal:

goaccess 17_50_0 17_55_0 18_00_0 18_05_0 18_10_0 > logreport.html

Opening up the logreport.html file will display the log information in a set of graphs: