How to create a centralized logging server with Syslog-ng?

2021.04.08

In this article, we will build a centralized logging server with Syslog-ng in Ubuntu 20.04.


Firstly, we need to prepare the environment. The network looks like this:

The client1, client2, and server are created by multipass. You can read the documentation of this tool here.

To access shell in each virtual machine (VM), typing following command:

multipass exec <machine-name> -- /bin/bash

Notice: The IP address in the below image was found by accessing the shell in each VM and type: 

ip addr show


Secondly, we install Syslog-ng in every VM with the command:

sudo apt-get update
sudo apt-get install syslog-ng -y

Enable syslog-ng

sudo systemctl enable syslog-ng

After that, we need to configure each VM in /etc/syslog-ng/syslog-ng.conf.

sudo mv /etc/syslog-ng/syslog-ng.conf /etc/syslog-ng/syslog-ng.conf.bak
sudo nano /etc/syslog-ng/syslog-ng.conf

In client1 and client2 the content should be like this:

@version: 3.5
@include "scl.conf"

options {
    chain_hostnames(off);
    flush_lines(0);
    use_dns(no);
    use_fqdn(no);
    owner("root");
    group("adm");
    perm(0640);
    stats_freq(0);
    bad_hostname("^gconfd$");
    keep-hostname(yes);
    create_dirs(yes);
};

source s_local {
    system();
    internal();
};

destination d_logs {
    file("/var/log/syslog-ng/${YEAR}/${MONTH}/${DAY}/all.log");
};

destination d_syslog_tcp {
    syslog("10.88.82.20" transport("tcp") port(514));
};

log { source(s_local); destination(d_logs); };
log { source(s_local); destination(d_syslog_tcp); };

Notice: create_dirs(yes); in options block is mean that automatically create directories if not exists.

In server it should be like this one:

@version: 3.5
@include "scl.conf"

options {
    chain_hostnames(off);
    flush_lines(0);
    use_dns(no);
    use_fqdn(no);
    owner("root");
    group("adm");
    perm(0640);
    stats_freq(0);
    bad_hostname("^gconfd$");
    keep-hostname(yes);
    create-dirs(yes);
};

source s_local {
    system();
    internal();
};

source s_network {
    syslog(transport(tcp) port(514));
};

destination d_logs {
    file("/var/log/syslog-ng/${HOST}/${YEAR}/${MONTH}/${DAY}/all.log");
};

log { source(s_local); source(s_network); destination(d_logs); };

Type Ctrl+O and Enter to save the file. Ctrl+X to exit to bash.

Restart the syslog-ng service:

sudo systemctl restart syslog-ng

Now, you can read continuously the log by a command (for example on my current date):

In each client:

sudo tail -f /var/log/syslog-ng/2021/02/16/all.log

In the server:

sudo tail -f /var/log/syslog-ng/server/2021/02/16/all.log


Finally, we can see the log is written to the server.

sudo ls /var/log/syslog-ng

And watching the client log by (for example on my current date):

sudo tail -f /var/log/syslog-ng/client1/2021/02/16/all.log


By: Anh Hao