What is Graylog?

Graylog is defined in terms of log management platform for collecting, indexing, and analyzing both structured and unstructured data from almost any source.

Why do we need Graylog?

Nowadays most of the applications following microservice architecture .where many of microservices are hosted on different machines. So, it will take very huge time for the user to get the logs from each microservice on different machines every time.

To avoid this we have an application called Graylog .which works by configuring at one place and get the logs of all microservices at one centralized location.

Is it Paid software?

No, Its Open source log management tool.

How it works?


As illustrated in the above diagram, the following are the components –




  1. ElasticSearch: ElasticSearch is useful for storing logs and searching text.
  2. MongoDB: MongoDB is useful for Metadata Management.
  3. Graylog: Graylog can help you to better understand the use made within your applications, improve their security, and reduce costs.

Architectural Considerations:

There are a few rules of thumb when scaling resources for Graylog:

  1. graylog-server nodes should have a focus on CPU power.
  2. Elasticsearch nodes should have as much RAM as possible and the fastest disks you can get. Everything depends on I/O speed here.
  3. MongoDB is only being used to store configuration and the dead letter messages and can be sized fairly small.
  4. graylog-web-interface nodes are mostly waiting for HTTP answers of the rest of the system and can also be rather small.
  5. graylog-radio nodes act as workers. They don’t know each other and you can shut them down at any point in time without changing the cluster state at all.

Also, keep in mind that messages are only stored in Elasticsearch. If you have data loss on Elasticsearch, the messages are gone — except if you have created backups of the indices.

MongoDB is only storing meta information and will be abstracted with a general database layer in future versions. This will allow you to use other databases like MySQL instead

How to Install and Configure graylog with easy steps?

Usually, graylog will be supporting for below of the platforms

  • Ubuntu 12.04, 14.04, 16.04
  • Debian 7, 8, 9
  • RHEL/CentOS 6, 7

Let us refer to install graylog for CentOS

Note, if we need to install for other OS levels please refer this


Make sure to install and configure the following software before installing and starting any Graylog services:

  • Java (>= 8)
  • MongoDB (>= 2.4)
  • Elasticsearch (>= 2.x)

Let’s start with installing Java

$ sudo yum install java-1.8.0-openjdk-headless.x86_64

If you want to use pwgen later on you need to Setup EPEL on your system with sudo yum install epel-release and install the package with

$ sudo yum install pwgen

How to add MongoDB? 

Before installing the MongoDB First add the below content to repository file /etc/yum.repos.d/mongodb-org-3.6.repo


name=MongoDB Repository





After that, install the latest release of MongoDB with

$ sudo yum install -y mongodb-org

Additionally, run these last steps to start MongoDB during the operating system’s boot and start it right away:

$ sudo chkconfig — add mongod

$ sudo systemctl daemon-reload

$ sudo systemctl enable mongod.service

$ sudo systemctl start mongod.service

How to Start Elasticsearch?

First, install the Elastic GPG key with

rpm — import then add the repository file /etc/yum.repos.d/elasticsearch.repo with the following contents:


name=Elasticsearch repository for 5.x packages







followed by the installation of the latest release with

$ sudo yum install elasticsearch.

Make sure to modify the Elasticsearch configuration file (/etc/elasticsearch/elasticsearch.yml) and set the cluster name to graylog additionally you need to uncomment (remove the # as the first character) the line: graylog

After you have modified the configuration, you can start Elasticsearch:

$ sudo chkconfig — add elasticsearch

$ sudo systemctl daemon-reload

$ sudo systemctl enable elasticsearch.service

$ sudo systemctl restart elasticsearch.service


How to configure Graylog server?

Now install the Graylog repository configuration and Graylog itself with the following commands:

$ sudo rpm -Uvh

$ sudo yum install graylog-server

Follow the instructions in your /etc/graylog/server/server.conf and add password_secret and root_password_sha2. These settings are mandatory and without them, Graylog will not start!

You need to use the following command to create your root_password_sha2:

echo -n yourpassword | sha256sum

To be able to connect to Graylog you should set rest_listen_uri and web_listen_uri to the public host name or a public IP address of the machine you can connect to.

The last step is to enable Graylog during the operating system’s startup

$ sudo chkconfig — add graylog-server

$ sudo systemctl daemon-reload

$ sudo systemctl enable graylog-server.service

$ sudo systemctl start graylog-server.service

What is collector sidecar in graylog?

Graylog Collector Sidecar is a lightweight configuration management system for different log collectors, also called Backends. The Graylog node(s) act as a centralized hub containing the configurations of log collectors.

How to add collector sidecar on the client side?

Its very simple just add the RPM package for centos

$ sudo rpm -i collector-sidecar-0.0.9-1.x86_64.rpm


Activate the Sidecar as a system service:

$ sudo graylog-collector-sidecar -service install
$ sudo systemctl start collector-sidecar


After Installing the collector sidecar just complete the configuration by



Server_url URL to the Graylog API, e.g. which is same as your rest listen URL
node_id Name of the Sidecar instance, will also show up in the web interface. Hostname will be used if not set.
 collector_id  Unique ID (UUID) of the instance. This can be a string or a path to an ID file
 log_path  A path to a directory where the Sidecar can store the output of each running collector backend
 tags  List of configuration tags. All configurations on the server side that match the tag list will be fetched and merged by this instance
 backends  A list of collector backends the user wants to run on the target host

Finally, we are done with the configuration.

How do we launch graylog?

  •  Open your favourite browser and hit the URL which you mentioned as web_listen_uri i.e
  • Enter the username and password as per the configuration
  • Go to System/Inputs and create the global inputs to start the process.

For example


  • Go to System/Collectors click on your collector and make sure your collector which is hosted in  your client side should be up and running

  • Create configuration of filebeat  by creating the output and Input of the microservice by specifying the log path to retrieve the log messages
  • Output filebeat example
  • Input Filebeat example

Now all is yours to set up the Input-output messages from the frontend of Graylog and get the log messages.

To be continued to set up the Streams on Each microservice…

  • Diagramatic References-


Written by-
Goutham R
QA Team

Also published on Medium.

Leave a Reply

Your email address will not be published. Required fields are marked *