Elasticsearch and Kibana installation and configuration

I want us to explore how to install and configure Elasticsearch and Kibana, to get a cluster ready for ingesting logs for any server. I will be using Rocky Linux (Redhat family) for this setup, and installing Elasticsearch and Kibana as RPM packages.

Elasticsearch is an open-source search and analytics engine. It is a free tool to store and quickly search data, manage and view it with Kibana, with a lot of features such as: indexing several types of data from different appliances, data analytics and observability, machine learning, security information and event management (SIEM), endpoint security and many more. A cluster consists of a number of nodes, but for the purpose of this setup, I will be configuring a one-node cluster in addition to one Kibana instance.

Installing Elasticsearch

The installation on this article is based on RHEL version 8, you can use yum instead of dnf.

Switch to superuser

sudo su

Update applications installed on a system

dnf update

Install vim editor - the colored editor may indicate errors in the config files

dnf -y install vim

Import the elasticsearch GnuPG key

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Create a file called elasticsearch.repo in the /etc/yum.repos.d/ directory with the command

vim /etc/yum.repos.d/elasticsearch.repo

and paste the following then save the file:

[elasticsearch]
name=Elasticsearch repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md

Install Elasticsearch

dnf install --enablerepo=elasticsearch elasticsearch
systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch.service

Change the “elastic” user password:

/usr/share/elasticsearch/bin/elasticsearch-reset-password -i -u elastic

Use -i for interactive, which allows you to type in your own password

To check if the node is running, run the following command:

curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic https://localhost:9200

Configuring Elasticsearch

The configuration file is in /etc/elasticsearch/

vim /etc/elasticsearch/elasticsearch.yml

Use the following settings to set up the node:

node.name: node1
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
# IP of the Elasticsearch node:
network.host: 192.0.0.1
http.port: 9200

Configure the firewall on this machine, allow communication to port 9200/tcp:

firewall-cmd --add-port=9200/tcp --permanent
firewall-cmd --reload

Other configurations:

To configure the heap size, add the Xms and Xmx JVM arguments to a custom JVM options file with the extension .options and store it in the jvm.options.d/ directory. For example, to set the maximum heap size to 2GB, set both Xms and Xmx to 2g:

-Xms2g
-Xmx2g

Installing Kibana

Kibana is the open-source user interface tool used to view the stored information in Elasticsearch and manage the cluster. It has many types of visualizations that help you analyze the data and make sense of what the it looks like. In this setup, Kibana will let us look at the Syslogs coming from the firewall by connecting to the Elasticsearch node and sending queries to it in the background.

On the Kibana VM, run the following commands:

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Create the following file and copy the content below to it:

vim /etc/yum.repos.d/kibana.repo

[kibana-8.x]
name=Kibana repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Run the following command to start installing Kibana:

dnf install kibana

Enable and start the Kibana service:

/bin/systemctl daemon-reload
/bin/systemctl enable kibana.service
systemctl start kibana.service

Configuring Kibana

The configuration file is in /etc/kibana/

vim /etc/kibana/kibana.yml

server.port: 5601
# IP for the Kibana server
server.host: “192.168.25.120”
# remove a warning at the login page:
server.publicBaseUrl: “http://192.0.0.2:5601”
server.name: “kibana”
# use https
elasticsearch.hosts: [“https://192.0.0.1:9200”]
elasticsearch.username: “kibana_system”
## The password can be turned off, and be stored in the keystore instead
## Check down for more instructions
# elasticsearch.password: “<password>”
# Copy http_ca.crt from the Elasticsearch node:
elasticsearch.ssl.certificateAuthorities: [ “/etc/kibana/certs/http_ca.crt” ]

From the Elasticsearch node, copy the CA certificate to Kibana:

scp /etc/elasticsearch/certs/http_ca.crt root@192.0.0.2:/etc/kibana/certs

Also, run the following command to change the password for the “kibana_system” built-in user:

/usr/share/elasticsearch/bin/elasticsearch-reset-password -i -u kibana_system

Use the -i option for interactive, which lets you choose your own password

Create the keystore [if it does not exist]

To create the kibana.keystore, use the create command:

/usr/share/kibana/bin/kibana-keystore create

The file kibana.keystore will be created in the config directory defined by the environment variable KBN_PATH_CONF.

To add the elasticsearch.password variable to the keystore, use the following command

/usr/share/kibana/bin/kibana-keystore add elasticsearch.username

Finally, allow connections to port 5601 in Kibana:

firewall-cmd --add-port=5601/tcp --permanent
firewall-cmd --reload

To check the status from journal for kibana, run the following command:

journalctl -xe --unit kibana

For the UAT setup, if you want to by-pass the security ssl checks, you can change this on the

# elasticsearch.ssl.verificationMode: full
# to
elasticsearch.ssl.verificationMode: none

At this point, the Elasticsearch node and the Kibana instance are ready. After changes to the .yml files, you should restart the service using “systemctl restart elasticsearch.service”, then checking the status to see if there are any errors or issues.

You can browse to the Kibana IP address and port combination to log in to the Kibana user interface

Use the “elastic” built-in user and the password you set after the Elasticsearch installation to log in.

In this blog post, we talked about how to install and configure Elasticsearch and Kibana.

Errors and solutions

1. Elasticsearch fails to start when tmp directory defined

Mount /tmp folder to run exec options. The following command achieves this goal.

mount -o remount,exec /tmp

Exec options for the folder /tmp reset on reboot. So, the solution must be run at every start.

Reference issue