Running Elasticsearch in a VM

This document describes installation and configuration of Elasticsearch in a VM running CentOS 8.

Create CentOS VM

Create new VM and install minimal CentOS 8 server without GUI. Configure static IP.

Min RAM 2GB
Min CPU 1
Min HDD 20GB
Operating System CentOS 8
NIC Static IP

Install Java

Install Java and 'lsof'.

yum install java-11-openjdk
yum install lsof

Install Elasticsearch

Step 1: Download and install Elasticsearch RPM

Download and install Elasticsearch RPM

yum install ./elasticsearch-7.8.1-x86_64.rpm

Reload systemd daemon

systemctl daemon-reload

Step 2: Edit Elasticsearch configuration file

Set following propeerties in /etc/elasticsearch/elasticsearch.yml file. Adjust values for your environment.

# Should be unique on your LAN. 
# If you're running ELK on the same LAN, make sure that ELK and PDS Registy use different cluster names.
cluster.name: pds-registry

# Can be any name, but we used the server DNS name
node.name: es1.test.local

# Can be "0.0.0.0" to listen on all network interfaces, DNS name or IP address
network.host: 0.0.0.0

# A list of nodes in the cluster. Enter DNS name or IP address.
discovery.seed_hosts: ["es1.test.local"]
cluster.initial_master_nodes: ["es1.test.local"]

Step 3: Start and test the service

Start the service:

systemctl start elasticsearch

Check the service status:

systemctl status elasticsearch

Check Elasticsearch cluster status

curl localhost:9200

You should get a message similar to this

{
  "name" : "es1.test.local",
  "cluster_name" : "pds-registry",
  "cluster_uuid" : "OpWsTs1rRxq390Ljjoqv0Q",
  "version" : {
    "number" : "7.8.1",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "b5ca9c58fb664ca8bf9e4057fc229b3396bf3a89",
    "build_date" : "2020-07-21T16:40:44.668009Z",
    "build_snapshot" : false,
    "lucene_version" : "8.5.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Firewall Configuration

To open Elasticsearch port (9200), run the following commands

firewall-cmd --zone=public --add-port=9200/tcp --permanent
firewall-cmd --reload
firewall-cmd --zone=public --list-ports

Troubleshooting

Check logs in /var/log/elasticsearch.

3-Node Cluster

Step 1: Install Elasticsearch

Create 3 VMs and install Java and Elasticsearch RPM on each node as described above. In this tutorial we are using 3 VMs with the following IPs and host names:

192.168.75.11   es1.test.local
192.168.75.12   es2.test.local
192.168.75.13   es3.test.local

Step 2: Configure DNS

Add the following entries in /etc/hosts or configure a DNS server.

192.168.75.11   es1.test.local
192.168.75.12   es2.test.local
192.168.75.13   es3.test.local

Step 3: Configure firewall

Open two Elasticsearch ports:

  • 9200 - clients connect to this port
  • 9300 - Elasticsearch nodes communicate over this port

Run the following commands to open ports:

firewall-cmd --zone=public --add-port=9200/tcp --permanent
firewall-cmd --zone=public --add-port=9300/tcp --permanent
firewall-cmd --reload
firewall-cmd --zone=public --list-ports

Step 4: Edit Elasticsearch configuration file

Set following propeerties in /etc/elasticsearch/elasticsearch.yml file on each node. Adjust values for your environment and each node.

# Should be unique on your LAN. 
# If you're running ELK on the same LAN, make sure that ELK and PDS Registy use different cluster names.
cluster.name: pds-registry

# Each node should have different name
node.name: es1.test.local

# Listen on all network interfaces
network.host: 0.0.0.0

# List all elasticsearch nodes
discovery.seed_hosts: ["es1.test.local", "es2.test.local", "es3.test.local"]
cluster.initial_master_nodes: ["es1.test.local", "es2.test.local", "es3.test.local"]

# To prevent network partitioning, require 2 masters (out of 3) to form the cluster
# before accepting user requests
discovery.zen.minimum_master_nodes: 2

Step 5: Start and test elasticsearch service on all nodes

Start the service:

systemctl start elasticsearch

Test the service:

systemctl status elasticsearch

Step 6: List all nodes

Call the following API on one of elasticsearch nodes (e.g., es1.test.local):

curl "http://es1.test.local:9200/_cat/nodes?v"

You should see a response similar to this. Make sure all 3 nodes joined the cluster.

ip            heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.75.13           20          96   1    0.04    0.08     0.09 dilmrt    -      es3.test.local
192.168.75.11           22          93   1    0.08    0.04     0.04 dilmrt    *      es1.test.local
192.168.75.12           53          96   1    0.09    0.08     0.05 dilmrt    -      es2.test.local