Building a Multi-Node Elasticsearch Cluster on Ubuntu

In the ever-expanding world of big data, Elasticsearch has emerged as a front-runner in the search and analytics engine space. Its ability to handle large volumes of data efficiently makes it a go-to choice for many organizations. This article delves into the process of setting up a multi-node Elasticsearch cluster on an Ubuntu server, providing a detailed and comprehensive guide for systems administrators, DevOps engineers, and data professionals.

This guide focuses on establishing a three-node Elasticsearch cluster on Ubuntu, specifically configured for the following nodes:

Advertisement

NODE_1: 192.168.10.101
NODE_2: 192.168.10.102
NODE_3: 192.168.10.103

What is Elasticsearch?

Elasticsearch is an open-source, distributed search and analytics engine designed for horizontal scalability, reliability, and real-time search. It’s often used for log analytics, full-text search, and operational intelligence use cases.

Prerequisites

Before embarking on setting up a multi-node Elasticsearch cluster, ensure you have the following:

• Multiple Ubuntu servers (physical or virtual) for the nodes.
• Basic knowledge of Linux command line and networking.
• Sudo or root access on the servers.

Step 1: Installing Elasticsearch (On Each Node)

1. Update and Install Java: Elasticsearch requires Java. Install OpenJDK:

   sudo apt update 
   sudo apt install openjdk-11-jdk 
   

2. Import Elasticsearch PGP Key: Securely download and install Elasticsearch using the official PGP key:

   wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg 
   

3. You may need to install the apt-transport-https package on Debian before proceeding:

   sudo apt-get install apt-transport-https 
   

4. Add Elasticsearch Repository: Save the repository definition to /etc/apt/sources.list.d/elastic-8.x.list:

   echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list 
   

5. Install Elasticsearch: Update the Apt cache and install Elasticsearch.

   sudo apt update  
   sudo apt install elasticsearch 
   

Step 2: Configuring Elasticsearch

1. Edit the Elasticsearch Configuration: Modify /etc/elasticsearch/elasticsearch.yml on each node.
   • On NODE_1:

     
     node.name: "NODE_1"
     network.host: 192.168.10.101
     cluster.initial_master_nodes: ["NODE_1", "NODE_2", "NODE_3"]
     
     

   • On NODE_2:

     
     node.name: "NODE_2"
     network.host: 192.168.10.102
     cluster.initial_master_nodes: ["NODE_1", "NODE_2", "NODE_3"]
     
     

   • On NODE_3:

     
     node.name: "NODE_3"
     network.host: 192.168.10.103
     cluster.initial_master_nodes: ["NODE_1", "NODE_2", "NODE_3"]
     
     

2. Configure JVM Options: Adjust JVM settings in /etc/elasticsearch/jvm.options if needed.

Step 3: Setting Up Cluster Networking

• Open Necessary Ports: Ensure that the ports Elasticsearch uses (default 9200 and 9300) are open on each node.
• Configure Firewall (if applicable): Adjust firewall settings to allow traffic on these ports.

Step 4: Starting and Verifying the Cluster

1. Start Elasticsearch Service:

   sudo systemctl start elasticsearch 
   

2. Enable Elasticsearch on Boot:

   sudo systemctl enable elasticsearch 
   

3. Check Cluster Health: Use curl to verify the cluster’s health:

   curl -X GET "localhost:9200/_cluster/health?pretty"
    
   

Step 5: Cluster Maintenance and Management

• Monitoring: Regularly monitor cluster health and performance.
• Scaling: Add more nodes as required.
• Backup and Recovery: Implement a backup and recovery plan.

Conclusion

Setting up a multi-node Elasticsearch cluster on Ubuntu involves careful planning and execution. By following the steps outlined in this guide, you can establish a robust, scalable, and efficient search and analytics platform that leverages the power of Elasticsearch. Remember, regular maintenance and monitoring are key to ensuring the long-term health and performance of your Elasticsearch cluster.

Further Reading:

• Elasticsearch Official Documentation
• Networking Basics for Elasticsearch