Cluster Setup on AWS

Please setup the 3 individual SearchBlox Servers and apply the license keys to all servers before proceeding with the cluster setup instructions.

To create a 3-server SearchBlox Cluster on AWS EC2 (1 indexing server and 2 search servers), follow the step-by-step instructions.

Configure EC2 instances and SearchBlox

1. Create or edit an existing AWS EC2 security group for the three SearchBlox servers, with the following rules:

📘

Note

• sg-xxxxxxxx is the ID of the security group (Search is the name of the group); all instances started with this security group allow internal traffic.

2. Create three servers in Amazon EC2: one Index Server and two Search Servers.

3. Install SearchBlox on all three instances using the instructions in Installing on Amazon Linux 2.

4. If you want to use internal IPs for communication between the servers for better security, you do not need to follow the changes described in the next four steps. If you choose this option, ensure that all three servers can access each other’s Elasticsearch service using the internal IP addresses.

5. If you prefer to use external IPs for Elasticsearch communication, allocate a static public IP to each EC2 instance using Elastic IP in AWS.

6. Go to /etc/hosts and allocate the public IP to public DNS name, as following and restart

34.xxx.xx.78  ec2-xx-xxx-3-78.compute-1.amazonaws.com
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
#::1         localhost6 localhost6.localdomain6

7. Now Go to /etc/hostname and make sure that the public DNS name is given in the file

8. Restart the instance and the SearchBlox service. The SearchBlox Admin Console will now display the public IP address. Request a license key for this IP address.
   Apply this change on all servers in the cluster.
9. If no changes are made, the internal IP address will appear on the Admin Console License page. If you plan to use this IP for the cluster setup, request the license key for this internal IP address.
10. Access the SearchBlox Admin Console and upload the license on all servers/
11. Restart all three SearchBlox servers.

❗️

Security Requirements:

• Traffic must be allowed on ports 9200 and 9300 among these 3 servers.
• Disallow external traffic access to 9200 and 9300.
• Allow traffic port 8443 to access the SearchBlox service.
• Allow traffic port 8444 to access the SearchBlox Analytics service.

Configure Cluster Setup through Index Server Admin Console

1. Go to Dashboard -> Administration -> Cluster in Index Server as shown below:

2. Select Multi-Node as in the screenshot below:

3. Enter the IP addresses for Index and Search Servers to join the cluster through Index Server.

Index Server:
Enter the IP address of the current server where you are configuring the cluster. This server will act as the Index Server.

Search Servers:
Enter the IP addresses of Search Server 1 and Search Server 2, which will act as the Search Servers.

Cluster Name (Optional):
You can provide a unique name for your SearchBlox cluster if needed.

4. If you are using the internal IP for Elasticsearch and license in SearchBlox you can skip this step.
   If you are using public IP for Elasticsearch then
   Stop SearchBlox
   go to <SEARCHBLOX_INSTALLATION_PATH>/elasticsearch/config/elasticsearch.yml file make the changes as shown below:

network.bind_host: xxx.xx.x.x
network.publish_host: xx.xxx.x.xx

where network.bind_host value is the private IP and network.publish_host is the public IP which is now mapped in /etc/hosts.

...
searchblox.master: xx.xxx.x.xx
network.bind_host: xxx.xx.x.x
network.publish_host: xx.xxx.x.xx
discovery.zen.ping.unicast.hosts: [xx.xxx.x.xx, xx.xx.xxx.xxx]

5. Stop and Start SearchBlox in all the three servers (which might take a few minutes).

🚧

Important Information on Restart:

• Be sure to stop and start all the servers after adding the cluster nodes and changing the cluster name by logging into the AWS server.

• To stop the service, use the following command:
  systemctl stop searchblox

• To start the service, use the following command:
  systemctl start searchblox

6. Login to the Index server, create a new collection and start indexing the collection. The new collection will now be available on the 2 other search servers for search within a few minutes if the setup is successful.

📘

Note:

Search Server's Admin Console will not be accessible after successful cluster setup. You can use base search page link https://<Search-Server-IP-or-DNS>:<Port>/search/index.html to access the collection documents.

Continue with the following troubleshooting steps if the new collection is not shown on the search servers

7. Run the following command on all servers to check the cluster status:
   curl -XGET 'https://<IP>:9200/_cluster/health?pretty=true'
   The url can also be accessed at https://<IP>:9200/_cluster/health?pretty=true

8. The cluster health status should be green or yellow.
9. The number_of_nodes should be 3 for a three-server cluster setup. If the number_of_nodes is 3 and the cluster health is green or yellow, the cluster setup is successful.
10. Create additional collections, configure them, and run indexing on the Index Server. The same collections will automatically be created on the Search Servers and will be available for search.

Indexing and Searching in a Cluster Setup

• Indexing and Admin Console configuration can only be performed on the Index Server.
• Search operations can be performed from both the Index Server and the Search Servers.
• If the Index Server goes down, searching can still continue through the Search Servers until the indexing node is restored.
• If a Search Server goes down, indexing will continue on the Index Server, and searching can still be performed through the Index Server and the remaining Search Server. When the failed Search Server restarts, the indexed data will automatically replicate and synchronize with the other servers.

Upgrade Cluster

A clean cluster installation, similar to a standalone server installation, is recommended. Cluster configuration is a one-time setup.

If you need to reconfigure the cluster using the same servers, switching from Multi Node to Single Node in the Admin Console will not work.

Best Practices for Cluster Setup

• Configure a load balancer to distribute traffic between the Search Servers.
• Perform searches only on the Search Servers for optimal performance.
• Ensure that the Index Server and Search Servers can communicate with each other on the same network.
• Ports 9200 and 9300 must be open on all three servers, and traffic should be allowed between them.
• Ensure that Elasticsearch URLs are accessible from all three servers.
• Restart all servers after adding cluster nodes or changing the cluster name.
• After configuring the cluster, avoid switching to Single Node mode, as removing the cluster configuration requires additional manual steps.