SearchBlox for Amazon OpenSearch Service is an enterprise search platform for the AWS Cloud thats uses the Amazon OpenSearch Service, the fully managed and scalable Opensearch service available on Amazon Web Services (AWS). SearchBlox for Amazon OpenSearch Service can crawl, index and search content across multiple datasources including file systems, websites, databases and applications.

Architecture

This service consists of two types of SearchBlox servers that are available through the AWS marketplace. The first is SearchBlox IndexServer. The SearchBlox IndexServer can crawl and index content in over 40 document formats including PDFs, HTML and Microsoft Word, Excel, Powerpoint directly into Amazon OpenSearch Service. The second type of server is the SearchBlox SearchServer. The SearchBlox SearchServer provides ready-to-use, fully customizable search front-ends including faceted search for the indices created by the SearchBlox IndexServer in the Amazon OpenSearch Service.

Setup

❗️

AWS Region

Please make sure to select the same AWS Region in all the following steps mentioned. For example, we have chosen "us-east-1" for creating opensearch, SearchBlox IndexServer , SearchBlox SearchServer, etc.

1. Create VPC

Create a VPC, which needs to be mentioned while creating a SearchBlox IndexServer at AWS Marketplace.

Instructions:

1. Log in to your AWS Console.
2. Navigate to VPC Dashboard.
3. Click on “Create VPC”.
4. Select VPC only.
5. Provide a Name tag, such as SearchBloxVPC.
6. Choose IPv4 CIDR block (e.g., 10.0.0.0/16).
7. Leave other settings as default unless you have specific network requirements.
8. Click Create VPC.

2. Create KeyPair

Create a Key Pair and store it safely. It will be used to access your AWS instance.

Instructions:

1. Navigate to the EC2 Dashboard.
2. In the left-hand menu, click on “Key Pairs”.
3. Click “Create key pair”.
4. Enter a name, like SearchBloxKey.
5. Click Create key pair and download the file.

3. Create IAM Role

Create an IAM role named SearchBlox_AmazonES and attach the AmazonESFullAccess policy to it, as shown in the screenshot.

This role must be configured after launching the SearchBlox IndexServer (and Search Server, if available) from the AWS Marketplace.

This role grants the necessary permissions for the instance to interact with Amazon OpenSearch.

Instructions:

• Go to the IAM Console.
• Click “Roles” → Create role.
• Choose:
o Trusted entity type: AWS service
o Use case: EC2
• Click Next to attach permissions.

• Search and select AmazonESFullAccess policy.
• Click Next, then give your role a name: SearchBlox_AmazonES.
• Click Create role.

4. Create AWS Opensearch Domain

Instructions:

• Navigate to the AWS Management Console.
• Search for and open “Amazon OpenSearch Service”.
• Click “Create domain” on the dashboard.
• Enter the domain name
• Choose a Domain Creation Method

• You will see two options under “Domain creation method”:
• o Standard create (Default — manual setup step by step)
  o Quick create (Preconfigured settings)

Deployment Option

• Domain with standby:
Keeps extra standby nodes ready. If something goes wrong, it automatically switches to those nodes. This option offers higher availability (99.99%) and better reliability.

• Domain without standby:
Distributes nodes across availability zones without standby. It’s a lower-cost option.

• Select the instance type for your nodes, define the desired number of nodes, and configure the storage settings—including type, size, and IOPS

• Select the master node instance type and specify the number of master nodes.

• Select the access and security type—either VPC access for private networking or Public access if you want the domain to be reachable over the internet.

• Choose an authentication method from the options below, if needed.

• Enable all encryption options to ensure secure communication and protect your data both in transit and at rest.

• Set an off-peak window to allow AWS to perform maintenance during periods of low traffic.

• Enable Auto-Tune to allow Amazon OpenSearch to automatically optimize your cluster performance based on workload patterns.

• Once all settings are configured, Click ‘Create’ to begin provisioning the OpenSearch domain.
• After approximately 10 to 15 minutes, the OpenSearch Service Dashboard will display your newly created domain. Once available, you will see the following status indicators:"

o Domain processing status: Active
o Configuration change status: Completed
o Cluster health: Green

These status indicate that the domain is fully operational and healthy

---

5.Launch SearchBlox IndexServer

• In the AWS Management Console, navigate to the EC2 service.
• Click Launch Instance to begin the process of creating a new EC2 instance.
• In the Amazon Machine Image (AMI) section, click on "Browse more AMIs".

• Select the AWS Marketplace AMIs tab.
• In the search bar, type “SearchBlox Enterprise Search”

Once you’ve found the SearchBlox Enterprise Search AMI, click on it to select it. Then, configure the instance details such as instance type, network settings, and storage.

• Ensure your security group allows ports 8443, 8444, 8446, and 9200 (TCP) for inbound traffic.
• After configuring the instance, proceed to review and launch it.

Integrate IAM Role with SearchBlox Server

To allow the EC2 instance to access Amazon OpenSearch:

• Go to the EC2 Dashboard and find the SearchBlox IndexServer instance.
• Right-click the instance (or select it and click Actions).

• Choose Instance Settings →Security→ Modify IAM Role.

• Select the IAM role you created earlier
• Click Update IAM Role to finish attaching the role.

SSH into the SearchBlox IndexServer

• Connect to the SearchBlox Server instance using SSH with the ec2-user and your .pem or .ppk key file.
• Switch to the SearchBlox user account:
sudo su – SearchBlox
• Open the awssettings.yml file to update the OpenSearch domain details:
/opt/searchblox/webapps/ROOT/WEB-INF/awssettings.yml

• The aws.region should match the region where you created both the SearchBlox IndexServer and the OpenSearch instance.

• The aws.url should be set to the OpenSearch domain endpoint provided in the AWS OpenSearch dashboard.
• Update searchblox.yml
o Open the following file:
o /opt/searchblox/webapps/ROOT/WEB-INF/searchblox.yml
o Update the deployment type by modifying the following line:
o searchblox.deployment.type: AWS

• Save the file after making the changes.

• Restart SearchBlox as follows:

• Access the SearchBlox Admin Console at https://xxxx:8443/console where xxxx is the Public DNS of the SearchBlox IndexServer instance.

• Access the SearchBlox Search URL as follows:

SearchBlox Search url : https://xxxx:8443/search
where xxxx is the Public DNS of the SearchBlox SearchServer instance.

Increase RAM memory for SearchBlox in AWS.

• To allocate or modify RAM for SearchBlox, edit the JVM configuration file located at:
/opt/searchblox/start.d/jvm.ini
• Adjust Values (e.g., -Xms16g -Xmx16g) based on your EC2 instance’s total RAM.

---

Kibana and Amazon OpenSearch Service

• Data indexed, as well as logs, are stored in the Opensearch domain. To view the logs, you can map the Opensearch index named sbindexlog in Kibana and search for the entries.
  The Kibana link will be available in the Domain dashboard. Refer to the following screenshot:

• Click the link and access Kibana.

• Adding log indices in Kibana.
  The two logs that can be added in Kibana are sbindexlog and sbstatuslog. You can add both logs in one index pattern.

Alternatively, you can create a separate index pattern for each log.

You can also query the logs based on URL, timestamp, etc.

• It is also possible to delete indices via Kibana. Go to Dev Tools in the left-hand menu. * To delete the Opensearch indices, click Get to Work .