Running on AWS Fargate

📘

Important:

• For Single ECS, use IP license.
• For managed ECS, Unlimited License is required. For the license, please contact SearchBlox support.

To use Fargate with ECS to run a series of containerized tasks or steps, follow the below steps:

🚧

Important:

• We can get Image URI in two ways:

1. Getting a Image URI on AWS Marketplace
2. Importing Custom Docker Image into Registry

NOTE: You can follow any one of the above step and Creating a Fargate Cluster is common for both the above steps.

Getting Image URI from AWS Marketplace

• Go to AWS Marketplace and Subcribe to SearchBlox Enterprise Search, by clicking the below link.
• Link to Fargate version
• Click on Continue to Subscribe.

• Click on Continue to Configuration.

• Click on Continue to Launch

• Scroll down to the Container images section and copy the commands that are shown below as shown in the image.

aws ecr get-login-password  
    --region us-east-1 | docker login  
    --username AWS  
    --password-stdin xxxxxxx.dkr.ecr.us-east-1.amazonaws.com

CONTAINER_IMAGES="xxxxxx.dkr.ecr.us-east-1.amazonaws.com/searchblox/searchblox:v10.7"   

for i in $(echo $CONTAINER_IMAGES | sed "s/,/ /g"); do docker pull $i; done

• After executing the above command the below response can be seen and copy the last line, which is the Image URI used for the next steps.

v10.7: Pulling from searchblox/searchblox
Digest: sha256:a13abcd5f970e253c8be2b7a093c215728046d2f4a0505343710a80bae39801d
Status: Image is up to date for xxxxxx.dkr.ecr.us-east-1.amazonaws.com/searchblox/searchblox:v10.7
xxxxxx.dkr.ecr.us-east-1.amazonaws.com/searchblox/searchblox:v10.7

Importing Custom Docker Image into the Registry

• Sign in to the AWS Management Console and navigate to the Amazon Elastic Container Service console.

• After selecting the ECS Container, select Repositories on the left pane. You will be redirected to Amazon Elastic Container Registry.

• Select Create repository.

Choose the image tag mutability setting:

• Mutable
  Allows image tags to be overwritten. Suitable during active development.
• Immutable
  Prevents tag overwrites. Recommended for production use to avoid accidental updates.

Note: Name should be same as the Docker image.

• Leave everything as default and Create repository.
• Once Repository is Successfully created, select View push commands.

• Read the instructions carefully on the page below.

Note: Highlighted text in the above image needs to be changed as per the Docker image Tag name.

Follow the steps below to authenticate, build, tag, and push your Docker image to Amazon ECR.
Note: Ensure that the AWS CLI and Docker are installed and properly configured on your terminal before proceeding.

1. Authenticate Docker to Your ECR Registry
   Run the following command to retrieve an authentication token and log in to your Amazon ECR registry:
   aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin .dkr.ecr.us-east-1.amazonaws.com
2. Build the Docker Image
   Use the command below to build your Docker image:
   docker build -t searchblox-ai-v11.0 .
3. Tag the Docker Image
   Tag the image with your ECR repository URI so it's ready to be pushed:
   docker tag searchblox-ai-v11.0:latest .dkr.ecr.us-east-1.amazonaws.com/searchblox-ai-v11.0:latest
4. Push the Image to Amazon ECR
   Finally, push your image to the ECR repository:
   docker push .dkr.ecr.us-east-1.amazonaws.com/searchblox-ai-v11.0:latest

Once the push is complete, your image will be available in the ECR repository and ready to use in your ECS task definition.

Creating Fargate Cluster

• Navigate to the ECS Console.
• Select Clusters to create a new cluster.

• Select Create cluster.

• Enter the Cluster name.
• Under Infrastructure, Select AWS Fargate (serverless).

Under Monitoring, you can enable CloudWatch Container Insights, which provides detailed metrics for your tasks and containers.

• Container Insights with enhanced observability:
Select this option if you want detailed metrics on CPU, memory, network, and disk usage at the task and container level. This helps in troubleshooting and performance optimization.

• Container Insights:
Provides basic metrics at the cluster and service level. Suitable for less intensive monitoring needs.

• Turned off:
Monitoring is disabled.

• Click on the Create button to create the cluster

Create a Task Definition

• Navigate back to the Amazon ECS main menu, then select Task Definitions from the sidebar

• Under Task definitions, select Create new task definition

• Specify a Task definition family and choose AWS FARGATE

• Choose Linux/86_64 and specify the CPU size as 16 CPU and Memory as 64GB (As per the docker image size).

• Create a task role in AWS IAM. Allow full permissions for ECS and Cloud Watch.

• To create a role in AWS, navigate to IAM service, click on left-side panel roles. Click on Create a Role. Select Trusted entity type as AWS Service and select usecase as lamda, click on Next button.

• In the Add Permission section search for AmazonECSTaskExecutionRolePolicy.

• Select the ecsTaskExecutionRole created in IAM console.

NOTE: Task size should be 16 CPU and 64GB Memory.

Under Container-1 settings, specify a Name for the container and paste the Image URI copied from the ECR repositories page.

Example:
<AWS_ACCOUNT_ID>.dkr.ecr..amazonaws.com/:

• Under Port mappings, add the following container port numbers: 8443, 8444, 8446, and 9200 and set the App protocol as HTTP.

• Allocate Memory hard limit as 40, Memory soft limit as 35 and the CPU cores as 16 CPU, can be increased based on the requirements.

• Set the storage size to a minimum of 100 GB, as shown in the image below

• Select Create.

Run the Task

• After creating the task definition, the following page will appear.

• Click on the Deploy dropdown menu and select Run task to launch the container.

• Choose the existing cluster name.
• Under Compute options, select Capacity provider strategy

Under Capacity provider strategy,

• choose FARGATE as the capacity provider.
• Set Base to 1
• Set Platform revision to LATEST

• Specify the number of desired tasks to be launched. This value can be adjusted based on your scaling needs.

• Ensure the task name is unique to avoid conflicts within the cluster.

• Under Networking, choose the default VPC ,Subnets or based on your specific requirements.

• Select Create new security group, then provide a Security group name and Security group description.

Under Inbound rules, do the following:
• Set Type to Custom TCP
• Add the following Port Range values: 8443, 8444, and 9200
• Set Source to Anywhere (0.0.0.0/0) — or restrict it based on your security requirements.

• Ensure that Public IP is set to Enabled to allow external access to the task.
• Select Create.

• After the task is created, navigate to the task page shown below.
• Monitor the task’s progress under the Last status column.
• Wait until the task status changes to Running to confirm that it is up and running successfully.

• Once the task status is COMPLETED, you can find the Public IP address under the Network bindings section.

• To access the SearchBlox console, use the Public IP on the browser
• https://:8443/console

Important:

If you are not able to access the SearchBlox Console, check the security group.

🚧

Important:

• If you are not able to access the SearchBlox Console, check the security group.