Installing Private LLM on CentOS/RHEL

Follow the following steps to install Private LLM on RedHat/CentOS:

1. Install OpenJDK 17 by running the following command:
   sudo yum install java-17-openjdk -y

2. Install System Monitoring Tool by running the following command:
   sudo dnf copr enable atim/bottom -y
sudo dnf install bottom -y

3. Install unzip tool by running the following command:(If already installed skip this step)
   sudo yum install unzip -y

4. Private LLM has to be installed in /opt folder, so change the directory to /opt by running the following command:
   cd /opt

5. Move the Private LLM Package(privatellm.zip) collected from SearchBlox Support Team into /opt/ directory.

6. Unzip the privatellm.zip file inside /opt/ directory, by running the following command:
   cd /opt/unzip privatellm.zip

7. Create a SearchAI Chatbot user by running the following command:
   useradd searchai

8. To create a service file for Private LLM edit the file /etc/systemd/system/privatellm.service with the following scripts:

   privatellm.service

   [Unit]
   Description=Private LLM
   Requires=network.target remote-fs.target
   After=network.target remote-fs.target
   
   [Service]
   Type=simple
   User=searchai
   WorkingDirectory=/opt/privatellm/
   ExecStart=/usr/bin/java -jar /opt/privatellm/privatellm.jar
   Restart=on-failure
   RestartSec=10
   
   [Install]
   WantedBy=multi-user.target
   
   

9. To create a service file for CPP server, edit the file /etc/system/system/cppserver.service with the following scripts:

   cppserver.service

   [Unit]
   Description=Private LLM
   Requires=network.target remote-fs.target
   After=network.target remote-fs.target
   [Service]
   Type=simple
   User=searchai
   WorkingDirectory=/opt/privatellm/
   ExecStart=/usr/bin/bash /opt/privatellm/start_cpp_server.sh
   Restart=on-failure
   RestartSec=10
   [Install]
   WantedBy=multi-user.target
   

10. Change permission for service files by running the following commands:
    sudo chmod -R 755 /etc/system/system/cppserver.service
sudo chmod -R 755 /etc/systemd/system/privatellm.service

11. Start CPP service by running the following command:
    systemctl start cppserver

12. Start Private LLM service by running the following command:
    systemctl start privatellm

🚧

NOTE:

Both Private LLM and CPP service must be running for the chatbot to function correctly.

13. Stop Private LLM and CPP service by running the following command:
    systemctl stop cppserver
systemctl stop privatellm