Installing Private LLM on CentOS/RHEL
Follow the following steps to install Private LLM on RedHat/CentOS:
-
Install OpenJDK 17 by running the following command:
sudo yum install java-17-openjdk -y
-
Install System Monitoring Tool by running the following command:
sudo dnf copr enable atim/bottom -y
sudo dnf install bottom -y
-
Install unzip tool by running the following command:(If already installed skip this step)
sudo yum install unzip -y
-
Private LLM has to be installed in /opt folder, so change the directory to
/opt
by running the following command:
cd /opt
-
Move the Private LLM Package(
privatellm.zip
) collected from SearchBlox Support Team into/opt/
directory. -
Unzip the
privatellm.zip
file inside/opt/
directory, by running the following command:
cd /opt/unzip privatellm.zip
-
Create a SearchAI Chatbot user by running the following command:
useradd searchai
-
To create a service file for Private LLM edit the file
/etc/systemd/system/privatellm.service
with the following scripts:[Unit] Description=Private LLM Requires=network.target remote-fs.target After=network.target remote-fs.target [Service] Type=simple User=searchai WorkingDirectory=/opt/privatellm/ ExecStart=/usr/bin/java -jar /opt/privatellm/privatellm.jar Restart=on-failure RestartSec=10 [Install] WantedBy=multi-user.target
-
To create a service file for CPP server, edit the file
/etc/system/system/cppserver.service
with the following scripts:[Unit] Description=Private LLM Requires=network.target remote-fs.target After=network.target remote-fs.target [Service] Type=simple User=searchai WorkingDirectory=/opt/privatellm/ ExecStart=/usr/bin/bash /opt/privatellm/start_cpp_server.sh Restart=on-failure RestartSec=10 [Install] WantedBy=multi-user.target
-
Change permission for service files by running the following commands:
sudo chmod -R 755 /etc/system/system/cppserver.service
sudo chmod -R 755 /etc/systemd/system/privatellm.service
-
Start CPP service by running the following command:
systemctl start cppserver
-
Start Private LLM service by running the following command:
systemctl start privatellm
NOTE:
Both Private LLM and CPP service must be running for the chatbot to function correctly.
- Stop Private LLM and CPP service by running the following command:
systemctl stop cppserver
systemctl stop privatellm
Updated 4 months ago