Installing Private LLM on Ubuntu

Follow the following steps to install Private LLM on RedHat/CentOS:

  1. Install OpenJDK 17 by running the following command:
    sudo apt install openjdk-17-jdk -y

  2. Install System Monitoring Tool by running the following command:
    sudo apt update -y
    sudo apt install bottom -y

  3. Install unzip tool by running the following command:(If already installed skip this step)
    sudo apt-get install unzip -y

  4. Private LLM has to be installed in /opt folder, so change the directory to /opt by running the following command:
    cd /opt

  5. Move the Private LLM Package(privatellm.zip) collected from SearchBlox Support Team into /opt/ directory.

  6. Unzip the privatellm.zip file inside /opt/ directory, by running the following command:
    cd /opt/unzip privatellm.zip

  7. Create a SearchAI Chatbot user by running the following command:
    useradd searchai

  8. To create a service file for Private LLM edit the file /etc/systemd/system/privatellm.service with the following scripts:

    [Unit]
    Description=Private LLM
    Requires=network.target remote-fs.target
    After=network.target remote-fs.target
    
    [Service]
    Type=simple
    User=searchai
    WorkingDirectory=/opt/privatellm/
    ExecStart=/usr/bin/java -jar /opt/privatellm/privatellm.jar
    Restart=on-failure
    RestartSec=10
    
    [Install]
    WantedBy=multi-user.target
    
  9. To create a service file for CPP server, edit the file /etc/system/system/cppserver.service with the following scripts:

    [Unit]
    Description=Private LLM
    Requires=network.target remote-fs.target
    After=network.target remote-fs.target
    [Service]
    Type=simple
    User=searchai
    WorkingDirectory=/opt/privatellm/
    ExecStart=/usr/bin/bash /opt/privatellm/start_cpp_server.sh
    Restart=on-failure
    RestartSec=10
    [Install]
    WantedBy=multi-user.target
    
  10. Change permission for service files by running the following commands:
    sudo chmod -R 755 /etc/system/system/cppserver.service
    sudo chmod -R 755 /etc/systemd/system/privatellm.service

  11. Start CPP service by running the following command:
    systemctl start cppserver

  12. Start Private LLM service by running the following command:
    systemctl start privatellm

    🚧

    NOTE:

    Both Private LLM and CPP service must be running for the chatbot to function correctly.

  13. Stop Private LLM and CPP service by running the following command:
    systemctl stop cppserver
    systemctl stop privatellm