Deploy Ollama-based GenAI LLMs (AI stack)
First published: Saturday, July 4, 2026 | Last updated: Saturday, July 4, 2026Deploy Ollama-based GenAI LLMs (AI stack) using the SloopStash Kubernetes kit.
|
Previous: Deploy Kafka cluster (Data Lake stack) |
Deploy and manage AI stack (Ollama-based GenAI LLMs) environments
Each Kubernetes worker node must have atleast 4 GB RAM for better performance while running these Ollama-based GenAI LLMs.
Kubernetes
Prepare Kubernetes worker nodes
# Create directories for Kubernetes persistent-volumes.
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/deep-seek/0/model
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/deep-seek/0/data
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/deep-seek/0/log
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/deep-seek-coder/0/model
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/deep-seek-coder/0/data
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/deep-seek-coder/0/log
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/gemma/0/model
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/gemma/0/data
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/gemma/0/log
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/llama/0/model
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/llama/0/data
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/llama/0/log
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/qwen-coder/0/model
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/qwen-coder/0/data
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/qwen-coder/0/log
$ sudo chmod -R ugo+rwx /mnt/sloopstash
Manage Kubernetes resources
# Switch to SloopStash Kubernetes kit directory.
$ cd /opt/kickstart-kubernetes
# Create Kubernetes namespace.
$ kubectl create namespace sloopstash-${ENVIRONMENT}-ai-s1
# Create Kubernetes storage-class.
$ envsubst < storage-class/ai/ollama.yml | kubectl apply -f -
# Create Kubernetes persistent-volume.
$ envsubst < persistent-volume/ai/ollama/deep-seek.yml | kubectl apply -f -
$ envsubst < persistent-volume/ai/ollama/deep-seek-coder.yml | kubectl apply -f -
$ envsubst < persistent-volume/ai/ollama/gemma.yml | kubectl apply -f -
$ envsubst < persistent-volume/ai/ollama/llama.yml | kubectl apply -f -
$ envsubst < persistent-volume/ai/ollama/qwen-coder.yml | kubectl apply -f -
# Create Kubernetes config-map.
$ kubectl create configmap ollama \
--from-file=workload/ollama/${AI_OLLAMA_VERSION}/conf/ \
--from-file=supervisor-server=workload/supervisor/conf/server.conf \
-n sloopstash-${ENVIRONMENT}-ai-s1
# Create Kubernetes service.
$ kubectl apply -f service/ai/ollama.yml -n sloopstash-${ENVIRONMENT}-ai-s1
# Create Kubernetes stateful-set.
$ envsubst < stateful-set/ai/ollama/deep-seek.yml | kubectl apply -f - -n sloopstash-${ENVIRONMENT}-ai-s1
$ envsubst < stateful-set/ai/ollama/deep-seek-coder.yml | kubectl apply -f - -n sloopstash-${ENVIRONMENT}-ai-s1
$ envsubst < stateful-set/ai/ollama/gemma.yml | kubectl apply -f - -n sloopstash-${ENVIRONMENT}-ai-s1
$ envsubst < stateful-set/ai/ollama/llama.yml | kubectl apply -f - -n sloopstash-${ENVIRONMENT}-ai-s1
$ envsubst < stateful-set/ai/ollama/qwen-coder.yml | kubectl apply -f - -n sloopstash-${ENVIRONMENT}-ai-s1
# List Kubernetes resources.
$ kubectl get sc,pv,ns -o wide
# List resources under Kubernetes namespace.
$ kubectl get pvc,cm,sts,deploy,rs,ds,po,svc,ep,ing -o wide -n sloopstash-${ENVIRONMENT}-ai-s1
# Delete Kubernetes namespace.
$ kubectl delete namespace sloopstash-${ENVIRONMENT}-ai-s1
# Delete Kubernetes persistent-volume.
$ envsubst < persistent-volume/ai/ollama/deep-seek.yml | kubectl delete -f -
$ envsubst < persistent-volume/ai/ollama/deep-seek-coder.yml | kubectl delete -f -
$ envsubst < persistent-volume/ai/ollama/gemma.yml | kubectl delete -f -
$ envsubst < persistent-volume/ai/ollama/llama.yml | kubectl delete -f -
$ envsubst < persistent-volume/ai/ollama/qwen-coder.yml | kubectl delete -f -
# Delete Kubernetes storage-class.
$ envsubst < storage-class/ai/ollama.yml | kubectl delete -f -
Ollama
Run DeepSeek
# Access Bash shell of OCI container (Ollama - DeepSeek).
$ kubectl exec -ti -n sloopstash-${ENVIRONMENT}-ai-s1 ollama-deep-seek-0 -c main -- /bin/bash
# Pull DeepSeek GenAI LLM from Ollama registry.
$ ollama pull deepseek-r1:1.5b
# Run DeepSeek GenAI LLM using Ollama.
$ ollama run deepseek-r1:1.5b
# Exit shell.
$ exit
Run DeepSeek coder
# Access Bash shell of OCI container (Ollama - DeepSeek coder).
$ kubectl exec -ti -n sloopstash-${ENVIRONMENT}-ai-s1 ollama-deep-seek-coder-0 -c main -- /bin/bash
# Pull DeepSeek coder GenAI LLM from Ollama registry.
$ ollama pull deepseek-coder:1.3b
# Run DeepSeek coder GenAI LLM using Ollama.
$ ollama run deepseek-coder:1.3b
# Exit shell.
$ exit
Run Gemma
# Access Bash shell of OCI container (Ollama - Gemma).
$ kubectl exec -ti -n sloopstash-${ENVIRONMENT}-ai-s1 ollama-gemma-0 -c main -- /bin/bash
# Pull Gemma GenAI LLM from Ollama registry.
$ ollama pull gemma2:2b
# Run Gemma GenAI LLM using Ollama.
$ ollama run gemma2:2b
# Exit shell.
$ exit
Run Llama
# Access Bash shell of OCI container (Ollama - Llama).
$ kubectl exec -ti -n sloopstash-${ENVIRONMENT}-ai-s1 ollama-llama-0 -c main -- /bin/bash
# Pull Llama GenAI LLM from Ollama registry.
$ ollama pull llama3.2:3b
# Run Llama GenAI LLM using Ollama.
$ ollama run llama3.2:3b
# Exit shell.
$ exit
Run Qwen coder
# Access Bash shell of OCI container (Ollama - Qwen coder).
$ kubectl exec -ti -n sloopstash-${ENVIRONMENT}-ai-s1 ollama-qwen-coder-0 -c main -- /bin/bash
# Pull Qwen coder GenAI LLM from Ollama registry.
$ ollama pull qwen2.5-coder:3b
# Run Qwen coder GenAI LLM using Ollama.
$ ollama run qwen2.5-coder:3b
# Exit shell.
$ exit
|
Previous: Deploy Kafka cluster (Data Lake stack) |