Deploy Ollama-based GenAI LLMs (AI stack)

First published: Saturday, July 4, 2026 | Last updated: Saturday, July 4, 2026

Deploy Ollama-based GenAI LLMs (AI stack) using the SloopStash Kubernetes kit.


Previous: Deploy Kafka cluster (Data Lake stack)

Deploy and manage AI stack (Ollama-based GenAI LLMs) environments

Each Kubernetes worker node must have atleast 4 GB RAM for better performance while running these Ollama-based GenAI LLMs.

Kubernetes

Prepare Kubernetes worker nodes

# Create directories for Kubernetes persistent-volumes.
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/deep-seek/0/model
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/deep-seek/0/data
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/deep-seek/0/log
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/deep-seek-coder/0/model
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/deep-seek-coder/0/data
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/deep-seek-coder/0/log
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/gemma/0/model
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/gemma/0/data
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/gemma/0/log
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/llama/0/model
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/llama/0/data
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/llama/0/log
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/qwen-coder/0/model
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/qwen-coder/0/data
$ sudo mkdir -p /mnt/sloopstash/${ENVIRONMENT}/ai/ollama/qwen-coder/0/log
$ sudo chmod -R ugo+rwx /mnt/sloopstash

Manage Kubernetes resources

# Switch to SloopStash Kubernetes kit directory.
$ cd /opt/kickstart-kubernetes

# Create Kubernetes namespace.
$ kubectl create namespace sloopstash-${ENVIRONMENT}-ai-s1

# Create Kubernetes storage-class.
$ envsubst < storage-class/ai/ollama.yml | kubectl apply -f -

# Create Kubernetes persistent-volume.
$ envsubst < persistent-volume/ai/ollama/deep-seek.yml | kubectl apply -f -
$ envsubst < persistent-volume/ai/ollama/deep-seek-coder.yml | kubectl apply -f -
$ envsubst < persistent-volume/ai/ollama/gemma.yml | kubectl apply -f -
$ envsubst < persistent-volume/ai/ollama/llama.yml | kubectl apply -f -
$ envsubst < persistent-volume/ai/ollama/qwen-coder.yml | kubectl apply -f -

# Create Kubernetes config-map.
$ kubectl create configmap ollama \
--from-file=workload/ollama/${AI_OLLAMA_VERSION}/conf/ \
--from-file=supervisor-server=workload/supervisor/conf/server.conf \
-n sloopstash-${ENVIRONMENT}-ai-s1

# Create Kubernetes service.
$ kubectl apply -f service/ai/ollama.yml -n sloopstash-${ENVIRONMENT}-ai-s1

# Create Kubernetes stateful-set.
$ envsubst < stateful-set/ai/ollama/deep-seek.yml | kubectl apply -f - -n sloopstash-${ENVIRONMENT}-ai-s1
$ envsubst < stateful-set/ai/ollama/deep-seek-coder.yml | kubectl apply -f - -n sloopstash-${ENVIRONMENT}-ai-s1
$ envsubst < stateful-set/ai/ollama/gemma.yml | kubectl apply -f - -n sloopstash-${ENVIRONMENT}-ai-s1
$ envsubst < stateful-set/ai/ollama/llama.yml | kubectl apply -f - -n sloopstash-${ENVIRONMENT}-ai-s1
$ envsubst < stateful-set/ai/ollama/qwen-coder.yml | kubectl apply -f - -n sloopstash-${ENVIRONMENT}-ai-s1

# List Kubernetes resources.
$ kubectl get sc,pv,ns -o wide

# List resources under Kubernetes namespace.
$ kubectl get pvc,cm,sts,deploy,rs,ds,po,svc,ep,ing -o wide -n sloopstash-${ENVIRONMENT}-ai-s1

# Delete Kubernetes namespace.
$ kubectl delete namespace sloopstash-${ENVIRONMENT}-ai-s1

# Delete Kubernetes persistent-volume.
$ envsubst < persistent-volume/ai/ollama/deep-seek.yml | kubectl delete -f -
$ envsubst < persistent-volume/ai/ollama/deep-seek-coder.yml | kubectl delete -f -
$ envsubst < persistent-volume/ai/ollama/gemma.yml | kubectl delete -f -
$ envsubst < persistent-volume/ai/ollama/llama.yml | kubectl delete -f -
$ envsubst < persistent-volume/ai/ollama/qwen-coder.yml | kubectl delete -f -

# Delete Kubernetes storage-class.
$ envsubst < storage-class/ai/ollama.yml | kubectl delete -f -

Ollama

Run DeepSeek

# Access Bash shell of OCI container (Ollama - DeepSeek).
$ kubectl exec -ti -n sloopstash-${ENVIRONMENT}-ai-s1 ollama-deep-seek-0 -c main -- /bin/bash

# Pull DeepSeek GenAI LLM from Ollama registry.
$ ollama pull deepseek-r1:1.5b

# Run DeepSeek GenAI LLM using Ollama.
$ ollama run deepseek-r1:1.5b

# Exit shell.
$ exit

Run DeepSeek coder

# Access Bash shell of OCI container (Ollama - DeepSeek coder).
$ kubectl exec -ti -n sloopstash-${ENVIRONMENT}-ai-s1 ollama-deep-seek-coder-0 -c main -- /bin/bash

# Pull DeepSeek coder GenAI LLM from Ollama registry.
$ ollama pull deepseek-coder:1.3b

# Run DeepSeek coder GenAI LLM using Ollama.
$ ollama run deepseek-coder:1.3b

# Exit shell.
$ exit

Run Gemma

# Access Bash shell of OCI container (Ollama - Gemma).
$ kubectl exec -ti -n sloopstash-${ENVIRONMENT}-ai-s1 ollama-gemma-0 -c main -- /bin/bash

# Pull Gemma GenAI LLM from Ollama registry.
$ ollama pull gemma2:2b

# Run Gemma GenAI LLM using Ollama.
$ ollama run gemma2:2b

# Exit shell.
$ exit

Run Llama

# Access Bash shell of OCI container (Ollama - Llama).
$ kubectl exec -ti -n sloopstash-${ENVIRONMENT}-ai-s1 ollama-llama-0 -c main -- /bin/bash

# Pull Llama GenAI LLM from Ollama registry.
$ ollama pull llama3.2:3b

# Run Llama GenAI LLM using Ollama.
$ ollama run llama3.2:3b

# Exit shell.
$ exit

Run Qwen coder

# Access Bash shell of OCI container (Ollama - Qwen coder).
$ kubectl exec -ti -n sloopstash-${ENVIRONMENT}-ai-s1 ollama-qwen-coder-0 -c main -- /bin/bash

# Pull Qwen coder GenAI LLM from Ollama registry.
$ ollama pull qwen2.5-coder:3b

# Run Qwen coder GenAI LLM using Ollama.
$ ollama run qwen2.5-coder:3b

# Exit shell.
$ exit

Previous: Deploy Kafka cluster (Data Lake stack)