Homelab Copilot Checkpoint 0, a local AI box that actually uses the GPU

I revived a spare desktop, put Proxmox on it, and got Ollama plus Open WebUI running on an Ubuntu VM with real RTX 3070 inference. What worked, what is still loose, and why Intune is next.

homelab proxmox ollama intune

Goal

I wanted to know what a local AI server feels like on hardware I already owned. Not a cloud demo. Not a screenshot of someone else's rig. A spare desktop that had been sitting dead, brought back as a small proof of concept I could break safely.

The same box also becomes the place for an Intune playground later. First I needed the host alive and a local model path that was honestly using the GPU.

What it is

Homelab Copilot Checkpoint 0 is the first public milestone of that journey.

  1. Host runs Proxmox VE 9.2.2 on a revived spare desktop.
  2. Guest VM ubuntu-ai runs Ubuntu Server 26.04.1 LTS with 4 vCPU, 8 GiB RAM and 100 GiB disk.
  3. NVIDIA driver 595.99.02 is installed so nvidia-smi sees the RTX 3070 8 GB.
  4. Ollama 0.40.15 serves qwen3:4b (Q4_K_M, about 2.5 GB).
  5. Open WebUI 0.13.0 (main-slim) runs via Docker Compose, published on localhost only.
  6. Access from Windows is over an SSH local forward. The UI is not LAN exposed.
  7. Ollama listens on the Docker bridge only, not on all interfaces.

Host kit for honesty: Ryzen 5 5600X, about 16 GB usable RAM (one stick), RTX 3070 8 GB, 1 TB Gen4 NVMe.

What I actually saw

During chat, nvidia-smi dmon showed SM utilisation up to 100%, memory utilisation around 93%, about 3.2 GB of framebuffer in use, power around 219 W and temperature around 75°C. That is real GPU inference, not CPU pretending.

A /no_think hint in the prompt helped responsiveness. I am not claiming thinking is fully disabled.

How I keep it contained (for now)

  1. Open WebUI stays on localhost on the VM.
  2. I reach it from Windows with an SSH local forward.
  3. Ollama is bound to the Docker bridge only.
  4. No RAG, no agents, no external APIs in this milestone.

Honest limits

This is a proof of concept. It is not production hardened.

  1. CORS is still wide open (*). That is a known follow up, not a feature.
  2. Usable host RAM is about 16 GB because one stick is out. Treat that as the real ceiling for now.
  3. The model is a small Qwen quant. It proves the path. It is not a claim about training or large model serving.
  4. Network path details stay private. The public story is localhost plus SSH forward, not a topology dump.

Weak spots

  1. Security hardening is incomplete while CORS stays open.
  2. One bad RAM stick still limits what I can stack on the host.
  3. Intune lab work has not started on this box yet.

What I'd change next

  1. Harden the AI stack (CORS and related exposure).
  2. Add Windows Server VMs for an Intune playground on the same Proxmox host so I can study for MD-102.
  3. Keep weekly Lab notes honest when the limits change.

About demos

Screenshots from this milestone show local chat and GPU telemetry. They are not a claim that the box is ready for untrusted users or the open internet.

← All lab entries