I tested 42 Ollama models on a Raspberry Pi 5 (16GB RAM)

1 Thinking Mode is a Killer on CPU

Models with built-in "thinking mode" (qwen3, qwen3.5, deepseek-r1 series) generate long internal reasoning chains before answering. On a CPU-only Raspberry Pi 5, this is devastating:

• A 0.8B model (qwen3.5:0.8b) takes 250 seconds to answer "What is 2+2?" — it generates thousands of characters of "Thinking..." text before the actual answer
• A 9.7B model (qwen3.5:9b) times out at 300 seconds on every single test
• Even a tiny 751M model (qwen3:0.6b) takes 36 seconds — 4x slower than a comparable non-thinking model

The exceptions: ministral-3:8b (24s) and lfm2.5:8b (45s) both have thinking mode but are remarkably fast. These are the only thinking-mode models I'd consider usable on RPi 5.

2 Q4_K_M is the Sweet Spot

Quantization matters enormously on CPU:

• Q4_K_M (4-bit): Fast, good quality, the default for most models
• Q8_0 (8-bit): 2x larger, 2x slower — qwen3.5:0.8b with Q8_0 takes 250s vs what Q4 would be ~125s
• Q4_0 (basic 4-bit): Slightly faster but lower quality than Q4_K_M

If a model only offers Q8_0 (like qwen3.5:0.8b and qwen3.5:2b), think twice before running on Pi.

3 8B Models Are Actually Usable

This was the biggest surprise. On RPi 5 with 16GB RAM:

• llama3.1:8b — 37s average, all 5 tests passed
• minicpm-v4.5:8b — 43s, all passed
• qwen2.5:7b — 46s, all passed
• mistral:7b — 50s, all passed

37 seconds for an 8B model on a $80 board is genuinely impressive. They use 4–6GB RAM, leaving plenty for the OS and context window.

4 The 3B Sweet Spot

If I had to pick one model for daily use on RPi 5, it would be llama3.2:3b:

• 14 seconds average response time
• 3.2B parameters — smart enough for most tasks
• Only 2.8GB RAM usage
• All 5 tests passed including Czech

At 14 seconds, it's only 5 seconds slower than 1B models, but with 3x the parameters. That's the sweet spot.

5 Code Models Are Slow

Specialized coding models (codegemma, starcoder2) are noticeably slower than general models of similar size:

• codegemma:7b — 30s (but 9B params, 7GB RAM)
• codegemma:2b — 51s (Q4_0, slower quantization)
• starcoder2:3b — 85s, timed out on Czech
• starcoder2:7b — 125s, timed out on Czech

For coding on RPi 5, codegemma:7b is the best option at 30s, but it's heavy (7GB RAM).

6 12B Works! (Barely)

gemma3:12b completed all tests in 42s average using 9.3GB RAM. That's 12.2 billion parameters running on a Raspberry Pi. Not fast, but it works and leaves enough RAM for the OS (~6GB free with 16GB total).

7 Vision Models Are Impractical

Both vision models tested (qwen3-vl:2b and qwen3-vl:4b) combine vision capability with thinking mode, making them extremely slow (176s and 210s). Several tests timed out. If you need vision on RPi 5, you'd need to find a non-thinking vision model.