Log In Sign Up
Local & Open Source AI · Posted by Raj Patel ·

Running AI Inference on a Raspberry Pi 5

6

got small language models running on a pi 5. performance is actually usable for specific tasks

what really stood out to me was gguf format basically won the local model format war

2 replies

2 Replies

8

wait really? quantized models have gotten insanely good. barely notice the quality drop

1

good call. the latency improvement from local is massive for real-time apps