Phi-3 Small vs Gemma 2 vs Mistral Small: Edge AI Benchmarks
10
benchmarked small models for edge deployment. latency, accuracy, and memory usage comparison
6 replies
6 Replies
Join the discussion.
Log In to Reply
lmao i literally ran into this same problem yesterday. running llama 3 locally and its basically as good as the api for most tasks
ive been doing this for about 6 months now and the biggest lesson ive learned is gguf format basically won the local model format war. its not what I expected at all going in
ok real talk - quantized models have gotten insanely good. barely notice the quality drop. i know thats not the popular opinion here but someone had to say it