Gemma 4 31B
Google · 30.7B params · Apache 2.0Q4Quant
18 GBDownload
~33tok/s est.
Best local LLMs for RTX 4090 (24GB)
With RTX 4090, 24GB VRAM the strongest open models that run well are Gemma 4 31B, Gemma 4 E4B. Speeds are estimates based on memory bandwidth.
Your machine
Showing models for RTX 4090, 24GB VRAM
Chat and writing
Top pick
Official model page
Qwen3.6 27B
Qwen · 27B params · See model cardQ5Quant
19 GBDownload
~31tok/s est.
Check the license on the model card before commercial use.
Official model pageQwen3.6 35B A3B
Qwen · 35B params · See model cardQ4Quant
21 GBDownload
~336tok/s est.
Check the license on the model card before commercial use.
Official model pageGemma 4 26B A4B
Google · 25.2B params · Apache 2.0Q5Quant
18 GBDownload
~223tok/s est.
Mixture of experts: 25.2B total, 3.8B active per token, so it runs fast for its size.
Official model pageCoding
Gemma 4 31B
Google · 30.7B params · Apache 2.0Q4Quant
18 GBDownload
~33tok/s est.
Qwen3.6 27B
Qwen · 27B params · See model cardQ5Quant
19 GBDownload
~31tok/s est.
Check the license on the model card before commercial use.
Official model pageQwen3.6 35B A3B
Qwen · 35B params · See model cardQ4Quant
21 GBDownload
~336tok/s est.
Check the license on the model card before commercial use.
Official model pageQwen3 Coder 30B A3B
Qwen · 30.5B params · Apache 2.0Q4Quant
18 GBDownload
~305tok/s est.
Mixture of experts: 30.5B total, 3.3B active per token, so it runs fast for its size.
Official model pageSmall and fast
Gemma 4 E4B
Google · 8B params · Apache 2.0Q8Quant
8 GBDownload
~126tok/s est.
Mixture of experts: 8B total, 4.5B active per token, so it runs fast for its size.
Official model pageMinistral 3 8B
Mistral · 8B params · Apache 2.0Q8Quant
8 GBDownload
~71tok/s est.
Qwen3 8B
Qwen · 8.2B params · Apache 2.0Q8Quant
9 GBDownload
~69tok/s est.
Qwen3 4B
Qwen · 4B params · Apache 2.0Q8Quant
4 GBDownload
~142tok/s est.
Quick facts
How much memory can an RTX 4090 (24GB) use for AI models?
About 23 GB of its 24 GB, after leaving room for the system. That is our estimate of what is usable for model weights.
What is the largest model an RTX 4090 (24GB) can run?
In our list, Qwen3.6 35B A3B at Q4 (about 21 GB) fits. Big models may run slowly, so check the speed estimate.
What is the fastest good model on an RTX 4090 (24GB)?
Qwen3.6 35B A3B at Q4 runs at an estimated 336 tokens per second, based on memory bandwidth.
Other setups: Mac mini M4 Pro 64GB MacBook Air M4 16GB MacBook Pro M4 Max 128GB RTX 3060 12GB RTX 5090 (32GB) MacBook Pro M4 Pro 24GB MacBook Pro M4 Pro 48GB Mac Studio M4 Max 64GB RTX 4070 (12GB) RTX 4080 (16GB) PC with 32GB RAM (no GPU)
How the estimates work
Fit: a model fits if its download size plus about 1.5 GB for context fits in usable memory. On a Mac that is about two thirds of RAM (three quarters from 32GB up), since macOS keeps some for itself. On a PC it is your GPU's VRAM minus 1 GB, or 60% of RAM when running on the CPU only.
Quant: we pick the highest quality that fits: Q8 (about 8.5 bits per weight), then Q5 (5.7), then Q4 (4.8). Q4 and Q5 keep most of the quality at a fraction of the size.
Speed (estimate): generating text is limited by memory bandwidth, because every token reads the active weights once. We use bandwidth x 0.6 / bytes per token. Mixture-of-experts models only read their active experts, so they run much faster than their total size suggests. Real speed varies with the app, context length and settings, so treat it as a rough guide.
How to run a local model
Ollama: install from ollama.com, then find the model in the Ollama library and run it with one command. LM Studio: install from lmstudio.ai, search the model name, and pick the quant shown here. On a Mac, LM Studio's MLX builds are usually fastest.
FAQ
What is the best local LLM for a Mac?
It depends on RAM. With 16GB, small models like Gemma 4 E4B or Qwen3 8B run well. With 64GB, mixture-of-experts models like gpt-oss-120b don't fit, but Gemma 4 31B and Qwen3.6 27B do, and 128GB opens up gpt-oss-120b and Mistral Small 4. Pick your Mac above for the exact list.
How much RAM do I need to run a local AI model?
About 0.6 GB per billion parameters at Q4, plus a little for context. An 8B model needs about 6 GB, a 30B model about 20 GB, and a 70B model about 44 GB.
Can I detect my specs automatically?
Partly. Browsers share your platform, CPU core count, a rough RAM hint and sometimes the GPU maker. They hide the exact chip and memory size, so the finder makes a best guess and you confirm it with a tap.
Is a local model cheaper than an API?
If you already own the hardware, local models cost only electricity. For top quality, hosted models still lead; compare them on our AI API prices page.