It’s a bit misleading.

Qwen 27B has way less “world knowledge” than GPT-5. Ask it random trivia without internet search access, and GPT would know waaay more.

This is generally true of small vs large models.

…But honestly, Qwen 27B is better at tool use or agentic stuff. It’s hyper optimized for just that and coding assistance, basically.

This is often true of old vs new. Most newer models have hyper focused on agents/coding, often to the detriment of other use cases.

Quantization for practically running Qwen 27V also has an impact. A off-the-shelf Q4_K_M is not the same as the unquantized weights in real-world use, or even an “optimized” quantization like a custom exl3.

I mean baking knowledge into a model isn't really all that useful to begin with. Just download wikipedia locally and have it access it through tool use, it's way more efficient and more accurate. And yeah, I find Q6 tends to be the sweet spot where it's close enough to full 16 bit in performance, but doesn't chew up too much memory.

This is for qwen 3.5, not 3.6, but the vram requirements are a little out there for most:

Qwen 3.5 27B VRAM Requirements — Dense Model Hardware Guide (Q4/Q5/Q6/Q8) | Will It Run AI Blog

Qwen 3.5 27B needs ~16.5 GB at Q4_K_M on RTX 4090. See also the newer Qwen3.6-27B (April 22, 2026) which needs 16.8 GB Q4 and beats it on coding benchmarks.

Will It Run AI (willitrunai.com)

16 to 55GB...

You don't want to run these models in RAM. I started using them on an RTX 3060-12Gb VRAM, and quickly added another, for 24Gb RAM. I also have 64 Gb RAM. Now it works well. Not lightning fast, but quite useable. VRAM is the key.

I bought two RTX 3060 with 12 Gb each for a total of 24Gb VRAM. Sionce the cards are not top shelf anymore for gaming, they can be had used in the 200€ range, so my setup was 400€ over the cost of my existing PC. My PC, while not last gen was already pretty beefy, with a Ryzen 9 and 64 Gb RAM. I think 400€ to have a pretty good local AI setup is not bad at all.

What version do you use and how do you run Qwen3.6? I've played around a bit with the Q4 version in LM studio +Zed, but I was not happy with the results. It looses track very often and often enters infinite loops or just stops...

I very much regret getting a 3070, with only 8GB of ram... Only reason I am hesitant about 3060s is that dual cards will use more power, and the economics are already pretty dicey when the hosted models are being sold below cost.

I learnt a few years back that the most important driver in GPU. or any other non upgradable memory device obsolescence, is RAM. I got a banger of a deal a few years on a RX 480, but it was 4Gb, adequate at the time. The card was obsolete way faster than the 8GB.
As for the power draw, are you planning to have your AI setup running 24/7? Nvtop (linux GPU monitoring app) reports 50-80% of the cards top power draw. There is also the fact that my data doesn't leave my setup.

Which one of them made these overlapping bar labels?

Ok, but all of them are far too inefficient for what they deliver.

This is a pretty good idea I hadn't thought about. The whole wikipedia is about 37Gb.

Kimi is better. Waiting for it to appear on Ollama Cloud.

Those models are for general use. If you have business use case and data related to it you can finetune model for specific use that will outperform all of frontier models and run at fraction of cost.

what kinda hardware do you need to "run this on your desktop"?

Full quantisation? I've only got a 8GB 3070, but I'll give it a go

Edit: Tried the unsloth/qwen3.6 with llama.CPP, and it failed to allocate a 26GB Vulcan buffer and died. Dunno what magic your using, no luck for me though

I use an exl3, with 4 bit MLPs but higher bit depth attention layers. And I force some custom sampling so I can lower the temperature a bit while keeping it out of loops.

This won’t work in LM Studio though. You have to run such a thing in TabbyAPI or some other backend that supports exllamav3.