How do you get into this? Any articles you can share? OP says you can run on your desktop... How? Doesn't this stuff require huge data centers?

what kinda hardware do you need to "run this on your desktop"?

A P40 with 24GB is ~150€, a V100 (32GB) is ~600€. Both of these fit Qwen3.6 27B (The P40 is about 3x slower though). The V100 even fits 400k context with a Q4 KV-Cache , which means you can have two slots for parallel processing (llama-cpp). You don't even have to use system memory.
One of my inference servers is running with 8gb of DDR3 and a 2nd Gen i7, so my old hardware has a good use again.

thanks, good stuff! sadly, got none of these locally available.

It’s a bit misleading.

Qwen 27B has way less “world knowledge” than GPT-5. Ask it random trivia without internet search access, and GPT would know waaay more.

This is generally true of small vs large models.

…But honestly, Qwen 27B is better at tool use or agentic stuff. It’s hyper optimized for just that and coding assistance, basically.

This is often true of old vs new. Most newer models have hyper focused on agents/coding, often to the detriment of other use cases.

Quantization for practically running Qwen 27V also has an impact. A off-the-shelf Q4_K_M is not the same as the unquantized weights in real-world use, or even an “optimized” quantization like a custom exl3.

Sure, a benchmark doesn't capture all the subtleties and different use cases, but it does give a general idea of the capabilities of a model. Obviously, you have to run the model and see if it does what you need. But the chart isn't really about the nuance, it's showing how drastically the efficiency of the models has improved in just a year. The fact that we can even reasonably compare a model you can run on a desktop to one that needed a data center just a year ago is phenomenal.

i see what your saying. i didnt mean to discredit standard benchmarks entirely.
i guess its obvious that it measures capability regardless of imprecision.
2 major proposed changes:
**first, i dont really know. aside from saying "benchmark your own prompt+usecase"
a proposed plan:

• approach one: pay attention and credit new or improved architecture designs and research.
• approach two: spend more attention on benchmarks. especially specific benchmarks ( that are not focused with industrial domain tasks.) **domain task pursuit, is useful!.. but it depends on if your interest align to popular domains.
• approach three: if willing to utilize remotely hosted models. rating should also take in consideration.. tools and everything else: websearch performance, RAG performance, smooth interface, pref/balance between speed vs comprehensiveness, cost (if relevant), etc.. .

Running model that is good at everything require huge amount of energy and huge data center. Those models are mixture of experts. Latest Kimi K3 have 896 experts. Imagine you have company with 896 employees. Each question involves 16 employees to figure out what to do in what area of your business. Like a brainstorm to solve problem. Now if you know exactly what you want and in which area you actually need only 1-5 people. Like an agile team instead of all those people that you have. So you can hire just couple Kimi K3 experts. 16 experts are 100B parameters so roughly 1 expert in frontier open source model is 6B parameters. 5 experts is 30B parameters. You can run 27B Qwen 3.6 quantized into int4 on your computer like other people are doing right now.

I posted link below to example where they fine tuned model ( take it like a employee training ) for specific task.

It’s a bit misleading.

Qwen 27B has way less “world knowledge” than GPT-5. Ask it random trivia without internet search access, and GPT would know waaay more.

This is generally true of small vs large models.

…But honestly, Qwen 27B is better at tool use or agentic stuff. It’s hyper optimized for just that and coding assistance, basically.

This is often true of old vs new. Most newer models have hyper focused on agents/coding, often to the detriment of other use cases.

Quantization for practically running Qwen 27V also has an impact. A off-the-shelf Q4_K_M is not the same as the unquantized weights in real-world use, or even an “optimized” quantization like a custom exl3.