CroZone
Release wireHermes Agent v0.23.3  ·  Stable channel patchImmich v3.4.2  ·  Library monitoring fixes
Guides

Which Qwen3.8 quant belongs on your GPU?

Seven actual GGUF downloads show what GSQ-RCO, IQ3_S, Q3_K_S and ternary packing buy you on a 24 GB card.

CroZone · Published Nov 20, 2026
In this article
Qwen3.8-27B family · Publisher-reported text GGUF sizes
BuildFile sizeSource
GSQ-RCO IQ2_XS 8.40 GBnpario
GSQ-RCO IQ2_S 9.30 GBnpario
GSQ-RCO IQ3_XXS 10.10 GBnpario
GSQ-RCO IQ3_S 11.80 GBnpario
Q3_K_S 12.74 GBbartowski
Ternary Q2_0 7.70 GBoshiteku
Ternary PTQ1_0 6.00 GBoshiteku

Checked Nov 20, 2026 · Decimal GB · Excludes the vision projector and runtime memory.

Decode the filename

In npario’s GSQ-RCO builds, GSQ (Gumbel-Softmax Quantization) chooses how to represent weights within a tensor. RCO (Riemannian Constrained Optimization) decides which tensors receive more precision under a file-size budget. IQ3_S is the resulting build label. Its whole-file average is 3.50 bits per weight.

What does “Q3S” mean?

Check the full suffix and publisher before downloading. The GSQ-RCO IQ3_S file is 11.8 GB. Bartowski’s Q3_K_S is a distinct K-quant build at 12.74 GB. Even matching suffixes across repositories can hide different tensor layouts. Keep the exact filename alongside your results.

Ternary weights, different runtimes

A ternary weight takes one of three values: negative, zero or positive, with scaling. Sanshoku is a ternary derivative of Qwen3.8-27B. Its publisher offers the same weights in several packings. Q2_0 is 7.70 GB and lists upstream llama.cpp support. PTQ1_0 shrinks the file to 6.00 GB and requires the PrismML fork. Changing packing alone gives no evidence of better answers.

The first trial on a 24 GB GPU

Start with IQ3_S if you want room for context. Run the same prompt, context length and tool schema against your current higher-precision build. Record peak VRAM, prompt processing speed, generation speed, and whether the answer passes the same check. Try the smaller ternary build if memory or concurrency is the limiting factor. A GGUF download size excludes the live context cache, runtime buffers and any separate vision projector.

Turn the table into a shortlist

On a 24 GB card, test a known-good quant first and record its peak memory with your normal workload. The numbers above describe bytes on disk, reported in decimal GB. Your GPU monitor may use MiB or GiB. Subtracting a decimal download size directly from a binary monitor total gives only a rough impression. The runtime may also place some tensors on the CPU, keep multiple models resident or allocate cache differently as context grows.

Choose two candidates that answer different questions. A GSQ-RCO IQ3_S trial asks whether you can retain the quality you need with a smaller file. A ternary trial asks whether you can trade quality or compatibility for more room. On each candidate, confirm the exact file loads in your installed runtime. Run at least one short prompt and one near your expected context length. Record whether all layers stay on the GPU and whether a second conversation pushes the setup over its memory budget.

Score the failures you would notice in daily use

For coding, use a small task with an existing test and inspect the patch. For structured extraction, parse the result and check fields against the source document. For long-context evaluation, ask a question about a fact buried in the middle of a fixture. Use the same prompt and reasoning settings, and repeat runs if generation is variable. Record warm and cold runs separately so the time to load weights does not dominate a generation-speed comparison. If a candidate breaks tool formatting, treat that as a failure even when its prose looks good. Compare the actual task pass rate before deciding which candidate earns the smaller footprint.

Primary sources

npario · GSQ-RCO files and evaluation · bartowski · GGUF layouts · oshiteku · Sanshoku ternary builds

Read next   Local or cloud? Follow the work.

Search CroZone