Server-only Offline Coding Models
Open-weight models on an existing server can provide coding help without a per-token cloud bill. A frontier model can orchestrate bounded workers when available; the worker should also have a documented standalone path. Downloading weights and dependencies requires connectivity initially. Inference can subsequently run offline if all runtime dependencies and auxiliary services remain inside the boundary.
This workflow targets dedicated inference guests and private GPU servers. Model weights and generated code stay on the selected server; no inference on a developer laptop or a virtualization hypervisor is needed.
Start with hardware and model identity
Section titled “Start with hardware and model identity”Record CPU, usable RAM, GPU type, dedicated VRAM, shared memory, free disk, quantization, runtime revision, and competing workloads. Integrated-GPU shared memory is not equivalent to the same amount of dedicated VRAM. Do not consume production guest capacity to run a benchmark: use an approved separate guest and leave existing services unchanged.
For mixture-of-experts models, active parameters help describe computation, but all required weights still need memory or offload. Context and concurrency add memory demand. Mistral’s offline-model guidance recommends Devstral Small 2, a dense 24B model, for coding and distinguishes its hardware needs from the much larger 119B Mistral Small 4.
Candidate matrix
Section titled “Candidate matrix”The following eight candidates illustrate different server memory and reasoning tradeoffs. These are primary-source specifications checked on 2026-10-19, not a measured coding-quality ranking. Seven upstream model cards identify Apache 2.0; GLM-4.7-Flash identifies MIT. The separately linked GGUF repositories identify the quantization publisher.
| Upstream model | Published context | Thinking and reasoning controls |
|---|---|---|
| Qwen3-Coder-30B-A3B-Instruct | 262,144 native; optional YaRN extension | Nonthinking only; no reasoning-effort levels |
| Qwen3.5-9B | 262,144 native; extension to 1,010,000 | Thinking on/off; named effort levels not established by the reviewed card |
| gpt-oss-20b | 131,072 | Reasoning low, medium, high; no documented off mode; requires Harmony formatting |
| Devstral-Small-2-24B-Instruct-2512 | Advertised 256K; configuration differs, see note below | Configurable thinking/effort levels not documented in the reviewed card |
| Qwen3.8-27B | 262,144 native; extension to 1,000,000 | Thinking on/off; effort low, medium, xhigh (default), not a uniform CLI level list |
| Qwen3-Coder-Next | 262,144 native | Nonthinking only; no reasoning-effort levels |
| Qwen3.6-35B-A3B | 262,144 native; extension to 1,010,000 | Thinking on/off; named effort levels not established by the reviewed card |
| GLM-4.7-Flash | Configuration allows 202,752 positions; independent card limit not established here | Thinking on/off; named effort levels not established |
Qwen3.5, Qwen3.6, and Qwen3.8 can disable thinking through chat_template_kwargs.enable_thinking=false when supported by the serving framework. Qwen3.5 does not officially support Qwen3’s soft prompt switches. Qwen3.6/3.8 also control retention of reasoning history separately with preserve_thinking. For gpt-oss, the server must correctly implement Harmony and map the requested reasoning level; assigning a client label of off does not create a supported model mode.
GLM’s pinned configuration is not evidence of a locally qualified 200K service. Its template supports enable_thinking=false; preserving thinking history is a separate control.
Devstral’s card advertises a 256K window, while its configuration has max_position_embeddings=393216 and its generation configuration has total max_length=262144. These fields do not establish a separately qualified 393,216-token service or an independent maximum output length.
Pinned quantized weight files
Section titled “Pinned quantized weight files”These are downloaded weight-file sizes, not RAM or VRAM requirements. Context caches, runtime buffers, concurrent slots, operating-system memory, and other workloads need additional capacity. GB uses 10⁹ bytes; GiB uses 2³⁰ bytes. Each link pins the quantization repository revision; sizes are rounded from the selected files, not the repository’s entire contents.
| Model | Quantization publisher and pinned revision | Weights GB / GiB |
|---|---|---|
| Qwen3-Coder-30B-A3B-Instruct | Unsloth Q4_K_M | 18.557 / 17.282 |
| Qwen3.5-9B | Unsloth Q4_K_M | 5.681 / 5.290 |
| gpt-oss-20b | ggml-org MXFP4 | 12.110 / 11.278 |
| Devstral Small 2 | Unsloth Q4_K_M | 14.334 / 13.350 |
| Qwen3.8-27B | Unsloth UD-Q4_K_M | 16.464 / 15.334 |
| Qwen3-Coder-Next | Qwen Q4_K_M, all four shards | 48.411 / 45.086 |
| Qwen3.6-35B-A3B | Unsloth UD-Q4_K_M | 22.135 / 20.614 |
| GLM-4.7-Flash | Unsloth Q4_K_M | 18.312 / 17.055 |
Upstream model identity, license, and capabilities are distinct from a converter’s artifact metadata. Pin every selected file’s size and SHA256, verify downloaded bytes before activation, and do not overwrite a known-good canonical file. A few active MoE parameters do not make the total weight footprint disappear.
For inference on an owned server, the provider per-token price is $0 for all eight. This excludes hardware purchase or rental, electricity, cooling, storage, administration, and the server’s other uses. A third party hosting the same model may charge for its service; these zero prices do not describe hosted APIs.
Advertised context versus usable context
Section titled “Advertised context versus usable context”Record three separate limits: the model’s published maximum, the runtime’s allocated context per slot, and the client’s configured context. The client must not advertise more than the served allocation. A model card saying 256K does not make an 8K runtime accept 256K tokens.
Thinking and sampling are model-specific
Section titled “Thinking and sampling are model-specific”A bounded first screen
Section titled “A bounded first screen”The CPU screen pins llama.cpp v0.7.0 at commit 620d2225. One reproducible starting profile uses 8 CPU threads, 8,192 context tokens, 1,024 generated tokens per response, one slot, and concurrency one. These are test allocations, not the models’ published limits or a claim of best quality. Use the served 8,192-token limit in the client preset even when the model card advertises a much larger window. Keep total session-token accounting separate from context size.
A short response budget can exhaust itself on reasoning before producing code. Qwen3.5’s card advises substantially more context for thinking workloads. Record truncation separately and give finalists an explicitly budgeted larger-context/output experiment before drawing broader conclusions.
| Model | Initial test mode | Temperature / top-p / top-k |
|---|---|---|
| Qwen3-Coder-30B-A3B-Instruct | Nonthinking | 0.7 / 0.8 / 20 |
| Qwen3.5-9B | Nonthinking; presence-penalty departure noted below | 0.7 / 0.8 / 20 |
| gpt-oss-20b | Reasoning low | 1.0 / 1.0 / 0 (disabled) |
| Devstral Small 2 | No claimed effort level | 0.15 / 0.95 / 20 |
| Qwen3.8-27B | Nonthinking | 0.7 / 0.8 / 20 |
| Qwen3-Coder-Next | Nonthinking | 1.0 / 0.95 / 40 |
| Qwen3.6-35B-A3B | Nonthinking | 0.7 / 0.8 / 20 |
| GLM-4.7-Flash | Nonthinking; controlled departures below | 0.7 / 1.0 / 0 (disabled) |
The screen adapter explicitly sends min_p=0 for every model. Qwen3.5/3.6/3.8 publish that value; for the other reviewed models it is a controlled test choice. Coder-30B explicitly receives repetition penalty 1.05; Qwen3.6 and Qwen3.8 explicitly receive presence penalty 1.5. Other penalties are omitted, using the pinned runtime’s defaults of repetition penalty 1.0 and presence penalty 0. The Qwen3.5 screen therefore departs from its published nonthinking general-task presence penalty of 1.5. Record that departure rather than calling it the complete vendor recipe. Devstral’s top-p/top-k and gpt-oss’s disabled top-k are controlled test choices. Devstral’s examples/generation defaults specify temperature 0.15. OpenAI’s gpt-oss repository recommends temperature 1.0 and top-p 1.0.
GLM’s official coding evaluations use temperature 0.7 and top-p 1.0, without establishing a separate nonthinking-only recipe. Its disabled top-k is a controlled choice; the adapter’s min_p=0 also differs from the linked quantization publisher’s recommendation of 0.01. These departures need measurement, not an assumption of equivalent behavior.
Do not silently reuse these profiles for thinking runs. Qwen3.8’s thinking profile uses temperature 1.0, top-p 0.95, top-k 20, and presence penalty 0. Qwen3.5 documents separate thinking recipes for general and precise coding tasks; its two nonthinking-reasoning recipe sections also disagree, so record the exact chosen recipe. The table uses its nonthinking general-task temperature/top-p/top-k values, with the presence-penalty departure documented above.
Output recommendations are not hard maxima
Section titled “Output recommendations are not hard maxima”| Model | What the reviewed primary source establishes |
|---|---|
| Qwen3-Coder-30B-A3B-Instruct | Recommends 65,536 output tokens; separate hard maximum not stated |
| Qwen3.5-9B | Recommends 32,768 normally and 81,920 for complex problems; separate hard maximum not stated |
| gpt-oss-20b | OpenAI lists 131,072 maximum output tokens, subject to the shared context budget including prompt and reasoning |
| Devstral Small 2 | Independent hard output maximum not established by the reviewed card; total max_length is not max_new_tokens |
| Qwen3.8-27B | Suggests separate 262,144 reasoning / 131,072 final-response budgets with a supporting framework and 1M context; separate hard maximum not stated |
| Qwen3-Coder-Next | Examples use 65,536 generated tokens; separate hard maximum not stated |
| Qwen3.6-35B-A3B | Independent hard output maximum not established in this comparison |
| GLM-4.7-Flash | Coding evaluations use 16,384 generated tokens; independent hard maximum not established |
None of those larger values is allocated by the 1,024-token first screen. Inspect the actual request and server template: client settings, runtime defaults, and effective inference limits can differ.
Carry the profile into the client
Section titled “Carry the profile into the client”Direct benchmark requests do not configure an interactive agent automatically. Vibe 3.0’s Unified Harness sends configured top_p and model output caps, but its generic chat adapter omits effort for thinking = "off" rather than sending enable_thinking=false. Verify thinking at the endpoint, and supply any required top_k, min_p, penalty, and seed settings through verified server defaults or explicit request translation. Generic reasoning history is preserved; template history controls remain separate. Vibe’s max label is not a model-independent synonym for xhigh. See generic request details.
Keep a per-response cap, such as max_output_tokens = 1024, separate from the CLI’s cumulative prompt-plus-completion run budget. Reported-usage guards can overshoot; an external supervisor and server resource limits provide independent containment. See run-limit semantics.
Use separate qualification gates
Section titled “Use separate qualification gates”- Runtime: verify pinned binaries, model checksums, health, usable context, service isolation, and the effective request limits.
- Protocol: send a small text request and a synthetic tool-selection request. Validate structured arguments without executing the proposed tool.
- Function screening: use the same bounded, independently checked problems for every model. Keep tests out of the prompt. Run correct and deliberately broken controls to prove the oracle can pass and fail.
- Coding worker: require a fresh read/edit/test canary through the actual agent configuration, not just direct completion calls.
- Repository acceptance: inspect the exact diff, rerun independent checks, and review sensitive changes before any merge or deployment.
Specify accepted source/module formats before grading. Keep harness or loader errors and model protocol failures separate from logic failures; preserve original outputs when revalidating a corrected harness.
Give model startup/readiness and each response separate deadlines. Loading cold weights can consume the readiness budget before a prompt is sent; record that as a startup/readiness failure with no coding score, rather than a failed implementation. Extending readiness alone must leave task prompts, response deadlines, context, and output caps unchanged.
Run generated functions as an unprivileged validator with a read-only job directory, no provider credentials or host home mounts, no network, and explicit memory/CPU/process/output/time caps. Keep the inference service separate from this validator. A worktree is useful for review; it does not establish these security boundaries.
Retain per-task correctness, first-pass result, model/settings, tokens, prefill/generation timings, wall time, finish reason, truncation, timeout, and failed checks. Run sequentially to avoid resource contention. For final timings, wait for downloads and builds to finish, record other shared-host workloads, and distinguish cold versus cached weight loading. Record CPU/thread allocation and cache conditions with the result; a cached first load is not a general startup guarantee. Repeats help identify variability; a handful of synthetic functions cannot establish full repository-agent reliability or final security-review competence.
CPU-only case study: a bounded first screen
Section titled “CPU-only case study: a bounded first screen”Measured on 2026-10-19–20 UTC.
A CPU-only guest with 16 vCPUs and 64 GiB RAM, backed by an Intel Xeon Gold 6242R, screened the eight pinned model/quantization combinations listed above. The initial inference setting used eight threads, an 8,192-token configured context, a 1,024-token response cap, and a 120-second response deadline. Model readiness had a separate deadline; a later profile allowed 300 seconds for startup while retaining the same response deadline, tasks, context, and output cap.
Each model received one attempt at each of four synthetic functions distilled from coding problem types: secret-name checks, verified identity, private-row visibility, and calendar validation. These were small executable checks, not complete repositories or end-to-end agent tasks. Raw passes below count only the original four-function suite; returned code and unavailable outputs are shown separately. Rows are alphabetical, not a quality ranking. This case reports aggregates from private fixtures; the exact contracts and source outputs are not published here, so it is not an independently reproducible public leaderboard.
| Model | Raw passes / 4 scheduled | Returned function sources | Other outcomes |
|---|---|---|---|
| Devstral Small 2 | 1 / 4 | 4 / 4 | Three executable-check failures |
| GLM-4.7-Flash | 3 / 4 | 4 / 4 | One executable-check failure |
| GPT-OSS-20B | 2 / 4 | 3 / 4 | One loader failure; one protocol failure. Corrected-loader revalidation of the unchanged saved source gives a separate 3 / 4 result. |
| Qwen3-Coder-30B | 2 / 4 | 4 / 4 | Two executable-check failures |
| Qwen3-Coder-Next | 3 / 4 | 4 / 4 | One executable-check failure under the later startup profile |
| Qwen3.5-9B | 2 / 4 | 4 / 4 | Two executable-check failures |
| Qwen3.6-35B-A3B | 3 / 4 | 4 / 4 | One executable-check failure |
| Qwen3.8-27B | 1 / 4 | 1 / 4 | Three response deadlines; unranked for coding quality |
Qwen3-Coder-Next’s earlier 120-second readiness attempt produced no coding responses. That attempt is a startup/readiness outcome with no coding score; the table uses its completed 300-second readiness profile. GPT-OSS-20B’s protocol failure returned no candidate to validate. A response deadline or protocol failure supplies no evidence that an implementation was logically wrong.
Separate authorization follow-up
Section titled “Separate authorization follow-up”A separately scored follow-up reused unchanged saved code, without new model calls. Of 16 authorization-task records, 13 contained eligible source; three had no returned source because of response deadlines or a protocol failure. The eligible sources yielded six passes when rechecked against the original frozen tests. Added explicit-contract assertions yielded one pass out of 13 eligible outputs: Qwen3-Coder-Next’s identity implementation. All four positive/negative validator controls behaved as expected.
These supplemental results do not replace the initial four-function scores. The added checks covered selected session-ID types, zero IDs, literal account linkage, visibility, and limited mutation checks; ambiguous ownership normalization stayed unscored. Passing this limited follow-up did not resolve every finding from static review or establish full-prompt compliance.
Separate short throughput comparison
Section titled “Separate short throughput comparison”After downloads and builds finished, GLM-4.7-Flash and Qwen3.6-35B-A3B ran serial CPU-only profiles on 2026-10-20 UTC, using the pinned quantizations above. The pinned llama-bench method separately measured 128-token prompt processing (pp128) and 32-token generation (tg32) at eight and 16 threads, with three repetitions per measurement. The table reports means in tokens per second.
| Model | Threads | pp128 (tokens/s) | tg32 (tokens/s) |
|---|---|---|---|
| GLM-4.7-Flash | 8 | 59.16 | 11.35 |
| GLM-4.7-Flash | 16 | 88.09 | 9.18 |
| Qwen3.6-35B-A3B | 8 | 58.47 | 8.76 |
| Qwen3.6-35B-A3B | 16 | 97.90 | 9.19 |
Default warmup remained enabled. There was no cache flush or cache-state isolation; these were shared-host measurements with load and available-memory snapshots, not controlled cold-load tests. llama-bench excludes tokenization and sampling time. These short tests do not measure full 8K-context latency, API or agent overhead, coding quality, or long-generation reliability.
Eight threads were retained for comparable, conservative quality repeats and lower thread allocation. This does not mean eight threads were universally fastest: GLM generated faster at eight, while Qwen3.6 gained about 5% in generation and 67% in prompt processing at 16, using twice as many threads. GLM’s 16-thread prompt-processing gain was about 49%. Choose separately for prompt-heavy and generation-heavy workloads, then verify with representative requests.
Separate repeatability and guided-repair checks
Section titled “Separate repeatability and guided-repair checks”GLM-4.7-Flash and Qwen3.6-35B-A3B then repeated the same four functions at eight threads, retaining the initial context, response cap, sampling settings, and 120-second response deadline. Each model used three fixed seeds: 19, 47, and 73. Seed 19 reproduced the initial source outputs; the other two seeds added samples. These are repeated samples of four contracts, not 12 distinct tasks or three independent trials.
| Model | Seed 19 passes / 4 | Seed 47 passes / 4 | Seed 73 passes / 4 | Total passes / 12 |
|---|---|---|---|---|
| GLM-4.7-Flash | 3 / 4 | 1 / 4 | 1 / 4 | 5 / 12 |
| Qwen3.6-35B-A3B | 3 / 4 | 3 / 4 | 3 / 4 | 9 / 12 |
All 24 function responses returned without response timeouts or output truncation. All returned sources received independent static review. The main checks missed explicit-contract defects in some passing implementations, including identity validation and a secret-name check that depended on an environment value. One Qwen visibility implementation had no defect identified in the focused review; that limited result does not certify the complete model or agent.
A separate frontier-guided GLM repair attempt supplied its original failed visibility function and specific guidance, while retaining the contract and checking both the original suite and added explicit requirements. It returned code but failed the original suite before the supplemental assertions were reached. This is one failed guided attempt, not evidence that supervision can never help; its score does not replace the initial results.
A final GLM attempt reused that guided prompt and the same checks, with thinking enabled, a 2,048-token response cap, and a 300-second response deadline. It returned nonempty reasoning but no code, ending with finish_reason=length at 2,048 completion tokens. No candidate was available for validation. Usage did not provide a separate reasoning-token count. This was output-budget exhaustion, not a logical test failure or an HTTP timeout. Thinking mode, output cap, and response deadline changed together, so this single attempt cannot isolate the effect of thinking or establish a generally useful thinking budget. The earlier results remain unchanged.
Qwen3.6 is the more promising CPU candidate to qualify next within this narrow setup. Its repeated main-suite result was stronger than GLM’s, but all three Qwen identity implementations still had supported contract defects in static review. Use small mechanical tasks with executable checks and independent review; these results do not establish unattended repository-agent reliability or suitability for final security review.
What the screen establishes
Section titled “What the screen establishes”Passing the small suite did not establish full-prompt compliance. In the initial eight-model screen, independent static review of every returned authorization implementation found supported edge-case defects against explicit requirements, including implementations that passed the original checks. Original scores were preserved; ambiguous or unrequested semantics were left as unscored caveats. Retain independent frontier-model review and human approval for security-sensitive changes, supported by executable project checks.
This was direct function-generation screening. It did not qualify Vibe’s complete agent loop, repository editing, tool execution, or unattended operation. Initial timing observations also shared host resources with staging or builds, so they do not support a clean startup-speed ranking. The separate short throughput comparison above has a different measurement scope. Repeat measurements should record thread allocation, shared workloads, and cold versus cached loading separately.
Before publishing results
Section titled “Before publishing results”Publish only approved aggregate results. Keep private endpoints, infrastructure availability, credentials, and repository-source-bearing prompts or outputs out of public documentation.
Before presenting a comparison, check that every included candidate completed the same executable checks under recorded settings. Report the number of tasks and repeats, correctness, latency, token usage, and failure categories. Keep runtime installation, protocol compatibility, function screening, and full agent qualification as separate evidence layers. Exclude interrupted or incomplete runs from quality rankings and explain the missing evidence without exposing private operational details.