Tech Digest Deep Dive - Aug 25, 2026

Ox Alpha: The Free 1M-Context Mystery Model, Benchmarked With Real Evidence

Utkarsh Bhangale 14 min read2,770 words

Key facts

Ox Alpha (OpenRouter id `stealth/ox-alpha`) is an anonymous, free, 1M-context reasoning model that appeared on OpenRouter and the Nous portal around August 20, 2026 with no lab listed, no HuggingFace weights, and no official benchmark scores - only a provider named "Stealth". It was repeatedly called a frontier coder on the strength of an ~80% score on a 10-task DeepSWE subset, but the same developer later reported a full DeepSWE run at ~63%. This post does what the hype skipped: we ran Ox Alpha and nine other models through one controlled 12-task harness (executed code, not claims), verified the community benchmark numbers against primary sources, and separated our own transparent spot-check from published leaderboard figures so the two are never conflated. Bottom line: on equal footing Ox Alpha posts 11/12, statistically tied with DeepSeek V4 Pro, Kimi K3, GLM-5.3, MiniMax M3 and Qwen 3.8 Max on this set, and every one of those models - including Ox Alpha - failed the same subtle JSON-escape task. Ox Alpha is a strong mid-tier reasoning coder, not the anonymous frontier model the early subset scores implied.

Ox Alpha (OpenRouter id stealth/ox-alpha) is an anonymous, free, 1M-context reasoning model that appeared on OpenRouter and the Nous portal around August 20, 2026 with no lab listed, no HuggingFace weights, and no official benchmark scores - only a provider named "Stealth". It was repeatedly called a frontier coder on the strength of an ~80% score on a 10-task DeepSWE subset, but the same developer later reported a full DeepSWE run at ~63%. This post does what the hype skipped: we ran Ox Alpha and nine other models through one controlled 12-task harness (executed code, not claims), verified the community benchmark numbers against primary sources, and separated our own transparent spot-check from published leaderboard figures so the two are never conflated. Bottom line: on equal footing Ox Alpha posts 11/12, statistically tied with DeepSeek V4 Pro, Kimi K3, GLM-5.3, MiniMax M3 and Qwen 3.8 Max on this set, and every one of those models - including Ox Alpha - failed the same subtle JSON-escape task. Ox Alpha is a strong mid-tier reasoning coder, not the anonymous frontier model the early subset scores implied.

What Ox Alpha Is

How Big It Got (Real Adoption Data)

However the benchmarks are read, the adoption numbers are not in dispute. OpenRouter's own Apps tab and global token rankings show Ox Alpha received enormous real production traffic almost immediately:

The capacity claim behind it is also striking and operator-supplied: roughly 100 trillion tokens per day, a figure that would put the operator among the largest inference providers anywhere (Business Insider notes it is ~100x Visa's stated monthly AI-token volume). We repeat that as an operator claim, not a verified spec. The traffic you can check yourself; the architecture you cannot.

The Hype vs The Retraction

The two headline numbers are real community claims, but both are small-sample and neither is officially published.

80% on a 10-task DeepSWE subset. Developer Ben Davis reported ~80% on a 10-task subset of DeepSWE, a long-horizon software-engineering benchmark, with reference models in that subset at ~65% (Fable 5) and ~52% (GPT-5.6 Sol). Davis flagged the sample as small enough to carry real variance.

Then the retraction. On Aug 22 Davis reported the actual full DeepSWE run: ended at ~63%, not the 80% his first subset test got. He clarified the fuller run was done by @winkey_h (DataCurve), described as the actual creator of DeepSWE. On the same thread, another commenter put Ox Alpha "more or less on par with GPT-5.6 Sol mid", while a third (Abe Burnett) said they had not seen Sol-mid performance and compared it to "Terra low" instead.

87.5% on "Kingbench". A community YouTube run reported Ox Alpha at 70/80 = 87.5%, second behind GLM-5.3 at 91.25%. Kingbench is not a standard benchmark: it has no published, auditable methodology and no independent lab runs it.

Officially absent. The OpenRouter model page lists no Artificial Analysis scores, no coding index, and no independent benchmark block. Ox Alpha does not appear on the public DeepSWE/BenchSift leaderboard as of late August 2026.

The honest read: Ox Alpha has no official scores, its most-cited 80% was retracted to ~63% by the same source, and the Kingbench figure is non-standard. Any claim that it tops coding benchmarks is not backed by verifiable evidence.

Who Is Behind It (Circumstantial Only)

Ox Alpha's maker has made no official statement. OpenRouter, which routes it, states plainly that it "is not its developer, owner, or provider" and that the model "is developed and operated by a third-party provider who has chosen to remain anonymous." Zhipu, OpenRouter and the anonymous provider have all declined to confirm or deny (as of Aug 23). Here is the honest state of the evidence:

Bottom line: the weight of forensic evidence tilts toward Zhipu and an unreleased GLM-5.x variant, and the anomalous-release playbook matches, but no named party has confirmed it. We treat this as hypothesis, not fact.

Our Controlled Eval: 12 Identical Tasks, 10 Models

To test Ox Alpha in a way that is reproducible and fair, we built a harness that (1) runs identical prompts and hidden test cases against every model, (2) actually executes generated code in a subprocess and grades it against the hidden tests (not the model's word), and (3) applies the exact same corrected rubric to every model. Plain cargo test-style "it passed" is easy to fake; executed verdicts are not.

The set: 7 coding tasks (two-sum, LRU cache, number of islands, median of two sorted arrays, reverse words, a subtle JSON-string-escaping task, multi-key stable sort), 4 reasoning tasks (river crossing, Monty Hall, sum-of-squares formula, a partial-fractions telescoping trap), and 1 tool-calling task (weather lookup with JSON tool calls).

Service notes: Ox Alpha was measured on the Nous portal (the primary measure) and again as ox-alpha-free on the opencode-go gateway (a control). DeepSeek V4 Flash/Pro, Kimi K3, GLM-5.3, MiniMax M3 and Qwen 3.8 Max ran through the opencode-go gateway. step-3.7-flash:free and tencent/hy3:free ran on the Nous portal. Same prompts, same executors, same rubric throughout.

Table: pass/fail per task

Task OxAlpha DS V4 Pro Kimi K3 GLM 5.3 MiniMax M3 Qwen3.8 Max DS V4 Flash step-3.7 hy3
two_sum P P P P P P P P F
LRU cache P P P P P P P P P
num islands P P P P P P P P P
median 2 arrays P P F P P P P F P
reverse words P P P P P P P P P
JSON escape F F F F F F F F F
multi-key sort P P P P P P P P P
river crossing P P P P P P P P P
Monty Hall P P P P P P P P P
sum of squares P P P P P P P P P
partial fractions P P P P P P P P P
tool weather P P P P P P P P P
Total 11/12 11/12 11/12 11/12 11/12 11/12 11/12 10/12 10/12

(Ox Alpha's ox-alpha-free control: 11/12, identical. The one apparent miss there, partial fractions, was a token-budget truncation: the gateway returned an empty final answer after spending its cap on reasoning; with a larger budget it answered correctly. Both ox-alpha endpoints behave the same when given equal room.)

What this set actually shows

On these 12 tasks every midline-and-up model collapses into a tight 11/12 cluster. The set does not separate frontier coders; it separates one thing: correct JSON string escaping, the single subtle, spec-heavy task that every single model failed. That is a real pattern worth reporting - "frontier" coding chops do not guarantee correctness on fiddly serialization edge cases - but it also means this eval is a coarse screen, not a frontier ranking.

Ox Alpha's 11/12 ties DeepSeek V4 Pro and Kimi K3 and beats the two smaller Nous free models (10/12 each). It does not out-tie the paid/open mids on this easy-ish set. That is consistent with the community full-DeepSWE result (~63%, a strong-but-not-top coder) and NOT with the inflated 80% subset figure.

A methodology warning worth stating plainly: we first ran the harness and scored Ox Alpha 8/12, then found three of those "failures" were bugs in our own grader (LaTeX-rendered numbers not matched, a wrong expected value on our side, and case-sensitivity), not Ox Alpha failures. After correcting the rubric to accept valid alternate phrasings, Ox Alpha is 11/12, and the corrected rubric was then applied identically to all ten models. Raw unfixed evidence and the audit trail are archived next to the harness. We disclose this loudly because a rubric that cannot tell a right answer from a wrong one is worth less than a flattering number.

Published Leaderboard Scores (Separate Axis)

These are third-party published figures for named models, using different benchmarks (SWE-bench+, SWE-bench Verified, Artificial Analysis indices) and different graders than our spot-check. We present them in a separate section on purpose: comparing a 12-task hand-run to a 500-issue SWE-bench run would be apples-to-oranges.

SWE-bench+ (OpenLM, as of Aug 2026): Claude Opus 5 at 97, DeepSeek V4 Pro at 96.4, GPT-5.6 Sol at 96.2, Grok 4.6 at 95.6, GLM-5.3 at 95.4, Claude Fable 5 at 95, Kimi K3 at 93.4, DeepSeek V4 Flash at 88.8, Claude Opus 4.8 at 88.6, GLM-5.2 at 82.8. Source: openlm.ai/swe-bench.

SWE-bench Verified (Vals AI independent harness, July 2026): GPT-5.6 Sol at 96.2, Claude Fable 5 at 95.0, Kimi K3 at 93.4. Source: morphllm.com/best-ai-model-for-coding (referencing Vals AI).

Artificial Analysis Coding Index (Aug 2026): GPT-5.6 Sol (xhigh) leads at 78.3 with Claude Opus 5 (max) at 78.0, close enough to call a tie; DeepSeek V4-Flash 0731 is the price-performance pick. Source: felloai.com/best-ai-models (referencing Artificial Analysis).

The relevant point for Ox Alpha: it has no entry on any of these. Its 87.5% Kingbench and 80% DeepSWE-subset figures run on non-standard or small samples and were not reproduced on official leaderboards. Until a third party runs Ox Alpha on an audited SWE-bench harness, its true coding standing is measured only by small community runs and evaluations like ours.

A Plausible Architecture (Hypothesis, Clearly Labeled)

We cannot see Ox Alpha's internals, so this is inference from observable surface plus circumstantial fingerprints, not a claim about its real construction.

What is observed: 1M context, 131K max output, multimodal input (text/image/video) to text, mandatory reasoning with an effort knob, tool calling, $0 preview, tokenizer labeled "Other", no weights.

If the Zhipu fingerprints hold, the underlying model is plausibly from the GLM-5 family. The GLM-5 paper ("GLM-5: from Vibe Coding to Agentic Engineering", arXiv 2602.15763) describes a sparse Mixture-of-Experts transformer: 744B total / ~40B active, 80 layers, ~256 experts, trained on ~28.5T tokens. Its attention uses Multi-latent Attention (MLA) with a compact latent KV-cache plus DeepSeek-style content-addressed sparse attention (DSA) via a "lightning indexer" - and the paper explicitly rejects naive sliding-window or linear-attention variants as lossy for long-context recall. Context grows 4K to 200K in stages (with GLM-5.2 being the first in the line to claim a solid 1M context with 131,072 max output). A Multi-token-prediction (MTP) draft layer drives speculative decoding.

There is a real tension worth naming: GLM-5.3 (released 6 days before Ox Alpha) is explicitly text-only, and the community's top ask - vision - was not included. Ox Alpha takes text + image + video input. That is precisely why several analysts read Ox Alpha as an unreleased upstream multimodal GLM variant rather than the public GLM-5.3: it would already contain the encoder stack GLM-5.3's public release lacks. It is also worth noting the anomalous-release playbook matches: Zhipu's own GLM-5 first appeared anonymously as "Pony Alpha" in Feb 2026 before Zhipu claimed it.

Independently of the maker, any 1M-context reasoning model at that profile almost certainly uses some combination of: a sparse Mixture-of-Experts decoder (active experts per token to keep serving cost manageable, which is also what makes a free preview economically plausible), long-context attention from content-based sparse attention (DSA-style) rather than a naive sliding window, a compact latent KV-cache so that 1M-token serving and a claimed ~100T tokens/day of capacity are feasible, a shared image + video encoder stack feeding a projector into the language trunk with a text-only decode head, and speculative/MTP decoding to sustain measured throughput around 22 tok/s. A 100T-tokens/day capacity claim, near-100% 3-day uptime and fast time-to-first-token all point to a massive, well-provisioned fleet - consistent with a major lab's preview.

What we explicitly cannot conclude: parameter count, expert layout, exact attention mechanism, training recipe, or maker. All of the above is a hypothesis consistent with the evidence, not a finding.

Bottom Line

Ox Alpha is a genuinely good, free, 1M-context reasoning coder that clears midline tasks cleanly and ties DeepSeek V4 Pro and Kimi K3 on our controlled 12-task run (11/12). But the narrative around it outran the evidence: the celebrated 80% DeepSWE-subset score was retracted to ~63% on the full run by the same developer, the Kingbench figure is non-standard, and it has no official benchmark entries at all. The strongest circumstantial evidence points to Zhipu and an unreleased GLM-5.x variant, but nobody has confirmed the maker. Treat Ox Alpha as what the evidence shows: a strong mid-tier reasoning model that is free - and treat the frontier-coder headline as hype until a third party runs it on an audited, full-size benchmark.

Method And Reproducibility

Sources