DeviceMark — On-Device LLM Leaderboard changelog https://devicemark.github.io/changelog.html Board updates: new models, new device measurements, methodology changes. en Scoring made bit-reproducible; two upstream IFEval items disclosed https://devicemark.github.io/methodology.html#ci devicemark-2026-07-14-determinism Tue, 14 Jul 2026 02:00:00 GMT The official IFEval checkers are non-reproducible unseeded (langdetect, hash order, and upstream items 1122/1129 whose non-alphabet letters the checker replaces with a random letter). All pinned; the board now regenerates bit-identically from the published raw outputs. Values shifted at most 0.3pp, inside every CI. v0 launch — 8 entries, full battery, device-measured https://devicemark.github.io/ devicemark-2026-07-13-v0 Sun, 13 Jul 2026 06:00:00 GMT 8 entries (5 open-model vendors + Apple's built-in Foundation Model), full 596-item battery, 95% CIs, 7/8 rows device-measured on iPhone 17 Pro. Findings: five-way statistical tie at the top; Google's official QAT int4 at parity on MMLU-Pro/MATH; quantization loss not monotone with size. Dataset: https://huggingface.co/datasets/devicemark/results Per-model badges https://devicemark.github.io/changelog.html devicemark-2026-07-13-badges Sun, 13 Jul 2026 09:00:00 GMT Embeddable per-row badges at /badge/<slug>.svg — intelligence composite + device tok/s, regenerated on every deploy.