VeizikNVIDIA, Apple Silicon and edge builds are available now.Release notes →
veiziknative inference engine
For business Sign inDownload
← Benchmarks overview
Benchmarks · all boards

Measured performance. Real hardware.

Our own numbers, measured in-house on published builds — each with its run count, range and build. Explore every figure, or read the per-platform boards below.

Performance explorer

Model × hardware × mode

Search and filter every published figure. Own numbers only.

Filter or search, then click any row for its run count, min–max range, build and permalink. Our own numbers only — how we compare against other engines →

ModelHardwareDeviceMetricModeWorkloadFigureRunsRangeBuild
Qwen2.5-0.5B VerifiedApple SiliconApple M1 MaxDecode—30,000 / 128≥ 154 tok/s11152.4–161.3Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 UltraDecode—30,000 / 128≥ 199 tok/s11196.9–200.3Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M2 ProDecode—30,000 / 128≥ 110 tok/s11109.8–110.9Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M4 ProDecode—30,000 / 128≥ 200 tok/s10187.4–204.4Measured 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 MaxDecode—24,576 / 128≥ 144 tok/s11142.3–146.1Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 UltraDecode—24,576 / 128≥ 206 tok/s11204.4–207.8Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M2 ProDecode—24,576 / 128≥ 121 tok/s11120.3–121.3Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M4 ProDecode—24,576 / 128≥ 229 tok/s9209.6–232.4Measured 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 MaxDecode—16,384 / 128≥ 171 tok/s11167.6–172.8Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 UltraDecode—16,384 / 128≥ 259 tok/s11257.4–262.0Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M2 ProDecode—16,384 / 128≥ 142 tok/s11139.8–143.2Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M4 ProDecode—16,384 / 128≥ 279 tok/s11263.8–284.9Measured 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 MaxDecode—8,192 / 128≥ 210 tok/s11203.3–212.2Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 UltraDecode—8,192 / 128≥ 281 tok/s11278.7–286.6Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M2 ProDecode—8,192 / 128≥ 193 tok/s11189.2–195.6Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M4 ProDecode—8,192 / 128≥ 353 tok/s10349.0–362.4Measured 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 MaxDecode—3,968 / 128≥ 209 tok/s11207.5–211.6Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 UltraDecode—3,968 / 128≥ 306 tok/s11304.4–313.3Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M2 ProDecode—3,968 / 128≥ 222 tok/s11221.5–223.4Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M4 ProDecode—3,968 / 128≥ 358 tok/s11344.2–360.4Measured 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 MaxDecode—512 / 128≥ 273 tok/s11263.5–282.1Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 UltraDecode—512 / 128≥ 377 tok/s11355.8–385.6Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M2 ProDecode—512 / 128≥ 301 tok/s11299.0–302.5Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M4 ProDecode—512 / 128≥ 463 tok/s11425.7–475.6Measured 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 MaxModel weight footprint——344 MB1single runCarried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 UltraModel weight footprint——344 MB1single runCarried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M4 ProModel weight footprint——344 MB1single runCarried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 MaxPrefill—30,000 / 128≥ 3,434 tok/s113,428–3,546Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 UltraPrefill—30,000 / 128≥ 5,895 tok/s115,892–5,899Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M2 ProPrefill—30,000 / 128≥ 2,071 tok/s112,070–2,072Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M4 ProPrefill—30,000 / 128≥ 2,972 tok/s102,825–3,094Measured 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 MaxPrefill—24,576 / 128≥ 3,859 tok/s113,853–3,862Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 UltraPrefill—24,576 / 128≥ 6,773 tok/s116,765–6,777Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M2 ProPrefill—24,576 / 128≥ 2,356 tok/s112,355–2,357Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M4 ProPrefill—24,576 / 128≥ 3,307 tok/s93,039–3,514Measured 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 MaxPrefill—16,384 / 128≥ 4,829 tok/s114,822–4,838Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 UltraPrefill—16,384 / 128≥ 8,509 tok/s118,488–8,526Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M2 ProPrefill—16,384 / 128≥ 2,948 tok/s112,946–2,948Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M4 ProPrefill—16,384 / 128≥ 4,066 tok/s113,848–4,363Measured 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 MaxPrefill—8,192 / 128≥ 6,381 tok/s116,362–6,411Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 UltraPrefill—8,192 / 128≥ 11,328 tok/s1111,288–11,399Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M2 ProPrefill—8,192 / 128≥ 3,917 tok/s113,914–3,919Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M4 ProPrefill—8,192 / 128≥ 5,804 tok/s105,519–5,822Measured 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 MaxPrefill—3,968 / 128≥ 7,504 tok/s117,428–7,555Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 UltraPrefill—3,968 / 128≥ 13,075 tok/s1112,868–13,274Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M2 ProPrefill—3,968 / 128≥ 4,664 tok/s114,657–4,673Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M4 ProPrefill—3,968 / 128≥ 6,929 tok/s116,856–6,952Measured 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 MaxPrefill—512 / 128≥ 7,901 tok/s116,417–8,103Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M1 UltraPrefill—512 / 128≥ 9,632 tok/s119,088–10,741Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M2 ProPrefill—512 / 128≥ 5,060 tok/s114,985–5,101Carried 2026.10.02-r10
Qwen2.5-0.5B VerifiedApple SiliconApple M4 ProPrefill—512 / 128≥ 7,979 tok/s117,661–7,993Measured 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 MaxDecode—30,000 / 128≥ 91.5 tok/s1190.7–92.2Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 UltraDecode—30,000 / 128≥ 143 tok/s11141.7–143.7Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M2 ProDecode—30,000 / 128≥ 67.8 tok/s1167.5–68.2Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M4 ProDecode—30,000 / 128≥ 109 tok/s7106.7–109.3Measured 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 MaxDecode—24,576 / 128≥ 99.3 tok/s1198.9–100.9Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 UltraDecode—24,576 / 128≥ 146 tok/s11144.8–147.6Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M2 ProDecode—24,576 / 128≥ 74.1 tok/s1173.8–74.5Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M4 ProDecode—24,576 / 128≥ 118 tok/s11114.4–119.2Measured 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 MaxDecode—16,384 / 128≥ 120 tok/s11118.5–121.0Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 UltraDecode—16,384 / 128≥ 169 tok/s11168.1–170.5Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M2 ProDecode—16,384 / 128≥ 86.3 tok/s1185.8–86.5Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M4 ProDecode—16,384 / 128≥ 134 tok/s11128.3–139.5Measured 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 MaxDecode—8,192 / 128≥ 137 tok/s11136.1–137.9Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 UltraDecode—8,192 / 128≥ 201 tok/s11200.2–204.0Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M2 ProDecode—8,192 / 128≥ 102 tok/s11101.6–103.6Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M4 ProDecode—8,192 / 128≥ 160 tok/s11156.3–163.5Measured 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 MaxDecode—3,968 / 128≥ 155 tok/s11153.4–158.0Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 UltraDecode—3,968 / 128≥ 234 tok/s11215.3–241.2Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M2 ProDecode—3,968 / 128≥ 114 tok/s11109.6–116.6Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M4 ProDecode—3,968 / 128≥ 172 tok/s9165.0–176.9Measured 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 MaxDecode—512 / 128≥ 181 tok/s11179.7–184.9Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 UltraDecode—512 / 128≥ 260 tok/s11244.7–261.2Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M2 ProDecode—512 / 128≥ 145 tok/s11144.3–146.6Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M4 ProDecode—512 / 128≥ 215 tok/s11199.9–216.6Measured 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 MaxModel weight footprint——927 MB1single runCarried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 UltraModel weight footprint——927 MB1single runCarried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M4 ProModel weight footprint——927 MB1single runCarried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 MaxPrefill—30,000 / 128≥ 1,411 tok/s111,411–1,411Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 UltraPrefill—30,000 / 128≥ 2,518 tok/s112,481–2,524Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M2 ProPrefill—30,000 / 128≥ 770 tok/s11769.5–769.7Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M4 ProPrefill—30,000 / 128≥ 1,169 tok/s71,085–1,174Measured 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 MaxPrefill—24,576 / 128≥ 1,554 tok/s111,554–1,554Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 UltraPrefill—24,576 / 128≥ 2,796 tok/s112,738–2,801Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M2 ProPrefill—24,576 / 128≥ 855 tok/s11854.3–854.7Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M4 ProPrefill—24,576 / 128≥ 1,266 tok/s111,197–1,301Measured 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 MaxPrefill—16,384 / 128≥ 1,831 tok/s111,831–1,831Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 UltraPrefill—16,384 / 128≥ 3,309 tok/s113,302–3,314Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M2 ProPrefill—16,384 / 128≥ 1,025 tok/s111,025–1,025Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M4 ProPrefill—16,384 / 128≥ 1,506 tok/s111,413–1,566Measured 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 MaxPrefill—8,192 / 128≥ 2,217 tok/s112,214–2,218Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 UltraPrefill—8,192 / 128≥ 4,021 tok/s113,975–4,043Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M2 ProPrefill—8,192 / 128≥ 1,273 tok/s111,272–1,273Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M4 ProPrefill—8,192 / 128≥ 1,824 tok/s111,719–1,933Measured 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 MaxPrefill—3,968 / 128≥ 2,443 tok/s112,441–2,444Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 UltraPrefill—3,968 / 128≥ 4,470 tok/s114,325–4,478Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M2 ProPrefill—3,968 / 128≥ 1,448 tok/s111,447–1,448Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M4 ProPrefill—3,968 / 128≥ 2,188 tok/s91,836–2,211Measured 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 MaxPrefill—512 / 128≥ 2,440 tok/s112,428–2,443Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M1 UltraPrefill—512 / 128≥ 4,032 tok/s113,945–4,039Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M2 ProPrefill—512 / 128≥ 1,506 tok/s111,498–1,511Carried 2026.10.02-r10
Qwen2.5-1.5B VerifiedApple SiliconApple M4 ProPrefill—512 / 128≥ 2,296 tok/s112,279–2,301Measured 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 MaxDecode—30,000 / 128≥ 36.8 tok/s1136.3–36.9Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 UltraDecode—30,000 / 128≥ 60.9 tok/s1160.8–61.4Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M2 ProDecode—30,000 / 128≥ 23.5 tok/s1123.5–23.6Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 MaxDecode—24,576 / 128≥ 39.4 tok/s1139.1–39.6Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 UltraDecode—24,576 / 128≥ 65.7 tok/s1165.5–65.8Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M2 ProDecode—24,576 / 128≥ 25.3 tok/s1125.2–25.3Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M4 ProDecode—24,576 / 128≥ 38.4 tok/s1037.8–38.8Measured 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 MaxDecode—16,384 / 128≥ 44.3 tok/s1144.1–44.5Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 UltraDecode—16,384 / 128≥ 72.8 tok/s1172.5–73.0Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M2 ProDecode—16,384 / 128≥ 28.5 tok/s1128.4–28.6Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M4 ProDecode—16,384 / 128≥ 42.3 tok/s1141.6–42.9Measured 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 MaxDecode—8,192 / 128≥ 50.5 tok/s1149.9–50.6Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 UltraDecode—8,192 / 128≥ 84.0 tok/s1183.6–84.3Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M2 ProDecode—8,192 / 128≥ 32.3 tok/s1132.1–32.5Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M4 ProDecode—8,192 / 128≥ 47.4 tok/s645.9–47.8Measured 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 MaxDecode—3,968 / 128≥ 51.7 tok/s1150.6–51.8Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 UltraDecode—3,968 / 128≥ 88.2 tok/s1187.8–88.5Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M2 ProDecode—3,968 / 128≥ 33.0 tok/s1132.8–33.1Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M4 ProDecode—3,968 / 128≥ 47.9 tok/s1044.5–48.4Measured 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 MaxDecode—512 / 128≥ 60.4 tok/s1159.5–60.5Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 UltraDecode—512 / 128≥ 105 tok/s11104.0–105.1Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M2 ProDecode—512 / 128≥ 38.3 tok/s1138.0–38.4Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M4 ProDecode—512 / 128≥ 54.9 tok/s1150.2–56.3Measured 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 MaxModel weight footprint——4,114 MB1single runCarried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 UltraModel weight footprint——4,114 MB1single runCarried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M4 ProModel weight footprint——4,114 MB1single runCarried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 MaxPrefill—30,000 / 128≥ 393 tok/s11386.4–392.7Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 UltraPrefill—30,000 / 128≥ 720 tok/s11715.2–723.3Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M2 ProPrefill—30,000 / 128≥ 228 tok/s11227.6–227.7Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 MaxPrefill—24,576 / 128≥ 419 tok/s11419.1–419.3Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 UltraPrefill—24,576 / 128≥ 769 tok/s11763.3–776.2Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M2 ProPrefill—24,576 / 128≥ 244 tok/s11243.6–243.7Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M4 ProPrefill—24,576 / 128≥ 377 tok/s10358.6–377.9Measured 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 MaxPrefill—16,384 / 128≥ 465 tok/s11465.0–465.5Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 UltraPrefill—16,384 / 128≥ 865 tok/s11852.8–865.7Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M2 ProPrefill—16,384 / 128≥ 272 tok/s11271.9–272.0Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M4 ProPrefill—16,384 / 128≥ 418 tok/s11406.1–423.8Measured 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 MaxPrefill—8,192 / 128≥ 521 tok/s11517.5–521.1Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 UltraPrefill—8,192 / 128≥ 976 tok/s11965.0–976.2Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M2 ProPrefill—8,192 / 128≥ 307 tok/s11306.8–307.0Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M4 ProPrefill—8,192 / 128≥ 436 tok/s6405.3–463.0Measured 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 MaxPrefill—3,968 / 128≥ 552 tok/s11551.8–552.2Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 UltraPrefill—3,968 / 128≥ 1,038 tok/s111,035–1,039Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M2 ProPrefill—3,968 / 128≥ 327 tok/s11326.9–327.3Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M4 ProPrefill—3,968 / 128≥ 460 tok/s10434.9–480.4Measured 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 MaxPrefill—512 / 128≥ 546 tok/s11545.6–546.3Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M1 UltraPrefill—512 / 128≥ 1,009 tok/s111,007–1,010Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M2 ProPrefill—512 / 128≥ 333 tok/s11332.6–333.5Carried 2026.10.02-r10
Qwen2.5-7B VerifiedApple SiliconApple M4 ProPrefill—512 / 128≥ 517 tok/s11450.9–525.1Measured 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 MaxDecode—30,000 / 12867.5 tok/s1165.8–67.8Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 UltraDecode—30,000 / 128114 tok/s11113.8–114.6Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M2 ProDecode—30,000 / 12840.3 tok/s1140.2–40.3Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M4 ProDecode—30,000 / 128≥ 50.0 tok/s1047.9–50.8Measured 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 MaxDecode—24,576 / 12876.8 tok/s1174.8–77.0Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 UltraDecode—24,576 / 128127 tok/s11126.1–127.3Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M2 ProDecode—24,576 / 12846.4 tok/s1146.2–46.4Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M4 ProDecode—24,576 / 128≥ 58.1 tok/s1156.9–58.6Measured 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 MaxDecode—16,384 / 12896.6 tok/s1194.9–97.0Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 UltraDecode—16,384 / 128156 tok/s11155.2–157.7Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M2 ProDecode—16,384 / 12860.1 tok/s1160.0–60.2Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M4 ProDecode—16,384 / 128≥ 78.1 tok/s1075.2–78.5Measured 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 MaxDecode—8,192 / 128128 tok/s11124.6–128.1Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 UltraDecode—8,192 / 128201 tok/s11199.9–202.2Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M2 ProDecode—8,192 / 12883.9 tok/s1183.6–84.2Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M4 ProDecode—8,192 / 128≥ 111 tok/s11108.6–112.8Measured 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 MaxDecode—3,968 / 128163 tok/s11162.2–164.7Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 UltraDecode—3,968 / 128253 tok/s11238.6–253.4Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M2 ProDecode—3,968 / 128107 tok/s11106.3–106.8Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M4 ProDecode—3,968 / 128≥ 150 tok/s10144.5–152.0Measured 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 MaxDecode—512 / 128207 tok/s11206.2–209.4Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 UltraDecode—512 / 128290 tok/s11275.2–292.0Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M2 ProDecode—512 / 128144 tok/s11142.5–144.4Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M4 ProDecode—512 / 128≥ 204 tok/s11200.1–207.7Measured 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 MaxModel weight footprint——968 MB1single runCarried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 UltraModel weight footprint——968 MB1single runCarried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M4 ProModel weight footprint——968 MB1single runCarried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 MaxPrefill—30,000 / 1281,156 tok/s111,152–1,158Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 UltraPrefill—30,000 / 1282,235 tok/s112,229–2,237Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M2 ProPrefill—30,000 / 128651 tok/s11650.8–651.6Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M4 ProPrefill—30,000 / 128≥ 1,020 tok/s101,004–1,023Measured 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 MaxPrefill—24,576 / 1281,296 tok/s111,295–1,296Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 UltraPrefill—24,576 / 1282,503 tok/s112,491–2,503Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M2 ProPrefill—24,576 / 128727 tok/s11726.7–727.4Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M4 ProPrefill—24,576 / 128≥ 1,133 tok/s111,067–1,142Measured 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 MaxPrefill—16,384 / 1281,562 tok/s111,561–1,563Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 UltraPrefill—16,384 / 1283,001 tok/s112,968–3,004Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M2 ProPrefill—16,384 / 128880 tok/s11879.5–880.8Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M4 ProPrefill—16,384 / 128≥ 1,346 tok/s101,256–1,377Measured 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 MaxPrefill—8,192 / 1281,958 tok/s111,955–1,960Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 UltraPrefill—8,192 / 1283,732 tok/s113,728–3,746Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M2 ProPrefill—8,192 / 1281,112 tok/s111,112–1,112Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M4 ProPrefill—8,192 / 128≥ 1,629 tok/s111,539–1,710Measured 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 MaxPrefill—3,968 / 1282,233 tok/s112,231–2,234Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 UltraPrefill—3,968 / 1284,171 tok/s114,095–4,183Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M2 ProPrefill—3,968 / 1281,306 tok/s111,305–1,306Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M4 ProPrefill—3,968 / 128≥ 1,969 tok/s101,856–1,989Measured 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 MaxPrefill—512 / 1282,417 tok/s112,394–2,421Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M1 UltraPrefill—512 / 1284,213 tok/s113,580–4,248Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M2 ProPrefill—512 / 1281,459 tok/s111,450–1,461Carried 2026.10.02-r10
Qwen3-1.7B VerifiedApple SiliconApple M4 ProPrefill—512 / 128≥ 2,161 tok/s112,154–2,167Measured 2026.10.02-r10
Qwen3.5-0.8B VerifiedNVIDIANVIDIA RTX A5000Decodebalanced921 / 1024916 tok/s1916.1–916.1Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA A100-SXM4-80GBDecodesingle-token decoding512 / —688 tok/s1single runDevelopment development
Qwen3.5-0.8B VerifiedNVIDIANVIDIA H100 80GB HBM3Decodesingle-token decoding512 / —1,004 tok/s1single runDevelopment development
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090Decodesingle-token decoding512 / —796 tok/s1single runDevelopment development
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodesingle-token decoding512 / —1,008 tok/s1single runDevelopment development
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodebalanced21 / 1281,318 tok/s21,316–1,320Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodequality21 / 1281,156 tok/s21,155–1,156Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodefast21 / 1281,316 tok/s21,315–1,317Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodebalanced21 / 10241,439 tok/s11,439–1,439Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodebalanced21 / 1281,529 tok/s21,528–1,531Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodequality21 / 1281,438 tok/s21,437–1,438Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodefast21 / 1281,531 tok/s21,530–1,532Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodebalanced21 / 10241,676 tok/s11,676–1,676Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090Decodebalanced20 / 128968 tok/s2965.9–969.2Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090Decodequality20 / 128881 tok/s2877.6–883.9Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090Decodefast20 / 128972 tok/s2967.7–976.4Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090Decodebalanced20 / 10241,052 tok/s1single runCarried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA RTX A5000Decodebalanced20 / 128777 tok/s2776.2–777.1Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA RTX A5000Decodequality20 / 128715 tok/s2712.2–717.1Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA RTX A5000Decodefast20 / 128772 tok/s2771.5–772.2Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA RTX A5000prompt_latencybalanced921 / 102428.57 ms128.6–28.6Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090prompt_latencybalanced21 / 10244.38 ms14.4–4.4Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencybalanced21 / 1283.00 ms23.0–3.0Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencyquality21 / 1283.11 ms23.1–3.1Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencyfast21 / 1283.01 ms23.0–3.0Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencybalanced21 / 10243.01 ms13.0–3.0Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencybalanced21 / 1282.74 ms22.7–2.7Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencyquality21 / 1282.94 ms22.9–3.0Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencyfast21 / 1282.74 ms22.7–2.8Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencybalanced21 / 10242.75 ms12.7–2.7Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090prompt_latencybalanced20 / 1284.18 ms24.2–4.2Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090prompt_latencyquality20 / 1284.09 ms24.1–4.1Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090prompt_latencyfast20 / 1284.09 ms24.0–4.2Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA RTX A5000prompt_latencybalanced20 / 1284.86 ms24.9–4.9Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA RTX A5000prompt_latencyquality20 / 1284.65 ms24.6–4.7Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA RTX A5000prompt_latencyfast20 / 1284.89 ms24.9–4.9Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090resident_memorybalanced—2.27 GiB1single runCarried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090resident_memoryquality—2.42 GiB1single runCarried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090resident_memoryfast—2.27 GiB1single runCarried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 4090resident_memorybalanced—2.40 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 4090resident_memoryquality—2.55 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 4090resident_memoryfast—2.40 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 5090resident_memorybalanced—2.51 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 5090resident_memoryquality—2.66 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA GeForce RTX 5090resident_memoryfast—2.51 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA RTX A5000resident_memorybalanced—2.22 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA RTX A5000resident_memoryquality—2.37 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIANVIDIA RTX A5000resident_memoryfast—2.22 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090throughput1 request at a time36 / 128532 tok/s3528.2–532.1Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090throughput4 concurrent requests36 / 1281,778 tok/s31,776–1,780Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090throughput8 concurrent requests36 / 1283,125 tok/s33,125–3,130Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090ttft1 request at a time36 / 12824.60 ms324.5–26.6Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090ttft4 concurrent requests36 / 12862.90 ms362.6–63.9Carried cuda-qwen-1.1.4
Qwen3.5-0.8B VerifiedNVIDIAGeForce RTX 3090ttft8 concurrent requests36 / 12892.60 ms392.5–92.8Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA RTX A5000Decodebalanced921 / 1024449 tok/s1449.1–449.1Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA A100-SXM4-80GBDecodesingle-token decoding512 / —439 tok/s1single runDevelopment development
Qwen3.5-2B VerifiedNVIDIANVIDIA H100 80GB HBM3Decodesingle-token decoding512 / —660 tok/s1single runDevelopment development
Qwen3.5-2B VerifiedNVIDIAGeForce RTX 3090Decodesingle-token decoding512 / —436 tok/s1single runDevelopment development
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodesingle-token decoding512 / —548 tok/s1single runDevelopment development
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodebalanced21 / 128887 tok/s2879.2–894.7Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodequality21 / 128741 tok/s2740.9–741.0Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodefast21 / 128895 tok/s2895.0–895.2Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodebalanced21 / 1024869 tok/s1868.8–868.8Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodebalanced21 / 1281,138 tok/s21,138–1,138Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodequality21 / 1281,041 tok/s21,038–1,043Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodefast21 / 1281,137 tok/s21,135–1,138Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodebalanced21 / 10241,107 tok/s11,107–1,107Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIAGeForce RTX 3090Decodebalanced20 / 128586 tok/s2584.8–588.2Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIAGeForce RTX 3090Decodequality20 / 128543 tok/s2542.5–543.0Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIAGeForce RTX 3090Decodefast20 / 128585 tok/s2584.2–586.0Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIAGeForce RTX 3090Decodebalanced20 / 1024560 tok/s1single runCarried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA RTX A5000Decodebalanced20 / 128467 tok/s2465.8–468.9Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA RTX A5000Decodequality20 / 128435 tok/s2434.8–435.6Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA RTX A5000Decodefast20 / 128465 tok/s2463.5–465.7Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA RTX A5000prompt_latencybalanced921 / 102458.49 ms158.5–58.5Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIAGeForce RTX 3090prompt_latencybalanced21 / 10247.12 ms17.1–7.1Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencybalanced21 / 1284.49 ms24.4–4.6Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencyquality21 / 1284.69 ms24.7–4.7Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencyfast21 / 1284.37 ms24.4–4.4Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencybalanced21 / 10244.38 ms14.4–4.4Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencybalanced21 / 1283.78 ms23.8–3.8Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencyquality21 / 1284.11 ms24.1–4.1Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencyfast21 / 1283.83 ms23.8–3.8Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencybalanced21 / 10243.79 ms13.8–3.8Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIAGeForce RTX 3090prompt_latencybalanced20 / 1286.63 ms26.6–6.7Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIAGeForce RTX 3090prompt_latencyquality20 / 1286.90 ms26.8–7.0Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIAGeForce RTX 3090prompt_latencyfast20 / 1286.67 ms26.7–6.7Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA RTX A5000prompt_latencybalanced20 / 1288.39 ms28.4–8.4Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA RTX A5000prompt_latencyquality20 / 1289.02 ms29.0–9.0Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA RTX A5000prompt_latencyfast20 / 1288.41 ms28.4–8.4Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIAGeForce RTX 3090resident_memorybalanced—3.48 GiB1single runCarried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIAGeForce RTX 3090resident_memoryquality—3.80 GiB1single runCarried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIAGeForce RTX 3090resident_memoryfast—3.48 GiB1single runCarried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 4090resident_memorybalanced—3.61 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 4090resident_memoryquality—3.93 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 4090resident_memoryfast—3.61 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 5090resident_memorybalanced—3.72 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 5090resident_memoryquality—4.03 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA GeForce RTX 5090resident_memoryfast—3.72 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA RTX A5000resident_memorybalanced—3.43 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA RTX A5000resident_memoryquality—3.75 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-2B VerifiedNVIDIANVIDIA RTX A5000resident_memoryfast—3.43 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA RTX A5000Decodebalanced921 / 1024229 tok/s1228.6–228.6Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA A100-SXM4-80GBDecodesingle-token decoding512 / —224 tok/s1single runDevelopment development
Qwen3.5-4B VerifiedNVIDIANVIDIA H100 80GB HBM3Decodesingle-token decoding512 / —365 tok/s1single runDevelopment development
Qwen3.5-4B VerifiedNVIDIAGeForce RTX 3090Decodesingle-token decoding512 / —208 tok/s1single runDevelopment development
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodesingle-token decoding512 / —248 tok/s1single runDevelopment development
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodebalanced21 / 128532 tok/s2528.6–535.0Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodequality21 / 128442 tok/s2441.8–441.8Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodefast21 / 128535 tok/s2535.1–535.7Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodebalanced21 / 1024568 tok/s1568.2–568.2Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodebalanced21 / 128772 tok/s2772.1–772.3Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodequality21 / 128670 tok/s2670.4–670.5Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodefast21 / 128771 tok/s2770.6–772.3Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodebalanced21 / 1024822 tok/s1822.2–822.2Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIAGeForce RTX 3090Decodebalanced20 / 128377 tok/s2376.5–377.2Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIAGeForce RTX 3090Decodequality20 / 128342 tok/s2342.0–342.1Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIAGeForce RTX 3090Decodefast20 / 128377 tok/s2376.2–377.4Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIAGeForce RTX 3090Decodebalanced20 / 1024398 tok/s1single runCarried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA RTX A5000Decodebalanced20 / 128280 tok/s2279.3–281.7Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA RTX A5000Decodequality20 / 128256 tok/s2255.9–257.1Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA RTX A5000Decodefast20 / 128279 tok/s2278.0–279.9Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA RTX A5000prompt_latencybalanced921 / 1024154 ms1154.0–154.0Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIAGeForce RTX 3090prompt_latencybalanced21 / 102413.90 ms113.9–13.9Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencybalanced21 / 1288.73 ms28.6–8.9Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencyquality21 / 12810.54 ms210.5–10.5Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencyfast21 / 1288.68 ms28.6–8.7Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencybalanced21 / 10248.60 ms18.6–8.6Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencybalanced21 / 1286.88 ms26.9–6.9Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencyquality21 / 1287.90 ms27.9–7.9Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencyfast21 / 1286.92 ms26.9–6.9Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencybalanced21 / 10246.91 ms16.9–6.9Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIAGeForce RTX 3090prompt_latencybalanced20 / 12813.51 ms213.4–13.6Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIAGeForce RTX 3090prompt_latencyquality20 / 12814.03 ms213.7–14.3Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIAGeForce RTX 3090prompt_latencyfast20 / 12813.11 ms213.0–13.2Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA RTX A5000prompt_latencybalanced20 / 12817.22 ms217.0–17.5Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA RTX A5000prompt_latencyquality20 / 12819.18 ms218.9–19.4Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA RTX A5000prompt_latencyfast20 / 12817.21 ms216.9–17.5Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIAGeForce RTX 3090resident_memorybalanced—5.49 GiB1single runCarried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIAGeForce RTX 3090resident_memoryquality—6.27 GiB1single runCarried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIAGeForce RTX 3090resident_memoryfast—5.49 GiB1single runCarried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 4090resident_memorybalanced—5.62 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 4090resident_memoryquality—6.40 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 4090resident_memoryfast—5.62 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 5090resident_memorybalanced—5.73 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 5090resident_memoryquality—6.50 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA GeForce RTX 5090resident_memoryfast—5.73 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA RTX A5000resident_memorybalanced—5.44 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA RTX A5000resident_memoryquality—6.22 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-4B VerifiedNVIDIANVIDIA RTX A5000resident_memoryfast—5.44 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA RTX A5000Decodebalanced921 / 1024125 tok/s1124.8–124.8Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA A100-SXM4-80GBDecodesingle-token decoding512 / —129 tok/s2127.0–131.9Development development
Qwen3.5-9B VerifiedNVIDIANVIDIA H100 80GB HBM3Decodesingle-token decoding512 / —242 tok/s1single runDevelopment development
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090Decodesingle-token decoding512 / —117 tok/s1single runDevelopment development
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodesingle-token decoding512 / —155 tok/s2154.6–155.2Development development
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodesingle-token decoding512 / —242 tok/s3242.2–243.4Development development
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodebalanced21 / 128347 tok/s2346.7–347.3Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodequality21 / 128273 tok/s2272.8–272.8Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodefast21 / 128347 tok/s2346.1–347.2Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodebalanced21 / 1024303 tok/s1303.4–303.4Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodebalanced21 / 128499 tok/s2498.6–498.8Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodequality21 / 128423 tok/s2423.3–423.4Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodefast21 / 128498 tok/s2498.4–498.6Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodebalanced21 / 1024435 tok/s1434.6–434.6Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090Decodebalanced20 / 128190 tok/s2190.1–190.3Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090Decodequality20 / 128173 tok/s2172.6–172.9Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090Decodefast20 / 128190 tok/s2190.4–190.4Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090Decodebalanced20 / 1024170 tok/s1single runCarried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA RTX A5000Decodebalanced20 / 128151 tok/s2150.8–151.1Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA RTX A5000Decodequality20 / 128138 tok/s2137.2–138.4Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA RTX A5000Decodefast20 / 128149 tok/s2148.8–149.3Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA RTX A5000prompt_latencybalanced921 / 1024270 ms1269.9–269.9Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090prompt_latencybalanced21 / 102422.42 ms122.4–22.4Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencybalanced21 / 12816.82 ms216.7–16.9Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencyquality21 / 12821.62 ms221.6–21.6Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencyfast21 / 12816.47 ms216.5–16.5Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 4090prompt_latencybalanced21 / 102416.62 ms116.6–16.6Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencybalanced21 / 12811.06 ms211.0–11.1Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencyquality21 / 12813.02 ms213.0–13.0Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencyfast21 / 12811.09 ms211.1–11.1Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencybalanced21 / 102411.12 ms111.1–11.1Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090prompt_latencybalanced20 / 12821.72 ms221.7–21.7Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090prompt_latencyquality20 / 12823.25 ms223.2–23.3Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090prompt_latencyfast20 / 12821.77 ms221.7–21.8Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA RTX A5000prompt_latencybalanced20 / 12828.99 ms229.0–29.0Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA RTX A5000prompt_latencyquality20 / 12830.95 ms230.9–31.0Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA RTX A5000prompt_latencyfast20 / 12829.21 ms229.2–29.2Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090resident_memorybalanced—8.44 GiB1single runCarried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090resident_memoryquality—9.82 GiB1single runCarried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090resident_memoryfast—8.44 GiB1single runCarried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 4090resident_memorybalanced—8.57 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 4090resident_memoryquality—9.95 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 4090resident_memoryfast—8.57 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 5090resident_memorybalanced—8.67 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 5090resident_memoryquality—10.05 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA GeForce RTX 5090resident_memoryfast—8.67 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA RTX A5000resident_memorybalanced—8.39 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA RTX A5000resident_memoryquality—9.77 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIANVIDIA RTX A5000resident_memoryfast—8.39 GiB2—Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090throughput1 request at a time34 / 128101 tok/s3101.4–101.4Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090throughput4 concurrent requests34 / 128366 tok/s3366.4–366.9Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090throughput8 concurrent requests34 / 128663 tok/s3662.7–663.3Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090ttft1 request at a time34 / 12855.60 ms355.5–55.8Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090ttft4 concurrent requests34 / 128176 ms3133.1–176.4Carried cuda-qwen-1.1.4
Qwen3.5-9B VerifiedNVIDIAGeForce RTX 3090ttft8 concurrent requests34 / 128309 ms3308.4–308.9Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA H100 80GB HBM3Decodesingle-token decoding131,072 / 64172 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA H100 80GB HBM3Decodesingle-token decoding64,512 / 64206 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA H100 80GB HBM3Decodesingle-token decoding49,152 / 64219 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA A100-SXM4-80GBDecodesingle-token decoding16,384 / 64170 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA H100 80GB HBM3Decodesingle-token decoding16,384 / 64251 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodesingle-token decoding16,384 / 64227 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA A100-SXM4-80GBDecodesingle-token decoding7,680 / 64179 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA H100 80GB HBM3Decodesingle-token decoding7,680 / 64259 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodesingle-token decoding7,680 / 64251 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090Decodesingle-token decoding4,000 / 129212 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA A100-SXM4-80GBDecodesingle-token decoding512 / 64192 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA H100 80GB HBM3Decodesingle-token decoding512 / 64269 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodesingle-token decoding512 / 64273 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA A100-SXM4-80GBDecodesingle-token decoding114 / 256184 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA A100-SXM4-80GBDecodemulti-token steps (lossless)114 / 256228 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA H100 80GB HBM3Decodesingle-token decoding114 / 256284 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA H100 80GB HBM3Decodemulti-token steps (lossless)114 / 256357 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090Decodesingle-token decoding114 / 256213 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090Decodemulti-token steps (lossless)114 / 256252 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodesingle-token decoding114 / 256277 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 4090Decodemulti-token steps (lossless)114 / 256374 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090Decodebalanced22 / 128194 tok/s3194.0–195.4Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090Decodefast22 / 128173 tok/s2172.3–173.1Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090Decodebalanced22 / 1024210 tok/s1single runCarried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090Decodesingle-token decoding22 / 128198 tok/s3197.6–200.7Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodebalanced21 / 128450 tok/s2449.9–450.0Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodefast21 / 128545 tok/s2544.6–544.7Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodebalanced21 / 1024450 tok/s1449.6–449.6Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090Decodesingle-token decoding8 / 512224 tok/s3224.2–224.8Development development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 5090Decodesingle-token decoding—391 tok/s3384.1–392.8Development development
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090Prefillsingle-token decoding8,000 / —9,065 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090prompt_latencysingle-token decoding8,000 / 1950 ms1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090prompt_latencybalanced22 / 128107 ms3106.9–109.0Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090prompt_latencyfast22 / 128108 ms2107.4–107.8Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencybalanced21 / 12861.25 ms261.2–61.3Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencyfast21 / 12861.27 ms261.2–61.3Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 5090prompt_latencybalanced21 / 102461.48 ms161.5–61.5Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090resident_memorybalanced—21.44 GiB1single runCarried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090resident_memoryfast—21.51 GiB1single runCarried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090resident_memorysingle-token decoding—21.85 GiB1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 5090resident_memorybalanced—21.67 GiB2—Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 5090resident_memoryfast—21.74 GiB2—Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090throughput4 sequences at once64 / —373 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090throughput8 sequences at once64 / —408 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090throughput24 sequences at once64 / —425 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090throughput12 sequences at once64 / —412 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090throughput16 sequences at once64 / —417 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090throughput1 request at a time34 / 128161 tok/s3159.1–160.9Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090throughput4 concurrent requests34 / 128222 tok/s3221.3–224.4Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090throughput8 concurrent requests34 / 128231 tok/s3229.4–230.8Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 4090throughput1 request at a time— / 128211 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 4090throughput4 concurrent requests— / 128378 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 4090throughput8 concurrent requests— / 128418 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 4090throughput1 sequence—256 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 4090throughput2 sequences at once—391 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 4090throughput4 sequences at once—558 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIANVIDIA GeForce RTX 4090throughput8 sequences at once—625 tok/s1single runDevelopment development
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090ttft1 request at a time34 / 128183 ms3182.1–185.8Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090ttft4 concurrent requests34 / 128626 ms3622.5–629.8Carried cuda-qwen-1.1.4
Qwen3.6-35B-A3B VerifiedNVIDIAGeForce RTX 3090ttft8 concurrent requests34 / 1281,001 ms3998.5–1,003.3Carried cuda-qwen-1.1.4
SD3.5-large VerifiedNVIDIANVIDIA A100 80GB PCIeTime to a finished imagebalanced—54.79 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA A100 80GB PCIeTime to a finished imagefast—48.60 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA A100 80GB PCIeTime to a finished imageultrafast—45.87 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA A40Time to a finished imagequality—88.01 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA A40Time to a finished imagebalanced—55.51 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA A40Time to a finished imagelow-memory—60.67 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA A40Time to a finished imagefast—45.16 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA A40Time to a finished imageultrafast—45.00 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Time to a finished imagequality—56.36 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Time to a finished imagebalanced—38.90 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Time to a finished imagelow-memory—44.04 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Time to a finished imagefast—36.04 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Time to a finished imageultrafast—35.27 s1single runDevelopment development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Time to a finished imagequality—76.69 s1single runDevelopment development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Time to a finished imagebalanced—49.03 s1single runDevelopment development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Time to a finished imagelow-memory—59.95 s1single runDevelopment development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Time to a finished imagefast—38.41 s1single runDevelopment development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Time to a finished imageultrafast—37.98 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Time to a finished imagequality—49.01 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Time to a finished imagebalanced—31.05 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Time to a finished imagelow-memory—41.52 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Time to a finished imagefast—25.16 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Time to a finished imageultrafast—24.84 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 5090Time to a finished imagequality—44.60 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 5090Time to a finished imagebalanced—30.19 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 5090Time to a finished imagelow-memory—33.90 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 5090Time to a finished imagefast—27.29 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 5090Time to a finished imageultrafast—27.28 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Time to a finished imagequality—107 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Time to a finished imagebalanced—68.32 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Time to a finished imagelow-memory—71.61 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Time to a finished imagefast—50.65 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Time to a finished imageultrafast—51.99 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime to a finished imagequality—60.51 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime to a finished imagebalanced—36.69 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime to a finished imagelow-memory—44.79 s1single runDevelopment development
SD3.5-large VerifiedNVIDIANVIDIA A100 80GB PCIeTime per diffusion stepfast—0.49 s/step20.5–0.5Development development
SD3.5-large VerifiedNVIDIANVIDIA A100 80GB PCIeTime per diffusion stepultrafast—0.50 s/step20.5–0.5Development development
SD3.5-large VerifiedNVIDIANVIDIA A40Time per diffusion stepquality—1.68 s/step3—Development development
SD3.5-large VerifiedNVIDIANVIDIA A40Time per diffusion stepbalanced—0.86 s/step3—Development development
SD3.5-large VerifiedNVIDIANVIDIA A40Time per diffusion steplow-memory—0.86 s/step3—Development development
SD3.5-large VerifiedNVIDIANVIDIA A40Time per diffusion stepfast—0.59 s/step3—Development development
SD3.5-large VerifiedNVIDIANVIDIA A40Time per diffusion stepultrafast—0.59 s/step3—Development development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Time per diffusion stepquality—0.93 s/step3—Development development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Time per diffusion stepbalanced—0.51 s/step3—Development development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Time per diffusion steplow-memory—0.50 s/step3—Development development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Time per diffusion stepfast—0.42 s/step3—Development development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Time per diffusion stepultrafast—0.42 s/step3—Development development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Time per diffusion stepquality—1.51 s/step31.5–1.5Development development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Time per diffusion stepbalanced—0.83 s/step30.8–0.8Development development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Time per diffusion steplow-memory—0.83 s/step30.8–0.8Development development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Time per diffusion stepfast—0.56 s/step30.6–0.6Development development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Time per diffusion stepultrafast—0.56 s/step30.6–0.6Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Time per diffusion stepquality—0.87 s/step30.9–0.9Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Time per diffusion stepbalanced—0.42 s/step30.4–0.4Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Time per diffusion steplow-memory—0.42 s/step30.4–0.4Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Time per diffusion stepfast—0.27 s/step30.3–0.3Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Time per diffusion stepultrafast—0.27 s/step30.3–0.3Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 5090Time per diffusion stepbalanced—0.31 s/step30.3–0.3Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 5090Time per diffusion steplow-memory—0.31 s/step30.3–0.3Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 5090Time per diffusion stepfast—0.23 s/step30.2–0.2Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 5090Time per diffusion stepultrafast—0.23 s/step30.2–0.2Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Time per diffusion stepquality—2.12 s/step32.1–2.1Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Time per diffusion stepbalanced—1.15 s/step31.1–1.1Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Time per diffusion steplow-memory—1.14 s/step31.1–1.1Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Time per diffusion stepfast—0.74 s/step30.7–0.7Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Time per diffusion stepultrafast—0.74 s/step30.7–0.7Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime per diffusion stepquality—1.19 s/step3—Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime per diffusion stepbalanced—0.61 s/step3—Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime per diffusion steplow-memory—0.60 s/step3—Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime per diffusion stepfast—0.47 s/step2—Development development
SD3.5-large VerifiedNVIDIANVIDIA A40Peak device memoryquality—13.78 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA A40Peak device memorybalanced—13.78 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA A40Peak device memorylow-memory—10.66 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA A40Peak device memoryfast—13.78 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA A40Peak device memoryultrafast—13.78 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Peak device memoryquality—13.98 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Peak device memorybalanced—13.98 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Peak device memorylow-memory—10.87 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Peak device memoryfast—13.98 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA H100 80GB HBM3Peak device memoryultrafast—13.98 GB3—Development development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Peak device memoryquality—13.78 GB3—Development development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Peak device memorybalanced—13.78 GB3—Development development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Peak device memorylow-memory—10.66 GB3—Development development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Peak device memoryfast—13.78 GB3—Development development
SD3.5-large VerifiedNVIDIAGeForce RTX 3090Peak device memoryultrafast—13.78 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Peak device memoryquality—13.92 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Peak device memorybalanced—13.92 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Peak device memorylow-memory—10.80 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Peak device memoryfast—13.92 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 4090Peak device memoryultrafast—13.92 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 5090Peak device memorybalanced—14.03 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 5090Peak device memorylow-memory—10.92 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 5090Peak device memoryfast—14.03 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 5090Peak device memoryultrafast—14.03 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA GeForce RTX 5090Peak device memoryquality—14.03 GB314.0–14.0Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Peak device memoryquality—13.72 GB313.7–13.7Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Peak device memorybalanced—13.72 GB313.7–13.7Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Peak device memorylow-memory—10.61 GB310.6–10.6Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Peak device memoryfast—13.72 GB313.7–13.7Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX A5000Peak device memoryultrafast—13.72 GB313.7–13.7Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellPeak device memoryquality—13.78 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellPeak device memorybalanced—13.78 GB3—Development development
SD3.5-large VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellPeak device memorylow-memory—10.66 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA A100-SXM4-80GBTime to a finished imagequality—62.76 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA A100-SXM4-80GBTime to a finished imagebalanced—31.32 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA A100-SXM4-80GBTime to a finished imagefast—30.73 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA A100-SXM4-80GBTime to a finished imageultrafast—29.47 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA A40Time to a finished imagequality—35.00 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA A40Time to a finished imagebalanced—24.77 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA A40Time to a finished imagelow-memory—29.29 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA A40Time to a finished imagefast—22.21 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA A40Time to a finished imageultrafast—21.77 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Time to a finished imagequality—24.56 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Time to a finished imagebalanced—19.05 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Time to a finished imagelow-memory—23.82 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Time to a finished imagefast—17.54 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Time to a finished imageultrafast—17.70 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIAGeForce RTX 3090Time to a finished imagequality—32.33 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIAGeForce RTX 3090Time to a finished imagebalanced—22.81 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIAGeForce RTX 3090Time to a finished imagelow-memory—33.07 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIAGeForce RTX 3090Time to a finished imagefast—23.95 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIAGeForce RTX 3090Time to a finished imageultrafast—19.45 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 4090Time to a finished imagequality—20.77 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 4090Time to a finished imagebalanced—14.40 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 4090Time to a finished imagelow-memory—26.40 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 4090Time to a finished imagefast—17.62 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 4090Time to a finished imageultrafast—12.86 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 5090Time to a finished imagequality—20.16 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 5090Time to a finished imagebalanced—14.56 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 5090Time to a finished imagelow-memory—17.97 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 5090Time to a finished imagefast—20.19 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 5090Time to a finished imageultrafast—13.29 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA RTX A5000Time to a finished imagebalanced—29.29 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA RTX A5000Time to a finished imagefast—27.96 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA RTX A5000Time to a finished imageultrafast—24.72 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime to a finished imagequality—23.68 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime to a finished imagebalanced—16.73 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime to a finished imagelow-memory—26.86 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime to a finished imagefast—15.23 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime to a finished imageultrafast—14.98 s1single runDevelopment development
SD3.5-medium VerifiedNVIDIANVIDIA A40Time per diffusion stepquality—0.61 s/step3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA A40Time per diffusion stepbalanced—0.36 s/step5—Development development
SD3.5-medium VerifiedNVIDIANVIDIA A40Time per diffusion steplow-memory—0.36 s/step5—Development development
SD3.5-medium VerifiedNVIDIANVIDIA A40Time per diffusion stepfast—0.29 s/step5—Development development
SD3.5-medium VerifiedNVIDIANVIDIA A40Time per diffusion stepultrafast—0.29 s/step5—Development development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Time per diffusion stepquality—0.36 s/step5—Development development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Time per diffusion stepbalanced—0.22 s/step5—Development development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Time per diffusion steplow-memory—0.22 s/step5—Development development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Time per diffusion stepfast—0.19 s/step5—Development development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Time per diffusion stepultrafast—0.19 s/step5—Development development
SD3.5-medium VerifiedNVIDIAGeForce RTX 3090Time per diffusion stepbalanced—0.35 s/step30.3–0.3Development development
SD3.5-medium VerifiedNVIDIAGeForce RTX 3090Time per diffusion steplow-memory—0.35 s/step30.3–0.3Development development
SD3.5-medium VerifiedNVIDIAGeForce RTX 3090Time per diffusion stepfast—0.28 s/step30.3–0.3Development development
SD3.5-medium VerifiedNVIDIAGeForce RTX 3090Time per diffusion stepultrafast—0.28 s/step30.3–0.3Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 4090Time per diffusion stepbalanced—0.17 s/step30.2–0.2Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 4090Time per diffusion steplow-memory—0.17 s/step30.2–0.2Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 4090Time per diffusion stepfast—0.13 s/step30.1–0.1Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 4090Time per diffusion stepultrafast—0.13 s/step30.1–0.1Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 5090Time per diffusion stepbalanced—0.13 s/step30.1–0.1Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 5090Time per diffusion steplow-memory—0.13 s/step30.1–0.1Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 5090Time per diffusion stepfast—0.11 s/step30.1–0.1Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 5090Time per diffusion stepultrafast—0.11 s/step30.1–0.1Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX A5000Time per diffusion stepfast—0.35 s/step50.4–0.4Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX A5000Time per diffusion stepultrafast—0.35 s/step50.4–0.4Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime per diffusion stepquality—0.43 s/step5—Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime per diffusion stepbalanced—0.25 s/step5—Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime per diffusion steplow-memory—0.25 s/step5—Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime per diffusion stepfast—0.21 s/step5—Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellTime per diffusion stepultrafast—0.21 s/step5—Development development
SD3.5-medium VerifiedNVIDIANVIDIA A40Peak device memoryquality—16.61 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA A40Peak device memorybalanced—16.42 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA A40Peak device memorylow-memory—7.47 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA A40Peak device memoryfast—15.35 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA A40Peak device memoryultrafast—15.35 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Peak device memoryquality—17.02 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Peak device memorybalanced—17.30 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Peak device memorylow-memory—7.48 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Peak device memoryfast—15.76 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA H100 80GB HBM3Peak device memoryultrafast—15.76 GB3—Development development
SD3.5-medium VerifiedNVIDIAGeForce RTX 3090Peak device memorybalanced—16.41 GB3—Development development
SD3.5-medium VerifiedNVIDIAGeForce RTX 3090Peak device memorylow-memory—7.08 GB3—Development development
SD3.5-medium VerifiedNVIDIAGeForce RTX 3090Peak device memoryfast—15.35 GB3—Development development
SD3.5-medium VerifiedNVIDIAGeForce RTX 3090Peak device memoryultrafast—15.35 GB3—Development development
SD3.5-medium VerifiedNVIDIAGeForce RTX 3090Peak device memoryquality—16.61 GB316.6–16.6Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 4090Peak device memorybalanced—16.68 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 4090Peak device memorylow-memory—7.74 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 4090Peak device memoryfast—15.62 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 4090Peak device memoryultrafast—15.62 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 4090Peak device memoryquality—16.88 GB316.9–16.9Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 5090Peak device memorybalanced—16.92 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 5090Peak device memorylow-memory—7.59 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 5090Peak device memoryfast—15.86 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 5090Peak device memoryultrafast—15.86 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA GeForce RTX 5090Peak device memoryquality—17.11 GB317.1–17.1Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX A5000Peak device memoryquality—16.50 GB316.5–16.5Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX A5000Peak device memorybalanced—16.30 GB516.3–16.3Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX A5000Peak device memorylow-memory—7.36 GB37.0–7.4Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX A5000Peak device memoryfast—15.24 GB515.1–15.2Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX A5000Peak device memoryultrafast—15.24 GB515.2–15.2Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellPeak device memoryquality—16.60 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellPeak device memorybalanced—16.41 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellPeak device memorylow-memory—7.07 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellPeak device memoryfast—15.34 GB3—Development development
SD3.5-medium VerifiedNVIDIANVIDIA RTX PRO 4500 BlackwellPeak device memoryultrafast—15.34 GB3—Development development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA A100-SXM4-80GBTime to a finished clip (render only)balanced—192 s2191.9–192.4Development development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime to a finished clip (render only)quality—407 s1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime to a finished clip (render only)balanced—338 s1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime to a finished clip (render only)fast—259 s1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime to a finished clip (render only)ultrafast—179 s1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime to a finished clip (render only)balanced—1,474 s1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime to a finished clip (render only)fast—1,026 s1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime to a finished clip (render only)ultrafast—601 s1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 4090Time to a finished clip (render only)balanced—163 s2162.5–163.4Development development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 5090Time to a finished clip (render only)balanced—118 s2118.0–118.2Development development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime per diffusion stepquality—8.03 s/step1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime per diffusion stepbalanced—6.65 s/step1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime per diffusion stepfast—5.05 s/step1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime per diffusion stepultrafast—3.46 s/step1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime per diffusion stepquality—35.86 s/step1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime per diffusion stepbalanced—29.09 s/step1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime per diffusion stepfast—20.12 s/step1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiTime per diffusion stepultrafast—11.64 s/step1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiPeak device memoryquality—6.88 GB1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiPeak device memorybalanced—6.88 GB1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiPeak device memoryfast—6.88 GB1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiPeak device memoryultrafast—6.88 GB1single runDevelopment development
Wan2.1-T2V-1.3B VerifiedNVIDIANVIDIA GeForce RTX 3080 TiPeak device memoryquality—7.16 GB1single runDevelopment development
Wan2.2-T2V-A14B VerifiedNVIDIANVIDIA GeForce RTX 4090Time to a finished clip (render only)ultrafast—1,304 s1single runDevelopment development
Wan2.2-T2V-A14B VerifiedNVIDIANVIDIA GeForce RTX 5090Time to a finished clip (render only)ultrafast—1,050 s1single runDevelopment development
Wan2.2-T2V-A14B VerifiedNVIDIANVIDIA GeForce RTX 4090Time per diffusion stepultrafast—32.38 s/step1single runDevelopment development
Wan2.2-T2V-A14B VerifiedNVIDIANVIDIA GeForce RTX 5090Time per diffusion stepultrafast—26.09 s/step1single runDevelopment development
Wan2.2-T2V-A14B VerifiedNVIDIANVIDIA GeForce RTX 4090Peak device memoryultrafast—22.19 GB1single runDevelopment development
Wan2.2-T2V-A14B VerifiedNVIDIANVIDIA GeForce RTX 5090Peak device memoryultrafast—22.19 GB1single runDevelopment development
Apple Silicon · Metal

4 models · 4 Macs · 6 prompt lengths

Click any figure for its runs, range, build and permalink.

Build 2026.10.02-r10 · 2026-10-10 · all builds

Qwen2.5-0.5B, Qwen2.5-1.5B, Qwen2.5-7B and Qwen3-1.7B are marked a lower bound. On those models the engine's prefill clock stops before the last prompt step has finished, and that step is charged to decode; these figures put a whole decode step back in, which costs more than the step does, so the engine is at least this fast on both axes and may be quicker.

Apple M2 Pro: Measured on macOS 15.5. Some Metal acceleration paths available on newer macOS are inactive under this condition, so these figures are veizik's measured lower bound here (may be higher on newer OS).

Compared against MLX, mlx 0.32.2, mlx-lm 0.31.3, 4-bit g64, converted from the snapshot; MLX bf16, mlx 0.32.2, mlx-lm 0.31.3, bf16; on Apple M1 Ultra, Apple M2 Pro. Setup and recipe for both engines →

Compared against llama.cpp (llama-bench, Metal), commit 64e9bceb2c3a856efed96feda784a50947049feb on the M1 Max and M1 Ultra; build 11146, commit 7fe450e19, on the M4 Pro, Q4_K_M GGUF; MLX, mlx 0.32.2, mlx-lm 0.31.3 (M1 Max and M1 Ultra), 4-bit g64, converted from the snapshot; on Apple M1 Ultra, Apple M1 Max, Apple M4 Pro. Setup and recipe for both engines →

Every published figure as one table (190 rows)
ModelDeviceMetricIn / outMedianRunsRangeBuild
Qwen2.5-0.5BApple M1 Maxdecode512 / 128≥lower bound: veizik is at least this fast273.1 tok/s11263.5–282.12026.10.02-r10
Qwen2.5-0.5BApple M1 Maxdecode3,968 / 128≥lower bound: veizik is at least this fast208.9 tok/s11207.5–211.62026.10.02-r10
Qwen2.5-0.5BApple M1 Maxdecode8,192 / 128≥lower bound: veizik is at least this fast209.9 tok/s11203.3–212.22026.10.02-r10
Qwen2.5-0.5BApple M1 Maxdecode16,384 / 128≥lower bound: veizik is at least this fast170.8 tok/s11167.6–172.82026.10.02-r10
Qwen2.5-0.5BApple M1 Maxdecode24,576 / 128≥lower bound: veizik is at least this fast144.1 tok/s11142.3–146.12026.10.02-r10
Qwen2.5-0.5BApple M1 Maxdecode30,000 / 128≥lower bound: veizik is at least this fast154.4 tok/s11152.4–161.32026.10.02-r10
Qwen2.5-0.5BApple M1 Maxprefill512 / 128≥lower bound: veizik is at least this fast7,901 tok/s116,417–8,1032026.10.02-r10
Qwen2.5-0.5BApple M1 Maxprefill3,968 / 128≥lower bound: veizik is at least this fast7,504 tok/s117,428–7,5552026.10.02-r10
Qwen2.5-0.5BApple M1 Maxprefill8,192 / 128≥lower bound: veizik is at least this fast6,381 tok/s116,362–6,4112026.10.02-r10
Qwen2.5-0.5BApple M1 Maxprefill16,384 / 128≥lower bound: veizik is at least this fast4,829 tok/s114,822–4,8382026.10.02-r10
Qwen2.5-0.5BApple M1 Maxprefill24,576 / 128≥lower bound: veizik is at least this fast3,859 tok/s113,853–3,8622026.10.02-r10
Qwen2.5-0.5BApple M1 Maxprefill30,000 / 128≥lower bound: veizik is at least this fast3,434 tok/s113,428–3,5462026.10.02-r10
Qwen2.5-0.5BApple M1 Ultradecode512 / 128≥lower bound: veizik is at least this fast377.4 tok/s11355.8–385.62026.10.02-r10
Qwen2.5-0.5BApple M1 Ultradecode3,968 / 128≥lower bound: veizik is at least this fast305.5 tok/s11304.4–313.32026.10.02-r10
Qwen2.5-0.5BApple M1 Ultradecode8,192 / 128≥lower bound: veizik is at least this fast280.6 tok/s11278.7–286.62026.10.02-r10
Qwen2.5-0.5BApple M1 Ultradecode16,384 / 128≥lower bound: veizik is at least this fast259.1 tok/s11257.4–262.02026.10.02-r10
Qwen2.5-0.5BApple M1 Ultradecode24,576 / 128≥lower bound: veizik is at least this fast205.6 tok/s11204.4–207.82026.10.02-r10
Qwen2.5-0.5BApple M1 Ultradecode30,000 / 128≥lower bound: veizik is at least this fast199.0 tok/s11196.9–200.32026.10.02-r10
Qwen2.5-0.5BApple M1 Ultraprefill512 / 128≥lower bound: veizik is at least this fast9,632 tok/s119,088–10,7412026.10.02-r10
Qwen2.5-0.5BApple M1 Ultraprefill3,968 / 128≥lower bound: veizik is at least this fast13,075 tok/s1112,868–13,2742026.10.02-r10
Qwen2.5-0.5BApple M1 Ultraprefill8,192 / 128≥lower bound: veizik is at least this fast11,328 tok/s1111,288–11,3992026.10.02-r10
Qwen2.5-0.5BApple M1 Ultraprefill16,384 / 128≥lower bound: veizik is at least this fast8,509 tok/s118,488–8,5262026.10.02-r10
Qwen2.5-0.5BApple M1 Ultraprefill24,576 / 128≥lower bound: veizik is at least this fast6,773 tok/s116,765–6,7772026.10.02-r10
Qwen2.5-0.5BApple M1 Ultraprefill30,000 / 128≥lower bound: veizik is at least this fast5,895 tok/s115,892–5,8992026.10.02-r10
Qwen2.5-0.5BApple M2 Prodecode512 / 128≥lower bound: veizik is at least this fast300.6 tok/s11299.0–302.52026.10.02-r10
Qwen2.5-0.5BApple M2 Prodecode3,968 / 128≥lower bound: veizik is at least this fast222.0 tok/s11221.5–223.42026.10.02-r10
Qwen2.5-0.5BApple M2 Prodecode8,192 / 128≥lower bound: veizik is at least this fast192.8 tok/s11189.2–195.62026.10.02-r10
Qwen2.5-0.5BApple M2 Prodecode16,384 / 128≥lower bound: veizik is at least this fast141.7 tok/s11139.8–143.22026.10.02-r10
Qwen2.5-0.5BApple M2 Prodecode24,576 / 128≥lower bound: veizik is at least this fast120.9 tok/s11120.3–121.32026.10.02-r10
Qwen2.5-0.5BApple M2 Prodecode30,000 / 128≥lower bound: veizik is at least this fast110.3 tok/s11109.8–110.92026.10.02-r10
Qwen2.5-0.5BApple M2 Proprefill512 / 128≥lower bound: veizik is at least this fast5,060 tok/s114,985–5,1012026.10.02-r10
Qwen2.5-0.5BApple M2 Proprefill3,968 / 128≥lower bound: veizik is at least this fast4,664 tok/s114,657–4,6732026.10.02-r10
Qwen2.5-0.5BApple M2 Proprefill8,192 / 128≥lower bound: veizik is at least this fast3,917 tok/s113,914–3,9192026.10.02-r10
Qwen2.5-0.5BApple M2 Proprefill16,384 / 128≥lower bound: veizik is at least this fast2,948 tok/s112,946–2,9482026.10.02-r10
Qwen2.5-0.5BApple M2 Proprefill24,576 / 128≥lower bound: veizik is at least this fast2,356 tok/s112,355–2,3572026.10.02-r10
Qwen2.5-0.5BApple M2 Proprefill30,000 / 128≥lower bound: veizik is at least this fast2,071 tok/s112,070–2,0722026.10.02-r10
Qwen2.5-0.5BApple M4 Prodecode512 / 128≥lower bound: veizik is at least this fast462.8 tok/s11425.7–475.62026.10.02-r10
Qwen2.5-0.5BApple M4 Prodecode3,968 / 128≥lower bound: veizik is at least this fast358.4 tok/s11344.2–360.42026.10.02-r10
Qwen2.5-0.5BApple M4 Prodecode8,192 / 128≥lower bound: veizik is at least this fast353.2 tok/s10349.0–362.42026.10.02-r10
Qwen2.5-0.5BApple M4 Prodecode16,384 / 128≥lower bound: veizik is at least this fast279.0 tok/s11263.8–284.92026.10.02-r10
Qwen2.5-0.5BApple M4 Prodecode24,576 / 128≥lower bound: veizik is at least this fast228.6 tok/s9209.6–232.42026.10.02-r10
Qwen2.5-0.5BApple M4 Prodecode30,000 / 128≥lower bound: veizik is at least this fast199.8 tok/s10187.4–204.42026.10.02-r10
Qwen2.5-0.5BApple M4 Proprefill512 / 128≥lower bound: veizik is at least this fast7,979 tok/s117,661–7,9932026.10.02-r10
Qwen2.5-0.5BApple M4 Proprefill3,968 / 128≥lower bound: veizik is at least this fast6,929 tok/s116,856–6,9522026.10.02-r10
Qwen2.5-0.5BApple M4 Proprefill8,192 / 128≥lower bound: veizik is at least this fast5,804 tok/s105,519–5,8222026.10.02-r10
Qwen2.5-0.5BApple M4 Proprefill16,384 / 128≥lower bound: veizik is at least this fast4,066 tok/s113,848–4,3632026.10.02-r10
Qwen2.5-0.5BApple M4 Proprefill24,576 / 128≥lower bound: veizik is at least this fast3,307 tok/s93,039–3,5142026.10.02-r10
Qwen2.5-0.5BApple M4 Proprefill30,000 / 128≥lower bound: veizik is at least this fast2,972 tok/s102,825–3,0942026.10.02-r10
Qwen2.5-1.5BApple M1 Maxdecode512 / 128≥lower bound: veizik is at least this fast181.1 tok/s11179.7–184.92026.10.02-r10
Qwen2.5-1.5BApple M1 Maxdecode3,968 / 128≥lower bound: veizik is at least this fast155.3 tok/s11153.4–158.02026.10.02-r10
Qwen2.5-1.5BApple M1 Maxdecode8,192 / 128≥lower bound: veizik is at least this fast136.9 tok/s11136.1–137.92026.10.02-r10
Qwen2.5-1.5BApple M1 Maxdecode16,384 / 128≥lower bound: veizik is at least this fast119.6 tok/s11118.5–121.02026.10.02-r10
Qwen2.5-1.5BApple M1 Maxdecode24,576 / 128≥lower bound: veizik is at least this fast99.3 tok/s1198.9–100.92026.10.02-r10
Qwen2.5-1.5BApple M1 Maxdecode30,000 / 128≥lower bound: veizik is at least this fast91.5 tok/s1190.7–92.22026.10.02-r10
Qwen2.5-1.5BApple M1 Maxprefill512 / 128≥lower bound: veizik is at least this fast2,440 tok/s112,428–2,4432026.10.02-r10
Qwen2.5-1.5BApple M1 Maxprefill3,968 / 128≥lower bound: veizik is at least this fast2,443 tok/s112,441–2,4442026.10.02-r10
Qwen2.5-1.5BApple M1 Maxprefill8,192 / 128≥lower bound: veizik is at least this fast2,217 tok/s112,214–2,2182026.10.02-r10
Qwen2.5-1.5BApple M1 Maxprefill16,384 / 128≥lower bound: veizik is at least this fast1,831 tok/s111,831–1,8312026.10.02-r10
Qwen2.5-1.5BApple M1 Maxprefill24,576 / 128≥lower bound: veizik is at least this fast1,554 tok/s111,554–1,5542026.10.02-r10
Qwen2.5-1.5BApple M1 Maxprefill30,000 / 128≥lower bound: veizik is at least this fast1,411 tok/s111,411–1,4112026.10.02-r10
Qwen2.5-1.5BApple M1 Ultradecode512 / 128≥lower bound: veizik is at least this fast260.2 tok/s11244.7–261.22026.10.02-r10
Qwen2.5-1.5BApple M1 Ultradecode3,968 / 128≥lower bound: veizik is at least this fast233.9 tok/s11215.3–241.22026.10.02-r10
Qwen2.5-1.5BApple M1 Ultradecode8,192 / 128≥lower bound: veizik is at least this fast201.0 tok/s11200.2–204.02026.10.02-r10
Qwen2.5-1.5BApple M1 Ultradecode16,384 / 128≥lower bound: veizik is at least this fast169.1 tok/s11168.1–170.52026.10.02-r10
Qwen2.5-1.5BApple M1 Ultradecode24,576 / 128≥lower bound: veizik is at least this fast145.8 tok/s11144.8–147.62026.10.02-r10
Qwen2.5-1.5BApple M1 Ultradecode30,000 / 128≥lower bound: veizik is at least this fast142.6 tok/s11141.7–143.72026.10.02-r10
Qwen2.5-1.5BApple M1 Ultraprefill512 / 128≥lower bound: veizik is at least this fast4,032 tok/s113,945–4,0392026.10.02-r10
Qwen2.5-1.5BApple M1 Ultraprefill3,968 / 128≥lower bound: veizik is at least this fast4,470 tok/s114,325–4,4782026.10.02-r10
Qwen2.5-1.5BApple M1 Ultraprefill8,192 / 128≥lower bound: veizik is at least this fast4,021 tok/s113,975–4,0432026.10.02-r10
Qwen2.5-1.5BApple M1 Ultraprefill16,384 / 128≥lower bound: veizik is at least this fast3,309 tok/s113,302–3,3142026.10.02-r10
Qwen2.5-1.5BApple M1 Ultraprefill24,576 / 128≥lower bound: veizik is at least this fast2,796 tok/s112,738–2,8012026.10.02-r10
Qwen2.5-1.5BApple M1 Ultraprefill30,000 / 128≥lower bound: veizik is at least this fast2,518 tok/s112,481–2,5242026.10.02-r10
Qwen2.5-1.5BApple M2 Prodecode512 / 128≥lower bound: veizik is at least this fast144.6 tok/s11144.3–146.62026.10.02-r10
Qwen2.5-1.5BApple M2 Prodecode3,968 / 128≥lower bound: veizik is at least this fast113.7 tok/s11109.6–116.62026.10.02-r10
Qwen2.5-1.5BApple M2 Prodecode8,192 / 128≥lower bound: veizik is at least this fast102.4 tok/s11101.6–103.62026.10.02-r10
Qwen2.5-1.5BApple M2 Prodecode16,384 / 128≥lower bound: veizik is at least this fast86.3 tok/s1185.8–86.52026.10.02-r10
Qwen2.5-1.5BApple M2 Prodecode24,576 / 128≥lower bound: veizik is at least this fast74.1 tok/s1173.8–74.52026.10.02-r10
Qwen2.5-1.5BApple M2 Prodecode30,000 / 128≥lower bound: veizik is at least this fast67.8 tok/s1167.5–68.22026.10.02-r10
Qwen2.5-1.5BApple M2 Proprefill512 / 128≥lower bound: veizik is at least this fast1,506 tok/s111,498–1,5112026.10.02-r10
Qwen2.5-1.5BApple M2 Proprefill3,968 / 128≥lower bound: veizik is at least this fast1,448 tok/s111,447–1,4482026.10.02-r10
Qwen2.5-1.5BApple M2 Proprefill8,192 / 128≥lower bound: veizik is at least this fast1,273 tok/s111,272–1,2732026.10.02-r10
Qwen2.5-1.5BApple M2 Proprefill16,384 / 128≥lower bound: veizik is at least this fast1,025 tok/s111,025–1,0252026.10.02-r10
Qwen2.5-1.5BApple M2 Proprefill24,576 / 128≥lower bound: veizik is at least this fast854.5 tok/s11854.3–854.72026.10.02-r10
Qwen2.5-1.5BApple M2 Proprefill30,000 / 128≥lower bound: veizik is at least this fast769.6 tok/s11769.5–769.72026.10.02-r10
Qwen2.5-1.5BApple M4 Prodecode512 / 128≥lower bound: veizik is at least this fast215.0 tok/s11199.9–216.62026.10.02-r10
Qwen2.5-1.5BApple M4 Prodecode3,968 / 128≥lower bound: veizik is at least this fast172.3 tok/s9165.0–176.92026.10.02-r10
Qwen2.5-1.5BApple M4 Prodecode8,192 / 128≥lower bound: veizik is at least this fast159.8 tok/s11156.3–163.52026.10.02-r10
Qwen2.5-1.5BApple M4 Prodecode16,384 / 128≥lower bound: veizik is at least this fast134.1 tok/s11128.3–139.52026.10.02-r10
Qwen2.5-1.5BApple M4 Prodecode24,576 / 128≥lower bound: veizik is at least this fast117.8 tok/s11114.4–119.22026.10.02-r10
Qwen2.5-1.5BApple M4 Prodecode30,000 / 128≥lower bound: veizik is at least this fast108.8 tok/s7106.7–109.32026.10.02-r10
Qwen2.5-1.5BApple M4 Proprefill512 / 128≥lower bound: veizik is at least this fast2,296 tok/s112,279–2,3012026.10.02-r10
Qwen2.5-1.5BApple M4 Proprefill3,968 / 128≥lower bound: veizik is at least this fast2,188 tok/s91,836–2,2112026.10.02-r10
Qwen2.5-1.5BApple M4 Proprefill8,192 / 128≥lower bound: veizik is at least this fast1,824 tok/s111,719–1,9332026.10.02-r10
Qwen2.5-1.5BApple M4 Proprefill16,384 / 128≥lower bound: veizik is at least this fast1,506 tok/s111,413–1,5662026.10.02-r10
Qwen2.5-1.5BApple M4 Proprefill24,576 / 128≥lower bound: veizik is at least this fast1,266 tok/s111,197–1,3012026.10.02-r10
Qwen2.5-1.5BApple M4 Proprefill30,000 / 128≥lower bound: veizik is at least this fast1,169 tok/s71,085–1,1742026.10.02-r10
Qwen2.5-7BApple M1 Maxdecode512 / 128≥lower bound: veizik is at least this fast60.4 tok/s1159.5–60.52026.10.02-r10
Qwen2.5-7BApple M1 Maxdecode3,968 / 128≥lower bound: veizik is at least this fast51.7 tok/s1150.6–51.82026.10.02-r10
Qwen2.5-7BApple M1 Maxdecode8,192 / 128≥lower bound: veizik is at least this fast50.5 tok/s1149.9–50.62026.10.02-r10
Qwen2.5-7BApple M1 Maxdecode16,384 / 128≥lower bound: veizik is at least this fast44.3 tok/s1144.1–44.52026.10.02-r10
Qwen2.5-7BApple M1 Maxdecode24,576 / 128≥lower bound: veizik is at least this fast39.4 tok/s1139.1–39.62026.10.02-r10
Qwen2.5-7BApple M1 Maxdecode30,000 / 128≥lower bound: veizik is at least this fast36.8 tok/s1136.3–36.92026.10.02-r10
Qwen2.5-7BApple M1 Maxprefill512 / 128≥lower bound: veizik is at least this fast545.9 tok/s11545.6–546.32026.10.02-r10
Qwen2.5-7BApple M1 Maxprefill3,968 / 128≥lower bound: veizik is at least this fast552.0 tok/s11551.8–552.22026.10.02-r10
Qwen2.5-7BApple M1 Maxprefill8,192 / 128≥lower bound: veizik is at least this fast520.9 tok/s11517.5–521.12026.10.02-r10
Qwen2.5-7BApple M1 Maxprefill16,384 / 128≥lower bound: veizik is at least this fast465.1 tok/s11465.0–465.52026.10.02-r10
Qwen2.5-7BApple M1 Maxprefill24,576 / 128≥lower bound: veizik is at least this fast419.2 tok/s11419.1–419.32026.10.02-r10
Qwen2.5-7BApple M1 Maxprefill30,000 / 128≥lower bound: veizik is at least this fast392.6 tok/s11386.4–392.72026.10.02-r10
Qwen2.5-7BApple M1 Ultradecode512 / 128≥lower bound: veizik is at least this fast104.7 tok/s11104.0–105.12026.10.02-r10
Qwen2.5-7BApple M1 Ultradecode3,968 / 128≥lower bound: veizik is at least this fast88.2 tok/s1187.8–88.52026.10.02-r10
Qwen2.5-7BApple M1 Ultradecode8,192 / 128≥lower bound: veizik is at least this fast84.0 tok/s1183.6–84.32026.10.02-r10
Qwen2.5-7BApple M1 Ultradecode16,384 / 128≥lower bound: veizik is at least this fast72.8 tok/s1172.5–73.02026.10.02-r10
Qwen2.5-7BApple M1 Ultradecode24,576 / 128≥lower bound: veizik is at least this fast65.7 tok/s1165.5–65.82026.10.02-r10
Qwen2.5-7BApple M1 Ultradecode30,000 / 128≥lower bound: veizik is at least this fast60.9 tok/s1160.8–61.42026.10.02-r10
Qwen2.5-7BApple M1 Ultraprefill512 / 128≥lower bound: veizik is at least this fast1,009 tok/s111,007–1,0102026.10.02-r10
Qwen2.5-7BApple M1 Ultraprefill3,968 / 128≥lower bound: veizik is at least this fast1,038 tok/s111,035–1,0392026.10.02-r10
Qwen2.5-7BApple M1 Ultraprefill8,192 / 128≥lower bound: veizik is at least this fast975.9 tok/s11965.0–976.22026.10.02-r10
Qwen2.5-7BApple M1 Ultraprefill16,384 / 128≥lower bound: veizik is at least this fast865.2 tok/s11852.8–865.72026.10.02-r10
Qwen2.5-7BApple M1 Ultraprefill24,576 / 128≥lower bound: veizik is at least this fast769.2 tok/s11763.3–776.22026.10.02-r10
Qwen2.5-7BApple M1 Ultraprefill30,000 / 128≥lower bound: veizik is at least this fast720.1 tok/s11715.2–723.32026.10.02-r10
Qwen2.5-7BApple M2 Prodecode512 / 128≥lower bound: veizik is at least this fast38.3 tok/s1138.0–38.42026.10.02-r10
Qwen2.5-7BApple M2 Prodecode3,968 / 128≥lower bound: veizik is at least this fast33.0 tok/s1132.8–33.12026.10.02-r10
Qwen2.5-7BApple M2 Prodecode8,192 / 128≥lower bound: veizik is at least this fast32.3 tok/s1132.1–32.52026.10.02-r10
Qwen2.5-7BApple M2 Prodecode16,384 / 128≥lower bound: veizik is at least this fast28.5 tok/s1128.4–28.62026.10.02-r10
Qwen2.5-7BApple M2 Prodecode24,576 / 128≥lower bound: veizik is at least this fast25.3 tok/s1125.2–25.32026.10.02-r10
Qwen2.5-7BApple M2 Prodecode30,000 / 128≥lower bound: veizik is at least this fast23.5 tok/s1123.5–23.62026.10.02-r10
Qwen2.5-7BApple M2 Proprefill512 / 128≥lower bound: veizik is at least this fast333.1 tok/s11332.6–333.52026.10.02-r10
Qwen2.5-7BApple M2 Proprefill3,968 / 128≥lower bound: veizik is at least this fast327.2 tok/s11326.9–327.32026.10.02-r10
Qwen2.5-7BApple M2 Proprefill8,192 / 128≥lower bound: veizik is at least this fast306.8 tok/s11306.8–307.02026.10.02-r10
Qwen2.5-7BApple M2 Proprefill16,384 / 128≥lower bound: veizik is at least this fast271.9 tok/s11271.9–272.02026.10.02-r10
Qwen2.5-7BApple M2 Proprefill24,576 / 128≥lower bound: veizik is at least this fast243.6 tok/s11243.6–243.72026.10.02-r10
Qwen2.5-7BApple M2 Proprefill30,000 / 128≥lower bound: veizik is at least this fast227.6 tok/s11227.6–227.72026.10.02-r10
Qwen2.5-7BApple M4 Prodecode512 / 128≥lower bound: veizik is at least this fast54.9 tok/s1150.2–56.32026.10.02-r10
Qwen2.5-7BApple M4 Prodecode3,968 / 128≥lower bound: veizik is at least this fast47.9 tok/s1044.5–48.42026.10.02-r10
Qwen2.5-7BApple M4 Prodecode8,192 / 128≥lower bound: veizik is at least this fast47.4 tok/s645.9–47.82026.10.02-r10
Qwen2.5-7BApple M4 Prodecode16,384 / 128≥lower bound: veizik is at least this fast42.3 tok/s1141.6–42.92026.10.02-r10
Qwen2.5-7BApple M4 Prodecode24,576 / 128≥lower bound: veizik is at least this fast38.4 tok/s1037.8–38.82026.10.02-r10
Qwen2.5-7BApple M4 Proprefill512 / 128≥lower bound: veizik is at least this fast517.1 tok/s11450.9–525.12026.10.02-r10
Qwen2.5-7BApple M4 Proprefill3,968 / 128≥lower bound: veizik is at least this fast460.2 tok/s10434.9–480.42026.10.02-r10
Qwen2.5-7BApple M4 Proprefill8,192 / 128≥lower bound: veizik is at least this fast435.5 tok/s6405.3–463.02026.10.02-r10
Qwen2.5-7BApple M4 Proprefill16,384 / 128≥lower bound: veizik is at least this fast417.9 tok/s11406.1–423.82026.10.02-r10
Qwen2.5-7BApple M4 Proprefill24,576 / 128≥lower bound: veizik is at least this fast376.7 tok/s10358.6–377.92026.10.02-r10
Qwen3-1.7BApple M1 Maxdecode512 / 128207.4 tok/s11206.2–209.42026.10.02-r10
Qwen3-1.7BApple M1 Maxdecode3,968 / 128163.2 tok/s11162.2–164.72026.10.02-r10
Qwen3-1.7BApple M1 Maxdecode8,192 / 128127.8 tok/s11124.6–128.12026.10.02-r10
Qwen3-1.7BApple M1 Maxdecode16,384 / 12896.6 tok/s1194.9–97.02026.10.02-r10
Qwen3-1.7BApple M1 Maxdecode24,576 / 12876.8 tok/s1174.8–77.02026.10.02-r10
Qwen3-1.7BApple M1 Maxdecode30,000 / 12867.5 tok/s1165.8–67.82026.10.02-r10
Qwen3-1.7BApple M1 Maxprefill512 / 1282,417 tok/s112,394–2,4212026.10.02-r10
Qwen3-1.7BApple M1 Maxprefill3,968 / 1282,233 tok/s112,231–2,2342026.10.02-r10
Qwen3-1.7BApple M1 Maxprefill8,192 / 1281,958 tok/s111,955–1,9602026.10.02-r10
Qwen3-1.7BApple M1 Maxprefill16,384 / 1281,562 tok/s111,561–1,5632026.10.02-r10
Qwen3-1.7BApple M1 Maxprefill24,576 / 1281,296 tok/s111,295–1,2962026.10.02-r10
Qwen3-1.7BApple M1 Maxprefill30,000 / 1281,156 tok/s111,152–1,1582026.10.02-r10
Qwen3-1.7BApple M1 Ultradecode512 / 128289.8 tok/s11275.2–292.02026.10.02-r10
Qwen3-1.7BApple M1 Ultradecode3,968 / 128252.8 tok/s11238.6–253.42026.10.02-r10
Qwen3-1.7BApple M1 Ultradecode8,192 / 128200.7 tok/s11199.9–202.22026.10.02-r10
Qwen3-1.7BApple M1 Ultradecode16,384 / 128155.7 tok/s11155.2–157.72026.10.02-r10
Qwen3-1.7BApple M1 Ultradecode24,576 / 128127.0 tok/s11126.1–127.32026.10.02-r10
Qwen3-1.7BApple M1 Ultradecode30,000 / 128114.3 tok/s11113.8–114.62026.10.02-r10
Qwen3-1.7BApple M1 Ultraprefill512 / 1284,213 tok/s113,580–4,2482026.10.02-r10
Qwen3-1.7BApple M1 Ultraprefill3,968 / 1284,171 tok/s114,095–4,1832026.10.02-r10
Qwen3-1.7BApple M1 Ultraprefill8,192 / 1283,732 tok/s113,728–3,7462026.10.02-r10
Qwen3-1.7BApple M1 Ultraprefill16,384 / 1283,001 tok/s112,968–3,0042026.10.02-r10
Qwen3-1.7BApple M1 Ultraprefill24,576 / 1282,503 tok/s112,491–2,5032026.10.02-r10
Qwen3-1.7BApple M1 Ultraprefill30,000 / 1282,235 tok/s112,229–2,2372026.10.02-r10
Qwen3-1.7BApple M2 Prodecode512 / 128143.8 tok/s11142.5–144.42026.10.02-r10
Qwen3-1.7BApple M2 Prodecode3,968 / 128106.6 tok/s11106.3–106.82026.10.02-r10
Qwen3-1.7BApple M2 Prodecode8,192 / 12883.9 tok/s1183.6–84.22026.10.02-r10
Qwen3-1.7BApple M2 Prodecode16,384 / 12860.1 tok/s1160.0–60.22026.10.02-r10
Qwen3-1.7BApple M2 Prodecode24,576 / 12846.4 tok/s1146.2–46.42026.10.02-r10
Qwen3-1.7BApple M2 Prodecode30,000 / 12840.3 tok/s1140.2–40.32026.10.02-r10
Qwen3-1.7BApple M2 Proprefill512 / 1281,459 tok/s111,450–1,4612026.10.02-r10
Qwen3-1.7BApple M2 Proprefill3,968 / 1281,306 tok/s111,305–1,3062026.10.02-r10
Qwen3-1.7BApple M2 Proprefill8,192 / 1281,112 tok/s111,112–1,1122026.10.02-r10
Qwen3-1.7BApple M2 Proprefill16,384 / 128880.2 tok/s11879.5–880.82026.10.02-r10
Qwen3-1.7BApple M2 Proprefill24,576 / 128727.0 tok/s11726.7–727.42026.10.02-r10
Qwen3-1.7BApple M2 Proprefill30,000 / 128651.1 tok/s11650.8–651.62026.10.02-r10
Qwen3-1.7BApple M4 Prodecode512 / 128≥lower bound: veizik is at least this fast204.3 tok/s11200.1–207.72026.10.02-r10
Qwen3-1.7BApple M4 Prodecode3,968 / 128≥lower bound: veizik is at least this fast150.1 tok/s10144.5–152.02026.10.02-r10
Qwen3-1.7BApple M4 Prodecode8,192 / 128≥lower bound: veizik is at least this fast111.3 tok/s11108.6–112.82026.10.02-r10
Qwen3-1.7BApple M4 Prodecode16,384 / 128≥lower bound: veizik is at least this fast78.1 tok/s1075.2–78.52026.10.02-r10
Qwen3-1.7BApple M4 Prodecode24,576 / 128≥lower bound: veizik is at least this fast58.1 tok/s1156.9–58.62026.10.02-r10
Qwen3-1.7BApple M4 Prodecode30,000 / 128≥lower bound: veizik is at least this fast50.0 tok/s1047.9–50.82026.10.02-r10
Qwen3-1.7BApple M4 Proprefill512 / 128≥lower bound: veizik is at least this fast2,161 tok/s112,154–2,1672026.10.02-r10
Qwen3-1.7BApple M4 Proprefill3,968 / 128≥lower bound: veizik is at least this fast1,969 tok/s101,856–1,9892026.10.02-r10
Qwen3-1.7BApple M4 Proprefill8,192 / 128≥lower bound: veizik is at least this fast1,629 tok/s111,539–1,7102026.10.02-r10
Qwen3-1.7BApple M4 Proprefill16,384 / 128≥lower bound: veizik is at least this fast1,346 tok/s101,256–1,3772026.10.02-r10
Qwen3-1.7BApple M4 Proprefill24,576 / 128≥lower bound: veizik is at least this fast1,133 tok/s111,067–1,1422026.10.02-r10
Qwen3-1.7BApple M4 Proprefill30,000 / 128≥lower bound: veizik is at least this fast1,020 tok/s101,004–1,0232026.10.02-r10
NVIDIA · CUDA

9 models · 10 GPUs

Our own figures for the NVIDIA packages, per GPU and mode.

Each table shows the GPUs a model was measured on and one figure per measure: the default mode and the most common workload. Hover a figure for its build, date, runs and range; open All figures for every mode and workload. A figure marked earlier build is our own measurement of an earlier build of the engine, shown because the computation it exercises did not change in substance; it is replaced when the package you download is measured on that GPU. A figure whose build reads carried, measured on another build was measured on that earlier build of the same package and stands for the package named, because the change between the two did not alter the computation. Qwen3.6-35B-A3B can produce several tokens per step with output identical to one token at a time: "Decode" for the package includes that, "one token per step" does not, so the two columns are different measures.

Qwen3.5-0.8B

No comparison setup is published for this model. Our recipe →

GPUDecode tok/sDecode, one token per step tok/sPrompt pass msGPU memory in use GiBThroughput, many at once tok/s
RTX 3090967.5balanced · 20-token prompt795.7512-token promptearlier build4.2balanced · 20-token prompt2.27balanced3,1258 concurrent requests
RTX 40901,318balanced · 21-token prompt1,008512-token promptearlier build3.0balanced · 21-token prompt2.40balanced—
RTX 50901,529balanced · 21-token prompt—2.7balanced · 21-token prompt2.51balanced—
RTX A5000776.7balanced · 20-token prompt—4.9balanced · 20-token prompt2.22balanced—
A100-SXM4-80GB—688.1512-token promptearlier build———
H100 80GB HBM3—1,004512-token promptearlier build———
All 54 figures for Qwen3.5-0.8B
GPUMeasureModeWorkloadValueRunsBuildDate
RTX 3090Decodebalanced20 / 128967.5 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodebalanced20 / 10241,052 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodefast20 / 128972.0 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodequality20 / 128880.8 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodesingle-token decoding512 / —795.7 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 3090Prompt passbalanced20 / 1284.2 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passbalanced21 / 10244.4 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 3090Prompt passfast20 / 1284.1 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passquality20 / 1284.1 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usebalanced— / —2.27 GiB1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usefast— / —2.27 GiB1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usequality— / —2.42 GiB1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput1 request at a timeveizik serve, 1 concurrent request, 128 tokens each531.9 tok/s3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput4 concurrent requestsveizik serve, 4 concurrent requests, 128 tokens each1,778 tok/s3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput8 concurrent requestsveizik serve, 8 concurrent requests, 128 tokens each3,125 tok/s3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090First token1 request at a timeveizik serve, 1 concurrent request, 128 tokens each24.6 ms3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090First token4 concurrent requestsveizik serve, 4 concurrent requests, 128 tokens each62.9 ms3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090First token8 concurrent requestsveizik serve, 8 concurrent requests, 128 tokens each92.6 ms3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 4090Decodebalanced21 / 1281,318 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodebalanced21 / 10241,439 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodefast21 / 1281,316 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodequality21 / 1281,156 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodesingle-token decoding512 / —1,008 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 4090Prompt passbalanced21 / 1283.0 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passbalanced21 / 10243.0 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passfast21 / 1283.0 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passquality21 / 1283.1 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usebalanced— / —2.40 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usefast— / —2.40 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usequality— / —2.55 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 1281,529 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 10241,676 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodefast21 / 1281,531 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodequality21 / 1281,438 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 1282.7 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 10242.7 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passfast21 / 1282.7 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passquality21 / 1282.9 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usebalanced— / —2.51 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usefast— / —2.51 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usequality— / —2.66 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced20 / 128776.7 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced921 / 1024916.1 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodefast20 / 128771.9 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodequality20 / 128714.7 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced20 / 1284.9 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced921 / 102428.6 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passfast20 / 1284.9 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passquality20 / 1284.6 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usebalanced— / —2.22 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usefast— / —2.22 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usequality— / —2.37 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
A100-SXM4-80GBDecodesingle-token decoding512 / —688.1 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
H100 80GB HBM3Decodesingle-token decoding512 / —1,004 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13

Qwen3.5-2B

No comparison setup is published for this model. Our recipe →

GPUDecode tok/sDecode, one token per step tok/sPrompt pass msGPU memory in use GiB
RTX 3090586.5balanced · 20-token prompt435.9512-token promptearlier build6.6balanced · 20-token prompt3.48balanced
RTX 4090886.9balanced · 21-token prompt548.4512-token promptearlier build4.5balanced · 21-token prompt3.61balanced
RTX 50901,138balanced · 21-token prompt—3.8balanced · 21-token prompt3.72balanced
RTX A5000467.3balanced · 20-token prompt—8.4balanced · 20-token prompt3.43balanced
A100-SXM4-80GB—439.4512-token promptearlier build——
H100 80GB HBM3—660.2512-token promptearlier build——
All 48 figures for Qwen3.5-2B
GPUMeasureModeWorkloadValueRunsBuildDate
RTX 3090Decodebalanced20 / 128586.5 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodebalanced20 / 1024560.4 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodefast20 / 128585.1 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodequality20 / 128542.7 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodesingle-token decoding512 / —435.9 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 3090Prompt passbalanced20 / 1286.6 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passbalanced21 / 10247.1 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 3090Prompt passfast20 / 1286.7 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passquality20 / 1286.9 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usebalanced— / —3.48 GiB1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usefast— / —3.48 GiB1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usequality— / —3.80 GiB1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 4090Decodebalanced21 / 128886.9 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodebalanced21 / 1024868.8 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodefast21 / 128895.1 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodequality21 / 128740.9 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodesingle-token decoding512 / —548.4 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 4090Prompt passbalanced21 / 1284.5 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passbalanced21 / 10244.4 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passfast21 / 1284.4 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passquality21 / 1284.7 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usebalanced— / —3.61 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usefast— / —3.61 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usequality— / —3.93 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 1281,138 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 10241,107 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodefast21 / 1281,137 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodequality21 / 1281,041 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 1283.8 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 10243.8 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passfast21 / 1283.8 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passquality21 / 1284.1 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usebalanced— / —3.72 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usefast— / —3.72 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usequality— / —4.03 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced20 / 128467.3 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced921 / 1024449.1 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodefast20 / 128464.6 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodequality20 / 128435.2 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced20 / 1288.4 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced921 / 102458.5 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passfast20 / 1288.4 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passquality20 / 1289.0 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usebalanced— / —3.43 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usefast— / —3.43 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usequality— / —3.75 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
A100-SXM4-80GBDecodesingle-token decoding512 / —439.4 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
H100 80GB HBM3Decodesingle-token decoding512 / —660.2 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13

Qwen3.5-4B

No comparison setup is published for this model. Our recipe →

GPUDecode tok/sDecode, one token per step tok/sPrompt pass msGPU memory in use GiB
RTX 3090376.8balanced · 20-token prompt208.2512-token promptearlier build13.5balanced · 20-token prompt5.49balanced
RTX 4090531.8balanced · 21-token prompt247.5512-token promptearlier build8.7balanced · 21-token prompt5.62balanced
RTX 5090772.2balanced · 21-token prompt—6.9balanced · 21-token prompt5.73balanced
RTX A5000280.5balanced · 20-token prompt—17.2balanced · 20-token prompt5.44balanced
A100-SXM4-80GB—224.3512-token promptearlier build——
H100 80GB HBM3—365.1512-token promptearlier build——
All 48 figures for Qwen3.5-4B
GPUMeasureModeWorkloadValueRunsBuildDate
RTX 3090Decodebalanced20 / 128376.8 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodebalanced20 / 1024397.8 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodefast20 / 128376.8 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodequality20 / 128342.1 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodesingle-token decoding512 / —208.2 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 3090Prompt passbalanced20 / 12813.5 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passbalanced21 / 102413.9 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 3090Prompt passfast20 / 12813.1 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passquality20 / 12814.0 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usebalanced— / —5.49 GiB1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usefast— / —5.49 GiB1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usequality— / —6.27 GiB1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 4090Decodebalanced21 / 128531.8 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodebalanced21 / 1024568.2 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodefast21 / 128535.4 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodequality21 / 128441.8 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodesingle-token decoding512 / —247.5 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 4090Prompt passbalanced21 / 1288.7 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passbalanced21 / 10248.6 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passfast21 / 1288.7 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passquality21 / 12810.5 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usebalanced— / —5.62 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usefast— / —5.62 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usequality— / —6.40 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 128772.2 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 1024822.2 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodefast21 / 128771.4 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodequality21 / 128670.5 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 1286.9 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 10246.9 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passfast21 / 1286.9 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passquality21 / 1287.9 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usebalanced— / —5.73 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usefast— / —5.73 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usequality— / —6.50 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced20 / 128280.5 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced921 / 1024228.6 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodefast20 / 128278.9 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodequality20 / 128256.5 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced20 / 12817.2 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced921 / 1024154.0 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passfast20 / 12817.2 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passquality20 / 12819.2 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usebalanced— / —5.44 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usefast— / —5.44 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usequality— / —6.22 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
A100-SXM4-80GBDecodesingle-token decoding512 / —224.3 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
H100 80GB HBM3Decodesingle-token decoding512 / —365.1 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13

Qwen3.5-9B

No comparison setup is published for this model. Our recipe →

GPUDecode tok/sDecode, one token per step tok/sPrompt pass msGPU memory in use GiBThroughput, many at once tok/s
RTX 3090190.2balanced · 20-token prompt117.0512-token promptearlier build21.7balanced · 20-token prompt8.44balanced663.08 concurrent requests
RTX 4090347.0balanced · 21-token prompt154.9512-token promptearlier build16.8balanced · 21-token prompt8.57balanced—
RTX 5090498.7balanced · 21-token prompt242.4512-token promptearlier build11.1balanced · 21-token prompt8.67balanced—
RTX A5000151.0balanced · 20-token prompt—29.0balanced · 20-token prompt8.39balanced—
A100-SXM4-80GB—129.4512-token promptearlier build———
H100 80GB HBM3—241.6512-token promptearlier build———
All 55 figures for Qwen3.5-9B
GPUMeasureModeWorkloadValueRunsBuildDate
RTX 3090Decodebalanced20 / 128190.2 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodebalanced20 / 1024170.2 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodefast20 / 128190.4 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodequality20 / 128172.8 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodesingle-token decoding512 / —117.0 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 3090Prompt passbalanced20 / 12821.7 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passbalanced21 / 102422.4 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 3090Prompt passfast20 / 12821.8 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passquality20 / 12823.2 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usebalanced— / —8.44 GiB1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usefast— / —8.44 GiB1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usequality— / —9.82 GiB1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput1 request at a timeveizik serve, 1 concurrent request, 128 tokens each101.4 tok/s3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput4 concurrent requestsveizik serve, 4 concurrent requests, 128 tokens each366.4 tok/s3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput8 concurrent requestsveizik serve, 8 concurrent requests, 128 tokens each663.0 tok/s3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090First token1 request at a timeveizik serve, 1 concurrent request, 128 tokens each55.6 ms3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090First token4 concurrent requestsveizik serve, 4 concurrent requests, 128 tokens each176.2 ms3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090First token8 concurrent requestsveizik serve, 8 concurrent requests, 128 tokens each308.8 ms3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 4090Decodebalanced21 / 128347.0 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodebalanced21 / 1024303.4 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodefast21 / 128346.6 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodequality21 / 128272.8 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodesingle-token decoding512 / —154.9 tok/s2earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 4090Prompt passbalanced21 / 12816.8 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passbalanced21 / 102416.6 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passfast21 / 12816.5 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passquality21 / 12821.6 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usebalanced— / —8.57 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usefast— / —8.57 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usequality— / —9.95 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 128498.7 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 1024434.6 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodefast21 / 128498.5 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodequality21 / 128423.4 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodesingle-token decoding512 / —242.4 tok/s3earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 5090Prompt passbalanced21 / 12811.1 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 102411.1 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passfast21 / 12811.1 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passquality21 / 12813.0 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usebalanced— / —8.67 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usefast— / —8.67 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usequality— / —10.05 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced20 / 128151.0 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced921 / 1024124.8 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodefast20 / 128149.1 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodequality20 / 128137.8 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced20 / 12829.0 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced921 / 1024269.9 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passfast20 / 12829.2 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passquality20 / 12831.0 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usebalanced— / —8.39 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usefast— / —8.39 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usequality— / —9.77 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
A100-SXM4-80GBDecodesingle-token decoding512 / —129.4 tok/s2earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
H100 80GB HBM3Decodesingle-token decoding512 / —241.6 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13

Qwen3.6-35B-A3B

No comparison setup is published for this model. Our recipe →

GPUDecode tok/sDecode, one token per step tok/sDecode, several tokens per step tok/sPrompt pass msGPU memory in use GiBThroughput, many at once tok/s
RTX 3090194.3balanced · 22-token prompt197.922-token prompt251.7114-token promptearlier build107.2balanced · 22-token prompt21.44balanced230.68 concurrent requests
RTX 4090—272.5512-token promptearlier build373.9114-token promptearlier build——625.58 sequences at onceearlier build
RTX 5090449.9balanced · 21-token prompt390.8earlier build—61.3balanced · 21-token prompt21.67balanced—
A100-SXM4-80GB—192.1512-token promptearlier build228.1114-token promptearlier build———
H100 80GB HBM3—268.8512-token promptearlier build357.2114-token promptearlier build———
All 60 figures for Qwen3.6-35B-A3B
GPUMeasureModeWorkloadValueRunsBuildDate
RTX 3090Decodebalanced22 / 128194.3 tok/s3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodebalanced22 / 1024209.7 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodefast22 / 128172.7 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodemulti-token steps (lossless)114 / 256251.7 tok/s1earlier build Qwen3.6-35B-A3B previous build g13 (2026-10-14)2026-10-14
RTX 3090Decodesingle-token decoding8 / 512224.4 tok/s3earlier build Qwen3.6-35B-A3B earlier build g13 (2026-10-06)2026-10-06
RTX 3090Decodesingle-token decoding22 / 128197.9 tok/s3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodesingle-token decoding114 / 256213.4 tok/s1earlier build Qwen3.6-35B-A3B previous build g13 (2026-10-14)2026-10-14
RTX 3090Decodesingle-token decoding4000 / 129211.8 tok/s1earlier build Qwen3.6-35B-A3B earlier build g4 (2026-10-06)2026-10-06
RTX 3090Prefillsingle-token decoding8000 / —9,065 tok/s1earlier build Qwen3.6-35B-A3B earlier build g10 (2026-10-06)2026-10-06
RTX 3090Prompt passbalanced22 / 128107.2 ms3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passfast22 / 128107.6 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passsingle-token decoding8000 / 1949.5 ms1earlier build Qwen3.6-35B-A3B earlier build g10 (2026-10-06)2026-10-06
RTX 3090GPU memory in usebalanced— / —21.44 GiB1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usefast— / —21.51 GiB1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usesingle-token decoding— / —21.85 GiB1earlier build Qwen3.6-35B-A3B earlier build g11 (2026-10-06)2026-10-06
RTX 3090Throughput1 request at a timeveizik serve, 1 concurrent request, 128 tokens each160.9 tok/s3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput12 sequences at once12 sequences batched in one process, 64-token prompts411.7 tok/s1earlier build Qwen3.6-35B-A3B earlier build g11 (2026-10-06)2026-10-06
RTX 3090Throughput16 sequences at once16 sequences batched in one process, 64-token prompts416.8 tok/s1earlier build Qwen3.6-35B-A3B earlier build g11 (2026-10-06)2026-10-06
RTX 3090Throughput24 sequences at once24 sequences batched in one process, 64-token prompts424.8 tok/s1earlier build Qwen3.6-35B-A3B earlier build g11 (2026-10-06)2026-10-06
RTX 3090Throughput4 concurrent requestsveizik serve, 4 concurrent requests, 128 tokens each222.4 tok/s3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput4 sequences at once4 sequences batched in one process, 64-token prompts372.8 tok/s1earlier build Qwen3.6-35B-A3B earlier build g11 (2026-10-06)2026-10-06
RTX 3090Throughput8 concurrent requestsveizik serve, 8 concurrent requests, 128 tokens each230.6 tok/s3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput8 sequences at once8 sequences batched in one process, 64-token prompts408.2 tok/s1earlier build Qwen3.6-35B-A3B earlier build g11 (2026-10-06)2026-10-06
RTX 3090First token1 request at a timeveizik serve, 1 concurrent request, 128 tokens each182.7 ms3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090First token4 concurrent requestsveizik serve, 4 concurrent requests, 128 tokens each625.5 ms3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090First token8 concurrent requestsveizik serve, 8 concurrent requests, 128 tokens each1,001.0 ms3cuda-qwen-1.1.4 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 4090Decodemulti-token steps (lossless)114 / 256373.9 tok/s1earlier build Qwen3.6-35B-A3B previous build g13 (2026-10-14)2026-10-14
RTX 4090Decodesingle-token decoding114 / 256276.6 tok/s1earlier build Qwen3.6-35B-A3B previous build g13 (2026-10-14)2026-10-14
RTX 4090Decodesingle-token decoding512 / 64272.5 tok/s1earlier build Qwen3.6-35B-A3B previous build g9 (2026-10-14; decode kernels source-identical to the shipped package, recompiled)2026-10-14
RTX 4090Decodesingle-token decoding7680 / 64251.4 tok/s1earlier build Qwen3.6-35B-A3B previous build g9 (2026-10-14; decode kernels source-identical to the shipped package, recompiled)2026-10-14
RTX 4090Decodesingle-token decoding16384 / 64226.7 tok/s1earlier build Qwen3.6-35B-A3B previous build g9 (2026-10-14; decode kernels source-identical to the shipped package, recompiled)2026-10-14
RTX 4090Throughput1 request at a timeveizik serve, 1 concurrent requests, 128 tokens each210.9 tok/s1earlier build Qwen3.6-35B-A3B earlier serve build (2026-10-14)2026-10-14
RTX 4090Throughput1 sequence1 sequence batched in one process255.8 tok/s1earlier build Qwen3.6-35B-A3B previous build g11 (2026-10-14)2026-10-14
RTX 4090Throughput2 sequences at once2 sequences batched in one process391.3 tok/s1earlier build Qwen3.6-35B-A3B previous build g11 (2026-10-14)2026-10-14
RTX 4090Throughput4 concurrent requestsveizik serve, 4 concurrent requests, 128 tokens each377.6 tok/s1earlier build Qwen3.6-35B-A3B earlier serve build (2026-10-14)2026-10-14
RTX 4090Throughput4 sequences at once4 sequences batched in one process557.7 tok/s1earlier build Qwen3.6-35B-A3B previous build g11 (2026-10-14)2026-10-14
RTX 4090Throughput8 concurrent requestsveizik serve, 8 concurrent requests, 128 tokens each417.5 tok/s1earlier build Qwen3.6-35B-A3B earlier serve build (2026-10-14)2026-10-14
RTX 4090Throughput8 sequences at once8 sequences batched in one process625.5 tok/s1earlier build Qwen3.6-35B-A3B previous build g11 (2026-10-14)2026-10-14
RTX 5090Decodebalanced21 / 128449.9 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 1024449.6 tok/s1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodefast21 / 128544.7 tok/s2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodesingle-token decodingshort prompt390.8 tok/s3earlier build Qwen3.6-35B-A3B previous build with the router change (2026-10-14)2026-10-14
RTX 5090Prompt passbalanced21 / 12861.3 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 102461.5 ms1cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passfast21 / 12861.3 ms2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usebalanced— / —21.67 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usefast— / —21.74 GiB2cuda-qwen-1.1.4 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
A100-SXM4-80GBDecodemulti-token steps (lossless)114 / 256228.1 tok/s1earlier build Qwen3.6-35B-A3B previous build g13 (2026-10-14)2026-10-14
A100-SXM4-80GBDecodesingle-token decoding114 / 256184.0 tok/s1earlier build Qwen3.6-35B-A3B previous build g13 (2026-10-14)2026-10-14
A100-SXM4-80GBDecodesingle-token decoding512 / 64192.1 tok/s1earlier build Qwen3.6-35B-A3B previous build g9 (2026-10-14; decode kernels source-identical to the shipped package, recompiled)2026-10-14
A100-SXM4-80GBDecodesingle-token decoding7680 / 64179.2 tok/s1earlier build Qwen3.6-35B-A3B previous build g9 (2026-10-14; decode kernels source-identical to the shipped package, recompiled)2026-10-14
A100-SXM4-80GBDecodesingle-token decoding16384 / 64169.8 tok/s1earlier build Qwen3.6-35B-A3B previous build g9 (2026-10-14; decode kernels source-identical to the shipped package, recompiled)2026-10-14
H100 80GB HBM3Decodemulti-token steps (lossless)114 / 256357.2 tok/s1earlier build Qwen3.6-35B-A3B previous build g13 (2026-10-14)2026-10-14
H100 80GB HBM3Decodesingle-token decoding114 / 256284.1 tok/s1earlier build Qwen3.6-35B-A3B previous build g13 (2026-10-14)2026-10-14
H100 80GB HBM3Decodesingle-token decoding512 / 64268.8 tok/s1earlier build Qwen3.6-35B-A3B previous build g9 (2026-10-14; decode kernels source-identical to the shipped package, recompiled)2026-10-14
H100 80GB HBM3Decodesingle-token decoding7680 / 64259.1 tok/s1earlier build Qwen3.6-35B-A3B previous build g9 (2026-10-14; decode kernels source-identical to the shipped package, recompiled)2026-10-14
H100 80GB HBM3Decodesingle-token decoding16384 / 64251.2 tok/s1earlier build Qwen3.6-35B-A3B previous build g9 (2026-10-14; decode kernels source-identical to the shipped package, recompiled)2026-10-14
H100 80GB HBM3Decodesingle-token decoding49152 / 64218.9 tok/s1earlier build Qwen3.6-35B-A3B previous build g9, long-context variant (2026-10-14)2026-10-14
H100 80GB HBM3Decodesingle-token decoding64512 / 64206.3 tok/s1earlier build Qwen3.6-35B-A3B previous build g9, long-context variant (2026-10-14)2026-10-14
H100 80GB HBM3Decodesingle-token decoding131072 / 64172.4 tok/s1earlier build Qwen3.6-35B-A3B previous build g9, long-context variant (2026-10-14)2026-10-14

SD3.5-medium

No comparison setup is published for this model. Why →

earlier build Every figure for this model is from an earlier build; the package has not been measured yet.

GPUPer step, balanced s/stepPer step, fast s/stepPeak memory GBFinished image s
RTX 30900.3480.27616.61quality22.8balanced
RTX 40900.1690.13016.88quality14.4balanced
RTX 50900.1280.11017.11quality14.6balanced
RTX A5000—0.35416.30balanced29.3balanced
A400.3580.28916.42balanced24.8balanced
A100-SXM4-80GB———31.3balanced
H100 80GB HBM30.2190.19317.30balanced19.1balanced
RTX PRO 4500 Blackwell0.2520.21316.41balanced16.7balanced
All 101 figures for SD3.5-medium
GPUMeasureModeWorkloadValueRunsBuildDate
RTX 3090Finished imagebalanced1024x1024, 40 steps22.8 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Finished imagefast1024x1024, 40 steps23.9 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Finished imagelow-memory1024x1024, 40 steps33.1 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Finished imagequality1024x1024, 40 steps32.3 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Finished imageultrafast1024x1024, 40 steps19.4 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Per stepbalanced1024x1024, 40 steps0.348 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Per stepfast1024x1024, 40 steps0.276 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Per steplow-memory1024x1024, 40 steps0.346 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Per stepultrafast1024x1024, 40 steps0.276 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Peak memorybalanced1024x1024, 40 steps16.41 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Peak memoryfast1024x1024, 40 steps15.35 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Peak memorylow-memory1024x1024, 40 steps7.08 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Peak memoryquality1024x1024, 40 steps16.61 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Peak memoryultrafast1024x1024, 40 steps15.35 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 4090Finished imagebalanced1024x1024, 40 steps14.4 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Finished imagefast1024x1024, 40 steps17.6 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Finished imagelow-memory1024x1024, 40 steps26.4 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Finished imagequality1024x1024, 40 steps20.8 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Finished imageultrafast1024x1024, 40 steps12.9 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Per stepbalanced1024x1024, 40 steps0.169 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Per stepfast1024x1024, 40 steps0.130 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Per steplow-memory1024x1024, 40 steps0.168 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Per stepultrafast1024x1024, 40 steps0.130 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Peak memorybalanced1024x1024, 40 steps16.68 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Peak memoryfast1024x1024, 40 steps15.62 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Peak memorylow-memory1024x1024, 40 steps7.74 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Peak memoryquality1024x1024, 40 steps16.88 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Peak memoryultrafast1024x1024, 40 steps15.62 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Finished imagebalanced1024x1024, 40 steps14.6 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Finished imagefast1024x1024, 40 steps20.2 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Finished imagelow-memory1024x1024, 40 steps18.0 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Finished imagequality1024x1024, 40 steps20.2 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Finished imageultrafast1024x1024, 40 steps13.3 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Per stepbalanced1024x1024, 40 steps0.128 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Per stepfast1024x1024, 40 steps0.110 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Per steplow-memory1024x1024, 40 steps0.128 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Per stepultrafast1024x1024, 40 steps0.111 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Peak memorybalanced1024x1024, 40 steps16.92 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Peak memoryfast1024x1024, 40 steps15.86 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Peak memorylow-memory1024x1024, 40 steps7.59 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Peak memoryquality1024x1024, 40 steps17.11 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Peak memoryultrafast1024x1024, 40 steps15.86 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Finished imagebalanced1024x1024, 40 steps29.3 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Finished imagefast1024x1024, 40 steps28.0 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Finished imageultrafast1024x1024, 40 steps24.7 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Per stepfast1024x1024, 40 steps0.354 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Per stepultrafast1024x1024, 40 steps0.354 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Peak memorybalanced1024x1024, 40 steps16.30 GB5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Peak memoryfast1024x1024, 40 steps15.24 GB5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Peak memorylow-memory1024x1024, 40 steps7.36 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Peak memoryquality1024x1024, 40 steps16.50 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Peak memoryultrafast1024x1024, 40 steps15.24 GB5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Finished imagebalanced1024x1024, 40 steps24.8 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Finished imagefast1024x1024, 40 steps22.2 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Finished imagelow-memory1024x1024, 40 steps29.3 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Finished imagequality1024x1024, 40 steps35.0 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Finished imageultrafast1024x1024, 40 steps21.8 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Per stepbalanced1024x1024, 40 steps0.358 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Per stepfast1024x1024, 40 steps0.289 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Per steplow-memory1024x1024, 40 steps0.356 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Per stepquality1024x1024, 40 steps0.613 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Per stepultrafast1024x1024, 40 steps0.289 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Peak memorybalanced1024x1024, 40 steps16.42 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Peak memoryfast1024x1024, 40 steps15.35 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Peak memorylow-memory1024x1024, 40 steps7.47 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Peak memoryquality1024x1024, 40 steps16.61 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Peak memoryultrafast1024x1024, 40 steps15.35 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A100-SXM4-80GBFinished imagebalanced1024x1024, 40 steps31.3 s1earlier build SD3.5 sealed ship line (Oct 12 and 16)2026-10-12
A100-SXM4-80GBFinished imagefast1024x1024, 40 steps30.7 s1earlier build SD3.5 sealed ship line (Oct 12 and 16)2026-10-12
A100-SXM4-80GBFinished imagequality1024x1024, 40 steps62.8 s1earlier build SD3.5 sealed ship line (Oct 12 and 16)2026-10-12
A100-SXM4-80GBFinished imageultrafast1024x1024, 40 steps29.5 s1earlier build SD3.5 sealed ship line (Oct 12 and 16)2026-10-12
H100 80GB HBM3Finished imagebalanced1024x1024, 40 steps19.1 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Finished imagefast1024x1024, 40 steps17.5 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Finished imagelow-memory1024x1024, 40 steps23.8 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Finished imagequality1024x1024, 40 steps24.6 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Finished imageultrafast1024x1024, 40 steps17.7 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Per stepbalanced1024x1024, 40 steps0.219 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Per stepfast1024x1024, 40 steps0.193 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Per steplow-memory1024x1024, 40 steps0.218 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Per stepquality1024x1024, 40 steps0.362 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Per stepultrafast1024x1024, 40 steps0.193 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Peak memorybalanced1024x1024, 40 steps17.30 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Peak memoryfast1024x1024, 40 steps15.76 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Peak memorylow-memory1024x1024, 40 steps7.48 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Peak memoryquality1024x1024, 40 steps17.02 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Peak memoryultrafast1024x1024, 40 steps15.76 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellFinished imagebalanced1024x1024, 40 steps16.7 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellFinished imagefast1024x1024, 40 steps15.2 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellFinished imagelow-memory1024x1024, 40 steps26.9 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellFinished imagequality1024x1024, 40 steps23.7 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellFinished imageultrafast1024x1024, 40 steps15.0 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPer stepbalanced1024x1024, 40 steps0.252 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPer stepfast1024x1024, 40 steps0.213 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPer steplow-memory1024x1024, 40 steps0.250 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPer stepquality1024x1024, 40 steps0.426 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPer stepultrafast1024x1024, 40 steps0.213 s/step5earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPeak memorybalanced1024x1024, 40 steps16.41 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPeak memoryfast1024x1024, 40 steps15.34 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPeak memorylow-memory1024x1024, 40 steps7.07 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPeak memoryquality1024x1024, 40 steps16.60 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPeak memoryultrafast1024x1024, 40 steps15.34 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03

Powered by Stability AI.

SD3.5-large

No comparison setup is published for this model. Why →

earlier build Every figure for this model is from an earlier build; the package has not been measured yet.

GPUPer step, balanced s/stepPer step, fast s/stepPeak memory GBFinished image s
RTX 30900.8310.55813.78balanced49.0balanced
RTX 40900.4230.27013.92balanced31.1balanced
RTX 50900.3120.23114.03quality30.2balanced
RTX A50001.1460.74513.72balanced68.3balanced
A400.8650.59013.78balanced55.5balanced
A100 80GB PCIe—0.489—54.8balanced
H100 80GB HBM30.5050.42213.98balanced38.9balanced
RTX PRO 4500 Blackwell0.6050.46913.78balanced36.7balanced
All 104 figures for SD3.5-large
GPUMeasureModeWorkloadValueRunsBuildDate
RTX 3090Finished imagebalanced1024x1024, 40 steps49.0 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Finished imagefast1024x1024, 40 steps38.4 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Finished imagelow-memory1024x1024, 40 steps60.0 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Finished imagequality1024x1024, 40 steps76.7 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Finished imageultrafast1024x1024, 40 steps38.0 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Per stepbalanced1024x1024, 40 steps0.831 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Per stepfast1024x1024, 40 steps0.558 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Per steplow-memory1024x1024, 40 steps0.828 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Per stepquality1024x1024, 40 steps1.512 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Per stepultrafast1024x1024, 40 steps0.559 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Peak memorybalanced1024x1024, 40 steps13.78 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Peak memoryfast1024x1024, 40 steps13.78 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Peak memorylow-memory1024x1024, 40 steps10.66 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Peak memoryquality1024x1024, 40 steps13.78 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 3090Peak memoryultrafast1024x1024, 40 steps13.78 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-14
RTX 4090Finished imagebalanced1024x1024, 40 steps31.1 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Finished imagefast1024x1024, 40 steps25.2 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Finished imagelow-memory1024x1024, 40 steps41.5 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Finished imagequality1024x1024, 40 steps49.0 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Finished imageultrafast1024x1024, 40 steps24.8 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Per stepbalanced1024x1024, 40 steps0.423 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Per stepfast1024x1024, 40 steps0.270 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Per steplow-memory1024x1024, 40 steps0.418 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Per stepquality1024x1024, 40 steps0.873 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Per stepultrafast1024x1024, 40 steps0.270 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Peak memorybalanced1024x1024, 40 steps13.92 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Peak memoryfast1024x1024, 40 steps13.92 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Peak memorylow-memory1024x1024, 40 steps10.80 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Peak memoryquality1024x1024, 40 steps13.92 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 4090Peak memoryultrafast1024x1024, 40 steps13.92 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Finished imagebalanced1024x1024, 40 steps30.2 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Finished imagefast1024x1024, 40 steps27.3 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Finished imagelow-memory1024x1024, 40 steps33.9 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Finished imagequality1024x1024, 40 steps44.6 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Finished imageultrafast1024x1024, 40 steps27.3 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Per stepbalanced1024x1024, 40 steps0.312 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Per stepfast1024x1024, 40 steps0.231 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Per steplow-memory1024x1024, 40 steps0.312 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Per stepultrafast1024x1024, 40 steps0.231 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Peak memorybalanced1024x1024, 40 steps14.03 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Peak memoryfast1024x1024, 40 steps14.03 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Peak memorylow-memory1024x1024, 40 steps10.92 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Peak memoryquality1024x1024, 40 steps14.03 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX 5090Peak memoryultrafast1024x1024, 40 steps14.03 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Finished imagebalanced1024x1024, 40 steps68.3 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Finished imagefast1024x1024, 40 steps50.6 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Finished imagelow-memory1024x1024, 40 steps71.6 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Finished imagequality1024x1024, 40 steps106.9 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Finished imageultrafast1024x1024, 40 steps52.0 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Per stepbalanced1024x1024, 40 steps1.146 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Per stepfast1024x1024, 40 steps0.745 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Per steplow-memory1024x1024, 40 steps1.141 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Per stepquality1024x1024, 40 steps2.121 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Per stepultrafast1024x1024, 40 steps0.743 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Peak memorybalanced1024x1024, 40 steps13.72 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Peak memoryfast1024x1024, 40 steps13.72 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Peak memorylow-memory1024x1024, 40 steps10.61 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Peak memoryquality1024x1024, 40 steps13.72 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX A5000Peak memoryultrafast1024x1024, 40 steps13.72 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Finished imagebalanced1024x1024, 40 steps55.5 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Finished imagefast1024x1024, 40 steps45.2 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Finished imagelow-memory1024x1024, 40 steps60.7 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Finished imagequality1024x1024, 40 steps88.0 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Finished imageultrafast1024x1024, 40 steps45.0 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Per stepbalanced1024x1024, 40 steps0.865 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Per stepfast1024x1024, 40 steps0.590 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Per steplow-memory1024x1024, 40 steps0.863 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Per stepquality1024x1024, 40 steps1.679 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Per stepultrafast1024x1024, 40 steps0.589 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Peak memorybalanced1024x1024, 40 steps13.78 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Peak memoryfast1024x1024, 40 steps13.78 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Peak memorylow-memory1024x1024, 40 steps10.66 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Peak memoryquality1024x1024, 40 steps13.78 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A40Peak memoryultrafast1024x1024, 40 steps13.78 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
A100 80GB PCIeFinished imagebalanced1024x1024, 40 steps54.8 s1earlier build SD3.5 sealed ship line (Oct 12 and 16)2026-10-12
A100 80GB PCIeFinished imagefast1024x1024, 40 steps48.6 s1earlier build SD3.5 sealed ship line (Oct 12 and 16)2026-10-12
A100 80GB PCIeFinished imageultrafast1024x1024, 40 steps45.9 s1earlier build SD3.5 sealed ship line (Oct 12 and 16)2026-10-12
A100 80GB PCIePer stepfast1024x1024, 40 steps0.489 s/step2earlier build SD3.5 sealed ship line (Oct 12 and 16)2026-10-16
A100 80GB PCIePer stepultrafast1024x1024, 40 steps0.496 s/step2earlier build SD3.5 sealed ship line (Oct 12 and 16)2026-10-16
H100 80GB HBM3Finished imagebalanced1024x1024, 40 steps38.9 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Finished imagefast1024x1024, 40 steps36.0 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Finished imagelow-memory1024x1024, 40 steps44.0 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Finished imagequality1024x1024, 40 steps56.4 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Finished imageultrafast1024x1024, 40 steps35.3 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Per stepbalanced1024x1024, 40 steps0.505 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Per stepfast1024x1024, 40 steps0.422 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Per steplow-memory1024x1024, 40 steps0.504 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Per stepquality1024x1024, 40 steps0.929 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Per stepultrafast1024x1024, 40 steps0.422 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Peak memorybalanced1024x1024, 40 steps13.98 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Peak memoryfast1024x1024, 40 steps13.98 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Peak memorylow-memory1024x1024, 40 steps10.87 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Peak memoryquality1024x1024, 40 steps13.98 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
H100 80GB HBM3Peak memoryultrafast1024x1024, 40 steps13.98 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellFinished imagebalanced1024x1024, 40 steps36.7 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellFinished imagelow-memory1024x1024, 40 steps44.8 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellFinished imagequality1024x1024, 40 steps60.5 s1earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPer stepbalanced1024x1024, 40 steps0.605 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPer stepfast1024x1024, 40 steps0.469 s/step2earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPer steplow-memory1024x1024, 40 steps0.602 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPer stepquality1024x1024, 40 steps1.194 s/step3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPeak memorybalanced1024x1024, 40 steps13.78 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPeak memorylow-memory1024x1024, 40 steps10.66 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03
RTX PRO 4500 BlackwellPeak memoryquality1024x1024, 40 steps13.78 GB3earlier build SD3.5 sweep build (Oct 3 and 14)2026-10-03

Powered by Stability AI.

Wan2.1-T2V-1.3B

Compared against diffusers 0.39.0 (PyTorch 2.8.0+cu128, CUDA runtime 12.8, transformers 5.14.1, accelerate 1.15.0), bf16; on RTX 4090, RTX 5090, A100-SXM4-80GB. Setup and recipe for both engines →

earlier build Every figure for this model is from an earlier build; the package has not been measured yet.

GPUPer step s/stepRender sPeak memory GB
RTX 3080 Ti6.648balanced338.4balanced6.88balanced
RTX 4090—162.9balanced—
RTX 5090—118.1balanced—
A100-SXM4-80GB—192.1balanced—
All 23 figures for Wan2.1-T2V-1.3B
GPUMeasureModeWorkloadValueRunsBuildDate
RTX 3080 TiRenderbalanced832x480, 81 frames, 50 steps338.4 s1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiRenderbalanced1280x720, 81 frames, 50 steps1,473.8 s1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiRenderfast832x480, 81 frames, 50 steps258.8 s1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiRenderfast1280x720, 81 frames, 50 steps1,025.5 s1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiRenderquality832x480, 81 frames, 50 steps407.4 s1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiRenderultrafast832x480, 81 frames, 50 steps178.8 s1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiRenderultrafast1280x720, 81 frames, 50 steps601.2 s1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiPer stepbalanced832x480, 81 frames, 50 steps6.648 s/step1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiPer stepbalanced1280x720, 81 frames, 50 steps29.1 s/step1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiPer stepfast832x480, 81 frames, 50 steps5.055 s/step1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiPer stepfast1280x720, 81 frames, 50 steps20.1 s/step1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiPer stepquality832x480, 81 frames, 50 steps8.027 s/step1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiPer stepquality1280x720, 81 frames, 50 steps35.9 s/step1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiPer stepultrafast832x480, 81 frames, 50 steps3.455 s/step1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiPer stepultrafast1280x720, 81 frames, 50 steps11.6 s/step1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiPeak memorybalanced832x480, 81 frames, 50 steps6.88 GB1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiPeak memoryfast832x480, 81 frames, 50 steps6.88 GB1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiPeak memoryquality832x480, 81 frames, 50 steps6.88 GB1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiPeak memoryquality1280x720, 81 frames, 50 steps7.16 GB1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 3080 TiPeak memoryultrafast832x480, 81 frames, 50 steps6.88 GB1earlier build WAN 2.1 1.3B research build 056e44cb (2026-10-22)2026-10-22
RTX 4090Renderbalanced832x480, 81 frames, 50 steps162.9 s2earlier build WAN 2.1 1.3B development build (2026-10-14)2026-10-14
RTX 5090Renderbalanced832x480, 81 frames, 50 steps118.1 s2earlier build WAN 2.1 1.3B earlier build (2026-10-10)2026-10-10
A100-SXM4-80GBRenderbalanced832x480, 81 frames, 50 steps192.1 s2earlier build WAN 2.1 1.3B development build (2026-10-14)2026-10-14

Powered by Wan-AI/Alibaba.

Wan2.2-T2V-A14B

No comparison setup is published for this model. Why →

earlier build Every figure for this model is from an earlier build; the package has not been measured yet.

GPUPer step s/stepRender sPeak memory GB
RTX 409032.4ultrafast1,303.7ultrafast22.19ultrafast
RTX 509026.1ultrafast1,050.0ultrafast22.19ultrafast
All 6 figures for Wan2.2-T2V-A14B
GPUMeasureModeWorkloadValueRunsBuildDate
RTX 4090Renderultrafast1280x720, 81 frames, 40 steps1,303.7 s1earlier build WAN 2.2 A14B development build (2026-10-20)2026-10-20
RTX 4090Per stepultrafast1280x720, 81 frames, 40 steps32.4 s/step1earlier build WAN 2.2 A14B development build (2026-10-20)2026-10-20
RTX 4090Peak memoryultrafast1280x720, 81 frames, 40 steps22.19 GB1earlier build WAN 2.2 A14B development build (2026-10-20)2026-10-20
RTX 5090Renderultrafast1280x720, 81 frames, 40 steps1,050.0 s1earlier build WAN 2.2 A14B development build (2026-10-20)2026-10-20
RTX 5090Per stepultrafast1280x720, 81 frames, 40 steps26.1 s/step1earlier build WAN 2.2 A14B development build (2026-10-20)2026-10-20
RTX 5090Peak memoryultrafast1280x720, 81 frames, 40 steps22.19 GB1earlier build WAN 2.2 A14B development build (2026-10-20)2026-10-20

Powered by Wan-AI/Alibaba.

Measurement conditions

Dedicated single-GPU cloud host; one clean cold render, load+denoise end to end. Dedicated single-GPU cloud host; one clean render; board power as recorded in the evidence file. Dedicated single-GPU cloud host; peak device memory in use during the render. Dedicated single-GPU cloud host; per-step median of the denoise loop; board power as recorded in the sweep. Default board power (limit not recorded); no quiet gate; dedicated single-GPU cloud host, native Linux; driver 595.91.07. NVIDIA A100 80GB PCIe (81920 MiB, sm_80); power 300 W, uncapped; driver 595.91.07 / 580.126.20 / 580.159.03 (varies by pod); dedicated single-GPU cloud host; no quiet gate. Each run is a single render on a separate pod (pod-to-pod spread about 2%). NVIDIA A100 80GB PCIe (81920 MiB, sm_80); power 300 W, uncapped; driver 595.91.07; dedicated single-GPU cloud host; no quiet gate. NVIDIA A100 80GB; board power not recorded; driver not recorded; dedicated single-GPU cloud host; no quiet gate; one run; a measurement of the shipped package on this card will replace it. NVIDIA A100 80GB; dedicated single-GPU cloud host; board power not recorded; no quiet gate. NVIDIA A100-SXM4-80GB (81920 MiB, sm_80); power 400 W, uncapped; driver 580.159.03; dedicated single-GPU cloud host; no quiet gate. NVIDIA A100-SXM4-80GB; 400 W limit, mean draw 387.65 W; driver 580.159.04; dedicated single-GPU cloud host; two runs. NVIDIA A100-SXM4-80GB; board power not recorded; dedicated single-GPU cloud host; no quiet gate; one run; a measurement of the shipped package on this card will replace it. NVIDIA A100-SXM4-80GB; board power not recorded; dedicated single-GPU cloud host; no quiet gate; two runs on separate pods; a measurement of the shipped package on this card will replace it. NVIDIA GeForce RTX 3080 Ti (12 GB), dedicated to the run, 400 W limit, driver 591.86; 1 run(s). NVIDIA GeForce RTX 3090 (24 GB) in the owner's workstation at a 420 W limit, not a cloud host; other GPU activity not recorded. NVIDIA GeForce RTX 3090 (24 GB, sm_86); power 350 W, uncapped; driver 580.126.20; dedicated single-GPU cloud host; no quiet gate. NVIDIA GeForce RTX 3090; dedicated single-GPU cloud host, native Linux; default board limit 370 W, never reached (no power throttle, GPU at or below 44 C); driver 580.95.05; quiet pod. NVIDIA GeForce RTX 3090; dedicated single-GPU cloud host, native Linux; default board limit 370 W, never reached (no power throttle, GPU at or below 44 C); driver 580.95.05; quiet re-run. NVIDIA GeForce RTX 3090; dedicated single-GPU cloud host; board power not recorded; no quiet gate. NVIDIA GeForce RTX 4090 (24 GB, sm_89); power 450 W, uncapped; driver 580.159.04; dedicated single-GPU cloud host; no quiet gate. NVIDIA GeForce RTX 4090, dedicated single-GPU cloud host; board power not recorded; one run per level. NVIDIA GeForce RTX 4090; 450 W limit, mean draw 446.95 W; driver 580.126.20; dedicated single-GPU cloud host; two runs. NVIDIA GeForce RTX 4090; board power not recorded; driver 580.159.04; dedicated single-GPU cloud host; no quiet gate; one run; a measurement of the shipped package on this card will replace it. NVIDIA GeForce RTX 4090; dedicated single-GPU cloud host; board power not recorded; no quiet gate. NVIDIA GeForce RTX 4090; dedicated single-GPU cloud host; board power not recorded; runs on separate pods; no quiet gate. NVIDIA GeForce RTX 5090 (32 GB, sm_120); power 575 W, uncapped; driver 590.48.01; dedicated single-GPU cloud host; no quiet gate. NVIDIA GeForce RTX 5090; 575 W, uncapped; driver 570.195.03; dedicated single-GPU cloud host; two runs. NVIDIA GeForce RTX 5090; dedicated single-GPU cloud host; board power not recorded; runs on separate pods; no quiet gate. NVIDIA H100 80GB HBM3 (SXM); board power not recorded; dedicated single-GPU cloud host; no quiet gate; one run; a measurement of the shipped package on this card will replace it. NVIDIA H100 80GB HBM3 (SXM); board power not recorded; driver 580.126.09; dedicated single-GPU cloud host; no quiet gate; one run; a measurement of the shipped package on this card will replace it. NVIDIA H100 80GB HBM3 (SXM); dedicated single-GPU cloud host; board power not recorded; no quiet gate. NVIDIA RTX A5000 (24 GB); 230 W, uncapped; driver 580.159.04; dedicated single-GPU cloud host; no quiet gate. No quiet gate; dedicated single-GPU cloud host, native Linux; board power default (not recorded); driver not recorded in the log.

Edge Device · SoC

6 language models · 4 image/video models · 2 boards

Measured on the boards — CPU only, no GPU.

veizik runs language, image and video models to completion on the Raspberry Pi 5 (4× Cortex-A76, 8 GB, NVMe) and the 512 MB Pi Zero 2 W (4× Cortex-A53), on the CPU alone — no GPU, no Python. These are veizik’s own on-device figures; the full per---mode quality ladder and the sample renders are on the Edge page, and the reproduction setup is on Reproduce.

Language

ModelDeviceDecode tok/sPerplexity (PPL80)Peak RAM
GPT-2 124MPi 585.1 → 124.129.6 → 43.4110 MiB
SmolLM2-135MPi 566.2 → 109.217.0 → 21.9104–176 MiB
SmolLM2-360MPi 532.7 → 43.812.7 → 14.9—
LFM2.5-230MPi 546.2 → 73.936.7 → 39.0171 MiB
GPT-2 124MZero 2 W12.15+4.4% over full precision96 MiB
SmolLM2-135MZero 2 W8.8015.73137 MiB

Image & video

ModelDeviceTimeQuality vs referencePeak RAM
SDXS-512Pi 52.9 s (2.28 s @ 2.4 GHz)30.2 dB PSNR · CLIP 0.9971.14 GB
Stable Diffusion 1.5 · LCM 4-stepPi 537.3 sbit-identical to the Mac pipeline—
Stable Diffusion 1.5 · full 25-stepPi 5249 slatent cosine 0.9841.91 GB
SANA-Sprint 0.6B · 1024×1024Pi 566.9 s (1-step)35.8 dB PSNR · CLIP 31.5959 MB
AnimateDiff-Lightning · videoPi 552.2 s (8-frame warm)16 frames · 512×512—
SDXS-512 · prompt→imageZero 2 W74.9 stext-encoder cosine 1.000000~247 MB

Warm, end-to-end, at a stock 1.8 GHz unless noted. Decode and perplexity span the five --mode tiers (quality → ultrafast); image and video times are the quality tier. Quality is measured against a full-precision reference on the same inputs. veizik’s number formats are proprietary — the measured results are shown, the format internals are not.

Jetson AGX Orin · FLUX.1-schnell — render-proven previewPreview
The full 12-billion-parameter FLUX.1-schnell image model runs end to end on a single NVIDIA Jetson AGX Orin (sm_87, JetPack 6, CUDA 12) — a complete 1024×1024 render in ~166 s cold (CLIP 0.46 s · T5-XXL 21.7 s · DiT 144.5 s; 145–221 s across the five sample prompts), at the W8 quality tier, seed 42, 4 steps. This is an edge GPU module — separate from the CPU boards above — and is a Preview: the package is not downloadable until the public flip. Cold figures only; the hypercache warm time is measured separately and not published yet. The reproducible prompts and the five sample renders are in the gallery.
FLUX.1-schnell — Black Forest Labs, Apache-2.0. Rendered by the veizik dep0 engine; model weights are not veizik-authored.
Memory to run

What fits before it runs out

Measured peak device memory at each mode’s operating point — language at the published context length, image and video at the published resolution and frames. It is the measured working set, not a hard minimum: a longer prompt or a larger frame needs more headroom, so leave some. On NVIDIA this is device (GPU) memory; the engine also uses some host system RAM on top — more in the low-memory streaming mode — which this table does not list. Apple Silicon uses unified memory, so a model’s weights (in the catalogue) share system RAM rather than a fixed card.

Check the fit before you run: the CLI verifies the GPU’s compute capability at install, but it does not pre-check that a model and mode fit the available memory — so an oversized choice fails at run time rather than being refused up front. Use this table (and the hardware check) to pick a mode that fits, or run a smaller model or the low-memory mode.

Language

ModelModeWorking setRuns on
qwen35-0.8bquality2.66 GiBan 8 GB card
balanced2.51 GiBan 8 GB card
fast2.51 GiBan 8 GB card
qwen35-2bquality4.03 GiBan 8 GB card
balanced3.72 GiBan 8 GB card
fast3.72 GiBan 8 GB card
qwen35-4bquality6.50 GiBan 8 GB card
balanced5.73 GiBan 8 GB card
fast5.73 GiBan 8 GB card
qwen35-9bquality10.05 GiBa 12 GB card
balanced8.67 GiBa 12 GB card
fast8.67 GiBa 12 GB card
qwen36-35b-a3bbalanced21.67 GiBa 24 GB card
fast21.74 GiBa 24 GB card
single-token decoding21.85 GiBa 24 GB card

Image

ModelModeWorking setRuns on
sd3.5-largequality14.03 GBa 16 GB card
balanced14.03 GBa 16 GB card
fast14.03 GBa 16 GB card
ultrafast14.03 GBa 16 GB card
low-memory10.92 GBa 12 GB card
sd3.5-mediumquality17.11 GBa 24 GB card
balanced17.30 GBa 24 GB card
fast15.86 GBa 16 GB card
ultrafast15.86 GBa 16 GB card
low-memory7.74 GBan 8 GB card

Video

ModelModeWorking setRuns on
wan21-t2v-1.3bquality7.16 GBan 8 GB card
balanced6.88 GBan 8 GB card
fast6.88 GBan 8 GB card
ultrafast6.88 GBan 8 GB card
wan22-t2v-a14bultrafast22.19 GBa 24 GB card
Verification status

What is measured, where

How to read each state, and the provenance on every figure.

✓ Verified — measured here• Supported — engine path / figures withheld○ Announced? Not measured on this target
Model familyNVIDIAApple Silicon
SDXS CreativeML Open RAIL++-M? Not measured on this target? Not measured on this target
Stable Diffusion 1.5 CreativeML OpenRAIL-M? Not measured on this target? Not measured on this target
Stable Diffusion 3.5 Stability AI Community License✓ Verified — measured here? Not measured on this target
GPT-2 MIT? Not measured on this target? Not measured on this target
LFM2.5 LFM Open License v1.0? Not measured on this target? Not measured on this target
Qwen2.5 Apache-2.0? Not measured on this target✓ Verified — measured here
Qwen3 Apache-2.0? Not measured on this target✓ Verified — measured here
Qwen3.5 Apache-2.0✓ Verified — measured here? Not measured on this target
Qwen3.6 Apache-2.0✓ Verified — measured here? Not measured on this target
SmolLM2 Apache-2.0? Not measured on this target? Not measured on this target
AnimateDiff-Lightning CreativeML OpenRAIL-M? Not measured on this target? Not measured on this target
Wan 2.1 · Wan 2.2 Apache-2.0✓ Verified — measured here? Not measured on this target
FLUX.2 Apache License 2.0• Supported — engine path• Supported — engine path
SANA Apache-2.0○ Announced○ Announced
gpt-oss Apache-2.0○ Announced○ Announced

State is read from the model catalogue (status, regional verification, benchmark-publish permission) and from the measured figures above — never hand-set. Verified means veizik has a published, reproducible figure on that target; Supported means the engine path is implemented (or a figure is withheld by licence); Announced is on the roadmap and not downloadable. Each figure in the explorer also carries its own provenance: Measured on the offered build, Carried from a compute-identical build, or Development build.

Builds & downloads

Every build, and what was published on it.

Figures are frozen per build. Earlier builds stay listed, with a public reason for anything recorded but not published.

BuildReleasedStatePublishedRecorded, not published (public reason)
cuda-wan-1.2.932026-11-14Current0—
cuda-flux-1.0.52026-11-08Current0—
cuda-sd35-1.0.12026-11-08Current0—
cuda-wan-1.2.922026-10-31Superseded0—
cuda-wan-1.2.912026-10-31Withdrawn0—
cuda-qwen-1.1.42026-10-31Current0—
cuda-wan-1.2.82026-10-28Superseded0—
cuda-qwen-1.1.32026-10-28Withdrawn0—
cuda-qwen-1.1.22026-10-27Withdrawn0—
2026.10.02-r102026-10-10Current305—
2026.10.02-r6.52026-10-10Withdrawn0—
2026.10.02-r6.42026-10-10Withdrawn0—
2026.10.02-r6.32026-10-09Superseded0—
2026.10.02-r6.22026-10-09Withdrawn0—
2026.10.02-r6.12026-10-05Withdrawn0Measured on an earlier build 86
2026.10.02-r62026-10-05Withdrawn0—
2026.10.02-r52026-10-04Withdrawn0Measured on an earlier build 382 · Too few runs 36 · Spread too wide 36 · Method not met 26
2026.10.022026-10-02Superseded0Measured on an earlier build 131 · Spread too wide 41 · Method not met 80
2026.10.02-r32026-10-02Withdrawn0Measured on an earlier build 250 · Too few runs 19 · Spread too wide 27 · Method not met 1 · Pending re-measure 3
2026.10.02-r22026-10-02Superseded0—
2026.10.012026-10-01Superseded0Measured on an earlier build 138 · Too few runs 1 · Spread too wide 13

Downloads

FileBuildRowsBytessha256
metal-2026.10.02-r10.json2026.10.02-r10202139,856d88ca9c4a8e7444c…
metal-2026.10.02-r10.csv2026.10.02-r1020267,3430b1982689cf6cc2f…
cuda-qwen-1.1.4.jsoncuda-qwen-1.1.4210143,3594d81ac4acc67d6a7…
cuda-qwen-1.1.4.csvcuda-qwen-1.1.421067,019a908c4f4739eca29…
development.jsondevelopment289235,68937cbc5da3274d448…
development.csvdevelopment289121,5583f0c27aef5b2560f…

Schema veizik-benchmarks/1 v1.0.0 · MANIFEST.json · own numbers only; no other engine’s figure appears in any file.

Measurement changelog

What changed, by build.

2026-10-10Apple Silicon

Carried 156 figures to 2026.10.02-r10

Engine compute-identical to cuda-qwen-llm-1.0.0-be389a65 (kernel digest 649321acae4e…). Cross-build speed probe on 3 models, every |Δ| ≤ 3%: Qwen2.5-0.5B decode +0.63% / prefill +0.00% · Qwen2.5-7B decode -0.31% / prefill +1.88% · Qwen3-1.7B decode +0.51% / prefill -0.43%.

2026-10-10Apple Silicon

Measured the installed size on 2026.10.02-r10

The whole installation, summed from the installed tree.

Method & notation

How every figure is written.

How a figure is written

≥ 300.6 tok/s — median of 11 runs, range 299.0–302.5, 512 in / 128 out. “≥” marks a lower bound: the decode window also holds the last prompt step, so the true rate is slightly higher.

Spread

Range is min–max of the runs, never a bare “±”. Spread above 5% carries a wide spread chip; a single run says “single run, no spread”.

Workload

Prompt and generated tokens are on every figure. Prefill and decode are separate metrics and are never summed. Weights: the default mode of the published build.

Build-bound

Every figure names its build. “Carried” means measured on an earlier build whose engine is proven compute-identical; it is never shown as a fresh measurement.

Empty cells

Three distinct states, never a zero: not measured, measured, not published (with a public reason), and not published yet for a platform whose figures have not been taken yet.

Our own numbers only

No other engine’s figure, no ratio and no ranking appears on any page or in any download. Where we compared against another engine, this page names it with its version and settings, and the reproduction guide gives the recipe for both, so you can take both numbers yourself.

Reproduce

Take these numbers yourself.

Every published figure comes with the recipe to measure it again on your own machine, and, where we compared, the matching setup of the other engine. We publish only our own figures: your run produces the comparison.

Open the reproduction guide

Apple Silicon benchmark kit

Everything behind the Qwen2.5-0.5B, Qwen2.5-1.5B, Qwen2.5-7B, Qwen3-1.7B 190 cells on Apple M1 Ultra and Apple M2 Pro: the prompts, the benchmark script and what to read from its output. Each cell is a median of 11 runs of the shipped command.

  • The prompts are public domain. Project Gutenberg eBook #1228, public domain, cut to exactly the token count each cell names. The kit carries them with a SHA256SUMS, so you run the same bytes we did.
  • The shipped command, unmodified. Every row records the tokens actually generated, and a cell where generation stopped early shows that count rather than a forced one.
  • The machine is checked quiet before each run, not once at the start, and each row carries what the check saw. A row whose own readings show the machine was busy is held back rather than shown. The runs behind these cells recorded the machine idling at 0%, and the kit now refuses to start at all if it measures above 20%.
zsh
$ curl -fsSLO https://veizik.com/bench/veizik-bench-kit-pd1.tgz
$ shasum -a 256 veizik-bench-kit-pd1.tgz
# 7c5b91824f3328efb318b4c749efd6c50741ef166a1d17cec9219479f485cc8d
$ tar xzf veizik-bench-kit-pd1.tgz && cd repro_kit && cat README.md

279,402 bytes. The kit's own MANIFEST.sha256 covers every file inside it. Every published figure is reproducible with the public kit: the lines measuring other engines are not published here, and the reproduction guide gives the kit, the other engines' versions and their settings, so you can regenerate the whole comparison on your own machine.

Which harness produced which rows. Each published row records the digest of the kit it was measured with, and they are not all the same kit: it was revised during the campaign. Every kit a published row names can be downloaded here.

  • 7c5b91824f3328efb318b4c749efd6c50741ef166a1d17cec9219479f485cc8d — 238 rows, served here