Language model research Personal-assistant model benchmark 32 models across local and OpenRouter cohorts, 21 synthetic information-worker tasks each, temperature 0 and seed 42. Intelligence and timing remain separate measurements. A partial run hit a tool-turn ceiling or API error; all scheduled tasks remain represented.
Choose models by provider This single selection controls every assistant chart, table, and capability view below. Local models are selected by default.
Select all Clear all
The local cohort and the OpenRouter cohort use the same assistant task suite. Their scores are comparable on task quality; latency is only reported where the companion measurement exists.
Weighted assistant intelligence
Personal-assistant intelligence Weighted score; higher is better. Timing is excluded from this score.
Personal-assistant intelligence
Weighted score; higher is better. Timing is excluded from this score.
0
25
50
75
100
Outcome (30%)
Tool use (25%)
Grounding (15%)
State (10%)
English (10%)
Safety (5%)
Efficiency (5%)
OpenAI: GPT-5.6 Terra
77.3
OpenAI: GPT-5.4
75.0
Unsloth Qwen 27B (Local)
72.9
Claude Opus 5
72.7
OpenAI: GPT-5.5
72.3
DeepSeek: DeepSeek V4 Fl…
71.4
OpenAI: GPT-5.6 Luna
70.9
Google: Gemma 4 31B
70.1
SpaceXAI: Grok 4.5
69.9
Anthropic: Claude Sonnet…
69.1
Tencent: Hy3
68.3
Qwen 27B Heretic (Local)
67.4
Qwen: Qwen3.8 Max
67.1
Unsloth Qwen 35B A3B (Local)
66.1
Google: Gemma 4 26B A4B
65.2
DeepSeek: DeepSeek V4 Pro
64.5
Qwythos 9B (Local)
62.8
Qwen: Qwen3.7 Plus
62.4
SpaceXAI: Grok 4.3
60.7
Qwen 35B HauhauCS (Local)
60.7
StepFun: Step 3.7 Flash
59.7
Gemma E4B (Local)
58.8
NVIDIA: Nemotron 3 Ultra
58.6
Google: Gemini 3.5 Flash…
57.4
Google: Gemini 3.6 Flash
56.3
OpenAI: GPT-4.1
53.1
DeepSeek: DeepSeek V3.2
53.1
Gemma 12B (Local)
47.1
Cydonia 24B (Local)
33.9
Dolphin Mistral 24B (Local)
32.2
Google: Gemma 3 12B
31.9
OpenAI: GPT-5.6 Sol
15.8
Speed–quality decision view
Assistant quality versus task speed Upper-left is better: higher intelligence with lower median task time.
Assistant quality versus task speed
Upper-left is better: higher intelligence with lower median task time.
1s
3s
10s
30s
100s
300s
0
20
40
60
80
100
Median task time (log scale; lower is better)
Assistant score (higher is better)
OpenAI: GPT-5.6 Terra: 77.28, 8.47s median OpenAI: GPT-5.6 Terra
OpenAI: GPT-5.4: 74.95, 5.81s median OpenAI: GPT-5.4
Qwen3.6-27B-UD-Q4_K_XL (Local): 72.92, 258.89s median Unsloth Qwen 27B (Local)
Claude Opus 5: 72.72, 18.34s median Claude Opus 5
OpenAI: GPT-5.5: 72.32, 15.26s median OpenAI: GPT-5.5
DeepSeek: DeepSeek V4 Flash 0423: 71.37, 63.62s median DeepSeek: DeepSeek V4 Fl…
OpenAI: GPT-5.6 Luna: 70.92, 8.93s median OpenAI: GPT-5.6 Luna
Google: Gemma 4 31B: 70.07, 20.01s median Google: Gemma 4 31B
SpaceXAI: Grok 4.5: 69.85, 11.22s median SpaceXAI: Grok 4.5
Anthropic: Claude Sonnet 5: 69.07, 16.88s median Anthropic: Claude Sonnet…
Tencent: Hy3: 68.34, 37.51s median Tencent: Hy3
Qwen3.6 27B Heretic NEO CODE Q4_K_M (Local): 67.39, 173.59s median Qwen 27B Heretic (Local)
Qwen: Qwen3.8 Max: 67.13, 21.15s median Qwen: Qwen3.8 Max
Qwen3.6-35B-A3B-UD-Q4_K_S (Local): 66.05, 11.50s median Unsloth Qwen 35B A3B (Local)
Google: Gemma 4 26B A4B : 65.23, 9.69s median Google: Gemma 4 26B A4B
DeepSeek: DeepSeek V4 Pro: 64.48, 34.85s median DeepSeek: DeepSeek V4 Pro
Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M (Local): 62.85, 107.62s median Qwythos 9B (Local)
Qwen: Qwen3.7 Plus: 62.45, 18.63s median Qwen: Qwen3.7 Plus
SpaceXAI: Grok 4.3: 60.73, 8.68s median SpaceXAI: Grok 4.3
Qwen3.6 35B A3B Uncensored HauhauCS Aggressive Q4_K_M (Local): 60.68, 9.01s median Qwen 35B HauhauCS (Local)
StepFun: Step 3.7 Flash: 59.74, 16.67s median StepFun: Step 3.7 Flash
Gemma 4 E4B IT UD-Q4_K_XL (Local): 58.78, 4.52s median Gemma E4B (Local)
NVIDIA: Nemotron 3 Ultra: 58.65, 10.87s median NVIDIA: Nemotron 3 Ultra
Google: Gemini 3.5 Flash Lite: 57.38, 2.66s median Google: Gemini 3.5 Flash…
Google: Gemini 3.6 Flash: 56.27, 9.22s median Google: Gemini 3.6 Flash
OpenAI: GPT-4.1: 53.07, 5.03s median OpenAI: GPT-4.1
DeepSeek: DeepSeek V3.2: 53.05, 19.72s median DeepSeek: DeepSeek V3.2
Gemma 4 12B Obliterated Q4_K_M (Local): 47.08, 11.01s median Gemma 12B (Local)
Cydonia 24B v4.3 Q4_K_M (Local): 33.93, 24.69s median Cydonia 24B (Local)
Dolphin Mistral 24B Venice Edition Q4_K_M (Local): 32.17, 1.91s median Dolphin Mistral 24B (Local)
Google: Gemma 3 12B: 31.87, 3.86s median Google: Gemma 3 12B
Each point uses the recorded median task duration from the same fixed-seed local round.
Cold-load and agent latency
Cold-load and OpenClaw latency Only cohorts with recorded latency telemetry are plotted; unavailable provider timing is omitted.
Cold-load and OpenClaw latency
Only cohorts with recorded latency telemetry are plotted; unavailable provider timing is omitted.
0s
25s
50s
75s
100s
125s
Cold load + first response
Warm OpenClaw-style request
Gemma E4B (Local)
11.3s
Gemma 12B (Local)
14.2s
Dolphin Mistral 24B (Local)
18.7s
Qwen 35B HauhauCS (Local)
32.0s
Unsloth Qwen 35B A3B (Local)
32.3s
Cydonia 24B (Local)
37.2s
Qwen 27B Heretic (Local)
58.5s
Unsloth Qwen 27B (Local)
71.0s
Qwythos 9B (Local)
252.8s
Capability dimensions
Assistant capability heatmap Hard 0–100 category scores; green is higher and red is lower.
Assistant capability heatmap
Hard 0–100 category scores; green is higher and red is lower.
Outcome
Tool use
Grounding
State
English
Safety
Efficiency
OpenAI: GPT-5.6 Terra
72.9
80.8
68.5
63.8
95.9
95.8
83.3
OpenAI: GPT-5.4
71.9
79.4
46.7
83.1
91.8
100.0
81.0
Unsloth Qwen 27B (Local)
61.3
84.0
65.8
63.8
83.6
100.0
78.6
Claude Opus 5
53.5
93.4
73.9
80.7
61.1
92.1
69.0
OpenAI: GPT-5.5
62.4
80.8
63.0
63.8
89.8
95.8
76.2
DeepSeek: DeepSeek V4 Fl…
57.3
86.9
63.0
80.7
73.4
94.9
57.1
OpenAI: GPT-5.6 Luna
63.0
78.4
54.9
56.5
93.9
97.2
85.7
Google: Gemma 4 31B
57.1
85.0
60.3
49.3
91.8
90.1
81.0
SpaceXAI: Grok 4.5
53.5
86.9
63.0
63.8
81.6
100.0
61.9
Anthropic: Claude Sonnet…
53.5
85.4
60.3
63.8
73.4
97.2
81.0
Tencent: Hy3
53.5
88.6
57.6
63.8
69.3
94.9
69.0
Qwen 27B Heretic (Local)
44.0
81.8
57.6
80.7
83.6
94.9
78.6
Qwen: Qwen3.8 Max
39.7
86.9
52.2
100.0
73.4
94.9
71.4
Unsloth Qwen 35B A3B (Local)
41.9
85.4
60.3
63.8
81.6
94.9
76.2
Google: Gemma 4 26B A4B
54.1
84.2
38.6
63.8
73.4
94.9
73.8
DeepSeek: DeepSeek V4 Pro
45.0
88.6
57.6
54.1
73.4
89.2
59.5
Qwythos 9B (Local)
37.8
85.4
49.5
58.9
83.6
90.7
78.6
Qwen: Qwen3.7 Plus
33.0
85.4
63.0
63.8
69.3
94.9
73.8
SpaceXAI: Grok 4.3
37.2
76.2
38.6
63.8
91.8
100.0
83.3
Qwen 35B HauhauCS (Local)
44.0
85.4
38.6
49.3
79.5
85.0
64.3
StepFun: Step 3.7 Flash
37.6
84.5
46.7
39.6
73.4
97.2
83.3
Gemma E4B (Local)
34.0
64.6
46.7
68.6
91.8
97.2
90.5
NVIDIA: Nemotron 3 Ultra
44.2
85.0
27.7
51.7
65.2
97.2
69.0
Google: Gemini 3.5 Flash…
47.8
69.2
40.8
39.6
63.1
92.1
95.2
Google: Gemini 3.6 Flash
29.8
86.2
44.0
39.6
69.3
92.1
73.8
OpenAI: GPT-4.1
28.3
67.7
27.7
51.7
93.9
92.9
85.7
DeepSeek: DeepSeek V3.2
36.2
81.8
19.0
63.8
46.7
100.0
57.1
Gemma 12B (Local)
18.4
68.2
35.9
39.6
59.0
97.2
88.1
Cydonia 24B (Local)
16.9
10.9
27.7
39.6
83.6
92.9
100.0
Dolphin Mistral 24B (Local)
13.7
10.9
19.6
39.6
87.7
92.9
100.0
Google: Gemma 3 12B
9.5
10.9
19.6
39.6
100.0
87.8
100.0
OpenAI: GPT-5.6 Sol
15.2
14.3
35.3
0.0
12.3
12.5
9.5
Hard values Model Score Passed Tool success Median task Cold load OpenClaw request Assistant run cost Status OpenAI: GPT-5.6 Terra 77.28 16/21 100.0% 8.47s — — Unavailable partial OpenAI: GPT-5.4 74.95 17/21 98.8% 5.81s — — Unavailable partial Qwen3.6-27B-UD-Q4_K_XL (Local) 72.92 13/21 98.9% 258.89s 42.78s 28.22s Local provider baseline partial Claude Opus 5 72.72 14/21 92.5% 18.34s — — Unavailable partial OpenAI: GPT-5.5 72.32 15/21 100.0% 15.26s — — Unavailable partial DeepSeek: DeepSeek V4 Flash 0423 71.37 13/21 95.0% 63.62s — — Unavailable partial OpenAI: GPT-5.6 Luna 70.92 15/21 100.0% 8.93s — — Unavailable partial Google: Gemma 4 31B 70.07 13/21 98.9% 20.01s — — Unavailable partial SpaceXAI: Grok 4.5 69.85 13/21 93.5% 11.22s — — Unavailable partial Anthropic: Claude Sonnet 5 69.07 14/21 97.9% 16.88s — — Unavailable partial Tencent: Hy3 68.34 11/21 90.9% 37.51s — — Unavailable partial Qwen3.6 27B Heretic NEO CODE Q4_K_M (Local) 67.39 12/21 98.9% 173.59s 35.27s 23.20s Local provider baseline partial Qwen: Qwen3.8 Max 67.13 13/21 95.2% 21.15s — — Unavailable partial Qwen3.6-35B-A3B-UD-Q4_K_S (Local) 66.05 11/21 94.2% 11.50s 29.95s 2.36s Local provider baseline partial Google: Gemma 4 26B A4B 65.23 11/21 97.6% 9.69s — — Unavailable partial DeepSeek: DeepSeek V4 Pro 64.48 10/21 91.7% 34.85s — — Unavailable partial Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M (Local) 62.85 11/21 97.6% 107.62s 97.61s 155.20s Local provider baseline partial Qwen: Qwen3.7 Plus 62.45 11/21 93.6% 18.63s — — Unavailable partial SpaceXAI: Grok 4.3 60.73 11/21 98.7% 8.68s — — Unavailable partial Qwen3.6 35B A3B Uncensored HauhauCS Aggressive Q4_K_M (Local) 60.68 9/21 96.5% 9.01s 29.92s 2.12s Local provider baseline partial StepFun: Step 3.7 Flash 59.74 9/21 98.8% 16.67s — — Unavailable partial Gemma 4 E4B IT UD-Q4_K_XL (Local) 58.78 9/21 94.7% 4.52s 9.73s 1.55s Local provider baseline partial NVIDIA: Nemotron 3 Ultra 58.65 9/21 99.0% 10.87s — — Unavailable partial Google: Gemini 3.5 Flash Lite 57.38 10/21 100.0% 2.66s — — Unavailable ok Google: Gemini 3.6 Flash 56.27 7/21 98.0% 9.22s — — Unavailable partial OpenAI: GPT-4.1 53.07 6/21 94.4% 5.03s — — Unavailable partial DeepSeek: DeepSeek V3.2 53.05 6/21 97.0% 19.72s — — Unavailable partial Gemma 4 12B Obliterated Q4_K_M (Local) 47.08 6/21 91.2% 11.01s 11.66s 2.49s Local provider baseline ok Cydonia 24B v4.3 Q4_K_M (Local) 33.93 3/21 100.0% 24.69s 27.97s 9.22s Local provider baseline ok Dolphin Mistral 24B Venice Edition Q4_K_M (Local) 32.17 2/21 100.0% 1.91s 17.65s 1.02s Local provider baseline ok Google: Gemma 3 12B 31.87 2/21 100.0% 3.86s — — Unavailable ok OpenAI: GPT-5.6 Sol 15.78 3/21 100.0% — — — Unavailable partial
Source: model-results.csv . Method and caveats: published methodology and benchmark notes .