LLM SQL Generation Benchmark Results

We assessed the ability of popular LLMs to generate accurate and efficient SQL from natural language prompts. Using a 200 million record dataset from the GH Archive uploaded to Tinybird, we asked the LLMs to generate SQL based on 50 prompts. The results are shown below and can be compared to a human baseline.

--
human
human
--
--
--
--
--
332.6 ms
31,006,852
759.83 MB
#1
anthropic
82.55
99.61
65.48
4.738
1.00
2,800.88 ms
34,437,823
811.31 MB
#2
openai
82.47
99.90
65.05
1.793
1.00
818.72 ms
39,159,059
941.02 MB
#3
anthropic
82.46
99.86
65.05
7.349
1.14
633.94 ms
38,050,598
949.38 MB
#4
~x-ai
81.91
99.80
64.02
13.747
1.24
2,811 ms
45,211,063
418.09 MB
#5
google
80.04
99.78
60.30
49.440
1.02
682.04 ms
46,150,625
1,055.25 MB
#6
qwen
79.90
99.63
60.17
67.609
1.00
1,201 ms
46,773,375
984.67 MB
#7
google
79.87
99.69
60.05
9.736
1.00
1,484.88 ms
50,048,646
1,060.75 MB
#8
anthropic
79.54
99.54
59.54
3.100
1.02
2,951.48 ms
36,182,829
938.35 MB
#9
moonshotai
79.40
99.75
59.06
4.547
1.02
1,397.327 ms
46,877,411
848.80 MB
#10
~anthropic
79.39
99.91
58.87
7.406
1.02
2,658.08 ms
35,558,485
301.20 MB
#11
x-ai
79.37
99.02
59.71
31.499
1.04
3,613.62 ms
37,203,325
921.56 MB
#12
google
79.33
99.73
58.94
21.273
1.00
825.714 ms
41,502,252
947.30 MB
#13
google
79.26
99.03
59.48
20.360
1.02
3,833.3 ms
49,317,675
1,056.97 MB
#14
google
79.16
98.84
59.48
7.875
1.00
5,507.52 ms
41,838,459
1,045.51 MB
#15
google
78.95
99.91
57.98
19.100
1.06
443.22 ms
42,878,115
826.88 MB
#16
anthropic
78.85
99.93
57.77
2.780
1.00
585.18 ms
34,526,743
811.92 MB
#17
google
78.76
99.18
58.35
2.406
1.00
6,401.3 ms
42,874,619
927.15 MB
#18
z-ai
78.72
99.82
57.62
50.516
1.06
2,970.88 ms
39,150,641
318.93 MB
#19
openai
78.59
99.97
57.21
3.757
1.40
525.58 ms
41,771,932
394.36 MB
#20
meta
78.57
99.92
57.22
6.670
1.08
2,934.5 ms
34,692,912
254.72 MB
#21
~google
78.56
99.31
57.81
11.087
1.04
3,468.7 ms
40,340,295
914.03 MB
#22
bytedance-seed
78.37
99.39
57.35
11.170
1.06
2,954.24 ms
46,000,305
964.12 MB
#23
openrouter
78.32
99.85
56.78
1.401
1.02
1,202.46 ms
72,041,510
1,141.62 MB
#24
~google
78.28
99.46
57.10
9.730
1.04
3,234.08 ms
32,730,132
781.68 MB
#25
anthropic
78.27
99.60
56.93
40.858
1.00
1,425.82 ms
40,046,627
947.37 MB
#26
z-ai
78.22
99.48
56.95
16.908
1.00
2,150.16 ms
45,225,460
1,036.61 MB
#27
anthropic
78.01
99.93
56.08
3.149
1.10
374.224 ms
40,099,998
824.57 MB
#28
z-ai
77.96
99.70
56.21
32.466
1.02
968.06 ms
45,783,003
1,078.88 MB
#29
deepseek
77.91
99.91
55.91
20.368
1.02
2,224.64 ms
42,412,877
299.77 MB
#30
qwen
77.85
99.73
55.97
20.423
1.12
1,065.78 ms
50,827,717
997.11 MB
#31
moonshotai
77.84
99.92
55.77
4.265
1.06
589.22 ms
49,539,148
903.11 MB
#32
x-ai
77.78
99.75
55.81
61.602
1.00
677.06 ms
49,360,869
1,145.95 MB
#33
openai
77.73
99.53
55.92
9.575
1.02
2,624.06 ms
45,682,344
923.23 MB
#34
x-ai
77.53
99.92
55.14
0.896
1.02
794.26 ms
35,500,350
846.01 MB
#35
google
77.48
99.38
55.57
5.771
1.14
4,475.24 ms
37,623,293
736.10 MB
#36
qwen
77.47
99.71
55.23
78.487
1.02
809.92 ms
45,777,471
1,046.79 MB
#37
meta
77.46
99.51
55.42
5.558
1.00
2,944.1 ms
44,070,185
969.39 MB
#38
x-ai
77.45
99.79
55.12
16.323
1.06
998.54 ms
42,709,671
910.85 MB
#39
x-ai
77.43
99.49
55.37
17.513
1.00
2,168.32 ms
44,026,726
1,009.68 MB
#40
z-ai
77.38
99.84
54.92
32.316
1.02
2,991.12 ms
38,051,000
315.34 MB
#41
openai
77.38
99.92
54.83
9.886
1.14
448.84 ms
49,432,133
844.29 MB
#42
qwen
77.34
99.80
54.89
72.053
1.04
2,453.62 ms
38,011,304
365.27 MB
#43
google
77.34
98.43
56.25
20.550
1.00
2,995.771 ms
51,942,301
1,111.37 MB
#44
openai
77.32
99.69
54.95
3.188
1.00
1,898.78 ms
57,024,811
1,137.19 MB
#45
deepseek
77.29
99.82
54.77
4.786
1.00
1,371.82 ms
42,265,330
838.05 MB
#46
xiaomi
77.20
99.67
54.74
23.199
1.06
1,395.92 ms
43,289,653
927.15 MB
#47
moonshotai
77.16
99.77
54.54
8.252
1.00
1,468.84 ms
46,652,241
877.99 MB
#48
openai
77.11
99.85
54.36
19.254
1.04
625.82 ms
43,602,960
960.85 MB
#49
mistralai
77.10
99.71
54.50
5.209
1.02
1,331.286 ms
52,340,224
974.02 MB
#50
anthropic
77.07
99.88
54.26
2.907
1.02
4,677.22 ms
42,411,762
294.93 MB
#51
thinkingmachines
77.07
99.82
54.31
6.680
1.06
4,048.72 ms
46,538,689
375.01 MB
#52
moonshotai
77.06
99.46
54.67
68.072
1.00
1,175.633 ms
44,153,829
997.26 MB
#53
qwen
76.97
99.69
54.26
50.610
1.00
1,024.12 ms
40,394,423
964.23 MB
#54
qwen
76.97
99.85
54.09
14.378
1.10
3,435.22 ms
39,311,235
310.13 MB
#55
sakana
76.96
99.63
54.30
58.429
1.06
1,059.8 ms
38,309,339
990.27 MB
#56
qwen
76.95
99.50
54.40
35.404
1.00
1,217.265 ms
42,908,205
1,021.26 MB
#57
deepseek
76.93
99.71
54.16
61.820
1.02
682.673 ms
42,768,724
937.48 MB
#58
nex-agi
76.89
99.48
54.29
30.772
1.10
1,749.78 ms
54,863,792
1,079.00 MB
#59
openai
76.88
99.67
54.09
10.389
1.00
1,022.388 ms
53,083,092
1,134.93 MB
#60
anthropic
76.88
98.42
55.35
5.410
1.02
6,141.408 ms
36,399,824
954.63 MB
#61
~z-ai
76.86
99.81
53.91
24.979
1.04
3,387.44 ms
42,864,145
353.49 MB
#62
anthropic
76.80
99.30
54.30
2.799
1.02
3,187.48 ms
39,847,072
984.97 MB
#63
qwen
76.79
99.70
53.88
51.377
1.00
1,144.52 ms
45,274,360
904.00 MB
#64
openai
76.76
99.87
53.65
9.049
1.02
639.54 ms
41,901,897
971.73 MB
#65
qwen
76.71
99.38
54.05
2.089
1.00
6,131.14 ms
38,332,239
776.42 MB
#66
deepseek
76.64
99.77
53.51
6.753
1.08
1,397.02 ms
46,005,139
899.56 MB
#67
qwen
76.61
99.58
53.64
93.889
1.04
1,244.28 ms
42,483,080
932.90 MB
#68
kwaipilot
76.57
99.44
53.71
2.012
1.00
5,157.86 ms
51,659,367
909.16 MB
#69
deepcogito
76.54
99.57
53.50
1.165
1.00
4,554.34 ms
36,881,329
806.31 MB
#70
stepfun
76.53
99.04
54.03
35.870
1.06
1,416.312 ms
46,041,589
1,111.22 MB
#71
qwen
76.51
99.61
53.41
5.172
1.17
550.043 ms
40,419,858
895.38 MB
#72
openai
76.50
99.77
53.23
5.045
1.00
1,374.06 ms
47,296,114
1,000.48 MB
#73
tencent
76.42
99.43
53.41
34.712
1.20
1,251.122 ms
45,276,609
982.15 MB
#74
anthropic
76.40
99.83
52.97
7.003
1.00
3,823.7 ms
41,901,088
384.21 MB
#75
openai
76.38
98.74
54.01
6.711
1.02
6,556.7 ms
45,940,212
1,029.91 MB
#76
qwen
76.36
99.77
52.95
37.553
1.06
761.347 ms
44,676,197
795.72 MB
#77
qwen
76.34
99.54
53.14
23.787
1.00
1,869.66 ms
38,823,710
936.95 MB
#78
deepseek
76.25
98.35
54.16
44.340
1.12
3,905.245 ms
49,847,738
974.66 MB
#79
openai
76.20
98.94
53.47
9.382
1.00
5,260.18 ms
39,558,560
969.10 MB
#80
anthropic
76.19
99.96
52.41
3.234
1.02
388.96 ms
37,145,042
684.44 MB
#81
openai
76.18
99.78
52.59
76.620
1.04
746.8 ms
52,804,037
936.55 MB
#82
z-ai
76.17
99.86
52.49
56.839
1.14
1,280.245 ms
44,533,261
368.54 MB
#83
perceptron
76.14
99.47
52.81
4.372
1.02
3,587.02 ms
66,915,860
1,055.43 MB
#84
qwen
76.11
99.90
52.31
8.620
1.18
397.755 ms
38,751,330
781.80 MB
#85
anthropic
76.09
99.93
52.25
3.423
1.02
2,997 ms
37,271,990
261.77 MB
#86
qwen
76.07
99.84
52.30
54.757
1.16
2,508.6 ms
37,320,037
295.97 MB
#87
z-ai
76.05
99.62
52.48
12.758
1.02
1,676.94 ms
45,577,866
1,035.83 MB
#88
kwaipilot
75.94
99.03
52.85
4.666
1.08
6,085.74 ms
43,355,598
915.92 MB
#89
qwen
75.87
99.87
51.86
13.982
1.02
2,741.04 ms
50,261,089
362.24 MB
#90
openai
75.86
99.85
51.87
1.534
1.02
1,386.16 ms
60,323,408
988.77 MB
#91
openai
75.79
99.89
51.69
10.228
1.08
613.66 ms
52,581,751
940.75 MB
#92
deepseek
75.76
99.91
51.62
3.690
1.14
729.48 ms
42,566,756
789.60 MB
#93
anthropic
75.75
99.85
51.65
6.342
1.04
580.51 ms
39,294,543
936.76 MB
#94
qwen
75.75
99.69
51.81
47.981
1.02
1,044.3 ms
46,539,910
999.42 MB
#95
qwen
75.73
99.98
51.48
4.571
1.04
457.224 ms
46,666,126
333.42 MB
#96
qwen
75.68
99.97
51.39
2.303
1.02
679.163 ms
53,829,416
387.08 MB
#97
openai
75.68
99.76
51.60
5.261
1.08
1,351.14 ms
51,060,147
1,021.13 MB
#98
openrouter
75.67
99.84
51.51
2.754
1.00
1,160.78 ms
71,383,218
1,133.85 MB
#99
poolside
75.66
99.75
51.56
7.356
1.14
1,255.76 ms
46,603,715
962.52 MB
#100
openrouter
75.62
99.83
51.40
1.362
1.02
1,358.24 ms
67,797,316
1,137.85 MB
#101
qwen
75.59
99.79
51.40
29.091
1.00
782.62 ms
52,951,289
1,073.90 MB
#102
z-ai
75.58
99.40
51.77
18.891
1.00
2,731.18 ms
50,021,624
994.13 MB
#103
qwen
75.56
99.48
51.65
13.946
1.06
2,107 ms
45,772,163
1,056.49 MB
#104
openai
75.54
99.79
51.29
1.375
1.02
1,946.04 ms
49,630,116
923.17 MB
#105
qwen
75.52
99.78
51.26
1.292
1.02
1,383.265 ms
46,171,856
959.85 MB
#106
anthracite-org
75.51
98.86
52.16
3.023
1.12
5,948.776 ms
38,123,778
821.08 MB
#107
moonshotai
75.45
99.63
51.28
24.452
1.00
1,374.34 ms
36,653,795
986.71 MB
#108
deepseek
75.44
99.89
50.99
27.582
1.06
3,025.64 ms
33,901,580
241.42 MB
#109
openai
75.44
99.86
51.01
11.124
1.00
596.52 ms
48,389,329
1,097.99 MB
#110
microsoft
75.42
96.90
53.93
6.842
1.47
9,964.571 ms
35,931,491
812.51 MB
#111
aion-labs
75.41
98.89
51.92
24.331
1.00
4,130.38 ms
57,308,234
1,133.15 MB
#112
anthropic
75.36
99.91
50.82
2.813
1.02
2,053.426 ms
34,047,552
206.91 MB
#113
nvidia
75.30
99.44
51.17
22.996
1.06
2,175.16 ms
45,879,249
994.61 MB
#114
google
75.29
99.78
50.80
1.556
1.02
1,936.7 ms
57,137,045
1,002.74 MB
#115
google
75.29
99.19
51.39
1.517
1.00
7,438.06 ms
53,824,558
965.34 MB
#116
aion-labs
75.26
99.70
50.81
59.689
1.06
847.02 ms
45,570,498
1,048.70 MB
#117
openai
75.22
99.94
50.51
2.955
1.00
442.98 ms
41,636,677
756.28 MB
#118
google
75.20
99.76
50.63
5.060
1.06
6,086.8 ms
56,772,354
393.05 MB
#119
x-ai
75.19
99.87
50.52
7.127
1.06
651.74 ms
55,296,404
869.75 MB
#120
openai
75.18
99.60
50.77
1.503
1.02
3,542.34 ms
42,878,575
910.97 MB
#121
~openai
75.18
99.94
50.43
1.540
1.06
2,515.34 ms
37,502,809
303.93 MB
#122
mistralai
75.17
99.85
50.50
2.621
1.06
1,349.88 ms
54,701,026
878.39 MB
#123
~moonshotai
75.16
99.15
51.18
19.888
1.02
3,525.92 ms
40,731,360
961.27 MB
#124
openai
75.06
99.86
50.26
21.133
1.04
702.64 ms
68,364,075
1,005.01 MB
#125
openai
75.03
99.99
50.08
2.074
1.00
421.6 ms
52,027,773
246.69 MB
#126
qwen
75.03
99.85
50.22
43.359
1.18
2,708.94 ms
30,035,905
260.80 MB
#127
qwen
75.01
99.73
50.28
45.733
1.04
846.5 ms
49,315,858
1,050.31 MB
#128
tencent
75.01
98.46
51.55
15.523
1.04
6,150.5 ms
49,211,334
1,078.85 MB
#129
openai
75.01
99.77
50.24
0.876
1.06
1,866.388 ms
55,625,851
880.60 MB
#130
qwen
74.98
99.95
50.00
1.474
1.00
421.14 ms
38,561,447
879.98 MB
#131
x-ai
74.95
99.76
50.15
10.611
1.00
762.122 ms
50,156,062
1,088.51 MB
#132
deepseek
74.95
99.79
50.11
3.681
1.04
1,430.3 ms
43,504,553
917.37 MB
#133
openai
74.95
99.89
50.00
2.190
1.06
818.38 ms
54,736,481
995.56 MB
#134
bytedance-seed
74.91
99.65
50.18
26.392
1.04
1,477.74 ms
46,878,338
944.20 MB
#135
openai
74.91
99.98
49.84
16.292
1.04
549.54 ms
53,315,039
303.04 MB
#136
anthropic
74.88
99.92
49.84
3.702
1.02
585.62 ms
43,365,288
907.62 MB
#137
openai
74.86
99.80
49.93
1.158
1.00
1,920.74 ms
61,160,789
1,027.66 MB
#138
qwen
74.85
99.75
49.96
17.344
1.02
720.837 ms
54,897,195
1,106.61 MB
#139
qwen
74.85
98.95
50.75
3.224
1.08
8,653.66 ms
44,233,987
815.52 MB
#140
google
74.83
98.84
50.82
5.276
1.00
7,081.24 ms
56,772,354
1,050.70 MB
#141
xiaomi
74.83
99.49
50.17
11.241
1.00
2,737.28 ms
47,678,693
957.11 MB
#142
openai
74.83
99.05
50.61
18.564
1.04
4,009.1 ms
45,921,518
989.20 MB
#143
z-ai
74.81
99.44
50.18
55.855
1.04
1,656.82 ms
50,110,333
1,071.67 MB
#144
qwen
74.80
99.67
49.93
31.219
1.04
1,253.26 ms
47,306,065
990.21 MB
#145
nex-agi
74.76
99.61
49.92
57.277
1.02
819.755 ms
40,479,420
1,012.16 MB
#146
openai
74.75
99.68
49.82
14.122
1.27
610.104 ms
46,853,009
965.28 MB
#147
perplexity
74.72
99.63
49.82
46.034
1.36
862.54 ms
53,631,153
1,119.54 MB
#148
nvidia
74.72
99.69
49.75
24.582
1.08
1,274.32 ms
44,080,993
919.44 MB
#149
nousresearch
74.68
99.92
49.45
7.260
1.12
552.64 ms
52,968,329
862.73 MB
#150
qwen
74.63
99.91
49.34
36.262
1.04
439.38 ms
45,468,824
791.67 MB
#151
openai
74.57
99.71
49.42
4.156
1.02
1,685.42 ms
55,308,771
1,106.43 MB
#152
deepseek
74.53
99.96
49.10
5.366
1.24
362.62 ms
39,914,537
612.03 MB
#153
anthropic
74.53
99.23
49.83
3.549
1.00
5,336.72 ms
46,682,298
961.64 MB
#154
openai
74.45
99.95
48.95
8.630
1.06
1,214.36 ms
47,908,602
340.50 MB
#155
writer
74.44
99.58
49.30
9.108
1.18
2,368.32 ms
38,942,989
791.79 MB
#156
perplexity
74.44
99.75
49.13
8.937
1.02
1,131.3 ms
54,592,943
1,142.75 MB
#157
google
74.42
99.85
49.00
1.281
1.00
1,465.88 ms
53,680,420
964.11 MB
#158
qwen
74.38
99.04
49.72
58.591
1.06
2,758.8 ms
45,622,670
1,037.56 MB
#159
mistralai
74.37
99.93
48.81
2.088
1.04
666.02 ms
53,051,447
878.95 MB
#160
x-ai
74.36
99.83
48.89
17.702
1.00
686.06 ms
44,422,695
1,033.66 MB
#161
minimax
74.34
99.51
49.16
25.143
1.18
1,680.46 ms
47,270,248
996.89 MB
#162
deepseek
74.27
99.74
48.79
62.020
1.02
869.52 ms
50,574,856
983.35 MB
#163
openai
74.25
98.55
49.94
3.996
1.04
6,177.531 ms
51,609,499
1,045.21 MB
#164
deepcogito
74.21
99.90
48.51
2.851
1.10
847.14 ms
42,613,427
792.70 MB
#165
qwen
74.19
99.90
48.49
2.456
1.08
732.878 ms
46,841,414
767.00 MB
#166
z-ai
74.17
99.60
48.73
27.030
1.08
934.408 ms
50,746,043
1,101.56 MB
#167
aion-labs
74.14
99.68
48.60
74.917
1.04
885.66 ms
48,400,008
1,052.70 MB
#168
meta-llama
74.12
99.95
48.30
3.095
1.04
410.78 ms
40,161,866
793.26 MB
#169
google
74.10
99.89
48.32
20.782
1.04
579.36 ms
38,815,820
806.77 MB
#170
deepseek
74.08
99.95
48.22
3.914
1.04
608.681 ms
45,651,463
322.46 MB
#171
perplexity
74.07
99.89
48.26
2.611
1.06
1,000.28 ms
37,440,965
778.30 MB
#172
openai
74.06
99.60
48.51
10.003
1.02
1,443.469 ms
54,867,793
1,033.70 MB
#173
qwen
74.03
99.10
48.96
1.642
1.06
8,332.72 ms
40,598,612
833.41 MB
#174
openai
73.98
99.86
48.10
1.366
1.04
1,364.88 ms
49,226,881
913.08 MB
#175
poolside
73.94
99.98
47.91
16.848
1.12
788.96 ms
38,618,194
228.47 MB
#176
qwen
73.93
99.79
48.08
7.308
1.06
4,844.7 ms
46,407,274
370.56 MB
#177
nousresearch
73.93
98.20
49.66
3.680
1.02
11,993.694 ms
43,291,775
752.60 MB
#178
x-ai
73.93
98.89
48.97
7.519
1.00
6,369.62 ms
57,388,979
1,035.02 MB
#179
openai
73.92
99.75
48.08
1.101
1.04
1,539.143 ms
47,348,539
989.91 MB
#180
inclusionai
73.90
99.21
48.59
8.477
1.00
3,858 ms
51,586,642
1,089.58 MB
#181
google
73.88
99.80
47.95
1.286
1.02
1,285.714 ms
50,589,629
969.49 MB
#182
mistralai
73.85
98.88
48.82
2.728
1.09
1,068.091 ms
41,646,956
860.77 MB
#183
poolside
73.84
99.98
47.69
5.713
1.10
719.58 ms
42,657,871
296.97 MB
#184
openai
73.76
99.01
48.52
1.926
1.06
4,263.938 ms
53,931,690
930.29 MB
#185
arcee-ai
73.72
99.53
47.90
8.240
1.20
2,119.44 ms
48,245,941
996.87 MB
#186
anthropic
73.67
99.17
48.18
3.491
1.00
6,180.14 ms
43,114,181
899.14 MB
#187
nousresearch
73.65
98.26
49.04
2.094
1.02
9,009.143 ms
47,704,275
1,008.86 MB
#188
cohere
73.65
99.01
48.29
1.612
1.04
6,164.959 ms
44,226,012
899.85 MB
#189
mistralai
73.59
99.96
47.22
0.855
1.00
775.14 ms
42,657,411
620.15 MB
#190
qwen
73.59
99.88
47.30
19.002
1.02
602.48 ms
45,928,106
890.50 MB
#191
z-ai
73.54
99.17
47.92
10.920
1.14
4,103.58 ms
44,778,371
908.40 MB
#192
z-ai
73.53
99.73
47.33
13.738
1.10
5,415.82 ms
52,774,370
366.60 MB
#193
openai
73.51
99.66
47.36
9.993
1.02
1,506.08 ms
53,506,834
1,119.95 MB
#194
openrouter
73.50
98.15
48.85
82.452
1.22
673.4 ms
37,498,477
1,044.59 MB
#195
mistralai
73.48
99.40
47.56
2.237
1.04
6,024.58 ms
44,102,484
778.48 MB
#196
deepseek
73.47
98.77
48.17
4.662
1.04
6,194.184 ms
46,010,640
876.42 MB
#197
~anthropic
73.40
99.96
46.84
1.905
1.10
2,559.22 ms
39,114,591
219.54 MB
#198
openai
73.37
99.73
47.00
4.821
1.04
1,703 ms
59,148,959
1,016.79 MB
#199
nvidia
73.33
99.31
47.35
75.122
1.10
1,321.306 ms
42,687,966
986.25 MB
#200
openrouter
73.27
99.56
46.97
17.596
1.04
1,977.66 ms
47,054,403
968.46 MB
#201
openai
73.25
99.70
46.80
1.300
1.02
1,981.02 ms
36,763,125
1,070.25 MB
#202
qwen
73.25
99.93
46.56
2.453
1.02
556.74 ms
42,185,121
868.81 MB
#203
openai
73.22
99.75
46.68
3.205
1.09
533.957 ms
41,234,766
980.50 MB
#204
x-ai
73.15
99.84
46.46
6.570
1.02
830.98 ms
51,488,460
1,077.86 MB
#205
z-ai
73.13
99.59
46.68
10.752
1.04
838.771 ms
48,329,302
1,130.98 MB
#206
~deepseek
73.08
98.54
47.61
29.448
1.14
3,637.286 ms
49,408,498
991.80 MB
#207
baidu
73.08
99.02
47.13
4.773
1.22
5,744.94 ms
40,362,238
856.35 MB
#208
inception
73.06
99.24
46.88
0.966
1.02
6,492.7 ms
49,413,821
1,050.30 MB
#209
openai
73.04
99.02
47.07
9.276
1.04
4,358.28 ms
40,287,810
1,027.15 MB
#210
google
73.04
99.99
46.09
2.126
1.02
337.4 ms
36,295,667
262.45 MB
#211
perplexity
73.04
99.89
46.19
2.310
1.06
902.08 ms
37,171,993
846.08 MB
#212
moonshotai
73.01
99.50
46.51
30.577
1.00
2,093.58 ms
37,188,519
875.48 MB
#213
anthropic
72.96
99.99
45.93
2.731
1.08
522.38 ms
47,370,988
297.58 MB
#214
poolside
72.95
98.74
47.15
8.874
1.10
4,107.306 ms
45,822,565
1,011.69 MB
#215
inclusionai
72.91
99.90
45.92
5.361
1.02
2,885.84 ms
37,442,248
327.69 MB
#216
minimax
72.90
98.96
46.83
6.706
1.14
5,725.68 ms
51,112,554
952.89 MB
#217
qwen
72.84
99.69
45.99
8.095
1.20
1,400.5 ms
49,184,648
1,009.49 MB
#218
thedrummer
72.80
99.93
45.68
1.966
1.10
412.306 ms
36,265,794
823.24 MB
#219
google
72.77
99.85
45.70
5.021
1.06
694.878 ms
41,212,019
869.17 MB
#220
deepseek
72.77
99.52
46.01
5.875
1.11
383.682 ms
38,010,973
813.72 MB
#221
qwen
72.75
99.47
46.04
1.800
1.02
4,211.94 ms
41,619,269
938.67 MB
#222
qwen
72.71
99.15
46.26
30.579
1.12
1,997.531 ms
53,919,075
1,064.75 MB
#223
bytedance-seed
72.59
99.84
45.34
8.785
1.06
2,173 ms
44,485,951
312.69 MB
#224
minimax
72.58
99.71
45.45
8.115
1.04
3,911.204 ms
46,994,209
415.62 MB
#225
qwen
72.56
98.88
46.24
2.308
1.06
6,441.857 ms
35,227,076
816.64 MB
#226
poolside
72.50
99.75
45.25
20.571
1.28
730.26 ms
38,514,356
1,011.07 MB
#227
meta-llama
72.48
100.00
44.96
2.048
1.04
289.875 ms
39,101,618
134.66 MB
#228
mistralai
72.45
99.24
45.66
2.166
1.06
6,698.14 ms
44,059,393
840.20 MB
#229
openai
72.44
99.78
45.10
1.945
1.00
919.562 ms
44,180,776
913.74 MB
#230
bytedance-seed
72.43
98.70
46.16
22.165
1.10
2,919.347 ms
54,494,944
1,182.43 MB
#231
qwen
72.42
99.60
45.23
31.462
1.04
1,591.3 ms
53,444,620
993.79 MB
#232
openai
72.41
99.99
44.83
2.145
1.04
690.28 ms
54,131,214
193.58 MB
#233
meituan
72.38
99.62
45.15
26.224
1.04
964.837 ms
46,725,377
1,047.87 MB
#234
qwen
72.37
99.47
45.26
18.700
1.06
2,507.64 ms
44,319,995
887.56 MB
#235
google
72.36
99.99
44.74
1.301
1.00
408.34 ms
39,751,249
369.30 MB
#236
x-ai
72.32
99.89
44.76
1.701
1.04
633.612 ms
42,572,577
720.40 MB
#237
tencent
72.27
98.93
45.61
91.202
1.14
3,274.82 ms
38,592,063
835.50 MB
#238
z-ai
72.27
99.86
44.67
39.634
1.10
1,245.061 ms
47,340,745
410.78 MB
#239
openai
72.23
99.82
44.64
25.613
1.04
643.3 ms
61,356,069
1,161.59 MB
#240
nousresearch
72.11
99.76
44.47
1.959
1.17
3,230.065 ms
27,041,989
205.24 MB
#241
mistralai
72.10
99.67
44.53
0.875
1.13
1,680 ms
37,772,037
814.39 MB
#242
inclusionai
72.04
99.86
44.22
17.933
1.08
2,776.878 ms
42,950,877
267.84 MB
#243
openai
72.01
99.79
44.24
10.635
1.00
976.7 ms
46,652,690
1,054.90 MB
#244
z-ai
71.92
99.24
44.60
16.719
1.02
3,428.94 ms
47,166,161
978.76 MB
#245
nex-agi
71.87
98.93
44.81
18.496
1.06
2,941.5 ms
45,188,794
1,294.56 MB
#246
qwen
71.81
99.48
44.14
14.099
1.14
2,444.84 ms
46,373,865
913.66 MB
#247
anthropic
71.75
99.93
43.57
1.671
1.12
553.44 ms
49,735,847
918.65 MB
#248
google
71.71
99.84
43.57
1.611
1.08
1,410.02 ms
43,979,391
864.07 MB
#249
openai
71.67
99.94
43.39
2.137
1.04
2,280.84 ms
54,031,752
342.74 MB
#250
qwen
71.55
98.92
44.18
1.423
1.08
6,878.225 ms
39,065,170
842.06 MB
#251
mistralai
71.51
99.40
43.63
2.117
1.04
6,525.08 ms
46,866,961
759.81 MB
#252
qwen
71.46
99.30
43.62
3.073
1.06
4,973 ms
34,608,542
848.96 MB
#253
thedrummer
71.42
98.76
44.07
2.614
1.04
4,995.833 ms
39,136,379
842.22 MB
#254
kwaipilot
71.35
98.71
43.99
3.518
1.06
5,266.775 ms
50,403,744
1,075.42 MB
#255
inclusionai
71.31
99.05
43.58
1.648
1.23
3,362.187 ms
40,359,621
896.80 MB
#256
relace
71.30
99.64
42.95
1.236
1.08
3,351.32 ms
34,859,649
819.65 MB
#257
openai
71.27
99.97
42.58
2.766
1.06
748.437 ms
31,706,247
214.85 MB
#258
nvidia
71.27
99.22
43.32
10.142
1.12
4,084.56 ms
48,768,188
919.86 MB
#259
sao10k
71.26
96.36
46.17
29.030
1.00
5,946.468 ms
43,271,705
869.27 MB
#260
mistralai
71.26
98.50
44.02
1.809
1.00
376.5 ms
37,893,118
912.60 MB
#261
tencent
71.25
94.21
48.30
25.097
1.50
4,016.065 ms
43,534,014
956.02 MB
#262
nvidia
71.22
99.85
42.59
12.717
1.33
483.347 ms
40,823,966
813.29 MB
#263
mistralai
71.15
97.17
45.14
1.856
1.02
2,076.14 ms
34,303,920
846.49 MB
#264
anthropic
71.11
99.42
42.80
3.657
1.22
761.778 ms
36,590,481
232.57 MB
#265
openai
71.06
99.67
42.45
1.792
1.14
1,843.9 ms
45,378,797
1,138.96 MB
#266
openai
71.03
99.37
42.69
2.211
1.04
5,273.02 ms
57,313,535
959.36 MB
#267
mistralai
71.02
99.92
42.12
0.902
1.02
610.163 ms
41,840,418
837.25 MB
#268
google
71.02
99.99
42.04
1.426
1.02
350.146 ms
44,547,543
181.54 MB
#269
mistralai
71.01
99.95
42.08
1.405
1.08
420.714 ms
44,380,748
715.11 MB
#270
nousresearch
70.98
99.85
42.10
1.140
1.12
7,162.714 ms
36,607,875
216.41 MB
#271
openai
70.89
99.66
42.12
27.104
1.08
1,015.46 ms
45,844,074
1,126.98 MB
#272
minimax
70.86
99.64
42.08
8.530
1.08
1,863.64 ms
43,702,513
933.60 MB
#273
cohere
70.80
98.69
42.91
2.220
1.02
9,529.776 ms
46,089,837
789.28 MB
#274
qwen
70.70
99.96
41.44
2.806
1.09
308 ms
31,184,916
374.88 MB
#275
inclusionai
70.62
99.62
41.62
2.387
1.02
3,063.36 ms
42,522,173
892.15 MB
#276
anthropic
70.50
98.76
42.24
1.544
1.31
4,766.958 ms
34,045,851
774.89 MB
#277
mistralai
70.43
99.89
40.96
12.425
1.18
522.531 ms
39,072,130
681.69 MB
#278
qwen
70.40
99.74
41.05
1.229
1.06
938.298 ms
46,924,142
852.80 MB
#279
mistralai
70.06
99.59
40.54
0.913
1.07
841.891 ms
34,809,725
941.72 MB
#280
tencent
69.97
99.92
40.01
1.592
1.04
2,482.75 ms
33,982,224
219.58 MB
#281
google
69.93
99.95
39.90
1.622
1.00
384.551 ms
42,309,547
735.32 MB
#282
poolside
69.80
99.03
40.56
17.002
1.33
745.043 ms
41,118,378
1,012.30 MB
#283
poolside
69.78
99.45
40.10
57.933
1.02
612.319 ms
37,472,080
934.91 MB
#284
inclusionai
69.77
99.82
39.72
1.591
1.19
3,602.521 ms
57,601,522
327.10 MB
#285
mistralai
69.71
99.85
39.58
2.412
1.09
372.696 ms
44,597,846
757.05 MB
#286
nvidia
69.69
99.47
39.91
27.559
1.16
2,047.6 ms
41,923,462
880.96 MB
#287
z-ai
69.66
99.73
39.58
15.652
1.17
1,055.304 ms
51,805,270
403.62 MB
#288
openai
69.63
99.97
39.30
1.272
1.33
673.958 ms
37,688,322
273.55 MB
#289
amazon
69.61
99.39
39.84
1.238
1.10
5,630.68 ms
39,007,357
840.21 MB
#290
dots-studio
69.51
99.76
39.27
20.536
1.37
3,084.224 ms
35,365,304
279.66 MB
#291
minimax
69.50
98.87
40.12
8.381
1.10
4,454.68 ms
44,511,023
1,127.07 MB
#292
mistralai
69.44
99.94
38.95
1.251
1.04
1,188.128 ms
45,278,108
285.40 MB
#293
mistralai
69.30
99.67
38.93
0.942
1.10
2,089.51 ms
34,677,350
856.21 MB
#294
nvidia
69.12
97.78
40.47
9.416
1.21
3,259.936 ms
53,660,951
1,062.85 MB
#295
microsoft
68.96
97.43
40.50
1.074
1.16
4,285.511 ms
35,244,379
793.67 MB
#296
amazon
68.95
99.66
38.23
0.975
1.20
4,654.5 ms
40,480,727
253.11 MB
#297
ai21
68.85
99.73
37.98
2.841
1.18
6,607.449 ms
40,653,494
295.37 MB
#298
mistralai
68.84
99.64
38.05
1.397
1.21
1,240.521 ms
47,439,900
1,011.42 MB
#299
meta-llama
68.77
99.88
37.66
1.786
1.14
772.49 ms
52,201,709
857.02 MB
#300
openrouter
68.71
94.82
42.60
42.194
1.35
3,291.913 ms
43,647,901
998.75 MB
#301
amazon
68.54
98.74
38.33
1.147
1.02
4,954.766 ms
40,098,946
795.77 MB
#302
minimax
68.52
98.91
38.13
11.278
1.59
4,790.717 ms
49,453,309
320.59 MB
#303
openai
68.45
99.69
37.20
20.092
1.02
906.12 ms
61,641,565
1,386.39 MB
#304
google
68.40
99.98
36.83
1.512
1.04
850.96 ms
39,578,736
261.53 MB
#305
deepseek
68.39
94.87
41.91
3.206
1.56
950.439 ms
53,138,787
1,064.91 MB
#306
sao10k
68.11
95.98
40.24
1.679
1.11
9,357.957 ms
43,740,018
819.98 MB
#307
qwen
68.06
99.36
36.76
13.037
1.12
2,157.531 ms
46,780,909
924.89 MB
#308
cognitivecomputations
67.91
95.34
40.47
2.454
1.26
7,461.022 ms
31,506,499
812.88 MB
#309
ibm-granite
67.77
95.77
39.77
0.985
1.12
3,330.326 ms
54,786,273
990.76 MB
#310
amazon
67.73
99.22
36.24
1.141
1.04
5,754.449 ms
46,477,940
865.11 MB
#311
upstage
67.56
97.12
38.00
6.896
1.20
1,715.727 ms
41,230,113
939.81 MB
#312
mistralai
67.41
99.97
34.86
0.925
1.00
385.911 ms
40,043,041
257.63 MB
#313
meta-llama
66.36
97.15
35.56
3.501
1.21
445.242 ms
38,658,489
992.39 MB
#314
openrouter
66.22
94.19
38.24
1.237
1.22
932.811 ms
56,016,230
1,309.17 MB
#315
qwen
66.01
97.79
34.23
1.355
1.15
6,749.128 ms
49,552,058
909.97 MB
#316
openai
65.61
92.94
38.29
241.404
1.11
1,570.629 ms
43,522,261
426.47 MB
#317
meta-llama
65.56
90.63
40.49
9.571
1.00
3,343.071 ms
38,164,715
870.23 MB
#318
openrouter
64.98
88.70
41.26
25.916
1.06
1,119.273 ms
34,595,145
1,015.30 MB
#319
tencent
64.97
97.38
32.57
3.219
1.17
6,919.255 ms
65,822,716
941.41 MB
#320
mistralai
64.97
99.94
30.00
3.307
1.09
680.644 ms
48,641,279
222.69 MB
#321
cohere
64.33
99.20
29.47
7.228
1.11
6,343.457 ms
42,009,355
280.73 MB
#322
openai
63.96
99.99
27.93
1.538
1.06
445.694 ms
52,428,071
239.26 MB
#323
alibaba
63.53
94.77
32.29
20.128
1.13
553.026 ms
57,570,886
1,153.90 MB
#324
thedrummer
63.20
99.70
26.70
2.113
1.17
3,204.543 ms
37,674,236
264.42 MB
#325
perplexity
61.63
88.30
34.95
3.717
2.45
683.182 ms
56,964,923
1,347.49 MB
#326
moonshotai
61.54
97.81
25.27
1.305
1.57
348.686 ms
18,091,302
755.15 MB
#327
baidu
61.53
92.38
30.68
54.468
1.15
847.854 ms
48,091,065
1,108.87 MB
#328
openai
61.39
99.39
23.39
1.150
1.80
1,126.878 ms
36,206,482
218.84 MB
#329
qwen
61.21
83.54
38.89
7.176
1.30
9,341.25 ms
54,006,425
1,005.05 MB
#330
bytedance
61.01
97.62
24.39
0.928
1.07
7,555.711 ms
44,234,522
601.89 MB
#331
google
61.00
99.24
22.76
2.972
1.26
2,179.022 ms
31,151,872
532.00 MB
#332
deepseek
60.79
94.05
27.53
23.636
1.36
491.97 ms
51,371,378
1,121.80 MB
#333
mistralai
60.13
95.59
24.67
0.917
1.08
898.5 ms
34,734,695
1,081.71 MB
#334
arcee-ai
59.50
98.40
20.60
5.629
1.75
1,228.208 ms
54,051,699
278.08 MB
#335
cohere
58.66
98.51
18.82
1.907
1.13
3,657.15 ms
63,295,865
217.31 MB
#336
undi95
58.35
99.65
17.06
2.912
1.09
1,882.125 ms
57,309,914
118.89 MB
#337
thedrummer
58.34
91.75
24.92
3.834
1.05
6,221.114 ms
35,948,066
929.78 MB
#338
aion-labs
57.86
99.71
16.00
3.787
1.22
320.135 ms
19,347,326
209.48 MB
#339
moonshotai
57.17
94.82
19.51
39.871
1.71
1,145.429 ms
39,286,658
419.18 MB
#340
openrouter
57.15
99.30
15.00
28.858
1.82
358.176 ms
37,959,195
240.96 MB
#341
google
56.55
93.85
19.24
2.848
1.21
1,773.879 ms
24,522,122
586.93 MB
#342
meta-llama
56.39
99.92
12.86
1.256
2.35
452.45 ms
16,735,360
53.18 MB
#343
mancer
56.25
99.78
12.72
3.973
1.29
985.676 ms
70,315,978
122.06 MB
#344
sao10k
55.11
80.20
30.02
1.159
1.28
6,060.744 ms
31,923,004
689.41 MB
#345
meta-llama
54.91
99.20
10.61
0.564
1.27
1,001.962 ms
69,010,677
394.73 MB
#346
tencent
53.41
99.82
7.00
1.134
1.57
862.143 ms
71,557,778
124.40 MB
#347
google
53.34
99.61
7.08
4.106
1.67
954.867 ms
12,995,860
87.26 MB
#348
liquid
52.93
99.85
6.00
1.393
1.07
428.357 ms
77,197,005
224.10 MB
#349
nvidia
52.81
87.59
18.03
23.357
1.86
487.333 ms
57,185,740
1,558.26 MB
#350
qwen
51.07
99.83
2.30
5.664
1.00
264 ms
29,050,068
205.58 MB
#351
liquid
50.97
99.93
2.00
1.389
1.08
294.462 ms
37,560,298
133.52 MB
#352
inclusionai
50.88
99.76
2.00
26.389
1.00
293 ms
7,321,600
116.24 MB
#353
meta-llama
50.10
99.96
0.25
1.557
1.08
274.769 ms
86,662,200
136.94 MB
#355
openai
49.91
99.81
0.00
4.652
1.08
1,130.64 ms
40,869,345
918.65 MB
#356
poolside
49.77
99.53
0.00
11.675
1.04
2,422.32 ms
42,920,255
910.97 MB
#357
thinkingmachines
49.75
99.49
0.00
9.172
1.04
2,728.54 ms
45,832,574
945.40 MB
#358
nvidia
49.74
99.48
0.00
6.639
1.02
3,119.28 ms
38,819,411
890.54 MB
#359
openai
49.71
99.41
0.00
3.255
1.02
4,224.68 ms
49,033,170
956.22 MB
#360
liquid
49.43
98.29
0.57
0.865
1.48
3,961.652 ms
153,028,682
288.32 MB
#361
meta-llama
48.11
89.45
6.78
1.693
1.32
1,332.88 ms
69,602,499
1,484.41 MB
#362
tencent
47.51
88.12
6.89
1.688
1.42
6,844.167 ms
117,428,064
613.30 MB
#363
meta-llama
34.45
60.04
8.86
3.030
1.15
3,823.8 ms
50,342,319
1,595.67 MB
#364
nvidia
30.39
51.27
9.50
8.782
2.33
5,478.167 ms
105,344,275
957.80 MB
#365
google
3.71
0.00
7.42
1.609
1.83
3,631.083 ms
56,145,491
2,570.86 MB