SELF-HOSTING

Self-Hosting LLMs: VRAM & GPU Sizing

VRAM, GPU count and speed for running open-weights models yourself, next to the vendor's API price and measured speed.

Data updated: 2026-11-05

SortPopularity = Hugging Face downloads of the official repo, last 30 days (2026-11-05)

Alibaba Qwen3 models · 11.8M downloads (30d) · latest 2026-08-24

Qwen3.8 27B

28B total · 28B active · apache-2.0

Popularity: 8.0MHF downloads (30d) · 19,154 likes
Released: 2026-08-05HF repo created
Public benchmarks: AA 20.1 · LiveBench 60.5
Vendor API: $0.50 / $3.00input / output per 1M tokens
FP16 VRAM: ≈67 GBINT4 ≈17 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈67 GB VRAM
L20 48GB×2List price $6,985–$7,025Market price $10,472
Ascend 910B 64GB×2List price n/aMarket price $32,911–$35,903
RTX PRO 6000 Blackwell 96GB×1List price $16,025Market price $17,586Cloud rental (live): $1.20–$2.27/hr (2026-11-05)
RTX 5090 32GB×4List price n/aMarket price $26,927Cloud rental (live) ×4: $1.77–$2.13/hr (2026-11-05)
RTX 5090D 24GB×4List price $9,873Market price $16,755–$23,935Cloud rental (live) ×4: $1.77–$2.12/hr (2026-11-05)
INT8: ≈33 GB VRAM
L20 48GB×1List price $3,493–$3,513Market price $5,236
Ascend 910B 64GB×1List price n/aMarket price $16,455–$17,951
RTX PRO 6000 Blackwell 96GB×1List price $16,025Market price $17,586Cloud rental (live): $1.20–$2.27/hr (2026-11-05)
RTX 5090 32GB×2List price n/aMarket price $13,463Cloud rental (live) ×2: $0.88–$1.06/hr (2026-11-05)
RTX 5090D 24GB×2List price $4,936Market price $8,377–$11,968Cloud rental (live) ×2: $0.88–$1.06/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈17 GB VRAM
L20 48GB×1List price $3,493–$3,513Market price $5,236
Ascend 910B 64GB×1List price n/aMarket price $16,455–$17,951
RTX PRO 6000 Blackwell 96GB×1List price $16,025Market price $17,586Cloud rental (live): $1.20–$2.27/hr (2026-11-05)
RTX 5090 32GB×1List price n/aMarket price $6,732Cloud rental (live): $0.44–$0.53/hr (2026-11-05)
RTX 5090D 24GB×1List price $2,468Market price $4,189–$5,984Cloud rental (live): $0.44–$0.53/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

Qwen3.8-Flash-Next

180B total · 6B active · other

Popularity: 3.8MHF downloads (30d) · 6,785 likes
Released: 2026-08-24HF repo created
Public benchmarks: No public benchmark yet
Vendor API: —input / output per 1M tokens
FP16 VRAM: ≈432 GBINT4 ≈108 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈432 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×8List price n/aMarket price $131,644–$143,612
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×16List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT8: ≈216 GB VRAM
L20 48GB×8List price $27,941–$28,101Market price $41,887
Ascend 910B 64GB×4List price n/aMarket price $65,822–$71,806
RTX PRO 6000 Blackwell 96GB×4List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
RTX 5090 32GB×8List price n/aMarket price $53,853Cloud rental (live) ×8: $3.54–$4.26/hr (2026-11-05)
RTX 5090D 24GB×16List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈108 GB VRAM
L20 48GB×4List price $13,970–$14,051Market price $20,943
Ascend 910B 64GB×2List price n/aMarket price $32,911–$35,903
RTX PRO 6000 Blackwell 96GB×2List price $32,050Market price $35,173Cloud rental (live) ×2: $2.41–$4.54/hr (2026-11-05)
RTX 5090 32GB×4List price n/aMarket price $26,927Cloud rental (live) ×4: $1.77–$2.13/hr (2026-11-05)
RTX 5090D 24GB×8List price $19,745Market price $33,509–$47,871Cloud rental (live) ×8: $3.54–$4.24/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

Qwen3.8 2.4T A95B

2,446B total · 95B active · Qwen custom open-source license agreement (HF labeled license:other, non-standard OSI protocol, specific terms must be reviewed before use)

Popularity: 26.3KHF downloads (30d) · 1,364 likes
Released: 2026-08-08HF repo created
Public benchmarks: No public benchmark yet
Vendor API: $2.00 / $6.00input / output per 1M tokens
FP16 VRAM: ≈5,871 GBINT4 ≈1,468 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈5,871 GB VRAM
L20 48GB×123List price $429,590–$432,056Market price $644,008
Ascend 910B 64GB×92List price n/aMarket price $1,513,905–$1,651,533
RTX PRO 6000 Blackwell 96GB×64List price $1,025,597Market price $1,125,531Cloud rental (live) ×64: $76.99–$145.22/hr (2026-11-05)
RTX 5090 32GB×184List price n/aMarket price $1,238,622Cloud rental (live) ×184: $81.33–$97.89/hr (2026-11-05)
RTX 5090D 24GB×245List price $604,703Market price $1,026,224–$1,466,034Cloud rental (live) ×245: $108.29–$129.85/hr (2026-11-05)
INT8: ≈2,935 GB VRAM
L20 48GB×64List price $223,527–$224,809Market price $335,094
Ascend 910B 64GB×64List price n/aMarket price $1,053,151–$1,148,892
RTX PRO 6000 Blackwell 96GB×32List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
RTX 5090 32GB×92List price n/aMarket price $619,311Cloud rental (live) ×92: $40.66–$48.94/hr (2026-11-05)
RTX 5090D 24GB×123List price $303,585Market price $515,206–$736,009Cloud rental (live) ×123: $54.37–$65.19/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈1,468 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×32List price n/aMarket price $526,576–$574,446
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×64List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

Zhipu AI (GLM)5 models · 11.3M downloads (30d) · latest 2026-08-25

GLM-5.3-Flash

321B total · 18B active · mit

Popularity: 6.3MHF downloads (30d) · 3,050 likes
Released: 2026-08-25HF repo created
Public benchmarks: AA 27.8 · LiveBench 65.8
Vendor API: $0.15 / $0.50input / output per 1M tokens
FP16 VRAM: ≈771 GBINT4 ≈193 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈771 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT8: ≈386 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×8List price n/aMarket price $131,644–$143,612
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×16List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈193 GB VRAM
L20 48GB×8List price $27,941–$28,101Market price $41,887
Ascend 910B 64GB×4List price n/aMarket price $65,822–$71,806
RTX PRO 6000 Blackwell 96GB×4List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
RTX 5090 32GB×8List price n/aMarket price $53,853Cloud rental (live) ×8: $3.54–$4.26/hr (2026-11-05)
RTX 5090D 24GB×16List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

GLM-5.3

753B total · 40B active (est.) · MIT License

Popularity: 2.3MHF downloads (30d) · 2,381 likes
Released: 2026-08-25HF repo created
Public benchmarks: AA 31.7 · LiveBench 0.0
Vendor API: $1.40 / $4.40input / output per 1M tokens
FP16 VRAM: ≈1,808 GBINT4 ≈452 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈1,808 GB VRAM
L20 48GB×64List price $223,527–$224,809Market price $335,094
Ascend 910B 64GB×32List price n/aMarket price $526,576–$574,446
RTX PRO 6000 Blackwell 96GB×32List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
RTX 5090 32GB×64List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
RTX 5090D 24GB×76List price $187,581Market price $318,339–$454,770Cloud rental (live) ×76: $33.59–$40.28/hr (2026-11-05)
INT8: ≈904 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈452 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×8List price n/aMarket price $131,644–$143,612
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×16List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

GLM-5

754B total · 40B active · mit

Popularity: 1.5MHF downloads (30d) · 2,141 likes
Released: 2026-02-11HF repo created
Public benchmarks: AA — · LiveBench 0.0
Vendor API: $1.00 / $3.20input / output per 1M tokens
FP16 VRAM: ≈1,809 GBINT4 ≈452 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈1,809 GB VRAM
L20 48GB×64List price $223,527–$224,809Market price $335,094
Ascend 910B 64GB×32List price n/aMarket price $526,576–$574,446
RTX PRO 6000 Blackwell 96GB×32List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
RTX 5090 32GB×64List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
RTX 5090D 24GB×76List price $187,581Market price $318,339–$454,770Cloud rental (live) ×76: $33.59–$40.28/hr (2026-11-05)
INT8: ≈905 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈452 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×8List price n/aMarket price $131,644–$143,612
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×16List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

GLM-5.1

754B total · 40B active · MIT License

Popularity: 922.6KHF downloads (30d) · 1,850 likes
Released: 2026-04-03HF repo created
Public benchmarks: AA — · LiveBench 0.0
Vendor API: $1.40 / $4.40input / output per 1M tokens
FP16 VRAM: ≈1,809 GBINT4 ≈452 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈1,809 GB VRAM
L20 48GB×64List price $223,527–$224,809Market price $335,094
Ascend 910B 64GB×32List price n/aMarket price $526,576–$574,446
RTX PRO 6000 Blackwell 96GB×32List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
RTX 5090 32GB×64List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
RTX 5090D 24GB×76List price $187,581Market price $318,339–$454,770Cloud rental (live) ×76: $33.59–$40.28/hr (2026-11-05)
INT8: ≈905 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈452 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×8List price n/aMarket price $131,644–$143,612
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×16List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

GLM-5.2

753B total · 39B active · MIT License

Popularity: 332.2KHF downloads (30d) · 5,202 likes
Released: 2026-06-16HF repo created
Public benchmarks: AA — · LiveBench 0.0
Vendor API: $1.40 / $4.40input / output per 1M tokens
FP16 VRAM: ≈1,808 GBINT4 ≈452 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈1,808 GB VRAM
L20 48GB×64List price $223,527–$224,809Market price $335,094
Ascend 910B 64GB×32List price n/aMarket price $526,576–$574,446
RTX PRO 6000 Blackwell 96GB×32List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
RTX 5090 32GB×64List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
RTX 5090D 24GB×76List price $187,581Market price $318,339–$454,770Cloud rental (live) ×76: $33.59–$40.28/hr (2026-11-05)
INT8: ≈904 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈452 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×8List price n/aMarket price $131,644–$143,612
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×16List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

DeepSeek2 models · 2.3M downloads (30d) · latest 2026-09-10

DeepSeek V4.1 Flash

763B total · 16B active · MIT

Popularity: 2.0MHF downloads (30d) · 4,863 likes
Released: 2026-09-10HF repo created
Public benchmarks: AA 32.6 · LiveBench 75.8
Vendor API: $0.30 / $1.20input / output per 1M tokens
FP16 VRAM: ≈1,832 GBINT4 ≈458 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈1,832 GB VRAM
L20 48GB×64List price $223,527–$224,809Market price $335,094
Ascend 910B 64GB×32List price n/aMarket price $526,576–$574,446
RTX PRO 6000 Blackwell 96GB×32List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
RTX 5090 32GB×64List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
RTX 5090D 24GB×77List price $190,049Market price $322,528–$460,754Cloud rental (live) ×77: $34.03–$40.81/hr (2026-11-05)
INT8: ≈916 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈458 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×8List price n/aMarket price $131,644–$143,612
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×16List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

DeepSeek V4 Pro

1,599B total · 49B active · DeepSeek open-source license agreement

Popularity: 336.9KHF downloads (30d) · 5,696 likes
Released: 2026-04-22HF repo created
Public benchmarks: AA 23.1 · LiveBench 0.0
Vendor API: $1.32 / $3.96input / output per 1M tokens
FP16 VRAM: ≈3,837 GBINT4 ≈959 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈3,837 GB VRAM
L20 48GB×80List price $279,408–$281,012Market price $418,867
Ascend 910B 64GB×64List price n/aMarket price $1,053,151–$1,148,892
RTX PRO 6000 Blackwell 96GB×64List price $1,025,597Market price $1,125,531Cloud rental (live) ×64: $76.99–$145.22/hr (2026-11-05)
RTX 5090 32GB×120List price n/aMarket price $807,797Cloud rental (live) ×120: $53.04–$63.84/hr (2026-11-05)
RTX 5090D 24GB×160List price $394,908Market price $670,187–$957,410Cloud rental (live) ×160: $70.72–$84.80/hr (2026-11-05)
INT8: ≈1,919 GB VRAM
L20 48GB×64List price $223,527–$224,809Market price $335,094
Ascend 910B 64GB×32List price n/aMarket price $526,576–$574,446
RTX PRO 6000 Blackwell 96GB×32List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
RTX 5090 32GB×64List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
RTX 5090D 24GB×80List price $197,454Market price $335,094–$478,705Cloud rental (live) ×80: $35.36–$42.40/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈959 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

Moonshot AI (Kimi)5 models · 1.5M downloads (30d) · latest 2026-11-02

Kimi K3

2,780B total · 104B active · Modified MIT License

Popularity: 768.7KHF downloads (30d) · 11,939 likes
Released: 2026-06-13HF repo created
Public benchmarks: AA — · LiveBench 69.5
Vendor API: $3.00 / $15.00input / output per 1M tokens
FP16 VRAM: ≈6,672 GBINT4 ≈1,668 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈6,672 GB VRAM
L20 48GB×139List price $485,472–$488,258Market price $727,781
Ascend 910B 64GB×105List price n/aMarket price $1,727,826–$1,884,901
RTX PRO 6000 Blackwell 96GB×70List price $1,121,747Market price $1,231,050Cloud rental (live) ×70: $84.21–$158.83/hr (2026-11-05)
RTX 5090 32GB×209List price n/aMarket price $1,406,913Cloud rental (live) ×209: $92.38–$111.19/hr (2026-11-05)
RTX 5090D 24GB×278List price $686,152Market price $1,164,450–$1,663,500Cloud rental (live) ×278: $122.88–$147.34/hr (2026-11-05)
INT8: ≈3,336 GB VRAM
L20 48GB×70List price $244,482–$245,885Market price $366,509
Ascend 910B 64GB×64List price n/aMarket price $1,053,151–$1,148,892
RTX PRO 6000 Blackwell 96GB×64List price $1,025,597Market price $1,125,531Cloud rental (live) ×64: $76.99–$145.22/hr (2026-11-05)
RTX 5090 32GB×105List price n/aMarket price $706,822Cloud rental (live) ×105: $46.41–$55.86/hr (2026-11-05)
RTX 5090D 24GB×139List price $343,076Market price $582,225–$831,750Cloud rental (live) ×139: $61.44–$73.67/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈1,668 GB VRAM
L20 48GB×64List price $223,527–$224,809Market price $335,094
Ascend 910B 64GB×32List price n/aMarket price $526,576–$574,446
RTX PRO 6000 Blackwell 96GB×32List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
RTX 5090 32GB×64List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
RTX 5090D 24GB×70List price $172,772Market price $293,207–$418,867Cloud rental (live) ×70: $30.94–$37.10/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

Kimi K2.5

1,027B total · 32B active · Open-source weight license

Popularity: 395.4KHF downloads (30d) · 2,898 likes
Released: 2026-01-01HF repo created
Public benchmarks: AA — · LiveBench 0.0
Vendor API: $0.60 / $3.00input / output per 1M tokens
FP16 VRAM: ≈2,465 GBINT4 ≈616 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈2,465 GB VRAM
L20 48GB×64List price $223,527–$224,809Market price $335,094
Ascend 910B 64GB×64List price n/aMarket price $1,053,151–$1,148,892
RTX PRO 6000 Blackwell 96GB×32List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
RTX 5090 32GB×78List price n/aMarket price $525,068Cloud rental (live) ×78: $34.48–$41.50/hr (2026-11-05)
RTX 5090D 24GB×103List price $254,222Market price $431,433–$616,333Cloud rental (live) ×103: $45.53–$54.59/hr (2026-11-05)
INT8: ≈1,232 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×32List price n/aMarket price $526,576–$574,446
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×64List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈616 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

Kimi K2.6

1,027B total · 32B active · Modified MIT License

Popularity: 244.2KHF downloads (30d) · 1,640 likes
Released: 2026-04-14HF repo created
Public benchmarks: AA — · LiveBench 0.0
Vendor API: $0.95 / $4.00input / output per 1M tokens
FP16 VRAM: ≈2,465 GBINT4 ≈616 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈2,465 GB VRAM
L20 48GB×64List price $223,527–$224,809Market price $335,094
Ascend 910B 64GB×64List price n/aMarket price $1,053,151–$1,148,892
RTX PRO 6000 Blackwell 96GB×32List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
RTX 5090 32GB×78List price n/aMarket price $525,068Cloud rental (live) ×78: $34.48–$41.50/hr (2026-11-05)
RTX 5090D 24GB×103List price $254,222Market price $431,433–$616,333Cloud rental (live) ×103: $45.53–$54.59/hr (2026-11-05)
INT8: ≈1,232 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×32List price n/aMarket price $526,576–$574,446
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×64List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈616 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

Kimi K2.7 Code

1,027B total · 32B active · Modified MIT License

Popularity: 81.8KHF downloads (30d) · 1,429 likes
Released: 2026-06-11HF repo created
Public benchmarks: AA — · LiveBench 0.0
Vendor API: $0.95 / $4.00input / output per 1M tokens
FP16 VRAM: ≈2,465 GBINT4 ≈616 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈2,465 GB VRAM
L20 48GB×64List price $223,527–$224,809Market price $335,094
Ascend 910B 64GB×64List price n/aMarket price $1,053,151–$1,148,892
RTX PRO 6000 Blackwell 96GB×32List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
RTX 5090 32GB×78List price n/aMarket price $525,068Cloud rental (live) ×78: $34.48–$41.50/hr (2026-11-05)
RTX 5090D 24GB×103List price $254,222Market price $431,433–$616,333Cloud rental (live) ×103: $45.53–$54.59/hr (2026-11-05)
INT8: ≈1,232 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×32List price n/aMarket price $526,576–$574,446
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×64List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈616 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

Kimi K3.1 Pro

2,780B total · 104B active · other

Popularity: 797HF downloads (30d) · 510 likes
Released: 2026-11-02HF repo created
Public benchmarks: AA 46.5 · LiveBench 74.7
Vendor API: $2.00 / $10.00input / output per 1M tokens
FP16 VRAM: ≈6,672 GBINT4 ≈1,668 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈6,672 GB VRAM
L20 48GB×140List price $488,964–$491,771Market price $733,017
Ascend 910B 64GB×105List price n/aMarket price $1,727,826–$1,884,901
RTX PRO 6000 Blackwell 96GB×70List price $1,121,747Market price $1,231,050Cloud rental (live) ×70: $84.21–$158.83/hr (2026-11-05)
RTX 5090 32GB×209List price n/aMarket price $1,406,913Cloud rental (live) ×209: $92.38–$111.19/hr (2026-11-05)
RTX 5090D 24GB×279List price $688,620Market price $1,168,639–$1,669,484Cloud rental (live) ×279: $123.32–$147.87/hr (2026-11-05)
INT8: ≈3,336 GB VRAM
L20 48GB×70List price $244,482–$245,885Market price $366,509
Ascend 910B 64GB×64List price n/aMarket price $1,053,151–$1,148,892
RTX PRO 6000 Blackwell 96GB×64List price $1,025,597Market price $1,125,531Cloud rental (live) ×64: $76.99–$145.22/hr (2026-11-05)
RTX 5090 32GB×105List price n/aMarket price $706,822Cloud rental (live) ×105: $46.41–$55.86/hr (2026-11-05)
RTX 5090D 24GB×140List price $345,544Market price $586,414–$837,734Cloud rental (live) ×140: $61.88–$74.20/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈1,668 GB VRAM
L20 48GB×64List price $223,527–$224,809Market price $335,094
Ascend 910B 64GB×32List price n/aMarket price $526,576–$574,446
RTX PRO 6000 Blackwell 96GB×32List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
RTX 5090 32GB×64List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
RTX 5090D 24GB×70List price $172,772Market price $293,207–$418,867Cloud rental (live) ×70: $30.94–$37.10/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

MiniMax2 models · 206.6K downloads (30d) · latest 2026-10-17

MiniMax M3

427B total · 23B active · MIT License

Popularity: 176.4KHF downloads (30d) · 1,601 likes
Released: 2026-06-02HF repo created
Public benchmarks: AA 18.9 · LiveBench 0.0
Vendor API: $0.30 / $1.20input / output per 1M tokens
FP16 VRAM: ≈1,025 GBINT4 ≈256 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈1,025 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×32List price n/aMarket price $526,576–$574,446
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×64List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT8: ≈512 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈256 GB VRAM
L20 48GB×8List price $27,941–$28,101Market price $41,887
Ascend 910B 64GB×8List price n/aMarket price $131,644–$143,612
RTX PRO 6000 Blackwell 96GB×4List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
RTX 5090 32GB×16List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
RTX 5090D 24GB×16List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

MiniMax M3.1 Flash

220B total · 23B active · other

Popularity: 30.2KHF downloads (30d) · 548 likes
Released: 2026-10-17HF repo created
Public benchmarks: AA 29.1 · LiveBench 70.6
Vendor API: $0.15 / $0.60input / output per 1M tokens
FP16 VRAM: ≈529 GBINT4 ≈132 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈529 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT8: ≈265 GB VRAM
L20 48GB×8List price $27,941–$28,101Market price $41,887
Ascend 910B 64GB×8List price n/aMarket price $131,644–$143,612
RTX PRO 6000 Blackwell 96GB×4List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
RTX 5090 32GB×16List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
RTX 5090D 24GB×16List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈132 GB VRAM
L20 48GB×4List price $13,970–$14,051Market price $20,943
Ascend 910B 64GB×4List price n/aMarket price $65,822–$71,806
RTX PRO 6000 Blackwell 96GB×2List price $32,050Market price $35,173Cloud rental (live) ×2: $2.41–$4.54/hr (2026-11-05)
RTX 5090 32GB×8List price n/aMarket price $53,853Cloud rental (live) ×8: $3.54–$4.26/hr (2026-11-05)
RTX 5090D 24GB×8List price $19,745Market price $33,509–$47,871Cloud rental (live) ×8: $3.54–$4.24/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

Tencent Hunyuan1 model · 67.5K downloads (30d) · latest 2025-06-25

Hunyuan A13B

80B total · 13B active · Tencent Hunyuan Open Source License Agreement

Popularity: 67.5KHF downloads (30d) · 795 likes
Released: 2025-06-25HF repo created
Public benchmarks: No public benchmark yet
Vendor API: $0.07 / $0.30input / output per 1M tokens
FP16 VRAM: ≈192 GBINT4 ≈48 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈192 GB VRAM
L20 48GB×4List price $13,970–$14,051Market price $20,943
Ascend 910B 64GB×4List price n/aMarket price $65,822–$71,806
RTX PRO 6000 Blackwell 96GB×2List price $32,050Market price $35,173Cloud rental (live) ×2: $2.41–$4.54/hr (2026-11-05)
RTX 5090 32GB×8List price n/aMarket price $53,853Cloud rental (live) ×8: $3.54–$4.26/hr (2026-11-05)
RTX 5090D 24GB×8List price $19,745Market price $33,509–$47,871Cloud rental (live) ×8: $3.54–$4.24/hr (2026-11-05)
INT8: ≈96 GB VRAM
L20 48GB×2List price $6,985–$7,025Market price $10,472
Ascend 910B 64GB×2List price n/aMarket price $32,911–$35,903
RTX PRO 6000 Blackwell 96GB×1List price $16,025Market price $17,586Cloud rental (live): $1.20–$2.27/hr (2026-11-05)
RTX 5090 32GB×4List price n/aMarket price $26,927Cloud rental (live) ×4: $1.77–$2.13/hr (2026-11-05)
RTX 5090D 24GB×4List price $9,873Market price $16,755–$23,935Cloud rental (live) ×4: $1.77–$2.12/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈48 GB VRAM
L20 48GB×1List price $3,493–$3,513Market price $5,236
Ascend 910B 64GB×1List price n/aMarket price $16,455–$17,951
RTX PRO 6000 Blackwell 96GB×1List price $16,025Market price $17,586Cloud rental (live): $1.20–$2.27/hr (2026-11-05)
RTX 5090 32GB×2List price n/aMarket price $13,463Cloud rental (live) ×2: $0.88–$1.06/hr (2026-11-05)
RTX 5090D 24GB×2List price $4,936Market price $8,377–$11,968Cloud rental (live) ×2: $0.88–$1.06/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

IFM(MBZUAI)1 model · 9.8K downloads (30d) · latest 2026-09-01

K2 Horizon 375B A23B

379B total · 23B active · apache-2.0

Popularity: 9.8KHF downloads (30d) · 118 likes
Released: 2026-09-01HF repo created
Public benchmarks: No public benchmark yet
Vendor API: —input / output per 1M tokens
FP16 VRAM: ≈910 GBINT4 ≈228 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈910 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT8: ≈455 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×8List price n/aMarket price $131,644–$143,612
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×16List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈228 GB VRAM
L20 48GB×8List price $27,941–$28,101Market price $41,887
Ascend 910B 64GB×4List price n/aMarket price $65,822–$71,806
RTX PRO 6000 Blackwell 96GB×4List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
RTX 5090 32GB×8List price n/aMarket price $53,853Cloud rental (live) ×8: $3.54–$4.26/hr (2026-11-05)
RTX 5090D 24GB×16List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

阶跃星辰1 model · 6.0K downloads (30d) · latest 2026-09-20

Step 5 Preview

604B total · 27B active · other

Popularity: 6.0KHF downloads (30d) · 137 likes
Released: 2026-09-20HF repo created
Public benchmarks: AA 28.5 · LiveBench 61.0
Vendor API: $1.05 / $2.99input / output per 1M tokens
FP16 VRAM: ≈1,450 GBINT4 ≈363 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈1,450 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×32List price n/aMarket price $526,576–$574,446
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×64List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT8: ≈725 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈363 GB VRAM
L20 48GB×8List price $27,941–$28,101Market price $41,887
Ascend 910B 64GB×8List price n/aMarket price $131,644–$143,612
RTX PRO 6000 Blackwell 96GB×4List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
RTX 5090 32GB×16List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
RTX 5090D 24GB×16List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

Motif Technologies1 model · 1.8K downloads (30d) · latest 2026-08-07

Motif 3

315B total · 13B active · mit

Popularity: 1.8KHF downloads (30d) · 154 likes
Released: 2026-08-07HF repo created
Public benchmarks: No public benchmark yet
Vendor API: —input / output per 1M tokens
FP16 VRAM: ≈756 GBINT4 ≈189 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈756 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT8: ≈378 GB VRAM
L20 48GB×8List price $27,941–$28,101Market price $41,887
Ascend 910B 64GB×8List price n/aMarket price $131,644–$143,612
RTX PRO 6000 Blackwell 96GB×4List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
RTX 5090 32GB×16List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
RTX 5090D 24GB×16List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈189 GB VRAM
L20 48GB×4List price $13,970–$14,051Market price $20,943
Ascend 910B 64GB×4List price n/aMarket price $65,822–$71,806
RTX PRO 6000 Blackwell 96GB×2List price $32,050Market price $35,173Cloud rental (live) ×2: $2.41–$4.54/hr (2026-11-05)
RTX 5090 32GB×8List price n/aMarket price $53,853Cloud rental (live) ×8: $3.54–$4.26/hr (2026-11-05)
RTX 5090D 24GB×8List price $19,745Market price $33,509–$47,871Cloud rental (live) ×8: $3.54–$4.24/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

Nex AGI1 model · 880 downloads (30d) · latest 2026-06-03

Nex-N2-Pro

397B total · 18B active (est.) · apache-2.0

Popularity: 880HF downloads (30d) · 376 likes
Released: 2026-06-03HF repo created
Public benchmarks: No public benchmark yet
Vendor API: —input / output per 1M tokens
FP16 VRAM: ≈952 GBINT4 ≈238 GB

VRAM and reference setups (FP16 / INT8 / INT4)

FP16/BF16 (full precision): ≈952 GB VRAM
L20 48GB×32List price $111,763–$112,405Market price $167,547
Ascend 910B 64GB×16List price n/aMarket price $263,288–$287,223
RTX PRO 6000 Blackwell 96GB×16List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
RTX 5090 32GB×32List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
RTX 5090D 24GB×64List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT8: ≈476 GB VRAM
L20 48GB×16List price $55,882–$56,202Market price $83,773
Ascend 910B 64GB×8List price n/aMarket price $131,644–$143,612
RTX PRO 6000 Blackwell 96GB×8List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
RTX 5090 32GB×16List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
RTX 5090D 24GB×32List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT4 (AWQ/GPTQ): ≈238 GB VRAM
L20 48GB×8List price $27,941–$28,101Market price $41,887
Ascend 910B 64GB×4List price n/aMarket price $65,822–$71,806
RTX PRO 6000 Blackwell 96GB×4List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
RTX 5090 32GB×8List price n/aMarket price $53,853Cloud rental (live) ×8: $3.54–$4.26/hr (2026-11-05)
RTX 5090D 24GB×16List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)

Full setup across 18 GPUs → Official repo →

INFERENCE ENGINES

Open-source inference enginesWhich software runs the model when you self-host · from each engine's official repo/docs, checked 2026-10-24

Engine	By	Models	Hardware	Online serving	License	Notes
vLLM	Community (UC Berkeley origin)	Most open-weights models	NVIDIA / AMD / TPU / Intel etc.; Ascend via vllm-ascend plugin	Yes (OpenAI-compatible API)	Apache-2.0	PagedAttention; de-facto default, new models usually supported on day one
SGLang	Community (LMSYS origin)	Most open-weights models	NVIDIA / AMD / Ascend / TPU etc.	Yes (OpenAI-compatible API)	Apache-2.0	RadixAttention prefix cache; common for large MoE deployments
TensorRT-LLM	NVIDIA	Major open-weights models (per official support list)	NVIDIA only	Yes (trtllm-serve / Triton)	Apache-2.0	NVIDIA-optimized; first to use new GPU features
MindIE	Huawei	Major open-weights models (per Ascend support list)	Ascend only	Yes (MindIE Service)	华为软件许可	Huawei's official Ascend inference suite
llama.cpp	Community	GGUF quantized models	CPU / NVIDIA / AMD / Apple etc., mostly consumer hardware	Yes (llama-server, single-node)	MIT	Low-bit quantization for PCs and single GPUs; Ollama and others build on it
DeepInferreleased 2026-10-20	DeepSeek	DeepSeek V4.1 Flash only	NVIDIA SM100 (B200/B300) / Ascend 950	No (offline inference only in this release)	Apache-2.0	Powers DeepSeek's own production; KV cache system, prefill/decode disaggregation and load balancing not included

GPU prices

List price, market price (SXM GPUs per 8-GPU node) and cloud rental for 18 GPUs: GPU prices & cloud rental →

FAQ

How is the VRAM requirement calculated?

VRAM = total parameters (B) × bytes per parameter × 1.2 runtime headroom. It covers model weights only, not concurrency or long-context KV cache; leave another 20–30% and load-test before buying.

How are GPU counts derived?

GPU counts use reference cards (L20 48 GB, Ascend 910B 64 GB, RTX PRO 6000 96 GB, RTX 5090 32 GB, RTX 5090D 24 GB) as examples; any GPU of similar VRAM works. Every model page covers all GPUs.

What do “active parameters” mean for MoE models?

Active parameters set compute and speed but do not reduce weight VRAM, so sizing uses total parameters; “est.” marks active-parameter counts inferred from sibling releases.

Where do the GPU prices come from?

SXM data-center GPUs (B300/B200/H200/H100/H800) are quoted per 8-GPU HGX node, as the industry does; per-GPU equivalents are shown and sizing bills whole nodes. USD converted from CNY at ¥6.68, taken from public reports and re-searched weekly; they swing widely and are indicative only.