{"api_version":"v1","generated":"2026-10-24T05:40:31+00:00","source":"https://tensorust.com","license":"CC BY 4.0, credit Tensorust with a link","docs":"https://tensorust.com/developers/","models":[{"id":"llama-3.2-1b","n":"Llama 3.2 1B","p":1.236,"r":"meta-llama/Llama-3.2-1B-Instruct","g":[[16,8,64,0]],"cx":131072},{"id":"llama-3.2-3b","n":"Llama 3.2 3B","p":3.213,"r":"meta-llama/Llama-3.2-3B-Instruct","g":[[28,8,128,0]],"cx":131072},{"id":"llama-3.1-8b","n":"Llama 3.1 8B","p":8.03,"r":"meta-llama/Llama-3.1-8B-Instruct","g":[[32,8,128,0]],"cx":131072},{"id":"llama-3.3-70b","n":"Llama 3.3 70B","p":70.554,"r":"meta-llama/Llama-3.3-70B-Instruct","g":[[80,8,128,0]],"cx":131072},{"id":"qwen3-4b","n":"Qwen3 4B","p":4.022,"r":"Qwen/Qwen3-4B","g":[[36,8,128,0]],"cx":40960},{"id":"qwen3-8b","n":"Qwen3 8B","p":8.191,"r":"Qwen/Qwen3-8B","g":[[36,8,128,0]],"cx":40960},{"id":"qwen3-14b","n":"Qwen3 14B","p":14.768,"r":"Qwen/Qwen3-14B","g":[[40,8,128,0]],"cx":40960},{"id":"qwen3-32b","n":"Qwen3 32B","p":32.762,"r":"Qwen/Qwen3-32B","g":[[64,8,128,0]],"cx":40960},{"id":"qwen3-30b-a3b","n":"Qwen3 30B-A3B (MoE)","p":30.532,"r":"Qwen/Qwen3-30B-A3B","g":[[48,4,128,0]],"cx":40960,"note":"Mixture of experts: about 3B parameters are active per token, but all 30.5B must sit in memory."},{"id":"qwen3.6-27b","n":"Qwen3.6 27B","p":27.781,"r":"Qwen/Qwen3.6-27B","g":[[16,4,256,0]],"lin":150994944,"cx":262144,"note":"Hybrid attention: 16 of 64 layers keep a KV cache; the 48 linear-attention layers keep a fixed state (about 0.14 GB per sequence) that does not grow with context."},{"id":"qwen3.6-35b-a3b","n":"Qwen3.6 35B-A3B (MoE)","p":35.952,"r":"Qwen/Qwen3.6-35B-A3B","g":[[10,2,256,0]],"lin":62914560,"cx":262144,"note":"Hybrid attention: 10 of 40 layers keep a KV cache; the other 30 keep a fixed state. Mixture of experts: every expert must sit in memory."},{"id":"qwen2.5-coder-32b","n":"Qwen2.5 Coder 32B","p":32.764,"r":"Qwen/Qwen2.5-Coder-32B-Instruct","g":[[64,8,128,0]],"cx":32768},{"id":"gemma-3-4b","n":"Gemma 3 4B","p":4.3,"r":"google/gemma-3-4b-it","g":[[5,4,256,0],[29,4,256,1024]],"cx":131072,"note":"5 of every 6 layers use a 1,024-token sliding window, so the KV cache stays small at long context. Parameter count includes the vision encoder."},{"id":"gemma-3-12b","n":"Gemma 3 12B","p":12.187,"r":"google/gemma-3-12b-it","g":[[8,8,256,0],[40,8,256,1024]],"cx":131072,"note":"5 of every 6 layers use a 1,024-token sliding window. Parameter count includes the vision encoder."},{"id":"gemma-3-27b","n":"Gemma 3 27B","p":27.432,"r":"google/gemma-3-27b-it","g":[[10,16,128,0],[52,16,128,1024]],"cx":131072,"note":"5 of every 6 layers use a 1,024-token sliding window. Parameter count includes the vision encoder."},{"id":"gemma-4-31b","n":"Gemma 4 31B","p":31.273,"r":"google/gemma-4-31B-it","g":[[10,4,512,0],[50,16,256,1024]],"cx":262144,"note":"50 of 60 layers use a 1,024-token sliding window. Gemma 4 can share keys and values in attention; we count both, so the KV number is an upper bound."},{"id":"gemma-4-26b-a4b","n":"Gemma 4 26B-A4B (MoE)","p":25.806,"r":"google/gemma-4-26B-A4B-it","g":[[5,2,512,0],[25,8,256,1024]],"cx":262144,"note":"25 of 30 layers use a 1,024-token sliding window; KV counted as an upper bound. Mixture of experts: every expert must sit in memory."},{"id":"mistral-7b","n":"Mistral 7B v0.3","p":7.248,"r":"mistralai/Mistral-7B-Instruct-v0.3","g":[[32,8,128,0]],"cx":32768},{"id":"mistral-small-3.1","n":"Mistral Small 3.1 24B","p":24.011,"r":"mistralai/Mistral-Small-3.1-24B-Instruct-2503","g":[[40,8,128,0]],"cx":131072},{"id":"phi-4","n":"Phi-4 14B","p":14.66,"r":"microsoft/phi-4","g":[[40,10,128,0]],"cx":16384},{"id":"gpt-oss-20b","n":"gpt-oss-20b (MoE)","p":21.0,"r":"openai/gpt-oss-20b","g":[[12,8,64,0],[12,8,64,128]],"nat":13761316904,"cx":131072,"note":"Ships with MXFP4 expert weights: the checkpoint on Hugging Face is 13.76 GB (12.8 GB in the unit used here). Half the layers use a 128-token sliding window."},{"id":"gpt-oss-120b","n":"gpt-oss-120b (MoE)","p":117.0,"r":"openai/gpt-oss-120b","g":[[18,8,64,0],[18,8,64,128]],"nat":65248893184,"cx":131072,"note":"Ships with MXFP4 expert weights: the checkpoint on Hugging Face is 65.25 GB (60.8 GB in the unit used here). Half the layers use a 128-token sliding window."},{"id":"glm-4.7-flash","n":"GLM-4.7-Flash (MoE)","p":31.221,"r":"zai-org/GLM-4.7-Flash","mla":[47,576],"cx":202752,"note":"Latent attention (MLA): each layer caches one 576-number vector per token instead of full keys and values. Assumes your runtime stores the compressed cache."},{"id":"deepseek-r1","n":"DeepSeek-R1 671B (MoE)","p":671.0,"r":"deepseek-ai/DeepSeek-R1","mla":[61,576],"cx":163840,"note":"671B total parameters per the model card (37B active per token). Latent attention (MLA) keeps the KV cache small; assumes your runtime stores the compressed cache."}],"quants":[{"id":"native","n":"As shipped (native checkpoint)","b":0},{"id":"fp32","n":"FP32","b":32},{"id":"fp16","n":"FP16 / BF16","b":16},{"id":"fp8","n":"FP8 / INT8","b":8},{"id":"q8_0","n":"Q8_0 (GGUF)","b":8.5008},{"id":"q6_k","n":"Q6_K (GGUF)","b":6.5633},{"id":"q5_k_m","n":"Q5_K_M (GGUF)","b":5.7036},{"id":"q4_k_m","n":"Q4_K_M (GGUF)","b":4.8944},{"id":"nvfp4","n":"NVFP4","b":4.5},{"id":"mxfp4","n":"MXFP4","b":4.25},{"id":"q3_k_m","n":"Q3_K_M (GGUF)","b":3.996},{"id":"q2_k","n":"Q2_K (GGUF)","b":3.1593}],"gpus":[{"name":"RTX 4060 / 5060","gb":8},{"name":"RTX 3060 12GB","gb":12},{"name":"RTX 4070 / 5070","gb":12},{"name":"RTX 4060 Ti 16GB / 5060 Ti 16GB","gb":16},{"name":"RTX 4080 / 5080 / 5070 Ti","gb":16},{"name":"RTX 3090 / 4090","gb":24},{"name":"NVIDIA L4","gb":24},{"name":"RTX 5090","gb":32},{"name":"NVIDIA L40S","gb":48},{"name":"A100 80GB / H100","gb":80},{"name":"RTX PRO 6000 Blackwell","gb":96},{"name":"NVIDIA H200","gb":141}],"unit":"GiB (1024^3 bytes)","formula":"weights = params x bits / 8; kv = 2 x layers x kv_heads x head_dim x min(ctx, window) x kv_bytes x batch; total = (weights + kv) x (1 + overhead%)"}