{"version":1,"type":"story","url":"https://digestai.news/story/anyone-knows-anything-about-this-quantized-k3-1m-worth-it","json":"https://digestai.news/story/anyone-knows-anything-about-this-quantized-k3-1m-worth-it.json","markdown":"https://digestai.news/story/anyone-knows-anything-about-this-quantized-k3-1m-worth-it.md","slug":"anyone-knows-anything-about-this-quantized-k3-1m-worth-it","headline":"Anyone knows anything about this Quantized K3.1m Worth it?","summary":"A Reddit user posted about a quantized, expert-pruned checkpoint derived from moonshotai/Kimi-K3.1-Pro, noting it removes 75% of routed experts and is intended for vLLM serving and evaluation smoke tests. The user states it is a calibration-free, safetensors-only prototype with expected large quality regression relative to the source model. The checkpoint loads and serves on eight NVIDIA H200 GPUs using vLLM, with steady GPU memory around 127,797 MiB per GPU. Evaluation on GSM8K with 5-shot settings yielded a score, though the exact percentage is not clearly stated in the post. The model is distributed under the upstream Kimi K3.1 Pro License.","keyPoints":["Checkpoint removes 75% of routed experts, retaining 224 per MoE layer from original 896","On-disk size is about 443G across 96 safetensors shards plus weight index","Served on 8x NVIDIA H200 GPUs with TP=8, steady GPU memory ~127,797 MiB per GPU"],"whyItMatters":"The checkpoint offers a way to run large MoE models with reduced expert count for testing, though with significant expected quality loss, relevant for researchers exploring efficient inference.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["moonshotai","mgoin","DaoCloud","NVIDIA"],"models":["Kimi-K3.1-Pro","Kimi-K3-pruned75"],"people":[]},"firstPublishedAt":"2026-11-05T23:24:34Z","updatedAt":"2026-11-05T23:24:34Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"huggingface.co","title":"Anyone knows anything about this Quantized K3.1m Worth it?","url":"https://huggingface.co/DaoCloud/Kimi-K3.1-Pro-pruned75","publishedAt":"2026-11-05T23:24:34Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[{"site":"Reddit","url":"https://www.reddit.com/r/LocalLLaMA/comments/1xoxkq8/anyone_knows_anything_about_this_quantized_k31m/","points":null}],"thread":null,"cite":{"text":"Digest AI, \"Anyone knows anything about this Quantized K3.1m Worth it?\", 5 November 2026, https://digestai.news/story/anyone-knows-anything-about-this-quantized-k3-1m-worth-it","publisher":"Digest AI","title":"Anyone knows anything about this Quantized K3.1m Worth it?","datePublished":"2026-11-05T23:24:34Z","url":"https://digestai.news/story/anyone-knows-anything-about-this-quantized-k3-1m-worth-it"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}