{"version":1,"type":"story","url":"https://digestai.news/story/bitcos-layout-cuts-ternary-llm-storage-below-1-58-bits-per-weight","json":"https://digestai.news/story/bitcos-layout-cuts-ternary-llm-storage-below-1-58-bits-per-weight.json","markdown":"https://digestai.news/story/bitcos-layout-cuts-ternary-llm-storage-below-1-58-bits-per-weight.md","slug":"bitcos-layout-cuts-ternary-llm-storage-below-1-58-bits-per-weight","headline":"Bitcos layout cuts ternary LLM storage below 1.58 bits per weight","summary":"Researchers have introduced BITCOS, a distribution‑adaptive storage format for ternary large language models that leverages the high prevalence of zero weights. By encoding a dense presence bitmap together with a compact sign vector, BITCOS reduces the effective bit‑width to 2 – z bits per weight, where z is the zero density. In tests on 29 ternary LLMs, the method outperformed the traditional five‑trit packing in 26 cases, achieving as low as 1.485 bits per weight on the sparsest model.\n\nThe authors also optimized unpacking routines for modern CPUs and GPUs, including AVX‑512, AVX2, and Intel Xe2 GPUs. Benchmarks show up to a 1.28× speedup in matrix‑vector multiplication and end‑to‑end inference gains of 1.18× on CPUs and 1.27× on GPUs across five hardware platforms. These improvements make ternary LLMs more storage‑efficient and faster to run, potentially lowering deployment costs for edge and server environments.","keyPoints":["BITCOS reduces ternary LLM storage to as low as 1.485 bits per weight.","Achieves up to 1.28× faster matrix‑vector multiplication versus five‑trit packing.","Inference throughput improves up to 1.27× on Intel Xe2 GPUs."],"whyItMatters":"Lower storage and faster inference make ternary LLMs viable for cost‑sensitive deployments, expanding AI capabilities on limited‑resource hardware.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["Intel"],"models":[],"people":[]},"firstPublishedAt":"2026-09-16T20:59:24Z","updatedAt":"2026-09-16T20:59:24Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arxiv.org","title":"Breaking the 1.58-bit Barrier for Ternary LLMs","url":"https://arxiv.org/abs/2609.16338","publishedAt":"2026-09-16T20:59:24Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[{"site":"Hacker News","url":"https://news.ycombinator.com/item?id=49732931","points":67}],"thread":null,"cite":{"text":"Digest AI, \"Bitcos layout cuts ternary LLM storage below 1.58 bits per weight\", 16 September 2026, https://digestai.news/story/bitcos-layout-cuts-ternary-llm-storage-below-1-58-bits-per-weight","publisher":"Digest AI","title":"Bitcos layout cuts ternary LLM storage below 1.58 bits per weight","datePublished":"2026-09-16T20:59:24Z","url":"https://digestai.news/story/bitcos-layout-cuts-ternary-llm-storage-below-1-58-bits-per-weight"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}