{"version":1,"type":"story","url":"https://digestai.news/story/bartowski-releases-xing4-0-29b-a4b-quantized-models-for-llama-cpp","json":"https://digestai.news/story/bartowski-releases-xing4-0-29b-a4b-quantized-models-for-llama-cpp.json","markdown":"https://digestai.news/story/bartowski-releases-xing4-0-29b-a4b-quantized-models-for-llama-cpp.md","slug":"bartowski-releases-xing4-0-29b-a4b-quantized-models-for-llama-cpp","headline":"Bartowski releases Xing4.0-29B-A4B quantized models for llama.cpp","summary":"A user named bartowski has shared quantized versions of the **Xing4.0-29B-A4B** model on Hugging Face, optimized for **llama.cpp** with **speculative decoding** and **imatrix calibration**. The original model has **31 billion parameters** and supports text input. Quantized versions range from **20.06 GB (Q4\\(K\\)M)** to smaller sizes, designed for faster inference on CPUs and GPUs. The quantizations include custom layouts tuned for performance, with details on trade-offs between speed and quality, such as **Q4\\(K\\)M** (90% base precision) and **IQ3\\(M\\)** (GPU-optimized). The files are compatible with tools like **LM Studio, Jan AI, and Text Generation Web UI**, and the author provides calibration data and instructions for running them efficiently on hardware with limited RAM or VRAM.","keyPoints":["bartowski released quantized versions of **Xing4.0-29B-A4B** for **llama.cpp**, with **speculative decoding** and **imatrix calibration**","Largest quantized file is **20.06 GB (Q4\\(K\\)M)**, optimized for speed and quality trade-offs on CPUs/GPUs","Models support **text input only** and work with tools like **LM Studio, Jan AI, and Text Generation Web UI**"],"whyItMatters":"Quantized versions of large models like this enable developers and hobbyists to run advanced AI locally without high-end hardware. The speculative decoding and imatrix optimizations could improve inference speed for text generation tasks, making powerful models more accessible for research and small-scale applications.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["Hugging Face"],"models":["Xing4.0-29B-A4B","llama.cpp"],"people":["bartowski"]},"firstPublishedAt":"2026-11-06T19:19:24Z","updatedAt":"2026-11-06T19:19:24Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"huggingface.co","title":"bartowski/XingChen-AGI_Xing4.0-29B-A4B-GGUF · Hugging Face","url":"https://huggingface.co/bartowski/XingChen-AGI_Xing4.0-29B-A4B-GGUF","publishedAt":"2026-11-06T19:19:24Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[{"site":"Reddit","url":"https://www.reddit.com/r/LocalLLaMA/comments/1xplkdu/bartowskixingchenagi_xing4029ba4bgguf_hugging_face/","points":null}],"thread":{"title":"Bartowski releases Xing4.0-29B-A4B quantized models for llama.cpp","url":"https://digestai.news/thread/marktechpost-publishes-tutorial-on-running-underdog-saluki-27b-quantized-model","storyCount":2},"cite":{"text":"Digest AI, \"Bartowski releases Xing4.0-29B-A4B quantized models for llama.cpp\", 6 November 2026, https://digestai.news/story/bartowski-releases-xing4-0-29b-a4b-quantized-models-for-llama-cpp","publisher":"Digest AI","title":"Bartowski releases Xing4.0-29B-A4B quantized models for llama.cpp","datePublished":"2026-11-06T19:19:24Z","url":"https://digestai.news/story/bartowski-releases-xing4-0-29b-a4b-quantized-models-for-llama-cpp"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}