# Gemma 4 graph optimizations improve prompt processing performance

Digest AI · Research · published 2026-10-29T00:00:00Z

Canonical: https://digestai.news/story/gemma-4-graph-optimizations-improve-prompt-processing-performance

## Summary

A pull request by Max Krasnyansky to the `llama.cpp` repository introduced optimizations for Gemma 4’s graph processing. The changes, picked on November 2, 2026, focus on prompt processing, including pruning, mask cropping, and speculative decoding improvements. The updates aim to reduce unnecessary tensor copying and memory overhead during multi-layer processing, particularly when backends differ (e.g., GPU vs. CPU layers). Krasnyansky notes that unified-memory backends already avoid these copies, but the optimizations still apply to others, restoring `ggml_cont()` for contiguous tensor handling and refining speculative decoding logic for shared-layer attention.

The PR highlights ongoing work to further optimize microbatch processing and post-KV-cache operations. Key changes include pruning unused prompt rows before shared layers, avoiding redundant hidden-state extraction for output-free prompt chunks, and preserving KV prefix embeddings. The goal is to maintain performance gains in speculative decoding while preserving existing MTP and classic speculative decoding behavior. The pull request references co-assistance from Claude (Opus 5.5) for conflict resolution and notes unresolved tasks like refining speculative-simple mode for shared-KV MTP.

## Key points

- Pull request by Max Krasnyansky optimizes Gemma 4’s graph for prompt processing on November 2, 2026
- Changes restore `ggml_cont()` to reduce tensor copying and memory overhead across backends
- Speculative decoding logic refined to avoid redundant hidden-state extraction in shared-layer attention

## Why it matters

Optimizations could significantly reduce latency and memory usage for Gemma 4 inference, especially in mixed-backend setups, benefiting deployments requiring high throughput.

## Sources

1. [gemma4: graph optimizations (massive gains on pp for E2B and E4B) by max-krasnyansky · Pull Request #31356 · ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp/pull/31356) (github.com, 2026-10-29, primary source)

## Cite

Digest AI, "Gemma 4 graph optimizations improve prompt processing performance", 29 October 2026, https://digestai.news/story/gemma-4-graph-optimizations-improve-prompt-processing-performance

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/gemma-4-graph-optimizations-improve-prompt-processing-performance.json