Mobiles & Gadgets

Google Releases Gemma 4 QAT Models to Improve On-Device Efficiency

· GlobeWire Daily Editorial Desk · Source: blog.google

1 min read

Key takeawaysGoogle DeepMind has announced the release of new Quantization-Aware Training (QAT) checkpoints for the Gemma 4 family, aiming to reduce memory requirements and improve performance on edge devices, mobile phones, and consumer GPUs.
Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency

Image: Google · Photo source · Resized and letterboxed to 16:9; no AI generation

Google DeepMind has announced the release of new Quantization-Aware Training (QAT) checkpoints for the Gemma 4 family, aiming to reduce memory requirements and improve performance on edge devices, mobile phones, and consumer GPUs.

Since the release of Gemma 4 two months ago, updates have included Multi-Token Prediction (MTP) and a 12B model. The newly announced QAT checkpoints simulate quantization during training to minimize quality loss compared to standard post-training quantization methods.

Key Compression Techniques

  • Q4_0 and Mobile Formats: Includes checkpoints for the Q4_0 format and a specialized mobile schema.
  • Reduced Memory Footprint: The Gemma 4 E2B text-only model reportedly requires less than 1GB of memory under specific configurations.
  • Static Activations and Channel-Wise Quantization: Designed to lower processing overhead on mobile chips.
  • Targeted 2-bit Quantization: Compresses token generation components while maintaining higher precision in core reasoning layers.

Ecosystem Integration and Availability

Developers can access the model weights on Hugging Face. The release supports various integration workflows:

  • Desktop Tools: Compatibility with llama.cpp, Ollama, and LM Studio.
  • Runtimes: Support for Google's LiteRT-LM and Transformers.js for web deployment.
  • Ecosystem Frameworks: Integration options using SGLang, vLLM, MLX for Apple Silicon, and fine-tuning support via Unsloth.

Users can verify these updates and access technical documentation directly through the official Google developers blog and associated repository links.

Share this article
Read the original source: Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency - blog.google

AI-assisted reporting; source claims are attributed and have not been independently verified.

About the author

GlobeWire Daily Editorial Desk — AI-assisted news based on attributed published sources.

More to read