Apps & Software

Inside GitHub’s New Git Storage Architecture

· GlobeWire Daily Editorial Desk · Source: github.blog

2 min read

Key takeawaysGitHub has announced its new Git storage architecture designed to handle thousands of writes per second while scaling reads on demand. Built to support agentic development for hundreds of millions of developers, the system underpins continuous operations on GitHub while running in production.
Inside GitHub’s New Git Storage Architecture

Image: Karthik Raman, Brian Celenza · Photo source · Resized and letterboxed to 16:9; no AI generation

GitHub has announced its new Git storage architecture designed to handle thousands of writes per second while scaling reads on demand. Built to support agentic development for hundreds of millions of developers, the system underpins continuous operations on GitHub while running in production.

Core Principles and Scale

Over the past year, automated agents have changed the traffic patterns of busy repositories on GitHub. To prepare for sustained high-volume activity rather than brief bursts, the architecture follows two fundamental design principles:

  • Minimize coordination: Only reference updates require agreement, allowing data storage, request serving, and repository maintenance to scale independently.
  • Decouple storage from compute: Immutable Git packs and auxiliary read data reside in Azure Blob Storage, while references and pack manifests use a replicated log service called Ref-CAS.

Internal testing demonstrated support for 5,000 concurrent agents working on a single repository and 7,200 pushes per second, while maintaining consistently low read latency.

Component Workflow

Requests from Git clients, APIs, and the web UI arrive at Spokes for routing and load balancing. Spokes assigns traffic to virtual replicas—disposable compute nodes that serve reads and writes using a locally cached disk that can be rebuilt at any time.

  • Writes: A virtual replica indexes incoming packs, uploads them to Azure Blob Storage, and commits a single transaction to Ref-CAS listing the pack in the manifest and updating references.
  • Reads: A virtual replica checks its reference version with Ref-CAS, pulls necessary packs from Blob Storage into its local cache, and serves requests.
  • Compaction: Dedicated background nodes work directly against Blob Storage and Ref-CAS to merge packs and remove unreachable objects without consuming virtual replica compute resources.

Future Roadmap and Availability

GitHub plans several upcoming optimizations, including targeted object fetches from Blob Storage, type-organized packs, partial replicas, and global replica placement. Additionally, features such as work-without-a-clone capabilities and atomic commit APIs are in development.

The new architecture is currently available in a limited private preview, with a broader rollout scheduled for early next year. Users can verify announcements and updates directly through the official GitHub Blog.

Practical Implications

  • Unchanged Developer Workflow: Developers continue to push, fetch, and branch using standard Git commands and protections.
  • Independent Scaling: Repositories can dynamically scale maintenance and read-write workloads without requiring custom manual migrations.
Share this article
Read the original source: Inside GitHub’s new Git storage architecture

AI-assisted reporting; source claims are attributed and have not been independently verified.

About the author

GlobeWire Daily Editorial Desk — AI-assisted news based on attributed published sources.

More to read