From 2,500 Requests to 24: Rebuilding Star History Without Stargazers
· GlobeWire Daily Tech Desk · Source: github.blog

Image: Taylor Cuilty, Sarita Iyer · Photo source · Resized and letterboxed to 16:9; no AI generation
Maintainers look at star history to answer practical questions about project trajectories around launches or events. Historically, obtaining this curve via the GitHub API required requesting pages of individual stargazer records, keeping timestamps, and discarding identities. Over time, this design posed privacy risks from scraping and caused database performance issues on highly starred repositories due to expensive counting and pagination.
After restricting bulk stargazer and watcher list access to repository administrators and collaborators to mitigate immediate risks, GitHub consulted with maintainers and tool builders. They discovered that tools like Star History needed aggregate counts and dates rather than individual usernames. This collaboration led to the development of a purpose-built endpoint designed to deliver historical trends efficiently without exposing user identities.
Understanding the Original Design Limitations
The original stargazer endpoint was built to list people who starred a repository rather than to serve as a historical event log. Reconstructing growth curves meant retrieving thousands of individual identities and processing every current star. Pagination bugs further exacerbated database load, causing thousands of slow queries and timeouts.
Designing an Efficient Aggregate Endpoint
Initial attempts to return timestamps without identities still required pagination through thousands of individual records. Simple SQL aggregation with `LIMIT` and `OFFSET` reduced JSON payload sizes but failed to reduce expensive database grouping and sorting across millions of rows.
The final solution shifts the order of operations by filtering stars by calendar time before counting:
- Time-Range Filtering: Requests translate page parameters into indexed `created_at` date ranges.
- Optimized Requests: A single request now returns up to 30 weeks of daily star counts.
- Reduced Database Load: Instead of scanning all current stars repeatedly, popular repositories like `react/react` now require approximately 24 requests instead of 2,511.
Practical Implications and Verification
The new aggregate endpoint protects user privacy while providing maintainers with predictable performance. However, scaling limitations remain for exceptionally dense weeks with unusually high star activity. Developers can verify documentation and share feedback directly through official GitHub community discussions.
Key Takeaways
- Focus on the actual use case rather than legacy data structures.
- Minimize data exposure by returning only what is strictly necessary.
- Filter data before aggregation to reduce database query costs.
AI-assisted reporting; source claims are attributed and have not been independently verified.



