git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] parse_object: try internal cache before reading object db

From
Jeff King <peff@peff.net>
Date
Jan 6, 2012, 19:16 UTC
Message-ID
<20120106191654.GA11022@sigill.intra.peff.net>
In-Reply-To
<7vipkpn87d.fsf@alter.siamese.dyndns.org>
On Thu, Jan 05, 2012 at 01:35:50PM -0800, Junio C Hamano wrote:
Show 6 quoted lines
> > For example, GitHub's alternates repository for git.git has
> > ~120,000 refs, of which only ~3200 are unique. The time for
> > upload-pack to print its list of advertised refs dropped
> > from 3.4s to 0.76s.
> 
> Nice. I am more impressed by 120k/3.4 than 3.2k/0.76, though ;-)

Actually, we can do much better than that. Here are a few patches that avoid parsing objects when possible. They drop the 3.4s to 2.0s. If you combine them with the parse_object optimization, my 120K case drops to around 0.68s.

I don't know if it is really that worth it on top of the parse_object optimization. It's almost negligible for the normal case (though I get a tiny speedup on my ~900-ref git.git repo), and a minor speedup on the crazy alternates case. OTOH, if you had some totally insane ref structure, like 120K _unique_ refs (which would probably imply that you're making one ref per commit or something silly like that. But hey, people have suggested it in the past), then it could be a big improvement.

  [1/2]: upload-pack: avoid parsing objects during ref advertisement
  [2/2]: upload-pack: avoid parsing tag destinations
-Peff
Previous: Jeff KingNext: Junio C Hamano
Message 6 of 17 in “parse_object: try internal cache before reading object db”
  1. parse_object: try internal cache before reading object dbJeff King, Jan 5, 2012
  2. Junio C HamanoJan 5, 2012
  3. Jeff KingJan 5, 2012
  4. Junio C HamanoJan 5, 2012
  5. Jeff KingJan 5, 2012
  6. Jeff KingJan 6, 2012
  7. Junio C HamanoJan 6, 2012
  8. Jeff KingJan 6, 2012
  9. Junio C HamanoJan 6, 2012
  10. Jeff KingJan 6, 2012
  11. 1/2 upload-pack: avoid parsing objects during ref advertisementJeff King, Jan 6, 2012
  12. Junio C HamanoJan 18, 2013
  13. Jeff KingJan 24, 2013
  14. Junio C HamanoJan 24, 2013
  15. Shawn PearceJan 29, 2013
  16. Jeff KingJan 29, 2013
  17. 2/2 upload-pack: avoid parsing tag destinationsJeff King, Jan 6, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.