git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC/PATCH 0/2] Speed up fetch with large number of tags

From
Julian Phillips <julian@quantumfyre.co.uk>
Date
Sep 16, 2009, 07:53 UTC
Message-ID
<20090916074737.58044.42776.julian@quantumfyre.co.uk>

I have a repository at $dayjob where fetch was taking ~30s to tell me that there were no updates.

It turns out that I appear to have added a nasty linear search of all remote refs for every commit (i.e. tag^{}) tag ref way back in the original C implementation of fetch. This doesn't scale well to large numbers of refs, so this replaces it with a hash table based lookup instead, which brings the time down to a few seconds even for very large ref counts.

I haven't tested it with non-native transports, but there is no reason to believe that the code should be transport specific.

Julian Phillips (2):
  ref-dict: Add a set of functions for working with a ref dictionary
  fetch: Speed up fetch by using ref dictionary
 Makefile        |    1 +
 builtin-fetch.c |   19 ++++++-------
 ref-dict.c      |   76 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 ref-dict.h      |   13 +++++++++
 4 files changed, 99 insertions(+), 10 deletions(-)
 create mode 100644 ref-dict.c
 create mode 100644 ref-dict.h
Next: Julian Phillips
Message 1 of 17 in “Speed up fetch with large number of tags”
  1. 0/2 Speed up fetch with large number of tagsJulian Phillips, Sep 16, 2009
  2. 1/2 ref-dict: Add a set of functions for working with a ref dictionaryJulian Phillips, Sep 16, 2009
  3. 2/2 fetch: Speed up fetch by using ref dictionaryJulian Phillips, Sep 16, 2009
  4. Junio C HamanoSep 16, 2009
  5. Julian PhillipsSep 16, 2009
  6. Shawn O. PearceSep 16, 2009
  7. Junio C HamanoSep 16, 2009
  8. Shawn O. PearceSep 16, 2009
  9. Junio C HamanoSep 16, 2009
  10. fetch: Speed up fetch by rewriting find_non_local_tagsJulian Phillips, Sep 16, 2009
  11. Junio C HamanoSep 16, 2009
  12. Julian PhillipsSep 16, 2009
  13. Julian PhillipsSep 17, 2009
  14. Johan HerlandSep 17, 2009
  15. fetch: Speed up fetch by rewriting find_non_local_tagsJulian Phillips, Sep 17, 2009
  16. Shawn O. PearceSep 16, 2009
  17. Junio C HamanoSep 22, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.