git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2] fetch-pack: don't try to fetch peel values with --all

From
Jeff King <peff@peff.net>
Date
Jun 13, 2018, 21:13 UTC
Message-ID
<20180613211302.GB15574@sigill.intra.peff.net>
In-Reply-To
<20180612185413.GA21856@deco.navytux.spb.ru>
On Tue, Jun 12, 2018 at 06:54:17PM +0000, Kirill Smelkov wrote:
Show 21 quoted lines
> > If an extra connection isn't a problem, you might be better off with
> > "git ls-remote", and then picking through the results for refs of
> > interest, and then "git fetch-pack" to actually get the pack. That's how
> > git-fetch worked when it was a shell script (e.g., see c3a200120d, the
> > last shell version).
> 
> Yes, this is what I ended up doing:
> 
> https://lab.nexedi.com/kirr/git-backup/commit/899103bf
> 
> but for another reason - to avoid repeating for every fetched repository
> slow (in case of my "big" destination backup repository) quickfetch()
> checking in every spawned `git fetch`: git-backup can build index of
> objects we already have ourselves only once at startup, and then in
> fetch, after checking lsremote output, consult that index, and if we see
> we already have everything for an advertised reference - just avoid
> giving it to fetch-pack to process. It turns out for many pulled
> repositories there is usually no references changed at all and this way
> fetch-pack can be skipped completely:
> 
> https://lab.nexedi.com/kirr/git-backup/commit/3efed898

Thanks for sharing that, it's an interesting case. I'd hope that git-fetch is smart enough not to even bother with quickfetch() if there are no refs to update. But if we have even one change to fetch, then yeah, in the general case it makes sense to me that you could do better by amortizing the scan of local objects across many operations.

-Peff
Previous: Kirill SmelkovNext: Kirill Smelkov
Message 20 of 21 in “fetch-pack: demonstrate --all breakage when remote have tags to non-commit objects”
  1. fetch-pack: demonstrate --all breakage when remote have tags to non-commit objectsKirill Smelkov, Jun 10, 2018
  2. Jeff KingJun 11, 2018
  3. fetch-pack: don't try to fetch peeled values with --allJeff King, Jun 11, 2018
  4. Eric SunshineJun 11, 2018
  5. fetch-pack: don't try to fetch peel values with --allJeff King, Jun 11, 2018
  6. Kirill SmelkovJun 11, 2018
  7. Jeff KingJun 12, 2018
  8. Kirill SmelkovJun 12, 2018
  9. fetch-pack: test explicitly that --all can fetch tag references pointing to non-commitsKirill Smelkov, Jun 13, 2018
  10. Junio C HamanoJun 13, 2018
  11. Kirill SmelkovJun 13, 2018
  12. Jeff KingJun 13, 2018
  13. Jeff KingJun 13, 2018
  14. Kirill SmelkovJun 14, 2018
  15. Junio C HamanoJun 14, 2018
  16. Kirill SmelkovJun 14, 2018
  17. fetch-pack: demonstrate --all failure when remote is emptyKirill Smelkov, Jun 13, 2018
  18. Junio C HamanoJun 13, 2018
  19. Kirill SmelkovJun 13, 2018
  20. Jeff KingJun 13, 2018
  21. Kirill SmelkovJun 14, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.