git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2] fetch-pack: don't try to fetch peel values with --all

From
Kirill Smelkov <kirr@nexedi.com>
Date
Jun 14, 2018, 05:29 UTC
Message-ID
<20180614052938.GB2861@deco.navytux.spb.ru>
In-Reply-To
<20180613211302.GB15574@sigill.intra.peff.net>
On Wed, Jun 13, 2018 at 05:13:02PM -0400, Jeff King wrote:
Show 29 quoted lines
> On Tue, Jun 12, 2018 at 06:54:17PM +0000, Kirill Smelkov wrote:
> 
> > > If an extra connection isn't a problem, you might be better off with
> > > "git ls-remote", and then picking through the results for refs of
> > > interest, and then "git fetch-pack" to actually get the pack. That's how
> > > git-fetch worked when it was a shell script (e.g., see c3a200120d, the
> > > last shell version).
> > 
> > Yes, this is what I ended up doing:
> > 
> > https://lab.nexedi.com/kirr/git-backup/commit/899103bf
> > 
> > but for another reason - to avoid repeating for every fetched repository
> > slow (in case of my "big" destination backup repository) quickfetch()
> > checking in every spawned `git fetch`: git-backup can build index of
> > objects we already have ourselves only once at startup, and then in
> > fetch, after checking lsremote output, consult that index, and if we see
> > we already have everything for an advertised reference - just avoid
> > giving it to fetch-pack to process. It turns out for many pulled
> > repositories there is usually no references changed at all and this way
> > fetch-pack can be skipped completely:
> > 
> > https://lab.nexedi.com/kirr/git-backup/commit/3efed898
> 
> Thanks for sharing that, it's an interesting case. I'd hope that
> git-fetch is smart enough not to even bother with quickfetch() if there
> are no refs to update. But if we have even one change to fetch, then
> yeah, in the general case it makes sense to me that you could do better
> by amortizing the scan of local objects across many operations.

Thanks for feedback. For the reference in case of git-backup `git fetch` or `git fetch-pack` would have to always do quickfetch scan or equivalent, because in case of backup repo there is only one reference in it - its master - and references of backed up repositories do not have anything representing them in backup.git/refs/ .

Kirill
Previous: Jeff King
Message 21 of 21 in “fetch-pack: demonstrate --all breakage when remote have tags to non-commit objects”
  1. fetch-pack: demonstrate --all breakage when remote have tags to non-commit objectsKirill Smelkov, Jun 10, 2018
  2. Jeff KingJun 11, 2018
  3. fetch-pack: don't try to fetch peeled values with --allJeff King, Jun 11, 2018
  4. Eric SunshineJun 11, 2018
  5. fetch-pack: don't try to fetch peel values with --allJeff King, Jun 11, 2018
  6. Kirill SmelkovJun 11, 2018
  7. Jeff KingJun 12, 2018
  8. Kirill SmelkovJun 12, 2018
  9. fetch-pack: test explicitly that --all can fetch tag references pointing to non-commitsKirill Smelkov, Jun 13, 2018
  10. Junio C HamanoJun 13, 2018
  11. Kirill SmelkovJun 13, 2018
  12. Jeff KingJun 13, 2018
  13. Jeff KingJun 13, 2018
  14. Kirill SmelkovJun 14, 2018
  15. Junio C HamanoJun 14, 2018
  16. Kirill SmelkovJun 14, 2018
  17. fetch-pack: demonstrate --all failure when remote is emptyKirill Smelkov, Jun 13, 2018
  18. Junio C HamanoJun 13, 2018
  19. Kirill SmelkovJun 13, 2018
  20. Jeff KingJun 13, 2018
  21. Kirill SmelkovJun 14, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.