git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git gc --auto aquires *.lock files that make a subsequent git-fetch error out

From
Jeff King <peff@peff.net>
Date
Jul 12, 2017, 20:43 UTC
Message-ID
<20170712204308.2emzv5y3rndflmpg@sigill.intra.peff.net>
In-Reply-To
<87a849z9cu.fsf@gmail.com>
On Wed, Jul 12, 2017 at 10:30:25PM +0200, Ævar Arnfjörð Bjarmason wrote:
Show 27 quoted lines
> > Is it really "in a row" that's a problem? The second fetch should not
> > begin until the first one is done, including until its auto-gc exits.
> > And even with background gc, we do the ref-locking operations first, due
> > to 62aad1849 (gc --auto: do not lock refs in the background,
> > 2014-05-25).
> >
> >> I happen to run into this on a git.git which has a lot of remotes (most
> >> people on-list whose remotes I know about) and fetch them in parallel:
> >>
> >>     $ git config alias.pfetch
> >>     !parallel 'git fetch {}' ::: $(git remote)
> >
> > Ah, so it's not in a row. It's parallel. Then yes, you may run into
> > problems with the gc locks conflicting with real operations. This isn't
> > really unique to fetch. Any simultaneous operation can run into problems
> > (e.g., on a busy server repo you may see conflicts between pack-refs and
> > regular pushes).
> 
> This is what I thought at first, and I've only encountered the issue in
> this parallel mode (mainly because it's tedious to reproduce). But I
> think the traces below show that it would happen with "git fetch --all"
> & "git remote update" as well, so the parallel invocations didn't
> matter.
> 
> I.e. I'd just update my first remote, then git-gc would start in the
> background and lock refs for my other remotes, which I'd then fail to
> update.

No, it should be OK because of the commit I mentioned at the top of the quoted section. Each one runs sequentially.

-Peff
Previous: Ævar Arnfjörð Bjarmason
Message 21 of 21 in “optimizing pack access on "read only" fetch repos”
  1. 0/2 optimizing pack access on "read only" fetch reposJeff King, Jan 26, 2013
  2. 1/2 fetch: run gc --auto after fetchingJeff King, Jan 26, 2013
  3. Jonathan NiederJan 27, 2013
  4. 2/2 fetch-pack: avoid repeatedly re-scanning pack directoryJeff King, Jan 26, 2013
  5. Jonathan NiederJan 27, 2013
  6. Junio C HamanoJan 27, 2013
  7. Jonathan NiederJan 27, 2013
  8. Junio C HamanoJan 27, 2013
  9. Shawn PearceJan 29, 2013
  10. Jeff KingJan 29, 2013
  11. Martin FickJan 29, 2013
  12. Junio C HamanoJan 29, 2013
  13. Jeff KingJan 29, 2013
  14. Junio C HamanoJan 29, 2013
  15. Shawn PearceJan 31, 2013
  16. Jeff KingFeb 1, 2013
  17. Shawn PearceFeb 2, 2013
  18. Duy NguyenJan 29, 2013
  19. Jeff KingJul 12, 2017
  20. Ævar Arnfjörð BjarmasonJul 12, 2017
  21. Jeff KingJul 12, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.