git/list[1] front-page[2] threads[3] people[4] search[5] about
 

RE: [PATCH] repack: respect gc.pid lock

From
DTDavid Turner <david.turner@twosigma.com>
Date
Apr 18, 2017, 17:43 UTC
Message-ID
<710ded65bb8843ab838d9c52cd796317@exmbdft7.ad.twosigma.com>
In-Reply-To
<20170418171930.zad5wrbu5rvdsmg5@sigill.intra.peff.net>
Show 48 quoted lines
> -----Original Message-----
> From: Jeff King [mailto:peff@peff.net]
> Sent: Tuesday, April 18, 2017 1:20 PM
> To: David Turner <David.Turner@twosigma.com>
> Cc: git@vger.kernel.org; christian.couder@gmail.com; mfick@codeaurora.org;
> jacob.keller@gmail.com
> Subject: Re: [PATCH] repack: respect gc.pid lock
> 
> On Tue, Apr 18, 2017 at 05:16:52PM +0000, David Turner wrote:
> 
> > > -----Original Message-----
> > > From: Jeff King [mailto:peff@peff.net]
> > > Sent: Monday, April 17, 2017 11:42 PM
> > > To: David Turner <David.Turner@twosigma.com>
> > > Cc: git@vger.kernel.org; christian.couder@gmail.com;
> > > mfick@codeaurora.org; jacob.keller@gmail.com
> > > Subject: Re: [PATCH] repack: respect gc.pid lock
> > >
> > > On Mon, Apr 17, 2017 at 11:29:18PM +0000, David Turner wrote:
> > >
> > > > We saw this failure in the logs multiple  times (with three
> > > > different shas, while a gc was running):
> > > > April 12, 2017 06:45 -> ERROR -> 'git -c repack.writeBitmaps=true
> > > > repack -A -d
> > > --pack-kept-objects' in [repo] failed:
> > > > fatal: packfile ./objects/pack/pack-[sha].pack cannot be accessed
> > > > Possibly some other repack was also running at the time as well.
> > > >
> > > > My colleague also saw it while manually doing gc (again while
> > > > repacks were likely to be running):
> > >
> > > This is sort of a side question, but...why are you running other
> > > repacks alongside git-gc? It seems like you ought to be doing one or the
> other.
> >
> > But actually, it would be kind of nice if git would help protect us from doing
> this?
> 
> A lock can catch the racy cases where both run at the same time. But I think that
> even:
> 
>   git -c repack.writeBitmaps=true repack -Ad
>   [...wait...]
>   git gc
> 
> is questionable, because that gc will erase your bitmaps. How does git-gc know
> that it's doing a bad thing by repacking without bitmaps, and that you didn't
> simply change your configuration or want to get rid of them?

Sorry, the gc in Gitlab does keep bitmaps. The one I quoted in a previous message doesn't, because the person typing the command was just doing some manual testing and I guess didn't realize that bitmaps were important. Or perhaps he knew that repack.writeBitmaps was already set in the config.

So given that the lock will catch the races, might it be a good idea (if Implemented to avoid locking on repack -d)?

Previous: Jeff KingNext: Jeff King
Message 10 of 13 in “repack: respect gc.pid lock”
  1. repack: respect gc.pid lockDavid Turner, Apr 13, 2017
  2. Jacob KellerApr 14, 2017
  3. Jeff KingApr 14, 2017
  4. David TurnerApr 17, 2017
  5. Jeff KingApr 18, 2017
  6. David TurnerApr 18, 2017
  7. Jeff KingApr 18, 2017
  8. David TurnerApr 18, 2017
  9. Jeff KingApr 18, 2017
  10. David TurnerApr 18, 2017
  11. Jeff KingApr 18, 2017
  12. David TurnerApr 20, 2017
  13. Jeff KingApr 20, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.