# Simultaneous gc and repack 7 messages from 2017-04-13 to 2017-04-13. Participants: David Turner, Jacob Keller, Martin Fick. Thread: https://gitlist.dev/t/45683 ## David Turner, 2017-04-13 17:31 Subject: Simultaneous gc and repack Message-ID: <1492104698.1527.79.camel@novalis.org> URL: https://gitlist.dev/e/1492104698.1527.79.camel%40novalis.org ``` Git gc locks the repository (using a gc.pid file) so that other gcs don't run concurrently. But git repack doesn't respect this lock, so it's possible to have a repack running at the same time as a gc. This makes the gc sad when its packs are deleted out from under it with: "fatal: ./objects/pack/pack-$sha.pack cannot be accessed". Then it dies, leaving a large temp file hanging around. Does the following seem reasonable? 1. Make git repack, by default, check for a gc.pid file (using the same logic as git gc itself does). 2. Provide a --force option to git repack to ignore said check. 3. Make git gc provide that --force option when it calls repack under its own lock. This came up because Gitlab runs a repack after every N pushes and a gc after every M commits, where M >> N. Sometimes, when pushes come in rapidly, the repack catches the still-running gc and the above badness happens. At least, that's my understanding: I don't run our Gitlab servers, but I talked to the person who does and that's what he said. Of course, Gitlab could do its own locking, but the general approach seems like it would help other folks too. ``` ## Jacob Keller, 2017-04-13 18:03 Subject: Re: Simultaneous gc and repack Message-ID: URL: https://gitlist.dev/e/CA%2BP7%2Bxo4mwA8DMeObq_T9%3DP9m7tG1d6afDS_Md56_EnhuXLVgw%40mail.gmail.com In-Reply-To: <1492104698.1527.79.camel@novalis.org> ``` On Thu, Apr 13, 2017 at 10:31 AM, David Turner wrote: > Git gc locks the repository (using a gc.pid file) so that other gcs > don't run concurrently. But git repack doesn't respect this lock, so > it's possible to have a repack running at the same time as a gc. This > makes the gc sad when its packs are deleted out from under it with: > "fatal: ./objects/pack/pack-$sha.pack cannot be accessed". Then it > dies, leaving a large temp file hanging around. > > Does the following seem reasonable? > > 1. Make git repack, by default, check for a gc.pid file (using the same > logic as git gc itself does). > 2. Provide a --force option to git repack to ignore said check. > 3. Make git gc provide that --force option when it calls repack under > its own lock. > What about just making the code that calls repack today just call gc instead? I guess it's more work if you don't strictly need it but..? Thanks, Jake > This came up because Gitlab runs a repack after every N pushes and a gc > after every M commits, where M >> N. Sometimes, when pushes come in > rapidly, the repack catches the still-running gc and the above badness > happens. At least, that's my understanding: I don't run our Gitlab > servers, but I talked to the person who does and that's what he said. > > Of course, Gitlab could do its own locking, but the general approach > seems like it would help other folks too. ``` ## Martin Fick, 2017-04-13 18:08 Subject: Re: Simultaneous gc and repack Message-ID: <10832632.7AIzXehjUi@mfick1-lnx> URL: https://gitlist.dev/e/10832632.7AIzXehjUi%40mfick1-lnx In-Reply-To: ``` On Thursday, April 13, 2017 11:03:14 AM Jacob Keller wrote: > On Thu, Apr 13, 2017 at 10:31 AM, David Turner wrote: > > Git gc locks the repository (using a gc.pid file) so > > that other gcs don't run concurrently. But git repack > > doesn't respect this lock, so it's possible to have a > > repack running at the same time as a gc. This makes > > the gc sad when its packs are deleted out from under it > > with: "fatal: ./objects/pack/pack-$sha.pack cannot be > > accessed". Then it dies, leaving a large temp file > > hanging around. > > > > Does the following seem reasonable? > > > > 1. Make git repack, by default, check for a gc.pid file > > (using the same logic as git gc itself does). > > 2. Provide a --force option to git repack to ignore said > > check. 3. Make git gc provide that --force option when > > it calls repack under its own lock. > > What about just making the code that calls repack today > just call gc instead? I guess it's more work if you don't > strictly need it but..? There are many scanerios where this does not achieve the same thing. On the obvious side, gc does more than repacking, but on the other side, repacking has many switches that are not available via gc. Would it make more sense to move the lock to repack instead of to gc? -Martin -- The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation ``` ## David Turner, 2017-04-13 18:28 Subject: Re: Simultaneous gc and repack Message-ID: <1492108087.1527.82.camel@novalis.org> URL: https://gitlist.dev/e/1492108087.1527.82.camel%40novalis.org In-Reply-To: <10832632.7AIzXehjUi@mfick1-lnx> ``` On Thu, 2017-04-13 at 12:08 -0600, Martin Fick wrote: > On Thursday, April 13, 2017 11:03:14 AM Jacob Keller wrote: > > On Thu, Apr 13, 2017 at 10:31 AM, David Turner  > > wrote: > > > Git gc locks the repository (using a gc.pid file) so > > > that other gcs don't run concurrently. But git repack > > > doesn't respect this lock, so it's possible to have a > > > repack running at the same time as a gc.  This makes > > > the gc sad when its packs are deleted out from under it > > > with: "fatal: ./objects/pack/pack-$sha.pack cannot be > > > accessed".  Then it dies, leaving a large temp file > > > hanging around. > > > > > > Does the following seem reasonable? > > > > > > 1. Make git repack, by default, check for a gc.pid file > > > (using the same logic as git gc itself does). > > > 2. Provide a --force option to git repack to ignore said > > > check. 3. Make git gc provide that --force option when > > > it calls repack under its own lock. > > > > What about just making the code that calls repack today > > just call gc instead? I guess it's more work if you don't > > strictly need it but..? > > There are many scanerios where this does not achieve the  > same thing.  On the obvious side, gc does more than  > repacking, but on the other side, repacking has many  > switches that are not available via gc. > > Would it make more sense to move the lock to repack instead  > of to gc? Other gc operations might step on each other too (e.g. packing refs). That would be less bad (and less common), but it still seems worth avoiding. ``` ## Jacob Keller, 2017-04-13 18:35 Subject: Re: Simultaneous gc and repack Message-ID: URL: https://gitlist.dev/e/CA%2BP7%2BxoQtGyWpuUKFDQzy1RhdzT7mhES%2BVOczNSSQceiQUkDbw%40mail.gmail.com In-Reply-To: <1492108087.1527.82.camel@novalis.org> ``` On Thu, Apr 13, 2017 at 11:28 AM, David Turner wrote: > On Thu, 2017-04-13 at 12:08 -0600, Martin Fick wrote: >> On Thursday, April 13, 2017 11:03:14 AM Jacob Keller wrote: >> > On Thu, Apr 13, 2017 at 10:31 AM, David Turner >> >> wrote: >> > > Git gc locks the repository (using a gc.pid file) so >> > > that other gcs don't run concurrently. But git repack >> > > doesn't respect this lock, so it's possible to have a >> > > repack running at the same time as a gc. This makes >> > > the gc sad when its packs are deleted out from under it >> > > with: "fatal: ./objects/pack/pack-$sha.pack cannot be >> > > accessed". Then it dies, leaving a large temp file >> > > hanging around. >> > > >> > > Does the following seem reasonable? >> > > >> > > 1. Make git repack, by default, check for a gc.pid file >> > > (using the same logic as git gc itself does). >> > > 2. Provide a --force option to git repack to ignore said >> > > check. 3. Make git gc provide that --force option when >> > > it calls repack under its own lock. >> > >> > What about just making the code that calls repack today >> > just call gc instead? I guess it's more work if you don't >> > strictly need it but..? >> >> There are many scanerios where this does not achieve the >> same thing. On the obvious side, gc does more than >> repacking, but on the other side, repacking has many >> switches that are not available via gc. >> >> Would it make more sense to move the lock to repack instead >> of to gc? > > Other gc operations might step on each other too (e.g. packing refs). > That would be less bad (and less common), but it still seems worth > avoiding. It sounds like your original solution would work, though I wouldn't use "force" and I would either not document or document with "this is only meant to be used by git-gc internally" Thanks, Jake ``` ## Martin Fick, 2017-04-13 18:36 Subject: Re: Simultaneous gc and repack Message-ID: <1553777.vJallt5N6j@mfick1-lnx> URL: https://gitlist.dev/e/1553777.vJallt5N6j%40mfick1-lnx In-Reply-To: <1492108087.1527.82.camel@novalis.org> ``` On Thursday, April 13, 2017 02:28:07 PM David Turner wrote: > On Thu, 2017-04-13 at 12:08 -0600, Martin Fick wrote: > > On Thursday, April 13, 2017 11:03:14 AM Jacob Keller wrote: > > > On Thu, Apr 13, 2017 at 10:31 AM, David Turner > > > > wrote: > > > > Git gc locks the repository (using a gc.pid file) so > > > > that other gcs don't run concurrently. But git > > > > repack > > > > doesn't respect this lock, so it's possible to have > > > > a > > > > repack running at the same time as a gc. This makes > > > > the gc sad when its packs are deleted out from under > > > > it > > > > with: "fatal: ./objects/pack/pack-$sha.pack cannot > > > > be > > > > accessed". Then it dies, leaving a large temp file > > > > hanging around. > > > > > > > > Does the following seem reasonable? > > > > > > > > 1. Make git repack, by default, check for a gc.pid > > > > file > > > > (using the same logic as git gc itself does). > > > > 2. Provide a --force option to git repack to ignore > > > > said > > > > check. 3. Make git gc provide that --force option > > > > when > > > > it calls repack under its own lock. > > > > > > What about just making the code that calls repack > > > today > > > just call gc instead? I guess it's more work if you > > > don't > > > strictly need it but..? > > > > There are many scanerios where this does not achieve > > the > > same thing. On the obvious side, gc does more than > > repacking, but on the other side, repacking has many > > switches that are not available via gc. > > > > Would it make more sense to move the lock to repack > > instead of to gc? > > Other gc operations might step on each other too (e.g. > packing refs). That would be less bad (and less common), > but it still seems worth avoiding. Yes, but all of thsoe operations need to be self protected already, or they risk the same issue. -Martin -- The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation ``` ## David Turner, 2017-04-13 19:05 Subject: Re: Simultaneous gc and repack Message-ID: <1492110320.1527.84.camel@novalis.org> URL: https://gitlist.dev/e/1492110320.1527.84.camel%40novalis.org In-Reply-To: <1553777.vJallt5N6j@mfick1-lnx> ``` On Thu, 2017-04-13 at 12:36 -0600, Martin Fick wrote: > On Thursday, April 13, 2017 02:28:07 PM David Turner wrote: > > On Thu, 2017-04-13 at 12:08 -0600, Martin Fick wrote: > > > On Thursday, April 13, 2017 11:03:14 AM Jacob Keller  > > wrote: > > > > On Thu, Apr 13, 2017 at 10:31 AM, David Turner  > > > > > > wrote: > > > > > Git gc locks the repository (using a gc.pid file) so > > > > > that other gcs don't run concurrently. But git > > > > > repack > > > > > doesn't respect this lock, so it's possible to have > > > > > a > > > > > repack running at the same time as a gc.  This makes > > > > > the gc sad when its packs are deleted out from under > > > > > it > > > > > with: "fatal: ./objects/pack/pack-$sha.pack cannot > > > > > be > > > > > accessed".  Then it dies, leaving a large temp file > > > > > hanging around. > > > > > > > > > > Does the following seem reasonable? > > > > > > > > > > 1. Make git repack, by default, check for a gc.pid > > > > > file > > > > > (using the same logic as git gc itself does). > > > > > 2. Provide a --force option to git repack to ignore > > > > > said > > > > > check. 3. Make git gc provide that --force option > > > > > when > > > > > it calls repack under its own lock. > > > > > > > > What about just making the code that calls repack > > > > today > > > > just call gc instead? I guess it's more work if you > > > > don't > > > > strictly need it but..? > > > > > > There are many scanerios where this does not achieve > > > the  > > > same thing.  On the obvious side, gc does more than  > > > repacking, but on the other side, repacking has many  > > > switches that are not available via gc. > > > > > > Would it make more sense to move the lock to repack > > > instead  of to gc? > > > > Other gc operations might step on each other too (e.g. > > packing refs). That would be less bad (and less common), > > but it still seems worth avoiding. > > Yes, but all of thsoe operations need to be self protected  > already, or they risk the same issue. They are individually protected. But I would rather fail fast. And I'm not sure what happens if git prune happens during a git repack -a; might the repack fail if an object that it expects to pack is pruned out from under it? ```