{"thread":{"id":"45683","subject":"Simultaneous gc and repack","startedAt":"2017-04-13T17:31:45Z","lastAt":"2017-04-13T19:05:27Z","messageCount":7,"participants":["David Turner","Jacob Keller","Martin Fick"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"316736","messageId":"1492104698.1527.79.camel@novalis.org","threadId":"45683","inReplyTo":null,"subject":"Simultaneous gc and repack","fromName":"David Turner","fromEmail":"novalis@novalis.org","sentAt":"2017-04-13T17:31:38Z","receivedAt":"2017-04-13T17:31:45Z","isPatch":false,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Git gc locks the repository (using a gc.pid file) so that other gcs\ndon't run concurrently. But git repack doesn't respect this lock, so\nit's possible to have a repack running at the same time as a gc. This\nmakes the gc sad when its packs are deleted out from under it with:\n\"fatal: ./objects/pack/pack-$sha.pack cannot be accessed\". Then it\ndies, leaving a large temp file hanging around.\n\nDoes the following seem reasonable?\n\n1. Make git repack, by default, check for a gc.pid file (using the same\nlogic as git gc itself does).\n2. Provide a --force option to git repack to ignore said check.\n3. Make git gc provide that --force option when it calls repack under\nits own lock.\n\nThis came up because Gitlab runs a repack after every N pushes and a gc\nafter every M commits, where M >> N. Sometimes, when pushes come in\nrapidly, the repack catches the still-running gc and the above badness\nhappens. At least, that's my understanding: I don't run our Gitlab\nservers, but I talked to the person who does and that's what he said.\n\nOf course, Gitlab could do its own locking, but the general approach\nseems like it would help other folks too.\n"},{"id":"316739","messageId":"CA+P7+xo4mwA8DMeObq_T9=P9m7tG1d6afDS_Md56_EnhuXLVgw@mail.gmail.com","threadId":"45683","inReplyTo":"1492104698.1527.79.camel@novalis.org","subject":"Re: Simultaneous gc and repack","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-04-13T18:03:14Z","receivedAt":"2017-04-13T18:03:42Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Thu, Apr 13, 2017 at 10:31 AM, David Turner wrote:\n> Git gc locks the repository (using a gc.pid file) so that other gcs\n> don't run concurrently. But git repack doesn't respect this lock, so\n> it's possible to have a repack running at the same time as a gc. This\n> makes the gc sad when its packs are deleted out from under it with:\n> \"fatal: ./objects/pack/pack-$sha.pack cannot be accessed\". Then it\n> dies, leaving a large temp file hanging around.\n>\n> Does the following seem reasonable?\n>\n> 1. Make git repack, by default, check for a gc.pid file (using the same\n> logic as git gc itself does).\n> 2. Provide a --force option to git repack to ignore said check.\n> 3. Make git gc provide that --force option when it calls repack under\n> its own lock.\n>\n\nWhat about just making the code that calls repack today just call gc\ninstead? I guess it's more work if you don't strictly need it but..?\n\nThanks,\nJake\n\n> This came up because Gitlab runs a repack after every N pushes and a gc\n> after every M commits, where M >> N. Sometimes, when pushes come in\n> rapidly, the repack catches the still-running gc and the above badness\n> happens. At least, that's my understanding: I don't run our Gitlab\n> servers, but I talked to the person who does and that's what he said.\n>\n> Of course, Gitlab could do its own locking, but the general approach\n> seems like it would help other folks too.\n"},{"id":"316741","messageId":"10832632.7AIzXehjUi@mfick1-lnx","threadId":"45683","inReplyTo":"CA+P7+xo4mwA8DMeObq_T9=P9m7tG1d6afDS_Md56_EnhuXLVgw@mail.gmail.com","subject":"Re: Simultaneous gc and repack","fromName":"Martin Fick","fromEmail":"mfick@codeaurora.org","sentAt":"2017-04-13T18:08:10Z","receivedAt":"2017-04-13T18:08:24Z","isPatch":false,"sender":{"key":"mfick@codeaurora.org","avatar":null},"body":"On Thursday, April 13, 2017 11:03:14 AM Jacob Keller wrote:\n> On Thu, Apr 13, 2017 at 10:31 AM, David Turner \n wrote:\n> > Git gc locks the repository (using a gc.pid file) so\n> > that other gcs don't run concurrently. But git repack\n> > doesn't respect this lock, so it's possible to have a\n> > repack running at the same time as a gc. This makes\n> > the gc sad when its packs are deleted out from under it\n> > with: \"fatal: ./objects/pack/pack-$sha.pack cannot be\n> > accessed\". Then it dies, leaving a large temp file\n> > hanging around.\n> > \n> > Does the following seem reasonable?\n> > \n> > 1. Make git repack, by default, check for a gc.pid file\n> > (using the same logic as git gc itself does).\n> > 2. Provide a --force option to git repack to ignore said\n> > check. 3. Make git gc provide that --force option when\n> > it calls repack under its own lock.\n> \n> What about just making the code that calls repack today\n> just call gc instead? I guess it's more work if you don't\n> strictly need it but..?\n\nThere are many scanerios where this does not achieve the \nsame thing. On the obvious side, gc does more than \nrepacking, but on the other side, repacking has many \nswitches that are not available via gc.\n\nWould it make more sense to move the lock to repack instead \nof to gc?\n\n-Martin\n\n-- \nThe Qualcomm Innovation Center, Inc. is a member of Code \nAurora Forum, hosted by The Linux Foundation\n\n"},{"id":"316746","messageId":"1492108087.1527.82.camel@novalis.org","threadId":"45683","inReplyTo":"10832632.7AIzXehjUi@mfick1-lnx","subject":"Re: Simultaneous gc and repack","fromName":"David Turner","fromEmail":"novalis@novalis.org","sentAt":"2017-04-13T18:28:07Z","receivedAt":"2017-04-13T18:28:13Z","isPatch":false,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Thu, 2017-04-13 at 12:08 -0600, Martin Fick wrote:\n> On Thursday, April 13, 2017 11:03:14 AM Jacob Keller wrote:\n> > On Thu, Apr 13, 2017 at 10:31 AM, David Turner \n> \n> wrote:\n> > > Git gc locks the repository (using a gc.pid file) so\n> > > that other gcs don't run concurrently. But git repack\n> > > doesn't respect this lock, so it's possible to have a\n> > > repack running at the same time as a gc.  This makes\n> > > the gc sad when its packs are deleted out from under it\n> > > with: \"fatal: ./objects/pack/pack-$sha.pack cannot be\n> > > accessed\".  Then it dies, leaving a large temp file\n> > > hanging around.\n> > > \n> > > Does the following seem reasonable?\n> > > \n> > > 1. Make git repack, by default, check for a gc.pid file\n> > > (using the same logic as git gc itself does).\n> > > 2. Provide a --force option to git repack to ignore said\n> > > check. 3. Make git gc provide that --force option when\n> > > it calls repack under its own lock.\n> > \n> > What about just making the code that calls repack today\n> > just call gc instead? I guess it's more work if you don't\n> > strictly need it but..?\n> \n> There are many scanerios where this does not achieve the \n> same thing.  On the obvious side, gc does more than \n> repacking, but on the other side, repacking has many \n> switches that are not available via gc.\n> \n> Would it make more sense to move the lock to repack instead \n> of to gc?\n\nOther gc operations might step on each other too (e.g. packing refs). \nThat would be less bad (and less common), but it still seems worth\navoiding.\n"},{"id":"316757","messageId":"CA+P7+xoQtGyWpuUKFDQzy1RhdzT7mhES+VOczNSSQceiQUkDbw@mail.gmail.com","threadId":"45683","inReplyTo":"1492108087.1527.82.camel@novalis.org","subject":"Re: Simultaneous gc and repack","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-04-13T18:35:16Z","receivedAt":"2017-04-13T18:35:46Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Thu, Apr 13, 2017 at 11:28 AM, David Turner wrote:\n> On Thu, 2017-04-13 at 12:08 -0600, Martin Fick wrote:\n>> On Thursday, April 13, 2017 11:03:14 AM Jacob Keller wrote:\n>> > On Thu, Apr 13, 2017 at 10:31 AM, David Turner\n>>\n>> wrote:\n>> > > Git gc locks the repository (using a gc.pid file) so\n>> > > that other gcs don't run concurrently. But git repack\n>> > > doesn't respect this lock, so it's possible to have a\n>> > > repack running at the same time as a gc. This makes\n>> > > the gc sad when its packs are deleted out from under it\n>> > > with: \"fatal: ./objects/pack/pack-$sha.pack cannot be\n>> > > accessed\". Then it dies, leaving a large temp file\n>> > > hanging around.\n>> > >\n>> > > Does the following seem reasonable?\n>> > >\n>> > > 1. Make git repack, by default, check for a gc.pid file\n>> > > (using the same logic as git gc itself does).\n>> > > 2. Provide a --force option to git repack to ignore said\n>> > > check. 3. Make git gc provide that --force option when\n>> > > it calls repack under its own lock.\n>> >\n>> > What about just making the code that calls repack today\n>> > just call gc instead? I guess it's more work if you don't\n>> > strictly need it but..?\n>>\n>> There are many scanerios where this does not achieve the\n>> same thing. On the obvious side, gc does more than\n>> repacking, but on the other side, repacking has many\n>> switches that are not available via gc.\n>>\n>> Would it make more sense to move the lock to repack instead\n>> of to gc?\n>\n> Other gc operations might step on each other too (e.g. packing refs).\n> That would be less bad (and less common), but it still seems worth\n> avoiding.\n\nIt sounds like your original solution would work, though I wouldn't\nuse \"force\" and I would either not document or document with \"this is\nonly meant to be used by git-gc internally\"\n\nThanks,\nJake\n"},{"id":"316758","messageId":"1553777.vJallt5N6j@mfick1-lnx","threadId":"45683","inReplyTo":"1492108087.1527.82.camel@novalis.org","subject":"Re: Simultaneous gc and repack","fromName":"Martin Fick","fromEmail":"mfick@codeaurora.org","sentAt":"2017-04-13T18:36:20Z","receivedAt":"2017-04-13T18:36:27Z","isPatch":false,"sender":{"key":"mfick@codeaurora.org","avatar":null},"body":"On Thursday, April 13, 2017 02:28:07 PM David Turner wrote:\n> On Thu, 2017-04-13 at 12:08 -0600, Martin Fick wrote:\n> > On Thursday, April 13, 2017 11:03:14 AM Jacob Keller \nwrote:\n> > > On Thu, Apr 13, 2017 at 10:31 AM, David Turner \n> > \n> > wrote:\n> > > > Git gc locks the repository (using a gc.pid file) so\n> > > > that other gcs don't run concurrently. But git\n> > > > repack\n> > > > doesn't respect this lock, so it's possible to have\n> > > > a\n> > > > repack running at the same time as a gc. This makes\n> > > > the gc sad when its packs are deleted out from under\n> > > > it\n> > > > with: \"fatal: ./objects/pack/pack-$sha.pack cannot\n> > > > be\n> > > > accessed\". Then it dies, leaving a large temp file\n> > > > hanging around.\n> > > > \n> > > > Does the following seem reasonable?\n> > > > \n> > > > 1. Make git repack, by default, check for a gc.pid\n> > > > file\n> > > > (using the same logic as git gc itself does).\n> > > > 2. Provide a --force option to git repack to ignore\n> > > > said\n> > > > check. 3. Make git gc provide that --force option\n> > > > when\n> > > > it calls repack under its own lock.\n> > > \n> > > What about just making the code that calls repack\n> > > today\n> > > just call gc instead? I guess it's more work if you\n> > > don't\n> > > strictly need it but..?\n> > \n> > There are many scanerios where this does not achieve\n> > the \n> > same thing. On the obvious side, gc does more than \n> > repacking, but on the other side, repacking has many \n> > switches that are not available via gc.\n> > \n> > Would it make more sense to move the lock to repack\n> > instead of to gc?\n> \n> Other gc operations might step on each other too (e.g.\n> packing refs). That would be less bad (and less common),\n> but it still seems worth avoiding.\n\nYes, but all of thsoe operations need to be self protected \nalready, or they risk the same issue.\n\n-Martin\n\n-- \nThe Qualcomm Innovation Center, Inc. is a member of Code \nAurora Forum, hosted by The Linux Foundation\n\n"},{"id":"316762","messageId":"1492110320.1527.84.camel@novalis.org","threadId":"45683","inReplyTo":"1553777.vJallt5N6j@mfick1-lnx","subject":"Re: Simultaneous gc and repack","fromName":"David Turner","fromEmail":"novalis@novalis.org","sentAt":"2017-04-13T19:05:20Z","receivedAt":"2017-04-13T19:05:27Z","isPatch":false,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Thu, 2017-04-13 at 12:36 -0600, Martin Fick wrote:\n> On Thursday, April 13, 2017 02:28:07 PM David Turner wrote:\n> > On Thu, 2017-04-13 at 12:08 -0600, Martin Fick wrote:\n> > > On Thursday, April 13, 2017 11:03:14 AM Jacob Keller \n> \n> wrote:\n> > > > On Thu, Apr 13, 2017 at 10:31 AM, David Turner \n> > > \n> > > wrote:\n> > > > > Git gc locks the repository (using a gc.pid file) so\n> > > > > that other gcs don't run concurrently. But git\n> > > > > repack\n> > > > > doesn't respect this lock, so it's possible to have\n> > > > > a\n> > > > > repack running at the same time as a gc.  This makes\n> > > > > the gc sad when its packs are deleted out from under\n> > > > > it\n> > > > > with: \"fatal: ./objects/pack/pack-$sha.pack cannot\n> > > > > be\n> > > > > accessed\".  Then it dies, leaving a large temp file\n> > > > > hanging around.\n> > > > > \n> > > > > Does the following seem reasonable?\n> > > > > \n> > > > > 1. Make git repack, by default, check for a gc.pid\n> > > > > file\n> > > > > (using the same logic as git gc itself does).\n> > > > > 2. Provide a --force option to git repack to ignore\n> > > > > said\n> > > > > check. 3. Make git gc provide that --force option\n> > > > > when\n> > > > > it calls repack under its own lock.\n> > > > \n> > > > What about just making the code that calls repack\n> > > > today\n> > > > just call gc instead? I guess it's more work if you\n> > > > don't\n> > > > strictly need it but..?\n> > > \n> > > There are many scanerios where this does not achieve\n> > > the \n> > > same thing.  On the obvious side, gc does more than \n> > > repacking, but on the other side, repacking has many \n> > > switches that are not available via gc.\n> > > \n> > > Would it make more sense to move the lock to repack\n> > > instead  of to gc?\n> > \n> > Other gc operations might step on each other too (e.g.\n> > packing refs). That would be less bad (and less common),\n> > but it still seems worth avoiding.\n> \n> Yes, but all of thsoe operations need to be self protected \n> already, or they risk the same issue.\n\nThey are individually protected. But I would rather fail fast. And\nI'm not sure what happens if git prune happens during a git repack -a;\nmight the repack fail if an object that it expects to pack is pruned\nout from under it?\n\n\n"}]}