{"thread":{"id":"50045","subject":"[PATCH 0/1] gc/repack: release packs when needed","startedAt":"2018-12-15T22:04:03Z","lastAt":"2019-01-11T17:24:14Z","messageCount":5,"participants":["Johannes Schindelin via GitGitGadget","Junio C Hamano","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":1},"messages":[{"id":"365437","messageId":"pull.95.git.gitgitgadget@gmail.com","threadId":"50045","inReplyTo":null,"subject":"[PATCH 0/1] gc/repack: release packs when needed","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-12-15T22:04:00Z","receivedAt":"2018-12-15T22:04:03Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This fixes more \"can't delete files while they are still open\" issues on\nWindows.\n\nJohannes Schindelin (1):\n gc/repack: release packs when needed\n\n builtin/gc.c | 4 +++-\n builtin/repack.c | 2 ++\n 2 files changed, 5 insertions(+), 1 deletion(-)\n\n\nbase-commit: 5d826e972970a784bd7a7bdf587512510097b8c7\nPublished-As: https://github.com/gitgitgadget/git/releases/tags/pr-95%2Fdscho%2Frelease-gc-repack-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-95/dscho/release-gc-repack-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/95\n-- \ngitgitgadget\n"},{"id":"365438","messageId":"7eee3d107927b30bd3e1ec422e833111627252ce.1544911438.git.gitgitgadget@gmail.com","threadId":"50045","inReplyTo":"pull.95.git.gitgitgadget@gmail.com","subject":"[PATCH 1/1] gc/repack: release packs when needed","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-12-15T22:04:01Z","receivedAt":"2018-12-15T22:04:06Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin \n\nOn Windows, files cannot be removed nor renamed if there are still\nhandles held by a process. To remedy that, we introduced the\nclose_all_packs() function.\n\nEarlier, we made sure that the packs are released just before `git gc`\nis spawned, in case that gc wants to remove no-longer needed packs.\n\nBut this developer forgot that gc itself also needs to let go of packs,\ne.g. when consolidating all packs via the --aggressive option.\n\nLikewise, `git repack -d` wants to delete obsolete packs and therefore\nneeds to close all pack handles, too.\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/gc.c | 4 +++-\n builtin/repack.c | 2 ++\n 2 files changed, 5 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 871a56f1c5..df90fd7f51 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -659,8 +659,10 @@ int cmd_gc(int argc, const char **argv, const char *prefix)\n \n \treport_garbage = report_pack_garbage;\n \treprepare_packed_git(the_repository);\n-\tif (pack_garbage.nr > 0)\n+\tif (pack_garbage.nr > 0) {\n+\t\tclose_all_packs(the_repository->objects);\n \t\tclean_pack_garbage();\n+\t}\n \n \tif (gc_write_commit_graph)\n \t\twrite_commit_graph_reachable(get_object_directory(), 0,\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex 45583683ee..f9319defe4 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -419,6 +419,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix)\n \tif (!names.nr && !po_args.quiet)\n \t\tprintf(\"Nothing new to pack.\\n\");\n \n+\tclose_all_packs(the_repository->objects);\n+\n \t/*\n \t * Ok we have prepared all new packfiles.\n \t * First see if there are packs of the same name and if so\n-- \ngitgitgadget\n"},{"id":"366532","messageId":"xmqqmuo81b2n.fsf@gitster-ct.c.googlers.com","threadId":"50045","inReplyTo":"7eee3d107927b30bd3e1ec422e833111627252ce.1544911438.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/1] gc/repack: release packs when needed","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-01-10T21:01:36Z","receivedAt":"2019-01-10T21:01:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Johannes Schindelin via GitGitGadget\" \nwrites:\n\n> From: Johannes Schindelin \n>\n> On Windows, files cannot be removed nor renamed if there are still\n> handles held by a process. To remedy that, we introduced the\n> close_all_packs() function.\n>\n> Earlier, we made sure that the packs are released just before `git gc`\n> is spawned, in case that gc wants to remove no-longer needed packs.\n>\n> But this developer forgot that gc itself also needs to let go of packs,\n> e.g. when consolidating all packs via the --aggressive option.\n>\n> Likewise, `git repack -d` wants to delete obsolete packs and therefore\n> needs to close all pack handles, too.\n>\n> Signed-off-by: Johannes Schindelin \n> ---\n> builtin/gc.c | 4 +++-\n> builtin/repack.c | 2 ++\n> 2 files changed, 5 insertions(+), 1 deletion(-)\n>\n> diff --git a/builtin/gc.c b/builtin/gc.c\n> index 871a56f1c5..df90fd7f51 100644\n> --- a/builtin/gc.c\n> +++ b/builtin/gc.c\n> @@ -659,8 +659,10 @@ int cmd_gc(int argc, const char **argv, const char *prefix)\n> \n> \treport_garbage = report_pack_garbage;\n> \treprepare_packed_git(the_repository);\n> -\tif (pack_garbage.nr > 0)\n> +\tif (pack_garbage.nr > 0) {\n> +\t\tclose_all_packs(the_repository->objects);\n> \t\tclean_pack_garbage();\n> +\t}\n\nClosing before removing does make sense, but wouldn't we want to\nmove reprepare_packed_git() after clean_pack_garbage() while at it?\nAfter all, the logical sequence is that we used the current set of\npacks to figure out whihch ones are garbage, then now we are about\nto discard. We close the packs in the current set (i.e. the fix\nmade in this patch), discard the garbage packs. It would make sense\nto start using the new set (i.e. \"reprepare\") after all that is\ndone, no? Especially, given that the next step (write-commit-graph)\nstill wants to read quite a lot of data from now the latest set of\npackfiles...\n\n> \tif (gc_write_commit_graph)\n> \t\twrite_commit_graph_reachable(get_object_directory(), 0,\n> diff --git a/builtin/repack.c b/builtin/repack.c\n> index 45583683ee..f9319defe4 100644\n> --- a/builtin/repack.c\n> +++ b/builtin/repack.c\n> @@ -419,6 +419,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix)\n> \tif (!names.nr && !po_args.quiet)\n> \t\tprintf(\"Nothing new to pack.\\n\");\n> \n> +\tclose_all_packs(the_repository->objects);\n> +\n\nOn the other hand, This one is added to the ideal and perfect\nlocation, I think.\n\nThanks.\n\n> \t/*\n> \t * Ok we have prepared all new packfiles.\n> \t * First see if there are packs of the same name and if so\n"},{"id":"366563","messageId":"20190111161021.GE16754@sigill.intra.peff.net","threadId":"50045","inReplyTo":"xmqqmuo81b2n.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 1/1] gc/repack: release packs when needed","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-01-11T16:10:22Z","receivedAt":"2019-01-11T16:10:25Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jan 10, 2019 at 01:01:36PM -0800, Junio C Hamano wrote:\n\n> > diff --git a/builtin/gc.c b/builtin/gc.c\n> > index 871a56f1c5..df90fd7f51 100644\n> > --- a/builtin/gc.c\n> > +++ b/builtin/gc.c\n> > @@ -659,8 +659,10 @@ int cmd_gc(int argc, const char **argv, const char *prefix)\n> > \n> > \treport_garbage = report_pack_garbage;\n> > \treprepare_packed_git(the_repository);\n> > -\tif (pack_garbage.nr > 0)\n> > +\tif (pack_garbage.nr > 0) {\n> > +\t\tclose_all_packs(the_repository->objects);\n> > \t\tclean_pack_garbage();\n> > +\t}\n> \n> Closing before removing does make sense, but wouldn't we want to\n> move reprepare_packed_git() after clean_pack_garbage() while at it?\n> After all, the logical sequence is that we used the current set of\n> packs to figure out whihch ones are garbage, then now we are about\n> to discard. We close the packs in the current set (i.e. the fix\n> made in this patch), discard the garbage packs. It would make sense\n> to start using the new set (i.e. \"reprepare\") after all that is\n> done, no? Especially, given that the next step (write-commit-graph)\n> still wants to read quite a lot of data from now the latest set of\n> packfiles...\n\nI agree that your suggested ordering makes more sense, but I don't think\nit matters in practice with the current code. reprepare_packed_git()\nnever throws away old pack entries (and if they're mmap'd, we might even\ncontinue to use them). So the end result is the same either way.\n\n-Peff\n"},{"id":"366564","messageId":"xmqqlg3ryuo7.fsf@gitster-ct.c.googlers.com","threadId":"50045","inReplyTo":"20190111161021.GE16754@sigill.intra.peff.net","subject":"Re: [PATCH 1/1] gc/repack: release packs when needed","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-01-11T17:24:08Z","receivedAt":"2019-01-11T17:24:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Thu, Jan 10, 2019 at 01:01:36PM -0800, Junio C Hamano wrote:\n>\n>> > diff --git a/builtin/gc.c b/builtin/gc.c\n>> > index 871a56f1c5..df90fd7f51 100644\n>> > --- a/builtin/gc.c\n>> > +++ b/builtin/gc.c\n>> > @@ -659,8 +659,10 @@ int cmd_gc(int argc, const char **argv, const char *prefix)\n>> > \n>> > \treport_garbage = report_pack_garbage;\n>> > \treprepare_packed_git(the_repository);\n>> > -\tif (pack_garbage.nr > 0)\n>> > +\tif (pack_garbage.nr > 0) {\n>> > +\t\tclose_all_packs(the_repository->objects);\n>> > \t\tclean_pack_garbage();\n>> > +\t}\n>> \n>> Closing before removing does make sense, but wouldn't we want to\n>> move reprepare_packed_git() after clean_pack_garbage() while at it?\n>> After all, the logical sequence is that we used the current set of\n>> packs to figure out whihch ones are garbage, then now we are about\n>> to discard. We close the packs in the current set (i.e. the fix\n>> made in this patch), discard the garbage packs. It would make sense\n>> to start using the new set (i.e. \"reprepare\") after all that is\n>> done, no? Especially, given that the next step (write-commit-graph)\n>> still wants to read quite a lot of data from now the latest set of\n>> packfiles...\n>\n> I agree that your suggested ordering makes more sense, but I don't think\n> it matters in practice with the current code. reprepare_packed_git()\n> never throws away old pack entries (and if they're mmap'd, we might even\n> continue to use them). So the end result is the same either way.\n\nYeah, it would not make difference to the machine. I was trying to\nbe more helpful to human readers.\n\nIn any case, this patch from Dec 15 last year is where my backlog\nsweeping is at right now X-<.\n"}]}