{"thread":{"id":"4650","subject":"[RFC] git-fetch - repack in the background after fetching","startedAt":"2006-06-24T11:30:00Z","lastAt":"2006-06-25T17:29:49Z","messageCount":7,"participants":["Martin Langhoff","Junio C Hamano","Linus Torvalds","Johannes Schindelin","Ryan Anderson"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"22441","messageId":"11511486003924-git-send-email-martin@catalyst.net.nz","threadId":"4650","inReplyTo":null,"subject":"[RFC] git-fetch - repack in the background after fetching","fromName":"Martin Langhoff","fromEmail":"martin@catalyst.net.nz","sentAt":"2006-06-24T11:30:00Z","receivedAt":"2006-06-24T11:30:00Z","isPatch":false,"sender":{"key":"martin@laptop.org","avatar":null},"body":"Check whether we have a large set of unpacked objects and repack\nafter the fetch, but don't for the user to wait for us. Conditional\non core.autorepack =! no.\n\nHaving ' handle concurrent pruning of packed objects'\n(637cdd9d1d997fca34a1fc668fed1311e30fe95f) from Jeff King it should\nbe safe to repack and prune in the background.\n\nSigned-off-by: Martin Langhoff <martin@catalyst.net.nz>\n\n---\n\nThis is a follow up to a similar patch earlier\nhttp://www.gelato.unsw.edu.au/archives/git/0605/21401.html -- is there \ninterest in making GIT more friendly to users who don't know or care\nabout packing and repacking their repos?\n\nI loathe to do this conditionally only on the count of unpacked\nobjects. If there's a quick'n'dirty way of asking portably whether\nthe machine is busy or otherwise resource-constrained (ie: on battery)\nit should use it to avoid running repack at inconvenient times.\n\n---\n git-fetch.sh |    9 +++++++++\n 1 files changed, 9 insertions(+), 0 deletions(-)\n\ndiff --git a/git-fetch.sh b/git-fetch.sh\nindex 48818f8..7211318 100755\n--- a/git-fetch.sh\n+++ b/git-fetch.sh\n@@ -427,3 +427,12 @@ case \",$update_head_ok,$orig_head,\" in\n \tfi\n \t;;\n esac\n+\n+if test \"$(git-repo-config --get core.autorepack)\" != 'no'\n+then\n+\tif test $(git rev-list --unpacked --all | wc -l) -gt 1000\n+\tthen\n+\t\techo \"Repacking in the background\"\n+\t\tnice git repack -a -d -q &\n+\tfi\n+fi\n-- \n1.4.1.rc1.g59c8\n"},{"id":"22514","messageId":"7vy7vmormn.fsf@assigned-by-dhcp.cox.net","threadId":"4650","inReplyTo":"11511486003924-git-send-email-martin@catalyst.net.nz","subject":"Re: [RFC] git-fetch - repack in the background after fetching","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-06-25T03:12:32Z","receivedAt":"2006-06-25T03:12:32Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Martin Langhoff <martin@catalyst.net.nz> writes:\n\n> This is a follow up to a similar patch earlier\n> http://www.gelato.unsw.edu.au/archives/git/0605/21401.html -- is there \n> interest in making GIT more friendly to users who don't know or care\n> about packing and repacking their repos?\n\nI would be a bit worried about the niced background repack\nracing against another instance of itself spawned by the same\nparent.\n\n> I loathe to do this conditionally only on the count of unpacked\n> objects. If there's a quick'n'dirty way of asking portably whether\n> the machine is busy or otherwise resource-constrained (ie: on battery)\n> it should use it to avoid running repack at inconvenient times.\n\ncount-objects might be lighter weight than rev-list --unpacked.\n\nIf you mean to make core.autorepack to be boolean, checking for\nstring 'no' is not the right way.\n\n\tgit repo-config --bool --get core.autorepack\n\nBut it does not matter if that variable is a string that is\nalmost always true unless the value is \"no\".\n"},{"id":"22519","messageId":"Pine.LNX.4.64.0606242049500.3747@g5.osdl.org","threadId":"4650","inReplyTo":"11511486003924-git-send-email-martin@catalyst.net.nz","subject":"Re: [RFC] git-fetch - repack in the background after fetching","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-06-25T03:53:16Z","receivedAt":"2006-06-25T03:53:16Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sat, 24 Jun 2006, Martin Langhoff wrote:\n>\n> Check whether we have a large set of unpacked objects and repack\n> after the fetch, but don't for the user to wait for us. Conditional\n> on core.autorepack =! no.\n\nI don't think this is safe.\n\nIt's also done stupidly.\n\nInstead of askign how many unpacked objects we have with the (expensive) \ngit-rev-list, why not just do\n\n\tls \"$GIT_DIR/objects/00\" | wc -l\n\nwhich is pretty much guaranteed to be faster and easier.\n\nHowever, the more worrisome thing about background repacking is that while \nit should be safe against normal users, if you have two _repacks_ at the \nsame time, they can decide to remove each others packs. Yeah, yeah, that's \npretty damn unlikely, but hey, \"pretty damn unlikely\" is not \"impossible\".\n\nAlso, I think you'd want to repack with \"-l\", in case the thing is set up \nwith an alternate object directory.\n\n\t\t\tLinus\n"},{"id":"22527","messageId":"Pine.LNX.4.63.0606251122260.29667@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"4650","inReplyTo":"Pine.LNX.4.64.0606242049500.3747@g5.osdl.org","subject":"Re: [RFC] git-fetch - repack in the background after fetching","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-06-25T09:25:02Z","receivedAt":"2006-06-25T09:25:02Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sat, 24 Jun 2006, Linus Torvalds wrote:\n\n> However, the more worrisome thing about background repacking is that while \n> it should be safe against normal users, if you have two _repacks_ at the \n> same time, they can decide to remove each others packs. Yeah, yeah, that's \n> pretty damn unlikely, but hey, \"pretty damn unlikely\" is not \"impossible\".\n\nWhy not introduce a lock file for repack?\n\nCiao,\nDscho\n"},{"id":"22535","messageId":"11512302144123-git-send-email-ryan@michonline.com","threadId":"4650","inReplyTo":"7vy7vmormn.fsf@assigned-by-dhcp.cox.net","subject":"[PATCH] Repack should try to prevent itself from running twice, concurrently.","fromName":"Ryan Anderson","fromEmail":"ryan@michonline.com","sentAt":"2006-06-25T10:10:14Z","receivedAt":"2006-06-25T10:10:14Z","isPatch":true,"sender":{"key":"ryan@michonline.com","avatar":null},"body":"Signed-off-by: Ryan Anderson <ryan@michonline.com>\n---\n git-repack.sh |   11 +++++++++++\n 1 files changed, 11 insertions(+), 0 deletions(-)\n\ndiff --git a/git-repack.sh b/git-repack.sh\nindex eb75c8c..20f9b55 100755\n--- a/git-repack.sh\n+++ b/git-repack.sh\n@@ -24,6 +24,15 @@ do\n \tshift\n done\n \n+if [ -f $GIT_DIR/repack.lock ]\n+then\n+\techo \"Existing repack job appears to be running.\"\n+\techo \"Remove $GIT_DIR/repack.lock if this is not the case.\"\n+\texit 1\n+else\n+\techo $$ > $GIT_DIR/repack.lock\n+fi\n+\n rm -f .tmp-pack-*\n PACKDIR=\"$GIT_OBJECT_DIRECTORY/pack\"\n \n@@ -83,3 +92,5 @@ case \"$no_update_info\" in\n t) : ;;\n *) git-update-server-info ;;\n esac\n+\n+rm $GIT_DIR/repack.lock\n-- \n1.4.1.rc1.gacb70\n"},{"id":"22537","messageId":"Pine.LNX.4.63.0606251216250.29667@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"4650","inReplyTo":"11512302144123-git-send-email-ryan@michonline.com","subject":"Re: [PATCH] Repack should try to prevent itself from running twice, concurrently.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-06-25T10:17:22Z","receivedAt":"2006-06-25T10:17:22Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sun, 25 Jun 2006, Ryan Anderson wrote:\n\n> +if [ -f $GIT_DIR/repack.lock ]\n> +then\n> +\techo \"Existing repack job appears to be running.\"\n> +\techo \"Remove $GIT_DIR/repack.lock if this is not the case.\"\n> +\texit 1\n> +else\n> +\techo $$ > $GIT_DIR/repack.lock\n> +fi\n\nIt is not like it is being an atomic operation, but then, we are not going \nto call repack multiple times a second. I'd say it is sufficient.\n\nCiao,\nDscho\n"},{"id":"22573","messageId":"Pine.LNX.4.64.0606251025100.3747@g5.osdl.org","threadId":"4650","inReplyTo":"Pine.LNX.4.63.0606251122260.29667@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [RFC] git-fetch - repack in the background after fetching","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-06-25T17:29:49Z","receivedAt":"2006-06-25T17:29:49Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 25 Jun 2006, Johannes Schindelin wrote:\n> \n> On Sat, 24 Jun 2006, Linus Torvalds wrote:\n> \n> > However, the more worrisome thing about background repacking is that while \n> > it should be safe against normal users, if you have two _repacks_ at the \n> > same time, they can decide to remove each others packs. Yeah, yeah, that's \n> > pretty damn unlikely, but hey, \"pretty damn unlikely\" is not \"impossible\".\n> \n> Why not introduce a lock file for repack?\n\nYou can do that. The problem is, lock-files are really hard to do \nright, and portably. Especially from scripts.\n\nBut _I_ think the basic issue is that it's wrong to even try to do this \nbackground repack.\n\nGit does explicit repacking. That's just how it is. If the worry is that \npeople forget to pack often enough, why not just have the \"git pull\" \nscript _tell_ the user, something like\n\n\tif [lots of unpacked objects]; then\n\t\techo \"You've got a boatload of unpacked objects now.\"\n\t\techo \"Maybe you'd like to repack using\"\n\t\techo \"   git repack -a -d\"\n\t\techo \"Thank you for not smoking\"\n\tfi >&2\n\nwhich is educational on so many levels.\n\n\t\tLinus\n"}]}