{"thread":{"id":"35376","subject":"Can a git push over ssh trigger a gc/repack? Diagnosing pack explosion","startedAt":"2013-11-21T15:21:26Z","lastAt":"2013-11-21T21:57:38Z","messageCount":4,"participants":["Martin Langhoff","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"230889","messageId":"CACPiFC+TqD_DhMaG+posoK4fTOLCoi=3jhJUPjt_72HTm9xjeQ@mail.gmail.com","threadId":"35376","inReplyTo":null,"subject":"Can a git push over ssh trigger a gc/repack? Diagnosing pack explosion","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2013-11-21T15:21:26Z","receivedAt":"2013-11-21T15:21:26Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"Hi git list,\n\nI am trying to diagnose a strange problem in a VM running as a 'git\nover ssh server', with one repo which periodically grows very quickly.\n\nThe complete dataset packs to a single pack+index of ~650MB. Growth is\nslow, these are ASCII text reports that use a template -- highly\ncompressible. Reports come from a few dozen machines that log in every\nhour.\n\nHowever, something is happening that explodes the efficient pack into\nan ungodly mess.\n\nDo client pushes over git+ssh ever trigger a repack on the server? If\nso, these repacking processes are racing with each other and taking\n650MB to 7GB at which point we hit ENOSPC, sometimes pom killer joins\nthe party, etc.\n\npack dir looks like this, ordered by timestamp:\nhttp://fpaste.org/55730/04636313/\n\ncheers,\n\n\n\nm\n-- \n martin.langhoff@gmail.com\n -  ask interesting questions\n - don't get distracted with shiny stuff  - working code first\n ~ http://docs.moodle.org/en/User:Martin_Langhoff\n"},{"id":"230891","messageId":"CACPiFC+xbnYjZUG49Em=aDUXnS_3_Cp=ZZBCrQCHM-sL78HCdA@mail.gmail.com","threadId":"35376","inReplyTo":"CACPiFC+TqD_DhMaG+posoK4fTOLCoi=3jhJUPjt_72HTm9xjeQ@mail.gmail.com","subject":"Re: Can a git push over ssh trigger a gc/repack? Diagnosing pack explosion","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2013-11-21T15:35:39Z","receivedAt":"2013-11-21T15:35:39Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On Thu, Nov 21, 2013 at 10:21 AM, Martin Langhoff\n<martin.langhoff@gmail.com> wrote:\n> Do client pushes over git+ssh ever trigger a repack on the server?\n\nman git-config\n[snip]\n\n       receive.autogc\n           By default, git-receive-pack will run \"git-gc --auto\" after\n           receiving data from git-push and updating refs. You can stop it by\n           setting this variable to false.\n\nOooooops!\n\nOk, couple problems here:\n\n - if it's receiving from many pushers, it races with itself; needs\nsome lock or back-off mechanism\n\n - alternatively, an splay mechanism. We have a \"hard\" threshold...\ngiven many \"pushers\" acting in parallel, they'll all hit the threshold\nat the same time. There is no need for this, we could randomize the\nthreshold by 20%; that would radically reduce the racy-ness\n\n - auto repack in this scenario has a reasonable likelihood if being\nvisited by the OOM killer -- therefore it needs to fail more\ngracefully, for example with tmpfile cleanup. Perhaps by having the\ntmpfiles places in a tmpdir named with the pid of the child would make\nthis easier...\n\nNaturally, I'll move quickly to disable this evil-spawn-automagic\nsetting and setup a cronjob. But I think it is possible to have\ndefaults that work more reliably and with lower risk of explosion.\n\nthoughts?\n\n\n\nm\n-- \n martin.langhoff@gmail.com\n -  ask interesting questions\n - don't get distracted with shiny stuff  - working code first\n ~ http://docs.moodle.org/en/User:Martin_Langhoff\n"},{"id":"230899","messageId":"xmqq61rl4jo0.fsf@gitster.dls.corp.google.com","threadId":"35376","inReplyTo":"CACPiFC+xbnYjZUG49Em=aDUXnS_3_Cp=ZZBCrQCHM-sL78HCdA@mail.gmail.com","subject":"Re: Can a git push over ssh trigger a gc/repack? Diagnosing pack explosion","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-11-21T19:52:15Z","receivedAt":"2013-11-21T19:52:15Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Martin Langhoff <martin.langhoff@gmail.com> writes:\n\n> On Thu, Nov 21, 2013 at 10:21 AM, Martin Langhoff\n> <martin.langhoff@gmail.com> wrote:\n>> Do client pushes over git+ssh ever trigger a repack on the server?\n>\n> man git-config\n> [snip]\n>\n>        receive.autogc\n>            By default, git-receive-pack will run \"git-gc --auto\" after\n>            receiving data from git-push and updating refs. You can stop it by\n>            setting this variable to false.\n>\n> Oooooops!\n>\n> Ok, couple problems here:\n>\n>  - if it's receiving from many pushers, it races with itself; needs\n> some lock or back-off mechanism\n\nSurely.\n\nI think these should help:\n\n    64a99eb4 (gc: reject if another gc is running, unless --force is given, 2013-08-08)\n    4c5baf02 (gc: remove gc.pid file at end of execution, 2013-10-16)\n\nThey should be in the upcoming v1.8.5.\n"},{"id":"230917","messageId":"CACPiFC+RmmhH9GuE+3ecP7Hj4C+iRfyk1px9EP+j45kOBExM5Q@mail.gmail.com","threadId":"35376","inReplyTo":"xmqq61rl4jo0.fsf@gitster.dls.corp.google.com","subject":"Re: Can a git push over ssh trigger a gc/repack? Diagnosing pack explosion","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2013-11-21T21:57:38Z","receivedAt":"2013-11-21T21:57:38Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On Thu, Nov 21, 2013 at 2:52 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>>  - if it's receiving from many pushers, it races with itself; needs\n>> some lock or back-off mechanism\n>\n> Surely.\n>\n> I think these should help:\n>\n>     64a99eb4 (gc: reject if another gc is running, unless --force is given, 2013-08-08)\n>     4c5baf02 (gc: remove gc.pid file at end of execution, 2013-10-16)\n>\n> They should be in the upcoming v1.8.5.\n\nAh, great to hear. For the record, this hit me on git 1.7.1, current on RHEL6.\n\nthanks!\n\n\n\nm\n-- \n martin.langhoff@gmail.com\n -  ask interesting questions\n - don't get distracted with shiny stuff  - working code first\n ~ http://docs.moodle.org/en/User:Martin_Langhoff\n"}]}