{"thread":{"id":"40911","subject":"Detecting pushes originating from shallow clones?","startedAt":"2015-11-30T22:27:31Z","lastAt":"2015-11-30T23:11:41Z","messageCount":3,"participants":["Owen Jacobson","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"273843","messageId":"147913FA-1158-40BE-8365-B90E3CBD79C7@heroku.com","threadId":"40911","inReplyTo":null,"subject":"Detecting pushes originating from shallow clones?","fromName":"Owen Jacobson","fromEmail":"ojacobson@heroku.com","sentAt":"2015-11-30T22:27:31Z","receivedAt":"2015-11-30T22:27:31Z","isPatch":false,"sender":{"key":"ojacobson@heroku.com","avatar":null},"body":"Hi folks,\n\nHere at Heroku, we ingest code from users by supplying them with a Git server they can `git push` into. Occasionally, users will attempt to push to us from a shallow clone, which causes numerous problems for us and, often, for our users (incomplete repositories, gaps in history, failed builds, and a host of other problems). We’ve been investigating what we can do to give these users clearer advice on how to fix the problem.\n\nWithin the constraints that\n\n* we cannot control which version of Git our users have installed, and\n* we run Git v1.9.1, obtained from the Ubuntu 14.04 LTS .deb repositories\n\nwhat can we do in an update/pre-receive hook to detect that an incoming push originates from a shallow repository and reject it?\n\nRight now, the best strategy we have is to observe whether\n\n    git rev-list OLD NEW\n\nfails, and if it does fail, whether the stderr output includes the phrase \"revision walk setup failed”. This feels like a fairly weak fix, so I thought I’d see if anyone here has a better idea.\n\nThanks,\n\n-o\n\n"},{"id":"273845","messageId":"xmqqr3j76qxp.fsf@gitster.mtv.corp.google.com","threadId":"40911","inReplyTo":"147913FA-1158-40BE-8365-B90E3CBD79C7@heroku.com","subject":"Re: Detecting pushes originating from shallow clones?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-11-30T22:52:18Z","receivedAt":"2015-11-30T22:52:18Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Owen Jacobson <ojacobson@heroku.com> writes:\n\n> Hi folks,\n>\n> Here at Heroku, we ingest code from users by supplying them with a\n> Git server they can `git push` into. Occasionally, users will\n> attempt to push to us from a shallow clone, which causes numerous\n> problems for us and, often, for our users (incomplete\n> repositories, gaps in history, failed builds, and a host of other\n> problems). We’ve been investigating what we can do to give these\n> users clearer advice on how to fix the problem.\n>\n> Within the constraints that\n>\n> * we cannot control which version of Git our users have installed, and\n> * we run Git v1.9.1, obtained from the Ubuntu 14.04 LTS .deb repositories\n>\n> what can we do in an update/pre-receive hook to detect that an\n> incoming push originates from a shallow repository and reject it?\n\nHmm, I would have suggested to set receive.fsckObjects which has\nbeen around since early 2008 (it is in v1.5.6, so it would likely be\nin v1.9.1 as well).\n\nBut even without that configuration set, \"push\" shouldn't leave the\nreceiving repository in an inconsistent state (e.g. incomplete\nrepository, gaps in history) in the first place.\n\nDoes anybody recall us having such a bug in the distant past in\n1.9.1 and fixing it?  I do not offhand recall but I wouldn't be\nsurprised.\n\n> Right now, the best strategy we have is to observe whether\n>\n>     git rev-list OLD NEW\n>\n> fails, and if it does fail, whether the stderr output includes the\n> phrase \"revision walk setup failed”. This feels like a fairly\n> weak fix ...\n\nActually, that (with \"--objects\" option) is essentially the test the\nreceiving end does internally to detect the \"gaps in history\" when\nreceive.fsckObjects configuration is set.\n"},{"id":"273846","messageId":"5C8F7871-D25C-4DDD-9F15-8EB9DF1F3103@heroku.com","threadId":"40911","inReplyTo":"xmqqr3j76qxp.fsf@gitster.mtv.corp.google.com","subject":"Re: Detecting pushes originating from shallow clones?","fromName":"Owen Jacobson","fromEmail":"ojacobson@heroku.com","sentAt":"2015-11-30T23:11:41Z","receivedAt":"2015-11-30T23:11:41Z","isPatch":false,"sender":{"key":"ojacobson@heroku.com","avatar":null},"body":"Owen Jacobson <ojacobson@heroku.com> wrote:\n\n> Within the constraints that\n> \n> * we cannot control which version of Git our users have installed, and\n> * we run Git v1.9.1, obtained from the Ubuntu 14.04 LTS .deb repositories\n> \n> what can we do in an update/pre-receive hook to detect that an\n> incoming push originates from a shallow repository and reject it?\n\nJunio C Hamano <gitster@pobox.com> wrote:\n\n> Hmm, I would have suggested to set receive.fsckObjects which has\n> been around since early 2008 (it is in v1.5.6, so it would likely be\n> in v1.9.1 as well).\n> \n> But even without that configuration set, \"push\" shouldn't leave the\n> receiving repository in an inconsistent state (e.g. incomplete\n> repository, gaps in history) in the first place.\n> \n> Does anybody recall us having such a bug in the distant past in\n> 1.9.1 and fixing it?  I do not offhand recall but I wouldn't be\n> surprised.\n\nIt’s possible that that would fix the problem.\n\nWe do see pushes get rejected occasionally with output similar to\n\n    To git@heroku.com:example.git\n     ! [remote rejected] master -> master (missing necessary objects)\n\n(We rely on customers reporting it to us, since right now we have no good way to spot this ourselves — we don’t get the output.)\n\nHowever, this happens -after- the pre-receive and update hooks have already run. We’ve historically assumed that the hooks are the last step prior to accepting a push, so we assume that if the hook passes, then it’s safe to take side-effect-ful actions like deploying the newly-pushed branch tip to servers.\n\nGit’s native error message also provides very little guidance on how users should resolve the problem. While I don’t think that’s Git’s responsibility in this case, it’d be nice to be able to insert our own messaging here, or to detect the problem earlier.\n\nOwen Jacobson <ojacobson@heroku.com> wrote:\n\n> Right now, the best strategy we have is to observe whether\n> \n>    git rev-list OLD NEW\n> \n> fails, and if it does fail, whether the stderr output includes the\n> phrase \"revision walk setup failed”. This feels like a fairly\n> weak fix …\n\nJunio C Hamano <gitster@pobox.com> wrote:\n\n> Actually, that (with \"--objects\" option) is essentially the test the\n> receiving end does internally to detect the \"gaps in history\" when\n> receive.fsckObjects configuration is set.\n\nHm. Well, if it’s essentially the same check, then perhaps running it ourselves in our hooks is the right fix.\n\nThanks for the input! I’m also interested in any other thoughts the list might have.\n\n-o\n\n"}]}