{"thread":{"id":"25183","subject":"Deleting of the specified ref during the post-receive hook","startedAt":"2010-09-21T14:40:50Z","lastAt":"2010-09-21T16:00:24Z","messageCount":4,"participants":["Joshua Jensen","Shawn O. Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"151215","messageId":"4C98C3F2.9050006@workspacewhiz.com","threadId":"25183","inReplyTo":null,"subject":"Deleting of the specified ref during the post-receive hook","fromName":"Joshua Jensen","fromEmail":"jjensen@workspacewhiz.com","sentAt":"2010-09-21T14:40:50Z","receivedAt":"2010-09-21T14:40:50Z","isPatch":false,"sender":{"key":"jjensen@workspacewhiz.com","avatar":"https://avatars.githubusercontent.com/u/111687?v=4"},"body":"  I've recently asked whether there was a public script to act as an \nautomatic 'maintainer', something akin to Gerrit's push always \nsucceeding without having to pull first when using Git as a central \nrepository.  I received a number of suggestions, and I have begun trial \nimplementations.\n\nMy current line of thought has an auto-merging script that monitors the \nrefs/for/ namespace (similar to Gerrit) and then applies --no-ff merges \nto the appropriate branch.  For instance, when the user pushes to \nrefs/for/master, the post-receive hook creates a secondary ref called \nrefs/for/master-SHA1-timestamp and then deletes the refs/for/master ref:\n\n#!bin/sh\n# post-receive hook\nwhile read oldrev newrev ref\ndo\n     case $ref in\n         refs/for/*)\n             timestamp=`date +%s`\n             `git update-ref $ref-$newrev-$timestamp $newrev`\n             `git update-ref -d $ref`\n         ;;\n     esac\ndone\n\nIf you'll pardon my lacking shell script skills (I'm open to learn!), my \nprimary question concerns safety.  When receiving a ref via an SSH-based \nserver (which happens to be Gitolite, but I don't think that is relevant \nhere), is the post-receive hook guaranteed to be run in a lockstep \nmanner?  That is, if two people push to 'refs/for/master' at the same \ntime, is there a lock to process one user and then the other user?\n\nThe auto-merging script is just simple at the moment.  It runs 'git \nfetch origin refs/for/*:refs/for/*', sorts the refs/for/ entries by \ntimestamp, and merges into the specified branch, emailing the user on \nsuccess or failure (not implemented yet... I'm sure Gitolite gives me \naccess to the username, but I haven't looked it up yet).\n\nBefore I go too much deeper down this path, am I way off base here?\n\nThanks.\n\nJosh\n"},{"id":"151218","messageId":"20100921145014.GH32601@spearce.org","threadId":"25183","inReplyTo":"4C98C3F2.9050006@workspacewhiz.com","subject":"Re: Deleting of the specified ref during the post-receive hook","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2010-09-21T14:50:14Z","receivedAt":"2010-09-21T14:50:14Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Joshua Jensen <jjensen@workspacewhiz.com> wrote:\n>  I've recently asked whether there was a public script to act as an  \n> automatic 'maintainer', something akin to Gerrit's push always  \n> succeeding without having to pull first when using Git as a central  \n> repository.  I received a number of suggestions, and I have begun trial  \n> implementations.\n>\n> My current line of thought has an auto-merging script that monitors the  \n> refs/for/ namespace (similar to Gerrit) and then applies --no-ff merges  \n> to the appropriate branch.  For instance, when the user pushes to  \n> refs/for/master, the post-receive hook creates a secondary ref called  \n> refs/for/master-SHA1-timestamp and then deletes the refs/for/master ref:\n>\n> #!bin/sh\n> # post-receive hook\n> while read oldrev newrev ref\n> do\n>     case $ref in\n>         refs/for/*)\n>             timestamp=`date +%s`\n>             `git update-ref $ref-$newrev-$timestamp $newrev`\n>             `git update-ref -d $ref`\n\nFWIW, you don't need `` around the git update-ref calls, because you\naren't using the output of the command as input to another command.\n\n> If you'll pardon my lacking shell script skills (I'm open to learn!), my  \n> primary question concerns safety.  When receiving a ref via an SSH-based  \n> server (which happens to be Gitolite, but I don't think that is relevant  \n> here), is the post-receive hook guaranteed to be run in a lockstep  \n> manner?\n\nNo.  If there are two concurrent pushes occurring, the script may\nexecute in parallel.\n\nBut you'll actually get something much worse.  receive-pack will\ncreate refs/for/master for the first user... and then might be put\nto sleep while another user's receive-pack starts.  That second\nuser will see refs/for/master existing, and will fail their push\nbecause their concept of $newrev doesn't match what is currently\nat refs/for/master.  Then the first user wakes up, runs your\npost-receive, and the ref is cleared.\n\nThe better strategy would be to use an update hook that refuses to\npermit the creation of refs/for/master:\n\n  #!/bin/sh\n  ref=$1\n  old=$2\n  new=$3\n\n  case $ref in\n  refs/for/*)\n    timestamp=`date +%s`\n    git update-ref $ref-$new-$timestamp $new\n    echo \"Created $ref-$new-$timestamp\"\n    exit 1\n    ;;\n  *)\n    exit 0\n    ;;\n  esac\n\n\nBy exit 1 here we refuse the push attempt, so receive-pack won't\ncreate refs/for/master, and another user pushing won't see that\nfalse failure.  However, unlike with Gerrit, every user is now going\nto receive a push failure message because the hook has appeared to\nreject the value, even though it accepted it.\n\n> That is, if two people push to 'refs/for/master' at the same  \n> time, is there a lock to process one user and then the other user?\n\nThere is, but only for the duration of the update hook.  So if you\nused an update hook instead, one user would succeed with the push\nand the other would get a lock failure, and would need to retry.\n\n> Before I go too much deeper down this path, am I way off base here?\n\nI'd have to ask why you are using gitolite and trying to abuse\ngit-receive-pack to do something that Gerrit does out of the box...\n\n-- \nShawn.\n"},{"id":"151221","messageId":"4C98D558.9090900@workspacewhiz.com","threadId":"25183","inReplyTo":"20100921145014.GH32601@spearce.org","subject":"Re: Deleting of the specified ref during the post-receive hook","fromName":"Joshua Jensen","fromEmail":"jjensen@workspacewhiz.com","sentAt":"2010-09-21T15:55:04Z","receivedAt":"2010-09-21T15:55:04Z","isPatch":false,"sender":{"key":"jjensen@workspacewhiz.com","avatar":"https://avatars.githubusercontent.com/u/111687?v=4"},"body":"  ----- Original Message -----\nFrom: Shawn O. Pearce\nDate: 9/21/2010 8:50 AM\n> Joshua Jensen<jjensen@workspacewhiz.com>  wrote:\n>> My current line of thought has an auto-merging script that monitors the\n>> refs/for/ namespace (similar to Gerrit) and then applies --no-ff merges\n>> to the appropriate branch.  For instance, when the user pushes to\n>> refs/for/master, the post-receive hook creates a secondary ref called\n>> refs/for/master-SHA1-timestamp and then deletes the refs/for/master ref\n>>\n> FWIW, you don't need `` around the git update-ref calls, because you\n> aren't using the output of the command as input to another command.\nYou are right.  I believe I was early on, and it just got copied and \npasted around.\n> No.  If there are two concurrent pushes occurring, the script may\n> execute in parallel.\nOkay, so what I'm doing is bad.  Got it.\n> But you'll actually get something much worse.  receive-pack will\n> create refs/for/master for the first user... and then might be put\n> to sleep while another user's receive-pack starts.  That second\n> user will see refs/for/master existing, and will fail their push\n> because their concept of $newrev doesn't match what is currently\n> at refs/for/master.  Then the first user wakes up, runs your\n> post-receive, and the ref is cleared.\nYep, this is exactly what I was trying to avoid by deleting the ref.\n> The better strategy would be to use an update hook that refuses to\n> permit the creation of refs/for/master:\n>\n>    #!/bin/sh\n>    ref=$1\n>    old=$2\n>    new=$3\n>\n>    case $ref in\n>    refs/for/*)\n>      timestamp=`date +%s`\n>      git update-ref $ref-$new-$timestamp $new\n>      echo \"Created $ref-$new-$timestamp\"\n>      exit 1\n>      ;;\n>    *)\n>      exit 0\n>      ;;\n>    esac\n>\n>\n> By exit 1 here we refuse the push attempt, so receive-pack won't\n> create refs/for/master, and another user pushing won't see that\n> false failure.  However, unlike with Gerrit, every user is now going\n> to receive a push failure message because the hook has appeared to\n> reject the value, even though it accepted it.\nOkay, I'll try this.  I can train people in the push failure, if \nnecessary.  Thanks!\n\n>> Before I go too much deeper down this path, am I way off base here?\n> I'd have to ask why you are using gitolite and trying to abuse\n> git-receive-pack to do something that Gerrit does out of the box...\n>\nOh, how I would love to just leave it to Gerrit to handle this.  (I \n*really* like Gerrit.)  If you would like to just skip ahead, the actual \nquestion related to this email message is in the final paragraph.\n\nWe don't want to squash our topic branches before pushing them to \nGerrit.  That means we end up with a range of 'n' number of commits in \nthe Gerrit review screen.  Gerrit understands that commit 'n' is \ndependent on 'n-1' and so on, but unfortunately, it doesn't show them in \na tree/group on the main page.  That's a minor gripe, but here is the \nmajor issue.\n\nWhen I review Change 1, I can press a submit button, and Change 1 goes \nlive right then.  Unfortunately, changes 2 through n were left behind.\n\nIf I review Change 1 through n and then press the Submit button on \nChange n and then on Change n-1 and then on Change n-2, Gerrit does the \n'right' thing (for us) and makes live the entire dependency chain at the \nsame time.  Of course, I would prefer to just have a group Submit \nbutton, but that's another story.\n\nThis has come up on the mailing list a few times, and I even think there \nis an issue tracker item for it.\n\nLet's ignore the review+submit portion of Gerrit now.  Can Gerrit be \ncoaxed to take a refs/for/master and directly apply it to the master \nbranch WITHOUT the review cycle?  If so, then I'm wasting my time trying \nto right a script.\n\nJosh\n"},{"id":"151222","messageId":"20100921160024.GI32601@spearce.org","threadId":"25183","inReplyTo":"4C98D558.9090900@workspacewhiz.com","subject":"Re: Deleting of the specified ref during the post-receive hook","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2010-09-21T16:00:24Z","receivedAt":"2010-09-21T16:00:24Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Joshua Jensen <jjensen@workspacewhiz.com> wrote:\n> Let's ignore the review+submit portion of Gerrit now.  Can Gerrit be  \n> coaxed to take a refs/for/master and directly apply it to the master  \n> branch WITHOUT the review cycle?  If so, then I'm wasting my time trying  \n> to right a script.\n\nNope.  But only because nobody has proposed doing that before.\nIts not impossible to do...\n\n-- \nShawn.\n"}]}