{"thread":{"id":"38263","subject":"saving \"git push --signed\" certificate blobs","startedAt":"2014-12-30T04:09:42Z","lastAt":"2015-01-01T01:59:14Z","messageCount":3,"participants":["Sitaram Chamarty","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"254186","messageId":"54A22586.70001@gmail.com","threadId":"38263","inReplyTo":"54A10ED1.9020704@gmail.com","subject":"saving \"git push --signed\" certificate blobs","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2014-12-30T04:09:42Z","receivedAt":"2014-12-30T04:09:42Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"Hello,\n\nJust wanted to say there's a little script at [1] that saves the certificate\nblobs generated on the server side by \"git push --signed\".\n\nQuoting from the source:\n\n# Collects the cert blob on push and saves it, then, if a certain number of\n# signed pushes have been seen, processes all the \"saved\" blobs in one go,\n# adding them to the special ref 'refs/push-certs'.  This is done in a way\n# that allows searching for all the certs pertaining to one specific branch\n# (thanks to Junio Hamano for this idea plus general brainstorming).\n\nNote that although I posted it in the gitolite ML, this has very little to do\nwith gitolite.  Any git server can use it, with only one very minor change [2]\nneeded.\n\nsitaram\n\n[1]: https://groups.google.com/forum/#!topic/gitolite/7cSrU6JorEY\n\n[2]: Either set the GL_OPTIONS_GPC_PENDING environment variable by reading its\nvalue from 'git config', or replace the only line that uses that variable, with\nsome other \"test\".\n"},{"id":"254203","messageId":"xmqqiogtrptu.fsf@gitster.dls.corp.google.com","threadId":"38263","inReplyTo":"54A22586.70001@gmail.com","subject":"Re: saving \"git push --signed\" certificate blobs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-12-30T17:48:45Z","receivedAt":"2014-12-30T17:48:45Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Sitaram Chamarty <sitaramc@gmail.com> writes:\n\n> Just wanted to say there's a little script at [1] that saves the certificate\n> blobs generated on the server side by \"git push --signed\".\n>\n> Quoting from the source:\n>\n> # Collects the cert blob on push and saves it, then, if a certain number of\n> # signed pushes have been seen, processes all the \"saved\" blobs in one go,\n> # adding them to the special ref 'refs/push-certs'.  This is done in a way\n> # that allows searching for all the certs pertaining to one specific branch\n> # (thanks to Junio Hamano for this idea plus general brainstorming).\n>\n> Note that although I posted it in the gitolite ML, this has very little to do\n> with gitolite.  Any git server can use it, with only one very minor change [2]\n> needed.\n>\n> sitaram\n>\n> [1]: https://groups.google.com/forum/#!topic/gitolite/7cSrU6JorEY\n>\n> [2]: Either set the GL_OPTIONS_GPC_PENDING environment variable by reading its\n> value from 'git config', or replace the only line that uses that variable, with\n> some other \"test\".\n\nNicely done.\n\nWe'd need to give you a tool to make it easy to create a \"validated\nchain of certificates\" out of\n\n    $ git log refs/push-certs -- refs/heads/master\n\nto make the history this script creates truly useful, but I think it\nis a very good start.\n\nI can see that you tried to make the log output \"human readable\" by\nreformatting $cf, I am not sure if it gives us much value.  I would\nhave expected that you would just use the blob contents for the log\nmessage as-is, so that\n\n    $ git log --pretty=raw refs/push-certs -- refs/heads/master |\n      validate-cert-chain\n\ncan just work on blobs (shown in the \"log\" output) without having to\nextract the blobs by doing something like\n\n    $ git rev-list refs/push-certs -- refs/heads/master |\n      while read commit\n      do\n\t\tgit cat-file blob $commit:refs/heads/master |\n                validate-cert\n      done\n\nBy the way, you seem to like \"cat\" too much, though.  You don't have\nto cat a single file into a pipeline.\n\nThanks.\n"},{"id":"254243","messageId":"54A4A9F2.6020601@gmail.com","threadId":"38263","inReplyTo":"xmqqiogtrptu.fsf@gitster.dls.corp.google.com","subject":"Re: saving \"git push --signed\" certificate blobs","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2015-01-01T01:59:14Z","receivedAt":"2015-01-01T01:59:14Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On 12/30/2014 11:18 PM, Junio C Hamano wrote:\n> Sitaram Chamarty <sitaramc@gmail.com> writes:\n> \n>> Just wanted to say there's a little script at [1] that saves the certificate\n>> blobs generated on the server side by \"git push --signed\".\n>>\n>> Quoting from the source:\n>>\n>> # Collects the cert blob on push and saves it, then, if a certain number of\n>> # signed pushes have been seen, processes all the \"saved\" blobs in one go,\n>> # adding them to the special ref 'refs/push-certs'.  This is done in a way\n>> # that allows searching for all the certs pertaining to one specific branch\n>> # (thanks to Junio Hamano for this idea plus general brainstorming).\n>>\n>> Note that although I posted it in the gitolite ML, this has very little to do\n>> with gitolite.  Any git server can use it, with only one very minor change [2]\n>> needed.\n>>\n>> sitaram\n>>\n>> [1]: https://groups.google.com/forum/#!topic/gitolite/7cSrU6JorEY\n>>\n>> [2]: Either set the GL_OPTIONS_GPC_PENDING environment variable by reading its\n>> value from 'git config', or replace the only line that uses that variable, with\n>> some other \"test\".\n> \n> Nicely done.\n> \n> We'd need to give you a tool to make it easy to create a \"validated\n> chain of certificates\" out of\n> \n>     $ git log refs/push-certs -- refs/heads/master\n> \n> to make the history this script creates truly useful, but I think it\n> is a very good start.\n> \n> I can see that you tried to make the log output \"human readable\" by\n> reformatting $cf, I am not sure if it gives us much value.  I would\n> have expected that you would just use the blob contents for the log\n> message as-is, so that\n> \n>     $ git log --pretty=raw refs/push-certs -- refs/heads/master |\n>       validate-cert-chain\n> \n> can just work on blobs (shown in the \"log\" output) without having to\n> extract the blobs by doing something like\n> \n>     $ git rev-list refs/push-certs -- refs/heads/master |\n>       while read commit\n>       do\n> \t\tgit cat-file blob $commit:refs/heads/master |\n>                 validate-cert\n>       done\n\nI see what you mean.  And it looks like using \"--format=%B also works\npretty well.  Will fix.\n\n> By the way, you seem to like \"cat\" too much, though.  You don't have\n> to cat a single file into a pipeline.\n\nGee I hope Randal Schwartz is not on this list :)\n\nAnyway the previous fix also removes most of them.\n\nI'm attaching the current version so non-gitolite users can find it\nwithout having to go to the gitolite repo.  For gitolite users, it's\nsomewhere in \"contrib/\" in the source tree.\n\nsitaram\n\n> Thanks.\n> \n\n\n\n#!/bin/sh\n\n# ----------------------------------------------------------------------\n# post-receive hook to adopt push certs into 'refs/push-certs'\n\n# Collects the cert blob on push and saves it, then, if a certain number of\n# signed pushes have been seen, processes all the \"saved\" blobs in one go,\n# adding them to the special ref 'refs/push-certs'.  This is done in a way\n# that allows searching for all the certs pertaining to one specific branch\n# (thanks to Junio Hamano for this idea plus general brainstorming).\n\n# The \"collection\" happens only if $GIT_PUSH_CERT_NONCE_STATUS = OK; again,\n# thanks to Junio for pointing this out; see [1]\n#\n# [1]: https://groups.google.com/forum/#!topic/gitolite/7cSrU6JorEY\n\n# WARNINGS:\n#   Does not check that GIT_PUSH_CERT_STATUS = \"G\".  If you want to check that\n#   and FAIL the push, you'll have to write a simple pre-receive hook\n#   (post-receive is not the place for that; see 'man githooks').\n#\n#   Gitolite users: failing the hook cannot be done as a VREF because git does\n#   not set those environment variables in the update hook.  You'll have to\n#   write a trivial pre-receive hook and add that in.\n\n# Relevant gitolite doc links:\n#   repo-specific environment variables\n#       http://gitolite.com/gitolite/dev-notes.html#rsev\n#   repo-specific hooks\n#       http://gitolite.com/gitolite/non-core.html#rsh\n#       http://gitolite.com/gitolite/cookbook.html#v3.6-variation-repo-specific-hooks\n\n# Environment:\n#   GIT_PUSH_CERT_NONCE_STATUS should be \"OK\" (as mentioned above)\n#\n#   GL_OPTIONS_GPC_PENDING (optional; defaults to 1).  This is the number of\n#   git push certs that should be waiting in order to trigger the post\n#   processing.  You can set it within gitolite like so:\n#\n#       repo foo bar    # or maybe just 'repo @all'\n#           option ENV.GPC_PENDING = 5\n\n# Setup:\n#   Set up this code as a post-receive hook for whatever repos you need to.\n#   Then arrange to have the environment variable GL_OPTION_GPC_PENDING set to\n#   some number, as shown above.  (This is only required if you need it to be\n#   greater than 1.)  It could of course be different for different repos.\n#   Also see \"Invocation\" section below.\n\n# Invocation:\n#   Normally via git (see 'man githooks'), once it is setup as a post-receive\n#   hook.\n#\n#   However, if you set the \"pending\" limit high, and want to periodically\n#   \"clean up\" pending certs without necessarily waiting for the counter to\n#   trip, do the following (untested):\n#\n#       RB=$(gitolite query-rc GL_REPO_BASE)\n#       for r in $(gitolite list-phy-repos)\n#       do\n#           cd $RB/$repo.git\n#           unset GL_OPTIONS_GPC_PENDING    # if it is set higher up\n#           hooks/post-receive post_process\n#       done\n#\n#   That will take care of it.\n\n# Using without gitolite:\n#   Just set GL_OPTIONS_GPC_PENDING within the script (maybe read it from git\n#   config).  Everything else is independent of gitolite.\n\n# ----------------------------------------------------------------------\n# make it work on BSD also (but NOT YET TESTED on FreeBSD!)\nuname_s=`uname -s`\nif [ \"$uname_s\" = \"Linux\" ]\nthen\n    _lock() { flock \"$@\"; }\nelse\n    _lock() { lockf -k \"$@\"; }\n    # I'm assuming other BSDs also have this; I only have FreeBSD.\nfi\n\n# ----------------------------------------------------------------------\n# standard stuff\ndie() { echo \"$@\" >&2; exit 1; }\nwarn() { echo \"$@\" >&2; }\n\n# ----------------------------------------------------------------------\n# if there are no arguments, we're running as a \"post-receive\" hook\nif [ -z \"$1\" ]\nthen\n    # ignore if it may be a replay attack\n    [ \"$GIT_PUSH_CERT_NONCE_STATUS\" = \"OK\" ] || exit 1\n    # I don't think \"exit 1\" does anything in a post-receive anyway, so that's\n    # just a symbolic gesture!\n\n    # note the lock file used\n    _lock .gpc.lock $0 cat_blob\n\n    # if you want to initiate the post-processing ONLY from outside (for\n    # example via cron), comment out the next line.\n    exec $0 post_process\nfi\n\n# ----------------------------------------------------------------------\n# the 'post_process' part; see \"Invocation\" section in the doc at the top\nif [ \"$1\" = \"post_process\" ]\nthen\n    # this is the same lock file as above\n    _lock .gpc.lock $0 count_and_rotate $$\n\n    [ -d git-push-certs.$$ ] || exit 0\n\n    # but this is a different one\n    _lock .gpc.ref.lock $0 update_ref $$\n\n    exit 0\nfi\n\n# ----------------------------------------------------------------------\n# other values for \"$1\" are internal use only\n\nif [ \"$1\" = \"cat_blob\" ]\nthen\n    mkdir -p git-push-certs\n    git cat-file blob $GIT_PUSH_CERT > git-push-certs/$GIT_PUSH_CERT\n    echo $GIT_PUSH_CERT >> git-push-certs/.blob.list\nfi\n\nif [ \"$1\" = \"count_and_rotate\" ]\nthen\n    count=$(ls git-push-certs | wc -l)\n    if test $count -ge ${GL_OPTIONS_GPC_PENDING:-1}\n    then\n        # rotate the directory\n        mv git-push-certs git-push-certs.$2\n    fi\nfi\n\nif [ \"$1\" = \"update_ref\" ]\nthen\n    # use a different index file for all this\n    GIT_INDEX_FILE=push_certs_index; export GIT_INDEX_FILE\n\n    # prepare the special ref to receive commits\n    PUSH_CERTS=refs/push-certs\n    if git rev-parse -q --verify $PUSH_CERTS >/dev/null\n    then\n        git read-tree $PUSH_CERTS\n    else\n        git read-tree --empty\n        T=$(git write-tree)\n        C=$(echo 'start' | git commit-tree $T)\n        git update-ref $PUSH_CERTS $C\n    fi\n\n    # for each cert blob...\n    for b in `cat git-push-certs.$2/.blob.list`\n    do\n        cf=git-push-certs.$2/$b\n\n        # it's highly unlikely that the blob got GC-ed already but write it\n        # back anyway, just in case\n        B=$(git hash-object -w $cf)\n\n        # bit of a sanity check\n        [ \"$B\" = \"$b\" ] || warn \"this should not happen: $B is not equal to $b\"\n\n        # for each ref described within the cert, update the index\n        for ref in `cat $cf | egrep '^[a-f0-9]+ [a-f0-9]+ refs/' | cut -f3 -d' '`\n        do\n            git update-index --add --cacheinfo 100644,$b,$ref\n            # we're using the ref name as a \"fake\" filename, so people can,\n            # for example, 'git log refs/push-certs -- refs/heads/master', to\n            # see all the push certs pertaining to the master branch.  This\n            # idea came from Junio Hamano, the git maintanier (I certainly\n            # don't deal with git plumbing enough to have thought of it!)\n        done\n\n        T=$(git write-tree)\n        C=$( git commit-tree -p $PUSH_CERTS $T < $cf )\n        git update-ref $PUSH_CERTS $C\n\n        rm -f $cf\n    done\n    rm -f git-push-certs.$2/.blob.list\n    rmdir git-push-certs.$2\nfi\n"}]}