{"thread":{"id":"23199","subject":"CIA hook for contrib/","startedAt":"2010-03-27T10:26:32Z","lastAt":"2010-04-21T22:00:21Z","messageCount":8,"participants":["Eric Raymond","Junio C Hamano","Petr Baudis"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"137941","messageId":"20100327102632.GA5043@thyrsus.com","threadId":"23199","inReplyTo":null,"subject":"CIA hook for contrib/","fromName":"Eric Raymond","fromEmail":"esr@thyrsus.com","sentAt":"2010-03-27T10:26:32Z","receivedAt":"2010-03-27T10:26:32Z","isPatch":false,"sender":{"key":"esr@thyrsus.com","avatar":"https://avatars.githubusercontent.com/u/727961?v=4"},"body":"Summary:\n* The CIA hooks for git are presently dusty and somewhat broken.\n* I am requesting that they be given a home in git contrib/.\n* I am supplying working, updated versions.\n* I will accept maintainance responsibility if required.\n\nMany listmembers will be aware of CIA.vc, a useful service that echoes\ncommits from open-source projects to IRC channels and maintains\nvarious interesting statistics on commit activity. Under git, it is\nsupported by one of two hook scripts: ciabot.pl and ciabot.pl.\n\nIn the course of migrating the GPSD project from Subversion to git, I\ndiscovered that the 2006 version of ciabot.sh available from CIA.vc is\nbroken. It requires, but does not have, a PATH adjustment to cope\nwith the fact that various git tools it uses now go in a commands\ndirectory private to git. The 2008 version of ciabot.sh made\navailable on the git wiki also has no PATH adjustment.\n\nUpon investigating further, I find that ciabot.sh seems to be a\nderivative of ciabot.pl, which is a dangling bit of the moribund\nCogito project. I have been unable to get responses from the authors\nof either ciabot.sh or ciabot.pl.\n\nA fixed and working version in shell is enclosed. It has some advantages\nover the 2006 and 2008 versions:\n\n* It uses only POSIX features. Thus, it will work with Debian/Ubuntu\n dash and BSD's non-bash shells.\n\n* It's faster, because it uses a lighter-weight shell.\n\n* It's much closer to being autoconfiguring. In the normal case, all\n the person installing an instance need supply is the project name.\n\n* It is considerably better documented.\n\n* It can support cgit as well as gitweb installations.\n\nI am also enclosing a rewrite in Python. This has a serious functional\nadvantage over either Perl or sh; the batteries-included effect of the\nPython libraries means it's not dependent on things like the path\nlocation of sendmail or whether the hosting site will allow it to\nrun wget.\n\nI think the broad usage of CIA.vc justifies adding one of these as an\nofficial ciabot hook to git contrib/, and am so proposing. Should it\nbe required, I will accept maintainance responsibility; I expect the\namount of effort involved to be minimal.\n\nBoth versions have been live-tested on the GPSD project repo. The\nPython version is now in production use there.\n\nCode is enclosed.\n-- \n\t\tEric S. Raymond\n\n\n#!/usr/bin/env python\n# Distributed under the terms of the GNU General Public License v2\n# Copyright (c) 2010 Eric S. Raymond \n#\n# This script contains porcelain and porcelain byproducts.\n# It's Python because the Python standard libraries avoid portability/security\n# issues raised by callouts in the ancestral Perl and sh scripts. It should\n# be compatible back to Python 2.1.5.\n#\n# It is meant to be run either on a post-commit hook or in an update\n# hook:\n#\n# post-commit: It queries for current HEAD and latest commit ID to get the\n# information it needs.\n#\n# update: You have to call it once per merged commit:\n#\n# refname=$1\n# oldhead=$2\n# newhead=$3\n# for merged in $(git rev-list ${oldhead}..${newhead} | tac) ; do\n# /path/to/ciabot.py ${refname} ${merged}\n# done\n#\n# Note: this script uses mail, not XML-RPC, in order to avoid stalling\n# until timeout when the XML-RPC server is down. \n#\n# Call with -n to see the notification mail dumped to stdout rather\n# than shipped to CIA.\n#\nimport os, sys, commands, socket, urllib\n#\n# The project as known to CIA. You will want to change this:\n#\nproject=\"GPSD\"\n\n#\n# You may not need to change these:\n#\n\n# Name of the repository.\n# You can hardwire this to make the script faster.\nrepo = os.path.basename(os.getcwd())\n\n# Fully-qualified domain name of this host.\n# You can hardwire this to make the script faster.\nhost = socket.getfqdn()\n\n# Changeset URL prefix for your repo: when the commit ID is appended\n# to this, it should point at a CGI that will display the commit\n# through gitweb or something similar. The default will probably\n# work if you have a typical gitweb/cgit setup.\n#\n#urlprefix=\"http://%(host)s/cgi-bin/gitweb.cgi?p=%(repo)s;a=commit;h=\"%locals()\nurlprefix=\"http://%(host)s/cgi-bin/cgit.cgi/%(repo)s/commit/?id=\"%locals()\n\n#\n# No user-serviceable parts below this line:\n#\n\ndef do(command):\n return commands.getstatusoutput(command)[1]\n\n# Addresses for the e-mail. The from address is a dummy, since CIA\n# will never reply to this mail.\nfromaddr = \"CIABOT-NOREPLY@\" + host\ntoaddr = \"cia@cia.vc\"\n\n# Identify the generator script.\n# Should only change when the script itself has a new home\ngenerator=\"http://www.catb.org/~esr/ciabot.sh\"\n\n# Should add to the command path both places sendmail is likely to lurk, \n# and the git private command directory.\nos.environ[\"PATH\"] += \":/usr/sbin/:\" + do(\"git --exec-path\")\n\n# Call this script with -n tp dump the notification mail to stdout\nmailit = True \nif sys.argv[1] == '-n':\n mailit = False\n sys.argv.pop(1)\n\n# Script wants a reference to head followed by the commit ID to notify about. \nif len(sys.argv) == 1:\n refname = do(\"git symbolic-ref HEAD 2>/dev/null\")\n merged = do(\"git rev-parse HEAD\")\nelse:\n refname = sys.argv[1]\n merged = sys.argv[2]\n\n# This tries to turn your gitwebbish URL into a tinyurl so it will take up\n# less space on the IRC notification line. You can tweak the tinyfier\n# service if you need to.\ntinyifier = \"http://tinyurl.com/api-create.php?url=\"\ntry:\n url = open(urllib.urlretrieve(tinyifier + urlprefix + merged)[0]).read()\nexcept:\n url = urlprefix + merged\n\nrefname = os.path.basename(refname)\ngitver = do(\"git --version\").split()[0]\t# Git version number\n\nrev = do(\"git describe ${merged} 2>/dev/null\") or merged[:12]\nrawcommit = do(\"git cat-file commit \" + merged)\nfiles=do(\"git diff-tree -r --name-only '\"+ merged +\"' | sed -e '1d' -e 's-.*-&-'\")\n\ninheader = True\nheaders = {}\nlogmessage = \"\"\nfor line in rawcommit.split(\"\\n\"):\n if inheader:\n if line:\n fields = line.split()\n headers[fields[0]] = \" \".join(fields[1:])\n else:\n inheader = False\n else:\n logmessage = line\n break\n(author, ts) = headers[\"author\"].split(\">\")\nauthor = author.replace(\"<\", \"\").split(\"@\")[0].split()[-1]\nts = ts.strip()\n\nout = '''\\\n\n \n CIA Shell client for Git\n %(gitver)s\n %(generator)s\n \n \n %(project)s\n %(repo)s:%(refname)s\n \n %(ts)s\n \n \n %(author)s\n %(rev)s\n \n %(files)s\n \n %(logmessage)s %(url)s\n %(url)s\n \n \n\n''' % locals()\n\nmessage = '''\\\nMessage-ID: <%(merged)s.%(author)s@%(project)s>\nFrom: %(fromaddr)s\nTo: %(toaddr)s\nContent-type: text/xml\nSubject: DeliverXML\n\n%(out)s''' % locals()\n\nif mailit:\n import smtplib\n server = smtplib.SMTP('localhost')\n server.sendmail(fromaddr, [toaddr], message)\n server.quit()\nelse:\n print message\n\n#End\n"},{"id":"138054","messageId":"7veij3bqnq.fsf@alter.siamese.dyndns.org","threadId":"23199","inReplyTo":"20100327102632.GA5043@thyrsus.com","subject":"Re: CIA hook for contrib/","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-03-29T06:14:33Z","receivedAt":"2010-03-29T06:14:33Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Raymond writes:\n\n> Summary:\n> * The CIA hooks for git are presently dusty and somewhat broken.\n\nThanks; but I don't ship any ancient hooks, so this probably wants to be\nsent to whoever still have them displayed on their sites.\n\n> * I am requesting that they be given a home in git contrib/.\n\nI gave a quick read to the shell one, and I found it reasonable. Thanks\nfor sharing.\n\nOne tiny nit is the \"git --exec-path\" bit, though. You don't seem to use\nany ancient \"git-frotz\" form, so it is not just unnecessary but is\nmisleading. I'd recommend to just drop it.\n\nAnother tiny nit is that you _might_ want to make it fail loudly if\nsomebody copies this out of contrib/ without changing $project properly,\ninstead of spamming cia with bogus messages claiming that they pertain to\nthe GPSD project.\n\n> I am also enclosing a rewrite in Python. This has a serious functional\n> advantage over either Perl or sh; the batteries-included effect of the\n> Python libraries means it's not dependent on things like the path\n> location of sendmail or whether the hosting site will allow it to\n> run wget.\n> ...\n> Code is enclosed.\n\nThanks. Could you send a signed-off patch that creates contrib/ciabot/\ndirectory, and places these two files in there, with a README describing\nthat you are the maintainer of these tools?\n"},{"id":"138063","messageId":"20100329090413.GA10538@thyrsus.com","threadId":"23199","inReplyTo":"7veij3bqnq.fsf@alter.siamese.dyndns.org","subject":"Re: CIA hook for contrib/","fromName":"Eric Raymond","fromEmail":"esr@thyrsus.com","sentAt":"2010-03-29T09:04:13Z","receivedAt":"2010-03-29T09:04:13Z","isPatch":false,"sender":{"key":"esr@thyrsus.com","avatar":"https://avatars.githubusercontent.com/u/727961?v=4"},"body":"Hm...this showed up about two minutes after I sent you private mail \nwondering why nobody had responded to my post. Sorry about that.\n\nJunio C Hamano :\n> Eric Raymond writes:\n> \n> > Summary:\n> > * The CIA hooks for git are presently dusty and somewhat broken.\n> \n> Thanks; but I don't ship any ancient hooks, so this probably wants to be\n> sent to whoever still have them displayed on their sites.\n\nThe ancient hooks are in scattered locations - one in the Cogito repo, another\non some random project site called alphine, and teo stale copies on CIA.vc.\nI'm trying to getthese updated.\n \n> One tiny nit is the \"git --exec-path\" bit, though. You don't seem to use\n> any ancient \"git-frotz\" form, so it is not just unnecessary but is\n> misleading. I'd recommend to just drop it.\n\nI will test and drop that if possible.\n \n> Another tiny nit is that you _might_ want to make it fail loudly if\n> somebody copies this out of contrib/ without changing $project properly,\n> instead of spamming cia with bogus messages claiming that they pertain to\n> the GPSD project.\n\nGood idea. Will do.\n\n> > I am also enclosing a rewrite in Python. This has a serious functional\n> > advantage over either Perl or sh; the batteries-included effect of the\n> > Python libraries means it's not dependent on things like the path\n> > location of sendmail or whether the hosting site will allow it to\n> > run wget.\n> > ...\n> > Code is enclosed.\n> \n> Thanks. Could you send a signed-off patch that creates contrib/ciabot/\n> directory, and places these two files in there, with a README describing\n> that you are the maintainer of these tools?\n\nI will do so.\n-- \n\t\tEric S. Raymond\n"},{"id":"140024","messageId":"20100421101002.GD3563@machine.or.cz","threadId":"23199","inReplyTo":"20100327102632.GA5043@thyrsus.com","subject":"Re: CIA hook for contrib/","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2010-04-21T10:10:02Z","receivedAt":"2010-04-21T10:10:02Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":" Hi!\n\nOn Sat, Mar 27, 2010 at 06:26:32AM -0400, Eric Raymond wrote:\n> Upon investigating further, I find that ciabot.sh seems to be a\n> derivative of ciabot.pl, which is a dangling bit of the moribund\n> Cogito project. I have been unable to get responses from the authors\n> of either ciabot.sh or ciabot.pl.\n\n Strange, I received no mail from you. When did you send it?\n\n> A fixed and working version in shell is enclosed.\n\n It would have been easier to review if it was quotable. ;-)\n(I have also looked at the current checked in version.)\n\n I'm happy that someone resurrected the hook script, thanks for that!\nBy the way, you seem to drop support for XML::RPC altogether while the\noriginal ciabot.pl could use both. While XML::RPC requires another\ndependency (not sure if plain Python installation can do it) and it can\ntime out in case of CIA server trouble (not sure how common these are\nnowadays), the distinct advantage is that the commits will always end up\nin correct order, while it seemed to be common that by mail, push of\nmultiple commits would reorder them randomly.\n\n> # update: You have to call it once per merged commit:\n> #\n> # refname=$1\n> # oldhead=$2\n> # newhead=$3\n> # for merged in $(git rev-list ${oldhead}..${newhead} | tac) ; do\n> # /path/to/ciabot.bash ${refname} ${merged}\n ^^^^^^^^^^^^^^^^^^^^\n\nObsolete .bash reference.\n\n> rev=$(git describe ${merged} 2>/dev/null)\n> # ${merged:0:12} was the only bashism left in the 2008 version of this\n> # script, according to checkbashisms. Replace it with ${merged} here\n> # because it was just a fallback anyway, and it's worth taking accepting\n> # a longer fallback for faster execution and removing the bash deoendency.\n> [ -z ${rev} ] && rev=${merged}\n\nI personally find git-describe'd revspecs extremely ugly, unreadable and\nless practical than plain hex ids (maybe I'm weird?), it would be really\nnice to get a configuration choice between git describe and just:\n\n\trev=$(echo \"$merged\" | cut -c 1-12)\n\n(Not sure about the bashism rant since you can trivially just replace it\nwith the cut.)\n\n> # This discards the part of the author's address after @.\n> # Might be nice to ship the full email address, if not\n> # for spammers' address harvesters - getting this wrong\n> # would make the freenode #commits channel into harvester heaven.\n\nThe comment is not accurate since that code seems to be doing a lot of\nother things as well.\n\nI would personally prefer to have this configurable; I consider trying\nto protect your e-mail address against harvesters is a lost fight anyway\nand you'd be much better off just getting a good spam filter, rather\nthan making yours and others' life harder by trying to fight in vain.\nBut it's not too important for me since I can just disable this easily.\n\n> rawcommit=$(git cat-file commit ${merged})\n> author=$(echo \"$rawcommit\" | sed -n -e '/^author .*<\\([^@]*\\).*$/s--\\1-p')\n> logmessage=$(echo \"$rawcommit\" | sed -e '1,/^$/d' | head -n 1)\n> logmessage=$(echo \"$logmessage\" | sed 's/\\&/&\\;/g; s//>\\;/g')\n> ts=$(echo \"$rawcommit\" | sed -n -e '/^author .*> \\([0-9]\\+\\).*$/s--\\1-p')\n\nYour life would be much easier in both the shell and python script if\nyou used something like:\n\n\tgit log -1 '--pretty=format:%an <%ae>%n%at%n%s'\n\nYou would also get in the correct format, incl. timezone correction.\n\n Kind regards,\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nhttp://pasky.or.cz/ | \"Ars longa, vita brevis.\" -- Hippocrates\n"},{"id":"140038","messageId":"20100421155149.GA11223@thyrsus.com","threadId":"23199","inReplyTo":"20100421101002.GD3563@machine.or.cz","subject":"Re: CIA hook for contrib/","fromName":"Eric Raymond","fromEmail":"esr@thyrsus.com","sentAt":"2010-04-21T15:51:49Z","receivedAt":"2010-04-21T15:51:49Z","isPatch":false,"sender":{"key":"esr@thyrsus.com","avatar":"https://avatars.githubusercontent.com/u/727961?v=4"},"body":"Petr Baudis :\n> On Sat, Mar 27, 2010 at 06:26:32AM -0400, Eric Raymond wrote:\n> > Upon investigating further, I find that ciabot.sh seems to be a\n> > derivative of ciabot.pl, which is a dangling bit of the moribund\n> > Cogito project. I have been unable to get responses from the authors\n> > of either ciabot.sh or ciabot.pl.\n> \n> Strange, I received no mail from you. When did you send it?\n\nAbout three weeks ago now. Sorry, it appears I fat-fingered your\naddress. I did eventually hear from the author of the 2008 sh\nversion; he passed me the maintainer's baton on that one.\n \n> I'm happy that someone resurrected the hook script, thanks for that!\n> By the way, you seem to drop support for XML::RPC altogether while the\n> original ciabot.pl could use both. While XML::RPC requires another\n> dependency (not sure if plain Python installation can do it) and it can\n> time out in case of CIA server trouble (not sure how common these are\n> nowadays), the distinct advantage is that the commits will always end up\n> in correct order, while it seemed to be common that by mail, push of\n> multiple commits would reorder them randomly.\n\nAh, I did not know of that advantage, it wasn't documented anywhere.\nIt's a sufficient reason to bring back XML-RPC support...and I\nhave just done so in the Python version.\n\n> > # update: You have to call it once per merged commit:\n> > #\n> > # refname=$1\n> > # oldhead=$2\n> > # newhead=$3\n> > # for merged in $(git rev-list ${oldhead}..${newhead} | tac) ; do\n> > # /path/to/ciabot.bash ${refname} ${merged}\n> ^^^^^^^^^^^^^^^^^^^^\n> \n> Obsolete .bash reference.\n\nFixed, thanks.\n\n> I personally find git-describe'd revspecs extremely ugly, unreadable and\n> less practical than plain hex ids (maybe I'm weird?), it would be really\n> nice to get a configuration choice between git describe and just:\n> \n> \trev=$(echo \"$merged\" | cut -c 1-12)\n> \n> (Not sure about the bashism rant since you can trivially just replace it\n> with the cut.)\n\nGood point about use of cut; I'll do that. \n\nI've implemented a revformat variable, but defaulted it to 'describe'.\n \n> I would personally prefer to have this configurable; I consider trying\n> to protect your e-mail address against harvesters is a lost fight anyway\n> and you'd be much better off just getting a good spam filter, rather\n> than making yours and others' life harder by trying to fight in vain.\n> But it's not too important for me since I can just disable this easily.\n\nBut email name collisions within projects are vanishingly rare, so I\ndon't see a lot of benefit in publishing the FQDN.\n \n> Your life would be much easier in both the shell and python script if\n> you used something like:\n> \n> \tgit log -1 '--pretty=format:%an <%ae>%n%at%n%s'\n> \n> You would also get in the correct format, incl. timezone correction.\n\nGood point. I inherited that nasty code; perhaps this facility did not\nexist when it was written.\n-- \n\t\tEric S. Raymond\n"},{"id":"140052","messageId":"20100421210427.GF3563@machine.or.cz","threadId":"23199","inReplyTo":"20100421155149.GA11223@thyrsus.com","subject":"Re: CIA hook for contrib/","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2010-04-21T21:04:27Z","receivedAt":"2010-04-21T21:04:27Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Thanks for doing all the changes!\n\nOn Wed, Apr 21, 2010 at 11:51:49AM -0400, Eric Raymond wrote:\n> Petr Baudis :\n> > I would personally prefer to have this configurable; I consider trying\n> > to protect your e-mail address against harvesters is a lost fight anyway\n> > and you'd be much better off just getting a good spam filter, rather\n> > than making yours and others' life harder by trying to fight in vain.\n> > But it's not too important for me since I can just disable this easily.\n> \n> But email name collisions within projects are vanishingly rare, so I\n> don't see a lot of benefit in publishing the FQDN.\n\nThat's a good point, I guess I was just ranting in general. ;-)\n\n> > Your life would be much easier in both the shell and python script if\n> > you used something like:\n> > \n> > \tgit log -1 '--pretty=format:%an <%ae>%n%at%n%s'\n> > \n> > You would also get in the correct format, incl. timezone correction.\n> \n> Good point. I inherited that nasty code; perhaps this facility did not\n> exist when it was written.\n\nIt most probably wasn't, the original perl script in particular was\nwritten at the very dawn of time from git history perspective.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nhttp://pasky.or.cz/ | \"Ars longa, vita brevis.\" -- Hippocrates\n"},{"id":"140063","messageId":"20100421215510.GB16643@thyrsus.com","threadId":"23199","inReplyTo":"20100421210427.GF3563@machine.or.cz","subject":"Re: CIA hook for contrib/","fromName":"Eric Raymond","fromEmail":"esr@thyrsus.com","sentAt":"2010-04-21T21:55:10Z","receivedAt":"2010-04-21T21:55:10Z","isPatch":false,"sender":{"key":"esr@thyrsus.com","avatar":"https://avatars.githubusercontent.com/u/727961?v=4"},"body":"Petr Baudis :\n> Thanks for doing all the changes!\n\nThanks for the review. You pushed me into doing two things I was\nnaggingly aware I ought to do but hadn't gotten around to: (1) XML-RPC\nsupport from the Python version (alas, it's too hard from shell), and\n(2) fixing the scripts so that they can be entirely informed by config\nvariables, no modifications or command-line switches required.\n\nYou can read about the current state of play at \n\n .\n\nI've spent the last couple of hours testing\n-- \n\t\tEric S. Raymond\n"},{"id":"140067","messageId":"20100421220021.GA16820@thyrsus.com","threadId":"23199","inReplyTo":"20100421215510.GB16643@thyrsus.com","subject":"Re: CIA hook for contrib/","fromName":"Eric Raymond","fromEmail":"esr@thyrsus.com","sentAt":"2010-04-21T22:00:21Z","receivedAt":"2010-04-21T22:00:21Z","isPatch":false,"sender":{"key":"esr@thyrsus.com","avatar":"https://avatars.githubusercontent.com/u/727961?v=4"},"body":"Eric Raymond :\n> I've spent the last couple of hours testing\n\nSentence truncated...\n\nI've spent the last couple of hours testing and epect to mail a patch\nto the list shortly. Well, after I get dinner, now.\n-- \n\t\tEric S. Raymond\n"}]}