{"thread":{"id":"31312","subject":"[PATCH] Get ciabot configuration from git variables.","startedAt":"2012-08-22T10:52:30Z","lastAt":"2012-08-22T16:47:54Z","messageCount":2,"participants":["Eric S. Raymond","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"197574","messageId":"20120822105230.GA30382@thyrsus.com","threadId":"31312","inReplyTo":null,"subject":"[PATCH] Get ciabot configuration from git variables.","fromName":"Eric S. Raymond","fromEmail":"esr@thyrsus.com","sentAt":"2012-08-22T10:52:30Z","receivedAt":"2012-08-22T10:52:30Z","isPatch":true,"sender":{"key":"esr@thyrsus.com","avatar":"https://avatars.githubusercontent.com/u/727961?v=4"},"body":"These changes remove all need to modify the ciabot scripts for installation.\nInstead, per-project configuration can be dome via variables in a [ciabot]\nsection of the config file.\n\nAlso, correct for the new server address.\n\nSigned-off-by: Eric S. Raymond \n---\n contrib/ciabot/ciabot.py | 161 +++++++++++++++++++++++++---------------------\n contrib/ciabot/ciabot.sh | 117 +++++++++++++++++++--------------\n 2 files changed, 158 insertions(+), 120 deletions(-)\n\ndiff --git a/contrib/ciabot/ciabot.py b/contrib/ciabot/ciabot.py\nindex 9775dff..8ce04eb 100755\n--- a/contrib/ciabot/ciabot.py\n+++ b/contrib/ciabot/ciabot.py\n@@ -11,43 +11,41 @@\n #\n # This script is meant to be run either in a post-commit hook or in an\n # update hook. If there's nothing unusual about your hosting setup,\n-# you can specify the project name with a -p option and avoid having\n-# to modify this script. Try it with -n to see the notification mail\n-# dumped to stdout and verify that it looks sane. With -V it dumps its\n-# version and exits.\n+# you can specify the project name and repo with config variables and\n+# avoid having to modify this script. Try it with -n to see the\n+# notification mail dumped to stdout and verify that it looks\n+# sane. With -V it dumps its version and exits.\n #\n-# In post-commit, run it without arguments (other than possibly a -p\n-# option). It will query for current HEAD and the latest commit ID to\n-# get the information it needs.\n+# In post-commit, run it without arguments. It will query for\n+# current HEAD and the latest commit ID to get the information it\n+# needs.\n #\n # In update, call it with a refname followed by a list of commits:\n-# You want to reverse the order git rev-list emits becxause it lists\n+# You want to reverse the order git rev-list emits because it lists\n # from most recent to oldest.\n #\n # /path/to/ciabot.py ${refname} $(git rev-list ${oldhead}..${newhead} | tac)\n #\n-# Note: this script uses mail, not XML-RPC, in order to avoid stalling\n-# until timeout when the CIA XML-RPC server is down.\n+# Configuration variables affecting this script:\n+# ciabot.project = name of the project (required)\n+# ciabot.repo = name of the project repo for gitweb/cgit purposes\n+# ciabot.xmlrpc = if true (default), ship notifications via XML-RPC\n+# ciabot.revformat = format in which the revision is shown\n #\n-\n-#\n-# The project as known to CIA. You will either want to change this\n-# or invoke the script with a -p option to set it.\n+# The ciabot.repo value defaults to ciabot.project lowercased.\n #\n-project=None\n-\n+# The revformat variable may have the following values\n+# raw -> full hex ID of commit\n+# short -> first 12 chars of hex ID\n+# describe = -> describe relative to last tag, falling back to short\n+# The default is 'describe'.\n #\n-# You may not need to change these:\n+# Note: the CIA project now says only XML-RPC is reliable, so\n+# we default to that.\n #\n-import os, sys, commands, socket, urllib\n-\n-# Name of the repository.\n-# You can hardwire this to make the script faster.\n-repo = os.path.basename(os.getcwd())\n \n-# Fully-qualified domain name of this host.\n-# You can hardwire this to make the script faster.\n-host = socket.getfqdn()\n+import os, sys, commands, socket, urllib\n+from xml.sax.saxutils import escape\n \n # Changeset URL prefix for your repo: when the commit ID is appended\n # to this, it should point at a CGI that will display the commit\n@@ -72,7 +70,7 @@ xml = '''\\\n \n \n CIA Python client for Git\n- %(gitver)s\n+ %(version)s\n %(generator)s\n \n \n@@ -98,19 +96,18 @@ xml = '''\\\n # No user-serviceable parts below this line:\n #\n \n-# Addresses for the e-mail. The from address is a dummy, since CIA\n-# will never reply to this mail.\n-fromaddr = \"CIABOT-NOREPLY@\" + host\n-toaddr = \"cia@cia.navi.cx\"\n+# Where to ship e-mail notifications.\n+toaddr = \"cia@cia.vc\"\n \n # Identify the generator script.\n # Should only change when the script itself gets a new home and maintainer.\n-generator=\"http://www.catb.org/~esr/ciabot.py\"\n+generator = \"http://www.catb.org/~esr/ciabot.py\"\n+version = \"3.5\"\n \n def do(command):\n return commands.getstatusoutput(command)[1]\n \n-def report(refname, merged):\n+def report(refname, merged, xmlrpc=True):\n \"Generate a commit notification to be reported to CIA\"\n \n # Try to tinyfy a reference to a web view for this commit.\n@@ -121,32 +118,27 @@ def report(refname, merged):\n \n branch = os.path.basename(refname)\n \n- # Compute a shortnane for the revision\n- rev = do(\"git describe '\"+ merged +\"' 2>/dev/null\") or merged[:12]\n-\n- # Extract the neta-information for the commit\n- rawcommit = do(\"git cat-file commit \" + merged)\n+ # Compute a description for the revision\n+ if revformat == 'raw':\n+ rev = merged\n+ elif revformat == 'short':\n+ rev = ''\n+ else: # revformat == 'describe'\n+ rev = do(\"git describe %s 2>/dev/null\" % merged)\n+ if not rev:\n+ rev = merged[:12]\n+\n+ # Extract the meta-information for the commit\n files=do(\"git diff-tree -r --name-only '\"+ merged +\"' | sed -e '1d' -e 's-.*-&-'\")\n- inheader = True\n- headers = {}\n- logmsg = \"\"\n- for line in rawcommit.split(\"\\n\"):\n- if inheader:\n- if line:\n- fields = line.split()\n- headers[fields[0]] = \" \".join(fields[1:])\n- else:\n- inheader = False\n- else:\n- logmsg = line\n- break\n- (author, ts) = headers[\"author\"].split(\">\")\n+ metainfo = do(\"git log -1 '--pretty=format:%an <%ae>%n%at%n%s' \" + merged)\n+ (author, ts, logmsg) = metainfo.split(\"\\n\")\n+ logmsg = escape(logmsg)\n \n- # This discards the part of the authors addrsss after @.\n- # Might be bnicece to ship the full email address, if not\n+ # This discards the part of the author's address after @.\n+ # Might be be nice to ship the full email address, if not\n # for spammers' address harvesters - getting this wrong\n # would make the freenode #commits channel into harvester heaven.\n- author = author.replace(\"<\", \"\").split(\"@\")[0].split()[-1]\n+ author = escape(author.replace(\"<\", \"\").split(\"@\")[0].split()[-1])\n \n # This ignores the timezone. Not clear what to do with it...\n ts = ts.strip().split()[0]\n@@ -155,8 +147,7 @@ def report(refname, merged):\n context.update(globals())\n \n out = xml % context\n-\n- message = '''\\\n+ mail = '''\\\n Message-ID: <%(merged)s.%(author)s@%(project)s>\n From: %(fromaddr)s\n To: %(toaddr)s\n@@ -165,25 +156,40 @@ Subject: DeliverXML\n \n %(out)s''' % locals()\n \n- return message\n+ if xmlrpc:\n+ return out\n+ else:\n+ return mail\n \n if __name__ == \"__main__\":\n import getopt\n \n+ # Get all config variables\n+ revformat = do(\"git config --get ciabot.revformat\")\n+ project = do(\"git config --get ciabot.project\")\n+ repo = do(\"git config --get ciabot.repo\")\n+ xmlrpc = do(\"git config --get ciabot.xmlrpc\")\n+ xmlrpc = not (xmlrpc and xmlrpc == \"false\")\n+\n+ host = socket.getfqdn()\n+ fromaddr = \"CIABOT-NOREPLY@\" + host\n+\n try:\n- (options, arguments) = getopt.getopt(sys.argv[1:], \"np:V\")\n+ (options, arguments) = getopt.getopt(sys.argv[1:], \"np:xV\")\n except getopt.GetoptError, msg:\n print \"ciabot.py: \" + str(msg)\n raise SystemExit, 1\n \n- mailit = True\n+ notify = True\n for (switch, val) in options:\n if switch == '-p':\n project = val\n elif switch == '-n':\n- mailit = False\n+ notify = False\n+ elif switch == '-x':\n+ xmlrpc = True\n elif switch == '-V':\n- print \"ciabot.py: version 3.2\"\n+ print \"ciabot.py: version\", version\n sys.exit(0)\n \n # Cough and die if user has not specified a project\n@@ -191,8 +197,8 @@ if __name__ == \"__main__\":\n sys.stderr.write(\"ciabot.py: no project specified, bailing out.\\n\")\n sys.exit(1)\n \n- # We'll need the git version number.\n- gitver = do(\"git --version\").split()[0]\n+ if not repo:\n+ repo = project.lower()\n \n urlprefix = urlprefix % globals()\n \n@@ -205,18 +211,29 @@ if __name__ == \"__main__\":\n refname = arguments[0]\n merges = arguments[1:]\n \n- if mailit:\n- import smtplib\n- server = smtplib.SMTP('localhost')\n+ if notify:\n+ if xmlrpc:\n+ import xmlrpclib\n+ server = xmlrpclib.Server('http://cia.vc/RPC2');\n+ else:\n+ import smtplib\n+ server = smtplib.SMTP('localhost')\n \n for merged in merges:\n- message = report(refname, merged)\n- if mailit:\n- server.sendmail(fromaddr, [toaddr], message)\n- else:\n+ message = report(refname, merged, xmlrpc)\n+ if not notify:\n print message\n+ elif xmlrpc:\n+ try:\n+ # RPC server is flaky, this can fail due to timeout.\n+ server.hub.deliver(message)\n+ except socket.error, e:\n+ sys.stderr.write(\"%s\\n\" % e)\n+ else:\n+ server.sendmail(fromaddr, [toaddr], message)\n \n- if mailit:\n- server.quit()\n+ if notify:\n+ if not xmlrpc:\n+ server.quit()\n \n #End\ndiff --git a/contrib/ciabot/ciabot.sh b/contrib/ciabot/ciabot.sh\nindex eb87bba..39afb47 100755\n--- a/contrib/ciabot/ciabot.sh\n+++ b/contrib/ciabot/ciabot.sh\n@@ -3,6 +3,8 @@\n # Copyright (c) 2006 Fernando J. Pereda \n # Copyright (c) 2008 Natanael Copa \n # Copyright (c) 2010 Eric S. Raymond \n+# Assistance and review by Petr Baudis, author of ciabot.pl,\n+# is gratefully acknowledged.\n #\n # This is a version 3.x of ciabot.sh; use -V to find the exact\n # version. Versions 1 and 2 were shipped in 2006 and 2008 and are not\n@@ -11,6 +13,7 @@\n # Note: This script should be considered obsolete.\n # There is a faster, better-documented rewrite in Python: find it as ciabot.py\n # Use this only if your hosting site forbids Python hooks.\n+# It requires: git(1), hostname(1), cut(1), sendmail(1), and wget(1).\n #\n # Originally based on Git ciabot.pl by Petr Baudis.\n # This script contains porcelain and porcelain byproducts.\n@@ -19,14 +22,14 @@\n #\n # This script is meant to be run either in a post-commit hook or in an\n # update hook. If there's nothing unusual about your hosting setup,\n-# you can specify the project name with a -p option and avoid having\n-# to modify this script. Try it with -n first to see the notification\n-# mail dumped to stdout and verify that it looks sane. Use -V to dump\n-# the version and exit.\n+# you can specify the project name and repo with config variables and\n+# avoid having to modify this script. Try it with -n to see the\n+# notification mail dumped to stdout and verify that it looks\n+# sane. With -V it dumps its version and exits.\n #\n-# In post-commit, run it without arguments (other than possibly a -p\n-# option). It will query for current HEAD and the latest commit ID to\n-# get the information it needs.\n+# In post-commit, run it without arguments. It will query for\n+# current HEAD and the latest commit ID to get the information it\n+# needs.\n #\n # In update, you have to call it once per merged commit:\n #\n@@ -34,33 +37,54 @@\n # oldhead=$2\n # newhead=$3\n # for merged in $(git rev-list ${oldhead}..${newhead} | tac) ; do\n-# /path/to/ciabot.bash ${refname} ${merged}\n+# /path/to/ciabot.sh ${refname} ${merged}\n # done\n #\n-# The reason for the tac call ids that git rev-list emits commits from\n+# The reason for the tac call is that git rev-list emits commits from\n # most recent to least - better to ship notifactions from oldest to newest.\n #\n-# Note: this script uses mail, not XML-RPC, in order to avoid stalling\n-# until timeout when the CIA XML-RPC server is down.\n+# Configuration variables affecting this script:\n+# ciabot.project = name of the project (makes -p option unnecessary)\n+# ciabot.repo = name of the project repo for gitweb/cgit purposes\n+# ciabot.revformat = format in which the revision is shown\n #\n-\n-#\n-# The project as known to CIA. You will either want to change this\n-# or set the project name with a -p option.\n+# The ciabot.repo defaults to ciabot.project lowercased.\n #\n-project=\n-\n+# The revformat variable may have the following values\n+# raw -> full hex ID of commit\n+# short -> first 12 chars of hex ID\n+# describe = -> describe relative to last tag, falling back to short\n+# The default is 'describe'.\n #\n-# You may not need to change these:\n+# Note: the shell ancestors of this script used mail, not XML-RPC, in\n+# order to avoid stalling until timeout when the CIA XML-RPC server is\n+# down. It is unknown whether this is still an issue in 2010, but\n+# XML-RPC would be annoying to do from sh in any case. (XML-RPC does\n+# have the advantage that it guarantees notification of multiple commits\n+# shpped from an update in their actual order.)\n #\n \n-# Name of the repository.\n-# You can hardwire this to make the script faster.\n-repo=\"`basename ${PWD}`\"\n+# The project as known to CIA. You can also hardwire this or set it with a\n+# -p option.\n+project=$(git config --get ciabot.project)\n+\n+# Name of the repo for gitweb/cgit purposes\n+repo=$(git config --get ciabot.repo)\n+[ -z $repo] && repo=$(echo \"${project}\" | tr '[A-Z]' '[a-z]')\n+\n+# What revision format do we want in the summary?\n+revformat=$(git config --get ciabot.revformat)\n \n-# Fully qualified domain name of the repo host.\n-# You can hardwire this to make the script faster.\n-host=`hostname --fqdn`\n+# Fully qualified domain name of the repo host. You can hardwire this\n+# to make the script faster. The -f option works under Linux and FreeBSD,\n+# but not OpenBSD and NetBSD. But under OpenBSD and NetBSD,\n+# hostname without options gives the FQDN.\n+if hostname -f >/dev/null 2>&1\n+then\n+ hostname=`hostname -f`\n+else\n+ hostname=`hostname`\n+fi\n \n # Changeset URL prefix for your repo: when the commit ID is appended\n # to this, it should point at a CGI that will display the commit\n@@ -73,13 +97,14 @@ urlprefix=\"http://${host}/cgi-bin/cgit.cgi/${repo}/commit/?id=\"\n # You probably will not need to change the following:\n #\n \n-# Identify the script. Should change only when the script itself\n-# gets a new home and maintainer.\n+# Identify the script. The 'generator' variable should change only\n+# when the script itself gets a new home and maintainer.\n generator=\"http://www.catb.org/~esr/ciabot/ciabot.sh\"\n+version=3.4\n \n # Addresses for the e-mail\n-from=\"CIABOT-NOREPLY@${host}\"\n-to=\"cia@cia.navi.cx\"\n+from=\"CIABOT-NOREPLY@${hostname}\"\n+to=\"cia@cia.vc\"\n \n # SMTP client to use - may need to edit the absolute pathname for your system\n sendmail=\"sendmail -t -f ${from}\"\n@@ -95,9 +120,9 @@ mode=mailit\n while getopts pnV opt\n do\n case $opt in\n-\tp) project=$2; shift ; shift ;;\n+ p) project=$2; shift ; shift ;;\n \tn) mode=dumpit; shift ;;\n-\tV) echo \"ciabot.sh: version 3.2\"; exit 0; shift ;;\n+\tV) echo \"ciabot.sh: version $version\"; exit 0; shift ;;\n esac\n done\n \n@@ -128,33 +153,29 @@ fi\n \n refname=${refname##refs/heads/}\n \n-gitver=$(git --version)\n-gitver=${gitver##* }\n+case $revformat in\n+raw) rev=$merged ;;\n+short) rev='' ;;\n+*) rev=$(git describe ${merged} 2>/dev/null) ;;\n+esac\n+[ -z ${rev} ] && rev=$(echo \"$merged\" | cut -c 1-12)\n \n-rev=$(git describe ${merged} 2>/dev/null)\n-# ${merged:0:12} was the only bashism left in the 2008 version of this\n-# script, according to checkbashisms. Replace it with ${merged} here\n-# because it was just a fallback anyway, and it's worth accepting a\n-# longer fallback for faster execution and removing the bash\n-# dependency.\n-[ -z ${rev} ] && rev=${merged}\n-\n-# This discards the part of the author's address after @.\n+# We discard the part of the author's address after @.\n # Might be nice to ship the full email address, if not\n # for spammers' address harvesters - getting this wrong\n # would make the freenode #commits channel into harvester heaven.\n-rawcommit=$(git cat-file commit ${merged})\n-author=$(echo \"$rawcommit\" | sed -n -e '/^author .*<\\([^@]*\\).*$/s--\\1-p')\n-logmessage=$(echo \"$rawcommit\" | sed -e '1,/^$/d' | head -n 1)\n-logmessage=$(echo \"$logmessage\" | sed 's/\\&/&\\;/g; s//>\\;/g')\n-ts=$(echo \"$rawcommit\" | sed -n -e '/^author .*> \\([0-9]\\+\\).*$/s--\\1-p')\n+author=$(git log -1 '--pretty=format:%an <%ae>' $merged)\n+author=$(echo \"$author\" | sed -n -e '/^.*<\\([^@]*\\).*$/s--\\1-p')\n+\n+logmessage=$(git log -1 '--pretty=format:%s' $merged)\n+ts=$(git log -1 '--pretty=format:%at' $merged)\n files=$(git diff-tree -r --name-only ${merged} | sed -e '1d' -e 's-.*-&-')\n \n out=\"\n \n \n CIA Shell client for Git\n- ${gitver}\n+ ${version}\n ${generator}\n \n \n@@ -167,7 +188,7 @@ out=\"\n ${author}\n ${rev}\n \n-\t${files}\n+ ${files}\n \n ${logmessage} ${url}\n ${url}\n-- \n1.7.9.5\n\n\n-- \n\t\tEric S. Raymond\n"},{"id":"197594","messageId":"7vpq6ian5x.fsf@alter.siamese.dyndns.org","threadId":"31312","inReplyTo":"20120822105230.GA30382@thyrsus.com","subject":"Re: [PATCH] Get ciabot configuration from git variables.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-08-22T16:47:54Z","receivedAt":"2012-08-22T16:47:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Thanks, will update 'maint' with it.\n"}]}