threads / patch / 7365

patchConvert emailing part of hooks--update to hooks--post-receive

Subject: [PATCH] Convert emailing part of hooks--update to hooks--post-receive

## tl;dr

11 messages between Mar 23, 2007 and Mar 31, 2007. Diffs are folded; open one to read it.

replies: 10people: 4as markdown or json

Andy Parkins· Mar 23, 2007, 10:23 UTC · lore

The update hook is no longer the correct place to generate emails; there is now the hooks/post-receive script which is run automatically after a ref has been updated.

This patch is to make use of that new location, and to address some faults in the old update hook.

The primary problem in the conversion was that in the update hook, the ref has not actually been changed, but is about to be. In the post-receive hook the ref has already been updated. That meant that where we previously had lines like:

 git rev-list --not --all

would now fail because "--all" in the post-receive hook includes the ref that we are making the email for. This made it more difficult to show only the new revisions added by this update.

The solution is not pretty; however it does work and doesn't need any changes to git-rev-list itself. It also fixes (more accurately: reduces the likelihood of) a nasty race when another update occurs while this script is running. The solution, in short, looks like this (see the source code for a longer explanation)

 git rev-parse --not --all | grep -v $(<$refname) |
 git rev-list --pretty --stdin $newrev --not $baserev

This uses git-rev-parse followed by grep to filter out the revision of the ref in question before it gets to rev-list and inhibits the output of itself. By using $(<$refname) rather than $newrev as the filter, it also takes care of the situation where another update to the same ref has been made since $refname was $newrev.

The second problem that is addressed is that of tags inhibiting the correct output of an update email. Consider this, with somebranch and sometag pointing at the same revision:

 git push origin somebranch
 git push origin sometag

That would work fine; the push of the branch would generate an email containing all the new commits introduced by the update, then the push of the tag would generate the shortlog formatted tag email. Now consider:

 git push origin sometag
 git push origin somebranch

When some branch comes to run its "--not --all" line, it will find sometag, and filter those commits from the email - leaving nothing. That meant that those commits would not show (in full) on any email. The fix is to not use "--all", and instead use "--branches" in the git-rev-parse command.

Other changes
 * Lose the monstrous one-giant-script layout and put things in easy to
   digest functions.  This makes it much easier to find the place you
   need to change if you wanted to customise the output.  I've also
   tried to write more verbose comments for the same reason.  The hook
   script is big, mainly because of all the different cases that it has
   to handle, so being easy to navigate is important.
 * All uses of "git-command" changed to "git command", to cope better
   if a user decided not to install all the hard links to git;
 * Cleaned up some of the English in the email
 * The fact that the receive hook makes the ref available also allows me
   to use Shawn Pearce's fantastic suggestion that an annotated tag can
   be parsed with git-for-each-ref.  This removes the potentially
   non-portable use of "<<<" heredocs and the nasty messing around with
   "date" to convert numbers of seconds UTC to a real date
 * Deletions are now caught and notified (briefly)
 * To help with debugging, I've retained the command line mode from the
   update hook; but made it so that the output is not emailed, it's just
   printed to the screen.  This could then be redirected if the user
   wanted
 * Removed the "Hello" from the beginning of the email - it's just
   noise, and no one seriously has their day made happier by "friendly"
   programs
 * The fact that it doesn't rely on repository state as an indicator any
   more means that it's far more stable in its output; hopefully the
   same arguments will always generate the same email - even if the
   repository changes in the future.  This means you can easily recreate
   an email should you want to.
Signed-off-by: Andy Parkins <andyparkins@gmail.com>
---

I'm sorry this is so big and nasty Junio; but that's mostly because the main body of it is a cut and paste from the old update hook.

It includes a whole set of little fixes as well as a reorganisation though, so I don't think the content tracking would have detected it as a rename.

 templates/hooks--post-receive |  554 +++++++++++++++++++++++++++++++++++++++++
 1 files changed, 554 insertions(+), 0 deletions(-)
 create mode 100644 templates/hooks--post-receive
Show changes to templates/hooks--post-receive +554 −0
diff --git a/templates/hooks--post-receive b/templates/hooks--post-receive
new file mode 100644
index 0000000..1e988ee
--- /dev/null
+++ b/templates/hooks--post-receive
@@ -0,0 +1,554 @@
+#!/bin/sh
+#
+# Copyright (c) 2007 Andy Parkins
+#
+# An example hook script to mail out commit update information.
+#
+# To enable this hook, make this file executable by "chmod +x update".
+#
+# Config
+# ------
+# hooks.mailinglist
+#   This is the list that all pushes will go to; leave it blank to not send
+#   emails for every ref update.
+# hooks.announcelist
+#   This is the list that all pushes of annotated tags will go to.  Leave it
+#   blank to default to the mailinglist field.  The announce emails lists the
+#   short log summary of the changes since the last annotated tag.
+#
+# Notes
+# -----
+# All emails have their subjects prefixed with "[SCM]" to aid filtering.
+# All emails include the headers "X-Git-Refname", "X-Git-Oldrev",
+# "X-Git-Newrev", and "X-Git-Reftype" to enable fine tuned filtering and
+# give information for debugging.
+#
+
+# ---------------------------- Functions
+
+#
+# Top level email generation function.  This decides what type of update
+# this is and calls the appropriate body-generation routine after outputting
+# the common header
+#
+# Note this function doesn't actually generate any email output, that is taken
+# care of by the functions it calls:
+#  - generate_email_header
+#  - generate_create_XXXX_email
+#  - generate_update_XXXX_email
+#  - generate_delete_XXXX_email
+#  - generate_email_footer
+#
+generate_email()
+{
+	# --- Arguments
+	oldrev=$(git rev-parse $1)
+	newrev=$(git rev-parse $2)
+	refname="$3"
+
+	# --- Interpret
+	# 0000->1234 (create)
+	# 1234->2345 (update)
+	# 2345->0000 (delete)
+	if expr "$oldrev" : '0*$' >/dev/null
+	then
+		change_type="create"
+	else
+		if expr "$newrev" : '0*$' >/dev/null
+		then
+			change_type="delete"
+		else
+			change_type="update"
+		fi
+	fi
+
+	# --- Get the revision types
+	newrev_type=$(git cat-file -t $newrev 2> /dev/null)
+	oldrev_type=$(git cat-file -t "$oldrev" 2> /dev/null)
+	case "$change_type" in
+	create|update)
+		rev="$newrev"
+		rev_type="$newrev_type"
+		;;
+	delete)
+		rev="$oldrev"
+		rev_type="$oldrev_type"
+		;;
+	esac
+
+	# The revision type tells us what type the commit is, combined with
+	# the location of the ref we can decide between
+	#  - working branch
+	#  - tracking branch
+	#  - unannoted tag
+	#  - annotated tag
+	case "$refname","$rev_type" in
+		refs/tags/*,commit)
+			# un-annotated tag
+			refname_type="tag"
+			short_refname=${refname##refs/tags/}
+			;;
+		refs/tags/*,tag)
+			# annotated tag
+			refname_type="annotated tag"
+			short_refname=${refname##refs/tags/}
+			# change recipients
+			if [ -n "$announcerecipients" ]; then
+				recipients="$announcerecipients"
+			fi
+			;;
+		refs/heads/*,commit)
+			# branch
+			refname_type="branch"
+			short_refname=${refname##refs/heads/}
+			;;
+		refs/remotes/*,commit)
+			# tracking branch
+			refname_type="tracking branch"
+			short_refname=${refname##refs/remotes/}
+			echo "*** Push-update of tracking branch, $refname"
+			echo "***  - no email generated." >&2
+			exit 0
+			;;
+		*)
+			# Anything else (is there anything else?)
+			echo "*** Unknown type of update to $refname ($rev_type)" >&2
+			echo "***  - no email generated"
+			exit 1
+			;;
+	esac
+
+	# Check if we've got anyone to send to
+	if [ -z "$recipients" ]; then
+		echo "*** hooks.recipients is not set so no email will be sent" >&2
+		echo "*** for $refname update $oldrev->$newrev" >&2
+		exit 0
+	fi
+
+	# Email parameters
+	# The committer will be obtained from the latest existing rev; so
+	# for a deletion it will be the oldrev, for the others, then newrev
+	committer=$(git show --pretty=full -s $rev | grep "^Commit: " | sed -e "s/^Commit: //")
+	# The email subject will contain the best description of the ref
+	# that we can build from the parameters
+	describe=$(git describe $rev 2>/dev/null)
+	if [ -z "$describe" ]; then
+		describe=$rev
+	fi
+
+	generate_email_header
+
+	# Call the correct body generation function - object oriented programming
+	# in a script - overkill perhaps?
+	fn_name=general
+	case "$refname_type" in
+		# --- Branches
+		"tracking branch"|branch)
+			fn_name=branch
+			;;
+		# --- Annotated tags
+		"annotated tag")
+			fn_name=atag
+			;;
+	esac
+	generate_${change_type}_${fn_name}_email
+
+	generate_email_footer
+}
+
+generate_email_header()
+{
+	# --- Email (all stdout will be the email)
+	# Generate header
+	cat <<-EOF
+	From: $committer
+	To: $recipients
+	Subject: ${EMAILPREFIX}$projectdesc $refname_type, $short_refname, ${change_type}d. $describe
+	X-Git-Refname: $refname
+	X-Git-Reftype: $refname_type
+	X-Git-Oldrev: $oldrev
+	X-Git-Newrev: $newrev
+
+	This is an automated email from the git hooks/post-receive script. It was
+	generated because a ref change was pushed to the repository containing
+	the project "$projectdesc".
+
+	The $refname_type, $short_refname has been ${change_type}d
+	EOF
+}
+
+generate_email_footer()
+{
+	cat <<-EOF
+
+	hooks/post-receive
+	--
+	Git Source Code Management System
+	EOF
+}
+
+# --------------- Branches
+
+#
+# Called for the creation of a branch
+#
+generate_create_branch_email()
+{
+	# This is a new branch and so oldrev is not valid
+	echo "        at  $newrev ($newrev_type)"
+	echo ""
+
+	echo $LOGBEGIN
+	# This shows all log entries that are not already covered by
+	# another ref - i.e. commits that are now accessible from this
+	# ref that were previously not accessible (see generate_update_branch_email
+	# for the explanation of this command)
+	git rev-parse --not --branches | grep -v $(<${GIT_DIR}/$refname) |
+	git rev-list --pretty --stdin $newrev --not $baserev
+	echo $LOGEND
+}
+
+#
+# Called for the change of a pre-existing branch
+#
+generate_update_branch_email()
+{
+	# Consider this:
+	#   1 --- 2 --- O --- X --- 3 --- 4 --- N
+	#
+	# O is $oldrev for $refname
+	# N is $newrev for $refname
+	# X is a revision pointed to by some other ref, for which we may
+	#   assume that an email has already been generated.
+	# In this case we want to issue an email containing only revisions
+	# 3, 4, and N.  Given (almost) by
+	#
+	#  git-rev-list N ^O --not --all
+	#
+	# The reason for the "almost", is that the "--not --all" will take
+	# precedence over the "N", and effectively will translate to
+	#
+	#  git-rev-list N ^O ^X ^N
+	#
+	# So, we need to build up the list more carefully.  git-rev-parse will
+	# generate a list of revs that may be fed into git-rev-list.  We can get
+	# it to make the "--not --all" part and then filter out the "^N" with:
+	#
+	#  git-rev-parse --not --all | grep -v N
+	#
+	# Then, using the --stdin switch to git-rev-list we have effectively
+	# manufactured
+	#
+	#  git-rev-list N ^O ^X
+	#
+	# This leaves a problem when someone else updates the repository
+	# while this script is running.  Their new value of the ref we're working
+	# on would be included in the "--not --all" output; and as our $newrev
+	# would be an ancestor of that commit, it would exclude all of our
+	# commits.  What we really want is to exclude the current value of
+	# $refname from the --not list, rather than N itself.  So:
+	#
+	#  git-rev-parse --not --all | grep -v $(< $refname)
+	#
+	# Get's us to something pretty safe (apart from the small time between
+	# refname being read, and git-rev-parse running - for that, I give up)
+	#
+	#
+	# Next problem, consider this:
+	#   * --- B --- * --- O ($oldrev)
+	#          \
+	#           * --- X --- * --- N ($newrev)
+	#
+	# That is to say, there is no guarantee that newrev is a strict subset of
+	# oldrev (it would have required a --force, but that's allowed).  So, we
+	# can't simply say rev-list $oldrev..$newrev.  Instead we find the common
+	# base of the two revs and list from there.
+	#
+	# As above, we need to take into account the presence of X; if another
+	# branch is already in the repository and points at some of the revisions
+	# that we are about to output - we don't want them.  The solution is as
+	# before: git-rev-parse output filtered.
+	#
+	# Finally, tags:
+	#   1 --- 2 --- O --- T --- 3 --- 4 --- N
+	#
+	# Tags pushed into the repository generate nice shortlog emails that
+	# summarise the commits between them and the previous tag.  However,
+	# those emails don't include the full commit messages that we output
+	# for a branch update.  Therefore we still want to output revisions
+	# that have been output on a tag email.
+	#
+	# Luckily, git-rev-parse includes just the tool.  Instead of using "--all"
+	# we use "--branches"; this has the added benefit that "remotes/" will
+	# be ignored as well.
+
+	baserev=$(git merge-base $oldrev $newrev)
+
+	# List all the revisions from baserev to newrev in a kind of
+	# "table-of-contents"; note this list can include revisions that have
+	# already had notification emails and is present to show the full detail
+	# of the change from rolling back the old revision to the base revision and
+	# then forward to the new revision
+	for rev in $(git rev-list $newrev --not $baserev)
+	do
+		revtype=$(git cat-file -t "$rev")
+		echo "       via  $rev ($revtype)"
+	done
+
+	# If the baserev is equal to the oldrev, then we know that this was a
+	# fast forward update; otherwise "--force" was used, and we give a more
+	# detailed explanation of what happened.
+	if [ "$baserev" = "$oldrev" ]; then
+		echo "      from  $oldrev ($oldrev_type)"
+	else
+		echo "      from  $baserev"
+		echo "  discards  $oldrev ($oldrev_type) and its branch"
+		echo ""
+		echo "This update added new revisions after undoing old revisions.  That is to"
+		echo "say, the old revision is not a strict subset of the new revision.  This"
+		echo "situation occurs when you --force push a change and generate a"
+		echo "repository containing something like this:"
+		echo ""
+		echo " * -- * -- B -- O -- O -- O ($oldrev)"
+		echo "            \\"
+		echo "             N -- N -- N ($newrev)"
+		echo ""
+		echo "When this happens we assume that you've already had alert emails for all"
+		echo "of the O revisions, and so we here report only the revisions in the N"
+		echo "branch from the common base, B ($baserev),"
+		echo "up to the new branch endpoint."
+	fi
+
+	echo ""
+	echo "Those revisions listed above that are new to this repository have"
+	echo "not appeared on any other notification email; so we list those"
+	echo "revisions in full, below."
+
+	echo ""
+	echo $LOGBEGIN
+	git rev-parse --not --branches | grep -v $(<${GIT_DIR}/$refname) |
+	git rev-list --pretty --stdin $newrev --not $baserev
+
+	# XXX: Need a way of detecting whether git rev-list actually outputted
+	# anything, so that we can issue a "no new revisions added by this
+	# update" message
+
+	echo $LOGEND
+
+	# The diffstat is shown from the old revision to the new revision.  This
+	# is to show the truth of what happened in this change.  There's no point
+	# showing the stat from the base to the new revision because the base
+	# is effectively a random revision at this point - the user will be
+	# interested in what this revision changed - including the undoing of
+	# previous revisions in the case of non-fast forward updates.
+	echo ""
+	echo "Summary of changes:"
+	git diff-tree --no-color --stat -M -C --find-copies-harder $baserev..$newrev
+}
+
+#
+# Called for the deletion of a branch
+#
+generate_delete_branch_email()
+{
+	echo "       was  $oldrev"
+	echo ""
+}
+
+# --------------- Annotated tags
+
+#
+# Called for the creation of an annotated tag
+#
+generate_create_atag_email()
+{
+	echo "        at  $newrev ($newrev_type)"
+
+	generate_atag_email
+}
+
+#
+# Called for the update of an annotated tag (this is probably a rare event
+# and may not even be allowed)
+#
+generate_update_atag_email()
+{
+	echo "        to  $newrev ($newrev_type)"
+	echo "      from  $oldrev (which is now obsolete)"
+
+	generate_atag_email
+}
+
+#
+# Called when an annotated tag is created or changed
+#
+generate_atag_email()
+{
+	# Use git-for-each-ref to pull out the individual fields from the tag
+	eval $(git-for-each-ref --format='
+	tagobject="%(*objectname)"
+	tagtype="%(*objecttype)"
+	tagger="%(taggername)"
+	tagged="%(taggerdate)"' $refname
+	)
+
+	echo "   tagging  $tagobject ($tagtype)"
+	case "$tagtype" in
+	commit)
+		# If the tagged object is a commit, then we assume this is a
+		# release, and so we calculate which tag this tag is replacing
+		prevtag=$(git describe --abbrev=0 $newrev^ 2>/dev/null)
+
+		if [ -n "$prevtag" ]; then
+			echo "  replaces  $prevtag"
+		fi
+		;;
+	*)
+		echo "    length  $(git cat-file -s $tagobject) bytes"
+		;;
+	esac
+	echo " tagged by  $tagger"
+	echo "        on  $tagged"
+
+	echo ""
+	echo $LOGBEGIN
+
+	# Show the content of the tag message; this might contain a change log
+	# or release notes so is worth displaying.  "tail -n +5" is there to
+	# remove the first 4 lines of the tag object, those details have
+	# already been summarised above
+	git cat-file tag $newrev | tail -q -n +5
+
+	echo ""
+	case "$tagtype" in
+	commit)
+		# Only commit tags make sense to have rev-list operations performed
+		# on them
+		if [ -n "$prevtag" ]; then
+			# Show changes since the previous release
+			git rev-list --pretty=short "$prevtag..$newrev" | git shortlog
+		else
+			# No previous tag, show all the changes since time began
+			git rev-list --pretty=short $newrev | git shortlog
+		fi
+		;;
+	*)
+		# XXX: Is there anything useful we can do for non-commit objects?
+		;;
+	esac
+
+	echo $LOGEND
+	echo ""
+}
+
+#
+# Called for the deletion of an annotated tag
+#
+generate_delete_atag_email()
+{
+	echo "       was  $oldrev"
+	echo ""
+}
+
+# --------------- General references
+
+#
+# Called when any other type of reference is created (most likely a
+# non-annotated tag)
+#
+generate_create_general_email()
+{
+	echo "        at  $newrev ($newrev_type)"
+
+	generate_general_email
+}
+
+#
+# Called when any other type of reference is updated (most likely a
+# non-annotated tag)
+#
+generate_update_general_email()
+{
+	echo "        to  $newrev ($newrev_type)"
+	echo "      from  $oldrev"
+
+	generate_general_email
+}
+
+#
+# Called for creation or update of any other type of reference
+#
+generate_general_email()
+{
+	# Unannotated tags are more about marking a point than releasing a version;
+	# therefore we don't do the shortlog summary that we do for annotated tags
+	# above - we simply show that the point has been marked, and print the log
+	# message for the marked point for reference purposes
+	#
+	# Note this section also catches any other reference type (although there
+	# aren't any) and deals with them in the same way.
+
+	echo ""
+	if [ "$newrev_type" = "commit" ]; then
+		echo $LOGBEGIN
+		git show --no-color --root -s $newrev
+		echo $LOGEND
+	else
+		# What can we do here?  The tag marks an object that is not a commit,
+		# so there is no log for us to display.  It's probably not wise to
+		# output git-cat-file as it could be a binary blob.  We'll just say how
+		# big it is
+		echo "$newrev is a $newrev_type, and is $(git cat-file -s $newrev) bytes long."
+	fi
+	echo ""
+}
+
+#
+# Called for the deletion of any other type of reference
+#
+generate_delete_general_email()
+{
+	echo "       was  $oldrev"
+	echo ""
+}
+
+# ---------------------------- main()
+
+# --- Constants
+EMAILPREFIX="[SCM] "
+LOGBEGIN="- Log -----------------------------------------------------------------"
+LOGEND="-----------------------------------------------------------------------"
+
+# --- Config
+# Set GIT_DIR either from the working directory, or from the environment
+# variable.
+GIT_DIR=$(git-rev-parse --git-dir 2>/dev/null)
+if [ -z "$GIT_DIR" ]; then
+	echo "fatal: post-receive: GIT_DIR not set"
+	exit 1
+fi
+
+projectdesc=$(cat $GIT_DIR/description)
+# Check if the description is unchanged from it's default, and shorten it to a
+# more manageable length if it is
+if expr "$projectdesc" : "Unnamed repository.*$" >/dev/null
+then
+	projectdesc="UNNAMED PROJECT"
+fi
+
+recipients=$(git repo-config hooks.mailinglist)
+announcerecipients=$(git repo-config hooks.announcelist)
+
+# --- Main loop
+# Allow dual mode: run from the command line just like the update hook, or if
+# no arguments are given then run as a hook script
+if [ -n "$1" -a -n "$2" -a -n "$3" ]; then
+	# Output to the terminal in command line mode - if someone wanted to
+	# resend an email; they could redirect the output to sendmail themselves
+	generate_email $2 $3 $1
+else
+	while read oldrev newrev refname
+	do
+		generate_email $oldrev $newrev $refname | /usr/sbin/sendmail -t
+	done
+fi
-- 
1.5.1.rc1.28.gedbf0
Junio C Hamano· Mar 24, 2007, 12:43 UTC · re: Andy Parkins · lore

Re: [PATCH] Convert emailing part of hooks--update to hooks--post-receive

Andy Parkins <andyparkins@gmail.com> writes:
Show 8 quoted lines
>  git rev-parse --not --all | grep -v $(<$refname) |
>  git rev-list --pretty --stdin $newrev --not $baserev
>
> This uses git-rev-parse followed by grep to filter out the revision of
> the ref in question before it gets to rev-list and inhibits the output
> of itself.  By using $(<$refname) rather than $newrev as the filter, it
> also takes care of the situation where another update to the same ref
> has been made since $refname was $newrev.

Is $(<$refname) even POSIX? Are we sure $refname is always a file (not a packed ref)?

If the other push updates the same ref you are filtering out, that would make "grep -v" to remove it from the exclusion set, which is a neat trick. However, wouldn't that cause the rev-list to report the effect of the other push, and not your push? I am wondering if the mail would say "Alice pushed and this branch has these commits as the result" and ends up including Bob's commits in it when that happens.

>  * All uses of "git-command" changed to "git command", to cope better
>    if a user decided not to install all the hard links to git;

I think that is a good discipline in general, but I wonder if it is strictly necessary. Doesn't hook already get invoked with environment that has GIT_EXEC_PATH at the beginning of the PATH?

Show 9 quoted lines
> +generate_email()
> +{
> ...
> +		refs/remotes/*,commit)
> +			# tracking branch
> +			refname_type="tracking branch"
> +			short_refname=${refname##refs/remotes/}
> +			echo "*** Push-update of tracking branch, $refname"
> +			echo "***  - no email generated." >&2
Huh?  Did you mean to send the first one also to stderr?
Show 6 quoted lines
> +			exit 0
> +			;;
> +		*)
> +			# Anything else (is there anything else?)
> +			echo "*** Unknown type of update to $refname ($rev_type)" >&2
> +			echo "***  - no email generated"
Huh?  Did you mean to send the second one also to stderr?

It usually is easier to spot this kind of gotchas if you consistently write like this instead:

	echo >&2 "your message comes here..."
	echo >&2 "... and the second line is here"
Show 15 quoted lines
> +			exit 1
> +			;;
> +	esac
> +
> +	# Check if we've got anyone to send to
> +	if [ -z "$recipients" ]; then
> +		echo "*** hooks.recipients is not set so no email will be sent" >&2
> +		echo "*** for $refname update $oldrev->$newrev" >&2
> +		exit 0
> +	fi
> +
> +	# Email parameters
> +	# The committer will be obtained from the latest existing rev; so
> +	# for a deletion it will be the oldrev, for the others, then newrev
> +	committer=$(git show --pretty=full -s $rev | grep "^Commit: " | sed -e "s/^Commit: //")

Please never pipe grep into sed, or the other way around. It makes the script look amateurish.

I think you meant:
	.... | sed -ne 's/^Commit: //p'

Is it assumed that pushing developers do not cross-pull, bypassing the repository this hook is in? Otherwise, if Alice does a lot of work, Bob pulls from her (bypassing this repository), makes a tiny fix, and Alice pulls (bypassing this repository) back from Bob, the last pull by Alice would result in a fast-forward, and when Alice pushes the results back, this repository will see the Bob's tiny fix at the tip, the script takes that commit as the representative, and From: line would have Bob's name on it.

It would not be a problem as long as you use this repository as the _ONLY_ place developers meet, but that's a big assumption: centralized repo with CVS-mentality.

The assumption is not wrong per-se, but I think that needs to be documented at one place, near the top: "if you subscribe to this kind of workflow, then this e-mail script is for you, otherwise, it is not".

Similarly the other assumption your script makes should be documented: the "list only new commits to the entire repository" design. I do not think that behaviour is for everybody, but you already know that, and I would not make further comments on this point, other than saying that I do agree it is good for some people. It all depends on the workflow and recipients' interests.

Show 11 quoted lines
> +	# The email subject will contain the best description of the ref
> +	# that we can build from the parameters
> +	describe=$(git describe $rev 2>/dev/null)
> +	if [ -z "$describe" ]; then
> +		describe=$rev
> +	fi
> +
> +	generate_email_header
> +
> +	# Call the correct body generation function - object oriented programming
> +	# in a script - overkill perhaps?

I do not think so. It is a good style as long as you do not botch it and leave the variables empty -- even in that case the runtime would notice and say "generate___email: no such command" ;-).

Show 11 quoted lines
> +	fn_name=general
> +	case "$refname_type" in
> +		# --- Branches
> +		"tracking branch"|branch)
> +			fn_name=branch
> +			;;
> +		# --- Annotated tags
> +		"annotated tag")
> +			fn_name=atag
> +			;;
> +	esac

Please indent the case arms at the same level as "case/esac", like this:

	case $var in
        foo | bar)
        	do this
                ;;
	esac
which is in line with:
	switch (x) {
        case 1:
        	foo;
                break;
	}
Show 25 quoted lines
> +	generate_${change_type}_${fn_name}_email
> +
> +	generate_email_footer
> +}
> +
> +generate_email_header()
> +{
> +	# --- Email (all stdout will be the email)
> +	# Generate header
> +	cat <<-EOF
> +	From: $committer
> +	To: $recipients
> +	Subject: ${EMAILPREFIX}$projectdesc $refname_type, $short_refname, ${change_type}d. $describe
> +	X-Git-Refname: $refname
> +	X-Git-Reftype: $refname_type
> +	X-Git-Oldrev: $oldrev
> +	X-Git-Newrev: $newrev
> +
> +	This is an automated email from the git hooks/post-receive script. It was
> +	generated because a ref change was pushed to the repository containing
> +	the project "$projectdesc".
> +
> +	The $refname_type, $short_refname has been ${change_type}d
> +	EOF
> +}

Micronit. Some committer names (GIT_COMMITTER_NAME) that are not quoted may need to be quoted to appear on the From: header (most notable is a name with dot after middle initial).

Show 10 quoted lines
> +
> +generate_email_footer()
> +{
> +	cat <<-EOF
> +
> +	hooks/post-receive
> +	--
> +	Git Source Code Management System
> +	EOF
> +}
A signature separator line is usually "^-- $", not "^--$".

I'd rather not to have name Git there, for two reasons. If you are the lead of the development group who is also contact to the QA group, you would probably get e-mail notifications from your development repo and QA group repo. Instead of the generic 'Git' name, which does not add any information of real value, it would be better to have an identifier of the repository (you would configure it in description perhaps). Granted, the same information is on Subject: line, but the body of the message may be saved in a separate file for later perusal.

Also, this is (as described in the first part of the script) only an example script, and although I am willing to help making sure this is bug-free (like spending time writing this message right now -- wow, I've been sitting here typing this e-mail for almost two hours), I do not want to be held responsible for any bugs in it, nor I do not want anybody to mistakenly think that what this script does is the way to run things recommended by the Git community. It really is just "a useful script to use WITH git Andy wrote", not "THE hook Git gives us".

Show 10 quoted lines
> +# --------------- Branches
> +
> +#
> +# Called for the creation of a branch
> +#
> +generate_create_branch_email()
> +{
> +	# This is a new branch and so oldrev is not valid
> +	echo "        at  $newrev ($newrev_type)"
> +	echo ""

I kind of like the attention to the detail of giving an empty argv[1] to echo (it used to be customary to do this for portability), but I wonder if it is still needed...

Show 10 quoted lines
> +
> +	echo $LOGBEGIN
> +	# This shows all log entries that are not already covered by
> +	# another ref - i.e. commits that are now accessible from this
> +	# ref that were previously not accessible (see generate_update_branch_email
> +	# for the explanation of this command)
> +	git rev-parse --not --branches | grep -v $(<${GIT_DIR}/$refname) |
> +	git rev-list --pretty --stdin $newrev --not $baserev
> +	echo $LOGEND
> +}

Where does $baserev come from? Random leftover garbage from invocation for the branch the main while loop handled before this branch?

I think "--not $baserev" is useless for a new branch ;-).
Show 13 quoted lines
> +
> +#
> +# Called for the change of a pre-existing branch
> +#
> +generate_update_branch_email()
> +{
> +	# Consider this:
> +	#   1 --- 2 --- O --- X --- 3 --- 4 --- N
> +	#
> +	# O is $oldrev for $refname
> +	# N is $newrev for $refname
> +	# X is a revision pointed to by some other ref, for which we may
> +	#   assume that an email has already been generated.

You are doing this one branch at a time even though a single push may update more than one branch. I wonder what happens if branch A and B are pushed, and they both contain shared new material? When this is run for A, the entire contribution of B (after push happened) is excluded. Then when this is run for B, the entire contribution of A is excluded. Doesn't that mean the commits new to this repository that are shared between A and B appear for neither branches? There must be something obvious I am missing...

Show 9 quoted lines
> +	# Next problem, consider this:
> +	#   * --- B --- * --- O ($oldrev)
> +	#          \
> +	#           * --- X --- * --- N ($newrev)
> +	#
> +	# That is to say, there is no guarantee that newrev is a strict subset of
> +	# oldrev (it would have required a --force, but that's allowed).  So, we
> +	# can't simply say rev-list $oldrev..$newrev.  Instead we find the common
> +	# base of the two revs and list from there.

This comment and logic is wrong. In the above graph, if you say "rev-list O..N" (which is "rev-list N ^O"), you would get commits that are reachable from N, excluding the ones that are also reachable from O, so you would get N, * before it, X and * before it.

Show 5 quoted lines
> +	#
> +	# As above, we need to take into account the presence of X; if another
> +	# branch is already in the repository and points at some of the revisions
> +	# that we are about to output - we don't want them.  The solution is as
> +	# before: git-rev-parse output filtered.

So, I think "rev-list N ^O ^X" is what you would want here, which is the same as the first "single strand of pearls" case. And notice that I did not have to say anything about base? I do not think it is necessary to compute base if you are only interested in showing which commits are new.

It is however useful to compute it for another reason, namely that you would want to give different messages depending on fast-forwardness of O->N transition. However, even for that purpose, merge-base may not be necessarily the best way to see if it is a fast forward for this application.

Show 13 quoted lines
> +	#
> +	# Finally, tags:
> +	#   1 --- 2 --- O --- T --- 3 --- 4 --- N
> +	#
> +	# Tags pushed into the repository generate nice shortlog emails that
> +	# summarise the commits between them and the previous tag.  However,
> +	# those emails don't include the full commit messages that we output
> +	# for a branch update.  Therefore we still want to output revisions
> +	# that have been output on a tag email.
> +	#
> +	# Luckily, git-rev-parse includes just the tool.  Instead of using "--all"
> +	# we use "--branches"; this has the added benefit that "remotes/" will
> +	# be ignored as well.
Ok.
Show 17 quoted lines
> +
> +	baserev=$(git merge-base $oldrev $newrev)
> +
> +	# List all the revisions from baserev to newrev in a kind of
> +	# "table-of-contents"; note this list can include revisions that have
> +	# already had notification emails and is present to show the full detail
> +	# of the change from rolling back the old revision to the base revision and
> +	# then forward to the new revision
> +	for rev in $(git rev-list $newrev --not $baserev)
> +	do
> +		revtype=$(git cat-file -t "$rev")
> +		echo "       via  $rev ($revtype)"
> +	done
> +
> +	# If the baserev is equal to the oldrev, then we know that this was a
> +	# fast forward update; otherwise "--force" was used, and we give a more
> +	# detailed explanation of what happened.

That's bog-standard way we use to see if O->N transition is a fast forward, but for this application, since it is far more likely to be than it isn't, I think computing "rev-list N..O" and checking if it reports anything is a better check (especially because its results can be used in a way I'll describe later).

N..O lists commits that are reachable from O but not from N, in other words, the commits lost by moving head from O to N. If that is not an empty set, it is not a fast-forward.

Show 20 quoted lines
> +	if [ "$baserev" = "$oldrev" ]; then
> +		echo "      from  $oldrev ($oldrev_type)"
> +	else
> +		echo "      from  $baserev"
> +		echo "  discards  $oldrev ($oldrev_type) and its branch"
> +		echo ""
> +		echo "This update added new revisions after undoing old revisions.  That is to"
> +		echo "say, the old revision is not a strict subset of the new revision.  This"
> +		echo "situation occurs when you --force push a change and generate a"
> +		echo "repository containing something like this:"
> +		echo ""
> +		echo " * -- * -- B -- O -- O -- O ($oldrev)"
> +		echo "            \\"
> +		echo "             N -- N -- N ($newrev)"
> +		echo ""
> +		echo "When this happens we assume that you've already had alert emails for all"
> +		echo "of the O revisions, and so we here report only the revisions in the N"
> +		echo "branch from the common base, B ($baserev),"
> +		echo "up to the new branch endpoint."
> +	fi

I suspect the recipients would want to know which ones are discarded, so perhaps list O with:

	"rev-list N..O"

Notice you do not need to use B to compute this set either. I think you can replace "--not $baserev" with "--not $oldrev" in all these "rev-list --pretty --stdin $newrev" invocations.

Show 10 quoted lines
> +
> +	echo ""
> +	echo "Those revisions listed above that are new to this repository have"
> +	echo "not appeared on any other notification email; so we list those"
> +	echo "revisions in full, below."
> +
> +	echo ""
> +	echo $LOGBEGIN
> +	git rev-parse --not --branches | grep -v $(<${GIT_DIR}/$refname) |
> +	git rev-list --pretty --stdin $newrev --not $baserev

I am recommending to get rid of merge-base, which means you won't have $baserev, but it can be replaced with $oldrev.

"O..N" looks more gittish than "$(merge-base O N)..N"
Show 13 quoted lines
> +
> +	# XXX: Need a way of detecting whether git rev-list actually outputted
> +	# anything, so that we can issue a "no new revisions added by this
> +	# update" message
> +
> +	echo $LOGEND
> +
> +	# The diffstat is shown from the old revision to the new revision.  This
> +	# is to show the truth of what happened in this change.  There's no point
> +	# showing the stat from the base to the new revision because the base
> +	# is effectively a random revision at this point - the user will be
> +	# interested in what this revision changed - including the undoing of
> +	# previous revisions in the case of non-fast forward updates.
> +	echo ""
> +	echo "Summary of changes:"
> +	git diff-tree --no-color --stat -M -C --find-copies-harder $baserev..$newrev
> +}
Comment and code mismatch between $baserev and $oldrev?

"-M -C" is redundant. Just "-C" should do. If you really want to spend extra cycles, then "--find-copies-harder" is fine, but I think that implies "-C".

Do you even need --no-color? I thought we specifically tried not to do the color or any UI stuff for the lower-level diff-{tree,files,index} brothers.

I would add --summary to the list, though.
Show 9 quoted lines
> +
> +#
> +# Called for the deletion of a branch
> +#
> +generate_delete_branch_email()
> +{
> +	echo "       was  $oldrev"
> +	echo ""
> +}
Perhaps with a one-line?
Show 12 quoted lines
> +
> +# --------------- Annotated tags
> +
> +#
> +# Called for the creation of an annotated tag
> +#
> +generate_create_atag_email()
> +{
> +	echo "        at  $newrev ($newrev_type)"
> +
> +	generate_atag_email
> +}
Perhaps with a one-line?
Show 25 quoted lines
> +
> +#
> +# Called for the update of an annotated tag (this is probably a rare event
> +# and may not even be allowed)
> +#
> +generate_update_atag_email()
> +{
> +	echo "        to  $newrev ($newrev_type)"
> +	echo "      from  $oldrev (which is now obsolete)"
> +
> +	generate_atag_email
> +}
> +
> +#
> +# Called when an annotated tag is created or changed
> +#
> +generate_atag_email()
> +{
> +	# Use git-for-each-ref to pull out the individual fields from the tag
> +	eval $(git-for-each-ref --format='
> +	tagobject="%(*objectname)"
> +	tagtype="%(*objecttype)"
> +	tagger="%(taggername)"
> +	tagged="%(taggerdate)"' $refname
> +	)

Perfect example of using for-each-ref and nicely done. Contrary to what you said earlier, I see a hyphen between git and for-each-ref, though.

Show 7 quoted lines
> +
> +	echo "   tagging  $tagobject ($tagtype)"
> +	case "$tagtype" in
> +	commit)
> +		# If the tagged object is a commit, then we assume this is a
> +		# release, and so we calculate which tag this tag is replacing
> +		prevtag=$(git describe --abbrev=0 $newrev^ 2>/dev/null)
What if $newrev is a merge, I wonder...
Show 20 quoted lines
> +
> +		if [ -n "$prevtag" ]; then
> +			echo "  replaces  $prevtag"
> +		fi
> +		;;
> +	*)
> +		echo "    length  $(git cat-file -s $tagobject) bytes"
> +		;;
> +	esac
> +	echo " tagged by  $tagger"
> +	echo "        on  $tagged"
> +
> +	echo ""
> +	echo $LOGBEGIN
> +
> +	# Show the content of the tag message; this might contain a change log
> +	# or release notes so is worth displaying.  "tail -n +5" is there to
> +	# remove the first 4 lines of the tag object, those details have
> +	# already been summarised above
> +	git cat-file tag $newrev | tail -q -n +5
Didn't you mean to update this with:
	... | sed -e '1,/^$/d'
?
Show 17 quoted lines
> +
> +	echo ""
> +	case "$tagtype" in
> +	commit)
> +		# Only commit tags make sense to have rev-list operations performed
> +		# on them
> +		if [ -n "$prevtag" ]; then
> +			# Show changes since the previous release
> +			git rev-list --pretty=short "$prevtag..$newrev" | git shortlog
> +		else
> +			# No previous tag, show all the changes since time began
> +			git rev-list --pretty=short $newrev | git shortlog
> +		fi
> +		;;
> +	*)
> +		# XXX: Is there anything useful we can do for non-commit objects?
> +		;;

You won't get a tag to another tag with %(*objecttype), so you only have to worry about trees and blobs. But in addition to the "length XX bytes" you already said the type of what it tags upfront, so I do not think of anything else to say here.

Show 10 quoted lines
> +# --- Config
> +# Set GIT_DIR either from the working directory, or from the environment
> +# variable.
> +GIT_DIR=$(git-rev-parse --git-dir 2>/dev/null)
> +if [ -z "$GIT_DIR" ]; then
> +	echo "fatal: post-receive: GIT_DIR not set"
> +	exit 1
> +fi
> +
> +projectdesc=$(cat $GIT_DIR/description)

This goes on the Subject: header; you'd better make sure the repository is not misconfigured to have more than one lines in it.

Show 6 quoted lines
> +# Check if the description is unchanged from it's default, and shorten it to a
> +# more manageable length if it is
> +if expr "$projectdesc" : "Unnamed repository.*$" >/dev/null
> +then
> +	projectdesc="UNNAMED PROJECT"
> +fi
I vaguely recall you made this die instead of leaving unnamed...
Andy Parkins· Mar 24, 2007, 15:44 UTC · re: Junio C Hamano · lore

Re: [PATCH] Convert emailing part of hooks--update to hooks--post-receive

On Saturday 2007, March 24, Junio C Hamano wrote:
> Is $(<$refname) even POSIX?  Are we sure $refname is always a
> file (not a packed ref)?

Oh dear; I had no idea that packed refs made the ref file itself disappear. Would

 git-rev-parse $refname
Be a better way of getting the revision?
Show 7 quoted lines
> If the other push updates the same ref you are filtering out,
> that would make "grep -v" to remove it from the exclusion set,
> which is a neat trick.  However, wouldn't that cause the
> rev-list to report the effect of the other push, and not your
> push?  I am wondering if the mail would say "Alice pushed and
> this branch has these commits as the result" and ends up
> including Bob's commits in it when that happens.
I hope not, 
  o --- O --- n --- n --- N --- m --- m --- M (refname)
The revs O and N come from stdin so git-receive-pack will pass (with 
refname pointing at M in both cases)
  O N refname
  N M refname
on stdin.  In case 1 (Alice),
  git rev-parse --not --all | grep -v $(git rev-parse $refname) |
  git rev-list --pretty --stdin $newrev

filters out the "^M" from the git-rev-parse output, and the git-rev-list adds N in. So although "^M" isn't the "^N" that the grep is normally there to remove, it doesn't matter because the $newrev makes the rev-list start at "N", ignoring the "m" commits.

> I think that is a good discipline in general, but I wonder if it
> is strictly necessary.  Doesn't hook already get invoked with
> environment that has GIT_EXEC_PATH at the beginning of the PATH?

It's certainly not necessary; just an idea I picked up off the mailing list, but I forget from whom I got it.

Show 8 quoted lines
> Huh?  Did you mean to send the first one also to stderr?
> Huh?  Did you mean to send the second one also to stderr?
>
> It usually is easier to spot this kind of gotchas if you
> consistently write like this instead:
>
> 	echo >&2 "your message comes here..."
> 	echo >&2 "... and the second line is here"
Done, done and done.
> Please never pipe grep into sed, or the other way around.  It
> makes the script look amateurish.

Okay; my unfamiliarity with sed is showing up; I don't use it a lot so I'd say I am an amateur. Fixed.

> Is it assumed that pushing developers do not cross-pull,
> bypassing the repository this hook is in?  Otherwise, if Alice

I did think about that problem, but I couldn't think of a truely "right" solution. It seemed strange to have emails appearing to be from someone who wasn't mentioned in the commits. Also; I couldn't see that there was any sensible way of finding the email address of the person doing the push.

> It would not be a problem as long as you use this repository as
> the _ONLY_ place developers meet, but that's a big assumption:
> centralized repo with CVS-mentality.

I thought it was safe to make that assumption because the use of this hook script implies a centralised-repo mentality.

> The assumption is not wrong per-se, but I think that needs to be
> documented at one place, near the top: "if you subscribe to this
Done.
> Similarly the other assumption your script makes should be
> documented: the "list only new commits to the entire repository"
Done.
> I do not think so.  It is a good style as long as you do not
> botch it and leave the variables empty -- even in that case the
> runtime would notice and say "generate___email: no such command"
> ;-).

I can't see a path through the script that leaves either $change_type or $fn_name unset; so I hope that won't be a problem.

> Please indent the case arms at the same level as "case/esac",
> like this:
Done.
> Micronit.  Some committer names (GIT_COMMITTER_NAME) that are
> not quoted may need to be quoted to appear on the From: header
> (most notable is a name with dot after middle initial).
How does
  committer=$(git show --pretty=full -s $rev | sed -ne "s/^Commit: //" |
		sed -e 's/\(.*\) </"\1" </')
Look?
> A signature separator line is usually "^-- $", not "^--$".
Done.
> I'd rather not to have name Git there, for two reasons.  If you
Done.  $projectdesc used instead.
Show 5 quoted lines
> Also, this is (as described in the first part of the script)
> only an example script, and although I am willing to help making
> sure this is bug-free (like spending time writing this message
> right now -- wow, I've been sitting here typing this e-mail for
> almost two hours), I do not want to be held responsible for any
Wow; that is a long time.
> bugs in it, nor I do not want anybody to mistakenly think that
> what this script does is the way to run things recommended by
> the Git community.  It really is just "a useful script to use
> WITH git Andy wrote", not "THE hook Git gives us".
No problem at all.
> I kind of like the attention to the detail of giving an empty
> argv[1] to echo (it used to be customary to do this for
> portability), but I wonder if it is still needed...

I don't know were I picked that habbit up from; I think some early scripts I wrote needed it, so from then on I just put it in whenever I need a blank.

On ocassion it's also helped for find and replace operations.
> Where does $baserev come from?  Random leftover garbage from
> invocation for the branch the main while loop handled before
> this branch?
Oops.  Done; baserev removed.
Show 9 quoted lines
> You are doing this one branch at a time even though a single
> push may update more than one branch.  I wonder what happens if
> branch A and B are pushed, and they both contain shared new
> material?  When this is run for A, the entire contribution of B
> (after push happened) is excluded.  Then when this is run for
> B, the entire contribution of A is excluded.  Doesn't that mean
> the commits new to this repository that are shared between A and
> B appear for neither branches?  There must be something obvious
> I am missing...

Hmmm, that's a good point. That's an assumption from the old update hook that is no longer true. I have no answer for that.

Show 5 quoted lines
> This comment and logic is wrong.  In the above graph, if you say
> "rev-list O..N" (which is "rev-list N ^O"), you would get
> commits that are reachable from N, excluding the ones that are
> also reachable from O, so you would get N, * before it, X and *
> before it.

The comment was indeed the wrong way round; but I got the baserev stuff from the old hook script:

$ git-blame v1.4.4 -- templates/hooks--update (Junio C Hamano 75) base=$(git-merge-base "$2" "$3") (Junio C Hamano 86) git-rev-list --pretty "$3" "^$base"

I just assumed there was good reason for it and copied it. If rev-list O..N is fine, I'll do that.

Show 6 quoted lines
> That's bog-standard way we use to see if O->N transition is a
> fast forward, but for this application, since it is far more
> likely to be than it isn't, I think computing "rev-list N..O"
> and checking if it reports anything is a better check
> (especially because its results can be used in a way I'll
> describe later).
I like it.

I've thrown away all the baserev stuff and used the N..O notation to list the discarded revisions. This also makes it easy to detect the fast forward as you say.

Show 6 quoted lines
> > +	echo ""
> > +	echo "Summary of changes:"
> > +	git diff-tree --no-color --stat -M -C --find-copies-harder
> > $baserev..$newrev +}
>
> Comment and code mismatch between $baserev and $oldrev?

Comment was right; what I was actually trying to show was the difference between oldrev and newrev, whatever the path it took to get from old to new.

> "-M -C" is redundant.  Just "-C" should do.  If you really want
> to spend extra cycles, then "--find-copies-harder" is fine, but

Done. I think wasting the extra cycles is okay, because no one is going to notice if the notification email is a bit delayed in its generation, but they might appreciate the extra detail in the email that will be there to refer to forever.

> Do you even need --no-color?  I thought we specifically tried
Paranoia on my part.  Removed.
> I would add --summary to the list, though.

Done; but I think the interesting extra information in the summary (i.e. the creations/deletions) would be better if it were on the --stat and the --summary can go again.

Show 11 quoted lines
> > +
> > +#
> > +# Called for the deletion of a branch
> > +#
> > +generate_delete_branch_email()
> > +{
> > +	echo "       was  $oldrev"
> > +	echo ""
> > +}
>
> Perhaps with a one-line?

Don't understand. Do you mean throw the excessive size? I made it a function on purpose to keep the "example" quality. Makes it obvious were to put customisations.

> Perfect example of using for-each-ref and nicely done.  Contrary
> to what you said earlier, I see a hyphen between git and
> for-each-ref, though.
Fixed.  I wish I could take credit, but it was Shawn's idea :-)
Show 5 quoted lines
> > +		# If the tagged object is a commit, then we assume this is a
> > +		# release, and so we calculate which tag this tag is replacing
> > +		prevtag=$(git describe --abbrev=0 $newrev^ 2>/dev/null)
>
> What if $newrev is a merge, I wonder...
Oh yes; but that one is Andreas's fault:
(Andreas Ericsson   54)  prev=$(git describe "$3^" | sed 's/-g.*//')

Can we hope that even if it is a merge, that either parent would be on the same previous tag?

 X --- * -- * --- * --- * -- Y
        \                   /
         * --- * --- * --- *
Not guaranteed, but what else is valid?
> Didn't you mean to update this with:
>
> 	... | sed -e '1,/^$/d'
I did; and could swear I had.  Wonder how I managed to undo that one.
> You won't get a tag to another tag with %(*objecttype), so you
> only have to worry about trees and blobs.  But in addition to
> the "length XX bytes" you already said the type of what it tags
> upfront, so I do not think of anything else to say here.

I did think that when gitattributes arrive, they might offer some interesting possibilities for that section.

> This goes on the Subject: header; you'd better make sure the
> repository is not misconfigured to have more than one lines in
> it.
projectdesc=$(cat $GIT_DIR/description | sed -ne '1p')
?
> I vaguely recall you made this die instead of leaving unnamed...

I did for the update hook; when that would prevent the update; the post-receive hook can't stop the update so we /have/ to issue the email - there is no second chance.

Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com
Linus Torvalds· Mar 24, 2007, 16:12 UTC · re: Andy Parkins · lore

Re: [PATCH] Convert emailing part of hooks--update to hooks--post-receive

On Sat, 24 Mar 2007, Andy Parkins wrote:
> 
> Oh dear; I had no idea that packed refs made the ref file itself 
> disappear.
Everybody should do
	git pack-refs --all --prune

occasionally to see this. In fact, I think we should probably make that part of "git gc" (which currently doesn't pack branches at all, just tags, since it omits the "--all").

Show 5 quoted lines
> Would
> 
>  git-rev-parse $refname
> 
> Be a better way of getting the revision?
That's always the correct way to get a revision, although you should use
	git-rev-parse --verify "$refname"^0

to make sure that you get a commit (of course, if you don't want a commit, but any random ref-SHA1, remove the "^0" from the end!).

HOWEVER. "git-rev-parse" will take any arbitrary SHA1-expression, which may or may not be what you want. If you actually also want to verify that it's strictly a branch name (or other ref-name), rather than just a random SHA1 expression, you should do

	git show-ref [--verify] refname

where the "--verify" again enables strict checking. HOWEVER, it will not check that it's a commit, so if you need the resulting SHA1 to be of a specific type, you need to do that separately.

Side note: those two "--verify" calls do two different kinds of strict checking, because "show-ref" and "rev-parse" are different things. In "git-rev-parse" it checks that the argument is exactly *one* SHA1 value, and not some flag or a SHA1 range. Because "git rev-parse" can take a lot of different input formats. In contrast, in "git show-ref", the stricter checking enabled by "--verify" will just force it to not do any "pattern" for the ref, but it wants an *exact* refname.

		Linus
Junio C Hamano· Mar 25, 2007, 07:13 UTC · re: Linus Torvalds · lore

Re: [PATCH] Convert emailing part of hooks--update to hooks--post-receive

Linus Torvalds <torvalds@linux-foundation.org> writes:
Show 12 quoted lines
> On Sat, 24 Mar 2007, Andy Parkins wrote:
>> 
>> Oh dear; I had no idea that packed refs made the ref file itself 
>> disappear.
>
> Everybody should do
>
> 	git pack-refs --all --prune
>
> occasionally to see this. In fact, I think we should probably make that 
> part of "git gc" (which currently doesn't pack branches at all, just tags, 
> since it omits the "--all").
*Warning*

As described in 1.5.0 release notes, packed and then pruned refs are not visible by commit walkers older than 1.5.0. So everybody should refrain from doing the above on a public "distribution point" repository that can be accessed over http until everybody upgrades.

Junio C Hamano· Mar 25, 2007, 07:07 UTC · re: Andy Parkins · lore

Re: [PATCH] Convert emailing part of hooks--update to hooks--post-receive

Andy Parkins <andyparkins@gmail.com> writes:
Show 14 quoted lines
> On Saturday 2007, March 24, Junio C Hamano wrote:
> ...
>> You are doing this one branch at a time even though a single
>> push may update more than one branch.  I wonder what happens if
>> branch A and B are pushed, and they both contain shared new
>> material?  When this is run for A, the entire contribution of B
>> (after push happened) is excluded.  Then when this is run for
>> B, the entire contribution of A is excluded.  Doesn't that mean
>> the commits new to this repository that are shared between A and
>> B appear for neither branches?  There must be something obvious
>> I am missing...
>
> Hmmm, that's a good point.  That's an assumption from the old update 
> hook that is no longer true.  I have no answer for that.
I think you can do something along the following line.
 (1) You say "for-each-ref --all" to get the ref information
     that is after update.
 (2) Instead of looping and processing one at a time, grab all
     information upfront.  replace the object name you obtained
     in (1) for the ref being updated with the oldrev info you
     discovered.
 (3) Handle the branch one at a time, but instead of using --all,
     use the result of (2) as the "state of the repository and
     its refs before this update".

In other words, you have enough information to reconstruct the view of the repository before your push took place. After running "for-each-ref --all" once upfront, you reconstruct such a view, and after that you do not ever look at where the real refs are in the repository. That would perhaps alleviate the issue of racing pushers as a side effect.

Show 13 quoted lines
>> > +
>> > +#
>> > +# Called for the deletion of a branch
>> > +#
>> > +generate_delete_branch_email()
>> > +{
>> > +	echo "       was  $oldrev"
>> > +	echo ""
>> > +}
>>
>> Perhaps with a one-line?
>
> Don't understand.  Do you mean throw the excessive size?
Sorry, what I meant was:
	git show -s --pretty=oneline "$oldrev"
Show 5 quoted lines
>> Perfect example of using for-each-ref and nicely done.  Contrary
>> to what you said earlier, I see a hyphen between git and
>> for-each-ref, though.
>
> Fixed.  I wish I could take credit, but it was Shawn's idea :-)

Actually it was not quite "nicely done". If taggername has an unusual character then dq pair you hard coded there may not quote the string correctly.

Show 7 quoted lines
> +	# Use git-for-each-ref to pull out the individual fields from the tag
> +	eval $(git-for-each-ref --format='
> +	tagobject="%(*objectname)"
> +	tagtype="%(*objecttype)"
> +	tagger="%(taggername)"
> +	tagged="%(taggerdate)"' $refname
> +	)
I even suspect that a malicious tagger could do:
	GIT_COMMITTER_NAME='Andy `rm -fr .` Parkins' \
        git tag -a my-malicious-tag

and push it to you. Telling for-each-ref to quote properly for the host language, like this:

	git-for-each-ref --shell --format='
        	o=%(*objectname)
                t=%(*objecttype)
                n=%(taggername)
                d=%(taggerdate)' $refname
would be safer.
Show 18 quoted lines
>> > +		# If the tagged object is a commit, then we assume this is a
>> > +		# release, and so we calculate which tag this tag is replacing
>> > +		prevtag=$(git describe --abbrev=0 $newrev^ 2>/dev/null)
>>
>> What if $newrev is a merge, I wonder...
>
> Oh yes; but that one is Andreas's fault:
>
> (Andreas Ericsson   54)  prev=$(git describe "$3^" | sed 's/-g.*//')
>
> Can we hope that even if it is a merge, that either parent would be on 
> the same previous tag?
>
>  X --- * -- * --- * --- * -- Y
>         \                   /
>          * --- * --- * --- *
>
> Not guaranteed, but what else is valid?

You could describe all the parents and see if they differ. If they reach different tag, we could see which one is newer. Or something like that. In the special (but usual) case of a single parent commit, "describing all the parents" is what you are already doing, so it is not any more expensive in the normal case.

Show 7 quoted lines
>> This goes on the Subject: header; you'd better make sure the
>> repository is not misconfigured to have more than one lines in
>> it.
>
> projectdesc=$(cat $GIT_DIR/description | sed -ne '1p')
>
> ?

Please do not have cat on either side of a pipe. That makes a shell script look ... eh, you know the word ;-).

	sed -e 1q "$GIT_DIR/description"
Junio C Hamano· Mar 25, 2007, 07:50 UTC · re: Junio C Hamano · lore

Re: [PATCH] Convert emailing part of hooks--update to hooks--post-receive

Junio C Hamano <junkio@cox.net> writes:
Show 20 quoted lines
> I think you can do something along the following line.
>
>  (1) You say "for-each-ref --all" to get the ref information
>      that is after update.
>
>  (2) Instead of looping and processing one at a time, grab all
>      information upfront.  replace the object name you obtained
>      in (1) for the ref being updated with the oldrev info you
>      discovered.
>
>  (3) Handle the branch one at a time, but instead of using --all,
>      use the result of (2) as the "state of the repository and
>      its refs before this update".
>
> In other words, you have enough information to reconstruct the
> view of the repository before your push took place.  After
> running "for-each-ref --all" once upfront, you reconstruct such
> a view, and after that you do not ever look at where the real
> refs are in the repository.  That would perhaps alleviate the
> issue of racing pushers as a side effect.
A couple of issues around the above suggestion.

Incidentally, the above fixes the "pushing tag and then commit, and pushing commit and then tag, would give different results" problem, as "what's new" computation would not consider the commits the updated tag brought in as "old" commits anymore.

The above fixes "when pushing two branches that share new commits, the shared commits will not be shown on any reports" problem, but if you literally implement it the way I suggested, you would instead list such shared commits that are new to the repository in reports for both branches. This may or may not be a problem, depending on your definition of "new to the repository". On one hand, they *are* new commits introduced by this push (that updates both branches), so in that sense, showing them on reports for both branches may be the right thing to do. On the other hand, you would not see these shared commits on the report for the latter branch if you did the same push as two separate "git push $URL A; git push $URL B" invocations, so showing them only on the report for one of the branches may be the right thing to do, even when they are pushed together (by the way, this is one of the reasons that I think "new to the repository" is wrong. I do not think its semantics is well-defined).

If you want to implement the latter semantics, then after you handle one branch in step (3), you can replace the internal view of "what the repository looked like before this push" by replacing the commit associated with the branch you just have dealt with (which you replaced with its $oldrev in step (2) in the data read from 'for-each-ref --all') with its $newrev. Then the shared commits would be part of the old state of the repository when you handle the other ref, and would not show up in its report. If you do this, you would not want to replace the data after handing a tag, in order to preserve the fix for "new tag hides new commits" problem.

Andy Parkins· Mar 25, 2007, 08:51 UTC · re: Junio C Hamano · lore

Re: [PATCH] Convert emailing part of hooks--update to hooks--post-receive

On Sunday 2007, March 25, Junio C Hamano wrote:
> I think you can do something along the following line.
>
>  (1) You say "for-each-ref --all" to get the ref information
>      that is after update.

Would you mind if I delayed that to a separate patch? It's a significant difference from the original update hook, so I think it's worth having that change separately documented.

> Sorry, what I meant was:
>
> 	git show -s --pretty=oneline "$oldrev"
Ah - no problem.  Done.
> Actually it was not quite "nicely done".  If taggername has an
> unusual character then dq pair you hard coded there may not
> quote the string correctly.
Devious.  Fixed as you suggest.
Show 9 quoted lines
> > (Andreas Ericsson   54)  prev=$(git describe "$3^" | sed
> > 's/-g.*//')
>
> You could describe all the parents and see if they differ.  If
> they reach different tag, we could see which one is newer.  Or
> something like that.  In the special (but usual) case of a
> single parent commit, "describing all the parents" is what you
> are already doing, so it is not any more expensive in the normal
> case.

Again; I think I'll fix that one as a separate patch as it is a change from the behaviour of hooks/update.

> Please do not have cat on either side of a pipe.  That makes a
> shell script look ... eh, you know the word ;-).
>
> 	sed -e 1q "$GIT_DIR/description"

Done. I've taken the liberty of adding a pipe-to-cat, that I hope you'll forgive...

# Output to the terminal in command line mode - if someone wanted to
# resend an email; they could redirect the output to sendmail themselves
  generate_email $2 $3 $1 | cat

This is to force the deactivation of the pager for all the git commands that generate_email calls.

Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com
Junio C Hamano· Mar 25, 2007, 09:13 UTC · re: Andy Parkins · lore

Re: [PATCH] Convert emailing part of hooks--update to hooks--post-receive

Andy Parkins <andyparkins@gmail.com> writes:
Show 8 quoted lines
> On Sunday 2007, March 25, Junio C Hamano wrote:
> ...
>> I think you can do something along the following line.
>>
>>  (1) You say "for-each-ref --all" to get the ref information
>>      that is after update.
>
> Would you mind if I delayed that to a separate patch?
Surely.

As we discussed, I would prefer a patch to create this as a new file in

	contrib/examples/hooks/
directory.  A single-liner 
	# see contrib/example/hooks for examples.

in another new file "templates/hooks--post-receive" would be also nice.

Show 5 quoted lines
>> Actually it was not quite "nicely done".  If taggername has an
>> unusual character then dq pair you hard coded there may not
>> quote the string correctly.
>
> Devious.
and funny, isn't it?
>   generate_email $2 $3 $1 | cat
>
> This is to force the deactivation of the pager for all the git commands 
> that generate_email calls.
I think you could probably do the same with:
	PAGER= generate_email "$2" "$3" "$1"
Andy Parkins· Mar 30, 2007, 19:16 UTC · re: Junio C Hamano · lore

[PATCH v2] Reimplement emailing part of hooks--update in contrib/hooks/post-receive-email

The update hook is no longer the correct place to generate emails; there is now the hooks/post-receive script which is run automatically after a ref has been updated.

This patch is to make use of that new location, and to address some faults in the old update hook.

The primary problem in the conversion was that in the update hook, the ref has not actually been changed, but is about to be. In the post-receive hook the ref has already been updated. That meant that where we previously had lines like:

 git rev-list --not --all

would now give the wrong list because "--all" in the post-receive hook includes the ref that we are making the email for. This made it more difficult to show only the new revisions added by this update.

The solution is not pretty; however it does work and doesn't need any changes to git-rev-list itself. It also fixes (more accurately: reduces the likelihood of) a nasty race when another update occurs while this script is running. The solution, in short, looks like this (see the source code for a longer explanation)

 git rev-parse --not --all | grep -v $(git rev-parse $refname) |
 git rev-list --pretty --stdin $oldrev..$newrev

This uses git-rev-parse followed by grep to filter out the revision of the ref in question before it gets to rev-list and inhibits the output of itself. By using $(git rev-parse $revname) rather than $newrev as the filter, it also takes care of the situation where another update to the same ref has been made since $refname was $newrev.

The second problem that is addressed is that of tags inhibiting the correct output of an update email. Consider this, with somebranch and sometag pointing at the same revision:

 git push origin somebranch
 git push origin sometag

That would work fine; the push of the branch would generate an email containing all the new commits introduced by the update, then the push of the tag would generate the shortlog formatted tag email. Now consider:

 git push origin sometag
 git push origin somebranch

When some branch comes to run its "--not --all" line, it will find sometag, and filter those commits from the email - leaving nothing. That meant that those commits would not show (in full) on any email. The fix is to not use "--all", and instead use "--branches" in the git-rev-parse command.

Other changes
 * Lose the monstrous one-giant-script layout and put things in easy to
   digest functions.  This makes it much easier to find the place you
   need to change if you wanted to customise the output.  I've also
   tried to write more verbose comments for the same reason.  The hook
   script is big, mainly because of all the different cases that it has
   to handle, so being easy to navigate is important.
 * All uses of "git-command" changed to "git command", to cope better
   if a user decided not to install all the hard links to git;
 * Cleaned up some of the English in the email
 * The fact that the receive hook makes the ref available also allows me
   to use Shawn Pearce's fantastic suggestion that an annotated tag can
   be parsed with git-for-each-ref.  This removes the potentially
   non-portable use of "<<<" heredocs and the nasty messing around with
   "date" to convert numbers of seconds UTC to a real date
 * Deletions are now caught and notified (briefly)
 * To help with debugging, I've retained the command line mode from the
   update hook; but made it so that the output is not emailed, it's just
   printed to the screen.  This could then be redirected if the user
   wanted
 * Removed the "Hello" from the beginning of the email - it's just
   noise, and no one seriously has their day made happier by "friendly"
   programs
 * The fact that it doesn't rely on repository state as an indicator any
   more means that it's far more stable in its output; hopefully the
   same arguments will always generate the same email - even if the
   repository changes in the future.  This means you can easily recreate
   an email should you want to.
 * Included Jim Meyering's envelope sender option for the sendmail call
 * The hook is now so big that it was inappropriate to copy it
   to every repository by keeping it in the templates directory.
   Instead, I've put a comment saying to look in contrib/hooks, and
   given an example of calling the script from that template hook.  The
   advantage of calling the script residing at some fixed location is
   that if a future package of git included a bug fixed version of the
   script, that would be picked up automatically, and the user would not
   have to notice and manually copy the new hook to every repository
   that uses it.
Signed-off-by: Andy Parkins <andyparkins@gmail.com>
---
 contrib/hooks/post-receieve-email |  588 +++++++++++++++++++++++++++++++++++++
 templates/hooks--post-receive     |   17 +
 2 files changed, 605 insertions(+), 0 deletions(-)
 create mode 100644 contrib/hooks/post-receieve-email
 create mode 100644 templates/hooks--post-receive
Show changes to 2 files +605 −0

contrib/hooks/post-receieve-email, templates/hooks--post-receive

diff --git a/contrib/hooks/post-receieve-email b/contrib/hooks/post-receieve-email
new file mode 100644
index 0000000..6516015
--- /dev/null
+++ b/contrib/hooks/post-receieve-email
@@ -0,0 +1,588 @@
+#!/bin/sh
+#
+# Copyright (c) 2007 Andy Parkins
+#
+# An example hook script to mail out commit update information.  This hook sends emails
+# listing new revisions to the repository introduced by the change being reported.  The
+# rule is that (for branch updates) each commit will appear on one email and one email
+# only.
+#
+# This hook is stored in the contrib/hooks directory.  Your distribution will have put
+# this somewhere standard.  You should make this script executable then link to it in
+# the repository you would like to use it in.  For example, on debian the hook is stored
+# in /usr/share/doc/git-core/contrib/hooks/post-receive-email:
+#
+#  chmod a+x post-receive-email
+#  cd /path/to/your/repository.git
+#  ln -sf /usr/share/doc/git-core/contrib/hooks/post-receive-email hooks/post-receive
+#
+# This hook script assumes it is enabled on the central repository of a project, with
+# all users pushing only to it and not between each other.  It will still work if you
+# don't operate in that style, but it would become possible for the email to be from
+# someone other than the person doing the push.
+#
+# Config
+# ------
+# hooks.mailinglist
+#   This is the list that all pushes will go to; leave it blank to not send
+#   emails for every ref update.
+# hooks.announcelist
+#   This is the list that all pushes of annotated tags will go to.  Leave it
+#   blank to default to the mailinglist field.  The announce emails lists the
+#   short log summary of the changes since the last annotated tag.
+# hook.envelopesender
+#   If set then the -f option is passed to sendmail to allow the envelope sender
+#   address to be set
+#
+# Notes
+# -----
+# All emails have their subjects prefixed with "[SCM]" to aid filtering.
+# All emails include the headers "X-Git-Refname", "X-Git-Oldrev",
+# "X-Git-Newrev", and "X-Git-Reftype" to enable fine tuned filtering and
+# give information for debugging.
+#
+
+# ---------------------------- Functions
+
+#
+# Top level email generation function.  This decides what type of update
+# this is and calls the appropriate body-generation routine after outputting
+# the common header
+#
+# Note this function doesn't actually generate any email output, that is taken
+# care of by the functions it calls:
+#  - generate_email_header
+#  - generate_create_XXXX_email
+#  - generate_update_XXXX_email
+#  - generate_delete_XXXX_email
+#  - generate_email_footer
+#
+generate_email()
+{
+	# --- Arguments
+	oldrev=$(git rev-parse $1)
+	newrev=$(git rev-parse $2)
+	refname="$3"
+
+	# --- Interpret
+	# 0000->1234 (create)
+	# 1234->2345 (update)
+	# 2345->0000 (delete)
+	if expr "$oldrev" : '0*$' >/dev/null
+	then
+		change_type="create"
+	else
+		if expr "$newrev" : '0*$' >/dev/null
+		then
+			change_type="delete"
+		else
+			change_type="update"
+		fi
+	fi
+
+	# --- Get the revision types
+	newrev_type=$(git cat-file -t $newrev 2> /dev/null)
+	oldrev_type=$(git cat-file -t "$oldrev" 2> /dev/null)
+	case "$change_type" in
+	create|update)
+		rev="$newrev"
+		rev_type="$newrev_type"
+		;;
+	delete)
+		rev="$oldrev"
+		rev_type="$oldrev_type"
+		;;
+	esac
+
+	# The revision type tells us what type the commit is, combined with
+	# the location of the ref we can decide between
+	#  - working branch
+	#  - tracking branch
+	#  - unannoted tag
+	#  - annotated tag
+	case "$refname","$rev_type" in
+		refs/tags/*,commit)
+			# un-annotated tag
+			refname_type="tag"
+			short_refname=${refname##refs/tags/}
+			;;
+		refs/tags/*,tag)
+			# annotated tag
+			refname_type="annotated tag"
+			short_refname=${refname##refs/tags/}
+			# change recipients
+			if [ -n "$announcerecipients" ]; then
+				recipients="$announcerecipients"
+			fi
+			;;
+		refs/heads/*,commit)
+			# branch
+			refname_type="branch"
+			short_refname=${refname##refs/heads/}
+			;;
+		refs/remotes/*,commit)
+			# tracking branch
+			refname_type="tracking branch"
+			short_refname=${refname##refs/remotes/}
+			echo >&2 "*** Push-update of tracking branch, $refname"
+			echo >&2 "***  - no email generated."
+			exit 0
+			;;
+		*)
+			# Anything else (is there anything else?)
+			echo >&2 "*** Unknown type of update to $refname ($rev_type)"
+			echo >&2 "***  - no email generated"
+			exit 1
+			;;
+	esac
+
+	# Check if we've got anyone to send to
+	if [ -z "$recipients" ]; then
+		echo >&2 "*** hooks.recipients is not set so no email will be sent"
+		echo >&2 "*** for $refname update $oldrev->$newrev"
+		exit 0
+	fi
+
+	# Email parameters
+	# The committer will be obtained from the latest existing rev; so
+	# for a deletion it will be the oldrev, for the others, then newrev
+	committer=$(git show --pretty=full -s $rev | sed -ne "s/^Commit: //p" |
+		sed -ne 's/\(.*\) </"\1" </p')
+	# The email subject will contain the best description of the ref
+	# that we can build from the parameters
+	describe=$(git describe $rev 2>/dev/null)
+	if [ -z "$describe" ]; then
+		describe=$rev
+	fi
+
+	generate_email_header
+
+	# Call the correct body generation function
+	fn_name=general
+	case "$refname_type" in
+	"tracking branch"|branch)
+		fn_name=branch
+		;;
+	"annotated tag")
+		fn_name=atag
+		;;
+	esac
+	generate_${change_type}_${fn_name}_email
+
+	generate_email_footer
+}
+
+generate_email_header()
+{
+	# --- Email (all stdout will be the email)
+	# Generate header
+	cat <<-EOF
+	From: $committer
+	To: $recipients
+	Subject: ${EMAILPREFIX}$projectdesc $refname_type, $short_refname, ${change_type}d. $describe
+	X-Git-Refname: $refname
+	X-Git-Reftype: $refname_type
+	X-Git-Oldrev: $oldrev
+	X-Git-Newrev: $newrev
+
+	This is an automated email from the git hooks/post-receive script. It was
+	generated because a ref change was pushed to the repository containing
+	the project "$projectdesc".
+
+	The $refname_type, $short_refname has been ${change_type}d
+	EOF
+}
+
+generate_email_footer()
+{
+	cat <<-EOF
+
+
+	hooks/post-receive
+	-- 
+	$projectdesc
+	EOF
+}
+
+# --------------- Branches
+
+#
+# Called for the creation of a branch
+#
+generate_create_branch_email()
+{
+	# This is a new branch and so oldrev is not valid
+	echo "        at  $newrev ($newrev_type)"
+	echo ""
+
+	echo $LOGBEGIN
+	# This shows all log entries that are not already covered by
+	# another ref - i.e. commits that are now accessible from this
+	# ref that were previously not accessible (see generate_update_branch_email
+	# for the explanation of this command)
+	git rev-parse --not --branches | grep -v $(git rev-parse $refname) |
+	git rev-list --pretty --stdin $newrev
+	echo $LOGEND
+}
+
+#
+# Called for the change of a pre-existing branch
+#
+generate_update_branch_email()
+{
+	# Consider this:
+	#   1 --- 2 --- O --- X --- 3 --- 4 --- N
+	#
+	# O is $oldrev for $refname
+	# N is $newrev for $refname
+	# X is a revision pointed to by some other ref, for which we may
+	#   assume that an email has already been generated.
+	# In this case we want to issue an email containing only revisions
+	# 3, 4, and N.  Given (almost) by
+	#
+	#  git-rev-list N ^O --not --all
+	#
+	# The reason for the "almost", is that the "--not --all" will take
+	# precedence over the "N", and effectively will translate to
+	#
+	#  git-rev-list N ^O ^X ^N
+	#
+	# So, we need to build up the list more carefully.  git-rev-parse will
+	# generate a list of revs that may be fed into git-rev-list.  We can get
+	# it to make the "--not --all" part and then filter out the "^N" with:
+	#
+	#  git-rev-parse --not --all | grep -v N
+	#
+	# Then, using the --stdin switch to git-rev-list we have effectively
+	# manufactured
+	#
+	#  git-rev-list N ^O ^X
+	#
+	# This leaves a problem when someone else updates the repository
+	# while this script is running.  Their new value of the ref we're working
+	# on would be included in the "--not --all" output; and as our $newrev
+	# would be an ancestor of that commit, it would exclude all of our
+	# commits.  What we really want is to exclude the current value of
+	# $refname from the --not list, rather than N itself.  So:
+	#
+	#  git-rev-parse --not --all | grep -v $(git-rev-parse $refname)
+	#
+	# Get's us to something pretty safe (apart from the small time between
+	# refname being read, and git-rev-parse running - for that, I give up)
+	#
+	#
+	# Next problem, consider this:
+	#   * --- B --- * --- O ($oldrev)
+	#          \
+	#           * --- X --- * --- N ($newrev)
+	#
+	# That is to say, there is no guarantee that oldrev is a strict subset of
+	# newrev (it would have required a --force, but that's allowed).  So, we
+	# can't simply say rev-list $oldrev..$newrev.  Instead we find the common
+	# base of the two revs and list from there.
+	#
+	# As above, we need to take into account the presence of X; if another
+	# branch is already in the repository and points at some of the revisions
+	# that we are about to output - we don't want them.  The solution is as
+	# before: git-rev-parse output filtered.
+	#
+	# Finally, tags:
+	#   1 --- 2 --- O --- T --- 3 --- 4 --- N
+	#
+	# Tags pushed into the repository generate nice shortlog emails that
+	# summarise the commits between them and the previous tag.  However,
+	# those emails don't include the full commit messages that we output
+	# for a branch update.  Therefore we still want to output revisions
+	# that have been output on a tag email.
+	#
+	# Luckily, git-rev-parse includes just the tool.  Instead of using "--all"
+	# we use "--branches"; this has the added benefit that "remotes/" will
+	# be ignored as well.
+
+	# List all of the revisions that were removed by this update, in a fast forward
+	# update, this list will be empty, because rev-list O ^N is empty.  For a non
+	# fast forward, O ^N is the list of removed revisions
+	fastforward=""
+	rev=""
+	for rev in $(git rev-list $newrev..$oldrev)
+	do
+		revtype=$(git cat-file -t "$rev")
+		echo "  discards  $rev ($revtype)"
+	done
+	if [ -z "$rev" ]; then
+		fast_forward=1
+	fi
+
+	# List all the revisions from baserev to newrev in a kind of
+	# "table-of-contents"; note this list can include revisions that have
+	# already had notification emails and is present to show the full detail
+	# of the change from rolling back the old revision to the base revision and
+	# then forward to the new revision
+	for rev in $(git rev-list $oldrev..$newrev)
+	do
+		revtype=$(git cat-file -t "$rev")
+		echo "       via  $rev ($revtype)"
+	done
+
+	if [ -z "$fastforward" ]; then
+		echo "      from  $oldrev ($oldrev_type)"
+	else
+		echo ""
+		echo "This update added new revisions after undoing old revisions.  That is to"
+		echo "say, the old revision is not a strict subset of the new revision.  This"
+		echo "situation occurs when you --force push a change and generate a"
+		echo "repository containing something like this:"
+		echo ""
+		echo " * -- * -- B -- O -- O -- O ($oldrev)"
+		echo "            \\"
+		echo "             N -- N -- N ($newrev)"
+		echo ""
+		echo "When this happens we assume that you've already had alert emails for all"
+		echo "of the O revisions, and so we here report only the revisions in the N"
+		echo "branch from the common base, B."
+	fi
+
+	echo ""
+	echo "Those revisions listed above that are new to this repository have"
+	echo "not appeared on any other notification email; so we list those"
+	echo "revisions in full, below."
+
+	echo ""
+	echo $LOGBEGIN
+	git rev-parse --not --branches | grep -v $(git rev-parse $refname) |
+	git rev-list --pretty --stdin $oldrev..$newrev
+
+	# XXX: Need a way of detecting whether git rev-list actually outputted
+	# anything, so that we can issue a "no new revisions added by this
+	# update" message
+
+	echo $LOGEND
+
+	# The diffstat is shown from the old revision to the new revision.  This
+	# is to show the truth of what happened in this change.  There's no point
+	# showing the stat from the base to the new revision because the base
+	# is effectively a random revision at this point - the user will be
+	# interested in what this revision changed - including the undoing of
+	# previous revisions in the case of non-fast forward updates.
+	echo ""
+	echo "Summary of changes:"
+	git diff-tree --stat --summary --find-copies-harder $oldrev..$newrev
+}
+
+#
+# Called for the deletion of a branch
+#
+generate_delete_branch_email()
+{
+	echo "       was  $oldrev"
+	echo ""
+	echo $LOGEND
+	git show -s --pretty=oneline $oldrev
+	echo $LOGEND
+}
+
+# --------------- Annotated tags
+
+#
+# Called for the creation of an annotated tag
+#
+generate_create_atag_email()
+{
+	echo "        at  $newrev ($newrev_type)"
+
+	generate_atag_email
+}
+
+#
+# Called for the update of an annotated tag (this is probably a rare event
+# and may not even be allowed)
+#
+generate_update_atag_email()
+{
+	echo "        to  $newrev ($newrev_type)"
+	echo "      from  $oldrev (which is now obsolete)"
+
+	generate_atag_email
+}
+
+#
+# Called when an annotated tag is created or changed
+#
+generate_atag_email()
+{
+	# Use git-for-each-ref to pull out the individual fields from the tag
+	eval $(git for-each-ref --shell --format='
+	tagobject=%(*objectname)
+	tagtype=%(*objecttype)
+	tagger=%(taggername)
+	tagged=%(taggerdate)' $refname
+	)
+
+	echo "   tagging  $tagobject ($tagtype)"
+	case "$tagtype" in
+	commit)
+		# If the tagged object is a commit, then we assume this is a
+		# release, and so we calculate which tag this tag is replacing
+		prevtag=$(git describe --abbrev=0 $newrev^ 2>/dev/null)
+
+		if [ -n "$prevtag" ]; then
+			echo "  replaces  $prevtag"
+		fi
+		;;
+	*)
+		echo "    length  $(git cat-file -s $tagobject) bytes"
+		;;
+	esac
+	echo " tagged by  $tagger"
+	echo "        on  $tagged"
+
+	echo ""
+	echo $LOGBEGIN
+
+	# Show the content of the tag message; this might contain a change log
+	# or release notes so is worth displaying.
+	git cat-file tag $newrev | sed -e '1,/^$/d'
+
+	echo ""
+	case "$tagtype" in
+	commit)
+		# Only commit tags make sense to have rev-list operations performed
+		# on them
+		if [ -n "$prevtag" ]; then
+			# Show changes since the previous release
+			git rev-list --pretty=short "$prevtag..$newrev" | git shortlog
+		else
+			# No previous tag, show all the changes since time began
+			git rev-list --pretty=short $newrev | git shortlog
+		fi
+		;;
+	*)
+		# XXX: Is there anything useful we can do for non-commit objects?
+		;;
+	esac
+
+	echo $LOGEND
+}
+
+#
+# Called for the deletion of an annotated tag
+#
+generate_delete_atag_email()
+{
+	echo "       was  $oldrev"
+	echo ""
+	echo $LOGEND
+	git show -s --pretty=oneline $oldrev
+	echo $LOGEND
+}
+
+# --------------- General references
+
+#
+# Called when any other type of reference is created (most likely a
+# non-annotated tag)
+#
+generate_create_general_email()
+{
+	echo "        at  $newrev ($newrev_type)"
+
+	generate_general_email
+}
+
+#
+# Called when any other type of reference is updated (most likely a
+# non-annotated tag)
+#
+generate_update_general_email()
+{
+	echo "        to  $newrev ($newrev_type)"
+	echo "      from  $oldrev"
+
+	generate_general_email
+}
+
+#
+# Called for creation or update of any other type of reference
+#
+generate_general_email()
+{
+	# Unannotated tags are more about marking a point than releasing a version;
+	# therefore we don't do the shortlog summary that we do for annotated tags
+	# above - we simply show that the point has been marked, and print the log
+	# message for the marked point for reference purposes
+	#
+	# Note this section also catches any other reference type (although there
+	# aren't any) and deals with them in the same way.
+
+	echo ""
+	if [ "$newrev_type" = "commit" ]; then
+		echo $LOGBEGIN
+		git show --no-color --root -s $newrev
+		echo $LOGEND
+	else
+		# What can we do here?  The tag marks an object that is not a commit,
+		# so there is no log for us to display.  It's probably not wise to
+		# output git-cat-file as it could be a binary blob.  We'll just say how
+		# big it is
+		echo "$newrev is a $newrev_type, and is $(git cat-file -s $newrev) bytes long."
+	fi
+}
+
+#
+# Called for the deletion of any other type of reference
+#
+generate_delete_general_email()
+{
+	echo "       was  $oldrev"
+	echo ""
+	echo $LOGEND
+	git show -s --pretty=oneline $oldrev
+	echo $LOGEND
+}
+
+# ---------------------------- main()
+
+# --- Constants
+EMAILPREFIX="[SCM] "
+LOGBEGIN="- Log -----------------------------------------------------------------"
+LOGEND="-----------------------------------------------------------------------"
+
+# --- Config
+# Set GIT_DIR either from the working directory, or from the environment
+# variable.
+GIT_DIR=$(git rev-parse --git-dir 2>/dev/null)
+if [ -z "$GIT_DIR" ]; then
+	echo >&2 "fatal: post-receive: GIT_DIR not set"
+	exit 1
+fi
+
+projectdesc=$(sed -e '1p' "$GIT_DIR/description")
+# Check if the description is unchanged from it's default, and shorten it to a
+# more manageable length if it is
+if expr "$projectdesc" : "Unnamed repository.*$" >/dev/null
+then
+	projectdesc="UNNAMED PROJECT"
+fi
+
+recipients=$(git repo-config hooks.mailinglist)
+announcerecipients=$(git repo-config hooks.announcelist)
+envelopesender=$(git-repo-config hooks.envelopesender)
+
+# --- Main loop
+# Allow dual mode: run from the command line just like the update hook, or if
+# no arguments are given then run as a hook script
+if [ -n "$1" -a -n "$2" -a -n "$3" ]; then
+	# Output to the terminal in command line mode - if someone wanted to
+	# resend an email; they could redirect the output to sendmail themselves
+	PAGER= generate_email $2 $3 $1
+else
+	if [ -n "$envelopesender" ]; then
+		envelopesender="-f '$envelopesender'"
+	fi
+
+	while read oldrev newrev refname
+	do
+		generate_email $oldrev $newrev $refname |
+		/usr/sbin/sendmail -t $envelopesender
+	done
+fi
diff --git a/templates/hooks--post-receive b/templates/hooks--post-receive
new file mode 100644
index 0000000..190de26
--- /dev/null
+++ b/templates/hooks--post-receive
@@ -0,0 +1,17 @@
+#!/bin/sh
+#
+# An example hook script for the post-receive event
+#
+# This script is run after receive-pack has accepted a pack and the
+# repository has been updated.  It is passed arguments in through stdin
+# in the form
+#  <oldrev> <newrev> <refname>
+# For example:
+#  aa453216d1b3e49e7f6f98441fa56946ddcd6a20 68f7abf4e6f922807889f52bc043ecd31b79f814 refs/heads/master
+#
+# see contrib/hooks/ for an sample, or uncomment the next line (on debian)
+#
+
+
+#. /usr/share/doc/git-core/contrib/hooks/post-receive-email
+
-- 
1.5.1.rc1.27.g1d848
Jakub Narebski· Mar 31, 2007, 11:36 UTC · re: Andy Parkins · lore

Re: [PATCH v2] Reimplement emailing part of hooks--update in contrib/hooks/post-receive-email

Andy Parkins wrote:
Show 5 quoted lines
>  git rev-list --not --all
> 
> would now give the wrong list because "--all" in the post-receive hook
> includes the ref that we are making the email for.  This made it more
> difficult to show only the new revisions added by this update.
What about "git rev-list --not --all ^included" ?
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

← back to recent threads