git-feed-mail-list.sh

I've just updated the script which feeds the git-commits-{head,24}@vger.kernel.org lists -- it now looks like this...

- Git-only; no longer uses cg-*
- No longer uses its own clone repo -- works directly from the original
- Handles MIME encoding of Subject: headers when necessary
  (although this is a bit icky in shell; especially _my_ shell).

Should I be using git-format-patch for this?

David Woodhouse <dwmw2@infradead.org> writes:

I am not sure if this part is tested..

Maybe you meant 'if test -n "$GIT_COMMITTER_EMAIL"' here?

If you were to do bashism local, don't you want to also localize other variables like key, SUBHEX, NEWSUB,...?

It may make sense to enhance format-patch to do the Q encoding, so that you do not have to do this part by hand...

With GIT 1.3.0 and later:

	git diff --patch-with-stat $parent..$commit

would be simpler here.

lastmail=`git rev-parse --default "$MAILTAG" ${2+"$2"}`

I am not sure if earlier base=$(git-rev-parse $1) is needed if you do this here...

Junio C Hamano <junkio@cox.net> writes:

Or at least lose "diffstat -p1" and replace it with

	git-apply --stat --status

which would be more pleasant.

As I wrote it this is broken, sorry.

This assumes you stop doing "MAILTAG=.git/refs/tags/MailDone" by hand and lose "do we have GIT_DIR" logic as well. Instead define MAILTAG=tags/MailDone or maybe refs/tags/MailDone and let "git rev-parse --default refs/tags/MailDone" figure out what to do when GIT_DIR is set or unset.

On Wed, 2006-05-03 at 21:35 -0700, Junio C Hamano wrote:

Yes, that would be useful. We should perhaps to the From: and To: headers too. Here's my current version (thanks for the feedback)...

The remaining problem is that the invocation of 'date' doesn't work with new versions of coreutils. This...

   date=(${rest#*> })
   sec=${date[0]}; tz=${date[1]}
   dtz=${tz/+/+ }; dtz=${dtz/-/- }
   pdate="$(date -Rud "1970-01-01 UTC + $sec sec $dtz" 2>/dev/null)"

... doesn't work any more on FC-5, because:

 $ date -Rud '1970-01-01 UTC + 1147104611 sec + 0100'
date: invalid date `1970-01-01 UTC + 1147104611 sec + 0100'

On Tue, 9 May 2006, David Woodhouse wrote:

Well, you might choose to just not use "git-cat-file commit" but instead ask git to format the thing for you.

Ie you could probably more easily parse the data from something like

	git show -B --patch-with-stat --pretty=fuller $commit

instead of using "git-cat-file commit $commit" and generating the stat and diff manually.

That way you get the dates etc pretty-printed for you by git.

			Linus

On Mon, 2006-05-08 at 17:19 -0700, Linus Torvalds wrote:

Aha, thanks. Git has learned to do a lot more since I first started hacking up a copy of git-log.sh to feed the mailing lists, and it even had to walk the commit tree manually :)

The output of (the undocumented) '--pretty=fuller' is probably good enough that I can just feed the mailing list with it directly. I think I have to add the commit and the parent manually, but that's easy enough to do -- the commit is obviously known, and the parent is just $(git-rev-parse $commit^1).

Anyone got any objections to switching the kernel git-commits-* lists to this format?

David Woodhouse <dwmw2@infradead.org> writes:

If you are going to parse it anyway, this would probably be easier.

	$ git show --patch-with-stat --pretty=fuller --parents "$commit"

Why would you want -B, by the way?

On 5/9/06, Junio C Hamano <junkio@cox.net> wrote:

What is --parents for ?

Is there a way to have a --pretty=fuller with date append to Author and Commiter name and email instead of have 1 lign after each ?

-- Beber #e.fr@freenode

"Bertrand Jacquin" <beber.mailing@gmail.com> writes:

In the part you did not quote, David wanetd a way to grab the parents of the commit in question, so I gave it to him.

sed is your friend.

On 5/9/06, Junio C Hamano <junkio@cox.net> wrote:

Ok, sorry. I read too fast.

-- Beber #e.fr@freenode

On Tue, 9 May 2006, David Woodhouse wrote:

As long as the "commit <sha1>" id is there (and "--pretty=fuller" does have it), I'll be happy. At some point, the commit mailing list didn't actually mention the commit ID itself, just the tree/parent IDs.

The "fuller" format should be fine, if you care about committer. Otherwise just use the standard "--pretty", which drops committer info.

		Linus

On Mon, 2006-05-08 at 17:45 -0700, Linus Torvalds wrote:

Ah, right. Those are _commit_ IDs in that strange first line. I'll reformat those to 'Commit:' and 'Parent:' for the mailing list.

Having 'git-show --pretty=email' would be nice. I think Junio is working on something which will achieve that, right?

Yeah, I know -- and I got complaints :)

David Woodhouse <dwmw2@infradead.org> writes:

That's the replacement of "git format-patch". If you have a chance, please try out what's in "next". Johannes did quite a nice enhancements.

On Mon, 2006-05-08 at 18:05 -0700, Junio C Hamano wrote:

I think I'd best wait for it to turn up in the release; preferably already capable of MIME quoting. I don't like touching the scripts which feed the mailing lists :)

This is what I currently have... since it changes the format I think I'll won't deploy it yet -- I'll wait until the "replacement of git-format-patch" is done, in case that would change the format _again_. I'd rather not change the format I send to the list twice within a few weeks.

Hello!

Wouldn't it be easier to feed the output to a MUA and letting it handle the MIME stuff for you?

I am using mutt for this purpose:

mutt -x -e 'set charset="utf-8"; set send_charset="us-ascii:iso-8859-2:utf-8"' -s "$subj" "$recipient" <$out

				Have a nice fortnight

On Tue, 9 May 2006, David Woodhouse wrote:

Right. That first line (that starts with "commit") lists the commit ID, and if you say "--parents", the commit ID's of the parents will be appended.

So if you want to turn that into "Commit: <id>" and "Parent: <id>", you'll want to do something like this:

  git show --no-abbrev -C --patch-with-stat --pretty=fuller --parents $commit |
	sed '1 s/commit \([0-9a-f]*\)/Commit:     \1\nParent:    /'

which should look pretty (count the spaces to make sure it lines up right with the other fields).

(And if you ever want to report on merges, you'll want to change that a bit, but it should be reasonably close to the above)

		Linus

On 5/9/06, Linus Torvalds <torvalds@osdl.org> wrote:

Is there a way to track merge like that ? Documentation is not very clear and near from empty.

-- Beber #e.fr@freenode

"Bertrand Jacquin" <beber.mailing@gmail.com> writes:

The command line you quoted shows the relevant information for people who want to know what happened in that merge.

Namely:

 * it always shows the header and the message

 * it shows the changes that are not trivial (i.e. merge parents
   have overlapping different versions and manual resolution
   resulted in something different from either parents).

It is not a replacement for format-patch, but I think the commit mailing list is not for machines to receive and apply the received patches, but for humans to inspect, so it would be more suitable than a naive alternative of showing diff from all parents concatenated together.

On 5/9/06, Junio C Hamano <junkio@cox.net> wrote:

That's right. And don't want to do that.

But I would like to send an email after merge to inform people that:

o tree ``a'' and ``b'' have been merged. o made by John Doe at a time o show a diffstat. o show a --short-log=oneline from merge base.

-- Beber #e.fr@freenode

"Bertrand Jacquin" <beber.mailing@gmail.com> writes:

Forgetting about a fast-forward merge, (1) and (2) are available in the commit header and the commit log, so is (4) if you enable merge.summary configuration like Linus does in his kernel repository.

The comment on diffstat Linus already made applies to (3), but if you want you could do "git diff --stat HEAD^..HEAD" to see what happened to that branch by merging the other branch into it.

On Mon, 8 May 2006, Junio C Hamano wrote:

NOTE! Please don't enable "merge.summary" if you ever merge from the upstream tree. That just looks ugly. Your merge messages will be just filled with crap that has nothing to do with your tree - and everything to do with all the _unrelated_ normal development that happened in the tree.

So in general, "merge.summary" makes sense only for trees that pull from downstreams, and never merge with anything upstream. My tree obviously does that for the kernel. Think of it as a "top-level maintainer" flag, although it works find also for sub-maintainers as long as they synchronize upwards _purely_ by being pulled from, not by pulling.

But if you want to get it for any random merges, you can always just do

	git log -11 --pretty=oneline ^$commit^ $commit^@ |
		sed 's/[0-9a-f]* // ; 11 s/.*/\.\.\./' 

which will show up to the ten first commits that were merged (and turn the eleventh one, if it exists, into "..." - that's a pretty disgusting trick to make it show when you left things out).

That "^$commit^ $commit^@" part is important. It may look like some deranged git smiley, but it does exactly what you want it to do: take all the parents of the commit, but ignore any commit reachable from the first one (the "mainline" of the person who did the commit).

The ^@ syntax is obviously pretty new, so it requires a modern git.

		Linus

Linus Torvalds <torvalds@osdl.org> writes:

It is indeed very quite new. Merged into "master" branch at the beginning of this month.

I often wish we had a straightforward way to tell when a given feature went into the mainline, not just appeared on a topic branch. In this case, I said:

	$ git whatchanged -p -S'"^@"' master -- revision.c

to find ea4a19 commit (Apr 30 00:54:29 2006 -0700). But that was when the feature was first made on one of my topic branches, which is not what I was looking for.

By looking at gitk, I can then tell 83262e (May 1 01:54:27) merged it to "next", and 746437 (May 1 22:55:40) merged it to "master".

In general this is an unsolvable question, because I can have a topic branch forked off of the tip of "master", cook it for a few days without advancing "master" at all, and merge it to "master" after that. But such a merge will be a fast-forward.

On Tue, 9 May 2006, Bertrand Jacquin wrote:

Sure.

If you want to track merges and get their patches, add the "--cc" flag, which tells git to use the "conflict combination patch" that shows any visible conflicts.

(NOTE NOTE NOTE! This is _not_ the same as showing what conflicted: if you edited the result to match one of the original branches, it will be quiet in --cc, but if the result of a conflict was something that was in _neither_ branch, it will be shown! So most clean merges will not show any conflict diff at all, but the diffstat will be shown against the "first parent").

And you probably don't want to abbreviate the parent commit SHA1's (which are shown for merges, but not regular commits), so add "--no-abbrev".

If you want to show parents for _all_ commits, you could do something like

  git show --no-abbrev --cc -C --patch-with-stat --pretty=fuller --parents |
	sed '1 s/commit [0-9a-f]*/\0\nParents:   / ; /^Merge: / d'

which removes a potential "Merge: " line in favour of listing the parents on a "Parents:" line, and which also shows merges nicely.

That said, the diffstat for merges is usually just a lot of noise. It's sometimes nice (you've merged from a topic branch), but if you have merged from the mainline _into_ a topic branch, it's just annoying.

So the above is just a wild suggestion. Caveat emptor.

		Linus

Linus Torvalds <torvalds@osdl.org> writes:

Actually, show defaults to --cc so what's shown is good as is.

True. We made --cc --patch-with-stat to do the combined patch text with diffstat for first-parent-diff to make it most natural for merging into upstream, not merging from upstream.