threads / discuss / 28232

Idea: "git format-patch" should get more information out of git

Subject: Idea: "git format-patch" should get more information out of git

## tl;dr

12 messages between Aug 27, 2011 and Sep 1, 2011.

replies: 11people: 5as markdown or json

Michael Haggerty· Aug 27, 2011, 05:12 UTC · lore

It is a nuisance that the emails for a patch series have to be hand-edited after they are generated by "git format-patch". Some of the problems with this workflow are

1. The extra information typed into the email series is not archived in
the git repository, and therefore the emails have to be kept around.
2. If it is discovered between "git format-patch" and "git send-email"
that one of the patches needs a tweak, then the email series has to be
generated anew and all of the hand-edits need to be redone.
3. If a "v2" of the patch series has to be created, all of the
hand-edits have to be done again.
4. There is no place to store the "additional information" (the part
that comes in patch emails between the "---" and the diffstat) while
working on the patch series; it all has to be remembered or noted
somewhere else until just before sending off the patch emails.

Wouldn't it be nice if more of the information needed by "git format-patch" could be extracted from the git repository? For example:

* The subject and body of the cover letter could be written to the log
message of an annotated tag at the tip of the patch series.  "git
format-patch" could retrieve it if the "--cover-letter" option is used.
* The CC and ACK lists could be written as specially-formatted lines in
the annotated tag's log message, or perhaps in a git note.
* The "additional information" could be written to git notes.

I don't know whether git notes are the right vessel for such information. For example, are they carried along when a commit is rebased?

I don't have time to implement any of this, but perhaps somebody thinks this is a good idea and is inspired to work on it.

Michael
-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
Junio C Hamano· Aug 27, 2011, 18:46 UTC · re: Michael Haggerty · lore

Re: Idea: "git format-patch" should get more information out of git

Michael Haggerty <mhagger@alum.mit.edu> writes:
> 1. The extra information typed into the email series is not archived in
> the git repository, and therefore the emails have to be kept around.

The convention is to have information that is not meant to be kept around after '---', so...

> 2. If it is discovered between "git format-patch" and "git send-email"
> that one of the patches needs a tweak, then the email series has to be
> generated anew and all of the hand-edits need to be redone.

I actually often work this backwards. Tweak the patch files, apply them to commit that they are meant to be applied, retest and then finally send. There is no "generating anew" involved.

> 4. There is no place to store the "additional information" (the part
> that comes in patch emails between the "---" and the diffstat) while
> working on the patch series;...

I thought there was a RFC floating around to do this using notes and also teach it to "commit -e" a few months ago? I vaguelly recall that Peff and one of the J's were involved, so I am CC'ing them.

Michael J Gruber· Aug 27, 2011, 20:35 UTC · re: Junio C Hamano · lore

Re: Idea: "git format-patch" should get more information out of git

Junio C Hamano venit, vidit, dixit 27.08.2011 20:46:
Show 23 quoted lines
> Michael Haggerty <mhagger@alum.mit.edu> writes:
> 
>> 1. The extra information typed into the email series is not archived in
>> the git repository, and therefore the emails have to be kept around.
> 
> The convention is to have information that is not meant to be kept around
> after '---', so...
> 
>> 2. If it is discovered between "git format-patch" and "git send-email"
>> that one of the patches needs a tweak, then the email series has to be
>> generated anew and all of the hand-edits need to be redone.
> 
> I actually often work this backwards. Tweak the patch files, apply them to
> commit that they are meant to be applied, retest and then finally send.
> There is no "generating anew" involved.
> 
>> 4. There is no place to store the "additional information" (the part
>> that comes in patch emails between the "---" and the diffstat) while
>> working on the patch series;...
> 
> I thought there was a RFC floating around to do this using notes and also
> teach it to "commit -e" a few months ago? I vaguelly recall that Peff and
> one of the J's were involved, so I am CC'ing them.
Don't forget the MJ ;)
My current setup is:

A. I put the additional per commit info in notes. "git format-patch --notes" takes it from there and adds it where it belongs.

B. I put series title and cover letter text into a ref note. "git branch --notes" displays them as a branch note, and "git format-patch --cover --notes" fills in the cover letter for me.

A patch for A has been on the list, but Junio didn't like the additional "---" which I had for technical reasons, and I haven't had time to revisit that patch yet. (The code would have to store a flag about notes having been inserted or not.) Nobody has complained about the patches I've created with that form, though ;)

The stuff for B is part of my "ref notes" stuff which I haven't submitted so far. I've been using it for quite a while now, but am still unsure about some design aspects.

A is simply my branch "format-patch-notes".

B uses these: vob/virtual-objects vob/refrev-hash vob/branch-notes vob/format-patch-branch-note (Merge in this order. Last one is the most immature.)

Note that I rebase frequently these days because of overlapping work in these areas.

I think Jeff had an interesting patch allowing to edit the note along side (OK: at the bottom of) the commit message.

Cheers, Michael

Oh, repo, yes... Any of these:

git://github.com/gitigit/git.git git://gitorious.org/~mjg/git/mjg.git git://repo.or.cz/git/mjg.git

Michael Haggerty· Aug 28, 2011, 06:21 UTC · re: Michael J Gruber · lore

Re: Idea: "git format-patch" should get more information out of git

On 08/27/2011 10:35 PM, Michael J Gruber wrote:
Show 8 quoted lines
> My current setup is:
> 
> A. I put the additional per commit info in notes. "git format-patch
> --notes" takes it from there and adds it where it belongs.
> 
> B. I put series title and cover letter text into a ref note. "git branch
> --notes" displays them as a branch note, and "git format-patch --cover
> --notes" fills in the cover letter for me.
Very cool.  I will check this out.
Michael
-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
Junio C Hamano· Aug 28, 2011, 06:34 UTC · re: Junio C Hamano · lore

Re: Idea: "git format-patch" should get more information out of git

Junio C Hamano <gitster@pobox.com> writes:
Show 9 quoted lines
> Michael Haggerty <mhagger@alum.mit.edu> writes:
>
>> 4. There is no place to store the "additional information" (the part
>> that comes in patch emails between the "---" and the diffstat) while
>> working on the patch series;...
>
> I thought there was a RFC floating around to do this using notes and also
> teach it to "commit -e" a few months ago? I vaguelly recall that Peff and
> one of the J's were involved, so I am CC'ing them.
Also, when I prepare a commit to be sent with an additional piece of
information, I often write "---" and the additional message after my
S-o-b: line while preparing the commit log message. Unlike format-patch
that strips that off, commit keeps it, which is handy.
Jeff King· Aug 29, 2011, 18:55 UTC · re: Junio C Hamano · lore

Re: Idea: "git format-patch" should get more information out of git

On Sat, Aug 27, 2011 at 11:34:13PM -0700, Junio C Hamano wrote:
Show 16 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Michael Haggerty <mhagger@alum.mit.edu> writes:
> >
> >> 4. There is no place to store the "additional information" (the part
> >> that comes in patch emails between the "---" and the diffstat) while
> >> working on the patch series;...
> >
> > I thought there was a RFC floating around to do this using notes and also
> > teach it to "commit -e" a few months ago? I vaguelly recall that Peff and
> > one of the J's were involved, so I am CC'ing them.
> 
> Also, when I prepare a commit to be sent with an additional piece of
> information, I often write "---" and the additional message after my
> S-o-b: line while preparing the commit log message. Unlike format-patch
> that strips that off, commit keeps it, which is handy.

After playing around a bit with my earlier series, I made the realization (perhaps obvious to others :) ), that if you are in a pure-patch workflow, keeping the "---" in your commit message locally is much simpler. It follows the commit around through rebases automatically, it gets put into format-patch output automatically, and so forth.

The only real downside is that you can never tell git "don't show me the cover letter cruft". Which is probably OK for your own local patches. But the point of the "---" is that information should never make it into a repo, which means in any workflow that involves pulling actual git commits, it won't work (after reading Michael's response in another thread, though, I think he would be interested in a hybrid pull-or-apply-via-mail system).

-Peff
Michael J Gruber· Aug 30, 2011, 12:21 UTC · re: Jeff King · lore

Re: Idea: "git format-patch" should get more information out of git

Jeff King venit, vidit, dixit 29.08.2011 20:55:
Show 25 quoted lines
> On Sat, Aug 27, 2011 at 11:34:13PM -0700, Junio C Hamano wrote:
> 
>> Junio C Hamano <gitster@pobox.com> writes:
>>
>>> Michael Haggerty <mhagger@alum.mit.edu> writes:
>>>
>>>> 4. There is no place to store the "additional information" (the part
>>>> that comes in patch emails between the "---" and the diffstat) while
>>>> working on the patch series;...
>>>
>>> I thought there was a RFC floating around to do this using notes and also
>>> teach it to "commit -e" a few months ago? I vaguelly recall that Peff and
>>> one of the J's were involved, so I am CC'ing them.
>>
>> Also, when I prepare a commit to be sent with an additional piece of
>> information, I often write "---" and the additional message after my
>> S-o-b: line while preparing the commit log message. Unlike format-patch
>> that strips that off, commit keeps it, which is handy.
> 
> After playing around a bit with my earlier series, I made the
> realization (perhaps obvious to others :) ), that if you are in a
> pure-patch workflow, keeping the "---" in your commit message locally is
> much simpler. It follows the commit around through rebases
> automatically, it gets put into format-patch output automatically, and
> so forth.
I'm confused: Does format-patch keep it or strip it as you say above?

Anyways, notes survive rebase etc., and at format-patch time you can decide whether you want to include them or not (with my patch).

> The only real downside is that you can never tell git "don't show me the
> cover letter cruft". Which is probably OK for your own local patches.
You can with notes...
Show 5 quoted lines
> But the point of the "---" is that information should never make it into
> a repo, which means in any workflow that involves pulling actual git
> commits, it won't work (after reading Michael's response in another
> thread, though, I think he would be interested in a hybrid
> pull-or-apply-via-mail system).

The only difficulty would be sharing notes in a push-pull workflow (for lack of tool support, though we do have merge support).

Cheers, Michael

Jeff King· Aug 30, 2011, 15:22 UTC · re: Michael J Gruber · lore

Re: Idea: "git format-patch" should get more information out of git

On Tue, Aug 30, 2011 at 02:21:13PM +0200, Michael J Gruber wrote:
Show 8 quoted lines
> > After playing around a bit with my earlier series, I made the
> > realization (perhaps obvious to others :) ), that if you are in a
> > pure-patch workflow, keeping the "---" in your commit message locally is
> > much simpler. It follows the commit around through rebases
> > automatically, it gets put into format-patch output automatically, and
> > so forth.
> 
> I'm confused: Does format-patch keep it or strip it as you say above?

Format-patch will include the whole commit message, including "---" and what comes after. But an "am" recipient will strip it out of the commit message. IOW, you locally treat your "---" notes as part of the commit message, but in a pure-patch workflow, upstream treats it as something to be stripped out.

So there is asymmetry in what is in your commit versus what is in upstream's commit. But that's OK, because how you treat the commit (something to be tweaked and rebased, and then mailed out) and how upstream treats it (something to go into the long-term project history) are different.

But that breaks down if you ever want to share actual commit objects with upstream.

> Anyways, notes survive rebase etc.
You still have to manually configure:
  git config notes.rewriteRef refs/notes/commits
Perhaps that is something that should be changed.
> and at format-patch time you can decide whether you want to include
> them or not (with my patch).

Yeah. Don't get me wrong, I think putting "---" notes into git-notes is way more flexible. It's just that it's also more complex, and unnecessarily so for many use cases.

We have some patches to hide that complexity, but I was at one point wondering if all of the complexity was worthwhile. But as I mentioned above, if you want to build anything more complex than a pure-patch workflow, the simple solution breaks down. So it probably is worth pursuing.

-Peff
Michael J Gruber· Aug 30, 2011, 15:41 UTC · re: Jeff King · lore

Re: Idea: "git format-patch" should get more information out of git

Jeff King venit, vidit, dixit 30.08.2011 17:22:
> On Tue, Aug 30, 2011 at 02:21:13PM +0200, Michael J Gruber wrote:
...
Show 7 quoted lines
>> Anyways, notes survive rebase etc.
> 
> You still have to manually configure:
> 
>   git config notes.rewriteRef refs/notes/commits
> 
> Perhaps that is something that should be changed.
Right, I've done that quite a while ago and forgot.
Show 12 quoted lines
>> and at format-patch time you can decide whether you want to include
>> them or not (with my patch).
> 
> Yeah. Don't get me wrong, I think putting "---" notes into git-notes is
> way more flexible. It's just that it's also more complex, and
> unnecessarily so for many use cases.
> 
> We have some patches to hide that complexity, but I was at one point
> wondering if all of the complexity was worthwhile. But as I mentioned
> above, if you want to build anything more complex than a pure-patch
> workflow, the simple solution breaks down. So it probably is worth
> pursuing.

Reminds me of the ref namespace restructuring which could help sharing notes... Oh, lots to do before git 3.0!

Michael
Johan Herland· Aug 30, 2011, 17:39 UTC · re: Michael J Gruber · lore

Re: Idea: "git format-patch" should get more information out of git

On Tuesday 30. August 2011, Michael J Gruber wrote:
> Reminds me of the ref namespace restructuring which could help
> sharing notes... Oh, lots to do before git 3.0!

Indeed. I am very sorry to not have been able to properly follow up on that proposal (or a couple of other patches for that matter), but I'm currently in the middle of changing jobs and moving to a new apartment, which unfortunately leaves little if any spare time for git work. I don't believe I'll find the time to pick this up in the next month or two, so if someone wants to run with it, you are very welcome to do so.

I will try to participate in discussions where I'm CCed, but the rest of the mailing list is mostly a blur, these days.

...Johan
-- 
Johan Herland, <johan@herland.net>
www.herland.net
Michael Haggerty· Sep 1, 2011, 04:32 UTC · re: Johan Herland · lore

Re: Idea: "git format-patch" should get more information out of git

On 08/30/2011 07:39 PM, Johan Herland wrote:
Show 6 quoted lines
> On Tuesday 30. August 2011, Michael J Gruber wrote:
>> Reminds me of the ref namespace restructuring which could help
>> sharing notes... Oh, lots to do before git 3.0!
> 
> Indeed. I am very sorry to not have been able to properly follow up on 
> that proposal [...]

What "ref rename restructuring" are you talking about? Can you give a mailing list link?

(I've been working on refs lately and want to know whether your proposal is relevant to my work.)

Michael
-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
Michael J Gruber· Sep 1, 2011, 06:44 UTC · re: Michael Haggerty · lore

Re: Idea: "git format-patch" should get more information out of git

Michael Haggerty venit, vidit, dixit 01.09.2011 06:32:
Show 13 quoted lines
> On 08/30/2011 07:39 PM, Johan Herland wrote:
>> On Tuesday 30. August 2011, Michael J Gruber wrote:
>>> Reminds me of the ref namespace restructuring which could help
>>> sharing notes... Oh, lots to do before git 3.0!
>>
>> Indeed. I am very sorry to not have been able to properly follow up on 
>> that proposal [...]
> 
> What "ref rename restructuring" are you talking about?  Can you give a
> mailing list link?
> 
> (I've been working on refs lately and want to know whether your proposal
> is relevant to my work.)

Searching for "namespace" in the subject should give you all relevant proposals plus the new "namespaces" which might affect you also:

http://permalink.gmane.org/gmane.comp.version-control.git/140681 http://permalink.gmane.org/gmane.comp.version-control.git/165813 http://permalink.gmane.org/gmane.comp.version-control.git/165799

http://permalink.gmane.org/gmane.comp.version-control.git/176808
Michael

← back to recent threads