threads / discuss / 18021

serious problem with `git format-patch' & `git am'

Subject: serious problem with `git format-patch' & `git am'

## tl;dr

8 messages between Feb 25, 2009 and Feb 27, 2009.

replies: 7people: 5as markdown or json

Werner LEMBERG· Feb 25, 2009, 22:03 UTC · lore
[both version 1.6.0.2 and git from 2009-01-24]
Folks,

I'm going to convert the FreeType CVS repository to git, using Keith Packard's `parsecvs'. It sometimes happens that there are git entries titled

  *** empty log message ***

I want to massage the git repository before publishing it, replacing those entries with something more meaningful.

The last time I tested this (using git 1.5.something, I no longer remember the exact version) I did the following:

  . git format-patch <start>..HEAD
    git reset --hard <start>
  . Edit the `Subject:' field in the created *.patch files where
    necessary.
  . git am --whitespace=nowarn *.patch

Unfortunately, this no longer works as before, and since I can't find a hint in the docs about this change I wonder whether it is a bug.

With git 1.5.something, if the first paragraph of the commit message (as output by parsecvs) looks like this

   foo foo foo foo
   bar bar bar bar
   baz baz baz baz

it is emitted exactly as-is (after `git format-patch' & `git am'); gitk shows `foo foo foo foo' as the first line. However, git version 1.6.0.2 and the current git both convert newlines to spaces in the first paragraph! I now get

  foo foo foo foo bar bar bar bar baz baz baz baz

as the beginning of the commit message which is VERY bad since CVS-style entries often have a long first paragraph, causing ugly-looking, overlong lines.

In case this is an intended change I ask you urgently to provide an option to get back the old behaviour of `git format-patch' & `git am'.

    Werner
Johannes Schindelin· Feb 26, 2009, 10:42 UTC · re: Werner LEMBERG · lore

Re: serious problem with `git format-patch' & `git am'

Hi,
[welcome on the Git list, Werner!]
On Wed, 25 Feb 2009, Werner LEMBERG wrote:
Show 49 quoted lines
> 
> [both version 1.6.0.2 and git from 2009-01-24]
> 
> Folks,
> 
> 
> I'm going to convert the FreeType CVS repository to git, using Keith
> Packard's `parsecvs'.  It sometimes happens that there are git entries
> titled
> 
>   *** empty log message ***
> 
> I want to massage the git repository before publishing it, replacing
> those entries with something more meaningful.
> 
> The last time I tested this (using git 1.5.something, I no longer
> remember the exact version) I did the following:
> 
>   . git format-patch <start>..HEAD
>     git reset --hard <start>
> 
>   . Edit the `Subject:' field in the created *.patch files where
>     necessary.
> 
>   . git am --whitespace=nowarn *.patch
> 
> Unfortunately, this no longer works as before, and since I can't find
> a hint in the docs about this change I wonder whether it is a bug.
> 
> With git 1.5.something, if the first paragraph of the commit message
> (as output by parsecvs) looks like this
> 
>    foo foo foo foo
>    bar bar bar bar
>    baz baz baz baz
> 
> it is emitted exactly as-is (after `git format-patch' & `git am');
> gitk shows `foo foo foo foo' as the first line.  However, git version
> 1.6.0.2 and the current git both convert newlines to spaces in the
> first paragraph!  I now get
> 
>   foo foo foo foo bar bar bar bar baz baz baz baz
> 
> as the beginning of the commit message which is VERY bad since
> CVS-style entries often have a long first paragraph, causing
> ugly-looking, overlong lines.
> 
> In case this is an intended change I ask you urgently to provide an
> option to get back the old behaviour of `git format-patch' & `git am'.

It is an intended change, for the _opposite_ case: some people thrive on overly-long subject lines, and the change was supposed to help them.

I was not sure if it would be a good change back then, but I did not have anything concrete I could present as an example where the behavior hurts.

In your case, I suggest using something different, though. Either use "rebase -i" and mark the commits you want to massage as "edit", or use "fast-export > dump", edit "dump" by hand, and then reimport the whole thing using fast-import.

IMHO your use case is not a good candidate for format-patch, as that program was really meant for patch submission (even if it also serves other purposes pretty well).

Ciao, Dscho

Brandon Casey· Feb 26, 2009, 18:00 UTC · re: Werner LEMBERG · lore

Re: serious problem with `git format-patch' & `git am'

Werner LEMBERG wrote:
Show 48 quoted lines
> [both version 1.6.0.2 and git from 2009-01-24]
> 
> Folks,
> 
> 
> I'm going to convert the FreeType CVS repository to git, using Keith
> Packard's `parsecvs'.  It sometimes happens that there are git entries
> titled
> 
>   *** empty log message ***
> 
> I want to massage the git repository before publishing it, replacing
> those entries with something more meaningful.
> 
> The last time I tested this (using git 1.5.something, I no longer
> remember the exact version) I did the following:
> 
>   . git format-patch <start>..HEAD
>     git reset --hard <start>
> 
>   . Edit the `Subject:' field in the created *.patch files where
>     necessary.
> 
>   . git am --whitespace=nowarn *.patch
> 
> Unfortunately, this no longer works as before, and since I can't find
> a hint in the docs about this change I wonder whether it is a bug.
> 
> With git 1.5.something, if the first paragraph of the commit message
> (as output by parsecvs) looks like this
> 
>    foo foo foo foo
>    bar bar bar bar
>    baz baz baz baz
> 
> it is emitted exactly as-is (after `git format-patch' & `git am');
> gitk shows `foo foo foo foo' as the first line.  However, git version
> 1.6.0.2 and the current git both convert newlines to spaces in the
> first paragraph!  I now get
> 
>   foo foo foo foo bar bar bar bar baz baz baz baz
> 
> as the beginning of the commit message which is VERY bad since
> CVS-style entries often have a long first paragraph, causing
> ugly-looking, overlong lines.
> 
> In case this is an intended change I ask you urgently to provide an
> option to get back the old behaviour of `git format-patch' & `git am'.

I was not aware of this behavior, but it looks like the '-k' option for both format-patch and git-am is what you want to use. It will prevent mailinfo from munging the commit message.

-brandon
Jakub Narebski· Feb 27, 2009, 01:11 UTC · re: Brandon Casey · lore

Re: serious problem with `git format-patch' & `git am'

Brandon Casey <casey@nrlssc.navy.mil> writes:
Show 5 quoted lines
> Werner LEMBERG wrote:
> > [both version 1.6.0.2 and git from 2009-01-24]
> >
> > I'm going to convert the FreeType CVS repository to git, using Keith
> > Packard's `parsecvs'.
>From what I understand it is not maintained, is not incremental, and

requires access to *,v files. Why not use more modern and actively developed cvs2git tool from cvs2svn project[1]?

[1] http://cvs2svn.tigris.org
[2] http://git.or.cz/gitwiki/InterfacesFrontendsAndTools
    "Interaction with other Revision Control Systems" section
Show 18 quoted lines
> >  It sometimes happens that there are git entries
> > titled
> > 
> >   *** empty log message ***
> > 
> > I want to massage the git repository before publishing it, replacing
> > those entries with something more meaningful.
> > 
> > The last time I tested this (using git 1.5.something, I no longer
> > remember the exact version) I did the following:
> > 
> >   . git format-patch <start>..HEAD
> >     git reset --hard <start>
> > 
> >   . Edit the `Subject:' field in the created *.patch files where
> >     necessary.
> > 
> >   . git am --whitespace=nowarn *.patch
You can use the following alternate solutions:
 * git rebase --interactive (perhaps with --root option if your git
   supports it, or cherry-pick root commit by hand if it does not),
   changing 'pick' to 'edit' for patches (commits) that you want to
   modify commit message.
 * if you can automate the changes, i.e. if you can script it, the
   best solution would be to use git-filter-branch instead; or use
   fast-export / fast-import pair with your filter in the middle.
Show 21 quoted lines
> > 
> > Unfortunately, this no longer works as before, and since I can't find
> > a hint in the docs about this change I wonder whether it is a bug.
> > 
> > With git 1.5.something, if the first paragraph of the commit message
> > (as output by parsecvs) looks like this
> > 
> >    foo foo foo foo
> >    bar bar bar bar
> >    baz baz baz baz
> > 
> > it is emitted exactly as-is (after `git format-patch' & `git am');
> > gitk shows `foo foo foo foo' as the first line.  However, git version
> > 1.6.0.2 and the current git both convert newlines to spaces in the
> > first paragraph!  I now get
> > 
> >   foo foo foo foo bar bar bar bar baz baz baz baz
> > 
> > as the beginning of the commit message which is VERY bad since
> > CVS-style entries often have a long first paragraph, causing
> > ugly-looking, overlong lines.

This is caused by the fact that git-am assumes that subject line goes as first line of commit message, with empty line separating it from the rest of commit message which comes from the body of email.

It used to preserve line breaking in Subject: line, if it was broken into lines using continuation style for email headers, but that was wrong solution in the intended use case (applying patches from email, patches conforming to git commit message conventions), as Subject: line could have been broken into multiple lines by the _transport_ layer, and not intended to be broken.

Show 7 quoted lines
> > 
> > In case this is an intended change I ask you urgently to provide an
> > option to get back the old behaviour of `git format-patch' & `git am'.
> 
> I was not aware of this behavior, but it looks like the '-k' option for
> both format-patch and git-am is what you want to use.  It will prevent
> mailinfo from munging the commit message.

I think you want undocumented but for the git-am usage (e.g. 'git am -h') --rebasing option to git-am, which makes git-am preserve Subject: line formatting, _including_ line breaking (I think that -k/--keep does preserve subject, but does not preserve line breaking; I haven't checked this, though). Note that in this case you would have to make git-format-patch to not output subject prefix ([PATCH]).

See how git-rebase.sh does it.
-- 
Jakub Narebski
Poland
ShadeHawk on #git
Junio C Hamano· Feb 27, 2009, 02:10 UTC · re: Jakub Narebski · lore

Re: serious problem with `git format-patch' & `git am'

Jakub Narebski <jnareb@gmail.com> writes:
> I think you want undocumented but for the git-am usage (e.g. 'git am
> -h') --rebasing option to git-am, which makes git-am preserve Subject:
> line formatting,...

I have to warn that that one requires you to have proper commit object to begin with. IOW, the first line "From [0-9a-f]{40} date" must record an existing object name.

As you said, "am" is for applying e-mailed patches and filter-branch would be the best fit for fixing imported history, but whichever tool is used, I think it is a mistake to try preserving a "one-line-per-paragraph" format the original poster seems to have, i.e.:

   foo foo foo foo
   bar bar bar bar
   baz baz baz baz

Yes, the low-level could create a commit object with a message like that, but all the Porcelains rely on "paragraphs are separated with a blank line" format. So no matter how the conversion is done, I think it would be a good idea to turn the above into:

   foo foo foo foo
   bar bar bar bar
   baz baz baz baz
if the user wants to keep the resulting history any useful.
To do it with "format-patch | am", you would format it like:
	Subject: foo foo foo foo
        From: author name <author.name@domain>
        Date: author date
        bar bar bar bar
        baz baz baz baz
        
Brandon Casey· Feb 27, 2009, 15:47 UTC · re: Junio C Hamano · lore

Re: serious problem with `git format-patch' & `git am'

Junio,
I think you are the author of git-am.sh.
What does this line do?
git-am.sh line 375:
    case "$keep_subject" in -k)  SUBJECT="[PATCH] $SUBJECT" ;; esac

It appears to do nothing. It has existed since the beginning of git-am.sh, and there has never been another reference to $keep_subject in this script. Is it possible that $keep_subject is supposed to be $keep?

-brandon
Junio C Hamano· Feb 27, 2009, 17:53 UTC · re: Brandon Casey · lore

Re: serious problem with `git format-patch' & `git am'

Brandon Casey <casey@nrlssc.navy.mil> writes:
Show 9 quoted lines
> What does this line do?
>
> git-am.sh line 375:
>
>     case "$keep_subject" in -k)  SUBJECT="[PATCH] $SUBJECT" ;; esac
>
> It appears to do nothing.  It has existed since the beginning of git-am.sh,
> and there has never been another reference to $keep_subject in this script.
> Is it possible that $keep_subject is supposed to be $keep?

I do not think it is doing anything, and I do not think it makes sense to "fix" it to allow adding "[PATCH] " in front either. I'd vote for removing the entire thing.

The $keep_subject variable and "add [PATCH] only to later strip" logic was borrowed from git-applymbox and git-applypatch pair, see 6bff6a6 (Teach applymbox to keep the Subject: line., 2005-08-16) especially for the change that affects tools/git-applypatch in the commit. I think there was a reason that -k refrained from adding the prefix in git-applypatch but the option to git-am needed to add the prefix which seems inconsistent but I do not recall the details. Maybe the list archive may know.

Werner LEMBERG· Feb 27, 2009, 09:16 UTC · re: Brandon Casey · lore

Re: serious problem with `git format-patch' & `git am'

> I was not aware of this behavior, but it looks like the '-k' option
> for both format-patch and git-am is what you want to use.  It will
> prevent mailinfo from munging the commit message.
Tried it already.  It doesn't help.
    Werner

← back to recent threads