threads / patch / 32906

patchDocumentation: filter-branch env-filter example

Subject: [PATCH] Documentation: filter-branch env-filter example

## tl;dr

5 messages between Feb 14, 2013 and Feb 14, 2013. Diffs are folded; open one to read it.

replies: 4people: 3as markdown or json

Tadeusz Andrzej Kadłubowski· Feb 14, 2013, 19:34 UTC · lore
filter-branch --env-filter example that shows how to change the email address
in all commits by a certain developer.
---
 Documentation/git-filter-branch.txt | 13 +++++++++++++
 1 file changed, 13 insertions(+)
Show changes to Documentation/git-filter-branch.txt +13 −0
diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt
index dfd12c9..2664cec 100644
--- a/Documentation/git-filter-branch.txt
+++ b/Documentation/git-filter-branch.txt
@@ -329,6 +329,19 @@ git filter-branch --msg-filter '
 ' HEAD~10..HEAD
 --------------------------------------------------------

+You can modify committer/author personal information using `--env-filter`.
+For example, to update some developer's email address use this command:
+
+--------------------------------------------------------
+git filter-branch --env-filter '
+	if [ $GIT_AUTHOR_EMAIL = john@old.example.com ]
+	then
+		GIT_AUTHOR_EMAIL=john@new.example.com
+	fi
+	export GIT_AUTHOR_EMAIL
+' -- --all
+--------------------------------------------------------
+
 To restrict rewriting to only part of the history, specify a revision
 range in addition to the new branch name.  The new branch name will
 point to the top-most revision that a 'git rev-list' of this range
-- 
1.7.11.7
Junio C Hamano· Feb 14, 2013, 20:29 UTC · re: Tadeusz Andrzej Kadłubowski · lore

Re: [PATCH] Documentation: filter-branch env-filter example

Tadeusz Andrzej Kadłubowski  <yess@hell.org.pl> writes:
> filter-branch --env-filter example that shows how to change the email address
> in all commits by a certain developer.
> ---
Thanks.  Sign-off?
Show 17 quoted lines
>  Documentation/git-filter-branch.txt | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
>
> diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt
> index dfd12c9..2664cec 100644
> --- a/Documentation/git-filter-branch.txt
> +++ b/Documentation/git-filter-branch.txt
> @@ -329,6 +329,19 @@ git filter-branch --msg-filter '
>  ' HEAD~10..HEAD
>  --------------------------------------------------------
>
> +You can modify committer/author personal information using `--env-filter`.
> +For example, to update some developer's email address use this command:
> +
> +--------------------------------------------------------
> +git filter-branch --env-filter '
> +	if [ $GIT_AUTHOR_EMAIL = john@old.example.com ]
Quote the variable in double-quotes, like this:
	if [ "$GIT_AUTHOR_EMAIL" = john@old.example.com ]

Otherwise the comparison will break, if the e-mail part had a whitespace in it, or if it were empty, which is an example of a more likely situation where you would want to fix commits using a procedure like this, no?

But more on the example later...
Show 7 quoted lines
> +	then
> +		GIT_AUTHOR_EMAIL=john@new.example.com
> +	fi
> +	export GIT_AUTHOR_EMAIL
> +' -- --all
> +--------------------------------------------------------
> +

I do not think an illustration of env-filter is a bad addition per-se, but the sample scenario is not a realistic one. No sane project should be encouraged to rewrite their entire history every time one of the contributors change his e-mail address. That is what the mailmap mechanism is for.

The only scenario that justifies use of the given sample I can think of is to rewrite the author and committer in an unpublished project because you noticed that you forgot to set user.name and user.email up before you created these commits correctly.

Taking all of the above, the added text may look more like this, I think:

	The `--env-filter` can be used to modify committer and/or
	author identity.  For example, if you found out that your
	commits have wrong identity of yours due to misconfigured
	user.email, you can make correction, before publishing the
	project, like this:
	--------------------------------------------------------
        git filter-branch --env-filter '
        	if test "$GIT_AUTHOR_EMAIL" = "root@localhost"
                then
			GIT_AUTHOR_EMAIL=yess@example.com
			export GIT_AUTHOR_EMAIL
		fi
        	if test "$GIT_COMMITTER_EMAIL" = "root@localhost"
                then
			GIT_COMMITTER_EMAIL=yess@example.com
			export GIT_COMMITTER_EMAIL
		fi
	' -- --all
	--------------------------------------------------------

By the way, I left the "export" in; "git filter-branch --help" explicitly says that you need to re-export it. But I am not sure if they are necessary, especially after 3c730fab2cae (filter-branch: use git-sh-setup's ident parsing functions, 2012-10-18) by Peff, which added extra "export" to make sure all six identity variables are exported. After applying the above rewrite, we may want to do the following as a separate, follow-up patch.

 Documentation/git-filter-branch.txt | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
Show changes to Documentation/git-filter-branch.txt +1 −4
diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt
index 8ebe999..066548e 100644
--- a/Documentation/git-filter-branch.txt
+++ b/Documentation/git-filter-branch.txt
@@ -83,8 +83,7 @@ OPTIONS
 	This filter may be used if you only need to modify the environment
 	in which the commit will be performed.  Specifically, you might
 	want to rewrite the author/committer name/email/time environment
-	variables (see linkgit:git-commit-tree[1] for details).  Do not forget
-	to re-export the variables.
+	variables (see linkgit:git-commit-tree[1] for details).
 
 --tree-filter <command>::
 	This is the filter for rewriting the tree and its contents.
@@ -340,12 +339,10 @@ git filter-branch --env-filter '
 	if test "$GIT_AUTHOR_EMAIL" = "root@localhost"
 	then
 		GIT_AUTHOR_EMAIL=yess@example.com
-		export GIT_AUTHOR_EMAIL
 	fi
 	if test "$GIT_COMMITTER_EMAIL" = "root@localhost"
 	then
 		GIT_COMMITTER_EMAIL=yess@example.com
-		export GIT_COMMITTER_EMAIL
 	fi
 ' -- --all
 --------------------------------------------------------
Jeff King· Feb 14, 2013, 21:09 UTC · re: Junio C Hamano · lore

Re: [PATCH] Documentation: filter-branch env-filter example

On Thu, Feb 14, 2013 at 12:29:36PM -0800, Junio C Hamano wrote:
Show 8 quoted lines
> Quote the variable in double-quotes, like this:
> 
> 	if [ "$GIT_AUTHOR_EMAIL" = john@old.example.com ]
> 
> Otherwise the comparison will break, if the e-mail part had a
> whitespace in it, or if it were empty, which is an example of a more
> likely situation where you would want to fix commits using a
> procedure like this, no?

Yeah, definitely. If you are cleaning up a broken ident that cannot be parsed, the failure mode is to put the whole author line in $GIT_AUTHOR_EMAIL, which would almost certainly include spaces.

Show 23 quoted lines
> Taking all of the above, the added text may look more like this, I
> think:
> 
> 	The `--env-filter` can be used to modify committer and/or
> 	author identity.  For example, if you found out that your
> 	commits have wrong identity of yours due to misconfigured
> 	user.email, you can make correction, before publishing the
> 	project, like this:
> 
> 	--------------------------------------------------------
>         git filter-branch --env-filter '
>         	if test "$GIT_AUTHOR_EMAIL" = "root@localhost"
>                 then
> 			GIT_AUTHOR_EMAIL=yess@example.com
> 			export GIT_AUTHOR_EMAIL
> 		fi
>         	if test "$GIT_COMMITTER_EMAIL" = "root@localhost"
>                 then
> 			GIT_COMMITTER_EMAIL=yess@example.com
> 			export GIT_COMMITTER_EMAIL
> 		fi
> 	' -- --all
> 	--------------------------------------------------------

That looks better, though there are a few English nits; here's my edited version:

        The `--env-filter` option can be used to modify committer and/or
	author identity.  For example, if you found out that your
	commits have the wrong identity due to a misconfigured
        user.email, you can make a correction, before publishing the
	project, like this:
Show 7 quoted lines
> By the way, I left the "export" in; "git filter-branch --help"
> explicitly says that you need to re-export it.  But I am not sure if
> they are necessary, especially after 3c730fab2cae (filter-branch:
> use git-sh-setup's ident parsing functions, 2012-10-18) by Peff,
> which added extra "export" to make sure all six identity variables
> are exported.  After applying the above rewrite, we may want to do
> the following as a separate, follow-up patch.

I think it has always been the case that we export them after setting them; just look at the preimage from 3c730fab, and you can see exports there.

I think the advice in the documentation about re-exporting is because some versions of the bourne shell will not reliably pass the new version of the variable when you do this:

  VAR=old
  export VAR
  VAR=new
  some_subprocess ;# we see $VAR=old here!

I do not recall ever running across such a shell myself, but rather hearing about it third-hand in a portability guide somewhere. Apple's shell documentation seems to indicate that /bin/sh in older versions of OS X had this behavior:

  https://developer.apple.com/library/mac/documentation/opensource/conceptual/shellscripting/shell_scripts/shell_scripts.html#//apple_ref/doc/uid/TP40004268-CH237-SW11

which makes me think that BSD ash may behave that way. It is certainly not necessary to re-export under bash or dash. I shudder to think what horrible, 1980's-era behavior is codified in Solaris /bin/sh.

We could explicitly re-export all of the ident variables preemptively before calling commit-tree, just to save the user the hassle of remembering to do so. It would be a no-op on sane shells, and I doubt the runtime cost is very high. I suppose it would break somebody who explicitly did:

  unset GIT_COMMITTER_NAME ;# use the value from user.name
in their env-filter, but that seems like a pretty unlikely corner case.
-Peff
Jeff King· Feb 14, 2013, 21:24 UTC · re: Jeff King · lore

Re: [PATCH] Documentation: filter-branch env-filter example

On Thu, Feb 14, 2013 at 04:09:10PM -0500, Jeff King wrote:
Show 11 quoted lines
> I think the advice in the documentation about re-exporting is because
> some versions of the bourne shell will not reliably pass the new version
> of the variable when you do this:
> 
>   VAR=old
>   export VAR
>   VAR=new
>   some_subprocess ;# we see $VAR=old here!
> 
> I do not recall ever running across such a shell myself, but rather
> hearing about it third-hand in a portability guide somewhere.

The closest I could find in the autoconf shell guidelines[1] is that the automagic export marking for incoming variables is not always accurate:

	export
            The builtin export dubs a shell variable environment
            variable. Each update of exported variables corresponds to
            an update of the environment variables.  Conversely, each
            environment variable received by the shell when it is
            launched should be imported as a shell variable marked as
            exported.
            Alas, many shells, such as Solaris 2.5, IRIX 6.3, IRIX 5.2,
            AIX 4.1.5, and Digital UNIX 4.0, forget to export the
            environment variables they receive. As a result, two
            variables coexist: the environment variable and the shell
            variable. The following code demonstrates this failure:
	    #! /bin/sh
	    echo $FOO
	    FOO=bar
	    echo $FOO
	    exec /bin/sh $0
            when run with `FOO=foo' in the environment, these shells
            will print alternately `foo' and `bar', although it should
            only print `foo' and then a sequence of `bar's.
            Therefore you should export again each environment variable
            that you update.
I don't know what the behavior would be on such shells of:
	#!/bin/sh
	echo $FOO
	FOO=bar
	export FOO
	echo $FOO
	exec /bin/sh $0

I.e., would the "export" correctly reconcile the local and environment copies of the variable, or are they forever broken? I don't have such a system to test on. But that would more closely match what we are doing.

-Peff
[1] https://www.gnu.org/software/autoconf/manual/autoconf.html#Limitations-of-Builtins
Junio C Hamano· Feb 14, 2013, 21:51 UTC · re: Jeff King · lore

Re: [PATCH] Documentation: filter-branch env-filter example

Jeff King <peff@peff.net> writes:
> I think it has always been the case that we export them after setting
> them; just look at the preimage from 3c730fab, and you can see exports
> there.

Yeah, I think the only difference is a broken commit case where sed expression did not find what it was looking for, in which case we do not do the export.

> I think the advice in the documentation about re-exporting is because
> some versions of the bourne shell will not reliably pass the new version
> of the variable ...

Ahh, old and painful memory of Solaris days comes back to me. OK, let's keep the export then.

Thanks.

← back to recent threads