patchsubmodule: teach "foreach" command a --revision <tree-ish> option

>   $ git submodule foreach --revision v1.0 'git tag v1.0 $sha1'
>
> Previously, this would have required multiple steps:
>
>   $ git checkout v1.0
>   $ git submodule update
>   $ git submodule foreach 'git tag v1.0'

> @@ -379,6 +379,7 @@ Use -f if you really want to add it." >&2
>  cmd_foreach()
>  {
>  	# parse $args after "submodule ... foreach".
> +	revision=
>  	while test $# -ne 0
>  	do
>  		case "$1" in
> @@ -388,6 +389,11 @@ cmd_foreach()
>  		--recursive)
>  			recursive=1
>  			;;
> +		--revision)
> +			git rev-parse --quiet --verify "$2" >/dev/null || usage
> +			revision=$2

> +			shift
> +			;;
>  		-*)
>  			usage
>  			;;
> @@ -404,7 +410,17 @@ cmd_foreach()
>  	# command in the subshell (and a recursive call to this function)
>  	exec 3<&0
>  
> -	module_list |
> +	if test -n "$revision"
> +	then
> +		# make ls-tree output look like ls-files output
> +		git ls-tree -r $revision | grep '^160000 ' |
> +		while read mode unused sha1 sm_path
> +		do
> +			echo "$mode $sha1 0 $sm_path"
> +		done
> +	else
> +		module_list
> +	fi |

> Assuming that the above guess is correct (which is a huge
> assumption, given the lack of clarity in the description), I think
> the feature might make sense.  The example would have been a lot
> easier to follow if it were something like this:
>
>     $ git submodule foreach --revision v1.0 'git grep -e frotz $sha1'

> Junio C Hamano <gitster@pobox.com> writes:
>
>> Assuming that the above guess is correct (which is a huge
>> assumption, given the lack of clarity in the description), I think
>> the feature might make sense.  The example would have been a lot
>> easier to follow if it were something like this:
>>
>>     $ git submodule foreach --revision v1.0 'git grep -e frotz $sha1'
>
> Imagine you have a checkout of v2.0 of the superproject in your
> working tree, and you run "git submodule foreach --revision v1.0".
> Further imagine a submodule S that used to exist back when the
> superproject was at v1.0 no longer exists in the current codebase
> (hence there is no such submodule in the working tree).
>
> Shouldn't the above "foreach ... grep" still try to find 'frotz' in
> the submodule S that was bound to v1.0 of the superproject?
>
> Given that your patch does not touch the part of cmd_foreach where
> it decides which submodule to descend into, it still will base its
> decision solely on the set of submodules that are bound to and have
> been "git submodule init"ed in the version of the superproject that
> is _currently_ checked out, no?

> On Tue, Oct 9, 2012 at 2:12 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> Junio C Hamano <gitster@pobox.com> writes:
>>
>>> Assuming that the above guess is correct (which is a huge
>>> assumption, given the lack of clarity in the description), I think
>>> the feature might make sense.  The example would have been a lot
>>> easier to follow if it were something like this:
>>>
>>>     $ git submodule foreach --revision v1.0 'git grep -e frotz $sha1'
>>
>> Imagine you have a checkout of v2.0 of the superproject in your
>> working tree, and you run "git submodule foreach --revision v1.0".
>> Further imagine a submodule S that used to exist back when the
>> superproject was at v1.0 no longer exists in the current codebase
>> (hence there is no such submodule in the working tree).
>>
>> Shouldn't the above "foreach ... grep" still try to find 'frotz' in
>> the submodule S that was bound to v1.0 of the superproject?
>>
>> Given that your patch does not touch the part of cmd_foreach where
>> it decides which submodule to descend into, it still will base its
>> decision solely on the set of submodules that are bound to and have
>> been "git submodule init"ed in the version of the superproject that
>> is _currently_ checked out, no?
>
> That's a good observation. My use-case for this (poorly explained in
> ...
> As you previously stated, I need to improve the documentation that
> goes along with this patch, so I'll call-out this limitation. I'm not
> sure what else can be done. You can't descend into a submodule that
> isn't there.

> Jay Soffian <jaysoffian@gmail.com> writes:
> 
>> On Tue, Oct 9, 2012 at 2:12 AM, Junio C Hamano <gitster@pobox.com> wrote:
>>> Junio C Hamano <gitster@pobox.com> writes:
>>>
>>>> Assuming that the above guess is correct (which is a huge
>>>> assumption, given the lack of clarity in the description), I think
>>>> the feature might make sense.  The example would have been a lot
>>>> easier to follow if it were something like this:
>>>>
>>>>     $ git submodule foreach --revision v1.0 'git grep -e frotz $sha1'
>>>
>>> Imagine you have a checkout of v2.0 of the superproject in your
>>> working tree, and you run "git submodule foreach --revision v1.0".
>>> Further imagine a submodule S that used to exist back when the
>>> superproject was at v1.0 no longer exists in the current codebase
>>> (hence there is no such submodule in the working tree).
>>>
>>> Shouldn't the above "foreach ... grep" still try to find 'frotz' in
>>> the submodule S that was bound to v1.0 of the superproject?
>>>
>>> Given that your patch does not touch the part of cmd_foreach where
>>> it decides which submodule to descend into, it still will base its
>>> decision solely on the set of submodules that are bound to and have
>>> been "git submodule init"ed in the version of the superproject that
>>> is _currently_ checked out, no?
>>
>> That's a good observation. My use-case for this (poorly explained in
>> ...
>> As you previously stated, I need to improve the documentation that
>> goes along with this patch, so I'll call-out this limitation. I'm not
>> sure what else can be done. You can't descend into a submodule that
>> isn't there.
> 
> As recent "submodule rm" work by Jens indicates, since 501770e (Move
> git-dir for submodules, 2011-08-15), you should be able to peek into
> submodules that have been "git submodule init"ed but do not exist in
> the current checkout of the superproject.
> 
> I think the right approach to implement this "recurse foreach in the
> superproject tree that is not checkout out to the working tree"
> feature should be:
> 
>  - Advertise it so that it is crystal clear that the command run by
>    "foreach" may have to run in a bare repository of submodule to
>    look at submodule's commit bound to the historical tree of the
>    superproject;

>  - Locate submodule's $GIT_DIR in $GIT_DIR/modules/$sm_name of the
>    superproject that corresponds to the submodule found in the
>    historical tree in the superproject (or if it is the same
>    repository as that is currently checked out, use $sm_path/.git),
>    and error out when it is not available.

> An implementation that works only when all the submodules necessary
> in the historical tree in the superproject are still in the current
> checkout of the superproject may be fine as a quick throw-away hack,
> and it may even be acceptable as a good first step towards the real
> feature, but at least it needs to be protected by an error checking
> upfront (perhaps running "diff-tree -r" between the index and the
> historical tree to make sure there are no removed submodules that
> existed in the historical tree), if it does not bother to check with
> $GIT_DIR/modules/$sm_name in the superproject.
> 
> Jens, anything I missed?

> Nothing I can think of right now, the above is a pretty good summary.
> My gut feeling is that having "git submodule foreach --revision ..."
> recurse through submodules whose work trees are out of sync is pretty
> fragile and could easily lead to inconsistencies. So I tend to think
> adding a custom script to the release process Jay uses which does the
> tagging itself might be a better solution here. Opinions?

> Am 09.10.2012 20:24, schrieb Junio C Hamano:
> ...
>> I think the right approach to implement this "recurse foreach in the
>> superproject tree that is not checkout out to the working tree"
>> feature should be:
>> 
>>  - Advertise it so that it is crystal clear that the command run by
>>    "foreach" may have to run in a bare repository of submodule to
>>    look at submodule's commit bound to the historical tree of the
>>    superproject;
>
> I think we should even try to enforce that the user shouldn't use
> the work tree at all (although at the moment I can't come up with
> an idea how we could do that), as the work tree *will* be out of
> sync almost always when you need this option.

>>  - Locate submodule's $GIT_DIR in $GIT_DIR/modules/$sm_name of the
>>    superproject that corresponds to the submodule found in the
>>    historical tree in the superproject (or if it is the same
>>    repository as that is currently checked out, use $sm_path/.git),
>>    and error out when it is not available.
>
> Looking in $GIT_DIR/modules/$sm_name could make sense to tag even
> those submodules which aren't currently populated. But IIRC the
> tags in such repositories could not be pushed using current git
> even when you use the "--recurse-submodules" option because that
> only honors populated submodules. So for now it would suffice to
> only recurse into populated submodules.

>> Jens, anything I missed?
>
> Nothing I can think of right now, the above is a pretty good summary.
> My gut feeling is that having "git submodule foreach --revision ..."
> recurse through submodules whose work trees are out of sync is pretty
> fragile and could easily lead to inconsistencies. So I tend to think
> adding a custom script to the release process Jay uses which does the
> tagging itself might be a better solution here. Opinions?