threads / discuss / 37679

let git grep consider sub projects

Subject: let git grep consider sub projects

## tl;dr

7 messages between Oct 7, 2014 and Oct 8, 2014.

replies: 6people: 4as markdown or json

Olaf Hering· Oct 7, 2014, 08:29 UTC · lore
How can I teach 'git grep $string' to also consider subprojects?

The build system of xen.git clones 4 other trees into a directory in its source tree during build. It would be nice if my 'git grep' searches also in these cloned subdirs. Somehow there must be some knowledge because 'git clean -dfx' leaves them alone, while 'git clean -dffx' wipes everything.

Olaf
PS: Sometimes I miss a 'git clean -dfx --also-sub-repos' useful to
really clean everything before starting over.
Junio C Hamano· Oct 7, 2014, 18:25 UTC · re: Olaf Hering · lore

Re: let git grep consider sub projects

Olaf Hering <olaf@aepfle.de> writes:
Show 12 quoted lines
> How can I teach 'git grep $string' to also consider subprojects?
>
> The build system of xen.git clones 4 other trees into a directory in its
> source tree during build. It would be nice if my 'git grep' searches
> also in these cloned subdirs. Somehow there must be some knowledge
> because 'git clean -dfx' leaves them alone, while 'git clean -dffx'
> wipes everything.
>
> Olaf
>
> PS: Sometimes I miss a 'git clean -dfx --also-sub-repos' useful to
> really clean everything before starting over.

Is "submodule foreach" under-advertised or with less than adequate features?

Mikael Magnusson· Oct 7, 2014, 19:12 UTC · re: Junio C Hamano · lore

Re: let git grep consider sub projects

On 7 October 2014 20:25, Junio C Hamano <gitster@pobox.com> wrote:
Show 17 quoted lines
> Olaf Hering <olaf@aepfle.de> writes:
>
>> How can I teach 'git grep $string' to also consider subprojects?
>>
>> The build system of xen.git clones 4 other trees into a directory in its
>> source tree during build. It would be nice if my 'git grep' searches
>> also in these cloned subdirs. Somehow there must be some knowledge
>> because 'git clean -dfx' leaves them alone, while 'git clean -dffx'
>> wipes everything.
>>
>> Olaf
>>
>> PS: Sometimes I miss a 'git clean -dfx --also-sub-repos' useful to
>> really clean everything before starting over.
>
> Is "submodule foreach" under-advertised or with less than adequate
> features?

It sounds like in these use cases, you would want the commands to run on all the submodules but also in the parent repo, am I wrong in thinking that git submodule foreach does only the former part? So you would either need to make a wrapper thing yourself or run the command twice.

In the first case with the git grep, I can also imagine that with some nontrivial patterns, having to quote the metacharacters not only once, but twice, can be a significant annoyance. Eg, first protect it from git submodule foreach parsing it, and then from the shell running the individual commands.

-- 
Mikael Magnusson
Jens Lehmann· Oct 8, 2014, 06:48 UTC · re: Mikael Magnusson · lore

Re: let git grep consider sub projects

Am 07.10.2014 um 21:12 schrieb Mikael Magnusson:
Show 24 quoted lines
> On 7 October 2014 20:25, Junio C Hamano <gitster@pobox.com> wrote:
>> Olaf Hering <olaf@aepfle.de> writes:
>>
>>> How can I teach 'git grep $string' to also consider subprojects?
>>>
>>> The build system of xen.git clones 4 other trees into a directory in its
>>> source tree during build. It would be nice if my 'git grep' searches
>>> also in these cloned subdirs. Somehow there must be some knowledge
>>> because 'git clean -dfx' leaves them alone, while 'git clean -dffx'
>>> wipes everything.
>>>
>>> Olaf
>>>
>>> PS: Sometimes I miss a 'git clean -dfx --also-sub-repos' useful to
>>> really clean everything before starting over.
>>
>> Is "submodule foreach" under-advertised or with less than adequate
>> features?
>
> It sounds like in these use cases, you would want the commands to run
> on all the submodules but also in the parent repo, am I wrong in
> thinking that git submodule foreach does only the former part? So you
> would either need to make a wrapper thing yourself or run the command
> twice.

Yep, the workaround for a command "git foo" that hasn't learned the --recurse-submodules option yet is

    git foo && git submodule foreach --recursive git foo
Show 5 quoted lines
> In the first case with the git grep, I can also imagine that with some
> nontrivial patterns, having to quote the metacharacters not only once,
> but twice, can be a significant annoyance. Eg, first protect it from
> git submodule foreach parsing it, and then from the shell running the
> individual commands.

The long term goal is to teach git grep and git clean - and others - the --recurse-submodules option. I'd be happy to help if someone wants to work on that, as I'm busy working on another topic.

Olaf Hering· Oct 8, 2014, 06:59 UTC · re: Jens Lehmann · lore

Re: let git grep consider sub projects

On Wed, Oct 08, Jens Lehmann wrote:
>    git foo && git submodule foreach --recursive git foo

Looks like a submodule is yet another thing, or I have to learn what a submodule is:

olaf@bax:~/xen $ find . -name .git ./.git ./tools/qemu-xen-dir-remote/.git ./tools/qemu-xen-traditional-dir-remote/.git ./tools/firmware/seabios-dir-remote/.git olaf@bax:~/xen $ git submodule foreach --recursive git show olaf@bax:~/xen $

xen does essentially 'git clone $url1 dir/url1-dir;git clone $url2 dir/url2-dir'. So the top level .git does not really know about dir/*-dir.

Olaf
Jens Lehmann· Oct 8, 2014, 07:24 UTC · re: Olaf Hering · lore

Re: let git grep consider sub projects

Am 08.10.2014 um 08:59 schrieb Olaf Hering:
Show 18 quoted lines
> On Wed, Oct 08, Jens Lehmann wrote:
>
>>     git foo && git submodule foreach --recursive git foo
>
> Looks like a submodule is yet another thing, or I have to learn what a
> submodule is:
>
> olaf@bax:~/xen $ find . -name .git
> ./.git
> ./tools/qemu-xen-dir-remote/.git
> ./tools/qemu-xen-traditional-dir-remote/.git
> ./tools/firmware/seabios-dir-remote/.git
> olaf@bax:~/xen $ git submodule foreach --recursive git show
> olaf@bax:~/xen $
>
> xen does essentially 'git clone $url1 dir/url1-dir;git clone $url2
> dir/url2-dir'. So the top level .git does not really know about
> dir/*-dir.

Okay, I just checked that xen doesn't use submodules but uses simple embedded repos ignored by the .gitignore file. From a quick glance it looks like handling the sub projects is scripted in the Makefiles. This is perfectly fine, but then you can't really expect git commands to know about these sub projects. Or am I missing something?

Olaf Hering· Oct 8, 2014, 07:31 UTC · re: Jens Lehmann · lore

Re: let git grep consider sub projects

On Wed, Oct 08, Jens Lehmann wrote:
Show 5 quoted lines
> Okay, I just checked that xen doesn't use submodules but uses simple
> embedded repos ignored by the .gitignore file. From a quick glance
> it looks like handling the sub projects is scripted in the Makefiles.
> This is perfectly fine, but then you can't really expect git commands
> to know about these sub projects. Or am I missing something?

I just did not know about the submodule command. And I did not realize that they are in .gitignore, which should have been obvious.

I will check if using submodule instead of plain clone can be done, without breaking anyones workflow.

Olaf

← back to recent threads