threads / discuss / 53703

git clone --filter=blob:limit=X interaction with submodules?

Subject: git clone --filter=blob:limit=X interaction with submodules?

## tl;dr

3 messages between Jun 17, 2020 and Jun 19, 2020.

replies: 2people: 2as markdown or json

Luke Diamand· Jun 17, 2020, 19:14 UTC · lore

I noticed that if I do git clone with submodules, using "--filter=blob:limit=whatever" that the filter is applied in the super project, but appears to be ignored in the sub modules.

Is that expected behaviour? Or am I just doing it wrong?
I'm using git 2.27.0.90.geebb51ba8c.

Thanks Luke

Jonathan Tan· Jun 17, 2020, 23:20 UTC · re: Luke Diamand · lore

Re: git clone --filter=blob:limit=X interaction with submodules?

Show 10 quoted lines
> I noticed that if I do git clone with submodules, using
> "--filter=blob:limit=whatever" that the filter is applied in the super
> project, but appears to be ignored in the sub modules.
> 
> Is that expected behaviour? Or am I just doing it wrong?
> 
> I'm using git 2.27.0.90.geebb51ba8c.
> 
> Thanks
> Luke

Yes, that's expected. The main technical reason is that a lot of the subsequent submodule operations may read objects from the submodules, and those could trigger on-demand fetches (if the submodules were themselves partial clones) - but right now the fetch code is designed to run only on one repository.

For those who would like to improve this state of affairs, one step in the right direction would be to update the fetch and transport code to support an arbitrary "struct repository" passed in, and not just operate on the_repository.

Luke Diamand· Jun 19, 2020, 14:00 UTC · re: Jonathan Tan · lore

Re: git clone --filter=blob:limit=X interaction with submodules?

On Thu, 18 Jun 2020 at 00:20, Jonathan Tan <jonathantanmy@google.com> wrote:
Show 17 quoted lines
>
> > I noticed that if I do git clone with submodules, using
> > "--filter=blob:limit=whatever" that the filter is applied in the super
> > project, but appears to be ignored in the sub modules.
> >
> > Is that expected behaviour? Or am I just doing it wrong?
> >
> > I'm using git 2.27.0.90.geebb51ba8c.
> >
> > Thanks
> > Luke
>
> Yes, that's expected. The main technical reason is that a lot of the
> subsequent submodule operations may read objects from the submodules,
> and those could trigger on-demand fetches (if the submodules were
> themselves partial clones) - but right now the fetch code is designed to
> run only on one repository.
Would this also explain why I get errors when syncing my superproject:
$ git pull --rebase
remote: Enumerating objects: 5, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), 844 bytes | 844.00 KiB/s, done.
Resolving deltas: 100% (2/2), completed with 2 local objects.
From ssh://whatever/somerepo
   7a9c9e1eeee..3325285b122  master     -> origin/master
fatal: git upload-pack: not our ref edcedb543a8b85c659ab7cb507bdb0211fb21bdf
fatal: remote error: upload-pack: not our ref
edcedb543a8b85c659ab7cb507bdb0211fb21bdf
The SSH command failed, please check the server logs for details.

This happens consistently when pulling from a repo that was originally cloned using --filter. The same repo without the filtering does not have this problem.

Show 5 quoted lines
>
> For those who would like to improve this state of affairs, one step in
> the right direction would be to update the fetch and transport code to
> support an arbitrary "struct repository" passed in, and not just operate
> on the_repository.

← back to recent threads