threads / discuss / 53580

Git fetches whole repository and not just latest

Subject: Git fetches whole repository and not just latest

## tl;dr

5 messages between May 29, 2020 and May 30, 2020.

replies: 4people: 2as markdown or json

Andy Shevchenko· May 29, 2020, 10:45 UTC · lore

A bit of description what I have on my side. So, I have a local repository which has many remotes added (~20 or so). Most of them are from the same Git server, but few are from different servers.

To refresh the state of my local repository I run `git remote update -p`. One note that I'm behind the proxy. This schema works for few years until out of a sudden (approximately month ago) I have noticed that Git started fetching whole repository instead of only latest stuff. Note, that the repositories in question are from the main server. Yes, sometimes it goes well, sometimes some random repository gets downloaded in full.

Example:

(few days ago)

Fetching spi remote: Enumerating objects: 18064, done. remote: Counting objects: 100% (18064/18064), done. remote: Compressing objects: 100% (2784/2784), done. remote: Total 7343589 (delta 16227), reused 15523 (delta 15266), pack-reused 7325525 Receiving objects: 100% (7343589/7343589), 1.21 GiB | 2.36 MiB/s, done.

(today)

Fetching spi remote: Enumerating objects: 822, done. remote: Counting objects: 100% (686/686), done. remote: Compressing objects: 100% (323/323), done. remote: Total 602 (delta 485), reused 358 (delta 279), pack-reused 0 Receiving objects: 100% (602/602), 160.25 KiB | 896.00 KiB/s, done.

(also today) Fetching origin remote: Enumerating objects: 17806, done. remote: Counting objects: 100% (17806/17806), done. remote: Compressing objects: 100% (2681/2681), done. remote: Total 7343331 (delta 15992), reused 15321 (delta 15106), pack-reused 7325525 Receiving objects: 100% (7343331/7343331), 1.21 GiB | 7.59 MiB/s, done.

origin, for example, almost everyday now gets in full (1.21 GiB!), while others have no pattern.

I would like to know if it's problem of proxy, or that specific Git server or is it (new) bug in Git?

-- 
With Best Regards,
Andy Shevchenko

Jonathan Nieder· May 29, 2020, 17:29 UTC · re: Andy Shevchenko · lore

Re: Git fetches whole repository and not just latest

Hi Andy,

Andy Shevchenko wrote:

> So, I have a local repository which has many remotes added (~20 or
> so). Most of them are from the same Git server, but few are from
> different servers.

[...]

Show 5 quoted lines

> origin, for example, almost everyday now gets in full (1.21 GiB!),
> while others have no pattern.
>
> I would like to know if it's problem of proxy, or that specific Git
> server or is it (new) bug in Git?

What Git version are you using?  Can you test 2.27.0-rc2?

I believe this is fixed by

	commit 2f0a093dd640e0dad0b261dae2427f2541b5426c
	Author: Jonathan Tan <jonathantanmy@google.com>
	Date:   Mon Apr 27 17:01:10 2020 -0700

	    fetch-pack: in protocol v2, reset in_vain upon ACK

which is part of 2.27.0-rc0.

Thanks and hope that helps, Jonathan

Andy Shevchenko· May 29, 2020, 19:08 UTC · re: Jonathan Nieder · lore

Re: Git fetches whole repository and not just latest

On Fri, May 29, 2020 at 8:29 PM Jonathan Nieder <jrnieder@gmail.com> wrote:

Show 13 quoted lines

> Andy Shevchenko wrote:
>
> > So, I have a local repository which has many remotes added (~20 or
> > so). Most of them are from the same Git server, but few are from
> > different servers.
> [...]
> > origin, for example, almost everyday now gets in full (1.21 GiB!),
> > while others have no pattern.
> >
> > I would like to know if it's problem of proxy, or that specific Git
> > server or is it (new) bug in Git?
>
> What Git version are you using?

git version 2.26.2

>  Can you test 2.27.0-rc2?

Is there any deb package?

Show 11 quoted lines

> I believe this is fixed by
>
>         commit 2f0a093dd640e0dad0b261dae2427f2541b5426c
>         Author: Jonathan Tan <jonathantanmy@google.com>
>         Date:   Mon Apr 27 17:01:10 2020 -0700
>
>             fetch-pack: in protocol v2, reset in_vain upon ACK
>
> which is part of 2.27.0-rc0.
>
> Thanks and hope that helps,

I'll try to allocate slot to test later, if there is deb available, I can do it sooner.

Thanks!

-- 
With Best Regards,
Andy Shevchenko

Jonathan Nieder· May 29, 2020, 19:20 UTC · re: Andy Shevchenko · lore

Re: Git fetches whole repository and not just latest

Andy Shevchenko wrote:

> On Fri, May 29, 2020 at 8:29 PM Jonathan Nieder <jrnieder@gmail.com> wrote:

>> What Git version are you using?
>
> git version 2.26.2

Thanks, makes sense.

>>  Can you test 2.27.0-rc2?
>
> Is there any deb package?

Yes, you can use the package from Debian unstable or experimental (https://tracker.debian.org/pkg/git).

If you're on Ubuntu, you can use https://launchpad.net/~git-core/+archive/ubuntu/candidate once it updates (cc-ing the maintainers of the PPA in case they want to comment on when that will be --- e.g. Git 2.27 is planned to come out this Monday).

Thanks, Jonathan

Andy Shevchenko· May 30, 2020, 11:16 UTC · re: Jonathan Nieder · lore

Re: Git fetches whole repository and not just latest

On Fri, May 29, 2020 at 10:20 PM Jonathan Nieder <jrnieder@gmail.com> wrote:

> Andy Shevchenko wrote:
> > On Fri, May 29, 2020 at 8:29 PM Jonathan Nieder <jrnieder@gmail.com> wrote:

...

Show 6 quoted lines

> >>  Can you test 2.27.0-rc2?
> >
> > Is there any deb package?
>
> Yes, you can use the package from Debian unstable or experimental
> (https://tracker.debian.org/pkg/git).

I run old version, noticed the bug, Ctrl+C immediately. Then I have installed rc2 and re-run without noticing the bug. Since it's only one run, I can't be sure, but I'll definitely monitor the situation for a long run. Thanks!

-- 
With Best Regards,
Andy Shevchenko