Re: git-fetch takes forever on a slow network link. Can parallel mode help?

From: R. Diez <rdiez-2006@rd10.de>
Date: Mar 8, 2026, 21:08 UTC
Message-ID: <0ebf757b-eab5-424a-a58b-e654b1a2942e@rd10.de>
In-Reply-To: <aazUlMBj_IK41Ss2@fruit.crustytoothpaste.net>

Hi again:

Show 10 quoted lines

>> The log talks about "upload pack", but I gather this is actually a download operation. It wouldn't be the first confusing item in Git. Or have I got it wrong?
> 
> upload-pack refers to what's happening on the server.  If you contact a
> Git server over something like HTTPS or SSH, then it will use
> git-upload-pack to send data to you (a fetch or clone from your
> perspective) or git-receive-pack to receive data from you (a push from
> your perspective).
> 
> When you perform a local fetch, upload-pack is spawned in the remote
> repository to serve data.

My client computer has an SMB/CIFS connection to the remote file server. That means the client has mounted the file share with "mount.cifs", so in this scenario nothing is happening on the server, as the connection is not HTTPS or SSH. No process will be spawned on the remote server.

That is the reason why I am getting confused. From my point of view, my client computer is not "uploading" anything when doing a "git pull".

But I guess Git is designed for all scenarios and will probably not use the correct terminology in my case.

In case it helps, I am using Git version 2.53.0.

Show 15 quoted lines

>> I added "export GIT_TRACE_PACKET=true", and then I got a more useful breakdown:
>>
>> This takes around 13 seconds:
>>
>>    pkt-line.c:85           packet:  upload-pack< 0000
> 
> Is it just that line that takes 13 seconds or is the listing of
> references altogether that takes 13 seconds?  That particular line
> should not take 13 seconds because it's literally just writing and
> flushing 4 bytes.
> 
> It would be helpful if you can to include the entire trace output so we
> can see and analyze it ourselves.  It's very hard to analyze data from
> the different sections in isolation if one is not intimately familiar
> with the protocol.

The log does not really say which operation is taking how long. It does not say when the listing of references starts or finishes, which files it is reading and how many bytes it is reading from each file, or whether the files are read sequentially or in parallel.

Thanks for your feedback. I know it is hard to help without the whole log, but I would have to ask for permission to upload a log with file paths, hashes and tag names. Or clean them all manually.

Show 6 quoted lines

>> 7 seconds are spent with "upload-pack" and "fetch" operations, mainly for single "refs/tags". I'll check whether that improves after the next "git gc" on the server.
> 
> Okay, this is helpful.  You probably have the `peel` capability, which
> means that when you have a tag, you get a line like this:
> 
>      4a76996b9c60ca3f21e644d78e1e5089a06c6fb3 refs/tags/v0.1.0 peeled:b4c993704e90881bec9c217749be813c70ae2bb6

Yes, that is the case.

Show 5 quoted lines

> That `peeled` directive tells us what object the tag points to, but it
> means that the tag object has to be opened and read, which makes things
> much more expensive.  Unfortunately, there's no way to turn that
> capability off, since Git doesn't usually have capability control
> options for the protocol.

OK, but there is no protocol here, Git is accessing the files over the mount.

> _However_, if you pack references with `git pack-refs` or you use
> [...]

OK, I'll try with "git gc" on the remote server the next time I can.

Show 12 quoted lines

> Git is already downloading them as efficiently as possible.  The
> protocol has both sides advertise the references (branches, tags, etc.)
> that they have and then, in a fetch or clone, the client sends a list of
> what it has and what it wants, and the two sides negotiate to come to an
> agreement on what needs to be sent.  This shared understanding includes
> _all_ of the objects necessary for everything the client wants but
> doesn't have, and then those are all sent as part of one pack.
> 
> Parallelization would not help here because the limiting factor is the
> speed of the connection (and in your case, literally the speed of
> reading data off the file system).
> [...]

I don't think that is the case. Git is accessing the remote repository over a mount (a file share), so there is no protocol or negotiation, although I am guessing it is happening virtually with the current Git implementation.

If I understand it correctly, without "packed references", Git will have to access a number of small files on the remote server. Even with packet references, there will probably still be a few small files to access, in addition to some biggish packed references file.

In the past, on rotational hard disks, issuing many such read requests in parallel wasn't beneficial to performance, because of the disk head seek times. That is, jumping around would thrash the disk instead of increasing performance.

But that is not true anymore with SSDs, and especially with file mounts over a network connection with a high latency. In that scenario, issuing parallel requests (with multiple threads or async I/O) should actually increase performance.

Is my reasoning correct?

Another question: Would it help if I only fetched the 'master' branch? Something like "git fetch origin master". Most of the time, I am only interested in the main branch.

I am guessing that "git fetch" will download all other branches by default, because of this:

[remote "origin"] fetch = +refs/heads/*:refs/remotes/origin/*

I read the "git fetch" documentation, but I didn't understand whether it will fetch by default everything or just the current branch.

Thanks again,
  rdiez