threads / discuss / 50146

Suspicious fetch-pack behaviour

Subject: Suspicious fetch-pack behaviour

## tl;dr

4 messages between Jan 3, 2019 and Jan 8, 2019.

replies: 3people: 3as markdown or json

Guilhem Bonnefille· Jan 3, 2019, 09:52 UTC · lore
Hi,

One of my users reported a strange problem: a simple HTTPS clone did not work with Git 1.8.3.1 on RedHat 7. I did many tests and I was not able to understand why his clone don't work while I'm able to do it on other similar host.

Nevertheless, we did more investigations. One of them: a raw strace.
I discovered two strange behaviours:
- fetch-pack closes its standard input and standard output and then
tries to print the references on standard input and finaly dies.
- git-remote-https does not react to fetch-pack death and continue
polling an empty set of FD.

Reading fetch-pack code, the behaviour is explicit: When "--stateless-rpc" is provided, fd is filled with standard input and standard ouput which are then closed. https://git.kernel.org/pub/scm/git/git.git/tree/builtin/fetch-pack.c?h=v1.8.3.1#n156

Reading this, I did not understand why it could work. Any help appreciated.

Here is the strace's extract:

2801 getdents(3, /* 2 entries */, 32768) = 48 2801 getdents(3, /* 0 entries */, 32768) = 0 2801 close(3) = 0 2801 close(0) = 0 2801 close(1) = 0 2801 fstat(1, 0x7ffe10ab4730) = -1 EBADF (Bad file descriptor) 2801 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f220c9e3000 2801 fstat(1, 0x7ffe10ab5040) = -1 EBADF (Bad file descriptor) 2801 write(1, "4df1dbf3224064cc5dd2e4c095da2dda"..., 159) = -1 EBADF (Bad file descriptor) 2801 exit_group(0) = ? 2801 +++ exited with 0 +++ 2769 <... poll resumed> ) = ? ERESTART_RESTARTBLOCK (Interrupted by signal) 2769 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=2801, si_uid=7561, si_status=0, si_utime=0, si_stime=0} --- 2769 restart_syscall(<... resuming interrupted poll ...>) = 0 2769 poll(NULL, 0, 1000) = 0 (Timeout) 2769 poll(NULL, 0, 1000) = 0 (Timeout) 2769 poll(NULL, 0, 1000) = 0 (Timeout) 2769 poll(NULL, 0, 1000) = 0 (Timeout) 2769 poll(NULL, 0, 1000) = 0 (Timeout) 2769 poll(NULL, 0, 1000) = 0 (Timeout)

-- 
Guilhem BONNEFILLE
-=- JID: guyou@im.apinc.org MSN: guilhem_bonnefille@hotmail.com
-=- mailto:guilhem.bonnefille@gmail.com
-=- http://nathguil.free.fr/
brian m. carlson· Jan 7, 2019, 03:37 UTC · re: Guilhem Bonnefille · lore

Re: Suspicious fetch-pack behaviour

On Thu, Jan 03, 2019 at 10:52:48AM +0100, Guilhem Bonnefille wrote:
Show 21 quoted lines
> Hi,
> 
> One of my users reported a strange problem: a simple HTTPS clone did
> not work with Git 1.8.3.1 on RedHat 7.
> I did many tests and I was not able to understand why his clone don't
> work while I'm able to do it on other similar host.
> 
> Nevertheless, we did more investigations. One of them: a raw strace.
> I discovered two strange behaviours:
> - fetch-pack closes its standard input and standard output and then
> tries to print the references on standard input and finaly dies.
> - git-remote-https does not react to fetch-pack death and continue
> polling an empty set of FD.
> 
> Reading fetch-pack code, the behaviour is explicit:
> When "--stateless-rpc" is provided, fd is filled with standard input
> and standard ouput which are then closed.
> https://git.kernel.org/pub/scm/git/git.git/tree/builtin/fetch-pack.c?h=v1.8.3.1#n156
> 
> Reading this, I did not understand why it could work.
> Any help appreciated.

When --stateless-rpc is passed, git fetch-pack usually has its standard input and output wired up to the ends of a socket. Those file descriptors are then passed to do_fetch_pack, which calls get_common to negotiate refs with the remote side and get_pack to get the resulting pack data. The negotiation should function regardless of the final ref printing.

It's true that attempting to write to the standard output fails in that case, but that's okay, since we wouldn't have wanted to write that data to the socket anyway.

-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204
Jeff King· Jan 8, 2019, 06:34 UTC · re: Guilhem Bonnefille · lore

Re: Suspicious fetch-pack behaviour

On Thu, Jan 03, 2019 at 10:52:48AM +0100, Guilhem Bonnefille wrote:
Show 15 quoted lines
> One of my users reported a strange problem: a simple HTTPS clone did
> not work with Git 1.8.3.1 on RedHat 7.
> I did many tests and I was not able to understand why his clone don't
> work while I'm able to do it on other similar host.
> 
> Nevertheless, we did more investigations. One of them: a raw strace.
> I discovered two strange behaviours:
> - fetch-pack closes its standard input and standard output and then
> tries to print the references on standard input and finaly dies.
> - git-remote-https does not react to fetch-pack death and continue
> polling an empty set of FD.
>
> [...]
>
> 2769  poll(NULL, 0, 1000)               = 0 (Timeout)

We actually don't use poll() very much in Git. And poking around the v1.8.3.1 source, I do not see any places where remote-https would call poll(), and none outside of "git help" and "git credential-cache-daemon" that would ever provide a timeout like "1000".

I wonder if this poll is actually being run by libcurl. Is it possible to get a backtrace of the looping process with gdb?

I'd also point out that v1.8.3.1 is over 5 years old, and there have been quite a few http-related fixes over the years. There is a good chance that if this is a Git bug, it has long since been fixed. Is it possible to reproduce with a more modern version of Git?

-Peff
Guilhem Bonnefille· Jan 8, 2019, 07:26 UTC · re: Jeff King · lore

Re: Suspicious fetch-pack behaviour

Le mar. 8 janv. 2019 à 07:34, Jeff King <peff@peff.net> a écrit :
Show 26 quoted lines
>
> On Thu, Jan 03, 2019 at 10:52:48AM +0100, Guilhem Bonnefille wrote:
>
> > One of my users reported a strange problem: a simple HTTPS clone did
> > not work with Git 1.8.3.1 on RedHat 7.
> > I did many tests and I was not able to understand why his clone don't
> > work while I'm able to do it on other similar host.
> >
> > Nevertheless, we did more investigations. One of them: a raw strace.
> > I discovered two strange behaviours:
> > - fetch-pack closes its standard input and standard output and then
> > tries to print the references on standard input and finaly dies.
> > - git-remote-https does not react to fetch-pack death and continue
> > polling an empty set of FD.
> >
> > [...]
> >
> > 2769  poll(NULL, 0, 1000)               = 0 (Timeout)
>
> We actually don't use poll() very much in Git. And poking around the
> v1.8.3.1 source, I do not see any places where remote-https would call
> poll(), and none outside of "git help" and "git credential-cache-daemon"
> that would ever provide a timeout like "1000".
>
> I wonder if this poll is actually being run by libcurl.  Is it possible
> to get a backtrace of the looping process with gdb?

Quite hard to debug as it occured on a computer provided by a customer, far from me...

> I'd also point out that v1.8.3.1 is over 5 years old, and there have
> been quite a few http-related fixes over the years.
Yes, it is an old version, but it is the version provided with RedHat 7.
> There is a good
> chance that if this is a Git bug, it has long since been fixed. Is it
> possible to reproduce with a more modern version of Git?
>

What is surprisingly is that I was unable to reproduce with the same version on an other computer.

During this time, my user discovered the IT team of the customer prvide a much more recent version of Git (2.X). With this new version, the problem was not reproduced.

Thanks for all your investigations.
-- 
Guilhem BONNEFILLE
-=- JID: guyou@im.apinc.org MSN: guilhem_bonnefille@hotmail.com
-=- mailto:guilhem.bonnefille@gmail.com
-=- http://nathguil.free.fr/

← back to recent threads