threads / discuss / 15469

git-clone: path or ssh problem with git-upload-pack in 1.6.0?

Subject: git-clone: path or ssh problem with git-upload-pack in 1.6.0?

## tl;dr

11 messages between Sep 10, 2008 and Sep 16, 2008.

replies: 10people: 5as markdown or json

Paul Johnston· Sep 10, 2008, 02:29 UTC · lore

I'm having trouble with git-clone and I'm wondering if there's something I'm doing wrong or something wrong with git. Probably the former, hopefully someone can set me straight.

I'm an admittedly novice git user. I'm trying to clone a repository over ssh. The host machine 'imac' is my mac osx 10.4 with git 1.6.0 installed from a macports package. This installs into /opt/local/bin. I also cloned git from HEAD and 'make; make install'ed into ~/bin, this is the version shown below. Either way, these are nonstandard/non-system-wide installation locations, and it requires that my shell PATH reflect this, obviously.

----------------------------------------------

# About my installation imac:~ paul$ which git /Users/paul/bin/git

imac:~ paul$ git --version git version 1.6.0.1.285.g1070

imac:~ paul$ ls /opt/local/bin/git* /opt/local/bin/git /opt/local/bin/git-upload-archive /opt/local/bin/gitk /opt/local/bin/git-receive-pack /opt/local/bin/git-upload-pack

imac:~ paul$ ls ~/bin/git*
/Users/paul/bin/git                     /Users/paul/bin/git-shell
         /Users/paul/bin/git-upload-pack
/Users/paul/bin/git-receive-pack        /Users/paul/bin/git-upload-archive
# Make a simple reposirtory /tmp/test/.git
imac:~ paul$ cd /tmp
imac:/tmp paul$ mkdir test; cd test; echo 'Hello World' > README; git
init; git add README; git commit -a -m'Initial import'; cd ..
Initialized empty Git repository in /private/tmp/test/.git/
Created initial commit 49c10e6: Initial import
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 README

# OK, this fails expectedly imac:/tmp paul$ cd ~ imac:~ paul$ git clone ssh://localhost/tmp/test/.git Initialized empty Git repository in /Users/paul/test/.git/ Password: bash: line 1: git-upload-pack: command not found fatal: The remote end hung up unexpectedly

# This has been killing me... Have tried numerous variants of the same with similar results... imac:~ paul$ git clone --upload-pack /opt/local/bin/git-upload-pack ssh://localhost/tmp/test/.git Initialized empty Git repository in /Users/paul/test/.git/ Password: remote: fatal: exec pack-objects failed. error: git-upload-pack: git-pack-objects died with error. fatal: git-upload-pack: aborting due to possible repository corruption on the remote side. remote: aborting due to possible repository corruption on the remote side. fatal: early EOF fatal: index-pack failed

# Fix it with symlinks in /usr/bin imac:~ paul$ cd /usr/bin/ imac:/usr/bin paul$ sudo ln -s /opt/local/bin/git* . imac:/usr/bin paul$ ls -al git* lrwxr-xr-x 1 root wheel 18 Sep 9 19:18 git -> /opt/local/bin/git lrwxr-xr-x 1 root wheel 31 Sep 9 19:18 git-receive-pack -> /opt/local/bin/git-receive-pack lrwxr-xr-x 1 root wheel 33 Sep 9 19:18 git-upload-archive -> /opt/local/bin/git-upload-archive lrwxr-xr-x 1 root wheel 30 Sep 9 19:18 git-upload-pack -> /opt/local/bin/git-upload-pack lrwxr-xr-x 1 root wheel 19 Sep 9 19:18 gitk -> /opt/local/bin/gitk

# And now git-clone works as expected... imac:/usr/bin paul$ cd imac:~ paul$ git clone ssh://localhost/tmp/test/.git Initialized empty Git repository in /Users/paul/test/.git/ Password: remote: Counting objects: 3, done. remote: Total 3 (delta 0), reused 0 (delta 0) Receiving objects: 100% (3/3), done.

imac:~ paul$ find test/
test/
test//.git
   { file listing removed for brevity... }
test//README
-------------------------------------------

OK, great, now it works. I suspect I'm not using the --upload-pack option correctly or my ssh installation is suboptimal, but as I could find no existing documentation on the subject, I am posting this message. Any thoughts?

Paul Johnston
Paul Johnston· Sep 15, 2008, 08:30 UTC · re: Paul Johnston · lore

I'm having trouble with git-clone and I'm wondering if there's something I'm doing wrong or something wrong with git. Probably the former, hopefully someone can set me straight.

I'm an admittedly novice git user. I'm trying to clone a repository over ssh. The host machine 'imac' is my mac osx 10.4 with git 1.6.0 installed from a macports package. This installs into /opt/local/bin. I also cloned git from HEAD and 'make; make install'ed into ~/bin, this is the version shown below. Either way, these are nonstandard/non-system-wide installation locations, and it requires that my shell PATH reflect this, obviously.

----------------------------------------------

# About my installation imac:~ paul$ which git /Users/paul/bin/git

imac:~ paul$ git --version git version 1.6.0.1.285.g1070

imac:~ paul$ ls /opt/local/bin/git* /opt/local/bin/git /opt/local/bin/git-upload-archive /opt/local/bin/gitk /opt/local/bin/git-receive-pack /opt/local/bin/git-upload-pack

imac:~ paul$ ls ~/bin/git* /Users/paul/bin/git /Users/paul/bin/git-shell /Users/paul/bin/git-upload-pack /Users/paul/bin/git-receive-pack /Users/paul/bin/git-upload-archive

# Make a simple reposirtory /tmp/test/.git
imac:~ paul$ cd /tmp
imac:/tmp paul$ mkdir test; cd test; echo 'Hello World' > README; git
init; git add README; git commit -a -m'Initial import'; cd ..
Initialized empty Git repository in /private/tmp/test/.git/
Created initial commit 49c10e6: Initial import
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 README

# OK, this fails expectedly imac:/tmp paul$ cd ~ imac:~ paul$ git clone ssh://localhost/tmp/test/.git Initialized empty Git repository in /Users/paul/test/.git/ Password: bash: line 1: git-upload-pack: command not found fatal: The remote end hung up unexpectedly

# This has been killing me... Have tried numerous variants of the same with similar results... imac:~ paul$ git clone --upload-pack /opt/local/bin/git-upload-pack ssh://localhost/tmp/test/.git Initialized empty Git repository in /Users/paul/test/.git/ Password: remote: fatal: exec pack-objects failed. error: git-upload-pack: git-pack-objects died with error. fatal: git-upload-pack: aborting due to possible repository corruption on the remote side. remote: aborting due to possible repository corruption on the remote side. fatal: early EOF fatal: index-pack failed

# Fix it with symlinks in /usr/bin imac:~ paul$ cd /usr/bin/ imac:/usr/bin paul$ sudo ln -s /opt/local/bin/git* . imac:/usr/bin paul$ ls -al git* lrwxr-xr-x 1 root wheel 18 Sep 9 19:18 git -> /opt/local/bin/git lrwxr-xr-x 1 root wheel 31 Sep 9 19:18 git-receive-pack -> /opt/local/bin/git-receive-pack lrwxr-xr-x 1 root wheel 33 Sep 9 19:18 git-upload-archive -> /opt/local/bin/git-upload-archive lrwxr-xr-x 1 root wheel 30 Sep 9 19:18 git-upload-pack -> /opt/local/bin/git-upload-pack lrwxr-xr-x 1 root wheel 19 Sep 9 19:18 gitk -> /opt/local/bin/gitk

# And now git-clone works as expected... imac:/usr/bin paul$ cd imac:~ paul$ git clone ssh://localhost/tmp/test/.git Initialized empty Git repository in /Users/paul/test/.git/ Password: remote: Counting objects: 3, done. remote: Total 3 (delta 0), reused 0 (delta 0) Receiving objects: 100% (3/3), done.

imac:~ paul$ find test/
test/
test//.git
  { file listing removed for brevity... }
test//README
-------------------------------------------

OK, great, now it works. I suspect I'm not using the --upload-pack option correctly or my ssh installation is suboptimal, but as I could find no existing documentation on the subject, I am posting this message. Any thoughts?

Paul Johnston
Michael J Gruber· Sep 15, 2008, 16:24 UTC · re: Paul Johnston · lore

[PATCH] help git-upload-pack find git

The --upload-pack option to clone raises the expectation that finding git-upload-pack on the server side is enough. But git-upload-pack needs to find git (for forking "git pack-objects") which fails if git is not in $PATH.

This patch makes git-upload-pack use git_set_argv0_path() so that an explicitely specified path for git-upload-pack is added to the path which is used when fork()ing.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
 upload-pack.c |   15 +++++++++++++++
 1 files changed, 15 insertions(+), 0 deletions(-)

The corresponding bug was reported by Paul Johnston who wondered why "git clone" failed when specifying --upload-pack for an out-of-$PATH installation of git. I'm not sure whether we should encourage this, but the --upload-pack option clearly gives the impression that git-upload-pack is all that is needed.

I haven't looked at other server side programmes yet regarding this issue, but "git clone" works fine with this patch in the situation where it would not without.

diff --git a/upload-pack.c b/upload-pack.c
index e5adbc0..1fb59de 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -615,6 +615,21 @@ int main(int argc, char **argv)
 	char *dir;
 	int i;
 	int strict = 0;
+	const char *cmd = argv[0] && *argv[0] ? argv[0] : "git-upload-pack";
+	char *slash = (char *)cmd + strlen(cmd);
+
+	/*
+	 * Take the basename of argv[0] as the command
+	 * name, and the dirname as the default exec_path
+	 * if we don't have anything better.
+	 */
+	do
+		--slash;
+	while (cmd <= slash && !is_dir_sep(*slash));
+	if (cmd <= slash) {
+		*slash++ = 0;
+		git_set_argv0_path(cmd);
+	}
 
 	for (i = 1; i < argc; i++) {
 		char *arg = argv[i];
-- 
1.6.0.1.308.gede4c
Junio C Hamano· Sep 15, 2008, 19:34 UTC · re: Michael J Gruber · lore

Re: [PATCH] help git-upload-pack find git

Michael J Gruber <git@drmicha.warpmail.net> writes:
Show 5 quoted lines
> The corresponding bug was reported by Paul Johnston who wondered why
> "git clone" failed when specifying --upload-pack for an out-of-$PATH
> installation of git.  I'm not sure whether we should encourage this, but
> the --upload-pack option clearly gives the impression that
> git-upload-pack is all that is needed.

Another possibility would be to teach exec_cmd.c:setup_path() to add the directory specified by $(bindir) to PATH after we add GIT_EXEC_PATH to it. That should cover the case David Miller reported, shouldn't it?

Michael J Gruber· Sep 15, 2008, 20:22 UTC · re: Junio C Hamano · lore

Re: [PATCH] help git-upload-pack find git

Junio C Hamano venit, vidit, dixit 15.09.2008 21:34:
Show 11 quoted lines
> Michael J Gruber <git@drmicha.warpmail.net> writes:
> 
>> The corresponding bug was reported by Paul Johnston who wondered why
>> "git clone" failed when specifying --upload-pack for an out-of-$PATH
>> installation of git.  I'm not sure whether we should encourage this, but
>> the --upload-pack option clearly gives the impression that
>> git-upload-pack is all that is needed.
> 
> Another possibility would be to teach exec_cmd.c:setup_path() to add the
> directory specified by $(bindir) to PATH after we add GIT_EXEC_PATH to
> it.  That should cover the case David Miller reported, shouldn't it?

Probably. I read that thread only after submitting my patch for upload-pack but suspected that other server side incarnations may suffer from the same problem.

I was actually surprised that setup_path() uses argv0_path without setting it, same as with argv_exec_path. I assumed this is for a good reason, I'm lacking the code base overview to judge this myself.

In any case, git.c sets argv0_path early, messes a bit with argv[0] and calls setup_path afterwards anyways. So adding the path in setup_path() should not hurt any "git foo" command.

One could construe situations where even that wouldn't help, because git-upload-pack can't pass --exec-dir to git and they can be in different locations - but I think that's crazy.

git.c, upload-pack.c, receive.pack.c and shell.c are the only callers. setup_path() needs to get a parameter. If shell.c should profit from the change then it needs to be taught how to pass an absolute path to do_{generic,cvs}_cmd().

So, I guess the general approach (change setup() path and have every caller profit) is OK. OK with you?

Michael
Johannes Sixt· Sep 16, 2008, 06:17 UTC · re: Michael J Gruber · lore

Re: [PATCH] help git-upload-pack find git

Michael J Gruber schrieb:
> I was actually surprised that setup_path() uses argv0_path without
> setting it, same as with argv_exec_path. I assumed this is for a good
> reason, I'm lacking the code base overview to judge this myself.

It argv_exec_path comes from git --exec-path=..., hence, git (via git.c) is the only caller of setup_path() that is able to set it. We can leave that one out of the equation.

Show 15 quoted lines
> In any case, git.c sets argv0_path early, messes a bit with argv[0] and
> calls setup_path afterwards anyways. So adding the path in setup_path()
> should not hurt any "git foo" command.
> 
> One could construe situations where even that wouldn't help, because
> git-upload-pack can't pass --exec-dir to git and they can be in
> different locations - but I think that's crazy.
> 
> git.c, upload-pack.c, receive.pack.c and shell.c are the only callers.
> setup_path() needs to get a parameter. If shell.c should profit from the
> change then it needs to be taught how to pass an absolute path to
> do_{generic,cvs}_cmd().
> 
> So, I guess the general approach (change setup() path and have every
> caller profit) is OK. OK with you?

Have you studied the commit message of e1464ca7bb0d (Record the command invocation path early) and the context in which this commit occurs? It's about relocatable git installations and how system_path() derives various other paths from argv[0].

Please show how you think you could change setup_path(), but keep in mind that in git.c you neither can do the equivalent of git_set_argv0_path() later nor setup_path() earlier.

-- Hannes
Michael J Gruber· Sep 16, 2008, 13:15 UTC · re: Johannes Sixt · lore

Re: [PATCH] help git-upload-pack find git

Johannes Sixt venit, vidit, dixit 16.09.2008 08:17:
Show 29 quoted lines
> Michael J Gruber schrieb:
>> I was actually surprised that setup_path() uses argv0_path without
>> setting it, same as with argv_exec_path. I assumed this is for a good
>> reason, I'm lacking the code base overview to judge this myself.
> 
> It argv_exec_path comes from git --exec-path=..., hence, git (via git.c)
> is the only caller of setup_path() that is able to set it. We can leave
> that one out of the equation.
> 
>> In any case, git.c sets argv0_path early, messes a bit with argv[0] and
>> calls setup_path afterwards anyways. So adding the path in setup_path()
>> should not hurt any "git foo" command.
>>
>> One could construe situations where even that wouldn't help, because
>> git-upload-pack can't pass --exec-dir to git and they can be in
>> different locations - but I think that's crazy.
>>
>> git.c, upload-pack.c, receive.pack.c and shell.c are the only callers.
>> setup_path() needs to get a parameter. If shell.c should profit from the
>> change then it needs to be taught how to pass an absolute path to
>> do_{generic,cvs}_cmd().
>>
>> So, I guess the general approach (change setup() path and have every
>> caller profit) is OK. OK with you?
> 
> Have you studied the commit message of e1464ca7bb0d (Record the command
> invocation path early) and the context in which this commit occurs? It's
> about relocatable git installations and how system_path() derives various
> other paths from argv[0].

That is what I meant by my lack of overview ;) I might have implemented what amounts to a revert of e1464ca7bb0d otherwise.

> Please show how you think you could change setup_path(), but keep in mind
> that in git.c you neither can do the equivalent of git_set_argv0_path()
> later nor setup_path() earlier.

Well, I'd be happy with the change in upload-pack as submitted. Junio suggested the more general approach. In fact I'd prefer everyone set their path, but not everyone seems to be allowed to in their restricted environments.

Why is it that setup_path() cannot be done before commit_pager_choice()? We need the pager in case list_common_cmds_help() is triggered, but why can't setup_path() be before that? Not because of "--", really.

Michael
Johannes Sixt· Sep 16, 2008, 13:43 UTC · re: Michael J Gruber · lore

Re: [PATCH] help git-upload-pack find git

Michael J Gruber schrieb:
> Why is it that setup_path() cannot be done before commit_pager_choice()?
> We need the pager in case list_common_cmds_help() is triggered, but why
> can't setup_path() be before that? Not because of "--", really.

I think that setup_path() must be called *after* handle_options() because of this use-case:

  $ git --exec-path=/some/where foo

and git_set_argv0_path() must be called *before* handle_options() because of this use-case:

  $ git --exec-path
-- Hannes
Michael J Gruber· Sep 16, 2008, 15:38 UTC · re: Johannes Sixt · lore

Re: [PATCH] help git-upload-pack find git

Johannes Sixt venit, vidit, dixit 16.09.2008 15:43:
Show 16 quoted lines
> Michael J Gruber schrieb:
>> Why is it that setup_path() cannot be done before commit_pager_choice()?
>> We need the pager in case list_common_cmds_help() is triggered, but why
>> can't setup_path() be before that? Not because of "--", really.
> 
> I think that setup_path() must be called *after* handle_options() because
> of this use-case:
> 
>   $ git --exec-path=/some/where foo
> 
> and git_set_argv0_path() must be called *before* handle_options() because
> of this use-case:
> 
>   $ git --exec-path
> 
> -- Hannes

Tricky, I see. But still I'm getting more and more confused: handle_options() calls git_exec_path() in this case, which does (unless there is argv_exec_path or the variable): return system_path(GIT_EXEC_PATH) Duh, that comes from the Makefile, -DGIT_EXEC_PATH... Back to somewhat less confused state.

OK, now how about
A: path as submitted
or
B: leave git.c as is, except that setup_path(NULL) is used; have
setup_path(argv0path) call git_set_argv0_path() unless the arg is NULL;
tell upload-pack etc. to stuff (other setup_path() callers) their
argv[0] into setup_path(). Junio, is that what you meant?
Michael
Michael J Gruber· Sep 15, 2008, 16:25 UTC · re: Paul Johnston · lore

Re: git-clone: path or ssh problem with git-upload-pack in 1.6.0?

Paul Johnston venit, vidit, dixit 15.09.2008 10:30:
Show 98 quoted lines
> I'm having trouble with git-clone and I'm wondering if there's
> something I'm doing wrong or something wrong with git.  Probably the
> former, hopefully someone can set me straight.
> 
> I'm an admittedly novice git user. I'm trying to clone a repository
> over ssh. The host machine 'imac' is my mac osx 10.4 with git 1.6.0
> installed from a macports package.  This installs into /opt/local/bin.
> I also cloned git from HEAD and 'make; make install'ed into ~/bin,
> this is the version shown below.  Either way, these are
> nonstandard/non-system-wide installation locations, and it requires
> that my shell PATH reflect this, obviously.
> 
> ----------------------------------------------
> 
> # About my installation
> imac:~ paul$ which git
> /Users/paul/bin/git
> 
> imac:~ paul$ git --version
> git version 1.6.0.1.285.g1070
> 
> imac:~ paul$ ls /opt/local/bin/git*
> /opt/local/bin/git
> /opt/local/bin/git-upload-archive       /opt/local/bin/gitk
> /opt/local/bin/git-receive-pack         /opt/local/bin/git-upload-pack
> 
> imac:~ paul$ ls ~/bin/git*
> /Users/paul/bin/git
> /Users/paul/bin/git-shell
> /Users/paul/bin/git-upload-pack
> /Users/paul/bin/git-receive-pack
> /Users/paul/bin/git-upload-archive
> 
> # Make a simple reposirtory /tmp/test/.git
> imac:~ paul$ cd /tmp
> imac:/tmp paul$ mkdir test; cd test; echo 'Hello World' > README; git
> init; git add README; git commit -a -m'Initial import'; cd ..
> Initialized empty Git repository in /private/tmp/test/.git/
> Created initial commit 49c10e6: Initial import
>  1 files changed, 1 insertions(+), 0 deletions(-)
>  create mode 100644 README
> 
> # OK, this fails expectedly
> imac:/tmp paul$ cd ~
> imac:~ paul$ git clone ssh://localhost/tmp/test/.git
> Initialized empty Git repository in /Users/paul/test/.git/
> Password:
> bash: line 1: git-upload-pack: command not found
> fatal: The remote end hung up unexpectedly
> 
> # This has been killing me...  Have tried numerous variants of the
> same with similar results...
> imac:~ paul$ git clone --upload-pack /opt/local/bin/git-upload-pack
> ssh://localhost/tmp/test/.git
> Initialized empty Git repository in /Users/paul/test/.git/
> Password:
> remote: fatal: exec pack-objects failed.
> error: git-upload-pack: git-pack-objects died with error.
> fatal: git-upload-pack: aborting due to possible repository corruption
> on the remote side.
> remote: aborting due to possible repository corruption on the remote side.
> fatal: early EOF
> fatal: index-pack failed
> 
> # Fix it with symlinks in /usr/bin
> imac:~ paul$ cd /usr/bin/
> imac:/usr/bin paul$ sudo ln -s /opt/local/bin/git* .
> imac:/usr/bin paul$ ls -al git*
> lrwxr-xr-x   1 root  wheel  18 Sep  9 19:18 git -> /opt/local/bin/git
> lrwxr-xr-x   1 root  wheel  31 Sep  9 19:18 git-receive-pack ->
> /opt/local/bin/git-receive-pack
> lrwxr-xr-x   1 root  wheel  33 Sep  9 19:18 git-upload-archive ->
> /opt/local/bin/git-upload-archive
> lrwxr-xr-x   1 root  wheel  30 Sep  9 19:18 git-upload-pack ->
> /opt/local/bin/git-upload-pack
> lrwxr-xr-x   1 root  wheel  19 Sep  9 19:18 gitk -> /opt/local/bin/gitk
> 
> # And now git-clone works as expected...
> imac:/usr/bin paul$ cd
> imac:~ paul$ git clone ssh://localhost/tmp/test/.git
> Initialized empty Git repository in /Users/paul/test/.git/
> Password:
> remote: Counting objects: 3, done.
> remote: Total 3 (delta 0), reused 0 (delta 0)
> Receiving objects: 100% (3/3), done.
> 
> imac:~ paul$ find test/
> test/
> test//.git
>   { file listing removed for brevity... }
> test//README
> 
> -------------------------------------------
> 
> OK, great, now it works. I suspect I'm not using the --upload-pack
> option correctly or my ssh installation is suboptimal, but as I could
> find no existing documentation on the subject, I am posting this
> message.  Any thoughts?

re ssh: I'm surprised your PATH doesn't include $HOME/bin when sshing. That should make everything work.

re upload-pack: That indeed sounds as if upload-pack can't execute pack-objects. I checked by moving my bin away, and clone failed. Seems everyone is using proper PATHS ;)

In fact, upload-pack tries to call "git pack-objects" and fails if it can't find git. Patch upcoming...

Michael
Michael Wookey· Sep 15, 2008, 22:39 UTC · re: Paul Johnston · lore

Re: git-clone: path or ssh problem with git-upload-pack in 1.6.0?

Show 11 quoted lines
> I'm having trouble with git-clone and I'm wondering if there's
> something I'm doing wrong or something wrong with git.  Probably the
> former, hopefully someone can set me straight.
>
> I'm an admittedly novice git user. I'm trying to clone a repository
> over ssh. The host machine 'imac' is my mac osx 10.4 with git 1.6.0
> installed from a macports package.  This installs into /opt/local/bin.
> I also cloned git from HEAD and 'make; make install'ed into ~/bin,
> this is the version shown below.  Either way, these are
> nonstandard/non-system-wide installation locations, and it requires
> that my shell PATH reflect this, obviously.
The following might help..
http://marc.info/?l=git&m=121378876831164&w=2

← back to recent threads