threads / discuss / 2360

git binary directory?

Subject: git binary directory?

## tl;dr

17 messages between Nov 5, 2005 and Nov 10, 2005.

replies: 16people: 4as markdown or json

Linus Torvalds· Nov 5, 2005, 21:02 UTC · lore
Just for fun, I did "git-<tab><tab>" a moment ago, and it asked me
	Display all 171 possibilities? (y or n)

which was a bit scary. Now, part of that was because I hadn't cleaned up the old git names, so I removed everything, and did a clean "make install" with the current git tree. At which point the question became

	Display all 105 possibilities? (y or n)
and pressing "y" shows a really nice list of programs.

Now, this is actually very convenient, and since I tend to install things in my own ~/bin directory, it's all good. What's a hundred files more or less?

However, doing a "ls /usr/bin/ | wc" shows that on my laptop /usr/bin is already 2583 files, and git is actually part of the problem there (hey, it's got the git RPM installed - it's not my main machine)..

Now, I happen to think that 2500+ files in /usr/bin is a bit much (ever try to use the horrid gnome executable finder on it when you want to convince firefox to use xpdf instead of that broken crap called "evince"? Takes absolutely ages and is horrible).

And git made it about 4% worse all on its own.

So I'd really suggest that while the "git-<tab><tab>" thing is perhaps useful, we'd actually be better off with an /usr/lib/git directory where we put the git executables by default. And just put "git" into /usr/bin.

That way, people who _want_ to use "git-<tab><tab>" can just add the git binary directory to their path and directly access all of them. And others can just use the plain "git" interface.

That would mean that I'd have to learn to use "git whatchanged" and "git diff-tree" instead of "git-whatchanged" and "git-diff-tree", but hey, it's why we have that "git" script in the first place.

What do people think?
			Linus
Linus Torvalds· Nov 5, 2005, 23:52 UTC · re: Linus Torvalds · lore

Re: git binary directory?

On Sat, 5 Nov 2005, Linus Torvalds wrote:
> 
> So I'd really suggest that while the "git-<tab><tab>" thing is perhaps 
> useful, we'd actually be better off with an /usr/lib/git directory where 
> we put the git executables by default. And just put "git" into /usr/bin.

This is a guaranteed BUGGY thing to make the git RPM do something like that.

It adds a "gitdir" parameter to "make install", which defaults to the same as "bindir" if you don't specify it.

It installs only "git" in "bindir", and all other git programs in "gitdir".

It then makes the "git" script use "gitdir" for searching for the installed git scripts and programs, and forces PATH to have "gitdir" at the head while executing the sub-script (so that when the scripts do "git-diff-tree" etc, they'll always use the proper gitdir version).

The RPM specfile is taught to make "gitdir" be "${_libdir}/git-@@VERSION"
This means that if I got everything right, the RPM will build with
	/usr/bin/git
	/usr/lib/git-0.99.9.GIT/git-add
	/usr/lib/git-0.99.9.GIT/git-am
	/usr/lib/git-0.99.9.GIT/git-apply
	..
	/usr/share/doc/git-core-0.99.9.GIT
	/usr/share/doc/git-core-0.99.9.GIT/COPYING
	..
	/usr/share/git-core
	/usr/share/git-core/python
	..
	/usr/share/git-core/templates
	/usr/share/git-core/templates/branches
	..
	/usr/share/man/man1/git-add.1.gz
	/usr/share/man/man1/git-am.1.gz
	..
where the new thing is that "/usr/lib/git-0.99.9.GIT/" usage.

I've tested building the rpm, and that part works. I haven't actually tested the end result, though.

Oh, and I have not a clue in hell about how Debian works. In fact, I don't have a real clue how RPM files work either, this was done by doing some pattern matching on the existing git-core.spec.in file. Somebody who knows better really should fix it.

WARNING! I just realized that "gitk" also ends up in that /usr/lib/git.. directory, and this isn't visible. My bad. I'm a retard.

Comments?
What do people think about this? Any daring souls willing to test this?

(Btw: word of caution on something that confused me at first: you have to _commit_ this diff in order for "make rpm" to work, since "make rpm" uses "git-tar-tree" to create the rpm file, not the current directory contents, heh ;)

		Linus
---
diff --git a/Makefile b/Makefile
index 6c01dc2..8df8b50 100644
--- a/Makefile
+++ b/Makefile
@@ -59,6 +59,7 @@ ALL_CFLAGS = $(CFLAGS) $(PLATFORM_DEFINE
 
 prefix = $(HOME)
 bindir = $(prefix)/bin
+gitdir = $(bindir)
 template_dir = $(prefix)/share/git-core/templates/
 GIT_PYTHON_DIR = $(prefix)/share/git-core/python
 # DESTDIR=
@@ -86,7 +87,7 @@ SCRIPT_SH = \
 	git-prune.sh git-pull.sh git-push.sh git-rebase.sh \
 	git-repack.sh git-request-pull.sh git-reset.sh \
 	git-resolve.sh git-revert.sh git-sh-setup.sh git-status.sh \
-	git-tag.sh git-verify-tag.sh git-whatchanged.sh git.sh \
+	git-tag.sh git-verify-tag.sh git-whatchanged.sh \
 	git-applymbox.sh git-applypatch.sh git-am.sh \
 	git-merge.sh git-merge-stupid.sh git-merge-octopus.sh \
 	git-merge-resolve.sh git-merge-ours.sh git-grep.sh
@@ -315,7 +316,7 @@ SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)
 export prefix TAR INSTALL DESTDIR SHELL_PATH template_dir
 ### Build rules
 
-all: $(PROGRAMS) $(SCRIPTS)
+all: git $(PROGRAMS) $(SCRIPTS)
 
 all:
 	$(MAKE) -C templates
@@ -323,6 +324,7 @@ all:
 git: git.sh Makefile
 	rm -f $@+ $@
 	sed -e '1s|#!.*/sh|#!$(call shq,$(SHELL_PATH))|' \
+	    -e 's:@@GITDIR@@:$(gitdir):g' \
 	    -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
 	    -e 's/@@X@@/$(X)/g' \
 	    $(GIT_LIST_TWEAK) <$@.sh >$@+
@@ -410,9 +412,11 @@ check:
 
 ### Installation rules
 
-install: $(PROGRAMS) $(SCRIPTS)
+install: git $(PROGRAMS) $(SCRIPTS)
 	$(INSTALL) -d -m755 $(call shellquote,$(DESTDIR)$(bindir))
-	$(INSTALL) $(PROGRAMS) $(SCRIPTS) $(call shellquote,$(DESTDIR)$(bindir))
+	$(INSTALL) git $(call shellquote,$(DESTDIR)$(bindir))
+	$(INSTALL) -d -m755 $(call shellquote,$(DESTDIR)$(gitdir))
+	$(INSTALL) $(PROGRAMS) $(SCRIPTS) $(call shellquote,$(DESTDIR)$(gitdir))
 	$(MAKE) -C templates install
 	$(INSTALL) -d -m755 $(call shellquote,$(DESTDIR)$(GIT_PYTHON_DIR))
 	$(INSTALL) $(PYMODULES) $(call shellquote,$(DESTDIR)$(GIT_PYTHON_DIR))
diff --git a/git-core.spec.in b/git-core.spec.in
index 5240dd2..0d0ddf3 100644
--- a/git-core.spec.in
+++ b/git-core.spec.in
@@ -19,17 +19,19 @@ distributed source code management syste
 rudimentary tools that can be used as a SCM, but you should look
 elsewhere for tools for ordinary humans layered on top of this.
 
+%define gitdir %{_libdir}/git-@@VERSION@@
+
 %prep
 %setup -q
 
 %build
 make %{_smp_mflags} CFLAGS="$RPM_OPT_FLAGS" WITH_OWN_SUBPROCESS_PY=YesPlease \
-     prefix=%{_prefix} all %{!?_without_docs: doc}
+     gitdir=%{gitdir} prefix=%{_prefix} all %{!?_without_docs: doc}
 
 %install
 rm -rf $RPM_BUILD_ROOT
 make %{_smp_mflags} DESTDIR=$RPM_BUILD_ROOT WITH_OWN_SUBPROCESS_PY=YesPlease \
-     prefix=%{_prefix} mandir=%{_mandir} \
+     gitdir=%{gitdir} prefix=%{_prefix} mandir=%{_mandir} \
      install %{!?_without_docs: install-doc}
 
 %clean
@@ -38,6 +40,7 @@ rm -rf $RPM_BUILD_ROOT
 %files
 %defattr(-,root,root)
 %{_bindir}/*
+%{gitdir}/*
 %{_datadir}/git-core/
 %doc README COPYING Documentation/*.txt
 %{!?_without_docs: %doc Documentation/*.html }
diff --git a/git.sh b/git.sh
index 94940ae..9ba1608 100755
--- a/git.sh
+++ b/git.sh
@@ -1,7 +1,8 @@
 #!/bin/sh
 
 cmd=
-path=$(dirname "$0")
+path="@@GITDIR@@"
+export PATH="$path:$PATH"
 case "$#" in
 0)	;;
 *)	cmd="$1"
Linus Torvalds· Nov 6, 2005, 00:27 UTC · re: Linus Torvalds · lore

Re: git binary directory?

On Sat, 5 Nov 2005, Linus Torvalds wrote:
Show 8 quoted lines
> 
> This is a guaranteed BUGGY thing to make the git RPM do something like 
> that.
>
> [ ... ]
> 
> WARNING! I just realized that "gitk" also ends up in that /usr/lib/git.. 
> directory, and this isn't visible. My bad. I'm a retard. 

This fixes the gitk thing, by installing it (along with git) in the regular $prefix/bin directory (ie default /usr/bin for the RPM).

I'm still a retard, and it's _probably_ still buggy, but at least it is no longer GUARANTEED to be buggy.

		Linus
---
diff --git a/Makefile b/Makefile
index 6c01dc2..6ec9dd2 100644
--- a/Makefile
+++ b/Makefile
@@ -59,6 +59,7 @@ ALL_CFLAGS = $(CFLAGS) $(PLATFORM_DEFINE
 
 prefix = $(HOME)
 bindir = $(prefix)/bin
+gitdir = $(bindir)
 template_dir = $(prefix)/share/git-core/templates/
 GIT_PYTHON_DIR = $(prefix)/share/git-core/python
 # DESTDIR=
@@ -86,7 +87,7 @@ SCRIPT_SH = \
 	git-prune.sh git-pull.sh git-push.sh git-rebase.sh \
 	git-repack.sh git-request-pull.sh git-reset.sh \
 	git-resolve.sh git-revert.sh git-sh-setup.sh git-status.sh \
-	git-tag.sh git-verify-tag.sh git-whatchanged.sh git.sh \
+	git-tag.sh git-verify-tag.sh git-whatchanged.sh \
 	git-applymbox.sh git-applypatch.sh git-am.sh \
 	git-merge.sh git-merge-stupid.sh git-merge-octopus.sh \
 	git-merge-resolve.sh git-merge-ours.sh git-grep.sh
@@ -310,12 +311,12 @@ DEFINES += -DSHA1_HEADER=$(call shellquo
 SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)) \
 	  $(patsubst %.perl,%,$(SCRIPT_PERL)) \
 	  $(patsubst %.py,%,$(SCRIPT_PYTHON)) \
-	  gitk git-cherry-pick
+	  git-cherry-pick
 
 export prefix TAR INSTALL DESTDIR SHELL_PATH template_dir
 ### Build rules
 
-all: $(PROGRAMS) $(SCRIPTS)
+all: git gitk $(PROGRAMS) $(SCRIPTS)
 
 all:
 	$(MAKE) -C templates
@@ -323,6 +324,7 @@ all:
 git: git.sh Makefile
 	rm -f $@+ $@
 	sed -e '1s|#!.*/sh|#!$(call shq,$(SHELL_PATH))|' \
+	    -e 's:@@GITDIR@@:$(gitdir):g' \
 	    -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
 	    -e 's/@@X@@/$(X)/g' \
 	    $(GIT_LIST_TWEAK) <$@.sh >$@+
@@ -410,9 +412,11 @@ check:
 
 ### Installation rules
 
-install: $(PROGRAMS) $(SCRIPTS)
+install: git gitk $(PROGRAMS) $(SCRIPTS)
 	$(INSTALL) -d -m755 $(call shellquote,$(DESTDIR)$(bindir))
-	$(INSTALL) $(PROGRAMS) $(SCRIPTS) $(call shellquote,$(DESTDIR)$(bindir))
+	$(INSTALL) git gitk $(call shellquote,$(DESTDIR)$(bindir))
+	$(INSTALL) -d -m755 $(call shellquote,$(DESTDIR)$(gitdir))
+	$(INSTALL) $(PROGRAMS) $(SCRIPTS) $(call shellquote,$(DESTDIR)$(gitdir))
 	$(MAKE) -C templates install
 	$(INSTALL) -d -m755 $(call shellquote,$(DESTDIR)$(GIT_PYTHON_DIR))
 	$(INSTALL) $(PYMODULES) $(call shellquote,$(DESTDIR)$(GIT_PYTHON_DIR))
@@ -450,7 +454,7 @@ deb: dist
 
 clean:
 	rm -f *.o mozilla-sha1/*.o ppc/*.o compat/*.o $(PROGRAMS) $(LIB_FILE)
-	rm -f $(filter-out gitk,$(SCRIPTS))
+	rm -f $(SCRIPTS)
 	rm -f git-core.spec *.pyc *.pyo
 	rm -rf $(GIT_TARNAME)
 	rm -f $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
diff --git a/git-core.spec.in b/git-core.spec.in
index 5240dd2..0d0ddf3 100644
--- a/git-core.spec.in
+++ b/git-core.spec.in
@@ -19,17 +19,19 @@ distributed source code management syste
 rudimentary tools that can be used as a SCM, but you should look
 elsewhere for tools for ordinary humans layered on top of this.
 
+%define gitdir %{_libdir}/git-@@VERSION@@
+
 %prep
 %setup -q
 
 %build
 make %{_smp_mflags} CFLAGS="$RPM_OPT_FLAGS" WITH_OWN_SUBPROCESS_PY=YesPlease \
-     prefix=%{_prefix} all %{!?_without_docs: doc}
+     gitdir=%{gitdir} prefix=%{_prefix} all %{!?_without_docs: doc}
 
 %install
 rm -rf $RPM_BUILD_ROOT
 make %{_smp_mflags} DESTDIR=$RPM_BUILD_ROOT WITH_OWN_SUBPROCESS_PY=YesPlease \
-     prefix=%{_prefix} mandir=%{_mandir} \
+     gitdir=%{gitdir} prefix=%{_prefix} mandir=%{_mandir} \
      install %{!?_without_docs: install-doc}
 
 %clean
@@ -38,6 +40,7 @@ rm -rf $RPM_BUILD_ROOT
 %files
 %defattr(-,root,root)
 %{_bindir}/*
+%{gitdir}/*
 %{_datadir}/git-core/
 %doc README COPYING Documentation/*.txt
 %{!?_without_docs: %doc Documentation/*.html }
diff --git a/git.sh b/git.sh
index 94940ae..9ba1608 100755
--- a/git.sh
+++ b/git.sh
@@ -1,7 +1,8 @@
 #!/bin/sh
 
 cmd=
-path=$(dirname "$0")
+path="@@GITDIR@@"
+export PATH="$path:$PATH"
 case "$#" in
 0)	;;
 *)	cmd="$1"
Junio C Hamano· Nov 6, 2005, 07:25 UTC · re: Linus Torvalds · lore

Re: git binary directory?

Linus Torvalds <torvalds@osdl.org> writes:
> This fixes the gitk thing, by installing it (along with git) in the 
> regular $prefix/bin directory (ie default /usr/bin for the RPM).

The other half of the fix needed to gitk is to teach it to use /usr/lib/git-@@VERSION@@ to locate git-rev-list and friends.

I really hate to do this, but here is an attempt on top of your patch, which:

 - makes gitk also use the @@GITDIR@@ to find git commands;
 - makes git and gitk honor $GIT_LIBEXEC environment, to let you
   override the hardcoded @@GITDIR@@;
 - makes "make clean" to clean generated file git again.

I am not sure how to handle gitk part, but the best approach would be to sell this change (including the rename) back to Paulus the upstream.

---
diff --git a/Makefile b/Makefile
index 2e78089..1948aa1 100644
--- a/Makefile
+++ b/Makefile
@@ -327,6 +327,15 @@ git: git.sh Makefile
 	chmod +x $@+
 	mv $@+ $@
 
+gitk: gitk.tcl Makefile
+	rm -f $@+ $@
+	sed -e '1s|#!.*/sh|#!$(call shq,$(SHELL_PATH))|' \
+	    -e 's:@@GITDIR@@:$(gitdir):g' \
+	    -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
+	    <$@.tcl >$@+
+	chmod +x $@+
+	mv $@+ $@
+
 $(filter-out git,$(patsubst %.sh,%,$(SCRIPT_SH))) : % : %.sh
 	rm -f $@
 	sed -e '1s|#!.*/sh|#!$(call shq,$(SHELL_PATH))|' \
@@ -450,7 +459,7 @@ deb: dist
 
 clean:
 	rm -f *.o mozilla-sha1/*.o ppc/*.o compat/*.o $(PROGRAMS) $(LIB_FILE)
-	rm -f $(SCRIPTS)
+	rm -f $(SCRIPTS) git gitk
 	rm -f git-core.spec *.pyc *.pyo
 	rm -rf $(GIT_TARNAME)
 	rm -f $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
diff --git a/git.sh b/git.sh
index 9ba1608..f39b373 100755
--- a/git.sh
+++ b/git.sh
@@ -1,7 +1,13 @@
 #!/bin/sh
 
 cmd=
-path="@@GITDIR@@"
+case "$GIT_LIBEXEC" in
+'')
+	GIT_LIBEXEC='@@GITDIR@@'
+	export GIT_LIBEXEC
+	;;
+esac
+path="$GIT_LIBEXEC"
 export PATH="$path:$PATH"
 case "$#" in
 0)	;;
diff --git a/gitk b/gitk.tcl
similarity index 100%
rename from gitk
rename to gitk.tcl
index a9d37d9..be9a79e 100755
--- a/gitk
+++ b/gitk.tcl
@@ -7,6 +7,11 @@ exec wish "$0" -- "$@"
 # and distributed under the terms of the GNU General Public Licence,
 # either version 2, or (at your option) any later version.
 
+if {![info exists env(GIT_LIBEXEC)]} {
+    set env(GIT_LIBEXEC) "@@GITDIR@@"
+}
+set env(PATH) "$env(GIT_LIBEXEC):$env(PATH)"
+
 proc gitdir {} {
     global env
     if {[info exists env(GIT_DIR)]} {
Junio C Hamano· Nov 6, 2005, 02:49 UTC · re: Linus Torvalds · lore

Re: git binary directory?

Linus Torvalds <torvalds@osdl.org> writes:
Show 6 quoted lines
> Now, I happen to think that 2500+ files in /usr/bin is a bit much (ever 
> try to use the horrid gnome executable finder on it when you want to 
> convince firefox to use xpdf instead of that broken crap called "evince"? 
> Takes absolutely ages and is horrible).
>
> And git made it about 4% worse all on its own.

My pragmatic half agrees with what you said. /usr/bin should not contain things that are never used by the end user -- things like git-sh-setup, git-fmt-merge-msg, and git-merge-recursive should not be there. Not having things like git-show-branch and git-update-index in /usr/bin is a regression because it needs an extra fork to call them through 'git' wrapper, but I could live with that.

My purist half, however, says that it is a wrong solution to the problem. If having many files in /usr/bin hurts performance, you should be using a filesystem that handles large directory better. Modern shells already know how to hash command names found in $PATH. It is just your gnome executable finder that is lacking the knowledge of which binaries are appropriate for what mimetype; perhaps your distribution could help by having a way for each package to register programs that handle particular mimetypes well with the system-wide database.

And my lazy remainder (yes, I add up to more then one ;-)) cheers on my purist side.

But common sense prevails at the end of the day. I would not fight a battle I know I would not be able to win. So what should we do about this problem? And when?

Since we do not have enough clout to have /usr/bin/git/ and ask the users to put that in their PATH like X11 does, we need to teach some of our commands that use other git commands to prepend /usr/lib/git/ (or /usr/libexec/git) on their PATH while they run. Although many of the Porcelainish commands include git-sh-setup, git-sh-setup itself is a prime candidate to be kicked out of /usr/bin, which means essentially everything needs to have that PATH trick.

This also is a bit inconvenient for our in-source-tree tests. We need to be testing what we just built and are about to install, not what is already installed, so every script needs to start with something like this:

	#!/bin/sh
	: ${GIT_BIN_DIR=@@GIT_BIN_DIR@@}
        PATH="$GIT_BIN_DIR:$PATH"
        git-sh-setup || die "not a git repository"
	...

and our test will run with GIT_BIN_DIR set to `pwd`/../../ (they run in t/trash and what we just built are found at the toplevel of the source). Also we need to do the same for binaries if they fork/exec other git commands.

Commands like upload-pack and receive-pack are directly executed from the ssh connection, and we need to arrange for them to be found on the PATH of users' non-login shells. This does not necessarily mean these commands need to stay in /usr/bin, but if we move them outside standard PATH, we would need to teach .bash_profile vs .bashrc workaround to all users, which I think is the yuckiest part of all of the above.

Linus Torvalds· Nov 6, 2005, 04:25 UTC · re: Junio C Hamano · lore

Re: git binary directory?

On Sat, 5 Nov 2005, Junio C Hamano wrote:
Show 5 quoted lines
> 
> My purist half, however, says that it is a wrong solution to the
> problem.  If having many files in /usr/bin hurts performance,
> you should be using a filesystem that handles large directory
> better.

I disagree with your arguments, even though I'm not convinced we necessarily need a directory of its own for git.

We have directories for a reason. You might as well argue that everybody should have a flat namespace, since it should be efficient.

The performance reason for directories is only secondary. The _real_ reason for directories is to keep related things together, and track them better. Havign a nice directory structure where programs keep their own files instead of putting them all in the same place is a good thing from an organization standpoint.

> Modern shells already know how to hash command names
> found in $PATH.

Right. And we could add the git directory to the path. In fact, that's exactly what the git wrapper script does: this allows the low-level git shell scripts to avoid havign to go through the wrapper, since they can now use the native programs directly.

(In fact, that PATH part of the patch is probably a bug-fix regardless: if I have two different versions of "git", and I ask for the one that isn't in my path explicitly, then it should use _its_ git programs, not the other versions).

> It is just your gnome executable finder that is
> lacking the knowledge of which binaries are appropriate for what
> mimetype;

The gnome file chooser is horrid, but even in the presense of a _nice_ file manager it's actually not very pleasant to have directories with thousands of files.

Show 5 quoted lines
> Since we do not have enough clout to have /usr/bin/git/ and ask
> the users to put that in their PATH like X11 does, we need to
> teach some of our commands that use other git commands to
> prepend /usr/lib/git/ (or /usr/libexec/git) on their PATH while
> they run.

Did you miss that part of my patch? That's exactly what this hunk of it does:

	diff --git a/git.sh b/git.sh
	index 94940ae..9ba1608 100755
	--- a/git.sh
	+++ b/git.sh
	@@ -1,7 +1,8 @@
	 #!/bin/sh
	 
	 cmd=
	-path=$(dirname "$0")
	+path="@@GITDIR@@"
	+export PATH="$path:$PATH"
	 case "$#" in
	 0)	;;
	 *)	cmd="$1"

and as mentioned, I actually think it's a bugfix regardless of anything else (do a "./git log", and it will _not_ execute "./git-rev-parse" and "./git-rev-list": it will execute whatever was in the path, usually /usr/bin/git-rev-xyzzy).

Now, gitk didn't do that, which is a bug.
> Although many of the Porcelainish commands include
> git-sh-setup, git-sh-setup itself is a prime candidate to be
> kicked out of /usr/bin, which means essentially everything needs
> to have that PATH trick.
Yes. All porcelain would need to do the PATH thing, I think.
> This also is a bit inconvenient for our in-source-tree tests.
> We need to be testing what we just built and are about to
> install, not what is already installed, so every script needs to
> start with something like this:

No, the actual programs and scripts themselves shouldn't change. Nothing that uses git-sh-setup should change, only the programs that are installed in /usr/bin would need to know that the helper programs are _not_ there, and set up the path properly.

But if we do that right, that should be just a couple of executables, which is the whole point of splitting this up.

But yes, you're right that right now we're not always set up for this, and git-upload-pack etc that execute directly from the shell would need help.

		Linus
Junio C Hamano· Nov 6, 2005, 05:37 UTC · re: Linus Torvalds · lore

Re: git binary directory?

Linus Torvalds <torvalds@osdl.org> writes:
Show 8 quoted lines
> We have directories for a reason. You might as well argue that everybody 
> should have a flat namespace, since it should be efficient.
>
> The performance reason for directories is only secondary. The _real_ 
> reason for directories is to keep related things together, and track them 
> better. Havign a nice directory structure where programs keep their own 
> files instead of putting them all in the same place is a good thing from 
> an organization standpoint.

My point (actually, my purist half's point) is that /usr/bin is that nice structure that keeps related things together --- the relatedness of them being "the end user would want to run them". Your initial hesitation that the change being discussed would force you to say "git whatchanged" when you are so accustomed to type git-whatchanged is valid. Unfortunately, we have far more commands in /usr/bin than good old V7 days, and while the _primary_ purpose of /usr/bin is to hold "the end user would want to run them" things together (hence, shell needs to know only about handful places to look at), having thousands of things in one place is inconvenient for purposes other than the primary purpose of that grouping (i.e. running them), such as browsing them.

You could deviate from the UNIX tradition and "keep related things together" in different ways; you _could_ have /usr/bin/pdf-viewers/, /usr/bin/html-viewers/, etc. to hold xpdf, acroread, firefox and iexplorer in them if you do not like /usr/bin/ that has many executables -- but we do not do that.

> (In fact, that PATH part of the patch is probably a bug-fix regardless: if 
> I have two different versions of "git", and I ask for the one that isn't 
> in my path explicitly, then it should use _its_ git programs, not the 
> other versions).
I think that is a valid change.
> The gnome file chooser is horrid, but even in the presense of a _nice_ 
> file manager it's actually not very pleasant to have directories with 
> thousands of files. 
Yes, but I think we should blame UNIX tradition for that ;-).
> Did you miss that part of my patch?

Sorry, my reply was prepared before I actually saw that patch (you would notice it was a reply to your first message). I think what you did in your patch makes sense. The end users should always say 'git update-index' (or know the lib/git path and it they want to say git-update-index), even when running the low-level commands, and as long as they use 'git' wrapper that approach would work.

> Yes. All porcelain would need to do the PATH thing, I think.

No, we could just say 'git commit' and 'git' is the only thing that needs to know the PATH thing, as you did.

> But yes, you're right that right now we're not always set up for this, and 
> git-upload-pack etc that execute directly from the shell would need help.
We _could_ even invoke 'git upload-pack' from 'git-fetch'.
Linus Torvalds· Nov 6, 2005, 16:20 UTC · re: Junio C Hamano · lore

Re: git binary directory?

On Sat, 5 Nov 2005, Junio C Hamano wrote:
> 
> My point (actually, my purist half's point) is that /usr/bin is
> that nice structure that keeps related things together --- the
> relatedness of them being "the end user would want to run them".
Yes. I wish there was some way around that.

Right now, for a 1.0 release, I suspect that the "put the git binaries somewhere else" just isn't worth it. It will break existing scripts that use the binaries directly (we've already broken the kernel.org snapshot scripts about a million times with just _renaming_ the binaries ;)

It would still be nice to not screw up peoples /usr/bin too badly. At least we have the nice property that our git programs sort together and can pretty much be wild-carded (not everybody uses package installers, and on one machine I had just done "make prefix=/usr install" and was happy to be able to basically remove it with "rm /usr/bin/git-*")

			Linus
Junio C Hamano· Nov 6, 2005, 20:15 UTC · re: Linus Torvalds · lore

Re: git binary directory?

Linus Torvalds <torvalds@osdl.org> writes:
> Right now, for a 1.0 release, I suspect that the "put the git binaries 
> somewhere else" just isn't worth it. It will break existing scripts that 
> use the binaries directly (we've already broken the kernel.org snapshot 
> scripts about a million times with just _renaming_ the binaries ;)

Although I _really_ wanted a 1.0 soonish, I personally feel that this change is better done now than later, if we are eventually going to do it anyway. "Never" _might_ be better than "now", but I suspect "later" or "post 1.0" is worse.

Petr Baudis· Nov 6, 2005, 22:19 UTC · re: Junio C Hamano · lore

Re: git binary directory?

Dear diary, on Sun, Nov 06, 2005 at 09:15:38PM CET, I got a letter where Junio C Hamano <junkio@cox.net> told me that...

Show 11 quoted lines
> Linus Torvalds <torvalds@osdl.org> writes:
> 
> > Right now, for a 1.0 release, I suspect that the "put the git binaries 
> > somewhere else" just isn't worth it. It will break existing scripts that 
> > use the binaries directly (we've already broken the kernel.org snapshot 
> > scripts about a million times with just _renaming_ the binaries ;)
> 
> Although I _really_ wanted a 1.0 soonish, I personally feel that
> this change is better done now than later, if we are eventually
> going to do it anyway.  "Never" _might_ be better than "now",
> but I suspect "later" or "post 1.0" is worse.

You are also going to break the porcelains (w/o manual user intervention), so I'm not happy about it but if you are doing it, do it now, please. :-)

BTW, can I easily get the patch from the 'git' tool, so that I can extend $PATH appropriately during Cogito initialization?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
Junio C Hamano· Nov 7, 2005, 00:08 UTC · re: Petr Baudis · lore

Re: git binary directory?

Petr Baudis <pasky@suse.cz> writes:
> You are also going to break the porcelains (w/o manual user
> intervention), so I'm not happy about it but if you are doing it, do it
> now, please. :-)

I thought Porcelains and people's script can just say 'git foo' everywhere where it currently says 'git-foo', and it should work before and after the change, and that is what I did to my private scripts. But you are right, it is nicer if 'git-foo' continued to work with a single patch to cg-Xlib or somewhere central like that.

> BTW, can I easily get the patch from the 'git' tool, so that I can
> extend $PATH appropriately during Cogito initialization?
Do you mean 'git --show-git-libdir' or something like that?
Petr Baudis· Nov 7, 2005, 00:43 UTC · re: Junio C Hamano · lore

Re: git binary directory?

Dear diary, on Mon, Nov 07, 2005 at 01:08:57AM CET, I got a letter where Junio C Hamano <junkio@cox.net> told me that...

Show 12 quoted lines
> Petr Baudis <pasky@suse.cz> writes:
> 
> > You are also going to break the porcelains (w/o manual user
> > intervention), so I'm not happy about it but if you are doing it, do it
> > now, please. :-)
> 
> I thought Porcelains and people's script can just say 'git foo'
> everywhere where it currently says 'git-foo', and it should work
> before and after the change, and that is what I did to my
> private scripts.  But you are right, it is nicer if 'git-foo'
> continued to work with a single patch to cg-Xlib or somewhere
> central like that.

I want to avoid extra fork()s and exec()s. They seem to routinely matter in orders of magnitude of speed in tight loops.

Besides, I'm more used to the dash form. ;-)
> > BTW, can I easily get the patch from the 'git' tool, so that I can
> > extend $PATH appropriately during Cogito initialization?
> 
> Do you mean 'git --show-git-libdir' or something like that?

Yes. Well... you said you have the gitdir changes in your 'pu' branch, but http://www.kernel.org/git/?p=git/git.git;a=tree;h=pu;hb=pu doesn't appear to have the changes...?

Is --show-git-libdir already implemented?
-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
Linus Torvalds· Nov 7, 2005, 00:54 UTC · re: Petr Baudis · lore

Re: git binary directory?

On Mon, 7 Nov 2005, Petr Baudis wrote:
> 
> I want to avoid extra fork()s and exec()s. They seem to routinely matter
> in orders of magnitude of speed in tight loops.

Yes. The more I think about it, the less I like the separate binary directory after all. The "git cmd" format is great for high-level commands, but we've always done "git-diff-tree" and "git-rev-list" etc without the "git cmd" indirection.

The downsides of a separate binary directory appear to be bigger than the upside..

		Linus
Petr Baudis· Nov 7, 2005, 09:45 UTC · re: Linus Torvalds · lore

Re: git binary directory?

Dear diary, on Mon, Nov 07, 2005 at 01:54:55AM CET, I got a letter where Linus Torvalds <torvalds@osdl.org> told me that...

Show 12 quoted lines
> On Mon, 7 Nov 2005, Petr Baudis wrote:
> > 
> > I want to avoid extra fork()s and exec()s. They seem to routinely matter
> > in orders of magnitude of speed in tight loops.
> 
> Yes. The more I think about it, the less I like the separate binary 
> directory after all. The "git cmd" format is great for high-level 
> commands, but we've always done "git-diff-tree" and "git-rev-list" etc 
> without the "git cmd" indirection.
> 
> The downsides of a separate binary directory appear to be bigger than the 
> upside..

Well, _one_ fork per user invocation (subsequent invocations wouldn't need it, I already "cache" this kind of stuff in the environment) of a Cogito command (to get the path to the gitdir) is not such a big deal, I think.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
Junio C Hamano· Nov 7, 2005, 17:27 UTC · re: Linus Torvalds · lore

Re: git binary directory?

Linus Torvalds <torvalds@osdl.org> writes:
Show 9 quoted lines
> On Mon, 7 Nov 2005, Petr Baudis wrote:
>> 
>> I want to avoid extra fork()s and exec()s. They seem to routinely matter
>> in orders of magnitude of speed in tight loops.
>
> Yes. The more I think about it, the less I like the separate binary 
> directory after all. The "git cmd" format is great for high-level 
> commands, but we've always done "git-diff-tree" and "git-rev-list" etc 
> without the "git cmd" indirection.

Extra fork() and exec() problem still exists, but "git cmd" format have worked with "git diff-tree" and "git rev-list" for quite some time (since Jun 14th), so you might've always done it with dash but that is not a requirement.

It looks like that both you and Pasky are not so enthusiastic about moving thing out of /usr/bin/, so let's keep things the way they are for 1.0 release and rethink when we do 2.0 (or whatever comes after 1.0).

Petr Baudis· Nov 10, 2005, 09:49 UTC · re: Junio C Hamano · lore

Re: git binary directory?

Dear diary, on Mon, Nov 07, 2005 at 06:27:48PM CET, I got a letter where Junio C Hamano <junkio@cox.net> said that...

> It looks like that both you and Pasky are not so enthusiastic
> about moving thing out of /usr/bin/, so let's keep things the
> way they are for 1.0 release and rethink when we do 2.0 (or
> whatever comes after 1.0).

Well, just to clarify - I'm personally not opposed to it, as long as it gets done in some timeframe where it won't drastically break backwards compatibility (that is, not in 1.0.1 or such, but e.g. at some right moment before 1.0 or at the start of the next development series whenever that happens), and as long as I get a way to get the path of the git commands by a single fork().

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
Andreas Ericsson· Nov 9, 2005, 13:32 UTC · re: Petr Baudis · lore

Re: git binary directory?

Petr Baudis wrote:
> 
> I want to avoid extra fork()s and exec()s. They seem to routinely matter
> in orders of magnitude of speed in tight loops.
> 

Give this a whirl. It's still slower than using the git-<something> form, but it's quite an improvement anyways.

#############
$ time for i in `seq 1 10000`; do git send-pack --help >/dev/null 2>&1; done

real 0m28.921s user 0m12.626s sys 0m14.219s $ time for i in `seq 1 10000`; do ./git send-pack --help >/dev/null 2>&1; done

real 0m12.129s user 0m4.444s sys 0m7.615s $ time for i in `seq 1 10000`; do git-send-pack --help >/dev/null 2>&1; done

real 0m9.069s user 0m3.481s sys 0m5.558s

##############
-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231


#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <limits.h>
#include <stdarg.h>
#include <glob.h>

#ifndef PATH_MAX
# define PATH_MAX 4096
#endif

static const char git_usage[] =
	"Usage: git [ --version ] [ --lib=<GIT_LIB> ] COMMAND [ OPTIONS ] [ TARGET ]";

struct string_list {
	size_t len;
	char *str;
	struct string_list *next;
};

/* most gui terms set COLUMNS (although some don't export it) */
static int columns(void)
{
	char *col_string = getenv("COLUMNS");
	int n_cols = 0;

	if (col_string && (n_cols = atoi(col_string)) > 0)
		return n_cols;

	return 80;
}

static inline void mput_char(char c, unsigned int num)
{
	unsigned int i;

	for(i = 0; i < num; i++)
		putchar(c);
}

static void fmt_print_string_list(struct string_list *list, int longest)
{
	int cols;
	int space = longest + 1; /* space between start of string1 and string2 */
	int max_cols = columns() - 1;

	cols = max_cols / space;

	if(cols < 1) {
		cols = 1;
		space = 0;
	}

	while (list) {
		int c = cols;
		printf("  ");

		for(c = cols; c; c--) {
			if (!list)
				break;

			printf("%s", list->str);

			if (space && c != 1)
				mput_char(' ', space - list->len);

			list = list->next;
		}

		putchar('\n');
	}
}

static void usage(char *path, const char *fmt, ...)
	__attribute__((__format__(__printf__, 2, 3)));

static void usage(char *path, const char *fmt, ...)
{
	struct string_list *list, *tail;
	unsigned int longest = 0, i;
	glob_t gl;

	list = tail = NULL;

	if (!fmt)
		puts(git_usage);
	else {
		printf("git: ");
		va_list ap;
		va_start(ap, fmt);
		vprintf(fmt, ap);
		va_end(ap);
	}

	putchar('\n');

	if (!path)
		exit(1);

	if (chdir(path) < 0) {
		printf("git: '%s': %s\n", path, strerror(errno));
		exit(1);
	}

	printf("\ngit commands available in '%s'\n", path);
	printf("----------------------------");
	mput_char('-', strlen(path));
	putchar('\n');

	glob("git-*", 0, NULL, &gl);
	for (i = 0; i < gl.gl_pathc; i++) {
		int len = strlen(gl.gl_pathv[i] + 4);

		if(access(gl.gl_pathv[i], X_OK))
			continue;

		if (longest < len)
			longest = len;

		if (!tail)
			tail = list = malloc(sizeof(struct string_list));
		else {
			tail->next = malloc(sizeof(struct string_list));
			tail = tail->next;
		}
		tail->len = len;
		tail->str = gl.gl_pathv[i] + 4;
		tail->next = NULL;
	}

	fmt_print_string_list(list, longest);

	puts("\nman-pages can be reached through 'man git-<COMMAND>'\n");

	exit(1);
}

#define DEFAULT_GIT_LIB "/usr/bin"
#define GIT_VERSION "0.99.9.GIT"

int main(int argc, char **argv, char **envp)
{
	char git_command[PATH_MAX + 1];
	char *git_lib = getenv("GIT_LIB");
	char wd[PATH_MAX + 1], libdir[PATH_MAX + 1];

	getcwd(wd, PATH_MAX);

	if(argc == 1) {
		if (git_lib)
			usage(git_lib, NULL);

		usage(DEFAULT_GIT_LIB, NULL);
	}

	argv++;
	/* always *argv since we inc it if we hit an option */
	while (argc-- && !strncmp(*argv, "--", 2)) {
		char *arg = (*argv++) + 2;

		if (!strncmp(arg, "lib=", 3))
			git_lib = arg + 4;
		else if (!strncmp(arg, "version", 7)) {
			printf("git version %s\n", GIT_VERSION);
			exit(0);
		}
		else
			usage (NULL, NULL);
	}

	if (!git_lib)
		git_lib = DEFAULT_GIT_LIB;

	/* allow relative paths, but run with exact */
	if (chdir(git_lib)) {
		printf("git: '%s': %s\n", git_lib, strerror(errno));
		exit (1);
	}

	getcwd(libdir, sizeof(libdir));
	chdir(wd);

	snprintf(git_command, sizeof(git_command), "%s/git-%s", libdir, *argv);

	if (access(git_command, X_OK))
		usage(git_lib, "'%s' is not a git-command", *argv);

	if (execve(git_command, argv, envp)) {
		printf("Failed to run command '%s': %s\n", git_command, strerror(errno));
		return 1;
	}

	/* not reached */

	return 0;
}

← back to recent threads