threads / patch / 2434

patchC implementation of the 'git' program.

Subject: [PATCH] C implementation of the 'git' program.

## tl;dr

9 messages between Nov 10, 2005 and Nov 11, 2005. Diffs are folded; open one to read it.

replies: 8people: 6as markdown or json

Andreas Ericsson· Nov 10, 2005, 18:26 UTC · lore
Subject: [PATCH] C implementation of the 'git' program.

This patch provides a C implementation of the 'git' program and introduces support for putting the git-* commands in their own directory. It also saves some time on executing git-* commands in a tight loop and it prints the currently available git commands in a nicely formatted list.

The location of the GIT_LIB can be obtained by running
	git --lib

which will hopefully give porcelainistas ample time to adapt their heavy-duty loops to use the programs in the library directly if they wish, although that's strictly speaking not really necessary any more.

Some timing values for 10000 invocations of git-var >&/dev/null; git.sh: 24.194s git.c: 9.044s git-var: 7.377s

The git-<tab><tab> behaviour can still be obtained by adding
	PATH=$PATH:$(git --lib)
to one's .profile.

Experimental libraries can be used by either setting the environment variable GIT_LIB, or by using

	git --lib=/some/experimental/lib
As a nice side-effect, the patch also fixes the problem with
	make
	make prefix=/usr install
Signed-off-by: Andreas Ericsson <ae@op5.se>
---
 Documentation/git.txt |    6 +
 Makefile              |   24 ++---
 git.c                 |  224 +++++++++++++++++++++++++++++++++++++++++++++++++
 git.sh                |   76 -----------------
 4 files changed, 239 insertions(+), 91 deletions(-)
 create mode 100644 git.c
 delete mode 100755 git.sh

applies-to: 6466c53ae80cddbb581c5fdb2332f9321fade867 b2b97ed9901915dce573516e3a6f10a60bc0c04a

Show changes to 4 files +239 −91

Documentation/git.txt, Makefile, git.c, git.sh

diff --git a/Documentation/git.txt b/Documentation/git.txt
index 2f9622f..848d09a 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -8,7 +8,7 @@ git - the stupid content tracker
 
 SYNOPSIS
 --------
-'git-<command>' <args>
+'git' [--version] [--lib[=GIT_LIB]] [--help] <command> <args>
 
 DESCRIPTION
 -----------
@@ -535,7 +535,9 @@ include::../README[]
 
 Author
 ------
-Written by Linus Torvalds <torvalds@osdl.org> and the git-list <git@vger.kernel.org>.
+git's founding father is Linus Torvalds <torvalds@osdl.org>.
+The current git nurse is Junio C. Hamano <junkio@cox.net>.
+Numerous contributions stem from the git-list <git@vger.kernel.org>.
 
 Documentation
 --------------
diff --git a/Makefile b/Makefile
index b202be1..80d865b 100644
--- a/Makefile
+++ b/Makefile
@@ -59,6 +59,8 @@ ALL_CFLAGS = $(CFLAGS)
 
 prefix = $(HOME)
 bindir = $(prefix)/bin
+# XXX set libdir to bindir for now to not break the porcelain
+libdir = $(bindir)
 template_dir = $(prefix)/share/git-core/templates/
 GIT_PYTHON_DIR = $(prefix)/share/git-core/python
 # DESTDIR=
@@ -86,7 +88,7 @@ SCRIPT_SH = \
 	git-prune.sh git-pull.sh git-push.sh git-rebase.sh \
 	git-repack.sh git-request-pull.sh git-reset.sh \
 	git-resolve.sh git-revert.sh git-sh-setup.sh git-status.sh \
-	git-tag.sh git-verify-tag.sh git-whatchanged.sh git.sh \
+	git-tag.sh git-verify-tag.sh git-whatchanged.sh \
 	git-applymbox.sh git-applypatch.sh git-am.sh \
 	git-merge.sh git-merge-stupid.sh git-merge-octopus.sh \
 	git-merge-resolve.sh git-merge-ours.sh git-grep.sh
@@ -323,19 +325,13 @@ SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)
 export prefix TAR INSTALL DESTDIR SHELL_PATH template_dir
 ### Build rules
 
-all: $(PROGRAMS) $(SCRIPTS)
+all: $(PROGRAMS) $(SCRIPTS) git
 
-all:
-	$(MAKE) -C templates
+git: git.c
+	$(CC) $(CFLAGS) $< -o $@ -DDEFAULT_GIT_LIB='"$(libdir)"' -DGIT_VERSION='"$(GIT_VERSION)"'
 
-git: git.sh Makefile
-	rm -f $@+ $@
-	sed -e '1s|#!.*/sh|#!$(call shq,$(SHELL_PATH))|' \
-	    -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
-	    -e 's/@@X@@/$(X)/g' \
-	    $(GIT_LIST_TWEAK) <$@.sh >$@+
-	chmod +x $@+
-	mv $@+ $@
+# Force rebuild so that "make prefix=<something> install" Just Works.
+.PHONY: git
 
 $(filter-out git,$(patsubst %.sh,%,$(SCRIPT_SH))) : % : %.sh
 	rm -f $@
@@ -420,8 +416,10 @@ check:
 ### Installation rules
 
 install: $(PROGRAMS) $(SCRIPTS)
+	$(INSTALL) -d -m755 $(call shellquote,$(DESTDIR)$(libdir))
+	$(INSTALL) $(PROGRAMS) $(SCRIPTS) $(call shellquote,$(DESTDIR)$(libdir))
 	$(INSTALL) -d -m755 $(call shellquote,$(DESTDIR)$(bindir))
-	$(INSTALL) $(PROGRAMS) $(SCRIPTS) $(call shellquote,$(DESTDIR)$(bindir))
+	$(INSTALL) git $(call shellquote,$(DESTDIR)$(bindir))
 	$(MAKE) -C templates install
 	$(INSTALL) -d -m755 $(call shellquote,$(DESTDIR)$(GIT_PYTHON_DIR))
 	$(INSTALL) $(PYMODULES) $(call shellquote,$(DESTDIR)$(GIT_PYTHON_DIR))
diff --git a/git.c b/git.c
new file mode 100644
index 0000000..8c01723
--- /dev/null
+++ b/git.c
@@ -0,0 +1,224 @@
+#include <stdio.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <string.h>
+#include <errno.h>
+#include <limits.h>
+#include <stdarg.h>
+#include <glob.h>
+
+#ifndef PATH_MAX
+# define PATH_MAX 4096
+#endif
+
+static const char git_usage[] =
+	"Usage: git [--version] [--lib[=<GIT_LIB>]] COMMAND [ OPTIONS ]";
+
+struct string_list {
+	size_t len;
+	char *str;
+	struct string_list *next;
+};
+
+/* most gui terms set COLUMNS (although some don't export it) */
+static int columns(void)
+{
+	char *col_string = getenv("COLUMNS");
+	int n_cols = 0;
+
+	if (col_string && (n_cols = atoi(col_string)) > 0)
+		return n_cols;
+
+	return 80;
+}
+
+static inline void mput_char(char c, unsigned int num)
+{
+	unsigned int i;
+
+	for(i = 0; i < num; i++)
+		putchar(c);
+}
+
+static void fmt_print_string_list(struct string_list *list, int longest)
+{
+	int cols;
+	int space = longest + 1; /* space between start of string1 and string2 */
+	int max_cols = columns() - 1;
+
+	cols = max_cols / space;
+
+	if(cols < 1) {
+		cols = 1;
+		space = 0;
+	}
+
+	while (list) {
+		int c = cols;
+		printf("  ");
+
+		for(c = cols; c; c--) {
+			if (!list)
+				break;
+
+			printf("%s", list->str);
+
+			if (space && c != 1)
+				mput_char(' ', space - list->len);
+
+			list = list->next;
+		}
+
+		putchar('\n');
+	}
+}
+
+#ifdef __GNUC__
+static void usage(char *path, const char *fmt, ...)
+	__attribute__((__format__(__printf__, 2, 3), __noreturn__));
+#endif
+
+static void usage(char *path, const char *fmt, ...)
+{
+	struct string_list *list, *tail;
+	unsigned int longest = 0, i;
+	glob_t gl;
+
+	list = tail = NULL;
+
+	if (!fmt)
+		puts(git_usage);
+	else {
+		printf("git: ");
+		va_list ap;
+		va_start(ap, fmt);
+		vprintf(fmt, ap);
+		va_end(ap);
+		putchar('\n');
+	}
+
+	putchar('\n');
+
+	if (!path)
+		exit(1);
+
+	if (chdir(path) < 0) {
+		printf("git: '%s': %s\n", path, strerror(errno));
+		exit(1);
+	}
+
+	i = glob("git-*", 0, NULL, &gl);
+	if (i) {
+		switch(i) {
+		case GLOB_NOSPACE:
+			puts("Out of memory while glob()'ing commands");
+			exit(2);
+		case GLOB_ABORTED:
+			printf("'%s': Read error: %s\n", path, strerror(errno));
+			exit(2);
+		case GLOB_NOMATCH:
+			printf("No git commands available in '%s'.\n", path);
+			printf("Do you need to specify --lib or set GIT_LIB?\n");
+			exit(1);
+		}
+	}
+
+	for (i = 0; i < gl.gl_pathc; i++) {
+		int len = strlen(gl.gl_pathv[i] + 4);
+
+		if(access(gl.gl_pathv[i], X_OK))
+			continue;
+
+		if (longest < len)
+			longest = len;
+
+		if (!tail)
+			tail = list = malloc(sizeof(struct string_list));
+		else {
+			tail->next = malloc(sizeof(struct string_list));
+			tail = tail->next;
+		}
+		tail->len = len;
+		tail->str = gl.gl_pathv[i] + 4;
+		tail->next = NULL;
+	}
+
+	printf("git commands available in '%s'\n", path);
+	printf("----------------------------");
+	mput_char('-', strlen(path));
+	putchar('\n');
+	fmt_print_string_list(list, longest);
+
+	puts("\nman-pages can be reached through 'man git-<COMMAND>'\n");
+
+	exit(1);
+}
+
+int main(int argc, char **argv, char **envp)
+{
+	char git_command[PATH_MAX + 1];
+	char wd[PATH_MAX + 1];
+	char *git_lib = getenv("GIT_LIB");
+	int i, len;
+
+	getcwd(wd, PATH_MAX);
+
+	if (!git_lib)
+		git_lib = DEFAULT_GIT_LIB;
+
+	if (argc == 1)
+		usage(git_lib, NULL);
+
+	for (i = 1; i < argc; i++) {
+		char *arg = argv[i];
+
+		if (strncmp(arg, "--", 2))
+			break;
+
+		arg += 2;
+
+		if (!strncmp(arg, "lib", 3)) {
+			arg += 3;
+			if (*arg == '=')
+				git_lib = arg + 1;
+			else {
+				puts(git_lib);
+				exit(0);
+			}
+		}
+		else if (!strcmp(arg, "version")) {
+			printf("git version %s\n", GIT_VERSION);
+			exit(0);
+		}
+		else if (!strcmp(arg, "help"))
+			usage(git_lib, NULL);
+		else
+			usage(NULL, NULL);
+	}
+
+	if (i >= argc)
+		usage(NULL, NULL);
+
+	/* allow relative paths, but run with exact */
+	if (chdir(git_lib)) {
+		printf("git: '%s': %s\n", git_lib, strerror(errno));
+		exit (1);
+	}
+
+	getcwd(git_command, sizeof(git_command));
+	chdir(wd);
+
+	len = strlen(git_command);
+	strncat(&git_command[len], "/git-", sizeof(git_command) - len);
+	len += 5;
+	strncat(&git_command[len], argv[i], sizeof(git_command) - len);
+
+	if (access(git_command, X_OK))
+		usage(git_lib, "'%s' is not a git-command", argv[i]);
+
+	/* execve() can only ever return if it fails */
+	execve(git_command, &argv[i], envp);
+	printf("Failed to run command '%s': %s\n", git_command, strerror(errno));
+
+	return 1;
+}
diff --git a/git.sh b/git.sh
deleted file mode 100755
index 94940ae..0000000
--- a/git.sh
+++ /dev/null
@@ -1,76 +0,0 @@
-#!/bin/sh
-
-cmd=
-path=$(dirname "$0")
-case "$#" in
-0)	;;
-*)	cmd="$1"
-	shift
-	case "$cmd" in
-	-v|--v|--ve|--ver|--vers|--versi|--versio|--version)
-		echo "git version @@GIT_VERSION@@"
-		exit 0 ;;
-	esac
-	
-	test -x "$path/git-$cmd" && exec "$path/git-$cmd" "$@"
-	
-	case '@@X@@' in
-	    '')
-		;;
-	    *)
-		test -x "$path/git-$cmd@@X@@" &&
-		exec "$path/git-$cmd@@X@@" "$@"
-		;;
-	esac
-	;;
-esac
-
-echo "Usage: git COMMAND [OPTIONS] [TARGET]"
-if [ -n "$cmd" ]; then
-    echo "git command '$cmd' not found."
-fi
-echo "git commands are:"
-
-fmt <<\EOF | sed -e 's/^/    /'
-add
-apply
-archimport
-bisect
-branch
-checkout
-cherry
-clone
-commit
-count-objects
-cvsimport
-diff
-fetch
-format-patch
-fsck-objects
-get-tar-commit-id
-init-db
-log
-ls-remote
-octopus
-pack-objects
-parse-remote
-patch-id
-prune
-pull
-push
-rebase
-relink
-rename
-repack
-request-pull
-reset
-resolve
-revert
-send-email
-shortlog
-show-branch
-status
-tag
-verify-tag
-whatchanged
-EOF
---
0.99.9.GIT
Linus Torvalds· Nov 10, 2005, 19:41 UTC · re: Andreas Ericsson · lore

Re: [PATCH] C implementation of the 'git' program.

On Thu, 10 Nov 2005, Andreas Ericsson wrote:
Show 5 quoted lines
> 
> This patch provides a C implementation of the 'git' program and introduces
> support for putting the git-* commands in their own directory. It also
> saves some time on executing git-* commands in a tight loop and it prints
> the currently available git commands in a nicely formatted list.

I think this makes sense - "git" may be a very simple program even in shell, but hey, it's simple even in C, and while performance may be a secondary issue, when it is in C we will have the option to extend in more effectively later.

For example, we might implement some things directly in the "git" wrapper.
And the performance difference does seem to be quite noticeable too..
> The location of the GIT_LIB can be obtained by running
> 
> 	git --lib

I think this might be a bit ambiguous. When I see "GIT_LIB", to me it implies traditional libraries (ie a "libgit.a" kind of thing), not the kind of "git executable plugin" directory.

So I'd suggest renaming "--lib" and "GIT_LIB" to be more of a "--libexec" kind of flavor, if only to avoid that confusion. Even if the actual directory ends up being /usr/lib/git-<version>/ as in my original suggestion, I think it's good to make it clear that it's not just some kind of "link type" library, but more of a combination of libraries and plugins and executables.

But maybe that's just me?
		Linus
Andreas Ericsson· Nov 10, 2005, 22:50 UTC · re: Linus Torvalds · lore

Re: [PATCH] C implementation of the 'git' program.

Linus Torvalds wrote:
> 
> And the performance difference does seem to be quite noticeable too..
> 

Yes. I was quite astonished when I noticed first, thinking the shell kept the parsed script in cache or some such. Apparently it doesn't.

Show 12 quoted lines
> 
>>The location of the GIT_LIB can be obtained by running
>>
>>	git --lib
> 
> 
> I think this might be a bit ambiguous. When I see "GIT_LIB", to me it 
> implies traditional libraries (ie a "libgit.a" kind of thing), not the 
> kind of "git executable plugin" directory.
> 
> So I'd suggest renaming "--lib" and "GIT_LIB" to be more of a "--libexec" 
> kind of flavor, if only to avoid that confusion.

Someone said libexec was moving out (of Linux, at least), so I thought I'd better avoid that. Perhaps GIT_LIBDIR?

Btw, I re-visited ye ole "git binary directory?" thread and noticed the nifty trick you did with PATH from the wrapper. I'll add that to git.c and, unless I get a better suggestion, rename GIT_LIB to GIT_LIBDIR (and git --libdir).

Show 8 quoted lines
> Even if the actual 
> directory ends up being /usr/lib/git-<version>/ as in my original 
> suggestion, I think it's good to make it clear that it's not just some 
> kind of "link type" library, but more of a combination of libraries and 
> plugins and executables.
> 
> But maybe that's just me?
> 
It's not just you, but GIT_DIR was taken and I felt lazy.
-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231
Petr Baudis· Nov 10, 2005, 23:37 UTC · re: Andreas Ericsson · lore

Re: [PATCH] C implementation of the 'git' program.

Dear diary, on Thu, Nov 10, 2005 at 11:50:16PM CET, I got a letter where Andreas Ericsson <ae@op5.se> said that...

Show 7 quoted lines
> Linus Torvalds wrote:
> >
> >And the performance difference does seem to be quite noticeable too..
> >
> 
> Yes. I was quite astonished when I noticed first, thinking the shell 
> kept the parsed script in cache or some such. Apparently it doesn't.

The bulk of the time is likely not spent parsing, but executing subprocesses, that's hideously expensive. Just for fun, you can try to measure the improvement when you remove the $(dirname "$0"). Here, for 1000 tries it's roughly 9s normal, 7s without dirname and 3s direct invocation, which seems to give you _very roughly_ 2s per subprocess, and 2s other shell overhead, which is unusually right.

Actually, I can bring the git.sh runtime from 7s to 4.7s:
-       case "$cmd" in
-       -v|--v|--ve|--ver|--vers|--versi|--versio|--version)
+       if [[ "$cmd" = "-v" ||
+             "$cmd" = "--v" ||
+             "$cmd" = "--ve" ||
+             "$cmd" = "--ver" ||
+             "$cmd" = "--vers" ||
+             "$cmd" = "--versi" ||
+             "$cmd" = "--versio" ||
+             "$cmd" = "--version" ]]; then
                echo "git version @@GIT_VERSION@@"
                exit 0 ;;
-       esac
+       fi
				(whitespace-mangled)
Well, subconsciously I never really trusted this case thing. ;-)

This leaves ~ 1.7s to other shell overhead and execve() (the main command call is without the fork(), while $(dirname) fork()s).

Show 15 quoted lines
> >>The location of the GIT_LIB can be obtained by running
> >>
> >>	git --lib
> >
> >
> >I think this might be a bit ambiguous. When I see "GIT_LIB", to me it 
> >implies traditional libraries (ie a "libgit.a" kind of thing), not the 
> >kind of "git executable plugin" directory.
> >
> >So I'd suggest renaming "--lib" and "GIT_LIB" to be more of a "--libexec" 
> >kind of flavor, if only to avoid that confusion.
> 
> 
> Someone said libexec was moving out (of Linux, at least), so I thought 
> I'd better avoid that. Perhaps GIT_LIBDIR?

This may not necessarily have anything in common with the actual directory name. I prefer libexec too (but personally don't care too much).

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
Junio C Hamano· Nov 11, 2005, 19:31 UTC · re: Andreas Ericsson · lore

Re: [PATCH] C implementation of the 'git' program.

Andreas Ericsson <ae@op5.se> writes:
Show 10 quoted lines
> Linus Torvalds wrote:
>
>> I think this might be a bit ambiguous. When I see "GIT_LIB", to me
>> it implies traditional libraries (ie a "libgit.a" kind of thing),
>> not the kind of "git executable plugin" directory.
>> So I'd suggest renaming "--lib" and "GIT_LIB" to be more of a
>> "--libexec" kind of flavor, if only to avoid that confusion.
>
> Someone said libexec was moving out (of Linux, at least), so I thought 
> I'd better avoid that. Perhaps GIT_LIBDIR?

More like "the rest of git executables". GIT_EXEC_PATH, with "git --exec-path" command, perhaps?

Raja R Harinath· Nov 11, 2005, 06:58 UTC · re: Andreas Ericsson · lore

Re: [PATCH] C implementation of the 'git' program.

Hi,
exon@op5.se (Andreas Ericsson) writes:
[snip]
Show 5 quoted lines
> As a nice side-effect, the patch also fixes the problem with
> 	make
> 	make prefix=/usr install
>
> Signed-off-by: Andreas Ericsson <ae@op5.se>
[snip]
> +# Force rebuild so that "make prefix=<something> install" Just Works.
> +.PHONY: git
This is bad, since it does the wrong thing for the correct way:
  make prefix=/usr
  sudo make prefix=/usr install
With your patch 'git' will be compiled as root.
- Hari
Junio C Hamano· Nov 11, 2005, 07:40 UTC · re: Raja R Harinath · lore

Re: [PATCH] C implementation of the 'git' program.

Raja R Harinath <rharinath@novell.com> writes:
Show 6 quoted lines
> This is bad, since it does the wrong thing for the correct way:
>
>   make prefix=/usr
>   sudo make prefix=/usr install
>
> With your patch 'git' will be compiled as root.
You are correct.

We should allow building as user and installing as root, with the installation step _not_ touching the build directory at all. A good litmus test is that you should be able to build and install from a root-squashed NFS mounted working tree, without giving write permissions to NFS-nobody user (obviously you need to give read and search permissions).

Andreas?
Andreas Ericsson· Nov 11, 2005, 12:04 UTC · re: Junio C Hamano · lore

Re: [PATCH] C implementation of the 'git' program.

Junio C Hamano wrote:
Show 15 quoted lines
> Raja R Harinath <rharinath@novell.com> writes:
> 
> 
>>This is bad, since it does the wrong thing for the correct way:
>>
>>  make prefix=/usr
>>  sudo make prefix=/usr install
>>
>>With your patch 'git' will be compiled as root.
> 
> 
> You are correct.
> 
> We should allow building as user and installing as root, with
> the installation step _not_ touching the build directory at all.

Sane point. This doesn't work without a ./configure script or some such though (something external to the Makefile that updates a config-file of some sort), as

make (make finishes, user remembers he/she must use prefix= to install somewhere) make prefix=/usr

wouldn't rebuild git so binaries produced would still try to use $HOME/bin for libdir. Granted, it's easy enough to do 'touch git.c' before rebuilding, but it's not exactly intuitive.

I'll hack up a configure-script if that's alright. It should be fairly short and just define the GIT_VERSION and GIT_LIBDIR macros.

I imagine this can be done through the config.mak file, included from 221 in Makefile, supposing it's forcibly included rather than just if it exists, and we'll have the 'git:' target depend on it. I think this is the least intrusive change. OTOH, it might not be bad to move some of the uname_? logic to a shell-script (and have all targets depend on config.mak).

Thoughts? I've got a feeling I've overlooked something here.
-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231
Junio C Hamano· Nov 11, 2005, 19:37 UTC · re: Andreas Ericsson · lore

Re: [PATCH] C implementation of the 'git' program.

Andreas Ericsson <ae@op5.se> writes:
> Thoughts? I've got a feeling I've overlooked something here.

If you are doing it, it might make sense to include it from Makefiles in subdirectories, so that on a platform with non GNU tar, you could say:

	$ cd t
	$ make
and get TAR=gtar definition from ../config.mk

I think Smurf's libize repository has something like that; it is found at:

        $ cat .git/remotes/smurf
        URL: http://netz.smurf.noris.de/git/git.git
        Pull: libize:libize

← back to recent threads