{"thread":{"id":"5453","subject":"[PATCH][RFC] Add git-archive-tree","startedAt":"2006-09-02T12:23:28Z","lastAt":"2006-09-17T11:54:13Z","messageCount":17,"participants":["Rene Scharfe","Franck Bui-Huu","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"26216","messageId":"44F977C0.4060901@lsrfire.ath.cx","threadId":"5453","inReplyTo":null,"subject":"[PATCH][RFC] Add git-archive-tree","fromName":"Rene Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2006-09-02T12:23:28Z","receivedAt":"2006-09-02T12:23:28Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"git-archive-tree is a command to make tar and ZIP archives of a git tree.\nIt helps prevent a proliferation of git-{format}-tree commands.  This is\nuseful e.g. for remote archive fetching because we only need to write a\nsingle upload and a single download program that simply pass on the\nformat option to git-archive-tree.\n\nSpeaking of remote, please note the absence of the --remote option of\ngit-tar-tree.  This is intentional; remote operations are special enough\nto deserve a separate (yet to be written) command.\n\nCurrently git-archive-tree -f tar is slower than git-tar-tree.  This is\nbecause it is welded to the side of the existing code to minimize patch\nsize, and I also suspect read_tree_recursive() to be quite a bit slower\nthan builtin-tar-tree.c::traverse_tree().\n\n\n Documentation/git-archive-tree.txt |   99 +++++++++++++++++++++++++++++++++++++\n Makefile                           |    3 -\n archive.h                          |    6 ++\n builtin-archive-tree.c             |   92 ++++++++++++++++++++++++++++++++++\n builtin-tar-tree.c                 |   66 ++++++++++++++++++++++++\n builtin-zip-tree.c                 |   28 ++++++++++\n builtin.h                          |    1 \n git.c                              |    1 \n 8 files changed, 295 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-archive-tree.txt b/Documentation/git-archive-tree.txt\nnew file mode 100644\nindex 0000000..122c482\n--- /dev/null\n+++ b/Documentation/git-archive-tree.txt\n@@ -0,0 +1,99 @@\n+git-archive-tree(1)\n+===============\n+\n+NAME\n+----\n+git-archive-tree - Creates a archive of the files in the named tree\n+\n+\n+SYNOPSIS\n+--------\n+'git-archive-tree' -f {tar|zip} [--prefix=<prefix>/] [-0|...|-9]\n+\t    <tree-ish> [path...]\n+\n+DESCRIPTION\n+-----------\n+Creates an archive of the specified format containing the tree structure\n+for the named tree.  If <prefix> is specified it is prepended to the\n+filenames in the archive.\n+\n+git-archive-tree behaves differently when given a tree ID versus when\n+given a commit ID or tag ID.  In the first case the current time is used\n+as modification time of each file in the archive.  In the latter case the\n+commit time as recorded in the referenced commit object is used instead.\n+Additionally the commit ID is stored in a global extended pax header if\n+the tar format is used; it can be extracted using git-get-tar-commit-id.\n+In ZIP files it is stored as a file comment.\n+\n+OPTIONS\n+-------\n+\n+-f::\n+\tFormat of the resulting archive, can be either 'tar' or 'zip'.\n+\n+<tree-ish>::\n+\tThe tree or commit to produce an archive for.\n+\n+path::\n+\tIf one or more paths are specified, include only these in the\n+\tarchive, otherwise include all files and subdirectories.\n+\n+--prefix=<prefix>/::\n+\tPrepend <prefix>/ to each filename in the archive.\n+\n+-0::\n+\tStore files in the archive instead of compressing them.  This\n+\toption has no effect when the tar format is used.\n+\n+-9::\n+        Highest and slowest compression level.  You can specify any\n+        number from 1 to 9 to adjust compression speed and ratio.  This\n+\toption has no effect when the tar format is used.\n+\n+CONFIGURATION\n+-------------\n+By default, file and directories modes are set to 0666 or 0777 in tar\n+archives.  It is possible to change this by setting the \"umask\" variable\n+in the repository configuration as follows :\n+\n+[tar]\n+        umask = 002\t;# group friendly\n+\n+The special umask value \"user\" indicates that the user's current umask\n+will be used instead. The default value remains 0, which means world\n+readable/writable files and directories.\n+\n+EXAMPLES\n+--------\n+git archive -f tar --prefix=junk/ HEAD | (cd /var/tmp/ && tar xf -)::\n+\n+\tCreate a tar archive that contains the contents of the\n+\tlatest commit on the current branch, and extracts it in\n+\t`/var/tmp/junk` directory.\n+\n+git archive -f tar --prefix=git-1.4.0/ v1.4.0 | gzip >git-1.4.0.tar.gz::\n+\n+\tCreate a compressed tarball for v1.4.0 release.\n+\n+git archive -f tar --prefix=git-1.4.0/ v1.4.0{caret}\\{tree\\} | gzip >git-1.4.0.tar.gz::\n+\n+\tCreate a compressed tarball for v1.4.0 release, but without a\n+\tglobal extended pax header.\n+\n+git archive -f zip --prefix=git-docs/ HEAD:Documentation/ > git-1.4.0-docs.zip::\n+\n+\tPut everything in the current head's Documentation/ directory\n+\tinto 'git-1.4.0-docs.zip', with the prefix 'git-docs/'.\n+\n+Author\n+------\n+Written by Rene Scharfe.\n+\n+Documentation\n+--------------\n+Documentation by David Greaves, Junio C Hamano and the git-list <git@vger.kernel.org>.\n+\n+GIT\n+---\n+Part of the gitlink:git[7] suite\n+\ndiff --git a/Makefile b/Makefile\nindex 05bd77f..d0a1055 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -231,7 +231,7 @@ LIB_FILE=libgit.a\n XDIFF_LIB=xdiff/lib.a\n \n LIB_H = \\\n-\tblob.h cache.h commit.h csum-file.h delta.h \\\n+\tarchive.h blob.h cache.h commit.h csum-file.h delta.h \\\n \tdiff.h object.h pack.h pkt-line.h quote.h refs.h \\\n \trun-command.h strbuf.h tag.h tree.h git-compat-util.h revision.h \\\n \ttree-walk.h log-tree.h dir.h path-list.h unpack-trees.h builtin.h\n@@ -255,6 +255,7 @@ LIB_OBJS = \\\n BUILTIN_OBJS = \\\n \tbuiltin-add.o \\\n \tbuiltin-apply.o \\\n+\tbuiltin-archive-tree.o \\\n \tbuiltin-cat-file.o \\\n \tbuiltin-checkout-index.o \\\n \tbuiltin-check-ref-format.o \\\ndiff --git a/archive.h b/archive.h\nnew file mode 100644\nindex 0000000..7813962\n--- /dev/null\n+++ b/archive.h\n@@ -0,0 +1,6 @@\n+#include \"tree.h\"\n+\n+typedef int (*write_archive_fn_t)(struct tree *tree, const unsigned char *commit_sha1, const char *prefix, time_t time, const char **pathspec);\n+\n+int write_tar_archive(struct tree *tree, const unsigned char *commit_sha1, const char *prefix, time_t time, const char **pathspec);\n+int write_zip_archive(struct tree *tree, const unsigned char *commit_sha1, const char *prefix, time_t time, const char **pathspec);\ndiff --git a/builtin-archive-tree.c b/builtin-archive-tree.c\nnew file mode 100644\nindex 0000000..2c6ee60\n--- /dev/null\n+++ b/builtin-archive-tree.c\n@@ -0,0 +1,92 @@\n+/*\n+ * Copyright (c) 2006 Rene Scharfe\n+ */\n+#include <time.h>\n+#include \"cache.h\"\n+#include \"builtin.h\"\n+#include \"commit.h\"\n+#include \"tree-walk.h\"\n+#include \"archive.h\"\n+\n+static const char archive_usage[] =\n+\"git-archive-tree -f {tar|zip} [--prefix=<prefix>/] [-0|...|-9] <tree-ish> [path...]\";\n+\n+static write_archive_fn_t parse_archive_format(const char *format)\n+{\n+\tif (!strcmp(format, \"tar\"))\n+\t\treturn write_tar_archive;\n+\tif (!strcmp(format, \"zip\"))\n+\t\treturn write_zip_archive;\n+\treturn NULL;\n+}\n+\n+int cmd_archive_tree(int argc, const char **argv, const char *prefix)\n+{\n+\tint more_args = 1;\n+\tconst char *archive_prefix = \"\";\n+\tunsigned char sha1[20];\n+\tstruct commit *commit;\n+\ttime_t archive_time;\n+\tconst char **pathspec;\n+\tconst unsigned char *commit_sha1 = NULL;\n+\twrite_archive_fn_t write_archive = NULL;\n+\tstruct tree *tree;\n+\tint result;\n+\n+\twhile (argc > 2 && more_args) {\n+\t\tconst char *arg = argv[1];\n+\t\tif (!strcmp(arg, \"-f\")) {\n+\t\t\twrite_archive = parse_archive_format(argv[2]);\n+\t\t\targv++;\n+\t\t\targc--;\n+\t\t} else if (!strncmp(arg, \"-f\", 2))\n+\t\t\twrite_archive = parse_archive_format(arg + 2);\n+\t\telse if (arg[0] == '-' && isdigit(arg[1]) && arg[2] == '\\0')\n+\t\t\tzlib_compression_level = arg[1] - '0';\n+\t\telse if (!strcmp(arg, \"--\"))\n+\t\t\tmore_args = 0;\n+\t\telse if (!strncmp(arg, \"--prefix=\", 9))\n+\t\t\tarchive_prefix = arg + 9;\n+\t\telse if (arg[0] == '-')\n+\t\t\tusage(archive_usage);\n+\t\telse\n+\t\t\tbreak;\n+\t\targv++;\n+\t\targc--;\n+\t}\n+\n+\tif (!write_archive)\n+\t\tusage(archive_usage);\n+\tif (argc < 2)\n+\t\tusage(archive_usage);\n+\tif (get_sha1(argv[1], sha1))\n+\t\tdie(\"Not a valid object name %s\", argv[1]);\n+\n+\tcommit = lookup_commit_reference_gently(sha1, 1);\n+\tif (commit)\n+\t\tcommit_sha1 = commit->object.sha1;\n+\n+\ttree = parse_tree_indirect(sha1);\n+\tif (!tree)\n+\t\tdie(\"not a tree object\");\n+\n+\tif (prefix) {\n+\t\tunsigned char tree_sha1[20];\n+\t\tunsigned int mode;\n+\t\tint err = get_tree_entry(tree->object.sha1, prefix,\n+\t\t                         tree_sha1, &mode);\n+\t\tif (err || !S_ISDIR(mode))\n+\t\t\tdie(\"current working directory is untracked\");\n+\t\tfree(tree);\n+\t\ttree = parse_tree_indirect(tree_sha1);\n+\t}\n+\n+\tarchive_time = commit ? commit->date : time(NULL);\n+\tpathspec = get_pathspec(archive_prefix, argv + 2);\n+\n+\tresult = write_archive(tree, commit_sha1, archive_prefix,\n+\t                       archive_time, pathspec);\n+\tfree(tree);\n+\n+\treturn result;\n+}\ndiff --git a/builtin-tar-tree.c b/builtin-tar-tree.c\nindex 61a4135..e0da01e 100644\n--- a/builtin-tar-tree.c\n+++ b/builtin-tar-tree.c\n@@ -9,6 +9,7 @@ #include \"strbuf.h\"\n #include \"tar.h\"\n #include \"builtin.h\"\n #include \"pkt-line.h\"\n+#include \"archive.h\"\n \n #define RECORDSIZE\t(512)\n #define BLOCKSIZE\t(RECORDSIZE * 20)\n@@ -338,6 +339,71 @@ static int generate_tar(int argc, const \n \treturn 0;\n }\n \n+static int write_tar_entry(const unsigned char *sha1,\n+                           const char *base, int baselen,\n+                           const char *filename, unsigned mode, int stage)\n+{\n+\tstatic struct strbuf path;\n+\tint filenamelen = strlen(filename);\n+\tvoid *buffer;\n+\tchar type[20];\n+\tunsigned long size;\n+\n+\tif (!path.alloc) {\n+\t\tpath.buf = xmalloc(PATH_MAX);\n+\t\tpath.alloc = PATH_MAX;\n+\t\tpath.len = path.eof = 0;\n+\t}\n+\tif (path.alloc < baselen + filenamelen) {\n+\t\tfree(path.buf);\n+\t\tpath.buf = xmalloc(baselen + filenamelen);\n+\t\tpath.alloc = baselen + filenamelen;\n+\t}\n+\tmemcpy(path.buf, base, baselen);\n+\tmemcpy(path.buf + baselen, filename, filenamelen);\n+\tpath.len = baselen + filenamelen;\n+\tif (S_ISDIR(mode)) {\n+\t\tstrbuf_append_string(&path, \"/\");\n+\t\tbuffer = NULL;\n+\t\tsize = 0;\n+\t} else {\n+\t\tbuffer = read_sha1_file(sha1, type, &size);\n+\t\tif (!buffer)\n+\t\t\tdie(\"cannot read %s\", sha1_to_hex(sha1));\n+\t}\n+\n+\twrite_entry(sha1, &path, mode, buffer, size);\n+\n+\treturn READ_TREE_RECURSIVE;\n+}\n+\n+int write_tar_archive(struct tree *tree, const unsigned char *commit_sha1,\n+                      const char *prefix, time_t time, const char **pathspec)\n+{\n+\tint plen = strlen(prefix);\n+\n+\tgit_config(git_tar_config);\n+\n+\tarchive_time = time;\n+\n+\tif (commit_sha1)\n+\t\twrite_global_extended_header(commit_sha1);\n+\n+\tif (prefix && plen > 0 && prefix[plen - 1] == '/') {\n+\t\tchar *base = strdup(prefix);\n+\t\tint baselen = strlen(base);\n+\n+\t\twhile (baselen > 0 && base[baselen - 1] == '/')\n+\t\t\tbase[--baselen] = '\\0';\n+\t\twrite_tar_entry(tree->object.sha1, \"\", 0, base, 040777, 0);\n+\t\tfree(base);\n+\t}\n+\tread_tree_recursive(tree, prefix, plen, 0, pathspec, write_tar_entry);\n+\twrite_trailer();\n+\n+\treturn 0;\n+}\n+\n static const char *exec = \"git-upload-tar\";\n \n static int remote_tar(int argc, const char **argv)\ndiff --git a/builtin-zip-tree.c b/builtin-zip-tree.c\nindex a5b834d..b142771 100644\n--- a/builtin-zip-tree.c\n+++ b/builtin-zip-tree.c\n@@ -8,6 +8,7 @@ #include \"blob.h\"\n #include \"tree.h\"\n #include \"quote.h\"\n #include \"builtin.h\"\n+#include \"archive.h\"\n \n static const char zip_tree_usage[] =\n \"git-zip-tree [-0|...|-9] <tree-ish> [ <base> ]\";\n@@ -351,3 +352,30 @@ int cmd_zip_tree(int argc, const char **\n \n \treturn 0;\n }\n+\n+int write_zip_archive(struct tree *tree, const unsigned char *commit_sha1,\n+                      const char *prefix, time_t time, const char **pathspec)\n+{\n+\tint plen = strlen(prefix);\n+\n+\tdos_time(&time, &zip_date, &zip_time);\n+\n+\tzip_dir = xmalloc(ZIP_DIRECTORY_MIN_SIZE);\n+\tzip_dir_size = ZIP_DIRECTORY_MIN_SIZE;\n+\n+\tif (prefix && plen > 0 && prefix[plen - 1] == '/') {\n+\t\tchar *base = strdup(prefix);\n+\t\tint baselen = strlen(base);\n+\n+\t\twhile (baselen > 0 && base[baselen - 1] == '/')\n+\t\t\tbase[--baselen] = '\\0';\n+\t\twrite_zip_entry(tree->object.sha1, \"\", 0, base, 040777, 0);\n+\t\tfree(base);\n+\t}\n+\tread_tree_recursive(tree, prefix, plen, 0, pathspec, write_zip_entry);\n+\twrite_zip_trailer(commit_sha1);\n+\n+\tfree(zip_dir);\n+\n+\treturn 0;\n+}\ndiff --git a/builtin.h b/builtin.h\nindex 25431d7..febb9d0 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -53,6 +53,7 @@ extern int cmd_stripspace(int argc, cons\n extern int cmd_symbolic_ref(int argc, const char **argv, const char *prefix);\n extern int cmd_tar_tree(int argc, const char **argv, const char *prefix);\n extern int cmd_zip_tree(int argc, const char **argv, const char *prefix);\n+extern int cmd_archive_tree(int argc, const char **argv, const char *prefix);\n extern int cmd_unpack_objects(int argc, const char **argv, const char *prefix);\n extern int cmd_update_index(int argc, const char **argv, const char *prefix);\n extern int cmd_update_ref(int argc, const char **argv, const char *prefix);\ndiff --git a/git.c b/git.c\nindex 05871ad..937b7f7 100644\n--- a/git.c\n+++ b/git.c\n@@ -264,6 +264,7 @@ static void handle_internal_command(int \n \t\t{ \"symbolic-ref\", cmd_symbolic_ref, RUN_SETUP },\n \t\t{ \"tar-tree\", cmd_tar_tree, RUN_SETUP },\n \t\t{ \"zip-tree\", cmd_zip_tree, RUN_SETUP },\n+\t\t{ \"archive-tree\", cmd_archive_tree, RUN_SETUP },\n \t\t{ \"unpack-objects\", cmd_unpack_objects, RUN_SETUP },\n \t\t{ \"update-index\", cmd_update_index, RUN_SETUP },\n \t\t{ \"update-ref\", cmd_update_ref, RUN_SETUP },\n\n-- \nVGER BF report: U 0.5\n"},{"id":"26217","messageId":"44F97B24.7090305@lsrfire.ath.cx","threadId":"5453","inReplyTo":"44F977C0.4060901@lsrfire.ath.cx","subject":"[PATCH] Add support for tgz archive format","fromName":"Rene Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2006-09-02T12:37:56Z","receivedAt":"2006-09-02T12:37:56Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":" Documentation/git-archive-tree.txt |    9 ++---\n archive.h                          |    2 +\n builtin-archive-tree.c             |    4 +-\n builtin-tar-tree.c                 |   63 ++++++++++++++++++++++++++++++++-----\n 4 files changed, 65 insertions(+), 13 deletions(-)\n\ndiff --git a/Documentation/git-archive-tree.txt b/Documentation/git-archive-tree.txt\nindex 122c482..25136d9 100644\n--- a/Documentation/git-archive-tree.txt\n+++ b/Documentation/git-archive-tree.txt\n@@ -8,7 +8,7 @@ git-archive-tree - Creates a archive of the f\n \n SYNOPSIS\n --------\n-'git-archive-tree' -f {tar|zip} [--prefix=<prefix>/] [-0|...|-9]\n+'git-archive-tree' -f {tar|tgz|zip} [--prefix=<prefix>/] [-0|...|-9]\n \t    <tree-ish> [path...]\n \n DESCRIPTION\n@@ -29,7 +29,8 @@ OPTIONS\n -------\n \n -f::\n-\tFormat of the resulting archive, can be either 'tar' or 'zip'.\n+\tFormat of the resulting archive, can be either 'tar', 'tgz'\n+\tor 'zip'.\n \n <tree-ish>::\n \tThe tree or commit to produce an archive for.\n@@ -71,11 +72,11 @@ git archive -f tar --prefix=junk/ HEAD |\n \tlatest commit on the current branch, and extracts it in\n \t`/var/tmp/junk` directory.\n \n-git archive -f tar --prefix=git-1.4.0/ v1.4.0 | gzip >git-1.4.0.tar.gz::\n+git archive -f tgz --prefix=git-1.4.0/ v1.4.0 >git-1.4.0.tar.gz::\n \n \tCreate a compressed tarball for v1.4.0 release.\n \n-git archive -f tar --prefix=git-1.4.0/ v1.4.0{caret}\\{tree\\} | gzip >git-1.4.0.tar.gz::\n+git archive -f tgz --prefix=git-1.4.0/ v1.4.0{caret}\\{tree\\} >git-1.4.0.tar.gz::\n \n \tCreate a compressed tarball for v1.4.0 release, but without a\n \tglobal extended pax header.\ndiff --git a/archive.h b/archive.h\nindex 7813962..6a03864 100644\n--- a/archive.h\n+++ b/archive.h\n@@ -3,4 +3,6 @@ #include \"tree.h\"\n typedef int (*write_archive_fn_t)(struct tree *tree, const unsigned char *commit_sha1, const char *prefix, time_t time, const char **pathspec);\n \n int write_tar_archive(struct tree *tree, const unsigned char *commit_sha1, const char *prefix, time_t time, const char **pathspec);\n+int write_tgz_archive(struct tree *tree, const unsigned char *commit_sha1,\n+                      const char *prefix, time_t time, const char **pathspec);\n int write_zip_archive(struct tree *tree, const unsigned char *commit_sha1, const char *prefix, time_t time, const char **pathspec);\ndiff --git a/builtin-archive-tree.c b/builtin-archive-tree.c\nindex 2c6ee60..f61e26d 100644\n--- a/builtin-archive-tree.c\n+++ b/builtin-archive-tree.c\n@@ -9,12 +9,14 @@ #include \"tree-walk.h\"\n #include \"archive.h\"\n \n static const char archive_usage[] =\n-\"git-archive-tree -f {tar|zip} [--prefix=<prefix>/] [-0|...|-9] <tree-ish> [path...]\";\n+\"git-archive-tree -f {tar|tgz|zip} [--prefix=<prefix>/] [-0|...|-9] <tree-ish> [path...]\";\n \n static write_archive_fn_t parse_archive_format(const char *format)\n {\n \tif (!strcmp(format, \"tar\"))\n \t\treturn write_tar_archive;\n+\tif (!strcmp(format, \"tgz\"))\n+\t\treturn write_tgz_archive;\n \tif (!strcmp(format, \"zip\"))\n \t\treturn write_zip_archive;\n \treturn NULL;\ndiff --git a/builtin-tar-tree.c b/builtin-tar-tree.c\nindex e0da01e..743d53a 100644\n--- a/builtin-tar-tree.c\n+++ b/builtin-tar-tree.c\n@@ -23,6 +23,10 @@ static unsigned long offset;\n static time_t archive_time;\n static int tar_umask;\n \n+static gzFile gzstdout;\n+\n+typedef void (*blocked_write_fn)(const void *data, unsigned long size);\n+\n /* writes out the whole block, but only if it is full */\n static void write_if_needed(void)\n {\n@@ -36,7 +40,7 @@ static void write_if_needed(void)\n  * queues up writes, so that all our write(2) calls write exactly one\n  * full block; pads writes to RECORDSIZE\n  */\n-static void write_blocked(const void *data, unsigned long size)\n+static void do_write_blocked(const void *data, unsigned long size)\n {\n \tconst char *buf = data;\n \tunsigned long tail;\n@@ -68,19 +72,20 @@ static void write_blocked(const void *da\n \twrite_if_needed();\n }\n \n+static blocked_write_fn write_blocked = do_write_blocked;\n+\n /*\n  * The end of tar archives is marked by 2*512 nul bytes and after that\n  * follows the rest of the block (if any).\n  */\n static void write_trailer(void)\n {\n-\tint tail = BLOCKSIZE - offset;\n-\tmemset(block + offset, 0, tail);\n-\twrite_or_die(1, block, BLOCKSIZE);\n-\tif (tail < 2 * RECORDSIZE) {\n-\t\tmemset(block, 0, offset);\n-\t\twrite_or_die(1, block, BLOCKSIZE);\n-\t}\n+\tchar zeroes[RECORDSIZE];\n+\tmemset(zeroes, 0, RECORDSIZE);\n+\twrite_blocked(zeroes, RECORDSIZE);\n+\twrite_blocked(zeroes, RECORDSIZE);\n+\twhile (offset)\n+\t\twrite_blocked(zeroes, RECORDSIZE);\n }\n \n static void strbuf_append_string(struct strbuf *sb, const char *s)\n@@ -404,6 +409,48 @@ int write_tar_archive(struct tree *tree,\n \treturn 0;\n }\n \n+static void write_blocked_gzip(const void *data, unsigned long size)\n+{\n+\tconst char *p = data;\n+\tint written;\n+\tunsigned long tail = size % RECORDSIZE;\n+\n+\twhile (size > 0) {\n+\t\twritten = gzwrite(gzstdout, p, size);\n+\t\tif (written == 0) {\n+\t\t\tif (errno == EPIPE)\n+\t\t\t\texit(0);\n+\t\t\tdie(\"gzwrite error (%s)\", strerror(errno));\n+\t\t}\n+\t\tsize -= written;\n+\t\tp += written;\n+\t}\n+\n+\tif (tail) {\n+\t\tz_off_t result = gzseek(gzstdout, RECORDSIZE - tail, SEEK_CUR);\n+\t\tif (result == -1)\n+\t\t\tdie(\"gzseek error (%s)\", strerror(errno));\n+\t}\n+}\n+\n+int write_tgz_archive(struct tree *tree, const unsigned char *commit_sha1,\n+                      const char *prefix, time_t time, const char **pathspec)\n+{\n+\tint result;\n+\n+\tgzstdout = gzdopen(1, \"wb\");\n+\tif (!gzstdout)\n+\t\tdie(\"zlib is unable to open stdout\");\n+\n+\twrite_blocked = write_blocked_gzip;\n+\tresult = write_tar_archive(tree, commit_sha1, prefix, time, pathspec); \n+\n+\tif (gzclose(gzstdout) != Z_OK)\n+\t\tresult = 1;\n+\n+\treturn result;\n+}\n+\n static const char *exec = \"git-upload-tar\";\n \n static int remote_tar(int argc, const char **argv)\n\n-- \nVGER BF report: U 0.497484\n"},{"id":"26218","messageId":"44F982BD.1050509@lsrfire.ath.cx","threadId":"5453","inReplyTo":"44F977C0.4060901@lsrfire.ath.cx","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Rene Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2006-09-02T13:10:21Z","receivedAt":"2006-09-02T13:10:21Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"The two patches I sent are what I have been able to come up with so far.\nThe next step would be to add archive-neutral upload and download support.\n\nHaving thought a bit about it I propose to keep git-archive-tree for\nlocal operations, only.  It can be called by the uploader just like\ngit-tar-tree is now called by git-upload-tar.  As Franck suggested, the\nuploader should allow the list of archive formats it supports to be\nrestricted in a config file.  The range of allowed compression levels\nshould also be configurable.\n\nDoes it make sense to change the wire protocol to simply send the\ncommand line options one by one?\n\nThe interface could be something like this:\n\n   git-download-archive <repo> <git-archive-tree options...>\n   git-upload-archive <directory>\n\nOr, if the big number of git command names is a concern:\n\n   git-archive-remote --upload|--download ...\n\nWhat do you all think?\n\nI won't have time for any of this over the next few days, so Franck, go\nwild. ;-)\n\nRené\n\n-- \nVGER BF report: U 0.499999\n"},{"id":"26219","messageId":"44F9928F.4070205@lsrfire.ath.cx","threadId":"5453","inReplyTo":"44F977C0.4060901@lsrfire.ath.cx","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Rene Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2006-09-02T14:17:51Z","receivedAt":"2006-09-02T14:17:51Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Oops, forgot to sign off.  Please consider both patches\n(git-archive-tree and tgz support) as\n\nSigned-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>\n\n-- \nVGER BF report: U 0.500007\n"},{"id":"26223","messageId":"cda58cb80609020824ja524444uc9fe65ea2e4febba@mail.gmail.com","threadId":"5453","inReplyTo":"44F977C0.4060901@lsrfire.ath.cx","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Franck Bui-Huu","fromEmail":"vagabon.xyz@gmail.com","sentAt":"2006-09-02T15:24:45Z","receivedAt":"2006-09-02T15:24:45Z","isPatch":true,"sender":{"key":"vagabon.xyz@gmail.com","avatar":null},"body":"Hi,\n\n2006/9/2, Rene Scharfe <rene.scharfe@lsrfire.ath.cx>:\n> git-archive-tree is a command to make tar and ZIP archives of a git tree.\n> It helps prevent a proliferation of git-{format}-tree commands.  This is\n> useful e.g. for remote archive fetching because we only need to write a\n> single upload and a single download program that simply pass on the\n> format option to git-archive-tree.\n\nOK, Rene you beat me. I started to cook up something this morning but\nhad no time to go further.\n\nI sent a starting implementation just because it seems complementary\nto the one you sent a couple hours ago: it supports '--remote' option.\nBut it does _not_ have path spec support you introduced. I think it's\ncool feature but I would have to dig into git's internal to implement\nit, which would have taken me a while.\n\n>From now, do you think we should import my work into your version or\n'vice verca' ?\n\n-- >8 --\n\nSubject: [PATCH] Add git-archive (other implementation)\n\nSigned-off-by: Franck Bui-Huu <vagabon.xyz@gmail.com>\n---\n .gitignore          |    1\n Makefile            |    1\n builtin-archive.c   |  171 +++++++++++++++++++++++++++++++++++++++++++++++++++\n builtin.h           |    1\n generate-cmdlist.sh |    1\n git.c               |    1\n 6 files changed, 176 insertions(+), 0 deletions(-)\n\ndiff --git a/.gitignore b/.gitignore\nindex 78cb671..a3e7ca1 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -8,6 +8,7 @@ git-apply\n git-applymbox\n git-applypatch\n git-archimport\n+git-archive\n git-bisect\n git-branch\n git-cat-file\ndiff --git a/Makefile b/Makefile\nindex 164dbcf..8d963e0 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -266,6 +266,7 @@ LIB_OBJS = \\\n BUILTIN_OBJS = \\\n \tbuiltin-add.o \\\n \tbuiltin-apply.o \\\n+\tbuiltin-archive.o \\\n \tbuiltin-cat-file.o \\\n \tbuiltin-checkout-index.o \\\n \tbuiltin-check-ref-format.o \\\ndiff --git a/builtin-archive.c b/builtin-archive.c\nnew file mode 100644\nindex 0000000..47898ee\n--- /dev/null\n+++ b/builtin-archive.c\n@@ -0,0 +1,171 @@\n+/*\n+ * Copyright (c) 2006 Franck Bui-Huu\n+ */\n+#include \"cache.h\"\n+#include \"builtin.h\"\n+#include \"exec_cmd.h\"\n+#include \"pkt-line.h\"\n+\n+#define MAX_ARGS\t32\n+\n+static const char *extra_argv[MAX_ARGS];\n+static int extra_argc = 1;\n+static const char *base;\n+static const char *remote;\n+\n+static const char archive_usage[] = \\\n+\"git-archive -f <format> [extra] [--remote=<repo>] [--prefix=<base>]\n<tree-ish>\";\n+\n+/* archiver's options */\n+#define NEED_BASE\t(1<<0)\n+\n+struct archiver_info {\n+\tconst char *name;\n+\tint (*fn)(int argc, const char **argv, const char *prefix);\n+\tint options;\n+};\n+\n+static struct archiver_info archivers[] = {\n+\t{ \"tar\", cmd_tar_tree, NEED_BASE },\n+\t{ \"zip\", cmd_zip_tree, NEED_BASE },\n+};\n+\n+static int run_remote_archiver(struct archiver_info *ar, char *url)\n+{\n+\tint fd[2], len, rv;\n+\tchar buf[1024];\n+\tpid_t pid;\n+\n+\t/*\n+\t * For now, remote operations does not support extra options\n+\t */\n+\tif (extra_argc > 2)\n+\t\tdie(\"Remote operation does not support extra options yet.\");\n+\n+\tsprintf(buf, \"git-upload-%s\", ar->name);\n+\tpid = git_connect(fd, url, buf);\n+\tif (pid < 0)\n+\t\treturn pid;\n+\n+\tpacket_write(fd[1], \"want %s\\n\", extra_argv[1]);\n+\tif (base)\n+\t\tpacket_write(fd[1], \"base %s\\n\", base);\n+\tpacket_flush(fd[1]);\n+\n+\tlen = packet_read_line(fd[0], buf, sizeof(buf));\n+\tif (!len)\n+\t\tdie(\"git-archive: expected ACK/NAK, got EOF\");\n+\tif (buf[len-1] == '\\n')\n+\t\tbuf[--len] = 0;\n+\tif (strcmp(buf, \"ACK\")) {\n+\t\tif (len > 5 && !strncmp(buf, \"NACK \", 5))\n+\t\t\tdie(\"git-archive: NACK %s\", buf + 5);\n+\t\tdie(\"git-archive: protocol error\");\n+\t}\n+\tif (packet_read_line(fd[0], buf, sizeof(buf)))\n+\t\tdie(\"git-archive: expected a flush\");\n+\n+\t/* Now, start reading from fd[0] and spit it out to stdout */\n+\trv = copy_fd(fd[0], 1);\n+\tclose(fd[0]);\n+\n+\treturn !!(rv | finish_connect(pid));\n+}\n+\n+void insert_extra_option(const char *option, int idx)\n+{\n+\tconst char **argv = extra_argv;\t/* some shortcuts... */\n+\tint argc = extra_argc;\n+\n+\tif (argc > MAX_ARGS - 2)\n+\t\tdie(\"Too many args\");\n+\tif (idx > MAX_ARGS - 1)\n+\t\tdie(\"idx overflow extra_argv buffer !\");\n+\n+\t/* insert at last position ? */\n+\tif (idx < 0) {\n+\t\targv[argc++] = option;\n+\t\targv[argc] = NULL;\n+\t} else {\n+\t\targv += idx;\n+\t\tmemmove(argv + 1, argv, argc * sizeof(*argv));\n+\t\t*argv = option;\n+\t}\n+\textra_argc++;\n+}\n+\n+int run_archiver(const char *format)\n+{\n+\tstruct archiver_info *ar;\n+\tchar buf[32];\n+\tint i;\n+\n+\tfor (i = 0; i < ARRAY_SIZE(archivers); i++) {\n+\t\tar = &archivers[i];\n+\t\tif (strcmp(format, ar->name))\n+\t\t\tcontinue;\n+\t\tgoto found;\n+\t}\n+\treturn error(\"Unknown archive format '%s'\\n\", format);\n+found:\n+\tif (remote) {\n+\t\tchar *url = strdup(remote+9);\t/* --remote=<repo> */\n+\t\treturn run_remote_archiver(ar, url);\n+\t}\n+\tif (base) {\n+\t\ti = 1;\n+\t\tif (ar->options & NEED_BASE) {\n+\t\t\tbase += 9; /* --prefix=<base> */\n+\t\t\ti = -1;\n+\t\t}\n+\t\tinsert_extra_option(base, i);\n+\t}\n+\tsprintf(buf, \"%s-tree\", ar->name);\n+\textra_argv[0] = buf;\n+\n+\treturn ar->fn(extra_argc, extra_argv, setup_git_directory());\n+}\n+\n+static int show_list_formats(void)\n+{\n+\tint i;\n+\t\n+\tfor (i = 0; i < ARRAY_SIZE(archivers); i++) {\n+\t\tprintf(\"%s\\n\", archivers[i].name);\n+\t}\n+\treturn 0;\n+}\n+\n+int cmd_archive(int argc, const char **argv, const char *prefix)\n+{\n+\tstatic const char *format;\n+\tint i;\n+\n+\tfor (i = 1; i < argc; i++) {\n+\t\tconst char *arg = argv[i];\n+\n+\t\tif (!strcmp(arg, \"--list\") || !strcmp(arg, \"-l\")) {\n+\t\t\treturn show_list_formats();\n+\t\t}\n+\t\tif (!strncmp(arg, \"--format=\", 9)) {\n+\t\t\tformat = arg+9;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (!strncmp(arg, \"--prefix=\", 9)) {\n+\t\t\tbase = arg;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (!strncmp(arg, \"--remote=\", 9)) {\n+\t\t\tremote = arg;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (extra_argc > MAX_ARGS - 2)\n+\t\t\tdie(\"Too many options\");\n+\t\textra_argv[extra_argc++] = arg;\n+\t}\n+\n+\tif (!format)\n+\t\tdie(\"You must specify an archive format\");\n+\n+\treturn run_archiver(format);\n+}\ndiff --git a/builtin.h b/builtin.h\nindex 25431d7..50852cd 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -15,6 +15,7 @@ extern int write_tree(unsigned char *sha\n\n extern int cmd_add(int argc, const char **argv, const char *prefix);\n extern int cmd_apply(int argc, const char **argv, const char *prefix);\n+extern int cmd_archive(int argc, const char **argv, const char *prefix);\n extern int cmd_cat_file(int argc, const char **argv, const char *prefix);\n extern int cmd_checkout_index(int argc, const char **argv, const char *prefix);\n extern int cmd_check_ref_format(int argc, const char **argv, const\nchar *prefix);\ndiff --git a/generate-cmdlist.sh b/generate-cmdlist.sh\nindex ec1eda2..5450918 100755\n--- a/generate-cmdlist.sh\n+++ b/generate-cmdlist.sh\n@@ -12,6 +12,7 @@ struct cmdname_help common_cmds[] = {\"\n sort <<\\EOF |\n add\n apply\n+archive\n bisect\n branch\n checkout\ndiff --git a/git.c b/git.c\nindex 403fb3a..ddb1620 100644\n--- a/git.c\n+++ b/git.c\n@@ -218,6 +218,7 @@ static void handle_internal_command(int\n \t} commands[] = {\n \t\t{ \"add\", cmd_add, RUN_SETUP },\n \t\t{ \"apply\", cmd_apply },\n+\t\t{ \"archive\", cmd_archive },\n \t\t{ \"cat-file\", cmd_cat_file, RUN_SETUP },\n \t\t{ \"checkout-index\", cmd_checkout_index, RUN_SETUP },\n \t\t{ \"check-ref-format\", cmd_check_ref_format },\n-- \n1.4.2.g8ed9-dirty\n\n-- \nVGER BF report: U 0.5\n"},{"id":"26225","messageId":"44F9AC75.3020302@lsrfire.ath.cx","threadId":"5453","inReplyTo":"cda58cb80609020824ja524444uc9fe65ea2e4febba@mail.gmail.com","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Rene Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2006-09-02T16:08:21Z","receivedAt":"2006-09-02T16:08:21Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Franck Bui-Huu schrieb:\n> Hi,\n> \n> 2006/9/2, Rene Scharfe <rene.scharfe@lsrfire.ath.cx>:\n>> git-archive-tree is a command to make tar and ZIP archives of a git tree.\n>> It helps prevent a proliferation of git-{format}-tree commands.  This is\n>> useful e.g. for remote archive fetching because we only need to write a\n>> single upload and a single download program that simply pass on the\n>> format option to git-archive-tree.\n> \n> OK, Rene you beat me. I started to cook up something this morning but\n> had no time to go further.\n\nI am sorry.  My excuse is I was so angry the other night (for unrelated\nreasons) that I started coding and I didn't stop until git-archive-tree\nwas working at three in the morning.\n\n> I sent a starting implementation just because it seems complementary\n> to the one you sent a couple hours ago: it supports '--remote' option.\n> But it does _not_ have path spec support you introduced. I think it's\n> cool feature but I would have to dig into git's internal to implement\n> it, which would have taken me a while.\n\nI cheated by using the already existing function read_tree_recursive().\n\n> From now, do you think we should import my work into your version or\n> 'vice verca' ?\n\nOf course I like mine better. :-)  But first we need to agree on a\ndirection.  I think it makes sense to keep the network stuff out of the\narchiver, and to keep the downloader as simple (\"dumb\") as possible: it\nshould pass on any options unaltered, including the format option.\n\nThat means the wire format changes completely, but since this feature\nhasn't been officially released, yet, this shouldn't be a problem, right?\n\nSo, I propose to not add anything (except new formats) to\ngit-archive-tree and to add a separate downloader and uploader (maybe\ncombined in one program) instead.  That way the remote archive\noperations are format agnostic.  OK, almost, because the uploader needs\nto determine the format and compression level in order to allow or\ndisallow uploads based on that.\n\nWhat do you think?\n\nThanks,\nRené\n\n-- \nVGER BF report: U 0.539485\n"},{"id":"26238","messageId":"cda58cb80609021313p5156a14du4895e53387dd24dc@mail.gmail.com","threadId":"5453","inReplyTo":"44F982BD.1050509@lsrfire.ath.cx","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Franck Bui-Huu","fromEmail":"vagabon.xyz@gmail.com","sentAt":"2006-09-02T20:13:14Z","receivedAt":"2006-09-02T20:13:14Z","isPatch":true,"sender":{"key":"vagabon.xyz@gmail.com","avatar":null},"body":"2006/9/2, Rene Scharfe <rene.scharfe@lsrfire.ath.cx>:\n> The two patches I sent are what I have been able to come up with so far.\n> The next step would be to add archive-neutral upload and download support.\n>\n> Having thought a bit about it I propose to keep git-archive-tree for\n> local operations, only.  It can be called by the uploader just like\n\nWell I don't see why putting the remote operations in an other file. I\nwas more thinking on something like this:\n\ngit-archive --format=<fmt> [--remote=<repo>] <tree-ish> [path...]\n\nThis main porcelain function would call directly functions provided by\narchivers lib. We will need to define an API which git-archive will\nuse for local operations.\n\nSymetrically, on the sever side we would have:\n\ngit-upload-archive --format=<fmt> <repo> [path...]\n\nused by git-daemon. It will deal with protocol, paths and use archiver's lib.\n\nEventually, we would have 2 commands:\n\ngit-archive\ngit-upload-archive\n\nand get ride of\n\ngit-tar-tree\ngit-zip-tree\ngit-upload-tar\ngit-upload-zip\n\n> git-tar-tree is now called by git-upload-tar.  As Franck suggested, the\n> uploader should allow the list of archive formats it supports to be\n> restricted in a config file.  The range of allowed compression levels\n> should also be configurable.\n>\n> Does it make sense to change the wire protocol to simply send the\n> command line options one by one?\n\nThat would make sense if the number of options grow up. Currently the\nremote protocol had been written by Junio, I just pick up that part\nfrom git-tar-tree and put it into git-archive. But if we allow\npathspec for remote operations, then  we need to send them to the\nuploader.\n\n>\n> The interface could be something like this:\n>\n>    git-download-archive <repo> <git-archive-tree options...>\n>    git-upload-archive <directory>\n>\n> Or, if the big number of git command names is a concern:\n>\n\nI think it is, IMHO. And that's why I think we could have only one\ncommand for building localy/remotely archive whatever the format.\ngit-archive should be a main procelain command, and we should get rid\nof git-{tar,zip}-tree commands.\n\n-- \n               Franck\n\n-- \nVGER BF report: U 0.742177\n"},{"id":"26242","messageId":"7v1wquynj5.fsf@assigned-by-dhcp.cox.net","threadId":"5453","inReplyTo":"44F982BD.1050509@lsrfire.ath.cx","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-09-02T21:19:26Z","receivedAt":"2006-09-02T21:19:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Rene Scharfe <rene.scharfe@lsrfire.ath.cx> writes:\n\n> Does it make sense to change the wire protocol to simply send the\n> command line options one by one?\n\nWhich wire protocol are you talking about?  The one between\nupload-tar and local \"tar-tree --remote\"?  I think that one was\na tar-tree specific hack and we do not want to mimic it.\n\nYour idea of making archiver neutral upload/download pair makes\nsense.  The daemon can invoke upload-archive with a single\nparameter (the repository, \".\"), just like upload_tar() in\n\"pu:daemon.c\" does [*1*], and upload-archive talks with the\nother end to know which archiver to run with what paramter.  We\nwould probably want to use some sort of side-band mechanism so\nthat we can do progress-bar as well.\n\n\n*1* Unrelated side note.  I find myself typing \"git less\" quite\n    often ;-)\n\n\tgit less pu:daemon.c\n\n    and get \"no such command, dummy\" response.  \n\n    Yes, I know I could alias it to \"-p cat-file -p\".  I am just\n    too lazy to do so.\n\n\n\n-- \nVGER BF report: U 0.709954\n"},{"id":"26243","messageId":"7vwt8mx8lb.fsf@assigned-by-dhcp.cox.net","threadId":"5453","inReplyTo":"44F977C0.4060901@lsrfire.ath.cx","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-09-02T21:27:28Z","receivedAt":"2006-09-02T21:27:28Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Rene Scharfe <rene.scharfe@lsrfire.ath.cx> writes:\n\n> Currently git-archive-tree -f tar is slower than git-tar-tree.  This is\n> because it is welded to the side of the existing code to minimize patch\n> size, and I also suspect read_tree_recursive() to be quite a bit slower\n> than builtin-tar-tree.c::traverse_tree().\n\nYes, I suspect \"struct object\" and friends are very inefficient\nto use for things like this.  \"struct tree_desc\" based traverser\nis much preferred.\n\n\n\n-- \nVGER BF report: U 0.772588\n"},{"id":"26313","messageId":"44FC6EDE.4010300@lsrfire.ath.cx","threadId":"5453","inReplyTo":"cda58cb80609021313p5156a14du4895e53387dd24dc@mail.gmail.com","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Rene Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2006-09-04T18:22:22Z","receivedAt":"2006-09-04T18:22:22Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Franck Bui-Huu schrieb:\n> 2006/9/2, Rene Scharfe <rene.scharfe@lsrfire.ath.cx>:\n>> The two patches I sent are what I have been able to come up with so far.\n>> The next step would be to add archive-neutral upload and download\n>> support.\n>>\n>> Having thought a bit about it I propose to keep git-archive-tree for\n>> local operations, only.  It can be called by the uploader just like\n> \n> Well I don't see why putting the remote operations in an other file. I\n> was more thinking on something like this:\n> \n> git-archive --format=<fmt> [--remote=<repo>] <tree-ish> [path...]\n\nMy intention was to put both halves of the wire protocol implementation\ninto the same source file.  I still think this is a good idea, but of\ncourse it's independent from any command line interface considerations.\n\nInternally, remote and local operations do completely different things,\nso it doesn't make sense to mix them.  But users probably don't care\nabout such details and may prefer a single command to express \"gimme\nthat archive, no matter if it's made here or there\".  OK, sort of.\n\n> This main porcelain function would call directly functions provided by\n> archivers lib. We will need to define an API which git-archive will\n> use for local operations.\n\nYes, this would be write_archive_fn_t in archive.h.\n\n> Symetrically, on the sever side we would have:\n> \n> git-upload-archive --format=<fmt> <repo> [path...]\n\nNo, git-upload-archive would accept only a single parameter: the path to\nthe repository.  It'd receive all other options via the wire protocol,\njust like git-upload-tar.\n\n> used by git-daemon. It will deal with protocol, paths and use archiver's\n> lib.\n\nExcept if you meant by this that git-daemon would handle the protocol\netc., which I'd disagree with (\"It\" is slightly ambiguous here).\n\n> Eventually, we would have 2 commands:\n> \n> git-archive\n> git-upload-archive\n> \n> and get ride of\n> \n> git-tar-tree\n> git-zip-tree\n> git-upload-tar\n> git-upload-zip\n\nLet's keep the git-<verb>-<object> nomenclature used for most commands\nfor the sake of consistency.  In both cases you'd only type\ngit-archiv<TAB> anyway. ;-)\n\nWe could remove the existing commands git-upload-tar and git-zip-tree\nright now, as they were never part of a release, yet.  git-upload-zip\ndoesn't exist, yet, and git-tar-tree would probably survive as legacy\ninterface, calling git-archive-tree internals.  Junio, am I correct\nregarding the cool-blooded killing of unreleased commands?\n\n>> git-tar-tree is now called by git-upload-tar.  As Franck suggested, the\n>> uploader should allow the list of archive formats it supports to be\n>> restricted in a config file.  The range of allowed compression levels\n>> should also be configurable.\n>>\n>> Does it make sense to change the wire protocol to simply send the\n>> command line options one by one?\n> \n> That would make sense if the number of options grow up. Currently the\n> remote protocol had been written by Junio, I just pick up that part\n> from git-tar-tree and put it into git-archive. But if we allow\n> pathspec for remote operations, then  we need to send them to the\n> uploader.\n\nIf we simply send each option to the downloader without understanding\nthem then the uploader can be kept really simple.  The protocol can be\nkept simple, and it would be future-proof: we will never have to update\nit.  I'm a bit worried about the security implications of such a setup,\nstrangely _because_ I can't see a way to exploit it right now (and\nbecause it allows arbitrary input, which we must be able to cope with\nanyway).\n\n>>\n>> The interface could be something like this:\n>>\n>>    git-download-archive <repo> <git-archive-tree options...>\n>>    git-upload-archive <directory>\n>>\n>> Or, if the big number of git command names is a concern:\n>>\n> \n> I think it is, IMHO. And that's why I think we could have only one\n> command for building localy/remotely archive whatever the format.\n> git-archive should be a main procelain command, and we should get rid\n> of git-{tar,zip}-tree commands.\n\nOK, makes sense; users will never need to call git-upload-archive\ndirectly, and having a single command for all archiving is a good thing.\n\nMy next steps will be to make traverse_tree() support path specs, in\norder to achieve feature parity with read_tree_recursive().  I hope that\nthe former keeps being significantly faster than the latter even after\nthat.  If it works out I'm going to convert git-zip-tree to use\ntraverse_tree().\n\nOther tasks: morphing git-upload-tar into git-upload-archive,\nreplacing/enhancing the protocol, and adding a matching uploader\n(--remote= option handler).  I won't start with that stuff until I get\nthe performance sorted out.  I hope this keeps me from stepping on your\ntoes (again), Franck. ;-)\n\nThanks,\nRené\n"},{"id":"26316","messageId":"7vy7szmm1u.fsf@assigned-by-dhcp.cox.net","threadId":"5453","inReplyTo":"44FC6EDE.4010300@lsrfire.ath.cx","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-09-04T20:09:01Z","receivedAt":"2006-09-04T20:09:01Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Rene Scharfe <rene.scharfe@lsrfire.ath.cx> writes:\n\n> Franck Bui-Huu schrieb:\n>>\n>> Well I don't see why putting the remote operations in an other file. I\n>> was more thinking on something like this:\n>> \n>> git-archive --format=<fmt> [--remote=<repo>] <tree-ish> [path...]\n>\n> My intention was to put both halves of the wire protocol implementation\n> into the same source file.  I still think this is a good idea, but of\n> course it's independent from any command line interface considerations.\n\nI think so too, and agree with your reasoning. \n\n> No, git-upload-archive would accept only a single parameter: the path to\n> the repository.  It'd receive all other options via the wire protocol,\n> just like git-upload-tar.\n\nSame here.\n\n> We could remove the existing commands git-upload-tar and git-zip-tree\n> right now, as they were never part of a release, yet.  git-upload-zip\n> doesn't exist, yet, and git-tar-tree would probably survive as legacy\n> interface, calling git-archive-tree internals.  Junio, am I correct\n> regarding the cool-blooded killing of unreleased commands?\n>>> git-tar-tree is now called by git-upload-tar.\n\nI do think leaving tar-tree vs upload-tar protocol alone would\nnot hurt development of this new archive vs upload-archive\nprotocol.  Also 'git-tar-tree --remote' has been part of git for\nsome time now (1.4.2 has it).  Having said that, I suspect\nnobody really relies on that, and I think this restructuring is\na good change.  Time to poll the userbase?\n\n\tDo people get upset if we said \"git-tar-tree\n\t--remote=xyzzy.example.com\" is no more, and you will\n\thave to say \"git-fetch-archive -f tar xyzzy.example.com\"?\n\n>>> Does it make sense to change the wire protocol to simply send the\n>>> command line options one by one?\n>> \n>> That would make sense if the number of options grow up.\n\n... or different archive backends want different kind of things\n(tar-tree without compression would not want -z6 but zip-tree\nwould).\n\n>> But if we allow\n>> pathspec for remote operations, then  we need to send them to the\n>> uploader.\n\nYup.\n\n> If we simply send each option to the downloader without understanding\n> them then the uploader can be kept really simple.  The protocol can be\n> kept simple, and it would be future-proof: we will never have to update\n> it.  I'm a bit worried about the security implications of such a setup,\n> strangely _because_ I can't see a way to exploit it right now (and\n> because it allows arbitrary input, which we must be able to cope with\n> anyway).\n\n>>>\n>>> The interface could be something like this:\n>>>\n>>>    git-download-archive <repo> <git-archive-tree options...>\n>>>    git-upload-archive <directory>\n>>>\n>> I think it is, IMHO. And that's why I think we could have only one\n>> command for building localy/remotely archive whatever the format.\n>> git-archive should be a main procelain command, and we should get rid\n>> of git-{tar,zip}-tree commands.\n>\n> OK, makes sense; users will never need to call git-upload-archive\n> directly, and having a single command for all archiving is a good thing.\n\nSo to recap:\n\n - \"git-archive-tree --format <foo> <foo specific options> <tree>\"\n   would know how to create <foo> format archive and send the result\n   to stdout.\n\n - \"git-download-archive <repo> <git-archive-tree-command-line>\"\n   would talk with \"git-upload-archive\" in the remote repository,\n   give archive-tree command line to it and receives the result.\n\n - \"git-upload-archive <repo>\" is not used by the end user.\n   Underlying git-archive-tree command line options are sent\n   over the protocol from download-archive, just like upload-tar\n   does.\n\nIf this is what you mean, I think three of us are in agreement\nhere.\n\n> My next steps will be to make traverse_tree() support path specs, in\n> order to achieve feature parity with read_tree_recursive().  I hope that\n> the former keeps being significantly faster than the latter even after\n> that.\n\nA thing that have been bothering me for some time in pathspec\narea is that we have two (eh perhaps three) quite different\npathspec semantics.\n\n - diff-tree family (and anything based on revision.c including\n   git-log) is a strict prefix directory match (e.g. no\n   wildcards, and \"Documentation/howto\" matches the directory\n   but not \"Documentation/howto-index.sh\")\n\n - ls-files family (ls-files used to be an one odd man out, but\n   git-grep mimics it, and \"git-commit <paths>\" uses ls-files\n   internally so they form a family) is prefix match with\n   fnmatch match upon wildcard (e.g. \"Documentation/howto\" still\n   matches the directory but not \"Documentation/howto-index.sh\",\n   but you can say \"Documentation/howto*\" to match both, and you\n   can even say \"Document*\").\n\nThe former semantics is very friendly with \"struct tree_desc\"\nbased traversal.  Allowing pathspecs with ls-files style\nmatching would probably more useful but more work.  And there\nare at least four implementations of pathspec matcher with\nslightly different interfaces (ugh, sorry) if I am not mistaken:\nbuiltin-ls-files.c has one (match), tree-diff.c has another\n(interesting), builtin-grep.c has one (pathspec_matches), dir.c\nhas another (match_pathspec).\n\nWe might be able to share code with para-walk.c in \"pu\" (it is\ndesigned to walk zero or more trees optionally with index and\nworking tree in parallel and we would be using it to walk only\nsingle tree without index nor working tree).  That would give me\nan incentive to clean up the code ;-) Currently it is a ten\n\"WIP\" patch series.\n\nOne thing it attempts to do on the side is to consolidate the\ndiff-tree style ones into one by introducing another in\nread-cache.c (pathname_included) but it does not convert\nexisting users of other matchers to use it yet.\n"},{"id":"26322","messageId":"44FCA272.7040907@lsrfire.ath.cx","threadId":"5453","inReplyTo":"7vy7szmm1u.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Rene Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2006-09-04T22:02:26Z","receivedAt":"2006-09-04T22:02:26Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Junio C Hamano schrieb:\n> So to recap:\n> \n> - \"git-archive-tree --format <foo> <foo specific options> <tree>\" \n> would know how to create <foo> format archive and send the result to\n> stdout.\n> \n> - \"git-download-archive <repo> <git-archive-tree-command-line>\" would\n> talk with \"git-upload-archive\" in the remote repository, give\n> archive-tree command line to it and receives the result.\n> \n> - \"git-upload-archive <repo>\" is not used by the end user. Underlying\n> git-archive-tree command line options are sent over the protocol from\n> download-archive, just like upload-tar does.\n> \n> If this is what you mean, I think three of us are in agreement here.\n\nWell, this is just _one_ of the positions I've taken on this topic, I\nhave to admit.  Franck then convinced me that merging downloader and\narchiver into one command is nice for users (less commands to remember,\nkeeps existing --remote option) even if it doesn't make sense\ntechnically, because their implementations have nothing in common.\n\nThis is a bikeshed colour discussion, and I'm bad at it.  I'll shut up\non this and simply follow the directions above.\n\n>> My next steps will be to make traverse_tree() support path specs,\n>> in order to achieve feature parity with read_tree_recursive().  I\n>> hope that the former keeps being significantly faster than the\n>> latter even after that.\n> \n> A thing that have been bothering me for some time in pathspec area is\n> that we have two (eh perhaps three) quite different pathspec\n> semantics.\n> \n> - diff-tree family (and anything based on revision.c including \n> git-log) is a strict prefix directory match (e.g. no wildcards, and\n> \"Documentation/howto\" matches the directory but not\n> \"Documentation/howto-index.sh\")\n> \n> - ls-files family (ls-files used to be an one odd man out, but \n> git-grep mimics it, and \"git-commit <paths>\" uses ls-files internally\n> so they form a family) is prefix match with fnmatch match upon\n> wildcard (e.g. \"Documentation/howto\" still matches the directory but\n> not \"Documentation/howto-index.sh\", but you can say\n> \"Documentation/howto*\" to match both, and you can even say\n> \"Document*\").\n> \n> The former semantics is very friendly with \"struct tree_desc\" based\n> traversal.  Allowing pathspecs with ls-files style matching would\n> probably more useful but more work.  And there are at least four\n> implementations of pathspec matcher with slightly different\n> interfaces (ugh, sorry) if I am not mistaken: builtin-ls-files.c has\n> one (match), tree-diff.c has another (interesting), builtin-grep.c\n> has one (pathspec_matches), dir.c has another (match_pathspec).\n> \n> We might be able to share code with para-walk.c in \"pu\" (it is \n> designed to walk zero or more trees optionally with index and working\n> tree in parallel and we would be using it to walk only single tree\n> without index nor working tree).  That would give me an incentive to\n> clean up the code ;-) Currently it is a ten \"WIP\" patch series.\n> \n> One thing it attempts to do on the side is to consolidate the \n> diff-tree style ones into one by introducing another in read-cache.c\n> (pathname_included) but it does not convert existing users of other\n> matchers to use it yet.\n\nInteresting.  OK, I'll check out the existing implementations with an\neye on consolidation and also take a look around that scary place named\n'pu'. ;-)\n\nSo far I have failed in creating a traverse_tree() function with path\nspec match support which also is faster than read_tree_recursive().\nMaybe the speed difference between 'git-tar-tree' and 'git-archive-tree\n-ftar' is caused by something else.  I keep on trying.\n\nRené\n"},{"id":"26323","messageId":"7vwt8jl1dv.fsf@assigned-by-dhcp.cox.net","threadId":"5453","inReplyTo":"44FCA272.7040907@lsrfire.ath.cx","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-09-04T22:20:44Z","receivedAt":"2006-09-04T22:20:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Rene Scharfe <rene.scharfe@lsrfire.ath.cx> writes:\n\n> Junio C Hamano schrieb:\n>...\n> Well, this is just _one_ of the positions I've taken on this topic, I\n> have to admit.  Franck then convinced me that merging downloader and\n> archiver into one command is nice for users (less commands to remember,\n> keeps existing --remote option) even if it doesn't make sense\n> technically, because their implementations have nothing in common.\n\nAh, I was not following the thread closely and I agree with you\nand Franck now in that smaller number of commands the better.\n(IOW, download-archive in my 3-item list is not needed in the\nend user UI and can be implemented as \"git-archive-tree\n--remote=<repo>).\n\nThanks for clarification.\n"},{"id":"26355","messageId":"cda58cb80609050443j26f2af21q6f75b2b4042c7b15@mail.gmail.com","threadId":"5453","inReplyTo":"7vwt8jl1dv.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Franck Bui-Huu","fromEmail":"vagabon.xyz@gmail.com","sentAt":"2006-09-05T11:43:16Z","receivedAt":"2006-09-05T11:43:16Z","isPatch":true,"sender":{"key":"vagabon.xyz@gmail.com","avatar":null},"body":"2006/9/5, Junio C Hamano <junkio@cox.net>:\n> Rene Scharfe <rene.scharfe@lsrfire.ath.cx> writes:\n>\n> > Junio C Hamano schrieb:\n> >...\n> > Well, this is just _one_ of the positions I've taken on this topic, I\n> > have to admit.  Franck then convinced me that merging downloader and\n> > archiver into one command is nice for users (less commands to remember,\n> > keeps existing --remote option) even if it doesn't make sense\n> > technically, because their implementations have nothing in common.\n>\n> Ah, I was not following the thread closely and I agree with you\n> and Franck now in that smaller number of commands the better.\n> (IOW, download-archive in my 3-item list is not needed in the\n> end user UI and can be implemented as \"git-archive-tree\n> --remote=<repo>).\n>\n\nOK, I'm sending what I've been able to do so far. It has been done\nbefore you sent some feedbacks one day ago, so please forgive me if,\nfor the moment, I called \"git-archive\" instead of \"git-archive-tree\"\nor if I missed some of your points; I'll change that.\n\nI'm sending 2 patches, which are sent to get some comments. They're\nnot for inclusion in anyways.\n\nThe pathspec area has been pick up from Rene's initial patch. I have\nnot enough git's internal knowledge to improve it. But I'll be glad to\nlook at this later (if you have any clues for starting point BTW).\n\n-- \n               Franck\n"},{"id":"26429","messageId":"44FF0DDE.7030700@lsrfire.ath.cx","threadId":"5453","inReplyTo":"7vwt8mx8lb.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Rene Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2006-09-06T18:05:18Z","receivedAt":"2006-09-06T18:05:18Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Junio C Hamano schrieb:\n> Rene Scharfe <rene.scharfe@lsrfire.ath.cx> writes:\n> \n>> Currently git-archive-tree -f tar is slower than git-tar-tree.  This is\n>> because it is welded to the side of the existing code to minimize patch\n>> size, and I also suspect read_tree_recursive() to be quite a bit slower\n>> than builtin-tar-tree.c::traverse_tree().\n> \n> Yes, I suspect \"struct object\" and friends are very inefficient\n> to use for things like this.  \"struct tree_desc\" based traverser\n> is much preferred.\n\nTurns out the reason for git-archive-tree -f tar being 10% slower than\ngit-tar-tree was a stupid memory leak in write_tar_entry(). *blush*\nIt caused lots of brk() calls (i.e. system calls).\n\nIn order to simplify measurement, I commented out the body of\nwrite_entry(), which both git-archive-tree and git-tar-tree are calling\nto write their output.  The rest left is basically two pure tree\ntraversers, git-archive-tree using read_tree_recursive() and git-tar-tree\nusing its struct tree_desc based traverse_tree().\n\nI then let the two chew away on the kernel repository.  And as\nkcachegrind impressively shows, all we do with our trees and objects is\ndwarfed by inflate().  In both cases more than 96.6% of the cost lies\nwithin libz.  That's not too surprising, because archivers need to\ndecompress _all_ objects, not only trees.\n\nSo for git-archive we can pretty much chose which traverser to use based\non convenience.\n\nAs a second experiment I wrote a struct tree_desc based traverser plus a\nmatching read_tree_recursive() compatibility wrapper (included below for\nreference, not for inclusion) and compared the performance of\n'git-ls-tree -r -t' on the kernel repository with and without it.  The\nresult is that the relative cost of all functions from tree.c combined\ndecreased from 0.93% to 0.66%.  Ugh.\n\nSo while a struct tree_desc based traverser can be significantly faster\nthan read_tree_recursive(), as soon as you actually start to do something\nto the trees that difference pales to insignificance.\n\nRené\n\n\n\ndiff --git a/tree.c b/tree.c\nindex ea386e5..977a4aa 100644\n--- a/tree.c\n+++ b/tree.c\n@@ -4,6 +4,7 @@ #include \"blob.h\"\n #include \"commit.h\"\n #include \"tag.h\"\n #include \"tree-walk.h\"\n+#include \"strbuf.h\"\n #include <stdlib.h>\n \n const char *tree_type = \"tree\";\n@@ -227,3 +228,99 @@ struct tree *parse_tree_indirect(const u\n \t\t\tparse_object(obj->sha1);\n \t} while (1);\n }\n+\n+static int do_read_tree_recursive_light(struct tree_desc *desc,\n+                                        struct strbuf *base,\n+                                        const char **match, read_tree_fn_t fn)\n+{\n+\tstruct name_entry entry;\n+\tint err = 0;\n+\tint baselen = base->len;\n+\n+\twhile (tree_entry(desc, &entry)) {\n+\t\tif (!match_tree_entry(base->buf, base->len, entry.path, entry.mode, match))\n+\t\t\tcontinue;\n+\n+\t\terr = fn(entry.sha1, base->buf, base->len, entry.path, entry.mode, 0);\n+\t\tswitch (err) {\n+\t\tcase 0:\n+\t\t\tcontinue;\n+\t\tcase READ_TREE_RECURSIVE:\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\treturn -1;\n+\t\t}\n+\n+\t\tif (S_ISDIR(entry.mode)) {\n+\t\t\tstruct tree_desc subtree;\n+\t\t\tchar type[20];\n+\t\t\tvoid *buf;\n+\t\t\tint newbaselen;\n+\n+\t\t\tbuf = read_sha1_file(entry.sha1, type, &subtree.size);\n+\t\t\tif (!buf)\n+\t\t\t\treturn error(\"cannot read %s\",\n+\t\t\t\t             sha1_to_hex(entry.sha1));\n+\t\t\tif (strcmp(type, tree_type)) {\n+\t\t\t\tfree(buf);\n+\t\t\t\treturn error(\"Object %s not a tree\",\n+\t\t\t\t             sha1_to_hex(entry.sha1));\n+\t\t\t}\n+\t\t\tsubtree.buf = buf;\n+\n+\t\t\tnewbaselen = baselen + entry.pathlen + 1;\n+\t\t\tif (newbaselen > base->alloc) {\n+\t\t\t\tbase->buf = xrealloc(base->buf, newbaselen);\n+\t\t\t\tbase->alloc = newbaselen;\n+\t\t\t}\n+\t\t\tmemcpy(base->buf + baselen, entry.path, entry.pathlen);\n+\t\t\tbase->buf[baselen + entry.pathlen] = '/';\n+\t\t\tbase->len = newbaselen;\n+\n+\t\t\terr = do_read_tree_recursive_light(&subtree,\n+\t\t\t                                   base,\n+\t\t\t                                   match, fn);\n+\t\t\tbase->len = baselen;\n+\t\t\tfree(buf);\n+\t\t\tif (err)\n+\t\t\t\tbreak;\n+\t\t}\n+\t}\n+\n+\treturn err;\n+}\n+\n+int read_tree_recursive_light(struct tree *tree,\n+                              const char *base, int baselen, int stage,\n+                              const char **match, read_tree_fn_t fn)\n+{\n+\tunsigned char *sha1 = tree->object.sha1;\n+\tstruct tree_desc desc;\n+\tchar type[20];\n+\tvoid *buf;\n+\tint err;\n+\tstruct strbuf sb;\n+\n+\tsb.buf = xmalloc(PATH_MAX);\n+\tsb.alloc = PATH_MAX;\n+\tsb.len = 0;\n+\tif (baselen > sb.alloc) {\n+\t\tsb.buf = xrealloc(sb.buf, baselen);\n+\t\tsb.alloc = baselen;\n+\t}\n+\tmemcpy(sb.buf, base, baselen);\n+\tsb.len = baselen;\n+\n+\tdesc.buf = buf = read_sha1_file(sha1, type, &desc.size);\n+\tif (!buf)\n+\t\treturn error(\"Could not read %s\", sha1_to_hex(sha1));\n+\tif (strcmp(type, tree_type)) {\n+\t\tfree(buf);\n+\t\treturn error(\"Object %s not a tree\", sha1_to_hex(sha1));\n+\t}\n+\n+\terr = do_read_tree_recursive_light(&desc, &sb, match, fn);\n+\tfree(buf);\n+\n+\treturn err;\n+}\ndiff --git a/tree.h b/tree.h\nindex dd25c53..2294bc2 100644\n--- a/tree.h\n+++ b/tree.h\n@@ -30,4 +30,8 @@ extern int read_tree_recursive(struct tr\n \n extern int read_tree(struct tree *tree, int stage, const char **paths);\n \n+extern int read_tree_recursive_light(struct tree *tree,\n+                                     const char *base, int baselen, int stage,\n+                                     const char **match, read_tree_fn_t fn);\n+\n #endif /* TREE_H */\n"},{"id":"26448","messageId":"7v4pvk7jmn.fsf@assigned-by-dhcp.cox.net","threadId":"5453","inReplyTo":"44FF0DDE.7030700@lsrfire.ath.cx","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-09-06T21:47:12Z","receivedAt":"2006-09-06T21:47:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Rene Scharfe <rene.scharfe@lsrfire.ath.cx> writes:\n\n> I then let the two chew away on the kernel repository.  And as\n> kcachegrind impressively shows, all we do with our trees and objects is\n> dwarfed by inflate().\n\nThe diff output codepath has a logic that says \"if the blob we\nare dealing with has the same object name as the corresponding\nblob in the index, and if the index entry is clean (i.e. it is\nknown that the file sitting in the working tree matches the\nblob), then do not inflate() but use data from that file\ninstead\".  It was originally done that way because we used to\nprepare temporary files out of blob and fed them to GNU diff,\nbut I think it is still kept that way.  I wonder if doing the\nsame may be cheaper for archivers.\n"},{"id":"27072","messageId":"450D3765.9090303@lsrfire.ath.cx","threadId":"5453","inReplyTo":"7v4pvk7jmn.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH][RFC] Add git-archive-tree","fromName":"Rene Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2006-09-17T11:54:13Z","receivedAt":"2006-09-17T11:54:13Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Junio C Hamano schrieb:\n> Rene Scharfe <rene.scharfe@lsrfire.ath.cx> writes:\n> \n>> I then let the two chew away on the kernel repository.  And as \n>> kcachegrind impressively shows, all we do with our trees and \n>> objects is dwarfed by inflate().\n> \n> The diff output codepath has a logic that says \"if the blob we are \n> dealing with has the same object name as the corresponding blob in \n> the index, and if the index entry is clean (i.e. it is known that the\n>  file sitting in the working tree matches the blob), then do not \n> inflate() but use data from that file instead\".\n\nNice idea.  The tree traverser would need to provide the filenames\nrelative to the current working directory in addition to the\nfilenames as they are written to the archive.  I guess your para-walk\ntree walker could be useful here.  I sadly haven't found the time to\nlook at it, yet, and now it even vanished from the pu branch.\n\nA read is an order of magnitude faster than a deflate of the same data,\nat least that's what I guess from comparing the runtimes of git-tar-tree\nand tar.  _However_, this doesn't account for I/O costs (in my tests the\nrepo and all checked-out files were cache hot) and for any compression\nthat would certainly be applied to the resulting archive.  So the full\nruntime of archive creation wouldn't be that much shorter.\n\nRené\n"}]}