{"thread":{"id":"280","subject":"[PATCH] Add archive-tree, a cpio archive creator","startedAt":"2005-04-24T01:43:46Z","lastAt":"2005-04-26T14:42:04Z","messageCount":3,"participants":["Rene Scharfe","Paul Dickson"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"1460","messageId":"20050424014346.GA23083@lsrfire.ath.cx","threadId":"280","inReplyTo":null,"subject":"[PATCH] Add archive-tree, a cpio archive creator","fromName":"Rene Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2005-04-24T01:43:46Z","receivedAt":"2005-04-24T01:43:46Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"This patch adds archive-tree, a program to create a cpio archive of all\nfiles referenced by a given tree or commit object.  Why cpio and not\ne.g., tar?  The cpio format I chose (GNU cpio calls it \"odc\") is dead\nsimple compared to any tar variant.  I was simply being lazy. :-P\n\narchive-tree optionally accepts a basedir, i.e. a prefix added to all\nnames in the archive.  Example usage:\n\n   $ ./archive-tree `tree-id` frob-1.1 | gzip -9 > frob-1.1.cpio.gz\n\nDirectory permissions are not handled properly, yet.  All directories\nget a mode of (at least) 0755.\n\nSigned-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>\n\n--- git-pasky/Makefile~\t2005-04-20 01:43:51.000000000 +0200\n+++ git-pasky/Makefile\t2005-04-24 02:02:22.000000000 +0200\n@@ -29,7 +29,7 @@\n PROG=   update-cache show-diff init-db write-tree read-tree commit-tree \\\n \tcat-file fsck-cache checkout-cache diff-tree rev-tree show-files \\\n \tcheck-files ls-tree merge-base merge-cache unpack-file git-export \\\n-\tdiff-cache convert-cache\n+\tdiff-cache convert-cache archive-tree\n \n SCRIPT=\tparent-id tree-id git gitXnormid.sh gitadd.sh gitaddremote.sh \\\n \tgitcommit.sh gitdiff-do gitdiff.sh gitlog.sh gitls.sh gitlsobj.sh \\\n--- /dev/null\t2004-12-03 19:50:49.000000000 +0100\n+++ git-pasky/archive-tree.c\t2005-04-24 03:31:39.000000000 +0200\n@@ -0,0 +1,206 @@\n+/*\n+ * GIT - The information manager from hell\n+ *\n+ * Copyright (C) Linus Torvalds, 2005\n+ */\n+#include \"cache.h\"\n+\n+const char *basedir = NULL;\n+\n+struct path_prefix {\n+\tstruct path_prefix *prev;\n+\tconst char *name;\n+};\n+\n+static void append_string(char **buffer, const char *s)\n+{\n+\tunsigned int len = strlen(s);\n+\tmemcpy(*buffer, s, len);\n+\t*buffer += len;\n+}\n+\n+static void append_char(char **buffer, char c)\n+{\n+\t**buffer = c;\n+\t*buffer += 1;\n+}\n+\n+static unsigned int path_prefix_len(struct path_prefix *prefix)\n+{\n+\tif (!prefix)\n+\t\treturn 0;\n+\treturn path_prefix_len(prefix->prev) + strlen(prefix->name) + 1;\n+}\n+\n+static void append_path_prefix(char **buffer, struct path_prefix *prefix)\n+{\n+\tif (!prefix)\n+\t\treturn;\n+\tappend_path_prefix(buffer, prefix->prev);\n+\tappend_string(buffer, prefix->name);\n+\tappend_char(buffer, '/');\n+}\n+\n+static unsigned long loopy_write(void *buf, unsigned long size)\n+{\n+\twhile (size > 0) {\n+\t\tlong ret = write(1, buf, size);\n+\t\tif (ret < 0) {\n+\t\t\tif (errno == EAGAIN)\n+\t\t\t\tcontinue;\n+\t\t\t/* Ignore epipe */\n+\t\t\tif (errno == EPIPE)\n+\t\t\t\tbreak;\n+\t\t\tdie(\"archive-tree: %s\", strerror(errno));\n+\t\t} else if (!ret) {\n+\t\t\tdie(\"archive-tree: disk full?\");\n+\t\t}\n+\t\tsize -= ret;\n+\t\tbuf += ret;\n+\t}\n+\treturn size;\n+}\n+\n+static long write_header(const char *basepath,\n+                         struct path_prefix *prefix,\n+                         const char *path,\n+                         unsigned int mode,\n+                         unsigned long size)\n+{\n+\tunsigned int namelen; \n+\tchar *header, *p;\n+\tlong ret;\n+\n+\tnamelen = path_prefix_len(prefix) + strlen(path) + 1;\n+\tif (basepath)\n+\t\tnamelen += strlen(basepath) + 1;\n+\n+\theader = malloc(76 + namelen);\n+\tif (!header)\n+\t\tdie(\"archive-tree: %s\", strerror(errno));\n+\n+\tsprintf(header, \"%06o%06o%06o%06o%06o%06o%06o%06o%011o%06o%011lo\",\n+\t\t070707,\t\t/* magic */\n+\t\t0,\t\t/* dev */\n+\t\t0,\t\t/* ino */\n+\t\tS_ISDIR(mode) ? mode | 0755 : mode,\t/* mode */ /*FIXME*/\n+\t\t0,\t\t/* uid */\n+\t\t0,\t\t/* gid */\n+\t\t1,\t\t/* nlink */\n+\t\t0,\t\t/* rdev */\n+\t\t0,\t\t/* mtime */\n+\t\tnamelen,\t/* namesize */\n+\t\tS_ISDIR(mode) ? 0 : size);\t/* filesize */\n+\n+\tp = header + 76;\n+\tif (basepath) {\n+\t\tappend_string(&p, basepath);\n+\t\tappend_char(&p, '/');\n+\t}\n+\tappend_path_prefix(&p, prefix);\n+\tappend_string(&p, path);\n+\tappend_char(&p, '\\0');\n+\n+\tret = loopy_write(header, 76 + namelen);\n+\tfree(header);\n+\treturn ret;\n+}\n+\n+static void write_trailer(void)\n+{\n+\twrite_header(NULL, NULL, \"TRAILER!!!\", 0, 0);\n+}\n+\n+static void list_recursive(void *buffer,\n+\t\t\t   const unsigned char *type,\n+\t\t\t   unsigned long size,\n+\t\t\t   struct path_prefix *prefix)\n+{\n+\tstruct path_prefix this_prefix;\n+\tthis_prefix.prev = prefix;\n+\n+\tif (strcmp(type, \"tree\"))\n+\t\tdie(\"expected a 'tree' node\");\n+\n+\twhile (size) {\n+\t\tint namelen = strlen(buffer)+1;\n+\t\tvoid *eltbuf;\n+\t\tchar elttype[20];\n+\t\tunsigned long eltsize;\n+\t\tunsigned char *sha1 = buffer + namelen;\n+\t\tchar *path = strchr(buffer, ' ') + 1;\n+\t\tunsigned int mode;\n+\n+\t\tif (size < namelen + 20 || sscanf(buffer, \"%o\", &mode) != 1)\n+\t\t\tdie(\"corrupt 'tree' file\");\n+\t\tbuffer = sha1 + 20;\n+\t\tsize -= namelen + 20;\n+\n+\t\tif (! (eltbuf = read_sha1_file(sha1, elttype, &eltsize)) ) {\n+\t\t\terror(\"cannot read %s\", sha1_to_hex(sha1));\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tif (write_header(basedir, prefix, path, mode, eltsize))\n+\t\t\tdie(\"archive-tree: could not write header\");\n+\n+\t\tif (S_ISDIR(mode)) {\n+\t\t\tthis_prefix.name = path;\n+\t\t\tlist_recursive(eltbuf, elttype, eltsize, &this_prefix);\n+\t\t} else {\n+\t\t\tif (loopy_write(eltbuf, eltsize))\n+\t\t\t\tdie(\"archive-tree: could not write body\");\n+\t\t}\n+\n+\t\tfree(eltbuf);\n+\t}\n+}\n+\n+static int list(unsigned char *sha1)\n+{\n+\tvoid *buffer;\n+\tunsigned long size;\n+\n+\tbuffer = read_tree_with_tree_or_commit_sha1(sha1, &size, 0);\n+\tif (!buffer)\n+\t\tdie(\"unable to read sha1 file\");\n+\tif (basedir)\n+\t\twrite_header(NULL, NULL, basedir, 040755, 0);\t/*FIXME*/\n+\tlist_recursive(buffer, \"tree\", size, NULL);\n+\tfree(buffer);\n+\treturn 0;\n+}\n+\n+static const char *archive_tree_usage = \"archive-tree <key> [basedir]\";\n+\n+int main(int argc, char **argv)\n+{\n+\tunsigned char sha1[20];\n+\n+\twhile (1 < argc && argv[1][0] == '-') {\n+\t\tswitch (argv[1][1]) {\n+\t\tdefault:\n+\t\t\tusage(archive_tree_usage);\n+\t\t}\n+\t\targc--; argv++;\n+\t}\n+\n+\tswitch (argc) {\n+\t\tcase 3:\n+\t\t\tbasedir = argv[2];\n+\t\t\t/* FALLTHROUGH */\n+\t\tcase 2:\n+\t\t\tif (get_sha1_hex(argv[1], sha1) < 0)\n+\t\t\t\tusage(archive_tree_usage);\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tusage(archive_tree_usage);\n+\t}\n+\tsha1_file_directory = getenv(DB_ENVIRONMENT);\n+\tif (!sha1_file_directory)\n+\t\tsha1_file_directory = DEFAULT_DB_ENVIRONMENT;\n+\tif (list(sha1) < 0)\n+\t\tdie(\"list failed\");\n+\twrite_trailer();\n+\treturn 0;\n+}\n"},{"id":"1669","messageId":"20050425153011.34c93b38.paul@permanentmail.com","threadId":"280","inReplyTo":"20050424014346.GA23083@lsrfire.ath.cx","subject":"Re: [PATCH] Add archive-tree, a cpio archive creator","fromName":"Paul Dickson","fromEmail":"paul@permanentmail.com","sentAt":"2005-04-25T22:30:11Z","receivedAt":"2005-04-25T22:30:11Z","isPatch":true,"sender":{"key":"paul@permanentmail.com","avatar":null},"body":"On Sun, 24 Apr 2005 03:43:46 +0200, Rene Scharfe wrote:\n\n> This patch adds archive-tree, a program to create a cpio archive of all\n> files referenced by a given tree or commit object.  Why cpio and not\n> e.g., tar?  The cpio format I chose (GNU cpio calls it \"odc\") is dead\n> simple compared to any tar variant.  I was simply being lazy. :-P\n\nUse the \"newc\" format.  The \"odc\" will generate a lot of errors on a\nlarge filesystem (which I discovered this past weekend).\n\n\t-Paul\n\n"},{"id":"1737","messageId":"426E533C.70502@lsrfire.ath.cx","threadId":"280","inReplyTo":"20050425153011.34c93b38.paul@permanentmail.com","subject":"Re: [PATCH] Add archive-tree, a cpio archive creator","fromName":"Rene Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2005-04-26T14:42:04Z","receivedAt":"2005-04-26T14:42:04Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Paul Dickson schrieb:\n> Use the \"newc\" format.  The \"odc\" will generate a lot of errors on a\n>  large filesystem (which I discovered this past weekend).\n\nHow is that?  I can only imagine problems with the size of individual\nfiles inside an archive because the size field in the header is limited\nto 33 bits.\n\nIn any case, I switched to the tar format.  It's more complicated but it\nturned out that creating these things is still easy.  Parsing them might\nbe more of a challenge, but we already have tar for that. :-)\n\nSee my other mail for more on the tar creator.  I also rebased it to\ncore GIT because it's no helper script.\n\nRene\n"}]}