{"thread":{"id":"4174","subject":"[PATCH] libify git-ls-files directory traversal","startedAt":"2006-05-17T02:02:14Z","lastAt":"2006-05-17T04:06:49Z","messageCount":4,"participants":["Linus Torvalds"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"20097","messageId":"Pine.LNX.4.64.0605161859050.16475@g5.osdl.org","threadId":"4174","inReplyTo":null,"subject":"[PATCH] libify git-ls-files directory traversal","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-05-17T02:02:14Z","receivedAt":"2006-05-17T02:02:14Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\nThis moves the core directory traversal and filename exclusion logic\ninto the general git library, making it available for other users\ndirectly.\n\nIf we ever want to do \"git commit\" or \"git add\" as a built-in (and we\ndo), we want to be able to handle most of git-ls-files as a library.\n\nNOTE! Not all of git-ls-files is libified by this.  The index matching\nand pathspec prefix calculation is still in ls-files.c, but this is a\nbig part of it.\n\nSigned-off-by: Linus Torvalds <torvalds@osdl.org>\n---\n Makefile   |    4 -\n dir.c      |  295 +++++++++++++++++++++++++++++++++++++++++++++++++\n dir.h      |   50 ++++++++\n ls-files.c |  363 +++++-------------------------------------------------------\n 4 files changed, 376 insertions(+), 336 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 9ba608c..f43ac63 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -199,7 +199,7 @@ LIB_H = \\\n \tblob.h cache.h commit.h csum-file.h delta.h \\\n \tdiff.h object.h pack.h pkt-line.h quote.h refs.h \\\n \trun-command.h strbuf.h tag.h tree.h git-compat-util.h revision.h \\\n-\ttree-walk.h log-tree.h\n+\ttree-walk.h log-tree.h dir.h\n \n DIFF_OBJS = \\\n \tdiff.o diff-lib.o diffcore-break.o diffcore-order.o \\\n@@ -210,7 +210,7 @@ LIB_OBJS = \\\n \tblob.o commit.o connect.o csum-file.o base85.o \\\n \tdate.o diff-delta.o entry.o exec_cmd.o ident.o index.o \\\n \tobject.o pack-check.o patch-delta.o path.o pkt-line.o \\\n-\tquote.o read-cache.o refs.o run-command.o \\\n+\tquote.o read-cache.o refs.o run-command.o dir.o \\\n \tserver-info.o setup.o sha1_file.o sha1_name.o strbuf.o \\\n \ttag.o tree.o usage.o config.o environment.o ctype.o copy.o \\\n \tfetch-clone.o revision.o pager.o tree-walk.o xdiff-interface.o \\\ndiff --git a/dir.c b/dir.c\nnew file mode 100644\nindex 0000000..3f41a5d\n--- /dev/null\n+++ b/dir.c\n@@ -0,0 +1,295 @@\n+/*\n+ * This handles recursive filename detection with exclude\n+ * files, index knowledge etc..\n+ *\n+ * Copyright (C) Linus Torvalds, 2005-2006\n+ *\t\t Junio Hamano, 2005-2006\n+ */\n+#include <dirent.h>\n+#include <fnmatch.h>\n+\n+#include \"cache.h\"\n+#include \"dir.h\"\n+\n+void add_exclude(const char *string, const char *base,\n+\t\t int baselen, struct exclude_list *which)\n+{\n+\tstruct exclude *x = xmalloc(sizeof (*x));\n+\n+\tx->pattern = string;\n+\tx->base = base;\n+\tx->baselen = baselen;\n+\tif (which->nr == which->alloc) {\n+\t\twhich->alloc = alloc_nr(which->alloc);\n+\t\twhich->excludes = realloc(which->excludes,\n+\t\t\t\t\t  which->alloc * sizeof(x));\n+\t}\n+\twhich->excludes[which->nr++] = x;\n+}\n+\n+static int add_excludes_from_file_1(const char *fname,\n+\t\t\t\t    const char *base,\n+\t\t\t\t    int baselen,\n+\t\t\t\t    struct exclude_list *which)\n+{\n+\tint fd, i;\n+\tlong size;\n+\tchar *buf, *entry;\n+\n+\tfd = open(fname, O_RDONLY);\n+\tif (fd < 0)\n+\t\tgoto err;\n+\tsize = lseek(fd, 0, SEEK_END);\n+\tif (size < 0)\n+\t\tgoto err;\n+\tlseek(fd, 0, SEEK_SET);\n+\tif (size == 0) {\n+\t\tclose(fd);\n+\t\treturn 0;\n+\t}\n+\tbuf = xmalloc(size+1);\n+\tif (read(fd, buf, size) != size)\n+\t\tgoto err;\n+\tclose(fd);\n+\n+\tbuf[size++] = '\\n';\n+\tentry = buf;\n+\tfor (i = 0; i < size; i++) {\n+\t\tif (buf[i] == '\\n') {\n+\t\t\tif (entry != buf + i && entry[0] != '#') {\n+\t\t\t\tbuf[i - (i && buf[i-1] == '\\r')] = 0;\n+\t\t\t\tadd_exclude(entry, base, baselen, which);\n+\t\t\t}\n+\t\t\tentry = buf + i + 1;\n+\t\t}\n+\t}\n+\treturn 0;\n+\n+ err:\n+\tif (0 <= fd)\n+\t\tclose(fd);\n+\treturn -1;\n+}\n+\n+void add_excludes_from_file(struct dir_struct *dir, const char *fname)\n+{\n+\tif (add_excludes_from_file_1(fname, \"\", 0,\n+\t\t\t\t     &dir->exclude_list[EXC_FILE]) < 0)\n+\t\tdie(\"cannot use %s as an exclude file\", fname);\n+}\n+\n+int push_exclude_per_directory(struct dir_struct *dir, const char *base, int baselen)\n+{\n+\tchar exclude_file[PATH_MAX];\n+\tstruct exclude_list *el = &dir->exclude_list[EXC_DIRS];\n+\tint current_nr = el->nr;\n+\n+\tif (dir->exclude_per_dir) {\n+\t\tmemcpy(exclude_file, base, baselen);\n+\t\tstrcpy(exclude_file + baselen, dir->exclude_per_dir);\n+\t\tadd_excludes_from_file_1(exclude_file, base, baselen, el);\n+\t}\n+\treturn current_nr;\n+}\n+\n+static void pop_exclude_per_directory(struct dir_struct *dir, int stk)\n+{\n+\tstruct exclude_list *el = &dir->exclude_list[EXC_DIRS];\n+\n+\twhile (stk < el->nr)\n+\t\tfree(el->excludes[--el->nr]);\n+}\n+\n+/* Scan the list and let the last match determines the fate.\n+ * Return 1 for exclude, 0 for include and -1 for undecided.\n+ */\n+static int excluded_1(const char *pathname,\n+\t\t      int pathlen,\n+\t\t      struct exclude_list *el)\n+{\n+\tint i;\n+\n+\tif (el->nr) {\n+\t\tfor (i = el->nr - 1; 0 <= i; i--) {\n+\t\t\tstruct exclude *x = el->excludes[i];\n+\t\t\tconst char *exclude = x->pattern;\n+\t\t\tint to_exclude = 1;\n+\n+\t\t\tif (*exclude == '!') {\n+\t\t\t\tto_exclude = 0;\n+\t\t\t\texclude++;\n+\t\t\t}\n+\n+\t\t\tif (!strchr(exclude, '/')) {\n+\t\t\t\t/* match basename */\n+\t\t\t\tconst char *basename = strrchr(pathname, '/');\n+\t\t\t\tbasename = (basename) ? basename+1 : pathname;\n+\t\t\t\tif (fnmatch(exclude, basename, 0) == 0)\n+\t\t\t\t\treturn to_exclude;\n+\t\t\t}\n+\t\t\telse {\n+\t\t\t\t/* match with FNM_PATHNAME:\n+\t\t\t\t * exclude has base (baselen long) implicitly\n+\t\t\t\t * in front of it.\n+\t\t\t\t */\n+\t\t\t\tint baselen = x->baselen;\n+\t\t\t\tif (*exclude == '/')\n+\t\t\t\t\texclude++;\n+\n+\t\t\t\tif (pathlen < baselen ||\n+\t\t\t\t    (baselen && pathname[baselen-1] != '/') ||\n+\t\t\t\t    strncmp(pathname, x->base, baselen))\n+\t\t\t\t    continue;\n+\n+\t\t\t\tif (fnmatch(exclude, pathname+baselen,\n+\t\t\t\t\t    FNM_PATHNAME) == 0)\n+\t\t\t\t\treturn to_exclude;\n+\t\t\t}\n+\t\t}\n+\t}\n+\treturn -1; /* undecided */\n+}\n+\n+int excluded(struct dir_struct *dir, const char *pathname)\n+{\n+\tint pathlen = strlen(pathname);\n+\tint st;\n+\n+\tfor (st = EXC_CMDL; st <= EXC_FILE; st++) {\n+\t\tswitch (excluded_1(pathname, pathlen, &dir->exclude_list[st])) {\n+\t\tcase 0:\n+\t\t\treturn 0;\n+\t\tcase 1:\n+\t\t\treturn 1;\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n+static void add_name(struct dir_struct *dir, const char *pathname, int len)\n+{\n+\tstruct dir_entry *ent;\n+\n+\tif (cache_name_pos(pathname, len) >= 0)\n+\t\treturn;\n+\n+\tif (dir->nr == dir->alloc) {\n+\t\tint alloc = alloc_nr(dir->alloc);\n+\t\tdir->alloc = alloc;\n+\t\tdir->entries = xrealloc(dir->entries, alloc*sizeof(ent));\n+\t}\n+\tent = xmalloc(sizeof(*ent) + len + 1);\n+\tent->len = len;\n+\tmemcpy(ent->name, pathname, len);\n+\tent->name[len] = 0;\n+\tdir->entries[dir->nr++] = ent;\n+}\n+\n+static int dir_exists(const char *dirname, int len)\n+{\n+\tint pos = cache_name_pos(dirname, len);\n+\tif (pos >= 0)\n+\t\treturn 1;\n+\tpos = -pos-1;\n+\tif (pos >= active_nr) /* can't */\n+\t\treturn 0;\n+\treturn !strncmp(active_cache[pos]->name, dirname, len);\n+}\n+\n+/*\n+ * Read a directory tree. We currently ignore anything but\n+ * directories, regular files and symlinks. That's because git\n+ * doesn't handle them at all yet. Maybe that will change some\n+ * day.\n+ *\n+ * Also, we ignore the name \".git\" (even if it is not a directory).\n+ * That likely will not change.\n+ */\n+static int read_directory_recursive(struct dir_struct *dir, const char *path, const char *base, int baselen)\n+{\n+\tDIR *fdir = opendir(path);\n+\tint contents = 0;\n+\n+\tif (fdir) {\n+\t\tint exclude_stk;\n+\t\tstruct dirent *de;\n+\t\tchar fullname[MAXPATHLEN + 1];\n+\t\tmemcpy(fullname, base, baselen);\n+\n+\t\texclude_stk = push_exclude_per_directory(dir, base, baselen);\n+\n+\t\twhile ((de = readdir(fdir)) != NULL) {\n+\t\t\tint len;\n+\n+\t\t\tif ((de->d_name[0] == '.') &&\n+\t\t\t    (de->d_name[1] == 0 ||\n+\t\t\t     !strcmp(de->d_name + 1, \".\") ||\n+\t\t\t     !strcmp(de->d_name + 1, \"git\")))\n+\t\t\t\tcontinue;\n+\t\t\tlen = strlen(de->d_name);\n+\t\t\tmemcpy(fullname + baselen, de->d_name, len+1);\n+\t\t\tif (excluded(dir, fullname) != dir->show_ignored) {\n+\t\t\t\tif (!dir->show_ignored || DTYPE(de) != DT_DIR) {\n+\t\t\t\t\tcontinue;\n+\t\t\t\t}\n+\t\t\t}\n+\n+\t\t\tswitch (DTYPE(de)) {\n+\t\t\tstruct stat st;\n+\t\t\tint subdir, rewind_base;\n+\t\t\tdefault:\n+\t\t\t\tcontinue;\n+\t\t\tcase DT_UNKNOWN:\n+\t\t\t\tif (lstat(fullname, &st))\n+\t\t\t\t\tcontinue;\n+\t\t\t\tif (S_ISREG(st.st_mode) || S_ISLNK(st.st_mode))\n+\t\t\t\t\tbreak;\n+\t\t\t\tif (!S_ISDIR(st.st_mode))\n+\t\t\t\t\tcontinue;\n+\t\t\t\t/* fallthrough */\n+\t\t\tcase DT_DIR:\n+\t\t\t\tmemcpy(fullname + baselen + len, \"/\", 2);\n+\t\t\t\tlen++;\n+\t\t\t\trewind_base = dir->nr;\n+\t\t\t\tsubdir = read_directory_recursive(dir, fullname, fullname,\n+\t\t\t\t                        baselen + len);\n+\t\t\t\tif (dir->show_other_directories &&\n+\t\t\t\t    (subdir || !dir->hide_empty_directories) &&\n+\t\t\t\t    !dir_exists(fullname, baselen + len)) {\n+\t\t\t\t\t// Rewind the read subdirectory\n+\t\t\t\t\twhile (dir->nr > rewind_base)\n+\t\t\t\t\t\tfree(dir->entries[--dir->nr]);\n+\t\t\t\t\tbreak;\n+\t\t\t\t}\n+\t\t\t\tcontents += subdir;\n+\t\t\t\tcontinue;\n+\t\t\tcase DT_REG:\n+\t\t\tcase DT_LNK:\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tadd_name(dir, fullname, baselen + len);\n+\t\t\tcontents++;\n+\t\t}\n+\t\tclosedir(fdir);\n+\n+\t\tpop_exclude_per_directory(dir, exclude_stk);\n+\t}\n+\n+\treturn contents;\n+}\n+\n+static int cmp_name(const void *p1, const void *p2)\n+{\n+\tconst struct dir_entry *e1 = *(const struct dir_entry **)p1;\n+\tconst struct dir_entry *e2 = *(const struct dir_entry **)p2;\n+\n+\treturn cache_name_compare(e1->name, e1->len,\n+\t\t\t\t  e2->name, e2->len);\n+}\n+\n+int read_directory(struct dir_struct *dir, const char *path, const char *base, int baselen)\n+{\n+\tread_directory_recursive(dir, path, base, baselen);\n+\tqsort(dir->entries, dir->nr, sizeof(struct dir_entry *), cmp_name);\n+\treturn dir->nr;\n+}\ndiff --git a/dir.h b/dir.h\nnew file mode 100644\nindex 0000000..e8fc441\n--- /dev/null\n+++ b/dir.h\n@@ -0,0 +1,50 @@\n+#ifndef DIR_H\n+#define DIR_H\n+\n+/*\n+ * We maintain three exclude pattern lists:\n+ * EXC_CMDL lists patterns explicitly given on the command line.\n+ * EXC_DIRS lists patterns obtained from per-directory ignore files.\n+ * EXC_FILE lists patterns from fallback ignore files.\n+ */\n+#define EXC_CMDL 0\n+#define EXC_DIRS 1\n+#define EXC_FILE 2\n+\n+\n+struct dir_entry {\n+\tint len;\n+\tchar name[FLEX_ARRAY]; /* more */\n+};\n+\n+struct exclude_list {\n+\tint nr;\n+\tint alloc;\n+\tstruct exclude {\n+\t\tconst char *pattern;\n+\t\tconst char *base;\n+\t\tint baselen;\n+\t} **excludes;\n+};\n+\n+struct dir_struct {\n+\tint nr, alloc;\n+\tunsigned int show_ignored:1,\n+\t\t     show_other_directories:1,\n+\t\t     hide_empty_directories:1;\n+\tstruct dir_entry **entries;\n+\n+\t/* Exclude info */\n+\tconst char *exclude_per_dir;\n+\tstruct exclude_list exclude_list[3];\n+};\n+\n+extern int read_directory(struct dir_struct *, const char *path, const char *base, int baselen);\n+extern int excluded(struct dir_struct *, const char *);\n+extern void add_excludes_from_file(struct dir_struct *, const char *fname);\n+extern void add_exclude(const char *string, const char *base,\n+\t\t\tint baselen, struct exclude_list *which);\n+extern int push_exclude_per_directory(struct dir_struct *,\n+\t\t\t\t      const char *base, int baselen);\n+\n+#endif\ndiff --git a/ls-files.c b/ls-files.c\nindex 4a4af1c..89941a3 100644\n--- a/ls-files.c\n+++ b/ls-files.c\n@@ -5,23 +5,20 @@\n  *\n  * Copyright (C) Linus Torvalds, 2005\n  */\n-#include <dirent.h>\n #include <fnmatch.h>\n \n #include \"cache.h\"\n #include \"quote.h\"\n+#include \"dir.h\"\n \n static int abbrev = 0;\n static int show_deleted = 0;\n static int show_cached = 0;\n static int show_others = 0;\n-static int show_ignored = 0;\n static int show_stage = 0;\n static int show_unmerged = 0;\n static int show_modified = 0;\n static int show_killed = 0;\n-static int show_other_directories = 0;\n-static int hide_empty_directories = 0;\n static int show_valid_bit = 0;\n static int line_terminator = '\\n';\n \n@@ -38,309 +35,6 @@ static const char *tag_other = \"\";\n static const char *tag_killed = \"\";\n static const char *tag_modified = \"\";\n \n-static const char *exclude_per_dir = NULL;\n-\n-/* We maintain three exclude pattern lists:\n- * EXC_CMDL lists patterns explicitly given on the command line.\n- * EXC_DIRS lists patterns obtained from per-directory ignore files.\n- * EXC_FILE lists patterns from fallback ignore files.\n- */\n-#define EXC_CMDL 0\n-#define EXC_DIRS 1\n-#define EXC_FILE 2\n-static struct exclude_list {\n-\tint nr;\n-\tint alloc;\n-\tstruct exclude {\n-\t\tconst char *pattern;\n-\t\tconst char *base;\n-\t\tint baselen;\n-\t} **excludes;\n-} exclude_list[3];\n-\n-static void add_exclude(const char *string, const char *base,\n-\t\t\tint baselen, struct exclude_list *which)\n-{\n-\tstruct exclude *x = xmalloc(sizeof (*x));\n-\n-\tx->pattern = string;\n-\tx->base = base;\n-\tx->baselen = baselen;\n-\tif (which->nr == which->alloc) {\n-\t\twhich->alloc = alloc_nr(which->alloc);\n-\t\twhich->excludes = realloc(which->excludes,\n-\t\t\t\t\t  which->alloc * sizeof(x));\n-\t}\n-\twhich->excludes[which->nr++] = x;\n-}\n-\n-static int add_excludes_from_file_1(const char *fname,\n-\t\t\t\t    const char *base,\n-\t\t\t\t    int baselen,\n-\t\t\t\t    struct exclude_list *which)\n-{\n-\tint fd, i;\n-\tlong size;\n-\tchar *buf, *entry;\n-\n-\tfd = open(fname, O_RDONLY);\n-\tif (fd < 0)\n-\t\tgoto err;\n-\tsize = lseek(fd, 0, SEEK_END);\n-\tif (size < 0)\n-\t\tgoto err;\n-\tlseek(fd, 0, SEEK_SET);\n-\tif (size == 0) {\n-\t\tclose(fd);\n-\t\treturn 0;\n-\t}\n-\tbuf = xmalloc(size+1);\n-\tif (read(fd, buf, size) != size)\n-\t\tgoto err;\n-\tclose(fd);\n-\n-\tbuf[size++] = '\\n';\n-\tentry = buf;\n-\tfor (i = 0; i < size; i++) {\n-\t\tif (buf[i] == '\\n') {\n-\t\t\tif (entry != buf + i && entry[0] != '#') {\n-\t\t\t\tbuf[i - (i && buf[i-1] == '\\r')] = 0;\n-\t\t\t\tadd_exclude(entry, base, baselen, which);\n-\t\t\t}\n-\t\t\tentry = buf + i + 1;\n-\t\t}\n-\t}\n-\treturn 0;\n-\n- err:\n-\tif (0 <= fd)\n-\t\tclose(fd);\n-\treturn -1;\n-}\n-\n-static void add_excludes_from_file(const char *fname)\n-{\n-\tif (add_excludes_from_file_1(fname, \"\", 0,\n-\t\t\t\t     &exclude_list[EXC_FILE]) < 0)\n-\t\tdie(\"cannot use %s as an exclude file\", fname);\n-}\n-\n-static int push_exclude_per_directory(const char *base, int baselen)\n-{\n-\tchar exclude_file[PATH_MAX];\n-\tstruct exclude_list *el = &exclude_list[EXC_DIRS];\n-\tint current_nr = el->nr;\n-\n-\tif (exclude_per_dir) {\n-\t\tmemcpy(exclude_file, base, baselen);\n-\t\tstrcpy(exclude_file + baselen, exclude_per_dir);\n-\t\tadd_excludes_from_file_1(exclude_file, base, baselen, el);\n-\t}\n-\treturn current_nr;\n-}\n-\n-static void pop_exclude_per_directory(int stk)\n-{\n-\tstruct exclude_list *el = &exclude_list[EXC_DIRS];\n-\n-\twhile (stk < el->nr)\n-\t\tfree(el->excludes[--el->nr]);\n-}\n-\n-/* Scan the list and let the last match determines the fate.\n- * Return 1 for exclude, 0 for include and -1 for undecided.\n- */\n-static int excluded_1(const char *pathname,\n-\t\t      int pathlen,\n-\t\t      struct exclude_list *el)\n-{\n-\tint i;\n-\n-\tif (el->nr) {\n-\t\tfor (i = el->nr - 1; 0 <= i; i--) {\n-\t\t\tstruct exclude *x = el->excludes[i];\n-\t\t\tconst char *exclude = x->pattern;\n-\t\t\tint to_exclude = 1;\n-\n-\t\t\tif (*exclude == '!') {\n-\t\t\t\tto_exclude = 0;\n-\t\t\t\texclude++;\n-\t\t\t}\n-\n-\t\t\tif (!strchr(exclude, '/')) {\n-\t\t\t\t/* match basename */\n-\t\t\t\tconst char *basename = strrchr(pathname, '/');\n-\t\t\t\tbasename = (basename) ? basename+1 : pathname;\n-\t\t\t\tif (fnmatch(exclude, basename, 0) == 0)\n-\t\t\t\t\treturn to_exclude;\n-\t\t\t}\n-\t\t\telse {\n-\t\t\t\t/* match with FNM_PATHNAME:\n-\t\t\t\t * exclude has base (baselen long) implicitly\n-\t\t\t\t * in front of it.\n-\t\t\t\t */\n-\t\t\t\tint baselen = x->baselen;\n-\t\t\t\tif (*exclude == '/')\n-\t\t\t\t\texclude++;\n-\n-\t\t\t\tif (pathlen < baselen ||\n-\t\t\t\t    (baselen && pathname[baselen-1] != '/') ||\n-\t\t\t\t    strncmp(pathname, x->base, baselen))\n-\t\t\t\t    continue;\n-\n-\t\t\t\tif (fnmatch(exclude, pathname+baselen,\n-\t\t\t\t\t    FNM_PATHNAME) == 0)\n-\t\t\t\t\treturn to_exclude;\n-\t\t\t}\n-\t\t}\n-\t}\n-\treturn -1; /* undecided */\n-}\n-\n-static int excluded(const char *pathname)\n-{\n-\tint pathlen = strlen(pathname);\n-\tint st;\n-\n-\tfor (st = EXC_CMDL; st <= EXC_FILE; st++) {\n-\t\tswitch (excluded_1(pathname, pathlen, &exclude_list[st])) {\n-\t\tcase 0:\n-\t\t\treturn 0;\n-\t\tcase 1:\n-\t\t\treturn 1;\n-\t\t}\n-\t}\n-\treturn 0;\n-}\n-\n-struct nond_on_fs {\n-\tint len;\n-\tchar name[FLEX_ARRAY]; /* more */\n-};\n-\n-static struct nond_on_fs **dir;\n-static int nr_dir;\n-static int dir_alloc;\n-\n-static void add_name(const char *pathname, int len)\n-{\n-\tstruct nond_on_fs *ent;\n-\n-\tif (cache_name_pos(pathname, len) >= 0)\n-\t\treturn;\n-\n-\tif (nr_dir == dir_alloc) {\n-\t\tdir_alloc = alloc_nr(dir_alloc);\n-\t\tdir = xrealloc(dir, dir_alloc*sizeof(ent));\n-\t}\n-\tent = xmalloc(sizeof(*ent) + len + 1);\n-\tent->len = len;\n-\tmemcpy(ent->name, pathname, len);\n-\tent->name[len] = 0;\n-\tdir[nr_dir++] = ent;\n-}\n-\n-static int dir_exists(const char *dirname, int len)\n-{\n-\tint pos = cache_name_pos(dirname, len);\n-\tif (pos >= 0)\n-\t\treturn 1;\n-\tpos = -pos-1;\n-\tif (pos >= active_nr) /* can't */\n-\t\treturn 0;\n-\treturn !strncmp(active_cache[pos]->name, dirname, len);\n-}\n-\n-/*\n- * Read a directory tree. We currently ignore anything but\n- * directories, regular files and symlinks. That's because git\n- * doesn't handle them at all yet. Maybe that will change some\n- * day.\n- *\n- * Also, we ignore the name \".git\" (even if it is not a directory).\n- * That likely will not change.\n- */\n-static int read_directory(const char *path, const char *base, int baselen)\n-{\n-\tDIR *fdir = opendir(path);\n-\tint contents = 0;\n-\n-\tif (fdir) {\n-\t\tint exclude_stk;\n-\t\tstruct dirent *de;\n-\t\tchar fullname[MAXPATHLEN + 1];\n-\t\tmemcpy(fullname, base, baselen);\n-\n-\t\texclude_stk = push_exclude_per_directory(base, baselen);\n-\n-\t\twhile ((de = readdir(fdir)) != NULL) {\n-\t\t\tint len;\n-\n-\t\t\tif ((de->d_name[0] == '.') &&\n-\t\t\t    (de->d_name[1] == 0 ||\n-\t\t\t     !strcmp(de->d_name + 1, \".\") ||\n-\t\t\t     !strcmp(de->d_name + 1, \"git\")))\n-\t\t\t\tcontinue;\n-\t\t\tlen = strlen(de->d_name);\n-\t\t\tmemcpy(fullname + baselen, de->d_name, len+1);\n-\t\t\tif (excluded(fullname) != show_ignored) {\n-\t\t\t\tif (!show_ignored || DTYPE(de) != DT_DIR) {\n-\t\t\t\t\tcontinue;\n-\t\t\t\t}\n-\t\t\t}\n-\n-\t\t\tswitch (DTYPE(de)) {\n-\t\t\tstruct stat st;\n-\t\t\tint subdir, rewind_base;\n-\t\t\tdefault:\n-\t\t\t\tcontinue;\n-\t\t\tcase DT_UNKNOWN:\n-\t\t\t\tif (lstat(fullname, &st))\n-\t\t\t\t\tcontinue;\n-\t\t\t\tif (S_ISREG(st.st_mode) || S_ISLNK(st.st_mode))\n-\t\t\t\t\tbreak;\n-\t\t\t\tif (!S_ISDIR(st.st_mode))\n-\t\t\t\t\tcontinue;\n-\t\t\t\t/* fallthrough */\n-\t\t\tcase DT_DIR:\n-\t\t\t\tmemcpy(fullname + baselen + len, \"/\", 2);\n-\t\t\t\tlen++;\n-\t\t\t\trewind_base = nr_dir;\n-\t\t\t\tsubdir = read_directory(fullname, fullname,\n-\t\t\t\t                        baselen + len);\n-\t\t\t\tif (show_other_directories &&\n-\t\t\t\t    (subdir || !hide_empty_directories) &&\n-\t\t\t\t    !dir_exists(fullname, baselen + len)) {\n-\t\t\t\t\t// Rewind the read subdirectory\n-\t\t\t\t\twhile (nr_dir > rewind_base)\n-\t\t\t\t\t\tfree(dir[--nr_dir]);\n-\t\t\t\t\tbreak;\n-\t\t\t\t}\n-\t\t\t\tcontents += subdir;\n-\t\t\t\tcontinue;\n-\t\t\tcase DT_REG:\n-\t\t\tcase DT_LNK:\n-\t\t\t\tbreak;\n-\t\t\t}\n-\t\t\tadd_name(fullname, baselen + len);\n-\t\t\tcontents++;\n-\t\t}\n-\t\tclosedir(fdir);\n-\n-\t\tpop_exclude_per_directory(exclude_stk);\n-\t}\n-\n-\treturn contents;\n-}\n-\n-static int cmp_name(const void *p1, const void *p2)\n-{\n-\tconst struct nond_on_fs *e1 = *(const struct nond_on_fs **)p1;\n-\tconst struct nond_on_fs *e2 = *(const struct nond_on_fs **)p2;\n-\n-\treturn cache_name_compare(e1->name, e1->len,\n-\t\t\t\t  e2->name, e2->len);\n-}\n \n /*\n  * Match a pathspec against a filename. The first \"len\" characters\n@@ -377,7 +71,7 @@ static int match(const char **spec, char\n \treturn 0;\n }\n \n-static void show_dir_entry(const char *tag, struct nond_on_fs *ent)\n+static void show_dir_entry(const char *tag, struct dir_entry *ent)\n {\n \tint len = prefix_len;\n \tint offset = prefix_offset;\n@@ -393,14 +87,14 @@ static void show_dir_entry(const char *t\n \tputchar(line_terminator);\n }\n \n-static void show_other_files(void)\n+static void show_other_files(struct dir_struct *dir)\n {\n \tint i;\n-\tfor (i = 0; i < nr_dir; i++) {\n+\tfor (i = 0; i < dir->nr; i++) {\n \t\t/* We should not have a matching entry, but we\n \t\t * may have an unmerged entry for this path.\n \t\t */\n-\t\tstruct nond_on_fs *ent = dir[i];\n+\t\tstruct dir_entry *ent = dir->entries[i];\n \t\tint pos = cache_name_pos(ent->name, ent->len);\n \t\tstruct cache_entry *ce;\n \t\tif (0 <= pos)\n@@ -416,11 +110,11 @@ static void show_other_files(void)\n \t}\n }\n \n-static void show_killed_files(void)\n+static void show_killed_files(struct dir_struct *dir)\n {\n \tint i;\n-\tfor (i = 0; i < nr_dir; i++) {\n-\t\tstruct nond_on_fs *ent = dir[i];\n+\tfor (i = 0; i < dir->nr; i++) {\n+\t\tstruct dir_entry *ent = dir->entries[i];\n \t\tchar *cp, *sp;\n \t\tint pos, len, killed = 0;\n \n@@ -461,7 +155,7 @@ static void show_killed_files(void)\n \t\t\t}\n \t\t}\n \t\tif (killed)\n-\t\t\tshow_dir_entry(tag_killed, dir[i]);\n+\t\t\tshow_dir_entry(tag_killed, dir->entries[i]);\n \t}\n }\n \n@@ -512,7 +206,7 @@ static void show_ce_entry(const char *ta\n \t}\n }\n \n-static void show_files(void)\n+static void show_files(struct dir_struct *dir)\n {\n \tint i;\n \n@@ -523,14 +217,14 @@ static void show_files(void)\n \n \t\tif (baselen) {\n \t\t\tpath = base = prefix;\n-\t\t\tif (exclude_per_dir) {\n+\t\t\tif (dir->exclude_per_dir) {\n \t\t\t\tchar *p, *pp = xmalloc(baselen+1);\n \t\t\t\tmemcpy(pp, prefix, baselen+1);\n \t\t\t\tp = pp;\n \t\t\t\twhile (1) {\n \t\t\t\t\tchar save = *p;\n \t\t\t\t\t*p = 0;\n-\t\t\t\t\tpush_exclude_per_directory(pp, p-pp);\n+\t\t\t\t\tpush_exclude_per_directory(dir, pp, p-pp);\n \t\t\t\t\t*p++ = save;\n \t\t\t\t\tif (!save)\n \t\t\t\t\t\tbreak;\n@@ -543,17 +237,16 @@ static void show_files(void)\n \t\t\t\tfree(pp);\n \t\t\t}\n \t\t}\n-\t\tread_directory(path, base, baselen);\n-\t\tqsort(dir, nr_dir, sizeof(struct nond_on_fs *), cmp_name);\n+\t\tread_directory(dir, path, base, baselen);\n \t\tif (show_others)\n-\t\t\tshow_other_files();\n+\t\t\tshow_other_files(dir);\n \t\tif (show_killed)\n-\t\t\tshow_killed_files();\n+\t\t\tshow_killed_files(dir);\n \t}\n \tif (show_cached | show_stage) {\n \t\tfor (i = 0; i < active_nr; i++) {\n \t\t\tstruct cache_entry *ce = active_cache[i];\n-\t\t\tif (excluded(ce->name) != show_ignored)\n+\t\t\tif (excluded(dir, ce->name) != dir->show_ignored)\n \t\t\t\tcontinue;\n \t\t\tif (show_unmerged && !ce_stage(ce))\n \t\t\t\tcontinue;\n@@ -565,7 +258,7 @@ static void show_files(void)\n \t\t\tstruct cache_entry *ce = active_cache[i];\n \t\t\tstruct stat st;\n \t\t\tint err;\n-\t\t\tif (excluded(ce->name) != show_ignored)\n+\t\t\tif (excluded(dir, ce->name) != dir->show_ignored)\n \t\t\t\tcontinue;\n \t\t\terr = lstat(ce->name, &st);\n \t\t\tif (show_deleted && err)\n@@ -652,7 +345,9 @@ int main(int argc, const char **argv)\n {\n \tint i;\n \tint exc_given = 0;\n+\tstruct dir_struct dir;\n \n+\tmemset(&dir, 0, sizeof(dir));\n \tprefix = setup_git_directory();\n \tif (prefix)\n \t\tprefix_offset = strlen(prefix);\n@@ -697,7 +392,7 @@ int main(int argc, const char **argv)\n \t\t\tcontinue;\n \t\t}\n \t\tif (!strcmp(arg, \"-i\") || !strcmp(arg, \"--ignored\")) {\n-\t\t\tshow_ignored = 1;\n+\t\t\tdir.show_ignored = 1;\n \t\t\tcontinue;\n \t\t}\n \t\tif (!strcmp(arg, \"-s\") || !strcmp(arg, \"--stage\")) {\n@@ -709,11 +404,11 @@ int main(int argc, const char **argv)\n \t\t\tcontinue;\n \t\t}\n \t\tif (!strcmp(arg, \"--directory\")) {\n-\t\t\tshow_other_directories = 1;\n+\t\t\tdir.show_other_directories = 1;\n \t\t\tcontinue;\n \t\t}\n \t\tif (!strcmp(arg, \"--no-empty-directory\")) {\n-\t\t\thide_empty_directories = 1;\n+\t\t\tdir.hide_empty_directories = 1;\n \t\t\tcontinue;\n \t\t}\n \t\tif (!strcmp(arg, \"-u\") || !strcmp(arg, \"--unmerged\")) {\n@@ -726,27 +421,27 @@ int main(int argc, const char **argv)\n \t\t}\n \t\tif (!strcmp(arg, \"-x\") && i+1 < argc) {\n \t\t\texc_given = 1;\n-\t\t\tadd_exclude(argv[++i], \"\", 0, &exclude_list[EXC_CMDL]);\n+\t\t\tadd_exclude(argv[++i], \"\", 0, &dir.exclude_list[EXC_CMDL]);\n \t\t\tcontinue;\n \t\t}\n \t\tif (!strncmp(arg, \"--exclude=\", 10)) {\n \t\t\texc_given = 1;\n-\t\t\tadd_exclude(arg+10, \"\", 0, &exclude_list[EXC_CMDL]);\n+\t\t\tadd_exclude(arg+10, \"\", 0, &dir.exclude_list[EXC_CMDL]);\n \t\t\tcontinue;\n \t\t}\n \t\tif (!strcmp(arg, \"-X\") && i+1 < argc) {\n \t\t\texc_given = 1;\n-\t\t\tadd_excludes_from_file(argv[++i]);\n+\t\t\tadd_excludes_from_file(&dir, argv[++i]);\n \t\t\tcontinue;\n \t\t}\n \t\tif (!strncmp(arg, \"--exclude-from=\", 15)) {\n \t\t\texc_given = 1;\n-\t\t\tadd_excludes_from_file(arg+15);\n+\t\t\tadd_excludes_from_file(&dir, arg+15);\n \t\t\tcontinue;\n \t\t}\n \t\tif (!strncmp(arg, \"--exclude-per-directory=\", 24)) {\n \t\t\texc_given = 1;\n-\t\t\texclude_per_dir = arg + 24;\n+\t\t\tdir.exclude_per_dir = arg + 24;\n \t\t\tcontinue;\n \t\t}\n \t\tif (!strcmp(arg, \"--full-name\")) {\n@@ -788,7 +483,7 @@ int main(int argc, const char **argv)\n \t\tps_matched = xcalloc(1, num);\n \t}\n \n-\tif (show_ignored && !exc_given) {\n+\tif (dir.show_ignored && !exc_given) {\n \t\tfprintf(stderr, \"%s: --ignored needs some exclude pattern\\n\",\n \t\t\targv[0]);\n \t\texit(1);\n@@ -802,7 +497,7 @@ int main(int argc, const char **argv)\n \tread_cache();\n \tif (prefix)\n \t\tprune_cache();\n-\tshow_files();\n+\tshow_files(&dir);\n \n \tif (ps_matched) {\n \t\t/* We need to make sure all pathspec matched otherwise\n"},{"id":"20099","messageId":"Pine.LNX.4.64.0605161931340.10823@g5.osdl.org","threadId":"4174","inReplyTo":"Pine.LNX.4.64.0605161859050.16475@g5.osdl.org","subject":"Re: [PATCH] libify git-ls-files directory traversal","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-05-17T02:36:52Z","receivedAt":"2006-05-17T02:36:52Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 16 May 2006, Linus Torvalds wrote:\n> \n> NOTE! Not all of git-ls-files is libified by this.  The index matching\n> and pathspec prefix calculation is still in ls-files.c, but this is a\n> big part of it.\n\nSide note: the reason I held off on the index matching is that \ngit-ls-files currently uses a pretty disgusting trick to make the index \naccesses faster for the common subdirectory case, namely it basically \nrewrites the index so that it only contains the entries defined by a \ncommon prefix.\n\nNow, that's fine for git-ls-files, but it's not fine for a library \nfunction where the caller may well want to actually use the index that it \nhas read in (eg \"git add\" and \"git commit\" both want to work with the \nfull index).\n\nSo libification of that part will require more than splitting things into \na new file and passing in a structure pointer that contains the data for \nthe function.\n\nThat said, a lot of the current shell scripts seem to use mainly \n\"git-ls-files\" with the \"--others\" flag, and in that case the current \nlibification should be already sufficient.\n\n\t\tLinus\n"},{"id":"20100","messageId":"Pine.LNX.4.64.0605161944480.10823@g5.osdl.org","threadId":"4174","inReplyTo":"Pine.LNX.4.64.0605161859050.16475@g5.osdl.org","subject":"[PATCH] Clean up git-ls-file directory walking library interface","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-05-17T02:46:16Z","receivedAt":"2006-05-17T02:46:16Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\nThis moves the code to add the per-directory ignore files for the base\ndirectory into the library routine.\n\nThat not only allows us to turn the function push_exclude_per_directory() \nstatic again, it also simplifies the library interface a lot (the caller \nno longer needs to worry about any of the per-directory exclude files at \nall).\n\nSigned-off-by: Linus Torvalds <torvalds@osdl.org>\n---\n dir.c      |   28 +++++++++++++++++++++++++++-\n dir.h      |    2 --\n ls-files.c |   22 +---------------------\n 3 files changed, 28 insertions(+), 24 deletions(-)\n\ndiff --git a/dir.c b/dir.c\nindex 3f41a5d..d40b62e 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -78,7 +78,7 @@ void add_excludes_from_file(struct dir_s\n \t\tdie(\"cannot use %s as an exclude file\", fname);\n }\n \n-int push_exclude_per_directory(struct dir_struct *dir, const char *base, int baselen)\n+static int push_exclude_per_directory(struct dir_struct *dir, const char *base, int baselen)\n {\n \tchar exclude_file[PATH_MAX];\n \tstruct exclude_list *el = &dir->exclude_list[EXC_DIRS];\n@@ -289,6 +289,32 @@ static int cmp_name(const void *p1, cons\n \n int read_directory(struct dir_struct *dir, const char *path, const char *base, int baselen)\n {\n+\t/*\n+\t * Make sure to do the per-directory exclude for all the\n+\t * directories leading up to our base.\n+\t */\n+\tif (baselen) {\n+\t\tif (dir->exclude_per_dir) {\n+\t\t\tchar *p, *pp = xmalloc(baselen+1);\n+\t\t\tmemcpy(pp, base, baselen+1);\n+\t\t\tp = pp;\n+\t\t\twhile (1) {\n+\t\t\t\tchar save = *p;\n+\t\t\t\t*p = 0;\n+\t\t\t\tpush_exclude_per_directory(dir, pp, p-pp);\n+\t\t\t\t*p++ = save;\n+\t\t\t\tif (!save)\n+\t\t\t\t\tbreak;\n+\t\t\t\tp = strchr(p, '/');\n+\t\t\t\tif (p)\n+\t\t\t\t\tp++;\n+\t\t\t\telse\n+\t\t\t\t\tp = pp + baselen;\n+\t\t\t}\n+\t\t\tfree(pp);\n+\t\t}\n+\t}\n+\n \tread_directory_recursive(dir, path, base, baselen);\n \tqsort(dir->entries, dir->nr, sizeof(struct dir_entry *), cmp_name);\n \treturn dir->nr;\ndiff --git a/dir.h b/dir.h\nindex e8fc441..4f65f57 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -44,7 +44,5 @@ extern int excluded(struct dir_struct *,\n extern void add_excludes_from_file(struct dir_struct *, const char *fname);\n extern void add_exclude(const char *string, const char *base,\n \t\t\tint baselen, struct exclude_list *which);\n-extern int push_exclude_per_directory(struct dir_struct *,\n-\t\t\t\t      const char *base, int baselen);\n \n #endif\ndiff --git a/ls-files.c b/ls-files.c\nindex 89941a3..dfe1481 100644\n--- a/ls-files.c\n+++ b/ls-files.c\n@@ -215,28 +215,8 @@ static void show_files(struct dir_struct\n \t\tconst char *path = \".\", *base = \"\";\n \t\tint baselen = prefix_len;\n \n-\t\tif (baselen) {\n+\t\tif (baselen)\n \t\t\tpath = base = prefix;\n-\t\t\tif (dir->exclude_per_dir) {\n-\t\t\t\tchar *p, *pp = xmalloc(baselen+1);\n-\t\t\t\tmemcpy(pp, prefix, baselen+1);\n-\t\t\t\tp = pp;\n-\t\t\t\twhile (1) {\n-\t\t\t\t\tchar save = *p;\n-\t\t\t\t\t*p = 0;\n-\t\t\t\t\tpush_exclude_per_directory(dir, pp, p-pp);\n-\t\t\t\t\t*p++ = save;\n-\t\t\t\t\tif (!save)\n-\t\t\t\t\t\tbreak;\n-\t\t\t\t\tp = strchr(p, '/');\n-\t\t\t\t\tif (p)\n-\t\t\t\t\t\tp++;\n-\t\t\t\t\telse\n-\t\t\t\t\t\tp = pp + baselen;\n-\t\t\t\t}\n-\t\t\t\tfree(pp);\n-\t\t\t}\n-\t\t}\n \t\tread_directory(dir, path, base, baselen);\n \t\tif (show_others)\n \t\t\tshow_other_files(dir);\n"},{"id":"20104","messageId":"Pine.LNX.4.64.0605162100570.10823@g5.osdl.org","threadId":"4174","inReplyTo":"Pine.LNX.4.64.0605161944480.10823@g5.osdl.org","subject":"Re: [PATCH] Clean up git-ls-file directory walking library interface","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-05-17T04:06:49Z","receivedAt":"2006-05-17T04:06:49Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 16 May 2006, Linus Torvalds wrote:\n> \n> That not only allows us to turn the function push_exclude_per_directory() \n> static again, it also simplifies the library interface a lot (the caller \n> no longer needs to worry about any of the per-directory exclude files at \n> all).\n\nJust as an example, here's all you need to basically do\n\n\tgit-ls-files --others --directory\n\t\t--exclude-from=\"$GIT_DIR/info/exclude\"\n\t\t--exclude-per-directory=.gitignore\n\nlike \"git status\" does (where the \"--exclude-from\" is conditional on \nwhether the file exists or not).\n\n\t\tLinus\n---\n#include \"cache.h\"\n#include \"dir.h\"\n\nint main(int argc, char **argv)\n{\n\tstruct dir_struct dir;\n\tconst char *prefix = setup_git_directory();\n\tconst char *base, *path;\n\tint baselen, i;\n\n\t/* Read the index */\n\tread_cache();\n\n\t/* Set up the \"struct dir_struct */\n\tmemset(&dir, 0, sizeof(dir));\n\tdir.show_other_directories = 1;\n\n\t/* normal git porcelain exclude patterns */\n\tdir.exclude_per_dir = \".gitignore\";\n\tpath = git_path(\"info/exclude\");\n\tif (!access(path, R_OK))\n\t\tadd_excludes_from_file(&dir, path);\n\n\t/* Set up read_directory() arguments and go go go! */\n\tpath = \".\";\n\tbase = \"\";\n\tbaselen = 0;\n\tif (prefix) {\n\t\tpath = base = prefix;\n\t\tbaselen = strlen(prefix);\n\t}\n\tread_directory(&dir, path, base, baselen);\n\n\t/* And print it all out */\n\tif (dir.nr)\n\t\tprintf(\"#\\n# Untracked files:\\n#\\n\");\n\tfor (i = 0; i < dir.nr; i++)\n\t\tprintf(\"# %s\\n\", dir.entries[i]->name);\n\treturn 0;\n}\n"}]}