{"thread":{"id":"28583","subject":"[PATCH WIP 0/3] git log --exclude","startedAt":"2011-10-05T07:18:05Z","lastAt":"2011-10-07T17:57:45Z","messageCount":12,"participants":["Nguyễn Thái Ngọc Duy","Matthieu Moy","Nguyen Thai Ngoc Duy","Junio C Hamano","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"176906","messageId":"1317799088-26626-1-git-send-email-pclouds@gmail.com","threadId":"28583","inReplyTo":null,"subject":"[PATCH WIP 0/3] git log --exclude","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-10-05T07:18:05Z","receivedAt":"2011-10-05T07:18:05Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This series adds --exclude that uses .gitignore mechanism remove\ncommits whose changes that are _entirely_ excluded. The main patch is\n2/3 where it teaches diff_tree_* about struct exclude_list.\n\nBecause it uses .gitignore mechanism, beware that these patterns do\nnot behave exactly like pathspecs (patterns without slashes match\nevery directory, for example)\n\nI tried these commands\n\n  time git log --stat >/dev/null\n  time git log --stat --exclude=Documentation >/dev/null\n\nThe former took 37 secs, the latter 40 secs. Not bad, but there is\ndefinitely room for improvement. skip_excludes() should be able to\npoint out whether an entire directory is excluded and skip the whole\ndirectory (as opposed to descending in and checking files one by one\nnow, in fear of negative patterns). These kinds of optimizations\nbenefit sparse checkout too.\n\nI think I made a mistake somewhere because the above command seems to\nremove more commits than it should... Regardless,\n\"git log --exclude=po\" on gnome-shell looks sooo clean.\n\nNguyễn Thái Ngọc Duy (3):\n  diff-no-index: rename read_directory to avoid conflict from dir.h\n  tree-diff: teach it to understand exclude patterns\n  log: add --exclude option\n\n diff-no-index.c |    6 ++--\n diff.h          |   11 ++++++++++\n revision.c      |    7 ++++-\n revision.h      |    2 +\n tree-diff.c     |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++---\n 5 files changed, 75 insertions(+), 9 deletions(-)\n\n-- \n1.7.3.1.256.g2539c.dirty\n"},{"id":"176907","messageId":"1317799088-26626-2-git-send-email-pclouds@gmail.com","threadId":"28583","inReplyTo":"1317799088-26626-1-git-send-email-pclouds@gmail.com","subject":"[PATCH WIP 1/3] diff-no-index: rename read_directory to avoid conflict from dir.h","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-10-05T07:18:06Z","receivedAt":"2011-10-05T07:18:06Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n diff-no-index.c |    6 +++---\n 1 files changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/diff-no-index.c b/diff-no-index.c\nindex 3a36144..927abb7 100644\n--- a/diff-no-index.c\n+++ b/diff-no-index.c\n@@ -16,7 +16,7 @@\n #include \"builtin.h\"\n #include \"string-list.h\"\n \n-static int read_directory(const char *path, struct string_list *list)\n+static int read_entire_directory(const char *path, struct string_list *list)\n {\n \tDIR *dir;\n \tstruct dirent *e;\n@@ -68,9 +68,9 @@ static int queue_diff(struct diff_options *o,\n \t\tstruct string_list p2 = STRING_LIST_INIT_DUP;\n \t\tint len1 = 0, len2 = 0, i1, i2, ret = 0;\n \n-\t\tif (name1 && read_directory(name1, &p1))\n+\t\tif (name1 && read_entire_directory(name1, &p1))\n \t\t\treturn -1;\n-\t\tif (name2 && read_directory(name2, &p2)) {\n+\t\tif (name2 && read_entire_directory(name2, &p2)) {\n \t\t\tstring_list_clear(&p1, 0);\n \t\t\treturn -1;\n \t\t}\n-- \n1.7.3.1.256.g2539c.dirty\n"},{"id":"176908","messageId":"1317799088-26626-3-git-send-email-pclouds@gmail.com","threadId":"28583","inReplyTo":"1317799088-26626-1-git-send-email-pclouds@gmail.com","subject":"[PATCH WIP 2/3] tree-diff: teach it to understand exclude patterns","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-10-05T07:18:07Z","receivedAt":"2011-10-05T07:18:07Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"We introduce a new set of API, diff_tree_*_with_exclude, that also\nexclude entries based on .gitignore patterns.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n diff.h      |   11 +++++++++++\n tree-diff.c |   58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++----\n 2 files changed, 65 insertions(+), 4 deletions(-)\n\ndiff --git a/diff.h b/diff.h\nindex 8c66b59..51c8a5f 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -12,6 +12,7 @@ struct diff_queue_struct;\n struct strbuf;\n struct diff_filespec;\n struct userdiff_driver;\n+struct exclude_list;\n \n typedef void (*change_fn_t)(struct diff_options *options,\n \t\t unsigned old_mode, unsigned new_mode,\n@@ -170,8 +171,18 @@ extern void diff_tree_setup_paths(const char **paths, struct diff_options *);\n extern void diff_tree_release_paths(struct diff_options *);\n extern int diff_tree(struct tree_desc *t1, struct tree_desc *t2,\n \t\t     const char *base, struct diff_options *opt);\n+extern int diff_tree_with_exclude(struct tree_desc *t1, struct tree_desc *t2,\n+\t\t\t\t  const char *base, struct diff_options *opt,\n+\t\t\t\t  struct exclude_list *el,\n+\t\t\t\t  int def_excl1, int def_excl2);\n extern int diff_tree_sha1(const unsigned char *old, const unsigned char *new,\n \t\t\t  const char *base, struct diff_options *opt);\n+extern int diff_tree_sha1_with_exclude(const unsigned char *old,\n+\t\t\t\t       const unsigned char *new,\n+\t\t\t\t       const char *base,\n+\t\t\t\t       struct diff_options *opt,\n+\t\t\t\t       struct exclude_list *el,\n+\t\t\t\t       int def_excl1, int def_excl2);\n extern int diff_root_tree_sha1(const unsigned char *new, const char *base,\n                                struct diff_options *opt);\n \ndiff --git a/tree-diff.c b/tree-diff.c\nindex b3cc2e4..9938ccf 100644\n--- a/tree-diff.c\n+++ b/tree-diff.c\n@@ -5,12 +5,14 @@\n #include \"diff.h\"\n #include \"diffcore.h\"\n #include \"tree.h\"\n+#include \"dir.h\"\n \n static void show_entry(struct diff_options *opt, const char *prefix,\n \t\t       struct tree_desc *desc, struct strbuf *base);\n \n static int compare_tree_entry(struct tree_desc *t1, struct tree_desc *t2,\n-\t\t\t      struct strbuf *base, struct diff_options *opt)\n+\t\t\t      struct strbuf *base, struct diff_options *opt,\n+\t\t\t      struct exclude_list *el, int def_excl1, int def_excl2)\n {\n \tunsigned mode1, mode2;\n \tconst char *path1, *path2;\n@@ -52,7 +54,8 @@ static int compare_tree_entry(struct tree_desc *t1, struct tree_desc *t2,\n \t\t\t\t    sha1, sha2, base->buf, 0, 0);\n \t\t}\n \t\tstrbuf_addch(base, '/');\n-\t\tdiff_tree_sha1(sha1, sha2, base->buf, opt);\n+\t\tdiff_tree_sha1_with_exclude(sha1, sha2, base->buf, opt,\n+\t\t\t\t\t    el, def_excl1, def_excl2);\n \t} else {\n \t\topt->change(opt, mode1, mode2, sha1, sha2, base->buf, 0, 0);\n \t}\n@@ -113,6 +116,31 @@ static void show_entry(struct diff_options *opt, const char *prefix,\n \tstrbuf_setlen(base, old_baselen);\n }\n \n+static int skip_excludes(struct tree_desc *t, struct strbuf *base,\n+\t\t\t struct exclude_list *el, int defval)\n+{\n+\tfor (; t->size; update_tree_entry(t)) {\n+\t\t/*\n+\t\t * excluded_from_list only cares whether dtype is\n+\t\t * DT_DIR or something else (except DT_UNKNOWN). Any\n+\t\t * other value would do\n+\t\t */\n+\t\tint dtype = S_ISDIR(t->entry.mode) ? DT_DIR : DT_REG;\n+\t\tint ret = excluded_from_list(base->buf, base->len, t->entry.path,\n+\t\t\t\t\t     &dtype, el);\n+\n+\t\t/* If undecided, use matching result of parent dir in defval */\n+\t\tif (ret < 0)\n+\t\t\tret = defval;\n+\n+\t\tif (ret == 1 && dtype == DT_REG)\n+\t\t\t;\n+\t\telse\n+\t\t\treturn ret;\n+\t}\n+\treturn defval;\n+}\n+\n static void skip_uninteresting(struct tree_desc *t, struct strbuf *base,\n \t\t\t       struct diff_options *opt, int *match)\n {\n@@ -130,9 +158,17 @@ static void skip_uninteresting(struct tree_desc *t, struct strbuf *base,\n int diff_tree(struct tree_desc *t1, struct tree_desc *t2,\n \t      const char *base_str, struct diff_options *opt)\n {\n+\treturn diff_tree_with_exclude(t1, t2, base_str, opt, NULL, 0, 0);\n+}\n+\n+int diff_tree_with_exclude(struct tree_desc *t1, struct tree_desc *t2,\n+\t\t\t   const char *base_str, struct diff_options *opt,\n+\t\t\t   struct exclude_list *el, int def_excl1, int def_excl2)\n+{\n \tstruct strbuf base;\n \tint baselen = strlen(base_str);\n \tint t1_match = 0, t2_match = 0;\n+\tint excl1 = 0, excl2 = 0;\n \n \t/* Enable recursion indefinitely */\n \topt->pathspec.recursive = DIFF_OPT_TST(opt, RECURSIVE);\n@@ -148,6 +184,10 @@ int diff_tree(struct tree_desc *t1, struct tree_desc *t2,\n \t\t\tskip_uninteresting(t1, &base, opt, &t1_match);\n \t\t\tskip_uninteresting(t2, &base, opt, &t2_match);\n \t\t}\n+\t\tif (el && el->nr) {\n+\t\t\texcl1 = skip_excludes(t1, &base, el, def_excl1);\n+\t\t\texcl2 = skip_excludes(t2, &base, el, def_excl2);\n+\t\t}\n \t\tif (!t1->size) {\n \t\t\tif (!t2->size)\n \t\t\t\tbreak;\n@@ -160,7 +200,7 @@ int diff_tree(struct tree_desc *t1, struct tree_desc *t2,\n \t\t\tupdate_tree_entry(t1);\n \t\t\tcontinue;\n \t\t}\n-\t\tswitch (compare_tree_entry(t1, t2, &base, opt)) {\n+\t\tswitch (compare_tree_entry(t1, t2, &base, opt, el, excl1, excl2)) {\n \t\tcase -1:\n \t\t\tupdate_tree_entry(t1);\n \t\t\tcontinue;\n@@ -267,6 +307,16 @@ static void try_to_follow_renames(struct tree_desc *t1, struct tree_desc *t2, co\n \n int diff_tree_sha1(const unsigned char *old, const unsigned char *new, const char *base, struct diff_options *opt)\n {\n+\treturn diff_tree_sha1_with_exclude(old, new, base, opt, NULL, 0, 0);\n+}\n+\n+int diff_tree_sha1_with_exclude(const unsigned char *old,\n+\t\t\t\tconst unsigned char *new,\n+\t\t\t\tconst char *base,\n+\t\t\t\tstruct diff_options *opt,\n+\t\t\t\tstruct exclude_list *el,\n+\t\t\t\tint def_excl1, int def_excl2)\n+{\n \tvoid *tree1, *tree2;\n \tstruct tree_desc t1, t2;\n \tunsigned long size1, size2;\n@@ -280,7 +330,7 @@ int diff_tree_sha1(const unsigned char *old, const unsigned char *new, const cha\n \t\tdie(\"unable to read destination tree (%s)\", sha1_to_hex(new));\n \tinit_tree_desc(&t1, tree1, size1);\n \tinit_tree_desc(&t2, tree2, size2);\n-\tretval = diff_tree(&t1, &t2, base, opt);\n+\tretval = diff_tree_with_exclude(&t1, &t2, base, opt, el, def_excl1, def_excl2);\n \tif (!*base && DIFF_OPT_TST(opt, FOLLOW_RENAMES) && diff_might_be_rename()) {\n \t\tinit_tree_desc(&t1, tree1, size1);\n \t\tinit_tree_desc(&t2, tree2, size2);\n-- \n1.7.3.1.256.g2539c.dirty\n"},{"id":"176909","messageId":"1317799088-26626-4-git-send-email-pclouds@gmail.com","threadId":"28583","inReplyTo":"1317799088-26626-1-git-send-email-pclouds@gmail.com","subject":"[PATCH WIP 3/3] log: add --exclude option","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-10-05T07:18:08Z","receivedAt":"2011-10-05T07:18:08Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This only helps filtering out commits whose changes are _entirely_ filtered out.\nThis does not affect patch generate (--patch, --stat and so on)\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n revision.c |    7 +++++--\n revision.h |    2 ++\n 2 files changed, 7 insertions(+), 2 deletions(-)\n\ndiff --git a/revision.c b/revision.c\nindex c46cfaa..8472553 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -334,8 +334,8 @@ static int rev_compare_tree(struct rev_info *revs, struct commit *parent, struct\n \n \ttree_difference = REV_TREE_SAME;\n \tDIFF_OPT_CLR(&revs->pruning, HAS_CHANGES);\n-\tif (diff_tree_sha1(t1->object.sha1, t2->object.sha1, \"\",\n-\t\t\t   &revs->pruning) < 0)\n+\tif (diff_tree_sha1_with_exclude(t1->object.sha1, t2->object.sha1, \"\",\n+\t\t\t\t\t&revs->pruning, &revs->el, 0, 0) < 0)\n \t\treturn REV_TREE_DIFFERENT;\n \treturn tree_difference;\n }\n@@ -1454,6 +1454,9 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg\n \t\treturn argcount;\n \t} else if (!strcmp(arg, \"--log-size\")) {\n \t\trevs->show_log_size = 1;\n+\t} else if (!prefixcmp(arg, \"--exclude=\")) {\n+\t\tadd_exclude(arg + 10, \"\", 0, &revs->el);\n+\t\trevs->prune = 1;\n \t}\n \t/*\n \t * Grepping the commit log\ndiff --git a/revision.h b/revision.h\nindex 3d64ada..3709959 100644\n--- a/revision.h\n+++ b/revision.h\n@@ -4,6 +4,7 @@\n #include \"parse-options.h\"\n #include \"grep.h\"\n #include \"notes.h\"\n+#include \"dir.h\"\n \n #define SEEN\t\t(1u<<0)\n #define UNINTERESTING   (1u<<1)\n@@ -133,6 +134,7 @@ struct rev_info {\n \t/* diff info for patches and for paths limiting */\n \tstruct diff_options diffopt;\n \tstruct diff_options pruning;\n+\tstruct exclude_list el;\n \n \tstruct reflog_walk_info *reflog_info;\n \tstruct decoration children;\n-- \n1.7.3.1.256.g2539c.dirty\n"},{"id":"176911","messageId":"vpqd3ebn9nc.fsf@bauges.imag.fr","threadId":"28583","inReplyTo":"1317799088-26626-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH WIP 0/3] git log --exclude","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2011-10-05T08:08:07Z","receivedAt":"2011-10-05T08:08:07Z","isPatch":true,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:\n\n> This series adds --exclude that uses .gitignore mechanism remove\n> commits whose changes that are _entirely_ excluded.\n\nI'd see this --exclude as the opposite of specifying files, i.e. in a\nrepository containing directories A, B and C,\n\ngit log --exclude=B\n\nwould be the same as\n\ngit log A C\n\nIf I understand correctly, it's not the case with your implementation\nbecause\n\n> Because it uses .gitignore mechanism, beware that these patterns do\n> not behave exactly like pathspecs\n\nand because \"git log --stat A C\" (or --patch) will show the diff only\nfor A and C for commits touching all directories.\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"176913","messageId":"CACsJy8DDcMPVFm2t3xTo1U3SthZEktOCQBBwwJwvSiQer9KuUg@mail.gmail.com","threadId":"28583","inReplyTo":"vpqd3ebn9nc.fsf@bauges.imag.fr","subject":"Re: [PATCH WIP 0/3] git log --exclude","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-10-05T08:28:40Z","receivedAt":"2011-10-05T08:28:40Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2011/10/5 Matthieu Moy <Matthieu.Moy@grenoble-inp.fr>:\n> Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:\n>\n>> This series adds --exclude that uses .gitignore mechanism remove\n>> commits whose changes that are _entirely_ excluded.\n>\n> I'd see this --exclude as the opposite of specifying files, i.e. in a\n> repository containing directories A, B and C,\n>\n> git log --exclude=B\n>\n> would be the same as\n>\n> git log A C\n\nI think I emphasized it too much. \"git log --exclude=B/ A B C\" should\nbe equivalent to \"git log A C\". If changes touch A or C, then no\nmatter they touch B, the commit will always be in. If changes only\ntouch B, neither A nor B, then the commit is removed, exactly the same\ncase with \"git log A C\".\n\n>> Because it uses .gitignore mechanism, beware that these patterns do\n>> not behave exactly like pathspecs\n\nif you specify --exclude=B, then A/.../B, C/.../B are all removed. A\nsubtle difference between pathspec and .gitignore.\n\n> and because \"git log --stat A C\" (or --patch) will show the diff only\n> for A and C for commits touching all directories.\n\nI'll take care of --patch and friends later. They both (--patch and\ncommit pruning) use the same diff mechanism, if we get it right for\nfor commit pruning, --patch will come nicely.\n-- \nDuy\n"},{"id":"176940","messageId":"7vhb3n8ie9.fsf@alter.siamese.dyndns.org","threadId":"28583","inReplyTo":"1317799088-26626-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH WIP 0/3] git log --exclude","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-10-05T17:20:30Z","receivedAt":"2011-10-05T17:20:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:\n\n> This series adds --exclude that uses .gitignore mechanism remove\n> commits whose changes that are _entirely_ excluded.\n\nI really do not want to see the exclude patterns nor mechanism used for\nthis. This will unnecessarily impact the long term design of narrow clones\nI had in mind.\n\nThe way I envisioned the narrow cloning would work like this [*1*]:\n\n * The narrowed set of paths is an attribute of the local repository. It\n   is not tied to the history nor the current working tree state, so the\n   information does not live in the index or in the history. A new file\n   $GIT_DIR/narrowed-paths specifies a list of pathspecs. We call a\n   repository with such a file \"a narrowed repository\".\n\n * The objects that live in a narrowed repository are subset of the\n   objects in an unnarrowed repository that records the same\n   history. Objects are not modified in any way when transferring into\n   a narrowed repository. E.g. if you clone git.git but limit the tree to\n   Documentation/ and builtin/, you will get _all_ commit objects, even\n   the ones that do _not_ touch these two directories, and the top level\n   tree objects. These top level tree objects _do_ record the object names\n   for paths outside the narrowed area. To facilitate local history\n   traversal, we may add either grafts or replace entries to \"gather\" away\n   commits that do not touch the narrowed area, but this is not essential.\n\n * \"struct pathspec\" gains a handful of new fields to carry the set of\n   narrowing pathspecs taken from $GIT_DIR/narrowed-paths file down the\n   callchain. When we decide to see if a given path matches the pathspec\n   and/or is worth descending into, we use these pathspecs to reject\n   anything that is outside of the narrowed area before applying user\n   supplied pathspec. We use narrowing pathspecs even for the whole tree\n   operation.\n\n * \"fsck\", \"prune\" and friends that do not use user-supplied pathspec will\n   learn the narrowed-paths so that the tree entries that describe objects\n   outside the narrowed area (e.g. \"Makefile\" in the top-level tree in a\n   repository narrowed to \"Documentation/\" and \"builtin/\") are ignored.\n\n   I looked at fsck and unpack-objects (surprise, but it does seem to need\n   some work) a few months ago for this exact purpose, and it appeared\n   doable.\n\nWe may want to be able to say something like \"I want Documentation/ but\nnot Documentaion/howto/\", and for that to work, narrowed-paths needs to be\nable to express negative matches. And in _that_ context, we will need\nnegative pathspec matches (and I think during the last round of pathspec\nupdates we identified where the support for \":(no):pathspec\" needs to be\nadded in the codepath). \n\nYou could say \"Use pathspec for inclusion and ignore patterns for\nexclusion\", but I do not think that inconsistency has merits.\n\nFor the purpose of \"log --exclude\" [*2*], I do not mind too much if the UI\nexpressed negative pathspecs using such a new command line option, but I\nthink it would be more natural to say (notations aside):\n\n\t$ git log -- ':(no):po' .\n\nand define the behaviour of user-supplied pathspec limiter this way:\n\n * Paths are matched from left to right;\n\n * First match determines the fate of the path;\n\n * A match with negative pathspec means \"the path in question does _not_\n   match\".\n\n\n[Footnote]\n\n*1* Unlike the \"This is _one_ possible way\" I usually give in discussions,\nthis is \"The narrow clone _must_ work like this, if we were to add it\".\n\n*2* I would suggest not using --exclude to avoid giving users a wrong\nimpression that this might have something to do with exclude patterns. How\nabout calling it \"--filter-out\" instead?\n"},{"id":"177032","messageId":"20111006143441.GA21558@sigill.intra.peff.net","threadId":"28583","inReplyTo":"7vhb3n8ie9.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH WIP 0/3] git log --exclude","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-10-06T14:34:41Z","receivedAt":"2011-10-06T14:34:41Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Oct 05, 2011 at 10:20:30AM -0700, Junio C Hamano wrote:\n\n> The way I envisioned the narrow cloning would work like this [*1*]:\n> \n>  * The narrowed set of paths is an attribute of the local repository. It\n>    is not tied to the history nor the current working tree state, so the\n>    information does not live in the index or in the history. A new file\n>    $GIT_DIR/narrowed-paths specifies a list of pathspecs. We call a\n>    repository with such a file \"a narrowed repository\".\n> \n>  * The objects that live in a narrowed repository are subset of the\n>    objects in an unnarrowed repository that records the same\n>    history. Objects are not modified in any way when transferring into\n>    a narrowed repository. E.g. if you clone git.git but limit the tree to\n>    Documentation/ and builtin/, you will get _all_ commit objects, even\n>    the ones that do _not_ touch these two directories, and the top level\n>    tree objects. These top level tree objects _do_ record the object names\n>    for paths outside the narrowed area. To facilitate local history\n>    traversal, we may add either grafts or replace entries to \"gather\" away\n>    commits that do not touch the narrowed area, but this is not essential.\n\nI'm really just a bystander on this topic, and haven't given it too much\nthought. But one stumbling block I see for narrow clone is how narrow\nrepositories will interact with object transfer from other repositories.\n\nFor example, if I have a narrow git.git that omits Documentation, and I\ndo a \"git fetch\" from a non-narrow repository, then how do we tell the\nnon-narrow remote that we don't have blobs in Documentation, and that\nthey should not be used as delta bases for any objects that are sent?\n\nThe current protocol relies on certain repository properties on the\nremote end that narrow clone will violate. I don't see a way around that\nwithout a protocol extension to communicate the narrowness. What will\nthat extension look like?\n\n-Peff\n"},{"id":"177064","messageId":"7vpqia11ds.fsf@alter.siamese.dyndns.org","threadId":"28583","inReplyTo":"20111006143441.GA21558@sigill.intra.peff.net","subject":"Re: [PATCH WIP 0/3] git log --exclude","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-10-06T17:22:07Z","receivedAt":"2011-10-06T17:22:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> The current protocol relies on certain repository properties on the\n> remote end that narrow clone will violate. I don't see a way around that\n> without a protocol extension to communicate the narrowness. What will\n> that extension look like?\n\nIt would have to involve exchanging \"I have/am interested in the paths\nthat match these pathspecs\". I do not mind if our initial implementation\ndid not support anything other than fetching into a narrow from full and\npushing from a narrow to full. The second iteration could add fetching and\npushing between two narrows with the same set of narrowing pathspecs.\n\nAs to widening the area after a clone is made, I do not mind and if our\ninitial impementation only supported widening the area in a stupid way\n(e.g. semi-equivalent of initial clone with the widened set of narrowing\npathspecs).\n"},{"id":"177115","messageId":"CACsJy8C9O-dGo=Y6HfQ5iYtsD8SMuiWB8y5ab=C+vT4Z01F+ag@mail.gmail.com","threadId":"28583","inReplyTo":"20111006143441.GA21558@sigill.intra.peff.net","subject":"Re: [PATCH WIP 0/3] git log --exclude","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-10-06T21:47:45Z","receivedAt":"2011-10-06T21:47:45Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2011/10/7 Jeff King <peff@peff.net>:\n> I'm really just a bystander on this topic, and haven't given it too much\n> thought. But one stumbling block I see for narrow clone is how narrow\n> repositories will interact with object transfer from other repositories.\n>\n> For example, if I have a narrow git.git that omits Documentation, and I\n> do a \"git fetch\" from a non-narrow repository, then how do we tell the\n> non-narrow remote that we don't have blobs in Documentation, and that\n> they should not be used as delta bases for any objects that are sent?\n\nPretty much how shallow cloned repos interacts. In shallow clone we\nsend depth info. In narrow clone, we send width info, in terms of\n(restricted) pathspec. With pathspec (possibly plus exclude rules but\nthis possibility is getting smaller), I can express \"fetch this\ndirectory only\", or \"fetch this directory, but not this subdirectory\n(because I already have it)\".\n\nWe can prohibit certain use cases at client side just like shallow clone.\n\n> The current protocol relies on certain repository properties on the\n> remote end that narrow clone will violate. I don't see a way around that\n> without a protocol extension to communicate the narrowness. What will\n> that extension look like?\n\nSomething like this\n\nhttp://thread.gmane.org/gmane.comp.version-control.git/155427/focus=155613\n-- \nDuy\n"},{"id":"177139","messageId":"CACsJy8DJs_cmCZegyPk=tB-bxWp4izrsTn8T=xeV1sU4fS5oTQ@mail.gmail.com","threadId":"28583","inReplyTo":"7vhb3n8ie9.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH WIP 0/3] git log --exclude","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-10-07T07:16:04Z","receivedAt":"2011-10-07T07:16:04Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2011/10/6 Junio C Hamano <gitster@pobox.com>:\n> For the purpose of \"log --exclude\" [*2*], I do not mind too much if the UI\n> expressed negative pathspecs using such a new command line option, but I\n> think it would be more natural to say (notations aside):\n>\n>        $ git log -- ':(no):po' .\n>\n> and define the behaviour of user-supplied pathspec limiter this way:\n>\n>  * Paths are matched from left to right;\n>\n>  * First match determines the fate of the path;\n>\n>  * A match with negative pathspec means \"the path in question does _not_\n>   match\".\n\nThings have changed a bit since last time I touched negative pathspec.\nBecause of depth limit support [1], pathspecs have to be sorted, which\nmeans we cannot keep pathspec in original order to match from left to\nright.\n\nThere may be a solution to mix depth limit and negative pathspec. I\nhaven't thought carefully about that because I lean towards a simpler\nbehaviour that only allows one negation level: a file is in if it\nmatches any positive pathspecs and none of negative ones.\n\nThis is enough if narrow clones consist of positive pathspec only. I\nreally doubt if users would want to make a narrow clone that \"contains\nA but not A/B, storage-wise\".\n\n[1] 86e4ca6 (tree_entry_interesting(): fix depth limit with\noverlapping pathspecs)\n-- \nDuy\n"},{"id":"177167","messageId":"7vaa9cwup2.fsf@alter.siamese.dyndns.org","threadId":"28583","inReplyTo":"CACsJy8DJs_cmCZegyPk=tB-bxWp4izrsTn8T=xeV1sU4fS5oTQ@mail.gmail.com","subject":"Re: [PATCH WIP 0/3] git log --exclude","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-10-07T17:57:45Z","receivedAt":"2011-10-07T17:57:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyen Thai Ngoc Duy <pclouds@gmail.com> writes:\n\n> There may be a solution to mix depth limit and negative pathspec. I\n> haven't thought carefully about that because I lean towards a simpler\n> behaviour that only allows one negation level: a file is in if it\n> matches any positive pathspecs and none of negative ones.\n\nI tend to think it probably is acceptable solution to define \"struct\npathspec\" to have one positive and one negative set, traversing and\ndeclaring a match on what matches something from the positive set only if\nit does not match anything in the negative set.\n\nThat is similar to how we define revision ranges in the sense that the\nrange notation to have two ranges (A..B C..D) does not mean union of two\nsets (A..B + C..D), but is difference between two unions ((B D)-(A C)).\n\n> This is enough if narrow clones consist of positive pathspec only. I\n> really doubt if users would want to make a narrow clone that \"contains\n> A but not A/B, storage-wise\".\n\nAnd if you define \"struct pathspec\" to have one positive and one negative\nset, you do not have to limit narrow clone only to positive. The repository\nspecific narrow pathspec will have one such \"struct pathspec\", but the\nuser may give us from the command line another \"struct pathspec\". At\nruntime, we would merge them to form into one negative and one positive\nset, and apply the same rule: nothing that matches any negative will\nappear in the traversal or in the output.\n"}]}