{"thread":{"id":"20522","subject":"[RFC PATCH v2 0/4] Sparse checkout","startedAt":"2009-08-10T15:19:18Z","lastAt":"2009-08-11T08:12:05Z","messageCount":19,"participants":["Nguyễn Thái Ngọc Duy","Johannes Schindelin","Nguyen Thai Ngoc Duy","Junio C Hamano"],"isPatch":true,"patchVersion":2,"patchTotal":4},"messages":[{"id":"120164","messageId":"1249917562-5931-1-git-send-email-pclouds@gmail.com","threadId":"20522","inReplyTo":null,"subject":"[RFC PATCH v2 0/4] Sparse checkout","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2009-08-10T15:19:18Z","receivedAt":"2009-08-10T15:19:18Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Another RFC, which is more usable than the last series I sent out.\nCompared to the last one, a few more patches have gone the way of\nDodo; 'git shape-workdir' has become a hook and should be easier\nto write too. All you need is a few\n\"git update-index --[no-]assume-unchanged\" in .git/hooks/sparse.\n\nNguyễn Thái Ngọc Duy (4):\n  Prevent diff machinery from examining assume-unchanged entries on\n    worktree\n  gitignore: read from index if .gitignore is assume-unchanged\n  unpack_trees(): add support for sparse checkout\n  read-tree: add --no-sparse to turn off sparse hook\n\n Documentation/technical/api-directory-listing.txt |    3 +\n builtin-clean.c                                   |    5 +-\n builtin-ls-files.c                                |    4 +-\n builtin-read-tree.c                               |    4 +-\n cache.h                                           |    3 +\n diff-lib.c                                        |    5 +-\n dir.c                                             |   70 +++++++---\n t/t1009-read-tree-sparse.sh                       |   48 +++++++\n t/t3001-ls-files-others-exclude.sh                |   20 +++\n t/t4039-diff-assume-unchanged.sh                  |   31 ++++\n t/t7300-clean.sh                                  |   19 +++\n unpack-trees.c                                    |  154 ++++++++++++++++++++-\n unpack-trees.h                                    |    3 +\n 13 files changed, 335 insertions(+), 34 deletions(-)\n create mode 100755 t/t1009-read-tree-sparse.sh\n create mode 100755 t/t4039-diff-assume-unchanged.sh\n"},{"id":"120165","messageId":"1249917562-5931-2-git-send-email-pclouds@gmail.com","threadId":"20522","inReplyTo":"1249917562-5931-1-git-send-email-pclouds@gmail.com","subject":"[RFC PATCH v2 1/4] Prevent diff machinery from examining assume-unchanged entries on worktree","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2009-08-10T15:19:19Z","receivedAt":"2009-08-10T15:19:19Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n diff-lib.c                       |    5 +++--\n t/t4039-diff-assume-unchanged.sh |   31 +++++++++++++++++++++++++++++++\n 2 files changed, 34 insertions(+), 2 deletions(-)\n create mode 100755 t/t4039-diff-assume-unchanged.sh\n\ndiff --git a/diff-lib.c b/diff-lib.c\nindex b7813af..f5787f6 100644\n--- a/diff-lib.c\n+++ b/diff-lib.c\n@@ -159,7 +159,7 @@ int run_diff_files(struct rev_info *revs, unsigned int option)\n \t\t\t\tcontinue;\n \t\t}\n \n-\t\tif (ce_uptodate(ce))\n+\t\tif (ce_uptodate(ce) || (ce->ce_flags & CE_VALID))\n \t\t\tcontinue;\n \n \t\tchanged = check_removed(ce, &st);\n@@ -337,6 +337,8 @@ static void do_oneway_diff(struct unpack_trees_options *o,\n \tstruct rev_info *revs = o->unpack_data;\n \tint match_missing, cached;\n \n+\t/* if the entry is not checked out, don't examine work tree */\n+\tcached = o->index_only || (idx && (idx->ce_flags & CE_VALID));\n \t/*\n \t * Backward compatibility wart - \"diff-index -m\" does\n \t * not mean \"do not ignore merges\", but \"match_missing\".\n@@ -344,7 +346,6 @@ static void do_oneway_diff(struct unpack_trees_options *o,\n \t * But with the revision flag parsing, that's found in\n \t * \"!revs->ignore_merges\".\n \t */\n-\tcached = o->index_only;\n \tmatch_missing = !revs->ignore_merges;\n \n \tif (cached && idx && ce_stage(idx)) {\ndiff --git a/t/t4039-diff-assume-unchanged.sh b/t/t4039-diff-assume-unchanged.sh\nnew file mode 100755\nindex 0000000..d0e46a7\n--- /dev/null\n+++ b/t/t4039-diff-assume-unchanged.sh\n@@ -0,0 +1,31 @@\n+#!/bin/sh\n+\n+test_description='diff with assume-unchanged entries'\n+\n+. ./test-lib.sh\n+\n+# external diff has been tested in t4020-diff-external.sh\n+\n+test_expect_success 'setup' '\n+\techo zero > zero &&\n+\tgit add zero &&\n+\tgit commit -m zero &&\n+\techo one > one &&\n+\techo two > two &&\n+\tgit add one two &&\n+\tgit commit -m onetwo &&\n+\tgit update-index --assume-unchanged one &&\n+\techo borked >> one &&\n+\ttest \"$(git ls-files -v one)\" = \"h one\"\n+'\n+\n+test_expect_success 'diff-index does not examine assume-unchanged entries' '\n+\tgit diff-index HEAD^ -- one | grep -q 5626abf0f72e58d7a153368ba57db4c673c0e171 \n+'\n+\n+# TODO ced_uptodate()\n+test_expect_success 'diff-files does not examine assume-unchanged entries' '\n+\t/usr/bin/git diff-files -- one\n+'\n+\n+test_done\n-- \n1.6.3.GIT\n"},{"id":"120168","messageId":"1249917562-5931-3-git-send-email-pclouds@gmail.com","threadId":"20522","inReplyTo":"1249917562-5931-2-git-send-email-pclouds@gmail.com","subject":"[RFC PATCH v2 2/4] gitignore: read from index if .gitignore is assume-unchanged","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2009-08-10T15:19:20Z","receivedAt":"2009-08-10T15:19:20Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"In sparse checkout mode (aka CE_VALID or assume-unchanged) some files\nmay be missing from working directory. If some of those files are\n.gitignore, it will affect how git excludes files.\n\nBecause those files are by definition \"assume unchanged\" we can\ninstead read them from index. This adds index as a prerequisite for\ndirectory listing. At the moment directory listing is used by \"git\nclean\", \"git add\", \"git ls-files\" and \"git status\"/\"git commit\" and\nunpack_trees()-related commands.  These commands have been\nchecked/modified to populate index before doing directory listing.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n Documentation/technical/api-directory-listing.txt |    3 +\n builtin-clean.c                                   |    5 +-\n builtin-ls-files.c                                |    4 +-\n dir.c                                             |   70 ++++++++++++++-------\n t/t3001-ls-files-others-exclude.sh                |   20 ++++++\n t/t7300-clean.sh                                  |   19 ++++++\n 6 files changed, 96 insertions(+), 25 deletions(-)\n\ndiff --git a/Documentation/technical/api-directory-listing.txt b/Documentation/technical/api-directory-listing.txt\nindex 5bbd18f..7d0e282 100644\n--- a/Documentation/technical/api-directory-listing.txt\n+++ b/Documentation/technical/api-directory-listing.txt\n@@ -58,6 +58,9 @@ The result of the enumeration is left in these fields::\n Calling sequence\n ----------------\n \n+* Ensure the_index is populated as it may have CE_VALID entries that\n+  affect directory listing.\n+\n * Prepare `struct dir_struct dir` and clear it with `memset(&dir, 0,\n   sizeof(dir))`.\n \ndiff --git a/builtin-clean.c b/builtin-clean.c\nindex 2d8c735..d917472 100644\n--- a/builtin-clean.c\n+++ b/builtin-clean.c\n@@ -71,8 +71,11 @@ int cmd_clean(int argc, const char **argv, const char *prefix)\n \n \tdir.flags |= DIR_SHOW_OTHER_DIRECTORIES;\n \n-\tif (!ignored)\n+\tif (!ignored) {\n+\t\tif (read_cache() < 0)\n+\t\t\tdie(\"index file corrupt\");\n \t\tsetup_standard_excludes(&dir);\n+\t}\n \n \tpathspec = get_pathspec(prefix, argv);\n \tread_cache();\ndiff --git a/builtin-ls-files.c b/builtin-ls-files.c\nindex f473220..d1a23c4 100644\n--- a/builtin-ls-files.c\n+++ b/builtin-ls-files.c\n@@ -481,6 +481,9 @@ int cmd_ls_files(int argc, const char **argv, const char *prefix)\n \t\tprefix_offset = strlen(prefix);\n \tgit_config(git_default_config, NULL);\n \n+\tif (read_cache() < 0)\n+\t\tdie(\"index file corrupt\");\n+\n \targc = parse_options(argc, argv, prefix, builtin_ls_files_options,\n \t\t\tls_files_usage, 0);\n \tif (show_tag || show_valid_bit) {\n@@ -508,7 +511,6 @@ int cmd_ls_files(int argc, const char **argv, const char *prefix)\n \tpathspec = get_pathspec(prefix, argv);\n \n \t/* be nice with submodule paths ending in a slash */\n-\tread_cache();\n \tif (pathspec)\n \t\tstrip_trailing_slash_from_submodules();\n \ndiff --git a/dir.c b/dir.c\nindex e05b850..e55344f 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -200,11 +200,36 @@ void add_exclude(const char *string, const char *base,\n \twhich->excludes[which->nr++] = x;\n }\n \n+static void *read_index_data(const char *path, size_t *size)\n+{\n+\tint pos, len;\n+\tunsigned long sz;\n+\tenum object_type type;\n+\tvoid *data;\n+\tstruct index_state *istate = &the_index;\n+\n+\tlen = strlen(path);\n+\tpos = index_name_pos(istate, path, len);\n+\tif (pos < 0)\n+\t\treturn NULL;\n+\t/* only applies to CE_VALID entries */\n+\tif (!(istate->cache[pos]->ce_flags & CE_VALID))\n+\t\treturn NULL;\n+\tdata = read_sha1_file(istate->cache[pos]->sha1, &type, &sz);\n+\tif (!data || type != OBJ_BLOB) {\n+\t\tfree(data);\n+\t\treturn NULL;\n+\t}\n+\t*size = xsize_t(sz);\n+\treturn data;\n+}\n+\n static int add_excludes_from_file_1(const char *fname,\n \t\t\t\t    const char *base,\n \t\t\t\t    int baselen,\n \t\t\t\t    char **buf_p,\n-\t\t\t\t    struct exclude_list *which)\n+\t\t\t\t    struct exclude_list *which,\n+\t\t\t\t    int check_index)\n {\n \tstruct stat st;\n \tint fd, i;\n@@ -212,27 +237,31 @@ static int add_excludes_from_file_1(const char *fname,\n \tchar *buf, *entry;\n \n \tfd = open(fname, O_RDONLY);\n-\tif (fd < 0 || fstat(fd, &st) < 0)\n-\t\tgoto err;\n-\tsize = xsize_t(st.st_size);\n-\tif (size == 0) {\n-\t\tclose(fd);\n-\t\treturn 0;\n+\tif (fd < 0 || fstat(fd, &st) < 0) {\n+\t\tif (0 <= fd)\n+\t\t\tclose(fd);\n+\t\tif (!check_index || (buf = read_index_data(fname, &size)) == NULL)\n+\t\t\treturn -1;\n \t}\n-\tbuf = xmalloc(size+1);\n-\tif (read_in_full(fd, buf, size) != size)\n-\t{\n-\t\tfree(buf);\n-\t\tgoto err;\n+\telse {\n+\t\tsize = xsize_t(st.st_size);\n+\t\tif (size == 0) {\n+\t\t\tclose(fd);\n+\t\t\treturn 0;\n+\t\t}\n+\t\tbuf = xmalloc(size);\n+\t\tif (read_in_full(fd, buf, size) != size) {\n+\t\t\tclose(fd);\n+\t\t\treturn -1;\n+\t\t}\n+\t\tclose(fd);\n \t}\n-\tclose(fd);\n \n \tif (buf_p)\n \t\t*buf_p = buf;\n-\tbuf[size++] = '\\n';\n \tentry = buf;\n-\tfor (i = 0; i < size; i++) {\n-\t\tif (buf[i] == '\\n') {\n+\tfor (i = 0; i <= size; i++) {\n+\t\tif (i == size || buf[i] == '\\n') {\n \t\t\tif (entry != buf + i && entry[0] != '#') {\n \t\t\t\tbuf[i - (i && buf[i-1] == '\\r')] = 0;\n \t\t\t\tadd_exclude(entry, base, baselen, which);\n@@ -241,17 +270,12 @@ static int add_excludes_from_file_1(const char *fname,\n \t\t}\n \t}\n \treturn 0;\n-\n- err:\n-\tif (0 <= fd)\n-\t\tclose(fd);\n-\treturn -1;\n }\n \n void add_excludes_from_file(struct dir_struct *dir, const char *fname)\n {\n \tif (add_excludes_from_file_1(fname, \"\", 0, NULL,\n-\t\t\t\t     &dir->exclude_list[EXC_FILE]) < 0)\n+\t\t\t\t     &dir->exclude_list[EXC_FILE], 0) < 0)\n \t\tdie(\"cannot use %s as an exclude file\", fname);\n }\n \n@@ -302,7 +326,7 @@ static void prep_exclude(struct dir_struct *dir, const char *base, int baselen)\n \t\tstrcpy(dir->basebuf + stk->baselen, dir->exclude_per_dir);\n \t\tadd_excludes_from_file_1(dir->basebuf,\n \t\t\t\t\t dir->basebuf, stk->baselen,\n-\t\t\t\t\t &stk->filebuf, el);\n+\t\t\t\t\t &stk->filebuf, el, 1);\n \t\tdir->exclude_stack = stk;\n \t\tcurrent = stk->baselen;\n \t}\ndiff --git a/t/t3001-ls-files-others-exclude.sh b/t/t3001-ls-files-others-exclude.sh\nindex c65bca8..88b69bc 100755\n--- a/t/t3001-ls-files-others-exclude.sh\n+++ b/t/t3001-ls-files-others-exclude.sh\n@@ -85,6 +85,26 @@ test_expect_success \\\n        >output &&\n      test_cmp expect output'\n \n+test_expect_success 'setup sparse gitignore' '\n+\tgit add .gitignore one/.gitignore one/two/.gitignore &&\n+\tgit update-index --assume-unchanged .gitignore one/.gitignore one/two/.gitignore &&\n+\trm .gitignore one/.gitignore one/two/.gitignore\n+'\n+\n+test_expect_success \\\n+    'git ls-files --others with various exclude options.' \\\n+    'git ls-files --others \\\n+       --exclude=\\*.6 \\\n+       --exclude-per-directory=.gitignore \\\n+       --exclude-from=.git/ignore \\\n+       >output &&\n+     test_cmp expect output'\n+\n+test_expect_success 'restore gitignore' '\n+\tgit checkout .gitignore one/.gitignore one/two/.gitignore &&\n+\trm .git/index\n+'\n+\n cat > excludes-file <<\\EOF\n *.[1-8]\n e*\ndiff --git a/t/t7300-clean.sh b/t/t7300-clean.sh\nindex 929d5d4..4886d5f 100755\n--- a/t/t7300-clean.sh\n+++ b/t/t7300-clean.sh\n@@ -22,6 +22,25 @@ test_expect_success 'setup' '\n \n '\n \n+test_expect_success 'git clean with assume-unchanged .gitignore' '\n+\tgit update-index --assume-unchanged .gitignore &&\n+\trm .gitignore &&\n+\tmkdir -p build docs &&\n+\ttouch a.out src/part3.c docs/manual.txt obj.o build/lib.so &&\n+\tgit clean &&\n+\ttest -f Makefile &&\n+\ttest -f README &&\n+\ttest -f src/part1.c &&\n+\ttest -f src/part2.c &&\n+\ttest ! -f a.out &&\n+\ttest ! -f src/part3.c &&\n+\ttest -f docs/manual.txt &&\n+\ttest -f obj.o &&\n+\ttest -f build/lib.so &&\n+\tgit update-index --no-assume-unchanged .gitignore &&\n+\tgit checkout .gitignore\n+'\n+\n test_expect_success 'git clean' '\n \n \tmkdir -p build docs &&\n-- \n1.6.3.GIT\n"},{"id":"120166","messageId":"1249917562-5931-4-git-send-email-pclouds@gmail.com","threadId":"20522","inReplyTo":"1249917562-5931-3-git-send-email-pclouds@gmail.com","subject":"[RFC PATCH v2 3/4] unpack_trees(): add support for sparse checkout","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2009-08-10T15:19:21Z","receivedAt":"2009-08-10T15:19:21Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This patch teaches unpack_trees() to checkout/remove entries on\nworking directories appropriately when sparse checkout area is\nchanged. Hook \"sparse\" is needed to help determine which entry will be\nchecked out, which will not be.\n\nWhen the hook is run, it is prepared with a pseudo index. The hook then\ncan use \"git update-index --[no-]assume-unchanged\" to manipulate the index.\nIt should not do anything else on the index. Assume unchanged information\nfrom the index will be used to shape working directory.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n cache.h                     |    3 +\n t/t1009-read-tree-sparse.sh |   42 ++++++++++++\n unpack-trees.c              |  152 +++++++++++++++++++++++++++++++++++++++++--\n unpack-trees.h              |    2 +\n 4 files changed, 193 insertions(+), 6 deletions(-)\n create mode 100755 t/t1009-read-tree-sparse.sh\n\ndiff --git a/cache.h b/cache.h\nindex 1a2a3c9..dfad54a 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -177,6 +177,9 @@ struct cache_entry {\n #define CE_HASHED    (0x100000)\n #define CE_UNHASHED  (0x200000)\n \n+/* Only remove in work directory, not index */\n+#define CE_WT_REMOVE (0x400000)\n+\n /*\n  * Extended on-disk flags\n  */\ndiff --git a/t/t1009-read-tree-sparse.sh b/t/t1009-read-tree-sparse.sh\nnew file mode 100755\nindex 0000000..b613a89\n--- /dev/null\n+++ b/t/t1009-read-tree-sparse.sh\n@@ -0,0 +1,42 @@\n+#!/bin/sh\n+\n+test_description='sparse hook tests'\n+\n+. ./test-lib.sh\n+\n+make_hook() {\n+\techo \"#!/bin/sh\" > .git/hooks/sparse &&\n+\techo \"$1\" >> .git/hooks/sparse &&\n+\tchmod u+x .git/hooks/sparse\n+}\n+\n+test_expect_success setup '\n+\techo one > one &&\n+\techo two > two &&\n+\tgit add one two &&\n+\tgit commit -m onetwo &&\n+\techo three > three &&\n+\tgit add three &&\n+\tgit commit -m three\n+'\n+\n+mkdir .git/hooks\n+\n+test_expect_success 'failed hook' '\n+\tmake_hook \"exit 1\" &&\n+\ttest_must_fail git read-tree -u -m HEAD\n+'\n+\n+test_expect_success 'remove one' '\n+\tmake_hook \"git update-index --assume-unchanged one\"\n+\tgit read-tree -u -m HEAD &&\n+\ttest ! -f one\n+'\n+\n+test_expect_success 're-add one' '\n+\tmake_hook \"git update-index --no-assume-unchanged one\" &&\n+\tgit read-tree -u -m HEAD &&\n+\ttest -f one\n+'\n+\n+test_done\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 720f7a1..f407bf5 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -8,6 +8,7 @@\n #include \"progress.h\"\n #include \"refs.h\"\n #include \"attr.h\"\n+#include \"run-command.h\"\n \n /*\n  * Error messages expected by scripts out of plumbing commands such as\n@@ -32,6 +33,12 @@ static struct unpack_trees_error_msgs unpack_plumbing_errors = {\n \n \t/* bind_overlap */\n \t\"Entry '%s' overlaps with '%s'.  Cannot bind.\",\n+\n+\t/* sparse_not_uptodate_file */\n+\t\"Entry '%s' not uptodate. Cannot update sparse checkout.\",\n+\n+\t/* would_lose_orphaned */\n+\t\"Orphaned working tree file '%s' would be %s by sparse checkout update.\",\n };\n \n #define ERRORMSG(o,fld) \\\n@@ -78,7 +85,7 @@ static int check_updates(struct unpack_trees_options *o)\n \tif (o->update && o->verbose_update) {\n \t\tfor (total = cnt = 0; cnt < index->cache_nr; cnt++) {\n \t\t\tstruct cache_entry *ce = index->cache[cnt];\n-\t\t\tif (ce->ce_flags & (CE_UPDATE | CE_REMOVE))\n+\t\t\tif (ce->ce_flags & (CE_UPDATE | CE_REMOVE | CE_WT_REMOVE))\n \t\t\t\ttotal++;\n \t\t}\n \n@@ -92,6 +99,13 @@ static int check_updates(struct unpack_trees_options *o)\n \tfor (i = 0; i < index->cache_nr; i++) {\n \t\tstruct cache_entry *ce = index->cache[i];\n \n+\t\tif (ce->ce_flags & CE_WT_REMOVE) {\n+\t\t\tdisplay_progress(progress, ++cnt);\n+\t\t\tif (o->update)\n+\t\t\t\tunlink_entry(ce);\n+\t\t\tcontinue;\n+\t\t}\n+\n \t\tif (ce->ce_flags & CE_REMOVE) {\n \t\t\tdisplay_progress(progress, ++cnt);\n \t\t\tif (o->update)\n@@ -118,6 +132,106 @@ static int check_updates(struct unpack_trees_options *o)\n \treturn errs != 0;\n }\n \n+static int verify_uptodate_sparse(struct cache_entry *ce, struct unpack_trees_options *o);\n+static int verify_absent_sparse(struct cache_entry *ce, const char *action, struct unpack_trees_options *o);\n+static int run_sparse_hook(struct unpack_trees_options *o)\n+{\n+\tstruct index_state *index = &o->result;\n+\tstruct index_state sparse_index = *index;\n+\tstruct cache_entry *ce, *sparse_ce;\n+\tchar sparse_index_file[PATH_MAX];\n+\tchar sparse_index_env[PATH_MAX];\n+\tconst char *argv[2], *env[2];\n+\tstruct child_process cp;\n+\tint fd, i, j;\n+\n+\tif (access(git_path(\"hooks/sparse\"), X_OK) < 0)\n+\t\treturn 0;\n+\n+\tstrcpy(sparse_index_file, git_path(\"sparse\"));\n+\tfd = open(sparse_index_file, O_WRONLY | O_CREAT, 0600);\n+\tif (fd < 0) {\n+\t\terror(\"Unable to open %s for writing\", sparse_index_file);\n+\t\treturn -1;\n+\t}\n+\t/* FIXME: write_index may change something */\n+\tif (write_index(&sparse_index, fd)) {\n+\t\terror(\"Unable to write index to %s\", sparse_index_file);\n+\t\tclose(fd);\n+\t\treturn -1;\n+\t}\n+\tclose(fd);\n+\n+\tmemset(&cp, 0, sizeof(cp));\n+\targv[0] = git_path(\"hooks/sparse\");\n+\targv[1] = NULL;\n+\tcp.argv = argv;\n+\tcp.no_stdin = 1;\n+\tcp.stdout_to_stderr = 1;\n+\tsnprintf(sparse_index_env, sizeof(sparse_index_env), \"GIT_INDEX_FILE=%s\", sparse_index_file);\n+\tenv[0] = sparse_index_env;\n+\tenv[1] = NULL;\n+\tcp.env = env;\n+\tif (run_command(&cp)) {\n+\t\terror(\"Failed to run hook 'sparse'\");\n+\t\tunlink(sparse_index_file);\n+\t\treturn -1;\n+\t}\n+\n+\tdiscard_index(&sparse_index);\n+\tread_index_from(&sparse_index, sparse_index_file);\n+\tunlink(sparse_index_file);\n+\n+\tce = index->cache[0];\n+\tsparse_ce = sparse_index.cache[0];\n+\tfor (i = j = 0; i < index->cache_nr; i++, ce++) {\n+\t\tint was_checkout = !(ce->ce_flags & CE_VALID);\n+\n+\t\tif (ce_stage(ce))\n+\t\t\tcontinue;\n+\n+\t\t/*\n+\t\t * We only care about files getting into the checkout area\n+\t\t * If merge strategies want to remove some, go ahead\n+\t\t */\n+\t\tif (ce->ce_flags & CE_REMOVE)\n+\t\t\tcontinue;\n+\n+\t\twhile (j < sparse_index.cache_nr &&\n+\t\t       cache_name_compare(sparse_ce->name, sparse_ce->ce_flags, ce->name, ce->ce_flags) < 0) {\n+\t\t\tsparse_ce++;\n+\t\t\tj++;\n+\t\t}\n+\t\tif (j < sparse_index.cache_nr &&\n+\t\t    !cache_name_compare(sparse_ce->name, sparse_ce->ce_flags, ce->name, ce->ce_flags))\n+\t\t\tce->ce_flags = (ce->ce_flags & ~CE_VALID) | (sparse_ce->ce_flags & CE_VALID);\n+\n+\t\t/* Update worktree, add/remove entries if needed */\n+\n+\t\tif (was_checkout && ce->ce_flags & CE_VALID) {\n+\t\t\t/*\n+\t\t\t * If CE_UPDATE is set, verify_uptodate() must be called already\n+\t\t\t * also stat info may have lost after merged_entry() so calling\n+\t\t\t * verify_uptodate() again may fail\n+\t\t\t */\n+\t\t\tif (!(ce->ce_flags & CE_UPDATE) && verify_uptodate_sparse(ce, o))\n+\t\t\t\treturn -1;\n+\t\t\tce->ce_flags |= CE_WT_REMOVE;\n+\t\t}\n+\t\tif (!was_checkout && !(ce->ce_flags & CE_VALID)) {\n+\t\t\tif (verify_absent_sparse(ce, \"overwritten\", o))\n+\t\t\t\treturn -1;\n+\t\t\tce->ce_flags |= CE_UPDATE;\n+\t\t}\n+\n+\t\t/* merge strategies may set CE_UPDATE outside checkout area */\n+\t\tif (ce->ce_flags & CE_VALID)\n+\t\t\tce->ce_flags &= ~CE_UPDATE;\n+\n+\t}\n+\treturn 0;\n+}\n+\n static inline int call_unpack_fn(struct cache_entry **src, struct unpack_trees_options *o)\n {\n \tint ret = o->fn(src, o);\n@@ -416,6 +530,9 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \tif (o->trivial_merges_only && o->nontrivial_merge)\n \t\treturn unpack_failed(o, \"Merge requires file-level merging\");\n \n+\tif (run_sparse_hook(o))\n+\t\treturn unpack_failed(o, NULL);\n+\n \to->src_index = NULL;\n \tret = check_updates(o) ? (-2) : 0;\n \tif (o->dst_index)\n@@ -445,8 +562,9 @@ static int same(struct cache_entry *a, struct cache_entry *b)\n  * When a CE gets turned into an unmerged entry, we\n  * want it to be up-to-date\n  */\n-static int verify_uptodate(struct cache_entry *ce,\n-\t\tstruct unpack_trees_options *o)\n+static int verify_uptodate_generic(struct cache_entry *ce,\n+\t\t\t\t   struct unpack_trees_options *o,\n+\t\t\t\t   const char *error_msg)\n {\n \tstruct stat st;\n \n@@ -471,7 +589,18 @@ static int verify_uptodate(struct cache_entry *ce,\n \tif (errno == ENOENT)\n \t\treturn 0;\n \treturn o->gently ? -1 :\n-\t\terror(ERRORMSG(o, not_uptodate_file), ce->name);\n+\t\terror(error_msg, ce->name);\n+}\n+\n+static int verify_uptodate(struct cache_entry *ce,\n+\t\t\t   struct unpack_trees_options *o)\n+{\n+\treturn verify_uptodate_generic(ce, o, ERRORMSG(o, not_uptodate_file));\n+}\n+static int verify_uptodate_sparse(struct cache_entry *ce,\n+\t\t\t\t  struct unpack_trees_options *o)\n+{\n+\treturn verify_uptodate_generic(ce, o, ERRORMSG(o, sparse_not_uptodate_file));\n }\n \n static void invalidate_ce_path(struct cache_entry *ce, struct unpack_trees_options *o)\n@@ -579,8 +708,9 @@ static int icase_exists(struct unpack_trees_options *o, struct cache_entry *dst,\n  * We do not want to remove or overwrite a working tree file that\n  * is not tracked, unless it is ignored.\n  */\n-static int verify_absent(struct cache_entry *ce, const char *action,\n-\t\t\t struct unpack_trees_options *o)\n+static int verify_absent_generic(struct cache_entry *ce, const char *action,\n+\t\t\t\t struct unpack_trees_options *o,\n+\t\t\t\t const char *error_msg)\n {\n \tstruct stat st;\n \n@@ -660,6 +790,16 @@ static int verify_absent(struct cache_entry *ce, const char *action,\n \t}\n \treturn 0;\n }\n+static int verify_absent(struct cache_entry *ce, const char *action,\n+\t\t\t struct unpack_trees_options *o)\n+{\n+\treturn verify_absent_generic(ce, action, o, ERRORMSG(o, would_lose_untracked));\n+}\n+static int verify_absent_sparse(struct cache_entry *ce, const char *action,\n+\t\t\t struct unpack_trees_options *o)\n+{\n+\treturn verify_absent_generic(ce, action, o, ERRORMSG(o, would_lose_orphaned));\n+}\n \n static int merged_entry(struct cache_entry *merge, struct cache_entry *old,\n \t\tstruct unpack_trees_options *o)\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex d19df44..ad21823 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -14,6 +14,8 @@ struct unpack_trees_error_msgs {\n \tconst char *not_uptodate_dir;\n \tconst char *would_lose_untracked;\n \tconst char *bind_overlap;\n+\tconst char *sparse_not_uptodate_file;\n+\tconst char *would_lose_orphaned;\n };\n \n struct unpack_trees_options {\n-- \n1.6.3.GIT\n"},{"id":"120167","messageId":"1249917562-5931-5-git-send-email-pclouds@gmail.com","threadId":"20522","inReplyTo":"1249917562-5931-4-git-send-email-pclouds@gmail.com","subject":"[RFC PATCH v2 4/4] read-tree: add --no-sparse to turn off sparse hook","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2009-08-10T15:19:22Z","receivedAt":"2009-08-10T15:19:22Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n builtin-read-tree.c         |    4 +++-\n t/t1009-read-tree-sparse.sh |    6 ++++++\n unpack-trees.c              |    6 ++++--\n unpack-trees.h              |    1 +\n 4 files changed, 14 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin-read-tree.c b/builtin-read-tree.c\nindex 9c2d634..4c1061e 100644\n--- a/builtin-read-tree.c\n+++ b/builtin-read-tree.c\n@@ -31,7 +31,7 @@ static int list_tree(unsigned char *sha1)\n }\n \n static const char * const read_tree_usage[] = {\n-\t\"git read-tree [[-m [--trivial] [--aggressive] | --reset | --prefix=<prefix>] [-u [--exclude-per-directory=<gitignore>] | -i]]  [--index-output=<file>] <tree-ish1> [<tree-ish2> [<tree-ish3>]]\",\n+\t\"git read-tree [[-m [--trivial] [--aggressive] | --reset | --prefix=<prefix>] [-u [--exclude-per-directory=<gitignore>] | -i]] [--no-sparse] [--index-output=<file>] <tree-ish1> [<tree-ish2> [<tree-ish3>]]\",\n \tNULL\n };\n \n@@ -98,6 +98,8 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n \t\t  PARSE_OPT_NONEG, exclude_per_directory_cb },\n \t\tOPT_SET_INT('i', NULL, &opts.index_only,\n \t\t\t    \"don't check the working tree after merging\", 1),\n+\t\tOPT_SET_INT(0, \"no-sparse\", &opts.no_sparse_hook,\n+\t\t\t    \"do not run sparse hook\", 1),\n \t\tOPT_END()\n \t};\n \ndiff --git a/t/t1009-read-tree-sparse.sh b/t/t1009-read-tree-sparse.sh\nindex b613a89..18115b1 100755\n--- a/t/t1009-read-tree-sparse.sh\n+++ b/t/t1009-read-tree-sparse.sh\n@@ -39,4 +39,10 @@ test_expect_success 're-add one' '\n \ttest -f one\n '\n \n+test_expect_success 'read-tree --no-sparse' '\n+\tmake_hook \"git update-index --assume-unchanged one\"\n+\tgit read-tree -u -m --no-sparse HEAD &&\n+\ttest -f one\n+'\n+\n test_done\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex f407bf5..d087112 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -530,8 +530,10 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \tif (o->trivial_merges_only && o->nontrivial_merge)\n \t\treturn unpack_failed(o, \"Merge requires file-level merging\");\n \n-\tif (run_sparse_hook(o))\n-\t\treturn unpack_failed(o, NULL);\n+\tif (!o->no_sparse_hook) {\n+\t\tif (run_sparse_hook(o))\n+\t\t\treturn unpack_failed(o, NULL);\n+\t}\n \n \to->src_index = NULL;\n \tret = check_updates(o) ? (-2) : 0;\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex ad21823..81eb2ef 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -30,6 +30,7 @@ struct unpack_trees_options {\n \t\t     skip_unmerged,\n \t\t     initial_checkout,\n \t\t     diff_index_cached,\n+\t\t     no_sparse_hook,\n \t\t     gently;\n \tconst char *prefix;\n \tint pos;\n-- \n1.6.3.GIT\n"},{"id":"120184","messageId":"alpine.DEB.1.00.0908101800480.8324@intel-tinevez-2-302","threadId":"20522","inReplyTo":"1249917562-5931-2-git-send-email-pclouds@gmail.com","subject":"Re: [RFC PATCH v2 1/4] Prevent diff machinery from examining assume-unchanged entries on worktree","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-08-10T16:20:19Z","receivedAt":"2009-08-10T16:20:19Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 10 Aug 2009, Nguyễn Thái Ngọc Duy wrote:\n\n> diff --git a/diff-lib.c b/diff-lib.c\n> index b7813af..f5787f6 100644\n> --- a/diff-lib.c\n> +++ b/diff-lib.c\n> @@ -337,6 +337,8 @@ static void do_oneway_diff(struct unpack_trees_options *o,\n>  \tstruct rev_info *revs = o->unpack_data;\n>  \tint match_missing, cached;\n>  \n> +\t/* if the entry is not checked out, don't examine work tree */\n> +\tcached = o->index_only || (idx && (idx->ce_flags & CE_VALID));\n>  \t/*\n>  \t * Backward compatibility wart - \"diff-index -m\" does\n>  \t * not mean \"do not ignore merges\", but \"match_missing\".\n> @@ -344,7 +346,6 @@ static void do_oneway_diff(struct unpack_trees_options *o,\n>  \t * But with the revision flag parsing, that's found in\n>  \t * \"!revs->ignore_merges\".\n>  \t */\n> -\tcached = o->index_only;\n>  \tmatch_missing = !revs->ignore_merges;\n>  \n>  \tif (cached && idx && ce_stage(idx)) {\n\nOut of curiosity: why did that line have to move up?\n\nCiao,\nDscho\n\n> diff --git a/t/t4039-diff-assume-unchanged.sh b/t/t4039-diff-assume-unchanged.sh\n> new file mode 100755\n> index 0000000..d0e46a7\n> --- /dev/null\n> +++ b/t/t4039-diff-assume-unchanged.sh\n> @@ -0,0 +1,31 @@\n> +#!/bin/sh\n> +\n> +test_description='diff with assume-unchanged entries'\n> +\n> +. ./test-lib.sh\n> +\n> +# external diff has been tested in t4020-diff-external.sh\n> +\n> +test_expect_success 'setup' '\n> +\techo zero > zero &&\n> +\tgit add zero &&\n> +\tgit commit -m zero &&\n> +\techo one > one &&\n> +\techo two > two &&\n> +\tgit add one two &&\n> +\tgit commit -m onetwo &&\n> +\tgit update-index --assume-unchanged one &&\n> +\techo borked >> one &&\n> +\ttest \"$(git ls-files -v one)\" = \"h one\"\n> +'\n\nMaybe use test_commit, to make it more readable?\n\n> +\n> +test_expect_success 'diff-index does not examine assume-unchanged entries' '\n> +\tgit diff-index HEAD^ -- one | grep -q 5626abf0f72e58d7a153368ba57db4c673c0e171 \n> +'\n> +\n> +# TODO ced_uptodate()\n\nWhat is this about?\n\n> +test_expect_success 'diff-files does not examine assume-unchanged entries' '\n> +\t/usr/bin/git diff-files -- one\n> +'\n> +\n> +test_done\n\nThanks,\nDscho\n"},{"id":"120190","messageId":"alpine.DEB.1.00.0908101826250.8324@intel-tinevez-2-302","threadId":"20522","inReplyTo":"1249917562-5931-3-git-send-email-pclouds@gmail.com","subject":"Re: [RFC PATCH v2 2/4] gitignore: read from index if .gitignore is assume-unchanged","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-08-10T16:33:10Z","receivedAt":"2009-08-10T16:33:10Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 10 Aug 2009, Nguyễn Thái Ngọc Duy wrote:\n\n> diff --git a/dir.c b/dir.c\n> index e05b850..e55344f 100644\n> --- a/dir.c\n> +++ b/dir.c\n> @@ -200,11 +200,36 @@ void add_exclude(const char *string, const char *base,\n>  \twhich->excludes[which->nr++] = x;\n>  }\n>  \n> +static void *read_index_data(const char *path, size_t *size)\n\nHow about calling it \"read_assume_unchanged_from_index()\" instead?  I \nsuggest this because it does not read the index from the data if the path \nis not marked assume unchanged...\n\n> @@ -212,27 +237,31 @@ static int add_excludes_from_file_1(const char *fname,\n>\n> [...]\n>  \n>  \tif (buf_p)\n>  \t\t*buf_p = buf;\n> -\tbuf[size++] = '\\n';\n>  \tentry = buf;\n> -\tfor (i = 0; i < size; i++) {\n> -\t\tif (buf[i] == '\\n') {\n> +\tfor (i = 0; i <= size; i++) {\n> +\t\tif (i == size || buf[i] == '\\n') {\n>  \t\t\tif (entry != buf + i && entry[0] != '#') {\n>  \t\t\t\tbuf[i - (i && buf[i-1] == '\\r')] = 0;\n>  \t\t\t\tadd_exclude(entry, base, baselen, which);\n\nShould this change not rather be a separate one?\n\n> @@ -241,17 +270,12 @@ static int add_excludes_from_file_1(const char *fname,\n>  \t\t}\n>  \t}\n>  \treturn 0;\n> -\n> - err:\n> -\tif (0 <= fd)\n> -\t\tclose(fd);\n> -\treturn -1;\n>  }\n>  \n>  void add_excludes_from_file(struct dir_struct *dir, const char *fname)\n>  {\n>  \tif (add_excludes_from_file_1(fname, \"\", 0, NULL,\n> -\t\t\t\t     &dir->exclude_list[EXC_FILE]) < 0)\n> +\t\t\t\t     &dir->exclude_list[EXC_FILE], 0) < 0)\n\nCould you mention in the commit message why this function does not want to \ncheck the index (I _guess_ it is because this code path only tries to read \n.git/info/exclude, but it would be better to be sure).\n\n>  \t\tdie(\"cannot use %s as an exclude file\", fname);\n>  }\n>  \n> @@ -302,7 +326,7 @@ static void prep_exclude(struct dir_struct *dir, const char *base, int baselen)\n>  \t\tstrcpy(dir->basebuf + stk->baselen, dir->exclude_per_dir);\n>  \t\tadd_excludes_from_file_1(dir->basebuf,\n>  \t\t\t\t\t dir->basebuf, stk->baselen,\n> -\t\t\t\t\t &stk->filebuf, el);\n> +\t\t\t\t\t &stk->filebuf, el, 1);\n>  \t\tdir->exclude_stack = stk;\n>  \t\tcurrent = stk->baselen;\n>  \t}\n> diff --git a/t/t3001-ls-files-others-exclude.sh b/t/t3001-ls-files-others-exclude.sh\n> index c65bca8..88b69bc 100755\n> --- a/t/t3001-ls-files-others-exclude.sh\n> +++ b/t/t3001-ls-files-others-exclude.sh\n> @@ -85,6 +85,26 @@ test_expect_success \\\n>         >output &&\n>       test_cmp expect output'\n>  \n> +test_expect_success 'setup sparse gitignore' '\n> +\tgit add .gitignore one/.gitignore one/two/.gitignore &&\n> +\tgit update-index --assume-unchanged .gitignore one/.gitignore one/two/.gitignore &&\n> +\trm .gitignore one/.gitignore one/two/.gitignore\n> +'\n\nYou're probably less sloppy than me; I'd have defined a variable like \nthis:\n\n\tignores=\".gitignore one/.gitignore one/two/.gitignore\"\n\nand used it for the three calls, just to make sure that I do not fsck \nanything up there due to fat fingers.\n\nThanks,\nDscho\n"},{"id":"120192","messageId":"alpine.DEB.1.00.0908101837510.8324@intel-tinevez-2-302","threadId":"20522","inReplyTo":"1249917562-5931-4-git-send-email-pclouds@gmail.com","subject":"Re: [RFC PATCH v2 3/4] unpack_trees(): add support for sparse checkout","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-08-10T16:41:17Z","receivedAt":"2009-08-10T16:41:17Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 10 Aug 2009, Nguyễn Thái Ngọc Duy wrote:\n\n> This patch teaches unpack_trees() to checkout/remove entries on working \n> directories appropriately when sparse checkout area is changed. Hook \n> \"sparse\" is needed to help determine which entry will be checked out, \n> which will not be.\n> \n> When the hook is run, it is prepared with a pseudo index. The hook then \n> can use \"git update-index --[no-]assume-unchanged\" to manipulate the \n> index. It should not do anything else on the index. Assume unchanged \n> information from the index will be used to shape working directory.\n\nIf I understand correctly, the purpose of the hook is to allow the user to \nmark something as unwanted preemptively, right?\n\nIf that is the sole reason for the hook, would it not be better to add \nsupport for a file .git/info/sparse which has the same syntax as \n.git/info/exclude, and which is used to determine if an \nadded/modified/deleted file is supposed to be in the \"sparse\" area or not?\n\nSomething like\n\n\t*\n\t!/Documentation/\n\ncomes to mind.\n\nThanks,\nDscho"},{"id":"120193","messageId":"alpine.DEB.1.00.0908101842530.8324@intel-tinevez-2-302","threadId":"20522","inReplyTo":"1249917562-5931-5-git-send-email-pclouds@gmail.com","subject":"Re: [RFC PATCH v2 4/4] read-tree: add --no-sparse to turn off sparse hook","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-08-10T16:46:05Z","receivedAt":"2009-08-10T16:46:05Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 10 Aug 2009, Nguyễn Thái Ngọc Duy wrote:\n\n> diff --git a/unpack-trees.c b/unpack-trees.c\n> index f407bf5..d087112 100644\n> --- a/unpack-trees.c\n> +++ b/unpack-trees.c\n> @@ -530,8 +530,10 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n>  \tif (o->trivial_merges_only && o->nontrivial_merge)\n>  \t\treturn unpack_failed(o, \"Merge requires file-level merging\");\n>  \n> -\tif (run_sparse_hook(o))\n> -\t\treturn unpack_failed(o, NULL);\n> +\tif (!o->no_sparse_hook) {\n> +\t\tif (run_sparse_hook(o))\n> +\t\t\treturn unpack_failed(o, NULL);\n> +\t}\n>  \n\nIMHO this would read nicelier as\n\n\tif (!o->no_sparse_hook && run_sparse_hook(o))\n\t\treturn unpack_failed(o, NULL);\n\n> diff --git a/unpack-trees.h b/unpack-trees.h\n> index ad21823..81eb2ef 100644\n> --- a/unpack-trees.h\n> +++ b/unpack-trees.h\n> @@ -30,6 +30,7 @@ struct unpack_trees_options {\n>  \t\t     skip_unmerged,\n>  \t\t     initial_checkout,\n>  \t\t     diff_index_cached,\n> +\t\t     no_sparse_hook,\n>  \t\t     gently;\n\nHmm.  I understand that the assumption is that memset(&opts, 0, \nsizeof(opts)); should give you a sensible default, but I cannot avoid \nnoticing that \"no_sparse_hook = 0\" is a double negation, something to be \navoided...\n\nThanks,\nDscho\n"},{"id":"120215","messageId":"fcaeb9bf0908101834n7cc7cfaakbf2d92fe8f32e9b1@mail.gmail.com","threadId":"20522","inReplyTo":"alpine.DEB.1.00.0908101800480.8324@intel-tinevez-2-302","subject":"Re: [RFC PATCH v2 1/4] Prevent diff machinery from examining assume-unchanged entries on worktree","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2009-08-11T01:34:11Z","receivedAt":"2009-08-11T01:34:11Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2009/8/10 Johannes Schindelin <Johannes.Schindelin@gmx.de>:\n>> +test_expect_success 'setup' '\n>> +     echo zero > zero &&\n>> +     git add zero &&\n>> +     git commit -m zero &&\n>> +     echo one > one &&\n>> +     echo two > two &&\n>> +     git add one two &&\n>> +     git commit -m onetwo &&\n>> +     git update-index --assume-unchanged one &&\n>> +     echo borked >> one &&\n>> +     test \"$(git ls-files -v one)\" = \"h one\"\n>> +'\n>\n> Maybe use test_commit, to make it more readable?\n\nThanks. I did not know that (or forgot already). Will use it.\n\n>\n>> +\n>> +test_expect_success 'diff-index does not examine assume-unchanged entries' '\n>> +     git diff-index HEAD^ -- one | grep -q 5626abf0f72e58d7a153368ba57db4c673c0e171\n>> +'\n>> +\n>> +# TODO ced_uptodate()\n>\n> What is this about?\n\nIt tests \"if (ce_uptodate(ce) || (ce->ce_flags & CE_VALID))\" and I was\npretty sure it hit ce_uptodate() first, so the second expression was\nnot tested.\n-- \nDuy\n"},{"id":"120216","messageId":"fcaeb9bf0908101838k37751fclac5c572eb042138e@mail.gmail.com","threadId":"20522","inReplyTo":"alpine.DEB.1.00.0908101842530.8324@intel-tinevez-2-302","subject":"Re: [RFC PATCH v2 4/4] read-tree: add --no-sparse to turn off sparse hook","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2009-08-11T01:38:03Z","receivedAt":"2009-08-11T01:38:03Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2009/8/10 Johannes Schindelin <Johannes.Schindelin@gmx.de>:\n> Hi,\n>\n> On Mon, 10 Aug 2009, Nguyễn Thái Ngọc Duy wrote:\n>\n>> diff --git a/unpack-trees.c b/unpack-trees.c\n>> index f407bf5..d087112 100644\n>> --- a/unpack-trees.c\n>> +++ b/unpack-trees.c\n>> @@ -530,8 +530,10 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n>>       if (o->trivial_merges_only && o->nontrivial_merge)\n>>               return unpack_failed(o, \"Merge requires file-level merging\");\n>>\n>> -     if (run_sparse_hook(o))\n>> -             return unpack_failed(o, NULL);\n>> +     if (!o->no_sparse_hook) {\n>> +             if (run_sparse_hook(o))\n>> +                     return unpack_failed(o, NULL);\n>> +     }\n>>\n>\n> IMHO this would read nicelier as\n>\n>        if (!o->no_sparse_hook && run_sparse_hook(o))\n>                return unpack_failed(o, NULL);\n\nRight.\n\n>> diff --git a/unpack-trees.h b/unpack-trees.h\n>> index ad21823..81eb2ef 100644\n>> --- a/unpack-trees.h\n>> +++ b/unpack-trees.h\n>> @@ -30,6 +30,7 @@ struct unpack_trees_options {\n>>                    skip_unmerged,\n>>                    initial_checkout,\n>>                    diff_index_cached,\n>> +                  no_sparse_hook,\n>>                    gently;\n>\n> Hmm.  I understand that the assumption is that memset(&opts, 0,\n> sizeof(opts)); should give you a sensible default, but I cannot avoid\n> noticing that \"no_sparse_hook = 0\" is a double negation, something to be\n> avoided...\n\nskip_sparse_hook then? :-)\n-- \nDuy\n"},{"id":"120217","messageId":"fcaeb9bf0908101847w7b3b22d9w84514a29ceae719e@mail.gmail.com","threadId":"20522","inReplyTo":"alpine.DEB.1.00.0908101837510.8324@intel-tinevez-2-302","subject":"Re: [RFC PATCH v2 3/4] unpack_trees(): add support for sparse checkout","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2009-08-11T01:47:53Z","receivedAt":"2009-08-11T01:47:53Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2009/8/10 Johannes Schindelin <Johannes.Schindelin@gmx.de>:\n> Hi,\n>\n> On Mon, 10 Aug 2009, Nguyễn Thái Ngọc Duy wrote:\n>\n>> This patch teaches unpack_trees() to checkout/remove entries on working\n>> directories appropriately when sparse checkout area is changed. Hook\n>> \"sparse\" is needed to help determine which entry will be checked out,\n>> which will not be.\n>>\n>> When the hook is run, it is prepared with a pseudo index. The hook then\n>> can use \"git update-index --[no-]assume-unchanged\" to manipulate the\n>> index. It should not do anything else on the index. Assume unchanged\n>> information from the index will be used to shape working directory.\n>\n> If I understand correctly, the purpose of the hook is to allow the user to\n> mark something as unwanted preemptively, right?\n>\n> If that is the sole reason for the hook, would it not be better to add\n> support for a file .git/info/sparse which has the same syntax as\n> .git/info/exclude, and which is used to determine if an\n> added/modified/deleted file is supposed to be in the \"sparse\" area or not?\n>\n> Something like\n>\n>        *\n>        !/Documentation/\n>\n> comes to mind.\n\nThat was what the original series was about (although I used git\nconfig instead of .git/info/sparse). The hook has two advantages:\n\n - flexibility: you can set things differently based on branch, for\nexample, or filter files based on certain file content.\n - less code bloat (and less intrusive too), compare ~1000 lines of\nchange of the original series with this series.\n-- \nDuy\n"},{"id":"120218","messageId":"fcaeb9bf0908101857x7a44d3dfge20e45d24daee9bf@mail.gmail.com","threadId":"20522","inReplyTo":"alpine.DEB.1.00.0908101826250.8324@intel-tinevez-2-302","subject":"Re: [RFC PATCH v2 2/4] gitignore: read from index if .gitignore is assume-unchanged","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2009-08-11T01:57:30Z","receivedAt":"2009-08-11T01:57:30Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2009/8/10 Johannes Schindelin <Johannes.Schindelin@gmx.de>:\n> Hi,\n>\n> On Mon, 10 Aug 2009, Nguyễn Thái Ngọc Duy wrote:\n>\n>> diff --git a/dir.c b/dir.c\n>> index e05b850..e55344f 100644\n>> --- a/dir.c\n>> +++ b/dir.c\n>> @@ -200,11 +200,36 @@ void add_exclude(const char *string, const char *base,\n>>       which->excludes[which->nr++] = x;\n>>  }\n>>\n>> +static void *read_index_data(const char *path, size_t *size)\n>\n> How about calling it \"read_assume_unchanged_from_index()\" instead?  I\n> suggest this because it does not read the index from the data if the path\n> is not marked assume unchanged...\n\nAgree.\n\n>> @@ -212,27 +237,31 @@ static int add_excludes_from_file_1(const char *fname,\n>>\n>> [...]\n>>\n>>       if (buf_p)\n>>               *buf_p = buf;\n>> -     buf[size++] = '\\n';\n>>       entry = buf;\n>> -     for (i = 0; i < size; i++) {\n>> -             if (buf[i] == '\\n') {\n>> +     for (i = 0; i <= size; i++) {\n>> +             if (i == size || buf[i] == '\\n') {\n>>                       if (entry != buf + i && entry[0] != '#') {\n>>                               buf[i - (i && buf[i-1] == '\\r')] = 0;\n>>                               add_exclude(entry, base, baselen, which);\n>\n> Should this change not rather be a separate one?\n\nYou meant a separate patch? It is tied to this patch, because if bus\nis read from read_index_data, it does not have extra space for '\\n' at\nthe end.\n\n>> @@ -241,17 +270,12 @@ static int add_excludes_from_file_1(const char *fname,\n>>               }\n>>       }\n>>       return 0;\n>> -\n>> - err:\n>> -     if (0 <= fd)\n>> -             close(fd);\n>> -     return -1;\n>>  }\n>>\n>>  void add_excludes_from_file(struct dir_struct *dir, const char *fname)\n>>  {\n>>       if (add_excludes_from_file_1(fname, \"\", 0, NULL,\n>> -                                  &dir->exclude_list[EXC_FILE]) < 0)\n>> +                                  &dir->exclude_list[EXC_FILE], 0) < 0)\n>\n> Could you mention in the commit message why this function does not want to\n> check the index (I _guess_ it is because this code path only tries to read\n> .git/info/exclude, but it would be better to be sure).\n\nTo retain old behaviour. But I have to check its callers. Maybe we\nwant to check the index too.\n\n>> @@ -85,6 +85,26 @@ test_expect_success \\\n>>         >output &&\n>>       test_cmp expect output'\n>>\n>> +test_expect_success 'setup sparse gitignore' '\n>> +     git add .gitignore one/.gitignore one/two/.gitignore &&\n>> +     git update-index --assume-unchanged .gitignore one/.gitignore one/two/.gitignore &&\n>> +     rm .gitignore one/.gitignore one/two/.gitignore\n>> +'\n>\n> You're probably less sloppy than me; I'd have defined a variable like\n> this:\n>\n>        ignores=\".gitignore one/.gitignore one/two/.gitignore\"\n>\n> and used it for the three calls, just to make sure that I do not fsck\n> anything up there due to fat fingers.\n\nI have slim ones :-) But \"git add $ignores && git update-index\n$ignores && rm $ignores\" is easier to read.\n-- \nDuy\n"},{"id":"120232","messageId":"7vhbwforvk.fsf@alter.siamese.dyndns.org","threadId":"20522","inReplyTo":"fcaeb9bf0908101838k37751fclac5c572eb042138e@mail.gmail.com","subject":"Re: [RFC PATCH v2 4/4] read-tree: add --no-sparse to turn off sparse hook","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-08-11T05:13:03Z","receivedAt":"2009-08-11T05:13:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyen Thai Ngoc Duy <pclouds@gmail.com> writes:\n\n>> Hmm.  I understand that the assumption is that memset(&opts, 0,\n>> sizeof(opts)); should give you a sensible default, but I cannot avoid\n>> noticing that \"no_sparse_hook = 0\" is a double negation, something to be\n>> avoided...\n>\n> skip_sparse_hook then? :-)\n\nWhy not making the hook to be skipped by default, and pass an explicit\noption to trigger the hook?\n\nI like Dscho's other suggestion to use patterns like .gitignore instead of\nusing hook scripts that needs to be ported across platforms, by the way.\n"},{"id":"120235","messageId":"alpine.DEB.1.00.0908110844160.4638@intel-tinevez-2-302","threadId":"20522","inReplyTo":"fcaeb9bf0908101834n7cc7cfaakbf2d92fe8f32e9b1@mail.gmail.com","subject":"Re: [RFC PATCH v2 1/4] Prevent diff machinery from examining assume-unchanged entries on worktree","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-08-11T06:45:21Z","receivedAt":"2009-08-11T06:45:21Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 11 Aug 2009, Nguyen Thai Ngoc Duy wrote:\n\n> >> +test_expect_success 'diff-index does not examine assume-unchanged entries' '\n> >> +     git diff-index HEAD^ -- one | grep -q 5626abf0f72e58d7a153368ba57db4c673c0e171\n> >> +'\n> >> +\n> >> +# TODO ced_uptodate()\n> >\n> > What is this about?\n> \n> It tests \"if (ce_uptodate(ce) || (ce->ce_flags & CE_VALID))\" and I was \n> pretty sure it hit ce_uptodate() first, so the second expression was not \n> tested.\n\nAh.  I was distracted by the \"d\" before the underscore.\n\nce_uptodate(ce) checks the time stamp amongst other things, right?  How \nabout using test-chmtime to force an mtime mismatch?\n\nCiao,\nDscho\n"},{"id":"120237","messageId":"alpine.DEB.1.00.0908110846050.4638@intel-tinevez-2-302","threadId":"20522","inReplyTo":"7vhbwforvk.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC PATCH v2 4/4] read-tree: add --no-sparse to turn off sparse hook","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-08-11T06:50:09Z","receivedAt":"2009-08-11T06:50:09Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 10 Aug 2009, Junio C Hamano wrote:\n\n> Nguyen Thai Ngoc Duy <pclouds@gmail.com> writes:\n> \n> >> Hmm.  I understand that the assumption is that memset(&opts, 0,\n> >> sizeof(opts)); should give you a sensible default, but I cannot avoid\n> >> noticing that \"no_sparse_hook = 0\" is a double negation, something to be\n> >> avoided...\n> >\n> > skip_sparse_hook then? :-)\n\nIt nicely avoids the double negation, indeed.\n\n> Why not making the hook to be skipped by default, and pass an explicit \n> option to trigger the hook?\n> \n> I like Dscho's other suggestion to use patterns like .gitignore instead \n> of using hook scripts that needs to be ported across platforms, by the \n> way.\n\nI forgot to mention that I checked dir.c to verify that \nadd_excludes_from_file_1() (which is static, so you'll either need to use \nit in the same file, or even better, export it renaming it to something \nlike add_excludes_from_file_to_list()) and excluded_1() (same here) \nalready do a large part of what you need.\n\nCiao,\nDscho\n"},{"id":"120243","messageId":"alpine.DEB.1.00.0908110850470.4638@intel-tinevez-2-302","threadId":"20522","inReplyTo":"fcaeb9bf0908101847w7b3b22d9w84514a29ceae719e@mail.gmail.com","subject":"Re: [RFC PATCH v2 3/4] unpack_trees(): add support for sparse checkout","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-08-11T07:02:06Z","receivedAt":"2009-08-11T07:02:06Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 11 Aug 2009, Nguyen Thai Ngoc Duy wrote:\n\n> 2009/8/10 Johannes Schindelin <Johannes.Schindelin@gmx.de>:\n>\n> > On Mon, 10 Aug 2009, Nguyễn Thái Ngọc Duy wrote:\n> >\n> >> This patch teaches unpack_trees() to checkout/remove entries on \n> >> working directories appropriately when sparse checkout area is \n> >> changed. Hook \"sparse\" is needed to help determine which entry will \n> >> be checked out, which will not be.\n> >>\n> >> When the hook is run, it is prepared with a pseudo index. The hook \n> >> then can use \"git update-index --[no-]assume-unchanged\" to manipulate \n> >> the index. It should not do anything else on the index. Assume \n> >> unchanged information from the index will be used to shape working \n> >> directory.\n> >\n> > If I understand correctly, the purpose of the hook is to allow the \n> > user to mark something as unwanted preemptively, right?\n> >\n> > If that is the sole reason for the hook, would it not be better to add \n> > support for a file .git/info/sparse which has the same syntax as \n> > .git/info/exclude, and which is used to determine if an \n> > added/modified/deleted file is supposed to be in the \"sparse\" area or \n> > not?\n> >\n> > Something like\n> >\n> >        *\n> >        !/Documentation/\n> >\n> > comes to mind.\n> \n> That was what the original series was about (although I used git\n> config instead of .git/info/sparse). The hook has two advantages:\n> \n>  - flexibility: you can set things differently based on branch, for \n>    example, or filter files based on certain file content.\n\nTrue, it is flexible.  The question is: is it not too flexible for its own \ngood?\n\nHaving to introduce a hook means that every user either must trust another \nperson and install the unmodified hook, or write the hook herself for \nevery narrow work tree.\n\nThe branch use case you mentioned is maybe a good example what I mean by \n\"too flexible\":\n\n- it is easy to get confused which parts are narrow or not if the hook can \n  change that depending on, say, the phase of the moon (okay, okay, \n  depending on the branch, but it is still confusing),\n\n- the whole purpose of narrow checkout is to avoid having to check out \n  stuff that does not affect you.  Chances are that 99.99% of all \"narrow\" \n  users do not need to modify which slice of the files they want to see, \n  and those who do are probably better off with two work trees, methinks.\n\n>  - less code bloat (and less intrusive too), compare ~1000 lines of \n>    change of the original series with this series.\n\nHeh, see my other mail.  I'm sorry I forgot to mention that the similarity \nto .gitignore spares you a lot of code...\n\nCiao,\nDscho\n"},{"id":"120305","messageId":"fcaeb9bf0908110008g1cbd3442p180557cbe6f24bb1@mail.gmail.com","threadId":"20522","inReplyTo":"alpine.DEB.1.00.0908110846050.4638@intel-tinevez-2-302","subject":"Re: [RFC PATCH v2 4/4] read-tree: add --no-sparse to turn off sparse hook","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2009-08-11T07:08:59Z","receivedAt":"2009-08-11T07:08:59Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Aug 11, 2009 at 1:50 PM, Johannes\nSchindelin<Johannes.Schindelin@gmx.de> wrote:\n>> Why not making the hook to be skipped by default, and pass an explicit\n>> option to trigger the hook?\n>>\n>> I like Dscho's other suggestion to use patterns like .gitignore instead\n>> of using hook scripts that needs to be ported across platforms, by the\n>> way.\n>\n> I forgot to mention that I checked dir.c to verify that\n> add_excludes_from_file_1() (which is static, so you'll either need to use\n> it in the same file, or even better, export it renaming it to something\n> like add_excludes_from_file_to_list()) and excluded_1() (same here)\n> already do a large part of what you need.\n\nIndeed (and I have that code too). I still like the hook though\nbecause you have more freedom in defining your worktree (and yes, less\nportable). I wanted to make something generic enough that you can\nbuild higher \"strategies\" on top, like .gitignore-based patterns.\nAnyway I'll send out another patch for .gitignore patterns tonight and\nsee how it goes.\n-- \nDuy\n"},{"id":"120234","messageId":"alpine.DEB.1.00.0908111009010.4638@intel-tinevez-2-302","threadId":"20522","inReplyTo":"fcaeb9bf0908101857x7a44d3dfge20e45d24daee9bf@mail.gmail.com","subject":"Re: [RFC PATCH v2 2/4] gitignore: read from index if .gitignore is assume-unchanged","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-08-11T08:12:05Z","receivedAt":"2009-08-11T08:12:05Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 11 Aug 2009, Nguyen Thai Ngoc Duy wrote:\n\n> 2009/8/10 Johannes Schindelin <Johannes.Schindelin@gmx.de>:\n>\n> > On Mon, 10 Aug 2009, Nguyễn Thái Ngọc Duy wrote:\n> >\n> >> @@ -212,27 +237,31 @@ static int add_excludes_from_file_1(const char *fname,\n> >>\n> >> [...]\n> >>\n> >>       if (buf_p)\n> >>               *buf_p = buf;\n> >> -     buf[size++] = '\\n';\n> >>       entry = buf;\n> >> -     for (i = 0; i < size; i++) {\n> >> -             if (buf[i] == '\\n') {\n> >> +     for (i = 0; i <= size; i++) {\n> >> +             if (i == size || buf[i] == '\\n') {\n> >>                       if (entry != buf + i && entry[0] != '#') {\n> >>                               buf[i - (i && buf[i-1] == '\\r')] = 0;\n> >>                               add_exclude(entry, base, baselen, which);\n> >\n> > Should this change not rather be a separate one?\n> \n> You meant a separate patch?\n\nYes, sorry, I meant exactly that.\n\n> It is tied to this patch, because if bus is read from read_index_data, \n> it does not have extra space for '\\n' at the end.\n\nBut you could separate it out as a patch that just avoids writing to buf.  \nIMHO this would make following the patch series easier.\n\n> >> @@ -241,17 +270,12 @@ static int add_excludes_from_file_1(const char *fname,\n> >>               }\n> >>       }\n> >>       return 0;\n> >> -\n> >> - err:\n> >> -     if (0 <= fd)\n> >> -             close(fd);\n> >> -     return -1;\n> >>  }\n> >>\n> >>  void add_excludes_from_file(struct dir_struct *dir, const char *fname)\n> >>  {\n> >>       if (add_excludes_from_file_1(fname, \"\", 0, NULL,\n> >> -                                  &dir->exclude_list[EXC_FILE]) < 0)\n> >> +                                  &dir->exclude_list[EXC_FILE], 0) < 0)\n> >\n> > Could you mention in the commit message why this function does not \n> > want to check the index (I _guess_ it is because this code path only \n> > tries to read .git/info/exclude, but it would be better to be sure).\n> \n> To retain old behaviour. But I have to check its callers. Maybe we\n> want to check the index too.\n\nYes, it would be good to illustrate in the commit message which code paths \nwant to check the index, and why.\n\n> >> @@ -85,6 +85,26 @@ test_expect_success \\\n> >>         >output &&\n> >>       test_cmp expect output'\n> >>\n> >> +test_expect_success 'setup sparse gitignore' '\n> >> +     git add .gitignore one/.gitignore one/two/.gitignore &&\n> >> +     git update-index --assume-unchanged .gitignore one/.gitignore one/two/.gitignore &&\n> >> +     rm .gitignore one/.gitignore one/two/.gitignore\n> >> +'\n> >\n> > You're probably less sloppy than me; I'd have defined a variable like\n> > this:\n> >\n> >        ignores=\".gitignore one/.gitignore one/two/.gitignore\"\n> >\n> > and used it for the three calls, just to make sure that I do not fsck\n> > anything up there due to fat fingers.\n> \n> I have slim ones :-)\n\nOh, don't get me wrong: my fingers are slim enough to play piano, and even \nto write pretty fast on my EeePC 701.\n\nBut darn, there are days when they feel as if they were thick like a \nbrick.\n\nCiao,\nDscho\n"}]}