{"thread":{"id":"59297","subject":"[PATCH 00/11] Clarify API for dir.[ch] and unpack-trees.[ch] -- mark relevant fields as internal","startedAt":"2023-02-23T09:15:07Z","lastAt":"2023-02-27T23:21:28Z","messageCount":57,"participants":["Elijah Newren via GitGitGadget","Derrick Stolee","Elijah Newren","Junio C Hamano","Jacob Keller","Jonathan Tan"],"isPatch":true,"patchVersion":1,"patchTotal":11},"messages":[{"id":"472508","messageId":"pull.1149.git.1677143700.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":null,"subject":"[PATCH 00/11] Clarify API for dir.[ch] and unpack-trees.[ch] -- mark relevant fields as internal","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-23T09:14:49Z","receivedAt":"2023-02-23T09:15:07Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"I wrote this patch series about a year and a half ago, but never submitted\nit. I rebased and updated it due to [0].\n\nSome time ago, I noticed that struct dir_struct and struct\nunpack_trees_options both have numerous fields meant for internal use only,\nmost of which are not marked as such. This has resulted in callers\naccidentally trying to initialize some of these fields, and in at least one\ncase required a fair amount of review to verify other changes were okay --\nreview that would have been simplified with the apriori knowledge that a\ncombination of multiple fields were internal-only[1]. Looking closer, I\nfound that only 6 out of 18 fields in dir_struct were actually meant to be\npublic[2], and noted that unpack_trees_options also had 11 internal-only\nfields (out of 36).\n\nThis patch is primarily about moving internal-only fields within these two\nstructs into an embedded internal struct. Patch breakdown:\n\n * Patches 1-3: Restructuring dir_struct\n   * Patch 1: Splitting off internal-use-only fields\n   * Patch 2: Add important usage note to avoid accidentally using\n     deprecated API\n   * Patch 3: Mark output-only fields as such\n * Patches 4-11: Restructuring unpack_trees_options\n   * Patches 4-6: Preparatory cleanup\n   * Patches 7-10: Splitting off internal-use-only fields\n   * Patch 11: Mark output-only field as such\n\nTo make the benefit more clear, here are compressed versions of dir_struct\nboth before and after the changes. First, before:\n\nstruct dir_struct {\n    int nr;\n    int alloc;\n    int ignored_nr;\n    int ignored_alloc;\n    enum [...] flags;\n    struct dir_entry **entries;\n    struct dir_entry **ignored;\n    const char *exclude_per_dir;\n#define EXC_CMDL 0\n#define EXC_DIRS 1\n#define EXC_FILE 2\n    struct exclude_list_group exclude_list_group[3];\n    struct exclude_stack *exclude_stack;\n    struct path_pattern *pattern;\n    struct strbuf basebuf;\n    struct untracked_cache *untracked;\n    struct oid_stat ss_info_exclude;\n    struct oid_stat ss_excludes_file;\n    unsigned unmanaged_exclude_files;\n    unsigned visited_paths;\n    unsigned visited_directories;\n};\n\n\nAnd after the changes:\n\nstruct dir_struct {\n    enum [...] flags;\n    int nr; /* output only */\n    int ignored_nr; /* output only */\n    struct dir_entry **entries; /* output only */\n    struct dir_entry **ignored; /* output only */\n    struct untracked_cache *untracked;\n    const char *exclude_per_dir; /* deprecated */\n    struct dir_struct_internal {\n        int alloc;\n        int ignored_alloc;\n#define EXC_CMDL 0\n#define EXC_DIRS 1\n#define EXC_FILE 2\n        struct exclude_list_group exclude_list_group[3];\n        struct exclude_stack *exclude_stack;\n        struct path_pattern *pattern;\n        struct strbuf basebuf;\n        struct oid_stat ss_info_exclude;\n        struct oid_stat ss_excludes_file;\n        unsigned unmanaged_exclude_files;\n        unsigned visited_paths;\n        unsigned visited_directories;\n    } internal;\n};\n\n\nThe former version has 18 fields (and 3 magic constants) which API users\nwill have to figure out. The latter makes it clear there are only at most 2\nfields you should be setting upon input, and at most 4 which you read at\noutput, and the rest (including all the magic constants) you can ignore.\n\n[0] Search for \"Extremely yes\" in\nhttps://lore.kernel.org/git/CAJoAoZm+TkCL0Jpg_qFgKottxbtiG2QOiY0qGrz3-uQy+=waPg@mail.gmail.com/\n[1]\nhttps://lore.kernel.org/git/CABPp-BFSFN3WM6q7KzkD5mhrwsz--St_-ej5LbaY8Yr2sZzj=w@mail.gmail.com/\n[2]\nhttps://lore.kernel.org/git/CABPp-BHgot=CPNyK_xNfog_SqsNPNoCGfiSb-gZoS2sn_741dQ@mail.gmail.com/\n\nElijah Newren (11):\n  dir: separate public from internal portion of dir_struct\n  dir: add a usage note to exclude_per_dir\n  dir: mark output only fields of dir_struct as such\n  unpack-trees: clean up some flow control\n  sparse-checkout: avoid using internal API of unpack-trees\n  sparse-checkout: avoid using internal API of unpack-trees, take 2\n  unpack_trees: start splitting internal fields from public API\n  unpack-trees: mark fields only used internally as internal\n  unpack-trees: rewrap a few overlong lines from previous patch\n  unpack-trees: special case read-tree debugging as internal usage\n  unpack-trees: add usage notices around df_conflict_entry\n\n builtin/read-tree.c       |  10 +-\n builtin/sparse-checkout.c |   4 +-\n dir.c                     | 114 +++++++++---------\n dir.h                     | 110 +++++++++--------\n unpack-trees.c            | 245 ++++++++++++++++++++------------------\n unpack-trees.h            |  42 ++++---\n 6 files changed, 275 insertions(+), 250 deletions(-)\n\n\nbase-commit: 06dd2baa8da4a73421b959ec026a43711b9d77f9\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1149%2Fnewren%2Fclarify-api-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1149/newren/clarify-api-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/1149\n-- \ngitgitgadget\n"},{"id":"472509","messageId":"7f59ad548d0cbeeffb3917765084418df520ec22.1677143700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.git.1677143700.gitgitgadget@gmail.com","subject":"[PATCH 01/11] dir: separate public from internal portion of dir_struct","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-23T09:14:50Z","receivedAt":"2023-02-23T09:15:08Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nIn order to make it clearer to callers what portions of dir_struct are\npublic API, and avoid errors from them setting fields that are meant as\ninternal API, split the fields used for internal implementation reasons\ninto a separate embedded struct.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n dir.c | 114 +++++++++++++++++++++++++++++-----------------------------\n dir.h |  86 +++++++++++++++++++++++---------------------\n 2 files changed, 104 insertions(+), 96 deletions(-)\n\ndiff --git a/dir.c b/dir.c\nindex 4e99f0c868f..7adf242026e 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -1190,7 +1190,7 @@ struct pattern_list *add_pattern_list(struct dir_struct *dir,\n \tstruct pattern_list *pl;\n \tstruct exclude_list_group *group;\n \n-\tgroup = &dir->exclude_list_group[group_type];\n+\tgroup = &dir->internal.exclude_list_group[group_type];\n \tALLOC_GROW(group->pl, group->nr + 1, group->alloc);\n \tpl = &group->pl[group->nr++];\n \tmemset(pl, 0, sizeof(*pl));\n@@ -1211,7 +1211,7 @@ static void add_patterns_from_file_1(struct dir_struct *dir, const char *fname,\n \t * differently when dir->untracked is non-NULL.\n \t */\n \tif (!dir->untracked)\n-\t\tdir->unmanaged_exclude_files++;\n+\t\tdir->internal.unmanaged_exclude_files++;\n \tpl = add_pattern_list(dir, EXC_FILE, fname);\n \tif (add_patterns(fname, \"\", 0, pl, NULL, 0, oid_stat) < 0)\n \t\tdie(_(\"cannot use %s as an exclude file\"), fname);\n@@ -1219,7 +1219,7 @@ static void add_patterns_from_file_1(struct dir_struct *dir, const char *fname,\n \n void add_patterns_from_file(struct dir_struct *dir, const char *fname)\n {\n-\tdir->unmanaged_exclude_files++; /* see validate_untracked_cache() */\n+\tdir->internal.unmanaged_exclude_files++; /* see validate_untracked_cache() */\n \tadd_patterns_from_file_1(dir, fname, NULL);\n }\n \n@@ -1519,7 +1519,7 @@ static struct path_pattern *last_matching_pattern_from_lists(\n \tstruct exclude_list_group *group;\n \tstruct path_pattern *pattern;\n \tfor (i = EXC_CMDL; i <= EXC_FILE; i++) {\n-\t\tgroup = &dir->exclude_list_group[i];\n+\t\tgroup = &dir->internal.exclude_list_group[i];\n \t\tfor (j = group->nr - 1; j >= 0; j--) {\n \t\t\tpattern = last_matching_pattern_from_list(\n \t\t\t\tpathname, pathlen, basename, dtype_p,\n@@ -1545,20 +1545,20 @@ static void prep_exclude(struct dir_struct *dir,\n \tstruct untracked_cache_dir *untracked;\n \tint current;\n \n-\tgroup = &dir->exclude_list_group[EXC_DIRS];\n+\tgroup = &dir->internal.exclude_list_group[EXC_DIRS];\n \n \t/*\n \t * Pop the exclude lists from the EXCL_DIRS exclude_list_group\n \t * which originate from directories not in the prefix of the\n \t * path being checked.\n \t */\n-\twhile ((stk = dir->exclude_stack) != NULL) {\n+\twhile ((stk = dir->internal.exclude_stack) != NULL) {\n \t\tif (stk->baselen <= baselen &&\n-\t\t    !strncmp(dir->basebuf.buf, base, stk->baselen))\n+\t\t    !strncmp(dir->internal.basebuf.buf, base, stk->baselen))\n \t\t\tbreak;\n-\t\tpl = &group->pl[dir->exclude_stack->exclude_ix];\n-\t\tdir->exclude_stack = stk->prev;\n-\t\tdir->pattern = NULL;\n+\t\tpl = &group->pl[dir->internal.exclude_stack->exclude_ix];\n+\t\tdir->internal.exclude_stack = stk->prev;\n+\t\tdir->internal.pattern = NULL;\n \t\tfree((char *)pl->src); /* see strbuf_detach() below */\n \t\tclear_pattern_list(pl);\n \t\tfree(stk);\n@@ -1566,7 +1566,7 @@ static void prep_exclude(struct dir_struct *dir,\n \t}\n \n \t/* Skip traversing into sub directories if the parent is excluded */\n-\tif (dir->pattern)\n+\tif (dir->internal.pattern)\n \t\treturn;\n \n \t/*\n@@ -1574,12 +1574,12 @@ static void prep_exclude(struct dir_struct *dir,\n \t * memset(dir, 0, sizeof(*dir)) before use. Changing all of\n \t * them seems lots of work for little benefit.\n \t */\n-\tif (!dir->basebuf.buf)\n-\t\tstrbuf_init(&dir->basebuf, PATH_MAX);\n+\tif (!dir->internal.basebuf.buf)\n+\t\tstrbuf_init(&dir->internal.basebuf, PATH_MAX);\n \n \t/* Read from the parent directories and push them down. */\n \tcurrent = stk ? stk->baselen : -1;\n-\tstrbuf_setlen(&dir->basebuf, current < 0 ? 0 : current);\n+\tstrbuf_setlen(&dir->internal.basebuf, current < 0 ? 0 : current);\n \tif (dir->untracked)\n \t\tuntracked = stk ? stk->ucd : dir->untracked->root;\n \telse\n@@ -1599,32 +1599,33 @@ static void prep_exclude(struct dir_struct *dir,\n \t\t\t\tdie(\"oops in prep_exclude\");\n \t\t\tcp++;\n \t\t\tuntracked =\n-\t\t\t\tlookup_untracked(dir->untracked, untracked,\n+\t\t\t\tlookup_untracked(dir->untracked,\n+\t\t\t\t\t\t untracked,\n \t\t\t\t\t\t base + current,\n \t\t\t\t\t\t cp - base - current);\n \t\t}\n-\t\tstk->prev = dir->exclude_stack;\n+\t\tstk->prev = dir->internal.exclude_stack;\n \t\tstk->baselen = cp - base;\n \t\tstk->exclude_ix = group->nr;\n \t\tstk->ucd = untracked;\n \t\tpl = add_pattern_list(dir, EXC_DIRS, NULL);\n-\t\tstrbuf_add(&dir->basebuf, base + current, stk->baselen - current);\n-\t\tassert(stk->baselen == dir->basebuf.len);\n+\t\tstrbuf_add(&dir->internal.basebuf, base + current, stk->baselen - current);\n+\t\tassert(stk->baselen == dir->internal.basebuf.len);\n \n \t\t/* Abort if the directory is excluded */\n \t\tif (stk->baselen) {\n \t\t\tint dt = DT_DIR;\n-\t\t\tdir->basebuf.buf[stk->baselen - 1] = 0;\n-\t\t\tdir->pattern = last_matching_pattern_from_lists(dir,\n+\t\t\tdir->internal.basebuf.buf[stk->baselen - 1] = 0;\n+\t\t\tdir->internal.pattern = last_matching_pattern_from_lists(dir,\n \t\t\t\t\t\t\t\t\tistate,\n-\t\t\t\tdir->basebuf.buf, stk->baselen - 1,\n-\t\t\t\tdir->basebuf.buf + current, &dt);\n-\t\t\tdir->basebuf.buf[stk->baselen - 1] = '/';\n-\t\t\tif (dir->pattern &&\n-\t\t\t    dir->pattern->flags & PATTERN_FLAG_NEGATIVE)\n-\t\t\t\tdir->pattern = NULL;\n-\t\t\tif (dir->pattern) {\n-\t\t\t\tdir->exclude_stack = stk;\n+\t\t\t\tdir->internal.basebuf.buf, stk->baselen - 1,\n+\t\t\t\tdir->internal.basebuf.buf + current, &dt);\n+\t\t\tdir->internal.basebuf.buf[stk->baselen - 1] = '/';\n+\t\t\tif (dir->internal.pattern &&\n+\t\t\t    dir->internal.pattern->flags & PATTERN_FLAG_NEGATIVE)\n+\t\t\t\tdir->internal.pattern = NULL;\n+\t\t\tif (dir->internal.pattern) {\n+\t\t\t\tdir->internal.exclude_stack = stk;\n \t\t\t\treturn;\n \t\t\t}\n \t\t}\n@@ -1647,15 +1648,15 @@ static void prep_exclude(struct dir_struct *dir,\n \t\t      */\n \t\t     !is_null_oid(&untracked->exclude_oid))) {\n \t\t\t/*\n-\t\t\t * dir->basebuf gets reused by the traversal, but we\n-\t\t\t * need fname to remain unchanged to ensure the src\n-\t\t\t * member of each struct path_pattern correctly\n+\t\t\t * dir->internal.basebuf gets reused by the traversal,\n+\t\t\t * but we need fname to remain unchanged to ensure the\n+\t\t\t * src member of each struct path_pattern correctly\n \t\t\t * back-references its source file.  Other invocations\n \t\t\t * of add_pattern_list provide stable strings, so we\n \t\t\t * strbuf_detach() and free() here in the caller.\n \t\t\t */\n \t\t\tstruct strbuf sb = STRBUF_INIT;\n-\t\t\tstrbuf_addbuf(&sb, &dir->basebuf);\n+\t\t\tstrbuf_addbuf(&sb, &dir->internal.basebuf);\n \t\t\tstrbuf_addstr(&sb, dir->exclude_per_dir);\n \t\t\tpl->src = strbuf_detach(&sb, NULL);\n \t\t\tadd_patterns(pl->src, pl->src, stk->baselen, pl, istate,\n@@ -1681,10 +1682,10 @@ static void prep_exclude(struct dir_struct *dir,\n \t\t\tinvalidate_gitignore(dir->untracked, untracked);\n \t\t\toidcpy(&untracked->exclude_oid, &oid_stat.oid);\n \t\t}\n-\t\tdir->exclude_stack = stk;\n+\t\tdir->internal.exclude_stack = stk;\n \t\tcurrent = stk->baselen;\n \t}\n-\tstrbuf_setlen(&dir->basebuf, baselen);\n+\tstrbuf_setlen(&dir->internal.basebuf, baselen);\n }\n \n /*\n@@ -1704,8 +1705,8 @@ struct path_pattern *last_matching_pattern(struct dir_struct *dir,\n \n \tprep_exclude(dir, istate, pathname, basename-pathname);\n \n-\tif (dir->pattern)\n-\t\treturn dir->pattern;\n+\tif (dir->internal.pattern)\n+\t\treturn dir->internal.pattern;\n \n \treturn last_matching_pattern_from_lists(dir, istate, pathname, pathlen,\n \t\t\tbasename, dtype_p);\n@@ -1742,7 +1743,7 @@ static struct dir_entry *dir_add_name(struct dir_struct *dir,\n \tif (index_file_exists(istate, pathname, len, ignore_case))\n \t\treturn NULL;\n \n-\tALLOC_GROW(dir->entries, dir->nr+1, dir->alloc);\n+\tALLOC_GROW(dir->entries, dir->nr+1, dir->internal.alloc);\n \treturn dir->entries[dir->nr++] = dir_entry_new(pathname, len);\n }\n \n@@ -1753,7 +1754,7 @@ struct dir_entry *dir_add_ignored(struct dir_struct *dir,\n \tif (!index_name_is_other(istate, pathname, len))\n \t\treturn NULL;\n \n-\tALLOC_GROW(dir->ignored, dir->ignored_nr+1, dir->ignored_alloc);\n+\tALLOC_GROW(dir->ignored, dir->ignored_nr+1, dir->internal.ignored_alloc);\n \treturn dir->ignored[dir->ignored_nr++] = dir_entry_new(pathname, len);\n }\n \n@@ -2569,7 +2570,7 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,\n \n \tif (open_cached_dir(&cdir, dir, untracked, istate, &path, check_only))\n \t\tgoto out;\n-\tdir->visited_directories++;\n+\tdir->internal.visited_directories++;\n \n \tif (untracked)\n \t\tuntracked->check_only = !!check_only;\n@@ -2578,7 +2579,7 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,\n \t\t/* check how the file or directory should be treated */\n \t\tstate = treat_path(dir, untracked, &cdir, istate, &path,\n \t\t\t\t   baselen, pathspec);\n-\t\tdir->visited_paths++;\n+\t\tdir->internal.visited_paths++;\n \n \t\tif (state > dir_state)\n \t\t\tdir_state = state;\n@@ -2586,7 +2587,8 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,\n \t\t/* recurse into subdir if instructed by treat_path */\n \t\tif (state == path_recurse) {\n \t\t\tstruct untracked_cache_dir *ud;\n-\t\t\tud = lookup_untracked(dir->untracked, untracked,\n+\t\t\tud = lookup_untracked(dir->untracked,\n+\t\t\t\t\t      untracked,\n \t\t\t\t\t      path.buf + baselen,\n \t\t\t\t\t      path.len - baselen);\n \t\t\tsubdir_state =\n@@ -2846,7 +2848,7 @@ static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct *d\n \t * condition also catches running setup_standard_excludes()\n \t * before setting dir->untracked!\n \t */\n-\tif (dir->unmanaged_exclude_files)\n+\tif (dir->internal.unmanaged_exclude_files)\n \t\treturn NULL;\n \n \t/*\n@@ -2875,7 +2877,7 @@ static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct *d\n \t * EXC_CMDL is not considered in the cache. If people set it,\n \t * skip the cache.\n \t */\n-\tif (dir->exclude_list_group[EXC_CMDL].nr)\n+\tif (dir->internal.exclude_list_group[EXC_CMDL].nr)\n \t\treturn NULL;\n \n \tif (!ident_in_untracked(dir->untracked)) {\n@@ -2935,15 +2937,15 @@ static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct *d\n \n \t/* Validate $GIT_DIR/info/exclude and core.excludesfile */\n \troot = dir->untracked->root;\n-\tif (!oideq(&dir->ss_info_exclude.oid,\n+\tif (!oideq(&dir->internal.ss_info_exclude.oid,\n \t\t   &dir->untracked->ss_info_exclude.oid)) {\n \t\tinvalidate_gitignore(dir->untracked, root);\n-\t\tdir->untracked->ss_info_exclude = dir->ss_info_exclude;\n+\t\tdir->untracked->ss_info_exclude = dir->internal.ss_info_exclude;\n \t}\n-\tif (!oideq(&dir->ss_excludes_file.oid,\n+\tif (!oideq(&dir->internal.ss_excludes_file.oid,\n \t\t   &dir->untracked->ss_excludes_file.oid)) {\n \t\tinvalidate_gitignore(dir->untracked, root);\n-\t\tdir->untracked->ss_excludes_file = dir->ss_excludes_file;\n+\t\tdir->untracked->ss_excludes_file = dir->internal.ss_excludes_file;\n \t}\n \n \t/* Make sure this directory is not dropped out at saving phase */\n@@ -2969,9 +2971,9 @@ static void emit_traversal_statistics(struct dir_struct *dir,\n \t}\n \n \ttrace2_data_intmax(\"read_directory\", repo,\n-\t\t\t   \"directories-visited\", dir->visited_directories);\n+\t\t\t   \"directories-visited\", dir->internal.visited_directories);\n \ttrace2_data_intmax(\"read_directory\", repo,\n-\t\t\t   \"paths-visited\", dir->visited_paths);\n+\t\t\t   \"paths-visited\", dir->internal.visited_paths);\n \n \tif (!dir->untracked)\n \t\treturn;\n@@ -2993,8 +2995,8 @@ int read_directory(struct dir_struct *dir, struct index_state *istate,\n \tstruct untracked_cache_dir *untracked;\n \n \ttrace2_region_enter(\"dir\", \"read_directory\", istate->repo);\n-\tdir->visited_paths = 0;\n-\tdir->visited_directories = 0;\n+\tdir->internal.visited_paths = 0;\n+\tdir->internal.visited_directories = 0;\n \n \tif (has_symlink_leading_path(path, len)) {\n \t\ttrace2_region_leave(\"dir\", \"read_directory\", istate->repo);\n@@ -3342,14 +3344,14 @@ void setup_standard_excludes(struct dir_struct *dir)\n \t\texcludes_file = xdg_config_home(\"ignore\");\n \tif (excludes_file && !access_or_warn(excludes_file, R_OK, 0))\n \t\tadd_patterns_from_file_1(dir, excludes_file,\n-\t\t\t\t\t dir->untracked ? &dir->ss_excludes_file : NULL);\n+\t\t\t\t\t dir->untracked ? &dir->internal.ss_excludes_file : NULL);\n \n \t/* per repository user preference */\n \tif (startup_info->have_repository) {\n \t\tconst char *path = git_path_info_exclude();\n \t\tif (!access_or_warn(path, R_OK, 0))\n \t\t\tadd_patterns_from_file_1(dir, path,\n-\t\t\t\t\t\t dir->untracked ? &dir->ss_info_exclude : NULL);\n+\t\t\t\t\t\t dir->untracked ? &dir->internal.ss_info_exclude : NULL);\n \t}\n }\n \n@@ -3405,7 +3407,7 @@ void dir_clear(struct dir_struct *dir)\n \tstruct dir_struct new = DIR_INIT;\n \n \tfor (i = EXC_CMDL; i <= EXC_FILE; i++) {\n-\t\tgroup = &dir->exclude_list_group[i];\n+\t\tgroup = &dir->internal.exclude_list_group[i];\n \t\tfor (j = 0; j < group->nr; j++) {\n \t\t\tpl = &group->pl[j];\n \t\t\tif (i == EXC_DIRS)\n@@ -3422,13 +3424,13 @@ void dir_clear(struct dir_struct *dir)\n \tfree(dir->ignored);\n \tfree(dir->entries);\n \n-\tstk = dir->exclude_stack;\n+\tstk = dir->internal.exclude_stack;\n \twhile (stk) {\n \t\tstruct exclude_stack *prev = stk->prev;\n \t\tfree(stk);\n \t\tstk = prev;\n \t}\n-\tstrbuf_release(&dir->basebuf);\n+\tstrbuf_release(&dir->internal.basebuf);\n \n \tmemcpy(dir, &new, sizeof(*dir));\n }\ndiff --git a/dir.h b/dir.h\nindex 8acfc044181..33fd848fc8d 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -215,14 +215,9 @@ struct dir_struct {\n \t/* The number of members in `entries[]` array. */\n \tint nr;\n \n-\t/* Internal use; keeps track of allocation of `entries[]` array.*/\n-\tint alloc;\n-\n \t/* The number of members in `ignored[]` array. */\n \tint ignored_nr;\n \n-\tint ignored_alloc;\n-\n \t/* bit-field of options */\n \tenum {\n \n@@ -296,51 +291,62 @@ struct dir_struct {\n \t */\n \tstruct dir_entry **ignored;\n \n+\t/* Enable/update untracked file cache if set */\n+\tstruct untracked_cache *untracked;\n+\n \t/**\n \t * The name of the file to be read in each directory for excluded files\n \t * (typically `.gitignore`).\n \t */\n \tconst char *exclude_per_dir;\n \n-\t/*\n-\t * We maintain three groups of exclude pattern lists:\n-\t *\n-\t * EXC_CMDL lists patterns explicitly given on the command line.\n-\t * EXC_DIRS lists patterns obtained from per-directory ignore files.\n-\t * EXC_FILE lists patterns from fallback ignore files, e.g.\n-\t *   - .git/info/exclude\n-\t *   - core.excludesfile\n-\t *\n-\t * Each group contains multiple exclude lists, a single list\n-\t * per source.\n-\t */\n+\tstruct dir_struct_internal {\n+\t\t/* Keeps track of allocation of `entries[]` array.*/\n+\t\tint alloc;\n+\n+\t\t/* Keeps track of allocation of `ignored[]` array. */\n+\t\tint ignored_alloc;\n+\n+\t\t/*\n+\t\t * We maintain three groups of exclude pattern lists:\n+\t\t *\n+\t\t * EXC_CMDL lists patterns explicitly given on the command line.\n+\t\t * EXC_DIRS lists patterns obtained from per-directory ignore\n+\t\t *          files.\n+\t\t * EXC_FILE lists patterns from fallback ignore files, e.g.\n+\t\t *   - .git/info/exclude\n+\t\t *   - core.excludesfile\n+\t\t *\n+\t\t * Each group contains multiple exclude lists, a single list\n+\t\t * per source.\n+\t\t */\n #define EXC_CMDL 0\n #define EXC_DIRS 1\n #define EXC_FILE 2\n-\tstruct exclude_list_group exclude_list_group[3];\n-\n-\t/*\n-\t * Temporary variables which are used during loading of the\n-\t * per-directory exclude lists.\n-\t *\n-\t * exclude_stack points to the top of the exclude_stack, and\n-\t * basebuf contains the full path to the current\n-\t * (sub)directory in the traversal. Exclude points to the\n-\t * matching exclude struct if the directory is excluded.\n-\t */\n-\tstruct exclude_stack *exclude_stack;\n-\tstruct path_pattern *pattern;\n-\tstruct strbuf basebuf;\n-\n-\t/* Enable untracked file cache if set */\n-\tstruct untracked_cache *untracked;\n-\tstruct oid_stat ss_info_exclude;\n-\tstruct oid_stat ss_excludes_file;\n-\tunsigned unmanaged_exclude_files;\n+\t\tstruct exclude_list_group exclude_list_group[3];\n \n-\t/* Stats about the traversal */\n-\tunsigned visited_paths;\n-\tunsigned visited_directories;\n+\t\t/*\n+\t\t * Temporary variables which are used during loading of the\n+\t\t * per-directory exclude lists.\n+\t\t *\n+\t\t * exclude_stack points to the top of the exclude_stack, and\n+\t\t * basebuf contains the full path to the current\n+\t\t * (sub)directory in the traversal. Exclude points to the\n+\t\t * matching exclude struct if the directory is excluded.\n+\t\t */\n+\t\tstruct exclude_stack *exclude_stack;\n+\t\tstruct path_pattern *pattern;\n+\t\tstruct strbuf basebuf;\n+\n+\t\t/* Additional metadata related to 'untracked' */\n+\t\tstruct oid_stat ss_info_exclude;\n+\t\tstruct oid_stat ss_excludes_file;\n+\t\tunsigned unmanaged_exclude_files;\n+\n+\t\t/* Stats about the traversal */\n+\t\tunsigned visited_paths;\n+\t\tunsigned visited_directories;\n+\t} internal;\n };\n \n #define DIR_INIT { 0 }\n-- \ngitgitgadget\n\n"},{"id":"472510","messageId":"239b10e11812d99be587265c0a5e283da45ca315.1677143700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.git.1677143700.gitgitgadget@gmail.com","subject":"[PATCH 02/11] dir: add a usage note to exclude_per_dir","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-23T09:14:51Z","receivedAt":"2023-02-23T09:15:10Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n dir.h | 8 ++++++--\n 1 file changed, 6 insertions(+), 2 deletions(-)\n\ndiff --git a/dir.h b/dir.h\nindex 33fd848fc8d..2196e12630c 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -295,8 +295,12 @@ struct dir_struct {\n \tstruct untracked_cache *untracked;\n \n \t/**\n-\t * The name of the file to be read in each directory for excluded files\n-\t * (typically `.gitignore`).\n+\t * Deprecated: ls-files is the only allowed caller; all other callers\n+\t * should leave this as NULL; it pre-dated the\n+\t * setup_standard_excludes() mechanism that replaces this.\n+\t *\n+\t * This field tracks the name of the file to be read in each directory\n+\t * for excluded files (typically `.gitignore`).\n \t */\n \tconst char *exclude_per_dir;\n \n-- \ngitgitgadget\n\n"},{"id":"472511","messageId":"b8aa14350d3411f63d6f02c2e2a857fd342cd8ff.1677143700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.git.1677143700.gitgitgadget@gmail.com","subject":"[PATCH 03/11] dir: mark output only fields of dir_struct as such","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-23T09:14:52Z","receivedAt":"2023-02-23T09:15:11Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nWhile at it, also group these fields together for convenience.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n dir.h | 16 ++++++++--------\n 1 file changed, 8 insertions(+), 8 deletions(-)\n\ndiff --git a/dir.h b/dir.h\nindex 2196e12630c..e8106e1ecac 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -212,12 +212,6 @@ struct untracked_cache {\n  */\n struct dir_struct {\n \n-\t/* The number of members in `entries[]` array. */\n-\tint nr;\n-\n-\t/* The number of members in `ignored[]` array. */\n-\tint ignored_nr;\n-\n \t/* bit-field of options */\n \tenum {\n \n@@ -282,14 +276,20 @@ struct dir_struct {\n \t\tDIR_SKIP_NESTED_GIT = 1<<9\n \t} flags;\n \n+\t/* The number of members in `entries[]` array. */\n+\tint nr; /* output only */\n+\n+\t/* The number of members in `ignored[]` array. */\n+\tint ignored_nr; /* output only */\n+\n \t/* An array of `struct dir_entry`, each element of which describes a path. */\n-\tstruct dir_entry **entries;\n+\tstruct dir_entry **entries; /* output only */\n \n \t/**\n \t * used for ignored paths with the `DIR_SHOW_IGNORED_TOO` and\n \t * `DIR_COLLECT_IGNORED` flags.\n \t */\n-\tstruct dir_entry **ignored;\n+\tstruct dir_entry **ignored; /* output only */\n \n \t/* Enable/update untracked file cache if set */\n \tstruct untracked_cache *untracked;\n-- \ngitgitgadget\n\n"},{"id":"472512","messageId":"f5a58123034a708b49aac7076197ad2ccdd3cba1.1677143700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.git.1677143700.gitgitgadget@gmail.com","subject":"[PATCH 04/11] unpack-trees: clean up some flow control","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-23T09:14:53Z","receivedAt":"2023-02-23T09:15:14Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nThe update_sparsity() function was introduced in commit 7af7a25853\n(\"unpack-trees: add a new update_sparsity() function\", 2020-03-27).\nPrior to that, unpack_trees() was used, but that had a few bugs because\nthe needs of the caller were different, and different enough that\nunpack_trees() could not easily be modified to handle both usecases.\n\nThe implementation detail that update_sparsity() was written by copying\nunpack_trees() and then streamlining it, and then modifying it in the\nneeded ways still shows through in that there are leftover vestiges in\nboth functions that are no longer needed.  Clean them up.  In\nparticular:\n\n  * update_sparsity() allows a pattern list to be passed in, but\n    unpack_trees() never should use a different pattern list.  Add a\n    check and a BUG() if this gets violated.\n  * update_sparsity() has a check early on that will BUG() if\n    o->skip_sparse_checkout is set; as such, there's no need to check\n    for that condition again later in the code.  We can simply remove\n    the check and its corresponding goto label.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 7 +++----\n 1 file changed, 3 insertions(+), 4 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 3d05e45a279..0887d157df4 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1873,6 +1873,8 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tdie(\"unpack_trees takes at most %d trees\", MAX_UNPACK_TREES);\n \tif (o->dir)\n \t\tBUG(\"o->dir is for internal use only\");\n+\tif (o->pl)\n+\t\tBUG(\"o->pl is for internal use only\");\n \n \ttrace_performance_enter();\n \ttrace2_region_enter(\"unpack_trees\", \"unpack_trees\", the_repository);\n@@ -1899,7 +1901,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \n \tif (!core_apply_sparse_checkout || !o->update)\n \t\to->skip_sparse_checkout = 1;\n-\tif (!o->skip_sparse_checkout && !o->pl) {\n+\tif (!o->skip_sparse_checkout) {\n \t\tmemset(&pl, 0, sizeof(pl));\n \t\tfree_pattern_list = 1;\n \t\tpopulate_from_existing_patterns(o, &pl);\n@@ -2113,8 +2115,6 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n \t\tmemset(&pl, 0, sizeof(pl));\n \t\tfree_pattern_list = 1;\n \t\tpopulate_from_existing_patterns(o, &pl);\n-\t\tif (o->skip_sparse_checkout)\n-\t\t\tgoto skip_sparse_checkout;\n \t}\n \n \t/* Expand sparse directories as needed */\n@@ -2142,7 +2142,6 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n \t\t\tret = UPDATE_SPARSITY_WARNINGS;\n \t}\n \n-skip_sparse_checkout:\n \tif (check_updates(o, o->src_index))\n \t\tret = UPDATE_SPARSITY_WORKTREE_UPDATE_FAILURES;\n \n-- \ngitgitgadget\n\n"},{"id":"472513","messageId":"508837fc182b0adba96c3fcbd468f47f8f0a0aef.1677143700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.git.1677143700.gitgitgadget@gmail.com","subject":"[PATCH 05/11] sparse-checkout: avoid using internal API of unpack-trees","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-23T09:14:54Z","receivedAt":"2023-02-23T09:15:16Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nstruct unpack_trees_options has the following field and comment:\n\n\tstruct pattern_list *pl; /* for internal use */\n\nDespite the internal-use comment, commit e091228e17 (\"sparse-checkout:\nupdate working directory in-process\", 2019-11-21) starting setting this\nfield from an external caller.  At the time, the only way around that\nwould have been to modify unpack_trees() to take an extra pattern_list\nargument, and there's a lot of callers of that function.  However, when\nwe split update_sparsity() off as a separate function, with\nsparse-checkout being the sole caller, the need to update other callers\nwent away.  Fix this API problem by adding a pattern_list argument to\nupdate_sparsity() and stop setting the internal o.pl field directly.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n builtin/sparse-checkout.c |  3 +--\n unpack-trees.c            | 17 ++++++++++-------\n unpack-trees.h            |  3 ++-\n 3 files changed, 13 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex c3738154918..4b7390ce367 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -219,14 +219,13 @@ static int update_working_directory(struct pattern_list *pl)\n \to.dst_index = r->index;\n \tindex_state_init(&o.result, r);\n \to.skip_sparse_checkout = 0;\n-\to.pl = pl;\n \n \tsetup_work_tree();\n \n \trepo_hold_locked_index(r, &lock_file, LOCK_DIE_ON_ERROR);\n \n \tsetup_unpack_trees_porcelain(&o, \"sparse-checkout\");\n-\tresult = update_sparsity(&o);\n+\tresult = update_sparsity(&o, pl);\n \tclear_unpack_trees_porcelain(&o);\n \n \tif (result == UPDATE_SPARSITY_WARNINGS)\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 0887d157df4..d9c9f330233 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -2091,10 +2091,10 @@ return_failed:\n  *\n  * CE_NEW_SKIP_WORKTREE is used internally.\n  */\n-enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n+enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n+\t\t\t\t\t    struct pattern_list *pl)\n {\n \tenum update_sparsity_result ret = UPDATE_SPARSITY_SUCCESS;\n-\tstruct pattern_list pl;\n \tint i;\n \tunsigned old_show_all_errors;\n \tint free_pattern_list = 0;\n@@ -2111,11 +2111,12 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n \ttrace_performance_enter();\n \n \t/* If we weren't given patterns, use the recorded ones */\n-\tif (!o->pl) {\n-\t\tmemset(&pl, 0, sizeof(pl));\n+\tif (!pl) {\n \t\tfree_pattern_list = 1;\n-\t\tpopulate_from_existing_patterns(o, &pl);\n+\t\tpl = xcalloc(1, sizeof(*pl));\n+\t\tpopulate_from_existing_patterns(o, pl);\n \t}\n+\to->pl = pl;\n \n \t/* Expand sparse directories as needed */\n \texpand_index(o->src_index, o->pl);\n@@ -2147,8 +2148,10 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n \n \tdisplay_warning_msgs(o);\n \to->show_all_errors = old_show_all_errors;\n-\tif (free_pattern_list)\n-\t\tclear_pattern_list(&pl);\n+\tif (free_pattern_list) {\n+\t\tclear_pattern_list(pl);\n+\t\tfree(pl);\n+\t}\n \ttrace_performance_leave(\"update_sparsity\");\n \treturn ret;\n }\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 3a7b3e5f007..f3a6e4f90ef 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -112,7 +112,8 @@ enum update_sparsity_result {\n \tUPDATE_SPARSITY_WORKTREE_UPDATE_FAILURES = -2\n };\n \n-enum update_sparsity_result update_sparsity(struct unpack_trees_options *options);\n+enum update_sparsity_result update_sparsity(struct unpack_trees_options *options,\n+\t\t\t\t\t    struct pattern_list *pl);\n \n int verify_uptodate(const struct cache_entry *ce,\n \t\t    struct unpack_trees_options *o);\n-- \ngitgitgadget\n\n"},{"id":"472514","messageId":"8955b45e35474e5feb826101423470d0b51e5470.1677143700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.git.1677143700.gitgitgadget@gmail.com","subject":"[PATCH 06/11] sparse-checkout: avoid using internal API of unpack-trees, take 2","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-23T09:14:55Z","receivedAt":"2023-02-23T09:15:18Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nCommit 2f6b1eb794 (\"cache API: add a \"INDEX_STATE_INIT\" macro/function,\nadd release_index()\", 2023-01-12) mistakenly added some initialization\nof a member of unpack_trees_options that was intended to be\ninternal-only.  Further, it served no purpose as it simply duplicated\nthe initialization that unpack-trees.c code was already doing.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n builtin/sparse-checkout.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex 4b7390ce367..8d5ae6f2a60 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -217,7 +217,6 @@ static int update_working_directory(struct pattern_list *pl)\n \to.head_idx = -1;\n \to.src_index = r->index;\n \to.dst_index = r->index;\n-\tindex_state_init(&o.result, r);\n \to.skip_sparse_checkout = 0;\n \n \tsetup_work_tree();\n-- \ngitgitgadget\n\n"},{"id":"472515","messageId":"63ee57478edfd731d4f257777ef5aec4c7a02f63.1677143700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.git.1677143700.gitgitgadget@gmail.com","subject":"[PATCH 07/11] unpack_trees: start splitting internal fields from public API","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-23T09:14:56Z","receivedAt":"2023-02-23T09:15:30Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nThis just splits the two fields already marked as internal-only into a\nseparate internal struct.  Future commits will add more fields that\nwere meant to be internal-only but were not explicitly marked as such\nto the same struct.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 40 ++++++++++++++++++++--------------------\n unpack-trees.h |  7 +++++--\n 2 files changed, 25 insertions(+), 22 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex d9c9f330233..e6b5fb980cb 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1809,7 +1809,7 @@ static void populate_from_existing_patterns(struct unpack_trees_options *o,\n \tif (get_sparse_checkout_patterns(pl) < 0)\n \t\to->skip_sparse_checkout = 1;\n \telse\n-\t\to->pl = pl;\n+\t\to->internal.pl = pl;\n }\n \n static void update_sparsity_for_prefix(const char *prefix,\n@@ -1871,10 +1871,10 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \n \tif (len > MAX_UNPACK_TREES)\n \t\tdie(\"unpack_trees takes at most %d trees\", MAX_UNPACK_TREES);\n-\tif (o->dir)\n-\t\tBUG(\"o->dir is for internal use only\");\n-\tif (o->pl)\n-\t\tBUG(\"o->pl is for internal use only\");\n+\tif (o->internal.dir)\n+\t\tBUG(\"o->internal.dir is for internal use only\");\n+\tif (o->internal.pl)\n+\t\tBUG(\"o->internal.pl is for internal use only\");\n \n \ttrace_performance_enter();\n \ttrace2_region_enter(\"unpack_trees\", \"unpack_trees\", the_repository);\n@@ -1891,9 +1891,9 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tBUG(\"UNPACK_RESET_OVERWRITE_UNTRACKED incompatible with preserved ignored files\");\n \n \tif (!o->preserve_ignored) {\n-\t\to->dir = &dir;\n-\t\to->dir->flags |= DIR_SHOW_IGNORED;\n-\t\tsetup_standard_excludes(o->dir);\n+\t\to->internal.dir = &dir;\n+\t\to->internal.dir->flags |= DIR_SHOW_IGNORED;\n+\t\tsetup_standard_excludes(o->internal.dir);\n \t}\n \n \tif (o->prefix)\n@@ -1943,7 +1943,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t * Sparse checkout loop #1: set NEW_SKIP_WORKTREE on existing entries\n \t */\n \tif (!o->skip_sparse_checkout)\n-\t\tmark_new_skip_worktree(o->pl, o->src_index, 0,\n+\t\tmark_new_skip_worktree(o->internal.pl, o->src_index, 0,\n \t\t\t\t       CE_NEW_SKIP_WORKTREE, o->verbose_update);\n \n \tif (!dfc)\n@@ -2009,7 +2009,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t * If they will have NEW_SKIP_WORKTREE, also set CE_SKIP_WORKTREE\n \t\t * so apply_sparse_checkout() won't attempt to remove it from worktree\n \t\t */\n-\t\tmark_new_skip_worktree(o->pl, &o->result,\n+\t\tmark_new_skip_worktree(o->internal.pl, &o->result,\n \t\t\t\t       CE_ADDED, CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE,\n \t\t\t\t       o->verbose_update);\n \n@@ -2067,9 +2067,9 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n done:\n \tif (free_pattern_list)\n \t\tclear_pattern_list(&pl);\n-\tif (o->dir) {\n-\t\tdir_clear(o->dir);\n-\t\to->dir = NULL;\n+\tif (o->internal.dir) {\n+\t\tdir_clear(o->internal.dir);\n+\t\to->internal.dir = NULL;\n \t}\n \ttrace2_region_leave(\"unpack_trees\", \"unpack_trees\", the_repository);\n \ttrace_performance_leave(\"unpack_trees\");\n@@ -2116,14 +2116,14 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n \t\tpl = xcalloc(1, sizeof(*pl));\n \t\tpopulate_from_existing_patterns(o, pl);\n \t}\n-\to->pl = pl;\n+\to->internal.pl = pl;\n \n \t/* Expand sparse directories as needed */\n-\texpand_index(o->src_index, o->pl);\n+\texpand_index(o->src_index, o->internal.pl);\n \n \t/* Set NEW_SKIP_WORKTREE on existing entries. */\n \tmark_all_ce_unused(o->src_index);\n-\tmark_new_skip_worktree(o->pl, o->src_index, 0,\n+\tmark_new_skip_worktree(o->internal.pl, o->src_index, 0,\n \t\t\t       CE_NEW_SKIP_WORKTREE, o->verbose_update);\n \n \t/* Then loop over entries and update/remove as needed */\n@@ -2338,8 +2338,8 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n \tpathbuf = xstrfmt(\"%.*s/\", namelen, ce->name);\n \n \tmemset(&d, 0, sizeof(d));\n-\tif (o->dir)\n-\t\td.exclude_per_dir = o->dir->exclude_per_dir;\n+\tif (o->internal.dir)\n+\t\td.exclude_per_dir = o->internal.dir->exclude_per_dir;\n \ti = read_directory(&d, o->src_index, pathbuf, namelen+1, NULL);\n \tdir_clear(&d);\n \tfree(pathbuf);\n@@ -2393,8 +2393,8 @@ static int check_ok_to_remove(const char *name, int len, int dtype,\n \tif (ignore_case && icase_exists(o, name, len, st))\n \t\treturn 0;\n \n-\tif (o->dir &&\n-\t    is_excluded(o->dir, o->src_index, name, &dtype))\n+\tif (o->internal.dir &&\n+\t    is_excluded(o->internal.dir, o->src_index, name, &dtype))\n \t\t/*\n \t\t * ce->name is explicitly excluded, so it is Ok to\n \t\t * overwrite it.\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex f3a6e4f90ef..5c1a9314a06 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -97,9 +97,12 @@ struct unpack_trees_options {\n \tstruct index_state *src_index;\n \tstruct index_state result;\n \n-\tstruct pattern_list *pl; /* for internal use */\n-\tstruct dir_struct *dir; /* for internal use only */\n \tstruct checkout_metadata meta;\n+\n+\tstruct unpack_trees_options_internal {\n+\t\tstruct pattern_list *pl;\n+\t\tstruct dir_struct *dir;\n+\t} internal;\n };\n \n int unpack_trees(unsigned n, struct tree_desc *t,\n-- \ngitgitgadget\n\n"},{"id":"472516","messageId":"f492ab27b19748a29cfcda88ea098797c512777a.1677143700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.git.1677143700.gitgitgadget@gmail.com","subject":"[PATCH 09/11] unpack-trees: rewrap a few overlong lines from previous patch","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-23T09:14:58Z","receivedAt":"2023-02-23T09:15:32Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nThe previous patch made many lines a little longer, resulting in four\nbecoming a bit too long.  They were left as-is for the previous patch\nto facilitate reviewers verifying that we were just adding \"internal.\"\nin a bunch of places, but rewrap them now.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 16 +++++++++-------\n 1 file changed, 9 insertions(+), 7 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex d89eb3d8bf0..fa186a27ccc 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1213,8 +1213,8 @@ static int unpack_single_entry(int n, unsigned long mask,\n \t\t * cache entry from the index aware logic.\n \t\t */\n \t\tsrc[i + o->merge] = create_ce_entry(info, names + i, stage,\n-\t\t\t\t\t\t    &o->internal.result, o->merge,\n-\t\t\t\t\t\t    bit & dirmask);\n+\t\t\t\t\t\t    &o->internal.result,\n+\t\t\t\t\t\t    o->merge, bit & dirmask);\n \t}\n \n \tif (o->merge) {\n@@ -1918,14 +1918,15 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t} else if (o->src_index == o->dst_index) {\n \t\t/*\n \t\t * o->dst_index (and thus o->src_index) will be discarded\n-\t\t * and overwritten with o->internal.result at the end of this function,\n-\t\t * so just use src_index's split_index to avoid having to\n-\t\t * create a new one.\n+\t\t * and overwritten with o->internal.result at the end of\n+\t\t * this function, so just use src_index's split_index to\n+\t\t * avoid having to create a new one.\n \t\t */\n \t\to->internal.result.split_index = o->src_index->split_index;\n \t\to->internal.result.split_index->refcount++;\n \t} else {\n-\t\to->internal.result.split_index = init_split_index(&o->internal.result);\n+\t\to->internal.result.split_index =\n+\t\t\tinit_split_index(&o->internal.result);\n \t}\n \toidcpy(&o->internal.result.oid, &o->src_index->oid);\n \to->merge_size = len;\n@@ -2049,7 +2050,8 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tmove_index_extensions(&o->internal.result, o->src_index);\n \t\tif (!ret) {\n \t\t\tif (git_env_bool(\"GIT_TEST_CHECK_CACHE_TREE\", 0))\n-\t\t\t\tcache_tree_verify(the_repository, &o->internal.result);\n+\t\t\t\tcache_tree_verify(the_repository,\n+\t\t\t\t\t\t  &o->internal.result);\n \t\t\tif (!o->skip_cache_tree_update &&\n \t\t\t    !cache_tree_fully_valid(o->internal.result.cache_tree))\n \t\t\t\tcache_tree_update(&o->internal.result,\n-- \ngitgitgadget\n\n"},{"id":"472517","messageId":"a5048ea00b23938a681f35ceffcae08e722e4a54.1677143700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.git.1677143700.gitgitgadget@gmail.com","subject":"[PATCH 10/11] unpack-trees: special case read-tree debugging as internal usage","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-23T09:14:59Z","receivedAt":"2023-02-23T09:15:34Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nbuiltin/read-tree.c has some special functionality explicitly designed\nfor debugging unpack-trees.[ch].  Associated with that is two fields\nthat no other external caller would or should use.  Mark these as\ninternal to unpack-trees, but allow builtin/read-tree to read or write\nthem for this special case.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n builtin/read-tree.c | 10 +++++-----\n unpack-trees.c      | 22 +++++++++++-----------\n unpack-trees.h      |  6 +++---\n 3 files changed, 19 insertions(+), 19 deletions(-)\n\ndiff --git a/builtin/read-tree.c b/builtin/read-tree.c\nindex 3ce75417833..6034408d486 100644\n--- a/builtin/read-tree.c\n+++ b/builtin/read-tree.c\n@@ -87,9 +87,9 @@ static int debug_merge(const struct cache_entry * const *stages,\n {\n \tint i;\n \n-\tprintf(\"* %d-way merge\\n\", o->merge_size);\n+\tprintf(\"* %d-way merge\\n\", o->internal.merge_size);\n \tdebug_stage(\"index\", stages[0], o);\n-\tfor (i = 1; i <= o->merge_size; i++) {\n+\tfor (i = 1; i <= o->internal.merge_size; i++) {\n \t\tchar buf[24];\n \t\txsnprintf(buf, sizeof(buf), \"ent#%d\", i);\n \t\tdebug_stage(buf, stages[i], o);\n@@ -144,7 +144,7 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix)\n \t\tOPT__DRY_RUN(&opts.dry_run, N_(\"don't update the index or the work tree\")),\n \t\tOPT_BOOL(0, \"no-sparse-checkout\", &opts.skip_sparse_checkout,\n \t\t\t N_(\"skip applying sparse checkout filter\")),\n-\t\tOPT_BOOL(0, \"debug-unpack\", &opts.debug_unpack,\n+\t\tOPT_BOOL(0, \"debug-unpack\", &opts.internal.debug_unpack,\n \t\t\t N_(\"debug unpack-trees\")),\n \t\tOPT_CALLBACK_F(0, \"recurse-submodules\", NULL,\n \t\t\t    \"checkout\", \"control recursive updating of submodules\",\n@@ -247,7 +247,7 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix)\n \t\t\topts.head_idx = 1;\n \t}\n \n-\tif (opts.debug_unpack)\n+\tif (opts.internal.debug_unpack)\n \t\topts.fn = debug_merge;\n \n \t/* If we're going to prime_cache_tree later, skip cache tree update */\n@@ -263,7 +263,7 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix)\n \tif (unpack_trees(nr_trees, t, &opts))\n \t\treturn 128;\n \n-\tif (opts.debug_unpack || opts.dry_run)\n+\tif (opts.internal.debug_unpack || opts.dry_run)\n \t\treturn 0; /* do not write the index out */\n \n \t/*\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex fa186a27ccc..60b6e38fd69 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -839,7 +839,7 @@ static int traverse_by_cache_tree(int pos, int nr_entries, int nr_names,\n \t\tmark_ce_used(src[0], o);\n \t}\n \tfree(tree_ce);\n-\tif (o->debug_unpack)\n+\tif (o->internal.debug_unpack)\n \t\tprintf(\"Unpacked %d entries from %s to %s using cache-tree\\n\",\n \t\t       nr_entries,\n \t\t       o->src_index->cache[pos]->name,\n@@ -1488,7 +1488,7 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str\n \twhile (!p->mode)\n \t\tp++;\n \n-\tif (o->debug_unpack)\n+\tif (o->internal.debug_unpack)\n \t\tdebug_unpack_callback(n, mask, dirmask, names, info);\n \n \t/* Are we supposed to look at the index too? */\n@@ -1929,7 +1929,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t\tinit_split_index(&o->internal.result);\n \t}\n \toidcpy(&o->internal.result.oid, &o->src_index->oid);\n-\to->merge_size = len;\n+\to->internal.merge_size = len;\n \tmark_all_ce_unused(o->src_index);\n \n \to->internal.result.fsmonitor_last_update =\n@@ -2880,9 +2880,9 @@ int twoway_merge(const struct cache_entry * const *src,\n \tconst struct cache_entry *oldtree = src[1];\n \tconst struct cache_entry *newtree = src[2];\n \n-\tif (o->merge_size != 2)\n+\tif (o->internal.merge_size != 2)\n \t\treturn error(\"Cannot do a twoway merge of %d trees\",\n-\t\t\t     o->merge_size);\n+\t\t\t     o->internal.merge_size);\n \n \tif (oldtree == o->df_conflict_entry)\n \t\toldtree = NULL;\n@@ -2962,9 +2962,9 @@ int bind_merge(const struct cache_entry * const *src,\n \tconst struct cache_entry *old = src[0];\n \tconst struct cache_entry *a = src[1];\n \n-\tif (o->merge_size != 1)\n+\tif (o->internal.merge_size != 1)\n \t\treturn error(\"Cannot do a bind merge of %d trees\",\n-\t\t\t     o->merge_size);\n+\t\t\t     o->internal.merge_size);\n \tif (a && old)\n \t\treturn o->quiet ? -1 :\n \t\t\terror(ERRORMSG(o, ERROR_BIND_OVERLAP),\n@@ -2988,9 +2988,9 @@ int oneway_merge(const struct cache_entry * const *src,\n \tconst struct cache_entry *old = src[0];\n \tconst struct cache_entry *a = src[1];\n \n-\tif (o->merge_size != 1)\n+\tif (o->internal.merge_size != 1)\n \t\treturn error(\"Cannot do a oneway merge of %d trees\",\n-\t\t\t     o->merge_size);\n+\t\t\t     o->internal.merge_size);\n \n \tif (!a || a == o->df_conflict_entry)\n \t\treturn deleted_entry(old, old, o);\n@@ -3025,8 +3025,8 @@ int stash_worktree_untracked_merge(const struct cache_entry * const *src,\n \tconst struct cache_entry *worktree = src[1];\n \tconst struct cache_entry *untracked = src[2];\n \n-\tif (o->merge_size != 2)\n-\t\tBUG(\"invalid merge_size: %d\", o->merge_size);\n+\tif (o->internal.merge_size != 2)\n+\t\tBUG(\"invalid merge_size: %d\", o->internal.merge_size);\n \n \tif (worktree && untracked)\n \t\treturn error(_(\"worktree and untracked commit have duplicate entries: %s\"),\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 0335c89bc75..e8737adfeda 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -65,7 +65,6 @@ struct unpack_trees_options {\n \t\t     skip_unmerged,\n \t\t     initial_checkout,\n \t\t     diff_index_cached,\n-\t\t     debug_unpack,\n \t\t     skip_sparse_checkout,\n \t\t     quiet,\n \t\t     exiting_early,\n@@ -78,7 +77,6 @@ struct unpack_trees_options {\n \tmerge_fn_t fn;\n \n \tint head_idx;\n-\tint merge_size;\n \n \tstruct cache_entry *df_conflict_entry;\n \tvoid *unpack_data;\n@@ -90,8 +88,10 @@ struct unpack_trees_options {\n \n \tstruct unpack_trees_options_internal {\n \t\tunsigned int nontrivial_merge,\n-\t\t\t     show_all_errors;\n+\t\t\t     show_all_errors,\n+\t\t\t     debug_unpack; /* used by read-tree debugging */\n \n+\t\tint merge_size; /* used by read-tree debugging */\n \t\tint cache_bottom;\n \t\tconst char *msgs[NB_UNPACK_TREES_WARNING_TYPES];\n \t\tstruct strvec msgs_to_free;\n-- \ngitgitgadget\n\n"},{"id":"472518","messageId":"081578b321063525be8b70013ef3d9f263619e0d.1677143700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.git.1677143700.gitgitgadget@gmail.com","subject":"[PATCH 08/11] unpack-trees: mark fields only used internally as internal","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-23T09:14:57Z","receivedAt":"2023-02-23T09:15:37Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nContinue the work from the previous patch by finding additional fields\nwhich are only used internally but not yet explicitly marked as such,\nand include them in the internal fields struct.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 157 +++++++++++++++++++++++++------------------------\n unpack-trees.h |  26 ++++----\n 2 files changed, 94 insertions(+), 89 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex e6b5fb980cb..d89eb3d8bf0 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -66,8 +66,8 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_WARNING_TYPES] = {\n };\n \n #define ERRORMSG(o,type) \\\n-\t( ((o) && (o)->msgs[(type)]) \\\n-\t  ? ((o)->msgs[(type)])      \\\n+\t( ((o) && (o)->internal.msgs[(type)]) \\\n+\t  ? ((o)->internal.msgs[(type)])      \\\n \t  : (unpack_plumbing_errors[(type)]) )\n \n static const char *super_prefixed(const char *path, const char *super_prefix)\n@@ -108,10 +108,10 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t\t\t  const char *cmd)\n {\n \tint i;\n-\tconst char **msgs = opts->msgs;\n+\tconst char **msgs = opts->internal.msgs;\n \tconst char *msg;\n \n-\tstrvec_init(&opts->msgs_to_free);\n+\tstrvec_init(&opts->internal.msgs_to_free);\n \n \tif (!strcmp(cmd, \"checkout\"))\n \t\tmsg = advice_enabled(ADVICE_COMMIT_BEFORE_MERGE)\n@@ -129,7 +129,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t\t  \"Please commit your changes or stash them before you %s.\")\n \t\t      : _(\"Your local changes to the following files would be overwritten by %s:\\n%%s\");\n \tmsgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOT_UPTODATE_FILE] =\n-\t\tstrvec_pushf(&opts->msgs_to_free, msg, cmd, cmd);\n+\t\tstrvec_pushf(&opts->internal.msgs_to_free, msg, cmd, cmd);\n \n \tmsgs[ERROR_NOT_UPTODATE_DIR] =\n \t\t_(\"Updating the following directories would lose untracked files in them:\\n%s\");\n@@ -153,7 +153,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t\t  \"Please move or remove them before you %s.\")\n \t\t      : _(\"The following untracked working tree files would be removed by %s:\\n%%s\");\n \tmsgs[ERROR_WOULD_LOSE_UNTRACKED_REMOVED] =\n-\t\tstrvec_pushf(&opts->msgs_to_free, msg, cmd, cmd);\n+\t\tstrvec_pushf(&opts->internal.msgs_to_free, msg, cmd, cmd);\n \n \tif (!strcmp(cmd, \"checkout\"))\n \t\tmsg = advice_enabled(ADVICE_COMMIT_BEFORE_MERGE)\n@@ -171,7 +171,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t\t  \"Please move or remove them before you %s.\")\n \t\t      : _(\"The following untracked working tree files would be overwritten by %s:\\n%%s\");\n \tmsgs[ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN] =\n-\t\tstrvec_pushf(&opts->msgs_to_free, msg, cmd, cmd);\n+\t\tstrvec_pushf(&opts->internal.msgs_to_free, msg, cmd, cmd);\n \n \t/*\n \t * Special case: ERROR_BIND_OVERLAP refers to a pair of paths, we\n@@ -189,16 +189,16 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \tmsgs[WARNING_SPARSE_ORPHANED_NOT_OVERWRITTEN] =\n \t\t_(\"The following paths were already present and thus not updated despite sparse patterns:\\n%s\");\n \n-\topts->show_all_errors = 1;\n+\topts->internal.show_all_errors = 1;\n \t/* rejected paths may not have a static buffer */\n-\tfor (i = 0; i < ARRAY_SIZE(opts->unpack_rejects); i++)\n-\t\topts->unpack_rejects[i].strdup_strings = 1;\n+\tfor (i = 0; i < ARRAY_SIZE(opts->internal.unpack_rejects); i++)\n+\t\topts->internal.unpack_rejects[i].strdup_strings = 1;\n }\n \n void clear_unpack_trees_porcelain(struct unpack_trees_options *opts)\n {\n-\tstrvec_clear(&opts->msgs_to_free);\n-\tmemset(opts->msgs, 0, sizeof(opts->msgs));\n+\tstrvec_clear(&opts->internal.msgs_to_free);\n+\tmemset(opts->internal.msgs, 0, sizeof(opts->internal.msgs));\n }\n \n static int do_add_entry(struct unpack_trees_options *o, struct cache_entry *ce,\n@@ -210,7 +210,7 @@ static int do_add_entry(struct unpack_trees_options *o, struct cache_entry *ce,\n \t\tset |= CE_WT_REMOVE;\n \n \tce->ce_flags = (ce->ce_flags & ~clear) | set;\n-\treturn add_index_entry(&o->result, ce,\n+\treturn add_index_entry(&o->internal.result, ce,\n \t\t\t       ADD_CACHE_OK_TO_ADD | ADD_CACHE_OK_TO_REPLACE);\n }\n \n@@ -218,7 +218,7 @@ static void add_entry(struct unpack_trees_options *o,\n \t\t      const struct cache_entry *ce,\n \t\t      unsigned int set, unsigned int clear)\n {\n-\tdo_add_entry(o, dup_cache_entry(ce, &o->result), set, clear);\n+\tdo_add_entry(o, dup_cache_entry(ce, &o->internal.result), set, clear);\n }\n \n /*\n@@ -233,7 +233,7 @@ static int add_rejected_path(struct unpack_trees_options *o,\n \tif (o->quiet)\n \t\treturn -1;\n \n-\tif (!o->show_all_errors)\n+\tif (!o->internal.show_all_errors)\n \t\treturn error(ERRORMSG(o, e), super_prefixed(path,\n \t\t\t\t\t\t\t    o->super_prefix));\n \n@@ -241,7 +241,7 @@ static int add_rejected_path(struct unpack_trees_options *o,\n \t * Otherwise, insert in a list for future display by\n \t * display_(error|warning)_msgs()\n \t */\n-\tstring_list_append(&o->unpack_rejects[e], path);\n+\tstring_list_append(&o->internal.unpack_rejects[e], path);\n \treturn -1;\n }\n \n@@ -253,7 +253,7 @@ static void display_error_msgs(struct unpack_trees_options *o)\n \tint e;\n \tunsigned error_displayed = 0;\n \tfor (e = 0; e < NB_UNPACK_TREES_ERROR_TYPES; e++) {\n-\t\tstruct string_list *rejects = &o->unpack_rejects[e];\n+\t\tstruct string_list *rejects = &o->internal.unpack_rejects[e];\n \n \t\tif (rejects->nr > 0) {\n \t\t\tint i;\n@@ -281,7 +281,7 @@ static void display_warning_msgs(struct unpack_trees_options *o)\n \tunsigned warning_displayed = 0;\n \tfor (e = NB_UNPACK_TREES_ERROR_TYPES + 1;\n \t     e < NB_UNPACK_TREES_WARNING_TYPES; e++) {\n-\t\tstruct string_list *rejects = &o->unpack_rejects[e];\n+\t\tstruct string_list *rejects = &o->internal.unpack_rejects[e];\n \n \t\tif (rejects->nr > 0) {\n \t\t\tint i;\n@@ -600,13 +600,14 @@ static void mark_ce_used(struct cache_entry *ce, struct unpack_trees_options *o)\n {\n \tce->ce_flags |= CE_UNPACKED;\n \n-\tif (o->cache_bottom < o->src_index->cache_nr &&\n-\t    o->src_index->cache[o->cache_bottom] == ce) {\n-\t\tint bottom = o->cache_bottom;\n+\tif (o->internal.cache_bottom < o->src_index->cache_nr &&\n+\t    o->src_index->cache[o->internal.cache_bottom] == ce) {\n+\t\tint bottom = o->internal.cache_bottom;\n+\n \t\twhile (bottom < o->src_index->cache_nr &&\n \t\t       o->src_index->cache[bottom]->ce_flags & CE_UNPACKED)\n \t\t\tbottom++;\n-\t\to->cache_bottom = bottom;\n+\t\to->internal.cache_bottom = bottom;\n \t}\n }\n \n@@ -652,7 +653,7 @@ static void mark_ce_used_same_name(struct cache_entry *ce,\n static struct cache_entry *next_cache_entry(struct unpack_trees_options *o)\n {\n \tconst struct index_state *index = o->src_index;\n-\tint pos = o->cache_bottom;\n+\tint pos = o->internal.cache_bottom;\n \n \twhile (pos < index->cache_nr) {\n \t\tstruct cache_entry *ce = index->cache[pos];\n@@ -711,7 +712,7 @@ static void restore_cache_bottom(struct traverse_info *info, int bottom)\n \n \tif (o->diff_index_cached)\n \t\treturn;\n-\to->cache_bottom = bottom;\n+\to->internal.cache_bottom = bottom;\n }\n \n static int switch_cache_bottom(struct traverse_info *info)\n@@ -721,13 +722,13 @@ static int switch_cache_bottom(struct traverse_info *info)\n \n \tif (o->diff_index_cached)\n \t\treturn 0;\n-\tret = o->cache_bottom;\n+\tret = o->internal.cache_bottom;\n \tpos = find_cache_pos(info->prev, info->name, info->namelen);\n \n \tif (pos < -1)\n-\t\to->cache_bottom = -2 - pos;\n+\t\to->internal.cache_bottom = -2 - pos;\n \telse if (pos < 0)\n-\t\to->cache_bottom = o->src_index->cache_nr;\n+\t\to->internal.cache_bottom = o->src_index->cache_nr;\n \treturn ret;\n }\n \n@@ -873,9 +874,9 @@ static int traverse_trees_recursive(int n, unsigned long dirmask,\n \t\t * save and restore cache_bottom anyway to not miss\n \t\t * unprocessed entries before 'pos'.\n \t\t */\n-\t\tbottom = o->cache_bottom;\n+\t\tbottom = o->internal.cache_bottom;\n \t\tret = traverse_by_cache_tree(pos, nr_entries, n, info);\n-\t\to->cache_bottom = bottom;\n+\t\to->internal.cache_bottom = bottom;\n \t\treturn ret;\n \t}\n \n@@ -1212,7 +1213,7 @@ static int unpack_single_entry(int n, unsigned long mask,\n \t\t * cache entry from the index aware logic.\n \t\t */\n \t\tsrc[i + o->merge] = create_ce_entry(info, names + i, stage,\n-\t\t\t\t\t\t    &o->result, o->merge,\n+\t\t\t\t\t\t    &o->internal.result, o->merge,\n \t\t\t\t\t\t    bit & dirmask);\n \t}\n \n@@ -1237,7 +1238,7 @@ static int unpack_single_entry(int n, unsigned long mask,\n \n static int unpack_failed(struct unpack_trees_options *o, const char *message)\n {\n-\tdiscard_index(&o->result);\n+\tdiscard_index(&o->internal.result);\n \tif (!o->quiet && !o->exiting_early) {\n \t\tif (message)\n \t\t\treturn error(\"%s\", message);\n@@ -1260,7 +1261,7 @@ static int find_cache_pos(struct traverse_info *info,\n \tstruct index_state *index = o->src_index;\n \tint pfxlen = info->pathlen;\n \n-\tfor (pos = o->cache_bottom; pos < index->cache_nr; pos++) {\n+\tfor (pos = o->internal.cache_bottom; pos < index->cache_nr; pos++) {\n \t\tconst struct cache_entry *ce = index->cache[pos];\n \t\tconst char *ce_name, *ce_slash;\n \t\tint cmp, ce_len;\n@@ -1271,8 +1272,8 @@ static int find_cache_pos(struct traverse_info *info,\n \t\t\t * we can never match it; don't check it\n \t\t\t * again.\n \t\t\t */\n-\t\t\tif (pos == o->cache_bottom)\n-\t\t\t\t++o->cache_bottom;\n+\t\t\tif (pos == o->internal.cache_bottom)\n+\t\t\t\t++o->internal.cache_bottom;\n \t\t\tcontinue;\n \t\t}\n \t\tif (!ce_in_traverse_path(ce, info)) {\n@@ -1450,7 +1451,7 @@ static int unpack_sparse_callback(int n, unsigned long mask, unsigned long dirma\n \t */\n \tif (!is_null_oid(&names[0].oid)) {\n \t\tsrc[0] = create_ce_entry(info, &names[0], 0,\n-\t\t\t\t\t&o->result, 1,\n+\t\t\t\t\t&o->internal.result, 1,\n \t\t\t\t\tdirmask & (1ul << 0));\n \t\tsrc[0]->ce_flags |= (CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE);\n \t}\n@@ -1560,7 +1561,7 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str\n \t\t\t\t * in 'mark_ce_used()'\n \t\t\t\t */\n \t\t\t\tif (!src[0] || !S_ISSPARSEDIR(src[0]->ce_mode))\n-\t\t\t\t\to->cache_bottom += matches;\n+\t\t\t\t\to->internal.cache_bottom += matches;\n \t\t\t\treturn mask;\n \t\t\t}\n \t\t}\n@@ -1907,37 +1908,37 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tpopulate_from_existing_patterns(o, &pl);\n \t}\n \n-\tindex_state_init(&o->result, o->src_index->repo);\n-\to->result.initialized = 1;\n-\to->result.timestamp.sec = o->src_index->timestamp.sec;\n-\to->result.timestamp.nsec = o->src_index->timestamp.nsec;\n-\to->result.version = o->src_index->version;\n+\tindex_state_init(&o->internal.result, o->src_index->repo);\n+\to->internal.result.initialized = 1;\n+\to->internal.result.timestamp.sec = o->src_index->timestamp.sec;\n+\to->internal.result.timestamp.nsec = o->src_index->timestamp.nsec;\n+\to->internal.result.version = o->src_index->version;\n \tif (!o->src_index->split_index) {\n-\t\to->result.split_index = NULL;\n+\t\to->internal.result.split_index = NULL;\n \t} else if (o->src_index == o->dst_index) {\n \t\t/*\n \t\t * o->dst_index (and thus o->src_index) will be discarded\n-\t\t * and overwritten with o->result at the end of this function,\n+\t\t * and overwritten with o->internal.result at the end of this function,\n \t\t * so just use src_index's split_index to avoid having to\n \t\t * create a new one.\n \t\t */\n-\t\to->result.split_index = o->src_index->split_index;\n-\t\to->result.split_index->refcount++;\n+\t\to->internal.result.split_index = o->src_index->split_index;\n+\t\to->internal.result.split_index->refcount++;\n \t} else {\n-\t\to->result.split_index = init_split_index(&o->result);\n+\t\to->internal.result.split_index = init_split_index(&o->internal.result);\n \t}\n-\toidcpy(&o->result.oid, &o->src_index->oid);\n+\toidcpy(&o->internal.result.oid, &o->src_index->oid);\n \to->merge_size = len;\n \tmark_all_ce_unused(o->src_index);\n \n-\to->result.fsmonitor_last_update =\n+\to->internal.result.fsmonitor_last_update =\n \t\txstrdup_or_null(o->src_index->fsmonitor_last_update);\n-\to->result.fsmonitor_has_run_once = o->src_index->fsmonitor_has_run_once;\n+\to->internal.result.fsmonitor_has_run_once = o->src_index->fsmonitor_has_run_once;\n \n \tif (!o->src_index->initialized &&\n \t    !repo->settings.command_requires_full_index &&\n-\t    is_sparse_index_allowed(&o->result, 0))\n-\t\to->result.sparse_index = 1;\n+\t    is_sparse_index_allowed(&o->internal.result, 0))\n+\t\to->internal.result.sparse_index = 1;\n \n \t/*\n \t * Sparse checkout loop #1: set NEW_SKIP_WORKTREE on existing entries\n@@ -1957,7 +1958,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tsetup_traverse_info(&info, prefix);\n \t\tinfo.fn = unpack_callback;\n \t\tinfo.data = o;\n-\t\tinfo.show_all_errors = o->show_all_errors;\n+\t\tinfo.show_all_errors = o->internal.show_all_errors;\n \t\tinfo.pathspec = o->pathspec;\n \n \t\tif (o->prefix) {\n@@ -1998,7 +1999,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t}\n \tmark_all_ce_unused(o->src_index);\n \n-\tif (o->trivial_merges_only && o->nontrivial_merge) {\n+\tif (o->trivial_merges_only && o->internal.nontrivial_merge) {\n \t\tret = unpack_failed(o, \"Merge requires file-level merging\");\n \t\tgoto done;\n \t}\n@@ -2009,13 +2010,13 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t * If they will have NEW_SKIP_WORKTREE, also set CE_SKIP_WORKTREE\n \t\t * so apply_sparse_checkout() won't attempt to remove it from worktree\n \t\t */\n-\t\tmark_new_skip_worktree(o->internal.pl, &o->result,\n+\t\tmark_new_skip_worktree(o->internal.pl, &o->internal.result,\n \t\t\t\t       CE_ADDED, CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE,\n \t\t\t\t       o->verbose_update);\n \n \t\tret = 0;\n-\t\tfor (i = 0; i < o->result.cache_nr; i++) {\n-\t\t\tstruct cache_entry *ce = o->result.cache[i];\n+\t\tfor (i = 0; i < o->internal.result.cache_nr; i++) {\n+\t\t\tstruct cache_entry *ce = o->internal.result.cache[i];\n \n \t\t\t/*\n \t\t\t * Entries marked with CE_ADDED in merged_entry() do not have\n@@ -2029,7 +2030,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t\t    verify_absent(ce, WARNING_SPARSE_ORPHANED_NOT_OVERWRITTEN, o))\n \t\t\t\tret = 1;\n \n-\t\t\tif (apply_sparse_checkout(&o->result, ce, o))\n+\t\t\tif (apply_sparse_checkout(&o->internal.result, ce, o))\n \t\t\t\tret = 1;\n \t\t}\n \t\tif (ret == 1) {\n@@ -2037,30 +2038,30 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t\t * Inability to sparsify or de-sparsify individual\n \t\t\t * paths is not an error, but just a warning.\n \t\t\t */\n-\t\t\tif (o->show_all_errors)\n+\t\t\tif (o->internal.show_all_errors)\n \t\t\t\tdisplay_warning_msgs(o);\n \t\t\tret = 0;\n \t\t}\n \t}\n \n-\tret = check_updates(o, &o->result) ? (-2) : 0;\n+\tret = check_updates(o, &o->internal.result) ? (-2) : 0;\n \tif (o->dst_index) {\n-\t\tmove_index_extensions(&o->result, o->src_index);\n+\t\tmove_index_extensions(&o->internal.result, o->src_index);\n \t\tif (!ret) {\n \t\t\tif (git_env_bool(\"GIT_TEST_CHECK_CACHE_TREE\", 0))\n-\t\t\t\tcache_tree_verify(the_repository, &o->result);\n+\t\t\t\tcache_tree_verify(the_repository, &o->internal.result);\n \t\t\tif (!o->skip_cache_tree_update &&\n-\t\t\t    !cache_tree_fully_valid(o->result.cache_tree))\n-\t\t\t\tcache_tree_update(&o->result,\n+\t\t\t    !cache_tree_fully_valid(o->internal.result.cache_tree))\n+\t\t\t\tcache_tree_update(&o->internal.result,\n \t\t\t\t\t\t  WRITE_TREE_SILENT |\n \t\t\t\t\t\t  WRITE_TREE_REPAIR);\n \t\t}\n \n-\t\to->result.updated_workdir = 1;\n+\t\to->internal.result.updated_workdir = 1;\n \t\tdiscard_index(o->dst_index);\n-\t\t*o->dst_index = o->result;\n+\t\t*o->dst_index = o->internal.result;\n \t} else {\n-\t\tdiscard_index(&o->result);\n+\t\tdiscard_index(&o->internal.result);\n \t}\n \to->src_index = NULL;\n \n@@ -2076,7 +2077,7 @@ done:\n \treturn ret;\n \n return_failed:\n-\tif (o->show_all_errors)\n+\tif (o->internal.show_all_errors)\n \t\tdisplay_error_msgs(o);\n \tmark_all_ce_unused(o->src_index);\n \tret = unpack_failed(o, NULL);\n@@ -2099,8 +2100,8 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n \tunsigned old_show_all_errors;\n \tint free_pattern_list = 0;\n \n-\told_show_all_errors = o->show_all_errors;\n-\to->show_all_errors = 1;\n+\told_show_all_errors = o->internal.show_all_errors;\n+\to->internal.show_all_errors = 1;\n \n \t/* Sanity checks */\n \tif (!o->update || o->index_only || o->skip_sparse_checkout)\n@@ -2147,7 +2148,7 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n \t\tret = UPDATE_SPARSITY_WORKTREE_UPDATE_FAILURES;\n \n \tdisplay_warning_msgs(o);\n-\to->show_all_errors = old_show_all_errors;\n+\to->internal.show_all_errors = old_show_all_errors;\n \tif (free_pattern_list) {\n \t\tclear_pattern_list(pl);\n \t\tfree(pl);\n@@ -2246,15 +2247,15 @@ static int verify_uptodate_sparse(const struct cache_entry *ce,\n }\n \n /*\n- * TODO: We should actually invalidate o->result, not src_index [1].\n+ * TODO: We should actually invalidate o->internal.result, not src_index [1].\n  * But since cache tree and untracked cache both are not copied to\n- * o->result until unpacking is complete, we invalidate them on\n+ * o->internal.result until unpacking is complete, we invalidate them on\n  * src_index instead with the assumption that they will be copied to\n  * dst_index at the end.\n  *\n  * [1] src_index->cache_tree is also used in unpack_callback() so if\n- * we invalidate o->result, we need to update it to use\n- * o->result.cache_tree as well.\n+ * we invalidate o->internal.result, we need to update it to use\n+ * o->internal.result.cache_tree as well.\n  */\n static void invalidate_ce_path(const struct cache_entry *ce,\n \t\t\t       struct unpack_trees_options *o)\n@@ -2422,7 +2423,7 @@ static int check_ok_to_remove(const char *name, int len, int dtype,\n \t * delete this path, which is in a subdirectory that\n \t * is being replaced with a blob.\n \t */\n-\tresult = index_file_exists(&o->result, name, len, 0);\n+\tresult = index_file_exists(&o->internal.result, name, len, 0);\n \tif (result) {\n \t\tif (result->ce_flags & CE_REMOVE)\n \t\t\treturn 0;\n@@ -2523,7 +2524,7 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\tstruct unpack_trees_options *o)\n {\n \tint update = CE_UPDATE;\n-\tstruct cache_entry *merge = dup_cache_entry(ce, &o->result);\n+\tstruct cache_entry *merge = dup_cache_entry(ce, &o->internal.result);\n \n \tif (!old) {\n \t\t/*\n@@ -2618,7 +2619,7 @@ static int merged_sparse_dir(const struct cache_entry * const *src, int n,\n \tsetup_traverse_info(&info, src[0]->name);\n \tinfo.fn = unpack_sparse_callback;\n \tinfo.data = o;\n-\tinfo.show_all_errors = o->show_all_errors;\n+\tinfo.show_all_errors = o->internal.show_all_errors;\n \tinfo.pathspec = o->pathspec;\n \n \t/* Get the tree descriptors of the sparse directory in each of the merging trees */\n@@ -2836,7 +2837,7 @@ int threeway_merge(const struct cache_entry * const *stages,\n \t\t\treturn -1;\n \t}\n \n-\to->nontrivial_merge = 1;\n+\to->internal.nontrivial_merge = 1;\n \n \t/* #2, #3, #4, #6, #7, #9, #10, #11. */\n \tcount = 0;\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 5c1a9314a06..0335c89bc75 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -59,7 +59,6 @@ struct unpack_trees_options {\n \t\t     preserve_ignored,\n \t\t     clone,\n \t\t     index_only,\n-\t\t     nontrivial_merge,\n \t\t     trivial_merges_only,\n \t\t     verbose_update,\n \t\t     aggressive,\n@@ -70,22 +69,13 @@ struct unpack_trees_options {\n \t\t     skip_sparse_checkout,\n \t\t     quiet,\n \t\t     exiting_early,\n-\t\t     show_all_errors,\n \t\t     dry_run,\n \t\t     skip_cache_tree_update;\n \tenum unpack_trees_reset_type reset;\n \tconst char *prefix;\n \tconst char *super_prefix;\n-\tint cache_bottom;\n \tstruct pathspec *pathspec;\n \tmerge_fn_t fn;\n-\tconst char *msgs[NB_UNPACK_TREES_WARNING_TYPES];\n-\tstruct strvec msgs_to_free;\n-\t/*\n-\t * Store error messages in an array, each case\n-\t * corresponding to a error message type\n-\t */\n-\tstruct string_list unpack_rejects[NB_UNPACK_TREES_WARNING_TYPES];\n \n \tint head_idx;\n \tint merge_size;\n@@ -95,11 +85,25 @@ struct unpack_trees_options {\n \n \tstruct index_state *dst_index;\n \tstruct index_state *src_index;\n-\tstruct index_state result;\n \n \tstruct checkout_metadata meta;\n \n \tstruct unpack_trees_options_internal {\n+\t\tunsigned int nontrivial_merge,\n+\t\t\t     show_all_errors;\n+\n+\t\tint cache_bottom;\n+\t\tconst char *msgs[NB_UNPACK_TREES_WARNING_TYPES];\n+\t\tstruct strvec msgs_to_free;\n+\n+\t\t/*\n+\t\t * Store error messages in an array, each case\n+\t\t * corresponding to a error message type\n+\t\t */\n+\t\tstruct string_list unpack_rejects[NB_UNPACK_TREES_WARNING_TYPES];\n+\n+\t\tstruct index_state result;\n+\n \t\tstruct pattern_list *pl;\n \t\tstruct dir_struct *dir;\n \t} internal;\n-- \ngitgitgadget\n\n"},{"id":"472519","messageId":"efec74c8b497998d6a4866b6ed4de4fbe22680c4.1677143700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.git.1677143700.gitgitgadget@gmail.com","subject":"[PATCH 11/11] unpack-trees: add usage notices around df_conflict_entry","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-23T09:15:00Z","receivedAt":"2023-02-23T09:15:39Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nAvoid making users believe they need to initialize df_conflict_entry\nto something (as happened with other output only fields before) with\na quick comment and a small sanity check.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 2 ++\n unpack-trees.h | 2 +-\n 2 files changed, 3 insertions(+), 1 deletion(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 60b6e38fd69..583132f1510 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1876,6 +1876,8 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tBUG(\"o->internal.dir is for internal use only\");\n \tif (o->internal.pl)\n \t\tBUG(\"o->internal.pl is for internal use only\");\n+\tif (o->df_conflict_entry)\n+\t\tBUG(\"o->df_conflict_entry is an output only field\");\n \n \ttrace_performance_enter();\n \ttrace2_region_enter(\"unpack_trees\", \"unpack_trees\", the_repository);\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex e8737adfeda..61c06eb7c50 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -78,7 +78,7 @@ struct unpack_trees_options {\n \n \tint head_idx;\n \n-\tstruct cache_entry *df_conflict_entry;\n+\tstruct cache_entry *df_conflict_entry; /* output only */\n \tvoid *unpack_data;\n \n \tstruct index_state *dst_index;\n-- \ngitgitgadget\n"},{"id":"472555","messageId":"16ff5069-0408-21cd-995c-8b47afb9810d@github.com","threadId":"59297","inReplyTo":"pull.1149.git.1677143700.gitgitgadget@gmail.com","subject":"Re: [PATCH 00/11] Clarify API for dir.[ch] and unpack-trees.[ch] -- mark relevant fields as internal","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2023-02-23T15:18:45Z","receivedAt":"2023-02-23T15:18:50Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 2/23/2023 4:14 AM, Elijah Newren via GitGitGadget wrote:\n> This patch is primarily about moving internal-only fields within these two\n> structs into an embedded internal struct. Patch breakdown:\n> \n>  * Patches 1-3: Restructuring dir_struct\n>    * Patch 1: Splitting off internal-use-only fields\n>    * Patch 2: Add important usage note to avoid accidentally using\n>      deprecated API\n>    * Patch 3: Mark output-only fields as such\n>  * Patches 4-11: Restructuring unpack_trees_options\n>    * Patches 4-6: Preparatory cleanup\n>    * Patches 7-10: Splitting off internal-use-only fields\n>    * Patch 11: Mark output-only field as such\n \n> And after the changes:\n> \n> struct dir_struct {\n>     enum [...] flags;\n>     int nr; /* output only */\n>     int ignored_nr; /* output only */\n>     struct dir_entry **entries; /* output only */\n>     struct dir_entry **ignored; /* output only */\n>     struct untracked_cache *untracked;\n>     const char *exclude_per_dir; /* deprecated */\n>     struct dir_struct_internal {\n>         int alloc;\n>         int ignored_alloc;\n> #define EXC_CMDL 0\n> #define EXC_DIRS 1\n> #define EXC_FILE 2\n>         struct exclude_list_group exclude_list_group[3];\n>         struct exclude_stack *exclude_stack;\n>         struct path_pattern *pattern;\n>         struct strbuf basebuf;\n>         struct oid_stat ss_info_exclude;\n>         struct oid_stat ss_excludes_file;\n>         unsigned unmanaged_exclude_files;\n>         unsigned visited_paths;\n>         unsigned visited_directories;\n>     } internal;\n> };\n\nThis does present a very clear structure to avoid callers being\nconfused when writing these changes. It doesn't, however, present\nany way to guarantee that callers can't mutate this state.\n\n...here I go on a side track thinking of an alternative...\n\nOne way to track this would be to anonymously declare 'struct\ndir_struct_internal' in the header file and let 'struct dir_struct'\ncontain a _pointer_ to the internal struct. The dir_struct_internal\ncan then be defined inside the .c file, limiting its scope. (It\nmust be a pointer in dir_struct or else callers would not be able\nto create a dir_struct without using a pointer and an initializer\nmethod.\n\nThe major downside to this pointer approach is that the internal\nstruct needs to be initialized within API calls and somehow cleared\nby all callers. The internal data could be initialized by the common\ninitializers read_directory() or fill_directory(). There is a\ndir_clear() that _should_ be called by all callers (but I notice we\nare leaking the struct in at least one place in add-interactive.c,\nand likely others).\n\nThis alternative adds some complexity to the structure, but\nprovides compiler-level guarantees that these internals are not used\noutside of dir.c. I thought it worth exploring, even if we decide\nthat the complexity is not worth those guarantees.\n\nThe best news is that your existing series makes it easier to flip\nto the internal pointer method in the future, since we can shift\nthe 'd->internal.member\" uses into \"d->internal->member\" in a\nmechanical way. Thus, the change you are proposing does not lock us\ninto this approach if we change our minds later.\n\nThanks,\n-Stolee\n"},{"id":"472556","messageId":"73d694c5-f2d9-c05b-c880-8d5650f36797@github.com","threadId":"59297","inReplyTo":"16ff5069-0408-21cd-995c-8b47afb9810d@github.com","subject":"Re: [PATCH 00/11] Clarify API for dir.[ch] and unpack-trees.[ch] -- mark relevant fields as internal","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2023-02-23T15:26:15Z","receivedAt":"2023-02-23T15:26:19Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 2/23/2023 10:18 AM, Derrick Stolee wrote:\n> On 2/23/2023 4:14 AM, Elijah Newren via GitGitGadget wrote:\n>> This patch is primarily about moving internal-only fields within these two\n>> structs into an embedded internal struct. Patch breakdown:\n>>\n>>  * Patches 1-3: Restructuring dir_struct\n>>    * Patch 1: Splitting off internal-use-only fields\n>>    * Patch 2: Add important usage note to avoid accidentally using\n>>      deprecated API\n>>    * Patch 3: Mark output-only fields as such\n>>  * Patches 4-11: Restructuring unpack_trees_options\n>>    * Patches 4-6: Preparatory cleanup\n>>    * Patches 7-10: Splitting off internal-use-only fields\n>>    * Patch 11: Mark output-only field as such\n...\n> The best news is that your existing series makes it easier to flip\n> to the internal pointer method in the future, since we can shift\n> the 'd->internal.member\" uses into \"d->internal->member\" in a\n> mechanical way. Thus, the change you are proposing does not lock us\n> into this approach if we change our minds later.\n\nAnd now that I've read the series in its entirety, I think it is\nwell organized and does not need any updates. It creates a better\nsituation than what we already have, and any changes to split the\ninternal structs to be anonymous to callers can be done as a\nfollow-up.\n\nThanks,\n-Stolee\n"},{"id":"472572","messageId":"CABPp-BF06RLazEjNfe0E=arM+q=dw9r478Q1ZLMeY-OxUf2CLg@mail.gmail.com","threadId":"59297","inReplyTo":"16ff5069-0408-21cd-995c-8b47afb9810d@github.com","subject":"Re: [PATCH 00/11] Clarify API for dir.[ch] and unpack-trees.[ch] -- mark relevant fields as internal","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2023-02-23T20:31:29Z","receivedAt":"2023-02-23T20:31:48Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Feb 23, 2023 at 7:18 AM Derrick Stolee <derrickstolee@github.com> wrote:\n>\n> On 2/23/2023 4:14 AM, Elijah Newren via GitGitGadget wrote:\n> > This patch is primarily about moving internal-only fields within these two\n> > structs into an embedded internal struct. Patch breakdown:\n> >\n> >  * Patches 1-3: Restructuring dir_struct\n> >    * Patch 1: Splitting off internal-use-only fields\n> >    * Patch 2: Add important usage note to avoid accidentally using\n> >      deprecated API\n> >    * Patch 3: Mark output-only fields as such\n> >  * Patches 4-11: Restructuring unpack_trees_options\n> >    * Patches 4-6: Preparatory cleanup\n> >    * Patches 7-10: Splitting off internal-use-only fields\n> >    * Patch 11: Mark output-only field as such\n>\n> > And after the changes:\n> >\n> > struct dir_struct {\n> >     enum [...] flags;\n> >     int nr; /* output only */\n> >     int ignored_nr; /* output only */\n> >     struct dir_entry **entries; /* output only */\n> >     struct dir_entry **ignored; /* output only */\n> >     struct untracked_cache *untracked;\n> >     const char *exclude_per_dir; /* deprecated */\n> >     struct dir_struct_internal {\n> >         int alloc;\n> >         int ignored_alloc;\n> > #define EXC_CMDL 0\n> > #define EXC_DIRS 1\n> > #define EXC_FILE 2\n> >         struct exclude_list_group exclude_list_group[3];\n> >         struct exclude_stack *exclude_stack;\n> >         struct path_pattern *pattern;\n> >         struct strbuf basebuf;\n> >         struct oid_stat ss_info_exclude;\n> >         struct oid_stat ss_excludes_file;\n> >         unsigned unmanaged_exclude_files;\n> >         unsigned visited_paths;\n> >         unsigned visited_directories;\n> >     } internal;\n> > };\n>\n> This does present a very clear structure to avoid callers being\n> confused when writing these changes. It doesn't, however, present\n> any way to guarantee that callers can't mutate this state.\n>\n> ...here I go on a side track thinking of an alternative...\n>\n> One way to track this would be to anonymously declare 'struct\n> dir_struct_internal' in the header file and let 'struct dir_struct'\n> contain a _pointer_ to the internal struct. The dir_struct_internal\n> can then be defined inside the .c file, limiting its scope. (It\n> must be a pointer in dir_struct or else callers would not be able\n> to create a dir_struct without using a pointer and an initializer\n> method.\n>\n> The major downside to this pointer approach is that the internal\n> struct needs to be initialized within API calls and somehow cleared\n> by all callers. The internal data could be initialized by the common\n> initializers read_directory() or fill_directory(). There is a\n> dir_clear() that _should_ be called by all callers (but I notice we\n> are leaking the struct in at least one place in add-interactive.c,\n> and likely others).\n>\n> This alternative adds some complexity to the structure, but\n> provides compiler-level guarantees that these internals are not used\n> outside of dir.c. I thought it worth exploring, even if we decide\n> that the complexity is not worth those guarantees.\n\nIn addition to the guarantees that others won't muck with those\nfields, such an approach would also buy us the following:\n\n  * The implementation can change and internal data structures be\nmodified/extended/appended-to, without requiring recompiling all files\nthat depend upon our header.\n  * Related to the last point, the ABI doesn't change when the\nimplementation and internal data structures change.  Might be helpful\nfor libification efforts.\n\nFor all three of these reasons, I pursued this alternate strategy you\nbring up in merge-recursive and merge-ort (they both use a \"struct\nmerge_options_internal *priv\" and then define _very_ different \"struct\nmerge_options_iternal\" in their respective C files).  I thought about\nusing this strategy you suggest here, but was worried at the time I\ncreated this patch that it might create more friction for Ævar who was\npushing his struct initialization work and memory leak cleanups in the\nsame area heavily at the time (and back then we had a few long threads\nback and forth because our work was overlapping and clashing\nslightly).  But I figured that doing at least this much was good,\nbecause of what you point out:\n\n> The best news is that your existing series makes it easier to flip\n> to the internal pointer method in the future, since we can shift\n> the 'd->internal.member\" uses into \"d->internal->member\" in a\n> mechanical way. Thus, the change you are proposing does not lock us\n> into this approach if we change our minds later.\n"},{"id":"472573","messageId":"CABPp-BGu8DKBcKCxLnBMBbPvo+DdFyad_iqTt1sbyRSm5tKH5Q@mail.gmail.com","threadId":"59297","inReplyTo":"73d694c5-f2d9-c05b-c880-8d5650f36797@github.com","subject":"Re: [PATCH 00/11] Clarify API for dir.[ch] and unpack-trees.[ch] -- mark relevant fields as internal","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2023-02-23T20:35:01Z","receivedAt":"2023-02-23T20:35:26Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Feb 23, 2023 at 7:26 AM Derrick Stolee <derrickstolee@github.com> wrote:\n>\n> On 2/23/2023 10:18 AM, Derrick Stolee wrote:\n> > On 2/23/2023 4:14 AM, Elijah Newren via GitGitGadget wrote:\n> >> This patch is primarily about moving internal-only fields within these two\n> >> structs into an embedded internal struct. Patch breakdown:\n> >>\n> >>  * Patches 1-3: Restructuring dir_struct\n> >>    * Patch 1: Splitting off internal-use-only fields\n> >>    * Patch 2: Add important usage note to avoid accidentally using\n> >>      deprecated API\n> >>    * Patch 3: Mark output-only fields as such\n> >>  * Patches 4-11: Restructuring unpack_trees_options\n> >>    * Patches 4-6: Preparatory cleanup\n> >>    * Patches 7-10: Splitting off internal-use-only fields\n> >>    * Patch 11: Mark output-only field as such\n> ...\n> > The best news is that your existing series makes it easier to flip\n> > to the internal pointer method in the future, since we can shift\n> > the 'd->internal.member\" uses into \"d->internal->member\" in a\n> > mechanical way. Thus, the change you are proposing does not lock us\n> > into this approach if we change our minds later.\n>\n> And now that I've read the series in its entirety, I think it is\n> well organized and does not need any updates. It creates a better\n> situation than what we already have, and any changes to split the\n> internal structs to be anonymous to callers can be done as a\n> follow-up.\n\nWow, I was a bit worried pushing a couple dozen patches last night\nthat it'd be weeks before anyone took a look, and perhaps even that\nI'd again get comments that I was pushing too many to the list.  You\nread and reviewed all of them across both series, including some\ncomments showing you read pretty carefully, all before I had even\nwoken up.  Very cool; thanks.\n"},{"id":"472605","messageId":"xmqqzg933lfw.fsf@gitster.g","threadId":"59297","inReplyTo":"16ff5069-0408-21cd-995c-8b47afb9810d@github.com","subject":"Re: [PATCH 00/11] Clarify API for dir.[ch] and unpack-trees.[ch] -- mark relevant fields as internal","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-02-24T01:24:03Z","receivedAt":"2023-02-24T01:24:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee <derrickstolee@github.com> writes:\n\n> The major downside to this pointer approach is that the internal\n> struct needs to be initialized within API calls and somehow cleared\n> by all callers. The internal data could be initialized by the common\n> initializers read_directory() or fill_directory(). There is a\n> dir_clear() that _should_ be called by all callers (but I notice we\n> are leaking the struct in at least one place in add-interactive.c,\n> and likely others).\n>\n> This alternative adds some complexity to the structure, but\n> provides compiler-level guarantees that these internals are not used\n> outside of dir.c. I thought it worth exploring, even if we decide\n> that the complexity is not worth those guarantees.\n\nI actually think the current structure may be a good place to stop\nat.  Or we could use the original flat structure, but with members\nthat are supposed to be private prefixed with a longer prefix that\nis very specific to the dir.c file, say \"private_to_dir_c_\".\n\nThen have a block of #define\n\n\t#define alloc private_to_dir_c_alloc\n\t#define ignored_alloc private_to_dir_c_ignored_alloc\n\t...\n\t#define visited_directories private_to_dir_c_visited_directories\n\nat the beginning dir.c to hide the cruft out of the implementation.\n\"git grep private_to_dir_c ':!dir.c'\" would catch any outsider\npeeking into the part of the struct that they shouldn't be touching.\n\n\n\n\n"},{"id":"472615","messageId":"CA+P7+xo02dGkjb5DwJ1Af_hoQ5HiuxASheZxoFz+r6B-6cQMug@mail.gmail.com","threadId":"59297","inReplyTo":"16ff5069-0408-21cd-995c-8b47afb9810d@github.com","subject":"Re: [PATCH 00/11] Clarify API for dir.[ch] and unpack-trees.[ch] -- mark relevant fields as internal","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2023-02-24T05:54:00Z","receivedAt":"2023-02-24T05:54:14Z","isPatch":true,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Thu, Feb 23, 2023 at 7:29 AM Derrick Stolee <derrickstolee@github.com> wrote:\n>\n> On 2/23/2023 4:14 AM, Elijah Newren via GitGitGadget wrote:\n> > This patch is primarily about moving internal-only fields within these two\n> > structs into an embedded internal struct. Patch breakdown:\n> >\n> >  * Patches 1-3: Restructuring dir_struct\n> >    * Patch 1: Splitting off internal-use-only fields\n> >    * Patch 2: Add important usage note to avoid accidentally using\n> >      deprecated API\n> >    * Patch 3: Mark output-only fields as such\n> >  * Patches 4-11: Restructuring unpack_trees_options\n> >    * Patches 4-6: Preparatory cleanup\n> >    * Patches 7-10: Splitting off internal-use-only fields\n> >    * Patch 11: Mark output-only field as such\n>\n> > And after the changes:\n> >\n> > struct dir_struct {\n> >     enum [...] flags;\n> >     int nr; /* output only */\n> >     int ignored_nr; /* output only */\n> >     struct dir_entry **entries; /* output only */\n> >     struct dir_entry **ignored; /* output only */\n> >     struct untracked_cache *untracked;\n> >     const char *exclude_per_dir; /* deprecated */\n> >     struct dir_struct_internal {\n> >         int alloc;\n> >         int ignored_alloc;\n> > #define EXC_CMDL 0\n> > #define EXC_DIRS 1\n> > #define EXC_FILE 2\n> >         struct exclude_list_group exclude_list_group[3];\n> >         struct exclude_stack *exclude_stack;\n> >         struct path_pattern *pattern;\n> >         struct strbuf basebuf;\n> >         struct oid_stat ss_info_exclude;\n> >         struct oid_stat ss_excludes_file;\n> >         unsigned unmanaged_exclude_files;\n> >         unsigned visited_paths;\n> >         unsigned visited_directories;\n> >     } internal;\n> > };\n>\n> This does present a very clear structure to avoid callers being\n> confused when writing these changes. It doesn't, however, present\n> any way to guarantee that callers can't mutate this state.\n>\n> ...here I go on a side track thinking of an alternative...\n>\n> One way to track this would be to anonymously declare 'struct\n> dir_struct_internal' in the header file and let 'struct dir_struct'\n> contain a _pointer_ to the internal struct. The dir_struct_internal\n> can then be defined inside the .c file, limiting its scope. (It\n> must be a pointer in dir_struct or else callers would not be able\n> to create a dir_struct without using a pointer and an initializer\n> method.\n>\n> The major downside to this pointer approach is that the internal\n> struct needs to be initialized within API calls and somehow cleared\n> by all callers. The internal data could be initialized by the common\n> initializers read_directory() or fill_directory(). There is a\n> dir_clear() that _should_ be called by all callers (but I notice we\n> are leaking the struct in at least one place in add-interactive.c,\n> and likely others).\n>\n> This alternative adds some complexity to the structure, but\n> provides compiler-level guarantees that these internals are not used\n> outside of dir.c. I thought it worth exploring, even if we decide\n> that the complexity is not worth those guarantees.\n>\n\nAnother approach, if you don't mind structure pointer math is to\ncreate two structures:\n\na) the external public one in the public header file\n\nstruct dir_entry {\n  <public stuff>\n};\n\nb) a private structure in the source file:\n\nstruct dir_entry_private {\n  struct dir_entry entry;\n  <private stuff>\n};\n\nIn the source file you also define a macro/function that can take a\npointer to dir_entry and get a pointer to dir_entry_private:\n\nstruct dir_entry_private *dir_entry_to_private(struct dir_entry *entry)\n{\n  return (struct dir_entry_private *)(<calculate the offset that entry\ninside dir_entry_private is, then subtract that from entry pointer>))\n}\n\nIn Linux kernel this is container_of, not sure if git has this already\ndefined and its a common pattern.\n\nThen you can set the code up such that the only way to allocate a\ndir_entry is to call some function in the dir code. If a new entry\nneeds to be allocated, implement an alloc function that has the full\nprivate structure definition.\n\nThis way you don't need an extra field in the dir_entry struct at all,\nbut at the cost of requiring special allocations. It works great for\ncode where the only way to get a dir_entry is already some other\nfunctions that would ensure the private version is setup correctly.\n\n> The best news is that your existing series makes it easier to flip\n> to the internal pointer method in the future, since we can shift\n> the 'd->internal.member\" uses into \"d->internal->member\" in a\n> mechanical way. Thus, the change you are proposing does not lock us\n> into this approach if we change our minds later.\n>\n\nI think either approach is good. I like container_of because I'm quite\nused to it in low level kernel code and its a good way to provide\nprivate/public split abstractions there. The private pointer variation\nis also a common approach to this problem and I think it sounds like\nwe already use it in a few places. Its perhaps better for those\nreasons.\n\n> Thanks,\n> -Stolee\n"},{"id":"472680","messageId":"20230224223116.1508846-1-jonathantanmy@google.com","threadId":"59297","inReplyTo":"239b10e11812d99be587265c0a5e283da45ca315.1677143700.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 02/11] dir: add a usage note to exclude_per_dir","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-02-24T22:31:16Z","receivedAt":"2023-02-24T22:31:23Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"\"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> diff --git a/dir.h b/dir.h\n> index 33fd848fc8d..2196e12630c 100644\n> --- a/dir.h\n> +++ b/dir.h\n> @@ -295,8 +295,12 @@ struct dir_struct {\n>  \tstruct untracked_cache *untracked;\n>  \n>  \t/**\n> -\t * The name of the file to be read in each directory for excluded files\n> -\t * (typically `.gitignore`).\n> +\t * Deprecated: ls-files is the only allowed caller; all other callers\n> +\t * should leave this as NULL; it pre-dated the\n> +\t * setup_standard_excludes() mechanism that replaces this.\n> +\t *\n> +\t * This field tracks the name of the file to be read in each directory\n> +\t * for excluded files (typically `.gitignore`).\n>  \t */\n>  \tconst char *exclude_per_dir;\n\nI'm not sure what is meant by \"allowed caller\", but I wouldn't have\nexpected this to also mean that unpack-trees would need to know to\npropagate this from o->internal.dir to d in verify_clean_subdirectory.\n\nI would be OK with excluding this from the patch set - I think the other\nchanges can stand independent of this.\n"},{"id":"472681","messageId":"20230224223313.1509428-1-jonathantanmy@google.com","threadId":"59297","inReplyTo":"f5a58123034a708b49aac7076197ad2ccdd3cba1.1677143700.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 04/11] unpack-trees: clean up some flow control","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-02-24T22:33:13Z","receivedAt":"2023-02-24T22:33:19Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"\"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n>   * update_sparsity() has a check early on that will BUG() if\n>     o->skip_sparse_checkout is set; as such, there's no need to check\n>     for that condition again later in the code.  We can simply remove\n>     the check and its corresponding goto label.\n\n[snip]\n\n> @@ -2113,8 +2115,6 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n>  \t\tmemset(&pl, 0, sizeof(pl));\n>  \t\tfree_pattern_list = 1;\n>  \t\tpopulate_from_existing_patterns(o, &pl);\n> -\t\tif (o->skip_sparse_checkout)\n> -\t\t\tgoto skip_sparse_checkout;\n\nI've verified that indeed there is a prior check that\no->skip_sparse_checkout is not set (not visible in the diff).\n\nUp to here, everything looks good.\n"},{"id":"472682","messageId":"20230224223730.1510266-1-jonathantanmy@google.com","threadId":"59297","inReplyTo":"508837fc182b0adba96c3fcbd468f47f8f0a0aef.1677143700.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 05/11] sparse-checkout: avoid using internal API of unpack-trees","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-02-24T22:37:30Z","receivedAt":"2023-02-24T22:38:05Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"\"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\n> index c3738154918..4b7390ce367 100644\n> --- a/builtin/sparse-checkout.c\n> +++ b/builtin/sparse-checkout.c\n> @@ -219,14 +219,13 @@ static int update_working_directory(struct pattern_list *pl)\n>  \to.dst_index = r->index;\n>  \tindex_state_init(&o.result, r);\n>  \to.skip_sparse_checkout = 0;\n> -\to.pl = pl;\n>  \n>  \tsetup_work_tree();\n>  \n>  \trepo_hold_locked_index(r, &lock_file, LOCK_DIE_ON_ERROR);\n>  \n>  \tsetup_unpack_trees_porcelain(&o, \"sparse-checkout\");\n> -\tresult = update_sparsity(&o);\n> +\tresult = update_sparsity(&o, pl);\n\nThis makes sense - setup_unpack_trees_porcelain() does not use o.pl, so\nthere is no logic change.\n\n> @@ -2111,11 +2111,12 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n>  \ttrace_performance_enter();\n>  \n>  \t/* If we weren't given patterns, use the recorded ones */\n> -\tif (!o->pl) {\n> -\t\tmemset(&pl, 0, sizeof(pl));\n> +\tif (!pl) {\n>  \t\tfree_pattern_list = 1;\n> -\t\tpopulate_from_existing_patterns(o, &pl);\n> +\t\tpl = xcalloc(1, sizeof(*pl));\n> +\t\tpopulate_from_existing_patterns(o, pl);\n>  \t}\n> +\to->pl = pl;\n>  \n>  \t/* Expand sparse directories as needed */\n>  \texpand_index(o->src_index, o->pl);\n> @@ -2147,8 +2148,10 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n>  \n>  \tdisplay_warning_msgs(o);\n>  \to->show_all_errors = old_show_all_errors;\n> -\tif (free_pattern_list)\n> -\t\tclear_pattern_list(&pl);\n> +\tif (free_pattern_list) {\n> +\t\tclear_pattern_list(pl);\n> +\t\tfree(pl);\n> +\t}\n\nWhen free_pattern_list is true, we free \"pl\" which was previously\nassigned to \"o->pl\". Do we thus need to also clear \"o->pl\"?\n\n"},{"id":"472684","messageId":"20230224232256.2038749-1-jonathantanmy@google.com","threadId":"59297","inReplyTo":"8955b45e35474e5feb826101423470d0b51e5470.1677143700.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 06/11] sparse-checkout: avoid using internal API of unpack-trees, take 2","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-02-24T23:22:56Z","receivedAt":"2023-02-24T23:23:03Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"\"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> From: Elijah Newren <newren@gmail.com>\n> \n> Commit 2f6b1eb794 (\"cache API: add a \"INDEX_STATE_INIT\" macro/function,\n> add release_index()\", 2023-01-12) mistakenly added some initialization\n> of a member of unpack_trees_options that was intended to be\n> internal-only.  Further, it served no purpose as it simply duplicated\n> the initialization that unpack-trees.c code was already doing.\n> \n> Signed-off-by: Elijah Newren <newren@gmail.com>\n> ---\n>  builtin/sparse-checkout.c | 1 -\n>  1 file changed, 1 deletion(-)\n> \n> diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\n> index 4b7390ce367..8d5ae6f2a60 100644\n> --- a/builtin/sparse-checkout.c\n> +++ b/builtin/sparse-checkout.c\n> @@ -217,7 +217,6 @@ static int update_working_directory(struct pattern_list *pl)\n>  \to.head_idx = -1;\n>  \to.src_index = r->index;\n>  \to.dst_index = r->index;\n> -\tindex_state_init(&o.result, r);\n>  \to.skip_sparse_checkout = 0;\n>  \n>  \tsetup_work_tree();\n\nThe commit message seems to imply that in this code path, there is some\ncode in unpack-trees.c that runs index_state_init(), but that doesn't\nseem to be the case. memset-ting the result field with a junk value\ncauses valgrind to fail with the following trace:\n\n  ==2035705== Invalid read of size 8\n  ==2035705==    at 0x30D982: lazy_init_name_hash (name-hash.c:602)\n  ==2035705==    by 0x30DDDA: index_file_exists (name-hash.c:721)\n  ==2035705==    by 0x3F71A8: check_ok_to_remove (unpack-trees.c:2430)\n  ==2035705==    by 0x3F74EE: verify_absent_1 (unpack-trees.c:2495)\n  ==2035705==    by 0x3F75C6: verify_absent_sparse (unpack-trees.c:2523)\n  ==2035705==    by 0x3F2A15: apply_sparse_checkout (unpack-trees.c:566)\n  ==2035705==    by 0x3F6849: update_sparsity (unpack-trees.c:2147)\n  ==2035705==    by 0x1FC105: update_working_directory (sparse-checkout.c:228)\n\nso it might be better to move the init invocation to update_sparsity()\ninstead of only removing it.\n"},{"id":"472685","messageId":"20230224233607.2040752-1-jonathantanmy@google.com","threadId":"59297","inReplyTo":"pull.1149.git.1677143700.gitgitgadget@gmail.com","subject":"Re: [PATCH 00/11] Clarify API for dir.[ch] and unpack-trees.[ch] -- mark relevant fields as internal","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-02-24T23:36:07Z","receivedAt":"2023-02-24T23:36:13Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"\"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> I wrote this patch series about a year and a half ago, but never submitted\n> it. I rebased and updated it due to [0].\n\n[snip]\n\n> [0] Search for \"Extremely yes\" in\n> https://lore.kernel.org/git/CAJoAoZm+TkCL0Jpg_qFgKottxbtiG2QOiY0qGrz3-uQy+=waPg@mail.gmail.com/\n\nI left some minor comments, but otherwise, this looks good. I do share\nthe desire to avoid unnecessary refactoring churn, but demarcation\nof private and public fields does make code much clearer and can\npotentially avoid a whole class of bugs, so I would be happy if this\npatch set was merged in.\n"},{"id":"472692","messageId":"CABPp-BG4KChJa0Kj8n84QBUt5Rc6_rX2Fus3R4MbmUuF6RNuOQ@mail.gmail.com","threadId":"59297","inReplyTo":"20230224223116.1508846-1-jonathantanmy@google.com","subject":"Re: [PATCH 02/11] dir: add a usage note to exclude_per_dir","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2023-02-25T00:23:42Z","receivedAt":"2023-02-25T00:24:01Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Feb 24, 2023 at 2:31 PM Jonathan Tan <jonathantanmy@google.com> wrote:\n>\n> \"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> > diff --git a/dir.h b/dir.h\n> > index 33fd848fc8d..2196e12630c 100644\n> > --- a/dir.h\n> > +++ b/dir.h\n> > @@ -295,8 +295,12 @@ struct dir_struct {\n> >       struct untracked_cache *untracked;\n> >\n> >       /**\n> > -      * The name of the file to be read in each directory for excluded files\n> > -      * (typically `.gitignore`).\n> > +      * Deprecated: ls-files is the only allowed caller; all other callers\n> > +      * should leave this as NULL; it pre-dated the\n> > +      * setup_standard_excludes() mechanism that replaces this.\n> > +      *\n> > +      * This field tracks the name of the file to be read in each directory\n> > +      * for excluded files (typically `.gitignore`).\n> >        */\n> >       const char *exclude_per_dir;\n>\n> I'm not sure what is meant by \"allowed caller\", but I wouldn't have\n> expected this to also mean that unpack-trees would need to know to\n> propagate this from o->internal.dir to d in verify_clean_subdirectory.\n\nAre you confusing fields that are internal to dir, with fields that\nare internal to unpack-trees?\n\nThis series does not make exclude_per_dir an internal field within dir_struct.\n\nLater in this series, we do make the embedded dir_struct within\nunpack_trees_options as internal.  Thus every access of any field of\ndir within unpack_trees will have an \"internal.\" prefix, but that has\nnothing to do with this patch and would still be true even if this\npatch were dropped.\n\n> I would be OK with excluding this from the patch set - I think the other\n> changes can stand independent of this.\n\nTrying to get a consistent look and feel between commands is\nimportant.  The \"setup_standard_excludes()\" is one small area where\nwe've achieved that, and it's shared by _almost_ all commands\n(builtin/ls-files being the only exception and it persists for\nbackward compatibility reasons).  So I definitely want to keep the\ndeprecation notice and warn people away from using this field.  That's\nwhat this patch is for.\n"},{"id":"472693","messageId":"CABPp-BHhWpo532fqO_cuVrTJX550UrEv0ZFCf003HjLFQ+wfFg@mail.gmail.com","threadId":"59297","inReplyTo":"20230224223730.1510266-1-jonathantanmy@google.com","subject":"Re: [PATCH 05/11] sparse-checkout: avoid using internal API of unpack-trees","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2023-02-25T00:33:02Z","receivedAt":"2023-02-25T00:33:21Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Feb 24, 2023 at 2:37 PM Jonathan Tan <jonathantanmy@google.com> wrote:\n>\n> \"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> > diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\n> > index c3738154918..4b7390ce367 100644\n> > --- a/builtin/sparse-checkout.c\n> > +++ b/builtin/sparse-checkout.c\n> > @@ -219,14 +219,13 @@ static int update_working_directory(struct pattern_list *pl)\n> >       o.dst_index = r->index;\n> >       index_state_init(&o.result, r);\n> >       o.skip_sparse_checkout = 0;\n> > -     o.pl = pl;\n> >\n> >       setup_work_tree();\n> >\n> >       repo_hold_locked_index(r, &lock_file, LOCK_DIE_ON_ERROR);\n> >\n> >       setup_unpack_trees_porcelain(&o, \"sparse-checkout\");\n> > -     result = update_sparsity(&o);\n> > +     result = update_sparsity(&o, pl);\n>\n> This makes sense - setup_unpack_trees_porcelain() does not use o.pl, so\n> there is no logic change.\n>\n> > @@ -2111,11 +2111,12 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n> >       trace_performance_enter();\n> >\n> >       /* If we weren't given patterns, use the recorded ones */\n> > -     if (!o->pl) {\n> > -             memset(&pl, 0, sizeof(pl));\n> > +     if (!pl) {\n> >               free_pattern_list = 1;\n> > -             populate_from_existing_patterns(o, &pl);\n> > +             pl = xcalloc(1, sizeof(*pl));\n> > +             populate_from_existing_patterns(o, pl);\n> >       }\n> > +     o->pl = pl;\n> >\n> >       /* Expand sparse directories as needed */\n> >       expand_index(o->src_index, o->pl);\n> > @@ -2147,8 +2148,10 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n> >\n> >       display_warning_msgs(o);\n> >       o->show_all_errors = old_show_all_errors;\n> > -     if (free_pattern_list)\n> > -             clear_pattern_list(&pl);\n> > +     if (free_pattern_list) {\n> > +             clear_pattern_list(pl);\n> > +             free(pl);\n> > +     }\n>\n> When free_pattern_list is true, we free \"pl\" which was previously\n> assigned to \"o->pl\". Do we thus need to also clear \"o->pl\"?\n\nYeah, I don't think the existing code will ever attempt to use o->pl\nagain under these circumstances, but setting it to NULL for\nfuture-proofing makes sense.  I'll make that tweak; thanks for reading\ncarefully!\n"},{"id":"472694","messageId":"CABPp-BEbZZjAE8N4OgWMEiV=MpGhEef0G4zytTTwrEjpGq0Qfw@mail.gmail.com","threadId":"59297","inReplyTo":"20230224232256.2038749-1-jonathantanmy@google.com","subject":"Re: [PATCH 06/11] sparse-checkout: avoid using internal API of unpack-trees, take 2","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2023-02-25T00:40:43Z","receivedAt":"2023-02-25T00:41:01Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Feb 24, 2023 at 3:22 PM Jonathan Tan <jonathantanmy@google.com> wrote:\n>\n> \"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> > From: Elijah Newren <newren@gmail.com>\n> >\n> > Commit 2f6b1eb794 (\"cache API: add a \"INDEX_STATE_INIT\" macro/function,\n> > add release_index()\", 2023-01-12) mistakenly added some initialization\n> > of a member of unpack_trees_options that was intended to be\n> > internal-only.  Further, it served no purpose as it simply duplicated\n> > the initialization that unpack-trees.c code was already doing.\n> >\n> > Signed-off-by: Elijah Newren <newren@gmail.com>\n> > ---\n> >  builtin/sparse-checkout.c | 1 -\n> >  1 file changed, 1 deletion(-)\n> >\n> > diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\n> > index 4b7390ce367..8d5ae6f2a60 100644\n> > --- a/builtin/sparse-checkout.c\n> > +++ b/builtin/sparse-checkout.c\n> > @@ -217,7 +217,6 @@ static int update_working_directory(struct pattern_list *pl)\n> >       o.head_idx = -1;\n> >       o.src_index = r->index;\n> >       o.dst_index = r->index;\n> > -     index_state_init(&o.result, r);\n> >       o.skip_sparse_checkout = 0;\n> >\n> >       setup_work_tree();\n>\n> The commit message seems to imply that in this code path, there is some\n> code in unpack-trees.c that runs index_state_init(), but that doesn't\n> seem to be the case. memset-ting the result field with a junk value\n> causes valgrind to fail with the following trace:\n>\n>   ==2035705== Invalid read of size 8\n>   ==2035705==    at 0x30D982: lazy_init_name_hash (name-hash.c:602)\n>   ==2035705==    by 0x30DDDA: index_file_exists (name-hash.c:721)\n>   ==2035705==    by 0x3F71A8: check_ok_to_remove (unpack-trees.c:2430)\n>   ==2035705==    by 0x3F74EE: verify_absent_1 (unpack-trees.c:2495)\n>   ==2035705==    by 0x3F75C6: verify_absent_sparse (unpack-trees.c:2523)\n>   ==2035705==    by 0x3F2A15: apply_sparse_checkout (unpack-trees.c:566)\n>   ==2035705==    by 0x3F6849: update_sparsity (unpack-trees.c:2147)\n>   ==2035705==    by 0x1FC105: update_working_directory (sparse-checkout.c:228)\n>\n> so it might be better to move the init invocation to update_sparsity()\n> instead of only removing it.\n\nActually, my commit message was implying o->result is only used by\nunpack_trees() and not update_sparsity().  While that implication is\n*mostly* true, I forgot about check_ok_to_remove().  Doh!\n\nThanks for reviewing so carefully; I'll move the initialization to\nupdate_sparsity() instead.\n"},{"id":"472698","messageId":"20230225015432.2065788-1-jonathantanmy@google.com","threadId":"59297","inReplyTo":"CABPp-BG4KChJa0Kj8n84QBUt5Rc6_rX2Fus3R4MbmUuF6RNuOQ@mail.gmail.com","subject":"Re: [PATCH 02/11] dir: add a usage note to exclude_per_dir","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-02-25T01:54:32Z","receivedAt":"2023-02-25T01:55:20Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Elijah Newren <newren@gmail.com> writes:\n> On Fri, Feb 24, 2023 at 2:31 PM Jonathan Tan <jonathantanmy@google.com> wrote:\n> >\n> > \"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> > > diff --git a/dir.h b/dir.h\n> > > index 33fd848fc8d..2196e12630c 100644\n> > > --- a/dir.h\n> > > +++ b/dir.h\n> > > @@ -295,8 +295,12 @@ struct dir_struct {\n> > >       struct untracked_cache *untracked;\n> > >\n> > >       /**\n> > > -      * The name of the file to be read in each directory for excluded files\n> > > -      * (typically `.gitignore`).\n> > > +      * Deprecated: ls-files is the only allowed caller; all other callers\n> > > +      * should leave this as NULL; it pre-dated the\n> > > +      * setup_standard_excludes() mechanism that replaces this.\n> > > +      *\n> > > +      * This field tracks the name of the file to be read in each directory\n> > > +      * for excluded files (typically `.gitignore`).\n> > >        */\n> > >       const char *exclude_per_dir;\n> >\n> > I'm not sure what is meant by \"allowed caller\", but I wouldn't have\n> > expected this to also mean that unpack-trees would need to know to\n> > propagate this from o->internal.dir to d in verify_clean_subdirectory.\n> \n> Are you confusing fields that are internal to dir, with fields that\n> are internal to unpack-trees?\n> \n> This series does not make exclude_per_dir an internal field within dir_struct.\n\nAgreed, but the comment says that ls-files is the only allowed caller,\nand I would have expected that non-\"allowed callers\" would not need to\nwrite to exclude_per_dir. But in unpack-trees.c:\n\n  2346          if (o->internal.dir)                                                                                                       \n  2347                  d.exclude_per_dir = o->internal.dir->exclude_per_dir;\n\nBoth \"d\" and \"o->internal.dir\" are of type \"struct dir_struct\" (well,\none is not a pointer and one is). I would not have expected such non-\nls-files code to read or write from this field. (But if unpack-trees\nis considered part of ls-files and/or copying the same field to another\nstruct is not considered \"calling\", then this patch is fine, I guess.)\n"},{"id":"472699","messageId":"239b10e11812d99be587265c0a5e283da45ca315.1677291961.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v2.git.1677291960.gitgitgadget@gmail.com","subject":"[PATCH v2 02/11] dir: add a usage note to exclude_per_dir","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-25T02:25:51Z","receivedAt":"2023-02-25T02:26:12Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n dir.h | 8 ++++++--\n 1 file changed, 6 insertions(+), 2 deletions(-)\n\ndiff --git a/dir.h b/dir.h\nindex 33fd848fc8d..2196e12630c 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -295,8 +295,12 @@ struct dir_struct {\n \tstruct untracked_cache *untracked;\n \n \t/**\n-\t * The name of the file to be read in each directory for excluded files\n-\t * (typically `.gitignore`).\n+\t * Deprecated: ls-files is the only allowed caller; all other callers\n+\t * should leave this as NULL; it pre-dated the\n+\t * setup_standard_excludes() mechanism that replaces this.\n+\t *\n+\t * This field tracks the name of the file to be read in each directory\n+\t * for excluded files (typically `.gitignore`).\n \t */\n \tconst char *exclude_per_dir;\n \n-- \ngitgitgadget\n\n"},{"id":"472700","messageId":"pull.1149.v2.git.1677291960.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.git.1677143700.gitgitgadget@gmail.com","subject":"[PATCH v2 00/11] Clarify API for dir.[ch] and unpack-trees.[ch] -- mark relevant fields as internal","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-25T02:25:49Z","receivedAt":"2023-02-25T02:26:12Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Changes since v1 (thanks to Jonathan Tan for the careful reviews!)\n\n * Clear o->pl when freeing pl, to avoid risking use-after-free.\n * Initialize o->result in update_sparsity() since it is actually used (by\n   check_ok_to_remove()).\n\nSome time ago, I noticed that struct dir_struct and struct\nunpack_trees_options both have numerous fields meant for internal use only,\nmost of which are not marked as such. This has resulted in callers\naccidentally trying to initialize some of these fields, and in at least one\ncase required a fair amount of review to verify other changes were okay --\nreview that would have been simplified with the apriori knowledge that a\ncombination of multiple fields were internal-only[1]. Looking closer, I\nfound that only 6 out of 18 fields in dir_struct were actually meant to be\npublic[2], and noted that unpack_trees_options also had 11 internal-only\nfields (out of 36).\n\nThis patch is primarily about moving internal-only fields within these two\nstructs into an embedded internal struct. Patch breakdown:\n\n * Patches 1-3: Restructuring dir_struct\n   * Patch 1: Splitting off internal-use-only fields\n   * Patch 2: Add important usage note to avoid accidentally using\n     deprecated API\n   * Patch 3: Mark output-only fields as such\n * Patches 4-11: Restructuring unpack_trees_options\n   * Patches 4-6: Preparatory cleanup\n   * Patches 7-10: Splitting off internal-use-only fields\n   * Patch 11: Mark output-only field as such\n\nTo make the benefit more clear, here are compressed versions of dir_struct\nboth before and after the changes. First, before:\n\nstruct dir_struct {\n    int nr;\n    int alloc;\n    int ignored_nr;\n    int ignored_alloc;\n    enum [...] flags;\n    struct dir_entry **entries;\n    struct dir_entry **ignored;\n    const char *exclude_per_dir;\n#define EXC_CMDL 0\n#define EXC_DIRS 1\n#define EXC_FILE 2\n    struct exclude_list_group exclude_list_group[3];\n    struct exclude_stack *exclude_stack;\n    struct path_pattern *pattern;\n    struct strbuf basebuf;\n    struct untracked_cache *untracked;\n    struct oid_stat ss_info_exclude;\n    struct oid_stat ss_excludes_file;\n    unsigned unmanaged_exclude_files;\n    unsigned visited_paths;\n    unsigned visited_directories;\n};\n\n\nAnd after the changes:\n\nstruct dir_struct {\n    enum [...] flags;\n    int nr; /* output only */\n    int ignored_nr; /* output only */\n    struct dir_entry **entries; /* output only */\n    struct dir_entry **ignored; /* output only */\n    struct untracked_cache *untracked;\n    const char *exclude_per_dir; /* deprecated */\n    struct dir_struct_internal {\n        int alloc;\n        int ignored_alloc;\n#define EXC_CMDL 0\n#define EXC_DIRS 1\n#define EXC_FILE 2\n        struct exclude_list_group exclude_list_group[3];\n        struct exclude_stack *exclude_stack;\n        struct path_pattern *pattern;\n        struct strbuf basebuf;\n        struct oid_stat ss_info_exclude;\n        struct oid_stat ss_excludes_file;\n        unsigned unmanaged_exclude_files;\n        unsigned visited_paths;\n        unsigned visited_directories;\n    } internal;\n};\n\n\nThe former version has 18 fields (and 3 magic constants) which API users\nwill have to figure out. The latter makes it clear there are only at most 2\nfields you should be setting upon input, and at most 4 which you read at\noutput, and the rest (including all the magic constants) you can ignore.\n\n[0] Search for \"Extremely yes\" in\nhttps://lore.kernel.org/git/CAJoAoZm+TkCL0Jpg_qFgKottxbtiG2QOiY0qGrz3-uQy+=waPg@mail.gmail.com/\n[1]\nhttps://lore.kernel.org/git/CABPp-BFSFN3WM6q7KzkD5mhrwsz--St_-ej5LbaY8Yr2sZzj=w@mail.gmail.com/\n[2]\nhttps://lore.kernel.org/git/CABPp-BHgot=CPNyK_xNfog_SqsNPNoCGfiSb-gZoS2sn_741dQ@mail.gmail.com/\n\nElijah Newren (11):\n  dir: separate public from internal portion of dir_struct\n  dir: add a usage note to exclude_per_dir\n  dir: mark output only fields of dir_struct as such\n  unpack-trees: clean up some flow control\n  sparse-checkout: avoid using internal API of unpack-trees\n  sparse-checkout: avoid using internal API of unpack-trees, take 2\n  unpack_trees: start splitting internal fields from public API\n  unpack-trees: mark fields only used internally as internal\n  unpack-trees: rewrap a few overlong lines from previous patch\n  unpack-trees: special case read-tree debugging as internal usage\n  unpack-trees: add usage notices around df_conflict_entry\n\n builtin/read-tree.c       |  10 +-\n builtin/sparse-checkout.c |   4 +-\n dir.c                     | 114 +++++++++---------\n dir.h                     | 110 +++++++++--------\n unpack-trees.c            | 247 ++++++++++++++++++++------------------\n unpack-trees.h            |  42 ++++---\n 6 files changed, 277 insertions(+), 250 deletions(-)\n\n\nbase-commit: 06dd2baa8da4a73421b959ec026a43711b9d77f9\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1149%2Fnewren%2Fclarify-api-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1149/newren/clarify-api-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/1149\n\nRange-diff vs v1:\n\n  1:  7f59ad548d0 =  1:  7f59ad548d0 dir: separate public from internal portion of dir_struct\n  2:  239b10e1181 =  2:  239b10e1181 dir: add a usage note to exclude_per_dir\n  3:  b8aa14350d3 =  3:  b8aa14350d3 dir: mark output only fields of dir_struct as such\n  4:  f5a58123034 =  4:  f5a58123034 unpack-trees: clean up some flow control\n  5:  508837fc182 !  5:  975dec0f0eb sparse-checkout: avoid using internal API of unpack-trees\n     @@ unpack-trees.c: enum update_sparsity_result update_sparsity(struct unpack_trees_\n      +\tif (free_pattern_list) {\n      +\t\tclear_pattern_list(pl);\n      +\t\tfree(pl);\n     ++\t\to->pl = NULL;\n      +\t}\n       \ttrace_performance_leave(\"update_sparsity\");\n       \treturn ret;\n  6:  8955b45e354 !  6:  429f195dcfe sparse-checkout: avoid using internal API of unpack-trees, take 2\n     @@ Commit message\n          Commit 2f6b1eb794 (\"cache API: add a \"INDEX_STATE_INIT\" macro/function,\n          add release_index()\", 2023-01-12) mistakenly added some initialization\n          of a member of unpack_trees_options that was intended to be\n     -    internal-only.  Further, it served no purpose as it simply duplicated\n     -    the initialization that unpack-trees.c code was already doing.\n     +    internal-only.  This initialization should be done within\n     +    update_sparsity() instead.\n     +\n     +    Note that while o->result is mostly meant for unpack_trees() and\n     +    update_sparsity() mostly operates without o->result,\n     +    check_ok_to_remove() does consult it so we need to ensure it is properly\n     +    initialized.\n      \n          Signed-off-by: Elijah Newren <newren@gmail.com>\n      \n     @@ builtin/sparse-checkout.c: static int update_working_directory(struct pattern_li\n       \to.skip_sparse_checkout = 0;\n       \n       \tsetup_work_tree();\n     +\n     + ## unpack-trees.c ##\n     +@@ unpack-trees.c: enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n     + \n     + \told_show_all_errors = o->show_all_errors;\n     + \to->show_all_errors = 1;\n     ++\tindex_state_init(&o->result, o->src_index->repo);\n     + \n     + \t/* Sanity checks */\n     + \tif (!o->update || o->index_only || o->skip_sparse_checkout)\n  7:  63ee57478ed !  7:  993da584dbb unpack_trees: start splitting internal fields from public API\n     @@ unpack-trees.c: enum update_sparsity_result update_sparsity(struct unpack_trees_\n       \t\t\t       CE_NEW_SKIP_WORKTREE, o->verbose_update);\n       \n       \t/* Then loop over entries and update/remove as needed */\n     +@@ unpack-trees.c: enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n     + \tif (free_pattern_list) {\n     + \t\tclear_pattern_list(pl);\n     + \t\tfree(pl);\n     +-\t\to->pl = NULL;\n     ++\t\to->internal.pl = NULL;\n     + \t}\n     + \ttrace_performance_leave(\"update_sparsity\");\n     + \treturn ret;\n      @@ unpack-trees.c: static int verify_clean_subdirectory(const struct cache_entry *ce,\n       \tpathbuf = xstrfmt(\"%.*s/\", namelen, ce->name);\n       \n  8:  081578b3210 !  8:  8ecb24a45f0 unpack-trees: mark fields only used internally as internal\n     @@ unpack-trees.c: enum update_sparsity_result update_sparsity(struct unpack_trees_\n       \n      -\told_show_all_errors = o->show_all_errors;\n      -\to->show_all_errors = 1;\n     +-\tindex_state_init(&o->result, o->src_index->repo);\n      +\told_show_all_errors = o->internal.show_all_errors;\n      +\to->internal.show_all_errors = 1;\n     ++\tindex_state_init(&o->internal.result, o->src_index->repo);\n       \n       \t/* Sanity checks */\n       \tif (!o->update || o->index_only || o->skip_sparse_checkout)\n  9:  f492ab27b19 =  9:  36ca49c3624 unpack-trees: rewrap a few overlong lines from previous patch\n 10:  a5048ea00b2 = 10:  5af04d7fe23 unpack-trees: special case read-tree debugging as internal usage\n 11:  efec74c8b49 = 11:  c4f31237634 unpack-trees: add usage notices around df_conflict_entry\n\n-- \ngitgitgadget\n"},{"id":"472702","messageId":"7f59ad548d0cbeeffb3917765084418df520ec22.1677291961.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v2.git.1677291960.gitgitgadget@gmail.com","subject":"[PATCH v2 01/11] dir: separate public from internal portion of dir_struct","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-25T02:25:50Z","receivedAt":"2023-02-25T02:26:18Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nIn order to make it clearer to callers what portions of dir_struct are\npublic API, and avoid errors from them setting fields that are meant as\ninternal API, split the fields used for internal implementation reasons\ninto a separate embedded struct.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n dir.c | 114 +++++++++++++++++++++++++++++-----------------------------\n dir.h |  86 +++++++++++++++++++++++---------------------\n 2 files changed, 104 insertions(+), 96 deletions(-)\n\ndiff --git a/dir.c b/dir.c\nindex 4e99f0c868f..7adf242026e 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -1190,7 +1190,7 @@ struct pattern_list *add_pattern_list(struct dir_struct *dir,\n \tstruct pattern_list *pl;\n \tstruct exclude_list_group *group;\n \n-\tgroup = &dir->exclude_list_group[group_type];\n+\tgroup = &dir->internal.exclude_list_group[group_type];\n \tALLOC_GROW(group->pl, group->nr + 1, group->alloc);\n \tpl = &group->pl[group->nr++];\n \tmemset(pl, 0, sizeof(*pl));\n@@ -1211,7 +1211,7 @@ static void add_patterns_from_file_1(struct dir_struct *dir, const char *fname,\n \t * differently when dir->untracked is non-NULL.\n \t */\n \tif (!dir->untracked)\n-\t\tdir->unmanaged_exclude_files++;\n+\t\tdir->internal.unmanaged_exclude_files++;\n \tpl = add_pattern_list(dir, EXC_FILE, fname);\n \tif (add_patterns(fname, \"\", 0, pl, NULL, 0, oid_stat) < 0)\n \t\tdie(_(\"cannot use %s as an exclude file\"), fname);\n@@ -1219,7 +1219,7 @@ static void add_patterns_from_file_1(struct dir_struct *dir, const char *fname,\n \n void add_patterns_from_file(struct dir_struct *dir, const char *fname)\n {\n-\tdir->unmanaged_exclude_files++; /* see validate_untracked_cache() */\n+\tdir->internal.unmanaged_exclude_files++; /* see validate_untracked_cache() */\n \tadd_patterns_from_file_1(dir, fname, NULL);\n }\n \n@@ -1519,7 +1519,7 @@ static struct path_pattern *last_matching_pattern_from_lists(\n \tstruct exclude_list_group *group;\n \tstruct path_pattern *pattern;\n \tfor (i = EXC_CMDL; i <= EXC_FILE; i++) {\n-\t\tgroup = &dir->exclude_list_group[i];\n+\t\tgroup = &dir->internal.exclude_list_group[i];\n \t\tfor (j = group->nr - 1; j >= 0; j--) {\n \t\t\tpattern = last_matching_pattern_from_list(\n \t\t\t\tpathname, pathlen, basename, dtype_p,\n@@ -1545,20 +1545,20 @@ static void prep_exclude(struct dir_struct *dir,\n \tstruct untracked_cache_dir *untracked;\n \tint current;\n \n-\tgroup = &dir->exclude_list_group[EXC_DIRS];\n+\tgroup = &dir->internal.exclude_list_group[EXC_DIRS];\n \n \t/*\n \t * Pop the exclude lists from the EXCL_DIRS exclude_list_group\n \t * which originate from directories not in the prefix of the\n \t * path being checked.\n \t */\n-\twhile ((stk = dir->exclude_stack) != NULL) {\n+\twhile ((stk = dir->internal.exclude_stack) != NULL) {\n \t\tif (stk->baselen <= baselen &&\n-\t\t    !strncmp(dir->basebuf.buf, base, stk->baselen))\n+\t\t    !strncmp(dir->internal.basebuf.buf, base, stk->baselen))\n \t\t\tbreak;\n-\t\tpl = &group->pl[dir->exclude_stack->exclude_ix];\n-\t\tdir->exclude_stack = stk->prev;\n-\t\tdir->pattern = NULL;\n+\t\tpl = &group->pl[dir->internal.exclude_stack->exclude_ix];\n+\t\tdir->internal.exclude_stack = stk->prev;\n+\t\tdir->internal.pattern = NULL;\n \t\tfree((char *)pl->src); /* see strbuf_detach() below */\n \t\tclear_pattern_list(pl);\n \t\tfree(stk);\n@@ -1566,7 +1566,7 @@ static void prep_exclude(struct dir_struct *dir,\n \t}\n \n \t/* Skip traversing into sub directories if the parent is excluded */\n-\tif (dir->pattern)\n+\tif (dir->internal.pattern)\n \t\treturn;\n \n \t/*\n@@ -1574,12 +1574,12 @@ static void prep_exclude(struct dir_struct *dir,\n \t * memset(dir, 0, sizeof(*dir)) before use. Changing all of\n \t * them seems lots of work for little benefit.\n \t */\n-\tif (!dir->basebuf.buf)\n-\t\tstrbuf_init(&dir->basebuf, PATH_MAX);\n+\tif (!dir->internal.basebuf.buf)\n+\t\tstrbuf_init(&dir->internal.basebuf, PATH_MAX);\n \n \t/* Read from the parent directories and push them down. */\n \tcurrent = stk ? stk->baselen : -1;\n-\tstrbuf_setlen(&dir->basebuf, current < 0 ? 0 : current);\n+\tstrbuf_setlen(&dir->internal.basebuf, current < 0 ? 0 : current);\n \tif (dir->untracked)\n \t\tuntracked = stk ? stk->ucd : dir->untracked->root;\n \telse\n@@ -1599,32 +1599,33 @@ static void prep_exclude(struct dir_struct *dir,\n \t\t\t\tdie(\"oops in prep_exclude\");\n \t\t\tcp++;\n \t\t\tuntracked =\n-\t\t\t\tlookup_untracked(dir->untracked, untracked,\n+\t\t\t\tlookup_untracked(dir->untracked,\n+\t\t\t\t\t\t untracked,\n \t\t\t\t\t\t base + current,\n \t\t\t\t\t\t cp - base - current);\n \t\t}\n-\t\tstk->prev = dir->exclude_stack;\n+\t\tstk->prev = dir->internal.exclude_stack;\n \t\tstk->baselen = cp - base;\n \t\tstk->exclude_ix = group->nr;\n \t\tstk->ucd = untracked;\n \t\tpl = add_pattern_list(dir, EXC_DIRS, NULL);\n-\t\tstrbuf_add(&dir->basebuf, base + current, stk->baselen - current);\n-\t\tassert(stk->baselen == dir->basebuf.len);\n+\t\tstrbuf_add(&dir->internal.basebuf, base + current, stk->baselen - current);\n+\t\tassert(stk->baselen == dir->internal.basebuf.len);\n \n \t\t/* Abort if the directory is excluded */\n \t\tif (stk->baselen) {\n \t\t\tint dt = DT_DIR;\n-\t\t\tdir->basebuf.buf[stk->baselen - 1] = 0;\n-\t\t\tdir->pattern = last_matching_pattern_from_lists(dir,\n+\t\t\tdir->internal.basebuf.buf[stk->baselen - 1] = 0;\n+\t\t\tdir->internal.pattern = last_matching_pattern_from_lists(dir,\n \t\t\t\t\t\t\t\t\tistate,\n-\t\t\t\tdir->basebuf.buf, stk->baselen - 1,\n-\t\t\t\tdir->basebuf.buf + current, &dt);\n-\t\t\tdir->basebuf.buf[stk->baselen - 1] = '/';\n-\t\t\tif (dir->pattern &&\n-\t\t\t    dir->pattern->flags & PATTERN_FLAG_NEGATIVE)\n-\t\t\t\tdir->pattern = NULL;\n-\t\t\tif (dir->pattern) {\n-\t\t\t\tdir->exclude_stack = stk;\n+\t\t\t\tdir->internal.basebuf.buf, stk->baselen - 1,\n+\t\t\t\tdir->internal.basebuf.buf + current, &dt);\n+\t\t\tdir->internal.basebuf.buf[stk->baselen - 1] = '/';\n+\t\t\tif (dir->internal.pattern &&\n+\t\t\t    dir->internal.pattern->flags & PATTERN_FLAG_NEGATIVE)\n+\t\t\t\tdir->internal.pattern = NULL;\n+\t\t\tif (dir->internal.pattern) {\n+\t\t\t\tdir->internal.exclude_stack = stk;\n \t\t\t\treturn;\n \t\t\t}\n \t\t}\n@@ -1647,15 +1648,15 @@ static void prep_exclude(struct dir_struct *dir,\n \t\t      */\n \t\t     !is_null_oid(&untracked->exclude_oid))) {\n \t\t\t/*\n-\t\t\t * dir->basebuf gets reused by the traversal, but we\n-\t\t\t * need fname to remain unchanged to ensure the src\n-\t\t\t * member of each struct path_pattern correctly\n+\t\t\t * dir->internal.basebuf gets reused by the traversal,\n+\t\t\t * but we need fname to remain unchanged to ensure the\n+\t\t\t * src member of each struct path_pattern correctly\n \t\t\t * back-references its source file.  Other invocations\n \t\t\t * of add_pattern_list provide stable strings, so we\n \t\t\t * strbuf_detach() and free() here in the caller.\n \t\t\t */\n \t\t\tstruct strbuf sb = STRBUF_INIT;\n-\t\t\tstrbuf_addbuf(&sb, &dir->basebuf);\n+\t\t\tstrbuf_addbuf(&sb, &dir->internal.basebuf);\n \t\t\tstrbuf_addstr(&sb, dir->exclude_per_dir);\n \t\t\tpl->src = strbuf_detach(&sb, NULL);\n \t\t\tadd_patterns(pl->src, pl->src, stk->baselen, pl, istate,\n@@ -1681,10 +1682,10 @@ static void prep_exclude(struct dir_struct *dir,\n \t\t\tinvalidate_gitignore(dir->untracked, untracked);\n \t\t\toidcpy(&untracked->exclude_oid, &oid_stat.oid);\n \t\t}\n-\t\tdir->exclude_stack = stk;\n+\t\tdir->internal.exclude_stack = stk;\n \t\tcurrent = stk->baselen;\n \t}\n-\tstrbuf_setlen(&dir->basebuf, baselen);\n+\tstrbuf_setlen(&dir->internal.basebuf, baselen);\n }\n \n /*\n@@ -1704,8 +1705,8 @@ struct path_pattern *last_matching_pattern(struct dir_struct *dir,\n \n \tprep_exclude(dir, istate, pathname, basename-pathname);\n \n-\tif (dir->pattern)\n-\t\treturn dir->pattern;\n+\tif (dir->internal.pattern)\n+\t\treturn dir->internal.pattern;\n \n \treturn last_matching_pattern_from_lists(dir, istate, pathname, pathlen,\n \t\t\tbasename, dtype_p);\n@@ -1742,7 +1743,7 @@ static struct dir_entry *dir_add_name(struct dir_struct *dir,\n \tif (index_file_exists(istate, pathname, len, ignore_case))\n \t\treturn NULL;\n \n-\tALLOC_GROW(dir->entries, dir->nr+1, dir->alloc);\n+\tALLOC_GROW(dir->entries, dir->nr+1, dir->internal.alloc);\n \treturn dir->entries[dir->nr++] = dir_entry_new(pathname, len);\n }\n \n@@ -1753,7 +1754,7 @@ struct dir_entry *dir_add_ignored(struct dir_struct *dir,\n \tif (!index_name_is_other(istate, pathname, len))\n \t\treturn NULL;\n \n-\tALLOC_GROW(dir->ignored, dir->ignored_nr+1, dir->ignored_alloc);\n+\tALLOC_GROW(dir->ignored, dir->ignored_nr+1, dir->internal.ignored_alloc);\n \treturn dir->ignored[dir->ignored_nr++] = dir_entry_new(pathname, len);\n }\n \n@@ -2569,7 +2570,7 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,\n \n \tif (open_cached_dir(&cdir, dir, untracked, istate, &path, check_only))\n \t\tgoto out;\n-\tdir->visited_directories++;\n+\tdir->internal.visited_directories++;\n \n \tif (untracked)\n \t\tuntracked->check_only = !!check_only;\n@@ -2578,7 +2579,7 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,\n \t\t/* check how the file or directory should be treated */\n \t\tstate = treat_path(dir, untracked, &cdir, istate, &path,\n \t\t\t\t   baselen, pathspec);\n-\t\tdir->visited_paths++;\n+\t\tdir->internal.visited_paths++;\n \n \t\tif (state > dir_state)\n \t\t\tdir_state = state;\n@@ -2586,7 +2587,8 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,\n \t\t/* recurse into subdir if instructed by treat_path */\n \t\tif (state == path_recurse) {\n \t\t\tstruct untracked_cache_dir *ud;\n-\t\t\tud = lookup_untracked(dir->untracked, untracked,\n+\t\t\tud = lookup_untracked(dir->untracked,\n+\t\t\t\t\t      untracked,\n \t\t\t\t\t      path.buf + baselen,\n \t\t\t\t\t      path.len - baselen);\n \t\t\tsubdir_state =\n@@ -2846,7 +2848,7 @@ static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct *d\n \t * condition also catches running setup_standard_excludes()\n \t * before setting dir->untracked!\n \t */\n-\tif (dir->unmanaged_exclude_files)\n+\tif (dir->internal.unmanaged_exclude_files)\n \t\treturn NULL;\n \n \t/*\n@@ -2875,7 +2877,7 @@ static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct *d\n \t * EXC_CMDL is not considered in the cache. If people set it,\n \t * skip the cache.\n \t */\n-\tif (dir->exclude_list_group[EXC_CMDL].nr)\n+\tif (dir->internal.exclude_list_group[EXC_CMDL].nr)\n \t\treturn NULL;\n \n \tif (!ident_in_untracked(dir->untracked)) {\n@@ -2935,15 +2937,15 @@ static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct *d\n \n \t/* Validate $GIT_DIR/info/exclude and core.excludesfile */\n \troot = dir->untracked->root;\n-\tif (!oideq(&dir->ss_info_exclude.oid,\n+\tif (!oideq(&dir->internal.ss_info_exclude.oid,\n \t\t   &dir->untracked->ss_info_exclude.oid)) {\n \t\tinvalidate_gitignore(dir->untracked, root);\n-\t\tdir->untracked->ss_info_exclude = dir->ss_info_exclude;\n+\t\tdir->untracked->ss_info_exclude = dir->internal.ss_info_exclude;\n \t}\n-\tif (!oideq(&dir->ss_excludes_file.oid,\n+\tif (!oideq(&dir->internal.ss_excludes_file.oid,\n \t\t   &dir->untracked->ss_excludes_file.oid)) {\n \t\tinvalidate_gitignore(dir->untracked, root);\n-\t\tdir->untracked->ss_excludes_file = dir->ss_excludes_file;\n+\t\tdir->untracked->ss_excludes_file = dir->internal.ss_excludes_file;\n \t}\n \n \t/* Make sure this directory is not dropped out at saving phase */\n@@ -2969,9 +2971,9 @@ static void emit_traversal_statistics(struct dir_struct *dir,\n \t}\n \n \ttrace2_data_intmax(\"read_directory\", repo,\n-\t\t\t   \"directories-visited\", dir->visited_directories);\n+\t\t\t   \"directories-visited\", dir->internal.visited_directories);\n \ttrace2_data_intmax(\"read_directory\", repo,\n-\t\t\t   \"paths-visited\", dir->visited_paths);\n+\t\t\t   \"paths-visited\", dir->internal.visited_paths);\n \n \tif (!dir->untracked)\n \t\treturn;\n@@ -2993,8 +2995,8 @@ int read_directory(struct dir_struct *dir, struct index_state *istate,\n \tstruct untracked_cache_dir *untracked;\n \n \ttrace2_region_enter(\"dir\", \"read_directory\", istate->repo);\n-\tdir->visited_paths = 0;\n-\tdir->visited_directories = 0;\n+\tdir->internal.visited_paths = 0;\n+\tdir->internal.visited_directories = 0;\n \n \tif (has_symlink_leading_path(path, len)) {\n \t\ttrace2_region_leave(\"dir\", \"read_directory\", istate->repo);\n@@ -3342,14 +3344,14 @@ void setup_standard_excludes(struct dir_struct *dir)\n \t\texcludes_file = xdg_config_home(\"ignore\");\n \tif (excludes_file && !access_or_warn(excludes_file, R_OK, 0))\n \t\tadd_patterns_from_file_1(dir, excludes_file,\n-\t\t\t\t\t dir->untracked ? &dir->ss_excludes_file : NULL);\n+\t\t\t\t\t dir->untracked ? &dir->internal.ss_excludes_file : NULL);\n \n \t/* per repository user preference */\n \tif (startup_info->have_repository) {\n \t\tconst char *path = git_path_info_exclude();\n \t\tif (!access_or_warn(path, R_OK, 0))\n \t\t\tadd_patterns_from_file_1(dir, path,\n-\t\t\t\t\t\t dir->untracked ? &dir->ss_info_exclude : NULL);\n+\t\t\t\t\t\t dir->untracked ? &dir->internal.ss_info_exclude : NULL);\n \t}\n }\n \n@@ -3405,7 +3407,7 @@ void dir_clear(struct dir_struct *dir)\n \tstruct dir_struct new = DIR_INIT;\n \n \tfor (i = EXC_CMDL; i <= EXC_FILE; i++) {\n-\t\tgroup = &dir->exclude_list_group[i];\n+\t\tgroup = &dir->internal.exclude_list_group[i];\n \t\tfor (j = 0; j < group->nr; j++) {\n \t\t\tpl = &group->pl[j];\n \t\t\tif (i == EXC_DIRS)\n@@ -3422,13 +3424,13 @@ void dir_clear(struct dir_struct *dir)\n \tfree(dir->ignored);\n \tfree(dir->entries);\n \n-\tstk = dir->exclude_stack;\n+\tstk = dir->internal.exclude_stack;\n \twhile (stk) {\n \t\tstruct exclude_stack *prev = stk->prev;\n \t\tfree(stk);\n \t\tstk = prev;\n \t}\n-\tstrbuf_release(&dir->basebuf);\n+\tstrbuf_release(&dir->internal.basebuf);\n \n \tmemcpy(dir, &new, sizeof(*dir));\n }\ndiff --git a/dir.h b/dir.h\nindex 8acfc044181..33fd848fc8d 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -215,14 +215,9 @@ struct dir_struct {\n \t/* The number of members in `entries[]` array. */\n \tint nr;\n \n-\t/* Internal use; keeps track of allocation of `entries[]` array.*/\n-\tint alloc;\n-\n \t/* The number of members in `ignored[]` array. */\n \tint ignored_nr;\n \n-\tint ignored_alloc;\n-\n \t/* bit-field of options */\n \tenum {\n \n@@ -296,51 +291,62 @@ struct dir_struct {\n \t */\n \tstruct dir_entry **ignored;\n \n+\t/* Enable/update untracked file cache if set */\n+\tstruct untracked_cache *untracked;\n+\n \t/**\n \t * The name of the file to be read in each directory for excluded files\n \t * (typically `.gitignore`).\n \t */\n \tconst char *exclude_per_dir;\n \n-\t/*\n-\t * We maintain three groups of exclude pattern lists:\n-\t *\n-\t * EXC_CMDL lists patterns explicitly given on the command line.\n-\t * EXC_DIRS lists patterns obtained from per-directory ignore files.\n-\t * EXC_FILE lists patterns from fallback ignore files, e.g.\n-\t *   - .git/info/exclude\n-\t *   - core.excludesfile\n-\t *\n-\t * Each group contains multiple exclude lists, a single list\n-\t * per source.\n-\t */\n+\tstruct dir_struct_internal {\n+\t\t/* Keeps track of allocation of `entries[]` array.*/\n+\t\tint alloc;\n+\n+\t\t/* Keeps track of allocation of `ignored[]` array. */\n+\t\tint ignored_alloc;\n+\n+\t\t/*\n+\t\t * We maintain three groups of exclude pattern lists:\n+\t\t *\n+\t\t * EXC_CMDL lists patterns explicitly given on the command line.\n+\t\t * EXC_DIRS lists patterns obtained from per-directory ignore\n+\t\t *          files.\n+\t\t * EXC_FILE lists patterns from fallback ignore files, e.g.\n+\t\t *   - .git/info/exclude\n+\t\t *   - core.excludesfile\n+\t\t *\n+\t\t * Each group contains multiple exclude lists, a single list\n+\t\t * per source.\n+\t\t */\n #define EXC_CMDL 0\n #define EXC_DIRS 1\n #define EXC_FILE 2\n-\tstruct exclude_list_group exclude_list_group[3];\n-\n-\t/*\n-\t * Temporary variables which are used during loading of the\n-\t * per-directory exclude lists.\n-\t *\n-\t * exclude_stack points to the top of the exclude_stack, and\n-\t * basebuf contains the full path to the current\n-\t * (sub)directory in the traversal. Exclude points to the\n-\t * matching exclude struct if the directory is excluded.\n-\t */\n-\tstruct exclude_stack *exclude_stack;\n-\tstruct path_pattern *pattern;\n-\tstruct strbuf basebuf;\n-\n-\t/* Enable untracked file cache if set */\n-\tstruct untracked_cache *untracked;\n-\tstruct oid_stat ss_info_exclude;\n-\tstruct oid_stat ss_excludes_file;\n-\tunsigned unmanaged_exclude_files;\n+\t\tstruct exclude_list_group exclude_list_group[3];\n \n-\t/* Stats about the traversal */\n-\tunsigned visited_paths;\n-\tunsigned visited_directories;\n+\t\t/*\n+\t\t * Temporary variables which are used during loading of the\n+\t\t * per-directory exclude lists.\n+\t\t *\n+\t\t * exclude_stack points to the top of the exclude_stack, and\n+\t\t * basebuf contains the full path to the current\n+\t\t * (sub)directory in the traversal. Exclude points to the\n+\t\t * matching exclude struct if the directory is excluded.\n+\t\t */\n+\t\tstruct exclude_stack *exclude_stack;\n+\t\tstruct path_pattern *pattern;\n+\t\tstruct strbuf basebuf;\n+\n+\t\t/* Additional metadata related to 'untracked' */\n+\t\tstruct oid_stat ss_info_exclude;\n+\t\tstruct oid_stat ss_excludes_file;\n+\t\tunsigned unmanaged_exclude_files;\n+\n+\t\t/* Stats about the traversal */\n+\t\tunsigned visited_paths;\n+\t\tunsigned visited_directories;\n+\t} internal;\n };\n \n #define DIR_INIT { 0 }\n-- \ngitgitgadget\n\n"},{"id":"472701","messageId":"b8aa14350d3411f63d6f02c2e2a857fd342cd8ff.1677291961.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v2.git.1677291960.gitgitgadget@gmail.com","subject":"[PATCH v2 03/11] dir: mark output only fields of dir_struct as such","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-25T02:25:52Z","receivedAt":"2023-02-25T02:26:19Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nWhile at it, also group these fields together for convenience.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n dir.h | 16 ++++++++--------\n 1 file changed, 8 insertions(+), 8 deletions(-)\n\ndiff --git a/dir.h b/dir.h\nindex 2196e12630c..e8106e1ecac 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -212,12 +212,6 @@ struct untracked_cache {\n  */\n struct dir_struct {\n \n-\t/* The number of members in `entries[]` array. */\n-\tint nr;\n-\n-\t/* The number of members in `ignored[]` array. */\n-\tint ignored_nr;\n-\n \t/* bit-field of options */\n \tenum {\n \n@@ -282,14 +276,20 @@ struct dir_struct {\n \t\tDIR_SKIP_NESTED_GIT = 1<<9\n \t} flags;\n \n+\t/* The number of members in `entries[]` array. */\n+\tint nr; /* output only */\n+\n+\t/* The number of members in `ignored[]` array. */\n+\tint ignored_nr; /* output only */\n+\n \t/* An array of `struct dir_entry`, each element of which describes a path. */\n-\tstruct dir_entry **entries;\n+\tstruct dir_entry **entries; /* output only */\n \n \t/**\n \t * used for ignored paths with the `DIR_SHOW_IGNORED_TOO` and\n \t * `DIR_COLLECT_IGNORED` flags.\n \t */\n-\tstruct dir_entry **ignored;\n+\tstruct dir_entry **ignored; /* output only */\n \n \t/* Enable/update untracked file cache if set */\n \tstruct untracked_cache *untracked;\n-- \ngitgitgadget\n\n"},{"id":"472703","messageId":"f5a58123034a708b49aac7076197ad2ccdd3cba1.1677291961.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v2.git.1677291960.gitgitgadget@gmail.com","subject":"[PATCH v2 04/11] unpack-trees: clean up some flow control","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-25T02:25:53Z","receivedAt":"2023-02-25T02:26:19Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nThe update_sparsity() function was introduced in commit 7af7a25853\n(\"unpack-trees: add a new update_sparsity() function\", 2020-03-27).\nPrior to that, unpack_trees() was used, but that had a few bugs because\nthe needs of the caller were different, and different enough that\nunpack_trees() could not easily be modified to handle both usecases.\n\nThe implementation detail that update_sparsity() was written by copying\nunpack_trees() and then streamlining it, and then modifying it in the\nneeded ways still shows through in that there are leftover vestiges in\nboth functions that are no longer needed.  Clean them up.  In\nparticular:\n\n  * update_sparsity() allows a pattern list to be passed in, but\n    unpack_trees() never should use a different pattern list.  Add a\n    check and a BUG() if this gets violated.\n  * update_sparsity() has a check early on that will BUG() if\n    o->skip_sparse_checkout is set; as such, there's no need to check\n    for that condition again later in the code.  We can simply remove\n    the check and its corresponding goto label.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 7 +++----\n 1 file changed, 3 insertions(+), 4 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 3d05e45a279..0887d157df4 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1873,6 +1873,8 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tdie(\"unpack_trees takes at most %d trees\", MAX_UNPACK_TREES);\n \tif (o->dir)\n \t\tBUG(\"o->dir is for internal use only\");\n+\tif (o->pl)\n+\t\tBUG(\"o->pl is for internal use only\");\n \n \ttrace_performance_enter();\n \ttrace2_region_enter(\"unpack_trees\", \"unpack_trees\", the_repository);\n@@ -1899,7 +1901,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \n \tif (!core_apply_sparse_checkout || !o->update)\n \t\to->skip_sparse_checkout = 1;\n-\tif (!o->skip_sparse_checkout && !o->pl) {\n+\tif (!o->skip_sparse_checkout) {\n \t\tmemset(&pl, 0, sizeof(pl));\n \t\tfree_pattern_list = 1;\n \t\tpopulate_from_existing_patterns(o, &pl);\n@@ -2113,8 +2115,6 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n \t\tmemset(&pl, 0, sizeof(pl));\n \t\tfree_pattern_list = 1;\n \t\tpopulate_from_existing_patterns(o, &pl);\n-\t\tif (o->skip_sparse_checkout)\n-\t\t\tgoto skip_sparse_checkout;\n \t}\n \n \t/* Expand sparse directories as needed */\n@@ -2142,7 +2142,6 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n \t\t\tret = UPDATE_SPARSITY_WARNINGS;\n \t}\n \n-skip_sparse_checkout:\n \tif (check_updates(o, o->src_index))\n \t\tret = UPDATE_SPARSITY_WORKTREE_UPDATE_FAILURES;\n \n-- \ngitgitgadget\n\n"},{"id":"472704","messageId":"975dec0f0ebaf72f0e5f5b5379f3d1b761045d2f.1677291961.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v2.git.1677291960.gitgitgadget@gmail.com","subject":"[PATCH v2 05/11] sparse-checkout: avoid using internal API of unpack-trees","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-25T02:25:54Z","receivedAt":"2023-02-25T02:26:24Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nstruct unpack_trees_options has the following field and comment:\n\n\tstruct pattern_list *pl; /* for internal use */\n\nDespite the internal-use comment, commit e091228e17 (\"sparse-checkout:\nupdate working directory in-process\", 2019-11-21) starting setting this\nfield from an external caller.  At the time, the only way around that\nwould have been to modify unpack_trees() to take an extra pattern_list\nargument, and there's a lot of callers of that function.  However, when\nwe split update_sparsity() off as a separate function, with\nsparse-checkout being the sole caller, the need to update other callers\nwent away.  Fix this API problem by adding a pattern_list argument to\nupdate_sparsity() and stop setting the internal o.pl field directly.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n builtin/sparse-checkout.c |  3 +--\n unpack-trees.c            | 18 +++++++++++-------\n unpack-trees.h            |  3 ++-\n 3 files changed, 14 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex c3738154918..4b7390ce367 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -219,14 +219,13 @@ static int update_working_directory(struct pattern_list *pl)\n \to.dst_index = r->index;\n \tindex_state_init(&o.result, r);\n \to.skip_sparse_checkout = 0;\n-\to.pl = pl;\n \n \tsetup_work_tree();\n \n \trepo_hold_locked_index(r, &lock_file, LOCK_DIE_ON_ERROR);\n \n \tsetup_unpack_trees_porcelain(&o, \"sparse-checkout\");\n-\tresult = update_sparsity(&o);\n+\tresult = update_sparsity(&o, pl);\n \tclear_unpack_trees_porcelain(&o);\n \n \tif (result == UPDATE_SPARSITY_WARNINGS)\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 0887d157df4..639e48cc6bb 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -2091,10 +2091,10 @@ return_failed:\n  *\n  * CE_NEW_SKIP_WORKTREE is used internally.\n  */\n-enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n+enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n+\t\t\t\t\t    struct pattern_list *pl)\n {\n \tenum update_sparsity_result ret = UPDATE_SPARSITY_SUCCESS;\n-\tstruct pattern_list pl;\n \tint i;\n \tunsigned old_show_all_errors;\n \tint free_pattern_list = 0;\n@@ -2111,11 +2111,12 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n \ttrace_performance_enter();\n \n \t/* If we weren't given patterns, use the recorded ones */\n-\tif (!o->pl) {\n-\t\tmemset(&pl, 0, sizeof(pl));\n+\tif (!pl) {\n \t\tfree_pattern_list = 1;\n-\t\tpopulate_from_existing_patterns(o, &pl);\n+\t\tpl = xcalloc(1, sizeof(*pl));\n+\t\tpopulate_from_existing_patterns(o, pl);\n \t}\n+\to->pl = pl;\n \n \t/* Expand sparse directories as needed */\n \texpand_index(o->src_index, o->pl);\n@@ -2147,8 +2148,11 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n \n \tdisplay_warning_msgs(o);\n \to->show_all_errors = old_show_all_errors;\n-\tif (free_pattern_list)\n-\t\tclear_pattern_list(&pl);\n+\tif (free_pattern_list) {\n+\t\tclear_pattern_list(pl);\n+\t\tfree(pl);\n+\t\to->pl = NULL;\n+\t}\n \ttrace_performance_leave(\"update_sparsity\");\n \treturn ret;\n }\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 3a7b3e5f007..f3a6e4f90ef 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -112,7 +112,8 @@ enum update_sparsity_result {\n \tUPDATE_SPARSITY_WORKTREE_UPDATE_FAILURES = -2\n };\n \n-enum update_sparsity_result update_sparsity(struct unpack_trees_options *options);\n+enum update_sparsity_result update_sparsity(struct unpack_trees_options *options,\n+\t\t\t\t\t    struct pattern_list *pl);\n \n int verify_uptodate(const struct cache_entry *ce,\n \t\t    struct unpack_trees_options *o);\n-- \ngitgitgadget\n\n"},{"id":"472705","messageId":"429f195dcfedbe0a54e9a2b7bbbc6c126ca4e485.1677291961.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v2.git.1677291960.gitgitgadget@gmail.com","subject":"[PATCH v2 06/11] sparse-checkout: avoid using internal API of unpack-trees, take 2","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-25T02:25:55Z","receivedAt":"2023-02-25T02:26:29Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nCommit 2f6b1eb794 (\"cache API: add a \"INDEX_STATE_INIT\" macro/function,\nadd release_index()\", 2023-01-12) mistakenly added some initialization\nof a member of unpack_trees_options that was intended to be\ninternal-only.  This initialization should be done within\nupdate_sparsity() instead.\n\nNote that while o->result is mostly meant for unpack_trees() and\nupdate_sparsity() mostly operates without o->result,\ncheck_ok_to_remove() does consult it so we need to ensure it is properly\ninitialized.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n builtin/sparse-checkout.c | 1 -\n unpack-trees.c            | 1 +\n 2 files changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex 4b7390ce367..8d5ae6f2a60 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -217,7 +217,6 @@ static int update_working_directory(struct pattern_list *pl)\n \to.head_idx = -1;\n \to.src_index = r->index;\n \to.dst_index = r->index;\n-\tindex_state_init(&o.result, r);\n \to.skip_sparse_checkout = 0;\n \n \tsetup_work_tree();\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 639e48cc6bb..4fca051cbea 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -2101,6 +2101,7 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n \n \told_show_all_errors = o->show_all_errors;\n \to->show_all_errors = 1;\n+\tindex_state_init(&o->result, o->src_index->repo);\n \n \t/* Sanity checks */\n \tif (!o->update || o->index_only || o->skip_sparse_checkout)\n-- \ngitgitgadget\n\n"},{"id":"472706","messageId":"36ca49c36242384a74c288389c27e5690fc73565.1677291961.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v2.git.1677291960.gitgitgadget@gmail.com","subject":"[PATCH v2 09/11] unpack-trees: rewrap a few overlong lines from previous patch","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-25T02:25:58Z","receivedAt":"2023-02-25T02:26:30Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nThe previous patch made many lines a little longer, resulting in four\nbecoming a bit too long.  They were left as-is for the previous patch\nto facilitate reviewers verifying that we were just adding \"internal.\"\nin a bunch of places, but rewrap them now.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 16 +++++++++-------\n 1 file changed, 9 insertions(+), 7 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex f5294194aa1..985896d6af6 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1213,8 +1213,8 @@ static int unpack_single_entry(int n, unsigned long mask,\n \t\t * cache entry from the index aware logic.\n \t\t */\n \t\tsrc[i + o->merge] = create_ce_entry(info, names + i, stage,\n-\t\t\t\t\t\t    &o->internal.result, o->merge,\n-\t\t\t\t\t\t    bit & dirmask);\n+\t\t\t\t\t\t    &o->internal.result,\n+\t\t\t\t\t\t    o->merge, bit & dirmask);\n \t}\n \n \tif (o->merge) {\n@@ -1918,14 +1918,15 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t} else if (o->src_index == o->dst_index) {\n \t\t/*\n \t\t * o->dst_index (and thus o->src_index) will be discarded\n-\t\t * and overwritten with o->internal.result at the end of this function,\n-\t\t * so just use src_index's split_index to avoid having to\n-\t\t * create a new one.\n+\t\t * and overwritten with o->internal.result at the end of\n+\t\t * this function, so just use src_index's split_index to\n+\t\t * avoid having to create a new one.\n \t\t */\n \t\to->internal.result.split_index = o->src_index->split_index;\n \t\to->internal.result.split_index->refcount++;\n \t} else {\n-\t\to->internal.result.split_index = init_split_index(&o->internal.result);\n+\t\to->internal.result.split_index =\n+\t\t\tinit_split_index(&o->internal.result);\n \t}\n \toidcpy(&o->internal.result.oid, &o->src_index->oid);\n \to->merge_size = len;\n@@ -2049,7 +2050,8 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tmove_index_extensions(&o->internal.result, o->src_index);\n \t\tif (!ret) {\n \t\t\tif (git_env_bool(\"GIT_TEST_CHECK_CACHE_TREE\", 0))\n-\t\t\t\tcache_tree_verify(the_repository, &o->internal.result);\n+\t\t\t\tcache_tree_verify(the_repository,\n+\t\t\t\t\t\t  &o->internal.result);\n \t\t\tif (!o->skip_cache_tree_update &&\n \t\t\t    !cache_tree_fully_valid(o->internal.result.cache_tree))\n \t\t\t\tcache_tree_update(&o->internal.result,\n-- \ngitgitgadget\n\n"},{"id":"472707","messageId":"993da584dbb92b682e86d5ce63451c8203eeecd7.1677291961.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v2.git.1677291960.gitgitgadget@gmail.com","subject":"[PATCH v2 07/11] unpack_trees: start splitting internal fields from public API","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-25T02:25:56Z","receivedAt":"2023-02-25T02:26:32Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nThis just splits the two fields already marked as internal-only into a\nseparate internal struct.  Future commits will add more fields that\nwere meant to be internal-only but were not explicitly marked as such\nto the same struct.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 42 +++++++++++++++++++++---------------------\n unpack-trees.h |  7 +++++--\n 2 files changed, 26 insertions(+), 23 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 4fca051cbea..c659af67c62 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1809,7 +1809,7 @@ static void populate_from_existing_patterns(struct unpack_trees_options *o,\n \tif (get_sparse_checkout_patterns(pl) < 0)\n \t\to->skip_sparse_checkout = 1;\n \telse\n-\t\to->pl = pl;\n+\t\to->internal.pl = pl;\n }\n \n static void update_sparsity_for_prefix(const char *prefix,\n@@ -1871,10 +1871,10 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \n \tif (len > MAX_UNPACK_TREES)\n \t\tdie(\"unpack_trees takes at most %d trees\", MAX_UNPACK_TREES);\n-\tif (o->dir)\n-\t\tBUG(\"o->dir is for internal use only\");\n-\tif (o->pl)\n-\t\tBUG(\"o->pl is for internal use only\");\n+\tif (o->internal.dir)\n+\t\tBUG(\"o->internal.dir is for internal use only\");\n+\tif (o->internal.pl)\n+\t\tBUG(\"o->internal.pl is for internal use only\");\n \n \ttrace_performance_enter();\n \ttrace2_region_enter(\"unpack_trees\", \"unpack_trees\", the_repository);\n@@ -1891,9 +1891,9 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tBUG(\"UNPACK_RESET_OVERWRITE_UNTRACKED incompatible with preserved ignored files\");\n \n \tif (!o->preserve_ignored) {\n-\t\to->dir = &dir;\n-\t\to->dir->flags |= DIR_SHOW_IGNORED;\n-\t\tsetup_standard_excludes(o->dir);\n+\t\to->internal.dir = &dir;\n+\t\to->internal.dir->flags |= DIR_SHOW_IGNORED;\n+\t\tsetup_standard_excludes(o->internal.dir);\n \t}\n \n \tif (o->prefix)\n@@ -1943,7 +1943,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t * Sparse checkout loop #1: set NEW_SKIP_WORKTREE on existing entries\n \t */\n \tif (!o->skip_sparse_checkout)\n-\t\tmark_new_skip_worktree(o->pl, o->src_index, 0,\n+\t\tmark_new_skip_worktree(o->internal.pl, o->src_index, 0,\n \t\t\t\t       CE_NEW_SKIP_WORKTREE, o->verbose_update);\n \n \tif (!dfc)\n@@ -2009,7 +2009,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t * If they will have NEW_SKIP_WORKTREE, also set CE_SKIP_WORKTREE\n \t\t * so apply_sparse_checkout() won't attempt to remove it from worktree\n \t\t */\n-\t\tmark_new_skip_worktree(o->pl, &o->result,\n+\t\tmark_new_skip_worktree(o->internal.pl, &o->result,\n \t\t\t\t       CE_ADDED, CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE,\n \t\t\t\t       o->verbose_update);\n \n@@ -2067,9 +2067,9 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n done:\n \tif (free_pattern_list)\n \t\tclear_pattern_list(&pl);\n-\tif (o->dir) {\n-\t\tdir_clear(o->dir);\n-\t\to->dir = NULL;\n+\tif (o->internal.dir) {\n+\t\tdir_clear(o->internal.dir);\n+\t\to->internal.dir = NULL;\n \t}\n \ttrace2_region_leave(\"unpack_trees\", \"unpack_trees\", the_repository);\n \ttrace_performance_leave(\"unpack_trees\");\n@@ -2117,14 +2117,14 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n \t\tpl = xcalloc(1, sizeof(*pl));\n \t\tpopulate_from_existing_patterns(o, pl);\n \t}\n-\to->pl = pl;\n+\to->internal.pl = pl;\n \n \t/* Expand sparse directories as needed */\n-\texpand_index(o->src_index, o->pl);\n+\texpand_index(o->src_index, o->internal.pl);\n \n \t/* Set NEW_SKIP_WORKTREE on existing entries. */\n \tmark_all_ce_unused(o->src_index);\n-\tmark_new_skip_worktree(o->pl, o->src_index, 0,\n+\tmark_new_skip_worktree(o->internal.pl, o->src_index, 0,\n \t\t\t       CE_NEW_SKIP_WORKTREE, o->verbose_update);\n \n \t/* Then loop over entries and update/remove as needed */\n@@ -2152,7 +2152,7 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n \tif (free_pattern_list) {\n \t\tclear_pattern_list(pl);\n \t\tfree(pl);\n-\t\to->pl = NULL;\n+\t\to->internal.pl = NULL;\n \t}\n \ttrace_performance_leave(\"update_sparsity\");\n \treturn ret;\n@@ -2340,8 +2340,8 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n \tpathbuf = xstrfmt(\"%.*s/\", namelen, ce->name);\n \n \tmemset(&d, 0, sizeof(d));\n-\tif (o->dir)\n-\t\td.exclude_per_dir = o->dir->exclude_per_dir;\n+\tif (o->internal.dir)\n+\t\td.exclude_per_dir = o->internal.dir->exclude_per_dir;\n \ti = read_directory(&d, o->src_index, pathbuf, namelen+1, NULL);\n \tdir_clear(&d);\n \tfree(pathbuf);\n@@ -2395,8 +2395,8 @@ static int check_ok_to_remove(const char *name, int len, int dtype,\n \tif (ignore_case && icase_exists(o, name, len, st))\n \t\treturn 0;\n \n-\tif (o->dir &&\n-\t    is_excluded(o->dir, o->src_index, name, &dtype))\n+\tif (o->internal.dir &&\n+\t    is_excluded(o->internal.dir, o->src_index, name, &dtype))\n \t\t/*\n \t\t * ce->name is explicitly excluded, so it is Ok to\n \t\t * overwrite it.\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex f3a6e4f90ef..5c1a9314a06 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -97,9 +97,12 @@ struct unpack_trees_options {\n \tstruct index_state *src_index;\n \tstruct index_state result;\n \n-\tstruct pattern_list *pl; /* for internal use */\n-\tstruct dir_struct *dir; /* for internal use only */\n \tstruct checkout_metadata meta;\n+\n+\tstruct unpack_trees_options_internal {\n+\t\tstruct pattern_list *pl;\n+\t\tstruct dir_struct *dir;\n+\t} internal;\n };\n \n int unpack_trees(unsigned n, struct tree_desc *t,\n-- \ngitgitgadget\n\n"},{"id":"472708","messageId":"c4f31237634023f872ea4e68e53b6c54cdd5ca7a.1677291961.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v2.git.1677291960.gitgitgadget@gmail.com","subject":"[PATCH v2 11/11] unpack-trees: add usage notices around df_conflict_entry","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-25T02:26:00Z","receivedAt":"2023-02-25T02:26:33Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nAvoid making users believe they need to initialize df_conflict_entry\nto something (as happened with other output only fields before) with\na quick comment and a small sanity check.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 2 ++\n unpack-trees.h | 2 +-\n 2 files changed, 3 insertions(+), 1 deletion(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex e58f0f6a867..aafc5eca791 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1876,6 +1876,8 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tBUG(\"o->internal.dir is for internal use only\");\n \tif (o->internal.pl)\n \t\tBUG(\"o->internal.pl is for internal use only\");\n+\tif (o->df_conflict_entry)\n+\t\tBUG(\"o->df_conflict_entry is an output only field\");\n \n \ttrace_performance_enter();\n \ttrace2_region_enter(\"unpack_trees\", \"unpack_trees\", the_repository);\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex e8737adfeda..61c06eb7c50 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -78,7 +78,7 @@ struct unpack_trees_options {\n \n \tint head_idx;\n \n-\tstruct cache_entry *df_conflict_entry;\n+\tstruct cache_entry *df_conflict_entry; /* output only */\n \tvoid *unpack_data;\n \n \tstruct index_state *dst_index;\n-- \ngitgitgadget\n"},{"id":"472709","messageId":"5af04d7fe2334d426dc0ba59f74ce2b082436773.1677291961.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v2.git.1677291960.gitgitgadget@gmail.com","subject":"[PATCH v2 10/11] unpack-trees: special case read-tree debugging as internal usage","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-25T02:25:59Z","receivedAt":"2023-02-25T02:26:35Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nbuiltin/read-tree.c has some special functionality explicitly designed\nfor debugging unpack-trees.[ch].  Associated with that is two fields\nthat no other external caller would or should use.  Mark these as\ninternal to unpack-trees, but allow builtin/read-tree to read or write\nthem for this special case.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n builtin/read-tree.c | 10 +++++-----\n unpack-trees.c      | 22 +++++++++++-----------\n unpack-trees.h      |  6 +++---\n 3 files changed, 19 insertions(+), 19 deletions(-)\n\ndiff --git a/builtin/read-tree.c b/builtin/read-tree.c\nindex 3ce75417833..6034408d486 100644\n--- a/builtin/read-tree.c\n+++ b/builtin/read-tree.c\n@@ -87,9 +87,9 @@ static int debug_merge(const struct cache_entry * const *stages,\n {\n \tint i;\n \n-\tprintf(\"* %d-way merge\\n\", o->merge_size);\n+\tprintf(\"* %d-way merge\\n\", o->internal.merge_size);\n \tdebug_stage(\"index\", stages[0], o);\n-\tfor (i = 1; i <= o->merge_size; i++) {\n+\tfor (i = 1; i <= o->internal.merge_size; i++) {\n \t\tchar buf[24];\n \t\txsnprintf(buf, sizeof(buf), \"ent#%d\", i);\n \t\tdebug_stage(buf, stages[i], o);\n@@ -144,7 +144,7 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix)\n \t\tOPT__DRY_RUN(&opts.dry_run, N_(\"don't update the index or the work tree\")),\n \t\tOPT_BOOL(0, \"no-sparse-checkout\", &opts.skip_sparse_checkout,\n \t\t\t N_(\"skip applying sparse checkout filter\")),\n-\t\tOPT_BOOL(0, \"debug-unpack\", &opts.debug_unpack,\n+\t\tOPT_BOOL(0, \"debug-unpack\", &opts.internal.debug_unpack,\n \t\t\t N_(\"debug unpack-trees\")),\n \t\tOPT_CALLBACK_F(0, \"recurse-submodules\", NULL,\n \t\t\t    \"checkout\", \"control recursive updating of submodules\",\n@@ -247,7 +247,7 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix)\n \t\t\topts.head_idx = 1;\n \t}\n \n-\tif (opts.debug_unpack)\n+\tif (opts.internal.debug_unpack)\n \t\topts.fn = debug_merge;\n \n \t/* If we're going to prime_cache_tree later, skip cache tree update */\n@@ -263,7 +263,7 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix)\n \tif (unpack_trees(nr_trees, t, &opts))\n \t\treturn 128;\n \n-\tif (opts.debug_unpack || opts.dry_run)\n+\tif (opts.internal.debug_unpack || opts.dry_run)\n \t\treturn 0; /* do not write the index out */\n \n \t/*\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 985896d6af6..e58f0f6a867 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -839,7 +839,7 @@ static int traverse_by_cache_tree(int pos, int nr_entries, int nr_names,\n \t\tmark_ce_used(src[0], o);\n \t}\n \tfree(tree_ce);\n-\tif (o->debug_unpack)\n+\tif (o->internal.debug_unpack)\n \t\tprintf(\"Unpacked %d entries from %s to %s using cache-tree\\n\",\n \t\t       nr_entries,\n \t\t       o->src_index->cache[pos]->name,\n@@ -1488,7 +1488,7 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str\n \twhile (!p->mode)\n \t\tp++;\n \n-\tif (o->debug_unpack)\n+\tif (o->internal.debug_unpack)\n \t\tdebug_unpack_callback(n, mask, dirmask, names, info);\n \n \t/* Are we supposed to look at the index too? */\n@@ -1929,7 +1929,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t\tinit_split_index(&o->internal.result);\n \t}\n \toidcpy(&o->internal.result.oid, &o->src_index->oid);\n-\to->merge_size = len;\n+\to->internal.merge_size = len;\n \tmark_all_ce_unused(o->src_index);\n \n \to->internal.result.fsmonitor_last_update =\n@@ -2882,9 +2882,9 @@ int twoway_merge(const struct cache_entry * const *src,\n \tconst struct cache_entry *oldtree = src[1];\n \tconst struct cache_entry *newtree = src[2];\n \n-\tif (o->merge_size != 2)\n+\tif (o->internal.merge_size != 2)\n \t\treturn error(\"Cannot do a twoway merge of %d trees\",\n-\t\t\t     o->merge_size);\n+\t\t\t     o->internal.merge_size);\n \n \tif (oldtree == o->df_conflict_entry)\n \t\toldtree = NULL;\n@@ -2964,9 +2964,9 @@ int bind_merge(const struct cache_entry * const *src,\n \tconst struct cache_entry *old = src[0];\n \tconst struct cache_entry *a = src[1];\n \n-\tif (o->merge_size != 1)\n+\tif (o->internal.merge_size != 1)\n \t\treturn error(\"Cannot do a bind merge of %d trees\",\n-\t\t\t     o->merge_size);\n+\t\t\t     o->internal.merge_size);\n \tif (a && old)\n \t\treturn o->quiet ? -1 :\n \t\t\terror(ERRORMSG(o, ERROR_BIND_OVERLAP),\n@@ -2990,9 +2990,9 @@ int oneway_merge(const struct cache_entry * const *src,\n \tconst struct cache_entry *old = src[0];\n \tconst struct cache_entry *a = src[1];\n \n-\tif (o->merge_size != 1)\n+\tif (o->internal.merge_size != 1)\n \t\treturn error(\"Cannot do a oneway merge of %d trees\",\n-\t\t\t     o->merge_size);\n+\t\t\t     o->internal.merge_size);\n \n \tif (!a || a == o->df_conflict_entry)\n \t\treturn deleted_entry(old, old, o);\n@@ -3027,8 +3027,8 @@ int stash_worktree_untracked_merge(const struct cache_entry * const *src,\n \tconst struct cache_entry *worktree = src[1];\n \tconst struct cache_entry *untracked = src[2];\n \n-\tif (o->merge_size != 2)\n-\t\tBUG(\"invalid merge_size: %d\", o->merge_size);\n+\tif (o->internal.merge_size != 2)\n+\t\tBUG(\"invalid merge_size: %d\", o->internal.merge_size);\n \n \tif (worktree && untracked)\n \t\treturn error(_(\"worktree and untracked commit have duplicate entries: %s\"),\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 0335c89bc75..e8737adfeda 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -65,7 +65,6 @@ struct unpack_trees_options {\n \t\t     skip_unmerged,\n \t\t     initial_checkout,\n \t\t     diff_index_cached,\n-\t\t     debug_unpack,\n \t\t     skip_sparse_checkout,\n \t\t     quiet,\n \t\t     exiting_early,\n@@ -78,7 +77,6 @@ struct unpack_trees_options {\n \tmerge_fn_t fn;\n \n \tint head_idx;\n-\tint merge_size;\n \n \tstruct cache_entry *df_conflict_entry;\n \tvoid *unpack_data;\n@@ -90,8 +88,10 @@ struct unpack_trees_options {\n \n \tstruct unpack_trees_options_internal {\n \t\tunsigned int nontrivial_merge,\n-\t\t\t     show_all_errors;\n+\t\t\t     show_all_errors,\n+\t\t\t     debug_unpack; /* used by read-tree debugging */\n \n+\t\tint merge_size; /* used by read-tree debugging */\n \t\tint cache_bottom;\n \t\tconst char *msgs[NB_UNPACK_TREES_WARNING_TYPES];\n \t\tstruct strvec msgs_to_free;\n-- \ngitgitgadget\n\n"},{"id":"472710","messageId":"8ecb24a45f08e58c081a53557eb147b5e65a42cf.1677291961.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v2.git.1677291960.gitgitgadget@gmail.com","subject":"[PATCH v2 08/11] unpack-trees: mark fields only used internally as internal","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-25T02:25:57Z","receivedAt":"2023-02-25T02:26:37Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nContinue the work from the previous patch by finding additional fields\nwhich are only used internally but not yet explicitly marked as such,\nand include them in the internal fields struct.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 159 +++++++++++++++++++++++++------------------------\n unpack-trees.h |  26 ++++----\n 2 files changed, 95 insertions(+), 90 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex c659af67c62..f5294194aa1 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -66,8 +66,8 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_WARNING_TYPES] = {\n };\n \n #define ERRORMSG(o,type) \\\n-\t( ((o) && (o)->msgs[(type)]) \\\n-\t  ? ((o)->msgs[(type)])      \\\n+\t( ((o) && (o)->internal.msgs[(type)]) \\\n+\t  ? ((o)->internal.msgs[(type)])      \\\n \t  : (unpack_plumbing_errors[(type)]) )\n \n static const char *super_prefixed(const char *path, const char *super_prefix)\n@@ -108,10 +108,10 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t\t\t  const char *cmd)\n {\n \tint i;\n-\tconst char **msgs = opts->msgs;\n+\tconst char **msgs = opts->internal.msgs;\n \tconst char *msg;\n \n-\tstrvec_init(&opts->msgs_to_free);\n+\tstrvec_init(&opts->internal.msgs_to_free);\n \n \tif (!strcmp(cmd, \"checkout\"))\n \t\tmsg = advice_enabled(ADVICE_COMMIT_BEFORE_MERGE)\n@@ -129,7 +129,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t\t  \"Please commit your changes or stash them before you %s.\")\n \t\t      : _(\"Your local changes to the following files would be overwritten by %s:\\n%%s\");\n \tmsgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOT_UPTODATE_FILE] =\n-\t\tstrvec_pushf(&opts->msgs_to_free, msg, cmd, cmd);\n+\t\tstrvec_pushf(&opts->internal.msgs_to_free, msg, cmd, cmd);\n \n \tmsgs[ERROR_NOT_UPTODATE_DIR] =\n \t\t_(\"Updating the following directories would lose untracked files in them:\\n%s\");\n@@ -153,7 +153,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t\t  \"Please move or remove them before you %s.\")\n \t\t      : _(\"The following untracked working tree files would be removed by %s:\\n%%s\");\n \tmsgs[ERROR_WOULD_LOSE_UNTRACKED_REMOVED] =\n-\t\tstrvec_pushf(&opts->msgs_to_free, msg, cmd, cmd);\n+\t\tstrvec_pushf(&opts->internal.msgs_to_free, msg, cmd, cmd);\n \n \tif (!strcmp(cmd, \"checkout\"))\n \t\tmsg = advice_enabled(ADVICE_COMMIT_BEFORE_MERGE)\n@@ -171,7 +171,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t\t  \"Please move or remove them before you %s.\")\n \t\t      : _(\"The following untracked working tree files would be overwritten by %s:\\n%%s\");\n \tmsgs[ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN] =\n-\t\tstrvec_pushf(&opts->msgs_to_free, msg, cmd, cmd);\n+\t\tstrvec_pushf(&opts->internal.msgs_to_free, msg, cmd, cmd);\n \n \t/*\n \t * Special case: ERROR_BIND_OVERLAP refers to a pair of paths, we\n@@ -189,16 +189,16 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \tmsgs[WARNING_SPARSE_ORPHANED_NOT_OVERWRITTEN] =\n \t\t_(\"The following paths were already present and thus not updated despite sparse patterns:\\n%s\");\n \n-\topts->show_all_errors = 1;\n+\topts->internal.show_all_errors = 1;\n \t/* rejected paths may not have a static buffer */\n-\tfor (i = 0; i < ARRAY_SIZE(opts->unpack_rejects); i++)\n-\t\topts->unpack_rejects[i].strdup_strings = 1;\n+\tfor (i = 0; i < ARRAY_SIZE(opts->internal.unpack_rejects); i++)\n+\t\topts->internal.unpack_rejects[i].strdup_strings = 1;\n }\n \n void clear_unpack_trees_porcelain(struct unpack_trees_options *opts)\n {\n-\tstrvec_clear(&opts->msgs_to_free);\n-\tmemset(opts->msgs, 0, sizeof(opts->msgs));\n+\tstrvec_clear(&opts->internal.msgs_to_free);\n+\tmemset(opts->internal.msgs, 0, sizeof(opts->internal.msgs));\n }\n \n static int do_add_entry(struct unpack_trees_options *o, struct cache_entry *ce,\n@@ -210,7 +210,7 @@ static int do_add_entry(struct unpack_trees_options *o, struct cache_entry *ce,\n \t\tset |= CE_WT_REMOVE;\n \n \tce->ce_flags = (ce->ce_flags & ~clear) | set;\n-\treturn add_index_entry(&o->result, ce,\n+\treturn add_index_entry(&o->internal.result, ce,\n \t\t\t       ADD_CACHE_OK_TO_ADD | ADD_CACHE_OK_TO_REPLACE);\n }\n \n@@ -218,7 +218,7 @@ static void add_entry(struct unpack_trees_options *o,\n \t\t      const struct cache_entry *ce,\n \t\t      unsigned int set, unsigned int clear)\n {\n-\tdo_add_entry(o, dup_cache_entry(ce, &o->result), set, clear);\n+\tdo_add_entry(o, dup_cache_entry(ce, &o->internal.result), set, clear);\n }\n \n /*\n@@ -233,7 +233,7 @@ static int add_rejected_path(struct unpack_trees_options *o,\n \tif (o->quiet)\n \t\treturn -1;\n \n-\tif (!o->show_all_errors)\n+\tif (!o->internal.show_all_errors)\n \t\treturn error(ERRORMSG(o, e), super_prefixed(path,\n \t\t\t\t\t\t\t    o->super_prefix));\n \n@@ -241,7 +241,7 @@ static int add_rejected_path(struct unpack_trees_options *o,\n \t * Otherwise, insert in a list for future display by\n \t * display_(error|warning)_msgs()\n \t */\n-\tstring_list_append(&o->unpack_rejects[e], path);\n+\tstring_list_append(&o->internal.unpack_rejects[e], path);\n \treturn -1;\n }\n \n@@ -253,7 +253,7 @@ static void display_error_msgs(struct unpack_trees_options *o)\n \tint e;\n \tunsigned error_displayed = 0;\n \tfor (e = 0; e < NB_UNPACK_TREES_ERROR_TYPES; e++) {\n-\t\tstruct string_list *rejects = &o->unpack_rejects[e];\n+\t\tstruct string_list *rejects = &o->internal.unpack_rejects[e];\n \n \t\tif (rejects->nr > 0) {\n \t\t\tint i;\n@@ -281,7 +281,7 @@ static void display_warning_msgs(struct unpack_trees_options *o)\n \tunsigned warning_displayed = 0;\n \tfor (e = NB_UNPACK_TREES_ERROR_TYPES + 1;\n \t     e < NB_UNPACK_TREES_WARNING_TYPES; e++) {\n-\t\tstruct string_list *rejects = &o->unpack_rejects[e];\n+\t\tstruct string_list *rejects = &o->internal.unpack_rejects[e];\n \n \t\tif (rejects->nr > 0) {\n \t\t\tint i;\n@@ -600,13 +600,14 @@ static void mark_ce_used(struct cache_entry *ce, struct unpack_trees_options *o)\n {\n \tce->ce_flags |= CE_UNPACKED;\n \n-\tif (o->cache_bottom < o->src_index->cache_nr &&\n-\t    o->src_index->cache[o->cache_bottom] == ce) {\n-\t\tint bottom = o->cache_bottom;\n+\tif (o->internal.cache_bottom < o->src_index->cache_nr &&\n+\t    o->src_index->cache[o->internal.cache_bottom] == ce) {\n+\t\tint bottom = o->internal.cache_bottom;\n+\n \t\twhile (bottom < o->src_index->cache_nr &&\n \t\t       o->src_index->cache[bottom]->ce_flags & CE_UNPACKED)\n \t\t\tbottom++;\n-\t\to->cache_bottom = bottom;\n+\t\to->internal.cache_bottom = bottom;\n \t}\n }\n \n@@ -652,7 +653,7 @@ static void mark_ce_used_same_name(struct cache_entry *ce,\n static struct cache_entry *next_cache_entry(struct unpack_trees_options *o)\n {\n \tconst struct index_state *index = o->src_index;\n-\tint pos = o->cache_bottom;\n+\tint pos = o->internal.cache_bottom;\n \n \twhile (pos < index->cache_nr) {\n \t\tstruct cache_entry *ce = index->cache[pos];\n@@ -711,7 +712,7 @@ static void restore_cache_bottom(struct traverse_info *info, int bottom)\n \n \tif (o->diff_index_cached)\n \t\treturn;\n-\to->cache_bottom = bottom;\n+\to->internal.cache_bottom = bottom;\n }\n \n static int switch_cache_bottom(struct traverse_info *info)\n@@ -721,13 +722,13 @@ static int switch_cache_bottom(struct traverse_info *info)\n \n \tif (o->diff_index_cached)\n \t\treturn 0;\n-\tret = o->cache_bottom;\n+\tret = o->internal.cache_bottom;\n \tpos = find_cache_pos(info->prev, info->name, info->namelen);\n \n \tif (pos < -1)\n-\t\to->cache_bottom = -2 - pos;\n+\t\to->internal.cache_bottom = -2 - pos;\n \telse if (pos < 0)\n-\t\to->cache_bottom = o->src_index->cache_nr;\n+\t\to->internal.cache_bottom = o->src_index->cache_nr;\n \treturn ret;\n }\n \n@@ -873,9 +874,9 @@ static int traverse_trees_recursive(int n, unsigned long dirmask,\n \t\t * save and restore cache_bottom anyway to not miss\n \t\t * unprocessed entries before 'pos'.\n \t\t */\n-\t\tbottom = o->cache_bottom;\n+\t\tbottom = o->internal.cache_bottom;\n \t\tret = traverse_by_cache_tree(pos, nr_entries, n, info);\n-\t\to->cache_bottom = bottom;\n+\t\to->internal.cache_bottom = bottom;\n \t\treturn ret;\n \t}\n \n@@ -1212,7 +1213,7 @@ static int unpack_single_entry(int n, unsigned long mask,\n \t\t * cache entry from the index aware logic.\n \t\t */\n \t\tsrc[i + o->merge] = create_ce_entry(info, names + i, stage,\n-\t\t\t\t\t\t    &o->result, o->merge,\n+\t\t\t\t\t\t    &o->internal.result, o->merge,\n \t\t\t\t\t\t    bit & dirmask);\n \t}\n \n@@ -1237,7 +1238,7 @@ static int unpack_single_entry(int n, unsigned long mask,\n \n static int unpack_failed(struct unpack_trees_options *o, const char *message)\n {\n-\tdiscard_index(&o->result);\n+\tdiscard_index(&o->internal.result);\n \tif (!o->quiet && !o->exiting_early) {\n \t\tif (message)\n \t\t\treturn error(\"%s\", message);\n@@ -1260,7 +1261,7 @@ static int find_cache_pos(struct traverse_info *info,\n \tstruct index_state *index = o->src_index;\n \tint pfxlen = info->pathlen;\n \n-\tfor (pos = o->cache_bottom; pos < index->cache_nr; pos++) {\n+\tfor (pos = o->internal.cache_bottom; pos < index->cache_nr; pos++) {\n \t\tconst struct cache_entry *ce = index->cache[pos];\n \t\tconst char *ce_name, *ce_slash;\n \t\tint cmp, ce_len;\n@@ -1271,8 +1272,8 @@ static int find_cache_pos(struct traverse_info *info,\n \t\t\t * we can never match it; don't check it\n \t\t\t * again.\n \t\t\t */\n-\t\t\tif (pos == o->cache_bottom)\n-\t\t\t\t++o->cache_bottom;\n+\t\t\tif (pos == o->internal.cache_bottom)\n+\t\t\t\t++o->internal.cache_bottom;\n \t\t\tcontinue;\n \t\t}\n \t\tif (!ce_in_traverse_path(ce, info)) {\n@@ -1450,7 +1451,7 @@ static int unpack_sparse_callback(int n, unsigned long mask, unsigned long dirma\n \t */\n \tif (!is_null_oid(&names[0].oid)) {\n \t\tsrc[0] = create_ce_entry(info, &names[0], 0,\n-\t\t\t\t\t&o->result, 1,\n+\t\t\t\t\t&o->internal.result, 1,\n \t\t\t\t\tdirmask & (1ul << 0));\n \t\tsrc[0]->ce_flags |= (CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE);\n \t}\n@@ -1560,7 +1561,7 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str\n \t\t\t\t * in 'mark_ce_used()'\n \t\t\t\t */\n \t\t\t\tif (!src[0] || !S_ISSPARSEDIR(src[0]->ce_mode))\n-\t\t\t\t\to->cache_bottom += matches;\n+\t\t\t\t\to->internal.cache_bottom += matches;\n \t\t\t\treturn mask;\n \t\t\t}\n \t\t}\n@@ -1907,37 +1908,37 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tpopulate_from_existing_patterns(o, &pl);\n \t}\n \n-\tindex_state_init(&o->result, o->src_index->repo);\n-\to->result.initialized = 1;\n-\to->result.timestamp.sec = o->src_index->timestamp.sec;\n-\to->result.timestamp.nsec = o->src_index->timestamp.nsec;\n-\to->result.version = o->src_index->version;\n+\tindex_state_init(&o->internal.result, o->src_index->repo);\n+\to->internal.result.initialized = 1;\n+\to->internal.result.timestamp.sec = o->src_index->timestamp.sec;\n+\to->internal.result.timestamp.nsec = o->src_index->timestamp.nsec;\n+\to->internal.result.version = o->src_index->version;\n \tif (!o->src_index->split_index) {\n-\t\to->result.split_index = NULL;\n+\t\to->internal.result.split_index = NULL;\n \t} else if (o->src_index == o->dst_index) {\n \t\t/*\n \t\t * o->dst_index (and thus o->src_index) will be discarded\n-\t\t * and overwritten with o->result at the end of this function,\n+\t\t * and overwritten with o->internal.result at the end of this function,\n \t\t * so just use src_index's split_index to avoid having to\n \t\t * create a new one.\n \t\t */\n-\t\to->result.split_index = o->src_index->split_index;\n-\t\to->result.split_index->refcount++;\n+\t\to->internal.result.split_index = o->src_index->split_index;\n+\t\to->internal.result.split_index->refcount++;\n \t} else {\n-\t\to->result.split_index = init_split_index(&o->result);\n+\t\to->internal.result.split_index = init_split_index(&o->internal.result);\n \t}\n-\toidcpy(&o->result.oid, &o->src_index->oid);\n+\toidcpy(&o->internal.result.oid, &o->src_index->oid);\n \to->merge_size = len;\n \tmark_all_ce_unused(o->src_index);\n \n-\to->result.fsmonitor_last_update =\n+\to->internal.result.fsmonitor_last_update =\n \t\txstrdup_or_null(o->src_index->fsmonitor_last_update);\n-\to->result.fsmonitor_has_run_once = o->src_index->fsmonitor_has_run_once;\n+\to->internal.result.fsmonitor_has_run_once = o->src_index->fsmonitor_has_run_once;\n \n \tif (!o->src_index->initialized &&\n \t    !repo->settings.command_requires_full_index &&\n-\t    is_sparse_index_allowed(&o->result, 0))\n-\t\to->result.sparse_index = 1;\n+\t    is_sparse_index_allowed(&o->internal.result, 0))\n+\t\to->internal.result.sparse_index = 1;\n \n \t/*\n \t * Sparse checkout loop #1: set NEW_SKIP_WORKTREE on existing entries\n@@ -1957,7 +1958,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tsetup_traverse_info(&info, prefix);\n \t\tinfo.fn = unpack_callback;\n \t\tinfo.data = o;\n-\t\tinfo.show_all_errors = o->show_all_errors;\n+\t\tinfo.show_all_errors = o->internal.show_all_errors;\n \t\tinfo.pathspec = o->pathspec;\n \n \t\tif (o->prefix) {\n@@ -1998,7 +1999,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t}\n \tmark_all_ce_unused(o->src_index);\n \n-\tif (o->trivial_merges_only && o->nontrivial_merge) {\n+\tif (o->trivial_merges_only && o->internal.nontrivial_merge) {\n \t\tret = unpack_failed(o, \"Merge requires file-level merging\");\n \t\tgoto done;\n \t}\n@@ -2009,13 +2010,13 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t * If they will have NEW_SKIP_WORKTREE, also set CE_SKIP_WORKTREE\n \t\t * so apply_sparse_checkout() won't attempt to remove it from worktree\n \t\t */\n-\t\tmark_new_skip_worktree(o->internal.pl, &o->result,\n+\t\tmark_new_skip_worktree(o->internal.pl, &o->internal.result,\n \t\t\t\t       CE_ADDED, CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE,\n \t\t\t\t       o->verbose_update);\n \n \t\tret = 0;\n-\t\tfor (i = 0; i < o->result.cache_nr; i++) {\n-\t\t\tstruct cache_entry *ce = o->result.cache[i];\n+\t\tfor (i = 0; i < o->internal.result.cache_nr; i++) {\n+\t\t\tstruct cache_entry *ce = o->internal.result.cache[i];\n \n \t\t\t/*\n \t\t\t * Entries marked with CE_ADDED in merged_entry() do not have\n@@ -2029,7 +2030,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t\t    verify_absent(ce, WARNING_SPARSE_ORPHANED_NOT_OVERWRITTEN, o))\n \t\t\t\tret = 1;\n \n-\t\t\tif (apply_sparse_checkout(&o->result, ce, o))\n+\t\t\tif (apply_sparse_checkout(&o->internal.result, ce, o))\n \t\t\t\tret = 1;\n \t\t}\n \t\tif (ret == 1) {\n@@ -2037,30 +2038,30 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t\t * Inability to sparsify or de-sparsify individual\n \t\t\t * paths is not an error, but just a warning.\n \t\t\t */\n-\t\t\tif (o->show_all_errors)\n+\t\t\tif (o->internal.show_all_errors)\n \t\t\t\tdisplay_warning_msgs(o);\n \t\t\tret = 0;\n \t\t}\n \t}\n \n-\tret = check_updates(o, &o->result) ? (-2) : 0;\n+\tret = check_updates(o, &o->internal.result) ? (-2) : 0;\n \tif (o->dst_index) {\n-\t\tmove_index_extensions(&o->result, o->src_index);\n+\t\tmove_index_extensions(&o->internal.result, o->src_index);\n \t\tif (!ret) {\n \t\t\tif (git_env_bool(\"GIT_TEST_CHECK_CACHE_TREE\", 0))\n-\t\t\t\tcache_tree_verify(the_repository, &o->result);\n+\t\t\t\tcache_tree_verify(the_repository, &o->internal.result);\n \t\t\tif (!o->skip_cache_tree_update &&\n-\t\t\t    !cache_tree_fully_valid(o->result.cache_tree))\n-\t\t\t\tcache_tree_update(&o->result,\n+\t\t\t    !cache_tree_fully_valid(o->internal.result.cache_tree))\n+\t\t\t\tcache_tree_update(&o->internal.result,\n \t\t\t\t\t\t  WRITE_TREE_SILENT |\n \t\t\t\t\t\t  WRITE_TREE_REPAIR);\n \t\t}\n \n-\t\to->result.updated_workdir = 1;\n+\t\to->internal.result.updated_workdir = 1;\n \t\tdiscard_index(o->dst_index);\n-\t\t*o->dst_index = o->result;\n+\t\t*o->dst_index = o->internal.result;\n \t} else {\n-\t\tdiscard_index(&o->result);\n+\t\tdiscard_index(&o->internal.result);\n \t}\n \to->src_index = NULL;\n \n@@ -2076,7 +2077,7 @@ done:\n \treturn ret;\n \n return_failed:\n-\tif (o->show_all_errors)\n+\tif (o->internal.show_all_errors)\n \t\tdisplay_error_msgs(o);\n \tmark_all_ce_unused(o->src_index);\n \tret = unpack_failed(o, NULL);\n@@ -2099,9 +2100,9 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n \tunsigned old_show_all_errors;\n \tint free_pattern_list = 0;\n \n-\told_show_all_errors = o->show_all_errors;\n-\to->show_all_errors = 1;\n-\tindex_state_init(&o->result, o->src_index->repo);\n+\told_show_all_errors = o->internal.show_all_errors;\n+\to->internal.show_all_errors = 1;\n+\tindex_state_init(&o->internal.result, o->src_index->repo);\n \n \t/* Sanity checks */\n \tif (!o->update || o->index_only || o->skip_sparse_checkout)\n@@ -2148,7 +2149,7 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n \t\tret = UPDATE_SPARSITY_WORKTREE_UPDATE_FAILURES;\n \n \tdisplay_warning_msgs(o);\n-\to->show_all_errors = old_show_all_errors;\n+\to->internal.show_all_errors = old_show_all_errors;\n \tif (free_pattern_list) {\n \t\tclear_pattern_list(pl);\n \t\tfree(pl);\n@@ -2248,15 +2249,15 @@ static int verify_uptodate_sparse(const struct cache_entry *ce,\n }\n \n /*\n- * TODO: We should actually invalidate o->result, not src_index [1].\n+ * TODO: We should actually invalidate o->internal.result, not src_index [1].\n  * But since cache tree and untracked cache both are not copied to\n- * o->result until unpacking is complete, we invalidate them on\n+ * o->internal.result until unpacking is complete, we invalidate them on\n  * src_index instead with the assumption that they will be copied to\n  * dst_index at the end.\n  *\n  * [1] src_index->cache_tree is also used in unpack_callback() so if\n- * we invalidate o->result, we need to update it to use\n- * o->result.cache_tree as well.\n+ * we invalidate o->internal.result, we need to update it to use\n+ * o->internal.result.cache_tree as well.\n  */\n static void invalidate_ce_path(const struct cache_entry *ce,\n \t\t\t       struct unpack_trees_options *o)\n@@ -2424,7 +2425,7 @@ static int check_ok_to_remove(const char *name, int len, int dtype,\n \t * delete this path, which is in a subdirectory that\n \t * is being replaced with a blob.\n \t */\n-\tresult = index_file_exists(&o->result, name, len, 0);\n+\tresult = index_file_exists(&o->internal.result, name, len, 0);\n \tif (result) {\n \t\tif (result->ce_flags & CE_REMOVE)\n \t\t\treturn 0;\n@@ -2525,7 +2526,7 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\tstruct unpack_trees_options *o)\n {\n \tint update = CE_UPDATE;\n-\tstruct cache_entry *merge = dup_cache_entry(ce, &o->result);\n+\tstruct cache_entry *merge = dup_cache_entry(ce, &o->internal.result);\n \n \tif (!old) {\n \t\t/*\n@@ -2620,7 +2621,7 @@ static int merged_sparse_dir(const struct cache_entry * const *src, int n,\n \tsetup_traverse_info(&info, src[0]->name);\n \tinfo.fn = unpack_sparse_callback;\n \tinfo.data = o;\n-\tinfo.show_all_errors = o->show_all_errors;\n+\tinfo.show_all_errors = o->internal.show_all_errors;\n \tinfo.pathspec = o->pathspec;\n \n \t/* Get the tree descriptors of the sparse directory in each of the merging trees */\n@@ -2838,7 +2839,7 @@ int threeway_merge(const struct cache_entry * const *stages,\n \t\t\treturn -1;\n \t}\n \n-\to->nontrivial_merge = 1;\n+\to->internal.nontrivial_merge = 1;\n \n \t/* #2, #3, #4, #6, #7, #9, #10, #11. */\n \tcount = 0;\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 5c1a9314a06..0335c89bc75 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -59,7 +59,6 @@ struct unpack_trees_options {\n \t\t     preserve_ignored,\n \t\t     clone,\n \t\t     index_only,\n-\t\t     nontrivial_merge,\n \t\t     trivial_merges_only,\n \t\t     verbose_update,\n \t\t     aggressive,\n@@ -70,22 +69,13 @@ struct unpack_trees_options {\n \t\t     skip_sparse_checkout,\n \t\t     quiet,\n \t\t     exiting_early,\n-\t\t     show_all_errors,\n \t\t     dry_run,\n \t\t     skip_cache_tree_update;\n \tenum unpack_trees_reset_type reset;\n \tconst char *prefix;\n \tconst char *super_prefix;\n-\tint cache_bottom;\n \tstruct pathspec *pathspec;\n \tmerge_fn_t fn;\n-\tconst char *msgs[NB_UNPACK_TREES_WARNING_TYPES];\n-\tstruct strvec msgs_to_free;\n-\t/*\n-\t * Store error messages in an array, each case\n-\t * corresponding to a error message type\n-\t */\n-\tstruct string_list unpack_rejects[NB_UNPACK_TREES_WARNING_TYPES];\n \n \tint head_idx;\n \tint merge_size;\n@@ -95,11 +85,25 @@ struct unpack_trees_options {\n \n \tstruct index_state *dst_index;\n \tstruct index_state *src_index;\n-\tstruct index_state result;\n \n \tstruct checkout_metadata meta;\n \n \tstruct unpack_trees_options_internal {\n+\t\tunsigned int nontrivial_merge,\n+\t\t\t     show_all_errors;\n+\n+\t\tint cache_bottom;\n+\t\tconst char *msgs[NB_UNPACK_TREES_WARNING_TYPES];\n+\t\tstruct strvec msgs_to_free;\n+\n+\t\t/*\n+\t\t * Store error messages in an array, each case\n+\t\t * corresponding to a error message type\n+\t\t */\n+\t\tstruct string_list unpack_rejects[NB_UNPACK_TREES_WARNING_TYPES];\n+\n+\t\tstruct index_state result;\n+\n \t\tstruct pattern_list *pl;\n \t\tstruct dir_struct *dir;\n \t} internal;\n-- \ngitgitgadget\n\n"},{"id":"472711","messageId":"CABPp-BFNd3ihQbMb+Sc270W=obnZkEgvwAuKZ8v_UBcFBoFUVg@mail.gmail.com","threadId":"59297","inReplyTo":"20230225015432.2065788-1-jonathantanmy@google.com","subject":"Re: [PATCH 02/11] dir: add a usage note to exclude_per_dir","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2023-02-25T03:23:02Z","receivedAt":"2023-02-25T03:23:21Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Feb 24, 2023 at 5:54 PM Jonathan Tan <jonathantanmy@google.com> wrote:\n>\n> Elijah Newren <newren@gmail.com> writes:\n> > On Fri, Feb 24, 2023 at 2:31 PM Jonathan Tan <jonathantanmy@google.com> wrote:\n> > >\n> > > \"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> > > > diff --git a/dir.h b/dir.h\n> > > > index 33fd848fc8d..2196e12630c 100644\n> > > > --- a/dir.h\n> > > > +++ b/dir.h\n> > > > @@ -295,8 +295,12 @@ struct dir_struct {\n> > > >       struct untracked_cache *untracked;\n> > > >\n> > > >       /**\n> > > > -      * The name of the file to be read in each directory for excluded files\n> > > > -      * (typically `.gitignore`).\n> > > > +      * Deprecated: ls-files is the only allowed caller; all other callers\n> > > > +      * should leave this as NULL; it pre-dated the\n> > > > +      * setup_standard_excludes() mechanism that replaces this.\n> > > > +      *\n> > > > +      * This field tracks the name of the file to be read in each directory\n> > > > +      * for excluded files (typically `.gitignore`).\n> > > >        */\n> > > >       const char *exclude_per_dir;\n> > >\n> > > I'm not sure what is meant by \"allowed caller\", but I wouldn't have\n> > > expected this to also mean that unpack-trees would need to know to\n> > > propagate this from o->internal.dir to d in verify_clean_subdirectory.\n> >\n> > Are you confusing fields that are internal to dir, with fields that\n> > are internal to unpack-trees?\n> >\n> > This series does not make exclude_per_dir an internal field within dir_struct.\n>\n> Agreed, but the comment says that ls-files is the only allowed caller,\n> and I would have expected that non-\"allowed callers\" would not need to\n> write to exclude_per_dir. But in unpack-trees.c:\n>\n>   2346          if (o->internal.dir)\n>   2347                  d.exclude_per_dir = o->internal.dir->exclude_per_dir;\n>\n> Both \"d\" and \"o->internal.dir\" are of type \"struct dir_struct\" (well,\n> one is not a pointer and one is). I would not have expected such non-\n> ls-files code to read or write from this field. (But if unpack-trees\n> is considered part of ls-files and/or copying the same field to another\n> struct is not considered \"calling\", then this patch is fine, I guess.)\n\nAh, gotcha, sorry for misunderstanding earlier.  And you are right;\nthe code in unpack_trees() is buggy and should not be using\nexclude_per_dir; the fact that it is using it directly actually hints\nthat it has bugs when exclusions are specified in .git/info/exclude\ninstead of .gitignore.  I've found a testcase that demonstrates this;\nI'll update the series with a new patch fixing it.\n"},{"id":"472736","messageId":"xmqqzg91tjb4.fsf@gitster.g","threadId":"59297","inReplyTo":"pull.1149.v2.git.1677291960.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 00/11] Clarify API for dir.[ch] and unpack-trees.[ch] -- mark relevant fields as internal","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-02-25T23:30:07Z","receivedAt":"2023-02-25T23:30:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> Changes since v1 (thanks to Jonathan Tan for the careful reviews!)\n>\n>  * Clear o->pl when freeing pl, to avoid risking use-after-free.\n>  * Initialize o->result in update_sparsity() since it is actually used (by\n>    check_ok_to_remove()).\n>\n> Some time ago, I noticed that struct dir_struct and struct\n> unpack_trees_options both have numerous fields meant for internal use only,\n> most of which are not marked as such. This has resulted in callers\n> accidentally trying to initialize some of these fields, and in at least one\n> case required a fair amount of review to verify other changes were okay --\n> review that would have been simplified with the apriori knowledge that a\n> combination of multiple fields were internal-only[1]. Looking closer, I\n> found that only 6 out of 18 fields in dir_struct were actually meant to be\n> public[2], and noted that unpack_trees_options also had 11 internal-only\n> fields (out of 36).\n\nNice.\n\nWill queue.\n\nThanks.\n"},{"id":"472782","messageId":"pull.1149.v3.git.1677511700.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v2.git.1677291960.gitgitgadget@gmail.com","subject":"[PATCH v3 00/13] Clarify API for dir.[ch] and unpack-trees.[ch] -- mark relevant fields as internal","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-27T15:28:07Z","receivedAt":"2023-02-27T15:28:30Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Changes since v2:\n\n * Two new patches:\n   * one patch (2nd in the series) that fixes a bug in unpack-trees due to\n     the code never having been updated to setup_standard_excludes() + adds\n     a test to avoid regressing that bug it\n   * a preliminary patch that fixes a separate latent issue in the modified\n     testfile referenced above, so that the new test listed above will\n     actually work on all platforms.\n\nChanges since v1 (thanks to Jonathan Tan for the careful reviews!)\n\n * Clear o->pl when freeing pl, to avoid risking use-after-free.\n * Initialize o->result in update_sparsity() since it is actually used (by\n   check_ok_to_remove()).\n\nSome time ago, I noticed that struct dir_struct and struct\nunpack_trees_options both have numerous fields meant for internal use only,\nmost of which are not marked as such. This has resulted in callers\naccidentally trying to initialize some of these fields, and in at least one\ncase required a fair amount of review to verify other changes were okay --\nreview that would have been simplified with the apriori knowledge that a\ncombination of multiple fields were internal-only[1]. Looking closer, I\nfound that only 6 out of 18 fields in dir_struct were actually meant to be\npublic[2], and noted that unpack_trees_options also had 11 internal-only\nfields (out of 36).\n\nThis patch is primarily about moving internal-only fields within these two\nstructs into an embedded internal struct. Patch breakdown:\n\n * Patches 1-3: Restructuring dir_struct\n   * Patch 1: Splitting off internal-use-only fields\n   * Patch 2: Add important usage note to avoid accidentally using\n     deprecated API\n   * Patch 3: Mark output-only fields as such\n * Patches 4-11: Restructuring unpack_trees_options\n   * Patches 4-6: Preparatory cleanup\n   * Patches 7-10: Splitting off internal-use-only fields\n   * Patch 11: Mark output-only field as such\n\nTo make the benefit more clear, here are compressed versions of dir_struct\nboth before and after the changes. First, before:\n\nstruct dir_struct {\n    int nr;\n    int alloc;\n    int ignored_nr;\n    int ignored_alloc;\n    enum [...] flags;\n    struct dir_entry **entries;\n    struct dir_entry **ignored;\n    const char *exclude_per_dir;\n#define EXC_CMDL 0\n#define EXC_DIRS 1\n#define EXC_FILE 2\n    struct exclude_list_group exclude_list_group[3];\n    struct exclude_stack *exclude_stack;\n    struct path_pattern *pattern;\n    struct strbuf basebuf;\n    struct untracked_cache *untracked;\n    struct oid_stat ss_info_exclude;\n    struct oid_stat ss_excludes_file;\n    unsigned unmanaged_exclude_files;\n    unsigned visited_paths;\n    unsigned visited_directories;\n};\n\n\nAnd after the changes:\n\nstruct dir_struct {\n    enum [...] flags;\n    int nr; /* output only */\n    int ignored_nr; /* output only */\n    struct dir_entry **entries; /* output only */\n    struct dir_entry **ignored; /* output only */\n    struct untracked_cache *untracked;\n    const char *exclude_per_dir; /* deprecated */\n    struct dir_struct_internal {\n        int alloc;\n        int ignored_alloc;\n#define EXC_CMDL 0\n#define EXC_DIRS 1\n#define EXC_FILE 2\n        struct exclude_list_group exclude_list_group[3];\n        struct exclude_stack *exclude_stack;\n        struct path_pattern *pattern;\n        struct strbuf basebuf;\n        struct oid_stat ss_info_exclude;\n        struct oid_stat ss_excludes_file;\n        unsigned unmanaged_exclude_files;\n        unsigned visited_paths;\n        unsigned visited_directories;\n    } internal;\n};\n\n\nThe former version has 18 fields (and 3 magic constants) which API users\nwill have to figure out. The latter makes it clear there are only at most 2\nfields you should be setting upon input, and at most 4 which you read at\noutput, and the rest (including all the magic constants) you can ignore.\n\n[0] Search for \"Extremely yes\" in\nhttps://lore.kernel.org/git/CAJoAoZm+TkCL0Jpg_qFgKottxbtiG2QOiY0qGrz3-uQy+=waPg@mail.gmail.com/\n[1]\nhttps://lore.kernel.org/git/CABPp-BFSFN3WM6q7KzkD5mhrwsz--St_-ej5LbaY8Yr2sZzj=w@mail.gmail.com/\n[2]\nhttps://lore.kernel.org/git/CABPp-BHgot=CPNyK_xNfog_SqsNPNoCGfiSb-gZoS2sn_741dQ@mail.gmail.com/\n\nElijah Newren (13):\n  t2021: fix platform-specific leftover cruft\n  unpack-trees: heed requests to overwrite ignored files\n  dir: separate public from internal portion of dir_struct\n  dir: add a usage note to exclude_per_dir\n  dir: mark output only fields of dir_struct as such\n  unpack-trees: clean up some flow control\n  sparse-checkout: avoid using internal API of unpack-trees\n  sparse-checkout: avoid using internal API of unpack-trees, take 2\n  unpack_trees: start splitting internal fields from public API\n  unpack-trees: mark fields only used internally as internal\n  unpack-trees: rewrap a few overlong lines from previous patch\n  unpack-trees: special case read-tree debugging as internal usage\n  unpack-trees: add usage notices around df_conflict_entry\n\n builtin/read-tree.c           |  10 +-\n builtin/sparse-checkout.c     |   4 +-\n dir.c                         | 114 ++++++++--------\n dir.h                         | 110 ++++++++-------\n t/t2021-checkout-overwrite.sh |  16 ++-\n unpack-trees.c                | 247 ++++++++++++++++++----------------\n unpack-trees.h                |  42 +++---\n 7 files changed, 292 insertions(+), 251 deletions(-)\n\n\nbase-commit: 06dd2baa8da4a73421b959ec026a43711b9d77f9\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1149%2Fnewren%2Fclarify-api-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1149/newren/clarify-api-v3\nPull-Request: https://github.com/gitgitgadget/git/pull/1149\n\nRange-diff vs v2:\n\n  -:  ----------- >  1:  7bbc4577a57 t2021: fix platform-specific leftover cruft\n  -:  ----------- >  2:  8ffdb6c8a8a unpack-trees: heed requests to overwrite ignored files\n  1:  7f59ad548d0 =  3:  879a93ac2d7 dir: separate public from internal portion of dir_struct\n  2:  239b10e1181 !  4:  4ce9fae5e7f dir: add a usage note to exclude_per_dir\n     @@ Metadata\n       ## Commit message ##\n          dir: add a usage note to exclude_per_dir\n      \n     +    As evidenced by the fix a couple commits ago, places in the code using\n     +    exclude_per_dir are likely buggy and should be adapted to call\n     +    setup_standard_excludes() instead.  Unfortunately, the usage of\n     +    exclude_per_dir has been hardcoded into the arguments ls-files accepts,\n     +    so we cannot actually remove it.  Add a note that it is deprecated and\n     +    no other callers should use it directly.\n     +\n          Signed-off-by: Elijah Newren <newren@gmail.com>\n      \n       ## dir.h ##\n  3:  b8aa14350d3 =  5:  12344400fa0 dir: mark output only fields of dir_struct as such\n  4:  f5a58123034 =  6:  4e86e39506c unpack-trees: clean up some flow control\n  5:  975dec0f0eb =  7:  cd3d4894afb sparse-checkout: avoid using internal API of unpack-trees\n  6:  429f195dcfe =  8:  09140cb2ac5 sparse-checkout: avoid using internal API of unpack-trees, take 2\n  7:  993da584dbb !  9:  27f2d477116 unpack_trees: start splitting internal fields from public API\n     @@ unpack-trees.c: static int verify_clean_subdirectory(const struct cache_entry *c\n       \n       \tmemset(&d, 0, sizeof(d));\n      -\tif (o->dir)\n     --\t\td.exclude_per_dir = o->dir->exclude_per_dir;\n      +\tif (o->internal.dir)\n     -+\t\td.exclude_per_dir = o->internal.dir->exclude_per_dir;\n     + \t\tsetup_standard_excludes(&d);\n       \ti = read_directory(&d, o->src_index, pathbuf, namelen+1, NULL);\n       \tdir_clear(&d);\n     - \tfree(pathbuf);\n      @@ unpack-trees.c: static int check_ok_to_remove(const char *name, int len, int dtype,\n       \tif (ignore_case && icase_exists(o, name, len, st))\n       \t\treturn 0;\n  8:  8ecb24a45f0 = 10:  4236c0d80c7 unpack-trees: mark fields only used internally as internal\n  9:  36ca49c3624 = 11:  76f4a544e4b unpack-trees: rewrap a few overlong lines from previous patch\n 10:  5af04d7fe23 = 12:  ee36935adb5 unpack-trees: special case read-tree debugging as internal usage\n 11:  c4f31237634 = 13:  6575c007577 unpack-trees: add usage notices around df_conflict_entry\n\n-- \ngitgitgadget\n"},{"id":"472783","messageId":"7bbc4577a57fc9cb9debc249af232282b6b3f24f.1677511700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v3.git.1677511700.gitgitgadget@gmail.com","subject":"[PATCH v3 01/13] t2021: fix platform-specific leftover cruft","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-27T15:28:08Z","receivedAt":"2023-02-27T15:28:33Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nt2021.6 existed to test the status of a symlink that was left around by\nprevious tests.  It tried to also clean up the symlink after it was done\nso that subsequent tests wouldn't be tripped up by it.  Unfortunately,\nsince this test had a SYMLINK prerequisite, that made the cleanup\nplatform dependent...and made a testcase I was trying to add to this\ntestsuite fail (that testcase will be included in the next patch).\nBefore we go and add new testcases, fix this cleanup by moving it into a\nseparate test.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n t/t2021-checkout-overwrite.sh | 5 ++++-\n 1 file changed, 4 insertions(+), 1 deletion(-)\n\ndiff --git a/t/t2021-checkout-overwrite.sh b/t/t2021-checkout-overwrite.sh\nindex 713c3fa6038..baca66e1a31 100755\n--- a/t/t2021-checkout-overwrite.sh\n+++ b/t/t2021-checkout-overwrite.sh\n@@ -50,10 +50,13 @@ test_expect_success 'checkout commit with dir must not remove untracked a/b' '\n \n test_expect_success SYMLINKS 'the symlink remained' '\n \n-\ttest_when_finished \"rm a/b\" &&\n \ttest -h a/b\n '\n \n+test_expect_success 'cleanup after previous symlink tests' '\n+\trm a/b\n+'\n+\n test_expect_success SYMLINKS 'checkout -f must not follow symlinks when removing entries' '\n \tgit checkout -f start &&\n \tmkdir dir &&\n-- \ngitgitgadget\n\n"},{"id":"472784","messageId":"8ffdb6c8a8a3b162c898ba759137d41bbfd44b39.1677511700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v3.git.1677511700.gitgitgadget@gmail.com","subject":"[PATCH v3 02/13] unpack-trees: heed requests to overwrite ignored files","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-27T15:28:09Z","receivedAt":"2023-02-27T15:28:34Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nWhen a directory exists but has only ignored files within it and we are\ntrying to switch to a branch that has a file where that directory is,\nthe behavior depends upon --[no]-overwrite-ignore.  If the user wants to\n--overwrite-ignore (the default), then we should delete the ignored file\nand directory and switch to the new branch.\n\nThe code to handle this in verify_clean_subdirectory() in unpack-trees\ntried to handle this via paying attention to the exclude_per_dir setting\nof the internal dir field.  This came from commit c81935348b (\"Fix\nswitching to a branch with D/F when current branch has file D.\",\n2007-03-15), which pre-dated 039bc64e88 (\"core.excludesfile clean-up\",\n2007-11-14), and thus did not pay attention to ignore patterns from\nother relevant files.  Change it to use setup_standard_excludes() so\nthat it is also aware of excludes specified in other locations.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n t/t2021-checkout-overwrite.sh | 11 +++++++++++\n unpack-trees.c                |  2 +-\n 2 files changed, 12 insertions(+), 1 deletion(-)\n\ndiff --git a/t/t2021-checkout-overwrite.sh b/t/t2021-checkout-overwrite.sh\nindex baca66e1a31..034f62c13c5 100755\n--- a/t/t2021-checkout-overwrite.sh\n+++ b/t/t2021-checkout-overwrite.sh\n@@ -69,4 +69,15 @@ test_expect_success SYMLINKS 'checkout -f must not follow symlinks when removing\n \ttest_path_is_file untracked/f\n '\n \n+test_expect_success 'checkout --overwrite-ignore should succeed if only ignored files in the way' '\n+\tgit checkout -b df_conflict &&\n+\ttest_commit contents some_dir &&\n+\tgit checkout start &&\n+\tmkdir some_dir &&\n+\techo autogenerated information >some_dir/ignore &&\n+\techo ignore >.git/info/exclude &&\n+\tgit checkout --overwrite-ignore df_conflict &&\n+\t! test_path_is_dir some_dir\n+'\n+\n test_done\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 3d05e45a279..4518d33ed99 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -2337,7 +2337,7 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n \n \tmemset(&d, 0, sizeof(d));\n \tif (o->dir)\n-\t\td.exclude_per_dir = o->dir->exclude_per_dir;\n+\t\tsetup_standard_excludes(&d);\n \ti = read_directory(&d, o->src_index, pathbuf, namelen+1, NULL);\n \tdir_clear(&d);\n \tfree(pathbuf);\n-- \ngitgitgadget\n\n"},{"id":"472786","messageId":"879a93ac2d7c9113489d6964e434aed73219651d.1677511700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v3.git.1677511700.gitgitgadget@gmail.com","subject":"[PATCH v3 03/13] dir: separate public from internal portion of dir_struct","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-27T15:28:10Z","receivedAt":"2023-02-27T15:28:37Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nIn order to make it clearer to callers what portions of dir_struct are\npublic API, and avoid errors from them setting fields that are meant as\ninternal API, split the fields used for internal implementation reasons\ninto a separate embedded struct.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n dir.c | 114 +++++++++++++++++++++++++++++-----------------------------\n dir.h |  86 +++++++++++++++++++++++---------------------\n 2 files changed, 104 insertions(+), 96 deletions(-)\n\ndiff --git a/dir.c b/dir.c\nindex 4e99f0c868f..7adf242026e 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -1190,7 +1190,7 @@ struct pattern_list *add_pattern_list(struct dir_struct *dir,\n \tstruct pattern_list *pl;\n \tstruct exclude_list_group *group;\n \n-\tgroup = &dir->exclude_list_group[group_type];\n+\tgroup = &dir->internal.exclude_list_group[group_type];\n \tALLOC_GROW(group->pl, group->nr + 1, group->alloc);\n \tpl = &group->pl[group->nr++];\n \tmemset(pl, 0, sizeof(*pl));\n@@ -1211,7 +1211,7 @@ static void add_patterns_from_file_1(struct dir_struct *dir, const char *fname,\n \t * differently when dir->untracked is non-NULL.\n \t */\n \tif (!dir->untracked)\n-\t\tdir->unmanaged_exclude_files++;\n+\t\tdir->internal.unmanaged_exclude_files++;\n \tpl = add_pattern_list(dir, EXC_FILE, fname);\n \tif (add_patterns(fname, \"\", 0, pl, NULL, 0, oid_stat) < 0)\n \t\tdie(_(\"cannot use %s as an exclude file\"), fname);\n@@ -1219,7 +1219,7 @@ static void add_patterns_from_file_1(struct dir_struct *dir, const char *fname,\n \n void add_patterns_from_file(struct dir_struct *dir, const char *fname)\n {\n-\tdir->unmanaged_exclude_files++; /* see validate_untracked_cache() */\n+\tdir->internal.unmanaged_exclude_files++; /* see validate_untracked_cache() */\n \tadd_patterns_from_file_1(dir, fname, NULL);\n }\n \n@@ -1519,7 +1519,7 @@ static struct path_pattern *last_matching_pattern_from_lists(\n \tstruct exclude_list_group *group;\n \tstruct path_pattern *pattern;\n \tfor (i = EXC_CMDL; i <= EXC_FILE; i++) {\n-\t\tgroup = &dir->exclude_list_group[i];\n+\t\tgroup = &dir->internal.exclude_list_group[i];\n \t\tfor (j = group->nr - 1; j >= 0; j--) {\n \t\t\tpattern = last_matching_pattern_from_list(\n \t\t\t\tpathname, pathlen, basename, dtype_p,\n@@ -1545,20 +1545,20 @@ static void prep_exclude(struct dir_struct *dir,\n \tstruct untracked_cache_dir *untracked;\n \tint current;\n \n-\tgroup = &dir->exclude_list_group[EXC_DIRS];\n+\tgroup = &dir->internal.exclude_list_group[EXC_DIRS];\n \n \t/*\n \t * Pop the exclude lists from the EXCL_DIRS exclude_list_group\n \t * which originate from directories not in the prefix of the\n \t * path being checked.\n \t */\n-\twhile ((stk = dir->exclude_stack) != NULL) {\n+\twhile ((stk = dir->internal.exclude_stack) != NULL) {\n \t\tif (stk->baselen <= baselen &&\n-\t\t    !strncmp(dir->basebuf.buf, base, stk->baselen))\n+\t\t    !strncmp(dir->internal.basebuf.buf, base, stk->baselen))\n \t\t\tbreak;\n-\t\tpl = &group->pl[dir->exclude_stack->exclude_ix];\n-\t\tdir->exclude_stack = stk->prev;\n-\t\tdir->pattern = NULL;\n+\t\tpl = &group->pl[dir->internal.exclude_stack->exclude_ix];\n+\t\tdir->internal.exclude_stack = stk->prev;\n+\t\tdir->internal.pattern = NULL;\n \t\tfree((char *)pl->src); /* see strbuf_detach() below */\n \t\tclear_pattern_list(pl);\n \t\tfree(stk);\n@@ -1566,7 +1566,7 @@ static void prep_exclude(struct dir_struct *dir,\n \t}\n \n \t/* Skip traversing into sub directories if the parent is excluded */\n-\tif (dir->pattern)\n+\tif (dir->internal.pattern)\n \t\treturn;\n \n \t/*\n@@ -1574,12 +1574,12 @@ static void prep_exclude(struct dir_struct *dir,\n \t * memset(dir, 0, sizeof(*dir)) before use. Changing all of\n \t * them seems lots of work for little benefit.\n \t */\n-\tif (!dir->basebuf.buf)\n-\t\tstrbuf_init(&dir->basebuf, PATH_MAX);\n+\tif (!dir->internal.basebuf.buf)\n+\t\tstrbuf_init(&dir->internal.basebuf, PATH_MAX);\n \n \t/* Read from the parent directories and push them down. */\n \tcurrent = stk ? stk->baselen : -1;\n-\tstrbuf_setlen(&dir->basebuf, current < 0 ? 0 : current);\n+\tstrbuf_setlen(&dir->internal.basebuf, current < 0 ? 0 : current);\n \tif (dir->untracked)\n \t\tuntracked = stk ? stk->ucd : dir->untracked->root;\n \telse\n@@ -1599,32 +1599,33 @@ static void prep_exclude(struct dir_struct *dir,\n \t\t\t\tdie(\"oops in prep_exclude\");\n \t\t\tcp++;\n \t\t\tuntracked =\n-\t\t\t\tlookup_untracked(dir->untracked, untracked,\n+\t\t\t\tlookup_untracked(dir->untracked,\n+\t\t\t\t\t\t untracked,\n \t\t\t\t\t\t base + current,\n \t\t\t\t\t\t cp - base - current);\n \t\t}\n-\t\tstk->prev = dir->exclude_stack;\n+\t\tstk->prev = dir->internal.exclude_stack;\n \t\tstk->baselen = cp - base;\n \t\tstk->exclude_ix = group->nr;\n \t\tstk->ucd = untracked;\n \t\tpl = add_pattern_list(dir, EXC_DIRS, NULL);\n-\t\tstrbuf_add(&dir->basebuf, base + current, stk->baselen - current);\n-\t\tassert(stk->baselen == dir->basebuf.len);\n+\t\tstrbuf_add(&dir->internal.basebuf, base + current, stk->baselen - current);\n+\t\tassert(stk->baselen == dir->internal.basebuf.len);\n \n \t\t/* Abort if the directory is excluded */\n \t\tif (stk->baselen) {\n \t\t\tint dt = DT_DIR;\n-\t\t\tdir->basebuf.buf[stk->baselen - 1] = 0;\n-\t\t\tdir->pattern = last_matching_pattern_from_lists(dir,\n+\t\t\tdir->internal.basebuf.buf[stk->baselen - 1] = 0;\n+\t\t\tdir->internal.pattern = last_matching_pattern_from_lists(dir,\n \t\t\t\t\t\t\t\t\tistate,\n-\t\t\t\tdir->basebuf.buf, stk->baselen - 1,\n-\t\t\t\tdir->basebuf.buf + current, &dt);\n-\t\t\tdir->basebuf.buf[stk->baselen - 1] = '/';\n-\t\t\tif (dir->pattern &&\n-\t\t\t    dir->pattern->flags & PATTERN_FLAG_NEGATIVE)\n-\t\t\t\tdir->pattern = NULL;\n-\t\t\tif (dir->pattern) {\n-\t\t\t\tdir->exclude_stack = stk;\n+\t\t\t\tdir->internal.basebuf.buf, stk->baselen - 1,\n+\t\t\t\tdir->internal.basebuf.buf + current, &dt);\n+\t\t\tdir->internal.basebuf.buf[stk->baselen - 1] = '/';\n+\t\t\tif (dir->internal.pattern &&\n+\t\t\t    dir->internal.pattern->flags & PATTERN_FLAG_NEGATIVE)\n+\t\t\t\tdir->internal.pattern = NULL;\n+\t\t\tif (dir->internal.pattern) {\n+\t\t\t\tdir->internal.exclude_stack = stk;\n \t\t\t\treturn;\n \t\t\t}\n \t\t}\n@@ -1647,15 +1648,15 @@ static void prep_exclude(struct dir_struct *dir,\n \t\t      */\n \t\t     !is_null_oid(&untracked->exclude_oid))) {\n \t\t\t/*\n-\t\t\t * dir->basebuf gets reused by the traversal, but we\n-\t\t\t * need fname to remain unchanged to ensure the src\n-\t\t\t * member of each struct path_pattern correctly\n+\t\t\t * dir->internal.basebuf gets reused by the traversal,\n+\t\t\t * but we need fname to remain unchanged to ensure the\n+\t\t\t * src member of each struct path_pattern correctly\n \t\t\t * back-references its source file.  Other invocations\n \t\t\t * of add_pattern_list provide stable strings, so we\n \t\t\t * strbuf_detach() and free() here in the caller.\n \t\t\t */\n \t\t\tstruct strbuf sb = STRBUF_INIT;\n-\t\t\tstrbuf_addbuf(&sb, &dir->basebuf);\n+\t\t\tstrbuf_addbuf(&sb, &dir->internal.basebuf);\n \t\t\tstrbuf_addstr(&sb, dir->exclude_per_dir);\n \t\t\tpl->src = strbuf_detach(&sb, NULL);\n \t\t\tadd_patterns(pl->src, pl->src, stk->baselen, pl, istate,\n@@ -1681,10 +1682,10 @@ static void prep_exclude(struct dir_struct *dir,\n \t\t\tinvalidate_gitignore(dir->untracked, untracked);\n \t\t\toidcpy(&untracked->exclude_oid, &oid_stat.oid);\n \t\t}\n-\t\tdir->exclude_stack = stk;\n+\t\tdir->internal.exclude_stack = stk;\n \t\tcurrent = stk->baselen;\n \t}\n-\tstrbuf_setlen(&dir->basebuf, baselen);\n+\tstrbuf_setlen(&dir->internal.basebuf, baselen);\n }\n \n /*\n@@ -1704,8 +1705,8 @@ struct path_pattern *last_matching_pattern(struct dir_struct *dir,\n \n \tprep_exclude(dir, istate, pathname, basename-pathname);\n \n-\tif (dir->pattern)\n-\t\treturn dir->pattern;\n+\tif (dir->internal.pattern)\n+\t\treturn dir->internal.pattern;\n \n \treturn last_matching_pattern_from_lists(dir, istate, pathname, pathlen,\n \t\t\tbasename, dtype_p);\n@@ -1742,7 +1743,7 @@ static struct dir_entry *dir_add_name(struct dir_struct *dir,\n \tif (index_file_exists(istate, pathname, len, ignore_case))\n \t\treturn NULL;\n \n-\tALLOC_GROW(dir->entries, dir->nr+1, dir->alloc);\n+\tALLOC_GROW(dir->entries, dir->nr+1, dir->internal.alloc);\n \treturn dir->entries[dir->nr++] = dir_entry_new(pathname, len);\n }\n \n@@ -1753,7 +1754,7 @@ struct dir_entry *dir_add_ignored(struct dir_struct *dir,\n \tif (!index_name_is_other(istate, pathname, len))\n \t\treturn NULL;\n \n-\tALLOC_GROW(dir->ignored, dir->ignored_nr+1, dir->ignored_alloc);\n+\tALLOC_GROW(dir->ignored, dir->ignored_nr+1, dir->internal.ignored_alloc);\n \treturn dir->ignored[dir->ignored_nr++] = dir_entry_new(pathname, len);\n }\n \n@@ -2569,7 +2570,7 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,\n \n \tif (open_cached_dir(&cdir, dir, untracked, istate, &path, check_only))\n \t\tgoto out;\n-\tdir->visited_directories++;\n+\tdir->internal.visited_directories++;\n \n \tif (untracked)\n \t\tuntracked->check_only = !!check_only;\n@@ -2578,7 +2579,7 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,\n \t\t/* check how the file or directory should be treated */\n \t\tstate = treat_path(dir, untracked, &cdir, istate, &path,\n \t\t\t\t   baselen, pathspec);\n-\t\tdir->visited_paths++;\n+\t\tdir->internal.visited_paths++;\n \n \t\tif (state > dir_state)\n \t\t\tdir_state = state;\n@@ -2586,7 +2587,8 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,\n \t\t/* recurse into subdir if instructed by treat_path */\n \t\tif (state == path_recurse) {\n \t\t\tstruct untracked_cache_dir *ud;\n-\t\t\tud = lookup_untracked(dir->untracked, untracked,\n+\t\t\tud = lookup_untracked(dir->untracked,\n+\t\t\t\t\t      untracked,\n \t\t\t\t\t      path.buf + baselen,\n \t\t\t\t\t      path.len - baselen);\n \t\t\tsubdir_state =\n@@ -2846,7 +2848,7 @@ static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct *d\n \t * condition also catches running setup_standard_excludes()\n \t * before setting dir->untracked!\n \t */\n-\tif (dir->unmanaged_exclude_files)\n+\tif (dir->internal.unmanaged_exclude_files)\n \t\treturn NULL;\n \n \t/*\n@@ -2875,7 +2877,7 @@ static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct *d\n \t * EXC_CMDL is not considered in the cache. If people set it,\n \t * skip the cache.\n \t */\n-\tif (dir->exclude_list_group[EXC_CMDL].nr)\n+\tif (dir->internal.exclude_list_group[EXC_CMDL].nr)\n \t\treturn NULL;\n \n \tif (!ident_in_untracked(dir->untracked)) {\n@@ -2935,15 +2937,15 @@ static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct *d\n \n \t/* Validate $GIT_DIR/info/exclude and core.excludesfile */\n \troot = dir->untracked->root;\n-\tif (!oideq(&dir->ss_info_exclude.oid,\n+\tif (!oideq(&dir->internal.ss_info_exclude.oid,\n \t\t   &dir->untracked->ss_info_exclude.oid)) {\n \t\tinvalidate_gitignore(dir->untracked, root);\n-\t\tdir->untracked->ss_info_exclude = dir->ss_info_exclude;\n+\t\tdir->untracked->ss_info_exclude = dir->internal.ss_info_exclude;\n \t}\n-\tif (!oideq(&dir->ss_excludes_file.oid,\n+\tif (!oideq(&dir->internal.ss_excludes_file.oid,\n \t\t   &dir->untracked->ss_excludes_file.oid)) {\n \t\tinvalidate_gitignore(dir->untracked, root);\n-\t\tdir->untracked->ss_excludes_file = dir->ss_excludes_file;\n+\t\tdir->untracked->ss_excludes_file = dir->internal.ss_excludes_file;\n \t}\n \n \t/* Make sure this directory is not dropped out at saving phase */\n@@ -2969,9 +2971,9 @@ static void emit_traversal_statistics(struct dir_struct *dir,\n \t}\n \n \ttrace2_data_intmax(\"read_directory\", repo,\n-\t\t\t   \"directories-visited\", dir->visited_directories);\n+\t\t\t   \"directories-visited\", dir->internal.visited_directories);\n \ttrace2_data_intmax(\"read_directory\", repo,\n-\t\t\t   \"paths-visited\", dir->visited_paths);\n+\t\t\t   \"paths-visited\", dir->internal.visited_paths);\n \n \tif (!dir->untracked)\n \t\treturn;\n@@ -2993,8 +2995,8 @@ int read_directory(struct dir_struct *dir, struct index_state *istate,\n \tstruct untracked_cache_dir *untracked;\n \n \ttrace2_region_enter(\"dir\", \"read_directory\", istate->repo);\n-\tdir->visited_paths = 0;\n-\tdir->visited_directories = 0;\n+\tdir->internal.visited_paths = 0;\n+\tdir->internal.visited_directories = 0;\n \n \tif (has_symlink_leading_path(path, len)) {\n \t\ttrace2_region_leave(\"dir\", \"read_directory\", istate->repo);\n@@ -3342,14 +3344,14 @@ void setup_standard_excludes(struct dir_struct *dir)\n \t\texcludes_file = xdg_config_home(\"ignore\");\n \tif (excludes_file && !access_or_warn(excludes_file, R_OK, 0))\n \t\tadd_patterns_from_file_1(dir, excludes_file,\n-\t\t\t\t\t dir->untracked ? &dir->ss_excludes_file : NULL);\n+\t\t\t\t\t dir->untracked ? &dir->internal.ss_excludes_file : NULL);\n \n \t/* per repository user preference */\n \tif (startup_info->have_repository) {\n \t\tconst char *path = git_path_info_exclude();\n \t\tif (!access_or_warn(path, R_OK, 0))\n \t\t\tadd_patterns_from_file_1(dir, path,\n-\t\t\t\t\t\t dir->untracked ? &dir->ss_info_exclude : NULL);\n+\t\t\t\t\t\t dir->untracked ? &dir->internal.ss_info_exclude : NULL);\n \t}\n }\n \n@@ -3405,7 +3407,7 @@ void dir_clear(struct dir_struct *dir)\n \tstruct dir_struct new = DIR_INIT;\n \n \tfor (i = EXC_CMDL; i <= EXC_FILE; i++) {\n-\t\tgroup = &dir->exclude_list_group[i];\n+\t\tgroup = &dir->internal.exclude_list_group[i];\n \t\tfor (j = 0; j < group->nr; j++) {\n \t\t\tpl = &group->pl[j];\n \t\t\tif (i == EXC_DIRS)\n@@ -3422,13 +3424,13 @@ void dir_clear(struct dir_struct *dir)\n \tfree(dir->ignored);\n \tfree(dir->entries);\n \n-\tstk = dir->exclude_stack;\n+\tstk = dir->internal.exclude_stack;\n \twhile (stk) {\n \t\tstruct exclude_stack *prev = stk->prev;\n \t\tfree(stk);\n \t\tstk = prev;\n \t}\n-\tstrbuf_release(&dir->basebuf);\n+\tstrbuf_release(&dir->internal.basebuf);\n \n \tmemcpy(dir, &new, sizeof(*dir));\n }\ndiff --git a/dir.h b/dir.h\nindex 8acfc044181..33fd848fc8d 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -215,14 +215,9 @@ struct dir_struct {\n \t/* The number of members in `entries[]` array. */\n \tint nr;\n \n-\t/* Internal use; keeps track of allocation of `entries[]` array.*/\n-\tint alloc;\n-\n \t/* The number of members in `ignored[]` array. */\n \tint ignored_nr;\n \n-\tint ignored_alloc;\n-\n \t/* bit-field of options */\n \tenum {\n \n@@ -296,51 +291,62 @@ struct dir_struct {\n \t */\n \tstruct dir_entry **ignored;\n \n+\t/* Enable/update untracked file cache if set */\n+\tstruct untracked_cache *untracked;\n+\n \t/**\n \t * The name of the file to be read in each directory for excluded files\n \t * (typically `.gitignore`).\n \t */\n \tconst char *exclude_per_dir;\n \n-\t/*\n-\t * We maintain three groups of exclude pattern lists:\n-\t *\n-\t * EXC_CMDL lists patterns explicitly given on the command line.\n-\t * EXC_DIRS lists patterns obtained from per-directory ignore files.\n-\t * EXC_FILE lists patterns from fallback ignore files, e.g.\n-\t *   - .git/info/exclude\n-\t *   - core.excludesfile\n-\t *\n-\t * Each group contains multiple exclude lists, a single list\n-\t * per source.\n-\t */\n+\tstruct dir_struct_internal {\n+\t\t/* Keeps track of allocation of `entries[]` array.*/\n+\t\tint alloc;\n+\n+\t\t/* Keeps track of allocation of `ignored[]` array. */\n+\t\tint ignored_alloc;\n+\n+\t\t/*\n+\t\t * We maintain three groups of exclude pattern lists:\n+\t\t *\n+\t\t * EXC_CMDL lists patterns explicitly given on the command line.\n+\t\t * EXC_DIRS lists patterns obtained from per-directory ignore\n+\t\t *          files.\n+\t\t * EXC_FILE lists patterns from fallback ignore files, e.g.\n+\t\t *   - .git/info/exclude\n+\t\t *   - core.excludesfile\n+\t\t *\n+\t\t * Each group contains multiple exclude lists, a single list\n+\t\t * per source.\n+\t\t */\n #define EXC_CMDL 0\n #define EXC_DIRS 1\n #define EXC_FILE 2\n-\tstruct exclude_list_group exclude_list_group[3];\n-\n-\t/*\n-\t * Temporary variables which are used during loading of the\n-\t * per-directory exclude lists.\n-\t *\n-\t * exclude_stack points to the top of the exclude_stack, and\n-\t * basebuf contains the full path to the current\n-\t * (sub)directory in the traversal. Exclude points to the\n-\t * matching exclude struct if the directory is excluded.\n-\t */\n-\tstruct exclude_stack *exclude_stack;\n-\tstruct path_pattern *pattern;\n-\tstruct strbuf basebuf;\n-\n-\t/* Enable untracked file cache if set */\n-\tstruct untracked_cache *untracked;\n-\tstruct oid_stat ss_info_exclude;\n-\tstruct oid_stat ss_excludes_file;\n-\tunsigned unmanaged_exclude_files;\n+\t\tstruct exclude_list_group exclude_list_group[3];\n \n-\t/* Stats about the traversal */\n-\tunsigned visited_paths;\n-\tunsigned visited_directories;\n+\t\t/*\n+\t\t * Temporary variables which are used during loading of the\n+\t\t * per-directory exclude lists.\n+\t\t *\n+\t\t * exclude_stack points to the top of the exclude_stack, and\n+\t\t * basebuf contains the full path to the current\n+\t\t * (sub)directory in the traversal. Exclude points to the\n+\t\t * matching exclude struct if the directory is excluded.\n+\t\t */\n+\t\tstruct exclude_stack *exclude_stack;\n+\t\tstruct path_pattern *pattern;\n+\t\tstruct strbuf basebuf;\n+\n+\t\t/* Additional metadata related to 'untracked' */\n+\t\tstruct oid_stat ss_info_exclude;\n+\t\tstruct oid_stat ss_excludes_file;\n+\t\tunsigned unmanaged_exclude_files;\n+\n+\t\t/* Stats about the traversal */\n+\t\tunsigned visited_paths;\n+\t\tunsigned visited_directories;\n+\t} internal;\n };\n \n #define DIR_INIT { 0 }\n-- \ngitgitgadget\n\n"},{"id":"472785","messageId":"12344400fa04d279de269785f62ba988e589a9f9.1677511700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v3.git.1677511700.gitgitgadget@gmail.com","subject":"[PATCH v3 05/13] dir: mark output only fields of dir_struct as such","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-27T15:28:12Z","receivedAt":"2023-02-27T15:28:38Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nWhile at it, also group these fields together for convenience.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n dir.h | 16 ++++++++--------\n 1 file changed, 8 insertions(+), 8 deletions(-)\n\ndiff --git a/dir.h b/dir.h\nindex 2196e12630c..e8106e1ecac 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -212,12 +212,6 @@ struct untracked_cache {\n  */\n struct dir_struct {\n \n-\t/* The number of members in `entries[]` array. */\n-\tint nr;\n-\n-\t/* The number of members in `ignored[]` array. */\n-\tint ignored_nr;\n-\n \t/* bit-field of options */\n \tenum {\n \n@@ -282,14 +276,20 @@ struct dir_struct {\n \t\tDIR_SKIP_NESTED_GIT = 1<<9\n \t} flags;\n \n+\t/* The number of members in `entries[]` array. */\n+\tint nr; /* output only */\n+\n+\t/* The number of members in `ignored[]` array. */\n+\tint ignored_nr; /* output only */\n+\n \t/* An array of `struct dir_entry`, each element of which describes a path. */\n-\tstruct dir_entry **entries;\n+\tstruct dir_entry **entries; /* output only */\n \n \t/**\n \t * used for ignored paths with the `DIR_SHOW_IGNORED_TOO` and\n \t * `DIR_COLLECT_IGNORED` flags.\n \t */\n-\tstruct dir_entry **ignored;\n+\tstruct dir_entry **ignored; /* output only */\n \n \t/* Enable/update untracked file cache if set */\n \tstruct untracked_cache *untracked;\n-- \ngitgitgadget\n\n"},{"id":"472787","messageId":"4ce9fae5e7f47e71d32d71cdbb9bc07b990e86c6.1677511700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v3.git.1677511700.gitgitgadget@gmail.com","subject":"[PATCH v3 04/13] dir: add a usage note to exclude_per_dir","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-27T15:28:11Z","receivedAt":"2023-02-27T15:28:40Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nAs evidenced by the fix a couple commits ago, places in the code using\nexclude_per_dir are likely buggy and should be adapted to call\nsetup_standard_excludes() instead.  Unfortunately, the usage of\nexclude_per_dir has been hardcoded into the arguments ls-files accepts,\nso we cannot actually remove it.  Add a note that it is deprecated and\nno other callers should use it directly.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n dir.h | 8 ++++++--\n 1 file changed, 6 insertions(+), 2 deletions(-)\n\ndiff --git a/dir.h b/dir.h\nindex 33fd848fc8d..2196e12630c 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -295,8 +295,12 @@ struct dir_struct {\n \tstruct untracked_cache *untracked;\n \n \t/**\n-\t * The name of the file to be read in each directory for excluded files\n-\t * (typically `.gitignore`).\n+\t * Deprecated: ls-files is the only allowed caller; all other callers\n+\t * should leave this as NULL; it pre-dated the\n+\t * setup_standard_excludes() mechanism that replaces this.\n+\t *\n+\t * This field tracks the name of the file to be read in each directory\n+\t * for excluded files (typically `.gitignore`).\n \t */\n \tconst char *exclude_per_dir;\n \n-- \ngitgitgadget\n\n"},{"id":"472788","messageId":"cd3d4894afbd4f98b777a2b56741e4915ae51e5d.1677511700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v3.git.1677511700.gitgitgadget@gmail.com","subject":"[PATCH v3 07/13] sparse-checkout: avoid using internal API of unpack-trees","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-27T15:28:14Z","receivedAt":"2023-02-27T15:28:42Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nstruct unpack_trees_options has the following field and comment:\n\n\tstruct pattern_list *pl; /* for internal use */\n\nDespite the internal-use comment, commit e091228e17 (\"sparse-checkout:\nupdate working directory in-process\", 2019-11-21) starting setting this\nfield from an external caller.  At the time, the only way around that\nwould have been to modify unpack_trees() to take an extra pattern_list\nargument, and there's a lot of callers of that function.  However, when\nwe split update_sparsity() off as a separate function, with\nsparse-checkout being the sole caller, the need to update other callers\nwent away.  Fix this API problem by adding a pattern_list argument to\nupdate_sparsity() and stop setting the internal o.pl field directly.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n builtin/sparse-checkout.c |  3 +--\n unpack-trees.c            | 18 +++++++++++-------\n unpack-trees.h            |  3 ++-\n 3 files changed, 14 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex c3738154918..4b7390ce367 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -219,14 +219,13 @@ static int update_working_directory(struct pattern_list *pl)\n \to.dst_index = r->index;\n \tindex_state_init(&o.result, r);\n \to.skip_sparse_checkout = 0;\n-\to.pl = pl;\n \n \tsetup_work_tree();\n \n \trepo_hold_locked_index(r, &lock_file, LOCK_DIE_ON_ERROR);\n \n \tsetup_unpack_trees_porcelain(&o, \"sparse-checkout\");\n-\tresult = update_sparsity(&o);\n+\tresult = update_sparsity(&o, pl);\n \tclear_unpack_trees_porcelain(&o);\n \n \tif (result == UPDATE_SPARSITY_WARNINGS)\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex bad3120a76e..6e4ca6fe800 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -2091,10 +2091,10 @@ return_failed:\n  *\n  * CE_NEW_SKIP_WORKTREE is used internally.\n  */\n-enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n+enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n+\t\t\t\t\t    struct pattern_list *pl)\n {\n \tenum update_sparsity_result ret = UPDATE_SPARSITY_SUCCESS;\n-\tstruct pattern_list pl;\n \tint i;\n \tunsigned old_show_all_errors;\n \tint free_pattern_list = 0;\n@@ -2111,11 +2111,12 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n \ttrace_performance_enter();\n \n \t/* If we weren't given patterns, use the recorded ones */\n-\tif (!o->pl) {\n-\t\tmemset(&pl, 0, sizeof(pl));\n+\tif (!pl) {\n \t\tfree_pattern_list = 1;\n-\t\tpopulate_from_existing_patterns(o, &pl);\n+\t\tpl = xcalloc(1, sizeof(*pl));\n+\t\tpopulate_from_existing_patterns(o, pl);\n \t}\n+\to->pl = pl;\n \n \t/* Expand sparse directories as needed */\n \texpand_index(o->src_index, o->pl);\n@@ -2147,8 +2148,11 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n \n \tdisplay_warning_msgs(o);\n \to->show_all_errors = old_show_all_errors;\n-\tif (free_pattern_list)\n-\t\tclear_pattern_list(&pl);\n+\tif (free_pattern_list) {\n+\t\tclear_pattern_list(pl);\n+\t\tfree(pl);\n+\t\to->pl = NULL;\n+\t}\n \ttrace_performance_leave(\"update_sparsity\");\n \treturn ret;\n }\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 3a7b3e5f007..f3a6e4f90ef 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -112,7 +112,8 @@ enum update_sparsity_result {\n \tUPDATE_SPARSITY_WORKTREE_UPDATE_FAILURES = -2\n };\n \n-enum update_sparsity_result update_sparsity(struct unpack_trees_options *options);\n+enum update_sparsity_result update_sparsity(struct unpack_trees_options *options,\n+\t\t\t\t\t    struct pattern_list *pl);\n \n int verify_uptodate(const struct cache_entry *ce,\n \t\t    struct unpack_trees_options *o);\n-- \ngitgitgadget\n\n"},{"id":"472789","messageId":"4e86e39506c65e5a84fcd655a632ddae21be51d0.1677511700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v3.git.1677511700.gitgitgadget@gmail.com","subject":"[PATCH v3 06/13] unpack-trees: clean up some flow control","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-27T15:28:13Z","receivedAt":"2023-02-27T15:28:42Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nThe update_sparsity() function was introduced in commit 7af7a25853\n(\"unpack-trees: add a new update_sparsity() function\", 2020-03-27).\nPrior to that, unpack_trees() was used, but that had a few bugs because\nthe needs of the caller were different, and different enough that\nunpack_trees() could not easily be modified to handle both usecases.\n\nThe implementation detail that update_sparsity() was written by copying\nunpack_trees() and then streamlining it, and then modifying it in the\nneeded ways still shows through in that there are leftover vestiges in\nboth functions that are no longer needed.  Clean them up.  In\nparticular:\n\n  * update_sparsity() allows a pattern list to be passed in, but\n    unpack_trees() never should use a different pattern list.  Add a\n    check and a BUG() if this gets violated.\n  * update_sparsity() has a check early on that will BUG() if\n    o->skip_sparse_checkout is set; as such, there's no need to check\n    for that condition again later in the code.  We can simply remove\n    the check and its corresponding goto label.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 7 +++----\n 1 file changed, 3 insertions(+), 4 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 4518d33ed99..bad3120a76e 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1873,6 +1873,8 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tdie(\"unpack_trees takes at most %d trees\", MAX_UNPACK_TREES);\n \tif (o->dir)\n \t\tBUG(\"o->dir is for internal use only\");\n+\tif (o->pl)\n+\t\tBUG(\"o->pl is for internal use only\");\n \n \ttrace_performance_enter();\n \ttrace2_region_enter(\"unpack_trees\", \"unpack_trees\", the_repository);\n@@ -1899,7 +1901,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \n \tif (!core_apply_sparse_checkout || !o->update)\n \t\to->skip_sparse_checkout = 1;\n-\tif (!o->skip_sparse_checkout && !o->pl) {\n+\tif (!o->skip_sparse_checkout) {\n \t\tmemset(&pl, 0, sizeof(pl));\n \t\tfree_pattern_list = 1;\n \t\tpopulate_from_existing_patterns(o, &pl);\n@@ -2113,8 +2115,6 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n \t\tmemset(&pl, 0, sizeof(pl));\n \t\tfree_pattern_list = 1;\n \t\tpopulate_from_existing_patterns(o, &pl);\n-\t\tif (o->skip_sparse_checkout)\n-\t\t\tgoto skip_sparse_checkout;\n \t}\n \n \t/* Expand sparse directories as needed */\n@@ -2142,7 +2142,6 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o)\n \t\t\tret = UPDATE_SPARSITY_WARNINGS;\n \t}\n \n-skip_sparse_checkout:\n \tif (check_updates(o, o->src_index))\n \t\tret = UPDATE_SPARSITY_WORKTREE_UPDATE_FAILURES;\n \n-- \ngitgitgadget\n\n"},{"id":"472790","messageId":"09140cb2ac53773eae19f06e51d6276e03689fc1.1677511700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v3.git.1677511700.gitgitgadget@gmail.com","subject":"[PATCH v3 08/13] sparse-checkout: avoid using internal API of unpack-trees, take 2","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-27T15:28:15Z","receivedAt":"2023-02-27T15:28:50Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nCommit 2f6b1eb794 (\"cache API: add a \"INDEX_STATE_INIT\" macro/function,\nadd release_index()\", 2023-01-12) mistakenly added some initialization\nof a member of unpack_trees_options that was intended to be\ninternal-only.  This initialization should be done within\nupdate_sparsity() instead.\n\nNote that while o->result is mostly meant for unpack_trees() and\nupdate_sparsity() mostly operates without o->result,\ncheck_ok_to_remove() does consult it so we need to ensure it is properly\ninitialized.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n builtin/sparse-checkout.c | 1 -\n unpack-trees.c            | 1 +\n 2 files changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex 4b7390ce367..8d5ae6f2a60 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -217,7 +217,6 @@ static int update_working_directory(struct pattern_list *pl)\n \to.head_idx = -1;\n \to.src_index = r->index;\n \to.dst_index = r->index;\n-\tindex_state_init(&o.result, r);\n \to.skip_sparse_checkout = 0;\n \n \tsetup_work_tree();\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 6e4ca6fe800..c8dacd76c5f 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -2101,6 +2101,7 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n \n \told_show_all_errors = o->show_all_errors;\n \to->show_all_errors = 1;\n+\tindex_state_init(&o->result, o->src_index->repo);\n \n \t/* Sanity checks */\n \tif (!o->update || o->index_only || o->skip_sparse_checkout)\n-- \ngitgitgadget\n\n"},{"id":"472791","messageId":"4236c0d80c7dd8b8c6b45462dea6bf88f844f50d.1677511700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v3.git.1677511700.gitgitgadget@gmail.com","subject":"[PATCH v3 10/13] unpack-trees: mark fields only used internally as internal","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-27T15:28:17Z","receivedAt":"2023-02-27T15:28:50Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nContinue the work from the previous patch by finding additional fields\nwhich are only used internally but not yet explicitly marked as such,\nand include them in the internal fields struct.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 159 +++++++++++++++++++++++++------------------------\n unpack-trees.h |  26 ++++----\n 2 files changed, 95 insertions(+), 90 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex ecf89d5bfeb..dd4b55ef49e 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -66,8 +66,8 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_WARNING_TYPES] = {\n };\n \n #define ERRORMSG(o,type) \\\n-\t( ((o) && (o)->msgs[(type)]) \\\n-\t  ? ((o)->msgs[(type)])      \\\n+\t( ((o) && (o)->internal.msgs[(type)]) \\\n+\t  ? ((o)->internal.msgs[(type)])      \\\n \t  : (unpack_plumbing_errors[(type)]) )\n \n static const char *super_prefixed(const char *path, const char *super_prefix)\n@@ -108,10 +108,10 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t\t\t  const char *cmd)\n {\n \tint i;\n-\tconst char **msgs = opts->msgs;\n+\tconst char **msgs = opts->internal.msgs;\n \tconst char *msg;\n \n-\tstrvec_init(&opts->msgs_to_free);\n+\tstrvec_init(&opts->internal.msgs_to_free);\n \n \tif (!strcmp(cmd, \"checkout\"))\n \t\tmsg = advice_enabled(ADVICE_COMMIT_BEFORE_MERGE)\n@@ -129,7 +129,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t\t  \"Please commit your changes or stash them before you %s.\")\n \t\t      : _(\"Your local changes to the following files would be overwritten by %s:\\n%%s\");\n \tmsgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOT_UPTODATE_FILE] =\n-\t\tstrvec_pushf(&opts->msgs_to_free, msg, cmd, cmd);\n+\t\tstrvec_pushf(&opts->internal.msgs_to_free, msg, cmd, cmd);\n \n \tmsgs[ERROR_NOT_UPTODATE_DIR] =\n \t\t_(\"Updating the following directories would lose untracked files in them:\\n%s\");\n@@ -153,7 +153,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t\t  \"Please move or remove them before you %s.\")\n \t\t      : _(\"The following untracked working tree files would be removed by %s:\\n%%s\");\n \tmsgs[ERROR_WOULD_LOSE_UNTRACKED_REMOVED] =\n-\t\tstrvec_pushf(&opts->msgs_to_free, msg, cmd, cmd);\n+\t\tstrvec_pushf(&opts->internal.msgs_to_free, msg, cmd, cmd);\n \n \tif (!strcmp(cmd, \"checkout\"))\n \t\tmsg = advice_enabled(ADVICE_COMMIT_BEFORE_MERGE)\n@@ -171,7 +171,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t\t  \"Please move or remove them before you %s.\")\n \t\t      : _(\"The following untracked working tree files would be overwritten by %s:\\n%%s\");\n \tmsgs[ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN] =\n-\t\tstrvec_pushf(&opts->msgs_to_free, msg, cmd, cmd);\n+\t\tstrvec_pushf(&opts->internal.msgs_to_free, msg, cmd, cmd);\n \n \t/*\n \t * Special case: ERROR_BIND_OVERLAP refers to a pair of paths, we\n@@ -189,16 +189,16 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \tmsgs[WARNING_SPARSE_ORPHANED_NOT_OVERWRITTEN] =\n \t\t_(\"The following paths were already present and thus not updated despite sparse patterns:\\n%s\");\n \n-\topts->show_all_errors = 1;\n+\topts->internal.show_all_errors = 1;\n \t/* rejected paths may not have a static buffer */\n-\tfor (i = 0; i < ARRAY_SIZE(opts->unpack_rejects); i++)\n-\t\topts->unpack_rejects[i].strdup_strings = 1;\n+\tfor (i = 0; i < ARRAY_SIZE(opts->internal.unpack_rejects); i++)\n+\t\topts->internal.unpack_rejects[i].strdup_strings = 1;\n }\n \n void clear_unpack_trees_porcelain(struct unpack_trees_options *opts)\n {\n-\tstrvec_clear(&opts->msgs_to_free);\n-\tmemset(opts->msgs, 0, sizeof(opts->msgs));\n+\tstrvec_clear(&opts->internal.msgs_to_free);\n+\tmemset(opts->internal.msgs, 0, sizeof(opts->internal.msgs));\n }\n \n static int do_add_entry(struct unpack_trees_options *o, struct cache_entry *ce,\n@@ -210,7 +210,7 @@ static int do_add_entry(struct unpack_trees_options *o, struct cache_entry *ce,\n \t\tset |= CE_WT_REMOVE;\n \n \tce->ce_flags = (ce->ce_flags & ~clear) | set;\n-\treturn add_index_entry(&o->result, ce,\n+\treturn add_index_entry(&o->internal.result, ce,\n \t\t\t       ADD_CACHE_OK_TO_ADD | ADD_CACHE_OK_TO_REPLACE);\n }\n \n@@ -218,7 +218,7 @@ static void add_entry(struct unpack_trees_options *o,\n \t\t      const struct cache_entry *ce,\n \t\t      unsigned int set, unsigned int clear)\n {\n-\tdo_add_entry(o, dup_cache_entry(ce, &o->result), set, clear);\n+\tdo_add_entry(o, dup_cache_entry(ce, &o->internal.result), set, clear);\n }\n \n /*\n@@ -233,7 +233,7 @@ static int add_rejected_path(struct unpack_trees_options *o,\n \tif (o->quiet)\n \t\treturn -1;\n \n-\tif (!o->show_all_errors)\n+\tif (!o->internal.show_all_errors)\n \t\treturn error(ERRORMSG(o, e), super_prefixed(path,\n \t\t\t\t\t\t\t    o->super_prefix));\n \n@@ -241,7 +241,7 @@ static int add_rejected_path(struct unpack_trees_options *o,\n \t * Otherwise, insert in a list for future display by\n \t * display_(error|warning)_msgs()\n \t */\n-\tstring_list_append(&o->unpack_rejects[e], path);\n+\tstring_list_append(&o->internal.unpack_rejects[e], path);\n \treturn -1;\n }\n \n@@ -253,7 +253,7 @@ static void display_error_msgs(struct unpack_trees_options *o)\n \tint e;\n \tunsigned error_displayed = 0;\n \tfor (e = 0; e < NB_UNPACK_TREES_ERROR_TYPES; e++) {\n-\t\tstruct string_list *rejects = &o->unpack_rejects[e];\n+\t\tstruct string_list *rejects = &o->internal.unpack_rejects[e];\n \n \t\tif (rejects->nr > 0) {\n \t\t\tint i;\n@@ -281,7 +281,7 @@ static void display_warning_msgs(struct unpack_trees_options *o)\n \tunsigned warning_displayed = 0;\n \tfor (e = NB_UNPACK_TREES_ERROR_TYPES + 1;\n \t     e < NB_UNPACK_TREES_WARNING_TYPES; e++) {\n-\t\tstruct string_list *rejects = &o->unpack_rejects[e];\n+\t\tstruct string_list *rejects = &o->internal.unpack_rejects[e];\n \n \t\tif (rejects->nr > 0) {\n \t\t\tint i;\n@@ -600,13 +600,14 @@ static void mark_ce_used(struct cache_entry *ce, struct unpack_trees_options *o)\n {\n \tce->ce_flags |= CE_UNPACKED;\n \n-\tif (o->cache_bottom < o->src_index->cache_nr &&\n-\t    o->src_index->cache[o->cache_bottom] == ce) {\n-\t\tint bottom = o->cache_bottom;\n+\tif (o->internal.cache_bottom < o->src_index->cache_nr &&\n+\t    o->src_index->cache[o->internal.cache_bottom] == ce) {\n+\t\tint bottom = o->internal.cache_bottom;\n+\n \t\twhile (bottom < o->src_index->cache_nr &&\n \t\t       o->src_index->cache[bottom]->ce_flags & CE_UNPACKED)\n \t\t\tbottom++;\n-\t\to->cache_bottom = bottom;\n+\t\to->internal.cache_bottom = bottom;\n \t}\n }\n \n@@ -652,7 +653,7 @@ static void mark_ce_used_same_name(struct cache_entry *ce,\n static struct cache_entry *next_cache_entry(struct unpack_trees_options *o)\n {\n \tconst struct index_state *index = o->src_index;\n-\tint pos = o->cache_bottom;\n+\tint pos = o->internal.cache_bottom;\n \n \twhile (pos < index->cache_nr) {\n \t\tstruct cache_entry *ce = index->cache[pos];\n@@ -711,7 +712,7 @@ static void restore_cache_bottom(struct traverse_info *info, int bottom)\n \n \tif (o->diff_index_cached)\n \t\treturn;\n-\to->cache_bottom = bottom;\n+\to->internal.cache_bottom = bottom;\n }\n \n static int switch_cache_bottom(struct traverse_info *info)\n@@ -721,13 +722,13 @@ static int switch_cache_bottom(struct traverse_info *info)\n \n \tif (o->diff_index_cached)\n \t\treturn 0;\n-\tret = o->cache_bottom;\n+\tret = o->internal.cache_bottom;\n \tpos = find_cache_pos(info->prev, info->name, info->namelen);\n \n \tif (pos < -1)\n-\t\to->cache_bottom = -2 - pos;\n+\t\to->internal.cache_bottom = -2 - pos;\n \telse if (pos < 0)\n-\t\to->cache_bottom = o->src_index->cache_nr;\n+\t\to->internal.cache_bottom = o->src_index->cache_nr;\n \treturn ret;\n }\n \n@@ -873,9 +874,9 @@ static int traverse_trees_recursive(int n, unsigned long dirmask,\n \t\t * save and restore cache_bottom anyway to not miss\n \t\t * unprocessed entries before 'pos'.\n \t\t */\n-\t\tbottom = o->cache_bottom;\n+\t\tbottom = o->internal.cache_bottom;\n \t\tret = traverse_by_cache_tree(pos, nr_entries, n, info);\n-\t\to->cache_bottom = bottom;\n+\t\to->internal.cache_bottom = bottom;\n \t\treturn ret;\n \t}\n \n@@ -1212,7 +1213,7 @@ static int unpack_single_entry(int n, unsigned long mask,\n \t\t * cache entry from the index aware logic.\n \t\t */\n \t\tsrc[i + o->merge] = create_ce_entry(info, names + i, stage,\n-\t\t\t\t\t\t    &o->result, o->merge,\n+\t\t\t\t\t\t    &o->internal.result, o->merge,\n \t\t\t\t\t\t    bit & dirmask);\n \t}\n \n@@ -1237,7 +1238,7 @@ static int unpack_single_entry(int n, unsigned long mask,\n \n static int unpack_failed(struct unpack_trees_options *o, const char *message)\n {\n-\tdiscard_index(&o->result);\n+\tdiscard_index(&o->internal.result);\n \tif (!o->quiet && !o->exiting_early) {\n \t\tif (message)\n \t\t\treturn error(\"%s\", message);\n@@ -1260,7 +1261,7 @@ static int find_cache_pos(struct traverse_info *info,\n \tstruct index_state *index = o->src_index;\n \tint pfxlen = info->pathlen;\n \n-\tfor (pos = o->cache_bottom; pos < index->cache_nr; pos++) {\n+\tfor (pos = o->internal.cache_bottom; pos < index->cache_nr; pos++) {\n \t\tconst struct cache_entry *ce = index->cache[pos];\n \t\tconst char *ce_name, *ce_slash;\n \t\tint cmp, ce_len;\n@@ -1271,8 +1272,8 @@ static int find_cache_pos(struct traverse_info *info,\n \t\t\t * we can never match it; don't check it\n \t\t\t * again.\n \t\t\t */\n-\t\t\tif (pos == o->cache_bottom)\n-\t\t\t\t++o->cache_bottom;\n+\t\t\tif (pos == o->internal.cache_bottom)\n+\t\t\t\t++o->internal.cache_bottom;\n \t\t\tcontinue;\n \t\t}\n \t\tif (!ce_in_traverse_path(ce, info)) {\n@@ -1450,7 +1451,7 @@ static int unpack_sparse_callback(int n, unsigned long mask, unsigned long dirma\n \t */\n \tif (!is_null_oid(&names[0].oid)) {\n \t\tsrc[0] = create_ce_entry(info, &names[0], 0,\n-\t\t\t\t\t&o->result, 1,\n+\t\t\t\t\t&o->internal.result, 1,\n \t\t\t\t\tdirmask & (1ul << 0));\n \t\tsrc[0]->ce_flags |= (CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE);\n \t}\n@@ -1560,7 +1561,7 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str\n \t\t\t\t * in 'mark_ce_used()'\n \t\t\t\t */\n \t\t\t\tif (!src[0] || !S_ISSPARSEDIR(src[0]->ce_mode))\n-\t\t\t\t\to->cache_bottom += matches;\n+\t\t\t\t\to->internal.cache_bottom += matches;\n \t\t\t\treturn mask;\n \t\t\t}\n \t\t}\n@@ -1907,37 +1908,37 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tpopulate_from_existing_patterns(o, &pl);\n \t}\n \n-\tindex_state_init(&o->result, o->src_index->repo);\n-\to->result.initialized = 1;\n-\to->result.timestamp.sec = o->src_index->timestamp.sec;\n-\to->result.timestamp.nsec = o->src_index->timestamp.nsec;\n-\to->result.version = o->src_index->version;\n+\tindex_state_init(&o->internal.result, o->src_index->repo);\n+\to->internal.result.initialized = 1;\n+\to->internal.result.timestamp.sec = o->src_index->timestamp.sec;\n+\to->internal.result.timestamp.nsec = o->src_index->timestamp.nsec;\n+\to->internal.result.version = o->src_index->version;\n \tif (!o->src_index->split_index) {\n-\t\to->result.split_index = NULL;\n+\t\to->internal.result.split_index = NULL;\n \t} else if (o->src_index == o->dst_index) {\n \t\t/*\n \t\t * o->dst_index (and thus o->src_index) will be discarded\n-\t\t * and overwritten with o->result at the end of this function,\n+\t\t * and overwritten with o->internal.result at the end of this function,\n \t\t * so just use src_index's split_index to avoid having to\n \t\t * create a new one.\n \t\t */\n-\t\to->result.split_index = o->src_index->split_index;\n-\t\to->result.split_index->refcount++;\n+\t\to->internal.result.split_index = o->src_index->split_index;\n+\t\to->internal.result.split_index->refcount++;\n \t} else {\n-\t\to->result.split_index = init_split_index(&o->result);\n+\t\to->internal.result.split_index = init_split_index(&o->internal.result);\n \t}\n-\toidcpy(&o->result.oid, &o->src_index->oid);\n+\toidcpy(&o->internal.result.oid, &o->src_index->oid);\n \to->merge_size = len;\n \tmark_all_ce_unused(o->src_index);\n \n-\to->result.fsmonitor_last_update =\n+\to->internal.result.fsmonitor_last_update =\n \t\txstrdup_or_null(o->src_index->fsmonitor_last_update);\n-\to->result.fsmonitor_has_run_once = o->src_index->fsmonitor_has_run_once;\n+\to->internal.result.fsmonitor_has_run_once = o->src_index->fsmonitor_has_run_once;\n \n \tif (!o->src_index->initialized &&\n \t    !repo->settings.command_requires_full_index &&\n-\t    is_sparse_index_allowed(&o->result, 0))\n-\t\to->result.sparse_index = 1;\n+\t    is_sparse_index_allowed(&o->internal.result, 0))\n+\t\to->internal.result.sparse_index = 1;\n \n \t/*\n \t * Sparse checkout loop #1: set NEW_SKIP_WORKTREE on existing entries\n@@ -1957,7 +1958,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tsetup_traverse_info(&info, prefix);\n \t\tinfo.fn = unpack_callback;\n \t\tinfo.data = o;\n-\t\tinfo.show_all_errors = o->show_all_errors;\n+\t\tinfo.show_all_errors = o->internal.show_all_errors;\n \t\tinfo.pathspec = o->pathspec;\n \n \t\tif (o->prefix) {\n@@ -1998,7 +1999,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t}\n \tmark_all_ce_unused(o->src_index);\n \n-\tif (o->trivial_merges_only && o->nontrivial_merge) {\n+\tif (o->trivial_merges_only && o->internal.nontrivial_merge) {\n \t\tret = unpack_failed(o, \"Merge requires file-level merging\");\n \t\tgoto done;\n \t}\n@@ -2009,13 +2010,13 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t * If they will have NEW_SKIP_WORKTREE, also set CE_SKIP_WORKTREE\n \t\t * so apply_sparse_checkout() won't attempt to remove it from worktree\n \t\t */\n-\t\tmark_new_skip_worktree(o->internal.pl, &o->result,\n+\t\tmark_new_skip_worktree(o->internal.pl, &o->internal.result,\n \t\t\t\t       CE_ADDED, CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE,\n \t\t\t\t       o->verbose_update);\n \n \t\tret = 0;\n-\t\tfor (i = 0; i < o->result.cache_nr; i++) {\n-\t\t\tstruct cache_entry *ce = o->result.cache[i];\n+\t\tfor (i = 0; i < o->internal.result.cache_nr; i++) {\n+\t\t\tstruct cache_entry *ce = o->internal.result.cache[i];\n \n \t\t\t/*\n \t\t\t * Entries marked with CE_ADDED in merged_entry() do not have\n@@ -2029,7 +2030,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t\t    verify_absent(ce, WARNING_SPARSE_ORPHANED_NOT_OVERWRITTEN, o))\n \t\t\t\tret = 1;\n \n-\t\t\tif (apply_sparse_checkout(&o->result, ce, o))\n+\t\t\tif (apply_sparse_checkout(&o->internal.result, ce, o))\n \t\t\t\tret = 1;\n \t\t}\n \t\tif (ret == 1) {\n@@ -2037,30 +2038,30 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t\t * Inability to sparsify or de-sparsify individual\n \t\t\t * paths is not an error, but just a warning.\n \t\t\t */\n-\t\t\tif (o->show_all_errors)\n+\t\t\tif (o->internal.show_all_errors)\n \t\t\t\tdisplay_warning_msgs(o);\n \t\t\tret = 0;\n \t\t}\n \t}\n \n-\tret = check_updates(o, &o->result) ? (-2) : 0;\n+\tret = check_updates(o, &o->internal.result) ? (-2) : 0;\n \tif (o->dst_index) {\n-\t\tmove_index_extensions(&o->result, o->src_index);\n+\t\tmove_index_extensions(&o->internal.result, o->src_index);\n \t\tif (!ret) {\n \t\t\tif (git_env_bool(\"GIT_TEST_CHECK_CACHE_TREE\", 0))\n-\t\t\t\tcache_tree_verify(the_repository, &o->result);\n+\t\t\t\tcache_tree_verify(the_repository, &o->internal.result);\n \t\t\tif (!o->skip_cache_tree_update &&\n-\t\t\t    !cache_tree_fully_valid(o->result.cache_tree))\n-\t\t\t\tcache_tree_update(&o->result,\n+\t\t\t    !cache_tree_fully_valid(o->internal.result.cache_tree))\n+\t\t\t\tcache_tree_update(&o->internal.result,\n \t\t\t\t\t\t  WRITE_TREE_SILENT |\n \t\t\t\t\t\t  WRITE_TREE_REPAIR);\n \t\t}\n \n-\t\to->result.updated_workdir = 1;\n+\t\to->internal.result.updated_workdir = 1;\n \t\tdiscard_index(o->dst_index);\n-\t\t*o->dst_index = o->result;\n+\t\t*o->dst_index = o->internal.result;\n \t} else {\n-\t\tdiscard_index(&o->result);\n+\t\tdiscard_index(&o->internal.result);\n \t}\n \to->src_index = NULL;\n \n@@ -2076,7 +2077,7 @@ done:\n \treturn ret;\n \n return_failed:\n-\tif (o->show_all_errors)\n+\tif (o->internal.show_all_errors)\n \t\tdisplay_error_msgs(o);\n \tmark_all_ce_unused(o->src_index);\n \tret = unpack_failed(o, NULL);\n@@ -2099,9 +2100,9 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n \tunsigned old_show_all_errors;\n \tint free_pattern_list = 0;\n \n-\told_show_all_errors = o->show_all_errors;\n-\to->show_all_errors = 1;\n-\tindex_state_init(&o->result, o->src_index->repo);\n+\told_show_all_errors = o->internal.show_all_errors;\n+\to->internal.show_all_errors = 1;\n+\tindex_state_init(&o->internal.result, o->src_index->repo);\n \n \t/* Sanity checks */\n \tif (!o->update || o->index_only || o->skip_sparse_checkout)\n@@ -2148,7 +2149,7 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n \t\tret = UPDATE_SPARSITY_WORKTREE_UPDATE_FAILURES;\n \n \tdisplay_warning_msgs(o);\n-\to->show_all_errors = old_show_all_errors;\n+\to->internal.show_all_errors = old_show_all_errors;\n \tif (free_pattern_list) {\n \t\tclear_pattern_list(pl);\n \t\tfree(pl);\n@@ -2248,15 +2249,15 @@ static int verify_uptodate_sparse(const struct cache_entry *ce,\n }\n \n /*\n- * TODO: We should actually invalidate o->result, not src_index [1].\n+ * TODO: We should actually invalidate o->internal.result, not src_index [1].\n  * But since cache tree and untracked cache both are not copied to\n- * o->result until unpacking is complete, we invalidate them on\n+ * o->internal.result until unpacking is complete, we invalidate them on\n  * src_index instead with the assumption that they will be copied to\n  * dst_index at the end.\n  *\n  * [1] src_index->cache_tree is also used in unpack_callback() so if\n- * we invalidate o->result, we need to update it to use\n- * o->result.cache_tree as well.\n+ * we invalidate o->internal.result, we need to update it to use\n+ * o->internal.result.cache_tree as well.\n  */\n static void invalidate_ce_path(const struct cache_entry *ce,\n \t\t\t       struct unpack_trees_options *o)\n@@ -2424,7 +2425,7 @@ static int check_ok_to_remove(const char *name, int len, int dtype,\n \t * delete this path, which is in a subdirectory that\n \t * is being replaced with a blob.\n \t */\n-\tresult = index_file_exists(&o->result, name, len, 0);\n+\tresult = index_file_exists(&o->internal.result, name, len, 0);\n \tif (result) {\n \t\tif (result->ce_flags & CE_REMOVE)\n \t\t\treturn 0;\n@@ -2525,7 +2526,7 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\tstruct unpack_trees_options *o)\n {\n \tint update = CE_UPDATE;\n-\tstruct cache_entry *merge = dup_cache_entry(ce, &o->result);\n+\tstruct cache_entry *merge = dup_cache_entry(ce, &o->internal.result);\n \n \tif (!old) {\n \t\t/*\n@@ -2620,7 +2621,7 @@ static int merged_sparse_dir(const struct cache_entry * const *src, int n,\n \tsetup_traverse_info(&info, src[0]->name);\n \tinfo.fn = unpack_sparse_callback;\n \tinfo.data = o;\n-\tinfo.show_all_errors = o->show_all_errors;\n+\tinfo.show_all_errors = o->internal.show_all_errors;\n \tinfo.pathspec = o->pathspec;\n \n \t/* Get the tree descriptors of the sparse directory in each of the merging trees */\n@@ -2838,7 +2839,7 @@ int threeway_merge(const struct cache_entry * const *stages,\n \t\t\treturn -1;\n \t}\n \n-\to->nontrivial_merge = 1;\n+\to->internal.nontrivial_merge = 1;\n \n \t/* #2, #3, #4, #6, #7, #9, #10, #11. */\n \tcount = 0;\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 5c1a9314a06..0335c89bc75 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -59,7 +59,6 @@ struct unpack_trees_options {\n \t\t     preserve_ignored,\n \t\t     clone,\n \t\t     index_only,\n-\t\t     nontrivial_merge,\n \t\t     trivial_merges_only,\n \t\t     verbose_update,\n \t\t     aggressive,\n@@ -70,22 +69,13 @@ struct unpack_trees_options {\n \t\t     skip_sparse_checkout,\n \t\t     quiet,\n \t\t     exiting_early,\n-\t\t     show_all_errors,\n \t\t     dry_run,\n \t\t     skip_cache_tree_update;\n \tenum unpack_trees_reset_type reset;\n \tconst char *prefix;\n \tconst char *super_prefix;\n-\tint cache_bottom;\n \tstruct pathspec *pathspec;\n \tmerge_fn_t fn;\n-\tconst char *msgs[NB_UNPACK_TREES_WARNING_TYPES];\n-\tstruct strvec msgs_to_free;\n-\t/*\n-\t * Store error messages in an array, each case\n-\t * corresponding to a error message type\n-\t */\n-\tstruct string_list unpack_rejects[NB_UNPACK_TREES_WARNING_TYPES];\n \n \tint head_idx;\n \tint merge_size;\n@@ -95,11 +85,25 @@ struct unpack_trees_options {\n \n \tstruct index_state *dst_index;\n \tstruct index_state *src_index;\n-\tstruct index_state result;\n \n \tstruct checkout_metadata meta;\n \n \tstruct unpack_trees_options_internal {\n+\t\tunsigned int nontrivial_merge,\n+\t\t\t     show_all_errors;\n+\n+\t\tint cache_bottom;\n+\t\tconst char *msgs[NB_UNPACK_TREES_WARNING_TYPES];\n+\t\tstruct strvec msgs_to_free;\n+\n+\t\t/*\n+\t\t * Store error messages in an array, each case\n+\t\t * corresponding to a error message type\n+\t\t */\n+\t\tstruct string_list unpack_rejects[NB_UNPACK_TREES_WARNING_TYPES];\n+\n+\t\tstruct index_state result;\n+\n \t\tstruct pattern_list *pl;\n \t\tstruct dir_struct *dir;\n \t} internal;\n-- \ngitgitgadget\n\n"},{"id":"472792","messageId":"27f2d4771166348f5dd73bdccb4eeba5b16a7416.1677511700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v3.git.1677511700.gitgitgadget@gmail.com","subject":"[PATCH v3 09/13] unpack_trees: start splitting internal fields from public API","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-27T15:28:16Z","receivedAt":"2023-02-27T15:29:05Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nThis just splits the two fields already marked as internal-only into a\nseparate internal struct.  Future commits will add more fields that\nwere meant to be internal-only but were not explicitly marked as such\nto the same struct.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 40 ++++++++++++++++++++--------------------\n unpack-trees.h |  7 +++++--\n 2 files changed, 25 insertions(+), 22 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex c8dacd76c5f..ecf89d5bfeb 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1809,7 +1809,7 @@ static void populate_from_existing_patterns(struct unpack_trees_options *o,\n \tif (get_sparse_checkout_patterns(pl) < 0)\n \t\to->skip_sparse_checkout = 1;\n \telse\n-\t\to->pl = pl;\n+\t\to->internal.pl = pl;\n }\n \n static void update_sparsity_for_prefix(const char *prefix,\n@@ -1871,10 +1871,10 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \n \tif (len > MAX_UNPACK_TREES)\n \t\tdie(\"unpack_trees takes at most %d trees\", MAX_UNPACK_TREES);\n-\tif (o->dir)\n-\t\tBUG(\"o->dir is for internal use only\");\n-\tif (o->pl)\n-\t\tBUG(\"o->pl is for internal use only\");\n+\tif (o->internal.dir)\n+\t\tBUG(\"o->internal.dir is for internal use only\");\n+\tif (o->internal.pl)\n+\t\tBUG(\"o->internal.pl is for internal use only\");\n \n \ttrace_performance_enter();\n \ttrace2_region_enter(\"unpack_trees\", \"unpack_trees\", the_repository);\n@@ -1891,9 +1891,9 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tBUG(\"UNPACK_RESET_OVERWRITE_UNTRACKED incompatible with preserved ignored files\");\n \n \tif (!o->preserve_ignored) {\n-\t\to->dir = &dir;\n-\t\to->dir->flags |= DIR_SHOW_IGNORED;\n-\t\tsetup_standard_excludes(o->dir);\n+\t\to->internal.dir = &dir;\n+\t\to->internal.dir->flags |= DIR_SHOW_IGNORED;\n+\t\tsetup_standard_excludes(o->internal.dir);\n \t}\n \n \tif (o->prefix)\n@@ -1943,7 +1943,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t * Sparse checkout loop #1: set NEW_SKIP_WORKTREE on existing entries\n \t */\n \tif (!o->skip_sparse_checkout)\n-\t\tmark_new_skip_worktree(o->pl, o->src_index, 0,\n+\t\tmark_new_skip_worktree(o->internal.pl, o->src_index, 0,\n \t\t\t\t       CE_NEW_SKIP_WORKTREE, o->verbose_update);\n \n \tif (!dfc)\n@@ -2009,7 +2009,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t * If they will have NEW_SKIP_WORKTREE, also set CE_SKIP_WORKTREE\n \t\t * so apply_sparse_checkout() won't attempt to remove it from worktree\n \t\t */\n-\t\tmark_new_skip_worktree(o->pl, &o->result,\n+\t\tmark_new_skip_worktree(o->internal.pl, &o->result,\n \t\t\t\t       CE_ADDED, CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE,\n \t\t\t\t       o->verbose_update);\n \n@@ -2067,9 +2067,9 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n done:\n \tif (free_pattern_list)\n \t\tclear_pattern_list(&pl);\n-\tif (o->dir) {\n-\t\tdir_clear(o->dir);\n-\t\to->dir = NULL;\n+\tif (o->internal.dir) {\n+\t\tdir_clear(o->internal.dir);\n+\t\to->internal.dir = NULL;\n \t}\n \ttrace2_region_leave(\"unpack_trees\", \"unpack_trees\", the_repository);\n \ttrace_performance_leave(\"unpack_trees\");\n@@ -2117,14 +2117,14 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n \t\tpl = xcalloc(1, sizeof(*pl));\n \t\tpopulate_from_existing_patterns(o, pl);\n \t}\n-\to->pl = pl;\n+\to->internal.pl = pl;\n \n \t/* Expand sparse directories as needed */\n-\texpand_index(o->src_index, o->pl);\n+\texpand_index(o->src_index, o->internal.pl);\n \n \t/* Set NEW_SKIP_WORKTREE on existing entries. */\n \tmark_all_ce_unused(o->src_index);\n-\tmark_new_skip_worktree(o->pl, o->src_index, 0,\n+\tmark_new_skip_worktree(o->internal.pl, o->src_index, 0,\n \t\t\t       CE_NEW_SKIP_WORKTREE, o->verbose_update);\n \n \t/* Then loop over entries and update/remove as needed */\n@@ -2152,7 +2152,7 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n \tif (free_pattern_list) {\n \t\tclear_pattern_list(pl);\n \t\tfree(pl);\n-\t\to->pl = NULL;\n+\t\to->internal.pl = NULL;\n \t}\n \ttrace_performance_leave(\"update_sparsity\");\n \treturn ret;\n@@ -2340,7 +2340,7 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n \tpathbuf = xstrfmt(\"%.*s/\", namelen, ce->name);\n \n \tmemset(&d, 0, sizeof(d));\n-\tif (o->dir)\n+\tif (o->internal.dir)\n \t\tsetup_standard_excludes(&d);\n \ti = read_directory(&d, o->src_index, pathbuf, namelen+1, NULL);\n \tdir_clear(&d);\n@@ -2395,8 +2395,8 @@ static int check_ok_to_remove(const char *name, int len, int dtype,\n \tif (ignore_case && icase_exists(o, name, len, st))\n \t\treturn 0;\n \n-\tif (o->dir &&\n-\t    is_excluded(o->dir, o->src_index, name, &dtype))\n+\tif (o->internal.dir &&\n+\t    is_excluded(o->internal.dir, o->src_index, name, &dtype))\n \t\t/*\n \t\t * ce->name is explicitly excluded, so it is Ok to\n \t\t * overwrite it.\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex f3a6e4f90ef..5c1a9314a06 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -97,9 +97,12 @@ struct unpack_trees_options {\n \tstruct index_state *src_index;\n \tstruct index_state result;\n \n-\tstruct pattern_list *pl; /* for internal use */\n-\tstruct dir_struct *dir; /* for internal use only */\n \tstruct checkout_metadata meta;\n+\n+\tstruct unpack_trees_options_internal {\n+\t\tstruct pattern_list *pl;\n+\t\tstruct dir_struct *dir;\n+\t} internal;\n };\n \n int unpack_trees(unsigned n, struct tree_desc *t,\n-- \ngitgitgadget\n\n"},{"id":"472793","messageId":"76f4a544e4bcf1207d2db68e26a2b12221c84abd.1677511700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v3.git.1677511700.gitgitgadget@gmail.com","subject":"[PATCH v3 11/13] unpack-trees: rewrap a few overlong lines from previous patch","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-27T15:28:18Z","receivedAt":"2023-02-27T15:29:07Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nThe previous patch made many lines a little longer, resulting in four\nbecoming a bit too long.  They were left as-is for the previous patch\nto facilitate reviewers verifying that we were just adding \"internal.\"\nin a bunch of places, but rewrap them now.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 16 +++++++++-------\n 1 file changed, 9 insertions(+), 7 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex dd4b55ef49e..cac5dd0da37 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1213,8 +1213,8 @@ static int unpack_single_entry(int n, unsigned long mask,\n \t\t * cache entry from the index aware logic.\n \t\t */\n \t\tsrc[i + o->merge] = create_ce_entry(info, names + i, stage,\n-\t\t\t\t\t\t    &o->internal.result, o->merge,\n-\t\t\t\t\t\t    bit & dirmask);\n+\t\t\t\t\t\t    &o->internal.result,\n+\t\t\t\t\t\t    o->merge, bit & dirmask);\n \t}\n \n \tif (o->merge) {\n@@ -1918,14 +1918,15 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t} else if (o->src_index == o->dst_index) {\n \t\t/*\n \t\t * o->dst_index (and thus o->src_index) will be discarded\n-\t\t * and overwritten with o->internal.result at the end of this function,\n-\t\t * so just use src_index's split_index to avoid having to\n-\t\t * create a new one.\n+\t\t * and overwritten with o->internal.result at the end of\n+\t\t * this function, so just use src_index's split_index to\n+\t\t * avoid having to create a new one.\n \t\t */\n \t\to->internal.result.split_index = o->src_index->split_index;\n \t\to->internal.result.split_index->refcount++;\n \t} else {\n-\t\to->internal.result.split_index = init_split_index(&o->internal.result);\n+\t\to->internal.result.split_index =\n+\t\t\tinit_split_index(&o->internal.result);\n \t}\n \toidcpy(&o->internal.result.oid, &o->src_index->oid);\n \to->merge_size = len;\n@@ -2049,7 +2050,8 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tmove_index_extensions(&o->internal.result, o->src_index);\n \t\tif (!ret) {\n \t\t\tif (git_env_bool(\"GIT_TEST_CHECK_CACHE_TREE\", 0))\n-\t\t\t\tcache_tree_verify(the_repository, &o->internal.result);\n+\t\t\t\tcache_tree_verify(the_repository,\n+\t\t\t\t\t\t  &o->internal.result);\n \t\t\tif (!o->skip_cache_tree_update &&\n \t\t\t    !cache_tree_fully_valid(o->internal.result.cache_tree))\n \t\t\t\tcache_tree_update(&o->internal.result,\n-- \ngitgitgadget\n\n"},{"id":"472794","messageId":"ee36935adb5e2e3bb78b4bdf36809d9edb47d5ff.1677511700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v3.git.1677511700.gitgitgadget@gmail.com","subject":"[PATCH v3 12/13] unpack-trees: special case read-tree debugging as internal usage","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-27T15:28:19Z","receivedAt":"2023-02-27T15:29:09Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nbuiltin/read-tree.c has some special functionality explicitly designed\nfor debugging unpack-trees.[ch].  Associated with that is two fields\nthat no other external caller would or should use.  Mark these as\ninternal to unpack-trees, but allow builtin/read-tree to read or write\nthem for this special case.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n builtin/read-tree.c | 10 +++++-----\n unpack-trees.c      | 22 +++++++++++-----------\n unpack-trees.h      |  6 +++---\n 3 files changed, 19 insertions(+), 19 deletions(-)\n\ndiff --git a/builtin/read-tree.c b/builtin/read-tree.c\nindex 3ce75417833..6034408d486 100644\n--- a/builtin/read-tree.c\n+++ b/builtin/read-tree.c\n@@ -87,9 +87,9 @@ static int debug_merge(const struct cache_entry * const *stages,\n {\n \tint i;\n \n-\tprintf(\"* %d-way merge\\n\", o->merge_size);\n+\tprintf(\"* %d-way merge\\n\", o->internal.merge_size);\n \tdebug_stage(\"index\", stages[0], o);\n-\tfor (i = 1; i <= o->merge_size; i++) {\n+\tfor (i = 1; i <= o->internal.merge_size; i++) {\n \t\tchar buf[24];\n \t\txsnprintf(buf, sizeof(buf), \"ent#%d\", i);\n \t\tdebug_stage(buf, stages[i], o);\n@@ -144,7 +144,7 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix)\n \t\tOPT__DRY_RUN(&opts.dry_run, N_(\"don't update the index or the work tree\")),\n \t\tOPT_BOOL(0, \"no-sparse-checkout\", &opts.skip_sparse_checkout,\n \t\t\t N_(\"skip applying sparse checkout filter\")),\n-\t\tOPT_BOOL(0, \"debug-unpack\", &opts.debug_unpack,\n+\t\tOPT_BOOL(0, \"debug-unpack\", &opts.internal.debug_unpack,\n \t\t\t N_(\"debug unpack-trees\")),\n \t\tOPT_CALLBACK_F(0, \"recurse-submodules\", NULL,\n \t\t\t    \"checkout\", \"control recursive updating of submodules\",\n@@ -247,7 +247,7 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix)\n \t\t\topts.head_idx = 1;\n \t}\n \n-\tif (opts.debug_unpack)\n+\tif (opts.internal.debug_unpack)\n \t\topts.fn = debug_merge;\n \n \t/* If we're going to prime_cache_tree later, skip cache tree update */\n@@ -263,7 +263,7 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix)\n \tif (unpack_trees(nr_trees, t, &opts))\n \t\treturn 128;\n \n-\tif (opts.debug_unpack || opts.dry_run)\n+\tif (opts.internal.debug_unpack || opts.dry_run)\n \t\treturn 0; /* do not write the index out */\n \n \t/*\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex cac5dd0da37..3e5f4bd2355 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -839,7 +839,7 @@ static int traverse_by_cache_tree(int pos, int nr_entries, int nr_names,\n \t\tmark_ce_used(src[0], o);\n \t}\n \tfree(tree_ce);\n-\tif (o->debug_unpack)\n+\tif (o->internal.debug_unpack)\n \t\tprintf(\"Unpacked %d entries from %s to %s using cache-tree\\n\",\n \t\t       nr_entries,\n \t\t       o->src_index->cache[pos]->name,\n@@ -1488,7 +1488,7 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str\n \twhile (!p->mode)\n \t\tp++;\n \n-\tif (o->debug_unpack)\n+\tif (o->internal.debug_unpack)\n \t\tdebug_unpack_callback(n, mask, dirmask, names, info);\n \n \t/* Are we supposed to look at the index too? */\n@@ -1929,7 +1929,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t\tinit_split_index(&o->internal.result);\n \t}\n \toidcpy(&o->internal.result.oid, &o->src_index->oid);\n-\to->merge_size = len;\n+\to->internal.merge_size = len;\n \tmark_all_ce_unused(o->src_index);\n \n \to->internal.result.fsmonitor_last_update =\n@@ -2882,9 +2882,9 @@ int twoway_merge(const struct cache_entry * const *src,\n \tconst struct cache_entry *oldtree = src[1];\n \tconst struct cache_entry *newtree = src[2];\n \n-\tif (o->merge_size != 2)\n+\tif (o->internal.merge_size != 2)\n \t\treturn error(\"Cannot do a twoway merge of %d trees\",\n-\t\t\t     o->merge_size);\n+\t\t\t     o->internal.merge_size);\n \n \tif (oldtree == o->df_conflict_entry)\n \t\toldtree = NULL;\n@@ -2964,9 +2964,9 @@ int bind_merge(const struct cache_entry * const *src,\n \tconst struct cache_entry *old = src[0];\n \tconst struct cache_entry *a = src[1];\n \n-\tif (o->merge_size != 1)\n+\tif (o->internal.merge_size != 1)\n \t\treturn error(\"Cannot do a bind merge of %d trees\",\n-\t\t\t     o->merge_size);\n+\t\t\t     o->internal.merge_size);\n \tif (a && old)\n \t\treturn o->quiet ? -1 :\n \t\t\terror(ERRORMSG(o, ERROR_BIND_OVERLAP),\n@@ -2990,9 +2990,9 @@ int oneway_merge(const struct cache_entry * const *src,\n \tconst struct cache_entry *old = src[0];\n \tconst struct cache_entry *a = src[1];\n \n-\tif (o->merge_size != 1)\n+\tif (o->internal.merge_size != 1)\n \t\treturn error(\"Cannot do a oneway merge of %d trees\",\n-\t\t\t     o->merge_size);\n+\t\t\t     o->internal.merge_size);\n \n \tif (!a || a == o->df_conflict_entry)\n \t\treturn deleted_entry(old, old, o);\n@@ -3027,8 +3027,8 @@ int stash_worktree_untracked_merge(const struct cache_entry * const *src,\n \tconst struct cache_entry *worktree = src[1];\n \tconst struct cache_entry *untracked = src[2];\n \n-\tif (o->merge_size != 2)\n-\t\tBUG(\"invalid merge_size: %d\", o->merge_size);\n+\tif (o->internal.merge_size != 2)\n+\t\tBUG(\"invalid merge_size: %d\", o->internal.merge_size);\n \n \tif (worktree && untracked)\n \t\treturn error(_(\"worktree and untracked commit have duplicate entries: %s\"),\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 0335c89bc75..e8737adfeda 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -65,7 +65,6 @@ struct unpack_trees_options {\n \t\t     skip_unmerged,\n \t\t     initial_checkout,\n \t\t     diff_index_cached,\n-\t\t     debug_unpack,\n \t\t     skip_sparse_checkout,\n \t\t     quiet,\n \t\t     exiting_early,\n@@ -78,7 +77,6 @@ struct unpack_trees_options {\n \tmerge_fn_t fn;\n \n \tint head_idx;\n-\tint merge_size;\n \n \tstruct cache_entry *df_conflict_entry;\n \tvoid *unpack_data;\n@@ -90,8 +88,10 @@ struct unpack_trees_options {\n \n \tstruct unpack_trees_options_internal {\n \t\tunsigned int nontrivial_merge,\n-\t\t\t     show_all_errors;\n+\t\t\t     show_all_errors,\n+\t\t\t     debug_unpack; /* used by read-tree debugging */\n \n+\t\tint merge_size; /* used by read-tree debugging */\n \t\tint cache_bottom;\n \t\tconst char *msgs[NB_UNPACK_TREES_WARNING_TYPES];\n \t\tstruct strvec msgs_to_free;\n-- \ngitgitgadget\n\n"},{"id":"472795","messageId":"6575c007577c0ae78125bc138ff0eb56ef1c4c1e.1677511700.git.gitgitgadget@gmail.com","threadId":"59297","inReplyTo":"pull.1149.v3.git.1677511700.gitgitgadget@gmail.com","subject":"[PATCH v3 13/13] unpack-trees: add usage notices around df_conflict_entry","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2023-02-27T15:28:20Z","receivedAt":"2023-02-27T15:29:11Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nAvoid making users believe they need to initialize df_conflict_entry\nto something (as happened with other output only fields before) with\na quick comment and a small sanity check.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n unpack-trees.c | 2 ++\n unpack-trees.h | 2 +-\n 2 files changed, 3 insertions(+), 1 deletion(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 3e5f4bd2355..a37ab292bbd 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1876,6 +1876,8 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tBUG(\"o->internal.dir is for internal use only\");\n \tif (o->internal.pl)\n \t\tBUG(\"o->internal.pl is for internal use only\");\n+\tif (o->df_conflict_entry)\n+\t\tBUG(\"o->df_conflict_entry is an output only field\");\n \n \ttrace_performance_enter();\n \ttrace2_region_enter(\"unpack_trees\", \"unpack_trees\", the_repository);\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex e8737adfeda..61c06eb7c50 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -78,7 +78,7 @@ struct unpack_trees_options {\n \n \tint head_idx;\n \n-\tstruct cache_entry *df_conflict_entry;\n+\tstruct cache_entry *df_conflict_entry; /* output only */\n \tvoid *unpack_data;\n \n \tstruct index_state *dst_index;\n-- \ngitgitgadget\n"},{"id":"472815","messageId":"0aa0562e-7aa6-9442-09da-46f303d76b08@github.com","threadId":"59297","inReplyTo":"7bbc4577a57fc9cb9debc249af232282b6b3f24f.1677511700.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 01/13] t2021: fix platform-specific leftover cruft","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2023-02-27T19:11:26Z","receivedAt":"2023-02-27T19:11:32Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 2/27/2023 10:28 AM, Elijah Newren via GitGitGadget wrote:\n> From: Elijah Newren <newren@gmail.com>\n\n>  test_expect_success SYMLINKS 'the symlink remained' '\n>  \n> -\ttest_when_finished \"rm a/b\" &&\n>  \ttest -h a/b\n>  '\n>  \n> +test_expect_success 'cleanup after previous symlink tests' '\n> +\trm a/b\n> +'\n\nI was confused why this worked without \"rm -f a/b\" and it seems\nthe path exists in all cases, it's just a symlink on the filesystem\nin the case of the SYMLINKS prerequisite.\n\nThanks,\n-Stolee\n"},{"id":"472830","messageId":"20230227232047.2441760-1-jonathantanmy@google.com","threadId":"59297","inReplyTo":"8ffdb6c8a8a3b162c898ba759137d41bbfd44b39.1677511700.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 02/13] unpack-trees: heed requests to overwrite ignored files","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-02-27T23:20:47Z","receivedAt":"2023-02-27T23:21:28Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"\"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> diff --git a/unpack-trees.c b/unpack-trees.c\n> index 3d05e45a279..4518d33ed99 100644\n> --- a/unpack-trees.c\n> +++ b/unpack-trees.c\n> @@ -2337,7 +2337,7 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n>  \n>  \tmemset(&d, 0, sizeof(d));\n>  \tif (o->dir)\n> -\t\td.exclude_per_dir = o->dir->exclude_per_dir;\n> +\t\tsetup_standard_excludes(&d);\n>  \ti = read_directory(&d, o->src_index, pathbuf, namelen+1, NULL);\n>  \tdir_clear(&d);\n>  \tfree(pathbuf);\n\nThanks to the later patches in this patch set, I only needed to look at\nunpack-trees.c to see how o->dir (later, o->internal.dir) is set. The\nonly place it is set is in unpack_trees(), in which a flag is set and\nsetup_standard_excludes() is called. So the RHS of the diff here does\neffectively the same thing as the LHS. (As for the flag, it is not set\nin the RHS, but it was not set in the LHS in the first place, so that's\nfine.)\n\nThanks - all 13 patches in this patch set look good.\n"}]}