{"thread":{"id":"58613","subject":"[PATCH] archive: add --recurse-submodules to git-archive command","startedAt":"2022-10-12T17:52:36Z","lastAt":"2022-10-28T18:18:34Z","messageCount":48,"participants":["Heather Lapointe via GitGitGadget","Alphadelta14 via GitGitGadget","René Scharfe","Heather Lapointe","Junio C Hamano","Phillip Wood","Glen Choo","Jonathan Tan","Ævar Arnfjörð Bjarmason"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"464733","messageId":"pull.1359.git.git.1665597148042.gitgitgadget@gmail.com","threadId":"58613","inReplyTo":null,"subject":"[PATCH] archive: add --recurse-submodules to git-archive command","fromName":"Heather Lapointe via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-12T17:52:27Z","receivedAt":"2022-10-12T17:52:36Z","isPatch":true,"sender":{"key":"alpha@alphaservcomputing.solutions","avatar":null},"body":"From: Alphadelta14 <alpha@alphaservcomputing.solutions>\n\nThis makes it possible to include submodule contents in an archive command.\n\nThis required updating the general read_tree callbacks to support sub-repos\nby not using the_repository global references where possible.\n\narchive: update streaming to use target repo\narchive: add test cases for git archive --recurse-submodules\n\nSigned-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n---\n    archive: Add --recurse-submodules to git-archive command\n    \n    This makes it possible to include submodule contents in an archive\n    command.\n    \n    The inspiration for this change comes from this Github thread,\n    https://github.com/dear-github/dear-github/issues/214, with at least 160\n    👍🏻 's at the time of writing. (I stumbled upon it because I wanted it\n    as well).\n    \n    I figured the underlying implementation wouldn't be too difficult with\n    most of the plumbing already in place, so I decided to add the relevant\n    logic to the client git-archive command.\n    \n    One of the trickier parts of this implementation involved teaching\n    read_tree about submodules. Some of the troublesome areas were still\n    using the the_repository references to look up commit or tree or oid\n    information. I ended up deciding that read_tree_fn_t would probably be\n    best off having a concrete repo reference since it allows changing the\n    context to a subrepo where needed (even though some of the usages did\n    not need it specifically).\n    \n    I am open to feedback since this is all quite new to me :)\n    \n    TODO:\n    \n     * working implementation\n     * valgrind\n     * add regression tests\n     * update documentation with new flag\n     * submit to mailing list\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1359%2FAlphadelta14%2Farchive-recurse-submodules-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1359/Alphadelta14/archive-recurse-submodules-v1\nPull-Request: https://github.com/git/git/pull/1359\n\n archive-tar.c                 | 14 +++--\n archive-zip.c                 | 14 ++---\n archive.c                     | 99 ++++++++++++++++++++++++-----------\n archive.h                     |  8 +--\n builtin/checkout.c            |  2 +-\n builtin/log.c                 |  2 +-\n builtin/ls-files.c            | 10 ++--\n builtin/ls-tree.c             | 16 +++---\n list-objects.c                |  2 +-\n merge-recursive.c             |  2 +-\n revision.c                    |  4 +-\n sparse-index.c                |  2 +-\n t/t5005-archive-submodules.sh | 84 +++++++++++++++++++++++++++++\n tree.c                        | 64 ++++++++++++++--------\n tree.h                        | 11 ++--\n wt-status.c                   |  2 +-\n 16 files changed, 246 insertions(+), 90 deletions(-)\n create mode 100755 t/t5005-archive-submodules.sh\n\ndiff --git a/archive-tar.c b/archive-tar.c\nindex 3e4822b6840..331fc10bca9 100644\n--- a/archive-tar.c\n+++ b/archive-tar.c\n@@ -18,6 +18,7 @@ static unsigned long offset;\n static int tar_umask = 002;\n \n static int write_tar_filter_archive(const struct archiver *ar,\n+\t\t\t\t    struct repository *repo,\n \t\t\t\t    struct archiver_args *args);\n \n /*\n@@ -246,7 +247,8 @@ static void write_extended_header(struct archiver_args *args,\n \twrite_blocked(buffer, size);\n }\n \n-static int write_tar_entry(struct archiver_args *args,\n+static int write_tar_entry(struct repository *repo,\n+\t\t\t   struct archiver_args *args,\n \t\t\t   const struct object_id *oid,\n \t\t\t   const char *path, size_t pathlen,\n \t\t\t   unsigned int mode,\n@@ -316,7 +318,7 @@ static int write_tar_entry(struct archiver_args *args,\n \t\tif (buffer)\n \t\t\twrite_blocked(buffer, size);\n \t\telse\n-\t\t\terr = stream_blocked(args->repo, oid);\n+\t\t\terr = stream_blocked(repo, oid);\n \t}\n \treturn err;\n }\n@@ -422,12 +424,13 @@ static int git_tar_config(const char *var, const char *value, void *cb)\n }\n \n static int write_tar_archive(const struct archiver *ar UNUSED,\n+\t\t\t     struct repository *repo,\n \t\t\t     struct archiver_args *args)\n {\n \tint err = 0;\n \n \twrite_global_extended_header(args);\n-\terr = write_archive_entries(args, write_tar_entry);\n+\terr = write_archive_entries(repo, args, write_tar_entry);\n \tif (!err)\n \t\twrite_trailer();\n \treturn err;\n@@ -462,6 +465,7 @@ static void tgz_write_block(const void *data)\n static const char internal_gzip_command[] = \"git archive gzip\";\n \n static int write_tar_filter_archive(const struct archiver *ar,\n+\t\t\t\t    struct repository *repo,\n \t\t\t\t    struct archiver_args *args)\n {\n #if ZLIB_VERNUM >= 0x1221\n@@ -484,7 +488,7 @@ static int write_tar_filter_archive(const struct archiver *ar,\n \t\tgzstream.next_out = outbuf;\n \t\tgzstream.avail_out = sizeof(outbuf);\n \n-\t\tr = write_tar_archive(ar, args);\n+\t\tr = write_tar_archive(ar, repo, args);\n \n \t\ttgz_deflate(Z_FINISH);\n \t\tgit_deflate_end(&gzstream);\n@@ -506,7 +510,7 @@ static int write_tar_filter_archive(const struct archiver *ar,\n \t\tdie_errno(_(\"unable to redirect descriptor\"));\n \tclose(filter.in);\n \n-\tr = write_tar_archive(ar, args);\n+\tr = write_tar_archive(ar, repo, args);\n \n \tclose(1);\n \tif (finish_command(&filter) != 0)\ndiff --git a/archive-zip.c b/archive-zip.c\nindex 0456f1ebf15..6b27d004b5d 100644\n--- a/archive-zip.c\n+++ b/archive-zip.c\n@@ -283,7 +283,8 @@ static int entry_is_binary(struct index_state *istate, const char *path,\n \n #define STREAM_BUFFER_SIZE (1024 * 16)\n \n-static int write_zip_entry(struct archiver_args *args,\n+static int write_zip_entry(struct repository *repo,\n+\t\t\t   struct archiver_args *args,\n \t\t\t   const struct object_id *oid,\n \t\t\t   const char *path, size_t pathlen,\n \t\t\t   unsigned int mode,\n@@ -340,7 +341,7 @@ static int write_zip_entry(struct archiver_args *args,\n \n \t\tif (!buffer) {\n \t\t\tenum object_type type;\n-\t\t\tstream = open_istream(args->repo, oid, &type, &size,\n+\t\t\tstream = open_istream(repo, oid, &type, &size,\n \t\t\t\t\t      NULL);\n \t\t\tif (!stream)\n \t\t\t\treturn error(_(\"cannot stream blob %s\"),\n@@ -349,7 +350,7 @@ static int write_zip_entry(struct archiver_args *args,\n \t\t\tout = NULL;\n \t\t} else {\n \t\t\tcrc = crc32(crc, buffer, size);\n-\t\t\tis_binary = entry_is_binary(args->repo->index,\n+\t\t\tis_binary = entry_is_binary(repo->index,\n \t\t\t\t\t\t    path_without_prefix,\n \t\t\t\t\t\t    buffer, size);\n \t\t\tout = buffer;\n@@ -426,7 +427,7 @@ static int write_zip_entry(struct archiver_args *args,\n \t\t\t\tbreak;\n \t\t\tcrc = crc32(crc, buf, readlen);\n \t\t\tif (is_binary == -1)\n-\t\t\t\tis_binary = entry_is_binary(args->repo->index,\n+\t\t\t\tis_binary = entry_is_binary(repo->index,\n \t\t\t\t\t\t\t    path_without_prefix,\n \t\t\t\t\t\t\t    buf, readlen);\n \t\t\twrite_or_die(1, buf, readlen);\n@@ -459,7 +460,7 @@ static int write_zip_entry(struct archiver_args *args,\n \t\t\t\tbreak;\n \t\t\tcrc = crc32(crc, buf, readlen);\n \t\t\tif (is_binary == -1)\n-\t\t\t\tis_binary = entry_is_binary(args->repo->index,\n+\t\t\t\tis_binary = entry_is_binary(repo->index,\n \t\t\t\t\t\t\t    path_without_prefix,\n \t\t\t\t\t\t\t    buf, readlen);\n \n@@ -619,6 +620,7 @@ static int archive_zip_config(const char *var, const char *value,\n }\n \n static int write_zip_archive(const struct archiver *ar UNUSED,\n+\t\t\t     struct repository *repo,\n \t\t\t     struct archiver_args *args)\n {\n \tint err;\n@@ -629,7 +631,7 @@ static int write_zip_archive(const struct archiver *ar UNUSED,\n \n \tstrbuf_init(&zip_dir, 0);\n \n-\terr = write_archive_entries(args, write_zip_entry);\n+\terr = write_archive_entries(repo, args, write_zip_entry);\n \tif (!err)\n \t\twrite_zip_trailer(args->commit_oid);\n \ndiff --git a/archive.c b/archive.c\nindex 61a79e4a227..2ee7205679d 100644\n--- a/archive.c\n+++ b/archive.c\n@@ -10,6 +10,7 @@\n #include \"unpack-trees.h\"\n #include \"dir.h\"\n #include \"quote.h\"\n+#include \"submodule.h\"\n \n static char const * const archive_usage[] = {\n \tN_(\"git archive [<options>] <tree-ish> [<path>...]\"),\n@@ -69,6 +70,7 @@ static void format_subst(const struct commit *commit,\n }\n \n static void *object_file_to_archive(const struct archiver_args *args,\n+\t\t\t\t    struct repository *repo,\n \t\t\t\t    const char *path,\n \t\t\t\t    const struct object_id *oid,\n \t\t\t\t    unsigned int mode,\n@@ -84,13 +86,13 @@ static void *object_file_to_archive(const struct archiver_args *args,\n \t\t\t       (args->tree ? &args->tree->object.oid : NULL), oid);\n \n \tpath += args->baselen;\n-\tbuffer = read_object_file(oid, type, sizep);\n+\tbuffer = repo_read_object_file(repo, oid, type, sizep);\n \tif (buffer && S_ISREG(mode)) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tsize_t size = 0;\n \n \t\tstrbuf_attach(&buf, buffer, *sizep, *sizep + 1);\n-\t\tconvert_to_working_tree(args->repo->index, path, buf.buf, buf.len, &buf, &meta);\n+\t\tconvert_to_working_tree(repo->index, path, buf.buf, buf.len, &buf, &meta);\n \t\tif (commit)\n \t\t\tformat_subst(commit, buf.buf, buf.len, &buf, args->pretty_ctx);\n \t\tbuffer = strbuf_detach(&buf, &size);\n@@ -134,7 +136,7 @@ static int check_attr_export_subst(const struct attr_check *check)\n \treturn check && ATTR_TRUE(check->items[1].value);\n }\n \n-static int write_archive_entry(const struct object_id *oid, const char *base,\n+static int write_archive_entry(struct repository *repo, const struct object_id *oid, const char *base,\n \t\tint baselen, const char *filename, unsigned mode,\n \t\tvoid *context)\n {\n@@ -160,7 +162,7 @@ static int write_archive_entry(const struct object_id *oid, const char *base,\n \n \tif (!S_ISDIR(mode)) {\n \t\tconst struct attr_check *check;\n-\t\tcheck = get_archive_attrs(args->repo->index, path_without_prefix);\n+\t\tcheck = get_archive_attrs(repo->index, path_without_prefix);\n \t\tif (check_attr_export_ignore(check))\n \t\t\treturn 0;\n \t\targs->convert = check_attr_export_subst(check);\n@@ -169,10 +171,10 @@ static int write_archive_entry(const struct object_id *oid, const char *base,\n \tif (S_ISDIR(mode) || S_ISGITLINK(mode)) {\n \t\tif (args->verbose)\n \t\t\tfprintf(stderr, \"%.*s\\n\", (int)path.len, path.buf);\n-\t\terr = write_entry(args, oid, path.buf, path.len, mode, NULL, 0);\n+\t\terr = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, 0);\n \t\tif (err)\n \t\t\treturn err;\n-\t\treturn (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);\n+\t\treturn READ_TREE_RECURSIVE;\n \t}\n \n \tif (args->verbose)\n@@ -180,14 +182,19 @@ static int write_archive_entry(const struct object_id *oid, const char *base,\n \n \t/* Stream it? */\n \tif (S_ISREG(mode) && !args->convert &&\n-\t    oid_object_info(args->repo, oid, &size) == OBJ_BLOB &&\n-\t    size > big_file_threshold)\n-\t\treturn write_entry(args, oid, path.buf, path.len, mode, NULL, size);\n+\t    oid_object_info(repo, oid, &size) == OBJ_BLOB &&\n+\t    size > big_file_threshold) {\n+\t\t\terr = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, size);\n+\t\t\tif (err) {\n+\t\t\t\tdie(\"Failed to write file %.*s\", (int)path.len, path.buf);\n+\t\t\t}\n+\t\t\treturn err;\n+\t\t}\n \n-\tbuffer = object_file_to_archive(args, path.buf, oid, mode, &type, &size);\n+\tbuffer = object_file_to_archive(args, repo, path.buf, oid, mode, &type, &size);\n \tif (!buffer)\n \t\treturn error(_(\"cannot read '%s'\"), oid_to_hex(oid));\n-\terr = write_entry(args, oid, path.buf, path.len, mode, buffer, size);\n+\terr = write_entry(repo, args, oid, path.buf, path.len, mode, buffer, size);\n \tfree(buffer);\n \treturn err;\n }\n@@ -207,7 +214,25 @@ static void queue_directory(const struct object_id *oid,\n \toidcpy(&d->oid, oid);\n }\n \n-static int write_directory(struct archiver_context *c)\n+static void queue_submodule(struct repository *superproject,\n+\t\tconst struct object_id *oid,\n+\t\tstruct strbuf *base, const char *filename,\n+\t\tunsigned mode, struct archiver_context *c)\n+{\n+\tstruct repository subrepo;\n+\n+\tif (repo_submodule_init(&subrepo, superproject, filename, null_oid()))\n+\t\treturn;\n+\n+\tif (repo_read_index(&subrepo) < 0)\n+\t\tdie(\"index file corrupt\");\n+\n+    queue_directory(oid, base, filename, mode, c);\n+\n+\trepo_clear(&subrepo);\n+}\n+\n+static int write_directory(struct repository *repo, struct archiver_context *c)\n {\n \tstruct directory *d = c->bottom;\n \tint ret;\n@@ -217,15 +242,18 @@ static int write_directory(struct archiver_context *c)\n \tc->bottom = d->up;\n \td->path[d->len - 1] = '\\0'; /* no trailing slash */\n \tret =\n-\t\twrite_directory(c) ||\n-\t\twrite_archive_entry(&d->oid, d->path, d->baselen,\n+\t\twrite_directory(repo, c) ||\n+\t\twrite_archive_entry(repo, &d->oid, d->path, d->baselen,\n \t\t\t\t    d->path + d->baselen, d->mode,\n-\t\t\t\t    c) != READ_TREE_RECURSIVE;\n+\t\t\t\t    c);\n \tfree(d);\n-\treturn ret ? -1 : 0;\n+\tif (ret == READ_TREE_RECURSIVE)\n+\t\treturn 0;\n+\treturn ret;\n }\n \n-static int queue_or_write_archive_entry(const struct object_id *oid,\n+static int queue_or_write_archive_entry(\n+\t\tstruct repository *repo, const struct object_id *oid,\n \t\tstruct strbuf *base, const char *filename,\n \t\tunsigned mode, void *context)\n {\n@@ -246,18 +274,23 @@ static int queue_or_write_archive_entry(const struct object_id *oid,\n \t\t/* Borrow base, but restore its original value when done. */\n \t\tstrbuf_addstr(base, filename);\n \t\tstrbuf_addch(base, '/');\n-\t\tcheck = get_archive_attrs(c->args->repo->index, base->buf);\n+\t\tcheck = get_archive_attrs(repo->index, base->buf);\n \t\tstrbuf_setlen(base, baselen);\n \n \t\tif (check_attr_export_ignore(check))\n \t\t\treturn 0;\n \t\tqueue_directory(oid, base, filename, mode, c);\n \t\treturn READ_TREE_RECURSIVE;\n+\t} else if (c->args->recurse_submodules && S_ISGITLINK(mode)) {\n+\t\tif (is_submodule_active(repo, filename)) {\n+\t\t\tqueue_submodule(repo, oid, base, filename, mode, c);\n+\t\t\treturn READ_TREE_RECURSIVE;\n+\t\t}\n \t}\n \n-\tif (write_directory(c))\n+\tif (write_directory(repo, c))\n \t\treturn -1;\n-\treturn write_archive_entry(oid, base->buf, base->len, filename, mode,\n+\treturn write_archive_entry(repo, oid, base->buf, base->len, filename, mode,\n \t\t\t\t   context);\n }\n \n@@ -267,7 +300,8 @@ struct extra_file_info {\n \tvoid *content;\n };\n \n-int write_archive_entries(struct archiver_args *args,\n+int write_archive_entries(struct repository *repo,\n+\t\tstruct archiver_args *args,\n \t\twrite_archive_entry_fn_t write_entry)\n {\n \tstruct archiver_context context;\n@@ -288,7 +322,7 @@ int write_archive_entries(struct archiver_args *args,\n \t\t\tlen--;\n \t\tif (args->verbose)\n \t\t\tfprintf(stderr, \"%.*s\\n\", (int)len, args->base);\n-\t\terr = write_entry(args, &args->tree->object.oid, args->base,\n+\t\terr = write_entry(repo, args, &args->tree->object.oid, args->base,\n \t\t\t\t  len, 040777, NULL, 0);\n \t\tif (err)\n \t\t\treturn err;\n@@ -305,8 +339,8 @@ int write_archive_entries(struct archiver_args *args,\n \t\tmemset(&opts, 0, sizeof(opts));\n \t\topts.index_only = 1;\n \t\topts.head_idx = -1;\n-\t\topts.src_index = args->repo->index;\n-\t\topts.dst_index = args->repo->index;\n+\t\topts.src_index = repo->index;\n+\t\topts.dst_index = repo->index;\n \t\topts.fn = oneway_merge;\n \t\tinit_tree_desc(&t, args->tree->buffer, args->tree->size);\n \t\tif (unpack_trees(1, &t, &opts))\n@@ -314,7 +348,7 @@ int write_archive_entries(struct archiver_args *args,\n \t\tgit_attr_set_direction(GIT_ATTR_INDEX);\n \t}\n \n-\terr = read_tree(args->repo, args->tree,\n+\terr = read_tree(repo, args->tree,\n \t\t\t&args->pathspec,\n \t\t\tqueue_or_write_archive_entry,\n \t\t\t&context);\n@@ -343,12 +377,12 @@ int write_archive_entries(struct archiver_args *args,\n \t\t\tif (strbuf_read_file(&content, path, info->stat.st_size) < 0)\n \t\t\t\terr = error_errno(_(\"cannot read '%s'\"), path);\n \t\t\telse\n-\t\t\t\terr = write_entry(args, &fake_oid, path_in_archive.buf,\n+\t\t\t\terr = write_entry(repo, args, &fake_oid, path_in_archive.buf,\n \t\t\t\t\t\t  path_in_archive.len,\n \t\t\t\t\t\t  canon_mode(info->stat.st_mode),\n \t\t\t\t\t\t  content.buf, content.len);\n \t\t} else {\n-\t\t\terr = write_entry(args, &fake_oid,\n+\t\t\terr = write_entry(repo, args, &fake_oid,\n \t\t\t\t\t  path, strlen(path),\n \t\t\t\t\t  canon_mode(info->stat.st_mode),\n \t\t\t\t\t  info->content, info->stat.st_size);\n@@ -382,7 +416,7 @@ struct path_exists_context {\n \tstruct archiver_args *args;\n };\n \n-static int reject_entry(const struct object_id *oid UNUSED,\n+static int reject_entry(struct repository *repo, const struct object_id *oid UNUSED,\n \t\t\tstruct strbuf *base,\n \t\t\tconst char *filename, unsigned mode,\n \t\t\tvoid *context)\n@@ -394,7 +428,7 @@ static int reject_entry(const struct object_id *oid UNUSED,\n \t\tstruct strbuf sb = STRBUF_INIT;\n \t\tstrbuf_addbuf(&sb, base);\n \t\tstrbuf_addstr(&sb, filename);\n-\t\tif (!match_pathspec(ctx->args->repo->index,\n+\t\tif (!match_pathspec(repo->index,\n \t\t\t\t    &ctx->pathspec,\n \t\t\t\t    sb.buf, sb.len, 0, NULL, 1))\n \t\t\tret = READ_TREE_RECURSIVE;\n@@ -431,6 +465,7 @@ static void parse_pathspec_arg(const char **pathspec,\n \t\t       PATHSPEC_PREFER_FULL,\n \t\t       \"\", pathspec);\n \tar_args->pathspec.recursive = 1;\n+\tar_args->pathspec.recurse_submodules = ar_args->recurse_submodules;\n \tif (pathspec) {\n \t\twhile (*pathspec) {\n \t\t\tif (**pathspec && !path_exists(ar_args, *pathspec))\n@@ -592,6 +627,7 @@ static int parse_archive_args(int argc, const char **argv,\n \tint verbose = 0;\n \tint i;\n \tint list = 0;\n+\tint recurse_submodules = 0;\n \tint worktree_attributes = 0;\n \tstruct option opts[] = {\n \t\tOPT_GROUP(\"\"),\n@@ -606,6 +642,8 @@ static int parse_archive_args(int argc, const char **argv,\n \t\t  add_file_cb, (intptr_t)&base },\n \t\tOPT_STRING('o', \"output\", &output, N_(\"file\"),\n \t\t\tN_(\"write the archive to this file\")),\n+\t\tOPT_BOOL(0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\tN_(\"include submodules in archive\")),\n \t\tOPT_BOOL(0, \"worktree-attributes\", &worktree_attributes,\n \t\t\tN_(\"read .gitattributes in working directory\")),\n \t\tOPT__VERBOSE(&verbose, N_(\"report archived files on stderr\")),\n@@ -670,6 +708,7 @@ static int parse_archive_args(int argc, const char **argv,\n \targs->base = base;\n \targs->baselen = strlen(base);\n \targs->worktree_attributes = worktree_attributes;\n+\targs->recurse_submodules = recurse_submodules;\n \n \treturn argc;\n }\n@@ -708,7 +747,7 @@ int write_archive(int argc, const char **argv, const char *prefix,\n \tparse_treeish_arg(argv, &args, prefix, remote);\n \tparse_pathspec_arg(argv + 1, &args);\n \n-\trc = ar->write_archive(ar, &args);\n+\trc = ar->write_archive(ar, repo, &args);\n \n \tstring_list_clear_func(&args.extra_files, extra_file_info_clear);\n \tfree(args.refname);\ndiff --git a/archive.h b/archive.h\nindex 08bed3ed3af..91b318f9ad8 100644\n--- a/archive.h\n+++ b/archive.h\n@@ -19,6 +19,7 @@ struct archiver_args {\n \ttimestamp_t time;\n \tstruct pathspec pathspec;\n \tunsigned int verbose : 1;\n+\tunsigned int recurse_submodules : 1;\n \tunsigned int worktree_attributes : 1;\n \tunsigned int convert : 1;\n \tint compression_level;\n@@ -41,7 +42,7 @@ const char *archive_format_from_filename(const char *filename);\n #define ARCHIVER_HIGH_COMPRESSION_LEVELS 4\n struct archiver {\n \tconst char *name;\n-\tint (*write_archive)(const struct archiver *, struct archiver_args *);\n+\tint (*write_archive)(const struct archiver *, struct repository *repo, struct archiver_args *);\n \tunsigned flags;\n \tchar *filter_command;\n };\n@@ -51,12 +52,13 @@ void init_tar_archiver(void);\n void init_zip_archiver(void);\n void init_archivers(void);\n \n-typedef int (*write_archive_entry_fn_t)(struct archiver_args *args,\n+typedef int (*write_archive_entry_fn_t)(struct repository *repo,\n+\t\t\t\t\tstruct archiver_args *args,\n \t\t\t\t\tconst struct object_id *oid,\n \t\t\t\t\tconst char *path, size_t pathlen,\n \t\t\t\t\tunsigned int mode,\n \t\t\t\t\tvoid *buffer, unsigned long size);\n \n-int write_archive_entries(struct archiver_args *args, write_archive_entry_fn_t write_entry);\n+int write_archive_entries(struct repository *repo, struct archiver_args *args, write_archive_entry_fn_t write_entry);\n \n #endif\t/* ARCHIVE_H */\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex 2a132392fbe..1238774b245 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -124,7 +124,7 @@ static int post_checkout_hook(struct commit *old_commit, struct commit *new_comm\n \n }\n \n-static int update_some(const struct object_id *oid, struct strbuf *base,\n+static int update_some(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n \t\t       const char *pathname, unsigned mode, void *context UNUSED)\n {\n \tint len;\ndiff --git a/builtin/log.c b/builtin/log.c\nindex ee19dc5d450..da73bbef836 100644\n--- a/builtin/log.c\n+++ b/builtin/log.c\n@@ -698,7 +698,7 @@ static int show_tag_object(const struct object_id *oid, struct rev_info *rev)\n \treturn 0;\n }\n \n-static int show_tree_object(const struct object_id *oid UNUSED,\n+static int show_tree_object(struct repository *repo UNUSED, const struct object_id *oid UNUSED,\n \t\t\t    struct strbuf *base UNUSED,\n \t\t\t    const char *pathname, unsigned mode,\n \t\t\t    void *context)\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex 4cf8a236483..a08768dbd2a 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -509,7 +509,7 @@ static int get_common_prefix_len(const char *common_prefix)\n \treturn common_prefix_len;\n }\n \n-static int read_one_entry_opt(struct index_state *istate,\n+static int read_one_entry_opt(struct repository *repo UNUSED, struct index_state *istate,\n \t\t\t      const struct object_id *oid,\n \t\t\t      struct strbuf *base,\n \t\t\t      const char *pathname,\n@@ -533,12 +533,12 @@ static int read_one_entry_opt(struct index_state *istate,\n \treturn add_index_entry(istate, ce, opt);\n }\n \n-static int read_one_entry(const struct object_id *oid, struct strbuf *base,\n+static int read_one_entry(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n \t\t\t  const char *pathname, unsigned mode,\n \t\t\t  void *context)\n {\n \tstruct index_state *istate = context;\n-\treturn read_one_entry_opt(istate, oid, base, pathname,\n+\treturn read_one_entry_opt(repo, istate, oid, base, pathname,\n \t\t\t\t  mode,\n \t\t\t\t  ADD_CACHE_OK_TO_ADD|ADD_CACHE_SKIP_DFCHECK);\n }\n@@ -547,12 +547,12 @@ static int read_one_entry(const struct object_id *oid, struct strbuf *base,\n  * This is used when the caller knows there is no existing entries at\n  * the stage that will conflict with the entry being added.\n  */\n-static int read_one_entry_quick(const struct object_id *oid, struct strbuf *base,\n+static int read_one_entry_quick(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n \t\t\t\tconst char *pathname, unsigned mode,\n \t\t\t\tvoid *context)\n {\n \tstruct index_state *istate = context;\n-\treturn read_one_entry_opt(istate, oid, base, pathname,\n+\treturn read_one_entry_opt(repo, istate, oid, base, pathname,\n \t\t\t\t  mode, ADD_CACHE_JUST_APPEND);\n }\n \ndiff --git a/builtin/ls-tree.c b/builtin/ls-tree.c\nindex c3ea09281af..c8d6ff95ac0 100644\n--- a/builtin/ls-tree.c\n+++ b/builtin/ls-tree.c\n@@ -141,7 +141,7 @@ static int show_recursive(const char *base, size_t baselen, const char *pathname\n \treturn 0;\n }\n \n-static int show_tree_fmt(const struct object_id *oid, struct strbuf *base,\n+static int show_tree_fmt(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n \t\t\t const char *pathname, unsigned mode, void *context UNUSED)\n {\n \tsize_t baselen;\n@@ -211,7 +211,7 @@ static void show_tree_common_default_long(struct strbuf *base,\n \tstrbuf_setlen(base, baselen);\n }\n \n-static int show_tree_default(const struct object_id *oid, struct strbuf *base,\n+static int show_tree_default(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n \t\t\t     const char *pathname, unsigned mode,\n \t\t\t     void *context UNUSED)\n {\n@@ -229,7 +229,7 @@ static int show_tree_default(const struct object_id *oid, struct strbuf *base,\n \treturn recurse;\n }\n \n-static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n+static int show_tree_long(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n \t\t\t  const char *pathname, unsigned mode,\n \t\t\t  void *context UNUSED)\n {\n@@ -244,7 +244,7 @@ static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n \n \tif (data.type == OBJ_BLOB) {\n \t\tunsigned long size;\n-\t\tif (oid_object_info(the_repository, data.oid, &size) == OBJ_BAD)\n+\t\tif (oid_object_info(repo, data.oid, &size) == OBJ_BAD)\n \t\t\txsnprintf(size_text, sizeof(size_text), \"BAD\");\n \t\telse\n \t\t\txsnprintf(size_text, sizeof(size_text),\n@@ -254,12 +254,12 @@ static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n \t}\n \n \tprintf(\"%06o %s %s %7s\\t\", data.mode, type_name(data.type),\n-\t       find_unique_abbrev(data.oid, abbrev), size_text);\n+\t       repo_find_unique_abbrev(repo, data.oid, abbrev), size_text);\n \tshow_tree_common_default_long(base, pathname, data.base->len);\n \treturn recurse;\n }\n \n-static int show_tree_name_only(const struct object_id *oid, struct strbuf *base,\n+static int show_tree_name_only(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n \t\t\t       const char *pathname, unsigned mode,\n \t\t\t       void *context UNUSED)\n {\n@@ -280,7 +280,7 @@ static int show_tree_name_only(const struct object_id *oid, struct strbuf *base,\n \treturn recurse;\n }\n \n-static int show_tree_object(const struct object_id *oid, struct strbuf *base,\n+static int show_tree_object(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n \t\t\t    const char *pathname, unsigned mode,\n \t\t\t    void *context UNUSED)\n {\n@@ -292,7 +292,7 @@ static int show_tree_object(const struct object_id *oid, struct strbuf *base,\n \tif (early >= 0)\n \t\treturn early;\n \n-\tprintf(\"%s%c\", find_unique_abbrev(oid, abbrev), line_termination);\n+\tprintf(\"%s%c\", repo_find_unique_abbrev(repo, oid, abbrev), line_termination);\n \treturn recurse;\n }\n \ndiff --git a/list-objects.c b/list-objects.c\nindex 250d9de41cb..4f463ae32ad 100644\n--- a/list-objects.c\n+++ b/list-objects.c\n@@ -185,7 +185,7 @@ static void process_tree(struct traversal_context *ctx,\n \t    !revs->include_check_obj(&tree->object, revs->include_check_data))\n \t\treturn;\n \n-\tfailed_parse = parse_tree_gently(tree, 1);\n+\tfailed_parse = parse_tree_gently(revs->repo, tree, 1);\n \tif (failed_parse) {\n \t\tif (revs->ignore_missing_links)\n \t\t\treturn;\ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex 4ddd3adea00..119e86090f9 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -456,7 +456,7 @@ static void unpack_trees_finish(struct merge_options *opt)\n \tclear_unpack_trees_porcelain(&opt->priv->unpack_opts);\n }\n \n-static int save_files_dirs(const struct object_id *oid UNUSED,\n+static int save_files_dirs(struct repository *repo UNUSED, const struct object_id *oid UNUSED,\n \t\t\t   struct strbuf *base, const char *path,\n \t\t\t   unsigned int mode, void *context)\n {\ndiff --git a/revision.c b/revision.c\nindex 36e31942cee..39e023a19ea 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -74,7 +74,7 @@ static void mark_tree_contents_uninteresting(struct repository *r,\n \tstruct tree_desc desc;\n \tstruct name_entry entry;\n \n-\tif (parse_tree_gently(tree, 1) < 0)\n+\tif (parse_tree_gently(r, tree, 1) < 0)\n \t\treturn;\n \n \tinit_tree_desc(&desc, tree->buffer, tree->size);\n@@ -181,7 +181,7 @@ static void add_children_by_path(struct repository *r,\n \tif (!tree)\n \t\treturn;\n \n-\tif (parse_tree_gently(tree, 1) < 0)\n+\tif (parse_tree_gently(r, tree, 1) < 0)\n \t\treturn;\n \n \tinit_tree_desc(&desc, tree->buffer, tree->size);\ndiff --git a/sparse-index.c b/sparse-index.c\nindex e4a54ce1943..05ca8aec1fb 100644\n--- a/sparse-index.c\n+++ b/sparse-index.c\n@@ -232,7 +232,7 @@ static void set_index_entry(struct index_state *istate, int nr, struct cache_ent\n \tadd_name_hash(istate, ce);\n }\n \n-static int add_path_to_index(const struct object_id *oid,\n+static int add_path_to_index(struct repository *repo UNUSED, const struct object_id *oid,\n \t\t\t     struct strbuf *base, const char *path,\n \t\t\t     unsigned int mode, void *context)\n {\ndiff --git a/t/t5005-archive-submodules.sh b/t/t5005-archive-submodules.sh\nnew file mode 100755\nindex 00000000000..49d5ff74ad5\n--- /dev/null\n+++ b/t/t5005-archive-submodules.sh\n@@ -0,0 +1,84 @@\n+#!/bin/sh\n+\n+test_description='git archive --recurse-submodules test'\n+\n+. ./test-lib.sh\n+. \"$TEST_DIRECTORY\"/lib-submodule-update.sh\n+\n+test_expect_success 'setup' '\n+\tcreate_lib_submodule_repo &&\n+\tgit -C submodule_update_repo checkout valid_sub1 &&\n+\tgit -C submodule_update_repo submodule update\n+'\n+\n+check_tar() {\n+\ttarfile=$1.tar\n+\tlistfile=$1.lst\n+\tdir=$1\n+\tdir_with_prefix=$dir/$2\n+\n+\ttest_expect_success ' extract tar archive' '\n+\t\t(mkdir $dir && cd $dir && \"$TAR\" xf -) <$tarfile\n+\t'\n+}\n+\n+check_added() {\n+\tdir=$1\n+\tpath_in_fs=$2\n+\tpath_in_archive=$3\n+\n+\ttest_expect_success \" validate extra file $path_in_archive\" '\n+\t\ttest -f $dir/$path_in_archive &&\n+\t\tdiff -r $path_in_fs $dir/$path_in_archive\n+\t'\n+}\n+\n+check_not_added() {\n+\tdir=$1\n+\tpath_in_archive=$2\n+\n+\ttest_expect_success \" validate unpresent file $path_in_archive\" '\n+\t\t! test -f $dir/$path_in_archive &&\n+\t\t! test -d $dir/$path_in_archive\n+\t'\n+}\n+\n+test_expect_success 'archive without recurse, non-init' '\n+\treset_work_tree_to valid_sub1 &&\n+\tgit -C submodule_update archive HEAD >b.tar\n+'\n+\n+check_tar b\n+check_added b submodule_update/file1 file1\n+check_not_added b sub1/file1\n+\n+test_expect_success 'archive with recurse, non-init' '\n+\treset_work_tree_to valid_sub1 &&\n+\t! git -C submodule_update archive --recurse-submodules HEAD >b2-err.tar\n+'\n+\n+test_expect_success 'archive with recurse, init' '\n+\treset_work_tree_to valid_sub1 &&\n+\tgit -C submodule_update submodule update --init &&\n+\tgit -C submodule_update ls-files --recurse-submodules &&\n+\tgit -C submodule_update ls-tree HEAD &&\n+\tgit -C submodule_update archive --recurse-submodules HEAD >b2.tar\n+'\n+\n+check_tar b2\n+check_added b2 submodule_update/sub1/file1 sub1/file1\n+\n+test_expect_success 'archive with recurse with big files' '\n+\treset_work_tree_to valid_sub1 &&\n+\ttest_config core.bigfilethreshold 1 &&\n+\tgit -C submodule_update submodule update --init &&\n+\tgit -C submodule_update ls-files --recurse-submodules &&\n+\tgit -C submodule_update ls-tree HEAD &&\n+\tgit -C submodule_update archive --recurse-submodules HEAD >b3.tar\n+'\n+\n+check_tar b3\n+check_added b3 submodule_update/sub1/file1 sub1/file1\n+\n+\n+test_done\ndiff --git a/tree.c b/tree.c\nindex 410e3b477e5..c5b5a0ac08f 100644\n--- a/tree.c\n+++ b/tree.c\n@@ -8,6 +8,7 @@\n #include \"alloc.h\"\n #include \"tree-walk.h\"\n #include \"repository.h\"\n+#include \"pathspec.h\"\n \n const char *tree_type = \"tree\";\n \n@@ -22,8 +23,8 @@ int read_tree_at(struct repository *r,\n \tint len, oldlen = base->len;\n \tenum interesting retval = entry_not_interesting;\n \n-\tif (parse_tree(tree))\n-\t\treturn -1;\n+\tif (repo_parse_tree(r, tree))\n+\t\tdie(\"Failed to parse tree\");\n \n \tinit_tree_desc(&desc, tree->buffer, tree->size);\n \n@@ -37,7 +38,7 @@ int read_tree_at(struct repository *r,\n \t\t\t\tcontinue;\n \t\t}\n \n-\t\tswitch (fn(&entry.oid, base,\n+\t\tswitch (fn(r, &entry.oid, base,\n \t\t\t   entry.path, entry.mode, context)) {\n \t\tcase 0:\n \t\t\tcontinue;\n@@ -47,36 +48,57 @@ int read_tree_at(struct repository *r,\n \t\t\treturn -1;\n \t\t}\n \n-\t\tif (S_ISDIR(entry.mode))\n+\t\tif (S_ISDIR(entry.mode)) {\n \t\t\toidcpy(&oid, &entry.oid);\n-\t\telse if (S_ISGITLINK(entry.mode)) {\n+\t\t\tlen = tree_entry_len(&entry);\n+\t\t\tstrbuf_add(base, entry.path, len);\n+\t\t\tstrbuf_addch(base, '/');\n+\t\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n+\t\t\t\t\t\tbase, pathspec,\n+\t\t\t\t\t\tfn, context);\n+\t\t\tstrbuf_setlen(base, oldlen);\n+\t\t\tif (retval)\n+\t\t\t\treturn -1;\n+\t\t} else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) {\n \t\t\tstruct commit *commit;\n+\t\t\tstruct repository subrepo;\n+\t\t\tstruct repository* subrepo_p = &subrepo;\n+\t\t\tstruct tree* submodule_tree;\n \n-\t\t\tcommit = lookup_commit(r, &entry.oid);\n+\t\t\tif (repo_submodule_init(subrepo_p, r, entry.path, null_oid()))\n+\t\t\t\tdie(\"couldn't init submodule %s%s\", base->buf, entry.path);\n+\n+\t\t\tif (repo_read_index(subrepo_p) < 0)\n+\t\t\t\tdie(\"index file corrupt\");\n+\n+\t\t\tcommit = lookup_commit(subrepo_p, &entry.oid);\n \t\t\tif (!commit)\n \t\t\t\tdie(\"Commit %s in submodule path %s%s not found\",\n \t\t\t\t    oid_to_hex(&entry.oid),\n \t\t\t\t    base->buf, entry.path);\n \n-\t\t\tif (parse_commit(commit))\n+\t\t\tif (repo_parse_commit(subrepo_p, commit))\n \t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n \t\t\t\t    oid_to_hex(&entry.oid),\n \t\t\t\t    base->buf, entry.path);\n \n-\t\t\toidcpy(&oid, get_commit_tree_oid(commit));\n+\t\t\tsubmodule_tree = repo_get_commit_tree(subrepo_p, commit);\n+\t\t\toidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);\n+\n+\t\t\tlen = tree_entry_len(&entry);\n+\t\t\tstrbuf_add(base, entry.path, len);\n+\t\t\tstrbuf_addch(base, '/');\n+\t\t\tretval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),\n+\t\t\t\t\t\tbase, pathspec,\n+\t\t\t\t\t\tfn, context);\n+\t\t\tif (retval) {\n+\t\t\t    die(\"failed to read tree for %s%s\", base->buf, entry.path);\n+\t\t\t    return -1;\n+\t\t\t}\n+\t\t\tstrbuf_setlen(base, oldlen);\n+\t\t\trepo_clear(subrepo_p);\n \t\t}\n-\t\telse\n-\t\t\tcontinue;\n \n-\t\tlen = tree_entry_len(&entry);\n-\t\tstrbuf_add(base, entry.path, len);\n-\t\tstrbuf_addch(base, '/');\n-\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n-\t\t\t\t      base, pathspec,\n-\t\t\t\t      fn, context);\n-\t\tstrbuf_setlen(base, oldlen);\n-\t\tif (retval)\n-\t\t\treturn -1;\n \t}\n \treturn 0;\n }\n@@ -121,7 +143,7 @@ int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size)\n \treturn 0;\n }\n \n-int parse_tree_gently(struct tree *item, int quiet_on_missing)\n+int parse_tree_gently(struct repository *r, struct tree *item, int quiet_on_missing)\n {\n \t enum object_type type;\n \t void *buffer;\n@@ -129,7 +151,7 @@ int parse_tree_gently(struct tree *item, int quiet_on_missing)\n \n \tif (item->object.parsed)\n \t\treturn 0;\n-\tbuffer = read_object_file(&item->object.oid, &type, &size);\n+\tbuffer = repo_read_object_file(r, &item->object.oid, &type, &size);\n \tif (!buffer)\n \t\treturn quiet_on_missing ? -1 :\n \t\t\terror(\"Could not read %s\",\ndiff --git a/tree.h b/tree.h\nindex 6efff003e21..3dc8f151760 100644\n--- a/tree.h\n+++ b/tree.h\n@@ -18,11 +18,14 @@ struct tree *lookup_tree(struct repository *r, const struct object_id *oid);\n \n int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size);\n \n-int parse_tree_gently(struct tree *tree, int quiet_on_missing);\n-static inline int parse_tree(struct tree *tree)\n+int parse_tree_gently(struct repository *r, struct tree *tree, int quiet_on_missing);\n+static inline int repo_parse_tree(struct repository *r, struct tree *tree)\n {\n-\treturn parse_tree_gently(tree, 0);\n+\treturn parse_tree_gently(r, tree, 0);\n }\n+#ifndef NO_THE_REPOSITORY_COMPATIBILITY_MACROS\n+#define parse_tree(tree) repo_parse_tree(the_repository, tree)\n+#endif\n void free_tree_buffer(struct tree *tree);\n \n /* Parses and returns the tree in the given ent, chasing tags and commits. */\n@@ -31,7 +34,7 @@ struct tree *parse_tree_indirect(const struct object_id *oid);\n int cmp_cache_name_compare(const void *a_, const void *b_);\n \n #define READ_TREE_RECURSIVE 1\n-typedef int (*read_tree_fn_t)(const struct object_id *, struct strbuf *, const char *, unsigned int, void *);\n+typedef int (*read_tree_fn_t)(struct repository *, const struct object_id *, struct strbuf *, const char *, unsigned int, void *);\n \n int read_tree_at(struct repository *r,\n \t\t struct tree *tree, struct strbuf *base,\ndiff --git a/wt-status.c b/wt-status.c\nindex 5813174896c..fefe4c55e1b 100644\n--- a/wt-status.c\n+++ b/wt-status.c\n@@ -665,7 +665,7 @@ static void wt_status_collect_changes_index(struct wt_status *s)\n \trelease_revisions(&rev);\n }\n \n-static int add_file_to_list(const struct object_id *oid,\n+static int add_file_to_list(struct repository *repo UNUSED, const struct object_id *oid,\n \t\t\t    struct strbuf *base, const char *path,\n \t\t\t    unsigned int mode, void *context)\n {\n\nbase-commit: e85701b4af5b7c2a9f3a1b07858703318dce365d\n-- \ngitgitgadget\n"},{"id":"464822","messageId":"68f7830c6d99187aa599dd608fa6c6a916bbd017.1665660960.git.gitgitgadget@gmail.com","threadId":"58613","inReplyTo":"pull.1359.v2.git.git.1665660960.gitgitgadget@gmail.com","subject":"[PATCH v2 2/2] archive: fix a case of submodule in submodule traversal","fromName":"Alphadelta14 via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-13T11:36:00Z","receivedAt":"2022-10-13T11:36:13Z","isPatch":true,"sender":{"key":"name:Alphadelta14","avatar":null},"body":"From: Alphadelta14 <alpha@alphaservcomputing.solutions>\n\nrepo_submodule_init actually expects the path relative to submodule_prefix.\nWe preform a simple strip to the correct path.\n\nSigned-off-by: Alphadelta14 <alpha@alphaservcomputing.solutions>\n---\n tree.c | 35 ++++++++++++++++++++++++++---------\n 1 file changed, 26 insertions(+), 9 deletions(-)\n\ndiff --git a/tree.c b/tree.c\nindex c5b5a0ac08f..c327d210f47 100644\n--- a/tree.c\n+++ b/tree.c\n@@ -64,35 +64,52 @@ int read_tree_at(struct repository *r,\n \t\t\tstruct repository subrepo;\n \t\t\tstruct repository* subrepo_p = &subrepo;\n \t\t\tstruct tree* submodule_tree;\n+\t\t\tchar *submodule_rel_path;\n+\t\t\tint name_base_len = 0;\n \n-\t\t\tif (repo_submodule_init(subrepo_p, r, entry.path, null_oid()))\n-\t\t\t\tdie(\"couldn't init submodule %s%s\", base->buf, entry.path);\n+\t\t\tlen = tree_entry_len(&entry);\n+\t\t\tstrbuf_add(base, entry.path, len);\n+\t\t\tsubmodule_rel_path = base->buf;\n+\t\t\t// repo_submodule_init expects a path relative to submodule_prefix\n+\t\t\tif (r->submodule_prefix) {\n+\t\t\t\tname_base_len = strlen(r->submodule_prefix);\n+\t\t\t\t// we should always expect to start with submodule_prefix\n+\t\t\t\tassert(!strncmp(submodule_rel_path, r->submodule_prefix, name_base_len));\n+\t\t\t\t// strip the prefix\n+\t\t\t\tsubmodule_rel_path += name_base_len;\n+\t\t\t\t// if submodule_prefix doesn't end with a /, we want to get rid of that too\n+\t\t\t\tif (is_dir_sep(submodule_rel_path[0])) {\n+\t\t\t\t\tsubmodule_rel_path++;\n+\t\t\t\t}\n+\t\t\t}\n+\n+\t\t\tif (repo_submodule_init(subrepo_p, r, submodule_rel_path, null_oid()))\n+\t\t\t\tdie(\"couldn't init submodule %s\", base->buf);\n \n \t\t\tif (repo_read_index(subrepo_p) < 0)\n \t\t\t\tdie(\"index file corrupt\");\n \n \t\t\tcommit = lookup_commit(subrepo_p, &entry.oid);\n \t\t\tif (!commit)\n-\t\t\t\tdie(\"Commit %s in submodule path %s%s not found\",\n+\t\t\t\tdie(\"Commit %s in submodule path %s not found\",\n \t\t\t\t    oid_to_hex(&entry.oid),\n-\t\t\t\t    base->buf, entry.path);\n+\t\t\t\t    base->buf);\n \n \t\t\tif (repo_parse_commit(subrepo_p, commit))\n-\t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n+\t\t\t\tdie(\"Invalid commit %s in submodule path %s\",\n \t\t\t\t    oid_to_hex(&entry.oid),\n-\t\t\t\t    base->buf, entry.path);\n+\t\t\t\t    base->buf);\n \n \t\t\tsubmodule_tree = repo_get_commit_tree(subrepo_p, commit);\n \t\t\toidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);\n \n-\t\t\tlen = tree_entry_len(&entry);\n-\t\t\tstrbuf_add(base, entry.path, len);\n \t\t\tstrbuf_addch(base, '/');\n+\n \t\t\tretval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),\n \t\t\t\t\t\tbase, pathspec,\n \t\t\t\t\t\tfn, context);\n \t\t\tif (retval) {\n-\t\t\t    die(\"failed to read tree for %s%s\", base->buf, entry.path);\n+\t\t\t    die(\"failed to read tree for %s\", base->buf);\n \t\t\t    return -1;\n \t\t\t}\n \t\t\tstrbuf_setlen(base, oldlen);\n-- \ngitgitgadget\n"},{"id":"464823","messageId":"pull.1359.v2.git.git.1665660960.gitgitgadget@gmail.com","threadId":"58613","inReplyTo":"pull.1359.git.git.1665597148042.gitgitgadget@gmail.com","subject":"[PATCH v2 0/2] archive: Add --recurse-submodules to git-archive command","fromName":"Heather Lapointe via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-13T11:35:58Z","receivedAt":"2022-10-13T11:36:16Z","isPatch":true,"sender":{"key":"alpha@alphaservcomputing.solutions","avatar":null},"body":"This makes it possible to include submodule contents in an archive command.\n\nThe inspiration for this change comes from this Github thread,\nhttps://github.com/dear-github/dear-github/issues/214, with at least 160\n👍🏻 's at the time of writing. (I stumbled upon it because I wanted it as\nwell).\n\nI figured the underlying implementation wouldn't be too difficult with most\nof the plumbing already in place, so I decided to add the relevant logic to\nthe client git-archive command.\n\nOne of the trickier parts of this implementation involved teaching read_tree\nabout submodules. Some of the troublesome areas were still using the\nthe_repository references to look up commit or tree or oid information. I\nended up deciding that read_tree_fn_t would probably be best off having a\nconcrete repo reference since it allows changing the context to a subrepo\nwhere needed (even though some of the usages did not need it specifically).\n\nI am open to feedback since this is all quite new to me :)\n\nTODO:\n\n * working implementation\n * valgrind\n * add regression tests\n * update documentation with new flag\n * submit to mailing list\n\nAlphadelta14 (2):\n  archive: add --recurse-submodules to git-archive command\n  archive: fix a case of submodule in submodule traversal\n\n archive-tar.c                 | 14 +++--\n archive-zip.c                 | 14 ++---\n archive.c                     | 99 ++++++++++++++++++++++++-----------\n archive.h                     |  8 +--\n builtin/checkout.c            |  2 +-\n builtin/log.c                 |  2 +-\n builtin/ls-files.c            | 10 ++--\n builtin/ls-tree.c             | 16 +++---\n list-objects.c                |  2 +-\n merge-recursive.c             |  2 +-\n revision.c                    |  4 +-\n sparse-index.c                |  2 +-\n t/t5005-archive-submodules.sh | 84 +++++++++++++++++++++++++++++\n tree.c                        | 93 ++++++++++++++++++++++----------\n tree.h                        | 11 ++--\n wt-status.c                   |  2 +-\n 16 files changed, 269 insertions(+), 96 deletions(-)\n create mode 100755 t/t5005-archive-submodules.sh\n\n\nbase-commit: e85701b4af5b7c2a9f3a1b07858703318dce365d\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1359%2FAlphadelta14%2Farchive-recurse-submodules-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1359/Alphadelta14/archive-recurse-submodules-v2\nPull-Request: https://github.com/git/git/pull/1359\n\nRange-diff vs v1:\n\n 1:  41664a59029 = 1:  41664a59029 archive: add --recurse-submodules to git-archive command\n -:  ----------- > 2:  68f7830c6d9 archive: fix a case of submodule in submodule traversal\n\n-- \ngitgitgadget\n"},{"id":"464824","messageId":"41664a59029e2492079bef3d3da10f44d315064b.1665660960.git.gitgitgadget@gmail.com","threadId":"58613","inReplyTo":"pull.1359.v2.git.git.1665660960.gitgitgadget@gmail.com","subject":"[PATCH v2 1/2] archive: add --recurse-submodules to git-archive command","fromName":"Alphadelta14 via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-13T11:35:59Z","receivedAt":"2022-10-13T11:36:18Z","isPatch":true,"sender":{"key":"name:Alphadelta14","avatar":null},"body":"From: Alphadelta14 <alpha@alphaservcomputing.solutions>\n\nThis makes it possible to include submodule contents in an archive command.\n\nThis required updating the general read_tree callbacks to support sub-repos\nby not using the_repository global references where possible.\n\narchive: update streaming to use target repo\narchive: add test cases for git archive --recurse-submodules\n\nSigned-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n---\n archive-tar.c                 | 14 +++--\n archive-zip.c                 | 14 ++---\n archive.c                     | 99 ++++++++++++++++++++++++-----------\n archive.h                     |  8 +--\n builtin/checkout.c            |  2 +-\n builtin/log.c                 |  2 +-\n builtin/ls-files.c            | 10 ++--\n builtin/ls-tree.c             | 16 +++---\n list-objects.c                |  2 +-\n merge-recursive.c             |  2 +-\n revision.c                    |  4 +-\n sparse-index.c                |  2 +-\n t/t5005-archive-submodules.sh | 84 +++++++++++++++++++++++++++++\n tree.c                        | 64 ++++++++++++++--------\n tree.h                        | 11 ++--\n wt-status.c                   |  2 +-\n 16 files changed, 246 insertions(+), 90 deletions(-)\n create mode 100755 t/t5005-archive-submodules.sh\n\ndiff --git a/archive-tar.c b/archive-tar.c\nindex 3e4822b6840..331fc10bca9 100644\n--- a/archive-tar.c\n+++ b/archive-tar.c\n@@ -18,6 +18,7 @@ static unsigned long offset;\n static int tar_umask = 002;\n \n static int write_tar_filter_archive(const struct archiver *ar,\n+\t\t\t\t    struct repository *repo,\n \t\t\t\t    struct archiver_args *args);\n \n /*\n@@ -246,7 +247,8 @@ static void write_extended_header(struct archiver_args *args,\n \twrite_blocked(buffer, size);\n }\n \n-static int write_tar_entry(struct archiver_args *args,\n+static int write_tar_entry(struct repository *repo,\n+\t\t\t   struct archiver_args *args,\n \t\t\t   const struct object_id *oid,\n \t\t\t   const char *path, size_t pathlen,\n \t\t\t   unsigned int mode,\n@@ -316,7 +318,7 @@ static int write_tar_entry(struct archiver_args *args,\n \t\tif (buffer)\n \t\t\twrite_blocked(buffer, size);\n \t\telse\n-\t\t\terr = stream_blocked(args->repo, oid);\n+\t\t\terr = stream_blocked(repo, oid);\n \t}\n \treturn err;\n }\n@@ -422,12 +424,13 @@ static int git_tar_config(const char *var, const char *value, void *cb)\n }\n \n static int write_tar_archive(const struct archiver *ar UNUSED,\n+\t\t\t     struct repository *repo,\n \t\t\t     struct archiver_args *args)\n {\n \tint err = 0;\n \n \twrite_global_extended_header(args);\n-\terr = write_archive_entries(args, write_tar_entry);\n+\terr = write_archive_entries(repo, args, write_tar_entry);\n \tif (!err)\n \t\twrite_trailer();\n \treturn err;\n@@ -462,6 +465,7 @@ static void tgz_write_block(const void *data)\n static const char internal_gzip_command[] = \"git archive gzip\";\n \n static int write_tar_filter_archive(const struct archiver *ar,\n+\t\t\t\t    struct repository *repo,\n \t\t\t\t    struct archiver_args *args)\n {\n #if ZLIB_VERNUM >= 0x1221\n@@ -484,7 +488,7 @@ static int write_tar_filter_archive(const struct archiver *ar,\n \t\tgzstream.next_out = outbuf;\n \t\tgzstream.avail_out = sizeof(outbuf);\n \n-\t\tr = write_tar_archive(ar, args);\n+\t\tr = write_tar_archive(ar, repo, args);\n \n \t\ttgz_deflate(Z_FINISH);\n \t\tgit_deflate_end(&gzstream);\n@@ -506,7 +510,7 @@ static int write_tar_filter_archive(const struct archiver *ar,\n \t\tdie_errno(_(\"unable to redirect descriptor\"));\n \tclose(filter.in);\n \n-\tr = write_tar_archive(ar, args);\n+\tr = write_tar_archive(ar, repo, args);\n \n \tclose(1);\n \tif (finish_command(&filter) != 0)\ndiff --git a/archive-zip.c b/archive-zip.c\nindex 0456f1ebf15..6b27d004b5d 100644\n--- a/archive-zip.c\n+++ b/archive-zip.c\n@@ -283,7 +283,8 @@ static int entry_is_binary(struct index_state *istate, const char *path,\n \n #define STREAM_BUFFER_SIZE (1024 * 16)\n \n-static int write_zip_entry(struct archiver_args *args,\n+static int write_zip_entry(struct repository *repo,\n+\t\t\t   struct archiver_args *args,\n \t\t\t   const struct object_id *oid,\n \t\t\t   const char *path, size_t pathlen,\n \t\t\t   unsigned int mode,\n@@ -340,7 +341,7 @@ static int write_zip_entry(struct archiver_args *args,\n \n \t\tif (!buffer) {\n \t\t\tenum object_type type;\n-\t\t\tstream = open_istream(args->repo, oid, &type, &size,\n+\t\t\tstream = open_istream(repo, oid, &type, &size,\n \t\t\t\t\t      NULL);\n \t\t\tif (!stream)\n \t\t\t\treturn error(_(\"cannot stream blob %s\"),\n@@ -349,7 +350,7 @@ static int write_zip_entry(struct archiver_args *args,\n \t\t\tout = NULL;\n \t\t} else {\n \t\t\tcrc = crc32(crc, buffer, size);\n-\t\t\tis_binary = entry_is_binary(args->repo->index,\n+\t\t\tis_binary = entry_is_binary(repo->index,\n \t\t\t\t\t\t    path_without_prefix,\n \t\t\t\t\t\t    buffer, size);\n \t\t\tout = buffer;\n@@ -426,7 +427,7 @@ static int write_zip_entry(struct archiver_args *args,\n \t\t\t\tbreak;\n \t\t\tcrc = crc32(crc, buf, readlen);\n \t\t\tif (is_binary == -1)\n-\t\t\t\tis_binary = entry_is_binary(args->repo->index,\n+\t\t\t\tis_binary = entry_is_binary(repo->index,\n \t\t\t\t\t\t\t    path_without_prefix,\n \t\t\t\t\t\t\t    buf, readlen);\n \t\t\twrite_or_die(1, buf, readlen);\n@@ -459,7 +460,7 @@ static int write_zip_entry(struct archiver_args *args,\n \t\t\t\tbreak;\n \t\t\tcrc = crc32(crc, buf, readlen);\n \t\t\tif (is_binary == -1)\n-\t\t\t\tis_binary = entry_is_binary(args->repo->index,\n+\t\t\t\tis_binary = entry_is_binary(repo->index,\n \t\t\t\t\t\t\t    path_without_prefix,\n \t\t\t\t\t\t\t    buf, readlen);\n \n@@ -619,6 +620,7 @@ static int archive_zip_config(const char *var, const char *value,\n }\n \n static int write_zip_archive(const struct archiver *ar UNUSED,\n+\t\t\t     struct repository *repo,\n \t\t\t     struct archiver_args *args)\n {\n \tint err;\n@@ -629,7 +631,7 @@ static int write_zip_archive(const struct archiver *ar UNUSED,\n \n \tstrbuf_init(&zip_dir, 0);\n \n-\terr = write_archive_entries(args, write_zip_entry);\n+\terr = write_archive_entries(repo, args, write_zip_entry);\n \tif (!err)\n \t\twrite_zip_trailer(args->commit_oid);\n \ndiff --git a/archive.c b/archive.c\nindex 61a79e4a227..2ee7205679d 100644\n--- a/archive.c\n+++ b/archive.c\n@@ -10,6 +10,7 @@\n #include \"unpack-trees.h\"\n #include \"dir.h\"\n #include \"quote.h\"\n+#include \"submodule.h\"\n \n static char const * const archive_usage[] = {\n \tN_(\"git archive [<options>] <tree-ish> [<path>...]\"),\n@@ -69,6 +70,7 @@ static void format_subst(const struct commit *commit,\n }\n \n static void *object_file_to_archive(const struct archiver_args *args,\n+\t\t\t\t    struct repository *repo,\n \t\t\t\t    const char *path,\n \t\t\t\t    const struct object_id *oid,\n \t\t\t\t    unsigned int mode,\n@@ -84,13 +86,13 @@ static void *object_file_to_archive(const struct archiver_args *args,\n \t\t\t       (args->tree ? &args->tree->object.oid : NULL), oid);\n \n \tpath += args->baselen;\n-\tbuffer = read_object_file(oid, type, sizep);\n+\tbuffer = repo_read_object_file(repo, oid, type, sizep);\n \tif (buffer && S_ISREG(mode)) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tsize_t size = 0;\n \n \t\tstrbuf_attach(&buf, buffer, *sizep, *sizep + 1);\n-\t\tconvert_to_working_tree(args->repo->index, path, buf.buf, buf.len, &buf, &meta);\n+\t\tconvert_to_working_tree(repo->index, path, buf.buf, buf.len, &buf, &meta);\n \t\tif (commit)\n \t\t\tformat_subst(commit, buf.buf, buf.len, &buf, args->pretty_ctx);\n \t\tbuffer = strbuf_detach(&buf, &size);\n@@ -134,7 +136,7 @@ static int check_attr_export_subst(const struct attr_check *check)\n \treturn check && ATTR_TRUE(check->items[1].value);\n }\n \n-static int write_archive_entry(const struct object_id *oid, const char *base,\n+static int write_archive_entry(struct repository *repo, const struct object_id *oid, const char *base,\n \t\tint baselen, const char *filename, unsigned mode,\n \t\tvoid *context)\n {\n@@ -160,7 +162,7 @@ static int write_archive_entry(const struct object_id *oid, const char *base,\n \n \tif (!S_ISDIR(mode)) {\n \t\tconst struct attr_check *check;\n-\t\tcheck = get_archive_attrs(args->repo->index, path_without_prefix);\n+\t\tcheck = get_archive_attrs(repo->index, path_without_prefix);\n \t\tif (check_attr_export_ignore(check))\n \t\t\treturn 0;\n \t\targs->convert = check_attr_export_subst(check);\n@@ -169,10 +171,10 @@ static int write_archive_entry(const struct object_id *oid, const char *base,\n \tif (S_ISDIR(mode) || S_ISGITLINK(mode)) {\n \t\tif (args->verbose)\n \t\t\tfprintf(stderr, \"%.*s\\n\", (int)path.len, path.buf);\n-\t\terr = write_entry(args, oid, path.buf, path.len, mode, NULL, 0);\n+\t\terr = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, 0);\n \t\tif (err)\n \t\t\treturn err;\n-\t\treturn (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);\n+\t\treturn READ_TREE_RECURSIVE;\n \t}\n \n \tif (args->verbose)\n@@ -180,14 +182,19 @@ static int write_archive_entry(const struct object_id *oid, const char *base,\n \n \t/* Stream it? */\n \tif (S_ISREG(mode) && !args->convert &&\n-\t    oid_object_info(args->repo, oid, &size) == OBJ_BLOB &&\n-\t    size > big_file_threshold)\n-\t\treturn write_entry(args, oid, path.buf, path.len, mode, NULL, size);\n+\t    oid_object_info(repo, oid, &size) == OBJ_BLOB &&\n+\t    size > big_file_threshold) {\n+\t\t\terr = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, size);\n+\t\t\tif (err) {\n+\t\t\t\tdie(\"Failed to write file %.*s\", (int)path.len, path.buf);\n+\t\t\t}\n+\t\t\treturn err;\n+\t\t}\n \n-\tbuffer = object_file_to_archive(args, path.buf, oid, mode, &type, &size);\n+\tbuffer = object_file_to_archive(args, repo, path.buf, oid, mode, &type, &size);\n \tif (!buffer)\n \t\treturn error(_(\"cannot read '%s'\"), oid_to_hex(oid));\n-\terr = write_entry(args, oid, path.buf, path.len, mode, buffer, size);\n+\terr = write_entry(repo, args, oid, path.buf, path.len, mode, buffer, size);\n \tfree(buffer);\n \treturn err;\n }\n@@ -207,7 +214,25 @@ static void queue_directory(const struct object_id *oid,\n \toidcpy(&d->oid, oid);\n }\n \n-static int write_directory(struct archiver_context *c)\n+static void queue_submodule(struct repository *superproject,\n+\t\tconst struct object_id *oid,\n+\t\tstruct strbuf *base, const char *filename,\n+\t\tunsigned mode, struct archiver_context *c)\n+{\n+\tstruct repository subrepo;\n+\n+\tif (repo_submodule_init(&subrepo, superproject, filename, null_oid()))\n+\t\treturn;\n+\n+\tif (repo_read_index(&subrepo) < 0)\n+\t\tdie(\"index file corrupt\");\n+\n+    queue_directory(oid, base, filename, mode, c);\n+\n+\trepo_clear(&subrepo);\n+}\n+\n+static int write_directory(struct repository *repo, struct archiver_context *c)\n {\n \tstruct directory *d = c->bottom;\n \tint ret;\n@@ -217,15 +242,18 @@ static int write_directory(struct archiver_context *c)\n \tc->bottom = d->up;\n \td->path[d->len - 1] = '\\0'; /* no trailing slash */\n \tret =\n-\t\twrite_directory(c) ||\n-\t\twrite_archive_entry(&d->oid, d->path, d->baselen,\n+\t\twrite_directory(repo, c) ||\n+\t\twrite_archive_entry(repo, &d->oid, d->path, d->baselen,\n \t\t\t\t    d->path + d->baselen, d->mode,\n-\t\t\t\t    c) != READ_TREE_RECURSIVE;\n+\t\t\t\t    c);\n \tfree(d);\n-\treturn ret ? -1 : 0;\n+\tif (ret == READ_TREE_RECURSIVE)\n+\t\treturn 0;\n+\treturn ret;\n }\n \n-static int queue_or_write_archive_entry(const struct object_id *oid,\n+static int queue_or_write_archive_entry(\n+\t\tstruct repository *repo, const struct object_id *oid,\n \t\tstruct strbuf *base, const char *filename,\n \t\tunsigned mode, void *context)\n {\n@@ -246,18 +274,23 @@ static int queue_or_write_archive_entry(const struct object_id *oid,\n \t\t/* Borrow base, but restore its original value when done. */\n \t\tstrbuf_addstr(base, filename);\n \t\tstrbuf_addch(base, '/');\n-\t\tcheck = get_archive_attrs(c->args->repo->index, base->buf);\n+\t\tcheck = get_archive_attrs(repo->index, base->buf);\n \t\tstrbuf_setlen(base, baselen);\n \n \t\tif (check_attr_export_ignore(check))\n \t\t\treturn 0;\n \t\tqueue_directory(oid, base, filename, mode, c);\n \t\treturn READ_TREE_RECURSIVE;\n+\t} else if (c->args->recurse_submodules && S_ISGITLINK(mode)) {\n+\t\tif (is_submodule_active(repo, filename)) {\n+\t\t\tqueue_submodule(repo, oid, base, filename, mode, c);\n+\t\t\treturn READ_TREE_RECURSIVE;\n+\t\t}\n \t}\n \n-\tif (write_directory(c))\n+\tif (write_directory(repo, c))\n \t\treturn -1;\n-\treturn write_archive_entry(oid, base->buf, base->len, filename, mode,\n+\treturn write_archive_entry(repo, oid, base->buf, base->len, filename, mode,\n \t\t\t\t   context);\n }\n \n@@ -267,7 +300,8 @@ struct extra_file_info {\n \tvoid *content;\n };\n \n-int write_archive_entries(struct archiver_args *args,\n+int write_archive_entries(struct repository *repo,\n+\t\tstruct archiver_args *args,\n \t\twrite_archive_entry_fn_t write_entry)\n {\n \tstruct archiver_context context;\n@@ -288,7 +322,7 @@ int write_archive_entries(struct archiver_args *args,\n \t\t\tlen--;\n \t\tif (args->verbose)\n \t\t\tfprintf(stderr, \"%.*s\\n\", (int)len, args->base);\n-\t\terr = write_entry(args, &args->tree->object.oid, args->base,\n+\t\terr = write_entry(repo, args, &args->tree->object.oid, args->base,\n \t\t\t\t  len, 040777, NULL, 0);\n \t\tif (err)\n \t\t\treturn err;\n@@ -305,8 +339,8 @@ int write_archive_entries(struct archiver_args *args,\n \t\tmemset(&opts, 0, sizeof(opts));\n \t\topts.index_only = 1;\n \t\topts.head_idx = -1;\n-\t\topts.src_index = args->repo->index;\n-\t\topts.dst_index = args->repo->index;\n+\t\topts.src_index = repo->index;\n+\t\topts.dst_index = repo->index;\n \t\topts.fn = oneway_merge;\n \t\tinit_tree_desc(&t, args->tree->buffer, args->tree->size);\n \t\tif (unpack_trees(1, &t, &opts))\n@@ -314,7 +348,7 @@ int write_archive_entries(struct archiver_args *args,\n \t\tgit_attr_set_direction(GIT_ATTR_INDEX);\n \t}\n \n-\terr = read_tree(args->repo, args->tree,\n+\terr = read_tree(repo, args->tree,\n \t\t\t&args->pathspec,\n \t\t\tqueue_or_write_archive_entry,\n \t\t\t&context);\n@@ -343,12 +377,12 @@ int write_archive_entries(struct archiver_args *args,\n \t\t\tif (strbuf_read_file(&content, path, info->stat.st_size) < 0)\n \t\t\t\terr = error_errno(_(\"cannot read '%s'\"), path);\n \t\t\telse\n-\t\t\t\terr = write_entry(args, &fake_oid, path_in_archive.buf,\n+\t\t\t\terr = write_entry(repo, args, &fake_oid, path_in_archive.buf,\n \t\t\t\t\t\t  path_in_archive.len,\n \t\t\t\t\t\t  canon_mode(info->stat.st_mode),\n \t\t\t\t\t\t  content.buf, content.len);\n \t\t} else {\n-\t\t\terr = write_entry(args, &fake_oid,\n+\t\t\terr = write_entry(repo, args, &fake_oid,\n \t\t\t\t\t  path, strlen(path),\n \t\t\t\t\t  canon_mode(info->stat.st_mode),\n \t\t\t\t\t  info->content, info->stat.st_size);\n@@ -382,7 +416,7 @@ struct path_exists_context {\n \tstruct archiver_args *args;\n };\n \n-static int reject_entry(const struct object_id *oid UNUSED,\n+static int reject_entry(struct repository *repo, const struct object_id *oid UNUSED,\n \t\t\tstruct strbuf *base,\n \t\t\tconst char *filename, unsigned mode,\n \t\t\tvoid *context)\n@@ -394,7 +428,7 @@ static int reject_entry(const struct object_id *oid UNUSED,\n \t\tstruct strbuf sb = STRBUF_INIT;\n \t\tstrbuf_addbuf(&sb, base);\n \t\tstrbuf_addstr(&sb, filename);\n-\t\tif (!match_pathspec(ctx->args->repo->index,\n+\t\tif (!match_pathspec(repo->index,\n \t\t\t\t    &ctx->pathspec,\n \t\t\t\t    sb.buf, sb.len, 0, NULL, 1))\n \t\t\tret = READ_TREE_RECURSIVE;\n@@ -431,6 +465,7 @@ static void parse_pathspec_arg(const char **pathspec,\n \t\t       PATHSPEC_PREFER_FULL,\n \t\t       \"\", pathspec);\n \tar_args->pathspec.recursive = 1;\n+\tar_args->pathspec.recurse_submodules = ar_args->recurse_submodules;\n \tif (pathspec) {\n \t\twhile (*pathspec) {\n \t\t\tif (**pathspec && !path_exists(ar_args, *pathspec))\n@@ -592,6 +627,7 @@ static int parse_archive_args(int argc, const char **argv,\n \tint verbose = 0;\n \tint i;\n \tint list = 0;\n+\tint recurse_submodules = 0;\n \tint worktree_attributes = 0;\n \tstruct option opts[] = {\n \t\tOPT_GROUP(\"\"),\n@@ -606,6 +642,8 @@ static int parse_archive_args(int argc, const char **argv,\n \t\t  add_file_cb, (intptr_t)&base },\n \t\tOPT_STRING('o', \"output\", &output, N_(\"file\"),\n \t\t\tN_(\"write the archive to this file\")),\n+\t\tOPT_BOOL(0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\tN_(\"include submodules in archive\")),\n \t\tOPT_BOOL(0, \"worktree-attributes\", &worktree_attributes,\n \t\t\tN_(\"read .gitattributes in working directory\")),\n \t\tOPT__VERBOSE(&verbose, N_(\"report archived files on stderr\")),\n@@ -670,6 +708,7 @@ static int parse_archive_args(int argc, const char **argv,\n \targs->base = base;\n \targs->baselen = strlen(base);\n \targs->worktree_attributes = worktree_attributes;\n+\targs->recurse_submodules = recurse_submodules;\n \n \treturn argc;\n }\n@@ -708,7 +747,7 @@ int write_archive(int argc, const char **argv, const char *prefix,\n \tparse_treeish_arg(argv, &args, prefix, remote);\n \tparse_pathspec_arg(argv + 1, &args);\n \n-\trc = ar->write_archive(ar, &args);\n+\trc = ar->write_archive(ar, repo, &args);\n \n \tstring_list_clear_func(&args.extra_files, extra_file_info_clear);\n \tfree(args.refname);\ndiff --git a/archive.h b/archive.h\nindex 08bed3ed3af..91b318f9ad8 100644\n--- a/archive.h\n+++ b/archive.h\n@@ -19,6 +19,7 @@ struct archiver_args {\n \ttimestamp_t time;\n \tstruct pathspec pathspec;\n \tunsigned int verbose : 1;\n+\tunsigned int recurse_submodules : 1;\n \tunsigned int worktree_attributes : 1;\n \tunsigned int convert : 1;\n \tint compression_level;\n@@ -41,7 +42,7 @@ const char *archive_format_from_filename(const char *filename);\n #define ARCHIVER_HIGH_COMPRESSION_LEVELS 4\n struct archiver {\n \tconst char *name;\n-\tint (*write_archive)(const struct archiver *, struct archiver_args *);\n+\tint (*write_archive)(const struct archiver *, struct repository *repo, struct archiver_args *);\n \tunsigned flags;\n \tchar *filter_command;\n };\n@@ -51,12 +52,13 @@ void init_tar_archiver(void);\n void init_zip_archiver(void);\n void init_archivers(void);\n \n-typedef int (*write_archive_entry_fn_t)(struct archiver_args *args,\n+typedef int (*write_archive_entry_fn_t)(struct repository *repo,\n+\t\t\t\t\tstruct archiver_args *args,\n \t\t\t\t\tconst struct object_id *oid,\n \t\t\t\t\tconst char *path, size_t pathlen,\n \t\t\t\t\tunsigned int mode,\n \t\t\t\t\tvoid *buffer, unsigned long size);\n \n-int write_archive_entries(struct archiver_args *args, write_archive_entry_fn_t write_entry);\n+int write_archive_entries(struct repository *repo, struct archiver_args *args, write_archive_entry_fn_t write_entry);\n \n #endif\t/* ARCHIVE_H */\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex 2a132392fbe..1238774b245 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -124,7 +124,7 @@ static int post_checkout_hook(struct commit *old_commit, struct commit *new_comm\n \n }\n \n-static int update_some(const struct object_id *oid, struct strbuf *base,\n+static int update_some(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n \t\t       const char *pathname, unsigned mode, void *context UNUSED)\n {\n \tint len;\ndiff --git a/builtin/log.c b/builtin/log.c\nindex ee19dc5d450..da73bbef836 100644\n--- a/builtin/log.c\n+++ b/builtin/log.c\n@@ -698,7 +698,7 @@ static int show_tag_object(const struct object_id *oid, struct rev_info *rev)\n \treturn 0;\n }\n \n-static int show_tree_object(const struct object_id *oid UNUSED,\n+static int show_tree_object(struct repository *repo UNUSED, const struct object_id *oid UNUSED,\n \t\t\t    struct strbuf *base UNUSED,\n \t\t\t    const char *pathname, unsigned mode,\n \t\t\t    void *context)\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex 4cf8a236483..a08768dbd2a 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -509,7 +509,7 @@ static int get_common_prefix_len(const char *common_prefix)\n \treturn common_prefix_len;\n }\n \n-static int read_one_entry_opt(struct index_state *istate,\n+static int read_one_entry_opt(struct repository *repo UNUSED, struct index_state *istate,\n \t\t\t      const struct object_id *oid,\n \t\t\t      struct strbuf *base,\n \t\t\t      const char *pathname,\n@@ -533,12 +533,12 @@ static int read_one_entry_opt(struct index_state *istate,\n \treturn add_index_entry(istate, ce, opt);\n }\n \n-static int read_one_entry(const struct object_id *oid, struct strbuf *base,\n+static int read_one_entry(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n \t\t\t  const char *pathname, unsigned mode,\n \t\t\t  void *context)\n {\n \tstruct index_state *istate = context;\n-\treturn read_one_entry_opt(istate, oid, base, pathname,\n+\treturn read_one_entry_opt(repo, istate, oid, base, pathname,\n \t\t\t\t  mode,\n \t\t\t\t  ADD_CACHE_OK_TO_ADD|ADD_CACHE_SKIP_DFCHECK);\n }\n@@ -547,12 +547,12 @@ static int read_one_entry(const struct object_id *oid, struct strbuf *base,\n  * This is used when the caller knows there is no existing entries at\n  * the stage that will conflict with the entry being added.\n  */\n-static int read_one_entry_quick(const struct object_id *oid, struct strbuf *base,\n+static int read_one_entry_quick(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n \t\t\t\tconst char *pathname, unsigned mode,\n \t\t\t\tvoid *context)\n {\n \tstruct index_state *istate = context;\n-\treturn read_one_entry_opt(istate, oid, base, pathname,\n+\treturn read_one_entry_opt(repo, istate, oid, base, pathname,\n \t\t\t\t  mode, ADD_CACHE_JUST_APPEND);\n }\n \ndiff --git a/builtin/ls-tree.c b/builtin/ls-tree.c\nindex c3ea09281af..c8d6ff95ac0 100644\n--- a/builtin/ls-tree.c\n+++ b/builtin/ls-tree.c\n@@ -141,7 +141,7 @@ static int show_recursive(const char *base, size_t baselen, const char *pathname\n \treturn 0;\n }\n \n-static int show_tree_fmt(const struct object_id *oid, struct strbuf *base,\n+static int show_tree_fmt(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n \t\t\t const char *pathname, unsigned mode, void *context UNUSED)\n {\n \tsize_t baselen;\n@@ -211,7 +211,7 @@ static void show_tree_common_default_long(struct strbuf *base,\n \tstrbuf_setlen(base, baselen);\n }\n \n-static int show_tree_default(const struct object_id *oid, struct strbuf *base,\n+static int show_tree_default(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n \t\t\t     const char *pathname, unsigned mode,\n \t\t\t     void *context UNUSED)\n {\n@@ -229,7 +229,7 @@ static int show_tree_default(const struct object_id *oid, struct strbuf *base,\n \treturn recurse;\n }\n \n-static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n+static int show_tree_long(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n \t\t\t  const char *pathname, unsigned mode,\n \t\t\t  void *context UNUSED)\n {\n@@ -244,7 +244,7 @@ static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n \n \tif (data.type == OBJ_BLOB) {\n \t\tunsigned long size;\n-\t\tif (oid_object_info(the_repository, data.oid, &size) == OBJ_BAD)\n+\t\tif (oid_object_info(repo, data.oid, &size) == OBJ_BAD)\n \t\t\txsnprintf(size_text, sizeof(size_text), \"BAD\");\n \t\telse\n \t\t\txsnprintf(size_text, sizeof(size_text),\n@@ -254,12 +254,12 @@ static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n \t}\n \n \tprintf(\"%06o %s %s %7s\\t\", data.mode, type_name(data.type),\n-\t       find_unique_abbrev(data.oid, abbrev), size_text);\n+\t       repo_find_unique_abbrev(repo, data.oid, abbrev), size_text);\n \tshow_tree_common_default_long(base, pathname, data.base->len);\n \treturn recurse;\n }\n \n-static int show_tree_name_only(const struct object_id *oid, struct strbuf *base,\n+static int show_tree_name_only(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n \t\t\t       const char *pathname, unsigned mode,\n \t\t\t       void *context UNUSED)\n {\n@@ -280,7 +280,7 @@ static int show_tree_name_only(const struct object_id *oid, struct strbuf *base,\n \treturn recurse;\n }\n \n-static int show_tree_object(const struct object_id *oid, struct strbuf *base,\n+static int show_tree_object(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n \t\t\t    const char *pathname, unsigned mode,\n \t\t\t    void *context UNUSED)\n {\n@@ -292,7 +292,7 @@ static int show_tree_object(const struct object_id *oid, struct strbuf *base,\n \tif (early >= 0)\n \t\treturn early;\n \n-\tprintf(\"%s%c\", find_unique_abbrev(oid, abbrev), line_termination);\n+\tprintf(\"%s%c\", repo_find_unique_abbrev(repo, oid, abbrev), line_termination);\n \treturn recurse;\n }\n \ndiff --git a/list-objects.c b/list-objects.c\nindex 250d9de41cb..4f463ae32ad 100644\n--- a/list-objects.c\n+++ b/list-objects.c\n@@ -185,7 +185,7 @@ static void process_tree(struct traversal_context *ctx,\n \t    !revs->include_check_obj(&tree->object, revs->include_check_data))\n \t\treturn;\n \n-\tfailed_parse = parse_tree_gently(tree, 1);\n+\tfailed_parse = parse_tree_gently(revs->repo, tree, 1);\n \tif (failed_parse) {\n \t\tif (revs->ignore_missing_links)\n \t\t\treturn;\ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex 4ddd3adea00..119e86090f9 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -456,7 +456,7 @@ static void unpack_trees_finish(struct merge_options *opt)\n \tclear_unpack_trees_porcelain(&opt->priv->unpack_opts);\n }\n \n-static int save_files_dirs(const struct object_id *oid UNUSED,\n+static int save_files_dirs(struct repository *repo UNUSED, const struct object_id *oid UNUSED,\n \t\t\t   struct strbuf *base, const char *path,\n \t\t\t   unsigned int mode, void *context)\n {\ndiff --git a/revision.c b/revision.c\nindex 36e31942cee..39e023a19ea 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -74,7 +74,7 @@ static void mark_tree_contents_uninteresting(struct repository *r,\n \tstruct tree_desc desc;\n \tstruct name_entry entry;\n \n-\tif (parse_tree_gently(tree, 1) < 0)\n+\tif (parse_tree_gently(r, tree, 1) < 0)\n \t\treturn;\n \n \tinit_tree_desc(&desc, tree->buffer, tree->size);\n@@ -181,7 +181,7 @@ static void add_children_by_path(struct repository *r,\n \tif (!tree)\n \t\treturn;\n \n-\tif (parse_tree_gently(tree, 1) < 0)\n+\tif (parse_tree_gently(r, tree, 1) < 0)\n \t\treturn;\n \n \tinit_tree_desc(&desc, tree->buffer, tree->size);\ndiff --git a/sparse-index.c b/sparse-index.c\nindex e4a54ce1943..05ca8aec1fb 100644\n--- a/sparse-index.c\n+++ b/sparse-index.c\n@@ -232,7 +232,7 @@ static void set_index_entry(struct index_state *istate, int nr, struct cache_ent\n \tadd_name_hash(istate, ce);\n }\n \n-static int add_path_to_index(const struct object_id *oid,\n+static int add_path_to_index(struct repository *repo UNUSED, const struct object_id *oid,\n \t\t\t     struct strbuf *base, const char *path,\n \t\t\t     unsigned int mode, void *context)\n {\ndiff --git a/t/t5005-archive-submodules.sh b/t/t5005-archive-submodules.sh\nnew file mode 100755\nindex 00000000000..49d5ff74ad5\n--- /dev/null\n+++ b/t/t5005-archive-submodules.sh\n@@ -0,0 +1,84 @@\n+#!/bin/sh\n+\n+test_description='git archive --recurse-submodules test'\n+\n+. ./test-lib.sh\n+. \"$TEST_DIRECTORY\"/lib-submodule-update.sh\n+\n+test_expect_success 'setup' '\n+\tcreate_lib_submodule_repo &&\n+\tgit -C submodule_update_repo checkout valid_sub1 &&\n+\tgit -C submodule_update_repo submodule update\n+'\n+\n+check_tar() {\n+\ttarfile=$1.tar\n+\tlistfile=$1.lst\n+\tdir=$1\n+\tdir_with_prefix=$dir/$2\n+\n+\ttest_expect_success ' extract tar archive' '\n+\t\t(mkdir $dir && cd $dir && \"$TAR\" xf -) <$tarfile\n+\t'\n+}\n+\n+check_added() {\n+\tdir=$1\n+\tpath_in_fs=$2\n+\tpath_in_archive=$3\n+\n+\ttest_expect_success \" validate extra file $path_in_archive\" '\n+\t\ttest -f $dir/$path_in_archive &&\n+\t\tdiff -r $path_in_fs $dir/$path_in_archive\n+\t'\n+}\n+\n+check_not_added() {\n+\tdir=$1\n+\tpath_in_archive=$2\n+\n+\ttest_expect_success \" validate unpresent file $path_in_archive\" '\n+\t\t! test -f $dir/$path_in_archive &&\n+\t\t! test -d $dir/$path_in_archive\n+\t'\n+}\n+\n+test_expect_success 'archive without recurse, non-init' '\n+\treset_work_tree_to valid_sub1 &&\n+\tgit -C submodule_update archive HEAD >b.tar\n+'\n+\n+check_tar b\n+check_added b submodule_update/file1 file1\n+check_not_added b sub1/file1\n+\n+test_expect_success 'archive with recurse, non-init' '\n+\treset_work_tree_to valid_sub1 &&\n+\t! git -C submodule_update archive --recurse-submodules HEAD >b2-err.tar\n+'\n+\n+test_expect_success 'archive with recurse, init' '\n+\treset_work_tree_to valid_sub1 &&\n+\tgit -C submodule_update submodule update --init &&\n+\tgit -C submodule_update ls-files --recurse-submodules &&\n+\tgit -C submodule_update ls-tree HEAD &&\n+\tgit -C submodule_update archive --recurse-submodules HEAD >b2.tar\n+'\n+\n+check_tar b2\n+check_added b2 submodule_update/sub1/file1 sub1/file1\n+\n+test_expect_success 'archive with recurse with big files' '\n+\treset_work_tree_to valid_sub1 &&\n+\ttest_config core.bigfilethreshold 1 &&\n+\tgit -C submodule_update submodule update --init &&\n+\tgit -C submodule_update ls-files --recurse-submodules &&\n+\tgit -C submodule_update ls-tree HEAD &&\n+\tgit -C submodule_update archive --recurse-submodules HEAD >b3.tar\n+'\n+\n+check_tar b3\n+check_added b3 submodule_update/sub1/file1 sub1/file1\n+\n+\n+test_done\ndiff --git a/tree.c b/tree.c\nindex 410e3b477e5..c5b5a0ac08f 100644\n--- a/tree.c\n+++ b/tree.c\n@@ -8,6 +8,7 @@\n #include \"alloc.h\"\n #include \"tree-walk.h\"\n #include \"repository.h\"\n+#include \"pathspec.h\"\n \n const char *tree_type = \"tree\";\n \n@@ -22,8 +23,8 @@ int read_tree_at(struct repository *r,\n \tint len, oldlen = base->len;\n \tenum interesting retval = entry_not_interesting;\n \n-\tif (parse_tree(tree))\n-\t\treturn -1;\n+\tif (repo_parse_tree(r, tree))\n+\t\tdie(\"Failed to parse tree\");\n \n \tinit_tree_desc(&desc, tree->buffer, tree->size);\n \n@@ -37,7 +38,7 @@ int read_tree_at(struct repository *r,\n \t\t\t\tcontinue;\n \t\t}\n \n-\t\tswitch (fn(&entry.oid, base,\n+\t\tswitch (fn(r, &entry.oid, base,\n \t\t\t   entry.path, entry.mode, context)) {\n \t\tcase 0:\n \t\t\tcontinue;\n@@ -47,36 +48,57 @@ int read_tree_at(struct repository *r,\n \t\t\treturn -1;\n \t\t}\n \n-\t\tif (S_ISDIR(entry.mode))\n+\t\tif (S_ISDIR(entry.mode)) {\n \t\t\toidcpy(&oid, &entry.oid);\n-\t\telse if (S_ISGITLINK(entry.mode)) {\n+\t\t\tlen = tree_entry_len(&entry);\n+\t\t\tstrbuf_add(base, entry.path, len);\n+\t\t\tstrbuf_addch(base, '/');\n+\t\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n+\t\t\t\t\t\tbase, pathspec,\n+\t\t\t\t\t\tfn, context);\n+\t\t\tstrbuf_setlen(base, oldlen);\n+\t\t\tif (retval)\n+\t\t\t\treturn -1;\n+\t\t} else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) {\n \t\t\tstruct commit *commit;\n+\t\t\tstruct repository subrepo;\n+\t\t\tstruct repository* subrepo_p = &subrepo;\n+\t\t\tstruct tree* submodule_tree;\n \n-\t\t\tcommit = lookup_commit(r, &entry.oid);\n+\t\t\tif (repo_submodule_init(subrepo_p, r, entry.path, null_oid()))\n+\t\t\t\tdie(\"couldn't init submodule %s%s\", base->buf, entry.path);\n+\n+\t\t\tif (repo_read_index(subrepo_p) < 0)\n+\t\t\t\tdie(\"index file corrupt\");\n+\n+\t\t\tcommit = lookup_commit(subrepo_p, &entry.oid);\n \t\t\tif (!commit)\n \t\t\t\tdie(\"Commit %s in submodule path %s%s not found\",\n \t\t\t\t    oid_to_hex(&entry.oid),\n \t\t\t\t    base->buf, entry.path);\n \n-\t\t\tif (parse_commit(commit))\n+\t\t\tif (repo_parse_commit(subrepo_p, commit))\n \t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n \t\t\t\t    oid_to_hex(&entry.oid),\n \t\t\t\t    base->buf, entry.path);\n \n-\t\t\toidcpy(&oid, get_commit_tree_oid(commit));\n+\t\t\tsubmodule_tree = repo_get_commit_tree(subrepo_p, commit);\n+\t\t\toidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);\n+\n+\t\t\tlen = tree_entry_len(&entry);\n+\t\t\tstrbuf_add(base, entry.path, len);\n+\t\t\tstrbuf_addch(base, '/');\n+\t\t\tretval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),\n+\t\t\t\t\t\tbase, pathspec,\n+\t\t\t\t\t\tfn, context);\n+\t\t\tif (retval) {\n+\t\t\t    die(\"failed to read tree for %s%s\", base->buf, entry.path);\n+\t\t\t    return -1;\n+\t\t\t}\n+\t\t\tstrbuf_setlen(base, oldlen);\n+\t\t\trepo_clear(subrepo_p);\n \t\t}\n-\t\telse\n-\t\t\tcontinue;\n \n-\t\tlen = tree_entry_len(&entry);\n-\t\tstrbuf_add(base, entry.path, len);\n-\t\tstrbuf_addch(base, '/');\n-\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n-\t\t\t\t      base, pathspec,\n-\t\t\t\t      fn, context);\n-\t\tstrbuf_setlen(base, oldlen);\n-\t\tif (retval)\n-\t\t\treturn -1;\n \t}\n \treturn 0;\n }\n@@ -121,7 +143,7 @@ int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size)\n \treturn 0;\n }\n \n-int parse_tree_gently(struct tree *item, int quiet_on_missing)\n+int parse_tree_gently(struct repository *r, struct tree *item, int quiet_on_missing)\n {\n \t enum object_type type;\n \t void *buffer;\n@@ -129,7 +151,7 @@ int parse_tree_gently(struct tree *item, int quiet_on_missing)\n \n \tif (item->object.parsed)\n \t\treturn 0;\n-\tbuffer = read_object_file(&item->object.oid, &type, &size);\n+\tbuffer = repo_read_object_file(r, &item->object.oid, &type, &size);\n \tif (!buffer)\n \t\treturn quiet_on_missing ? -1 :\n \t\t\terror(\"Could not read %s\",\ndiff --git a/tree.h b/tree.h\nindex 6efff003e21..3dc8f151760 100644\n--- a/tree.h\n+++ b/tree.h\n@@ -18,11 +18,14 @@ struct tree *lookup_tree(struct repository *r, const struct object_id *oid);\n \n int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size);\n \n-int parse_tree_gently(struct tree *tree, int quiet_on_missing);\n-static inline int parse_tree(struct tree *tree)\n+int parse_tree_gently(struct repository *r, struct tree *tree, int quiet_on_missing);\n+static inline int repo_parse_tree(struct repository *r, struct tree *tree)\n {\n-\treturn parse_tree_gently(tree, 0);\n+\treturn parse_tree_gently(r, tree, 0);\n }\n+#ifndef NO_THE_REPOSITORY_COMPATIBILITY_MACROS\n+#define parse_tree(tree) repo_parse_tree(the_repository, tree)\n+#endif\n void free_tree_buffer(struct tree *tree);\n \n /* Parses and returns the tree in the given ent, chasing tags and commits. */\n@@ -31,7 +34,7 @@ struct tree *parse_tree_indirect(const struct object_id *oid);\n int cmp_cache_name_compare(const void *a_, const void *b_);\n \n #define READ_TREE_RECURSIVE 1\n-typedef int (*read_tree_fn_t)(const struct object_id *, struct strbuf *, const char *, unsigned int, void *);\n+typedef int (*read_tree_fn_t)(struct repository *, const struct object_id *, struct strbuf *, const char *, unsigned int, void *);\n \n int read_tree_at(struct repository *r,\n \t\t struct tree *tree, struct strbuf *base,\ndiff --git a/wt-status.c b/wt-status.c\nindex 5813174896c..fefe4c55e1b 100644\n--- a/wt-status.c\n+++ b/wt-status.c\n@@ -665,7 +665,7 @@ static void wt_status_collect_changes_index(struct wt_status *s)\n \trelease_revisions(&rev);\n }\n \n-static int add_file_to_list(const struct object_id *oid,\n+static int add_file_to_list(struct repository *repo UNUSED, const struct object_id *oid,\n \t\t\t    struct strbuf *base, const char *path,\n \t\t\t    unsigned int mode, void *context)\n {\n-- \ngitgitgadget\n\n"},{"id":"464874","messageId":"ee64f463-c4f7-cd1e-50ca-7c302415cf52@web.de","threadId":"58613","inReplyTo":"41664a59029e2492079bef3d3da10f44d315064b.1665660960.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 1/2] archive: add --recurse-submodules to git-archive command","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2022-10-13T17:53:46Z","receivedAt":"2022-10-13T17:55:11Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 13.10.22 um 13:35 schrieb Alphadelta14 via GitGitGadget:\n> From: Alphadelta14 <alpha@alphaservcomputing.solutions>\n>\n\n> diff --git a/tree.c b/tree.c\n> index 410e3b477e5..c5b5a0ac08f 100644\n> --- a/tree.c\n> +++ b/tree.c\n> @@ -8,6 +8,7 @@\n>  #include \"alloc.h\"\n>  #include \"tree-walk.h\"\n>  #include \"repository.h\"\n> +#include \"pathspec.h\"\n>\n>  const char *tree_type = \"tree\";\n>\n> @@ -22,8 +23,8 @@ int read_tree_at(struct repository *r,\n>  \tint len, oldlen = base->len;\n>  \tenum interesting retval = entry_not_interesting;\n>\n> -\tif (parse_tree(tree))\n> -\t\treturn -1;\n> +\tif (repo_parse_tree(r, tree))\n> +\t\tdie(\"Failed to parse tree\");\n>\n>  \tinit_tree_desc(&desc, tree->buffer, tree->size);\n>\n> @@ -37,7 +38,7 @@ int read_tree_at(struct repository *r,\n>  \t\t\t\tcontinue;\n>  \t\t}\n>\n> -\t\tswitch (fn(&entry.oid, base,\n> +\t\tswitch (fn(r, &entry.oid, base,\n>  \t\t\t   entry.path, entry.mode, context)) {\n>  \t\tcase 0:\n>  \t\t\tcontinue;\n> @@ -47,36 +48,57 @@ int read_tree_at(struct repository *r,\n>  \t\t\treturn -1;\n>  \t\t}\n>\n> -\t\tif (S_ISDIR(entry.mode))\n> +\t\tif (S_ISDIR(entry.mode)) {\n>  \t\t\toidcpy(&oid, &entry.oid);\n> -\t\telse if (S_ISGITLINK(entry.mode)) {\n\nSo you remove the non-recursive handling of submodules here...\n\n> +\t\t\tlen = tree_entry_len(&entry);\n> +\t\t\tstrbuf_add(base, entry.path, len);\n> +\t\t\tstrbuf_addch(base, '/');\n> +\t\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n> +\t\t\t\t\t\tbase, pathspec,\n> +\t\t\t\t\t\tfn, context);\n> +\t\t\tstrbuf_setlen(base, oldlen);\n> +\t\t\tif (retval)\n> +\t\t\t\treturn -1;\n> +\t\t} else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) {\n\n... and add recursive handling here, and there is no further else\nbranch.  Why do we no longer need the non-recursive variant?\n\n>  \t\t\tstruct commit *commit;\n> +\t\t\tstruct repository subrepo;\n> +\t\t\tstruct repository* subrepo_p = &subrepo;\n> +\t\t\tstruct tree* submodule_tree;\n>\n> -\t\t\tcommit = lookup_commit(r, &entry.oid);\n> +\t\t\tif (repo_submodule_init(subrepo_p, r, entry.path, null_oid()))\n> +\t\t\t\tdie(\"couldn't init submodule %s%s\", base->buf, entry.path);\n> +\n> +\t\t\tif (repo_read_index(subrepo_p) < 0)\n> +\t\t\t\tdie(\"index file corrupt\");\n> +\n> +\t\t\tcommit = lookup_commit(subrepo_p, &entry.oid);\n>  \t\t\tif (!commit)\n>  \t\t\t\tdie(\"Commit %s in submodule path %s%s not found\",\n>  \t\t\t\t    oid_to_hex(&entry.oid),\n>  \t\t\t\t    base->buf, entry.path);\n>\n> -\t\t\tif (parse_commit(commit))\n> +\t\t\tif (repo_parse_commit(subrepo_p, commit))\n>  \t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n>  \t\t\t\t    oid_to_hex(&entry.oid),\n>  \t\t\t\t    base->buf, entry.path);\n>\n> -\t\t\toidcpy(&oid, get_commit_tree_oid(commit));\n> +\t\t\tsubmodule_tree = repo_get_commit_tree(subrepo_p, commit);\n> +\t\t\toidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);\n> +\n> +\t\t\tlen = tree_entry_len(&entry);\n> +\t\t\tstrbuf_add(base, entry.path, len);\n> +\t\t\tstrbuf_addch(base, '/');\n> +\t\t\tretval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),\n> +\t\t\t\t\t\tbase, pathspec,\n> +\t\t\t\t\t\tfn, context);\n> +\t\t\tif (retval) {\n> +\t\t\t    die(\"failed to read tree for %s%s\", base->buf, entry.path);\n> +\t\t\t    return -1;\n> +\t\t\t}\n> +\t\t\tstrbuf_setlen(base, oldlen);\n> +\t\t\trepo_clear(subrepo_p);\n>  \t\t}\n> -\t\telse\n> -\t\t\tcontinue;\n>\n> -\t\tlen = tree_entry_len(&entry);\n> -\t\tstrbuf_add(base, entry.path, len);\n> -\t\tstrbuf_addch(base, '/');\n> -\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n> -\t\t\t\t      base, pathspec,\n> -\t\t\t\t      fn, context);\n> -\t\tstrbuf_setlen(base, oldlen);\n> -\t\tif (retval)\n> -\t\t\treturn -1;\n>  \t}\n>  \treturn 0;\n>  }\n> @@ -121,7 +143,7 @@ int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size)\n>  \treturn 0;\n>  }\n>\n> -int parse_tree_gently(struct tree *item, int quiet_on_missing)\n> +int parse_tree_gently(struct repository *r, struct tree *item, int quiet_on_missing)\n>  {\n>  \t enum object_type type;\n>  \t void *buffer;\n> @@ -129,7 +151,7 @@ int parse_tree_gently(struct tree *item, int quiet_on_missing)\n>\n>  \tif (item->object.parsed)\n>  \t\treturn 0;\n> -\tbuffer = read_object_file(&item->object.oid, &type, &size);\n> +\tbuffer = repo_read_object_file(r, &item->object.oid, &type, &size);\n>  \tif (!buffer)\n>  \t\treturn quiet_on_missing ? -1 :\n>  \t\t\terror(\"Could not read %s\",\n\n"},{"id":"464875","messageId":"97a82675-22bb-b996-deac-3f13a91f3df4@web.de","threadId":"58613","inReplyTo":"pull.1359.v2.git.git.1665660960.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 0/2] archive: Add --recurse-submodules to git-archive command","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2022-10-13T17:53:44Z","receivedAt":"2022-10-13T17:55:13Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 13.10.22 um 13:35 schrieb Heather Lapointe via GitGitGadget:\n> This makes it possible to include submodule contents in an archive command.\n\nGreat!\n\n> The inspiration for this change comes from this Github thread,\n> https://github.com/dear-github/dear-github/issues/214, with at least 160\n> 👍🏻 's at the time of writing. (I stumbled upon it because I wanted it as\n> well).\n>\n> I figured the underlying implementation wouldn't be too difficult with most\n> of the plumbing already in place, so I decided to add the relevant logic to\n> the client git-archive command.\n>\n> One of the trickier parts of this implementation involved teaching read_tree\n> about submodules. Some of the troublesome areas were still using the\n> the_repository references to look up commit or tree or oid information. I\n> ended up deciding that read_tree_fn_t would probably be best off having a\n> concrete repo reference since it allows changing the context to a subrepo\n> where needed (even though some of the usages did not need it specifically).\n>\n> I am open to feedback since this is all quite new to me :)\n>\n> TODO:\n\nThis list confuses me:\n\n>  * working implementation\n\nWhat exactly is not working, yet?\n\n>  * valgrind\n\nWhat's up with it?  Does is report errors or leaks?\n\n>  * add regression tests\n\nThis series adds a new test script.  Do you plan to add more checks?\n\n>  * update documentation with new flag\n\nThat I can understand: Indeed Documentation/git-archive.txt would need\nan update.\n\n>  * submit to mailing list\n\nBut you already did submit two iterations of this series to the Git\nmailing list!?\n\n>\n> Alphadelta14 (2):\n>   archive: add --recurse-submodules to git-archive command\n>   archive: fix a case of submodule in submodule traversal\n\nWe prefer to keep known bugs out of the repo.  It helps when bisecting,\nfor example.  So it would be better to squash the fix into the patch\nthat adds the feature.  But...\n\n>  archive-tar.c                 | 14 +++--\n>  archive-zip.c                 | 14 ++---\n>  archive.c                     | 99 ++++++++++++++++++++++++-----------\n>  archive.h                     |  8 +--\n>  builtin/checkout.c            |  2 +-\n>  builtin/log.c                 |  2 +-\n>  builtin/ls-files.c            | 10 ++--\n>  builtin/ls-tree.c             | 16 +++---\n>  list-objects.c                |  2 +-\n>  merge-recursive.c             |  2 +-\n>  revision.c                    |  4 +-\n>  sparse-index.c                |  2 +-\n>  t/t5005-archive-submodules.sh | 84 +++++++++++++++++++++++++++++\n>  tree.c                        | 93 ++++++++++++++++++++++----------\n>  tree.h                        | 11 ++--\n>  wt-status.c                   |  2 +-\n>  16 files changed, 269 insertions(+), 96 deletions(-)\n>  create mode 100755 t/t5005-archive-submodules.sh\n\n... this is all a bit much for a single patch, I feel.  Giving\nparse_tree_gently() a repo parameter, adding repo_parse_tree(), using\nit in read_tree_at(), adding a repo parameter to read_tree_fn_t,\nletting read_tree_at() recurse into submodules and adding the new\noption to git archive all seem like topics worth their own patch and\nrationale.\n\nYou probably have all of that in your head right now, but at least my\nattention span and working memory capacity requires smaller morsels.\n\n>\n>\n> base-commit: e85701b4af5b7c2a9f3a1b07858703318dce365d\n> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1359%2FAlphadelta14%2Farchive-recurse-submodules-v2\n> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1359/Alphadelta14/archive-recurse-submodules-v2\n> Pull-Request: https://github.com/git/git/pull/1359\n>\n> Range-diff vs v1:\n>\n>  1:  41664a59029 = 1:  41664a59029 archive: add --recurse-submodules to git-archive command\n>  -:  ----------- > 2:  68f7830c6d9 archive: fix a case of submodule in submodule traversal\n>\n\n"},{"id":"464896","messageId":"183d348227c.c5ba226d125397.1082382798689125662@alphaservcomputing.solutions","threadId":"58613","inReplyTo":"ee64f463-c4f7-cd1e-50ca-7c302415cf52@web.de","subject":"Re: [PATCH v2 1/2] archive: add --recurse-submodules to git-archive command","fromName":"Heather Lapointe","fromEmail":"alpha@alphaservcomputing.solutions","sentAt":"2022-10-13T21:37:43Z","receivedAt":"2022-10-13T21:37:56Z","isPatch":true,"sender":{"key":"alpha@alphaservcomputing.solutions","avatar":null},"body":"\n---- On Thu, 13 Oct 2022 10:53:46 -0700 René Scharfe  wrote ---\n\n > Am 13.10.22 um 13:35 schrieb Alphadelta14 via GitGitGadget: \n > > From: Alphadelta14 alpha@alphaservcomputing.solutions> \n > > \n >  \n > > diff --git a/tree.c b/tree.c \n > > index 410e3b477e5..c5b5a0ac08f 100644 \n > > --- a/tree.c \n > > +++ b/tree.c \n > > @@ -8,6 +8,7 @@ \n > >  #include \"alloc.h\" \n > >  #include \"tree-walk.h\" \n > >  #include \"repository.h\" \n > > +#include \"pathspec.h\" \n > > \n > >  const char *tree_type = \"tree\"; \n > > \n > > @@ -22,8 +23,8 @@ int read_tree_at(struct repository *r, \n > >      int len, oldlen = base->len; \n > >      enum interesting retval = entry_not_interesting; \n > > \n > > -    if (parse_tree(tree)) \n > > -        return -1; \n > > +    if (repo_parse_tree(r, tree)) \n > > +        die(\"Failed to parse tree\"); \n > > \n > >      init_tree_desc(&desc, tree->buffer, tree->size); \n > > \n > > @@ -37,7 +38,7 @@ int read_tree_at(struct repository *r, \n > >                  continue; \n > >          } \n > > \n > > -        switch (fn(&entry.oid, base, \n > > +        switch (fn(r, &entry.oid, base, \n > >                 entry.path, entry.mode, context)) { \n > >          case 0: \n > >              continue; \n > > @@ -47,36 +48,57 @@ int read_tree_at(struct repository *r, \n > >              return -1; \n > >          } \n > > \n > > -        if (S_ISDIR(entry.mode)) \n > > +        if (S_ISDIR(entry.mode)) { \n > >              oidcpy(&oid, &entry.oid); \n > > -        else if (S_ISGITLINK(entry.mode)) { \n >  \n > So you remove the non-recursive handling of submodules here... \n\nThe original logic looked like it had not been thoroughly used in the past years.\nIt was performing a commit lookup within the submodule using the superproject\nrepository, which would not actually work unless there was global state\nbeing contextually pushed where `the_repository` was being replaced\nwith a submodule repository instance (it appeared to not be the case currently).\n\n >  \n > > +            len = tree_entry_len(&entry); \n > > +            strbuf_add(base, entry.path, len); \n > > +            strbuf_addch(base, '/'); \n > > +            retval = read_tree_at(r, lookup_tree(r, &oid), \n > > +                        base, pathspec, \n > > +                        fn, context); \n > > +            strbuf_setlen(base, oldlen); \n > > +            if (retval) \n > > +                return -1; \n > > +        } else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) { \n >  \n > ... and add recursive handling here, and there is no further else \n > branch.  Why do we no longer need the non-recursive variant? \n\nThe else case is that we either have submodules, but we don't have pathspec->recurse_submodules.\nBoth the dir case and submodule cases handle recursion\n(but with enough difference that it didn't make sense to use the same logic).\nThe other else case we simply not recurse (for files).\n\n >  \n > >              struct commit *commit; \n > > +            struct repository subrepo; \n > > +            struct repository* subrepo_p = &subrepo; \n > > +            struct tree* submodule_tree; \n > > \n > > -            commit = lookup_commit(r, &entry.oid); \n > > +            if (repo_submodule_init(subrepo_p, r, entry.path, null_oid())) \n > > +                die(\"couldn't init submodule %s%s\", base->buf, entry.path); \n > > + \n > > +            if (repo_read_index(subrepo_p) < 0) \n > > +                die(\"index file corrupt\"); \n > > + \n > > +            commit = lookup_commit(subrepo_p, &entry.oid); \n > >              if (!commit) \n > >                  die(\"Commit %s in submodule path %s%s not found\", \n > >                      oid_to_hex(&entry.oid), \n > >                      base->buf, entry.path); \n > > \n > > -            if (parse_commit(commit)) \n > > +            if (repo_parse_commit(subrepo_p, commit)) \n > >                  die(\"Invalid commit %s in submodule path %s%s\", \n > >                      oid_to_hex(&entry.oid), \n > >                      base->buf, entry.path); \n > > \n > > -            oidcpy(&oid, get_commit_tree_oid(commit)); \n > > +            submodule_tree = repo_get_commit_tree(subrepo_p, commit); \n > > +            oidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL); \n > > + \n > > +            len = tree_entry_len(&entry); \n > > +            strbuf_add(base, entry.path, len); \n > > +            strbuf_addch(base, '/'); \n > > +            retval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid), \n > > +                        base, pathspec, \n > > +                        fn, context); \n > > +            if (retval) { \n > > +                die(\"failed to read tree for %s%s\", base->buf, entry.path); \n > > +                return -1; \n > > +            } \n > > +            strbuf_setlen(base, oldlen); \n > > +            repo_clear(subrepo_p); \n > >          } \n > > -        else \n > > -            continue; \n > > \n > > -        len = tree_entry_len(&entry); \n > > -        strbuf_add(base, entry.path, len); \n > > -        strbuf_addch(base, '/'); \n > > -        retval = read_tree_at(r, lookup_tree(r, &oid), \n > > -                      base, pathspec, \n > > -                      fn, context); \n > > -        strbuf_setlen(base, oldlen); \n > > -        if (retval) \n > > -            return -1; \n > >      } \n > >      return 0; \n > >  } \n > > @@ -121,7 +143,7 @@ int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size) \n > >      return 0; \n > >  } \n > > \n > > -int parse_tree_gently(struct tree *item, int quiet_on_missing) \n > > +int parse_tree_gently(struct repository *r, struct tree *item, int quiet_on_missing) \n > >  { \n > >       enum object_type type; \n > >       void *buffer; \n > > @@ -129,7 +151,7 @@ int parse_tree_gently(struct tree *item, int quiet_on_missing) \n > > \n > >      if (item->object.parsed) \n > >          return 0; \n > > -    buffer = read_object_file(&item->object.oid, &type, &size); \n > > +    buffer = repo_read_object_file(r, &item->object.oid, &type, &size); \n > >      if (!buffer) \n > >          return quiet_on_missing ? -1 : \n > >              error(\"Could not read %s\", \n >  \n > \n"},{"id":"464897","messageId":"183d33ad4dc.b3e5b5ca123845.2982022328880074553@alphaservcomputing.solutions","threadId":"58613","inReplyTo":"97a82675-22bb-b996-deac-3f13a91f3df4@web.de","subject":"Re: [PATCH v2 0/2] archive: Add --recurse-submodules to git-archive command","fromName":"Heather Lapointe","fromEmail":"alpha@alphaservcomputing.solutions","sentAt":"2022-10-13T21:23:11Z","receivedAt":"2022-10-13T21:38:24Z","isPatch":true,"sender":{"key":"alpha@alphaservcomputing.solutions","avatar":null},"body":"---- On Thu, 13 Oct 2022 10:53:44 -0700 René Scharfe  wrote ---\n\n > > I am open to feedback since this is all quite new to me :) \n > > \n > > TODO: \n >  \n > This list confuses me: \n\nI apologize. I'm new to this repo and workflow.\nI had been using checkboxes in github, which look like `- [x]` for ones that I have completed.\nThey all got converted into items that look like they needed doing via GitGitGadget.\n\nThe only remaining one was to update documentation.\n\n >  \n > > \n > > Alphadelta14 (2): \n > >   archive: add --recurse-submodules to git-archive command \n > >   archive: fix a case of submodule in submodule traversal \n >  \n > We prefer to keep known bugs out of the repo.  It helps when bisecting, \n > for example.  So it would be better to squash the fix into the patch \n > that adds the feature.  But... \n\nAbsolutely can do.\n\n >  \n > >  archive-tar.c                 | 14 +++-- \n > >  archive-zip.c                 | 14 ++--- \n > >  archive.c                     | 99 ++++++++++++++++++++++++----------- \n > >  archive.h                     |  8 +-- \n > >  builtin/checkout.c            |  2 +- \n > >  builtin/log.c                 |  2 +- \n > >  builtin/ls-files.c            | 10 ++-- \n > >  builtin/ls-tree.c             | 16 +++--- \n > >  list-objects.c                |  2 +- \n > >  merge-recursive.c             |  2 +- \n > >  revision.c                    |  4 +- \n > >  sparse-index.c                |  2 +- \n > >  t/t5005-archive-submodules.sh | 84 +++++++++++++++++++++++++++++ \n > >  tree.c                        | 93 ++++++++++++++++++++++---------- \n > >  tree.h                        | 11 ++-- \n > >  wt-status.c                   |  2 +- \n > >  16 files changed, 269 insertions(+), 96 deletions(-) \n > >  create mode 100755 t/t5005-archive-submodules.sh \n >  \n > ... this is all a bit much for a single patch, I feel.  Giving \n > parse_tree_gently() a repo parameter, adding repo_parse_tree(), using \n > it in read_tree_at(), adding a repo parameter to read_tree_fn_t, \n > letting read_tree_at() recurse into submodules and adding the new \n > option to git archive all seem like topics worth their own patch and \n > rationale. \n > You probably have all of that in your head right now, but at least my \n > attention span and working memory capacity requires smaller morsels. \n \nDoes this mean I should create multiple PRs?\nOr should they just be split up into individual commits.\nI will work off assuming the latter.\n\nI am comfortable rewriting history as long as I understand the direction to go in.\n\nShould each individual patch be completely standalone?\n(To the point where each set, with the previous patches should produce a working application?\nOr is having the patch broken up by groups of changes, with some level of expecting the final\nfunctionality good?)\n\nBut thanks so far. I will get working on this (and review your next set of messages).\n\n"},{"id":"464931","messageId":"b0d36edc-cd9a-e4a1-3bc1-d35654f4843b@web.de","threadId":"58613","inReplyTo":"183d33ad4dc.b3e5b5ca123845.2982022328880074553@alphaservcomputing.solutions","subject":"Re: [PATCH v2 0/2] archive: Add --recurse-submodules to git-archive command","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2022-10-14T09:47:52Z","receivedAt":"2022-10-14T09:48:05Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 13.10.22 um 23:23 schrieb Heather Lapointe:\n> I had been using checkboxes in github, which look like `- [x]` for\n> ones that I have completed. They all got converted into items that\n> look like they needed doing via GitGitGadget.\n>\n> The only remaining one was to update documentation.\n\nI opened https://github.com/html-to-text/node-html-to-text/issues/260 to\nsee if they are willing to support this kind of to-do lists better.\n\n> Does this mean I should create multiple PRs? Or should they just be\n> split up into individual commits. I will work off assuming the\n> latter.\n\nRight, I meant multiple commits.\n\n> Should each individual patch be completely standalone? (To the point\n> where each set, with the previous patches should produce a working\n> application? Or is having the patch broken up by groups of changes,\n> with some level of expecting the final functionality good?)\n\nYes, each commit should leave Git in a working state.  Adding new global\nfunctions that are only used by later commits in the series is fine.\n\nRené\n"},{"id":"465056","messageId":"79959a54eb4c1a0812b1f4643530069a63e549f4.1665973401.git.gitgitgadget@gmail.com","threadId":"58613","inReplyTo":"pull.1359.v3.git.git.1665973401.gitgitgadget@gmail.com","subject":"[PATCH v3 1/9] tree: do not use the_repository for tree traversal methods.","fromName":"Alphadelta14 via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-17T02:23:13Z","receivedAt":"2022-10-17T02:23:34Z","isPatch":true,"sender":{"key":"name:Alphadelta14","avatar":null},"body":"From: Alphadelta14 <alpha@alphaservcomputing.solutions>\n\nExpect that tree walking may switch repository contexts for cases\nsuch as submodules.\nAdded compatibility macros for existing cases.\n\nAnnotate an existing issue where repo is wrong when traversing.\n\nSigned-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n---\n tree.c | 15 +++++++++------\n tree.h | 14 ++++++++++----\n 2 files changed, 19 insertions(+), 10 deletions(-)\n\ndiff --git a/tree.c b/tree.c\nindex 410e3b477e5..13f9173d45e 100644\n--- a/tree.c\n+++ b/tree.c\n@@ -22,7 +22,7 @@ int read_tree_at(struct repository *r,\n \tint len, oldlen = base->len;\n \tenum interesting retval = entry_not_interesting;\n \n-\tif (parse_tree(tree))\n+\tif (repo_parse_tree(r, tree))\n \t\treturn -1;\n \n \tinit_tree_desc(&desc, tree->buffer, tree->size);\n@@ -58,7 +58,11 @@ int read_tree_at(struct repository *r,\n \t\t\t\t    oid_to_hex(&entry.oid),\n \t\t\t\t    base->buf, entry.path);\n \n-\t\t\tif (parse_commit(commit))\n+\t\t\t// FIXME: This is the wrong repo instance (it refers to the superproject)\n+\t\t\t// it will always fail as is (will fix in later patch)\n+\t\t\t// This current codepath isn't executed by any existing callbacks\n+\t\t\t// so it wouldn't show up as an issue at this time.\n+\t\t\tif (repo_parse_commit(r, commit))\n \t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n \t\t\t\t    oid_to_hex(&entry.oid),\n \t\t\t\t    base->buf, entry.path);\n@@ -121,7 +125,7 @@ int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size)\n \treturn 0;\n }\n \n-int parse_tree_gently(struct tree *item, int quiet_on_missing)\n+int repo_parse_tree_gently(struct repository *r, struct tree *item, int quiet_on_missing)\n {\n \t enum object_type type;\n \t void *buffer;\n@@ -129,7 +133,7 @@ int parse_tree_gently(struct tree *item, int quiet_on_missing)\n \n \tif (item->object.parsed)\n \t\treturn 0;\n-\tbuffer = read_object_file(&item->object.oid, &type, &size);\n+\tbuffer = repo_read_object_file(r, &item->object.oid, &type, &size);\n \tif (!buffer)\n \t\treturn quiet_on_missing ? -1 :\n \t\t\terror(\"Could not read %s\",\n@@ -149,9 +153,8 @@ void free_tree_buffer(struct tree *tree)\n \ttree->object.parsed = 0;\n }\n \n-struct tree *parse_tree_indirect(const struct object_id *oid)\n+struct tree *repo_parse_tree_indirect(struct repository *r, const struct object_id *oid)\n {\n-\tstruct repository *r = the_repository;\n \tstruct object *obj = parse_object(r, oid);\n \treturn (struct tree *)repo_peel_to_type(r, NULL, 0, obj, OBJ_TREE);\n }\ndiff --git a/tree.h b/tree.h\nindex 6efff003e21..cc6402e4738 100644\n--- a/tree.h\n+++ b/tree.h\n@@ -18,15 +18,21 @@ struct tree *lookup_tree(struct repository *r, const struct object_id *oid);\n \n int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size);\n \n-int parse_tree_gently(struct tree *tree, int quiet_on_missing);\n-static inline int parse_tree(struct tree *tree)\n+int repo_parse_tree_gently(struct repository *r, struct tree *tree, int quiet_on_missing);\n+static inline int repo_parse_tree(struct repository *r, struct tree *tree)\n {\n-\treturn parse_tree_gently(tree, 0);\n+\treturn repo_parse_tree_gently(r, tree, 0);\n }\n+\n+#ifndef NO_THE_REPOSITORY_COMPATIBILITY_MACROS\n+#define parse_tree(tree) repo_parse_tree(the_repository, tree)\n+#define parse_tree_gently(tree, quiet_on_missing) repo_parse_tree_gently(the_repository, tree, quiet_on_missing)\n+#define parse_tree_indirect(oid) repo_parse_tree_indirect(the_repository, oid)\n+#endif\n void free_tree_buffer(struct tree *tree);\n \n /* Parses and returns the tree in the given ent, chasing tags and commits. */\n-struct tree *parse_tree_indirect(const struct object_id *oid);\n+struct tree *repo_parse_tree_indirect(struct repository *r, const struct object_id *oid);\n \n int cmp_cache_name_compare(const void *a_, const void *b_);\n \n-- \ngitgitgadget\n\n"},{"id":"465057","messageId":"2291e0f9b5c61f9668b206b85368829db9384bb3.1665973401.git.gitgitgadget@gmail.com","threadId":"58613","inReplyTo":"pull.1359.v3.git.git.1665973401.gitgitgadget@gmail.com","subject":"[PATCH v3 2/9] tree: update cases to use repo_ tree methods","fromName":"Heather Lapointe via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-17T02:23:14Z","receivedAt":"2022-10-17T02:23:34Z","isPatch":true,"sender":{"key":"alpha@alphaservcomputing.solutions","avatar":null},"body":"From: Heather Lapointe <alpha@alphaservcomputing.solutions>\n\nFor cases which had already had a repository instance,\nupdate those to use the repo_parse_tree* methods.\n\nLeave the remaining invocations that were already using the_repository\nalone.\n\nSigned-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n---\n merge.c     | 4 ++--\n reset.c     | 2 +-\n revision.c  | 4 ++--\n sequencer.c | 6 +++---\n 4 files changed, 8 insertions(+), 8 deletions(-)\n\ndiff --git a/merge.c b/merge.c\nindex 2382ff66d35..1efc4440c03 100644\n--- a/merge.c\n+++ b/merge.c\n@@ -63,12 +63,12 @@ int checkout_fast_forward(struct repository *r,\n \tmemset(&trees, 0, sizeof(trees));\n \tmemset(&t, 0, sizeof(t));\n \n-\ttrees[nr_trees] = parse_tree_indirect(head);\n+\ttrees[nr_trees] = repo_parse_tree_indirect(r, head);\n \tif (!trees[nr_trees++]) {\n \t\trollback_lock_file(&lock_file);\n \t\treturn -1;\n \t}\n-\ttrees[nr_trees] = parse_tree_indirect(remote);\n+\ttrees[nr_trees] = repo_parse_tree_indirect(r, remote);\n \tif (!trees[nr_trees++]) {\n \t\trollback_lock_file(&lock_file);\n \t\treturn -1;\ndiff --git a/reset.c b/reset.c\nindex e3383a93343..a0ac5e8a684 100644\n--- a/reset.c\n+++ b/reset.c\n@@ -153,7 +153,7 @@ int reset_head(struct repository *r, const struct reset_head_opts *opts)\n \t\tgoto leave_reset_head;\n \t}\n \n-\ttree = parse_tree_indirect(oid);\n+\ttree = repo_parse_tree_indirect(r, oid);\n \tprime_cache_tree(r, r->index, tree);\n \n \tif (write_locked_index(r->index, &lock, COMMIT_LOCK) < 0) {\ndiff --git a/revision.c b/revision.c\nindex 36e31942cee..dab5ddaf039 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -74,7 +74,7 @@ static void mark_tree_contents_uninteresting(struct repository *r,\n \tstruct tree_desc desc;\n \tstruct name_entry entry;\n \n-\tif (parse_tree_gently(tree, 1) < 0)\n+\tif (repo_parse_tree_gently(r, tree, 1) < 0)\n \t\treturn;\n \n \tinit_tree_desc(&desc, tree->buffer, tree->size);\n@@ -181,7 +181,7 @@ static void add_children_by_path(struct repository *r,\n \tif (!tree)\n \t\treturn;\n \n-\tif (parse_tree_gently(tree, 1) < 0)\n+\tif (repo_parse_tree_gently(r, tree, 1) < 0)\n \t\treturn;\n \n \tinit_tree_desc(&desc, tree->buffer, tree->size);\ndiff --git a/sequencer.c b/sequencer.c\nindex a4d85f1fbdd..a4c09dfa182 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -685,9 +685,9 @@ static int do_recursive_merge(struct repository *r,\n \t\to.buffer_output = 2;\n \to.show_rename_progress = 1;\n \n-\thead_tree = parse_tree_indirect(head);\n-\tnext_tree = next ? get_commit_tree(next) : empty_tree(r);\n-\tbase_tree = base ? get_commit_tree(base) : empty_tree(r);\n+\thead_tree = repo_parse_tree_indirect(r, head);\n+\tnext_tree = next ? repo_get_commit_tree(r, next) : empty_tree(r);\n+\tbase_tree = base ? repo_get_commit_tree(r, base) : empty_tree(r);\n \n \tfor (i = 0; i < opts->xopts_nr; i++)\n \t\tparse_merge_opt(&o, opts->xopts[i]);\n-- \ngitgitgadget\n\n"},{"id":"465058","messageId":"9a07c6932f4c7ef844df1fc4f5b6b9feb1810135.1665973401.git.gitgitgadget@gmail.com","threadId":"58613","inReplyTo":"pull.1359.v3.git.git.1665973401.gitgitgadget@gmail.com","subject":"[PATCH v3 3/9] tree: increase test coverage for tree.c","fromName":"Heather Lapointe via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-17T02:23:15Z","receivedAt":"2022-10-17T02:23:43Z","isPatch":true,"sender":{"key":"alpha@alphaservcomputing.solutions","avatar":null},"body":"From: Heather Lapointe <alpha@alphaservcomputing.solutions>\n\nThis highlights some buggy behavior from read_tree for submodules that\nwas not being executed.\n\nThis introduces a test-tool tree-read-tree-at command\n(the complex name is because it is not related to the read-tree command).\n\nSigned-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n---\n Makefile                          |  1 +\n t/helper/test-tool.c              |  1 +\n t/helper/test-tool.h              |  1 +\n t/helper/test-tree-read-tree-at.c | 40 +++++++++++++++++++\n t/t1023-tree-read-tree-at.sh      | 65 +++++++++++++++++++++++++++++++\n 5 files changed, 108 insertions(+)\n create mode 100644 t/helper/test-tree-read-tree-at.c\n create mode 100755 t/t1023-tree-read-tree-at.sh\n\ndiff --git a/Makefile b/Makefile\nindex 6bfb62cbe94..52d17ca7276 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -788,6 +788,7 @@ TEST_BUILTINS_OBJS += test-submodule-nested-repo-config.o\n TEST_BUILTINS_OBJS += test-submodule.o\n TEST_BUILTINS_OBJS += test-subprocess.o\n TEST_BUILTINS_OBJS += test-trace2.o\n+TEST_BUILTINS_OBJS += test-tree-read-tree-at.o\n TEST_BUILTINS_OBJS += test-urlmatch-normalization.o\n TEST_BUILTINS_OBJS += test-userdiff.o\n TEST_BUILTINS_OBJS += test-wildmatch.o\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex d1d013bcd92..a8a9bedec5f 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -82,6 +82,7 @@ static struct test_cmd cmds[] = {\n \t{ \"submodule-nested-repo-config\", cmd__submodule_nested_repo_config },\n \t{ \"subprocess\", cmd__subprocess },\n \t{ \"trace2\", cmd__trace2 },\n+\t{ \"tree-read-tree-at\", cmd__tree_read_tree_at },\n \t{ \"userdiff\", cmd__userdiff },\n \t{ \"urlmatch-normalization\", cmd__urlmatch_normalization },\n \t{ \"xml-encode\", cmd__xml_encode },\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex 6b46b6444b6..409fddfaeb8 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -76,6 +76,7 @@ int cmd__submodule_config(int argc, const char **argv);\n int cmd__submodule_nested_repo_config(int argc, const char **argv);\n int cmd__subprocess(int argc, const char **argv);\n int cmd__trace2(int argc, const char **argv);\n+int cmd__tree_read_tree_at(int argc, const char **argv);\n int cmd__userdiff(int argc, const char **argv);\n int cmd__urlmatch_normalization(int argc, const char **argv);\n int cmd__xml_encode(int argc, const char **argv);\ndiff --git a/t/helper/test-tree-read-tree-at.c b/t/helper/test-tree-read-tree-at.c\nnew file mode 100644\nindex 00000000000..bba759bb264\n--- /dev/null\n+++ b/t/helper/test-tree-read-tree-at.c\n@@ -0,0 +1,40 @@\n+/* This tests tree.c's read_tree / read_tree_at.\n+We call it tree-read-tree-at to disambiguate with the read-tree tool.\n+*/\n+#include \"cache.h\"\n+#include \"pathspec.h\"\n+#include \"test-tool.h\"\n+#include \"tree.h\"\n+\n+static int test_handle_entry(const struct object_id *oid,\n+\t\tstruct strbuf *base, const char *filename,\n+\t\tunsigned mode, void *context UNUSED) {\n+\tprintf(\"%i %s %s%s\\n\", mode, oid_to_hex(oid), base->buf, filename);\n+\tif (S_ISDIR(mode) || S_ISGITLINK(mode)) {\n+\t\treturn READ_TREE_RECURSIVE;\n+\t}\n+\treturn 0;\n+}\n+\n+int cmd__tree_read_tree_at(int argc UNUSED, const char **argv)\n+{\n+\tstruct pathspec pathspec;\n+\tstruct tree *tree;\n+\tstruct repository *repo;\n+\tstruct object_id oid;\n+\n+\tsetup_git_directory();\n+\trepo = the_repository;\n+\tassert(repo);\n+\n+\tparse_pathspec(&pathspec, 0,\n+\t\t       PATHSPEC_PREFER_FULL,\n+\t\t       \"\", argv);\n+\n+\tassert(repo_get_oid(repo, \"HEAD\", &oid) == 0);\n+\ttree = repo_parse_tree_indirect(repo, &oid);\n+\tassert(tree);\n+\tpathspec.recurse_submodules = 1;\n+\tread_tree(repo, tree, &pathspec, test_handle_entry, NULL);\n+\treturn 0;\n+}\ndiff --git a/t/t1023-tree-read-tree-at.sh b/t/t1023-tree-read-tree-at.sh\nnew file mode 100755\nindex 00000000000..9e5ce3abb4b\n--- /dev/null\n+++ b/t/t1023-tree-read-tree-at.sh\n@@ -0,0 +1,65 @@\n+#!/bin/sh\n+\n+# tests for tree.c (not read-tree.c)\n+test_description='Test read_tree / read_tree_at'\n+. ./test-lib.sh\n+\n+test_expect_success 'read_tree basic' '\n+\trm -rf walk_tree_basic &&\n+\tgit init walk_tree_basic &&\n+\t(\n+\t\tcd walk_tree_basic &&\n+\t\tset -x &&\n+\n+\t\tmkdir -p dir1/dirA &&\n+\t\tmkdir -p dir1/dirB &&\n+\t\tmkdir -p dir2 &&\n+\t\techo \"file1\" > file1.txt &&\n+\t\techo \"file2\" > file2.txt &&\n+\t\t# uncommitted\n+\t\techo \"file3\" > file3.txt &&\n+\n+\t\techo \"file1A1\" > dir1/dirA/file1.txt &&\n+\t\tgit add file1.txt file2.txt dir1/dirA/file1.txt &&\n+\t\tgit commit -m \"initial commit\" &&\n+\n+\t\ttest-tool tree-read-tree-at . > walk1.txt &&\n+\t\tgrep \" file1.txt\" walk1.txt &&\n+\t\t! grep \" file3.txt\" walk1.txt &&\n+\t\t! grep \" dir1/dirB\" walk1.txt &&\n+\t\tgrep \" dir1/dirA/file1.txt\" walk1.txt\n+\t)\n+'\n+\n+test_expect_success 'read_tree submodules' '\n+\trm -rf walk_tree_submodules &&\n+\tgit init submodule1 &&\n+\t(\n+\t\tcd submodule1 &&\n+\t\tmkdir -p dir1/dirA &&\n+\t\techo \"dir2/sub1/file1.txt\" > file1.txt &&\n+\t\techo \"dir2/sub1/file1A1.txt\" > dir1/dirA/file1.txt &&\n+\t\tgit add file1.txt dir1/dirA/file1.txt &&\n+\t\tgit commit -m \"initial commit\"\n+\t) &&\n+\tgit init walk_tree_submodules &&\n+\t(\n+\t\tcd walk_tree_submodules &&\n+\n+\t\tmkdir -p dir2 &&\n+\t\techo \"file1\" > file1.txt &&\n+\t\techo \"dir2/file2\" > dir2/file2.txt &&\n+\t\tgit add file1.txt dir2/file2.txt &&\n+\t\tgit commit -m \"initial commit\" &&\n+\n+\t\tgit submodule add ../submodule1 dir2/sub1 &&\n+\t\tgit commit -m \"add submodule1\" &&\n+\n+\t\ttest-tool tree-read-tree-at . > walk2.txt &&\n+\t\tgrep \" file1.txt\" walk2.txt &&\n+\t\tgrep \" dir2/sub1/file1.txt\" walk2.txt &&\n+\t\tgrep \" dir2/sub1/dir1/dirA/file1.txt\" walk2.txt\n+\t)\n+'\n+\n+test_done\n-- \ngitgitgadget\n\n"},{"id":"465059","messageId":"pull.1359.v3.git.git.1665973401.gitgitgadget@gmail.com","threadId":"58613","inReplyTo":"pull.1359.v2.git.git.1665660960.gitgitgadget@gmail.com","subject":"[PATCH v3 0/9] archive: Add --recurse-submodules to git-archive command","fromName":"Heather Lapointe via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-17T02:23:12Z","receivedAt":"2022-10-17T02:23:43Z","isPatch":true,"sender":{"key":"alpha@alphaservcomputing.solutions","avatar":null},"body":"This makes it possible to include submodule contents in an archive command.\n\nThe inspiration for this change comes from this Github thread,\nhttps://github.com/dear-github/dear-github/issues/214, with at least 160\n👍🏻 's at the time of writing. (I stumbled upon it because I wanted it as\nwell).\n\nI figured the underlying implementation wouldn't be too difficult with most\nof the plumbing already in place, so I decided to add the relevant logic to\nthe client git-archive command.\n\nOne of the trickier parts of this implementation involved teaching read_tree\nabout submodules. Some of the troublesome areas were still using the\nthe_repository references to look up commit or tree or oid information. I\nended up deciding that read_tree_fn_t would probably be best off having a\nconcrete repo reference since it allows changing the context to a subrepo\nwhere needed (even though some of the usages did not need it specifically).\n\nI am open to feedback since this is all quite new to me :)\n\nAlphadelta14 (1):\n  tree: do not use the_repository for tree traversal methods.\n\nHeather Lapointe (8):\n  tree: update cases to use repo_ tree methods\n  tree: increase test coverage for tree.c\n  tree: handle submodule case for read_tree_at properly\n  tree: add repository parameter to read_tree_fn_t\n  archive: pass repo objects to write_archive handlers\n  archive: remove global repository from archive_args\n  archive: add --recurse-submodules to git-archive command\n  archive: add tests for git archive --recurse-submodules\n\n Documentation/git-archive.txt     |   6 +-\n Makefile                          |   1 +\n archive-tar.c                     |  15 ++--\n archive-zip.c                     |  15 ++--\n archive.c                         | 138 ++++++++++++++++++++----------\n archive.h                         |  16 +++-\n builtin/checkout.c                |   4 +-\n builtin/log.c                     |   4 +-\n builtin/ls-files.c                |   8 +-\n builtin/ls-tree.c                 |  34 +++++---\n merge-recursive.c                 |   4 +-\n merge.c                           |   4 +-\n reset.c                           |   2 +-\n revision.c                        |   4 +-\n sequencer.c                       |   6 +-\n sparse-index.c                    |   4 +-\n t/helper/test-tool.c              |   1 +\n t/helper/test-tool.h              |   1 +\n t/helper/test-tree-read-tree-at.c |  41 +++++++++\n t/t1023-tree-read-tree-at.sh      |  65 ++++++++++++++\n t/t5005-archive-submodules.sh     |  83 ++++++++++++++++++\n tree.c                            |  93 ++++++++++++++------\n tree.h                            |  16 ++--\n wt-status.c                       |   4 +-\n 24 files changed, 448 insertions(+), 121 deletions(-)\n create mode 100644 t/helper/test-tree-read-tree-at.c\n create mode 100755 t/t1023-tree-read-tree-at.sh\n create mode 100755 t/t5005-archive-submodules.sh\n\n\nbase-commit: e85701b4af5b7c2a9f3a1b07858703318dce365d\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1359%2FAlphadelta14%2Farchive-recurse-submodules-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1359/Alphadelta14/archive-recurse-submodules-v3\nPull-Request: https://github.com/git/git/pull/1359\n\nRange-diff vs v2:\n\n  -:  ----------- >  1:  79959a54eb4 tree: do not use the_repository for tree traversal methods.\n  -:  ----------- >  2:  2291e0f9b5c tree: update cases to use repo_ tree methods\n  -:  ----------- >  3:  9a07c6932f4 tree: increase test coverage for tree.c\n  2:  68f7830c6d9 !  4:  d3d1738e670 archive: fix a case of submodule in submodule traversal\n     @@\n       ## Metadata ##\n     -Author: Alphadelta14 <alpha@alphaservcomputing.solutions>\n     +Author: Heather Lapointe <alpha@alphaservcomputing.solutions>\n      \n       ## Commit message ##\n     -    archive: fix a case of submodule in submodule traversal\n     +    tree: handle submodule case for read_tree_at properly\n      \n     -    repo_submodule_init actually expects the path relative to submodule_prefix.\n     -    We preform a simple strip to the correct path.\n     +    This supports traversal into an actual submodule for read_tree_at.\n     +    The logic is blocked on pathspec->recurse_submodules now,\n     +    but previously hadn't been executed due to all fn() cases\n     +    returning early for submodules.\n      \n     -    Signed-off-by: Alphadelta14 <alpha@alphaservcomputing.solutions>\n     +    Signed-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n      \n       ## tree.c ##\n     +@@\n     + #include \"alloc.h\"\n     + #include \"tree-walk.h\"\n     + #include \"repository.h\"\n     ++#include \"pathspec.h\"\n     + \n     + const char *tree_type = \"tree\";\n     + \n      @@ tree.c: int read_tree_at(struct repository *r,\n     - \t\t\tstruct repository subrepo;\n     - \t\t\tstruct repository* subrepo_p = &subrepo;\n     - \t\t\tstruct tree* submodule_tree;\n     + \t\t\treturn -1;\n     + \t\t}\n     + \n     +-\t\tif (S_ISDIR(entry.mode))\n     ++\t\tif (S_ISDIR(entry.mode)) {\n     + \t\t\toidcpy(&oid, &entry.oid);\n     +-\t\telse if (S_ISGITLINK(entry.mode)) {\n     +-\t\t\tstruct commit *commit;\n     + \n     +-\t\t\tcommit = lookup_commit(r, &entry.oid);\n     ++\t\t\tlen = tree_entry_len(&entry);\n     ++\t\t\tstrbuf_add(base, entry.path, len);\n     ++\t\t\tstrbuf_addch(base, '/');\n     ++\t\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n     ++\t\t\t\t\t\tbase, pathspec,\n     ++\t\t\t\t\t\tfn, context);\n     ++\t\t\tstrbuf_setlen(base, oldlen);\n     ++\t\t\tif (retval)\n     ++\t\t\t\treturn -1;\n     ++\t\t} else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) {\n     ++\t\t\tstruct commit *commit;\n     ++\t\t\tstruct repository subrepo;\n     ++\t\t\tstruct repository* subrepo_p = &subrepo;\n     ++\t\t\tstruct tree* submodule_tree;\n      +\t\t\tchar *submodule_rel_path;\n      +\t\t\tint name_base_len = 0;\n     - \n     --\t\t\tif (repo_submodule_init(subrepo_p, r, entry.path, null_oid()))\n     --\t\t\t\tdie(\"couldn't init submodule %s%s\", base->buf, entry.path);\n     ++\n      +\t\t\tlen = tree_entry_len(&entry);\n      +\t\t\tstrbuf_add(base, entry.path, len);\n      +\t\t\tsubmodule_rel_path = base->buf;\n     @@ tree.c: int read_tree_at(struct repository *r,\n      +\n      +\t\t\tif (repo_submodule_init(subrepo_p, r, submodule_rel_path, null_oid()))\n      +\t\t\t\tdie(\"couldn't init submodule %s\", base->buf);\n     - \n     - \t\t\tif (repo_read_index(subrepo_p) < 0)\n     - \t\t\t\tdie(\"index file corrupt\");\n     - \n     - \t\t\tcommit = lookup_commit(subrepo_p, &entry.oid);\n     ++\n     ++\t\t\tif (repo_read_index(subrepo_p) < 0)\n     ++\t\t\t\tdie(\"index file corrupt\");\n     ++\n     ++\t\t\tcommit = lookup_commit(subrepo_p, &entry.oid);\n       \t\t\tif (!commit)\n      -\t\t\t\tdie(\"Commit %s in submodule path %s%s not found\",\n      +\t\t\t\tdie(\"Commit %s in submodule path %s not found\",\n       \t\t\t\t    oid_to_hex(&entry.oid),\n      -\t\t\t\t    base->buf, entry.path);\n     -+\t\t\t\t    base->buf);\n     - \n     - \t\t\tif (repo_parse_commit(subrepo_p, commit))\n     +-\n     +-\t\t\t// FIXME: This is the wrong repo instance (it refers to the superproject)\n     +-\t\t\t// it will always fail as is (will fix in later patch)\n     +-\t\t\t// This current codepath isn't executed by any existing callbacks\n     +-\t\t\t// so it wouldn't show up as an issue at this time.\n     +-\t\t\tif (repo_parse_commit(r, commit))\n      -\t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n     ++\t\t\t\t    base->buf);\n     ++\n     ++\t\t\tif (repo_parse_commit(subrepo_p, commit))\n      +\t\t\t\tdie(\"Invalid commit %s in submodule path %s\",\n       \t\t\t\t    oid_to_hex(&entry.oid),\n      -\t\t\t\t    base->buf, entry.path);\n      +\t\t\t\t    base->buf);\n       \n     - \t\t\tsubmodule_tree = repo_get_commit_tree(subrepo_p, commit);\n     - \t\t\toidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);\n     +-\t\t\toidcpy(&oid, get_commit_tree_oid(commit));\n     +-\t\t}\n     +-\t\telse\n     +-\t\t\tcontinue;\n     ++\t\t\tsubmodule_tree = repo_get_commit_tree(subrepo_p, commit);\n     ++\t\t\toidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);\n       \n     --\t\t\tlen = tree_entry_len(&entry);\n     --\t\t\tstrbuf_add(base, entry.path, len);\n     - \t\t\tstrbuf_addch(base, '/');\n     +-\t\tlen = tree_entry_len(&entry);\n     +-\t\tstrbuf_add(base, entry.path, len);\n     +-\t\tstrbuf_addch(base, '/');\n     +-\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n     +-\t\t\t\t      base, pathspec,\n     +-\t\t\t\t      fn, context);\n     +-\t\tstrbuf_setlen(base, oldlen);\n     +-\t\tif (retval)\n     +-\t\t\treturn -1;\n     ++\t\t\tstrbuf_addch(base, '/');\n      +\n     - \t\t\tretval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),\n     - \t\t\t\t\t\tbase, pathspec,\n     - \t\t\t\t\t\tfn, context);\n     - \t\t\tif (retval) {\n     --\t\t\t    die(\"failed to read tree for %s%s\", base->buf, entry.path);\n     ++\t\t\tretval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),\n     ++\t\t\t\t\t\tbase, pathspec,\n     ++\t\t\t\t\t\tfn, context);\n     ++\t\t\tif (retval)\n      +\t\t\t    die(\"failed to read tree for %s\", base->buf);\n     - \t\t\t    return -1;\n     - \t\t\t}\n     - \t\t\tstrbuf_setlen(base, oldlen);\n     ++\t\t\tstrbuf_setlen(base, oldlen);\n     ++\t\t\trepo_clear(subrepo_p);\n     ++\t\t}\n     ++\t\t// else, this is a file (or a submodule, but no pathspec->recurse_submodules)\n     + \t}\n     + \treturn 0;\n     + }\n  -:  ----------- >  5:  376345fdf66 tree: add repository parameter to read_tree_fn_t\n  1:  41664a59029 !  6:  1b9b049d64f archive: add --recurse-submodules to git-archive command\n     @@\n       ## Metadata ##\n     -Author: Alphadelta14 <alpha@alphaservcomputing.solutions>\n     +Author: Heather Lapointe <alpha@alphaservcomputing.solutions>\n      \n       ## Commit message ##\n     -    archive: add --recurse-submodules to git-archive command\n     +    archive: pass repo objects to write_archive handlers\n      \n     -    This makes it possible to include submodule contents in an archive command.\n     -\n     -    This required updating the general read_tree callbacks to support sub-repos\n     -    by not using the_repository global references where possible.\n     -\n     -    archive: update streaming to use target repo\n     -    archive: add test cases for git archive --recurse-submodules\n     +    Use contextual repos instead of the_repository or args->repo\n     +    to ensure that submodules will be handled correctly\n     +    since they use multiple repo instances.\n      \n          Signed-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n      \n     @@ archive-tar.c: static unsigned long offset;\n       static int tar_umask = 002;\n       \n       static int write_tar_filter_archive(const struct archiver *ar,\n     -+\t\t\t\t    struct repository *repo,\n     ++\t\t\t\t\tstruct repository *repo,\n       \t\t\t\t    struct archiver_args *args);\n       \n       /*\n     @@ archive-tar.c: static void write_extended_header(struct archiver_args *args,\n       }\n       \n      -static int write_tar_entry(struct archiver_args *args,\n     -+static int write_tar_entry(struct repository *repo,\n     ++static int write_tar_entry(\n     ++\t\t\t   struct repository *repo,\n      +\t\t\t   struct archiver_args *args,\n       \t\t\t   const struct object_id *oid,\n       \t\t\t   const char *path, size_t pathlen,\n     @@ archive-tar.c: static void tgz_write_block(const void *data)\n       static const char internal_gzip_command[] = \"git archive gzip\";\n       \n       static int write_tar_filter_archive(const struct archiver *ar,\n     -+\t\t\t\t    struct repository *repo,\n     ++\t\t\t\t\tstruct repository *repo,\n       \t\t\t\t    struct archiver_args *args)\n       {\n       #if ZLIB_VERNUM >= 0x1221\n     @@ archive-zip.c: static int entry_is_binary(struct index_state *istate, const char\n       #define STREAM_BUFFER_SIZE (1024 * 16)\n       \n      -static int write_zip_entry(struct archiver_args *args,\n     -+static int write_zip_entry(struct repository *repo,\n     ++static int write_zip_entry(\n     ++\t\t\t   struct repository *repo,\n      +\t\t\t   struct archiver_args *args,\n       \t\t\t   const struct object_id *oid,\n       \t\t\t   const char *path, size_t pathlen,\n     @@ archive-zip.c: static int archive_zip_config(const char *var, const char *value,\n       }\n       \n       static int write_zip_archive(const struct archiver *ar UNUSED,\n     -+\t\t\t     struct repository *repo,\n     ++\t\t\t\t struct repository *repo,\n       \t\t\t     struct archiver_args *args)\n       {\n       \tint err;\n     @@ archive-zip.c: static int write_zip_archive(const struct archiver *ar UNUSED,\n       \n      \n       ## archive.c ##\n     -@@\n     - #include \"unpack-trees.h\"\n     - #include \"dir.h\"\n     - #include \"quote.h\"\n     -+#include \"submodule.h\"\n     - \n     - static char const * const archive_usage[] = {\n     - \tN_(\"git archive [<options>] <tree-ish> [<path>...]\"),\n     -@@ archive.c: static void format_subst(const struct commit *commit,\n     - }\n     - \n     - static void *object_file_to_archive(const struct archiver_args *args,\n     -+\t\t\t\t    struct repository *repo,\n     - \t\t\t\t    const char *path,\n     - \t\t\t\t    const struct object_id *oid,\n     - \t\t\t\t    unsigned int mode,\n     -@@ archive.c: static void *object_file_to_archive(const struct archiver_args *args,\n     - \t\t\t       (args->tree ? &args->tree->object.oid : NULL), oid);\n     - \n     - \tpath += args->baselen;\n     --\tbuffer = read_object_file(oid, type, sizep);\n     -+\tbuffer = repo_read_object_file(repo, oid, type, sizep);\n     - \tif (buffer && S_ISREG(mode)) {\n     - \t\tstruct strbuf buf = STRBUF_INIT;\n     - \t\tsize_t size = 0;\n     - \n     - \t\tstrbuf_attach(&buf, buffer, *sizep, *sizep + 1);\n     --\t\tconvert_to_working_tree(args->repo->index, path, buf.buf, buf.len, &buf, &meta);\n     -+\t\tconvert_to_working_tree(repo->index, path, buf.buf, buf.len, &buf, &meta);\n     - \t\tif (commit)\n     - \t\t\tformat_subst(commit, buf.buf, buf.len, &buf, args->pretty_ctx);\n     - \t\tbuffer = strbuf_detach(&buf, &size);\n      @@ archive.c: static int check_attr_export_subst(const struct attr_check *check)\n       \treturn check && ATTR_TRUE(check->items[1].value);\n       }\n       \n      -static int write_archive_entry(const struct object_id *oid, const char *base,\n     -+static int write_archive_entry(struct repository *repo, const struct object_id *oid, const char *base,\n     ++static int write_archive_entry(\n     ++\t\tstruct repository *repo,\n     ++\t\tconst struct object_id *oid, const char *base,\n       \t\tint baselen, const char *filename, unsigned mode,\n       \t\tvoid *context)\n       {\n     @@ archive.c: static int write_archive_entry(const struct object_id *oid, const cha\n      +\t\terr = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, 0);\n       \t\tif (err)\n       \t\t\treturn err;\n     --\t\treturn (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);\n     -+\t\treturn READ_TREE_RECURSIVE;\n     - \t}\n     - \n     - \tif (args->verbose)\n     + \t\treturn (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);\n      @@ archive.c: static int write_archive_entry(const struct object_id *oid, const char *base,\n       \n       \t/* Stream it? */\n       \tif (S_ISREG(mode) && !args->convert &&\n      -\t    oid_object_info(args->repo, oid, &size) == OBJ_BLOB &&\n     --\t    size > big_file_threshold)\n     --\t\treturn write_entry(args, oid, path.buf, path.len, mode, NULL, size);\n      +\t    oid_object_info(repo, oid, &size) == OBJ_BLOB &&\n     -+\t    size > big_file_threshold) {\n     -+\t\t\terr = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, size);\n     -+\t\t\tif (err) {\n     -+\t\t\t\tdie(\"Failed to write file %.*s\", (int)path.len, path.buf);\n     -+\t\t\t}\n     -+\t\t\treturn err;\n     -+\t\t}\n     + \t    size > big_file_threshold)\n     +-\t\treturn write_entry(args, oid, path.buf, path.len, mode, NULL, size);\n     ++\t\treturn write_entry(repo, args, oid, path.buf, path.len, mode, NULL, size);\n       \n     --\tbuffer = object_file_to_archive(args, path.buf, oid, mode, &type, &size);\n     -+\tbuffer = object_file_to_archive(args, repo, path.buf, oid, mode, &type, &size);\n     + \tbuffer = object_file_to_archive(args, path.buf, oid, mode, &type, &size);\n       \tif (!buffer)\n       \t\treturn error(_(\"cannot read '%s'\"), oid_to_hex(oid));\n      -\terr = write_entry(args, oid, path.buf, path.len, mode, buffer, size);\n     @@ archive.c: static void queue_directory(const struct object_id *oid,\n       }\n       \n      -static int write_directory(struct archiver_context *c)\n     -+static void queue_submodule(struct repository *superproject,\n     -+\t\tconst struct object_id *oid,\n     -+\t\tstruct strbuf *base, const char *filename,\n     -+\t\tunsigned mode, struct archiver_context *c)\n     -+{\n     -+\tstruct repository subrepo;\n     -+\n     -+\tif (repo_submodule_init(&subrepo, superproject, filename, null_oid()))\n     -+\t\treturn;\n     -+\n     -+\tif (repo_read_index(&subrepo) < 0)\n     -+\t\tdie(\"index file corrupt\");\n     -+\n     -+    queue_directory(oid, base, filename, mode, c);\n     -+\n     -+\trepo_clear(&subrepo);\n     -+}\n     -+\n     -+static int write_directory(struct repository *repo, struct archiver_context *c)\n     ++static int write_directory(\n     ++\t\tstruct repository *repo,\n     ++\t\tstruct archiver_context *c)\n       {\n       \tstruct directory *d = c->bottom;\n       \tint ret;\n     @@ archive.c: static int write_directory(struct archiver_context *c)\n      +\t\twrite_directory(repo, c) ||\n      +\t\twrite_archive_entry(repo, &d->oid, d->path, d->baselen,\n       \t\t\t\t    d->path + d->baselen, d->mode,\n     --\t\t\t\t    c) != READ_TREE_RECURSIVE;\n     -+\t\t\t\t    c);\n     + \t\t\t\t    c) != READ_TREE_RECURSIVE;\n       \tfree(d);\n     --\treturn ret ? -1 : 0;\n     -+\tif (ret == READ_TREE_RECURSIVE)\n     -+\t\treturn 0;\n     -+\treturn ret;\n     - }\n     - \n     --static int queue_or_write_archive_entry(const struct object_id *oid,\n     -+static int queue_or_write_archive_entry(\n     -+\t\tstruct repository *repo, const struct object_id *oid,\n     - \t\tstruct strbuf *base, const char *filename,\n     - \t\tunsigned mode, void *context)\n     - {\n     -@@ archive.c: static int queue_or_write_archive_entry(const struct object_id *oid,\n     - \t\t/* Borrow base, but restore its original value when done. */\n     - \t\tstrbuf_addstr(base, filename);\n     - \t\tstrbuf_addch(base, '/');\n     --\t\tcheck = get_archive_attrs(c->args->repo->index, base->buf);\n     -+\t\tcheck = get_archive_attrs(repo->index, base->buf);\n     - \t\tstrbuf_setlen(base, baselen);\n     - \n     - \t\tif (check_attr_export_ignore(check))\n     - \t\t\treturn 0;\n     - \t\tqueue_directory(oid, base, filename, mode, c);\n     +@@ archive.c: static int queue_or_write_archive_entry(\n       \t\treturn READ_TREE_RECURSIVE;\n     -+\t} else if (c->args->recurse_submodules && S_ISGITLINK(mode)) {\n     -+\t\tif (is_submodule_active(repo, filename)) {\n     -+\t\t\tqueue_submodule(repo, oid, base, filename, mode, c);\n     -+\t\t\treturn READ_TREE_RECURSIVE;\n     -+\t\t}\n       \t}\n       \n      -\tif (write_directory(c))\n     -+\tif (write_directory(repo, c))\n     ++\tif (write_directory(r, c))\n       \t\treturn -1;\n      -\treturn write_archive_entry(oid, base->buf, base->len, filename, mode,\n     -+\treturn write_archive_entry(repo, oid, base->buf, base->len, filename, mode,\n     ++\treturn write_archive_entry(r, oid, base->buf, base->len, filename, mode,\n       \t\t\t\t   context);\n       }\n       \n     @@ archive.c: struct extra_file_info {\n       };\n       \n      -int write_archive_entries(struct archiver_args *args,\n     -+int write_archive_entries(struct repository *repo,\n     ++int write_archive_entries(\n     ++\t\tstruct repository *repo,\n      +\t\tstruct archiver_args *args,\n       \t\twrite_archive_entry_fn_t write_entry)\n       {\n     @@ archive.c: int write_archive_entries(struct archiver_args *args,\n       \t\t\t\t  len, 040777, NULL, 0);\n       \t\tif (err)\n       \t\t\treturn err;\n     -@@ archive.c: int write_archive_entries(struct archiver_args *args,\n     - \t\tmemset(&opts, 0, sizeof(opts));\n     - \t\topts.index_only = 1;\n     - \t\topts.head_idx = -1;\n     --\t\topts.src_index = args->repo->index;\n     --\t\topts.dst_index = args->repo->index;\n     -+\t\topts.src_index = repo->index;\n     -+\t\topts.dst_index = repo->index;\n     - \t\topts.fn = oneway_merge;\n     - \t\tinit_tree_desc(&t, args->tree->buffer, args->tree->size);\n     - \t\tif (unpack_trees(1, &t, &opts))\n     -@@ archive.c: int write_archive_entries(struct archiver_args *args,\n     - \t\tgit_attr_set_direction(GIT_ATTR_INDEX);\n     - \t}\n     - \n     --\terr = read_tree(args->repo, args->tree,\n     -+\terr = read_tree(repo, args->tree,\n     - \t\t\t&args->pathspec,\n     - \t\t\tqueue_or_write_archive_entry,\n     - \t\t\t&context);\n      @@ archive.c: int write_archive_entries(struct archiver_args *args,\n       \t\t\tif (strbuf_read_file(&content, path, info->stat.st_size) < 0)\n       \t\t\t\terr = error_errno(_(\"cannot read '%s'\"), path);\n     @@ archive.c: int write_archive_entries(struct archiver_args *args,\n       \t\t\t\t\t  path, strlen(path),\n       \t\t\t\t\t  canon_mode(info->stat.st_mode),\n       \t\t\t\t\t  info->content, info->stat.st_size);\n     -@@ archive.c: struct path_exists_context {\n     - \tstruct archiver_args *args;\n     - };\n     - \n     --static int reject_entry(const struct object_id *oid UNUSED,\n     -+static int reject_entry(struct repository *repo, const struct object_id *oid UNUSED,\n     - \t\t\tstruct strbuf *base,\n     - \t\t\tconst char *filename, unsigned mode,\n     - \t\t\tvoid *context)\n     -@@ archive.c: static int reject_entry(const struct object_id *oid UNUSED,\n     - \t\tstruct strbuf sb = STRBUF_INIT;\n     - \t\tstrbuf_addbuf(&sb, base);\n     - \t\tstrbuf_addstr(&sb, filename);\n     --\t\tif (!match_pathspec(ctx->args->repo->index,\n     -+\t\tif (!match_pathspec(repo->index,\n     - \t\t\t\t    &ctx->pathspec,\n     - \t\t\t\t    sb.buf, sb.len, 0, NULL, 1))\n     - \t\t\tret = READ_TREE_RECURSIVE;\n     -@@ archive.c: static void parse_pathspec_arg(const char **pathspec,\n     - \t\t       PATHSPEC_PREFER_FULL,\n     - \t\t       \"\", pathspec);\n     - \tar_args->pathspec.recursive = 1;\n     -+\tar_args->pathspec.recurse_submodules = ar_args->recurse_submodules;\n     - \tif (pathspec) {\n     - \t\twhile (*pathspec) {\n     - \t\t\tif (**pathspec && !path_exists(ar_args, *pathspec))\n     -@@ archive.c: static int parse_archive_args(int argc, const char **argv,\n     - \tint verbose = 0;\n     - \tint i;\n     - \tint list = 0;\n     -+\tint recurse_submodules = 0;\n     - \tint worktree_attributes = 0;\n     - \tstruct option opts[] = {\n     - \t\tOPT_GROUP(\"\"),\n     -@@ archive.c: static int parse_archive_args(int argc, const char **argv,\n     - \t\t  add_file_cb, (intptr_t)&base },\n     - \t\tOPT_STRING('o', \"output\", &output, N_(\"file\"),\n     - \t\t\tN_(\"write the archive to this file\")),\n     -+\t\tOPT_BOOL(0, \"recurse-submodules\", &recurse_submodules,\n     -+\t\t\tN_(\"include submodules in archive\")),\n     - \t\tOPT_BOOL(0, \"worktree-attributes\", &worktree_attributes,\n     - \t\t\tN_(\"read .gitattributes in working directory\")),\n     - \t\tOPT__VERBOSE(&verbose, N_(\"report archived files on stderr\")),\n     -@@ archive.c: static int parse_archive_args(int argc, const char **argv,\n     - \targs->base = base;\n     - \targs->baselen = strlen(base);\n     - \targs->worktree_attributes = worktree_attributes;\n     -+\targs->recurse_submodules = recurse_submodules;\n     - \n     - \treturn argc;\n     - }\n      @@ archive.c: int write_archive(int argc, const char **argv, const char *prefix,\n       \tparse_treeish_arg(argv, &args, prefix, remote);\n       \tparse_pathspec_arg(argv + 1, &args);\n     @@ archive.c: int write_archive(int argc, const char **argv, const char *prefix,\n       \tfree(args.refname);\n      \n       ## archive.h ##\n     -@@ archive.h: struct archiver_args {\n     - \ttimestamp_t time;\n     - \tstruct pathspec pathspec;\n     - \tunsigned int verbose : 1;\n     -+\tunsigned int recurse_submodules : 1;\n     - \tunsigned int worktree_attributes : 1;\n     - \tunsigned int convert : 1;\n     - \tint compression_level;\n      @@ archive.h: const char *archive_format_from_filename(const char *filename);\n       #define ARCHIVER_HIGH_COMPRESSION_LEVELS 4\n       struct archiver {\n       \tconst char *name;\n      -\tint (*write_archive)(const struct archiver *, struct archiver_args *);\n     -+\tint (*write_archive)(const struct archiver *, struct repository *repo, struct archiver_args *);\n     ++\tint (*write_archive)(\n     ++\t\tconst struct archiver *,\n     ++\t\tstruct repository *,\n     ++\t\tstruct archiver_args *);\n       \tunsigned flags;\n       \tchar *filter_command;\n       };\n     @@ archive.h: void init_tar_archiver(void);\n       void init_archivers(void);\n       \n      -typedef int (*write_archive_entry_fn_t)(struct archiver_args *args,\n     -+typedef int (*write_archive_entry_fn_t)(struct repository *repo,\n     ++typedef int (*write_archive_entry_fn_t)(\n     ++\t\t\t\t\tstruct repository *repo,\n      +\t\t\t\t\tstruct archiver_args *args,\n       \t\t\t\t\tconst struct object_id *oid,\n       \t\t\t\t\tconst char *path, size_t pathlen,\n     @@ archive.h: void init_tar_archiver(void);\n       \t\t\t\t\tvoid *buffer, unsigned long size);\n       \n      -int write_archive_entries(struct archiver_args *args, write_archive_entry_fn_t write_entry);\n     -+int write_archive_entries(struct repository *repo, struct archiver_args *args, write_archive_entry_fn_t write_entry);\n     ++int write_archive_entries(\n     ++\tstruct repository *repo,\n     ++\tstruct archiver_args *args,\n     ++\twrite_archive_entry_fn_t write_entry);\n       \n       #endif\t/* ARCHIVE_H */\n     -\n     - ## builtin/checkout.c ##\n     -@@ builtin/checkout.c: static int post_checkout_hook(struct commit *old_commit, struct commit *new_comm\n     - \n     - }\n     - \n     --static int update_some(const struct object_id *oid, struct strbuf *base,\n     -+static int update_some(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n     - \t\t       const char *pathname, unsigned mode, void *context UNUSED)\n     - {\n     - \tint len;\n     -\n     - ## builtin/log.c ##\n     -@@ builtin/log.c: static int show_tag_object(const struct object_id *oid, struct rev_info *rev)\n     - \treturn 0;\n     - }\n     - \n     --static int show_tree_object(const struct object_id *oid UNUSED,\n     -+static int show_tree_object(struct repository *repo UNUSED, const struct object_id *oid UNUSED,\n     - \t\t\t    struct strbuf *base UNUSED,\n     - \t\t\t    const char *pathname, unsigned mode,\n     - \t\t\t    void *context)\n     -\n     - ## builtin/ls-files.c ##\n     -@@ builtin/ls-files.c: static int get_common_prefix_len(const char *common_prefix)\n     - \treturn common_prefix_len;\n     - }\n     - \n     --static int read_one_entry_opt(struct index_state *istate,\n     -+static int read_one_entry_opt(struct repository *repo UNUSED, struct index_state *istate,\n     - \t\t\t      const struct object_id *oid,\n     - \t\t\t      struct strbuf *base,\n     - \t\t\t      const char *pathname,\n     -@@ builtin/ls-files.c: static int read_one_entry_opt(struct index_state *istate,\n     - \treturn add_index_entry(istate, ce, opt);\n     - }\n     - \n     --static int read_one_entry(const struct object_id *oid, struct strbuf *base,\n     -+static int read_one_entry(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n     - \t\t\t  const char *pathname, unsigned mode,\n     - \t\t\t  void *context)\n     - {\n     - \tstruct index_state *istate = context;\n     --\treturn read_one_entry_opt(istate, oid, base, pathname,\n     -+\treturn read_one_entry_opt(repo, istate, oid, base, pathname,\n     - \t\t\t\t  mode,\n     - \t\t\t\t  ADD_CACHE_OK_TO_ADD|ADD_CACHE_SKIP_DFCHECK);\n     - }\n     -@@ builtin/ls-files.c: static int read_one_entry(const struct object_id *oid, struct strbuf *base,\n     -  * This is used when the caller knows there is no existing entries at\n     -  * the stage that will conflict with the entry being added.\n     -  */\n     --static int read_one_entry_quick(const struct object_id *oid, struct strbuf *base,\n     -+static int read_one_entry_quick(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n     - \t\t\t\tconst char *pathname, unsigned mode,\n     - \t\t\t\tvoid *context)\n     - {\n     - \tstruct index_state *istate = context;\n     --\treturn read_one_entry_opt(istate, oid, base, pathname,\n     -+\treturn read_one_entry_opt(repo, istate, oid, base, pathname,\n     - \t\t\t\t  mode, ADD_CACHE_JUST_APPEND);\n     - }\n     - \n     -\n     - ## builtin/ls-tree.c ##\n     -@@ builtin/ls-tree.c: static int show_recursive(const char *base, size_t baselen, const char *pathname\n     - \treturn 0;\n     - }\n     - \n     --static int show_tree_fmt(const struct object_id *oid, struct strbuf *base,\n     -+static int show_tree_fmt(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n     - \t\t\t const char *pathname, unsigned mode, void *context UNUSED)\n     - {\n     - \tsize_t baselen;\n     -@@ builtin/ls-tree.c: static void show_tree_common_default_long(struct strbuf *base,\n     - \tstrbuf_setlen(base, baselen);\n     - }\n     - \n     --static int show_tree_default(const struct object_id *oid, struct strbuf *base,\n     -+static int show_tree_default(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n     - \t\t\t     const char *pathname, unsigned mode,\n     - \t\t\t     void *context UNUSED)\n     - {\n     -@@ builtin/ls-tree.c: static int show_tree_default(const struct object_id *oid, struct strbuf *base,\n     - \treturn recurse;\n     - }\n     - \n     --static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n     -+static int show_tree_long(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n     - \t\t\t  const char *pathname, unsigned mode,\n     - \t\t\t  void *context UNUSED)\n     - {\n     -@@ builtin/ls-tree.c: static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n     - \n     - \tif (data.type == OBJ_BLOB) {\n     - \t\tunsigned long size;\n     --\t\tif (oid_object_info(the_repository, data.oid, &size) == OBJ_BAD)\n     -+\t\tif (oid_object_info(repo, data.oid, &size) == OBJ_BAD)\n     - \t\t\txsnprintf(size_text, sizeof(size_text), \"BAD\");\n     - \t\telse\n     - \t\t\txsnprintf(size_text, sizeof(size_text),\n     -@@ builtin/ls-tree.c: static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n     - \t}\n     - \n     - \tprintf(\"%06o %s %s %7s\\t\", data.mode, type_name(data.type),\n     --\t       find_unique_abbrev(data.oid, abbrev), size_text);\n     -+\t       repo_find_unique_abbrev(repo, data.oid, abbrev), size_text);\n     - \tshow_tree_common_default_long(base, pathname, data.base->len);\n     - \treturn recurse;\n     - }\n     - \n     --static int show_tree_name_only(const struct object_id *oid, struct strbuf *base,\n     -+static int show_tree_name_only(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n     - \t\t\t       const char *pathname, unsigned mode,\n     - \t\t\t       void *context UNUSED)\n     - {\n     -@@ builtin/ls-tree.c: static int show_tree_name_only(const struct object_id *oid, struct strbuf *base,\n     - \treturn recurse;\n     - }\n     - \n     --static int show_tree_object(const struct object_id *oid, struct strbuf *base,\n     -+static int show_tree_object(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n     - \t\t\t    const char *pathname, unsigned mode,\n     - \t\t\t    void *context UNUSED)\n     - {\n     -@@ builtin/ls-tree.c: static int show_tree_object(const struct object_id *oid, struct strbuf *base,\n     - \tif (early >= 0)\n     - \t\treturn early;\n     - \n     --\tprintf(\"%s%c\", find_unique_abbrev(oid, abbrev), line_termination);\n     -+\tprintf(\"%s%c\", repo_find_unique_abbrev(repo, oid, abbrev), line_termination);\n     - \treturn recurse;\n     - }\n     - \n     -\n     - ## list-objects.c ##\n     -@@ list-objects.c: static void process_tree(struct traversal_context *ctx,\n     - \t    !revs->include_check_obj(&tree->object, revs->include_check_data))\n     - \t\treturn;\n     - \n     --\tfailed_parse = parse_tree_gently(tree, 1);\n     -+\tfailed_parse = parse_tree_gently(revs->repo, tree, 1);\n     - \tif (failed_parse) {\n     - \t\tif (revs->ignore_missing_links)\n     - \t\t\treturn;\n     -\n     - ## merge-recursive.c ##\n     -@@ merge-recursive.c: static void unpack_trees_finish(struct merge_options *opt)\n     - \tclear_unpack_trees_porcelain(&opt->priv->unpack_opts);\n     - }\n     - \n     --static int save_files_dirs(const struct object_id *oid UNUSED,\n     -+static int save_files_dirs(struct repository *repo UNUSED, const struct object_id *oid UNUSED,\n     - \t\t\t   struct strbuf *base, const char *path,\n     - \t\t\t   unsigned int mode, void *context)\n     - {\n     -\n     - ## revision.c ##\n     -@@ revision.c: static void mark_tree_contents_uninteresting(struct repository *r,\n     - \tstruct tree_desc desc;\n     - \tstruct name_entry entry;\n     - \n     --\tif (parse_tree_gently(tree, 1) < 0)\n     -+\tif (parse_tree_gently(r, tree, 1) < 0)\n     - \t\treturn;\n     - \n     - \tinit_tree_desc(&desc, tree->buffer, tree->size);\n     -@@ revision.c: static void add_children_by_path(struct repository *r,\n     - \tif (!tree)\n     - \t\treturn;\n     - \n     --\tif (parse_tree_gently(tree, 1) < 0)\n     -+\tif (parse_tree_gently(r, tree, 1) < 0)\n     - \t\treturn;\n     - \n     - \tinit_tree_desc(&desc, tree->buffer, tree->size);\n     -\n     - ## sparse-index.c ##\n     -@@ sparse-index.c: static void set_index_entry(struct index_state *istate, int nr, struct cache_ent\n     - \tadd_name_hash(istate, ce);\n     - }\n     - \n     --static int add_path_to_index(const struct object_id *oid,\n     -+static int add_path_to_index(struct repository *repo UNUSED, const struct object_id *oid,\n     - \t\t\t     struct strbuf *base, const char *path,\n     - \t\t\t     unsigned int mode, void *context)\n     - {\n     -\n     - ## t/t5005-archive-submodules.sh (new) ##\n     -@@\n     -+#!/bin/sh\n     -+\n     -+test_description='git archive --recurse-submodules test'\n     -+\n     -+. ./test-lib.sh\n     -+. \"$TEST_DIRECTORY\"/lib-submodule-update.sh\n     -+\n     -+test_expect_success 'setup' '\n     -+\tcreate_lib_submodule_repo &&\n     -+\tgit -C submodule_update_repo checkout valid_sub1 &&\n     -+\tgit -C submodule_update_repo submodule update\n     -+'\n     -+\n     -+check_tar() {\n     -+\ttarfile=$1.tar\n     -+\tlistfile=$1.lst\n     -+\tdir=$1\n     -+\tdir_with_prefix=$dir/$2\n     -+\n     -+\ttest_expect_success ' extract tar archive' '\n     -+\t\t(mkdir $dir && cd $dir && \"$TAR\" xf -) <$tarfile\n     -+\t'\n     -+}\n     -+\n     -+check_added() {\n     -+\tdir=$1\n     -+\tpath_in_fs=$2\n     -+\tpath_in_archive=$3\n     -+\n     -+\ttest_expect_success \" validate extra file $path_in_archive\" '\n     -+\t\ttest -f $dir/$path_in_archive &&\n     -+\t\tdiff -r $path_in_fs $dir/$path_in_archive\n     -+\t'\n     -+}\n     -+\n     -+check_not_added() {\n     -+\tdir=$1\n     -+\tpath_in_archive=$2\n     -+\n     -+\ttest_expect_success \" validate unpresent file $path_in_archive\" '\n     -+\t\t! test -f $dir/$path_in_archive &&\n     -+\t\t! test -d $dir/$path_in_archive\n     -+\t'\n     -+}\n     -+\n     -+test_expect_success 'archive without recurse, non-init' '\n     -+\treset_work_tree_to valid_sub1 &&\n     -+\tgit -C submodule_update archive HEAD >b.tar\n     -+'\n     -+\n     -+check_tar b\n     -+check_added b submodule_update/file1 file1\n     -+check_not_added b sub1/file1\n     -+\n     -+test_expect_success 'archive with recurse, non-init' '\n     -+\treset_work_tree_to valid_sub1 &&\n     -+\t! git -C submodule_update archive --recurse-submodules HEAD >b2-err.tar\n     -+'\n     -+\n     -+test_expect_success 'archive with recurse, init' '\n     -+\treset_work_tree_to valid_sub1 &&\n     -+\tgit -C submodule_update submodule update --init &&\n     -+\tgit -C submodule_update ls-files --recurse-submodules &&\n     -+\tgit -C submodule_update ls-tree HEAD &&\n     -+\tgit -C submodule_update archive --recurse-submodules HEAD >b2.tar\n     -+'\n     -+\n     -+check_tar b2\n     -+check_added b2 submodule_update/sub1/file1 sub1/file1\n     -+\n     -+test_expect_success 'archive with recurse with big files' '\n     -+\treset_work_tree_to valid_sub1 &&\n     -+\ttest_config core.bigfilethreshold 1 &&\n     -+\tgit -C submodule_update submodule update --init &&\n     -+\tgit -C submodule_update ls-files --recurse-submodules &&\n     -+\tgit -C submodule_update ls-tree HEAD &&\n     -+\tgit -C submodule_update archive --recurse-submodules HEAD >b3.tar\n     -+'\n     -+\n     -+check_tar b3\n     -+check_added b3 submodule_update/sub1/file1 sub1/file1\n     -+\n     -+\n     -+test_done\n     -\n     - ## tree.c ##\n     -@@\n     - #include \"alloc.h\"\n     - #include \"tree-walk.h\"\n     - #include \"repository.h\"\n     -+#include \"pathspec.h\"\n     - \n     - const char *tree_type = \"tree\";\n     - \n     -@@ tree.c: int read_tree_at(struct repository *r,\n     - \tint len, oldlen = base->len;\n     - \tenum interesting retval = entry_not_interesting;\n     - \n     --\tif (parse_tree(tree))\n     --\t\treturn -1;\n     -+\tif (repo_parse_tree(r, tree))\n     -+\t\tdie(\"Failed to parse tree\");\n     - \n     - \tinit_tree_desc(&desc, tree->buffer, tree->size);\n     - \n     -@@ tree.c: int read_tree_at(struct repository *r,\n     - \t\t\t\tcontinue;\n     - \t\t}\n     - \n     --\t\tswitch (fn(&entry.oid, base,\n     -+\t\tswitch (fn(r, &entry.oid, base,\n     - \t\t\t   entry.path, entry.mode, context)) {\n     - \t\tcase 0:\n     - \t\t\tcontinue;\n     -@@ tree.c: int read_tree_at(struct repository *r,\n     - \t\t\treturn -1;\n     - \t\t}\n     - \n     --\t\tif (S_ISDIR(entry.mode))\n     -+\t\tif (S_ISDIR(entry.mode)) {\n     - \t\t\toidcpy(&oid, &entry.oid);\n     --\t\telse if (S_ISGITLINK(entry.mode)) {\n     -+\t\t\tlen = tree_entry_len(&entry);\n     -+\t\t\tstrbuf_add(base, entry.path, len);\n     -+\t\t\tstrbuf_addch(base, '/');\n     -+\t\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n     -+\t\t\t\t\t\tbase, pathspec,\n     -+\t\t\t\t\t\tfn, context);\n     -+\t\t\tstrbuf_setlen(base, oldlen);\n     -+\t\t\tif (retval)\n     -+\t\t\t\treturn -1;\n     -+\t\t} else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) {\n     - \t\t\tstruct commit *commit;\n     -+\t\t\tstruct repository subrepo;\n     -+\t\t\tstruct repository* subrepo_p = &subrepo;\n     -+\t\t\tstruct tree* submodule_tree;\n     - \n     --\t\t\tcommit = lookup_commit(r, &entry.oid);\n     -+\t\t\tif (repo_submodule_init(subrepo_p, r, entry.path, null_oid()))\n     -+\t\t\t\tdie(\"couldn't init submodule %s%s\", base->buf, entry.path);\n     -+\n     -+\t\t\tif (repo_read_index(subrepo_p) < 0)\n     -+\t\t\t\tdie(\"index file corrupt\");\n     -+\n     -+\t\t\tcommit = lookup_commit(subrepo_p, &entry.oid);\n     - \t\t\tif (!commit)\n     - \t\t\t\tdie(\"Commit %s in submodule path %s%s not found\",\n     - \t\t\t\t    oid_to_hex(&entry.oid),\n     - \t\t\t\t    base->buf, entry.path);\n     - \n     --\t\t\tif (parse_commit(commit))\n     -+\t\t\tif (repo_parse_commit(subrepo_p, commit))\n     - \t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n     - \t\t\t\t    oid_to_hex(&entry.oid),\n     - \t\t\t\t    base->buf, entry.path);\n     - \n     --\t\t\toidcpy(&oid, get_commit_tree_oid(commit));\n     -+\t\t\tsubmodule_tree = repo_get_commit_tree(subrepo_p, commit);\n     -+\t\t\toidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);\n     -+\n     -+\t\t\tlen = tree_entry_len(&entry);\n     -+\t\t\tstrbuf_add(base, entry.path, len);\n     -+\t\t\tstrbuf_addch(base, '/');\n     -+\t\t\tretval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),\n     -+\t\t\t\t\t\tbase, pathspec,\n     -+\t\t\t\t\t\tfn, context);\n     -+\t\t\tif (retval) {\n     -+\t\t\t    die(\"failed to read tree for %s%s\", base->buf, entry.path);\n     -+\t\t\t    return -1;\n     -+\t\t\t}\n     -+\t\t\tstrbuf_setlen(base, oldlen);\n     -+\t\t\trepo_clear(subrepo_p);\n     - \t\t}\n     --\t\telse\n     --\t\t\tcontinue;\n     - \n     --\t\tlen = tree_entry_len(&entry);\n     --\t\tstrbuf_add(base, entry.path, len);\n     --\t\tstrbuf_addch(base, '/');\n     --\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n     --\t\t\t\t      base, pathspec,\n     --\t\t\t\t      fn, context);\n     --\t\tstrbuf_setlen(base, oldlen);\n     --\t\tif (retval)\n     --\t\t\treturn -1;\n     - \t}\n     - \treturn 0;\n     - }\n     -@@ tree.c: int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size)\n     - \treturn 0;\n     - }\n     - \n     --int parse_tree_gently(struct tree *item, int quiet_on_missing)\n     -+int parse_tree_gently(struct repository *r, struct tree *item, int quiet_on_missing)\n     - {\n     - \t enum object_type type;\n     - \t void *buffer;\n     -@@ tree.c: int parse_tree_gently(struct tree *item, int quiet_on_missing)\n     - \n     - \tif (item->object.parsed)\n     - \t\treturn 0;\n     --\tbuffer = read_object_file(&item->object.oid, &type, &size);\n     -+\tbuffer = repo_read_object_file(r, &item->object.oid, &type, &size);\n     - \tif (!buffer)\n     - \t\treturn quiet_on_missing ? -1 :\n     - \t\t\terror(\"Could not read %s\",\n     -\n     - ## tree.h ##\n     -@@ tree.h: struct tree *lookup_tree(struct repository *r, const struct object_id *oid);\n     - \n     - int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size);\n     - \n     --int parse_tree_gently(struct tree *tree, int quiet_on_missing);\n     --static inline int parse_tree(struct tree *tree)\n     -+int parse_tree_gently(struct repository *r, struct tree *tree, int quiet_on_missing);\n     -+static inline int repo_parse_tree(struct repository *r, struct tree *tree)\n     - {\n     --\treturn parse_tree_gently(tree, 0);\n     -+\treturn parse_tree_gently(r, tree, 0);\n     - }\n     -+#ifndef NO_THE_REPOSITORY_COMPATIBILITY_MACROS\n     -+#define parse_tree(tree) repo_parse_tree(the_repository, tree)\n     -+#endif\n     - void free_tree_buffer(struct tree *tree);\n     - \n     - /* Parses and returns the tree in the given ent, chasing tags and commits. */\n     -@@ tree.h: struct tree *parse_tree_indirect(const struct object_id *oid);\n     - int cmp_cache_name_compare(const void *a_, const void *b_);\n     - \n     - #define READ_TREE_RECURSIVE 1\n     --typedef int (*read_tree_fn_t)(const struct object_id *, struct strbuf *, const char *, unsigned int, void *);\n     -+typedef int (*read_tree_fn_t)(struct repository *, const struct object_id *, struct strbuf *, const char *, unsigned int, void *);\n     - \n     - int read_tree_at(struct repository *r,\n     - \t\t struct tree *tree, struct strbuf *base,\n     -\n     - ## wt-status.c ##\n     -@@ wt-status.c: static void wt_status_collect_changes_index(struct wt_status *s)\n     - \trelease_revisions(&rev);\n     - }\n     - \n     --static int add_file_to_list(const struct object_id *oid,\n     -+static int add_file_to_list(struct repository *repo UNUSED, const struct object_id *oid,\n     - \t\t\t    struct strbuf *base, const char *path,\n     - \t\t\t    unsigned int mode, void *context)\n     - {\n  -:  ----------- >  7:  2443c9b1b6e archive: remove global repository from archive_args\n  -:  ----------- >  8:  4672e3d9586 archive: add --recurse-submodules to git-archive command\n  -:  ----------- >  9:  f88ebbaf17c archive: add tests for git archive --recurse-submodules\n\n-- \ngitgitgadget\n"},{"id":"465060","messageId":"d3d1738e670d5dbf1378fc5c3209b2e98234a771.1665973401.git.gitgitgadget@gmail.com","threadId":"58613","inReplyTo":"pull.1359.v3.git.git.1665973401.gitgitgadget@gmail.com","subject":"[PATCH v3 4/9] tree: handle submodule case for read_tree_at properly","fromName":"Heather Lapointe via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-17T02:23:16Z","receivedAt":"2022-10-17T02:23:46Z","isPatch":true,"sender":{"key":"alpha@alphaservcomputing.solutions","avatar":null},"body":"From: Heather Lapointe <alpha@alphaservcomputing.solutions>\n\nThis supports traversal into an actual submodule for read_tree_at.\nThe logic is blocked on pathspec->recurse_submodules now,\nbut previously hadn't been executed due to all fn() cases\nreturning early for submodules.\n\nSigned-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n---\n tree.c | 88 ++++++++++++++++++++++++++++++++++++++++------------------\n 1 file changed, 61 insertions(+), 27 deletions(-)\n\ndiff --git a/tree.c b/tree.c\nindex 13f9173d45e..2a087c010f9 100644\n--- a/tree.c\n+++ b/tree.c\n@@ -8,6 +8,7 @@\n #include \"alloc.h\"\n #include \"tree-walk.h\"\n #include \"repository.h\"\n+#include \"pathspec.h\"\n \n const char *tree_type = \"tree\";\n \n@@ -47,40 +48,73 @@ int read_tree_at(struct repository *r,\n \t\t\treturn -1;\n \t\t}\n \n-\t\tif (S_ISDIR(entry.mode))\n+\t\tif (S_ISDIR(entry.mode)) {\n \t\t\toidcpy(&oid, &entry.oid);\n-\t\telse if (S_ISGITLINK(entry.mode)) {\n-\t\t\tstruct commit *commit;\n \n-\t\t\tcommit = lookup_commit(r, &entry.oid);\n+\t\t\tlen = tree_entry_len(&entry);\n+\t\t\tstrbuf_add(base, entry.path, len);\n+\t\t\tstrbuf_addch(base, '/');\n+\t\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n+\t\t\t\t\t\tbase, pathspec,\n+\t\t\t\t\t\tfn, context);\n+\t\t\tstrbuf_setlen(base, oldlen);\n+\t\t\tif (retval)\n+\t\t\t\treturn -1;\n+\t\t} else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) {\n+\t\t\tstruct commit *commit;\n+\t\t\tstruct repository subrepo;\n+\t\t\tstruct repository* subrepo_p = &subrepo;\n+\t\t\tstruct tree* submodule_tree;\n+\t\t\tchar *submodule_rel_path;\n+\t\t\tint name_base_len = 0;\n+\n+\t\t\tlen = tree_entry_len(&entry);\n+\t\t\tstrbuf_add(base, entry.path, len);\n+\t\t\tsubmodule_rel_path = base->buf;\n+\t\t\t// repo_submodule_init expects a path relative to submodule_prefix\n+\t\t\tif (r->submodule_prefix) {\n+\t\t\t\tname_base_len = strlen(r->submodule_prefix);\n+\t\t\t\t// we should always expect to start with submodule_prefix\n+\t\t\t\tassert(!strncmp(submodule_rel_path, r->submodule_prefix, name_base_len));\n+\t\t\t\t// strip the prefix\n+\t\t\t\tsubmodule_rel_path += name_base_len;\n+\t\t\t\t// if submodule_prefix doesn't end with a /, we want to get rid of that too\n+\t\t\t\tif (is_dir_sep(submodule_rel_path[0])) {\n+\t\t\t\t\tsubmodule_rel_path++;\n+\t\t\t\t}\n+\t\t\t}\n+\n+\t\t\tif (repo_submodule_init(subrepo_p, r, submodule_rel_path, null_oid()))\n+\t\t\t\tdie(\"couldn't init submodule %s\", base->buf);\n+\n+\t\t\tif (repo_read_index(subrepo_p) < 0)\n+\t\t\t\tdie(\"index file corrupt\");\n+\n+\t\t\tcommit = lookup_commit(subrepo_p, &entry.oid);\n \t\t\tif (!commit)\n-\t\t\t\tdie(\"Commit %s in submodule path %s%s not found\",\n+\t\t\t\tdie(\"Commit %s in submodule path %s not found\",\n \t\t\t\t    oid_to_hex(&entry.oid),\n-\t\t\t\t    base->buf, entry.path);\n-\n-\t\t\t// FIXME: This is the wrong repo instance (it refers to the superproject)\n-\t\t\t// it will always fail as is (will fix in later patch)\n-\t\t\t// This current codepath isn't executed by any existing callbacks\n-\t\t\t// so it wouldn't show up as an issue at this time.\n-\t\t\tif (repo_parse_commit(r, commit))\n-\t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n+\t\t\t\t    base->buf);\n+\n+\t\t\tif (repo_parse_commit(subrepo_p, commit))\n+\t\t\t\tdie(\"Invalid commit %s in submodule path %s\",\n \t\t\t\t    oid_to_hex(&entry.oid),\n-\t\t\t\t    base->buf, entry.path);\n+\t\t\t\t    base->buf);\n \n-\t\t\toidcpy(&oid, get_commit_tree_oid(commit));\n-\t\t}\n-\t\telse\n-\t\t\tcontinue;\n+\t\t\tsubmodule_tree = repo_get_commit_tree(subrepo_p, commit);\n+\t\t\toidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);\n \n-\t\tlen = tree_entry_len(&entry);\n-\t\tstrbuf_add(base, entry.path, len);\n-\t\tstrbuf_addch(base, '/');\n-\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n-\t\t\t\t      base, pathspec,\n-\t\t\t\t      fn, context);\n-\t\tstrbuf_setlen(base, oldlen);\n-\t\tif (retval)\n-\t\t\treturn -1;\n+\t\t\tstrbuf_addch(base, '/');\n+\n+\t\t\tretval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),\n+\t\t\t\t\t\tbase, pathspec,\n+\t\t\t\t\t\tfn, context);\n+\t\t\tif (retval)\n+\t\t\t    die(\"failed to read tree for %s\", base->buf);\n+\t\t\tstrbuf_setlen(base, oldlen);\n+\t\t\trepo_clear(subrepo_p);\n+\t\t}\n+\t\t// else, this is a file (or a submodule, but no pathspec->recurse_submodules)\n \t}\n \treturn 0;\n }\n-- \ngitgitgadget\n\n"},{"id":"465061","messageId":"376345fdf66b274a7ce3dfff4d0a2b185858147c.1665973401.git.gitgitgadget@gmail.com","threadId":"58613","inReplyTo":"pull.1359.v3.git.git.1665973401.gitgitgadget@gmail.com","subject":"[PATCH v3 5/9] tree: add repository parameter to read_tree_fn_t","fromName":"Heather Lapointe via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-17T02:23:17Z","receivedAt":"2022-10-17T02:23:48Z","isPatch":true,"sender":{"key":"alpha@alphaservcomputing.solutions","avatar":null},"body":"From: Heather Lapointe <alpha@alphaservcomputing.solutions>\n\nAdd a repo paramter to read_tree_fn_t because most callbacks do\nneed some repo instance.\nThis avoids having to use the_repository functions otherwise\nand improves repo context switching for submodules.\n\nSigned-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n---\n archive.c                         | 11 ++++++----\n builtin/checkout.c                |  4 +++-\n builtin/log.c                     |  4 +++-\n builtin/ls-files.c                |  8 ++++++--\n builtin/ls-tree.c                 | 34 ++++++++++++++++++++-----------\n merge-recursive.c                 |  4 +++-\n sparse-index.c                    |  4 +++-\n t/helper/test-tree-read-tree-at.c |  3 ++-\n tree.c                            |  2 +-\n tree.h                            |  2 +-\n wt-status.c                       |  4 +++-\n 11 files changed, 54 insertions(+), 26 deletions(-)\n\ndiff --git a/archive.c b/archive.c\nindex 61a79e4a227..15f3ac92dfc 100644\n--- a/archive.c\n+++ b/archive.c\n@@ -225,7 +225,9 @@ static int write_directory(struct archiver_context *c)\n \treturn ret ? -1 : 0;\n }\n \n-static int queue_or_write_archive_entry(const struct object_id *oid,\n+static int queue_or_write_archive_entry(\n+\t\tstruct repository *r,\n+\t\tconst struct object_id *oid,\n \t\tstruct strbuf *base, const char *filename,\n \t\tunsigned mode, void *context)\n {\n@@ -246,7 +248,7 @@ static int queue_or_write_archive_entry(const struct object_id *oid,\n \t\t/* Borrow base, but restore its original value when done. */\n \t\tstrbuf_addstr(base, filename);\n \t\tstrbuf_addch(base, '/');\n-\t\tcheck = get_archive_attrs(c->args->repo->index, base->buf);\n+\t\tcheck = get_archive_attrs(r->index, base->buf);\n \t\tstrbuf_setlen(base, baselen);\n \n \t\tif (check_attr_export_ignore(check))\n@@ -382,7 +384,8 @@ struct path_exists_context {\n \tstruct archiver_args *args;\n };\n \n-static int reject_entry(const struct object_id *oid UNUSED,\n+static int reject_entry(\n+\t\t\tstruct repository *r, const struct object_id *oid UNUSED,\n \t\t\tstruct strbuf *base,\n \t\t\tconst char *filename, unsigned mode,\n \t\t\tvoid *context)\n@@ -394,7 +397,7 @@ static int reject_entry(const struct object_id *oid UNUSED,\n \t\tstruct strbuf sb = STRBUF_INIT;\n \t\tstrbuf_addbuf(&sb, base);\n \t\tstrbuf_addstr(&sb, filename);\n-\t\tif (!match_pathspec(ctx->args->repo->index,\n+\t\tif (!match_pathspec(r->index,\n \t\t\t\t    &ctx->pathspec,\n \t\t\t\t    sb.buf, sb.len, 0, NULL, 1))\n \t\t\tret = READ_TREE_RECURSIVE;\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex 2a132392fbe..ee98858afe6 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -124,7 +124,9 @@ static int post_checkout_hook(struct commit *old_commit, struct commit *new_comm\n \n }\n \n-static int update_some(const struct object_id *oid, struct strbuf *base,\n+static int update_some(\n+\t\t       struct repository *r UNUSED,\n+\t\t\t   const struct object_id *oid, struct strbuf *base,\n \t\t       const char *pathname, unsigned mode, void *context UNUSED)\n {\n \tint len;\ndiff --git a/builtin/log.c b/builtin/log.c\nindex ee19dc5d450..608a448fe4d 100644\n--- a/builtin/log.c\n+++ b/builtin/log.c\n@@ -698,7 +698,9 @@ static int show_tag_object(const struct object_id *oid, struct rev_info *rev)\n \treturn 0;\n }\n \n-static int show_tree_object(const struct object_id *oid UNUSED,\n+static int show_tree_object(\n+\t\t\t    struct repository *r UNUSED,\n+\t\t\t    const struct object_id *oid UNUSED,\n \t\t\t    struct strbuf *base UNUSED,\n \t\t\t    const char *pathname, unsigned mode,\n \t\t\t    void *context)\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex 4cf8a236483..fbb07fa08c2 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -533,7 +533,9 @@ static int read_one_entry_opt(struct index_state *istate,\n \treturn add_index_entry(istate, ce, opt);\n }\n \n-static int read_one_entry(const struct object_id *oid, struct strbuf *base,\n+static int read_one_entry(\n+\t\t\t  struct repository *r UNUSED,\n+\t\t\t  const struct object_id *oid, struct strbuf *base,\n \t\t\t  const char *pathname, unsigned mode,\n \t\t\t  void *context)\n {\n@@ -547,7 +549,9 @@ static int read_one_entry(const struct object_id *oid, struct strbuf *base,\n  * This is used when the caller knows there is no existing entries at\n  * the stage that will conflict with the entry being added.\n  */\n-static int read_one_entry_quick(const struct object_id *oid, struct strbuf *base,\n+static int read_one_entry_quick(\n+\t\t\t\tstruct repository *r UNUSED,\n+\t\t\t\tconst struct object_id *oid, struct strbuf *base,\n \t\t\t\tconst char *pathname, unsigned mode,\n \t\t\t\tvoid *context)\n {\ndiff --git a/builtin/ls-tree.c b/builtin/ls-tree.c\nindex c3ea09281af..dd571abad1c 100644\n--- a/builtin/ls-tree.c\n+++ b/builtin/ls-tree.c\n@@ -141,8 +141,10 @@ static int show_recursive(const char *base, size_t baselen, const char *pathname\n \treturn 0;\n }\n \n-static int show_tree_fmt(const struct object_id *oid, struct strbuf *base,\n-\t\t\t const char *pathname, unsigned mode, void *context UNUSED)\n+static int show_tree_fmt(\n+\t\t\tstruct repository *r UNUSED,\n+\t\t\tconst struct object_id *oid, struct strbuf *base,\n+\t\t\tconst char *pathname, unsigned mode, void *context UNUSED)\n {\n \tsize_t baselen;\n \tint recurse = 0;\n@@ -211,9 +213,11 @@ static void show_tree_common_default_long(struct strbuf *base,\n \tstrbuf_setlen(base, baselen);\n }\n \n-static int show_tree_default(const struct object_id *oid, struct strbuf *base,\n-\t\t\t     const char *pathname, unsigned mode,\n-\t\t\t     void *context UNUSED)\n+static int show_tree_default(\n+\t\tstruct repository *r,\n+\t\tconst struct object_id *oid, struct strbuf *base,\n+\t\tconst char *pathname, unsigned mode,\n+\t\tvoid *context UNUSED)\n {\n \tint early;\n \tint recurse;\n@@ -224,12 +228,14 @@ static int show_tree_default(const struct object_id *oid, struct strbuf *base,\n \t\treturn early;\n \n \tprintf(\"%06o %s %s\\t\", data.mode, type_name(data.type),\n-\t       find_unique_abbrev(data.oid, abbrev));\n+\t       repo_find_unique_abbrev(r, data.oid, abbrev));\n \tshow_tree_common_default_long(base, pathname, data.base->len);\n \treturn recurse;\n }\n \n-static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n+static int show_tree_long(\n+\t\t\t  struct repository *r,\n+\t\t\t  const struct object_id *oid, struct strbuf *base,\n \t\t\t  const char *pathname, unsigned mode,\n \t\t\t  void *context UNUSED)\n {\n@@ -244,7 +250,7 @@ static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n \n \tif (data.type == OBJ_BLOB) {\n \t\tunsigned long size;\n-\t\tif (oid_object_info(the_repository, data.oid, &size) == OBJ_BAD)\n+\t\tif (oid_object_info(r, data.oid, &size) == OBJ_BAD)\n \t\t\txsnprintf(size_text, sizeof(size_text), \"BAD\");\n \t\telse\n \t\t\txsnprintf(size_text, sizeof(size_text),\n@@ -254,12 +260,14 @@ static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n \t}\n \n \tprintf(\"%06o %s %s %7s\\t\", data.mode, type_name(data.type),\n-\t       find_unique_abbrev(data.oid, abbrev), size_text);\n+\t       repo_find_unique_abbrev(r, data.oid, abbrev), size_text);\n \tshow_tree_common_default_long(base, pathname, data.base->len);\n \treturn recurse;\n }\n \n-static int show_tree_name_only(const struct object_id *oid, struct strbuf *base,\n+static int show_tree_name_only(\n+\t\t\t       struct repository *r UNUSED,\n+\t\t\t       const struct object_id *oid, struct strbuf *base,\n \t\t\t       const char *pathname, unsigned mode,\n \t\t\t       void *context UNUSED)\n {\n@@ -280,7 +288,9 @@ static int show_tree_name_only(const struct object_id *oid, struct strbuf *base,\n \treturn recurse;\n }\n \n-static int show_tree_object(const struct object_id *oid, struct strbuf *base,\n+static int show_tree_object(\n+\t\t\t    struct repository *r,\n+\t\t\t\tconst struct object_id *oid, struct strbuf *base,\n \t\t\t    const char *pathname, unsigned mode,\n \t\t\t    void *context UNUSED)\n {\n@@ -292,7 +302,7 @@ static int show_tree_object(const struct object_id *oid, struct strbuf *base,\n \tif (early >= 0)\n \t\treturn early;\n \n-\tprintf(\"%s%c\", find_unique_abbrev(oid, abbrev), line_termination);\n+\tprintf(\"%s%c\", repo_find_unique_abbrev(r, oid, abbrev), line_termination);\n \treturn recurse;\n }\n \ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex 4ddd3adea00..dccde276655 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -456,7 +456,9 @@ static void unpack_trees_finish(struct merge_options *opt)\n \tclear_unpack_trees_porcelain(&opt->priv->unpack_opts);\n }\n \n-static int save_files_dirs(const struct object_id *oid UNUSED,\n+static int save_files_dirs(\n+\t\t\t   struct repository *r UNUSED,\n+\t\t\t   const struct object_id *oid UNUSED,\n \t\t\t   struct strbuf *base, const char *path,\n \t\t\t   unsigned int mode, void *context)\n {\ndiff --git a/sparse-index.c b/sparse-index.c\nindex e4a54ce1943..4187c7ce9c4 100644\n--- a/sparse-index.c\n+++ b/sparse-index.c\n@@ -232,7 +232,9 @@ static void set_index_entry(struct index_state *istate, int nr, struct cache_ent\n \tadd_name_hash(istate, ce);\n }\n \n-static int add_path_to_index(const struct object_id *oid,\n+static int add_path_to_index(\n+\t\t\t     struct repository *r UNUSED,\n+\t\t\t     const struct object_id *oid,\n \t\t\t     struct strbuf *base, const char *path,\n \t\t\t     unsigned int mode, void *context)\n {\ndiff --git a/t/helper/test-tree-read-tree-at.c b/t/helper/test-tree-read-tree-at.c\nindex bba759bb264..d2bcc8c849a 100644\n--- a/t/helper/test-tree-read-tree-at.c\n+++ b/t/helper/test-tree-read-tree-at.c\n@@ -6,7 +6,8 @@ We call it tree-read-tree-at to disambiguate with the read-tree tool.\n #include \"test-tool.h\"\n #include \"tree.h\"\n \n-static int test_handle_entry(const struct object_id *oid,\n+static int test_handle_entry(\n+\t\tstruct repository *r UNUSED, const struct object_id *oid,\n \t\tstruct strbuf *base, const char *filename,\n \t\tunsigned mode, void *context UNUSED) {\n \tprintf(\"%i %s %s%s\\n\", mode, oid_to_hex(oid), base->buf, filename);\ndiff --git a/tree.c b/tree.c\nindex 2a087c010f9..17c3af819e2 100644\n--- a/tree.c\n+++ b/tree.c\n@@ -38,7 +38,7 @@ int read_tree_at(struct repository *r,\n \t\t\t\tcontinue;\n \t\t}\n \n-\t\tswitch (fn(&entry.oid, base,\n+\t\tswitch (fn(r, &entry.oid, base,\n \t\t\t   entry.path, entry.mode, context)) {\n \t\tcase 0:\n \t\t\tcontinue;\ndiff --git a/tree.h b/tree.h\nindex cc6402e4738..94b1e11d9eb 100644\n--- a/tree.h\n+++ b/tree.h\n@@ -37,7 +37,7 @@ struct tree *repo_parse_tree_indirect(struct repository *r, const struct object_\n int cmp_cache_name_compare(const void *a_, const void *b_);\n \n #define READ_TREE_RECURSIVE 1\n-typedef int (*read_tree_fn_t)(const struct object_id *, struct strbuf *, const char *, unsigned int, void *);\n+typedef int (*read_tree_fn_t)(struct repository *r, const struct object_id *, struct strbuf *, const char *, unsigned int, void *);\n \n int read_tree_at(struct repository *r,\n \t\t struct tree *tree, struct strbuf *base,\ndiff --git a/wt-status.c b/wt-status.c\nindex 5813174896c..cff2a780f32 100644\n--- a/wt-status.c\n+++ b/wt-status.c\n@@ -665,7 +665,9 @@ static void wt_status_collect_changes_index(struct wt_status *s)\n \trelease_revisions(&rev);\n }\n \n-static int add_file_to_list(const struct object_id *oid,\n+static int add_file_to_list(\n+\t\t\t    struct repository *r UNUSED,\n+\t\t\t    const struct object_id *oid,\n \t\t\t    struct strbuf *base, const char *path,\n \t\t\t    unsigned int mode, void *context)\n {\n-- \ngitgitgadget\n\n"},{"id":"465062","messageId":"1b9b049d64fc4ea919c94e57b74a1760b3837892.1665973401.git.gitgitgadget@gmail.com","threadId":"58613","inReplyTo":"pull.1359.v3.git.git.1665973401.gitgitgadget@gmail.com","subject":"[PATCH v3 6/9] archive: pass repo objects to write_archive handlers","fromName":"Heather Lapointe via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-17T02:23:18Z","receivedAt":"2022-10-17T02:23:51Z","isPatch":true,"sender":{"key":"alpha@alphaservcomputing.solutions","avatar":null},"body":"From: Heather Lapointe <alpha@alphaservcomputing.solutions>\n\nUse contextual repos instead of the_repository or args->repo\nto ensure that submodules will be handled correctly\nsince they use multiple repo instances.\n\nSigned-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n---\n archive-tar.c | 15 ++++++++++-----\n archive-zip.c | 15 +++++++++------\n archive.c     | 38 ++++++++++++++++++++++----------------\n archive.h     | 14 +++++++++++---\n 4 files changed, 52 insertions(+), 30 deletions(-)\n\ndiff --git a/archive-tar.c b/archive-tar.c\nindex 3e4822b6840..5a2d42ff229 100644\n--- a/archive-tar.c\n+++ b/archive-tar.c\n@@ -18,6 +18,7 @@ static unsigned long offset;\n static int tar_umask = 002;\n \n static int write_tar_filter_archive(const struct archiver *ar,\n+\t\t\t\t\tstruct repository *repo,\n \t\t\t\t    struct archiver_args *args);\n \n /*\n@@ -246,7 +247,9 @@ static void write_extended_header(struct archiver_args *args,\n \twrite_blocked(buffer, size);\n }\n \n-static int write_tar_entry(struct archiver_args *args,\n+static int write_tar_entry(\n+\t\t\t   struct repository *repo,\n+\t\t\t   struct archiver_args *args,\n \t\t\t   const struct object_id *oid,\n \t\t\t   const char *path, size_t pathlen,\n \t\t\t   unsigned int mode,\n@@ -316,7 +319,7 @@ static int write_tar_entry(struct archiver_args *args,\n \t\tif (buffer)\n \t\t\twrite_blocked(buffer, size);\n \t\telse\n-\t\t\terr = stream_blocked(args->repo, oid);\n+\t\t\terr = stream_blocked(repo, oid);\n \t}\n \treturn err;\n }\n@@ -422,12 +425,13 @@ static int git_tar_config(const char *var, const char *value, void *cb)\n }\n \n static int write_tar_archive(const struct archiver *ar UNUSED,\n+\t\t\t     struct repository *repo,\n \t\t\t     struct archiver_args *args)\n {\n \tint err = 0;\n \n \twrite_global_extended_header(args);\n-\terr = write_archive_entries(args, write_tar_entry);\n+\terr = write_archive_entries(repo, args, write_tar_entry);\n \tif (!err)\n \t\twrite_trailer();\n \treturn err;\n@@ -462,6 +466,7 @@ static void tgz_write_block(const void *data)\n static const char internal_gzip_command[] = \"git archive gzip\";\n \n static int write_tar_filter_archive(const struct archiver *ar,\n+\t\t\t\t\tstruct repository *repo,\n \t\t\t\t    struct archiver_args *args)\n {\n #if ZLIB_VERNUM >= 0x1221\n@@ -484,7 +489,7 @@ static int write_tar_filter_archive(const struct archiver *ar,\n \t\tgzstream.next_out = outbuf;\n \t\tgzstream.avail_out = sizeof(outbuf);\n \n-\t\tr = write_tar_archive(ar, args);\n+\t\tr = write_tar_archive(ar, repo, args);\n \n \t\ttgz_deflate(Z_FINISH);\n \t\tgit_deflate_end(&gzstream);\n@@ -506,7 +511,7 @@ static int write_tar_filter_archive(const struct archiver *ar,\n \t\tdie_errno(_(\"unable to redirect descriptor\"));\n \tclose(filter.in);\n \n-\tr = write_tar_archive(ar, args);\n+\tr = write_tar_archive(ar, repo, args);\n \n \tclose(1);\n \tif (finish_command(&filter) != 0)\ndiff --git a/archive-zip.c b/archive-zip.c\nindex 0456f1ebf15..2c1f943a6cc 100644\n--- a/archive-zip.c\n+++ b/archive-zip.c\n@@ -283,7 +283,9 @@ static int entry_is_binary(struct index_state *istate, const char *path,\n \n #define STREAM_BUFFER_SIZE (1024 * 16)\n \n-static int write_zip_entry(struct archiver_args *args,\n+static int write_zip_entry(\n+\t\t\t   struct repository *repo,\n+\t\t\t   struct archiver_args *args,\n \t\t\t   const struct object_id *oid,\n \t\t\t   const char *path, size_t pathlen,\n \t\t\t   unsigned int mode,\n@@ -340,7 +342,7 @@ static int write_zip_entry(struct archiver_args *args,\n \n \t\tif (!buffer) {\n \t\t\tenum object_type type;\n-\t\t\tstream = open_istream(args->repo, oid, &type, &size,\n+\t\t\tstream = open_istream(repo, oid, &type, &size,\n \t\t\t\t\t      NULL);\n \t\t\tif (!stream)\n \t\t\t\treturn error(_(\"cannot stream blob %s\"),\n@@ -349,7 +351,7 @@ static int write_zip_entry(struct archiver_args *args,\n \t\t\tout = NULL;\n \t\t} else {\n \t\t\tcrc = crc32(crc, buffer, size);\n-\t\t\tis_binary = entry_is_binary(args->repo->index,\n+\t\t\tis_binary = entry_is_binary(repo->index,\n \t\t\t\t\t\t    path_without_prefix,\n \t\t\t\t\t\t    buffer, size);\n \t\t\tout = buffer;\n@@ -426,7 +428,7 @@ static int write_zip_entry(struct archiver_args *args,\n \t\t\t\tbreak;\n \t\t\tcrc = crc32(crc, buf, readlen);\n \t\t\tif (is_binary == -1)\n-\t\t\t\tis_binary = entry_is_binary(args->repo->index,\n+\t\t\t\tis_binary = entry_is_binary(repo->index,\n \t\t\t\t\t\t\t    path_without_prefix,\n \t\t\t\t\t\t\t    buf, readlen);\n \t\t\twrite_or_die(1, buf, readlen);\n@@ -459,7 +461,7 @@ static int write_zip_entry(struct archiver_args *args,\n \t\t\t\tbreak;\n \t\t\tcrc = crc32(crc, buf, readlen);\n \t\t\tif (is_binary == -1)\n-\t\t\t\tis_binary = entry_is_binary(args->repo->index,\n+\t\t\t\tis_binary = entry_is_binary(repo->index,\n \t\t\t\t\t\t\t    path_without_prefix,\n \t\t\t\t\t\t\t    buf, readlen);\n \n@@ -619,6 +621,7 @@ static int archive_zip_config(const char *var, const char *value,\n }\n \n static int write_zip_archive(const struct archiver *ar UNUSED,\n+\t\t\t\t struct repository *repo,\n \t\t\t     struct archiver_args *args)\n {\n \tint err;\n@@ -629,7 +632,7 @@ static int write_zip_archive(const struct archiver *ar UNUSED,\n \n \tstrbuf_init(&zip_dir, 0);\n \n-\terr = write_archive_entries(args, write_zip_entry);\n+\terr = write_archive_entries(repo, args, write_zip_entry);\n \tif (!err)\n \t\twrite_zip_trailer(args->commit_oid);\n \ndiff --git a/archive.c b/archive.c\nindex 15f3ac92dfc..2cca7bc5c8a 100644\n--- a/archive.c\n+++ b/archive.c\n@@ -134,7 +134,9 @@ static int check_attr_export_subst(const struct attr_check *check)\n \treturn check && ATTR_TRUE(check->items[1].value);\n }\n \n-static int write_archive_entry(const struct object_id *oid, const char *base,\n+static int write_archive_entry(\n+\t\tstruct repository *repo,\n+\t\tconst struct object_id *oid, const char *base,\n \t\tint baselen, const char *filename, unsigned mode,\n \t\tvoid *context)\n {\n@@ -160,7 +162,7 @@ static int write_archive_entry(const struct object_id *oid, const char *base,\n \n \tif (!S_ISDIR(mode)) {\n \t\tconst struct attr_check *check;\n-\t\tcheck = get_archive_attrs(args->repo->index, path_without_prefix);\n+\t\tcheck = get_archive_attrs(repo->index, path_without_prefix);\n \t\tif (check_attr_export_ignore(check))\n \t\t\treturn 0;\n \t\targs->convert = check_attr_export_subst(check);\n@@ -169,7 +171,7 @@ static int write_archive_entry(const struct object_id *oid, const char *base,\n \tif (S_ISDIR(mode) || S_ISGITLINK(mode)) {\n \t\tif (args->verbose)\n \t\t\tfprintf(stderr, \"%.*s\\n\", (int)path.len, path.buf);\n-\t\terr = write_entry(args, oid, path.buf, path.len, mode, NULL, 0);\n+\t\terr = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, 0);\n \t\tif (err)\n \t\t\treturn err;\n \t\treturn (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);\n@@ -180,14 +182,14 @@ static int write_archive_entry(const struct object_id *oid, const char *base,\n \n \t/* Stream it? */\n \tif (S_ISREG(mode) && !args->convert &&\n-\t    oid_object_info(args->repo, oid, &size) == OBJ_BLOB &&\n+\t    oid_object_info(repo, oid, &size) == OBJ_BLOB &&\n \t    size > big_file_threshold)\n-\t\treturn write_entry(args, oid, path.buf, path.len, mode, NULL, size);\n+\t\treturn write_entry(repo, args, oid, path.buf, path.len, mode, NULL, size);\n \n \tbuffer = object_file_to_archive(args, path.buf, oid, mode, &type, &size);\n \tif (!buffer)\n \t\treturn error(_(\"cannot read '%s'\"), oid_to_hex(oid));\n-\terr = write_entry(args, oid, path.buf, path.len, mode, buffer, size);\n+\terr = write_entry(repo, args, oid, path.buf, path.len, mode, buffer, size);\n \tfree(buffer);\n \treturn err;\n }\n@@ -207,7 +209,9 @@ static void queue_directory(const struct object_id *oid,\n \toidcpy(&d->oid, oid);\n }\n \n-static int write_directory(struct archiver_context *c)\n+static int write_directory(\n+\t\tstruct repository *repo,\n+\t\tstruct archiver_context *c)\n {\n \tstruct directory *d = c->bottom;\n \tint ret;\n@@ -217,8 +221,8 @@ static int write_directory(struct archiver_context *c)\n \tc->bottom = d->up;\n \td->path[d->len - 1] = '\\0'; /* no trailing slash */\n \tret =\n-\t\twrite_directory(c) ||\n-\t\twrite_archive_entry(&d->oid, d->path, d->baselen,\n+\t\twrite_directory(repo, c) ||\n+\t\twrite_archive_entry(repo, &d->oid, d->path, d->baselen,\n \t\t\t\t    d->path + d->baselen, d->mode,\n \t\t\t\t    c) != READ_TREE_RECURSIVE;\n \tfree(d);\n@@ -257,9 +261,9 @@ static int queue_or_write_archive_entry(\n \t\treturn READ_TREE_RECURSIVE;\n \t}\n \n-\tif (write_directory(c))\n+\tif (write_directory(r, c))\n \t\treturn -1;\n-\treturn write_archive_entry(oid, base->buf, base->len, filename, mode,\n+\treturn write_archive_entry(r, oid, base->buf, base->len, filename, mode,\n \t\t\t\t   context);\n }\n \n@@ -269,7 +273,9 @@ struct extra_file_info {\n \tvoid *content;\n };\n \n-int write_archive_entries(struct archiver_args *args,\n+int write_archive_entries(\n+\t\tstruct repository *repo,\n+\t\tstruct archiver_args *args,\n \t\twrite_archive_entry_fn_t write_entry)\n {\n \tstruct archiver_context context;\n@@ -290,7 +296,7 @@ int write_archive_entries(struct archiver_args *args,\n \t\t\tlen--;\n \t\tif (args->verbose)\n \t\t\tfprintf(stderr, \"%.*s\\n\", (int)len, args->base);\n-\t\terr = write_entry(args, &args->tree->object.oid, args->base,\n+\t\terr = write_entry(repo, args, &args->tree->object.oid, args->base,\n \t\t\t\t  len, 040777, NULL, 0);\n \t\tif (err)\n \t\t\treturn err;\n@@ -345,12 +351,12 @@ int write_archive_entries(struct archiver_args *args,\n \t\t\tif (strbuf_read_file(&content, path, info->stat.st_size) < 0)\n \t\t\t\terr = error_errno(_(\"cannot read '%s'\"), path);\n \t\t\telse\n-\t\t\t\terr = write_entry(args, &fake_oid, path_in_archive.buf,\n+\t\t\t\terr = write_entry(repo, args, &fake_oid, path_in_archive.buf,\n \t\t\t\t\t\t  path_in_archive.len,\n \t\t\t\t\t\t  canon_mode(info->stat.st_mode),\n \t\t\t\t\t\t  content.buf, content.len);\n \t\t} else {\n-\t\t\terr = write_entry(args, &fake_oid,\n+\t\t\terr = write_entry(repo, args, &fake_oid,\n \t\t\t\t\t  path, strlen(path),\n \t\t\t\t\t  canon_mode(info->stat.st_mode),\n \t\t\t\t\t  info->content, info->stat.st_size);\n@@ -711,7 +717,7 @@ int write_archive(int argc, const char **argv, const char *prefix,\n \tparse_treeish_arg(argv, &args, prefix, remote);\n \tparse_pathspec_arg(argv + 1, &args);\n \n-\trc = ar->write_archive(ar, &args);\n+\trc = ar->write_archive(ar, repo, &args);\n \n \tstring_list_clear_func(&args.extra_files, extra_file_info_clear);\n \tfree(args.refname);\ndiff --git a/archive.h b/archive.h\nindex 08bed3ed3af..bfbbd3274bd 100644\n--- a/archive.h\n+++ b/archive.h\n@@ -41,7 +41,10 @@ const char *archive_format_from_filename(const char *filename);\n #define ARCHIVER_HIGH_COMPRESSION_LEVELS 4\n struct archiver {\n \tconst char *name;\n-\tint (*write_archive)(const struct archiver *, struct archiver_args *);\n+\tint (*write_archive)(\n+\t\tconst struct archiver *,\n+\t\tstruct repository *,\n+\t\tstruct archiver_args *);\n \tunsigned flags;\n \tchar *filter_command;\n };\n@@ -51,12 +54,17 @@ void init_tar_archiver(void);\n void init_zip_archiver(void);\n void init_archivers(void);\n \n-typedef int (*write_archive_entry_fn_t)(struct archiver_args *args,\n+typedef int (*write_archive_entry_fn_t)(\n+\t\t\t\t\tstruct repository *repo,\n+\t\t\t\t\tstruct archiver_args *args,\n \t\t\t\t\tconst struct object_id *oid,\n \t\t\t\t\tconst char *path, size_t pathlen,\n \t\t\t\t\tunsigned int mode,\n \t\t\t\t\tvoid *buffer, unsigned long size);\n \n-int write_archive_entries(struct archiver_args *args, write_archive_entry_fn_t write_entry);\n+int write_archive_entries(\n+\tstruct repository *repo,\n+\tstruct archiver_args *args,\n+\twrite_archive_entry_fn_t write_entry);\n \n #endif\t/* ARCHIVE_H */\n-- \ngitgitgadget\n\n"},{"id":"465063","messageId":"2443c9b1b6efce4340f01930c179d2fafbc5bbb3.1665973401.git.gitgitgadget@gmail.com","threadId":"58613","inReplyTo":"pull.1359.v3.git.git.1665973401.gitgitgadget@gmail.com","subject":"[PATCH v3 7/9] archive: remove global repository from archive_args","fromName":"Heather Lapointe via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-17T02:23:19Z","receivedAt":"2022-10-17T02:23:54Z","isPatch":true,"sender":{"key":"alpha@alphaservcomputing.solutions","avatar":null},"body":"From: Heather Lapointe <alpha@alphaservcomputing.solutions>\n\nRemove archive_args.repo to ensure all functions are using local\nrepository instances.\nSince all functions now have access to repo, this access isn't\nneeded anymore.\n\nThe main issue is that submodules do not use the same repo\nas the subproject repo that is being passed around contextually.\n\nSigned-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n---\n archive.c | 51 +++++++++++++++++++++++++++++----------------------\n archive.h |  1 -\n 2 files changed, 29 insertions(+), 23 deletions(-)\n\ndiff --git a/archive.c b/archive.c\nindex 2cca7bc5c8a..34549d849f1 100644\n--- a/archive.c\n+++ b/archive.c\n@@ -36,7 +36,9 @@ void init_archivers(void)\n \tinit_zip_archiver();\n }\n \n-static void format_subst(const struct commit *commit,\n+static void format_subst(\n+\t\t\t struct repository *repo,\n+\t\t\t const struct commit *commit,\n \t\t\t const char *src, size_t len,\n \t\t\t struct strbuf *buf, struct pretty_print_context *ctx)\n {\n@@ -59,7 +61,7 @@ static void format_subst(const struct commit *commit,\n \t\tstrbuf_add(&fmt, b + 8, c - b - 8);\n \n \t\tstrbuf_add(buf, src, b - src);\n-\t\tformat_commit_message(commit, fmt.buf, buf, ctx);\n+\t\trepo_format_commit_message(repo, commit, fmt.buf, buf, ctx);\n \t\tlen -= c + 1 - src;\n \t\tsrc  = c + 1;\n \t}\n@@ -68,7 +70,9 @@ static void format_subst(const struct commit *commit,\n \tfree(to_free);\n }\n \n-static void *object_file_to_archive(const struct archiver_args *args,\n+static void *object_file_to_archive(\n+\t\t\t\t    struct repository *repo,\n+\t\t\t\t    const struct archiver_args *args,\n \t\t\t\t    const char *path,\n \t\t\t\t    const struct object_id *oid,\n \t\t\t\t    unsigned int mode,\n@@ -84,15 +88,15 @@ static void *object_file_to_archive(const struct archiver_args *args,\n \t\t\t       (args->tree ? &args->tree->object.oid : NULL), oid);\n \n \tpath += args->baselen;\n-\tbuffer = read_object_file(oid, type, sizep);\n+\tbuffer = repo_read_object_file(repo, oid, type, sizep);\n \tif (buffer && S_ISREG(mode)) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tsize_t size = 0;\n \n \t\tstrbuf_attach(&buf, buffer, *sizep, *sizep + 1);\n-\t\tconvert_to_working_tree(args->repo->index, path, buf.buf, buf.len, &buf, &meta);\n+\t\tconvert_to_working_tree(repo->index, path, buf.buf, buf.len, &buf, &meta);\n \t\tif (commit)\n-\t\t\tformat_subst(commit, buf.buf, buf.len, &buf, args->pretty_ctx);\n+\t\t\tformat_subst(repo, commit, buf.buf, buf.len, &buf, args->pretty_ctx);\n \t\tbuffer = strbuf_detach(&buf, &size);\n \t\t*sizep = size;\n \t}\n@@ -186,7 +190,7 @@ static int write_archive_entry(\n \t    size > big_file_threshold)\n \t\treturn write_entry(repo, args, oid, path.buf, path.len, mode, NULL, size);\n \n-\tbuffer = object_file_to_archive(args, path.buf, oid, mode, &type, &size);\n+\tbuffer = object_file_to_archive(repo, args, path.buf, oid, mode, &type, &size);\n \tif (!buffer)\n \t\treturn error(_(\"cannot read '%s'\"), oid_to_hex(oid));\n \terr = write_entry(repo, args, oid, path.buf, path.len, mode, buffer, size);\n@@ -313,8 +317,8 @@ int write_archive_entries(\n \t\tmemset(&opts, 0, sizeof(opts));\n \t\topts.index_only = 1;\n \t\topts.head_idx = -1;\n-\t\topts.src_index = args->repo->index;\n-\t\topts.dst_index = args->repo->index;\n+\t\topts.src_index = repo->index;\n+\t\topts.dst_index = repo->index;\n \t\topts.fn = oneway_merge;\n \t\tinit_tree_desc(&t, args->tree->buffer, args->tree->size);\n \t\tif (unpack_trees(1, &t, &opts))\n@@ -322,7 +326,7 @@ int write_archive_entries(\n \t\tgit_attr_set_direction(GIT_ATTR_INDEX);\n \t}\n \n-\terr = read_tree(args->repo, args->tree,\n+\terr = read_tree(repo, args->tree,\n \t\t\t&args->pathspec,\n \t\t\tqueue_or_write_archive_entry,\n \t\t\t&context);\n@@ -412,7 +416,7 @@ static int reject_entry(\n \treturn ret;\n }\n \n-static int path_exists(struct archiver_args *args, const char *path)\n+static int path_exists(struct repository *repo, struct archiver_args *args, const char *path)\n {\n \tconst char *paths[] = { path, NULL };\n \tstruct path_exists_context ctx;\n@@ -421,14 +425,16 @@ static int path_exists(struct archiver_args *args, const char *path)\n \tctx.args = args;\n \tparse_pathspec(&ctx.pathspec, 0, 0, \"\", paths);\n \tctx.pathspec.recursive = 1;\n-\tret = read_tree(args->repo, args->tree,\n+\tret = read_tree(repo, args->tree,\n \t\t\t&ctx.pathspec,\n \t\t\treject_entry, &ctx);\n \tclear_pathspec(&ctx.pathspec);\n \treturn ret != 0;\n }\n \n-static void parse_pathspec_arg(const char **pathspec,\n+static void parse_pathspec_arg(\n+\t\tstruct repository *repo,\n+\t\tconst char **pathspec,\n \t\tstruct archiver_args *ar_args)\n {\n \t/*\n@@ -442,14 +448,16 @@ static void parse_pathspec_arg(const char **pathspec,\n \tar_args->pathspec.recursive = 1;\n \tif (pathspec) {\n \t\twhile (*pathspec) {\n-\t\t\tif (**pathspec && !path_exists(ar_args, *pathspec))\n+\t\t\tif (**pathspec && !path_exists(repo, ar_args, *pathspec))\n \t\t\t\tdie(_(\"pathspec '%s' did not match any files\"), *pathspec);\n \t\t\tpathspec++;\n \t\t}\n \t}\n }\n \n-static void parse_treeish_arg(const char **argv,\n+static void parse_treeish_arg(\n+\t\tstruct repository *repo,\n+\t\tconst char **argv,\n \t\tstruct archiver_args *ar_args, const char *prefix,\n \t\tint remote)\n {\n@@ -475,7 +483,7 @@ static void parse_treeish_arg(const char **argv,\n \tif (get_oid(name, &oid))\n \t\tdie(_(\"not a valid object name: %s\"), name);\n \n-\tcommit = lookup_commit_reference_gently(ar_args->repo, &oid, 1);\n+\tcommit = lookup_commit_reference_gently(repo, &oid, 1);\n \tif (commit) {\n \t\tcommit_oid = &commit->object.oid;\n \t\tarchive_time = commit->date;\n@@ -484,7 +492,7 @@ static void parse_treeish_arg(const char **argv,\n \t\tarchive_time = time(NULL);\n \t}\n \n-\ttree = parse_tree_indirect(&oid);\n+\ttree = repo_parse_tree_indirect(repo, &oid);\n \tif (!tree)\n \t\tdie(_(\"not a tree object: %s\"), oid_to_hex(&oid));\n \n@@ -493,14 +501,14 @@ static void parse_treeish_arg(const char **argv,\n \t\tunsigned short mode;\n \t\tint err;\n \n-\t\terr = get_tree_entry(ar_args->repo,\n+\t\terr = get_tree_entry(repo,\n \t\t\t\t     &tree->object.oid,\n \t\t\t\t     prefix, &tree_oid,\n \t\t\t\t     &mode);\n \t\tif (err || !S_ISDIR(mode))\n \t\t\tdie(_(\"current working directory is untracked\"));\n \n-\t\ttree = parse_tree_indirect(&tree_oid);\n+\t\ttree = repo_parse_tree_indirect(repo, &tree_oid);\n \t}\n \tar_args->refname = ref;\n \tar_args->tree = tree;\n@@ -701,7 +709,6 @@ int write_archive(int argc, const char **argv, const char *prefix,\n \tctx.abbrev = DEFAULT_ABBREV;\n \tctx.describe_status = &describe_status;\n \targs.pretty_ctx = &ctx;\n-\targs.repo = repo;\n \targs.prefix = prefix;\n \tstring_list_init_dup(&args.extra_files);\n \targc = parse_archive_args(argc, argv, &ar, &args, name_hint, remote);\n@@ -714,8 +721,8 @@ int write_archive(int argc, const char **argv, const char *prefix,\n \t\tsetup_git_directory();\n \t}\n \n-\tparse_treeish_arg(argv, &args, prefix, remote);\n-\tparse_pathspec_arg(argv + 1, &args);\n+\tparse_treeish_arg(repo, argv, &args, prefix, remote);\n+\tparse_pathspec_arg(repo, argv + 1, &args);\n \n \trc = ar->write_archive(ar, repo, &args);\n \ndiff --git a/archive.h b/archive.h\nindex bfbbd3274bd..540a3b12130 100644\n--- a/archive.h\n+++ b/archive.h\n@@ -8,7 +8,6 @@ struct repository;\n struct pretty_print_context;\n \n struct archiver_args {\n-\tstruct repository *repo;\n \tchar *refname;\n \tconst char *prefix;\n \tconst char *base;\n-- \ngitgitgadget\n\n"},{"id":"465064","messageId":"f88ebbaf17cbf1a0b57336430bd43ade94406f38.1665973401.git.gitgitgadget@gmail.com","threadId":"58613","inReplyTo":"pull.1359.v3.git.git.1665973401.gitgitgadget@gmail.com","subject":"[PATCH v3 9/9] archive: add tests for git archive --recurse-submodules","fromName":"Heather Lapointe via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-17T02:23:21Z","receivedAt":"2022-10-17T02:23:56Z","isPatch":true,"sender":{"key":"alpha@alphaservcomputing.solutions","avatar":null},"body":"From: Heather Lapointe <alpha@alphaservcomputing.solutions>\n\nEnsuring functionality works with and without submodules.\nWe expect --recurse-submodules to fail if there are uninitialized submodules\npresent.\n\nSigned-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n---\n archive.c                     |  2 +-\n t/t5005-archive-submodules.sh | 83 +++++++++++++++++++++++++++++++++++\n 2 files changed, 84 insertions(+), 1 deletion(-)\n create mode 100755 t/t5005-archive-submodules.sh\n\ndiff --git a/archive.c b/archive.c\nindex f81ef741487..b0a3181f7f5 100644\n--- a/archive.c\n+++ b/archive.c\n@@ -179,7 +179,7 @@ static int write_archive_entry(\n \t\terr = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, 0);\n \t\tif (err)\n \t\t\treturn err;\n-\t\treturn (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);\n+\t\treturn READ_TREE_RECURSIVE;\n \t}\n \n \tif (args->verbose)\ndiff --git a/t/t5005-archive-submodules.sh b/t/t5005-archive-submodules.sh\nnew file mode 100755\nindex 00000000000..aad6cfd1082\n--- /dev/null\n+++ b/t/t5005-archive-submodules.sh\n@@ -0,0 +1,83 @@\n+#!/bin/sh\n+\n+test_description='git archive --recurse-submodules test'\n+\n+. ./test-lib.sh\n+\n+check_tar() {\n+\ttarfile=$1.tar\n+\tlistfile=$1.lst\n+\tdir=$1\n+\tdir_with_prefix=$dir/$2\n+\n+\ttest_expect_success ' extract tar archive' '\n+\t\t(mkdir $dir && cd $dir && \"$TAR\" xf -) <$tarfile\n+\t'\n+}\n+\n+check_added() {\n+\tdir=$1\n+\tpath_in_fs=$2\n+\tpath_in_archive=$3\n+\n+\ttest_expect_success \" validate extra file $path_in_archive\" '\n+\t\ttest -f $dir/$path_in_archive &&\n+\t\tdiff -r $path_in_fs $dir/$path_in_archive\n+\t'\n+}\n+\n+check_not_added() {\n+\tdir=$1\n+\tpath_in_archive=$2\n+\n+\ttest_expect_success \" validate unpresent file $path_in_archive\" '\n+\t\t! test -f $dir/$path_in_archive &&\n+\t\t! test -d $dir/$path_in_archive\n+\t'\n+}\n+\n+test_expect_success 'setup' '\n+\trm -rf repo_with_submodules submodule1 uninited_repo_with_submodules &&\n+\tgit init repo_with_submodules &&\n+\tgit init submodule1 &&\n+\t(\n+\t\tcd submodule1 &&\n+\t\techo \"dir1/sub1/file1.txt\" > \"file1.txt\" &&\n+\t\tgit add file1.txt &&\n+\t\tgit commit -m \"initialize with file1.txt\"\n+\t) &&\n+\t(\n+\t    cd repo_with_submodules &&\n+\t    echo \"file2\" > file2.txt &&\n+\t    git add file2.txt &&\n+\t    git commit -m \"initialize with file2.txt\" &&\n+\t    mkdir -p dir1 &&\n+\t    git submodule add ../submodule1 dir1/sub1 &&\n+\t    git commit -m \"add submodule1\"\n+\t) &&\n+\tgit clone repo_with_submodules uninited_repo_with_submodules\n+'\n+\n+test_expect_success 'archive without recurse, non-init' '\n+\tgit -C uninited_repo_with_submodules archive -v HEAD >b.tar\n+'\n+\n+check_tar b\n+check_added b uninited_repo_with_submodules/file2.txt file2.txt\n+check_not_added b uninited_repo_with_submodules/dir1/sub1/file1.txt\n+\n+# It is expected that --recurse-submodules will not work if submodules are not\n+# initialized.\n+test_expect_success 'archive with recurse, non-init' '\n+\t! git -C uninited_repo_with_submodules archive --recurse-submodules -v HEAD >b2-err.tar\n+'\n+\n+test_expect_success 'archive with recurse, init' '\n+\tgit -C repo_with_submodules archive --recurse-submodules -v HEAD >b3.tar\n+'\n+\n+check_tar b3\n+check_added b3 repo_with_submodules/file2.txt file2.txt\n+check_added b3 repo_with_submodules/dir1/sub1/file1.txt dir1/sub1/file1.txt\n+\n+test_done\n-- \ngitgitgadget\n"},{"id":"465065","messageId":"4672e3d958625cd76eb8056ab434e9a37f52661e.1665973401.git.gitgitgadget@gmail.com","threadId":"58613","inReplyTo":"pull.1359.v3.git.git.1665973401.gitgitgadget@gmail.com","subject":"[PATCH v3 8/9] archive: add --recurse-submodules to git-archive command","fromName":"Heather Lapointe via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-17T02:23:20Z","receivedAt":"2022-10-17T02:23:58Z","isPatch":true,"sender":{"key":"alpha@alphaservcomputing.solutions","avatar":null},"body":"From: Heather Lapointe <alpha@alphaservcomputing.solutions>\n\nThis makes it possible to include submodule contents in an archive command.\n\nThe default behavior remains the same, do not write submodule contents\nto the resulting archive.\n\nSigned-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n---\n Documentation/git-archive.txt |  6 +++++-\n archive.c                     | 36 +++++++++++++++++++++++++++++++++--\n archive.h                     |  1 +\n 3 files changed, 40 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-archive.txt b/Documentation/git-archive.txt\nindex 60c040988bb..22f54428b98 100644\n--- a/Documentation/git-archive.txt\n+++ b/Documentation/git-archive.txt\n@@ -10,7 +10,8 @@ SYNOPSIS\n --------\n [verse]\n 'git archive' [--format=<fmt>] [--list] [--prefix=<prefix>/] [<extra>]\n-\t      [-o <file> | --output=<file>] [--worktree-attributes]\n+\t      [-o <file> | --output=<file>]\n+\t      [--recurse-submodules] [--worktree-attributes]\n \t      [--remote=<repo> [--exec=<git-upload-archive>]] <tree-ish>\n \t      [<path>...]\n \n@@ -82,6 +83,9 @@ The file mode is limited to a regular file, and the option may be\n subject to platform-dependent command-line limits. For non-trivial\n cases, write an untracked file and use `--add-file` instead.\n \n+--recurse-submodules\n+\tInclude submodules recursively in archive.\n+\n --worktree-attributes::\n \tLook for attributes in .gitattributes files in the working tree\n \tas well (see <<ATTRIBUTES>>).\ndiff --git a/archive.c b/archive.c\nindex 34549d849f1..f81ef741487 100644\n--- a/archive.c\n+++ b/archive.c\n@@ -10,6 +10,7 @@\n #include \"unpack-trees.h\"\n #include \"dir.h\"\n #include \"quote.h\"\n+#include \"submodule.h\"\n \n static char const * const archive_usage[] = {\n \tN_(\"git archive [<options>] <tree-ish> [<path>...]\"),\n@@ -213,6 +214,25 @@ static void queue_directory(const struct object_id *oid,\n \toidcpy(&d->oid, oid);\n }\n \n+static void queue_submodule(\n+\t\tstruct repository *superproject,\n+\t\tconst struct object_id *oid,\n+\t\tstruct strbuf *base, const char *filename,\n+\t\tunsigned mode, struct archiver_context *c)\n+{\n+\tstruct repository subrepo;\n+\n+\tif (repo_submodule_init(&subrepo, superproject, filename, null_oid()))\n+\t\treturn;\n+\n+\tif (repo_read_index(&subrepo) < 0)\n+\t\tdie(\"index file corrupt\");\n+\n+    queue_directory(oid, base, filename, mode, c);\n+\n+\trepo_clear(&subrepo);\n+}\n+\n static int write_directory(\n \t\tstruct repository *repo,\n \t\tstruct archiver_context *c)\n@@ -228,9 +248,11 @@ static int write_directory(\n \t\twrite_directory(repo, c) ||\n \t\twrite_archive_entry(repo, &d->oid, d->path, d->baselen,\n \t\t\t\t    d->path + d->baselen, d->mode,\n-\t\t\t\t    c) != READ_TREE_RECURSIVE;\n+\t\t\t\t    c);\n \tfree(d);\n-\treturn ret ? -1 : 0;\n+\tif (ret == READ_TREE_RECURSIVE)\n+\t\treturn 0;\n+\treturn ret;\n }\n \n static int queue_or_write_archive_entry(\n@@ -263,6 +285,11 @@ static int queue_or_write_archive_entry(\n \t\t\treturn 0;\n \t\tqueue_directory(oid, base, filename, mode, c);\n \t\treturn READ_TREE_RECURSIVE;\n+\t} else if (c->args->recurse_submodules && S_ISGITLINK(mode)) {\n+\t\tif (is_submodule_active(r, filename)) {\n+\t\t\tqueue_submodule(r, oid, base, filename, mode, c);\n+\t\t\treturn READ_TREE_RECURSIVE;\n+\t\t}\n \t}\n \n \tif (write_directory(r, c))\n@@ -446,6 +473,7 @@ static void parse_pathspec_arg(\n \t\t       PATHSPEC_PREFER_FULL,\n \t\t       \"\", pathspec);\n \tar_args->pathspec.recursive = 1;\n+\tar_args->pathspec.recurse_submodules = ar_args->recurse_submodules;\n \tif (pathspec) {\n \t\twhile (*pathspec) {\n \t\t\tif (**pathspec && !path_exists(repo, ar_args, *pathspec))\n@@ -609,6 +637,7 @@ static int parse_archive_args(int argc, const char **argv,\n \tint verbose = 0;\n \tint i;\n \tint list = 0;\n+\tint recurse_submodules = 0;\n \tint worktree_attributes = 0;\n \tstruct option opts[] = {\n \t\tOPT_GROUP(\"\"),\n@@ -623,6 +652,8 @@ static int parse_archive_args(int argc, const char **argv,\n \t\t  add_file_cb, (intptr_t)&base },\n \t\tOPT_STRING('o', \"output\", &output, N_(\"file\"),\n \t\t\tN_(\"write the archive to this file\")),\n+\t\tOPT_BOOL(0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\tN_(\"include submodules in archive\")),\n \t\tOPT_BOOL(0, \"worktree-attributes\", &worktree_attributes,\n \t\t\tN_(\"read .gitattributes in working directory\")),\n \t\tOPT__VERBOSE(&verbose, N_(\"report archived files on stderr\")),\n@@ -686,6 +717,7 @@ static int parse_archive_args(int argc, const char **argv,\n \targs->verbose = verbose;\n \targs->base = base;\n \targs->baselen = strlen(base);\n+\targs->recurse_submodules = recurse_submodules;\n \targs->worktree_attributes = worktree_attributes;\n \n \treturn argc;\ndiff --git a/archive.h b/archive.h\nindex 540a3b12130..1b21484dda6 100644\n--- a/archive.h\n+++ b/archive.h\n@@ -18,6 +18,7 @@ struct archiver_args {\n \ttimestamp_t time;\n \tstruct pathspec pathspec;\n \tunsigned int verbose : 1;\n+\tunsigned int recurse_submodules : 1;\n \tunsigned int worktree_attributes : 1;\n \tunsigned int convert : 1;\n \tint compression_level;\n-- \ngitgitgadget\n\n"},{"id":"465097","messageId":"xmqq4jw237jm.fsf@gitster.g","threadId":"58613","inReplyTo":"79959a54eb4c1a0812b1f4643530069a63e549f4.1665973401.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 1/9] tree: do not use the_repository for tree traversal methods.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-17T13:26:37Z","receivedAt":"2022-10-17T13:26:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Alphadelta14 via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> From: Alphadelta14 <alpha@alphaservcomputing.solutions>\n\nI'll fix this line to match all the other patches in the series\nbefore applying.\n\n> Expect that tree walking may switch repository contexts for cases\n> such as submodules.\n> Added compatibility macros for existing cases.\n>\n> Annotate an existing issue where repo is wrong when traversing.\n>\n> Signed-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n> ---\n\n\n> @@ -58,7 +58,11 @@ int read_tree_at(struct repository *r,\n>  \t\t\t\t    oid_to_hex(&entry.oid),\n>  \t\t\t\t    base->buf, entry.path);\n>  \n> -\t\t\tif (parse_commit(commit))\n> +\t\t\t// FIXME: This is the wrong repo instance (it refers to the superproject)\n> +\t\t\t// it will always fail as is (will fix in later patch)\n> +\t\t\t// This current codepath isn't executed by any existing callbacks\n> +\t\t\t// so it wouldn't show up as an issue at this time.\n\n\t/*\n\t * We write our multi-line comments\n\t * this way.\n\t */\n\nMy suspicion is that the if/else if/ cascade for GITLINK assumes\nthat the caller earlier did add_submodule_odb() to make sure any\nobject it needs should be available via the_repository->objects\nobject store.  If your caller (presumably \"archive that is trying to\nlearn the --recurse-submodules option\") hasn't learned to do so yet\nat this step, it is understandable if it fails.\n\n> +\t\t\tif (repo_parse_commit(r, commit))\n>  \t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n>  \t\t\t\t    oid_to_hex(&entry.oid),\n>  \t\t\t\t    base->buf, entry.path);\n\n> +#ifndef NO_THE_REPOSITORY_COMPATIBILITY_MACROS\n> +#define parse_tree(tree) repo_parse_tree(the_repository, tree)\n> +#define parse_tree_gently(tree, quiet_on_missing) repo_parse_tree_gently(the_repository, tree, quiet_on_missing)\n> +#define parse_tree_indirect(oid) repo_parse_tree_indirect(the_repository, oid)\n> +#endif\n\nGood.\n"},{"id":"465099","messageId":"b8e2be95-0d42-aaae-4d9f-1825b62707ba@dunelm.org.uk","threadId":"58613","inReplyTo":"9a07c6932f4c7ef844df1fc4f5b6b9feb1810135.1665973401.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 3/9] tree: increase test coverage for tree.c","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-10-17T13:34:22Z","receivedAt":"2022-10-17T13:34:25Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Heather\n\nOn 17/10/2022 03:23, Heather Lapointe via GitGitGadget wrote:\n> From: Heather Lapointe <alpha@alphaservcomputing.solutions>\n> \n> This highlights some buggy behavior from read_tree for submodules that\n> was not being executed.\n\nThe commit message should explain the reason behind the change being \nmade. In this case it would be helpful to give an overview of what the \nbug is you're testing for. Given the description I was expecting to see \nsome failing tests that are fixed by a later patch but that doesn't seem \nto be the case, so I'm wondering what these tests do.\n\n> This introduces a test-tool tree-read-tree-at command\n> (the complex name is because it is not related to the read-tree command).\n\nIt would also be helpful to explain why we cannot reproduce the bug with \n\"git read-tree --recurse-submodules\"\n\nBest Wishes\n\nPhillip\n\n> Signed-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n> ---\n>   Makefile                          |  1 +\n>   t/helper/test-tool.c              |  1 +\n>   t/helper/test-tool.h              |  1 +\n>   t/helper/test-tree-read-tree-at.c | 40 +++++++++++++++++++\n>   t/t1023-tree-read-tree-at.sh      | 65 +++++++++++++++++++++++++++++++\n>   5 files changed, 108 insertions(+)\n>   create mode 100644 t/helper/test-tree-read-tree-at.c\n>   create mode 100755 t/t1023-tree-read-tree-at.sh\n> \n> diff --git a/Makefile b/Makefile\n> index 6bfb62cbe94..52d17ca7276 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -788,6 +788,7 @@ TEST_BUILTINS_OBJS += test-submodule-nested-repo-config.o\n>   TEST_BUILTINS_OBJS += test-submodule.o\n>   TEST_BUILTINS_OBJS += test-subprocess.o\n>   TEST_BUILTINS_OBJS += test-trace2.o\n> +TEST_BUILTINS_OBJS += test-tree-read-tree-at.o\n>   TEST_BUILTINS_OBJS += test-urlmatch-normalization.o\n>   TEST_BUILTINS_OBJS += test-userdiff.o\n>   TEST_BUILTINS_OBJS += test-wildmatch.o\n> diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\n> index d1d013bcd92..a8a9bedec5f 100644\n> --- a/t/helper/test-tool.c\n> +++ b/t/helper/test-tool.c\n> @@ -82,6 +82,7 @@ static struct test_cmd cmds[] = {\n>   \t{ \"submodule-nested-repo-config\", cmd__submodule_nested_repo_config },\n>   \t{ \"subprocess\", cmd__subprocess },\n>   \t{ \"trace2\", cmd__trace2 },\n> +\t{ \"tree-read-tree-at\", cmd__tree_read_tree_at },\n>   \t{ \"userdiff\", cmd__userdiff },\n>   \t{ \"urlmatch-normalization\", cmd__urlmatch_normalization },\n>   \t{ \"xml-encode\", cmd__xml_encode },\n> diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\n> index 6b46b6444b6..409fddfaeb8 100644\n> --- a/t/helper/test-tool.h\n> +++ b/t/helper/test-tool.h\n> @@ -76,6 +76,7 @@ int cmd__submodule_config(int argc, const char **argv);\n>   int cmd__submodule_nested_repo_config(int argc, const char **argv);\n>   int cmd__subprocess(int argc, const char **argv);\n>   int cmd__trace2(int argc, const char **argv);\n> +int cmd__tree_read_tree_at(int argc, const char **argv);\n>   int cmd__userdiff(int argc, const char **argv);\n>   int cmd__urlmatch_normalization(int argc, const char **argv);\n>   int cmd__xml_encode(int argc, const char **argv);\n> diff --git a/t/helper/test-tree-read-tree-at.c b/t/helper/test-tree-read-tree-at.c\n> new file mode 100644\n> index 00000000000..bba759bb264\n> --- /dev/null\n> +++ b/t/helper/test-tree-read-tree-at.c\n> @@ -0,0 +1,40 @@\n> +/* This tests tree.c's read_tree / read_tree_at.\n> +We call it tree-read-tree-at to disambiguate with the read-tree tool.\n> +*/\n> +#include \"cache.h\"\n> +#include \"pathspec.h\"\n> +#include \"test-tool.h\"\n> +#include \"tree.h\"\n> +\n> +static int test_handle_entry(const struct object_id *oid,\n> +\t\tstruct strbuf *base, const char *filename,\n> +\t\tunsigned mode, void *context UNUSED) {\n> +\tprintf(\"%i %s %s%s\\n\", mode, oid_to_hex(oid), base->buf, filename);\n> +\tif (S_ISDIR(mode) || S_ISGITLINK(mode)) {\n> +\t\treturn READ_TREE_RECURSIVE;\n> +\t}\n> +\treturn 0;\n> +}\n> +\n> +int cmd__tree_read_tree_at(int argc UNUSED, const char **argv)\n> +{\n> +\tstruct pathspec pathspec;\n> +\tstruct tree *tree;\n> +\tstruct repository *repo;\n> +\tstruct object_id oid;\n> +\n> +\tsetup_git_directory();\n> +\trepo = the_repository;\n> +\tassert(repo);\n> +\n> +\tparse_pathspec(&pathspec, 0,\n> +\t\t       PATHSPEC_PREFER_FULL,\n> +\t\t       \"\", argv);\n> +\n> +\tassert(repo_get_oid(repo, \"HEAD\", &oid) == 0);\n> +\ttree = repo_parse_tree_indirect(repo, &oid);\n> +\tassert(tree);\n> +\tpathspec.recurse_submodules = 1;\n> +\tread_tree(repo, tree, &pathspec, test_handle_entry, NULL);\n> +\treturn 0;\n> +}\n> diff --git a/t/t1023-tree-read-tree-at.sh b/t/t1023-tree-read-tree-at.sh\n> new file mode 100755\n> index 00000000000..9e5ce3abb4b\n> --- /dev/null\n> +++ b/t/t1023-tree-read-tree-at.sh\n> @@ -0,0 +1,65 @@\n> +#!/bin/sh\n> +\n> +# tests for tree.c (not read-tree.c)\n> +test_description='Test read_tree / read_tree_at'\n> +. ./test-lib.sh\n> +\n> +test_expect_success 'read_tree basic' '\n> +\trm -rf walk_tree_basic &&\n> +\tgit init walk_tree_basic &&\n> +\t(\n> +\t\tcd walk_tree_basic &&\n> +\t\tset -x &&\n> +\n> +\t\tmkdir -p dir1/dirA &&\n> +\t\tmkdir -p dir1/dirB &&\n> +\t\tmkdir -p dir2 &&\n> +\t\techo \"file1\" > file1.txt &&\n> +\t\techo \"file2\" > file2.txt &&\n> +\t\t# uncommitted\n> +\t\techo \"file3\" > file3.txt &&\n> +\n> +\t\techo \"file1A1\" > dir1/dirA/file1.txt &&\n> +\t\tgit add file1.txt file2.txt dir1/dirA/file1.txt &&\n> +\t\tgit commit -m \"initial commit\" &&\n> +\n> +\t\ttest-tool tree-read-tree-at . > walk1.txt &&\n> +\t\tgrep \" file1.txt\" walk1.txt &&\n> +\t\t! grep \" file3.txt\" walk1.txt &&\n> +\t\t! grep \" dir1/dirB\" walk1.txt &&\n> +\t\tgrep \" dir1/dirA/file1.txt\" walk1.txt\n> +\t)\n> +'\n> +\n> +test_expect_success 'read_tree submodules' '\n> +\trm -rf walk_tree_submodules &&\n> +\tgit init submodule1 &&\n> +\t(\n> +\t\tcd submodule1 &&\n> +\t\tmkdir -p dir1/dirA &&\n> +\t\techo \"dir2/sub1/file1.txt\" > file1.txt &&\n> +\t\techo \"dir2/sub1/file1A1.txt\" > dir1/dirA/file1.txt &&\n> +\t\tgit add file1.txt dir1/dirA/file1.txt &&\n> +\t\tgit commit -m \"initial commit\"\n> +\t) &&\n> +\tgit init walk_tree_submodules &&\n> +\t(\n> +\t\tcd walk_tree_submodules &&\n> +\n> +\t\tmkdir -p dir2 &&\n> +\t\techo \"file1\" > file1.txt &&\n> +\t\techo \"dir2/file2\" > dir2/file2.txt &&\n> +\t\tgit add file1.txt dir2/file2.txt &&\n> +\t\tgit commit -m \"initial commit\" &&\n> +\n> +\t\tgit submodule add ../submodule1 dir2/sub1 &&\n> +\t\tgit commit -m \"add submodule1\" &&\n> +\n> +\t\ttest-tool tree-read-tree-at . > walk2.txt &&\n> +\t\tgrep \" file1.txt\" walk2.txt &&\n> +\t\tgrep \" dir2/sub1/file1.txt\" walk2.txt &&\n> +\t\tgrep \" dir2/sub1/dir1/dirA/file1.txt\" walk2.txt\n> +\t)\n> +'\n> +\n> +test_done\n"},{"id":"465100","messageId":"xmqqy1te1sii.fsf@gitster.g","threadId":"58613","inReplyTo":"9a07c6932f4c7ef844df1fc4f5b6b9feb1810135.1665973401.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 3/9] tree: increase test coverage for tree.c","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-17T13:36:37Z","receivedAt":"2022-10-17T13:36:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Heather Lapointe via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> diff --git a/t/t1023-tree-read-tree-at.sh b/t/t1023-tree-read-tree-at.sh\n> new file mode 100755\n> index 00000000000..9e5ce3abb4b\n> --- /dev/null\n> +++ b/t/t1023-tree-read-tree-at.sh\n> @@ -0,0 +1,65 @@\n> +#!/bin/sh\n> +\n> +# tests for tree.c (not read-tree.c)\n> +test_description='Test read_tree / read_tree_at'\n> +. ./test-lib.sh\n> +\n> +test_expect_success 'read_tree basic' '\n> +\trm -rf walk_tree_basic &&\n> +\tgit init walk_tree_basic &&\n> +\t(\n> +\t\tcd walk_tree_basic &&\n> +\t\tset -x &&\n\nDo we need this, when we have '-x' option alongside with '-v', '-i',\nand '-d' available in the test harness already?\n\n> +\t\tmkdir -p dir1/dirA &&\n> +\t\tmkdir -p dir1/dirB &&\n> +\t\tmkdir -p dir2 &&\n\nCan't we have these three done by the same single \"mkdir -p\" process?\n\n> +\t\techo \"file1\" > file1.txt &&\n> +\t\techo \"file2\" > file2.txt &&\n\nLose the SP between redirection operator \">\" and its target, i.e.\n\n\t\techo file1 >file1.txt\n\ncf. Documentation/CodingGuidelines\n\nAlso you do not necessarily have to have dq around a single token.\n\n> +\t\t# uncommitted\n> +\t\techo \"file3\" > file3.txt &&\n> +\n> +\t\techo \"file1A1\" > dir1/dirA/file1.txt &&\n> +\t\tgit add file1.txt file2.txt dir1/dirA/file1.txt &&\n> +\t\tgit commit -m \"initial commit\" &&\n> +\n> +\t\ttest-tool tree-read-tree-at . > walk1.txt &&\n> +\t\tgrep \" file1.txt\" walk1.txt &&\n> +\t\t! grep \" file3.txt\" walk1.txt &&\n> +\t\t! grep \" dir1/dirB\" walk1.txt &&\n> +\t\tgrep \" dir1/dirA/file1.txt\" walk1.txt\n> +\t)\n> +'\n> +\n> +test_expect_success 'read_tree submodules' '\n> +\trm -rf walk_tree_submodules &&\n\nCurious why the above does not clean \"submodule1\", too.  After all,\nall the \"rm -rf\" we saw in this script above are removing what its\nearlier steps would never have created (but will create), just in\ncase.  Why not do the same?\n\nIf the pattern is to \"remove what we will need to create immediately\nbefore we actually try to create, just in case\", then shouldn't the\nabove be removing \"submodule1\", and we should have another \"rm -rf\"\nfor \"walk_tree_submodules\" immediately before we do \"git init\" on\nit several lines below?\n\n> +\tgit init submodule1 &&\n> +\t(\n> +\t\tcd submodule1 &&\n> +\t\tmkdir -p dir1/dirA &&\n> +\t\techo \"dir2/sub1/file1.txt\" > file1.txt &&\n> +\t\techo \"dir2/sub1/file1A1.txt\" > dir1/dirA/file1.txt &&\n> +\t\tgit add file1.txt dir1/dirA/file1.txt &&\n> +\t\tgit commit -m \"initial commit\"\n> +\t) &&\n> +\tgit init walk_tree_submodules &&\n> +\t(\n> +\t\tcd walk_tree_submodules &&\n> +\n> +\t\tmkdir -p dir2 &&\n> +\t\techo \"file1\" > file1.txt &&\n> +\t\techo \"dir2/file2\" > dir2/file2.txt &&\n> +\t\tgit add file1.txt dir2/file2.txt &&\n> +\t\tgit commit -m \"initial commit\" &&\n> +\n> +\t\tgit submodule add ../submodule1 dir2/sub1 &&\n> +\t\tgit commit -m \"add submodule1\" &&\n> +\n> +\t\ttest-tool tree-read-tree-at . > walk2.txt &&\n> +\t\tgrep \" file1.txt\" walk2.txt &&\n> +\t\tgrep \" dir2/sub1/file1.txt\" walk2.txt &&\n> +\t\tgrep \" dir2/sub1/dir1/dirA/file1.txt\" walk2.txt\n> +\t)\n> +'\n> +\n> +test_done\n"},{"id":"465101","messageId":"bf890f94-10b7-e344-fe20-8ce129923c4f@dunelm.org.uk","threadId":"58613","inReplyTo":"d3d1738e670d5dbf1378fc5c3209b2e98234a771.1665973401.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 4/9] tree: handle submodule case for read_tree_at properly","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-10-17T13:48:04Z","receivedAt":"2022-10-17T13:48:20Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Heather\n\nOn 17/10/2022 03:23, Heather Lapointe via GitGitGadget wrote:\n> From: Heather Lapointe <alpha@alphaservcomputing.solutions>\n> \n> This supports traversal into an actual submodule for read_tree_at.\n> The logic is blocked on pathspec->recurse_submodules now,\n\nI'm struggling to understand what this is saying.\n\n> but previously hadn't been executed due to all fn() cases\n> returning early for submodules.\n> \n> Signed-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n> ---\n>   tree.c | 88 ++++++++++++++++++++++++++++++++++++++++------------------\n>   1 file changed, 61 insertions(+), 27 deletions(-)\n> \n> diff --git a/tree.c b/tree.c\n> index 13f9173d45e..2a087c010f9 100644\n> --- a/tree.c\n> +++ b/tree.c\n> @@ -8,6 +8,7 @@\n>   #include \"alloc.h\"\n>   #include \"tree-walk.h\"\n>   #include \"repository.h\"\n> +#include \"pathspec.h\"\n>   \n>   const char *tree_type = \"tree\";\n>   \n> @@ -47,40 +48,73 @@ int read_tree_at(struct repository *r,\n>   \t\t\treturn -1;\n>   \t\t}\n>   \n> -\t\tif (S_ISDIR(entry.mode))\n> +\t\tif (S_ISDIR(entry.mode)) {\n>   \t\t\toidcpy(&oid, &entry.oid);\n> -\t\telse if (S_ISGITLINK(entry.mode)) {\n> -\t\t\tstruct commit *commit;\n>   \n> -\t\t\tcommit = lookup_commit(r, &entry.oid);\n> +\t\t\tlen = tree_entry_len(&entry);\n> +\t\t\tstrbuf_add(base, entry.path, len);\n> +\t\t\tstrbuf_addch(base, '/');\n> +\t\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n> +\t\t\t\t\t\tbase, pathspec,\n> +\t\t\t\t\t\tfn, context);\n> +\t\t\tstrbuf_setlen(base, oldlen);\n> +\t\t\tif (retval)\n> +\t\t\t\treturn -1;\n> +\t\t} else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) {\n> +\t\t\tstruct commit *commit;\n> +\t\t\tstruct repository subrepo;\n> +\t\t\tstruct repository* subrepo_p = &subrepo;\n\nNormally we'd just use &subrepo wherever we want a pointer rather than \ndefining an alias like this. For example it is common to see\n\n\tstruct strbuf buf = STRBUF_INIT;\n\n\tstrbuf_add(&buf, \"hello world\");\n\nwe don't define a buf_p variable\n\n> +\t\t\tstruct tree* submodule_tree;\n> +\t\t\tchar *submodule_rel_path;\n> +\t\t\tint name_base_len = 0;\n> +\n> +\t\t\tlen = tree_entry_len(&entry);\n> +\t\t\tstrbuf_add(base, entry.path, len);\n> +\t\t\tsubmodule_rel_path = base->buf;\n> +\t\t\t// repo_submodule_init expects a path relative to submodule_prefix\n\nI found the comments in this section code code helpful, but single line \ncomments should be formatted as\n\t/* single line comment */\n\n> +\t\t\tif (r->submodule_prefix) {\n> +\t\t\t\tname_base_len = strlen(r->submodule_prefix);\n> +\t\t\t\t// we should always expect to start with submodule_prefix\n> +\t\t\t\tassert(!strncmp(submodule_rel_path, r->submodule_prefix, name_base_len));\n\nRather than using assert() we tend to use BUG() as that then provides a \ngrep-able message. It also means that we wont have an out of bounds \naccess if the invariant is violated when compiling with NDEBUG. So we \ncould drop the comment and write\n\n\tif (strncmp(submodule_rel_path, r->submodule_prefix, name_base_len)\n\t\tBUG(\"missing submodule path prefix\");\n\n> +\t\t\t\t// strip the prefix\n> +\t\t\t\tsubmodule_rel_path += name_base_len;\n> +\t\t\t\t// if submodule_prefix doesn't end with a /, we want to get rid of that too\n\nI think there is a typo here - if the prefix does end with a / then \nwe're dropping it.\n\n> +\t\t\t\tif (is_dir_sep(submodule_rel_path[0])) {\n> +\t\t\t\t\tsubmodule_rel_path++;\n> +\t\t\t\t}\n> +\t\t\t}\n> +\n> +\t\t\tif (repo_submodule_init(subrepo_p, r, submodule_rel_path, null_oid()))\n> +\t\t\t\tdie(\"couldn't init submodule %s\", base->buf);\n> +\n> +\t\t\tif (repo_read_index(subrepo_p) < 0)\n> +\t\t\t\tdie(\"index file corrupt\");\n> +\n> +\t\t\tcommit = lookup_commit(subrepo_p, &entry.oid);\n>   \t\t\tif (!commit)\n> -\t\t\t\tdie(\"Commit %s in submodule path %s%s not found\",\n> +\t\t\t\tdie(\"Commit %s in submodule path %s not found\",\n>   \t\t\t\t    oid_to_hex(&entry.oid),\n> -\t\t\t\t    base->buf, entry.path);\n> -\n> -\t\t\t// FIXME: This is the wrong repo instance (it refers to the superproject)\n> -\t\t\t// it will always fail as is (will fix in later patch)\n> -\t\t\t// This current codepath isn't executed by any existing callbacks\n> -\t\t\t// so it wouldn't show up as an issue at this time.\n> -\t\t\tif (repo_parse_commit(r, commit))\n\nStyle comment for the patch that added this code. Multi-line comments \nshould be formatted as\n\t/*\n\t * Multi-line\n\t * comment\n\t */\n\nBest Wishes\n\nPhillip\n\n> -\t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n> +\t\t\t\t    base->buf);\n> +\n> +\t\t\tif (repo_parse_commit(subrepo_p, commit))\n> +\t\t\t\tdie(\"Invalid commit %s in submodule path %s\",\n>   \t\t\t\t    oid_to_hex(&entry.oid),\n> -\t\t\t\t    base->buf, entry.path);\n> +\t\t\t\t    base->buf);\n>   \n> -\t\t\toidcpy(&oid, get_commit_tree_oid(commit));\n> -\t\t}\n> -\t\telse\n> -\t\t\tcontinue;\n> +\t\t\tsubmodule_tree = repo_get_commit_tree(subrepo_p, commit);\n> +\t\t\toidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);\n>   \n> -\t\tlen = tree_entry_len(&entry);\n> -\t\tstrbuf_add(base, entry.path, len);\n> -\t\tstrbuf_addch(base, '/');\n> -\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n> -\t\t\t\t      base, pathspec,\n> -\t\t\t\t      fn, context);\n> -\t\tstrbuf_setlen(base, oldlen);\n> -\t\tif (retval)\n> -\t\t\treturn -1;\n> +\t\t\tstrbuf_addch(base, '/');\n> +\n> +\t\t\tretval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),\n> +\t\t\t\t\t\tbase, pathspec,\n> +\t\t\t\t\t\tfn, context);\n> +\t\t\tif (retval)\n> +\t\t\t    die(\"failed to read tree for %s\", base->buf);\n> +\t\t\tstrbuf_setlen(base, oldlen);\n> +\t\t\trepo_clear(subrepo_p);\n> +\t\t}\n> +\t\t// else, this is a file (or a submodule, but no pathspec->recurse_submodules)\n>   \t}\n>   \treturn 0;\n>   }\n"},{"id":"465102","messageId":"0f16b5cf-4903-ef26-1fac-982cd09d43b4@dunelm.org.uk","threadId":"58613","inReplyTo":"1b9b049d64fc4ea919c94e57b74a1760b3837892.1665973401.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 6/9] archive: pass repo objects to write_archive handlers","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-10-17T13:50:46Z","receivedAt":"2022-10-17T13:50:54Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Heather\n\nOn 17/10/2022 03:23, Heather Lapointe via GitGitGadget wrote:\n> From: Heather Lapointe <alpha@alphaservcomputing.solutions>\n> \n> Use contextual repos instead of the_repository or args->repo\n> to ensure that submodules will be handled correctly\n> since they use multiple repo instances.\n\nReading this I wondered whether we could get rid of args->repo. Having \nread ahead to the next patch the answer is yes. Perhaps you could expand \nthis commit message to mention that there are still uses of args->repo \nafter this change but they will be removed in the next patch at which \npoint we can remove that member.\n\n> \n> Signed-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n> ---\n>   archive-tar.c | 15 ++++++++++-----\n>   archive-zip.c | 15 +++++++++------\n>   archive.c     | 38 ++++++++++++++++++++++----------------\n>   archive.h     | 14 +++++++++++---\n>   4 files changed, 52 insertions(+), 30 deletions(-)\n> \n> diff --git a/archive-tar.c b/archive-tar.c\n> index 3e4822b6840..5a2d42ff229 100644\n> --- a/archive-tar.c\n> +++ b/archive-tar.c\n> @@ -18,6 +18,7 @@ static unsigned long offset;\n>   static int tar_umask = 002;\n>   \n>   static int write_tar_filter_archive(const struct archiver *ar,\n> +\t\t\t\t\tstruct repository *repo,\n>   \t\t\t\t    struct archiver_args *args);\n>   \n>   /*\n> @@ -246,7 +247,9 @@ static void write_extended_header(struct archiver_args *args,\n>   \twrite_blocked(buffer, size);\n>   }\n>   \n> -static int write_tar_entry(struct archiver_args *args,\n> +static int write_tar_entry(\n> +\t\t\t   struct repository *repo,\n\nThis argument should be on the same line as the function name\n\nBest Wishes\n\nPhillip\n\n> +\t\t\t   struct archiver_args *args,\n>   \t\t\t   const struct object_id *oid,\n>   \t\t\t   const char *path, size_t pathlen,\n>   \t\t\t   unsigned int mode,\n> @@ -316,7 +319,7 @@ static int write_tar_entry(struct archiver_args *args,\n>   \t\tif (buffer)\n>   \t\t\twrite_blocked(buffer, size);\n>   \t\telse\n> -\t\t\terr = stream_blocked(args->repo, oid);\n> +\t\t\terr = stream_blocked(repo, oid);\n>   \t}\n>   \treturn err;\n>   }\n> @@ -422,12 +425,13 @@ static int git_tar_config(const char *var, const char *value, void *cb)\n>   }\n>   \n>   static int write_tar_archive(const struct archiver *ar UNUSED,\n> +\t\t\t     struct repository *repo,\n>   \t\t\t     struct archiver_args *args)\n>   {\n>   \tint err = 0;\n>   \n>   \twrite_global_extended_header(args);\n> -\terr = write_archive_entries(args, write_tar_entry);\n> +\terr = write_archive_entries(repo, args, write_tar_entry);\n>   \tif (!err)\n>   \t\twrite_trailer();\n>   \treturn err;\n> @@ -462,6 +466,7 @@ static void tgz_write_block(const void *data)\n>   static const char internal_gzip_command[] = \"git archive gzip\";\n>   \n>   static int write_tar_filter_archive(const struct archiver *ar,\n> +\t\t\t\t\tstruct repository *repo,\n>   \t\t\t\t    struct archiver_args *args)\n>   {\n>   #if ZLIB_VERNUM >= 0x1221\n> @@ -484,7 +489,7 @@ static int write_tar_filter_archive(const struct archiver *ar,\n>   \t\tgzstream.next_out = outbuf;\n>   \t\tgzstream.avail_out = sizeof(outbuf);\n>   \n> -\t\tr = write_tar_archive(ar, args);\n> +\t\tr = write_tar_archive(ar, repo, args);\n>   \n>   \t\ttgz_deflate(Z_FINISH);\n>   \t\tgit_deflate_end(&gzstream);\n> @@ -506,7 +511,7 @@ static int write_tar_filter_archive(const struct archiver *ar,\n>   \t\tdie_errno(_(\"unable to redirect descriptor\"));\n>   \tclose(filter.in);\n>   \n> -\tr = write_tar_archive(ar, args);\n> +\tr = write_tar_archive(ar, repo, args);\n>   \n>   \tclose(1);\n>   \tif (finish_command(&filter) != 0)\n> diff --git a/archive-zip.c b/archive-zip.c\n> index 0456f1ebf15..2c1f943a6cc 100644\n> --- a/archive-zip.c\n> +++ b/archive-zip.c\n> @@ -283,7 +283,9 @@ static int entry_is_binary(struct index_state *istate, const char *path,\n>   \n>   #define STREAM_BUFFER_SIZE (1024 * 16)\n>   \n> -static int write_zip_entry(struct archiver_args *args,\n> +static int write_zip_entry(\n> +\t\t\t   struct repository *repo,\n> +\t\t\t   struct archiver_args *args,\n>   \t\t\t   const struct object_id *oid,\n>   \t\t\t   const char *path, size_t pathlen,\n>   \t\t\t   unsigned int mode,\n> @@ -340,7 +342,7 @@ static int write_zip_entry(struct archiver_args *args,\n>   \n>   \t\tif (!buffer) {\n>   \t\t\tenum object_type type;\n> -\t\t\tstream = open_istream(args->repo, oid, &type, &size,\n> +\t\t\tstream = open_istream(repo, oid, &type, &size,\n>   \t\t\t\t\t      NULL);\n>   \t\t\tif (!stream)\n>   \t\t\t\treturn error(_(\"cannot stream blob %s\"),\n> @@ -349,7 +351,7 @@ static int write_zip_entry(struct archiver_args *args,\n>   \t\t\tout = NULL;\n>   \t\t} else {\n>   \t\t\tcrc = crc32(crc, buffer, size);\n> -\t\t\tis_binary = entry_is_binary(args->repo->index,\n> +\t\t\tis_binary = entry_is_binary(repo->index,\n>   \t\t\t\t\t\t    path_without_prefix,\n>   \t\t\t\t\t\t    buffer, size);\n>   \t\t\tout = buffer;\n> @@ -426,7 +428,7 @@ static int write_zip_entry(struct archiver_args *args,\n>   \t\t\t\tbreak;\n>   \t\t\tcrc = crc32(crc, buf, readlen);\n>   \t\t\tif (is_binary == -1)\n> -\t\t\t\tis_binary = entry_is_binary(args->repo->index,\n> +\t\t\t\tis_binary = entry_is_binary(repo->index,\n>   \t\t\t\t\t\t\t    path_without_prefix,\n>   \t\t\t\t\t\t\t    buf, readlen);\n>   \t\t\twrite_or_die(1, buf, readlen);\n> @@ -459,7 +461,7 @@ static int write_zip_entry(struct archiver_args *args,\n>   \t\t\t\tbreak;\n>   \t\t\tcrc = crc32(crc, buf, readlen);\n>   \t\t\tif (is_binary == -1)\n> -\t\t\t\tis_binary = entry_is_binary(args->repo->index,\n> +\t\t\t\tis_binary = entry_is_binary(repo->index,\n>   \t\t\t\t\t\t\t    path_without_prefix,\n>   \t\t\t\t\t\t\t    buf, readlen);\n>   \n> @@ -619,6 +621,7 @@ static int archive_zip_config(const char *var, const char *value,\n>   }\n>   \n>   static int write_zip_archive(const struct archiver *ar UNUSED,\n> +\t\t\t\t struct repository *repo,\n>   \t\t\t     struct archiver_args *args)\n>   {\n>   \tint err;\n> @@ -629,7 +632,7 @@ static int write_zip_archive(const struct archiver *ar UNUSED,\n>   \n>   \tstrbuf_init(&zip_dir, 0);\n>   \n> -\terr = write_archive_entries(args, write_zip_entry);\n> +\terr = write_archive_entries(repo, args, write_zip_entry);\n>   \tif (!err)\n>   \t\twrite_zip_trailer(args->commit_oid);\n>   \n> diff --git a/archive.c b/archive.c\n> index 15f3ac92dfc..2cca7bc5c8a 100644\n> --- a/archive.c\n> +++ b/archive.c\n> @@ -134,7 +134,9 @@ static int check_attr_export_subst(const struct attr_check *check)\n>   \treturn check && ATTR_TRUE(check->items[1].value);\n>   }\n>   \n> -static int write_archive_entry(const struct object_id *oid, const char *base,\n> +static int write_archive_entry(\n> +\t\tstruct repository *repo,\n> +\t\tconst struct object_id *oid, const char *base,\n>   \t\tint baselen, const char *filename, unsigned mode,\n>   \t\tvoid *context)\n>   {\n> @@ -160,7 +162,7 @@ static int write_archive_entry(const struct object_id *oid, const char *base,\n>   \n>   \tif (!S_ISDIR(mode)) {\n>   \t\tconst struct attr_check *check;\n> -\t\tcheck = get_archive_attrs(args->repo->index, path_without_prefix);\n> +\t\tcheck = get_archive_attrs(repo->index, path_without_prefix);\n>   \t\tif (check_attr_export_ignore(check))\n>   \t\t\treturn 0;\n>   \t\targs->convert = check_attr_export_subst(check);\n> @@ -169,7 +171,7 @@ static int write_archive_entry(const struct object_id *oid, const char *base,\n>   \tif (S_ISDIR(mode) || S_ISGITLINK(mode)) {\n>   \t\tif (args->verbose)\n>   \t\t\tfprintf(stderr, \"%.*s\\n\", (int)path.len, path.buf);\n> -\t\terr = write_entry(args, oid, path.buf, path.len, mode, NULL, 0);\n> +\t\terr = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, 0);\n>   \t\tif (err)\n>   \t\t\treturn err;\n>   \t\treturn (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);\n> @@ -180,14 +182,14 @@ static int write_archive_entry(const struct object_id *oid, const char *base,\n>   \n>   \t/* Stream it? */\n>   \tif (S_ISREG(mode) && !args->convert &&\n> -\t    oid_object_info(args->repo, oid, &size) == OBJ_BLOB &&\n> +\t    oid_object_info(repo, oid, &size) == OBJ_BLOB &&\n>   \t    size > big_file_threshold)\n> -\t\treturn write_entry(args, oid, path.buf, path.len, mode, NULL, size);\n> +\t\treturn write_entry(repo, args, oid, path.buf, path.len, mode, NULL, size);\n>   \n>   \tbuffer = object_file_to_archive(args, path.buf, oid, mode, &type, &size);\n>   \tif (!buffer)\n>   \t\treturn error(_(\"cannot read '%s'\"), oid_to_hex(oid));\n> -\terr = write_entry(args, oid, path.buf, path.len, mode, buffer, size);\n> +\terr = write_entry(repo, args, oid, path.buf, path.len, mode, buffer, size);\n>   \tfree(buffer);\n>   \treturn err;\n>   }\n> @@ -207,7 +209,9 @@ static void queue_directory(const struct object_id *oid,\n>   \toidcpy(&d->oid, oid);\n>   }\n>   \n> -static int write_directory(struct archiver_context *c)\n> +static int write_directory(\n> +\t\tstruct repository *repo,\n> +\t\tstruct archiver_context *c)\n>   {\n>   \tstruct directory *d = c->bottom;\n>   \tint ret;\n> @@ -217,8 +221,8 @@ static int write_directory(struct archiver_context *c)\n>   \tc->bottom = d->up;\n>   \td->path[d->len - 1] = '\\0'; /* no trailing slash */\n>   \tret =\n> -\t\twrite_directory(c) ||\n> -\t\twrite_archive_entry(&d->oid, d->path, d->baselen,\n> +\t\twrite_directory(repo, c) ||\n> +\t\twrite_archive_entry(repo, &d->oid, d->path, d->baselen,\n>   \t\t\t\t    d->path + d->baselen, d->mode,\n>   \t\t\t\t    c) != READ_TREE_RECURSIVE;\n>   \tfree(d);\n> @@ -257,9 +261,9 @@ static int queue_or_write_archive_entry(\n>   \t\treturn READ_TREE_RECURSIVE;\n>   \t}\n>   \n> -\tif (write_directory(c))\n> +\tif (write_directory(r, c))\n>   \t\treturn -1;\n> -\treturn write_archive_entry(oid, base->buf, base->len, filename, mode,\n> +\treturn write_archive_entry(r, oid, base->buf, base->len, filename, mode,\n>   \t\t\t\t   context);\n>   }\n>   \n> @@ -269,7 +273,9 @@ struct extra_file_info {\n>   \tvoid *content;\n>   };\n>   \n> -int write_archive_entries(struct archiver_args *args,\n> +int write_archive_entries(\n> +\t\tstruct repository *repo,\n> +\t\tstruct archiver_args *args,\n>   \t\twrite_archive_entry_fn_t write_entry)\n>   {\n>   \tstruct archiver_context context;\n> @@ -290,7 +296,7 @@ int write_archive_entries(struct archiver_args *args,\n>   \t\t\tlen--;\n>   \t\tif (args->verbose)\n>   \t\t\tfprintf(stderr, \"%.*s\\n\", (int)len, args->base);\n> -\t\terr = write_entry(args, &args->tree->object.oid, args->base,\n> +\t\terr = write_entry(repo, args, &args->tree->object.oid, args->base,\n>   \t\t\t\t  len, 040777, NULL, 0);\n>   \t\tif (err)\n>   \t\t\treturn err;\n> @@ -345,12 +351,12 @@ int write_archive_entries(struct archiver_args *args,\n>   \t\t\tif (strbuf_read_file(&content, path, info->stat.st_size) < 0)\n>   \t\t\t\terr = error_errno(_(\"cannot read '%s'\"), path);\n>   \t\t\telse\n> -\t\t\t\terr = write_entry(args, &fake_oid, path_in_archive.buf,\n> +\t\t\t\terr = write_entry(repo, args, &fake_oid, path_in_archive.buf,\n>   \t\t\t\t\t\t  path_in_archive.len,\n>   \t\t\t\t\t\t  canon_mode(info->stat.st_mode),\n>   \t\t\t\t\t\t  content.buf, content.len);\n>   \t\t} else {\n> -\t\t\terr = write_entry(args, &fake_oid,\n> +\t\t\terr = write_entry(repo, args, &fake_oid,\n>   \t\t\t\t\t  path, strlen(path),\n>   \t\t\t\t\t  canon_mode(info->stat.st_mode),\n>   \t\t\t\t\t  info->content, info->stat.st_size);\n> @@ -711,7 +717,7 @@ int write_archive(int argc, const char **argv, const char *prefix,\n>   \tparse_treeish_arg(argv, &args, prefix, remote);\n>   \tparse_pathspec_arg(argv + 1, &args);\n>   \n> -\trc = ar->write_archive(ar, &args);\n> +\trc = ar->write_archive(ar, repo, &args);\n>   \n>   \tstring_list_clear_func(&args.extra_files, extra_file_info_clear);\n>   \tfree(args.refname);\n> diff --git a/archive.h b/archive.h\n> index 08bed3ed3af..bfbbd3274bd 100644\n> --- a/archive.h\n> +++ b/archive.h\n> @@ -41,7 +41,10 @@ const char *archive_format_from_filename(const char *filename);\n>   #define ARCHIVER_HIGH_COMPRESSION_LEVELS 4\n>   struct archiver {\n>   \tconst char *name;\n> -\tint (*write_archive)(const struct archiver *, struct archiver_args *);\n> +\tint (*write_archive)(\n> +\t\tconst struct archiver *,\n> +\t\tstruct repository *,\n> +\t\tstruct archiver_args *);\n>   \tunsigned flags;\n>   \tchar *filter_command;\n>   };\n> @@ -51,12 +54,17 @@ void init_tar_archiver(void);\n>   void init_zip_archiver(void);\n>   void init_archivers(void);\n>   \n> -typedef int (*write_archive_entry_fn_t)(struct archiver_args *args,\n> +typedef int (*write_archive_entry_fn_t)(\n> +\t\t\t\t\tstruct repository *repo,\n> +\t\t\t\t\tstruct archiver_args *args,\n>   \t\t\t\t\tconst struct object_id *oid,\n>   \t\t\t\t\tconst char *path, size_t pathlen,\n>   \t\t\t\t\tunsigned int mode,\n>   \t\t\t\t\tvoid *buffer, unsigned long size);\n>   \n> -int write_archive_entries(struct archiver_args *args, write_archive_entry_fn_t write_entry);\n> +int write_archive_entries(\n> +\tstruct repository *repo,\n> +\tstruct archiver_args *args,\n> +\twrite_archive_entry_fn_t write_entry);\n>   \n>   #endif\t/* ARCHIVE_H */\n"},{"id":"465103","messageId":"xmqqsfjm1rl6.fsf@gitster.g","threadId":"58613","inReplyTo":"d3d1738e670d5dbf1378fc5c3209b2e98234a771.1665973401.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 4/9] tree: handle submodule case for read_tree_at properly","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-17T13:56:37Z","receivedAt":"2022-10-17T13:56:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Heather Lapointe via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> From: Heather Lapointe <alpha@alphaservcomputing.solutions>\n>\n> This supports traversal into an actual submodule for read_tree_at.\n> The logic is blocked on pathspec->recurse_submodules now,\n> but previously hadn't been executed due to all fn() cases\n> returning early for submodules.\n>\n> Signed-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n> ---\n>  tree.c | 88 ++++++++++++++++++++++++++++++++++++++++------------------\n>  1 file changed, 61 insertions(+), 27 deletions(-)\n>\n> diff --git a/tree.c b/tree.c\n> index 13f9173d45e..2a087c010f9 100644\n> --- a/tree.c\n> +++ b/tree.c\n> @@ -8,6 +8,7 @@\n>  #include \"alloc.h\"\n>  #include \"tree-walk.h\"\n>  #include \"repository.h\"\n> +#include \"pathspec.h\"\n>  \n>  const char *tree_type = \"tree\";\n>  \n> @@ -47,40 +48,73 @@ int read_tree_at(struct repository *r,\n>  \t\t\treturn -1;\n>  \t\t}\n>  \n> -\t\tif (S_ISDIR(entry.mode))\n> +\t\tif (S_ISDIR(entry.mode)) {\n>  \t\t\toidcpy(&oid, &entry.oid);\n> +\t\t\tlen = tree_entry_len(&entry);\n> +\t\t\tstrbuf_add(base, entry.path, len);\n> +\t\t\tstrbuf_addch(base, '/');\n> +\t\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n> +\t\t\t\t\t\tbase, pathspec,\n> +\t\t\t\t\t\tfn, context);\n> +\t\t\tstrbuf_setlen(base, oldlen);\n> +\t\t\tif (retval)\n> +\t\t\t\treturn -1;\n\nThe diff output makes it appear as if we are now adding many extra\nprocessing to a normal directory case, but it actually folds the\ncode that was originally outside the if/else if/ cascade here.  So\nI think this is not breaking the normal directory case.\n\n> +\t\t} else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) {\n> +\t\t\tstruct commit *commit;\n> +\t\t\tstruct repository subrepo;\n> +\t\t\tstruct repository* subrepo_p = &subrepo;\n> +\t\t\tstruct tree* submodule_tree;\n\nIn our codebase, star/asterisk for a pointer declaration sticks to\nthe variable, not the type.\n\ncf. Documentation/CodingGuidelines\n\n> +\t\t\tchar *submodule_rel_path;\n\nFunny that the new code sometimes gets it right ;-)\n\n> +\t\t\tint name_base_len = 0;\n> +\n> +\t\t\tlen = tree_entry_len(&entry);\n> +\t\t\tstrbuf_add(base, entry.path, len);\n> +\t\t\tsubmodule_rel_path = base->buf;\n> +\t\t\t// repo_submodule_init expects a path relative to submodule_prefix\n\nWe avoid // comments.\n\n> +\t\t\tif (r->submodule_prefix) {\n> +\t\t\t\tname_base_len = strlen(r->submodule_prefix);\n> +\t\t\t\t// we should always expect to start with submodule_prefix\n> +\t\t\t\tassert(!strncmp(submodule_rel_path, r->submodule_prefix, name_base_len));\n> +\t\t\t\t// strip the prefix\n> +\t\t\t\tsubmodule_rel_path += name_base_len;\n> +\t\t\t\t// if submodule_prefix doesn't end with a /, we want to get rid of that too\n> +\t\t\t\tif (is_dir_sep(submodule_rel_path[0])) {\n> +\t\t\t\t\tsubmodule_rel_path++;\n> +\t\t\t\t}\n> +\t\t\t}\n> +\n> +\t\t\tif (repo_submodule_init(subrepo_p, r, submodule_rel_path, null_oid()))\n> +\t\t\t\tdie(\"couldn't init submodule %s\", base->buf);\n> +\n> +\t\t\tif (repo_read_index(subrepo_p) < 0)\n> +\t\t\t\tdie(\"index file corrupt\");\n\nWhy?  You are going to ask the object store of the submodule\nrepository, and to do so you do not need to have its index read into\nthe core.\n\n> +\t\t\tcommit = lookup_commit(subrepo_p, &entry.oid);\n>  \t\t\tif (!commit)\n> -\t\t\t\tdie(\"Commit %s in submodule path %s%s not found\",\n> +\t\t\t\tdie(\"Commit %s in submodule path %s not found\",\n>  \t\t\t\t    oid_to_hex(&entry.oid),\n> -\t\t\t\t    base->buf, entry.path);\n> -\n> -\t\t\t// FIXME: This is the wrong repo instance (it refers to the superproject)\n> -\t\t\t// it will always fail as is (will fix in later patch)\n> -\t\t\t// This current codepath isn't executed by any existing callbacks\n> -\t\t\t// so it wouldn't show up as an issue at this time.\n> -\t\t\tif (repo_parse_commit(r, commit))\n> -\t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n> +\t\t\t\t    base->buf);\n> +\n> +\t\t\tif (repo_parse_commit(subrepo_p, commit))\n> +\t\t\t\tdie(\"Invalid commit %s in submodule path %s\",\n>  \t\t\t\t    oid_to_hex(&entry.oid),\n> -\t\t\t\t    base->buf, entry.path);\n> +\t\t\t\t    base->buf);\n>  \n> -\t\t\toidcpy(&oid, get_commit_tree_oid(commit));\n> -\t\t}\n> -\t\telse\n> -\t\t\tcontinue;\n> +\t\t\tsubmodule_tree = repo_get_commit_tree(subrepo_p, commit);\n> +\t\t\toidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);\n>  \n> -\t\tlen = tree_entry_len(&entry);\n> -\t\tstrbuf_add(base, entry.path, len);\n> -\t\tstrbuf_addch(base, '/');\n> -\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n> -\t\t\t\t      base, pathspec,\n> -\t\t\t\t      fn, context);\n> -\t\tstrbuf_setlen(base, oldlen);\n> -\t\tif (retval)\n> -\t\t\treturn -1;\n> +\t\t\tstrbuf_addch(base, '/');\n> +\n> +\t\t\tretval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),\n> +\t\t\t\t\t\tbase, pathspec,\n> +\t\t\t\t\t\tfn, context);\n> +\t\t\tif (retval)\n> +\t\t\t    die(\"failed to read tree for %s\", base->buf);\n> +\t\t\tstrbuf_setlen(base, oldlen);\n> +\t\t\trepo_clear(subrepo_p);\n\nThis is a lot of new code, which must be done correctly.  An easier\nway out to use the add_submodule_odb() trick that the original code\nassumed becomes somewhat tempting (I guess we would do that in fn()\nthat would tell us to recurse into this codepath upon seeing a gitlink\nentry).  Then we wouldn't have had to touch any tree() calls that were\ntaught to take \"struct repository *\" in earlier steps in this series.\n\nBut at some point, we would need to bite the bullet and plumb the\nrepository pointer through the callchain of more APIs, and this may\nbe that point.  I dunno.\n\n> +\t\t}\n> +\t\t// else, this is a file (or a submodule, but no pathspec->recurse_submodules)\n>  \t}\n>  \treturn 0;\n>  }\n"},{"id":"465104","messageId":"8acd4fac-491f-7ab5-88be-b7804b37d23b@dunelm.org.uk","threadId":"58613","inReplyTo":"pull.1359.v3.git.git.1665973401.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 0/9] archive: Add --recurse-submodules to git-archive command","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-10-17T13:57:13Z","receivedAt":"2022-10-17T13:57:23Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Heather\n\nOn 17/10/2022 03:23, Heather Lapointe via GitGitGadget wrote:\n> This makes it possible to include submodule contents in an archive command.\n> \n> The inspiration for this change comes from this Github thread,\n> https://github.com/dear-github/dear-github/issues/214, with at least 160\n> 👍🏻 's at the time of writing. (I stumbled upon it because I wanted it as\n> well).\n> \n> I figured the underlying implementation wouldn't be too difficult with most\n> of the plumbing already in place, so I decided to add the relevant logic to\n> the client git-archive command.\n> \n> One of the trickier parts of this implementation involved teaching read_tree\n> about submodules. Some of the troublesome areas were still using the\n> the_repository references to look up commit or tree or oid information. I\n> ended up deciding that read_tree_fn_t would probably be best off having a\n> concrete repo reference since it allows changing the context to a subrepo\n> where needed (even though some of the usages did not need it specifically).\n> \n> I am open to feedback since this is all quite new to me :)\n\nI've had a quick read through and I thought this patch series was well \nstructured and easy to follow for someone like me who is not familiar \nwith the archive code. I've left a few mostly high-level comments on \nsome of the individual patches. The commit messages are all pretty brief \nand would benefit from a bit more explaintion of why the changes are \nbeing made. From a brief read through the code changes themselves all \nlooked pretty good.\n\nBest Wishes\n\nPhillip\n\n\n> Alphadelta14 (1):\n>    tree: do not use the_repository for tree traversal methods.\n >\n> Heather Lapointe (8):\n>    tree: update cases to use repo_ tree methods\n>    tree: increase test coverage for tree.c\n>    tree: handle submodule case for read_tree_at properly\n>    tree: add repository parameter to read_tree_fn_t\n>    archive: pass repo objects to write_archive handlers\n>    archive: remove global repository from archive_args\n>    archive: add --recurse-submodules to git-archive command\n>    archive: add tests for git archive --recurse-submodules\n> \n>   Documentation/git-archive.txt     |   6 +-\n>   Makefile                          |   1 +\n>   archive-tar.c                     |  15 ++--\n>   archive-zip.c                     |  15 ++--\n>   archive.c                         | 138 ++++++++++++++++++++----------\n>   archive.h                         |  16 +++-\n>   builtin/checkout.c                |   4 +-\n>   builtin/log.c                     |   4 +-\n>   builtin/ls-files.c                |   8 +-\n>   builtin/ls-tree.c                 |  34 +++++---\n>   merge-recursive.c                 |   4 +-\n>   merge.c                           |   4 +-\n>   reset.c                           |   2 +-\n>   revision.c                        |   4 +-\n>   sequencer.c                       |   6 +-\n>   sparse-index.c                    |   4 +-\n>   t/helper/test-tool.c              |   1 +\n>   t/helper/test-tool.h              |   1 +\n>   t/helper/test-tree-read-tree-at.c |  41 +++++++++\n>   t/t1023-tree-read-tree-at.sh      |  65 ++++++++++++++\n>   t/t5005-archive-submodules.sh     |  83 ++++++++++++++++++\n>   tree.c                            |  93 ++++++++++++++------\n>   tree.h                            |  16 ++--\n>   wt-status.c                       |   4 +-\n>   24 files changed, 448 insertions(+), 121 deletions(-)\n>   create mode 100644 t/helper/test-tree-read-tree-at.c\n>   create mode 100755 t/t1023-tree-read-tree-at.sh\n>   create mode 100755 t/t5005-archive-submodules.sh\n> \n> \n> base-commit: e85701b4af5b7c2a9f3a1b07858703318dce365d\n> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1359%2FAlphadelta14%2Farchive-recurse-submodules-v3\n> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1359/Alphadelta14/archive-recurse-submodules-v3\n> Pull-Request: https://github.com/git/git/pull/1359\n> \n> Range-diff vs v2:\n> \n>    -:  ----------- >  1:  79959a54eb4 tree: do not use the_repository for tree traversal methods.\n>    -:  ----------- >  2:  2291e0f9b5c tree: update cases to use repo_ tree methods\n>    -:  ----------- >  3:  9a07c6932f4 tree: increase test coverage for tree.c\n>    2:  68f7830c6d9 !  4:  d3d1738e670 archive: fix a case of submodule in submodule traversal\n>       @@\n>         ## Metadata ##\n>       -Author: Alphadelta14 <alpha@alphaservcomputing.solutions>\n>       +Author: Heather Lapointe <alpha@alphaservcomputing.solutions>\n>        \n>         ## Commit message ##\n>       -    archive: fix a case of submodule in submodule traversal\n>       +    tree: handle submodule case for read_tree_at properly\n>        \n>       -    repo_submodule_init actually expects the path relative to submodule_prefix.\n>       -    We preform a simple strip to the correct path.\n>       +    This supports traversal into an actual submodule for read_tree_at.\n>       +    The logic is blocked on pathspec->recurse_submodules now,\n>       +    but previously hadn't been executed due to all fn() cases\n>       +    returning early for submodules.\n>        \n>       -    Signed-off-by: Alphadelta14 <alpha@alphaservcomputing.solutions>\n>       +    Signed-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n>        \n>         ## tree.c ##\n>       +@@\n>       + #include \"alloc.h\"\n>       + #include \"tree-walk.h\"\n>       + #include \"repository.h\"\n>       ++#include \"pathspec.h\"\n>       +\n>       + const char *tree_type = \"tree\";\n>       +\n>        @@ tree.c: int read_tree_at(struct repository *r,\n>       - \t\t\tstruct repository subrepo;\n>       - \t\t\tstruct repository* subrepo_p = &subrepo;\n>       - \t\t\tstruct tree* submodule_tree;\n>       + \t\t\treturn -1;\n>       + \t\t}\n>       +\n>       +-\t\tif (S_ISDIR(entry.mode))\n>       ++\t\tif (S_ISDIR(entry.mode)) {\n>       + \t\t\toidcpy(&oid, &entry.oid);\n>       +-\t\telse if (S_ISGITLINK(entry.mode)) {\n>       +-\t\t\tstruct commit *commit;\n>       +\n>       +-\t\t\tcommit = lookup_commit(r, &entry.oid);\n>       ++\t\t\tlen = tree_entry_len(&entry);\n>       ++\t\t\tstrbuf_add(base, entry.path, len);\n>       ++\t\t\tstrbuf_addch(base, '/');\n>       ++\t\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n>       ++\t\t\t\t\t\tbase, pathspec,\n>       ++\t\t\t\t\t\tfn, context);\n>       ++\t\t\tstrbuf_setlen(base, oldlen);\n>       ++\t\t\tif (retval)\n>       ++\t\t\t\treturn -1;\n>       ++\t\t} else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) {\n>       ++\t\t\tstruct commit *commit;\n>       ++\t\t\tstruct repository subrepo;\n>       ++\t\t\tstruct repository* subrepo_p = &subrepo;\n>       ++\t\t\tstruct tree* submodule_tree;\n>        +\t\t\tchar *submodule_rel_path;\n>        +\t\t\tint name_base_len = 0;\n>       -\n>       --\t\t\tif (repo_submodule_init(subrepo_p, r, entry.path, null_oid()))\n>       --\t\t\t\tdie(\"couldn't init submodule %s%s\", base->buf, entry.path);\n>       ++\n>        +\t\t\tlen = tree_entry_len(&entry);\n>        +\t\t\tstrbuf_add(base, entry.path, len);\n>        +\t\t\tsubmodule_rel_path = base->buf;\n>       @@ tree.c: int read_tree_at(struct repository *r,\n>        +\n>        +\t\t\tif (repo_submodule_init(subrepo_p, r, submodule_rel_path, null_oid()))\n>        +\t\t\t\tdie(\"couldn't init submodule %s\", base->buf);\n>       -\n>       - \t\t\tif (repo_read_index(subrepo_p) < 0)\n>       - \t\t\t\tdie(\"index file corrupt\");\n>       -\n>       - \t\t\tcommit = lookup_commit(subrepo_p, &entry.oid);\n>       ++\n>       ++\t\t\tif (repo_read_index(subrepo_p) < 0)\n>       ++\t\t\t\tdie(\"index file corrupt\");\n>       ++\n>       ++\t\t\tcommit = lookup_commit(subrepo_p, &entry.oid);\n>         \t\t\tif (!commit)\n>        -\t\t\t\tdie(\"Commit %s in submodule path %s%s not found\",\n>        +\t\t\t\tdie(\"Commit %s in submodule path %s not found\",\n>         \t\t\t\t    oid_to_hex(&entry.oid),\n>        -\t\t\t\t    base->buf, entry.path);\n>       -+\t\t\t\t    base->buf);\n>       -\n>       - \t\t\tif (repo_parse_commit(subrepo_p, commit))\n>       +-\n>       +-\t\t\t// FIXME: This is the wrong repo instance (it refers to the superproject)\n>       +-\t\t\t// it will always fail as is (will fix in later patch)\n>       +-\t\t\t// This current codepath isn't executed by any existing callbacks\n>       +-\t\t\t// so it wouldn't show up as an issue at this time.\n>       +-\t\t\tif (repo_parse_commit(r, commit))\n>        -\t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n>       ++\t\t\t\t    base->buf);\n>       ++\n>       ++\t\t\tif (repo_parse_commit(subrepo_p, commit))\n>        +\t\t\t\tdie(\"Invalid commit %s in submodule path %s\",\n>         \t\t\t\t    oid_to_hex(&entry.oid),\n>        -\t\t\t\t    base->buf, entry.path);\n>        +\t\t\t\t    base->buf);\n>         \n>       - \t\t\tsubmodule_tree = repo_get_commit_tree(subrepo_p, commit);\n>       - \t\t\toidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);\n>       +-\t\t\toidcpy(&oid, get_commit_tree_oid(commit));\n>       +-\t\t}\n>       +-\t\telse\n>       +-\t\t\tcontinue;\n>       ++\t\t\tsubmodule_tree = repo_get_commit_tree(subrepo_p, commit);\n>       ++\t\t\toidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);\n>         \n>       --\t\t\tlen = tree_entry_len(&entry);\n>       --\t\t\tstrbuf_add(base, entry.path, len);\n>       - \t\t\tstrbuf_addch(base, '/');\n>       +-\t\tlen = tree_entry_len(&entry);\n>       +-\t\tstrbuf_add(base, entry.path, len);\n>       +-\t\tstrbuf_addch(base, '/');\n>       +-\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n>       +-\t\t\t\t      base, pathspec,\n>       +-\t\t\t\t      fn, context);\n>       +-\t\tstrbuf_setlen(base, oldlen);\n>       +-\t\tif (retval)\n>       +-\t\t\treturn -1;\n>       ++\t\t\tstrbuf_addch(base, '/');\n>        +\n>       - \t\t\tretval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),\n>       - \t\t\t\t\t\tbase, pathspec,\n>       - \t\t\t\t\t\tfn, context);\n>       - \t\t\tif (retval) {\n>       --\t\t\t    die(\"failed to read tree for %s%s\", base->buf, entry.path);\n>       ++\t\t\tretval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),\n>       ++\t\t\t\t\t\tbase, pathspec,\n>       ++\t\t\t\t\t\tfn, context);\n>       ++\t\t\tif (retval)\n>        +\t\t\t    die(\"failed to read tree for %s\", base->buf);\n>       - \t\t\t    return -1;\n>       - \t\t\t}\n>       - \t\t\tstrbuf_setlen(base, oldlen);\n>       ++\t\t\tstrbuf_setlen(base, oldlen);\n>       ++\t\t\trepo_clear(subrepo_p);\n>       ++\t\t}\n>       ++\t\t// else, this is a file (or a submodule, but no pathspec->recurse_submodules)\n>       + \t}\n>       + \treturn 0;\n>       + }\n>    -:  ----------- >  5:  376345fdf66 tree: add repository parameter to read_tree_fn_t\n>    1:  41664a59029 !  6:  1b9b049d64f archive: add --recurse-submodules to git-archive command\n>       @@\n>         ## Metadata ##\n>       -Author: Alphadelta14 <alpha@alphaservcomputing.solutions>\n>       +Author: Heather Lapointe <alpha@alphaservcomputing.solutions>\n>        \n>         ## Commit message ##\n>       -    archive: add --recurse-submodules to git-archive command\n>       +    archive: pass repo objects to write_archive handlers\n>        \n>       -    This makes it possible to include submodule contents in an archive command.\n>       -\n>       -    This required updating the general read_tree callbacks to support sub-repos\n>       -    by not using the_repository global references where possible.\n>       -\n>       -    archive: update streaming to use target repo\n>       -    archive: add test cases for git archive --recurse-submodules\n>       +    Use contextual repos instead of the_repository or args->repo\n>       +    to ensure that submodules will be handled correctly\n>       +    since they use multiple repo instances.\n>        \n>            Signed-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>\n>        \n>       @@ archive-tar.c: static unsigned long offset;\n>         static int tar_umask = 002;\n>         \n>         static int write_tar_filter_archive(const struct archiver *ar,\n>       -+\t\t\t\t    struct repository *repo,\n>       ++\t\t\t\t\tstruct repository *repo,\n>         \t\t\t\t    struct archiver_args *args);\n>         \n>         /*\n>       @@ archive-tar.c: static void write_extended_header(struct archiver_args *args,\n>         }\n>         \n>        -static int write_tar_entry(struct archiver_args *args,\n>       -+static int write_tar_entry(struct repository *repo,\n>       ++static int write_tar_entry(\n>       ++\t\t\t   struct repository *repo,\n>        +\t\t\t   struct archiver_args *args,\n>         \t\t\t   const struct object_id *oid,\n>         \t\t\t   const char *path, size_t pathlen,\n>       @@ archive-tar.c: static void tgz_write_block(const void *data)\n>         static const char internal_gzip_command[] = \"git archive gzip\";\n>         \n>         static int write_tar_filter_archive(const struct archiver *ar,\n>       -+\t\t\t\t    struct repository *repo,\n>       ++\t\t\t\t\tstruct repository *repo,\n>         \t\t\t\t    struct archiver_args *args)\n>         {\n>         #if ZLIB_VERNUM >= 0x1221\n>       @@ archive-zip.c: static int entry_is_binary(struct index_state *istate, const char\n>         #define STREAM_BUFFER_SIZE (1024 * 16)\n>         \n>        -static int write_zip_entry(struct archiver_args *args,\n>       -+static int write_zip_entry(struct repository *repo,\n>       ++static int write_zip_entry(\n>       ++\t\t\t   struct repository *repo,\n>        +\t\t\t   struct archiver_args *args,\n>         \t\t\t   const struct object_id *oid,\n>         \t\t\t   const char *path, size_t pathlen,\n>       @@ archive-zip.c: static int archive_zip_config(const char *var, const char *value,\n>         }\n>         \n>         static int write_zip_archive(const struct archiver *ar UNUSED,\n>       -+\t\t\t     struct repository *repo,\n>       ++\t\t\t\t struct repository *repo,\n>         \t\t\t     struct archiver_args *args)\n>         {\n>         \tint err;\n>       @@ archive-zip.c: static int write_zip_archive(const struct archiver *ar UNUSED,\n>         \n>        \n>         ## archive.c ##\n>       -@@\n>       - #include \"unpack-trees.h\"\n>       - #include \"dir.h\"\n>       - #include \"quote.h\"\n>       -+#include \"submodule.h\"\n>       -\n>       - static char const * const archive_usage[] = {\n>       - \tN_(\"git archive [<options>] <tree-ish> [<path>...]\"),\n>       -@@ archive.c: static void format_subst(const struct commit *commit,\n>       - }\n>       -\n>       - static void *object_file_to_archive(const struct archiver_args *args,\n>       -+\t\t\t\t    struct repository *repo,\n>       - \t\t\t\t    const char *path,\n>       - \t\t\t\t    const struct object_id *oid,\n>       - \t\t\t\t    unsigned int mode,\n>       -@@ archive.c: static void *object_file_to_archive(const struct archiver_args *args,\n>       - \t\t\t       (args->tree ? &args->tree->object.oid : NULL), oid);\n>       -\n>       - \tpath += args->baselen;\n>       --\tbuffer = read_object_file(oid, type, sizep);\n>       -+\tbuffer = repo_read_object_file(repo, oid, type, sizep);\n>       - \tif (buffer && S_ISREG(mode)) {\n>       - \t\tstruct strbuf buf = STRBUF_INIT;\n>       - \t\tsize_t size = 0;\n>       -\n>       - \t\tstrbuf_attach(&buf, buffer, *sizep, *sizep + 1);\n>       --\t\tconvert_to_working_tree(args->repo->index, path, buf.buf, buf.len, &buf, &meta);\n>       -+\t\tconvert_to_working_tree(repo->index, path, buf.buf, buf.len, &buf, &meta);\n>       - \t\tif (commit)\n>       - \t\t\tformat_subst(commit, buf.buf, buf.len, &buf, args->pretty_ctx);\n>       - \t\tbuffer = strbuf_detach(&buf, &size);\n>        @@ archive.c: static int check_attr_export_subst(const struct attr_check *check)\n>         \treturn check && ATTR_TRUE(check->items[1].value);\n>         }\n>         \n>        -static int write_archive_entry(const struct object_id *oid, const char *base,\n>       -+static int write_archive_entry(struct repository *repo, const struct object_id *oid, const char *base,\n>       ++static int write_archive_entry(\n>       ++\t\tstruct repository *repo,\n>       ++\t\tconst struct object_id *oid, const char *base,\n>         \t\tint baselen, const char *filename, unsigned mode,\n>         \t\tvoid *context)\n>         {\n>       @@ archive.c: static int write_archive_entry(const struct object_id *oid, const cha\n>        +\t\terr = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, 0);\n>         \t\tif (err)\n>         \t\t\treturn err;\n>       --\t\treturn (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);\n>       -+\t\treturn READ_TREE_RECURSIVE;\n>       - \t}\n>       -\n>       - \tif (args->verbose)\n>       + \t\treturn (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);\n>        @@ archive.c: static int write_archive_entry(const struct object_id *oid, const char *base,\n>         \n>         \t/* Stream it? */\n>         \tif (S_ISREG(mode) && !args->convert &&\n>        -\t    oid_object_info(args->repo, oid, &size) == OBJ_BLOB &&\n>       --\t    size > big_file_threshold)\n>       --\t\treturn write_entry(args, oid, path.buf, path.len, mode, NULL, size);\n>        +\t    oid_object_info(repo, oid, &size) == OBJ_BLOB &&\n>       -+\t    size > big_file_threshold) {\n>       -+\t\t\terr = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, size);\n>       -+\t\t\tif (err) {\n>       -+\t\t\t\tdie(\"Failed to write file %.*s\", (int)path.len, path.buf);\n>       -+\t\t\t}\n>       -+\t\t\treturn err;\n>       -+\t\t}\n>       + \t    size > big_file_threshold)\n>       +-\t\treturn write_entry(args, oid, path.buf, path.len, mode, NULL, size);\n>       ++\t\treturn write_entry(repo, args, oid, path.buf, path.len, mode, NULL, size);\n>         \n>       --\tbuffer = object_file_to_archive(args, path.buf, oid, mode, &type, &size);\n>       -+\tbuffer = object_file_to_archive(args, repo, path.buf, oid, mode, &type, &size);\n>       + \tbuffer = object_file_to_archive(args, path.buf, oid, mode, &type, &size);\n>         \tif (!buffer)\n>         \t\treturn error(_(\"cannot read '%s'\"), oid_to_hex(oid));\n>        -\terr = write_entry(args, oid, path.buf, path.len, mode, buffer, size);\n>       @@ archive.c: static void queue_directory(const struct object_id *oid,\n>         }\n>         \n>        -static int write_directory(struct archiver_context *c)\n>       -+static void queue_submodule(struct repository *superproject,\n>       -+\t\tconst struct object_id *oid,\n>       -+\t\tstruct strbuf *base, const char *filename,\n>       -+\t\tunsigned mode, struct archiver_context *c)\n>       -+{\n>       -+\tstruct repository subrepo;\n>       -+\n>       -+\tif (repo_submodule_init(&subrepo, superproject, filename, null_oid()))\n>       -+\t\treturn;\n>       -+\n>       -+\tif (repo_read_index(&subrepo) < 0)\n>       -+\t\tdie(\"index file corrupt\");\n>       -+\n>       -+    queue_directory(oid, base, filename, mode, c);\n>       -+\n>       -+\trepo_clear(&subrepo);\n>       -+}\n>       -+\n>       -+static int write_directory(struct repository *repo, struct archiver_context *c)\n>       ++static int write_directory(\n>       ++\t\tstruct repository *repo,\n>       ++\t\tstruct archiver_context *c)\n>         {\n>         \tstruct directory *d = c->bottom;\n>         \tint ret;\n>       @@ archive.c: static int write_directory(struct archiver_context *c)\n>        +\t\twrite_directory(repo, c) ||\n>        +\t\twrite_archive_entry(repo, &d->oid, d->path, d->baselen,\n>         \t\t\t\t    d->path + d->baselen, d->mode,\n>       --\t\t\t\t    c) != READ_TREE_RECURSIVE;\n>       -+\t\t\t\t    c);\n>       + \t\t\t\t    c) != READ_TREE_RECURSIVE;\n>         \tfree(d);\n>       --\treturn ret ? -1 : 0;\n>       -+\tif (ret == READ_TREE_RECURSIVE)\n>       -+\t\treturn 0;\n>       -+\treturn ret;\n>       - }\n>       -\n>       --static int queue_or_write_archive_entry(const struct object_id *oid,\n>       -+static int queue_or_write_archive_entry(\n>       -+\t\tstruct repository *repo, const struct object_id *oid,\n>       - \t\tstruct strbuf *base, const char *filename,\n>       - \t\tunsigned mode, void *context)\n>       - {\n>       -@@ archive.c: static int queue_or_write_archive_entry(const struct object_id *oid,\n>       - \t\t/* Borrow base, but restore its original value when done. */\n>       - \t\tstrbuf_addstr(base, filename);\n>       - \t\tstrbuf_addch(base, '/');\n>       --\t\tcheck = get_archive_attrs(c->args->repo->index, base->buf);\n>       -+\t\tcheck = get_archive_attrs(repo->index, base->buf);\n>       - \t\tstrbuf_setlen(base, baselen);\n>       -\n>       - \t\tif (check_attr_export_ignore(check))\n>       - \t\t\treturn 0;\n>       - \t\tqueue_directory(oid, base, filename, mode, c);\n>       +@@ archive.c: static int queue_or_write_archive_entry(\n>         \t\treturn READ_TREE_RECURSIVE;\n>       -+\t} else if (c->args->recurse_submodules && S_ISGITLINK(mode)) {\n>       -+\t\tif (is_submodule_active(repo, filename)) {\n>       -+\t\t\tqueue_submodule(repo, oid, base, filename, mode, c);\n>       -+\t\t\treturn READ_TREE_RECURSIVE;\n>       -+\t\t}\n>         \t}\n>         \n>        -\tif (write_directory(c))\n>       -+\tif (write_directory(repo, c))\n>       ++\tif (write_directory(r, c))\n>         \t\treturn -1;\n>        -\treturn write_archive_entry(oid, base->buf, base->len, filename, mode,\n>       -+\treturn write_archive_entry(repo, oid, base->buf, base->len, filename, mode,\n>       ++\treturn write_archive_entry(r, oid, base->buf, base->len, filename, mode,\n>         \t\t\t\t   context);\n>         }\n>         \n>       @@ archive.c: struct extra_file_info {\n>         };\n>         \n>        -int write_archive_entries(struct archiver_args *args,\n>       -+int write_archive_entries(struct repository *repo,\n>       ++int write_archive_entries(\n>       ++\t\tstruct repository *repo,\n>        +\t\tstruct archiver_args *args,\n>         \t\twrite_archive_entry_fn_t write_entry)\n>         {\n>       @@ archive.c: int write_archive_entries(struct archiver_args *args,\n>         \t\t\t\t  len, 040777, NULL, 0);\n>         \t\tif (err)\n>         \t\t\treturn err;\n>       -@@ archive.c: int write_archive_entries(struct archiver_args *args,\n>       - \t\tmemset(&opts, 0, sizeof(opts));\n>       - \t\topts.index_only = 1;\n>       - \t\topts.head_idx = -1;\n>       --\t\topts.src_index = args->repo->index;\n>       --\t\topts.dst_index = args->repo->index;\n>       -+\t\topts.src_index = repo->index;\n>       -+\t\topts.dst_index = repo->index;\n>       - \t\topts.fn = oneway_merge;\n>       - \t\tinit_tree_desc(&t, args->tree->buffer, args->tree->size);\n>       - \t\tif (unpack_trees(1, &t, &opts))\n>       -@@ archive.c: int write_archive_entries(struct archiver_args *args,\n>       - \t\tgit_attr_set_direction(GIT_ATTR_INDEX);\n>       - \t}\n>       -\n>       --\terr = read_tree(args->repo, args->tree,\n>       -+\terr = read_tree(repo, args->tree,\n>       - \t\t\t&args->pathspec,\n>       - \t\t\tqueue_or_write_archive_entry,\n>       - \t\t\t&context);\n>        @@ archive.c: int write_archive_entries(struct archiver_args *args,\n>         \t\t\tif (strbuf_read_file(&content, path, info->stat.st_size) < 0)\n>         \t\t\t\terr = error_errno(_(\"cannot read '%s'\"), path);\n>       @@ archive.c: int write_archive_entries(struct archiver_args *args,\n>         \t\t\t\t\t  path, strlen(path),\n>         \t\t\t\t\t  canon_mode(info->stat.st_mode),\n>         \t\t\t\t\t  info->content, info->stat.st_size);\n>       -@@ archive.c: struct path_exists_context {\n>       - \tstruct archiver_args *args;\n>       - };\n>       -\n>       --static int reject_entry(const struct object_id *oid UNUSED,\n>       -+static int reject_entry(struct repository *repo, const struct object_id *oid UNUSED,\n>       - \t\t\tstruct strbuf *base,\n>       - \t\t\tconst char *filename, unsigned mode,\n>       - \t\t\tvoid *context)\n>       -@@ archive.c: static int reject_entry(const struct object_id *oid UNUSED,\n>       - \t\tstruct strbuf sb = STRBUF_INIT;\n>       - \t\tstrbuf_addbuf(&sb, base);\n>       - \t\tstrbuf_addstr(&sb, filename);\n>       --\t\tif (!match_pathspec(ctx->args->repo->index,\n>       -+\t\tif (!match_pathspec(repo->index,\n>       - \t\t\t\t    &ctx->pathspec,\n>       - \t\t\t\t    sb.buf, sb.len, 0, NULL, 1))\n>       - \t\t\tret = READ_TREE_RECURSIVE;\n>       -@@ archive.c: static void parse_pathspec_arg(const char **pathspec,\n>       - \t\t       PATHSPEC_PREFER_FULL,\n>       - \t\t       \"\", pathspec);\n>       - \tar_args->pathspec.recursive = 1;\n>       -+\tar_args->pathspec.recurse_submodules = ar_args->recurse_submodules;\n>       - \tif (pathspec) {\n>       - \t\twhile (*pathspec) {\n>       - \t\t\tif (**pathspec && !path_exists(ar_args, *pathspec))\n>       -@@ archive.c: static int parse_archive_args(int argc, const char **argv,\n>       - \tint verbose = 0;\n>       - \tint i;\n>       - \tint list = 0;\n>       -+\tint recurse_submodules = 0;\n>       - \tint worktree_attributes = 0;\n>       - \tstruct option opts[] = {\n>       - \t\tOPT_GROUP(\"\"),\n>       -@@ archive.c: static int parse_archive_args(int argc, const char **argv,\n>       - \t\t  add_file_cb, (intptr_t)&base },\n>       - \t\tOPT_STRING('o', \"output\", &output, N_(\"file\"),\n>       - \t\t\tN_(\"write the archive to this file\")),\n>       -+\t\tOPT_BOOL(0, \"recurse-submodules\", &recurse_submodules,\n>       -+\t\t\tN_(\"include submodules in archive\")),\n>       - \t\tOPT_BOOL(0, \"worktree-attributes\", &worktree_attributes,\n>       - \t\t\tN_(\"read .gitattributes in working directory\")),\n>       - \t\tOPT__VERBOSE(&verbose, N_(\"report archived files on stderr\")),\n>       -@@ archive.c: static int parse_archive_args(int argc, const char **argv,\n>       - \targs->base = base;\n>       - \targs->baselen = strlen(base);\n>       - \targs->worktree_attributes = worktree_attributes;\n>       -+\targs->recurse_submodules = recurse_submodules;\n>       -\n>       - \treturn argc;\n>       - }\n>        @@ archive.c: int write_archive(int argc, const char **argv, const char *prefix,\n>         \tparse_treeish_arg(argv, &args, prefix, remote);\n>         \tparse_pathspec_arg(argv + 1, &args);\n>       @@ archive.c: int write_archive(int argc, const char **argv, const char *prefix,\n>         \tfree(args.refname);\n>        \n>         ## archive.h ##\n>       -@@ archive.h: struct archiver_args {\n>       - \ttimestamp_t time;\n>       - \tstruct pathspec pathspec;\n>       - \tunsigned int verbose : 1;\n>       -+\tunsigned int recurse_submodules : 1;\n>       - \tunsigned int worktree_attributes : 1;\n>       - \tunsigned int convert : 1;\n>       - \tint compression_level;\n>        @@ archive.h: const char *archive_format_from_filename(const char *filename);\n>         #define ARCHIVER_HIGH_COMPRESSION_LEVELS 4\n>         struct archiver {\n>         \tconst char *name;\n>        -\tint (*write_archive)(const struct archiver *, struct archiver_args *);\n>       -+\tint (*write_archive)(const struct archiver *, struct repository *repo, struct archiver_args *);\n>       ++\tint (*write_archive)(\n>       ++\t\tconst struct archiver *,\n>       ++\t\tstruct repository *,\n>       ++\t\tstruct archiver_args *);\n>         \tunsigned flags;\n>         \tchar *filter_command;\n>         };\n>       @@ archive.h: void init_tar_archiver(void);\n>         void init_archivers(void);\n>         \n>        -typedef int (*write_archive_entry_fn_t)(struct archiver_args *args,\n>       -+typedef int (*write_archive_entry_fn_t)(struct repository *repo,\n>       ++typedef int (*write_archive_entry_fn_t)(\n>       ++\t\t\t\t\tstruct repository *repo,\n>        +\t\t\t\t\tstruct archiver_args *args,\n>         \t\t\t\t\tconst struct object_id *oid,\n>         \t\t\t\t\tconst char *path, size_t pathlen,\n>       @@ archive.h: void init_tar_archiver(void);\n>         \t\t\t\t\tvoid *buffer, unsigned long size);\n>         \n>        -int write_archive_entries(struct archiver_args *args, write_archive_entry_fn_t write_entry);\n>       -+int write_archive_entries(struct repository *repo, struct archiver_args *args, write_archive_entry_fn_t write_entry);\n>       ++int write_archive_entries(\n>       ++\tstruct repository *repo,\n>       ++\tstruct archiver_args *args,\n>       ++\twrite_archive_entry_fn_t write_entry);\n>         \n>         #endif\t/* ARCHIVE_H */\n>       -\n>       - ## builtin/checkout.c ##\n>       -@@ builtin/checkout.c: static int post_checkout_hook(struct commit *old_commit, struct commit *new_comm\n>       -\n>       - }\n>       -\n>       --static int update_some(const struct object_id *oid, struct strbuf *base,\n>       -+static int update_some(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n>       - \t\t       const char *pathname, unsigned mode, void *context UNUSED)\n>       - {\n>       - \tint len;\n>       -\n>       - ## builtin/log.c ##\n>       -@@ builtin/log.c: static int show_tag_object(const struct object_id *oid, struct rev_info *rev)\n>       - \treturn 0;\n>       - }\n>       -\n>       --static int show_tree_object(const struct object_id *oid UNUSED,\n>       -+static int show_tree_object(struct repository *repo UNUSED, const struct object_id *oid UNUSED,\n>       - \t\t\t    struct strbuf *base UNUSED,\n>       - \t\t\t    const char *pathname, unsigned mode,\n>       - \t\t\t    void *context)\n>       -\n>       - ## builtin/ls-files.c ##\n>       -@@ builtin/ls-files.c: static int get_common_prefix_len(const char *common_prefix)\n>       - \treturn common_prefix_len;\n>       - }\n>       -\n>       --static int read_one_entry_opt(struct index_state *istate,\n>       -+static int read_one_entry_opt(struct repository *repo UNUSED, struct index_state *istate,\n>       - \t\t\t      const struct object_id *oid,\n>       - \t\t\t      struct strbuf *base,\n>       - \t\t\t      const char *pathname,\n>       -@@ builtin/ls-files.c: static int read_one_entry_opt(struct index_state *istate,\n>       - \treturn add_index_entry(istate, ce, opt);\n>       - }\n>       -\n>       --static int read_one_entry(const struct object_id *oid, struct strbuf *base,\n>       -+static int read_one_entry(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n>       - \t\t\t  const char *pathname, unsigned mode,\n>       - \t\t\t  void *context)\n>       - {\n>       - \tstruct index_state *istate = context;\n>       --\treturn read_one_entry_opt(istate, oid, base, pathname,\n>       -+\treturn read_one_entry_opt(repo, istate, oid, base, pathname,\n>       - \t\t\t\t  mode,\n>       - \t\t\t\t  ADD_CACHE_OK_TO_ADD|ADD_CACHE_SKIP_DFCHECK);\n>       - }\n>       -@@ builtin/ls-files.c: static int read_one_entry(const struct object_id *oid, struct strbuf *base,\n>       -  * This is used when the caller knows there is no existing entries at\n>       -  * the stage that will conflict with the entry being added.\n>       -  */\n>       --static int read_one_entry_quick(const struct object_id *oid, struct strbuf *base,\n>       -+static int read_one_entry_quick(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n>       - \t\t\t\tconst char *pathname, unsigned mode,\n>       - \t\t\t\tvoid *context)\n>       - {\n>       - \tstruct index_state *istate = context;\n>       --\treturn read_one_entry_opt(istate, oid, base, pathname,\n>       -+\treturn read_one_entry_opt(repo, istate, oid, base, pathname,\n>       - \t\t\t\t  mode, ADD_CACHE_JUST_APPEND);\n>       - }\n>       -\n>       -\n>       - ## builtin/ls-tree.c ##\n>       -@@ builtin/ls-tree.c: static int show_recursive(const char *base, size_t baselen, const char *pathname\n>       - \treturn 0;\n>       - }\n>       -\n>       --static int show_tree_fmt(const struct object_id *oid, struct strbuf *base,\n>       -+static int show_tree_fmt(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n>       - \t\t\t const char *pathname, unsigned mode, void *context UNUSED)\n>       - {\n>       - \tsize_t baselen;\n>       -@@ builtin/ls-tree.c: static void show_tree_common_default_long(struct strbuf *base,\n>       - \tstrbuf_setlen(base, baselen);\n>       - }\n>       -\n>       --static int show_tree_default(const struct object_id *oid, struct strbuf *base,\n>       -+static int show_tree_default(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n>       - \t\t\t     const char *pathname, unsigned mode,\n>       - \t\t\t     void *context UNUSED)\n>       - {\n>       -@@ builtin/ls-tree.c: static int show_tree_default(const struct object_id *oid, struct strbuf *base,\n>       - \treturn recurse;\n>       - }\n>       -\n>       --static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n>       -+static int show_tree_long(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n>       - \t\t\t  const char *pathname, unsigned mode,\n>       - \t\t\t  void *context UNUSED)\n>       - {\n>       -@@ builtin/ls-tree.c: static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n>       -\n>       - \tif (data.type == OBJ_BLOB) {\n>       - \t\tunsigned long size;\n>       --\t\tif (oid_object_info(the_repository, data.oid, &size) == OBJ_BAD)\n>       -+\t\tif (oid_object_info(repo, data.oid, &size) == OBJ_BAD)\n>       - \t\t\txsnprintf(size_text, sizeof(size_text), \"BAD\");\n>       - \t\telse\n>       - \t\t\txsnprintf(size_text, sizeof(size_text),\n>       -@@ builtin/ls-tree.c: static int show_tree_long(const struct object_id *oid, struct strbuf *base,\n>       - \t}\n>       -\n>       - \tprintf(\"%06o %s %s %7s\\t\", data.mode, type_name(data.type),\n>       --\t       find_unique_abbrev(data.oid, abbrev), size_text);\n>       -+\t       repo_find_unique_abbrev(repo, data.oid, abbrev), size_text);\n>       - \tshow_tree_common_default_long(base, pathname, data.base->len);\n>       - \treturn recurse;\n>       - }\n>       -\n>       --static int show_tree_name_only(const struct object_id *oid, struct strbuf *base,\n>       -+static int show_tree_name_only(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,\n>       - \t\t\t       const char *pathname, unsigned mode,\n>       - \t\t\t       void *context UNUSED)\n>       - {\n>       -@@ builtin/ls-tree.c: static int show_tree_name_only(const struct object_id *oid, struct strbuf *base,\n>       - \treturn recurse;\n>       - }\n>       -\n>       --static int show_tree_object(const struct object_id *oid, struct strbuf *base,\n>       -+static int show_tree_object(struct repository *repo, const struct object_id *oid, struct strbuf *base,\n>       - \t\t\t    const char *pathname, unsigned mode,\n>       - \t\t\t    void *context UNUSED)\n>       - {\n>       -@@ builtin/ls-tree.c: static int show_tree_object(const struct object_id *oid, struct strbuf *base,\n>       - \tif (early >= 0)\n>       - \t\treturn early;\n>       -\n>       --\tprintf(\"%s%c\", find_unique_abbrev(oid, abbrev), line_termination);\n>       -+\tprintf(\"%s%c\", repo_find_unique_abbrev(repo, oid, abbrev), line_termination);\n>       - \treturn recurse;\n>       - }\n>       -\n>       -\n>       - ## list-objects.c ##\n>       -@@ list-objects.c: static void process_tree(struct traversal_context *ctx,\n>       - \t    !revs->include_check_obj(&tree->object, revs->include_check_data))\n>       - \t\treturn;\n>       -\n>       --\tfailed_parse = parse_tree_gently(tree, 1);\n>       -+\tfailed_parse = parse_tree_gently(revs->repo, tree, 1);\n>       - \tif (failed_parse) {\n>       - \t\tif (revs->ignore_missing_links)\n>       - \t\t\treturn;\n>       -\n>       - ## merge-recursive.c ##\n>       -@@ merge-recursive.c: static void unpack_trees_finish(struct merge_options *opt)\n>       - \tclear_unpack_trees_porcelain(&opt->priv->unpack_opts);\n>       - }\n>       -\n>       --static int save_files_dirs(const struct object_id *oid UNUSED,\n>       -+static int save_files_dirs(struct repository *repo UNUSED, const struct object_id *oid UNUSED,\n>       - \t\t\t   struct strbuf *base, const char *path,\n>       - \t\t\t   unsigned int mode, void *context)\n>       - {\n>       -\n>       - ## revision.c ##\n>       -@@ revision.c: static void mark_tree_contents_uninteresting(struct repository *r,\n>       - \tstruct tree_desc desc;\n>       - \tstruct name_entry entry;\n>       -\n>       --\tif (parse_tree_gently(tree, 1) < 0)\n>       -+\tif (parse_tree_gently(r, tree, 1) < 0)\n>       - \t\treturn;\n>       -\n>       - \tinit_tree_desc(&desc, tree->buffer, tree->size);\n>       -@@ revision.c: static void add_children_by_path(struct repository *r,\n>       - \tif (!tree)\n>       - \t\treturn;\n>       -\n>       --\tif (parse_tree_gently(tree, 1) < 0)\n>       -+\tif (parse_tree_gently(r, tree, 1) < 0)\n>       - \t\treturn;\n>       -\n>       - \tinit_tree_desc(&desc, tree->buffer, tree->size);\n>       -\n>       - ## sparse-index.c ##\n>       -@@ sparse-index.c: static void set_index_entry(struct index_state *istate, int nr, struct cache_ent\n>       - \tadd_name_hash(istate, ce);\n>       - }\n>       -\n>       --static int add_path_to_index(const struct object_id *oid,\n>       -+static int add_path_to_index(struct repository *repo UNUSED, const struct object_id *oid,\n>       - \t\t\t     struct strbuf *base, const char *path,\n>       - \t\t\t     unsigned int mode, void *context)\n>       - {\n>       -\n>       - ## t/t5005-archive-submodules.sh (new) ##\n>       -@@\n>       -+#!/bin/sh\n>       -+\n>       -+test_description='git archive --recurse-submodules test'\n>       -+\n>       -+. ./test-lib.sh\n>       -+. \"$TEST_DIRECTORY\"/lib-submodule-update.sh\n>       -+\n>       -+test_expect_success 'setup' '\n>       -+\tcreate_lib_submodule_repo &&\n>       -+\tgit -C submodule_update_repo checkout valid_sub1 &&\n>       -+\tgit -C submodule_update_repo submodule update\n>       -+'\n>       -+\n>       -+check_tar() {\n>       -+\ttarfile=$1.tar\n>       -+\tlistfile=$1.lst\n>       -+\tdir=$1\n>       -+\tdir_with_prefix=$dir/$2\n>       -+\n>       -+\ttest_expect_success ' extract tar archive' '\n>       -+\t\t(mkdir $dir && cd $dir && \"$TAR\" xf -) <$tarfile\n>       -+\t'\n>       -+}\n>       -+\n>       -+check_added() {\n>       -+\tdir=$1\n>       -+\tpath_in_fs=$2\n>       -+\tpath_in_archive=$3\n>       -+\n>       -+\ttest_expect_success \" validate extra file $path_in_archive\" '\n>       -+\t\ttest -f $dir/$path_in_archive &&\n>       -+\t\tdiff -r $path_in_fs $dir/$path_in_archive\n>       -+\t'\n>       -+}\n>       -+\n>       -+check_not_added() {\n>       -+\tdir=$1\n>       -+\tpath_in_archive=$2\n>       -+\n>       -+\ttest_expect_success \" validate unpresent file $path_in_archive\" '\n>       -+\t\t! test -f $dir/$path_in_archive &&\n>       -+\t\t! test -d $dir/$path_in_archive\n>       -+\t'\n>       -+}\n>       -+\n>       -+test_expect_success 'archive without recurse, non-init' '\n>       -+\treset_work_tree_to valid_sub1 &&\n>       -+\tgit -C submodule_update archive HEAD >b.tar\n>       -+'\n>       -+\n>       -+check_tar b\n>       -+check_added b submodule_update/file1 file1\n>       -+check_not_added b sub1/file1\n>       -+\n>       -+test_expect_success 'archive with recurse, non-init' '\n>       -+\treset_work_tree_to valid_sub1 &&\n>       -+\t! git -C submodule_update archive --recurse-submodules HEAD >b2-err.tar\n>       -+'\n>       -+\n>       -+test_expect_success 'archive with recurse, init' '\n>       -+\treset_work_tree_to valid_sub1 &&\n>       -+\tgit -C submodule_update submodule update --init &&\n>       -+\tgit -C submodule_update ls-files --recurse-submodules &&\n>       -+\tgit -C submodule_update ls-tree HEAD &&\n>       -+\tgit -C submodule_update archive --recurse-submodules HEAD >b2.tar\n>       -+'\n>       -+\n>       -+check_tar b2\n>       -+check_added b2 submodule_update/sub1/file1 sub1/file1\n>       -+\n>       -+test_expect_success 'archive with recurse with big files' '\n>       -+\treset_work_tree_to valid_sub1 &&\n>       -+\ttest_config core.bigfilethreshold 1 &&\n>       -+\tgit -C submodule_update submodule update --init &&\n>       -+\tgit -C submodule_update ls-files --recurse-submodules &&\n>       -+\tgit -C submodule_update ls-tree HEAD &&\n>       -+\tgit -C submodule_update archive --recurse-submodules HEAD >b3.tar\n>       -+'\n>       -+\n>       -+check_tar b3\n>       -+check_added b3 submodule_update/sub1/file1 sub1/file1\n>       -+\n>       -+\n>       -+test_done\n>       -\n>       - ## tree.c ##\n>       -@@\n>       - #include \"alloc.h\"\n>       - #include \"tree-walk.h\"\n>       - #include \"repository.h\"\n>       -+#include \"pathspec.h\"\n>       -\n>       - const char *tree_type = \"tree\";\n>       -\n>       -@@ tree.c: int read_tree_at(struct repository *r,\n>       - \tint len, oldlen = base->len;\n>       - \tenum interesting retval = entry_not_interesting;\n>       -\n>       --\tif (parse_tree(tree))\n>       --\t\treturn -1;\n>       -+\tif (repo_parse_tree(r, tree))\n>       -+\t\tdie(\"Failed to parse tree\");\n>       -\n>       - \tinit_tree_desc(&desc, tree->buffer, tree->size);\n>       -\n>       -@@ tree.c: int read_tree_at(struct repository *r,\n>       - \t\t\t\tcontinue;\n>       - \t\t}\n>       -\n>       --\t\tswitch (fn(&entry.oid, base,\n>       -+\t\tswitch (fn(r, &entry.oid, base,\n>       - \t\t\t   entry.path, entry.mode, context)) {\n>       - \t\tcase 0:\n>       - \t\t\tcontinue;\n>       -@@ tree.c: int read_tree_at(struct repository *r,\n>       - \t\t\treturn -1;\n>       - \t\t}\n>       -\n>       --\t\tif (S_ISDIR(entry.mode))\n>       -+\t\tif (S_ISDIR(entry.mode)) {\n>       - \t\t\toidcpy(&oid, &entry.oid);\n>       --\t\telse if (S_ISGITLINK(entry.mode)) {\n>       -+\t\t\tlen = tree_entry_len(&entry);\n>       -+\t\t\tstrbuf_add(base, entry.path, len);\n>       -+\t\t\tstrbuf_addch(base, '/');\n>       -+\t\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n>       -+\t\t\t\t\t\tbase, pathspec,\n>       -+\t\t\t\t\t\tfn, context);\n>       -+\t\t\tstrbuf_setlen(base, oldlen);\n>       -+\t\t\tif (retval)\n>       -+\t\t\t\treturn -1;\n>       -+\t\t} else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) {\n>       - \t\t\tstruct commit *commit;\n>       -+\t\t\tstruct repository subrepo;\n>       -+\t\t\tstruct repository* subrepo_p = &subrepo;\n>       -+\t\t\tstruct tree* submodule_tree;\n>       -\n>       --\t\t\tcommit = lookup_commit(r, &entry.oid);\n>       -+\t\t\tif (repo_submodule_init(subrepo_p, r, entry.path, null_oid()))\n>       -+\t\t\t\tdie(\"couldn't init submodule %s%s\", base->buf, entry.path);\n>       -+\n>       -+\t\t\tif (repo_read_index(subrepo_p) < 0)\n>       -+\t\t\t\tdie(\"index file corrupt\");\n>       -+\n>       -+\t\t\tcommit = lookup_commit(subrepo_p, &entry.oid);\n>       - \t\t\tif (!commit)\n>       - \t\t\t\tdie(\"Commit %s in submodule path %s%s not found\",\n>       - \t\t\t\t    oid_to_hex(&entry.oid),\n>       - \t\t\t\t    base->buf, entry.path);\n>       -\n>       --\t\t\tif (parse_commit(commit))\n>       -+\t\t\tif (repo_parse_commit(subrepo_p, commit))\n>       - \t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n>       - \t\t\t\t    oid_to_hex(&entry.oid),\n>       - \t\t\t\t    base->buf, entry.path);\n>       -\n>       --\t\t\toidcpy(&oid, get_commit_tree_oid(commit));\n>       -+\t\t\tsubmodule_tree = repo_get_commit_tree(subrepo_p, commit);\n>       -+\t\t\toidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);\n>       -+\n>       -+\t\t\tlen = tree_entry_len(&entry);\n>       -+\t\t\tstrbuf_add(base, entry.path, len);\n>       -+\t\t\tstrbuf_addch(base, '/');\n>       -+\t\t\tretval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),\n>       -+\t\t\t\t\t\tbase, pathspec,\n>       -+\t\t\t\t\t\tfn, context);\n>       -+\t\t\tif (retval) {\n>       -+\t\t\t    die(\"failed to read tree for %s%s\", base->buf, entry.path);\n>       -+\t\t\t    return -1;\n>       -+\t\t\t}\n>       -+\t\t\tstrbuf_setlen(base, oldlen);\n>       -+\t\t\trepo_clear(subrepo_p);\n>       - \t\t}\n>       --\t\telse\n>       --\t\t\tcontinue;\n>       -\n>       --\t\tlen = tree_entry_len(&entry);\n>       --\t\tstrbuf_add(base, entry.path, len);\n>       --\t\tstrbuf_addch(base, '/');\n>       --\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n>       --\t\t\t\t      base, pathspec,\n>       --\t\t\t\t      fn, context);\n>       --\t\tstrbuf_setlen(base, oldlen);\n>       --\t\tif (retval)\n>       --\t\t\treturn -1;\n>       - \t}\n>       - \treturn 0;\n>       - }\n>       -@@ tree.c: int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size)\n>       - \treturn 0;\n>       - }\n>       -\n>       --int parse_tree_gently(struct tree *item, int quiet_on_missing)\n>       -+int parse_tree_gently(struct repository *r, struct tree *item, int quiet_on_missing)\n>       - {\n>       - \t enum object_type type;\n>       - \t void *buffer;\n>       -@@ tree.c: int parse_tree_gently(struct tree *item, int quiet_on_missing)\n>       -\n>       - \tif (item->object.parsed)\n>       - \t\treturn 0;\n>       --\tbuffer = read_object_file(&item->object.oid, &type, &size);\n>       -+\tbuffer = repo_read_object_file(r, &item->object.oid, &type, &size);\n>       - \tif (!buffer)\n>       - \t\treturn quiet_on_missing ? -1 :\n>       - \t\t\terror(\"Could not read %s\",\n>       -\n>       - ## tree.h ##\n>       -@@ tree.h: struct tree *lookup_tree(struct repository *r, const struct object_id *oid);\n>       -\n>       - int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size);\n>       -\n>       --int parse_tree_gently(struct tree *tree, int quiet_on_missing);\n>       --static inline int parse_tree(struct tree *tree)\n>       -+int parse_tree_gently(struct repository *r, struct tree *tree, int quiet_on_missing);\n>       -+static inline int repo_parse_tree(struct repository *r, struct tree *tree)\n>       - {\n>       --\treturn parse_tree_gently(tree, 0);\n>       -+\treturn parse_tree_gently(r, tree, 0);\n>       - }\n>       -+#ifndef NO_THE_REPOSITORY_COMPATIBILITY_MACROS\n>       -+#define parse_tree(tree) repo_parse_tree(the_repository, tree)\n>       -+#endif\n>       - void free_tree_buffer(struct tree *tree);\n>       -\n>       - /* Parses and returns the tree in the given ent, chasing tags and commits. */\n>       -@@ tree.h: struct tree *parse_tree_indirect(const struct object_id *oid);\n>       - int cmp_cache_name_compare(const void *a_, const void *b_);\n>       -\n>       - #define READ_TREE_RECURSIVE 1\n>       --typedef int (*read_tree_fn_t)(const struct object_id *, struct strbuf *, const char *, unsigned int, void *);\n>       -+typedef int (*read_tree_fn_t)(struct repository *, const struct object_id *, struct strbuf *, const char *, unsigned int, void *);\n>       -\n>       - int read_tree_at(struct repository *r,\n>       - \t\t struct tree *tree, struct strbuf *base,\n>       -\n>       - ## wt-status.c ##\n>       -@@ wt-status.c: static void wt_status_collect_changes_index(struct wt_status *s)\n>       - \trelease_revisions(&rev);\n>       - }\n>       -\n>       --static int add_file_to_list(const struct object_id *oid,\n>       -+static int add_file_to_list(struct repository *repo UNUSED, const struct object_id *oid,\n>       - \t\t\t    struct strbuf *base, const char *path,\n>       - \t\t\t    unsigned int mode, void *context)\n>       - {\n>    -:  ----------- >  7:  2443c9b1b6e archive: remove global repository from archive_args\n>    -:  ----------- >  8:  4672e3d9586 archive: add --recurse-submodules to git-archive command\n>    -:  ----------- >  9:  f88ebbaf17c archive: add tests for git archive --recurse-submodules\n> \n"},{"id":"465214","messageId":"xmqqo7u9t1zf.fsf@gitster.g","threadId":"58613","inReplyTo":"pull.1359.v3.git.git.1665973401.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 0/9] archive: Add --recurse-submodules to git-archive command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-18T18:34:28Z","receivedAt":"2022-10-18T18:34:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Today I was scheduled to be offline, so I won't dig further on the\nissues this topic has now, but the new tests this series introduces,\nnamely t1023 and t5005, both relies on being able to clone a nested\nsubmodule via file:// transport, which no longer is allowed.\n\nThe patches need to be updated to adjust to the new world order, of\ncourse, but we probably should take it as an example of what the\nmost recent update may be breaking for real world users.\n\nThanks.\n"},{"id":"465217","messageId":"CAFwvh9BwV=NEtF_gsh9Tr8gHz02=78m4MCBQbuXk5SzKCAm4RA@mail.gmail.com","threadId":"58613","inReplyTo":"xmqqo7u9t1zf.fsf@gitster.g","subject":"Re: [PATCH v3 0/9] archive: Add --recurse-submodules to git-archive command","fromName":"Heather Lapointe","fromEmail":"alpha@alphaservcomputing.solutions","sentAt":"2022-10-18T18:48:22Z","receivedAt":"2022-10-18T18:48:45Z","isPatch":true,"sender":{"key":"alpha@alphaservcomputing.solutions","avatar":null},"body":"Thanks for taking a look.\n\nOn Tue, Oct 18, 2022 at 2:34 PM Junio C Hamano <gitster@pobox.com> wrote:\n>\n> Today I was scheduled to be offline, so I won't dig further on the\n> issues this topic has now, but the new tests this series introduces,\n> namely t1023 and t5005, both relies on being able to clone a nested\n> submodule via file:// transport, which no longer is allowed.\n\nI was following the patterns of t/lib-submodule-update.sh. Are there\nbetter examples\nthat I can follow?\n\n> The patches need to be updated to adjust to the new world order, of\n> course, but we probably should take it as an example of what the\n> most recent update may be breaking for real world users.\n>\n> Thanks.\n"},{"id":"465253","messageId":"xmqqy1tbssa8.fsf@gitster.g","threadId":"58613","inReplyTo":"CAFwvh9BwV=NEtF_gsh9Tr8gHz02=78m4MCBQbuXk5SzKCAm4RA@mail.gmail.com","subject":"Re: [PATCH v3 0/9] archive: Add --recurse-submodules to git-archive command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-19T16:16:15Z","receivedAt":"2022-10-19T16:16:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Heather Lapointe <alpha@alphaservcomputing.solutions> writes:\n\n> Thanks for taking a look.\n>\n> On Tue, Oct 18, 2022 at 2:34 PM Junio C Hamano <gitster@pobox.com> wrote:\n>>\n>> Today I was scheduled to be offline, so I won't dig further on the\n>> issues this topic has now, but the new tests this series introduces,\n>> namely t1023 and t5005, both relies on being able to clone a nested\n>> submodule via file:// transport, which no longer is allowed.\n>\n> I was following the patterns of t/lib-submodule-update.sh. Are there\n> better examples\n> that I can follow?\n\nMimic what Taylor did to adjust to the new world order that was\nintroduced in the 2.38.1 update.\n\nLook at 9c32cfb4 (Sync with v2.38.1, 2022-10-17), which merges\n2.38.1 and updates the tests to adjust to the new world order, by\ncomparing the t/ directory of its first parent and the result of the\nmerge.  It shows what Taylor did to adjust the tests to adjust.\n\n    $ git diff 9c32cfb4^ 9c32cfb4 t/\n\nI personally doubt it is generally a good idea, as it sets a bad\npattern that tempts unsuspecting users to blindly copy and paste it\nto their $HOME/.gitconfig without realizing what its ramifications\nare, but the easiest workaround may be to mimic what was done in\nt/lib-submodule-update.sh that sets protocol.file.allow\nconfiguration knob globally.\n"},{"id":"465268","messageId":"xmqqwn8vpmr3.fsf@gitster.g","threadId":"58613","inReplyTo":"xmqqy1tbssa8.fsf@gitster.g","subject":"Re: [PATCH v3 0/9] archive: Add --recurse-submodules to git-archive command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-19T20:44:00Z","receivedAt":"2022-10-19T20:44:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> Mimic what Taylor did to adjust to the new world order that was\n> introduced in the 2.38.1 update.\n>\n> Look at 9c32cfb4 (Sync with v2.38.1, 2022-10-17), which merges\n> 2.38.1 and updates the tests to adjust to the new world order, by\n> comparing the t/ directory of its first parent and the result of the\n> merge.  It shows what Taylor did to adjust the tests to adjust.\n>\n>     $ git diff 9c32cfb4^ 9c32cfb4 t/\n>\n> I personally doubt it is generally a good idea, as it sets a bad\n> pattern that tempts unsuspecting users to blindly copy and paste it\n> to their $HOME/.gitconfig without realizing what its ramifications\n> are, but the easiest workaround may be to mimic what was done in\n> t/lib-submodule-update.sh that sets protocol.file.allow\n> configuration knob globally.\n\nI'll queue this at the tip of your topic when I rebuild 'seen' for\ntoday's integration run.\n\n t/t1023-tree-read-tree-at.sh  | 4 +++-\n t/t5005-archive-submodules.sh | 7 ++++---\n 2 files changed, 7 insertions(+), 4 deletions(-)\n\ndiff --git a/t/t1023-tree-read-tree-at.sh b/t/t1023-tree-read-tree-at.sh\nindex 9e5ce3abb4..cfe6c867e3 100755\n--- a/t/t1023-tree-read-tree-at.sh\n+++ b/t/t1023-tree-read-tree-at.sh\n@@ -32,7 +32,8 @@ test_expect_success 'read_tree basic' '\n '\n \n test_expect_success 'read_tree submodules' '\n-\trm -rf walk_tree_submodules &&\n+\tgit config --global protocol.file.allow always &&\n+\trm -rf submodule1 &&\n \tgit init submodule1 &&\n \t(\n \t\tcd submodule1 &&\n@@ -42,6 +43,7 @@ test_expect_success 'read_tree submodules' '\n \t\tgit add file1.txt dir1/dirA/file1.txt &&\n \t\tgit commit -m \"initial commit\"\n \t) &&\n+\trm -rf walk_tree_submodules &&\n \tgit init walk_tree_submodules &&\n \t(\n \t\tcd walk_tree_submodules &&\ndiff --git a/t/t5005-archive-submodules.sh b/t/t5005-archive-submodules.sh\nindex aad6cfd108..e1413e08a2 100755\n--- a/t/t5005-archive-submodules.sh\n+++ b/t/t5005-archive-submodules.sh\n@@ -4,7 +4,7 @@ test_description='git archive --recurse-submodules test'\n \n . ./test-lib.sh\n \n-check_tar() {\n+check_tar () {\n \ttarfile=$1.tar\n \tlistfile=$1.lst\n \tdir=$1\n@@ -15,7 +15,7 @@ check_tar() {\n \t'\n }\n \n-check_added() {\n+check_added () {\n \tdir=$1\n \tpath_in_fs=$2\n \tpath_in_archive=$3\n@@ -26,7 +26,7 @@ check_added() {\n \t'\n }\n \n-check_not_added() {\n+check_not_added () {\n \tdir=$1\n \tpath_in_archive=$2\n \n@@ -37,6 +37,7 @@ check_not_added() {\n }\n \n test_expect_success 'setup' '\n+\tgit config --global protocol.file.allow always &&\n \trm -rf repo_with_submodules submodule1 uninited_repo_with_submodules &&\n \tgit init repo_with_submodules &&\n \tgit init submodule1 &&\n-- \n2.38.1-236-gf47955814b\n\n"},{"id":"465275","messageId":"xmqqfsfjnvbu.fsf@gitster.g","threadId":"58613","inReplyTo":"xmqqwn8vpmr3.fsf@gitster.g","subject":"Re: [PATCH v3 0/9] archive: Add --recurse-submodules to git-archive command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-20T01:21:41Z","receivedAt":"2022-10-20T01:29:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n>> I personally doubt it is generally a good idea, as it sets a bad\n>> pattern that tempts unsuspecting users to blindly copy and paste it\n>> to their $HOME/.gitconfig without realizing what its ramifications\n>> are, but the easiest workaround may be to mimic what was done in\n>> t/lib-submodule-update.sh that sets protocol.file.allow\n>> configuration knob globally.\n>\n> I'll queue this at the tip of your topic when I rebuild 'seen' for\n> today's integration run.\n>\n>  t/t1023-tree-read-tree-at.sh  | 4 +++-\n>  t/t5005-archive-submodules.sh | 7 ++++---\n>  2 files changed, 7 insertions(+), 4 deletions(-)\n\nIt seems to have cleared the \"submodule tests no longer can use\nsubmodules with file:// without tweaking the config\" issue I saw\nearlier.  It seems to give us a segfault in win+VS test, though.\n\nhttps://github.com/git/git/actions/runs/3285647856/jobs/5413033844#step:5:245\n\nThanks.\n\n\n"},{"id":"465395","messageId":"xmqq8rlaossr.fsf@gitster.g","threadId":"58613","inReplyTo":"xmqqfsfjnvbu.fsf@gitster.g","subject":"Re: [PATCH v3 0/9] archive: Add --recurse-submodules to git-archive command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-21T01:43:16Z","receivedAt":"2022-10-21T01:43:24Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> It seems to have cleared the \"submodule tests no longer can use\n> submodules with file:// without tweaking the config\" issue I saw\n> earlier.  It seems to give us a segfault in win+VS test, though.\n>\n> https://github.com/git/git/actions/runs/3285647856/jobs/5413033844#step:5:245\n\nHere is a pair of CI run that attributes the breakage to this topic:\n\n  https://github.com/git/git/actions/runs/3293333066\n\nis one CI run on 'seen' that has this topic and everything else in\nflight.  This topic is at the tip of 'seen' when this CI run was\ndone, and win+VS test (8) seems to be failing the same way as the\nprevious one I reported earlier above.\n\nDropping the merge of this topic (i.e. \"git reset --hard HEAD^\") out\nof 'seen' and running CI again: \n\n  https://github.com/git/git/actions/runs/3293553109\n\nwe can see that all tests pass there, which unfortunately is a rare\nevent these days (well, the segfaulting code is something this topic\nadds, so it is not surprising that the rest of the topics in flight\nwould not segfault the same way).\n\nDo you need help from somebody equipped with Windows knowledge and\nbuild/test environment?  As I do not do Windows or macOS, I cannot\noffer to be one myself, but the development community is full of\ncapable folks and help is often a request away.\n\nThanks.\n"},{"id":"465844","messageId":"kl6lr0yuqlk0.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"58613","inReplyTo":"pull.1359.v3.git.git.1665973401.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 0/9] archive: Add --recurse-submodules to git-archive command","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-10-26T22:14:55Z","receivedAt":"2022-10-26T22:15:10Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Hi Heather!\n\nWe covered this series in Review Club [1]. We will leave review on\nthis thread, though you may find the notes [2] useful.\n\n[1] https://lore.kernel.org/git/kl6l35bbsubq.fsf@chooglen-macbookpro.roam.corp.google.com\n[2] https://docs.google.com/document/d/14L8BAumGTpsXpjDY8VzZ4rRtpAjuGrFSRqn3stCuS_w/edit?pli=1#\n\n\"Heather Lapointe via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> This makes it possible to include submodule contents in an archive command.\n>\n> The inspiration for this change comes from this Github thread,\n> https://github.com/dear-github/dear-github/issues/214, with at least 160\n> 👍🏻 's at the time of writing. (I stumbled upon it because I wanted it as\n> well).\n>\n> I figured the underlying implementation wouldn't be too difficult with most\n> of the plumbing already in place, so I decided to add the relevant logic to\n> the client git-archive command.\n>\n> One of the trickier parts of this implementation involved teaching read_tree\n> about submodules. Some of the troublesome areas were still using the\n> the_repository references to look up commit or tree or oid information. I\n> ended up deciding that read_tree_fn_t would probably be best off having a\n> concrete repo reference since it allows changing the context to a subrepo\n> where needed (even though some of the usages did not need it specifically).\n>\n> I am open to feedback since this is all quite new to me :)\n\nThe Review Club participants generally agreed that this is a really\nwell-structured and easy-to-follow series :) As far as new contributions\ngo, this is really good.\n\nI think this series broadly makes sense, i.e.:\n\n- the implementation of plumbing \"struct repository\" through read_tree()\n  (this might also be really helpful for future work)\n- the interface (using \"--recurse-submodules\")\n- the expected behavior\n\nSo I can see this going through with a bit of polish. The others have\ncovered style issues quite thoroughly, so I won't comment on those.\n"},{"id":"465846","messageId":"kl6lo7tyqkph.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"58613","inReplyTo":"79959a54eb4c1a0812b1f4643530069a63e549f4.1665973401.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 1/9] tree: do not use the_repository for tree traversal methods.","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-10-26T22:33:14Z","receivedAt":"2022-10-26T22:33:19Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"\"Alphadelta14 via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> diff --git a/tree.h b/tree.h\n> index 6efff003e21..cc6402e4738 100644\n> --- a/tree.h\n> +++ b/tree.h\n> @@ -18,15 +18,21 @@ struct tree *lookup_tree(struct repository *r, const struct object_id *oid);\n>  \n>  int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size);\n>  \n> -int parse_tree_gently(struct tree *tree, int quiet_on_missing);\n> -static inline int parse_tree(struct tree *tree)\n> +int repo_parse_tree_gently(struct repository *r, struct tree *tree, int quiet_on_missing);\n> +static inline int repo_parse_tree(struct repository *r, struct tree *tree)\n>  {\n> -\treturn parse_tree_gently(tree, 0);\n> +\treturn repo_parse_tree_gently(r, tree, 0);\n>  }\n> +\n> +#ifndef NO_THE_REPOSITORY_COMPATIBILITY_MACROS\n> +#define parse_tree(tree) repo_parse_tree(the_repository, tree)\n> +#define parse_tree_gently(tree, quiet_on_missing) repo_parse_tree_gently(the_repository, tree, quiet_on_missing)\n> +#define parse_tree_indirect(oid) repo_parse_tree_indirect(the_repository, oid)\n> +#endif\n\nTypically, when we have repo_* and non-repo_* variants, we use a \"static\ninline\" function, e.g. from refs.h:\n\n  int repo_dwim_ref(struct repository *r, const char *str, int len,\n        struct object_id *oid, char **ref, int nonfatal_dangling_mark);\n\n  static inline int dwim_ref(const char *str, int len, struct object_id *oid,\n          char **ref, int nonfatal_dangling_mark)\n  {\n    return repo_dwim_ref(the_repository, str, len, oid, ref,\n            nonfatal_dangling_mark);\n  }\n\nI think we should do the same here, instead of using \"#ifndef\nNO_THE_REPOSITORY_COMPATIBILITY_MACROS\".\n\nFrom I can gather from \"git log -S\nNO_THE_REPOSITORY_COMPATIBILITY_MACROS\", that macro was introduced\nin e675765235 (diff.c: remove implicit dependency on the_index,\n2018-09-21) and all instances of that macro were introduced around that\ntime. At the time, there was an effort to get rid of the_repository and\nthe_index almost everywhere (except builtins), and the macro would\nensure that we did this successfully.\n\nWe did such a good job with the_index that we flipped the default from\nNO_THE_INDEX_COMPATIBILITY_MACROS to USE_THE_INDEX_COMPATIBILITY_MACROS\n(f8adbec9fe (cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch,\n2019-01-24)) but it looks like we never got there with the_repository.\nI couldn't find any instances of \"#define\nNO_THE_REPOSITORY_COMPATIBILITY_MACROS\", so I think we should just use\n\"static inline\" instead.\n\nAlternatively, one could get rid of the non-repo_* variant and adjust\nall existing callers to use \"struct repository\", but that's a lot of\nchurn and may conflict with other in-flight series, so that's probably\nleft for another time.\n\n>  void free_tree_buffer(struct tree *tree);\n>  \n>  /* Parses and returns the tree in the given ent, chasing tags and commits. */\n> -struct tree *parse_tree_indirect(const struct object_id *oid);\n> +struct tree *repo_parse_tree_indirect(struct repository *r, const struct object_id *oid);\n>  \n>  int cmp_cache_name_compare(const void *a_, const void *b_);\n>  \n> -- \n> gitgitgadget\n"},{"id":"465847","messageId":"kl6lk04mqk0j.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"58613","inReplyTo":"d3d1738e670d5dbf1378fc5c3209b2e98234a771.1665973401.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 4/9] tree: handle submodule case for read_tree_at properly","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-10-26T22:48:12Z","receivedAt":"2022-10-26T22:48:25Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"\"Heather Lapointe via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> @@ -47,40 +48,73 @@ int read_tree_at(struct repository *r,\n>  \t\t\treturn -1;\n>  \t\t}\n>  \n> -\t\tif (S_ISDIR(entry.mode))\n> +\t\tif (S_ISDIR(entry.mode)) {\n>  \t\t\toidcpy(&oid, &entry.oid);\n> -\t\telse if (S_ISGITLINK(entry.mode)) {\n> -\t\t\tstruct commit *commit;\n>  \n> -\t\t\tcommit = lookup_commit(r, &entry.oid);\n> +\t\t\tlen = tree_entry_len(&entry);\n> +\t\t\tstrbuf_add(base, entry.path, len);\n> +\t\t\tstrbuf_addch(base, '/');\n> +\t\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n> +\t\t\t\t\t\tbase, pathspec,\n> +\t\t\t\t\t\tfn, context);\n> +\t\t\tstrbuf_setlen(base, oldlen);\n> +\t\t\tif (retval)\n> +\t\t\t\treturn -1;\n> +\t\t} else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) {\n> +\t\t\tstruct commit *commit;\n> +\t\t\tstruct repository subrepo;\n> +\t\t\tstruct repository* subrepo_p = &subrepo;\n> +\t\t\tstruct tree* submodule_tree;\n> +\t\t\tchar *submodule_rel_path;\n> +\t\t\tint name_base_len = 0;\n> +\n> +\t\t\tlen = tree_entry_len(&entry);\n> +\t\t\tstrbuf_add(base, entry.path, len);\n> +\t\t\tsubmodule_rel_path = base->buf;\n> +\t\t\t// repo_submodule_init expects a path relative to submodule_prefix\n> +\t\t\tif (r->submodule_prefix) {\n> +\t\t\t\tname_base_len = strlen(r->submodule_prefix);\n> +\t\t\t\t// we should always expect to start with submodule_prefix\n> +\t\t\t\tassert(!strncmp(submodule_rel_path, r->submodule_prefix, name_base_len));\n> +\t\t\t\t// strip the prefix\n> +\t\t\t\tsubmodule_rel_path += name_base_len;\n> +\t\t\t\t// if submodule_prefix doesn't end with a /, we want to get rid of that too\n> +\t\t\t\tif (is_dir_sep(submodule_rel_path[0])) {\n> +\t\t\t\t\tsubmodule_rel_path++;\n> +\t\t\t\t}\n> +\t\t\t}\n> +\n> +\t\t\tif (repo_submodule_init(subrepo_p, r, submodule_rel_path, null_oid()))\n> +\t\t\t\tdie(\"couldn't init submodule %s\", base->buf);\n> +\n> +\t\t\tif (repo_read_index(subrepo_p) < 0)\n> +\t\t\t\tdie(\"index file corrupt\");\n> +\n> +\t\t\tcommit = lookup_commit(subrepo_p, &entry.oid);\n>  \t\t\tif (!commit)\n> -\t\t\t\tdie(\"Commit %s in submodule path %s%s not found\",\n> +\t\t\t\tdie(\"Commit %s in submodule path %s not found\",\n>  \t\t\t\t    oid_to_hex(&entry.oid),\n> -\t\t\t\t    base->buf, entry.path);\n> -\n> -\t\t\t// FIXME: This is the wrong repo instance (it refers to the superproject)\n> -\t\t\t// it will always fail as is (will fix in later patch)\n> -\t\t\t// This current codepath isn't executed by any existing callbacks\n> -\t\t\t// so it wouldn't show up as an issue at this time.\n> -\t\t\tif (repo_parse_commit(r, commit))\n> -\t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n> +\t\t\t\t    base->buf);\n> +\n> +\t\t\tif (repo_parse_commit(subrepo_p, commit))\n> +\t\t\t\tdie(\"Invalid commit %s in submodule path %s\",\n>  \t\t\t\t    oid_to_hex(&entry.oid),\n> -\t\t\t\t    base->buf, entry.path);\n> +\t\t\t\t    base->buf);\n>  \n> -\t\t\toidcpy(&oid, get_commit_tree_oid(commit));\n> -\t\t}\n> -\t\telse\n> -\t\t\tcontinue;\n> +\t\t\tsubmodule_tree = repo_get_commit_tree(subrepo_p, commit);\n> +\t\t\toidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);\n>  \n> -\t\tlen = tree_entry_len(&entry);\n> -\t\tstrbuf_add(base, entry.path, len);\n> -\t\tstrbuf_addch(base, '/');\n> -\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n> -\t\t\t\t      base, pathspec,\n> -\t\t\t\t      fn, context);\n> -\t\tstrbuf_setlen(base, oldlen);\n> -\t\tif (retval)\n> -\t\t\treturn -1;\n> +\t\t\tstrbuf_addch(base, '/');\n> +\n> +\t\t\tretval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),\n> +\t\t\t\t\t\tbase, pathspec,\n> +\t\t\t\t\t\tfn, context);\n> +\t\t\tif (retval)\n> +\t\t\t    die(\"failed to read tree for %s\", base->buf);\n> +\t\t\tstrbuf_setlen(base, oldlen);\n> +\t\t\trepo_clear(subrepo_p);\n> +\t\t}\n> +\t\t// else, this is a file (or a submodule, but no pathspec->recurse_submodules)\n\nIn this patch, we say that we can ignore a submodule when\npathspec->recurse_submodules is 0, but unless I'm missing something, I\ndon't think that's the case. The preimage is:\n\n\t\telse if (S_ISGITLINK(entry.mode)) {\n\t\t\tstruct commit *commit;\n\n\t\t\tcommit = lookup_commit(r, &entry.oid);\n\t\t\tif (!commit)\n\t\t\t\tdie(\"Commit %s in submodule path %s%s not found\",\n\t\t\t\t    oid_to_hex(&entry.oid),\n\t\t\t\t    base->buf, entry.path);\n\n      /* ... */\n\t\t\tif (repo_parse_commit(r, commit))\n\t\t\t\tdie(\"Invalid commit %s in submodule path %s%s\",\n\t\t\t\t    oid_to_hex(&entry.oid),\n\t\t\t\t    base->buf, entry.path);\n\n\t\t\toidcpy(&oid, get_commit_tree_oid(commit));\n\t\t}\n\t\telse\n\t\t\tcontinue;\n\n\t\tlen = tree_entry_len(&entry);\n\t\tstrbuf_add(base, entry.path, len);\n\t\tstrbuf_addch(base, '/');\n\t\tretval = read_tree_at(r, lookup_tree(r, &oid),\n\t\t\t\t      base, pathspec,\n\t\t\t\t      fn, context);\n\nwhich isn't a no-op since we actually do recurse into the gitlink. I\ndon't know whether the subsequent call actually succeeds though (e.g.\nmaybe it always failed and it was just a de facto no-op?), but that's\nmuch harder to prove. Since this function has callers outside of \"git\narchive\", it would be better to be conservative and keep the original\nbehavior in the S_ISGITLINK(entry.mode) && !pathspec->recurse_submodules\ncase.\n\n>  \t}\n>  \treturn 0;\n>  }\n> -- \n> gitgitgadget\n"},{"id":"465849","messageId":"kl6lh6zqqhvm.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"58613","inReplyTo":"4672e3d958625cd76eb8056ab434e9a37f52661e.1665973401.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 8/9] archive: add --recurse-submodules to git-archive command","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-10-26T23:34:21Z","receivedAt":"2022-10-26T23:34:34Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"\"Heather Lapointe via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> index 34549d849f1..f81ef741487 100644\n> --- a/archive.c\n> +++ b/archive.c\n> @@ -213,6 +214,25 @@ static void queue_directory(const struct object_id *oid,\n>  \toidcpy(&d->oid, oid);\n>  }\n>  \n> +static void queue_submodule(\n> +\t\tstruct repository *superproject,\n> +\t\tconst struct object_id *oid,\n> +\t\tstruct strbuf *base, const char *filename,\n> +\t\tunsigned mode, struct archiver_context *c)\n> +{\n> +\tstruct repository subrepo;\n> +\n> +\tif (repo_submodule_init(&subrepo, superproject, filename, null_oid()))\n> +\t\treturn;\n> +\n> +\tif (repo_read_index(&subrepo) < 0)\n> +\t\tdie(\"index file corrupt\");\n> +\n> +    queue_directory(oid, base, filename, mode, c);\n> +\n> +\trepo_clear(&subrepo);\n> +}\n> +\n\nThis bit is puzzling to me because we init the submodule, read its\nindex, and then don't read objects from it at all. How does this work\nwhen we aren't reading objects from the submodule we init here? My guess\nis that read_tree() is already doing the heavy lifting of recursing into\nsubmodules, so we don't need to worry any more about init-ing submodules\nin archive.c, which is great.\n\nSo in effect, this is just checking whether we can read the submodule\nand its index. We can drop this check since we already do that check in\nread_tree().\n\nWhat's much more surprising is that you can delete the entire function\nbody (even queue_directory()!) and the tests still pass! The tests are\ndefinitely testing what they say they are (I've also checked the\ntarballs), so I'm not sure what's going on.\n\nI commented out queue_directory() in the S_ISDIR case, and the only test\nfailures I saw were:\n\n- t5000.68, which uses a glob in its pathspec. I tried using a glob for\n  in the archive submodule tests, but I couldn't reproduce the failure.\n- t5004.11, which is a really big test case that I didn't bother looking\n  deeply into.\n\nSo I'm at a loss as to what queue_directory() actually does. My best\nguess at a reproduction would be to make a subdirectory in t5000.68 a\nsubmodule. If we do find such a reproducing case, we should add it to\nthe test suite.\n\n>  static int write_directory(\n>  \t\tstruct repository *repo,\n>  \t\tstruct archiver_context *c)\n> @@ -228,9 +248,11 @@ static int write_directory(\n>  \t\twrite_directory(repo, c) ||\n>  \t\twrite_archive_entry(repo, &d->oid, d->path, d->baselen,\n>  \t\t\t\t    d->path + d->baselen, d->mode,\n> -\t\t\t\t    c) != READ_TREE_RECURSIVE;\n> +\t\t\t\t    c);\n>  \tfree(d);\n> -\treturn ret ? -1 : 0;\n> +\tif (ret == READ_TREE_RECURSIVE)\n> +\t\treturn 0;\n> +\treturn ret;\n>  }\n>  \n>  static int queue_or_write_archive_entry(\n> @@ -263,6 +285,11 @@ static int queue_or_write_archive_entry(\n>  \t\t\treturn 0;\n>  \t\tqueue_directory(oid, base, filename, mode, c);\n>  \t\treturn READ_TREE_RECURSIVE;\n> +\t} else if (c->args->recurse_submodules && S_ISGITLINK(mode)) {\n> +\t\tif (is_submodule_active(r, filename)) {\n> +\t\t\tqueue_submodule(r, oid, base, filename, mode, c);\n> +\t\t\treturn READ_TREE_RECURSIVE;\n> +\t\t}\n\nIf we are omitting inactive submodules from the archive, we should test\nthis behavior.\n\n>  \t}\n>  \n>  \tif (write_directory(r, c))\n> @@ -446,6 +473,7 @@ static void parse_pathspec_arg(\n>  \t\t       PATHSPEC_PREFER_FULL,\n>  \t\t       \"\", pathspec);\n>  \tar_args->pathspec.recursive = 1;\n> +\tar_args->pathspec.recurse_submodules = ar_args->recurse_submodules;\n>  \tif (pathspec) {\n>  \t\twhile (*pathspec) {\n>  \t\t\tif (**pathspec && !path_exists(repo, ar_args, *pathspec))\n> @@ -609,6 +637,7 @@ static int parse_archive_args(int argc, const char **argv,\n>  \tint verbose = 0;\n>  \tint i;\n>  \tint list = 0;\n> +\tint recurse_submodules = 0;\n>  \tint worktree_attributes = 0;\n>  \tstruct option opts[] = {\n>  \t\tOPT_GROUP(\"\"),\n> @@ -623,6 +652,8 @@ static int parse_archive_args(int argc, const char **argv,\n>  \t\t  add_file_cb, (intptr_t)&base },\n>  \t\tOPT_STRING('o', \"output\", &output, N_(\"file\"),\n>  \t\t\tN_(\"write the archive to this file\")),\n> +\t\tOPT_BOOL(0, \"recurse-submodules\", &recurse_submodules,\n> +\t\t\tN_(\"include submodules in archive\")),\n>  \t\tOPT_BOOL(0, \"worktree-attributes\", &worktree_attributes,\n>  \t\t\tN_(\"read .gitattributes in working directory\")),\n>  \t\tOPT__VERBOSE(&verbose, N_(\"report archived files on stderr\")),\n> @@ -686,6 +717,7 @@ static int parse_archive_args(int argc, const char **argv,\n>  \targs->verbose = verbose;\n>  \targs->base = base;\n>  \targs->baselen = strlen(base);\n> +\targs->recurse_submodules = recurse_submodules;\n>  \targs->worktree_attributes = worktree_attributes;\n>  \n>  \treturn argc;\n> diff --git a/archive.h b/archive.h\n> index 540a3b12130..1b21484dda6 100644\n> --- a/archive.h\n> +++ b/archive.h\n> @@ -18,6 +18,7 @@ struct archiver_args {\n>  \ttimestamp_t time;\n>  \tstruct pathspec pathspec;\n>  \tunsigned int verbose : 1;\n> +\tunsigned int recurse_submodules : 1;\n>  \tunsigned int worktree_attributes : 1;\n>  \tunsigned int convert : 1;\n>  \tint compression_level;\n> -- \n> gitgitgadget\n"},{"id":"465865","messageId":"fa90e7cf-4ddc-573a-1859-9e2b6b203f95@web.de","threadId":"58613","inReplyTo":"kl6lh6zqqhvm.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [PATCH v3 8/9] archive: add --recurse-submodules to git-archive command","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2022-10-27T07:09:06Z","receivedAt":"2022-10-27T07:09:19Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 27.10.22 um 01:34 schrieb Glen Choo:\n> \"Heather Lapointe via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n>\n>> index 34549d849f1..f81ef741487 100644\n>> --- a/archive.c\n>> +++ b/archive.c\n>> @@ -213,6 +214,25 @@ static void queue_directory(const struct object_id *oid,\n>>  \toidcpy(&d->oid, oid);\n>>  }\n>>\n>> +static void queue_submodule(\n>> +\t\tstruct repository *superproject,\n>> +\t\tconst struct object_id *oid,\n>> +\t\tstruct strbuf *base, const char *filename,\n>> +\t\tunsigned mode, struct archiver_context *c)\n>> +{\n>> +\tstruct repository subrepo;\n>> +\n>> +\tif (repo_submodule_init(&subrepo, superproject, filename, null_oid()))\n>> +\t\treturn;\n>> +\n>> +\tif (repo_read_index(&subrepo) < 0)\n>> +\t\tdie(\"index file corrupt\");\n>> +\n>> +    queue_directory(oid, base, filename, mode, c);\n>> +\n>> +\trepo_clear(&subrepo);\n>> +}\n>> +\n\n> What's much more surprising is that you can delete the entire function\n> body (even queue_directory()!) and the tests still pass! The tests are\n> definitely testing what they say they are (I've also checked the\n> tarballs), so I'm not sure what's going on.\n>\n> I commented out queue_directory() in the S_ISDIR case, and the only test\n> failures I saw were:\n>\n> - t5000.68, which uses a glob in its pathspec. I tried using a glob for\n>   in the archive submodule tests, but I couldn't reproduce the failure.\n> - t5004.11, which is a really big test case that I didn't bother looking\n>   deeply into.\n>\n> So I'm at a loss as to what queue_directory() actually does.\nAn archive doesn't strictly need directory entries.  If it contains a\nfile with a deeply nested path then extractors will create the parent\ndirectory hierarchy regardless.  diff(1) won't notice any difference.\nDirectory entries are mainly included to specify the permission bits.\n\nt5000.68 checks for the directory entries in the output given by the\noption --verbose of git archive.  t5004.11 checks the number of archive\nentries (including directories) using \"zipinfo -h\".\n\nRené\n"},{"id":"465882","messageId":"kl6lczadqins.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"58613","inReplyTo":"fa90e7cf-4ddc-573a-1859-9e2b6b203f95@web.de","subject":"Re: [PATCH v3 8/9] archive: add --recurse-submodules to git-archive command","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-10-27T17:29:43Z","receivedAt":"2022-10-27T17:29:50Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"René Scharfe <l.s.r@web.de> writes:\n\n> Am 27.10.22 um 01:34 schrieb Glen Choo:\n>> \"Heather Lapointe via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n>>\n>>> index 34549d849f1..f81ef741487 100644\n>>> --- a/archive.c\n>>> +++ b/archive.c\n>>> @@ -213,6 +214,25 @@ static void queue_directory(const struct object_id *oid,\n>>>  \toidcpy(&d->oid, oid);\n>>>  }\n>>>\n>>> +static void queue_submodule(\n>>> +\t\tstruct repository *superproject,\n>>> +\t\tconst struct object_id *oid,\n>>> +\t\tstruct strbuf *base, const char *filename,\n>>> +\t\tunsigned mode, struct archiver_context *c)\n>>> +{\n>>> +\tstruct repository subrepo;\n>>> +\n>>> +\tif (repo_submodule_init(&subrepo, superproject, filename, null_oid()))\n>>> +\t\treturn;\n>>> +\n>>> +\tif (repo_read_index(&subrepo) < 0)\n>>> +\t\tdie(\"index file corrupt\");\n>>> +\n>>> +    queue_directory(oid, base, filename, mode, c);\n>>> +\n>>> +\trepo_clear(&subrepo);\n>>> +}\n>>> +\n>\n>> What's much more surprising is that you can delete the entire function\n>> body (even queue_directory()!) and the tests still pass! The tests are\n>> definitely testing what they say they are (I've also checked the\n>> tarballs), so I'm not sure what's going on.\n>>\n>> I commented out queue_directory() in the S_ISDIR case, and the only test\n>> failures I saw were:\n>>\n>> - t5000.68, which uses a glob in its pathspec. I tried using a glob for\n>>   in the archive submodule tests, but I couldn't reproduce the failure.\n>> - t5004.11, which is a really big test case that I didn't bother looking\n>>   deeply into.\n>>\n>> So I'm at a loss as to what queue_directory() actually does.\n> An archive doesn't strictly need directory entries.  If it contains a\n> file with a deeply nested path then extractors will create the parent\n> directory hierarchy regardless.  diff(1) won't notice any difference.\n> Directory entries are mainly included to specify the permission bits.\n\nThanks. In that case, we should probably also test the case where there\nare empty directories (e.g. when a file is excluded by a pathspec), and\nwe should also check the permission bits.\n\n>\n> t5000.68 checks for the directory entries in the output given by the\n> option --verbose of git archive.  t5004.11 checks the number of archive\n> entries (including directories) using \"zipinfo -h\".\n>\n> René\n"},{"id":"465883","messageId":"kl6lbkpxqim1.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"58613","inReplyTo":"fa90e7cf-4ddc-573a-1859-9e2b6b203f95@web.de","subject":"Re: [PATCH v3 8/9] archive: add --recurse-submodules to git-archive command","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-10-27T17:30:46Z","receivedAt":"2022-10-27T17:30:54Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"René Scharfe <l.s.r@web.de> writes:\n\n> Am 27.10.22 um 01:34 schrieb Glen Choo:\n>> \"Heather Lapointe via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n>>\n>>> index 34549d849f1..f81ef741487 100644\n>>> --- a/archive.c\n>>> +++ b/archive.c\n>>> @@ -213,6 +214,25 @@ static void queue_directory(const struct object_id *oid,\n>>>  \toidcpy(&d->oid, oid);\n>>>  }\n>>>\n>>> +static void queue_submodule(\n>>> +\t\tstruct repository *superproject,\n>>> +\t\tconst struct object_id *oid,\n>>> +\t\tstruct strbuf *base, const char *filename,\n>>> +\t\tunsigned mode, struct archiver_context *c)\n>>> +{\n>>> +\tstruct repository subrepo;\n>>> +\n>>> +\tif (repo_submodule_init(&subrepo, superproject, filename, null_oid()))\n>>> +\t\treturn;\n>>> +\n>>> +\tif (repo_read_index(&subrepo) < 0)\n>>> +\t\tdie(\"index file corrupt\");\n>>> +\n>>> +    queue_directory(oid, base, filename, mode, c);\n>>> +\n>>> +\trepo_clear(&subrepo);\n>>> +}\n>>> +\n>\n>> What's much more surprising is that you can delete the entire function\n>> body (even queue_directory()!) and the tests still pass! The tests are\n>> definitely testing what they say they are (I've also checked the\n>> tarballs), so I'm not sure what's going on.\n>>\n>> I commented out queue_directory() in the S_ISDIR case, and the only test\n>> failures I saw were:\n>>\n>> - t5000.68, which uses a glob in its pathspec. I tried using a glob for\n>>   in the archive submodule tests, but I couldn't reproduce the failure.\n>> - t5004.11, which is a really big test case that I didn't bother looking\n>>   deeply into.\n>>\n>> So I'm at a loss as to what queue_directory() actually does.\n> An archive doesn't strictly need directory entries.  If it contains a\n> file with a deeply nested path then extractors will create the parent\n> directory hierarchy regardless.  diff(1) won't notice any difference.\n> Directory entries are mainly included to specify the permission bits.\n\nThanks. In that case, we should probably also test the case where there\nare empty directories (e.g. when a file is excluded by a pathspec), and\nwe should also check the permission bits.\n\n>\n> t5000.68 checks for the directory entries in the output given by the\n> option --verbose of git archive.  t5004.11 checks the number of archive\n> entries (including directories) using \"zipinfo -h\".\n>\n> René\n"},{"id":"465884","messageId":"kl6l8rl1qihg.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"58613","inReplyTo":"fa90e7cf-4ddc-573a-1859-9e2b6b203f95@web.de","subject":"Re: [PATCH v3 8/9] archive: add --recurse-submodules to git-archive command","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-10-27T17:33:31Z","receivedAt":"2022-10-27T17:33:36Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"René Scharfe <l.s.r@web.de> writes:\n\n> Am 27.10.22 um 01:34 schrieb Glen Choo:\n>> \"Heather Lapointe via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n>>\n>>> index 34549d849f1..f81ef741487 100644\n>>> --- a/archive.c\n>>> +++ b/archive.c\n>>> @@ -213,6 +214,25 @@ static void queue_directory(const struct object_id *oid,\n>>>  \toidcpy(&d->oid, oid);\n>>>  }\n>>>\n>>> +static void queue_submodule(\n>>> +\t\tstruct repository *superproject,\n>>> +\t\tconst struct object_id *oid,\n>>> +\t\tstruct strbuf *base, const char *filename,\n>>> +\t\tunsigned mode, struct archiver_context *c)\n>>> +{\n>>> +\tstruct repository subrepo;\n>>> +\n>>> +\tif (repo_submodule_init(&subrepo, superproject, filename, null_oid()))\n>>> +\t\treturn;\n>>> +\n>>> +\tif (repo_read_index(&subrepo) < 0)\n>>> +\t\tdie(\"index file corrupt\");\n>>> +\n>>> +    queue_directory(oid, base, filename, mode, c);\n>>> +\n>>> +\trepo_clear(&subrepo);\n>>> +}\n>>> +\n>\n>> What's much more surprising is that you can delete the entire function\n>> body (even queue_directory()!) and the tests still pass! The tests are\n>> definitely testing what they say they are (I've also checked the\n>> tarballs), so I'm not sure what's going on.\n>>\n>> I commented out queue_directory() in the S_ISDIR case, and the only test\n>> failures I saw were:\n>>\n>> - t5000.68, which uses a glob in its pathspec. I tried using a glob for\n>>   in the archive submodule tests, but I couldn't reproduce the failure.\n>> - t5004.11, which is a really big test case that I didn't bother looking\n>>   deeply into.\n>>\n>> So I'm at a loss as to what queue_directory() actually does.\n> An archive doesn't strictly need directory entries.  If it contains a\n> file with a deeply nested path then extractors will create the parent\n> directory hierarchy regardless.  diff(1) won't notice any difference.\n> Directory entries are mainly included to specify the permission bits.\n\nThanks. In that case, we should probably also test the case where there\nare empty directories (e.g. when a file is excluded by a pathspec), and\nwe should also check the permission bits.\n\n>\n> t5000.68 checks for the directory entries in the output given by the\n> option --verbose of git archive.  t5004.11 checks the number of archive\n> entries (including directories) using \"zipinfo -h\".\n>\n> René\n"},{"id":"465888","messageId":"20221027180949.1837457-1-jonathantanmy@google.com","threadId":"58613","inReplyTo":"79959a54eb4c1a0812b1f4643530069a63e549f4.1665973401.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 1/9] tree: do not use the_repository for tree traversal methods.","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2022-10-27T18:09:49Z","receivedAt":"2022-10-27T18:09:57Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"First of all, let me echo what Glen said [1], that this series is  \noverall well laid out and makes sense. \n\nOther reviewers have commented on style issues, but I'll hold off on \nmaking my comments on those and also possible improvements on commit \nmessages until I can say \"besides style and commit messages, I think \nthat this series is good to merge in\". \n\n[1] https://lore.kernel.org/git/kl6lr0yuqlk0.fsf@chooglen-macbookpro.roam.corp.google.com/\n\n\"Alphadelta14 via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> +\t\t\t// This current codepath isn't executed by any existing callbacks\n> +\t\t\t// so it wouldn't show up as an issue at this time.\n\nI was a bit confused by this comment, so I looked at the surrounding  \ncode. I think it could be better rephrased as: \n\n  All existing callbacks at the time of writing cause this part of the  \n  code to be skipped when S_ISGITLINK(entry.mode) is true, so this \n  wrong behavior does not call any issues. \n \n"},{"id":"465890","messageId":"20221027182857.1839448-1-jonathantanmy@google.com","threadId":"58613","inReplyTo":"9a07c6932f4c7ef844df1fc4f5b6b9feb1810135.1665973401.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 3/9] tree: increase test coverage for tree.c","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2022-10-27T18:28:57Z","receivedAt":"2022-10-27T18:29:39Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"\"Heather Lapointe via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> +\tsetup_git_directory();\n> +\trepo = the_repository;\n> +\tassert(repo);\n> +\n> +\tparse_pathspec(&pathspec, 0,\n> +\t\t       PATHSPEC_PREFER_FULL,\n> +\t\t       \"\", argv);\n\nHere, the repository is hardcoded to be \"the_repository\", and the C \ncode allows the pathspec to be varied but the shell test code always \nspecifies \".\" as the pathspec. Given that one of the main points of \nthis series is the repo varying, could the repo be taken in as a CLI \nargument? The pathspec can be left variable, but if it's not going to \nchange, you might as well hardcode it in the C code. \n\nThe existing test cases of a basic one and a recursing one is good, but \nit would be good also to have one where the repo being passed into the \nfunction is not the repo whose directory we're currently executing in \n(that is, different to the_repository). That way we can test that the \nfunction works for arbitrary repositories.\n\n \n"},{"id":"465891","messageId":"20221027184335.1840969-1-jonathantanmy@google.com","threadId":"58613","inReplyTo":"d3d1738e670d5dbf1378fc5c3209b2e98234a771.1665973401.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 4/9] tree: handle submodule case for read_tree_at properly","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2022-10-27T18:43:35Z","receivedAt":"2022-10-27T18:43:39Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"\"Heather Lapointe via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> From: Heather Lapointe <alpha@alphaservcomputing.solutions>\n> \n> This supports traversal into an actual submodule for read_tree_at.\n> The logic is blocked on pathspec->recurse_submodules now,\n\nWhat do you mean by \"blocked\"? Do you mean only that \npathspec->recurse_submodules needs to be specified, or do you also mean \nthat no caller specifies pathspec->recurse_submodules now, so this code \npath is never executed? \n\n> but previously hadn't been executed due to all fn() cases\n> returning early for submodules.\n\nWhat do you mean by \"previously hadn't been executed due to...\"? At a \nglance, I would think that this new logic is introduced in this patch, \nso of course it would never have been previously executed. \n\n> +\t\t\tif (repo_submodule_init(subrepo_p, r, submodule_rel_path, null_oid()))\n\nI don't think this can be null_oid() here - it has to match the tree \nfrom which you constructed submodule_rel_path. (That tree may not be \nthe tree at HEAD.) \n \n"},{"id":"465893","messageId":"xmqqwn8lksno.fsf@gitster.g","threadId":"58613","inReplyTo":"20221027180949.1837457-1-jonathantanmy@google.com","subject":"Re: [PATCH v3 1/9] tree: do not use the_repository for tree traversal methods.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-27T18:50:19Z","receivedAt":"2022-10-27T18:51:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan <jonathantanmy@google.com> writes:\n\n> First of all, let me echo what Glen said [1], that this series is  \n> overall well laid out and makes sense. \n>\n> Other reviewers have commented on style issues, but I'll hold off on \n> making my comments on those and also possible improvements on commit \n> messages until I can say \"besides style and commit messages, I think \n> that this series is good to merge in\". \n>\n> [1] https://lore.kernel.org/git/kl6lr0yuqlk0.fsf@chooglen-macbookpro.roam.corp.google.com/\n>\n> \"Alphadelta14 via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n>> +\t\t\t// This current codepath isn't executed by any existing callbacks\n>> +\t\t\t// so it wouldn't show up as an issue at this time.\n>\n> I was a bit confused by this comment, so I looked at the surrounding  \n> code. I think it could be better rephrased as: \n>\n>   All existing callbacks at the time of writing cause this part of the  \n>   code to be skipped when S_ISGITLINK(entry.mode) is true, so this \n>   wrong behavior does not call any issues. \n>  \n\nAs I already said, I do not think this is \"wrong behaviour\" to begin\nwith.  The current code requires that you'd use add_submodule_odb()\nto make the objects in them accessible and if your program fails to\ndo so, as a very natural consequence, you'd not see objects pointed\nby the gitlink.\n\nChanging that assumption is OK as long as existing callers that\ndepend on the current semantics are not broken by such a change, but\nI do not think \"wrong behaviour does not call any issues\" is a\ncorrect analysis of the problem.\n"},{"id":"465894","messageId":"20221027185437.1842331-1-jonathantanmy@google.com","threadId":"58613","inReplyTo":"f88ebbaf17cbf1a0b57336430bd43ade94406f38.1665973401.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 9/9] archive: add tests for git archive --recurse-submodules","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2022-10-27T18:54:37Z","receivedAt":"2022-10-27T18:54:51Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"\"Heather Lapointe via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> diff --git a/archive.c b/archive.c\n> index f81ef741487..b0a3181f7f5 100644\n> --- a/archive.c\n> +++ b/archive.c\n> @@ -179,7 +179,7 @@ static int write_archive_entry(\n>  \t\terr = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, 0);\n>  \t\tif (err)\n>  \t\t\treturn err;\n> -\t\treturn (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);\n> +\t\treturn READ_TREE_RECURSIVE;\n\nShould this change be in the previous commit, if this commit is about \ntests? \n\n> +check_tar() {\n> +\ttarfile=$1.tar\n> +\tlistfile=$1.lst\n> +\tdir=$1\n> +\tdir_with_prefix=$dir/$2\n> +\n> +\ttest_expect_success ' extract tar archive' '\n> +\t\t(mkdir $dir && cd $dir && \"$TAR\" xf -) <$tarfile\n> +\t'\n> +}\n\nIn the Git test codebase, there is a mix of styles in that some people \nwant each test_expect_success block to be individually runnable (I am \none of them), and to some, that's not as important. But this is \nextremely in the other direction. It would be better if each \ntest_expect_success block tests one thing, but inspecting the resulting \narchive should all go into the same test_expect_success block that \noriginally created the archive; we should not split each step of \ninspection into its own block. \n\nAlso, I don't think we need to extract the tar to check it; using \"tf\" \nand inspecting the resulting list with \"grep\" and \"! grep\" should do. \n\n"},{"id":"465917","messageId":"kl6ledusdety.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"58613","inReplyTo":"20221027185437.1842331-1-jonathantanmy@google.com","subject":"Re: [PATCH v3 9/9] archive: add tests for git archive --recurse-submodules","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-10-27T23:30:49Z","receivedAt":"2022-10-27T23:31:05Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Jonathan Tan <jonathantanmy@google.com> writes:\n\n> Also, I don't think we need to extract the tar to check it; using \"tf\" \n> and inspecting the resulting list with \"grep\" and \"! grep\" should do. \n\nIt might be even easier to inspect the output with test_cmp, instead of\nreasoning about which files should and should not be there.\n"},{"id":"465921","messageId":"221028.86wn8k94pu.gmgdl@evledraar.gmail.com","threadId":"58613","inReplyTo":"f88ebbaf17cbf1a0b57336430bd43ade94406f38.1665973401.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 9/9] archive: add tests for git archive --recurse-submodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-28T00:17:11Z","receivedAt":"2022-10-28T00:23:05Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Oct 17 2022, Heather Lapointe via GitGitGadget wrote:\n\n> From: Heather Lapointe <alpha@alphaservcomputing.solutions>\n\n[In addition to what others mentioned]\n\n> +test_description='git archive --recurse-submodules test'\n> +\n> +. ./test-lib.sh\n> +\n> +check_tar() {\n> +\ttarfile=$1.tar\n> +\tlistfile=$1.lst\n\nThis \"listfile\" is used nowhere?\"\n\n> +\tdir=$1\n> +\tdir_with_prefix=$dir/$2\n\nNor dir_with_prefix?\n\n> +\n> +\ttest_expect_success ' extract tar archive' '\n> +\t\t(mkdir $dir && cd $dir && \"$TAR\" xf -) <$tarfile\n\nAside from what Jonathan mentioned, maybe we can just use one variable\nhere then?\n\n\tmkdir $foo ... <$foo.tar\n\n> +\ttest_expect_success \" validate extra file $path_in_archive\" '\n> +\t\ttest -f $dir/$path_in_archive &&\n\nInstead use \"test_path_is_file\", and in general for \"test <whatever>\"\ncheck out if we have a wrapper in test-lib-functions.sh.\n\n> +check_not_added() {\n> +\tdir=$1\n> +\tpath_in_archive=$2\n> +\n> +\ttest_expect_success \" validate unpresent file $path_in_archive\" '\n> +\t\t! test -f $dir/$path_in_archive &&\n> +\t\t! test -d $dir/$path_in_archive\n\nDon't test for what a thing isn't, but what it is. Can't we do that\nhere?\n\n> +test_expect_success 'setup' '\n> +\trm -rf repo_with_submodules submodule1 uninited_repo_with_submodules &&\n\nDon't have a test rm -rf stuff from a previous block, but have\n\"test_when_finished\" clean up after that previous test instead.\n\n> +\tgit init repo_with_submodules &&\n> +\tgit init submodule1 &&\n> +\t(\n> +\t\tcd submodule1 &&\n\nThis:\n> +\t\techo \"dir1/sub1/file1.txt\" > \"file1.txt\" &&\n> +\t\tgit add file1.txt &&\n> +\t\tgit commit -m \"initialize with file1.txt\"\n\nLooks like you can use test_commit instead.\n\nAnd note you can use -C, so you won't need the sub-shell either, I think.\n> +\t) &&\n> +\t(\n> +\t    cd repo_with_submodules &&\n> +\t    echo \"file2\" > file2.txt &&\n> +\t    git add file2.txt &&\n> +\t    git commit -m \"initialize with file2.txt\" &&\n\nDitto.\n\n> +\t    mkdir -p dir1 &&\n\nLet's drop \"-p\" here, to check for errors.\n\n> +test_expect_success 'archive with recurse, non-init' '\n> +\t! git -C uninited_repo_with_submodules archive --recurse-submodules -v HEAD >b2-err.tar\n\nFor git, don't use !, use test_must_fail, ! hides segfaults.\n\nDoes this test pass when you build with SANITIZE=leak? Then you can do this at the top:\n\n\tTEST_PASSES_SANITIZE_LEAK=true\n\t. ./test-lib.sh\n\nIf you can't test that locally pushing to GitHub CI will tell you...\n"},{"id":"465964","messageId":"CAFwvh9A7V_OC-63f2mqEot4ybHeDUWQ5hvFUet3K+XCc3Xva_A@mail.gmail.com","threadId":"58613","inReplyTo":"kl6lr0yuqlk0.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [PATCH v3 0/9] archive: Add --recurse-submodules to git-archive command","fromName":"Heather Lapointe","fromEmail":"hthralpha@gmail.com","sentAt":"2022-10-28T18:18:19Z","receivedAt":"2022-10-28T18:18:34Z","isPatch":true,"sender":{"key":"hthralpha@gmail.com","avatar":null},"body":"On Wed, Oct 26, 2022 at 6:15 PM Glen Choo <chooglen@google.com> wrote:\n\n> The Review Club participants generally agreed that this is a really\n> well-structured and easy-to-follow series :) As far as new contributions\n> go, this is really good.\n>\n> I think this series broadly makes sense, i.e.:\n>\n> - the implementation of plumbing \"struct repository\" through read_tree()\n>   (this might also be really helpful for future work)\n> - the interface (using \"--recurse-submodules\")\n> - the expected behavior\n>\n> So I can see this going through with a bit of polish. The others have\n> covered style issues quite thoroughly, so I won't comment on those.\n\nThank you! I've started looking through a lot of these!\nI have been a bit swamped with my own work or I would have contributed\nanother patch series by now.\n"}]}