{"thread":{"id":"52970","subject":"[PATCH 0/8] Additional metadata for filter processes","startedAt":"2020-03-10T18:22:11Z","lastAt":"2020-03-15T19:30:57Z","messageCount":14,"participants":["brian m. carlson","Junio C Hamano","SZEDER Gábor"],"isPatch":true,"patchVersion":1,"patchTotal":8},"messages":[{"id":"393016","messageId":"20200310182046.748959-1-sandals@crustytoothpaste.net","threadId":"52970","inReplyTo":null,"subject":"[PATCH 0/8] Additional metadata for filter processes","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-03-10T18:20:38Z","receivedAt":"2020-03-10T18:22:11Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Smudge and clean filters are currently provided with one particular\npiece of data: the pathname of the file being smudged.  While this is\nhelpful, there are a variety of situations where people would like to\nhave more data.\n\nOne such situation is for users who would like to have a custom\nident-style filter that contains the branch name.  In many cases, it's\nsufficient to look up this information based on HEAD, but during\ncheckout, HEAD does not point to the right place, since it's updated\nafter the files are written.\n\nOther information users frequently want to know is the commit's object\nID and the object ID of the blob being filtered.  For example, if\nfiltering is expensive and the filter process sees duplicate blobs\nduring checkout, it may cache the results and avoid having to compute\nthe filter twice.\n\nThis series provides an additional set of metadata to the filter\nprocess with the keys \"ref\", \"treeish\", and \"blob\".  We prefer to\nprovide a commit as the treeish whenever possible, but in some cases,\nsuch as when git archive is invoked with a tree, there is no commit, and\nwe use the tree instead.\n\nNote that we don't provide this metadata in all cases.  Sometimes it is\ntrivial for the filter to do a simple \"git rev-parse HEAD\", and in such\ncases, metadata other than the blob may not be provided.  We also don't\nhandle the case where the user is using a smudge or clean command\ninstead of a filter process command: if the user wants the additional\nmetadata, it should be possible for them to write a small filter\nprocess, which is reasonably trivial in most languages.  Our\ndocumentation already permits us to add additional metadata and\nguarantees only that the pathname will be provided.\n\nMy particular use case for this is prefetching and precomputing data\nduring archive generation, since we don't permit delayed filters there\ndue to archives needing to be in a predictable order.  I have tried to\nmake it as generally applicable as possible, since I can imagine (and\nhave indeed seen requests for) many other useful applications of this\nelsewhere.\n\nFeedback is of course welcome.\n\nbrian m. carlson (8):\n  builtin/checkout: pass branch info down to checkout_worktree\n  convert: permit passing additional metadata to filter processes\n  convert: provide additional metadata to filters\n  builtin/checkout: compute checkout metadata for checkouts\n  builtin/clone: compute checkout metadata for clones\n  builtin/rebase: compute checkout metadata for rebases\n  builtin/reset: compute checkout metadata for reset\n  t0021: test filter metadata for additional cases\n\n apply.c                 |   2 +-\n archive.c               |  13 ++-\n archive.h               |   1 +\n builtin/cat-file.c      |   5 +-\n builtin/checkout.c      |  54 +++++++----\n builtin/clone.c         |   6 +-\n builtin/rebase.c        |   1 +\n builtin/reset.c         |  16 +++-\n cache.h                 |   1 +\n convert.c               |  66 ++++++++++++--\n convert.h               |  29 +++++-\n diff.c                  |   5 +-\n entry.c                 |   7 +-\n merge-recursive.c       |   2 +-\n merge.c                 |   1 +\n sequencer.c             |   1 +\n t/t0021-conversion.sh   | 198 ++++++++++++++++++++++++++++++++++------\n t/t0021/rot13-filter.pl |   6 ++\n unpack-trees.c          |   1 +\n unpack-trees.h          |   1 +\n 20 files changed, 341 insertions(+), 75 deletions(-)\n\n"},{"id":"393017","messageId":"20200310182046.748959-2-sandals@crustytoothpaste.net","threadId":"52970","inReplyTo":"20200310182046.748959-1-sandals@crustytoothpaste.net","subject":"[PATCH 1/8] builtin/checkout: pass branch info down to checkout_worktree","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-03-10T18:20:39Z","receivedAt":"2020-03-10T18:22:12Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"From: \"brian m. carlson\" <bk2204@github.com>\n\nIn the future, we're going to want to use the branch info in\ncheckout_worktree, so let's pass the whole struct branch_info down, not\njust the revision name.  We hoist the definition of struct branch_info\nso it's in scope.\n\nSigned-off-by: brian m. carlson <bk2204@github.com>\n---\n builtin/checkout.c | 33 +++++++++++++++++----------------\n 1 file changed, 17 insertions(+), 16 deletions(-)\n\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex d6773818b8..8a28f48d67 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -88,6 +88,17 @@ struct checkout_opts {\n \tstruct tree *source_tree;\n };\n \n+struct branch_info {\n+\tconst char *name; /* The short name used */\n+\tconst char *path; /* The full name of a real branch */\n+\tstruct commit *commit; /* The named commit */\n+\t/*\n+\t * if not null the branch is detached because it's already\n+\t * checked out in this checkout\n+\t */\n+\tchar *checkout;\n+};\n+\n static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,\n \t\t\t      int changed)\n {\n@@ -337,7 +348,8 @@ static void mark_ce_for_checkout_no_overlay(struct cache_entry *ce,\n \t}\n }\n \n-static int checkout_worktree(const struct checkout_opts *opts)\n+static int checkout_worktree(const struct checkout_opts *opts,\n+\t\t\t     const struct branch_info *info)\n {\n \tstruct checkout state = CHECKOUT_INIT;\n \tint nr_checkouts = 0, nr_unmerged = 0;\n@@ -396,7 +408,7 @@ static int checkout_worktree(const struct checkout_opts *opts)\n }\n \n static int checkout_paths(const struct checkout_opts *opts,\n-\t\t\t  const char *revision)\n+\t\t\t  const struct branch_info *new_branch_info)\n {\n \tint pos;\n \tstatic char *ps_matched;\n@@ -462,7 +474,7 @@ static int checkout_paths(const struct checkout_opts *opts,\n \t\telse\n \t\t\tBUG(\"either flag must have been set, worktree=%d, index=%d\",\n \t\t\t    opts->checkout_worktree, opts->checkout_index);\n-\t\treturn run_add_interactive(revision, patch_mode, &opts->pathspec);\n+\t\treturn run_add_interactive(new_branch_info->name, patch_mode, &opts->pathspec);\n \t}\n \n \trepo_hold_locked_index(the_repository, &lock_file, LOCK_DIE_ON_ERROR);\n@@ -523,7 +535,7 @@ static int checkout_paths(const struct checkout_opts *opts,\n \n \t/* Now we are committed to check them out */\n \tif (opts->checkout_worktree)\n-\t\terrs |= checkout_worktree(opts);\n+\t\terrs |= checkout_worktree(opts, new_branch_info);\n \telse\n \t\tremove_marked_cache_entries(&the_index, 1);\n \n@@ -620,17 +632,6 @@ static int reset_tree(struct tree *tree, const struct checkout_opts *o,\n \t}\n }\n \n-struct branch_info {\n-\tconst char *name; /* The short name used */\n-\tconst char *path; /* The full name of a real branch */\n-\tstruct commit *commit; /* The named commit */\n-\t/*\n-\t * if not null the branch is detached because it's already\n-\t * checked out in this checkout\n-\t */\n-\tchar *checkout;\n-};\n-\n static void setup_branch_path(struct branch_info *branch)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n@@ -1710,7 +1711,7 @@ static int checkout_main(int argc, const char **argv, const char *prefix,\n \n \tUNLEAK(opts);\n \tif (opts->patch_mode || opts->pathspec.nr)\n-\t\treturn checkout_paths(opts, new_branch_info.name);\n+\t\treturn checkout_paths(opts, &new_branch_info);\n \telse\n \t\treturn checkout_branch(opts, &new_branch_info);\n }\n"},{"id":"393018","messageId":"20200310182046.748959-4-sandals@crustytoothpaste.net","threadId":"52970","inReplyTo":"20200310182046.748959-1-sandals@crustytoothpaste.net","subject":"[PATCH 3/8] convert: provide additional metadata to filters","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-03-10T18:20:41Z","receivedAt":"2020-03-10T18:22:13Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"From: \"brian m. carlson\" <bk2204@github.com>\n\nNow that we have the codebase wired up to pass any additional metadata\nto filters, let's collect the additional metadata that we'd like to\npass.\n\nThe two main places we pass this metadata are checkouts and archives.\nIn these two situations, reading HEAD isn't a valid option, since HEAD\nisn't updated for checkouts until after the working tree is written and\narchives can accept an arbitrary tree.  In other situations, HEAD will\nusually reflect the refname of the branch in current use.\n\nWe pass a smaller amount of data in other cases, such as git cat-file,\nwhere we can really only logically know about the blob.\n\nThis commit updates only the parts of the checkout code where we don't\nuse unpack_trees.  That function and callers of it will be handled in a\nfuture commit.\n\nIn the archive code, we leak a small amount of memory, since nothing we\npass in the archiver argument structure is freed.\n\nSigned-off-by: brian m. carlson <bk2204@github.com>\n---\n archive.c               | 13 ++++++++++---\n archive.h               |  1 +\n builtin/cat-file.c      |  5 ++++-\n builtin/checkout.c      |  4 ++++\n cache.h                 |  1 +\n convert.c               | 22 ++++++++++++++++++++++\n convert.h               | 17 +++++++++++++++++\n diff.c                  |  5 ++++-\n entry.c                 |  7 +++++--\n t/t0021/rot13-filter.pl |  6 ++++++\n 10 files changed, 74 insertions(+), 7 deletions(-)\n\ndiff --git a/archive.c b/archive.c\nindex d9e92cce58..fb39706120 100644\n--- a/archive.c\n+++ b/archive.c\n@@ -77,6 +77,11 @@ void *object_file_to_archive(const struct archiver_args *args,\n {\n \tvoid *buffer;\n \tconst struct commit *commit = args->convert ? args->commit : NULL;\n+\tstruct checkout_metadata meta;\n+\n+\tinit_checkout_metadata(&meta, args->refname,\n+\t\t\t       args->commit_oid ? args->commit_oid :\n+\t\t\t       (args->tree ? &args->tree->object.oid : NULL), oid);\n \n \tpath += args->baselen;\n \tbuffer = read_object_file(oid, type, sizep);\n@@ -85,7 +90,7 @@ void *object_file_to_archive(const struct archiver_args *args,\n \t\tsize_t size = 0;\n \n \t\tstrbuf_attach(&buf, buffer, *sizep, *sizep + 1);\n-\t\tconvert_to_working_tree(args->repo->index, path, buf.buf, buf.len, &buf, NULL);\n+\t\tconvert_to_working_tree(args->repo->index, path, buf.buf, buf.len, &buf, &meta);\n \t\tif (commit)\n \t\t\tformat_subst(commit, buf.buf, buf.len, &buf);\n \t\tbuffer = strbuf_detach(&buf, &size);\n@@ -385,16 +390,17 @@ static void parse_treeish_arg(const char **argv,\n \tstruct tree *tree;\n \tconst struct commit *commit;\n \tstruct object_id oid;\n+\tchar *ref = NULL;\n \n \t/* Remotes are only allowed to fetch actual refs */\n \tif (remote && !remote_allow_unreachable) {\n-\t\tchar *ref = NULL;\n \t\tconst char *colon = strchrnul(name, ':');\n \t\tint refnamelen = colon - name;\n \n \t\tif (!dwim_ref(name, refnamelen, &oid, &ref))\n \t\t\tdie(_(\"no such ref: %.*s\"), refnamelen, name);\n-\t\tfree(ref);\n+\t} else {\n+\t\tdwim_ref(name, strlen(name), &oid, &ref);\n \t}\n \n \tif (get_oid(name, &oid))\n@@ -427,6 +433,7 @@ static void parse_treeish_arg(const char **argv,\n \n \t\ttree = parse_tree_indirect(&tree_oid);\n \t}\n+\tar_args->refname = ref;\n \tar_args->tree = tree;\n \tar_args->commit_oid = commit_oid;\n \tar_args->commit = commit;\ndiff --git a/archive.h b/archive.h\nindex e60e3dd31c..3bd96bf6bb 100644\n--- a/archive.h\n+++ b/archive.h\n@@ -8,6 +8,7 @@ struct repository;\n \n struct archiver_args {\n \tstruct repository *repo;\n+\tconst char *refname;\n \tconst char *base;\n \tsize_t baselen;\n \tstruct tree *tree;\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex 545fed4924..6ecc8ee6dc 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -42,7 +42,10 @@ static int filter_object(const char *path, unsigned mode,\n \t\t\t     oid_to_hex(oid), path);\n \tif ((type == OBJ_BLOB) && S_ISREG(mode)) {\n \t\tstruct strbuf strbuf = STRBUF_INIT;\n-\t\tif (convert_to_working_tree(&the_index, path, *buf, *size, &strbuf, NULL)) {\n+\t\tstruct checkout_metadata meta;\n+\n+\t\tinit_checkout_metadata(&meta, NULL, NULL, oid);\n+\t\tif (convert_to_working_tree(&the_index, path, *buf, *size, &strbuf, &meta)) {\n \t\t\tfree(*buf);\n \t\t\t*size = strbuf.len;\n \t\t\t*buf = strbuf_detach(&strbuf, NULL);\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex 8a28f48d67..a7bfa3688d 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -360,6 +360,10 @@ static int checkout_worktree(const struct checkout_opts *opts,\n \tstate.refresh_cache = 1;\n \tstate.istate = &the_index;\n \n+\tinit_checkout_metadata(&state.meta, info->path,\n+\t\t\t       info->commit ? &info->commit->object.oid : NULL,\n+\t\t\t       NULL);\n+\n \tenable_delayed_checkout(&state);\n \tfor (pos = 0; pos < active_nr; pos++) {\n \t\tstruct cache_entry *ce = active_cache[pos];\ndiff --git a/cache.h b/cache.h\nindex 37c899b53f..9b24e5d61f 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1679,6 +1679,7 @@ struct checkout {\n \tconst char *base_dir;\n \tint base_dir_len;\n \tstruct delayed_checkout *delayed_checkout;\n+\tstruct checkout_metadata meta;\n \tunsigned force:1,\n \t\t quiet:1,\n \t\t not_new:1,\ndiff --git a/convert.c b/convert.c\nindex 6261921cfb..5aa87d45e3 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -2006,3 +2006,25 @@ int stream_filter(struct stream_filter *filter,\n {\n \treturn filter->vtbl->filter(filter, input, isize_p, output, osize_p);\n }\n+\n+void init_checkout_metadata(struct checkout_metadata *meta, const char *refname,\n+\t\t\t    const struct object_id *treeish,\n+\t\t\t    const struct object_id *blob)\n+{\n+\tmemset(meta, 0, sizeof(*meta));\n+\tif (refname)\n+\t\tmeta->refname = refname;\n+\tif (treeish)\n+\t\toidcpy(&meta->treeish, treeish);\n+\tif (blob)\n+\t\toidcpy(&meta->blob, blob);\n+}\n+\n+void clone_checkout_metadata(struct checkout_metadata *dst,\n+\t\t\t     const struct checkout_metadata *src,\n+\t\t\t     const struct object_id *blob)\n+{\n+\tmemcpy(dst, src, sizeof(*dst));\n+\tif (blob)\n+\t\toidcpy(&dst->blob, blob);\n+}\ndiff --git a/convert.h b/convert.h\nindex 894e01c38b..e29d1026a6 100644\n--- a/convert.h\n+++ b/convert.h\n@@ -102,6 +102,23 @@ void convert_to_git_filter_fd(const struct index_state *istate,\n int would_convert_to_git_filter_fd(const struct index_state *istate,\n \t\t\t\t   const char *path);\n \n+/*\n+ * Initialize the checkout metadata with the given values.  Any argument may be\n+ * NULL if it is not applicable.  The treeish should be a commit if that is\n+ * available, and a tree otherwise.\n+ *\n+ * The refname is not copied and must be valid for the lifetime of the struct.\n+ * THe object IDs are copied.\n+ */\n+void init_checkout_metadata(struct checkout_metadata *meta, const char *refname,\n+\t\t\t    const struct object_id *treeish,\n+\t\t\t    const struct object_id *blob);\n+\n+/* Copy the metadata from src to dst, updating the blob. */\n+void clone_checkout_metadata(struct checkout_metadata *dst,\n+\t\t\t     const struct checkout_metadata *src,\n+\t\t\t     const struct object_id *blob);\n+\n /*\n  * Reset the internal list of attributes used by convert_to_git and\n  * convert_to_working_tree.\ndiff --git a/diff.c b/diff.c\nindex 12761c8017..1010d806f5 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -4062,6 +4062,9 @@ static void prep_temp_blob(struct index_state *istate,\n \tstruct strbuf tempfile = STRBUF_INIT;\n \tchar *path_dup = xstrdup(path);\n \tconst char *base = basename(path_dup);\n+\tstruct checkout_metadata meta;\n+\n+\tinit_checkout_metadata(&meta, NULL, NULL, oid);\n \n \t/* Generate \"XXXXXX_basename.ext\" */\n \tstrbuf_addstr(&tempfile, \"XXXXXX_\");\n@@ -4071,7 +4074,7 @@ static void prep_temp_blob(struct index_state *istate,\n \tif (!temp->tempfile)\n \t\tdie_errno(\"unable to create temp-file\");\n \tif (convert_to_working_tree(istate, path,\n-\t\t\t(const char *)blob, (size_t)size, &buf, NULL)) {\n+\t\t\t(const char *)blob, (size_t)size, &buf, &meta)) {\n \t\tblob = buf.buf;\n \t\tsize = buf.len;\n \t}\ndiff --git a/entry.c b/entry.c\nindex 4b2d9b2dad..00b4903366 100644\n--- a/entry.c\n+++ b/entry.c\n@@ -264,6 +264,9 @@ static int write_entry(struct cache_entry *ce,\n \tsize_t newsize = 0;\n \tstruct stat st;\n \tconst struct submodule *sub;\n+\tstruct checkout_metadata meta;\n+\n+\tclone_checkout_metadata(&meta, &state->meta, &ce->oid);\n \n \tif (ce_mode_s_ifmt == S_IFREG) {\n \t\tstruct stream_filter *filter = get_stream_filter(state->istate, ce->name,\n@@ -315,13 +318,13 @@ static int write_entry(struct cache_entry *ce,\n \t\t */\n \t\tif (dco && dco->state != CE_NO_DELAY) {\n \t\t\tret = async_convert_to_working_tree(state->istate, ce->name, new_blob,\n-\t\t\t\t\t\t\t    size, &buf, NULL, dco);\n+\t\t\t\t\t\t\t    size, &buf, &meta, dco);\n \t\t\tif (ret && string_list_has_string(&dco->paths, ce->name)) {\n \t\t\t\tfree(new_blob);\n \t\t\t\tgoto delayed;\n \t\t\t}\n \t\t} else\n-\t\t\tret = convert_to_working_tree(state->istate, ce->name, new_blob, size, &buf, NULL);\n+\t\t\tret = convert_to_working_tree(state->istate, ce->name, new_blob, size, &buf, &meta);\n \n \t\tif (ret) {\n \t\t\tfree(new_blob);\ndiff --git a/t/t0021/rot13-filter.pl b/t/t0021/rot13-filter.pl\nindex 470107248e..c43cf433cf 100644\n--- a/t/t0021/rot13-filter.pl\n+++ b/t/t0021/rot13-filter.pl\n@@ -135,7 +135,13 @@ sub rot13 {\n \t\t\t\tif ( exists $DELAY{$pathname} and $DELAY{$pathname}{\"requested\"} == 0 ) {\n \t\t\t\t\t$DELAY{$pathname}{\"requested\"} = 1;\n \t\t\t\t}\n+\t\t\t} elsif ($buffer =~ /^(ref|treeish|blob)=/) {\n+\t\t\t\t# Do nothing.\n \t\t\t} else {\n+\t\t\t\t# In general, filters need to be graceful about\n+\t\t\t\t# new metadata, since it's documented that we\n+\t\t\t\t# can pass any key-value pairs, but for tests,\n+\t\t\t\t# let's be a little stricter.\n \t\t\t\tdie \"Unknown message '$buffer'\";\n \t\t\t}\n \n"},{"id":"393019","messageId":"20200310182046.748959-6-sandals@crustytoothpaste.net","threadId":"52970","inReplyTo":"20200310182046.748959-1-sandals@crustytoothpaste.net","subject":"[PATCH 5/8] builtin/clone: compute checkout metadata for clones","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-03-10T18:20:43Z","receivedAt":"2020-03-10T18:22:15Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"From: \"brian m. carlson\" <bk2204@github.com>\n\nWhen checking out a commit, provide metadata to the filter process\nincluding the ref we're using.\n\nSigned-off-by: brian m. carlson <bk2204@github.com>\n---\n builtin/clone.c       |  6 +++++-\n t/t0021-conversion.sh | 10 +++++++---\n 2 files changed, 12 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 1ad26f4d8c..00e5427ef1 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -780,11 +780,12 @@ static int checkout(int submodule_progress)\n \tif (!strcmp(head, \"HEAD\")) {\n \t\tif (advice_detached_head)\n \t\t\tdetach_advice(oid_to_hex(&oid));\n+\t\tfree(head);\n+\t\thead = NULL;\n \t} else {\n \t\tif (!starts_with(head, \"refs/heads/\"))\n \t\t\tdie(_(\"HEAD not found below refs/heads!\"));\n \t}\n-\tfree(head);\n \n \t/* We need to be in the new work tree for the checkout */\n \tsetup_work_tree();\n@@ -799,6 +800,7 @@ static int checkout(int submodule_progress)\n \topts.verbose_update = (option_verbosity >= 0);\n \topts.src_index = &the_index;\n \topts.dst_index = &the_index;\n+\tinit_checkout_metadata(&opts.meta, head, &oid, NULL);\n \n \ttree = parse_tree_indirect(&oid);\n \tparse_tree(tree);\n@@ -806,6 +808,8 @@ static int checkout(int submodule_progress)\n \tif (unpack_trees(1, &t, &opts) < 0)\n \t\tdie(_(\"unable to checkout working tree\"));\n \n+\tfree(head);\n+\n \tif (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))\n \t\tdie(_(\"unable to write new index file\"));\n \ndiff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh\nindex 4b8d6a74a7..ad329044a3 100755\n--- a/t/t0021-conversion.sh\n+++ b/t/t0021-conversion.sh\n@@ -748,7 +748,8 @@ test_expect_success PERL 'delayed checkout in process filter' '\n \t) &&\n \n \tS=$(file_size \"$TEST_ROOT/test.o\") &&\n-\tM=\"blob=$(git -C repo rev-parse --verify master:test.a)\" &&\n+\tPM=\"ref=refs/heads/master treeish=$(git -C repo rev-parse --verify master) \" &&\n+\tM=\"${PM}blob=$(git -C repo rev-parse --verify master:test.a)\" &&\n \tcat >a.exp <<-EOF &&\n \t\tSTART\n \t\tinit handshake complete\n@@ -789,8 +790,11 @@ test_expect_success PERL 'delayed checkout in process filter' '\n \n \t\trm *.a *.b &&\n \t\tfilter_git checkout . &&\n-\t\ttest_cmp_count ../a.exp a.log &&\n-\t\ttest_cmp_count ../b.exp b.log &&\n+\t\t# We are not checking out a ref here, so filter out ref metadata.\n+\t\tsed -e \"s!$PM!!\" ../a.exp >a.exp.filtered &&\n+\t\tsed -e \"s!$PM!!\" ../b.exp >b.exp.filtered &&\n+\t\ttest_cmp_count a.exp.filtered a.log &&\n+\t\ttest_cmp_count b.exp.filtered b.log &&\n \n \t\ttest_cmp_committed_rot13 \"$TEST_ROOT/test.o\" test.a &&\n \t\ttest_cmp_committed_rot13 \"$TEST_ROOT/test.o\" test-delay10.a &&\n"},{"id":"393020","messageId":"20200310182046.748959-5-sandals@crustytoothpaste.net","threadId":"52970","inReplyTo":"20200310182046.748959-1-sandals@crustytoothpaste.net","subject":"[PATCH 4/8] builtin/checkout: compute checkout metadata for checkouts","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-03-10T18:20:42Z","receivedAt":"2020-03-10T18:22:16Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"From: \"brian m. carlson\" <bk2204@github.com>\n\nProvide commit metadata for checkout code paths that use unpack_trees\nand friends.  When we're checking out a commit, use the commit\ninformation, but don't provide commit information if we're checking out\nfrom the index, since there need not be any particular commit associated\nwith the index, and even if there is one, we can't know what it is.\n\nSigned-off-by: brian m. carlson <bk2204@github.com>\n---\n builtin/checkout.c      | 17 ++++++---\n merge.c                 |  1 +\n t/t0021-conversion.sh   | 78 ++++++++++++++++++++++++++---------------\n t/t0021/rot13-filter.pl |  2 +-\n unpack-trees.c          |  1 +\n unpack-trees.h          |  1 +\n 6 files changed, 67 insertions(+), 33 deletions(-)\n\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex a7bfa3688d..1e0919f80a 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -602,7 +602,8 @@ static void describe_detached_head(const char *msg, struct commit *commit)\n }\n \n static int reset_tree(struct tree *tree, const struct checkout_opts *o,\n-\t\t      int worktree, int *writeout_error)\n+\t\t      int worktree, int *writeout_error,\n+\t\t      struct branch_info *info)\n {\n \tstruct unpack_trees_options opts;\n \tstruct tree_desc tree_desc;\n@@ -617,6 +618,10 @@ static int reset_tree(struct tree *tree, const struct checkout_opts *o,\n \topts.verbose_update = o->show_progress;\n \topts.src_index = &the_index;\n \topts.dst_index = &the_index;\n+\tinit_checkout_metadata(&opts.meta, info->path,\n+\t\t\t       info->commit ? &info->commit->object.oid :\n+\t\t\t       &tree->object.oid,\n+\t\t\t       NULL);\n \tparse_tree(tree);\n \tinit_tree_desc(&tree_desc, tree->buffer, tree->size);\n \tswitch (unpack_trees(1, &tree_desc, &opts)) {\n@@ -668,7 +673,7 @@ static int merge_working_tree(const struct checkout_opts *opts,\n \t} else\n \t\tnew_tree = get_commit_tree(new_branch_info->commit);\n \tif (opts->discard_changes) {\n-\t\tret = reset_tree(new_tree, opts, 1, writeout_error);\n+\t\tret = reset_tree(new_tree, opts, 1, writeout_error, new_branch_info);\n \t\tif (ret)\n \t\t\treturn ret;\n \t} else {\n@@ -697,6 +702,10 @@ static int merge_working_tree(const struct checkout_opts *opts,\n \t\ttopts.quiet = opts->merge && old_branch_info->commit;\n \t\ttopts.verbose_update = opts->show_progress;\n \t\ttopts.fn = twoway_merge;\n+\t\tinit_checkout_metadata(&topts.meta, new_branch_info->path,\n+\t\t\t\t       new_branch_info->commit ?\n+\t\t\t\t       &new_branch_info->commit->object.oid :\n+\t\t\t\t       NULL, NULL);\n \t\tif (opts->overwrite_ignore) {\n \t\t\ttopts.dir = xcalloc(1, sizeof(*topts.dir));\n \t\t\ttopts.dir->flags |= DIR_SHOW_IGNORED;\n@@ -767,7 +776,7 @@ static int merge_working_tree(const struct checkout_opts *opts,\n \n \t\t\tret = reset_tree(new_tree,\n \t\t\t\t\t opts, 1,\n-\t\t\t\t\t writeout_error);\n+\t\t\t\t\t writeout_error, new_branch_info);\n \t\t\tif (ret)\n \t\t\t\treturn ret;\n \t\t\to.ancestor = old_branch_info->name;\n@@ -787,7 +796,7 @@ static int merge_working_tree(const struct checkout_opts *opts,\n \t\t\t\texit(128);\n \t\t\tret = reset_tree(new_tree,\n \t\t\t\t\t opts, 0,\n-\t\t\t\t\t writeout_error);\n+\t\t\t\t\t writeout_error, new_branch_info);\n \t\t\tstrbuf_release(&o.obuf);\n \t\t\tstrbuf_release(&old_commit_shortname);\n \t\t\tif (ret)\ndiff --git a/merge.c b/merge.c\nindex 7c1d756c3f..aa36de2f64 100644\n--- a/merge.c\n+++ b/merge.c\n@@ -94,6 +94,7 @@ int checkout_fast_forward(struct repository *r,\n \topts.verbose_update = 1;\n \topts.merge = 1;\n \topts.fn = twoway_merge;\n+\tinit_checkout_metadata(&opts.meta, NULL, remote, NULL);\n \tsetup_unpack_trees_porcelain(&opts, \"merge\");\n \n \tif (unpack_trees(nr_trees, t, &opts)) {\ndiff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh\nindex dc664da551..4b8d6a74a7 100755\n--- a/t/t0021-conversion.sh\n+++ b/t/t0021-conversion.sh\n@@ -364,6 +364,10 @@ test_expect_success PERL 'required process filter should filter data' '\n \t\tS=$(file_size test.r) &&\n \t\tS2=$(file_size test2.r) &&\n \t\tS3=$(file_size \"testsubdir/test3 '\\''sq'\\'',\\$x=.r\") &&\n+\t\tM=$(git hash-object test.r) &&\n+\t\tM2=$(git hash-object test2.r) &&\n+\t\tM3=$(git hash-object \"testsubdir/test3 '\\''sq'\\'',\\$x=.r\") &&\n+\t\tEMPTY=$(git hash-object /dev/null) &&\n \n \t\tfilter_git add . &&\n \t\tcat >expected.log <<-EOF &&\n@@ -378,14 +382,15 @@ test_expect_success PERL 'required process filter should filter data' '\n \t\ttest_cmp_count expected.log debug.log &&\n \n \t\tgit commit -m \"test commit 2\" &&\n+\t\tMETA=\"ref=refs/heads/master treeish=$(git rev-parse --verify master)\" &&\n \t\trm -f test2.r \"testsubdir/test3 '\\''sq'\\'',\\$x=.r\" &&\n \n \t\tfilter_git checkout --quiet --no-progress . &&\n \t\tcat >expected.log <<-EOF &&\n \t\t\tSTART\n \t\t\tinit handshake complete\n-\t\t\tIN: smudge test2.r $S2 [OK] -- OUT: $S2 . [OK]\n-\t\t\tIN: smudge testsubdir/test3 '\\''sq'\\'',\\$x=.r $S3 [OK] -- OUT: $S3 . [OK]\n+\t\t\tIN: smudge test2.r blob=$M2 $S2 [OK] -- OUT: $S2 . [OK]\n+\t\t\tIN: smudge testsubdir/test3 '\\''sq'\\'',\\$x=.r blob=$M3 $S3 [OK] -- OUT: $S3 . [OK]\n \t\t\tSTOP\n \t\tEOF\n \t\ttest_cmp_exclude_clean expected.log debug.log &&\n@@ -406,10 +411,10 @@ test_expect_success PERL 'required process filter should filter data' '\n \t\tcat >expected.log <<-EOF &&\n \t\t\tSTART\n \t\t\tinit handshake complete\n-\t\t\tIN: smudge test.r $S [OK] -- OUT: $S . [OK]\n-\t\t\tIN: smudge test2.r $S2 [OK] -- OUT: $S2 . [OK]\n-\t\t\tIN: smudge test4-empty.r 0 [OK] -- OUT: 0  [OK]\n-\t\t\tIN: smudge testsubdir/test3 '\\''sq'\\'',\\$x=.r $S3 [OK] -- OUT: $S3 . [OK]\n+\t\t\tIN: smudge test.r $META blob=$M $S [OK] -- OUT: $S . [OK]\n+\t\t\tIN: smudge test2.r $META blob=$M2 $S2 [OK] -- OUT: $S2 . [OK]\n+\t\t\tIN: smudge test4-empty.r $META blob=$EMPTY 0 [OK] -- OUT: 0  [OK]\n+\t\t\tIN: smudge testsubdir/test3 '\\''sq'\\'',\\$x=.r $META blob=$M3 $S3 [OK] -- OUT: $S3 . [OK]\n \t\t\tSTOP\n \t\tEOF\n \t\ttest_cmp_exclude_clean expected.log debug.log &&\n@@ -519,17 +524,22 @@ test_expect_success PERL 'required process filter should process multiple packet\n \t\tEOF\n \t\ttest_cmp_count expected.log debug.log &&\n \n-\t\trm -f *.file &&\n+\t\tM1=\"blob=$(git hash-object 1pkt_1__.file)\" &&\n+\t\tM2=\"blob=$(git hash-object 2pkt_1+1.file)\" &&\n+\t\tM3=\"blob=$(git hash-object 2pkt_2-1.file)\" &&\n+\t\tM4=\"blob=$(git hash-object 2pkt_2__.file)\" &&\n+\t\tM5=\"blob=$(git hash-object 3pkt_2+1.file)\" &&\n+\t\trm -f *.file debug.log &&\n \n \t\tfilter_git checkout --quiet --no-progress -- *.file &&\n \t\tcat >expected.log <<-EOF &&\n \t\t\tSTART\n \t\t\tinit handshake complete\n-\t\t\tIN: smudge 1pkt_1__.file $(($S    )) [OK] -- OUT: $(($S    )) . [OK]\n-\t\t\tIN: smudge 2pkt_1+1.file $(($S  +1)) [OK] -- OUT: $(($S  +1)) .. [OK]\n-\t\t\tIN: smudge 2pkt_2-1.file $(($S*2-1)) [OK] -- OUT: $(($S*2-1)) .. [OK]\n-\t\t\tIN: smudge 2pkt_2__.file $(($S*2  )) [OK] -- OUT: $(($S*2  )) .. [OK]\n-\t\t\tIN: smudge 3pkt_2+1.file $(($S*2+1)) [OK] -- OUT: $(($S*2+1)) ... [OK]\n+\t\t\tIN: smudge 1pkt_1__.file $M1 $(($S    )) [OK] -- OUT: $(($S    )) . [OK]\n+\t\t\tIN: smudge 2pkt_1+1.file $M2 $(($S  +1)) [OK] -- OUT: $(($S  +1)) .. [OK]\n+\t\t\tIN: smudge 2pkt_2-1.file $M3 $(($S*2-1)) [OK] -- OUT: $(($S*2-1)) .. [OK]\n+\t\t\tIN: smudge 2pkt_2__.file $M4 $(($S*2  )) [OK] -- OUT: $(($S*2  )) .. [OK]\n+\t\t\tIN: smudge 3pkt_2+1.file $M5 $(($S*2+1)) [OK] -- OUT: $(($S*2+1)) ... [OK]\n \t\t\tSTOP\n \t\tEOF\n \t\ttest_cmp_exclude_clean expected.log debug.log &&\n@@ -578,6 +588,10 @@ test_expect_success PERL 'process filter should restart after unexpected write f\n \t\tS=$(file_size test.r) &&\n \t\tS2=$(file_size test2.r) &&\n \t\tSF=$(file_size smudge-write-fail.r) &&\n+\t\tM=$(git hash-object test.r) &&\n+\t\tM2=$(git hash-object test2.r) &&\n+\t\tMF=$(git hash-object smudge-write-fail.r) &&\n+\t\trm -f debug.log &&\n \n \t\tgit add . &&\n \t\trm -f *.r &&\n@@ -591,11 +605,11 @@ test_expect_success PERL 'process filter should restart after unexpected write f\n \t\tcat >expected.log <<-EOF &&\n \t\t\tSTART\n \t\t\tinit handshake complete\n-\t\t\tIN: smudge smudge-write-fail.r $SF [OK] -- [WRITE FAIL]\n+\t\t\tIN: smudge smudge-write-fail.r blob=$MF $SF [OK] -- [WRITE FAIL]\n \t\t\tSTART\n \t\t\tinit handshake complete\n-\t\t\tIN: smudge test.r $S [OK] -- OUT: $S . [OK]\n-\t\t\tIN: smudge test2.r $S2 [OK] -- OUT: $S2 . [OK]\n+\t\t\tIN: smudge test.r blob=$M $S [OK] -- OUT: $S . [OK]\n+\t\t\tIN: smudge test2.r blob=$M2 $S2 [OK] -- OUT: $S2 . [OK]\n \t\t\tSTOP\n \t\tEOF\n \t\ttest_cmp_exclude_clean expected.log debug.log &&\n@@ -629,6 +643,10 @@ test_expect_success PERL 'process filter should not be restarted if it signals a\n \t\tS=$(file_size test.r) &&\n \t\tS2=$(file_size test2.r) &&\n \t\tSE=$(file_size error.r) &&\n+\t\tM=$(git hash-object test.r) &&\n+\t\tM2=$(git hash-object test2.r) &&\n+\t\tME=$(git hash-object error.r) &&\n+\t\trm -f debug.log &&\n \n \t\tgit add . &&\n \t\trm -f *.r &&\n@@ -637,9 +655,9 @@ test_expect_success PERL 'process filter should not be restarted if it signals a\n \t\tcat >expected.log <<-EOF &&\n \t\t\tSTART\n \t\t\tinit handshake complete\n-\t\t\tIN: smudge error.r $SE [OK] -- [ERROR]\n-\t\t\tIN: smudge test.r $S [OK] -- OUT: $S . [OK]\n-\t\t\tIN: smudge test2.r $S2 [OK] -- OUT: $S2 . [OK]\n+\t\t\tIN: smudge error.r blob=$ME $SE [OK] -- [ERROR]\n+\t\t\tIN: smudge test.r blob=$M $S [OK] -- OUT: $S . [OK]\n+\t\t\tIN: smudge test2.r blob=$M2 $S2 [OK] -- OUT: $S2 . [OK]\n \t\t\tSTOP\n \t\tEOF\n \t\ttest_cmp_exclude_clean expected.log debug.log &&\n@@ -665,18 +683,21 @@ test_expect_success PERL 'process filter abort stops processing of all further f\n \t\techo \"error this blob and all future blobs\" >abort.o &&\n \t\tcp abort.o abort.r &&\n \n+\t\tM=\"blob=$(git hash-object abort.r)\" &&\n+\t\trm -f debug.log &&\n \t\tSA=$(file_size abort.r) &&\n \n \t\tgit add . &&\n \t\trm -f *.r &&\n \n+\n \t\t# Note: This test assumes that Git filters files in alphabetical\n \t\t# order (\"abort.r\" before \"test.r\").\n \t\tfilter_git checkout --quiet --no-progress . &&\n \t\tcat >expected.log <<-EOF &&\n \t\t\tSTART\n \t\t\tinit handshake complete\n-\t\t\tIN: smudge abort.r $SA [OK] -- [ABORT]\n+\t\t\tIN: smudge abort.r $M $SA [OK] -- [ABORT]\n \t\t\tSTOP\n \t\tEOF\n \t\ttest_cmp_exclude_clean expected.log debug.log &&\n@@ -727,27 +748,28 @@ test_expect_success PERL 'delayed checkout in process filter' '\n \t) &&\n \n \tS=$(file_size \"$TEST_ROOT/test.o\") &&\n+\tM=\"blob=$(git -C repo rev-parse --verify master:test.a)\" &&\n \tcat >a.exp <<-EOF &&\n \t\tSTART\n \t\tinit handshake complete\n-\t\tIN: smudge test.a $S [OK] -- OUT: $S . [OK]\n-\t\tIN: smudge test-delay10.a $S [OK] -- [DELAYED]\n-\t\tIN: smudge test-delay11.a $S [OK] -- [DELAYED]\n-\t\tIN: smudge test-delay20.a $S [OK] -- [DELAYED]\n+\t\tIN: smudge test.a $M $S [OK] -- OUT: $S . [OK]\n+\t\tIN: smudge test-delay10.a $M $S [OK] -- [DELAYED]\n+\t\tIN: smudge test-delay11.a $M $S [OK] -- [DELAYED]\n+\t\tIN: smudge test-delay20.a $M $S [OK] -- [DELAYED]\n \t\tIN: list_available_blobs test-delay10.a test-delay11.a [OK]\n-\t\tIN: smudge test-delay10.a 0 [OK] -- OUT: $S . [OK]\n-\t\tIN: smudge test-delay11.a 0 [OK] -- OUT: $S . [OK]\n+\t\tIN: smudge test-delay10.a $M 0 [OK] -- OUT: $S . [OK]\n+\t\tIN: smudge test-delay11.a $M 0 [OK] -- OUT: $S . [OK]\n \t\tIN: list_available_blobs test-delay20.a [OK]\n-\t\tIN: smudge test-delay20.a 0 [OK] -- OUT: $S . [OK]\n+\t\tIN: smudge test-delay20.a $M 0 [OK] -- OUT: $S . [OK]\n \t\tIN: list_available_blobs [OK]\n \t\tSTOP\n \tEOF\n \tcat >b.exp <<-EOF &&\n \t\tSTART\n \t\tinit handshake complete\n-\t\tIN: smudge test-delay10.b $S [OK] -- [DELAYED]\n+\t\tIN: smudge test-delay10.b $M $S [OK] -- [DELAYED]\n \t\tIN: list_available_blobs test-delay10.b [OK]\n-\t\tIN: smudge test-delay10.b 0 [OK] -- OUT: $S . [OK]\n+\t\tIN: smudge test-delay10.b $M 0 [OK] -- OUT: $S . [OK]\n \t\tIN: list_available_blobs [OK]\n \t\tSTOP\n \tEOF\ndiff --git a/t/t0021/rot13-filter.pl b/t/t0021/rot13-filter.pl\nindex c43cf433cf..cd32a82da5 100644\n--- a/t/t0021/rot13-filter.pl\n+++ b/t/t0021/rot13-filter.pl\n@@ -136,7 +136,7 @@ sub rot13 {\n \t\t\t\t\t$DELAY{$pathname}{\"requested\"} = 1;\n \t\t\t\t}\n \t\t\t} elsif ($buffer =~ /^(ref|treeish|blob)=/) {\n-\t\t\t\t# Do nothing.\n+\t\t\t\tprint $debug \" $buffer\";\n \t\t\t} else {\n \t\t\t\t# In general, filters need to be graceful about\n \t\t\t\t# new metadata, since it's documented that we\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 1ecdab3304..3aba5da6b6 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -371,6 +371,7 @@ static int check_updates(struct unpack_trees_options *o)\n \tstate.quiet = 1;\n \tstate.refresh_cache = 1;\n \tstate.istate = index;\n+\tclone_checkout_metadata(&state.meta, &o->meta, NULL);\n \n \tif (!o->update || o->dry_run) {\n \t\tremove_marked_cache_entries(index, 0);\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex ae1557fb80..ad41b45a71 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -85,6 +85,7 @@ struct unpack_trees_options {\n \tstruct index_state result;\n \n \tstruct pattern_list *pl; /* for internal use */\n+\tstruct checkout_metadata meta;\n };\n \n int unpack_trees(unsigned n, struct tree_desc *t,\n"},{"id":"393021","messageId":"20200310182046.748959-7-sandals@crustytoothpaste.net","threadId":"52970","inReplyTo":"20200310182046.748959-1-sandals@crustytoothpaste.net","subject":"[PATCH 6/8] builtin/rebase: compute checkout metadata for rebases","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-03-10T18:20:44Z","receivedAt":"2020-03-10T18:22:16Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"From: \"brian m. carlson\" <bk2204@github.com>\n\nSigned-off-by: brian m. carlson <bk2204@github.com>\n---\n builtin/rebase.c      |  1 +\n sequencer.c           |  1 +\n t/t0021-conversion.sh | 39 ++++++++++++++++++++++++++++++++++++++-\n 3 files changed, 40 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/rebase.c b/builtin/rebase.c\nindex cb7aeae597..903267fff9 100644\n--- a/builtin/rebase.c\n+++ b/builtin/rebase.c\n@@ -868,6 +868,7 @@ static int reset_head(struct object_id *oid, const char *action,\n \tunpack_tree_opts.fn = reset_hard ? oneway_merge : twoway_merge;\n \tunpack_tree_opts.update = 1;\n \tunpack_tree_opts.merge = 1;\n+\tinit_checkout_metadata(&unpack_tree_opts.meta, switch_to_branch, oid, NULL);\n \tif (!detach_head)\n \t\tunpack_tree_opts.reset = 1;\n \ndiff --git a/sequencer.c b/sequencer.c\nindex 7477b15422..b05a010c0c 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -3288,6 +3288,7 @@ static int do_reset(struct repository *r,\n \tunpack_tree_opts.fn = oneway_merge;\n \tunpack_tree_opts.merge = 1;\n \tunpack_tree_opts.update = 1;\n+\tinit_checkout_metadata(&unpack_tree_opts.meta, name, &oid, NULL);\n \n \tif (repo_read_index_unmerged(r)) {\n \t\trollback_lock_file(&lock);\ndiff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh\nindex ad329044a3..3efb0dad20 100755\n--- a/t/t0021-conversion.sh\n+++ b/t/t0021-conversion.sh\n@@ -382,7 +382,8 @@ test_expect_success PERL 'required process filter should filter data' '\n \t\ttest_cmp_count expected.log debug.log &&\n \n \t\tgit commit -m \"test commit 2\" &&\n-\t\tMETA=\"ref=refs/heads/master treeish=$(git rev-parse --verify master)\" &&\n+\t\tMASTER=$(git rev-parse --verify master) &&\n+\t\tMETA=\"ref=refs/heads/master treeish=$MASTER\" &&\n \t\trm -f test2.r \"testsubdir/test3 '\\''sq'\\'',\\$x=.r\" &&\n \n \t\tfilter_git checkout --quiet --no-progress . &&\n@@ -425,6 +426,42 @@ test_expect_success PERL 'required process filter should filter data' '\n \t)\n '\n \n+test_expect_success PERL 'required process filter should filter data for various subcommands' '\n+\ttest_config_global filter.protocol.process \"rot13-filter.pl debug.log clean smudge\" &&\n+\ttest_config_global filter.protocol.required true &&\n+\t(\n+\t\tcd repo &&\n+\n+\t\tS=$(file_size test.r) &&\n+\t\tS2=$(file_size test2.r) &&\n+\t\tS3=$(file_size \"testsubdir/test3 '\\''sq'\\'',\\$x=.r\") &&\n+\t\tM=$(git hash-object test.r) &&\n+\t\tM2=$(git hash-object test2.r) &&\n+\t\tM3=$(git hash-object \"testsubdir/test3 '\\''sq'\\'',\\$x=.r\") &&\n+\t\tEMPTY=$(git hash-object /dev/null) &&\n+\n+\t\tMASTER=$(git rev-parse --verify master) &&\n+\n+\t\tcp \"$TEST_ROOT/test.o\" test5.r &&\n+\t\tgit add test5.r &&\n+\t\tgit commit -m \"test commit 3\" &&\n+\t\tgit checkout empty-branch &&\n+\t\tfilter_git rebase --onto empty-branch master^^ master &&\n+\t\tMETA=\"ref=refs/heads/master treeish=$(git rev-parse --verify master)\" &&\n+\t\tcat >expected.log <<-EOF &&\n+\t\t\tSTART\n+\t\t\tinit handshake complete\n+\t\t\tIN: smudge test.r $META blob=$M $S [OK] -- OUT: $S . [OK]\n+\t\t\tIN: smudge test2.r $META blob=$M2 $S2 [OK] -- OUT: $S2 . [OK]\n+\t\t\tIN: smudge test4-empty.r $META blob=$EMPTY 0 [OK] -- OUT: 0  [OK]\n+\t\t\tIN: smudge test5.r $META blob=$M $S [OK] -- OUT: $S . [OK]\n+\t\t\tIN: smudge testsubdir/test3 '\\''sq'\\'',\\$x=.r $META blob=$M3 $S3 [OK] -- OUT: $S3 . [OK]\n+\t\t\tSTOP\n+\t\tEOF\n+\t\ttest_cmp_exclude_clean expected.log debug.log\n+\t)\n+'\n+\n test_expect_success PERL 'required process filter takes precedence' '\n \ttest_config_global filter.protocol.clean false &&\n \ttest_config_global filter.protocol.process \"rot13-filter.pl debug.log clean\" &&\n"},{"id":"393022","messageId":"20200310182046.748959-3-sandals@crustytoothpaste.net","threadId":"52970","inReplyTo":"20200310182046.748959-1-sandals@crustytoothpaste.net","subject":"[PATCH 2/8] convert: permit passing additional metadata to filter processes","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-03-10T18:20:40Z","receivedAt":"2020-03-10T18:22:17Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"From: \"brian m. carlson\" <bk2204@github.com>\n\nThere are a variety of situations where a filter process can make use of\nsome additional metadata.  For example, some people find the ident\nfilter too limiting and would like to include the commit or the branch\nin their smudged files.  This information isn't available during\ncheckout as HEAD hasn't been updated at that point, and it wouldn't be\navailable in archives either.\n\nLet's add a way to pass this metadata down to the filter.  We pass the\nblob we're operating on, the treeish (preferring the commit over the\ntree if one exists), and the ref we're operating on.  Note that we won't\npass this information in all cases, such as when renormalizing or when\nwe're performing diffs, since it doesn't make sense in those cases.\n\nThis series wires up the code to print this information, but doesn't\npass any of it at this point.  In a future commit, we'll have various\ncode paths pass the actual useful data down.\n\nSigned-off-by: brian m. carlson <bk2204@github.com>\n---\n apply.c            |  2 +-\n archive.c          |  2 +-\n builtin/cat-file.c |  2 +-\n convert.c          | 44 ++++++++++++++++++++++++++++++++++----------\n convert.h          | 12 ++++++++++--\n diff.c             |  2 +-\n entry.c            |  4 ++--\n merge-recursive.c  |  2 +-\n 8 files changed, 51 insertions(+), 19 deletions(-)\n\ndiff --git a/apply.c b/apply.c\nindex bdc008fae2..144c19aaca 100644\n--- a/apply.c\n+++ b/apply.c\n@@ -4349,7 +4349,7 @@ static int try_create_file(struct apply_state *state, const char *path,\n \tif (fd < 0)\n \t\treturn 1;\n \n-\tif (convert_to_working_tree(state->repo->index, path, buf, size, &nbuf)) {\n+\tif (convert_to_working_tree(state->repo->index, path, buf, size, &nbuf, NULL)) {\n \t\tsize = nbuf.len;\n \t\tbuf  = nbuf.buf;\n \t}\ndiff --git a/archive.c b/archive.c\nindex a8da0fcc4f..d9e92cce58 100644\n--- a/archive.c\n+++ b/archive.c\n@@ -85,7 +85,7 @@ void *object_file_to_archive(const struct archiver_args *args,\n \t\tsize_t size = 0;\n \n \t\tstrbuf_attach(&buf, buffer, *sizep, *sizep + 1);\n-\t\tconvert_to_working_tree(args->repo->index, path, buf.buf, buf.len, &buf);\n+\t\tconvert_to_working_tree(args->repo->index, path, buf.buf, buf.len, &buf, NULL);\n \t\tif (commit)\n \t\t\tformat_subst(commit, buf.buf, buf.len, &buf);\n \t\tbuffer = strbuf_detach(&buf, &size);\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex 272f9fc6d7..545fed4924 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -42,7 +42,7 @@ static int filter_object(const char *path, unsigned mode,\n \t\t\t     oid_to_hex(oid), path);\n \tif ((type == OBJ_BLOB) && S_ISREG(mode)) {\n \t\tstruct strbuf strbuf = STRBUF_INIT;\n-\t\tif (convert_to_working_tree(&the_index, path, *buf, *size, &strbuf)) {\n+\t\tif (convert_to_working_tree(&the_index, path, *buf, *size, &strbuf, NULL)) {\n \t\t\tfree(*buf);\n \t\t\t*size = strbuf.len;\n \t\t\t*buf = strbuf_detach(&strbuf, NULL);\ndiff --git a/convert.c b/convert.c\nindex 5ead3ce678..6261921cfb 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -797,6 +797,7 @@ static void handle_filter_error(const struct strbuf *filter_status,\n static int apply_multi_file_filter(const char *path, const char *src, size_t len,\n \t\t\t\t   int fd, struct strbuf *dst, const char *cmd,\n \t\t\t\t   const unsigned int wanted_capability,\n+\t\t\t\t   const struct checkout_metadata *meta,\n \t\t\t\t   struct delayed_checkout *dco)\n {\n \tint err;\n@@ -855,6 +856,24 @@ static int apply_multi_file_filter(const char *path, const char *src, size_t len\n \tif (err)\n \t\tgoto done;\n \n+\tif (meta && meta->refname) {\n+\t\terr = packet_write_fmt_gently(process->in, \"ref=%s\\n\", meta->refname);\n+\t\tif (err)\n+\t\t\tgoto done;\n+\t}\n+\n+\tif (meta && !is_null_oid(&meta->treeish)) {\n+\t\terr = packet_write_fmt_gently(process->in, \"treeish=%s\\n\", oid_to_hex(&meta->treeish));\n+\t\tif (err)\n+\t\t\tgoto done;\n+\t}\n+\n+\tif (meta && !is_null_oid(&meta->blob)) {\n+\t\terr = packet_write_fmt_gently(process->in, \"blob=%s\\n\", oid_to_hex(&meta->blob));\n+\t\tif (err)\n+\t\t\tgoto done;\n+\t}\n+\n \tif ((entry->supported_capabilities & CAP_DELAY) &&\n \t    dco && dco->state == CE_CAN_DELAY) {\n \t\tcan_delay = 1;\n@@ -971,6 +990,7 @@ static struct convert_driver {\n static int apply_filter(const char *path, const char *src, size_t len,\n \t\t\tint fd, struct strbuf *dst, struct convert_driver *drv,\n \t\t\tconst unsigned int wanted_capability,\n+\t\t\tconst struct checkout_metadata *meta,\n \t\t\tstruct delayed_checkout *dco)\n {\n \tconst char *cmd = NULL;\n@@ -990,7 +1010,7 @@ static int apply_filter(const char *path, const char *src, size_t len,\n \t\treturn apply_single_file_filter(path, src, len, fd, dst, cmd);\n \telse if (drv->process && *drv->process)\n \t\treturn apply_multi_file_filter(path, src, len, fd, dst,\n-\t\t\tdrv->process, wanted_capability, dco);\n+\t\t\tdrv->process, wanted_capability, meta, dco);\n \n \treturn 0;\n }\n@@ -1368,7 +1388,7 @@ int would_convert_to_git_filter_fd(const struct index_state *istate, const char\n \tif (!ca.drv->required)\n \t\treturn 0;\n \n-\treturn apply_filter(path, NULL, 0, -1, NULL, ca.drv, CAP_CLEAN, NULL);\n+\treturn apply_filter(path, NULL, 0, -1, NULL, ca.drv, CAP_CLEAN, NULL, NULL);\n }\n \n const char *get_convert_attr_ascii(const struct index_state *istate, const char *path)\n@@ -1406,7 +1426,7 @@ int convert_to_git(const struct index_state *istate,\n \n \tconvert_attrs(istate, &ca, path);\n \n-\tret |= apply_filter(path, src, len, -1, dst, ca.drv, CAP_CLEAN, NULL);\n+\tret |= apply_filter(path, src, len, -1, dst, ca.drv, CAP_CLEAN, NULL, NULL);\n \tif (!ret && ca.drv && ca.drv->required)\n \t\tdie(_(\"%s: clean filter '%s' failed\"), path, ca.drv->name);\n \n@@ -1441,7 +1461,7 @@ void convert_to_git_filter_fd(const struct index_state *istate,\n \tassert(ca.drv);\n \tassert(ca.drv->clean || ca.drv->process);\n \n-\tif (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN, NULL))\n+\tif (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN, NULL, NULL))\n \t\tdie(_(\"%s: clean filter '%s' failed\"), path, ca.drv->name);\n \n \tencode_to_git(path, dst->buf, dst->len, dst, ca.working_tree_encoding, conv_flags);\n@@ -1452,7 +1472,9 @@ void convert_to_git_filter_fd(const struct index_state *istate,\n static int convert_to_working_tree_internal(const struct index_state *istate,\n \t\t\t\t\t    const char *path, const char *src,\n \t\t\t\t\t    size_t len, struct strbuf *dst,\n-\t\t\t\t\t    int normalizing, struct delayed_checkout *dco)\n+\t\t\t\t\t    int normalizing,\n+\t\t\t\t\t    const struct checkout_metadata *meta,\n+\t\t\t\t\t    struct delayed_checkout *dco)\n {\n \tint ret = 0, ret_filter = 0;\n \tstruct conv_attrs ca;\n@@ -1484,7 +1506,7 @@ static int convert_to_working_tree_internal(const struct index_state *istate,\n \t}\n \n \tret_filter = apply_filter(\n-\t\tpath, src, len, -1, dst, ca.drv, CAP_SMUDGE, dco);\n+\t\tpath, src, len, -1, dst, ca.drv, CAP_SMUDGE, meta, dco);\n \tif (!ret_filter && ca.drv && ca.drv->required)\n \t\tdie(_(\"%s: smudge filter %s failed\"), path, ca.drv->name);\n \n@@ -1494,22 +1516,24 @@ static int convert_to_working_tree_internal(const struct index_state *istate,\n int async_convert_to_working_tree(const struct index_state *istate,\n \t\t\t\t  const char *path, const char *src,\n \t\t\t\t  size_t len, struct strbuf *dst,\n+\t\t\t\t  const struct checkout_metadata *meta,\n \t\t\t\t  void *dco)\n {\n-\treturn convert_to_working_tree_internal(istate, path, src, len, dst, 0, dco);\n+\treturn convert_to_working_tree_internal(istate, path, src, len, dst, 0, meta, dco);\n }\n \n int convert_to_working_tree(const struct index_state *istate,\n \t\t\t    const char *path, const char *src,\n-\t\t\t    size_t len, struct strbuf *dst)\n+\t\t\t    size_t len, struct strbuf *dst,\n+\t\t\t    const struct checkout_metadata *meta)\n {\n-\treturn convert_to_working_tree_internal(istate, path, src, len, dst, 0, NULL);\n+\treturn convert_to_working_tree_internal(istate, path, src, len, dst, 0, meta, NULL);\n }\n \n int renormalize_buffer(const struct index_state *istate, const char *path,\n \t\t       const char *src, size_t len, struct strbuf *dst)\n {\n-\tint ret = convert_to_working_tree_internal(istate, path, src, len, dst, 1, NULL);\n+\tint ret = convert_to_working_tree_internal(istate, path, src, len, dst, 1, NULL, NULL);\n \tif (ret) {\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\ndiff --git a/convert.h b/convert.h\nindex 3710969d43..894e01c38b 100644\n--- a/convert.h\n+++ b/convert.h\n@@ -4,10 +4,10 @@\n #ifndef CONVERT_H\n #define CONVERT_H\n \n+#include \"hash.h\"\n #include \"string-list.h\"\n \n struct index_state;\n-struct object_id;\n struct strbuf;\n \n #define CONV_EOL_RNDTRP_DIE   (1<<0) /* Die if CRLF to LF to CRLF is different */\n@@ -57,6 +57,12 @@ struct delayed_checkout {\n \tstruct string_list paths;\n };\n \n+struct checkout_metadata {\n+\tconst char *refname;\n+\tstruct object_id treeish;\n+\tstruct object_id blob;\n+};\n+\n extern enum eol core_eol;\n extern char *check_roundtrip_encoding;\n const char *get_cached_convert_stats_ascii(const struct index_state *istate,\n@@ -71,10 +77,12 @@ int convert_to_git(const struct index_state *istate,\n \t\t   struct strbuf *dst, int conv_flags);\n int convert_to_working_tree(const struct index_state *istate,\n \t\t\t    const char *path, const char *src,\n-\t\t\t    size_t len, struct strbuf *dst);\n+\t\t\t    size_t len, struct strbuf *dst,\n+\t\t\t    const struct checkout_metadata *meta);\n int async_convert_to_working_tree(const struct index_state *istate,\n \t\t\t\t  const char *path, const char *src,\n \t\t\t\t  size_t len, struct strbuf *dst,\n+\t\t\t\t  const struct checkout_metadata *meta,\n \t\t\t\t  void *dco);\n int async_query_available_blobs(const char *cmd,\n \t\t\t\tstruct string_list *available_paths);\ndiff --git a/diff.c b/diff.c\nindex f2cfbf2214..12761c8017 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -4071,7 +4071,7 @@ static void prep_temp_blob(struct index_state *istate,\n \tif (!temp->tempfile)\n \t\tdie_errno(\"unable to create temp-file\");\n \tif (convert_to_working_tree(istate, path,\n-\t\t\t(const char *)blob, (size_t)size, &buf)) {\n+\t\t\t(const char *)blob, (size_t)size, &buf, NULL)) {\n \t\tblob = buf.buf;\n \t\tsize = buf.len;\n \t}\ndiff --git a/entry.c b/entry.c\nindex 53380bb614..4b2d9b2dad 100644\n--- a/entry.c\n+++ b/entry.c\n@@ -315,13 +315,13 @@ static int write_entry(struct cache_entry *ce,\n \t\t */\n \t\tif (dco && dco->state != CE_NO_DELAY) {\n \t\t\tret = async_convert_to_working_tree(state->istate, ce->name, new_blob,\n-\t\t\t\t\t\t\t    size, &buf, dco);\n+\t\t\t\t\t\t\t    size, &buf, NULL, dco);\n \t\t\tif (ret && string_list_has_string(&dco->paths, ce->name)) {\n \t\t\t\tfree(new_blob);\n \t\t\t\tgoto delayed;\n \t\t\t}\n \t\t} else\n-\t\t\tret = convert_to_working_tree(state->istate, ce->name, new_blob, size, &buf);\n+\t\t\tret = convert_to_working_tree(state->istate, ce->name, new_blob, size, &buf, NULL);\n \n \t\tif (ret) {\n \t\t\tfree(new_blob);\ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex 7a4e6f20fa..d92e2acf1e 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -958,7 +958,7 @@ static int update_file_flags(struct merge_options *opt,\n \t\tif (S_ISREG(contents->mode)) {\n \t\t\tstruct strbuf strbuf = STRBUF_INIT;\n \t\t\tif (convert_to_working_tree(opt->repo->index,\n-\t\t\t\t\t\t    path, buf, size, &strbuf)) {\n+\t\t\t\t\t\t    path, buf, size, &strbuf, NULL)) {\n \t\t\t\tfree(buf);\n \t\t\t\tsize = strbuf.len;\n \t\t\t\tbuf = strbuf_detach(&strbuf, NULL);\n"},{"id":"393023","messageId":"20200310182046.748959-9-sandals@crustytoothpaste.net","threadId":"52970","inReplyTo":"20200310182046.748959-1-sandals@crustytoothpaste.net","subject":"[PATCH 8/8] t0021: test filter metadata for additional cases","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-03-10T18:20:46Z","receivedAt":"2020-03-10T18:22:19Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"From: \"brian m. carlson\" <bk2204@github.com>\n\nCheck that we get the expected data when performing a merges or\ngenerating archives.  Note that we don't expect a ref for merges,\nbecause we won't be checking out any particular ref, but instead a tree\nof the merged data.  For archives, however, we expect a ref as normal if\nwe have one.\n\nSigned-off-by: brian m. carlson <bk2204@github.com>\n---\n t/t0021-conversion.sh | 45 +++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 45 insertions(+)\n\ndiff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh\nindex a026fd46b4..4bfffa9c31 100755\n--- a/t/t0021-conversion.sh\n+++ b/t/t0021-conversion.sh\n@@ -488,6 +488,51 @@ test_expect_success PERL 'required process filter should filter data for various\n \t\t\tIN: smudge testsubdir/test3 '\\''sq'\\'',\\$x=.r $META blob=$M3 $S3 [OK] -- OUT: $S3 . [OK]\n \t\t\tSTOP\n \t\tEOF\n+\t\ttest_cmp_exclude_clean expected.log debug.log &&\n+\n+\t\tgit checkout -b merge empty-branch &&\n+\t\tgit branch -f master $MASTER2 &&\n+\t\tfilter_git merge master &&\n+\t\tMETA=\"treeish=$MASTER2\" &&\n+\t\tcat >expected.log <<-EOF &&\n+\t\t\tSTART\n+\t\t\tinit handshake complete\n+\t\t\tIN: smudge test.r $META blob=$M $S [OK] -- OUT: $S . [OK]\n+\t\t\tIN: smudge test2.r $META blob=$M2 $S2 [OK] -- OUT: $S2 . [OK]\n+\t\t\tIN: smudge test4-empty.r $META blob=$EMPTY 0 [OK] -- OUT: 0  [OK]\n+\t\t\tIN: smudge test5.r $META blob=$M $S [OK] -- OUT: $S . [OK]\n+\t\t\tIN: smudge testsubdir/test3 '\\''sq'\\'',\\$x=.r $META blob=$M3 $S3 [OK] -- OUT: $S3 . [OK]\n+\t\t\tSTOP\n+\t\tEOF\n+\t\ttest_cmp_exclude_clean expected.log debug.log &&\n+\n+\t\tfilter_git archive master >/dev/null &&\n+\t\tMETA=\"ref=refs/heads/master treeish=$MASTER2\" &&\n+\t\tcat >expected.log <<-EOF &&\n+\t\t\tSTART\n+\t\t\tinit handshake complete\n+\t\t\tIN: smudge test.r $META blob=$M $S [OK] -- OUT: $S . [OK]\n+\t\t\tIN: smudge test2.r $META blob=$M2 $S2 [OK] -- OUT: $S2 . [OK]\n+\t\t\tIN: smudge test4-empty.r $META blob=$EMPTY 0 [OK] -- OUT: 0  [OK]\n+\t\t\tIN: smudge test5.r $META blob=$M $S [OK] -- OUT: $S . [OK]\n+\t\t\tIN: smudge testsubdir/test3 '\\''sq'\\'',\\$x=.r $META blob=$M3 $S3 [OK] -- OUT: $S3 . [OK]\n+\t\t\tSTOP\n+\t\tEOF\n+\t\ttest_cmp_exclude_clean expected.log debug.log &&\n+\n+\t\tTREE=\"$(git rev-parse $MASTER2^{tree})\" &&\n+\t\tfilter_git archive $TREE >/dev/null &&\n+\t\tMETA=\"treeish=$TREE\" &&\n+\t\tcat >expected.log <<-EOF &&\n+\t\t\tSTART\n+\t\t\tinit handshake complete\n+\t\t\tIN: smudge test.r $META blob=$M $S [OK] -- OUT: $S . [OK]\n+\t\t\tIN: smudge test2.r $META blob=$M2 $S2 [OK] -- OUT: $S2 . [OK]\n+\t\t\tIN: smudge test4-empty.r $META blob=$EMPTY 0 [OK] -- OUT: 0  [OK]\n+\t\t\tIN: smudge test5.r $META blob=$M $S [OK] -- OUT: $S . [OK]\n+\t\t\tIN: smudge testsubdir/test3 '\\''sq'\\'',\\$x=.r $META blob=$M3 $S3 [OK] -- OUT: $S3 . [OK]\n+\t\t\tSTOP\n+\t\tEOF\n \t\ttest_cmp_exclude_clean expected.log debug.log\n \t)\n '\n"},{"id":"393024","messageId":"20200310182046.748959-8-sandals@crustytoothpaste.net","threadId":"52970","inReplyTo":"20200310182046.748959-1-sandals@crustytoothpaste.net","subject":"[PATCH 7/8] builtin/reset: compute checkout metadata for reset","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-03-10T18:20:45Z","receivedAt":"2020-03-10T18:22:21Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"From: \"brian m. carlson\" <bk2204@github.com>\n\nPass the commit, and if we have it, the ref to the filters when we\nperform a checkout.  This should only be the case when we invoke git\nreset --hard; the metadata will be unused otherwise.\n\nSigned-off-by: brian m. carlson <bk2204@github.com>\n---\n builtin/reset.c       | 16 +++++++++++++---\n t/t0021-conversion.sh | 32 +++++++++++++++++++++++++++++++-\n 2 files changed, 44 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/reset.c b/builtin/reset.c\nindex 18228c312e..4c634111bd 100644\n--- a/builtin/reset.c\n+++ b/builtin/reset.c\n@@ -46,7 +46,7 @@ static inline int is_merge(void)\n \treturn !access(git_path_merge_head(the_repository), F_OK);\n }\n \n-static int reset_index(const struct object_id *oid, int reset_type, int quiet)\n+static int reset_index(const char *ref, const struct object_id *oid, int reset_type, int quiet)\n {\n \tint i, nr = 0;\n \tstruct tree_desc desc[2];\n@@ -60,6 +60,7 @@ static int reset_index(const struct object_id *oid, int reset_type, int quiet)\n \topts.dst_index = &the_index;\n \topts.fn = oneway_merge;\n \topts.merge = 1;\n+\tinit_checkout_metadata(&opts.meta, ref, oid, NULL);\n \tif (!quiet)\n \t\topts.verbose_update = 1;\n \tswitch (reset_type) {\n@@ -418,11 +419,20 @@ int cmd_reset(int argc, const char **argv, const char *prefix)\n \t\t\t\t}\n \t\t\t}\n \t\t} else {\n-\t\t\tint err = reset_index(&oid, reset_type, quiet);\n+\t\t\tstruct object_id dummy;\n+\t\t\tchar *ref = NULL;\n+\t\t\tint err;\n+\n+\t\t\tdwim_ref(rev, strlen(rev), &dummy, &ref);\n+\t\t\tif (ref && !starts_with(ref, \"refs/\"))\n+\t\t\t\tref = NULL;\n+\n+\t\t\terr = reset_index(ref, &oid, reset_type, quiet);\n \t\t\tif (reset_type == KEEP && !err)\n-\t\t\t\terr = reset_index(&oid, MIXED, quiet);\n+\t\t\t\terr = reset_index(ref, &oid, MIXED, quiet);\n \t\t\tif (err)\n \t\t\t\tdie(_(\"Could not reset index file to revision '%s'.\"), rev);\n+\t\t\tfree(ref);\n \t\t}\n \n \t\tif (write_locked_index(&the_index, &lock, COMMIT_LOCK))\ndiff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh\nindex 3efb0dad20..a026fd46b4 100755\n--- a/t/t0021-conversion.sh\n+++ b/t/t0021-conversion.sh\n@@ -447,7 +447,8 @@ test_expect_success PERL 'required process filter should filter data for various\n \t\tgit commit -m \"test commit 3\" &&\n \t\tgit checkout empty-branch &&\n \t\tfilter_git rebase --onto empty-branch master^^ master &&\n-\t\tMETA=\"ref=refs/heads/master treeish=$(git rev-parse --verify master)\" &&\n+\t\tMASTER2=$(git rev-parse --verify master) &&\n+\t\tMETA=\"ref=refs/heads/master treeish=$MASTER2\" &&\n \t\tcat >expected.log <<-EOF &&\n \t\t\tSTART\n \t\t\tinit handshake complete\n@@ -458,6 +459,35 @@ test_expect_success PERL 'required process filter should filter data for various\n \t\t\tIN: smudge testsubdir/test3 '\\''sq'\\'',\\$x=.r $META blob=$M3 $S3 [OK] -- OUT: $S3 . [OK]\n \t\t\tSTOP\n \t\tEOF\n+\t\ttest_cmp_exclude_clean expected.log debug.log &&\n+\n+\t\tgit reset --hard empty-branch &&\n+\t\tfilter_git reset --hard $MASTER &&\n+\t\tMETA=\"treeish=$MASTER\" &&\n+\t\tcat >expected.log <<-EOF &&\n+\t\t\tSTART\n+\t\t\tinit handshake complete\n+\t\t\tIN: smudge test.r $META blob=$M $S [OK] -- OUT: $S . [OK]\n+\t\t\tIN: smudge test2.r $META blob=$M2 $S2 [OK] -- OUT: $S2 . [OK]\n+\t\t\tIN: smudge test4-empty.r $META blob=$EMPTY 0 [OK] -- OUT: 0  [OK]\n+\t\t\tIN: smudge testsubdir/test3 '\\''sq'\\'',\\$x=.r $META blob=$M3 $S3 [OK] -- OUT: $S3 . [OK]\n+\t\t\tSTOP\n+\t\tEOF\n+\t\ttest_cmp_exclude_clean expected.log debug.log &&\n+\n+\t\tgit branch old-master $MASTER &&\n+\t\tgit reset --hard empty-branch &&\n+\t\tfilter_git reset --hard old-master &&\n+\t\tMETA=\"ref=refs/heads/old-master treeish=$MASTER\" &&\n+\t\tcat >expected.log <<-EOF &&\n+\t\t\tSTART\n+\t\t\tinit handshake complete\n+\t\t\tIN: smudge test.r $META blob=$M $S [OK] -- OUT: $S . [OK]\n+\t\t\tIN: smudge test2.r $META blob=$M2 $S2 [OK] -- OUT: $S2 . [OK]\n+\t\t\tIN: smudge test4-empty.r $META blob=$EMPTY 0 [OK] -- OUT: 0  [OK]\n+\t\t\tIN: smudge testsubdir/test3 '\\''sq'\\'',\\$x=.r $META blob=$M3 $S3 [OK] -- OUT: $S3 . [OK]\n+\t\t\tSTOP\n+\t\tEOF\n \t\ttest_cmp_exclude_clean expected.log debug.log\n \t)\n '\n"},{"id":"393093","messageId":"xmqq1rpyhbjd.fsf@gitster.c.googlers.com","threadId":"52970","inReplyTo":"20200310182046.748959-3-sandals@crustytoothpaste.net","subject":"Re: [PATCH 2/8] convert: permit passing additional metadata to filter processes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-03-11T20:38:46Z","receivedAt":"2020-03-11T20:38:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" <sandals@crustytoothpaste.net> writes:\n\n> From: \"brian m. carlson\" <bk2204@github.com>\n\nDo you want this name/address associated to your contributions?  I\nam asking because to me you have always been \"that toothpaste guy\" ;-)\n\n> There are a variety of situations where a filter process can make use of\n> some additional metadata.  For example, some people find the ident\n> filter too limiting and would like to include the commit or the branch\n> in their smudged files.  This information isn't available during\n> checkout as HEAD hasn't been updated at that point, and it wouldn't be\n> available in archives either.\n>\n> Let's add a way to pass this metadata down to the filter.  We pass the\n> blob we're operating on, the treeish (preferring the commit over the\n> tree if one exists), and the ref we're operating on.  Note that we won't\n> pass this information in all cases, such as when renormalizing or when\n> we're performing diffs, since it doesn't make sense in those cases.\n>\n> This series wires up the code to print this information, but doesn't\n> pass any of it at this point.  In a future commit, we'll have various\n> code paths pass the actual useful data down.\n> ...\n> +struct checkout_metadata {\n> +\tconst char *refname;\n> +\tstruct object_id treeish;\n> +\tstruct object_id blob;\n> +};\n\nThe \"filter\" you talk about is the clean/smudge kind of thing that\nworks on individual blobs, and the series is about giving a bit more\ninformation than the raw contents to be filtered.  I am assuming\nthat I got at least that part correctly.\n\nNow, a few questions.\n\nIs \"refname\" \"refs/heads/branch? when I run \"git checkout branch\" or\n\"git checkout branch -- Makefile\"?  Is \"treeish\" the same as the\noutput from \"rev-parse branch^{commit}\" in such a case?\n\nAssuming they are, what refname and treeish does my filter see, when\nthe user did these things?\n\n    git checkout origin/master\n    git checkout v1.2.3\n    git checkout v1.2.3~4\n\nAs a writer of a filter, do I get different clues when I am munging\nDocumentation/Makefile and t/Makefile, when the user does\n\n    git checkout master -- Makefile Documentation/Makefile\n\ni.e. grab these two files out of the tree of the commit at the tip\nof the master branch?\n\nOr do I just learn what the \"refname\" (presumably \"refs/heads/master\"?)\nand treeish (presumably \"rev-parse master^{commit}\") and I cannot\ntell where in that tree hierarchy the contents come from?\n\nThanks.\n\n"},{"id":"393100","messageId":"20200312003906.GC6198@camp.crustytoothpaste.net","threadId":"52970","inReplyTo":"xmqq1rpyhbjd.fsf@gitster.c.googlers.com","subject":"Re: [PATCH 2/8] convert: permit passing additional metadata to filter processes","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-03-12T00:39:06Z","receivedAt":"2020-03-12T00:39:22Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2020-03-11 at 20:38:46, Junio C Hamano wrote:\n> \"brian m. carlson\" <sandals@crustytoothpaste.net> writes:\n> \n> > From: \"brian m. carlson\" <bk2204@github.com>\n> \n> Do you want this name/address associated to your contributions?  I\n> am asking because to me you have always been \"that toothpaste guy\" ;-)\n\nYes, this is something I wrote as part of my job.  I mostly work on\nnon-Git things there, but occasionally I do some Git work, and I attach\na work email to commits I do at work and a personal email to things I do\noutside of work.  This is so for licensing reasons: I have the ability\nto grant libgit2 or other folks permission to use stuff I wrote on\npersonal time, but I would need to ask first for things written on\nGitHub's time, and I am really terrible at remembering why I wrote what\ncode.\n\nThere's already a .mailmap entry that maps everything to the same person.\n\nYou should continue to see most of my contributions come from my\npersonal address, since most of what I contribute to Git is just for\nfun.\n\n> The \"filter\" you talk about is the clean/smudge kind of thing that\n> works on individual blobs, and the series is about giving a bit more\n> information than the raw contents to be filtered.  I am assuming\n> that I got at least that part correctly.\n\nYou have indeed.\n\n> Now, a few questions.\n> \n> Is \"refname\" \"refs/heads/branch? when I run \"git checkout branch\" or\n> \"git checkout branch -- Makefile\"?  Is \"treeish\" the same as the\n> output from \"rev-parse branch^{commit}\" in such a case?\n\nYes, we'll get something like the following:\n\n  command=smudge\n  ref=refs/heads/master\n  treeish=16f09066000e3328fb2d5c54beb55fc25c1af15c\n  blob=c8e324426dccfcfebad5c33c8ffb97d5a2b54a67\n  can-delay=1\n  pathname=test99.bin\n\nThe treeish is, in this case, git rev-parse refs/heads/master^{commit}.\nIt will always be the commit unless we have no commit, in which case it\nwill be the tree.\n\n> Assuming they are, what refname and treeish does my filter see, when\n> the user did these things?\n> \n>     git checkout origin/master\n>     git checkout v1.2.3\n>     git checkout v1.2.3~4\n\nWe get no ref for these, because HEAD doesn't (well, won't) point to\nanything other than a branch, and a treeish pointing to a commit, plus a\nblob and a pathname for each individual blob.\n\nI tried to emulate the \"read HEAD\" behavior as much as possible for\nrefnames, but if people would like a ref in these cases, I can do that.\n\n> As a writer of a filter, do I get different clues when I am munging\n> Documentation/Makefile and t/Makefile, when the user does\n> \n>     git checkout master -- Makefile Documentation/Makefile\n> \n> i.e. grab these two files out of the tree of the commit at the tip\n> of the master branch?\n> \n> Or do I just learn what the \"refname\" (presumably \"refs/heads/master\"?)\n> and treeish (presumably \"rev-parse master^{commit}\") and I cannot\n> tell where in that tree hierarchy the contents come from?\n\nWe already pass the pathname when processing a file; this is guaranteed.\n\nI clearly need to update the commit message to explain this better for\nfuture readers, since you should not have to ask questions about what\noutput the series provides, so I'll do that when I reroll.  I'll see if\nanyone else has any feedback, and then do that.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"393262","messageId":"20200315103929.GH3122@szeder.dev","threadId":"52970","inReplyTo":"20200310182046.748959-6-sandals@crustytoothpaste.net","subject":"Re: [PATCH 5/8] builtin/clone: compute checkout metadata for clones","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2020-03-15T10:39:29Z","receivedAt":"2020-03-15T10:39:37Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Tue, Mar 10, 2020 at 06:20:43PM +0000, brian m. carlson wrote:\n> diff --git a/builtin/clone.c b/builtin/clone.c\n> index 1ad26f4d8c..00e5427ef1 100644\n> --- a/builtin/clone.c\n> +++ b/builtin/clone.c\n> @@ -780,11 +780,12 @@ static int checkout(int submodule_progress)\n>  \tif (!strcmp(head, \"HEAD\")) {\n>  \t\tif (advice_detached_head)\n>  \t\t\tdetach_advice(oid_to_hex(&oid));\n> +\t\tfree(head);\n> +\t\thead = NULL;\n\nCoccinelle suggests to use FREE_AND_NULL(head) instead.\n\n"},{"id":"393274","messageId":"20200315174445.GA6910@camp.crustytoothpaste.net","threadId":"52970","inReplyTo":"20200315103929.GH3122@szeder.dev","subject":"Re: [PATCH 5/8] builtin/clone: compute checkout metadata for clones","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-03-15T17:44:45Z","receivedAt":"2020-03-15T17:44:54Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2020-03-15 at 10:39:29, SZEDER Gábor wrote:\n> On Tue, Mar 10, 2020 at 06:20:43PM +0000, brian m. carlson wrote:\n> > diff --git a/builtin/clone.c b/builtin/clone.c\n> > index 1ad26f4d8c..00e5427ef1 100644\n> > --- a/builtin/clone.c\n> > +++ b/builtin/clone.c\n> > @@ -780,11 +780,12 @@ static int checkout(int submodule_progress)\n> >  \tif (!strcmp(head, \"HEAD\")) {\n> >  \t\tif (advice_detached_head)\n> >  \t\t\tdetach_advice(oid_to_hex(&oid));\n> > +\t\tfree(head);\n> > +\t\thead = NULL;\n> \n> Coccinelle suggests to use FREE_AND_NULL(head) instead.\n\nI'll do that.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"393283","messageId":"xmqqo8sxa00o.fsf@gitster.c.googlers.com","threadId":"52970","inReplyTo":"20200315103929.GH3122@szeder.dev","subject":"Re: [PATCH 5/8] builtin/clone: compute checkout metadata for clones","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-03-15T19:30:47Z","receivedAt":"2020-03-15T19:30:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"SZEDER Gábor <szeder.dev@gmail.com> writes:\n\n> On Tue, Mar 10, 2020 at 06:20:43PM +0000, brian m. carlson wrote:\n>> diff --git a/builtin/clone.c b/builtin/clone.c\n>> index 1ad26f4d8c..00e5427ef1 100644\n>> --- a/builtin/clone.c\n>> +++ b/builtin/clone.c\n>> @@ -780,11 +780,12 @@ static int checkout(int submodule_progress)\n>>  \tif (!strcmp(head, \"HEAD\")) {\n>>  \t\tif (advice_detached_head)\n>>  \t\t\tdetach_advice(oid_to_hex(&oid));\n>> +\t\tfree(head);\n>> +\t\thead = NULL;\n>\n> Coccinelle suggests to use FREE_AND_NULL(head) instead.\n\nThanks; squashed to update 'pu'.\n"}]}