git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/8] Additional metadata for filter processes

From
brian m. carlson <sandals@crustytoothpaste.net>
Date
Mar 10, 2020, 18:20 UTC
Message-ID
<20200310182046.748959-1-sandals@crustytoothpaste.net>

Smudge and clean filters are currently provided with one particular piece of data: the pathname of the file being smudged. While this is helpful, there are a variety of situations where people would like to have more data.

One such situation is for users who would like to have a custom ident-style filter that contains the branch name. In many cases, it's sufficient to look up this information based on HEAD, but during checkout, HEAD does not point to the right place, since it's updated after the files are written.

Other information users frequently want to know is the commit's object ID and the object ID of the blob being filtered. For example, if filtering is expensive and the filter process sees duplicate blobs during checkout, it may cache the results and avoid having to compute the filter twice.

This series provides an additional set of metadata to the filter process with the keys "ref", "treeish", and "blob". We prefer to provide a commit as the treeish whenever possible, but in some cases, such as when git archive is invoked with a tree, there is no commit, and we use the tree instead.

Note that we don't provide this metadata in all cases. Sometimes it is trivial for the filter to do a simple "git rev-parse HEAD", and in such cases, metadata other than the blob may not be provided. We also don't handle the case where the user is using a smudge or clean command instead of a filter process command: if the user wants the additional metadata, it should be possible for them to write a small filter process, which is reasonably trivial in most languages. Our documentation already permits us to add additional metadata and guarantees only that the pathname will be provided.

My particular use case for this is prefetching and precomputing data during archive generation, since we don't permit delayed filters there due to archives needing to be in a predictable order. I have tried to make it as generally applicable as possible, since I can imagine (and have indeed seen requests for) many other useful applications of this elsewhere.

Feedback is of course welcome.
brian m. carlson (8):
  builtin/checkout: pass branch info down to checkout_worktree
  convert: permit passing additional metadata to filter processes
  convert: provide additional metadata to filters
  builtin/checkout: compute checkout metadata for checkouts
  builtin/clone: compute checkout metadata for clones
  builtin/rebase: compute checkout metadata for rebases
  builtin/reset: compute checkout metadata for reset
  t0021: test filter metadata for additional cases
 apply.c                 |   2 +-
 archive.c               |  13 ++-
 archive.h               |   1 +
 builtin/cat-file.c      |   5 +-
 builtin/checkout.c      |  54 +++++++----
 builtin/clone.c         |   6 +-
 builtin/rebase.c        |   1 +
 builtin/reset.c         |  16 +++-
 cache.h                 |   1 +
 convert.c               |  66 ++++++++++++--
 convert.h               |  29 +++++-
 diff.c                  |   5 +-
 entry.c                 |   7 +-
 merge-recursive.c       |   2 +-
 merge.c                 |   1 +
 sequencer.c             |   1 +
 t/t0021-conversion.sh   | 198 ++++++++++++++++++++++++++++++++++------
 t/t0021/rot13-filter.pl |   6 ++
 unpack-trees.c          |   1 +
 unpack-trees.h          |   1 +
 20 files changed, 341 insertions(+), 75 deletions(-)
Next: brian m. carlson
Message 1 of 14 in “Additional metadata for filter processes”
  1. 0/8 Additional metadata for filter processesbrian m. carlson, Mar 10, 2020
  2. 1/8 builtin/checkout: pass branch info down to checkout_worktreebrian m. carlson, Mar 10, 2020
  3. 3/8 convert: provide additional metadata to filtersbrian m. carlson, Mar 10, 2020
  4. 5/8 builtin/clone: compute checkout metadata for clonesbrian m. carlson, Mar 10, 2020
  5. SZEDER GáborMar 15, 2020
  6. brian m. carlsonMar 15, 2020
  7. Junio C HamanoMar 15, 2020
  8. 4/8 builtin/checkout: compute checkout metadata for checkoutsbrian m. carlson, Mar 10, 2020
  9. 6/8 builtin/rebase: compute checkout metadata for rebasesbrian m. carlson, Mar 10, 2020
  10. 2/8 convert: permit passing additional metadata to filter processesbrian m. carlson, Mar 10, 2020
  11. Junio C HamanoMar 11, 2020
  12. brian m. carlsonMar 12, 2020
  13. 8/8 t0021: test filter metadata for additional casesbrian m. carlson, Mar 10, 2020
  14. 7/8 builtin/reset: compute checkout metadata for resetbrian m. carlson, Mar 10, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.