[PATCH v4 0/5] for-each-ref: add '--include-root-refs' option

This is the forth version of my patch series to print root refs in git-for-each-ref(1).

With the introduction of the reftable backend, it becomes ever so important to provide the necessary tooling for printing all refs associated with a worktree.

While regular refs stored within the "refs/" namespace are currently supported by multiple commands like git-for-each-ref(1), git-show-ref(1). Neither support printing root refs within the worktree.

This patch series is a follow up to the RFC/discussion we had earlier on the list [1].

The first 4 commits add the required functionality to ensure we can print all refs (regular, pseudo, HEAD). The 5th commit modifies the git-for-each-ref(1) command to add the "--include-root-refs" command which will include HEAD and pseudorefs with regular "refs/" refs.

[1]: https://lore.kernel.org/git/20231221170715.110565-1-karthik.188@gmail.com/#t

Changes from v3:
1. Move from using 'git for-each-ref ""' to print root refs to adding
the '--include-root-refs' option for git-for-each-ref(1). This provides better
UX for users.
2. Modify `is_pseudoref()` to use `refs_resolve_ref_unsafe`.
3. Includes reftable-backend changes and is now rebased on top of next (ed35d3359).  

Range-diff:

1:  2141a2a62b ! 1:  98130a7ad7 refs: introduce `is_pseudoref()` and `is_headref()`
    @@ Commit message
         related files like 'BISECT_START' to a new directory similar to the
         'rebase-merge' directory.
     
    +    Helped-by: Jeff King <peff@peff.net>
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
     
      ## refs.c ##
    @@ refs.c: static int is_pseudoref_syntax(const char *refname)
     +		return 0;
     +
     +	if (ends_with(refname, "_HEAD")) {
    -+		 read_ref_full(refname, RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE,
    -+		      &oid, NULL);
    -+		 return !is_null_oid(&oid);
    ++		refs_resolve_ref_unsafe(refs, refname,
    ++   					RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE,
    ++   					&oid, NULL);
    ++   		return !is_null_oid(&oid);
     +	}
     +
     +	for (i = 0; i < ARRAY_SIZE(irregular_pseudorefs); i++)
     +		if (!strcmp(refname, irregular_pseudorefs[i])) {
    -+			read_ref_full(refname, RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE,
    -+						  &oid, NULL);
    ++			refs_resolve_ref_unsafe(refs, refname,
    ++   						RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE,
    ++   						&oid, NULL);
     +			return !is_null_oid(&oid);
     +		}
     +
2:  c96f0a9c83 = 2:  060ab08af5 refs: extract out `loose_fill_ref_dir_regular_file()`
3:  d165358b83 ! 3:  40d2375ad9 refs: introduce `refs_for_each_all_refs()`
    @@ Metadata
     Author: Karthik Nayak <karthik.188@gmail.com>
     
      ## Commit message ##
    -    refs: introduce `refs_for_each_all_refs()`
    +    refs: introduce `refs_for_each_include_root_refs()`
     
    -    Introduce a new ref iteration flag `DO_FOR_EACH_INCLUDE_ALL_REFS`, which
    -    will be used to iterate over all refs. In the files backend this is
    -    limited to regular refs, pseudorefs and HEAD. For other backends like
    -    the reftable this is the universal set of all refs stored in the
    -    backend.
    +    Introduce a new ref iteration flag `DO_FOR_EACH_INCLUDE_ROOT_REFS`,
    +    which will be used to iterate over regular refs plus pseudorefs and
    +    HEAD.
     
         Refs which fall outside the `refs/` and aren't either pseudorefs or HEAD
         are more of a grey area. This is because we don't block the users from
    -    creating such refs but they are not officially supported. In the files
    -    backend, we can isolate such files from other files.
    +    creating such refs but they are not officially supported.
     
    -    Introduce `refs_for_each_all_refs()` which calls `do_for_each_ref()`
    -    with this newly introduced flag.
    +    Introduce `refs_for_each_include_root_refs()` which calls
    +    `do_for_each_ref()` with this newly introduced flag.
     
         In `refs/files-backend.c`, introduce a new function
         `add_pseudoref_and_head_entries()` to add pseudorefs and HEAD to the
         `ref_dir`. We then finally call `add_pseudoref_and_head_entries()`
    -    whenever the `DO_FOR_EACH_INCLUDE_ALL_REFS` flag is set. Any new ref
    +    whenever the `DO_FOR_EACH_INCLUDE_ROOT_REFS` flag is set. Any new ref
         backend will also have to implement similar changes on its end.
     
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
    @@ refs.c: int for_each_rawref(each_ref_fn fn, void *cb_data)
      	return refs_for_each_rawref(get_main_ref_store(the_repository), fn, cb_data);
      }
      
    -+int refs_for_each_all_refs(struct ref_store *refs, each_ref_fn fn,
    -+			   void *cb_data)
    ++int refs_for_each_include_root_refs(struct ref_store *refs, each_ref_fn fn,
    ++				    void *cb_data)
     +{
     +	return do_for_each_ref(refs, "", NULL, fn, 0,
    -+			       DO_FOR_EACH_INCLUDE_ALL_REFS, cb_data);
    ++			       DO_FOR_EACH_INCLUDE_ROOT_REFS, cb_data);
     +}
     +
      static int qsort_strcmp(const void *va, const void *vb)
    @@ refs.h: int for_each_namespaced_ref(const char **exclude_patterns,
      int for_each_rawref(each_ref_fn fn, void *cb_data);
      
     +/*
    -+ * Iterates over all ref types, regular, pseudorefs and HEAD.
    ++ * Iterates over all refs including root refs, i.e. pseudorefs and HEAD.
     + */
    -+int refs_for_each_all_refs(struct ref_store *refs, each_ref_fn fn,
    -+			   void *cb_data);
    ++int refs_for_each_include_root_refs(struct ref_store *refs, each_ref_fn fn,
    ++				    void *cb_data);
     +
      /*
       * Normalizes partial refs to their fully qualified form.
    @@ refs/files-backend.c: static struct ref_cache *get_loose_ref_cache(struct files_
      
     +		dir = get_ref_dir(refs->loose->root);
     +
    -+		if (flags & DO_FOR_EACH_INCLUDE_ALL_REFS)
    ++		if (flags & DO_FOR_EACH_INCLUDE_ROOT_REFS)
     +			add_pseudoref_and_head_entries(dir->cache->ref_store, dir,
    -+										   refs->loose->root->name);
    ++						       refs->loose->root->name);
     +
      		/*
      		 * Add an incomplete entry for "refs/" (to be filled
    @@ refs/refs-internal.h: enum do_for_each_ref_flags {
      	DO_FOR_EACH_OMIT_DANGLING_SYMREFS = (1 << 2),
     +
     +	/*
    -+	 * Include all refs in the $GIT_DIR in contrast to generally only listing
    -+	 * references having the "refs/" prefix. In the files-backend this is
    -+	 * limited to regular refs, pseudorefs and HEAD.
    ++	 * Include root refs i.e. HEAD and pseudorefs along with the regular
    ++	 * refs.
     +	 */
    -+	DO_FOR_EACH_INCLUDE_ALL_REFS = (1 << 3),
    ++	DO_FOR_EACH_INCLUDE_ROOT_REFS = (1 << 3),
      };
      
      /*
4:  a17983d0ba < -:  ---------- for-each-ref: avoid filtering on empty pattern
-:  ---------- > 4:  b4b9435505 ref-filter: rename 'FILTER_REFS_ALL' to 'FILTER_REFS_REGULAR'
-:  ---------- > 5:  ee99ac41ae for-each-ref: add new option to include root refs

Karthik Nayak (5):
  refs: introduce `is_pseudoref()` and `is_headref()`
  refs: extract out `loose_fill_ref_dir_regular_file()`
  refs: introduce `refs_for_each_include_root_refs()`
  ref-filter: rename 'FILTER_REFS_ALL' to 'FILTER_REFS_REGULAR'
  for-each-ref: add new option to include root refs

 Documentation/git-for-each-ref.txt |   5 +-
 builtin/for-each-ref.c             |  11 ++-
 ref-filter.c                       |  29 ++++++-
 ref-filter.h                       |   7 +-
 refs.c                             |  48 +++++++++++
 refs.h                             |   9 ++
 refs/files-backend.c               | 127 +++++++++++++++++++++--------
 refs/refs-internal.h               |   6 ++
 refs/reftable-backend.c            |  11 ++-
 t/t6302-for-each-ref-filter.sh     |  31 +++++++
 10 files changed, 238 insertions(+), 46 deletions(-)