Re: [PATCH v3 2/4] pack-refs: teach --exclude option to exclude refs from being packed
- From
Taylor Blau <me@ttaylorr.com>
- Date
- May 12, 2023, 00:00 UTC
- Message-ID
- <ZF2BmI/SnhfEDB2K@nand.local>
- In-Reply-To
- <8c5c66a3050ee1c0ce8a48a088f5ecc2df7d1e3a.1683828635.git.gitgitgadget@gmail.com>
On Thu, May 11, 2023 at 06:10:32PM +0000, John Cai via GitGitGadget wrote:
Show 53 quoted lines
> From: John Cai <johncai86@gmail.com> > > At GitLab, we have a system that creates ephemeral internal refs that > don't live long before getting deleted. Having an option to exclude > certain refs from a packed-refs file allows these internal references to > be deleted much more efficiently. > > Add an --exclude option to the pack-refs builtin, and use the ref > exclusions API to exclude certain refs from being packed into the final > packed-refs file > > Signed-off-by: John Cai <johncai86@gmail.com> > --- > Documentation/git-pack-refs.txt | 12 +++++++++++- > builtin/pack-refs.c | 20 ++++++++++++++++---- > refs.c | 4 ++-- > refs.h | 7 ++++++- > refs/debug.c | 4 ++-- > refs/files-backend.c | 16 ++++++++++------ > refs/packed-backend.c | 2 +- > refs/refs-internal.h | 3 ++- > revision.h | 2 +- > t/helper/test-ref-store.c | 3 ++- > t/t3210-pack-refs.sh | 16 ++++++++++++++++ > 11 files changed, 69 insertions(+), 20 deletions(-) > > diff --git a/Documentation/git-pack-refs.txt b/Documentation/git-pack-refs.txt > index e011e5fead3..c0f7426e519 100644 > --- a/Documentation/git-pack-refs.txt > +++ b/Documentation/git-pack-refs.txt > @@ -8,7 +8,7 @@ git-pack-refs - Pack heads and tags for efficient repository access > SYNOPSIS > -------- > [verse] > -'git pack-refs' [--all] [--no-prune] > +'git pack-refs' [--all] [--no-prune] [--exclude <pattern>] > > DESCRIPTION > ----------- > @@ -60,6 +60,16 @@ interests. > The command usually removes loose refs under `$GIT_DIR/refs` > hierarchy after packing them. This option tells it not to. > > +--exclude <pattern>:: > + > +Do not pack refs matching the given `glob(7)` pattern. Repetitions of this option > +accumulate exclusion patterns. Use `--no-exclude` to clear and reset the list of > +patterns. If a ref is already packed, including it with `--exclude` will not > +unpack it. > + > +When used with `--all`, it will use the difference between the set of all refs, > +and what is provided to `--exclude`. > +
I think this last paragraph could be simplified, though feel free to discard my suggestion if you think it makes things less clear.
When used with `--all`, pack only loose refs which do not match any of the provided `--exclude` patterns.
Show 7 quoted lines
> int cmd_pack_refs(int argc, const char **argv, const char *prefix)
> {
> unsigned int flags = PACK_REFS_PRUNE;
> + static struct ref_exclusions excludes = REF_EXCLUSIONS_INIT;
> + struct pack_refs_opts pack_refs_opts = {.exclusions = &excludes, .flags = flags};
> + static struct string_list option_excluded_refs = STRING_LIST_INIT_NODUP;
> + struct string_list_item *item;Since this list does not appear to be sensitive to its order, have you considered using the strvec API instead of the string_list one?
Show 27 quoted lines
> diff --git a/refs.c b/refs.c
> index d2a98e1c21f..881a0da65cf 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -2132,9 +2132,9 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
> }
>
> /* backend functions */
> -int refs_pack_refs(struct ref_store *refs, unsigned int flags)
> +int refs_pack_refs(struct ref_store *refs, struct pack_refs_opts *opts)
> {
> - return refs->be->pack_refs(refs, flags);
> + return refs->be->pack_refs(refs, opts);
> }
>
> int peel_iterated_oid(const struct object_id *base, struct object_id *peeled)
> diff --git a/refs.h b/refs.h
> index 123cfa44244..46020bd335c 100644
> --- a/refs.h
> +++ b/refs.h
> @@ -63,6 +63,11 @@ struct worktree;
> #define RESOLVE_REF_NO_RECURSE 0x02
> #define RESOLVE_REF_ALLOW_BAD_NAME 0x04
>
> +struct pack_refs_opts {
> + unsigned int flags;
> + struct ref_exclusions *exclusions;I think this would be OK to include directly in the struct instead of via a pointer, but either is fine.
Show 14 quoted lines
> @@ -1175,15 +1176,18 @@ static void prune_refs(struct files_ref_store *refs, struct ref_to_prune **refs_
> */
> static int should_pack_ref(const char *refname,
> const struct object_id *oid, unsigned int ref_flags,
> - unsigned int pack_flags)
> + struct pack_refs_opts *opts)
> {
> /* Do not pack per-worktree refs: */
> if (parse_worktree_ref(refname, NULL, NULL, NULL) !=
> REF_WORKTREE_SHARED)
> return 0;
>
> + if (opts->exclusions && ref_excluded(opts->exclusions, refname))
> + return 0;Looks good, here is where we throw out refs that we don't want. I wonder if ref_excluded() does the right thing with a zero-initialized argument (i.e. that it behaves as if nothing matches).
I wonder if it's possible to skip over certain loose references by avoiding traversal into the sub-directories for simple prefixes. That may be a premature optimization, though, so I don't think you necessarily need to worry about it in this round.
Show 15 quoted lines
> +test_expect_success 'test excluded refs are not packed' ' > + git branch dont_pack1 && > + git branch dont_pack2 && > + git branch pack_this && > + git pack-refs --all --exclude "refs/heads/dont_pack*" && > + test -f .git/refs/heads/dont_pack1 && > + test -f .git/refs/heads/dont_pack2 && > + ! test -f ./git/refs/heads/pack_this' > + > +test_expect_success 'test --no-exclude refs clears excluded refs' ' > + git branch dont_pack3 && > + git branch dont_pack4 && > + git pack-refs --all --exclude "refs/heads/dont_pack*" --no-exclude && > + ! test -f .git/refs/heads/dont_pack3 && > + ! test -f .git/refs/heads/dont_pack4'
Tests look good. The trailing quote is a little odd to be placed on the last line of the test instead of off on its own, but I suppose that is imitating existing style, which is OK.
Thanks, Taylor