git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 02/10] builtin/cat-file: wire up an option to filter objects

From
Toon Claes <toon@iotcl.com>
Date
Apr 1, 2025, 11:45 UTC
Message-ID
<87r02cf6l1.fsf@iotcl.com>
In-Reply-To
<20250327-pks-cat-file-object-type-filter-v2-2-4bbc7085d7c5@pks.im>
Patrick Steinhardt <ps@pks.im> writes:
Show 74 quoted lines
> In batch mode, git-cat-file(1) enumerates all objects and prints them
> by iterating through both loose and packed objects. This works without
> considering their reachability at all, and consequently most options to
> filter objects as they exist in e.g. git-rev-list(1) are not applicable.
> In some situations it may still be useful though to filter objects based
> on properties that are inherent to them. This includes the object size
> as well as its type.
>
> Such a filter already exists in git-rev-list(1) with the `--filter=`
> command line option. While this option supports a couple of filters that
> are not applicable to our usecase, some of them are quite a neat fit.
>
> Wire up the filter as an option for git-cat-file(1). This allows us to
> reuse the same syntax as in git-rev-list(1) so that we don't have to
> reinvent the wheel. For now, we die when any of the filter options has
> been passed by the user, but they will be wired up in subsequent
> commits.
>
> Further note that the filters that we are about to introduce don't
> significantly speed up the runtime of git-cat-file(1). While we can skip
> emitting a lot of objects in case they are uninteresting to us, the
> majority of time is spent reading the packfile, which is bottlenecked by
> I/O and not the processor. This will change though once we start to make
> use of bitmaps, which will allow us to skip reading the whole packfile.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  Documentation/git-cat-file.adoc |  6 ++++++
>  builtin/cat-file.c              | 37 +++++++++++++++++++++++++++++++++----
>  t/t1006-cat-file.sh             | 32 ++++++++++++++++++++++++++++++++
>  3 files changed, 71 insertions(+), 4 deletions(-)
>
> diff --git a/Documentation/git-cat-file.adoc b/Documentation/git-cat-file.adoc
> index d5890ae3686..f7f57b7f538 100644
> --- a/Documentation/git-cat-file.adoc
> +++ b/Documentation/git-cat-file.adoc
> @@ -81,6 +81,12 @@ OPTIONS
>  	end-of-line conversion, etc). In this case, `<object>` has to be of
>  	the form `<tree-ish>:<path>`, or `:<path>`.
>  
> +--filter=<filter-spec>::
> +--no-filter::
> +	Omit objects from the list of printed objects. This can only be used in
> +	combination with one of the batched modes. The '<filter-spec>' may be
> +	one of the following:
> +
>  --path=<path>::
>  	For use with `--textconv` or `--filters`, to allow specifying an object
>  	name and a path separately, e.g. when it is difficult to figure out
> diff --git a/builtin/cat-file.c b/builtin/cat-file.c
> index 8e40016dd24..940900d92ad 100644
> --- a/builtin/cat-file.c
> +++ b/builtin/cat-file.c
> @@ -936,10 +946,13 @@ int cmd_cat_file(int argc,
>  	int opt_cw = 0;
>  	int opt_epts = 0;
>  	const char *exp_type = NULL, *obj_name = NULL;
> -	struct batch_options batch = {0};
> +	struct batch_options batch = {
> +		.objects_filter = LIST_OBJECTS_FILTER_INIT,
> +	};
>  	int unknown_type = 0;
>  	int input_nul_terminated = 0;
>  	int nul_terminated = 0;
> +	int ret;
>  
>  	const char * const builtin_catfile_usage[] = {
>  		N_("git cat-file <type> <object>"),
> @@ -1000,6 +1013,8 @@ int cmd_cat_file(int argc,
>  			    N_("run filters on object's content"), 'w'),
>  		OPT_STRING(0, "path", &force_path, N_("blob|tree"),
>  			   N_("use a <path> for (--textconv | --filters); Not with 'batch'")),
> +		OPT_CALLBACK(0, "filter", &batch.objects_filter, N_("args"),
> +			     N_("object filtering"), opt_parse_list_objects_filter),

Because we've decided on `--filter` we can use `OPT_PARSE_LIST_OBJECTS_FILTER` here now.

-- Toon

Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 7 of 28 in “builtin/cat-file: allow filtering objects in batch mode”
  1. 00/10 builtin/cat-file: allow filtering objects in batch modePatrick Steinhardt, Mar 27, 2025
  2. 01/10 builtin/cat-file: rename variable that tracks usagePatrick Steinhardt, Mar 27, 2025
  3. Karthik NayakApr 1, 2025
  4. Patrick SteinhardtApr 2, 2025
  5. Junio C HamanoApr 7, 2025
  6. 02/10 builtin/cat-file: wire up an option to filter objectsPatrick Steinhardt, Mar 27, 2025
  7. Toon ClaesApr 1, 2025
  8. Patrick SteinhardtApr 2, 2025
  9. Karthik NayakApr 1, 2025
  10. Patrick SteinhardtApr 2, 2025
  11. 04/10 builtin/cat-file: support "blob:limit=" objects filterPatrick Steinhardt, Mar 27, 2025
  12. 03/10 builtin/cat-file: support "blob:none" objects filterPatrick Steinhardt, Mar 27, 2025
  13. Karthik NayakApr 1, 2025
  14. Karthik NayakApr 1, 2025
  15. Patrick SteinhardtApr 2, 2025
  16. 05/10 builtin/cat-file: support "object:type=" objects filterPatrick Steinhardt, Mar 27, 2025
  17. 06/10 pack-bitmap: allow passing payloads to `show_reachable_fn()`Patrick Steinhardt, Mar 27, 2025
  18. Toon ClaesApr 1, 2025
  19. Patrick SteinhardtApr 2, 2025
  20. 07/10 pack-bitmap: add function to iterate over filtered bitmapped objectsPatrick Steinhardt, Mar 27, 2025
  21. 08/10 pack-bitmap: introduce function to check whether a pack is bitmappedPatrick Steinhardt, Mar 27, 2025
  22. Toon ClaesApr 1, 2025
  23. Patrick SteinhardtApr 2, 2025
  24. 10/10 builtin/cat-file: use bitmaps to efficiently filter by object typePatrick Steinhardt, Mar 27, 2025
  25. 09/10 builtin/cat-file: deduplicate logic to iterate over all objectsPatrick Steinhardt, Mar 27, 2025
  26. Toon ClaesApr 1, 2025
  27. Patrick SteinhardtApr 2, 2025
  28. Toon ClaesApr 3, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.