git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 14/15] [GSOC] cat-file: re-implement --textconv, --filters options

From
ZheNing Hu via GitGitGadget <gitgitgadget@gmail.com>
Date
Jul 1, 2021, 16:08 UTC
Message-ID
<3aeb4d3d3ec9dc112a19a8b4adeaf213577b4028.1625155693.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.989.git.1625155693.gitgitgadget@gmail.com>
From: ZheNing Hu <adlternative@gmail.com>

After cat-file reuses the ref-filter logic, we re-implement the functions of --textconv and --filters options.

Add members `use_textconv` and `use_filters` in struct `ref_format`, and use global variables `use_filters` and `use_textconv` in `ref-filter.c`, so that we can filter the content of the object in get_object(). Use `actual_oi` to record the real expand_data: it may point to the original `oi` or the `act_oi` processed by `textconv_object()` or `convert_to_working_tree()`. `grab_values()` will grab the contents of `actual_oi` and `grab_common_values()` to grab the contents of origin `oi`, this ensures that `%(objectsize)` still uses the size of the unfiltered data.

In `get_object()`, we made an optimization: Firstly, get the size and type of the object instead of directly getting the object data. If using --textconv, after successfully obtaining the filtered object data, an extra oid_object_info_extended() will be skipped, which can reduce the cost of object data copy; If using --filter, the data of the object first will be getted first, and then convert_to_working_tree() will be used to get the filtered object data.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Hariom Verma <hariom18599@gmail.com>
Signed-off-by: ZheNing Hu <adlternative@gmail.com>
---
 builtin/cat-file.c |  6 +++++
 ref-filter.c       | 59 ++++++++++++++++++++++++++++++++++++++++++++--
 ref-filter.h       |  2 ++
 3 files changed, 65 insertions(+), 2 deletions(-)
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index dc604a9879d..710085b7d72 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -380,6 +380,12 @@ static int batch_objects(struct batch_options *batch, const struct option *optio
 	if (batch->print_contents)
 		strbuf_addstr(&format, "\n%(raw)");
 	batch->format.format = format.buf;
+
+	if (batch->cmdmode == 'c')
+		batch->format.use_textconv = 1;
+	else if (batch->cmdmode == 'w')
+		batch->format.use_filters = 1;
+
 	if (verify_ref_format(&batch->format))
 		usage_with_options(cat_file_usage, options);
 
diff --git a/ref-filter.c b/ref-filter.c
index 9ca3dd5557d..427552b8108 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -1,3 +1,4 @@
+#define USE_THE_INDEX_COMPATIBILITY_MACROS
 #include "builtin.h"
 #include "cache.h"
 #include "parse-options.h"
@@ -84,6 +85,9 @@ static struct expand_data {
 	struct object_info info;
 } oi, oi_deref;
 
+int use_filters;
+int use_textconv;
+
 struct ref_to_worktree_entry {
 	struct hashmap_entry ent;
 	struct worktree *wt; /* key is wt->head_ref */
@@ -1031,6 +1035,9 @@ int verify_ref_format(struct ref_format *format)
 					       used_atom[at].atom_type == ATOM_WORKTREEPATH)))
 			die(_("this command reject atom %%(%.*s)"), (int)(ep - sp - 2), sp + 2);
 
+		use_filters = format->use_filters;
+		use_textconv = format->use_textconv;
+
 		if ((format->quote_style == QUOTE_PYTHON ||
 		     format->quote_style == QUOTE_SHELL ||
 		     format->quote_style == QUOTE_TCL) &&
@@ -1742,10 +1749,38 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj
 {
 	/* parse_object_buffer() will set eaten to 0 if free() will be needed */
 	int eaten = 1;
+	struct expand_data *actual_oi = oi;
+	struct expand_data act_oi = {0};
+
 	if (oi->info.contentp) {
 		/* We need to know that to use parse_object_buffer properly */
+		void **temp_contentp = oi->info.contentp;
+		oi->info.contentp = NULL;
 		oi->info.sizep = &oi->size;
 		oi->info.typep = &oi->type;
+
+		/* get the type and size */
+		if (oid_object_info_extended(the_repository, &oi->oid, &oi->info,
+					OBJECT_INFO_LOOKUP_REPLACE))
+			return strbuf_addf_ret(err, 1, _("%s missing"),
+					       oid_to_hex(&oi->oid));
+
+		oi->info.sizep = NULL;
+		oi->info.typep = NULL;
+		oi->info.contentp = temp_contentp;
+
+		if (use_textconv && !ref->rest)
+			return strbuf_addf_ret(err, -1, _("missing path for '%s'"),
+					       oid_to_hex(&act_oi.oid));
+		if (use_textconv && oi->type == OBJ_BLOB) {
+			act_oi = *oi;
+			if (textconv_object(the_repository,
+					    ref->rest, 0100644, &act_oi.oid,
+					    1, (char **)(&act_oi.content), &act_oi.size)) {
+				actual_oi = &act_oi;
+				goto success;
+			}
+		}
 	}
 	if (oid_object_info_extended(the_repository, &oi->oid, &oi->info,
 				     OBJECT_INFO_LOOKUP_REPLACE))
@@ -1755,19 +1790,39 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj
 		BUG("Object size is less than zero.");
 
 	if (oi->info.contentp) {
-		*obj = parse_object_buffer(the_repository, &oi->oid, oi->type, oi->size, oi->content, &eaten);
+		if (use_filters && !ref->rest)
+			return strbuf_addf_ret(err, -1, _("missing path for '%s'"),
+					       oid_to_hex(&oi->oid));
+		if (use_filters && oi->type == OBJ_BLOB) {
+			struct strbuf strbuf = STRBUF_INIT;
+			struct checkout_metadata meta;
+			act_oi = *oi;
+
+			init_checkout_metadata(&meta, NULL, NULL, &act_oi.oid);
+			if (!convert_to_working_tree(&the_index, ref->rest, act_oi.content, act_oi.size, &strbuf, &meta))
+				die("could not convert '%s' %s",
+					oid_to_hex(&oi->oid), ref->rest);
+			act_oi.size = strbuf.len;
+			act_oi.content = strbuf_detach(&strbuf, NULL);
+			actual_oi = &act_oi;
+		}
+
+success:
+		*obj = parse_object_buffer(the_repository, &actual_oi->oid, actual_oi->type, actual_oi->size, actual_oi->content, &eaten);
 		if (!*obj) {
 			if (!eaten)
 				free(oi->content);
 			return strbuf_addf_ret(err, -1, _("parse_object_buffer failed on %s for %s"),
 					       oid_to_hex(&oi->oid), ref->refname);
 		}
-		grab_values(ref->value, deref, *obj, oi);
+		grab_values(ref->value, deref, *obj, actual_oi);
 	}
 
 	grab_common_values(ref->value, deref, oi);
 	if (!eaten)
 		free(oi->content);
+	if (actual_oi != oi)
+		free(actual_oi->content);
 	return 0;
 }
 
diff --git a/ref-filter.h b/ref-filter.h
index 053980a6a42..497e3e93632 100644
--- a/ref-filter.h
+++ b/ref-filter.h
@@ -80,6 +80,8 @@ struct ref_format {
 	const char *rest;
 	int cat_file_mode;
 	int quote_style;
+	int use_textconv;
+	int use_filters;
 	int use_rest;
 	int use_color;
 

-- 
gitgitgadget
Previous: ZheNing Hu via GitGitGadgetNext: Junio C Hamano
Message 33 of 39 in “[GSOC] cat-file: reuse ref-filter logic”
  1. 00/15 [GSOC] cat-file: reuse ref-filter logicZheNing Hu via GitGitGadget, Jul 1, 2021
  2. 01/15 [GSOC] ref-filter: add obj-type check in grab contentsZheNing Hu via GitGitGadget, Jul 1, 2021
  3. 03/15 [GSOC] ref-filter: --format=%(raw) re-support --perlZheNing Hu via GitGitGadget, Jul 1, 2021
  4. Ævar Arnfjörð BjarmasonJul 2, 2021
  5. ZheNing HuJul 3, 2021
  6. 02/15 [GSOC] ref-filter: add %(raw) atomZheNing Hu via GitGitGadget, Jul 1, 2021
  7. Ævar Arnfjörð BjarmasonJul 2, 2021
  8. ZheNing HuJul 3, 2021
  9. 05/15 [GSOC] ref-filter: add %(rest) atomZheNing Hu via GitGitGadget, Jul 1, 2021
  10. 04/15 [GSOC] ref-filter: use non-const ref_format in *_atom_parser()ZheNing Hu via GitGitGadget, Jul 1, 2021
  11. 06/15 [GSOC] ref-filter: pass get_object() return value to their callersZheNing Hu via GitGitGadget, Jul 1, 2021
  12. 08/15 [GSOC] ref-filter: add cat_file_mode in struct ref_formatZheNing Hu via GitGitGadget, Jul 1, 2021
  13. Ævar Arnfjörð BjarmasonJul 2, 2021
  14. Eric SunshineJul 2, 2021
  15. Christian CouderJul 2, 2021
  16. ZheNing HuJul 3, 2021
  17. Ævar Arnfjörð BjarmasonJul 5, 2021
  18. 07/15 [GSOC] ref-filter: introduce free_ref_array_item_value() functionZheNing Hu via GitGitGadget, Jul 1, 2021
  19. 09/15 [GSOC] ref-filter: modify the error message and value in get_objectZheNing Hu via GitGitGadget, Jul 1, 2021
  20. 11/15 [GSOC] cat-file: change batch_objects parameter nameZheNing Hu via GitGitGadget, Jul 1, 2021
  21. 12/15 [GSOC] cat-file: reuse ref-filter logicZheNing Hu via GitGitGadget, Jul 1, 2021
  22. Ævar Arnfjörð BjarmasonJul 2, 2021
  23. Christian CouderJul 2, 2021
  24. ZheNing HuJul 3, 2021
  25. ZheNing HuJul 3, 2021
  26. Ævar Arnfjörð BjarmasonJul 3, 2021
  27. ZheNing HuJul 4, 2021
  28. Ævar Arnfjörð BjarmasonJul 5, 2021
  29. 10/15 [GSOC] cat-file: add has_object_file() checkZheNing Hu via GitGitGadget, Jul 1, 2021
  30. Ævar Arnfjörð BjarmasonJul 2, 2021
  31. ZheNing HuJul 3, 2021
  32. 13/15 [GSOC] cat-file: reuse err buf in batch_object_write()ZheNing Hu via GitGitGadget, Jul 1, 2021
  33. 14/15 [GSOC] cat-file: re-implement --textconv, --filters optionsZheNing Hu via GitGitGadget, Jul 1, 2021
  34. Junio C HamanoJul 1, 2021
  35. Junio C HamanoJul 1, 2021
  36. ZheNing HuJul 2, 2021
  37. Junio C HamanoJul 2, 2021
  38. ZheNing HuJul 3, 2021
  39. 15/15 [GSOC] ref-filter: remove grab_oid() functionZheNing Hu via GitGitGadget, Jul 1, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.