{"thread":{"id":"56274","subject":"[PATCH 02/27] [GSOC] ref-filter: add %(raw) atom","startedAt":"2021-08-13T08:23:20Z","lastAt":"2021-08-13T08:23:53Z","messageCount":28,"participants":["ZheNing Hu via GitGitGadget"],"isPatch":true,"patchVersion":1,"patchTotal":27},"messages":[{"id":"432633","messageId":"59577ccf140900bd04177ffc7ef0afb92c2d45e2.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 02/27] [GSOC] ref-filter: add %(raw) atom","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:45Z","receivedAt":"2021-08-13T08:23:20Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nAdd new formatting option `%(raw)`, which will print the raw\nobject data without any changes. It will help further to migrate\nall cat-file formatting logic from cat-file to ref-filter.\n\nThe raw data of blob, tree objects may contain '\\0', but most of\nthe logic in `ref-filter` depends on the output of the atom being\ntext (specifically, no embedded NULs in it).\n\nE.g. `quote_formatting()` use `strbuf_addstr()` or `*._quote_buf()`\nadd the data to the buffer. The raw data of a tree object is\n`100644 one\\0...`, only the `100644 one` will be added to the buffer,\nwhich is incorrect.\n\nTherefore, we need to find a way to record the length of the\natom_value's member `s`. Although strbuf can already record the\nstring and its length, if we want to replace the type of atom_value's\nmember `s` with strbuf, many places in ref-filter that are filled\nwith dynamically allocated mermory in `v->s` are not easy to replace.\nAt the same time, we need to check if `v->s == NULL` in\npopulate_value(), and strbuf cannot easily distinguish NULL and empty\nstrings, but c-style \"const char *\" can do it. So add a new member in\n`struct atom_value`: `s_size`, which can record raw object size, it\ncan help us add raw object data to the buffer or compare two buffers\nwhich contain raw object data.\n\nNote that `--format=%(raw)` cannot be used with `--python`, `--shell`,\n`--tcl`, and `--perl` because if the binary raw data is passed to a\nvariable in such languages, these may not support arbitrary binary data\nin their string variable type.\n\nReviewed-by: Jacob Keller \nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nHelped-by: Bagas Sanjaya \nHelped-by: Ævar Arnfjörð Bjarmason \nHelped-by: Felipe Contreras \nHelped-by: Phillip Wood \nHelped-by: Junio C Hamano \nBased-on-patch-by: Olga Telezhnaya \nSigned-off-by: ZheNing Hu \n---\n Documentation/git-for-each-ref.txt | 9 ++\n ref-filter.c | 143 +++++++++++++++----\n t/t6300-for-each-ref.sh | 216 +++++++++++++++++++++++++++++\n 3 files changed, 340 insertions(+), 28 deletions(-)\n\ndiff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt\nindex 2ae2478de70..cbb6f87d13f 100644\n--- a/Documentation/git-for-each-ref.txt\n+++ b/Documentation/git-for-each-ref.txt\n@@ -235,6 +235,15 @@ and `date` to extract the named component. For email fields (`authoremail`,\n without angle brackets, and `:localpart` to get the part before the `@` symbol\n out of the trimmed email.\n \n+The raw data in an object is `raw`.\n+\n+raw:size::\n+\tThe raw data size of the object.\n+\n+Note that `--format=%(raw)` can not be used with `--python`, `--shell`, `--tcl`,\n+`--perl` because such language may not support arbitrary binary data in their\n+string variable type.\n+\n The message in a commit or a tag object is `contents`, from which\n `contents:` can be used to extract various parts out of:\n \ndiff --git a/ref-filter.c b/ref-filter.c\nindex e3715210e5c..1e660f60c4b 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -144,6 +144,7 @@ enum atom_type {\n \tATOM_BODY,\n \tATOM_TRAILERS,\n \tATOM_CONTENTS,\n+\tATOM_RAW,\n \tATOM_UPSTREAM,\n \tATOM_PUSH,\n \tATOM_SYMREF,\n@@ -189,6 +190,9 @@ static struct used_atom {\n \t\t\tstruct process_trailer_options trailer_opts;\n \t\t\tunsigned int nlines;\n \t\t} contents;\n+\t\tstruct {\n+\t\t\tenum { RAW_BARE, RAW_LENGTH } option;\n+\t\t} raw_data;\n \t\tstruct {\n \t\t\tcmp_status cmp_status;\n \t\t\tconst char *str;\n@@ -427,6 +431,18 @@ static int contents_atom_parser(const struct ref_format *format, struct used_ato\n \treturn 0;\n }\n \n+static int raw_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+\t\t\t\tconst char *arg, struct strbuf *err)\n+{\n+\tif (!arg)\n+\t\tatom->u.raw_data.option = RAW_BARE;\n+\telse if (!strcmp(arg, \"size\"))\n+\t\tatom->u.raw_data.option = RAW_LENGTH;\n+\telse\n+\t\treturn strbuf_addf_ret(err, -1, _(\"unrecognized %%(raw) argument: %s\"), arg);\n+\treturn 0;\n+}\n+\n static int oid_atom_parser(const struct ref_format *format, struct used_atom *atom,\n \t\t\t const char *arg, struct strbuf *err)\n {\n@@ -587,6 +603,7 @@ static struct {\n \t[ATOM_BODY] = { \"body\", SOURCE_OBJ, FIELD_STR, body_atom_parser },\n \t[ATOM_TRAILERS] = { \"trailers\", SOURCE_OBJ, FIELD_STR, trailers_atom_parser },\n \t[ATOM_CONTENTS] = { \"contents\", SOURCE_OBJ, FIELD_STR, contents_atom_parser },\n+\t[ATOM_RAW] = { \"raw\", SOURCE_OBJ, FIELD_STR, raw_atom_parser },\n \t[ATOM_UPSTREAM] = { \"upstream\", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser },\n \t[ATOM_PUSH] = { \"push\", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser },\n \t[ATOM_SYMREF] = { \"symref\", SOURCE_NONE, FIELD_STR, refname_atom_parser },\n@@ -621,12 +638,19 @@ struct ref_formatting_state {\n \n struct atom_value {\n \tconst char *s;\n+\tssize_t s_size;\n \tint (*handler)(struct atom_value *atomv, struct ref_formatting_state *state,\n \t\t struct strbuf *err);\n \tuintmax_t value; /* used for sorting when not FIELD_STR */\n \tstruct used_atom *atom;\n };\n \n+#define ATOM_SIZE_UNSPECIFIED (-1)\n+\n+#define ATOM_VALUE_INIT { \\\n+\t.s_size = ATOM_SIZE_UNSPECIFIED \\\n+}\n+\n /*\n * Used to parse format string and sort specifiers\n */\n@@ -645,13 +669,6 @@ static int parse_ref_filter_atom(const struct ref_format *format,\n \t\treturn strbuf_addf_ret(err, -1, _(\"malformed field name: %.*s\"),\n \t\t\t\t (int)(ep-atom), atom);\n \n-\t/* Do we have the atom already used elsewhere? */\n-\tfor (i = 0; i < used_atom_cnt; i++) {\n-\t\tint len = strlen(used_atom[i].name);\n-\t\tif (len == ep - atom && !memcmp(used_atom[i].name, atom, len))\n-\t\t\treturn i;\n-\t}\n-\n \t/*\n \t * If the atom name has a colon, strip it and everything after\n \t * it off - it specifies the format for this entry, and\n@@ -661,6 +678,13 @@ static int parse_ref_filter_atom(const struct ref_format *format,\n \targ = memchr(sp, ':', ep - sp);\n \tatom_len = (arg ? arg : ep) - sp;\n \n+\t/* Do we have the atom already used elsewhere? */\n+\tfor (i = 0; i < used_atom_cnt; i++) {\n+\t\tint len = strlen(used_atom[i].name);\n+\t\tif (len == ep - atom && !memcmp(used_atom[i].name, atom, len))\n+\t\t\treturn i;\n+\t}\n+\n \t/* Is the atom a valid one? */\n \tfor (i = 0; i < ARRAY_SIZE(valid_atom); i++) {\n \t\tint len = strlen(valid_atom[i].name);\n@@ -710,11 +734,14 @@ static int parse_ref_filter_atom(const struct ref_format *format,\n \treturn at;\n }\n \n-static void quote_formatting(struct strbuf *s, const char *str, int quote_style)\n+static void quote_formatting(struct strbuf *s, const char *str, ssize_t len, int quote_style)\n {\n \tswitch (quote_style) {\n \tcase QUOTE_NONE:\n-\t\tstrbuf_addstr(s, str);\n+\t\tif (len < 0)\n+\t\t\tstrbuf_addstr(s, str);\n+\t\telse\n+\t\t\tstrbuf_add(s, str, len);\n \t\tbreak;\n \tcase QUOTE_SHELL:\n \t\tsq_quote_buf(s, str);\n@@ -741,9 +768,11 @@ static int append_atom(struct atom_value *v, struct ref_formatting_state *state,\n \t * encountered.\n \t */\n \tif (!state->stack->prev)\n-\t\tquote_formatting(&state->stack->output, v->s, state->quote_style);\n-\telse\n+\t\tquote_formatting(&state->stack->output, v->s, v->s_size, state->quote_style);\n+\telse if (v->s_size < 0)\n \t\tstrbuf_addstr(&state->stack->output, v->s);\n+\telse\n+\t\tstrbuf_add(&state->stack->output, v->s, v->s_size);\n \treturn 0;\n }\n \n@@ -843,21 +872,23 @@ static int if_atom_handler(struct atom_value *atomv, struct ref_formatting_state\n \treturn 0;\n }\n \n-static int is_empty(const char *s)\n+static int is_empty(struct strbuf *buf)\n {\n-\twhile (*s != '\\0') {\n-\t\tif (!isspace(*s))\n-\t\t\treturn 0;\n-\t\ts++;\n-\t}\n-\treturn 1;\n-}\n+\tconst char *cur = buf->buf;\n+\tconst char *end = buf->buf + buf->len;\n+\n+\twhile (cur != end && (isspace(*cur)))\n+\t\tcur++;\n+\n+\treturn cur == end;\n+ }\n \n static int then_atom_handler(struct atom_value *atomv, struct ref_formatting_state *state,\n \t\t\t struct strbuf *err)\n {\n \tstruct ref_formatting_stack *cur = state->stack;\n \tstruct if_then_else *if_then_else = NULL;\n+\tsize_t str_len = 0;\n \n \tif (cur->at_end == if_then_else_handler)\n \t\tif_then_else = (struct if_then_else *)cur->at_end_data;\n@@ -868,18 +899,22 @@ static int then_atom_handler(struct atom_value *atomv, struct ref_formatting_sta\n \tif (if_then_else->else_atom_seen)\n \t\treturn strbuf_addf_ret(err, -1, _(\"format: %%(then) atom used after %%(else)\"));\n \tif_then_else->then_atom_seen = 1;\n+\tif (if_then_else->str)\n+\t\tstr_len = strlen(if_then_else->str);\n \t/*\n \t * If the 'equals' or 'notequals' attribute is used then\n \t * perform the required comparison. If not, only non-empty\n \t * strings satisfy the 'if' condition.\n \t */\n \tif (if_then_else->cmp_status == COMPARE_EQUAL) {\n-\t\tif (!strcmp(if_then_else->str, cur->output.buf))\n+\t\tif (str_len == cur->output.len &&\n+\t\t !memcmp(if_then_else->str, cur->output.buf, cur->output.len))\n \t\t\tif_then_else->condition_satisfied = 1;\n \t} else if (if_then_else->cmp_status == COMPARE_UNEQUAL) {\n-\t\tif (strcmp(if_then_else->str, cur->output.buf))\n+\t\tif (str_len != cur->output.len ||\n+\t\t memcmp(if_then_else->str, cur->output.buf, cur->output.len))\n \t\t\tif_then_else->condition_satisfied = 1;\n-\t} else if (cur->output.len && !is_empty(cur->output.buf))\n+\t} else if (cur->output.len && !is_empty(&cur->output))\n \t\tif_then_else->condition_satisfied = 1;\n \tstrbuf_reset(&cur->output);\n \treturn 0;\n@@ -925,7 +960,7 @@ static int end_atom_handler(struct atom_value *atomv, struct ref_formatting_stat\n \t * only on the topmost supporting atom.\n \t */\n \tif (!current->prev->prev) {\n-\t\tquote_formatting(&s, current->output.buf, state->quote_style);\n+\t\tquote_formatting(&s, current->output.buf, current->output.len, state->quote_style);\n \t\tstrbuf_swap(¤t->output, &s);\n \t}\n \tstrbuf_release(&s);\n@@ -975,6 +1010,10 @@ int verify_ref_format(struct ref_format *format)\n \t\tat = parse_ref_filter_atom(format, sp + 2, ep, &err);\n \t\tif (at < 0)\n \t\t\tdie(\"%s\", err.buf);\n+\t\tif (format->quote_style && used_atom[at].atom_type == ATOM_RAW &&\n+\t\t used_atom[at].u.raw_data.option == RAW_BARE)\n+\t\t\tdie(_(\"--format=%.*s cannot be used with\"\n+\t\t\t \"--python, --shell, --tcl, --perl\"), (int)(ep - sp - 2), sp + 2);\n \t\tcp = ep + 1;\n \n \t\tif (skip_prefix(used_atom[at].name, \"color:\", &color))\n@@ -1368,12 +1407,25 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp\n \t\tstruct used_atom *atom = &used_atom[i];\n \t\tconst char *name = atom->name;\n \t\tstruct atom_value *v = &val[i];\n+\t\tenum atom_type atom_type = atom->atom_type;\n \n \t\tif (!!deref != (*name == '*'))\n \t\t\tcontinue;\n \t\tif (deref)\n \t\t\tname++;\n \n+\t\tif (atom_type == ATOM_RAW) {\n+\t\t\tunsigned long buf_size = data->size;\n+\n+\t\t\tif (atom->u.raw_data.option == RAW_BARE) {\n+\t\t\t\tv->s = xmemdupz(buf, buf_size);\n+\t\t\t\tv->s_size = buf_size;\n+\t\t\t} else if (atom->u.raw_data.option == RAW_LENGTH) {\n+\t\t\t\tv->s = xstrfmt(\"%\"PRIuMAX, (uintmax_t)buf_size);\n+\t\t\t}\n+\t\t\tcontinue;\n+\t\t}\n+\n \t\tif ((data->type != OBJ_TAG &&\n \t\t data->type != OBJ_COMMIT) ||\n \t\t (strcmp(name, \"body\") &&\n@@ -1461,9 +1513,11 @@ static void grab_values(struct atom_value *val, int deref, struct object *obj, s\n \t\tbreak;\n \tcase OBJ_TREE:\n \t\t/* grab_tree_values(val, deref, obj, buf, sz); */\n+\t\tgrab_sub_body_contents(val, deref, data);\n \t\tbreak;\n \tcase OBJ_BLOB:\n \t\t/* grab_blob_values(val, deref, obj, buf, sz); */\n+\t\tgrab_sub_body_contents(val, deref, data);\n \t\tbreak;\n \tdefault:\n \t\tdie(\"Eh? Object of type %d?\", obj->type);\n@@ -1767,6 +1821,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\tconst char *refname;\n \t\tstruct branch *branch = NULL;\n \n+\t\tv->s_size = ATOM_SIZE_UNSPECIFIED;\n \t\tv->handler = append_atom;\n \t\tv->atom = atom;\n \n@@ -2374,6 +2429,19 @@ static int compare_detached_head(struct ref_array_item *a, struct ref_array_item\n \treturn 0;\n }\n \n+static int memcasecmp(const void *vs1, const void *vs2, size_t n)\n+{\n+\tconst char *s1 = vs1, *s2 = vs2;\n+\tconst char *end = s1 + n;\n+\n+\tfor (; s1 < end; s1++, s2++) {\n+\t\tint diff = tolower(*s1) - tolower(*s2);\n+\t\tif (diff)\n+\t\t\treturn diff;\n+\t}\n+\treturn 0;\n+}\n+\n static int cmp_ref_sorting(struct ref_sorting *s, struct ref_array_item *a, struct ref_array_item *b)\n {\n \tstruct atom_value *va, *vb;\n@@ -2394,10 +2462,29 @@ static int cmp_ref_sorting(struct ref_sorting *s, struct ref_array_item *a, stru\n \t} else if (s->sort_flags & REF_SORTING_VERSION) {\n \t\tcmp = versioncmp(va->s, vb->s);\n \t} else if (cmp_type == FIELD_STR) {\n-\t\tint (*cmp_fn)(const char *, const char *);\n-\t\tcmp_fn = s->sort_flags & REF_SORTING_ICASE\n-\t\t\t? strcasecmp : strcmp;\n-\t\tcmp = cmp_fn(va->s, vb->s);\n+\t\tif (va->s_size < 0 && vb->s_size < 0) {\n+\t\t\tint (*cmp_fn)(const char *, const char *);\n+\t\t\tcmp_fn = s->sort_flags & REF_SORTING_ICASE\n+\t\t\t\t? strcasecmp : strcmp;\n+\t\t\tcmp = cmp_fn(va->s, vb->s);\n+\t\t} else {\n+\t\t\tsize_t a_size = va->s_size < 0 ?\n+\t\t\t\t\tstrlen(va->s) : va->s_size;\n+\t\t\tsize_t b_size = vb->s_size < 0 ?\n+\t\t\t\t\tstrlen(vb->s) : vb->s_size;\n+\t\t\tint (*cmp_fn)(const void *, const void *, size_t);\n+\t\t\tcmp_fn = s->sort_flags & REF_SORTING_ICASE\n+\t\t\t\t? memcasecmp : memcmp;\n+\n+\t\t\tcmp = cmp_fn(va->s, vb->s, b_size > a_size ?\n+\t\t\t\t a_size : b_size);\n+\t\t\tif (!cmp) {\n+\t\t\t\tif (a_size > b_size)\n+\t\t\t\t\tcmp = 1;\n+\t\t\t\telse if (a_size < b_size)\n+\t\t\t\t\tcmp = -1;\n+\t\t\t}\n+\t\t}\n \t} else {\n \t\tif (va->value < vb->value)\n \t\t\tcmp = -1;\n@@ -2496,7 +2583,7 @@ int format_ref_array_item(struct ref_array_item *info,\n \t\tappend_literal(cp, sp, &state);\n \t}\n \tif (format->need_color_reset_at_eol) {\n-\t\tstruct atom_value resetv;\n+\t\tstruct atom_value resetv = ATOM_VALUE_INIT;\n \t\tresetv.s = GIT_COLOR_RESET;\n \t\tif (append_atom(&resetv, &state, error_buf)) {\n \t\t\tpop_stack_element(&state.stack);\ndiff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh\nindex 9e0214076b4..18554f62d94 100755\n--- a/t/t6300-for-each-ref.sh\n+++ b/t/t6300-for-each-ref.sh\n@@ -130,6 +130,8 @@ test_atom head parent:short=10 ''\n test_atom head numparent 0\n test_atom head object ''\n test_atom head type ''\n+test_atom head raw \"$(git cat-file commit refs/heads/main)\n+\"\n test_atom head '*objectname' ''\n test_atom head '*objecttype' ''\n test_atom head author 'A U Thor 1151968724 +0200'\n@@ -221,6 +223,15 @@ test_atom tag contents 'Tagging at 1151968727\n '\n test_atom tag HEAD ' '\n \n+test_expect_success 'basic atom: refs/tags/testtag *raw' '\n+\tgit cat-file commit refs/tags/testtag^{} >expected &&\n+\tgit for-each-ref --format=\"%(*raw)\" refs/tags/testtag >actual &&\n+\tsanitize_pgp expected.clean &&\n+\techo >>expected.clean &&\n+\tsanitize_pgp actual.clean &&\n+\ttest_cmp expected.clean actual.clean\n+'\n+\n test_expect_success 'Check invalid atoms names are errors' '\n \ttest_must_fail git for-each-ref --format=\"%(INVALID)\" refs/heads\n '\n@@ -686,6 +697,15 @@ test_atom refs/tags/signed-empty contents:body ''\n test_atom refs/tags/signed-empty contents:signature \"$sig\"\n test_atom refs/tags/signed-empty contents \"$sig\"\n \n+test_expect_success GPG 'basic atom: refs/tags/signed-empty raw' '\n+\tgit cat-file tag refs/tags/signed-empty >expected &&\n+\tgit for-each-ref --format=\"%(raw)\" refs/tags/signed-empty >actual &&\n+\tsanitize_pgp expected.clean &&\n+\techo >>expected.clean &&\n+\tsanitize_pgp actual.clean &&\n+\ttest_cmp expected.clean actual.clean\n+'\n+\n test_atom refs/tags/signed-short subject 'subject line'\n test_atom refs/tags/signed-short subject:sanitize 'subject-line'\n test_atom refs/tags/signed-short contents:subject 'subject line'\n@@ -695,6 +715,15 @@ test_atom refs/tags/signed-short contents:signature \"$sig\"\n test_atom refs/tags/signed-short contents \"subject line\n $sig\"\n \n+test_expect_success GPG 'basic atom: refs/tags/signed-short raw' '\n+\tgit cat-file tag refs/tags/signed-short >expected &&\n+\tgit for-each-ref --format=\"%(raw)\" refs/tags/signed-short >actual &&\n+\tsanitize_pgp expected.clean &&\n+\techo >>expected.clean &&\n+\tsanitize_pgp actual.clean &&\n+\ttest_cmp expected.clean actual.clean\n+'\n+\n test_atom refs/tags/signed-long subject 'subject line'\n test_atom refs/tags/signed-long subject:sanitize 'subject-line'\n test_atom refs/tags/signed-long contents:subject 'subject line'\n@@ -708,6 +737,15 @@ test_atom refs/tags/signed-long contents \"subject line\n body contents\n $sig\"\n \n+test_expect_success GPG 'basic atom: refs/tags/signed-long raw' '\n+\tgit cat-file tag refs/tags/signed-long >expected &&\n+\tgit for-each-ref --format=\"%(raw)\" refs/tags/signed-long >actual &&\n+\tsanitize_pgp expected.clean &&\n+\techo >>expected.clean &&\n+\tsanitize_pgp actual.clean &&\n+\ttest_cmp expected.clean actual.clean\n+'\n+\n test_expect_success 'set up refs pointing to tree and blob' '\n \tgit update-ref refs/mytrees/first refs/heads/main^{tree} &&\n \tgit update-ref refs/myblobs/first refs/heads/main:one\n@@ -720,6 +758,16 @@ test_atom refs/mytrees/first contents:body \"\"\n test_atom refs/mytrees/first contents:signature \"\"\n test_atom refs/mytrees/first contents \"\"\n \n+test_expect_success 'basic atom: refs/mytrees/first raw' '\n+\tgit cat-file tree refs/mytrees/first >expected &&\n+\techo >>expected &&\n+\tgit for-each-ref --format=\"%(raw)\" refs/mytrees/first >actual &&\n+\ttest_cmp expected actual &&\n+\tgit cat-file -s refs/mytrees/first >expected &&\n+\tgit for-each-ref --format=\"%(raw:size)\" refs/mytrees/first >actual &&\n+\ttest_cmp expected actual\n+'\n+\n test_atom refs/myblobs/first subject \"\"\n test_atom refs/myblobs/first contents:subject \"\"\n test_atom refs/myblobs/first body \"\"\n@@ -727,6 +775,174 @@ test_atom refs/myblobs/first contents:body \"\"\n test_atom refs/myblobs/first contents:signature \"\"\n test_atom refs/myblobs/first contents \"\"\n \n+test_expect_success 'basic atom: refs/myblobs/first raw' '\n+\tgit cat-file blob refs/myblobs/first >expected &&\n+\techo >>expected &&\n+\tgit for-each-ref --format=\"%(raw)\" refs/myblobs/first >actual &&\n+\ttest_cmp expected actual &&\n+\tgit cat-file -s refs/myblobs/first >expected &&\n+\tgit for-each-ref --format=\"%(raw:size)\" refs/myblobs/first >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'set up refs pointing to binary blob' '\n+\tprintf \"a\\0b\\0c\" >blob1 &&\n+\tprintf \"a\\0c\\0b\" >blob2 &&\n+\tprintf \"\\0a\\0b\\0c\" >blob3 &&\n+\tprintf \"abc\" >blob4 &&\n+\tprintf \"\\0 \\0 \\0 \" >blob5 &&\n+\tprintf \"\\0 \\0a\\0 \" >blob6 &&\n+\tprintf \" \" >blob7 &&\n+\t>blob8 &&\n+\tobj=$(git hash-object -w blob1) &&\n+\tgit update-ref refs/myblobs/blob1 \"$obj\" &&\n+\tobj=$(git hash-object -w blob2) &&\n+\tgit update-ref refs/myblobs/blob2 \"$obj\" &&\n+\tobj=$(git hash-object -w blob3) &&\n+\tgit update-ref refs/myblobs/blob3 \"$obj\" &&\n+\tobj=$(git hash-object -w blob4) &&\n+\tgit update-ref refs/myblobs/blob4 \"$obj\" &&\n+\tobj=$(git hash-object -w blob5) &&\n+\tgit update-ref refs/myblobs/blob5 \"$obj\" &&\n+\tobj=$(git hash-object -w blob6) &&\n+\tgit update-ref refs/myblobs/blob6 \"$obj\" &&\n+\tobj=$(git hash-object -w blob7) &&\n+\tgit update-ref refs/myblobs/blob7 \"$obj\" &&\n+\tobj=$(git hash-object -w blob8) &&\n+\tgit update-ref refs/myblobs/blob8 \"$obj\"\n+'\n+\n+test_expect_success 'Verify sorts with raw' '\n+\tcat >expected <<-EOF &&\n+\trefs/myblobs/blob8\n+\trefs/myblobs/blob5\n+\trefs/myblobs/blob6\n+\trefs/myblobs/blob3\n+\trefs/myblobs/blob7\n+\trefs/mytrees/first\n+\trefs/myblobs/first\n+\trefs/myblobs/blob1\n+\trefs/myblobs/blob2\n+\trefs/myblobs/blob4\n+\trefs/heads/main\n+\tEOF\n+\tgit for-each-ref --format=\"%(refname)\" --sort=raw \\\n+\t\trefs/heads/main refs/myblobs/ refs/mytrees/first >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'Verify sorts with raw:size' '\n+\tcat >expected <<-EOF &&\n+\trefs/myblobs/blob8\n+\trefs/myblobs/first\n+\trefs/myblobs/blob7\n+\trefs/heads/main\n+\trefs/myblobs/blob4\n+\trefs/myblobs/blob1\n+\trefs/myblobs/blob2\n+\trefs/myblobs/blob3\n+\trefs/myblobs/blob5\n+\trefs/myblobs/blob6\n+\trefs/mytrees/first\n+\tEOF\n+\tgit for-each-ref --format=\"%(refname)\" --sort=raw:size \\\n+\t\trefs/heads/main refs/myblobs/ refs/mytrees/first >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'validate raw atom with %(if:equals)' '\n+\tcat >expected <<-EOF &&\n+\tnot equals\n+\tnot equals\n+\tnot equals\n+\tnot equals\n+\tnot equals\n+\tnot equals\n+\trefs/myblobs/blob4\n+\tnot equals\n+\tnot equals\n+\tnot equals\n+\tnot equals\n+\tnot equals\n+\tEOF\n+\tgit for-each-ref --format=\"%(if:equals=abc)%(raw)%(then)%(refname)%(else)not equals%(end)\" \\\n+\t\trefs/myblobs/ refs/heads/ >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'validate raw atom with %(if:notequals)' '\n+\tcat >expected <<-EOF &&\n+\trefs/heads/ambiguous\n+\trefs/heads/main\n+\trefs/heads/newtag\n+\trefs/myblobs/blob1\n+\trefs/myblobs/blob2\n+\trefs/myblobs/blob3\n+\tequals\n+\trefs/myblobs/blob5\n+\trefs/myblobs/blob6\n+\trefs/myblobs/blob7\n+\trefs/myblobs/blob8\n+\trefs/myblobs/first\n+\tEOF\n+\tgit for-each-ref --format=\"%(if:notequals=abc)%(raw)%(then)%(refname)%(else)equals%(end)\" \\\n+\t\trefs/myblobs/ refs/heads/ >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'empty raw refs with %(if)' '\n+\tcat >expected <<-EOF &&\n+\trefs/myblobs/blob1 not empty\n+\trefs/myblobs/blob2 not empty\n+\trefs/myblobs/blob3 not empty\n+\trefs/myblobs/blob4 not empty\n+\trefs/myblobs/blob5 not empty\n+\trefs/myblobs/blob6 not empty\n+\trefs/myblobs/blob7 empty\n+\trefs/myblobs/blob8 empty\n+\trefs/myblobs/first not empty\n+\tEOF\n+\tgit for-each-ref --format=\"%(refname) %(if)%(raw)%(then)not empty%(else)empty%(end)\" \\\n+\t\trefs/myblobs/ >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success '%(raw) with --python must fail' '\n+\ttest_must_fail git for-each-ref --format=\"%(raw)\" --python\n+'\n+\n+test_expect_success '%(raw) with --tcl must fail' '\n+\ttest_must_fail git for-each-ref --format=\"%(raw)\" --tcl\n+'\n+\n+test_expect_success '%(raw) with --perl must fail' '\n+\ttest_must_fail git for-each-ref --format=\"%(raw)\" --perl\n+'\n+\n+test_expect_success '%(raw) with --shell must fail' '\n+\ttest_must_fail git for-each-ref --format=\"%(raw)\" --shell\n+'\n+\n+test_expect_success '%(raw) with --shell and --sort=raw must fail' '\n+\ttest_must_fail git for-each-ref --format=\"%(raw)\" --sort=raw --shell\n+'\n+\n+test_expect_success '%(raw:size) with --shell' '\n+\tgit for-each-ref --format=\"%(raw:size)\" | while read line\n+\tdo\n+\t\techo \"'\\''$line'\\''\" >>expect\n+\tdone &&\n+\tgit for-each-ref --format=\"%(raw:size)\" --shell >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'for-each-ref --format compare with cat-file --batch' '\n+\tgit rev-parse refs/mytrees/first | git cat-file --batch >expected &&\n+\tgit for-each-ref --format=\"%(objectname) %(objecttype) %(objectsize)\n+%(raw)\" refs/mytrees/first >actual &&\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'set up multiple-sort tags' '\n \tfor when in 100000 200000\n \tdo\n-- \ngitgitgadget\n\n"},{"id":"432635","messageId":"pull.1016.git.1628842990.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":null,"subject":"[PATCH 00/27] [GSOC] [RFC] cat-file: reuse ref-filter logic","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:43Z","receivedAt":"2021-08-13T08:23:20Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"This patch series makes cat-file reuse ref-filter logic. At the same time,\nsome performance optimizations have been carried out. It's last version is\nhere:\nhttps://lore.kernel.org/git/pull.993.v2.git.1626363626.gitgitgadget@gmail.com/#t\n\nIt seems that zh/ref-filter-raw-data is still hovering in the next branch\n(Because git is rc2) So I now want to show some recent performance\noptimizations first.\n\nChange from last version:\n\n 1. Use free_global_resource() to avoid memory leaks.\n 2. Skip parse_object_buffer() which bring 12.5% performance optimization.\n 3. Merge two for loop in grab_person() which bring 2% performance\n optimization.\n 4. Remove strlen from find_subpos.\n 5. Introducing xstrvfmt_len() and xstrfmt_len().\n 6. Remove second parsing in format_ref_array_item() which bring 1.9%\n performance optimization\n 7. Introduction ref_filter_slopbuf to instread xstrdup(\"\").\n 8. Add deref member to struct used_atom to simplify the logic of the\n program.\n 9. Introduce symref_atom_parser() to make the program logic more concise.\n 10. Use switch/case instread of if/else to increase the readability of the\n code.\n 11. Reuse finnal buffer which bring 2% performance optimization.\n 12. Add need_get_object_info flag to reduce memory comparing.\n\nThis is the result of the performance test after I did some optimization:\n\nTest upstream/master this tree\n------------------------------------------------------------------------------------\n1006.2: cat-file --batch-check 0.08(0.07+0.00) 0.09(0.08+0.01) +12.5%\n1006.3: cat-file --batch-check with atoms 0.06(0.04+0.02) 0.08(0.06+0.02) +33.3%\n1006.4: cat-file --batch 0.49(0.46+0.02) 0.50(0.47+0.03) +2.0%\n1006.5: cat-file --batch with atoms 0.47(0.45+0.01) 0.49(0.47+0.02) +4.3%\n\n\nWe can see that the performance of the current patch of git cat-file --batch\nis very close to upstream/master. The optimization of git cat-file\n--batch-check does not seem obvious, because its optimization degree will be\naffected by noise, which may appear in the range of +12.5% to +50.0%. From\nan optimistic point of view, the execution time of git cat-file\n--batch-check itself is relatively short, the optimization is of course not\nobvious.\n\nAs GSOC is about to end, this patch series is estimated to be adjusted for\nsome time, I can only wish this patch can be accepted in the future.\n\nNote: The previous part of this patch series is the duplicate content\nbelonging to zh/ref-filter-raw-data.\n\nZheNing Hu (27):\n [GSOC] ref-filter: add obj-type check in grab contents\n [GSOC] ref-filter: add %(raw) atom\n [GSOC] ref-filter: --format=%(raw) support --perl\n [GSOC] ref-filter: use non-const ref_format in *_atom_parser()\n [GSOC] ref-filter: add %(rest) atom\n [GSOC] ref-filter: pass get_object() return value to their callers\n [GSOC] ref-filter: introduce free_ref_array_item_value() function\n [GSOC] ref-filter: add cat_file_mode to ref_format\n [GSOC] ref-filter: modify the error message and value in get_object\n [GSOC] cat-file: add has_object_file() check\n [GSOC] cat-file: change batch_objects parameter name\n [GSOC] cat-file: create p1006-cat-file.sh\n [GSOC] cat-file: reuse ref-filter logic\n [GSOC] cat-file: reuse err buf in batch_object_write()\n [GSOC] cat-file: re-implement --textconv, --filters options\n [GSOC] ref-filter: remove grab_oid() function\n [GSOC] ref-filter: performance optimization by skip\n parse_object_buffer\n [GSOC] ref-filter: use atom_type and merge two for loop in grab_person\n [GSOC] ref-filter: remove strlen from find_subpos\n [GSOC] ref-filter: introducing xstrvfmt_len() and xstrfmt_len()\n [GSOC] ref-filter: remove second parsing in format_ref_array_item\n [GSOC] ref-filter: introduction ref_filter_slopbuf\n [GSOC] ref-filter: add deref member to struct used_atom\n [GSOC] ref-filter: introduce symref_atom_parser()\n [GSOC] ref-filter: use switch case instread of if else\n [GSOC] ref-filter: reuse finnal buffer if no stack need\n [GSOC] ref-filter: add need_get_object_info flag to struct expand_data\n\n Documentation/git-cat-file.txt | 6 +\n Documentation/git-for-each-ref.txt | 9 +\n builtin/branch.c | 2 +\n builtin/cat-file.c | 275 +++------\n builtin/for-each-ref.c | 3 +-\n builtin/tag.c | 4 +-\n builtin/verify-tag.c | 2 +\n quote.c | 17 +\n quote.h | 1 +\n ref-filter.c | 902 +++++++++++++++++++----------\n ref-filter.h | 30 +-\n strbuf.c | 21 +\n strbuf.h | 6 +\n t/perf/p1006-cat-file.sh | 28 +\n t/t1006-cat-file.sh | 239 ++++++++\n t/t3203-branch-output.sh | 4 +\n t/t6300-for-each-ref.sh | 235 ++++++++\n t/t6301-for-each-ref-errors.sh | 2 +-\n t/t7004-tag.sh | 4 +\n t/t7030-verify-tag.sh | 4 +\n 20 files changed, 1283 insertions(+), 511 deletions(-)\n create mode 100755 t/perf/p1006-cat-file.sh\n\n\nbase-commit: 5d213e46bb7b880238ff5ea3914e940a50ae9369\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1016%2Fadlternative%2Fcat-file-reuse-ref-filter-logic-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1016/adlternative/cat-file-reuse-ref-filter-logic-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/1016\n-- \ngitgitgadget\n"},{"id":"432634","messageId":"aea5c26cc403b34401fcdb87e3947b8cc37787ea.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 01/27] [GSOC] ref-filter: add obj-type check in grab contents","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:44Z","receivedAt":"2021-08-13T08:23:22Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nOnly tag and commit objects use `grab_sub_body_contents()` to grab\nobject contents in the current codebase. We want to teach the\nfunction to also handle blobs and trees to get their raw data,\nwithout parsing a blob (whose contents looks like a commit or a tag)\nincorrectly as a commit or a tag. So it's needed to pass a\n`struct expand_data *data` instread of only `void *buf` to both\n`grab_sub_body_contents()` and `grab_values()` to be able to check\nthe object type.\n\nSkip the block of code that is specific to handling commits and tags\nearly when the given object is of a wrong type to help later\naddition to handle other types of objects in this function.\n\nReviewed-by: Jacob Keller \nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nHelped-by: Junio C Hamano \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 24 +++++++++++++++---------\n 1 file changed, 15 insertions(+), 9 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 0cfef7b719b..e3715210e5c 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1357,11 +1357,12 @@ static void append_lines(struct strbuf *out, const char *buf, unsigned long size\n }\n \n /* See grab_values */\n-static void grab_sub_body_contents(struct atom_value *val, int deref, void *buf)\n+static void grab_sub_body_contents(struct atom_value *val, int deref, struct expand_data *data)\n {\n \tint i;\n \tconst char *subpos = NULL, *bodypos = NULL, *sigpos = NULL;\n \tsize_t sublen = 0, bodylen = 0, nonsiglen = 0, siglen = 0;\n+\tvoid *buf = data->content;\n \n \tfor (i = 0; i < used_atom_cnt; i++) {\n \t\tstruct used_atom *atom = &used_atom[i];\n@@ -1372,10 +1373,13 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, void *buf)\n \t\t\tcontinue;\n \t\tif (deref)\n \t\t\tname++;\n-\t\tif (strcmp(name, \"body\") &&\n-\t\t !starts_with(name, \"subject\") &&\n-\t\t !starts_with(name, \"trailers\") &&\n-\t\t !starts_with(name, \"contents\"))\n+\n+\t\tif ((data->type != OBJ_TAG &&\n+\t\t data->type != OBJ_COMMIT) ||\n+\t\t (strcmp(name, \"body\") &&\n+\t\t !starts_with(name, \"subject\") &&\n+\t\t !starts_with(name, \"trailers\") &&\n+\t\t !starts_with(name, \"contents\")))\n \t\t\tcontinue;\n \t\tif (!subpos)\n \t\t\tfind_subpos(buf,\n@@ -1439,17 +1443,19 @@ static void fill_missing_values(struct atom_value *val)\n * pointed at by the ref itself; otherwise it is the object the\n * ref (which is a tag) refers to.\n */\n-static void grab_values(struct atom_value *val, int deref, struct object *obj, void *buf)\n+static void grab_values(struct atom_value *val, int deref, struct object *obj, struct expand_data *data)\n {\n+\tvoid *buf = data->content;\n+\n \tswitch (obj->type) {\n \tcase OBJ_TAG:\n \t\tgrab_tag_values(val, deref, obj);\n-\t\tgrab_sub_body_contents(val, deref, buf);\n+\t\tgrab_sub_body_contents(val, deref, data);\n \t\tgrab_person(\"tagger\", val, deref, buf);\n \t\tbreak;\n \tcase OBJ_COMMIT:\n \t\tgrab_commit_values(val, deref, obj);\n-\t\tgrab_sub_body_contents(val, deref, buf);\n+\t\tgrab_sub_body_contents(val, deref, data);\n \t\tgrab_person(\"author\", val, deref, buf);\n \t\tgrab_person(\"committer\", val, deref, buf);\n \t\tbreak;\n@@ -1679,7 +1685,7 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj\n \t\t\treturn strbuf_addf_ret(err, -1, _(\"parse_object_buffer failed on %s for %s\"),\n \t\t\t\t\t oid_to_hex(&oi->oid), ref->refname);\n \t\t}\n-\t\tgrab_values(ref->value, deref, *obj, oi->content);\n+\t\tgrab_values(ref->value, deref, *obj, oi);\n \t}\n \n \tgrab_common_values(ref->value, deref, oi);\n-- \ngitgitgadget\n\n"},{"id":"432636","messageId":"452ea887e6123bc74b1a2f563ab0ea7c650867b9.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 03/27] [GSOC] ref-filter: --format=%(raw) support --perl","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:46Z","receivedAt":"2021-08-13T08:23:24Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nBecause the perl language can handle binary data correctly,\nadd the function perl_quote_buf_with_len(), which can specify\nthe length of the data and prevent the data from being truncated\nat '\\0' to help `--format=\"%(raw)\"` support `--perl`.\n\nReviewed-by: Jacob Keller \nHelped-by: Ævar Arnfjörð Bjarmason \nSigned-off-by: ZheNing Hu \n---\n Documentation/git-for-each-ref.txt | 4 ++--\n quote.c | 17 +++++++++++++++++\n quote.h | 1 +\n ref-filter.c | 15 +++++++++++----\n t/t6300-for-each-ref.sh | 19 +++++++++++++++++--\n 5 files changed, 48 insertions(+), 8 deletions(-)\n\ndiff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt\nindex cbb6f87d13f..6da899c6296 100644\n--- a/Documentation/git-for-each-ref.txt\n+++ b/Documentation/git-for-each-ref.txt\n@@ -241,8 +241,8 @@ raw:size::\n \tThe raw data size of the object.\n \n Note that `--format=%(raw)` can not be used with `--python`, `--shell`, `--tcl`,\n-`--perl` because such language may not support arbitrary binary data in their\n-string variable type.\n+because such language may not support arbitrary binary data in their string\n+variable type.\n \n The message in a commit or a tag object is `contents`, from which\n `contents:` can be used to extract various parts out of:\ndiff --git a/quote.c b/quote.c\nindex 8a3a5e39eb1..26719d21d1e 100644\n--- a/quote.c\n+++ b/quote.c\n@@ -471,6 +471,23 @@ void perl_quote_buf(struct strbuf *sb, const char *src)\n \tstrbuf_addch(sb, sq);\n }\n \n+void perl_quote_buf_with_len(struct strbuf *sb, const char *src, size_t len)\n+{\n+\tconst char sq = '\\'';\n+\tconst char bq = '\\\\';\n+\tconst char *c = src;\n+\tconst char *end = src + len;\n+\n+\tstrbuf_addch(sb, sq);\n+\twhile (c != end) {\n+\t\tif (*c == sq || *c == bq)\n+\t\t\tstrbuf_addch(sb, bq);\n+\t\tstrbuf_addch(sb, *c);\n+\t\tc++;\n+\t}\n+\tstrbuf_addch(sb, sq);\n+}\n+\n void python_quote_buf(struct strbuf *sb, const char *src)\n {\n \tconst char sq = '\\'';\ndiff --git a/quote.h b/quote.h\nindex 049d8dd0b3d..87ff458b06d 100644\n--- a/quote.h\n+++ b/quote.h\n@@ -95,6 +95,7 @@ char *quote_path(const char *in, const char *prefix, struct strbuf *out, unsigne\n \n /* quoting as a string literal for other languages */\n void perl_quote_buf(struct strbuf *sb, const char *src);\n+void perl_quote_buf_with_len(struct strbuf *sb, const char *src, size_t len);\n void python_quote_buf(struct strbuf *sb, const char *src);\n void tcl_quote_buf(struct strbuf *sb, const char *src);\n void basic_regex_quote_buf(struct strbuf *sb, const char *src);\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 1e660f60c4b..f2454d34ce8 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -747,7 +747,10 @@ static void quote_formatting(struct strbuf *s, const char *str, ssize_t len, int\n \t\tsq_quote_buf(s, str);\n \t\tbreak;\n \tcase QUOTE_PERL:\n-\t\tperl_quote_buf(s, str);\n+\t\tif (len < 0)\n+\t\t\tperl_quote_buf(s, str);\n+\t\telse\n+\t\t\tperl_quote_buf_with_len(s, str, len);\n \t\tbreak;\n \tcase QUOTE_PYTHON:\n \t\tpython_quote_buf(s, str);\n@@ -1010,10 +1013,14 @@ int verify_ref_format(struct ref_format *format)\n \t\tat = parse_ref_filter_atom(format, sp + 2, ep, &err);\n \t\tif (at < 0)\n \t\t\tdie(\"%s\", err.buf);\n-\t\tif (format->quote_style && used_atom[at].atom_type == ATOM_RAW &&\n-\t\t used_atom[at].u.raw_data.option == RAW_BARE)\n+\n+\t\tif ((format->quote_style == QUOTE_PYTHON ||\n+\t\t format->quote_style == QUOTE_SHELL ||\n+\t\t format->quote_style == QUOTE_TCL) &&\n+\t\t used_atom[at].atom_type == ATOM_RAW &&\n+\t\t used_atom[at].u.raw_data.option == RAW_BARE)\n \t\t\tdie(_(\"--format=%.*s cannot be used with\"\n-\t\t\t \"--python, --shell, --tcl, --perl\"), (int)(ep - sp - 2), sp + 2);\n+\t\t\t \"--python, --shell, --tcl\"), (int)(ep - sp - 2), sp + 2);\n \t\tcp = ep + 1;\n \n \t\tif (skip_prefix(used_atom[at].name, \"color:\", &color))\ndiff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh\nindex 18554f62d94..3d15d0a5360 100755\n--- a/t/t6300-for-each-ref.sh\n+++ b/t/t6300-for-each-ref.sh\n@@ -915,8 +915,23 @@ test_expect_success '%(raw) with --tcl must fail' '\n \ttest_must_fail git for-each-ref --format=\"%(raw)\" --tcl\n '\n \n-test_expect_success '%(raw) with --perl must fail' '\n-\ttest_must_fail git for-each-ref --format=\"%(raw)\" --perl\n+test_expect_success '%(raw) with --perl' '\n+\tgit for-each-ref --format=\"\\$name= %(raw);\n+print \\\"\\$name\\\"\" refs/myblobs/blob1 --perl | perl >actual &&\n+\tcmp blob1 actual &&\n+\tgit for-each-ref --format=\"\\$name= %(raw);\n+print \\\"\\$name\\\"\" refs/myblobs/blob3 --perl | perl >actual &&\n+\tcmp blob3 actual &&\n+\tgit for-each-ref --format=\"\\$name= %(raw);\n+print \\\"\\$name\\\"\" refs/myblobs/blob8 --perl | perl >actual &&\n+\tcmp blob8 actual &&\n+\tgit for-each-ref --format=\"\\$name= %(raw);\n+print \\\"\\$name\\\"\" refs/myblobs/first --perl | perl >actual &&\n+\tcmp one actual &&\n+\tgit cat-file tree refs/mytrees/first > expected &&\n+\tgit for-each-ref --format=\"\\$name= %(raw);\n+print \\\"\\$name\\\"\" refs/mytrees/first --perl | perl >actual &&\n+\tcmp expected actual\n '\n \n test_expect_success '%(raw) with --shell must fail' '\n-- \ngitgitgadget\n\n"},{"id":"432637","messageId":"0dc3cf095934e81071189f9c116a98b457f26335.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 04/27] [GSOC] ref-filter: use non-const ref_format in *_atom_parser()","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:47Z","receivedAt":"2021-08-13T08:23:25Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nUse non-const ref_format in *_atom_parser(), which can help us\nmodify the members of ref_format in *_atom_parser().\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n builtin/tag.c | 2 +-\n ref-filter.c | 44 ++++++++++++++++++++++----------------------\n ref-filter.h | 4 ++--\n 3 files changed, 25 insertions(+), 25 deletions(-)\n\ndiff --git a/builtin/tag.c b/builtin/tag.c\nindex 82fcfc09824..452558ec957 100644\n--- a/builtin/tag.c\n+++ b/builtin/tag.c\n@@ -146,7 +146,7 @@ static int verify_tag(const char *name, const char *ref,\n \t\t const struct object_id *oid, void *cb_data)\n {\n \tint flags;\n-\tconst struct ref_format *format = cb_data;\n+\tstruct ref_format *format = cb_data;\n \tflags = GPG_VERIFY_VERBOSE;\n \n \tif (format->format)\ndiff --git a/ref-filter.c b/ref-filter.c\nindex f2454d34ce8..3c101500ed1 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -227,7 +227,7 @@ static int strbuf_addf_ret(struct strbuf *sb, int ret, const char *fmt, ...)\n \treturn ret;\n }\n \n-static int color_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int color_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t const char *color_value, struct strbuf *err)\n {\n \tif (!color_value)\n@@ -265,7 +265,7 @@ static int refname_atom_parser_internal(struct refname_atom *atom, const char *a\n \treturn 0;\n }\n \n-static int remote_ref_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int remote_ref_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t\t const char *arg, struct strbuf *err)\n {\n \tstruct string_list params = STRING_LIST_INIT_DUP;\n@@ -312,7 +312,7 @@ static int remote_ref_atom_parser(const struct ref_format *format, struct used_a\n \treturn 0;\n }\n \n-static int objecttype_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int objecttype_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t\t const char *arg, struct strbuf *err)\n {\n \tif (arg)\n@@ -324,7 +324,7 @@ static int objecttype_atom_parser(const struct ref_format *format, struct used_a\n \treturn 0;\n }\n \n-static int objectsize_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int objectsize_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t\t const char *arg, struct strbuf *err)\n {\n \tif (!arg) {\n@@ -344,7 +344,7 @@ static int objectsize_atom_parser(const struct ref_format *format, struct used_a\n \treturn 0;\n }\n \n-static int deltabase_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int deltabase_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t\t const char *arg, struct strbuf *err)\n {\n \tif (arg)\n@@ -356,7 +356,7 @@ static int deltabase_atom_parser(const struct ref_format *format, struct used_at\n \treturn 0;\n }\n \n-static int body_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int body_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t const char *arg, struct strbuf *err)\n {\n \tif (arg)\n@@ -365,7 +365,7 @@ static int body_atom_parser(const struct ref_format *format, struct used_atom *a\n \treturn 0;\n }\n \n-static int subject_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int subject_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t const char *arg, struct strbuf *err)\n {\n \tif (!arg)\n@@ -377,7 +377,7 @@ static int subject_atom_parser(const struct ref_format *format, struct used_atom\n \treturn 0;\n }\n \n-static int trailers_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int trailers_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t\tconst char *arg, struct strbuf *err)\n {\n \tatom->u.contents.trailer_opts.no_divider = 1;\n@@ -403,7 +403,7 @@ static int trailers_atom_parser(const struct ref_format *format, struct used_ato\n \treturn 0;\n }\n \n-static int contents_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int contents_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t\tconst char *arg, struct strbuf *err)\n {\n \tif (!arg)\n@@ -431,7 +431,7 @@ static int contents_atom_parser(const struct ref_format *format, struct used_ato\n \treturn 0;\n }\n \n-static int raw_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int raw_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t\tconst char *arg, struct strbuf *err)\n {\n \tif (!arg)\n@@ -443,7 +443,7 @@ static int raw_atom_parser(const struct ref_format *format, struct used_atom *at\n \treturn 0;\n }\n \n-static int oid_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int oid_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t const char *arg, struct strbuf *err)\n {\n \tif (!arg)\n@@ -462,7 +462,7 @@ static int oid_atom_parser(const struct ref_format *format, struct used_atom *at\n \treturn 0;\n }\n \n-static int person_email_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int person_email_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t\t const char *arg, struct strbuf *err)\n {\n \tif (!arg)\n@@ -476,7 +476,7 @@ static int person_email_atom_parser(const struct ref_format *format, struct used\n \treturn 0;\n }\n \n-static int refname_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int refname_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t const char *arg, struct strbuf *err)\n {\n \treturn refname_atom_parser_internal(&atom->u.refname, arg, atom->name, err);\n@@ -493,7 +493,7 @@ static align_type parse_align_position(const char *s)\n \treturn -1;\n }\n \n-static int align_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int align_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t const char *arg, struct strbuf *err)\n {\n \tstruct align *align = &atom->u.align;\n@@ -545,7 +545,7 @@ static int align_atom_parser(const struct ref_format *format, struct used_atom *\n \treturn 0;\n }\n \n-static int if_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int if_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t const char *arg, struct strbuf *err)\n {\n \tif (!arg) {\n@@ -560,7 +560,7 @@ static int if_atom_parser(const struct ref_format *format, struct used_atom *ato\n \treturn 0;\n }\n \n-static int head_atom_parser(const struct ref_format *format, struct used_atom *atom,\n+static int head_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t const char *arg, struct strbuf *unused_err)\n {\n \tatom->u.head = resolve_refdup(\"HEAD\", RESOLVE_REF_READING, NULL, NULL);\n@@ -571,7 +571,7 @@ static struct {\n \tconst char *name;\n \tinfo_source source;\n \tcmp_type cmp_type;\n-\tint (*parser)(const struct ref_format *format, struct used_atom *atom,\n+\tint (*parser)(struct ref_format *format, struct used_atom *atom,\n \t\t const char *arg, struct strbuf *err);\n } valid_atom[] = {\n \t[ATOM_REFNAME] = { \"refname\", SOURCE_NONE, FIELD_STR, refname_atom_parser },\n@@ -654,7 +654,7 @@ struct atom_value {\n /*\n * Used to parse format string and sort specifiers\n */\n-static int parse_ref_filter_atom(const struct ref_format *format,\n+static int parse_ref_filter_atom(struct ref_format *format,\n \t\t\t\t const char *atom, const char *ep,\n \t\t\t\t struct strbuf *err)\n {\n@@ -2561,9 +2561,9 @@ static void append_literal(const char *cp, const char *ep, struct ref_formatting\n }\n \n int format_ref_array_item(struct ref_array_item *info,\n-\t\t\t const struct ref_format *format,\n-\t\t\t struct strbuf *final_buf,\n-\t\t\t struct strbuf *error_buf)\n+\t\t\t struct ref_format *format,\n+\t\t\t struct strbuf *final_buf,\n+\t\t\t struct strbuf *error_buf)\n {\n \tconst char *cp, *sp, *ep;\n \tstruct ref_formatting_state state = REF_FORMATTING_STATE_INIT;\n@@ -2607,7 +2607,7 @@ int format_ref_array_item(struct ref_array_item *info,\n }\n \n void pretty_print_ref(const char *name, const struct object_id *oid,\n-\t\t const struct ref_format *format)\n+\t\t struct ref_format *format)\n {\n \tstruct ref_array_item *ref_item;\n \tstruct strbuf output = STRBUF_INIT;\ndiff --git a/ref-filter.h b/ref-filter.h\nindex baf72a71896..74fb423fc89 100644\n--- a/ref-filter.h\n+++ b/ref-filter.h\n@@ -116,7 +116,7 @@ void ref_array_sort(struct ref_sorting *sort, struct ref_array *array);\n void ref_sorting_set_sort_flags_all(struct ref_sorting *sorting, unsigned int mask, int on);\n /* Based on the given format and quote_style, fill the strbuf */\n int format_ref_array_item(struct ref_array_item *info,\n-\t\t\t const struct ref_format *format,\n+\t\t\t struct ref_format *format,\n \t\t\t struct strbuf *final_buf,\n \t\t\t struct strbuf *error_buf);\n /* Parse a single sort specifier and add it to the list */\n@@ -137,7 +137,7 @@ void setup_ref_filter_porcelain_msg(void);\n * name must be a fully qualified refname.\n */\n void pretty_print_ref(const char *name, const struct object_id *oid,\n-\t\t const struct ref_format *format);\n+\t\t struct ref_format *format);\n \n /*\n * Push a single ref onto the array; this can be used to construct your own\n-- \ngitgitgadget\n\n"},{"id":"432638","messageId":"9a6e39c4c9994362212551ec99bb2291b27b78d9.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 07/27] [GSOC] ref-filter: introduce free_ref_array_item_value() function","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:50Z","receivedAt":"2021-08-13T08:23:26Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nWhen we use ref_array_item which is not dynamically allocated and\nwant to free the space of its member \"value\" after the end of use,\nfree_array_item() does not meet our needs, because it tries to free\nref_array_item itself and its member \"symref\".\n\nIntroduce free_ref_array_item_value() for freeing ref_array_item value.\nIt will be called internally by free_array_item(), and it will help\n`cat-file --batch` free ref_array_item's value memory later.\n\nHelped-by: Junio C Hamano \nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 11 ++++++++---\n ref-filter.h | 2 ++\n 2 files changed, 10 insertions(+), 3 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 2fd3b69f6d6..7df7f06002b 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -2300,16 +2300,21 @@ static int ref_filter_handler(const char *refname, const struct object_id *oid,\n \treturn 0;\n }\n \n-/* Free memory allocated for a ref_array_item */\n-static void free_array_item(struct ref_array_item *item)\n+void free_ref_array_item_value(struct ref_array_item *item)\n {\n-\tfree((char *)item->symref);\n \tif (item->value) {\n \t\tint i;\n \t\tfor (i = 0; i < used_atom_cnt; i++)\n \t\t\tfree((char *)item->value[i].s);\n \t\tfree(item->value);\n \t}\n+}\n+\n+/* Free memory allocated for a ref_array_item */\n+static void free_array_item(struct ref_array_item *item)\n+{\n+\tfree((char *)item->symref);\n+\tfree_ref_array_item_value(item);\n \tfree(item);\n }\n \ndiff --git a/ref-filter.h b/ref-filter.h\nindex c15dee8d6b9..44e6dc05ac2 100644\n--- a/ref-filter.h\n+++ b/ref-filter.h\n@@ -111,6 +111,8 @@ struct ref_format {\n int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int type);\n /* Clear all memory allocated to ref_array */\n void ref_array_clear(struct ref_array *array);\n+/* Free ref_array_item's value */\n+void free_ref_array_item_value(struct ref_array_item *item);\n /* Used to verify if the given format is correct and to parse out the used atoms */\n int verify_ref_format(struct ref_format *format);\n /* Sort the given ref_array as per the ref_sorting provided */\n-- \ngitgitgadget\n\n"},{"id":"432639","messageId":"cabe4690ca5a4ea4a6c3d2fd2f070253fd193975.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 05/27] [GSOC] ref-filter: add %(rest) atom","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:48Z","receivedAt":"2021-08-13T08:23:27Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\n%(rest) is a atom used for cat-file batch mode, which can split\nthe input lines at the first whitespace boundary, all characters\nbefore that whitespace are considered to be the object name;\ncharacters after that first run of whitespace (i.e., the \"rest\"\nof the line) are output in place of the %(rest) atom.\n\nIn order to let \"cat-file --batch=%(rest)\" use the ref-filter\ninterface, add %(rest) atom for ref-filter.\n\nIntroduce the reject_atom() to reject the atom %(rest) for\n\"git for-each-ref\", \"git branch\", \"git tag\" and \"git verify-tag\".\n\nReviewed-by: Jacob Keller \nSuggected-by: Jacob Keller \nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 25 +++++++++++++++++++++++++\n ref-filter.h | 5 ++++-\n t/t3203-branch-output.sh | 4 ++++\n t/t6300-for-each-ref.sh | 4 ++++\n t/t7004-tag.sh | 4 ++++\n t/t7030-verify-tag.sh | 4 ++++\n 6 files changed, 45 insertions(+), 1 deletion(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 3c101500ed1..93ce2a6ef2e 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -157,6 +157,7 @@ enum atom_type {\n \tATOM_IF,\n \tATOM_THEN,\n \tATOM_ELSE,\n+\tATOM_REST,\n };\n \n /*\n@@ -560,6 +561,15 @@ static int if_atom_parser(struct ref_format *format, struct used_atom *atom,\n \treturn 0;\n }\n \n+static int rest_atom_parser(struct ref_format *format, struct used_atom *atom,\n+\t\t\t const char *arg, struct strbuf *err)\n+{\n+\tif (arg)\n+\t\treturn strbuf_addf_ret(err, -1, _(\"%%(rest) does not take arguments\"));\n+\tformat->use_rest = 1;\n+\treturn 0;\n+}\n+\n static int head_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t const char *arg, struct strbuf *unused_err)\n {\n@@ -616,6 +626,7 @@ static struct {\n \t[ATOM_IF] = { \"if\", SOURCE_NONE, FIELD_STR, if_atom_parser },\n \t[ATOM_THEN] = { \"then\", SOURCE_NONE },\n \t[ATOM_ELSE] = { \"else\", SOURCE_NONE },\n+\t[ATOM_REST] = { \"rest\", SOURCE_NONE, FIELD_STR, rest_atom_parser },\n \t/*\n \t * Please update $__git_ref_fieldlist in git-completion.bash\n \t * when you add new atoms\n@@ -993,6 +1004,11 @@ static const char *find_next(const char *cp)\n \treturn NULL;\n }\n \n+static int reject_atom(enum atom_type atom_type)\n+{\n+\treturn atom_type == ATOM_REST;\n+}\n+\n /*\n * Make sure the format string is well formed, and parse out\n * the used atoms.\n@@ -1013,6 +1029,8 @@ int verify_ref_format(struct ref_format *format)\n \t\tat = parse_ref_filter_atom(format, sp + 2, ep, &err);\n \t\tif (at < 0)\n \t\t\tdie(\"%s\", err.buf);\n+\t\tif (reject_atom(used_atom[at].atom_type))\n+\t\t\tdie(_(\"this command reject atom %%(%.*s)\"), (int)(ep - sp - 2), sp + 2);\n \n \t\tif ((format->quote_style == QUOTE_PYTHON ||\n \t\t format->quote_style == QUOTE_SHELL ||\n@@ -1932,6 +1950,12 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\t\tv->handler = else_atom_handler;\n \t\t\tv->s = xstrdup(\"\");\n \t\t\tcontinue;\n+\t\t} else if (atom_type == ATOM_REST) {\n+\t\t\tif (ref->rest)\n+\t\t\t\tv->s = xstrdup(ref->rest);\n+\t\t\telse\n+\t\t\t\tv->s = xstrdup(\"\");\n+\t\t\tcontinue;\n \t\t} else\n \t\t\tcontinue;\n \n@@ -2149,6 +2173,7 @@ static struct ref_array_item *new_ref_array_item(const char *refname,\n \n \tFLEX_ALLOC_STR(ref, refname, refname);\n \toidcpy(&ref->objectname, oid);\n+\tref->rest = NULL;\n \n \treturn ref;\n }\ndiff --git a/ref-filter.h b/ref-filter.h\nindex 74fb423fc89..c15dee8d6b9 100644\n--- a/ref-filter.h\n+++ b/ref-filter.h\n@@ -38,6 +38,7 @@ struct ref_sorting {\n \n struct ref_array_item {\n \tstruct object_id objectname;\n+\tconst char *rest;\n \tint flag;\n \tunsigned int kind;\n \tconst char *symref;\n@@ -76,14 +77,16 @@ struct ref_format {\n \t * verify_ref_format() afterwards to finalize.\n \t */\n \tconst char *format;\n+\tconst char *rest;\n \tint quote_style;\n+\tint use_rest;\n \tint use_color;\n \n \t/* Internal state to ref-filter */\n \tint need_color_reset_at_eol;\n };\n \n-#define REF_FORMAT_INIT { NULL, 0, -1 }\n+#define REF_FORMAT_INIT { .use_color = -1 }\n \n /* Macros for checking --merged and --no-merged options */\n #define _OPT_MERGED_NO_MERGED(option, filter, h) \\\ndiff --git a/t/t3203-branch-output.sh b/t/t3203-branch-output.sh\nindex 5325b9f67a0..6e94c6db7b5 100755\n--- a/t/t3203-branch-output.sh\n+++ b/t/t3203-branch-output.sh\n@@ -340,6 +340,10 @@ test_expect_success 'git branch --format option' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'git branch with --format=%(rest) must fail' '\n+\ttest_must_fail git branch --format=\"%(rest)\" >actual\n+'\n+\n test_expect_success 'worktree colors correct' '\n \tcat >expect <<-EOF &&\n \t* (HEAD detached from fromtag)\ndiff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh\nindex 3d15d0a5360..0d2e062f791 100755\n--- a/t/t6300-for-each-ref.sh\n+++ b/t/t6300-for-each-ref.sh\n@@ -1211,6 +1211,10 @@ test_expect_success 'basic atom: head contents:trailers' '\n \ttest_cmp expect actual.clean\n '\n \n+test_expect_success 'basic atom: rest must fail' '\n+\ttest_must_fail git for-each-ref --format=\"%(rest)\" refs/heads/main\n+'\n+\n test_expect_success 'trailer parsing not fooled by --- line' '\n \tgit commit --allow-empty -F - <<-\\EOF &&\n \tthis is the subject\ndiff --git a/t/t7004-tag.sh b/t/t7004-tag.sh\nindex 2f72c5c6883..082be85dffc 100755\n--- a/t/t7004-tag.sh\n+++ b/t/t7004-tag.sh\n@@ -1998,6 +1998,10 @@ test_expect_success '--format should list tags as per format given' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'git tag -l with --format=\"%(rest)\" must fail' '\n+\ttest_must_fail git tag -l --format=\"%(rest)\" \"v1*\"\n+'\n+\n test_expect_success \"set up color tests\" '\n \techo \"v1.0\" >expect.color &&\n \techo \"v1.0\" >expect.bare &&\ndiff --git a/t/t7030-verify-tag.sh b/t/t7030-verify-tag.sh\nindex 3cefde9602b..10faa645157 100755\n--- a/t/t7030-verify-tag.sh\n+++ b/t/t7030-verify-tag.sh\n@@ -194,6 +194,10 @@ test_expect_success GPG 'verifying tag with --format' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success GPG 'verifying tag with --format=\"%(rest)\" must fail' '\n+\ttest_must_fail git verify-tag --format=\"%(rest)\" \"fourth-signed\"\n+'\n+\n test_expect_success GPG 'verifying a forged tag with --format should fail silently' '\n \ttest_must_fail git verify-tag --format=\"tagname : %(tag)\" $(cat forged1.tag) >actual-forged &&\n \ttest_must_be_empty actual-forged\n-- \ngitgitgadget\n\n"},{"id":"432640","messageId":"c82c2d7ceab00b85d90701d9fc6d2023fa7799be.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 06/27] [GSOC] ref-filter: pass get_object() return value to their callers","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:49Z","receivedAt":"2021-08-13T08:23:28Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nBecause in the refactor of `git cat-file --batch` later,\noid_object_info_extended() in get_object() will be used to obtain\nthe info of an object with it's oid. When the object cannot be\nobtained in the git repository, `cat-file --batch` expects to output\n\" missing\" and continue the next oid query instead of letting\nGit exit. In other error conditions, Git should exit normally. So we\ncan achieve this function by passing the return value of get_object().\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nHelped-by: Ævar Arnfjörð Bjarmason \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 17 +++++++++++------\n 1 file changed, 11 insertions(+), 6 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 93ce2a6ef2e..2fd3b69f6d6 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1825,6 +1825,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n {\n \tstruct object *obj;\n \tint i;\n+\tint ret;\n \tstruct object_info empty = OBJECT_INFO_INIT;\n \n \tCALLOC_ARRAY(ref->value, used_atom_cnt);\n@@ -1981,8 +1982,9 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \n \n \toi.oid = ref->objectname;\n-\tif (get_object(ref, 0, &obj, &oi, err))\n-\t\treturn -1;\n+\tret = get_object(ref, 0, &obj, &oi, err);\n+\tif (ret)\n+\t\treturn ret;\n \n \t/*\n \t * If there is no atom that wants to know about tagged\n@@ -2014,8 +2016,10 @@ static int get_ref_atom_value(struct ref_array_item *ref, int atom,\n \t\t\t struct atom_value **v, struct strbuf *err)\n {\n \tif (!ref->value) {\n-\t\tif (populate_value(ref, err))\n-\t\t\treturn -1;\n+\t\tint ret = populate_value(ref, err);\n+\n+\t\tif (ret)\n+\t\t\treturn ret;\n \t\tfill_missing_values(ref->value);\n \t}\n \t*v = &ref->value[atom];\n@@ -2592,6 +2596,7 @@ int format_ref_array_item(struct ref_array_item *info,\n {\n \tconst char *cp, *sp, *ep;\n \tstruct ref_formatting_state state = REF_FORMATTING_STATE_INIT;\n+\tint ret;\n \n \tstate.quote_style = format->quote_style;\n \tpush_stack_element(&state.stack);\n@@ -2604,10 +2609,10 @@ int format_ref_array_item(struct ref_array_item *info,\n \t\tif (cp < sp)\n \t\t\tappend_literal(cp, sp, &state);\n \t\tpos = parse_ref_filter_atom(format, sp + 2, ep, error_buf);\n-\t\tif (pos < 0 || get_ref_atom_value(info, pos, &atomv, error_buf) ||\n+\t\tif (pos < 0 || (ret = get_ref_atom_value(info, pos, &atomv, error_buf)) ||\n \t\t atomv->handler(atomv, &state, error_buf)) {\n \t\t\tpop_stack_element(&state.stack);\n-\t\t\treturn -1;\n+\t\t\treturn ret ? ret : -1;\n \t\t}\n \t}\n \tif (*cp) {\n-- \ngitgitgadget\n\n"},{"id":"432641","messageId":"305ef165064a9c39a53b8ad464bed27db5def355.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 08/27] [GSOC] ref-filter: add cat_file_mode to ref_format","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:51Z","receivedAt":"2021-08-13T08:23:29Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nAdd `cat_file_mode` member to struct `ref_format`. When\n`cat-file --batch` use ref-filter logic later, it can help us\nreject atoms in verify_ref_format() which cat-file cannot use,\ne.g. `%(refname)`, `%(push)`, `%(upstream)\"...\n\nAdd batch_test_atom() to t/t1006-cat-file.sh and add check\nfor cat-file --batch, this can help us clearly show which\natoms cat-file accepts and which atoms it rejects.\n\nHelped-by: Eric Sunshine \nHelped-by: Ævar Arnfjörð Bjarmason \nHelped-by: Christian Couder \nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 23 ++++-\n ref-filter.h | 1 +\n t/t1006-cat-file.sh | 226 ++++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 246 insertions(+), 4 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 7df7f06002b..e50acfc7378 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1004,9 +1004,24 @@ static const char *find_next(const char *cp)\n \treturn NULL;\n }\n \n-static int reject_atom(enum atom_type atom_type)\n-{\n-\treturn atom_type == ATOM_REST;\n+static int reject_atom(int cat_file_mode, enum atom_type atom_type)\n+{\n+\tif (!cat_file_mode)\n+\t\treturn atom_type == ATOM_REST;\n+\n+\t/* cat_file_mode */\n+\tswitch (atom_type) {\n+\tcase ATOM_FLAG:\n+\tcase ATOM_HEAD:\n+\tcase ATOM_PUSH:\n+\tcase ATOM_REFNAME:\n+\tcase ATOM_SYMREF:\n+\tcase ATOM_UPSTREAM:\n+\tcase ATOM_WORKTREEPATH:\n+\t\treturn 1;\n+\tdefault:\n+\t\treturn 0;\n+\t}\n }\n \n /*\n@@ -1029,7 +1044,7 @@ int verify_ref_format(struct ref_format *format)\n \t\tat = parse_ref_filter_atom(format, sp + 2, ep, &err);\n \t\tif (at < 0)\n \t\t\tdie(\"%s\", err.buf);\n-\t\tif (reject_atom(used_atom[at].atom_type))\n+\t\tif (reject_atom(format->cat_file_mode, used_atom[at].atom_type))\n \t\t\tdie(_(\"this command reject atom %%(%.*s)\"), (int)(ep - sp - 2), sp + 2);\n \n \t\tif ((format->quote_style == QUOTE_PYTHON ||\ndiff --git a/ref-filter.h b/ref-filter.h\nindex 44e6dc05ac2..053980a6a42 100644\n--- a/ref-filter.h\n+++ b/ref-filter.h\n@@ -78,6 +78,7 @@ struct ref_format {\n \t */\n \tconst char *format;\n \tconst char *rest;\n+\tint cat_file_mode;\n \tint quote_style;\n \tint use_rest;\n \tint use_color;\ndiff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh\nindex 18b3779ccb6..95d760652eb 100755\n--- a/t/t1006-cat-file.sh\n+++ b/t/t1006-cat-file.sh\n@@ -608,4 +608,230 @@ test_expect_success 'cat-file --batch=\"batman\" with --batch-all-objects will wor\n \tcmp expect actual\n '\n \n+. \"$TEST_DIRECTORY\"/lib-gpg.sh\n+. \"$TEST_DIRECTORY\"/lib-terminal.sh\n+\n+test_expect_success 'cat-file --batch|--batch-check setup' '\n+\techo 1>blob1 &&\n+\tprintf \"a\\0b\\0\\c\" >blob2 &&\n+\tgit add blob1 blob2 &&\n+\tgit commit -m \"Commit Message\" &&\n+\tgit branch -M main &&\n+\tgit tag -a -m \"v0.0.0\" testtag &&\n+\tgit update-ref refs/myblobs/blob1 HEAD:blob1 &&\n+\tgit update-ref refs/myblobs/blob2 HEAD:blob2 &&\n+\tgit update-ref refs/mytrees/tree1 HEAD^{tree}\n+'\n+\n+batch_test_atom() {\n+\tif test \"$3\" = \"fail\"\n+\tthen\n+\t\ttest_expect_${4:-success} $PREREQ \"basic atom: $1 $2 must fail\" \"\n+\t\t\ttest_must_fail git cat-file --batch-check='$2' >bad <<-EOF\n+\t\t\t$1\n+\t\t\tEOF\n+\t\t\"\n+\telse\n+\t\ttest_expect_${4:-success} $PREREQ \"basic atom: $1 $2\" \"\n+\t\t\tgit for-each-ref --format='$2' $1 >expected &&\n+\t\t\tgit cat-file --batch-check='$2' >actual <<-EOF &&\n+\t\t\t$1\n+\t\t\tEOF\n+\t\t\tsanitize_pgp actual.clean &&\n+\t\t\tcmp expected actual.clean\n+\t\t\"\n+\tfi\n+}\n+\n+batch_test_atom refs/heads/main '%(refname)' fail\n+batch_test_atom refs/heads/main '%(refname:)' fail\n+batch_test_atom refs/heads/main '%(refname:short)' fail\n+batch_test_atom refs/heads/main '%(refname:lstrip=1)' fail\n+batch_test_atom refs/heads/main '%(refname:lstrip=2)' fail\n+batch_test_atom refs/heads/main '%(refname:lstrip=-1)' fail\n+batch_test_atom refs/heads/main '%(refname:lstrip=-2)' fail\n+batch_test_atom refs/heads/main '%(refname:rstrip=1)' fail\n+batch_test_atom refs/heads/main '%(refname:rstrip=2)' fail\n+batch_test_atom refs/heads/main '%(refname:rstrip=-1)' fail\n+batch_test_atom refs/heads/main '%(refname:rstrip=-2)' fail\n+batch_test_atom refs/heads/main '%(refname:strip=1)' fail\n+batch_test_atom refs/heads/main '%(refname:strip=2)' fail\n+batch_test_atom refs/heads/main '%(refname:strip=-1)' fail\n+batch_test_atom refs/heads/main '%(refname:strip=-2)' fail\n+batch_test_atom refs/heads/main '%(upstream)' fail\n+batch_test_atom refs/heads/main '%(upstream:short)' fail\n+batch_test_atom refs/heads/main '%(upstream:lstrip=2)' fail\n+batch_test_atom refs/heads/main '%(upstream:lstrip=-2)' fail\n+batch_test_atom refs/heads/main '%(upstream:rstrip=2)' fail\n+batch_test_atom refs/heads/main '%(upstream:rstrip=-2)' fail\n+batch_test_atom refs/heads/main '%(upstream:strip=2)' fail\n+batch_test_atom refs/heads/main '%(upstream:strip=-2)' fail\n+batch_test_atom refs/heads/main '%(push)' fail\n+batch_test_atom refs/heads/main '%(push:short)' fail\n+batch_test_atom refs/heads/main '%(push:lstrip=1)' fail\n+batch_test_atom refs/heads/main '%(push:lstrip=-1)' fail\n+batch_test_atom refs/heads/main '%(push:rstrip=1)' fail\n+batch_test_atom refs/heads/main '%(push:rstrip=-1)' fail\n+batch_test_atom refs/heads/main '%(push:strip=1)' fail\n+batch_test_atom refs/heads/main '%(push:strip=-1)' fail\n+batch_test_atom refs/heads/main '%(objecttype)'\n+batch_test_atom refs/heads/main '%(objectsize)'\n+batch_test_atom refs/heads/main '%(objectsize:disk)'\n+batch_test_atom refs/heads/main '%(deltabase)'\n+batch_test_atom refs/heads/main '%(objectname)'\n+batch_test_atom refs/heads/main '%(objectname:short)' fail\n+batch_test_atom refs/heads/main '%(objectname:short=1)' fail\n+batch_test_atom refs/heads/main '%(objectname:short=10)' fail\n+batch_test_atom refs/heads/main '%(tree)' fail\n+batch_test_atom refs/heads/main '%(tree:short)' fail\n+batch_test_atom refs/heads/main '%(tree:short=1)' fail\n+batch_test_atom refs/heads/main '%(tree:short=10)' fail\n+batch_test_atom refs/heads/main '%(parent)' fail\n+batch_test_atom refs/heads/main '%(parent:short)' fail\n+batch_test_atom refs/heads/main '%(parent:short=1)' fail\n+batch_test_atom refs/heads/main '%(parent:short=10)' fail\n+batch_test_atom refs/heads/main '%(numparent)' fail\n+batch_test_atom refs/heads/main '%(object)' fail\n+batch_test_atom refs/heads/main '%(type)' fail\n+batch_test_atom refs/heads/main '%(raw)' fail\n+batch_test_atom refs/heads/main '%(*objectname)' fail\n+batch_test_atom refs/heads/main '%(*objecttype)' fail\n+batch_test_atom refs/heads/main '%(author)' fail\n+batch_test_atom refs/heads/main '%(authorname)' fail\n+batch_test_atom refs/heads/main '%(authoremail)' fail\n+batch_test_atom refs/heads/main '%(authoremail:trim)' fail\n+batch_test_atom refs/heads/main '%(authoremail:localpart)' fail\n+batch_test_atom refs/heads/main '%(authordate)' fail\n+batch_test_atom refs/heads/main '%(committer)' fail\n+batch_test_atom refs/heads/main '%(committername)' fail\n+batch_test_atom refs/heads/main '%(committeremail)' fail\n+batch_test_atom refs/heads/main '%(committeremail:trim)' fail\n+batch_test_atom refs/heads/main '%(committeremail:localpart)' fail\n+batch_test_atom refs/heads/main '%(committerdate)' fail\n+batch_test_atom refs/heads/main '%(tag)' fail\n+batch_test_atom refs/heads/main '%(tagger)' fail\n+batch_test_atom refs/heads/main '%(taggername)' fail\n+batch_test_atom refs/heads/main '%(taggeremail)' fail\n+batch_test_atom refs/heads/main '%(taggeremail:trim)' fail\n+batch_test_atom refs/heads/main '%(taggeremail:localpart)' fail\n+batch_test_atom refs/heads/main '%(taggerdate)' fail\n+batch_test_atom refs/heads/main '%(creator)' fail\n+batch_test_atom refs/heads/main '%(creatordate)' fail\n+batch_test_atom refs/heads/main '%(subject)' fail\n+batch_test_atom refs/heads/main '%(subject:sanitize)' fail\n+batch_test_atom refs/heads/main '%(contents:subject)' fail\n+batch_test_atom refs/heads/main '%(body)' fail\n+batch_test_atom refs/heads/main '%(contents:body)' fail\n+batch_test_atom refs/heads/main '%(contents:signature)' fail\n+batch_test_atom refs/heads/main '%(contents)' fail\n+batch_test_atom refs/heads/main '%(HEAD)' fail\n+batch_test_atom refs/heads/main '%(upstream:track)' fail\n+batch_test_atom refs/heads/main '%(upstream:trackshort)' fail\n+batch_test_atom refs/heads/main '%(upstream:track,nobracket)' fail\n+batch_test_atom refs/heads/main '%(upstream:nobracket,track)' fail\n+batch_test_atom refs/heads/main '%(push:track)' fail\n+batch_test_atom refs/heads/main '%(push:trackshort)' fail\n+batch_test_atom refs/heads/main '%(worktreepath)' fail\n+batch_test_atom refs/heads/main '%(symref)' fail\n+batch_test_atom refs/heads/main '%(flag)' fail\n+\n+batch_test_atom refs/tags/testtag '%(refname)' fail\n+batch_test_atom refs/tags/testtag '%(refname:short)' fail\n+batch_test_atom refs/tags/testtag '%(upstream)' fail\n+batch_test_atom refs/tags/testtag '%(push)' fail\n+batch_test_atom refs/tags/testtag '%(objecttype)'\n+batch_test_atom refs/tags/testtag '%(objectsize)'\n+batch_test_atom refs/tags/testtag '%(objectsize:disk)'\n+batch_test_atom refs/tags/testtag '%(*objectsize:disk)' fail\n+batch_test_atom refs/tags/testtag '%(deltabase)'\n+batch_test_atom refs/tags/testtag '%(*deltabase)' fail\n+batch_test_atom refs/tags/testtag '%(objectname)'\n+batch_test_atom refs/tags/testtag '%(objectname:short)' fail\n+batch_test_atom refs/tags/testtag '%(tree)' fail\n+batch_test_atom refs/tags/testtag '%(tree:short)' fail\n+batch_test_atom refs/tags/testtag '%(tree:short=1)' fail\n+batch_test_atom refs/tags/testtag '%(tree:short=10)' fail\n+batch_test_atom refs/tags/testtag '%(parent)' fail\n+batch_test_atom refs/tags/testtag '%(parent:short)' fail\n+batch_test_atom refs/tags/testtag '%(parent:short=1)' fail\n+batch_test_atom refs/tags/testtag '%(parent:short=10)' fail\n+batch_test_atom refs/tags/testtag '%(numparent)' fail\n+batch_test_atom refs/tags/testtag '%(object)' fail\n+batch_test_atom refs/tags/testtag '%(type)' fail\n+batch_test_atom refs/tags/testtag '%(*objectname)' fail\n+batch_test_atom refs/tags/testtag '%(*objecttype)' fail\n+batch_test_atom refs/tags/testtag '%(author)' fail\n+batch_test_atom refs/tags/testtag '%(authorname)' fail\n+batch_test_atom refs/tags/testtag '%(authoremail)' fail\n+batch_test_atom refs/tags/testtag '%(authoremail:trim)' fail\n+batch_test_atom refs/tags/testtag '%(authoremail:localpart)' fail\n+batch_test_atom refs/tags/testtag '%(authordate)' fail\n+batch_test_atom refs/tags/testtag '%(committer)' fail\n+batch_test_atom refs/tags/testtag '%(committername)' fail\n+batch_test_atom refs/tags/testtag '%(committeremail)' fail\n+batch_test_atom refs/tags/testtag '%(committeremail:trim)' fail\n+batch_test_atom refs/tags/testtag '%(committeremail:localpart)' fail\n+batch_test_atom refs/tags/testtag '%(committerdate)' fail\n+batch_test_atom refs/tags/testtag '%(tag)' fail\n+batch_test_atom refs/tags/testtag '%(tagger)' fail\n+batch_test_atom refs/tags/testtag '%(taggername)' fail\n+batch_test_atom refs/tags/testtag '%(taggeremail)' fail\n+batch_test_atom refs/tags/testtag '%(taggeremail:trim)' fail\n+batch_test_atom refs/tags/testtag '%(taggeremail:localpart)' fail\n+batch_test_atom refs/tags/testtag '%(taggerdate)' fail\n+batch_test_atom refs/tags/testtag '%(creator)' fail\n+batch_test_atom refs/tags/testtag '%(creatordate)' fail\n+batch_test_atom refs/tags/testtag '%(subject)' fail\n+batch_test_atom refs/tags/testtag '%(subject:sanitize)' fail\n+batch_test_atom refs/tags/testtag '%(contents:subject)' fail\n+batch_test_atom refs/tags/testtag '%(body)' fail\n+batch_test_atom refs/tags/testtag '%(contents:body)' fail\n+batch_test_atom refs/tags/testtag '%(contents:signature)' fail\n+batch_test_atom refs/tags/testtag '%(contents)' fail\n+batch_test_atom refs/tags/testtag '%(HEAD)' fail\n+\n+batch_test_atom refs/myblobs/blob1 '%(refname)' fail\n+batch_test_atom refs/myblobs/blob1 '%(upstream)' fail\n+batch_test_atom refs/myblobs/blob1 '%(push)' fail\n+batch_test_atom refs/myblobs/blob1 '%(HEAD)' fail\n+\n+batch_test_atom refs/myblobs/blob1 '%(objectname)'\n+batch_test_atom refs/myblobs/blob1 '%(objecttype)'\n+batch_test_atom refs/myblobs/blob1 '%(objectsize)'\n+batch_test_atom refs/myblobs/blob1 '%(objectsize:disk)'\n+batch_test_atom refs/myblobs/blob1 '%(deltabase)'\n+\n+batch_test_atom refs/myblobs/blob1 '%(contents)' fail\n+batch_test_atom refs/myblobs/blob2 '%(contents)' fail\n+\n+batch_test_atom refs/myblobs/blob1 '%(raw)' fail\n+batch_test_atom refs/myblobs/blob2 '%(raw)' fail\n+batch_test_atom refs/mytrees/tree1 '%(raw)' fail\n+\n+batch_test_atom refs/myblobs/blob1 '%(raw:size)' fail\n+batch_test_atom refs/myblobs/blob2 '%(raw:size)' fail\n+batch_test_atom refs/mytrees/tree1 '%(raw:size)' fail\n+\n+batch_test_atom refs/myblobs/blob1 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' fail\n+batch_test_atom refs/myblobs/blob2 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' fail\n+batch_test_atom refs/mytrees/tree1 '%(if:equals=tree)%(objecttype)%(then)tree%(else)not tree%(end)' fail\n+\n+batch_test_atom refs/heads/main '%(align:60) objectname is %(objectname)%(end)|%(objectname)' fail\n+batch_test_atom refs/heads/main '%(align:left,60) objectname is %(objectname)%(end)|%(objectname)' fail\n+batch_test_atom refs/heads/main '%(align:middle,60) objectname is %(objectname)%(end)|%(objectname)' fail\n+batch_test_atom refs/heads/main '%(align:60,right) objectname is %(objectname)%(end)|%(objectname)' fail\n+\n+batch_test_atom refs/heads/main 'VALID'\n+batch_test_atom refs/heads/main '%(INVALID)' fail\n+batch_test_atom refs/heads/main '%(authordate:INVALID)' fail\n+\n+batch_test_atom refs/heads/main '%(objectname) %(objecttype) %(objectsize)\n+%(raw)' fail\n+batch_test_atom refs/tags/testtag '%(objectname) %(objecttype) %(objectsize)\n+%(raw)' fail\n+batch_test_atom refs/myblobs/blob1 '%(objectname) %(objecttype) %(objectsize)\n+%(raw)' fail\n+batch_test_atom refs/myblobs/blob2 '%(objectname) %(objecttype) %(objectsize)\n+%(raw)' fail\n+\n test_done\n-- \ngitgitgadget\n\n"},{"id":"432642","messageId":"39d930361dbd87d1ddd2608e3346c10861653658.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 10/27] [GSOC] cat-file: add has_object_file() check","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:53Z","receivedAt":"2021-08-13T08:23:30Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nUse `has_object_file()` in `batch_one_object()` to check\nwhether the input object exists. This can help us reject\nthe missing oid when we let `cat-file --batch` use ref-filter\nlogic later.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n builtin/cat-file.c | 7 +++++++\n 1 file changed, 7 insertions(+)\n\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex 243fe6844bc..59a86412fd0 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -428,6 +428,13 @@ static void batch_one_object(const char *obj_name,\n \t\treturn;\n \t}\n \n+\tif (!has_object_file(&data->oid)) {\n+\t\tprintf(\"%s missing\\n\",\n+\t\t obj_name ? obj_name : oid_to_hex(&data->oid));\n+\t\tfflush(stdout);\n+\t\treturn;\n+\t}\n+\n \tbatch_object_write(obj_name, scratch, opt, data);\n }\n \n-- \ngitgitgadget\n\n"},{"id":"432643","messageId":"d477edb6f05672a85fd3eb907ca13cc4a524113d.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 09/27] [GSOC] ref-filter: modify the error message and value in get_object","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:52Z","receivedAt":"2021-08-13T08:23:30Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nLet get_object() return 1 and print \" missing\" instead\nof returning -1 and printing \"missing object for \"\nif oid_object_info_extended() unable to find the data corresponding\nto oid. When `cat-file --batch` use ref-filter logic later it can\nhelp `format_ref_array_item()` just report that the object is missing\nwithout letting Git exit.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 4 ++--\n t/t6301-for-each-ref-errors.sh | 2 +-\n 2 files changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex e50acfc7378..fd482e9ff20 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1766,8 +1766,8 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj\n \t}\n \tif (oid_object_info_extended(the_repository, &oi->oid, &oi->info,\n \t\t\t\t OBJECT_INFO_LOOKUP_REPLACE))\n-\t\treturn strbuf_addf_ret(err, -1, _(\"missing object %s for %s\"),\n-\t\t\t\t oid_to_hex(&oi->oid), ref->refname);\n+\t\treturn strbuf_addf_ret(err, 1, _(\"%s missing\"),\n+\t\t\t\t oid_to_hex(&oi->oid));\n \tif (oi->info.disk_sizep && oi->disk_size < 0)\n \t\tBUG(\"Object size is less than zero.\");\n \ndiff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh\nindex 40edf9dab53..3553f84a00c 100755\n--- a/t/t6301-for-each-ref-errors.sh\n+++ b/t/t6301-for-each-ref-errors.sh\n@@ -41,7 +41,7 @@ test_expect_success 'Missing objects are reported correctly' '\n \tr=refs/heads/missing &&\n \techo $MISSING >.git/$r &&\n \ttest_when_finished \"rm -f .git/$r\" &&\n-\techo \"fatal: missing object $MISSING for $r\" >missing-err &&\n+\techo \"fatal: $MISSING missing\" >missing-err &&\n \ttest_must_fail git for-each-ref 2>err &&\n \ttest_cmp missing-err err &&\n \t(\n-- \ngitgitgadget\n\n"},{"id":"432644","messageId":"a7aaf18357e615ea35f3933d098a11d78495081a.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 11/27] [GSOC] cat-file: change batch_objects parameter name","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:54Z","receivedAt":"2021-08-13T08:23:31Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nBecause later cat-file reuses ref-filter logic that will add\nparameter \"const struct option *options\" to batch_objects(),\nthe two synonymous parameters of \"opt\" and \"options\" may\nconfuse readers, so change batch_options parameter of\nbatch_objects() from \"opt\" to \"batch\".\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n builtin/cat-file.c | 20 ++++++++++----------\n 1 file changed, 10 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex 59a86412fd0..41d407638d5 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -495,7 +495,7 @@ static int batch_unordered_packed(const struct object_id *oid,\n \treturn batch_unordered_object(oid, data);\n }\n \n-static int batch_objects(struct batch_options *opt)\n+static int batch_objects(struct batch_options *batch)\n {\n \tstruct strbuf input = STRBUF_INIT;\n \tstruct strbuf output = STRBUF_INIT;\n@@ -503,8 +503,8 @@ static int batch_objects(struct batch_options *opt)\n \tint save_warning;\n \tint retval = 0;\n \n-\tif (!opt->format)\n-\t\topt->format = \"%(objectname) %(objecttype) %(objectsize)\";\n+\tif (!batch->format)\n+\t\tbatch->format = \"%(objectname) %(objecttype) %(objectsize)\";\n \n \t/*\n \t * Expand once with our special mark_query flag, which will prime the\n@@ -513,20 +513,20 @@ static int batch_objects(struct batch_options *opt)\n \t */\n \tmemset(&data, 0, sizeof(data));\n \tdata.mark_query = 1;\n-\tstrbuf_expand(&output, opt->format, expand_format, &data);\n+\tstrbuf_expand(&output, batch->format, expand_format, &data);\n \tdata.mark_query = 0;\n \tstrbuf_release(&output);\n-\tif (opt->cmdmode)\n+\tif (batch->cmdmode)\n \t\tdata.split_on_whitespace = 1;\n \n \t/*\n \t * If we are printing out the object, then always fill in the type,\n \t * since we will want to decide whether or not to stream.\n \t */\n-\tif (opt->print_contents)\n+\tif (batch->print_contents)\n \t\tdata.info.typep = &data.type;\n \n-\tif (opt->all_objects) {\n+\tif (batch->all_objects) {\n \t\tstruct object_cb_data cb;\n \t\tstruct object_info empty = OBJECT_INFO_INIT;\n \n@@ -536,11 +536,11 @@ static int batch_objects(struct batch_options *opt)\n \t\tif (has_promisor_remote())\n \t\t\twarning(\"This repository uses promisor remotes. Some objects may not be loaded.\");\n \n-\t\tcb.opt = opt;\n+\t\tcb.opt = batch;\n \t\tcb.expand = &data;\n \t\tcb.scratch = &output;\n \n-\t\tif (opt->unordered) {\n+\t\tif (batch->unordered) {\n \t\t\tstruct oidset seen = OIDSET_INIT;\n \n \t\t\tcb.seen = &seen;\n@@ -590,7 +590,7 @@ static int batch_objects(struct batch_options *opt)\n \t\t\tdata.rest = p;\n \t\t}\n \n-\t\tbatch_one_object(input.buf, &output, opt, &data);\n+\t\tbatch_one_object(input.buf, &output, batch, &data);\n \t}\n \n \tstrbuf_release(&input);\n-- \ngitgitgadget\n\n"},{"id":"432645","messageId":"1f72eb6e4e0f0e5eaeef2b45deebc81f2b611357.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 12/27] [GSOC] cat-file: create p1006-cat-file.sh","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:55Z","receivedAt":"2021-08-13T08:23:33Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nCreate p1006-cat-file.sh to provide performance testing for\n`git cat-file --batch` and `git cat-file --batch-check`. This\nwill help us compare the performance changes after we let\ncat-file reuse the ref-filter logic.\n\nHelped-by: Ævar Arnfjörð Bjarmason \nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n t/perf/p1006-cat-file.sh | 28 ++++++++++++++++++++++++++++\n 1 file changed, 28 insertions(+)\n create mode 100755 t/perf/p1006-cat-file.sh\n\ndiff --git a/t/perf/p1006-cat-file.sh b/t/perf/p1006-cat-file.sh\nnew file mode 100755\nindex 00000000000..b84ac31f9cc\n--- /dev/null\n+++ b/t/perf/p1006-cat-file.sh\n@@ -0,0 +1,28 @@\n+#!/bin/sh\n+\n+test_description='Basic sort performance tests'\n+. ./perf-lib.sh\n+\n+test_perf_default_repo\n+\n+test_expect_success 'setup' '\n+\tgit rev-list --all >rla\n+'\n+\n+test_perf 'cat-file --batch-check' '\n+\tgit cat-file --batch-check \n\nIn order to let cat-file use ref-filter logic, let's do the\nfollowing:\n\n1. Change the type of member `format` in struct `batch_options`\nto `ref_format`, we will pass it to ref-filter later.\n2. Let `batch_objects()` add atoms to format, and use\n`verify_ref_format()` to check atoms.\n3. Use `format_ref_array_item()` in `batch_object_write()` to\nget the formatted data corresponding to the object. If the\nreturn value of `format_ref_array_item()` is equals to zero,\nuse `batch_write()` to print object data; else if the return\nvalue is less than zero, use `die()` to print the error message\nand exit; else if return value is greater than zero, only print\nthe error message, but don't exit.\n4. Use free_ref_array_item_value() to free ref_array_item's\nvalue, and use free_global_resource() to free global memory\nallocated resource.\n\nMost of the atoms in `for-each-ref --format` are now supported,\nsuch as `%(tree)`, `%(parent)`, `%(author)`, `%(tagger)`, `%(if)`,\n`%(then)`, `%(else)`, `%(end)`. But these atoms will be rejected:\n`%(refname)`, `%(symref)`, `%(upstream)`, `%(push)`, `%(worktreepath)`,\n`%(flag)`, `%(HEAD)`, because these atoms are unique to those objects\nthat pointed to by a ref, \"for-each-ref\"'s family can naturally use\nthese atoms, but not all objects are pointed to be a ref, so \"cat-file\"\nwill not be able to use them.\n\nThe performance for `git cat-file --batch-all-objects\n--batch-check` on the Git repository itself with performance\ntesting tool `hyperfine` changes from 616.7 ms ± 8.9 ms to\n758.7 ms ± 16.4 ms.\n\nThe performance for `git cat-file --batch-all-objects --batch\n>/dev/null` on the Git repository itself with performance testing\ntool `time` change from \"25.26s user 0.30s system 98% cpu 25.840 total\"\nto \"28.79s user 0.83s system 99% cpu 29.829 total\".\n\nThe reasons for the performance degradation are as follows:\n1. There are a lot of data copies in the logic of ref-filter.\n2, In order to be able to support more useful formats, complex\ndata structure and parsing process are used in ref-filter.\n\nLater patches will gradually deal with the problem of performance\ndegradation.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n Documentation/git-cat-file.txt | 6 +\n builtin/cat-file.c | 245 +++++++--------------------------\n ref-filter.c | 20 ++-\n ref-filter.h | 2 +\n t/t1006-cat-file.sh | 229 +++++++++++++++---------------\n 5 files changed, 191 insertions(+), 311 deletions(-)\n\ndiff --git a/Documentation/git-cat-file.txt b/Documentation/git-cat-file.txt\nindex 4eb0421b3fd..ef8ab952b2f 100644\n--- a/Documentation/git-cat-file.txt\n+++ b/Documentation/git-cat-file.txt\n@@ -226,6 +226,12 @@ newline. The available atoms are:\n \tafter that first run of whitespace (i.e., the \"rest\" of the\n \tline) are output in place of the `%(rest)` atom.\n \n+Note that most of the atoms in `for-each-ref --format` are now supported,\n+such as `%(tree)`, `%(parent)`, `%(author)`, `%(tagger)`, `%(if)`,\n+`%(then)`, `%(else)`, `%(end)`. But these atoms will be rejected:\n+`%(refname)`, `%(symref)`, `%(upstream)`, `%(push)`, `%(worktreepath)`,\n+`%(flag)`, `%(HEAD)`. See linkgit:git-for-each-ref[1].\n+\n If no format is specified, the default format is `%(objectname)\n %(objecttype) %(objectsize)`.\n \ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex 41d407638d5..306b4d49493 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -16,6 +16,7 @@\n #include \"packfile.h\"\n #include \"object-store.h\"\n #include \"promisor-remote.h\"\n+#include \"ref-filter.h\"\n \n struct batch_options {\n \tint enabled;\n@@ -25,7 +26,7 @@ struct batch_options {\n \tint all_objects;\n \tint unordered;\n \tint cmdmode; /* may be 'w' or 'c' for --filters or --textconv */\n-\tconst char *format;\n+\tstruct ref_format format;\n };\n \n static const char *force_path;\n@@ -195,99 +196,10 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name,\n \n struct expand_data {\n \tstruct object_id oid;\n-\tenum object_type type;\n-\tunsigned long size;\n-\toff_t disk_size;\n \tconst char *rest;\n-\tstruct object_id delta_base_oid;\n-\n-\t/*\n-\t * If mark_query is true, we do not expand anything, but rather\n-\t * just mark the object_info with items we wish to query.\n-\t */\n-\tint mark_query;\n-\n-\t/*\n-\t * Whether to split the input on whitespace before feeding it to\n-\t * get_sha1; this is decided during the mark_query phase based on\n-\t * whether we have a %(rest) token in our format.\n-\t */\n \tint split_on_whitespace;\n-\n-\t/*\n-\t * After a mark_query run, this object_info is set up to be\n-\t * passed to oid_object_info_extended. It will point to the data\n-\t * elements above, so you can retrieve the response from there.\n-\t */\n-\tstruct object_info info;\n-\n-\t/*\n-\t * This flag will be true if the requested batch format and options\n-\t * don't require us to call oid_object_info, which can then be\n-\t * optimized out.\n-\t */\n-\tunsigned skip_object_info : 1;\n };\n \n-static int is_atom(const char *atom, const char *s, int slen)\n-{\n-\tint alen = strlen(atom);\n-\treturn alen == slen && !memcmp(atom, s, alen);\n-}\n-\n-static void expand_atom(struct strbuf *sb, const char *atom, int len,\n-\t\t\tvoid *vdata)\n-{\n-\tstruct expand_data *data = vdata;\n-\n-\tif (is_atom(\"objectname\", atom, len)) {\n-\t\tif (!data->mark_query)\n-\t\t\tstrbuf_addstr(sb, oid_to_hex(&data->oid));\n-\t} else if (is_atom(\"objecttype\", atom, len)) {\n-\t\tif (data->mark_query)\n-\t\t\tdata->info.typep = &data->type;\n-\t\telse\n-\t\t\tstrbuf_addstr(sb, type_name(data->type));\n-\t} else if (is_atom(\"objectsize\", atom, len)) {\n-\t\tif (data->mark_query)\n-\t\t\tdata->info.sizep = &data->size;\n-\t\telse\n-\t\t\tstrbuf_addf(sb, \"%\"PRIuMAX , (uintmax_t)data->size);\n-\t} else if (is_atom(\"objectsize:disk\", atom, len)) {\n-\t\tif (data->mark_query)\n-\t\t\tdata->info.disk_sizep = &data->disk_size;\n-\t\telse\n-\t\t\tstrbuf_addf(sb, \"%\"PRIuMAX, (uintmax_t)data->disk_size);\n-\t} else if (is_atom(\"rest\", atom, len)) {\n-\t\tif (data->mark_query)\n-\t\t\tdata->split_on_whitespace = 1;\n-\t\telse if (data->rest)\n-\t\t\tstrbuf_addstr(sb, data->rest);\n-\t} else if (is_atom(\"deltabase\", atom, len)) {\n-\t\tif (data->mark_query)\n-\t\t\tdata->info.delta_base_oid = &data->delta_base_oid;\n-\t\telse\n-\t\t\tstrbuf_addstr(sb,\n-\t\t\t\t oid_to_hex(&data->delta_base_oid));\n-\t} else\n-\t\tdie(\"unknown format element: %.*s\", len, atom);\n-}\n-\n-static size_t expand_format(struct strbuf *sb, const char *start, void *data)\n-{\n-\tconst char *end;\n-\n-\tif (*start != '(')\n-\t\treturn 0;\n-\tend = strchr(start + 1, ')');\n-\tif (!end)\n-\t\tdie(\"format element '%s' does not end in ')'\", start);\n-\n-\texpand_atom(sb, start + 1, end - start - 1, data);\n-\n-\treturn end - start + 1;\n-}\n-\n static void batch_write(struct batch_options *opt, const void *data, int len)\n {\n \tif (opt->buffer_output) {\n@@ -297,87 +209,34 @@ static void batch_write(struct batch_options *opt, const void *data, int len)\n \t\twrite_or_die(1, data, len);\n }\n \n-static void print_object_or_die(struct batch_options *opt, struct expand_data *data)\n-{\n-\tconst struct object_id *oid = &data->oid;\n-\n-\tassert(data->info.typep);\n-\n-\tif (data->type == OBJ_BLOB) {\n-\t\tif (opt->buffer_output)\n-\t\t\tfflush(stdout);\n-\t\tif (opt->cmdmode) {\n-\t\t\tchar *contents;\n-\t\t\tunsigned long size;\n-\n-\t\t\tif (!data->rest)\n-\t\t\t\tdie(\"missing path for '%s'\", oid_to_hex(oid));\n-\n-\t\t\tif (opt->cmdmode == 'w') {\n-\t\t\t\tif (filter_object(data->rest, 0100644, oid,\n-\t\t\t\t\t\t &contents, &size))\n-\t\t\t\t\tdie(\"could not convert '%s' %s\",\n-\t\t\t\t\t oid_to_hex(oid), data->rest);\n-\t\t\t} else if (opt->cmdmode == 'c') {\n-\t\t\t\tenum object_type type;\n-\t\t\t\tif (!textconv_object(the_repository,\n-\t\t\t\t\t\t data->rest, 0100644, oid,\n-\t\t\t\t\t\t 1, &contents, &size))\n-\t\t\t\t\tcontents = read_object_file(oid,\n-\t\t\t\t\t\t\t\t &type,\n-\t\t\t\t\t\t\t\t &size);\n-\t\t\t\tif (!contents)\n-\t\t\t\t\tdie(\"could not convert '%s' %s\",\n-\t\t\t\t\t oid_to_hex(oid), data->rest);\n-\t\t\t} else\n-\t\t\t\tBUG(\"invalid cmdmode: %c\", opt->cmdmode);\n-\t\t\tbatch_write(opt, contents, size);\n-\t\t\tfree(contents);\n-\t\t} else {\n-\t\t\tstream_blob(oid);\n-\t\t}\n-\t}\n-\telse {\n-\t\tenum object_type type;\n-\t\tunsigned long size;\n-\t\tvoid *contents;\n-\n-\t\tcontents = read_object_file(oid, &type, &size);\n-\t\tif (!contents)\n-\t\t\tdie(\"object %s disappeared\", oid_to_hex(oid));\n-\t\tif (type != data->type)\n-\t\t\tdie(\"object %s changed type!?\", oid_to_hex(oid));\n-\t\tif (data->info.sizep && size != data->size)\n-\t\t\tdie(\"object %s changed size!?\", oid_to_hex(oid));\n-\n-\t\tbatch_write(opt, contents, size);\n-\t\tfree(contents);\n-\t}\n-}\n \n static void batch_object_write(const char *obj_name,\n \t\t\t struct strbuf *scratch,\n \t\t\t struct batch_options *opt,\n \t\t\t struct expand_data *data)\n {\n-\tif (!data->skip_object_info &&\n-\t oid_object_info_extended(the_repository, &data->oid, &data->info,\n-\t\t\t\t OBJECT_INFO_LOOKUP_REPLACE) < 0) {\n-\t\tprintf(\"%s missing\\n\",\n-\t\t obj_name ? obj_name : oid_to_hex(&data->oid));\n-\t\tfflush(stdout);\n-\t\treturn;\n-\t}\n+\tint ret;\n+\tstruct strbuf err = STRBUF_INIT;\n+\tstruct ref_array_item item = { data->oid, data->rest };\n \n \tstrbuf_reset(scratch);\n-\tstrbuf_expand(scratch, opt->format, expand_format, data);\n-\tstrbuf_addch(scratch, '\\n');\n-\tbatch_write(opt, scratch->buf, scratch->len);\n \n-\tif (opt->print_contents) {\n-\t\tprint_object_or_die(opt, data);\n-\t\tbatch_write(opt, \"\\n\", 1);\n+\tret = format_ref_array_item(&item, &opt->format, scratch, &err);\n+\tif (ret < 0)\n+\t\tdie(\"%s\\n\", err.buf);\n+\tif (ret) {\n+\t\t/* ret > 0 means when the object corresponding to oid\n+\t\t * cannot be found in format_ref_array_item(), we only print\n+\t\t * the error message.\n+\t\t */\n+\t\tprintf(\"%s\\n\", err.buf);\n+\t\tfflush(stdout);\n+\t} else {\n+\t\tstrbuf_addch(scratch, '\\n');\n+\t\tbatch_write(opt, scratch->buf, scratch->len);\n \t}\n+\tfree_ref_array_item_value(&item);\n+\tstrbuf_release(&err);\n }\n \n static void batch_one_object(const char *obj_name,\n@@ -495,43 +354,37 @@ static int batch_unordered_packed(const struct object_id *oid,\n \treturn batch_unordered_object(oid, data);\n }\n \n-static int batch_objects(struct batch_options *batch)\n+static const char * const cat_file_usage[] = {\n+\tN_(\"git cat-file (-t [--allow-unknown-type] | -s [--allow-unknown-type] | -e | -p | | --textconv | --filters) [--path=] \"),\n+\tN_(\"git cat-file (--batch[=] | --batch-check[=]) [--follow-symlinks] [--textconv | --filters]\"),\n+\tNULL\n+};\n+\n+static int batch_objects(struct batch_options *batch, const struct option *options)\n {\n \tstruct strbuf input = STRBUF_INIT;\n \tstruct strbuf output = STRBUF_INIT;\n+\tstruct strbuf format = STRBUF_INIT;\n \tstruct expand_data data;\n \tint save_warning;\n \tint retval = 0;\n \n-\tif (!batch->format)\n-\t\tbatch->format = \"%(objectname) %(objecttype) %(objectsize)\";\n-\n-\t/*\n-\t * Expand once with our special mark_query flag, which will prime the\n-\t * object_info to be handed to oid_object_info_extended for each\n-\t * object.\n-\t */\n \tmemset(&data, 0, sizeof(data));\n-\tdata.mark_query = 1;\n-\tstrbuf_expand(&output, batch->format, expand_format, &data);\n-\tdata.mark_query = 0;\n-\tstrbuf_release(&output);\n-\tif (batch->cmdmode)\n-\t\tdata.split_on_whitespace = 1;\n-\n-\t/*\n-\t * If we are printing out the object, then always fill in the type,\n-\t * since we will want to decide whether or not to stream.\n-\t */\n+\tif (batch->format.format)\n+\t\tstrbuf_addstr(&format, batch->format.format);\n+\telse\n+\t\tstrbuf_addstr(&format, \"%(objectname) %(objecttype) %(objectsize)\");\n \tif (batch->print_contents)\n-\t\tdata.info.typep = &data.type;\n+\t\tstrbuf_addstr(&format, \"\\n%(raw)\");\n+\tbatch->format.format = format.buf;\n+\tif (verify_ref_format(&batch->format))\n+\t\tusage_with_options(cat_file_usage, options);\n+\n+\tif (batch->cmdmode || batch->format.use_rest)\n+\t\tdata.split_on_whitespace = 1;\n \n \tif (batch->all_objects) {\n \t\tstruct object_cb_data cb;\n-\t\tstruct object_info empty = OBJECT_INFO_INIT;\n-\n-\t\tif (!memcmp(&data.info, &empty, sizeof(empty)))\n-\t\t\tdata.skip_object_info = 1;\n \n \t\tif (has_promisor_remote())\n \t\t\twarning(\"This repository uses promisor remotes. Some objects may not be loaded.\");\n@@ -560,8 +413,9 @@ static int batch_objects(struct batch_options *batch)\n \n \t\t\toid_array_clear(&sa);\n \t\t}\n-\n+\t\tstrbuf_release(&format);\n \t\tstrbuf_release(&output);\n+\t\tfree_global_resource();\n \t\treturn 0;\n \t}\n \n@@ -593,18 +447,14 @@ static int batch_objects(struct batch_options *batch)\n \t\tbatch_one_object(input.buf, &output, batch, &data);\n \t}\n \n+\tstrbuf_release(&format);\n \tstrbuf_release(&input);\n \tstrbuf_release(&output);\n+\tfree_global_resource();\n \twarn_on_object_refname_ambiguity = save_warning;\n \treturn retval;\n }\n \n-static const char * const cat_file_usage[] = {\n-\tN_(\"git cat-file (-t [--allow-unknown-type] | -s [--allow-unknown-type] | -e | -p | | --textconv | --filters) [--path=] \"),\n-\tN_(\"git cat-file (--batch[=] | --batch-check[=]) [--follow-symlinks] [--textconv | --filters]\"),\n-\tNULL\n-};\n-\n static int git_cat_file_config(const char *var, const char *value, void *cb)\n {\n \tif (userdiff_config(var, value) < 0)\n@@ -627,7 +477,7 @@ static int batch_option_callback(const struct option *opt,\n \n \tbo->enabled = 1;\n \tbo->print_contents = !strcmp(opt->long_name, \"batch\");\n-\tbo->format = arg;\n+\tbo->format.format = arg;\n \n \treturn 0;\n }\n@@ -636,7 +486,9 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix)\n {\n \tint opt = 0;\n \tconst char *exp_type = NULL, *obj_name = NULL;\n-\tstruct batch_options batch = {0};\n+\tstruct batch_options batch = {\n+\t\t.format = REF_FORMAT_INIT\n+\t};\n \tint unknown_type = 0;\n \n \tconst struct option options[] = {\n@@ -675,6 +527,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix)\n \tgit_config(git_cat_file_config, NULL);\n \n \tbatch.buffer_output = -1;\n+\tbatch.format.cat_file_mode = 1;\n \targc = parse_options(argc, argv, prefix, options, cat_file_usage, 0);\n \n \tif (opt) {\n@@ -718,7 +571,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix)\n \t\tbatch.buffer_output = batch.all_objects;\n \n \tif (batch.enabled)\n-\t\treturn batch_objects(&batch);\n+\t\treturn batch_objects(&batch, options);\n \n \tif (unknown_type && opt != 't' && opt != 's')\n \t\tdie(\"git cat-file --allow-unknown-type: use with -s or -t\");\ndiff --git a/ref-filter.c b/ref-filter.c\nindex fd482e9ff20..f83b6f83494 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -2333,16 +2333,11 @@ static void free_array_item(struct ref_array_item *item)\n \tfree(item);\n }\n \n-/* Free all memory allocated for ref_array */\n-void ref_array_clear(struct ref_array *array)\n+/* Free global memory allocated resource */\n+void free_global_resource(void)\n {\n \tint i;\n \n-\tfor (i = 0; i < array->nr; i++)\n-\t\tfree_array_item(array->items[i]);\n-\tFREE_AND_NULL(array->items);\n-\tarray->nr = array->alloc = 0;\n-\n \tfor (i = 0; i < used_atom_cnt; i++) {\n \t\tstruct used_atom *atom = &used_atom[i];\n \t\tif (atom->atom_type == ATOM_HEAD)\n@@ -2359,6 +2354,17 @@ void ref_array_clear(struct ref_array *array)\n \t\tref_to_worktree_map.worktrees = NULL;\n \t}\n }\n+/* Free all memory allocated for ref_array */\n+void ref_array_clear(struct ref_array *array)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i < array->nr; i++)\n+\t\tfree_array_item(array->items[i]);\n+\tFREE_AND_NULL(array->items);\n+\tarray->nr = array->alloc = 0;\n+\tfree_global_resource();\n+}\n \n #define EXCLUDE_REACHED 0\n #define INCLUDE_REACHED 1\ndiff --git a/ref-filter.h b/ref-filter.h\nindex 053980a6a42..ef3d89d73d4 100644\n--- a/ref-filter.h\n+++ b/ref-filter.h\n@@ -112,6 +112,8 @@ struct ref_format {\n int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int type);\n /* Clear all memory allocated to ref_array */\n void ref_array_clear(struct ref_array *array);\n+/* Free global memory allocated resource */\n+void free_global_resource(void);\n /* Free ref_array_item's value */\n void free_ref_array_item_value(struct ref_array_item *item);\n /* Used to verify if the given format is correct and to parse out the used atoms */\ndiff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh\nindex 95d760652eb..8c1943011fb 100755\n--- a/t/t1006-cat-file.sh\n+++ b/t/t1006-cat-file.sh\n@@ -679,51 +679,51 @@ batch_test_atom refs/heads/main '%(objectsize)'\n batch_test_atom refs/heads/main '%(objectsize:disk)'\n batch_test_atom refs/heads/main '%(deltabase)'\n batch_test_atom refs/heads/main '%(objectname)'\n-batch_test_atom refs/heads/main '%(objectname:short)' fail\n-batch_test_atom refs/heads/main '%(objectname:short=1)' fail\n-batch_test_atom refs/heads/main '%(objectname:short=10)' fail\n-batch_test_atom refs/heads/main '%(tree)' fail\n-batch_test_atom refs/heads/main '%(tree:short)' fail\n-batch_test_atom refs/heads/main '%(tree:short=1)' fail\n-batch_test_atom refs/heads/main '%(tree:short=10)' fail\n-batch_test_atom refs/heads/main '%(parent)' fail\n-batch_test_atom refs/heads/main '%(parent:short)' fail\n-batch_test_atom refs/heads/main '%(parent:short=1)' fail\n-batch_test_atom refs/heads/main '%(parent:short=10)' fail\n-batch_test_atom refs/heads/main '%(numparent)' fail\n-batch_test_atom refs/heads/main '%(object)' fail\n-batch_test_atom refs/heads/main '%(type)' fail\n-batch_test_atom refs/heads/main '%(raw)' fail\n-batch_test_atom refs/heads/main '%(*objectname)' fail\n-batch_test_atom refs/heads/main '%(*objecttype)' fail\n-batch_test_atom refs/heads/main '%(author)' fail\n-batch_test_atom refs/heads/main '%(authorname)' fail\n-batch_test_atom refs/heads/main '%(authoremail)' fail\n-batch_test_atom refs/heads/main '%(authoremail:trim)' fail\n-batch_test_atom refs/heads/main '%(authoremail:localpart)' fail\n-batch_test_atom refs/heads/main '%(authordate)' fail\n-batch_test_atom refs/heads/main '%(committer)' fail\n-batch_test_atom refs/heads/main '%(committername)' fail\n-batch_test_atom refs/heads/main '%(committeremail)' fail\n-batch_test_atom refs/heads/main '%(committeremail:trim)' fail\n-batch_test_atom refs/heads/main '%(committeremail:localpart)' fail\n-batch_test_atom refs/heads/main '%(committerdate)' fail\n-batch_test_atom refs/heads/main '%(tag)' fail\n-batch_test_atom refs/heads/main '%(tagger)' fail\n-batch_test_atom refs/heads/main '%(taggername)' fail\n-batch_test_atom refs/heads/main '%(taggeremail)' fail\n-batch_test_atom refs/heads/main '%(taggeremail:trim)' fail\n-batch_test_atom refs/heads/main '%(taggeremail:localpart)' fail\n-batch_test_atom refs/heads/main '%(taggerdate)' fail\n-batch_test_atom refs/heads/main '%(creator)' fail\n-batch_test_atom refs/heads/main '%(creatordate)' fail\n-batch_test_atom refs/heads/main '%(subject)' fail\n-batch_test_atom refs/heads/main '%(subject:sanitize)' fail\n-batch_test_atom refs/heads/main '%(contents:subject)' fail\n-batch_test_atom refs/heads/main '%(body)' fail\n-batch_test_atom refs/heads/main '%(contents:body)' fail\n-batch_test_atom refs/heads/main '%(contents:signature)' fail\n-batch_test_atom refs/heads/main '%(contents)' fail\n+batch_test_atom refs/heads/main '%(objectname:short)'\n+batch_test_atom refs/heads/main '%(objectname:short=1)'\n+batch_test_atom refs/heads/main '%(objectname:short=10)'\n+batch_test_atom refs/heads/main '%(tree)'\n+batch_test_atom refs/heads/main '%(tree:short)'\n+batch_test_atom refs/heads/main '%(tree:short=1)'\n+batch_test_atom refs/heads/main '%(tree:short=10)'\n+batch_test_atom refs/heads/main '%(parent)'\n+batch_test_atom refs/heads/main '%(parent:short)'\n+batch_test_atom refs/heads/main '%(parent:short=1)'\n+batch_test_atom refs/heads/main '%(parent:short=10)'\n+batch_test_atom refs/heads/main '%(numparent)'\n+batch_test_atom refs/heads/main '%(object)'\n+batch_test_atom refs/heads/main '%(type)'\n+batch_test_atom refs/heads/main '%(raw)'\n+batch_test_atom refs/heads/main '%(*objectname)'\n+batch_test_atom refs/heads/main '%(*objecttype)'\n+batch_test_atom refs/heads/main '%(author)'\n+batch_test_atom refs/heads/main '%(authorname)'\n+batch_test_atom refs/heads/main '%(authoremail)'\n+batch_test_atom refs/heads/main '%(authoremail:trim)'\n+batch_test_atom refs/heads/main '%(authoremail:localpart)'\n+batch_test_atom refs/heads/main '%(authordate)'\n+batch_test_atom refs/heads/main '%(committer)'\n+batch_test_atom refs/heads/main '%(committername)'\n+batch_test_atom refs/heads/main '%(committeremail)'\n+batch_test_atom refs/heads/main '%(committeremail:trim)'\n+batch_test_atom refs/heads/main '%(committeremail:localpart)'\n+batch_test_atom refs/heads/main '%(committerdate)'\n+batch_test_atom refs/heads/main '%(tag)'\n+batch_test_atom refs/heads/main '%(tagger)'\n+batch_test_atom refs/heads/main '%(taggername)'\n+batch_test_atom refs/heads/main '%(taggeremail)'\n+batch_test_atom refs/heads/main '%(taggeremail:trim)'\n+batch_test_atom refs/heads/main '%(taggeremail:localpart)'\n+batch_test_atom refs/heads/main '%(taggerdate)'\n+batch_test_atom refs/heads/main '%(creator)'\n+batch_test_atom refs/heads/main '%(creatordate)'\n+batch_test_atom refs/heads/main '%(subject)'\n+batch_test_atom refs/heads/main '%(subject:sanitize)'\n+batch_test_atom refs/heads/main '%(contents:subject)'\n+batch_test_atom refs/heads/main '%(body)'\n+batch_test_atom refs/heads/main '%(contents:body)'\n+batch_test_atom refs/heads/main '%(contents:signature)'\n+batch_test_atom refs/heads/main '%(contents)'\n batch_test_atom refs/heads/main '%(HEAD)' fail\n batch_test_atom refs/heads/main '%(upstream:track)' fail\n batch_test_atom refs/heads/main '%(upstream:trackshort)' fail\n@@ -742,52 +742,52 @@ batch_test_atom refs/tags/testtag '%(push)' fail\n batch_test_atom refs/tags/testtag '%(objecttype)'\n batch_test_atom refs/tags/testtag '%(objectsize)'\n batch_test_atom refs/tags/testtag '%(objectsize:disk)'\n-batch_test_atom refs/tags/testtag '%(*objectsize:disk)' fail\n+batch_test_atom refs/tags/testtag '%(*objectsize:disk)'\n batch_test_atom refs/tags/testtag '%(deltabase)'\n-batch_test_atom refs/tags/testtag '%(*deltabase)' fail\n+batch_test_atom refs/tags/testtag '%(*deltabase)'\n batch_test_atom refs/tags/testtag '%(objectname)'\n-batch_test_atom refs/tags/testtag '%(objectname:short)' fail\n-batch_test_atom refs/tags/testtag '%(tree)' fail\n-batch_test_atom refs/tags/testtag '%(tree:short)' fail\n-batch_test_atom refs/tags/testtag '%(tree:short=1)' fail\n-batch_test_atom refs/tags/testtag '%(tree:short=10)' fail\n-batch_test_atom refs/tags/testtag '%(parent)' fail\n-batch_test_atom refs/tags/testtag '%(parent:short)' fail\n-batch_test_atom refs/tags/testtag '%(parent:short=1)' fail\n-batch_test_atom refs/tags/testtag '%(parent:short=10)' fail\n-batch_test_atom refs/tags/testtag '%(numparent)' fail\n-batch_test_atom refs/tags/testtag '%(object)' fail\n-batch_test_atom refs/tags/testtag '%(type)' fail\n-batch_test_atom refs/tags/testtag '%(*objectname)' fail\n-batch_test_atom refs/tags/testtag '%(*objecttype)' fail\n-batch_test_atom refs/tags/testtag '%(author)' fail\n-batch_test_atom refs/tags/testtag '%(authorname)' fail\n-batch_test_atom refs/tags/testtag '%(authoremail)' fail\n-batch_test_atom refs/tags/testtag '%(authoremail:trim)' fail\n-batch_test_atom refs/tags/testtag '%(authoremail:localpart)' fail\n-batch_test_atom refs/tags/testtag '%(authordate)' fail\n-batch_test_atom refs/tags/testtag '%(committer)' fail\n-batch_test_atom refs/tags/testtag '%(committername)' fail\n-batch_test_atom refs/tags/testtag '%(committeremail)' fail\n-batch_test_atom refs/tags/testtag '%(committeremail:trim)' fail\n-batch_test_atom refs/tags/testtag '%(committeremail:localpart)' fail\n-batch_test_atom refs/tags/testtag '%(committerdate)' fail\n-batch_test_atom refs/tags/testtag '%(tag)' fail\n-batch_test_atom refs/tags/testtag '%(tagger)' fail\n-batch_test_atom refs/tags/testtag '%(taggername)' fail\n-batch_test_atom refs/tags/testtag '%(taggeremail)' fail\n-batch_test_atom refs/tags/testtag '%(taggeremail:trim)' fail\n-batch_test_atom refs/tags/testtag '%(taggeremail:localpart)' fail\n-batch_test_atom refs/tags/testtag '%(taggerdate)' fail\n-batch_test_atom refs/tags/testtag '%(creator)' fail\n-batch_test_atom refs/tags/testtag '%(creatordate)' fail\n-batch_test_atom refs/tags/testtag '%(subject)' fail\n-batch_test_atom refs/tags/testtag '%(subject:sanitize)' fail\n-batch_test_atom refs/tags/testtag '%(contents:subject)' fail\n-batch_test_atom refs/tags/testtag '%(body)' fail\n-batch_test_atom refs/tags/testtag '%(contents:body)' fail\n-batch_test_atom refs/tags/testtag '%(contents:signature)' fail\n-batch_test_atom refs/tags/testtag '%(contents)' fail\n+batch_test_atom refs/tags/testtag '%(objectname:short)'\n+batch_test_atom refs/tags/testtag '%(tree)'\n+batch_test_atom refs/tags/testtag '%(tree:short)'\n+batch_test_atom refs/tags/testtag '%(tree:short=1)'\n+batch_test_atom refs/tags/testtag '%(tree:short=10)'\n+batch_test_atom refs/tags/testtag '%(parent)'\n+batch_test_atom refs/tags/testtag '%(parent:short)'\n+batch_test_atom refs/tags/testtag '%(parent:short=1)'\n+batch_test_atom refs/tags/testtag '%(parent:short=10)'\n+batch_test_atom refs/tags/testtag '%(numparent)'\n+batch_test_atom refs/tags/testtag '%(object)'\n+batch_test_atom refs/tags/testtag '%(type)'\n+batch_test_atom refs/tags/testtag '%(*objectname)'\n+batch_test_atom refs/tags/testtag '%(*objecttype)'\n+batch_test_atom refs/tags/testtag '%(author)'\n+batch_test_atom refs/tags/testtag '%(authorname)'\n+batch_test_atom refs/tags/testtag '%(authoremail)'\n+batch_test_atom refs/tags/testtag '%(authoremail:trim)'\n+batch_test_atom refs/tags/testtag '%(authoremail:localpart)'\n+batch_test_atom refs/tags/testtag '%(authordate)'\n+batch_test_atom refs/tags/testtag '%(committer)'\n+batch_test_atom refs/tags/testtag '%(committername)'\n+batch_test_atom refs/tags/testtag '%(committeremail)'\n+batch_test_atom refs/tags/testtag '%(committeremail:trim)'\n+batch_test_atom refs/tags/testtag '%(committeremail:localpart)'\n+batch_test_atom refs/tags/testtag '%(committerdate)'\n+batch_test_atom refs/tags/testtag '%(tag)'\n+batch_test_atom refs/tags/testtag '%(tagger)'\n+batch_test_atom refs/tags/testtag '%(taggername)'\n+batch_test_atom refs/tags/testtag '%(taggeremail)'\n+batch_test_atom refs/tags/testtag '%(taggeremail:trim)'\n+batch_test_atom refs/tags/testtag '%(taggeremail:localpart)'\n+batch_test_atom refs/tags/testtag '%(taggerdate)'\n+batch_test_atom refs/tags/testtag '%(creator)'\n+batch_test_atom refs/tags/testtag '%(creatordate)'\n+batch_test_atom refs/tags/testtag '%(subject)'\n+batch_test_atom refs/tags/testtag '%(subject:sanitize)'\n+batch_test_atom refs/tags/testtag '%(contents:subject)'\n+batch_test_atom refs/tags/testtag '%(body)'\n+batch_test_atom refs/tags/testtag '%(contents:body)'\n+batch_test_atom refs/tags/testtag '%(contents:signature)'\n+batch_test_atom refs/tags/testtag '%(contents)'\n batch_test_atom refs/tags/testtag '%(HEAD)' fail\n \n batch_test_atom refs/myblobs/blob1 '%(refname)' fail\n@@ -801,37 +801,50 @@ batch_test_atom refs/myblobs/blob1 '%(objectsize)'\n batch_test_atom refs/myblobs/blob1 '%(objectsize:disk)'\n batch_test_atom refs/myblobs/blob1 '%(deltabase)'\n \n-batch_test_atom refs/myblobs/blob1 '%(contents)' fail\n-batch_test_atom refs/myblobs/blob2 '%(contents)' fail\n+batch_test_atom refs/myblobs/blob1 '%(contents)'\n+batch_test_atom refs/myblobs/blob2 '%(contents)'\n \n-batch_test_atom refs/myblobs/blob1 '%(raw)' fail\n-batch_test_atom refs/myblobs/blob2 '%(raw)' fail\n-batch_test_atom refs/mytrees/tree1 '%(raw)' fail\n+batch_test_atom refs/myblobs/blob1 '%(raw)'\n+batch_test_atom refs/myblobs/blob2 '%(raw)'\n+batch_test_atom refs/mytrees/tree1 '%(raw)'\n \n-batch_test_atom refs/myblobs/blob1 '%(raw:size)' fail\n-batch_test_atom refs/myblobs/blob2 '%(raw:size)' fail\n-batch_test_atom refs/mytrees/tree1 '%(raw:size)' fail\n+batch_test_atom refs/myblobs/blob1 '%(raw:size)'\n+batch_test_atom refs/myblobs/blob2 '%(raw:size)'\n+batch_test_atom refs/mytrees/tree1 '%(raw:size)'\n \n-batch_test_atom refs/myblobs/blob1 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' fail\n-batch_test_atom refs/myblobs/blob2 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)' fail\n-batch_test_atom refs/mytrees/tree1 '%(if:equals=tree)%(objecttype)%(then)tree%(else)not tree%(end)' fail\n+batch_test_atom refs/myblobs/blob1 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)'\n+batch_test_atom refs/myblobs/blob2 '%(if:equals=blob)%(objecttype)%(then)commit%(else)not commit%(end)'\n+batch_test_atom refs/mytrees/tree1 '%(if:equals=tree)%(objecttype)%(then)tree%(else)not tree%(end)'\n \n-batch_test_atom refs/heads/main '%(align:60) objectname is %(objectname)%(end)|%(objectname)' fail\n-batch_test_atom refs/heads/main '%(align:left,60) objectname is %(objectname)%(end)|%(objectname)' fail\n-batch_test_atom refs/heads/main '%(align:middle,60) objectname is %(objectname)%(end)|%(objectname)' fail\n-batch_test_atom refs/heads/main '%(align:60,right) objectname is %(objectname)%(end)|%(objectname)' fail\n+batch_test_atom refs/heads/main '%(align:60) objectname is %(objectname)%(end)|%(objectname)'\n+batch_test_atom refs/heads/main '%(align:left,60) objectname is %(objectname)%(end)|%(objectname)'\n+batch_test_atom refs/heads/main '%(align:middle,60) objectname is %(objectname)%(end)|%(objectname)'\n+batch_test_atom refs/heads/main '%(align:60,right) objectname is %(objectname)%(end)|%(objectname)'\n \n batch_test_atom refs/heads/main 'VALID'\n batch_test_atom refs/heads/main '%(INVALID)' fail\n batch_test_atom refs/heads/main '%(authordate:INVALID)' fail\n \n batch_test_atom refs/heads/main '%(objectname) %(objecttype) %(objectsize)\n-%(raw)' fail\n+%(raw)'\n batch_test_atom refs/tags/testtag '%(objectname) %(objecttype) %(objectsize)\n-%(raw)' fail\n+%(raw)'\n batch_test_atom refs/myblobs/blob1 '%(objectname) %(objecttype) %(objectsize)\n-%(raw)' fail\n+%(raw)'\n batch_test_atom refs/myblobs/blob2 '%(objectname) %(objecttype) %(objectsize)\n-%(raw)' fail\n+%(raw)'\n+\n+test_expect_success 'cat-file --batch equals to --batch-check with atoms' '\n+\tgit cat-file --batch-check=\"%(objectname) %(objecttype) %(objectsize)\n+%(raw)\" >expected <<-EOF &&\n+\trefs/heads/main\n+\trefs/tags/testtag\n+\tEOF\n+\tgit cat-file --batch >actual <<-EOF &&\n+\trefs/heads/main\n+\trefs/tags/testtag\n+\tEOF\n+\tcmp expected actual\n+'\n \n test_done\n-- \ngitgitgadget\n\n"},{"id":"432647","messageId":"69493864d535941d31a41c145eb594fe38ea361d.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 14/27] [GSOC] cat-file: reuse err buf in batch_object_write()","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:57Z","receivedAt":"2021-08-13T08:23:37Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nReuse the `err` buffer in batch_object_write(), as the\nbuffer `scratch` does. This will reduce the overhead\nof multiple allocations of memory of the err buffer.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n builtin/cat-file.c | 22 ++++++++++++++--------\n 1 file changed, 14 insertions(+), 8 deletions(-)\n\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex 306b4d49493..034094dc640 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -212,35 +212,36 @@ static void batch_write(struct batch_options *opt, const void *data, int len)\n \n static void batch_object_write(const char *obj_name,\n \t\t\t struct strbuf *scratch,\n+\t\t\t struct strbuf *err,\n \t\t\t struct batch_options *opt,\n \t\t\t struct expand_data *data)\n {\n \tint ret;\n-\tstruct strbuf err = STRBUF_INIT;\n \tstruct ref_array_item item = { data->oid, data->rest };\n \n \tstrbuf_reset(scratch);\n+\tstrbuf_reset(err);\n \n-\tret = format_ref_array_item(&item, &opt->format, scratch, &err);\n+\tret = format_ref_array_item(&item, &opt->format, scratch, err);\n \tif (ret < 0)\n-\t\tdie(\"%s\\n\", err.buf);\n+\t\tdie(\"%s\\n\", err->buf);\n \tif (ret) {\n \t\t/* ret > 0 means when the object corresponding to oid\n \t\t * cannot be found in format_ref_array_item(), we only print\n \t\t * the error message.\n \t\t */\n-\t\tprintf(\"%s\\n\", err.buf);\n+\t\tprintf(\"%s\\n\", err->buf);\n \t\tfflush(stdout);\n \t} else {\n \t\tstrbuf_addch(scratch, '\\n');\n \t\tbatch_write(opt, scratch->buf, scratch->len);\n \t}\n \tfree_ref_array_item_value(&item);\n-\tstrbuf_release(&err);\n }\n \n static void batch_one_object(const char *obj_name,\n \t\t\t struct strbuf *scratch,\n+\t\t\t struct strbuf *err,\n \t\t\t struct batch_options *opt,\n \t\t\t struct expand_data *data)\n {\n@@ -294,7 +295,7 @@ static void batch_one_object(const char *obj_name,\n \t\treturn;\n \t}\n \n-\tbatch_object_write(obj_name, scratch, opt, data);\n+\tbatch_object_write(obj_name, scratch, err, opt, data);\n }\n \n struct object_cb_data {\n@@ -302,13 +303,14 @@ struct object_cb_data {\n \tstruct expand_data *expand;\n \tstruct oidset *seen;\n \tstruct strbuf *scratch;\n+\tstruct strbuf *err;\n };\n \n static int batch_object_cb(const struct object_id *oid, void *vdata)\n {\n \tstruct object_cb_data *data = vdata;\n \toidcpy(&data->expand->oid, oid);\n-\tbatch_object_write(NULL, data->scratch, data->opt, data->expand);\n+\tbatch_object_write(NULL, data->scratch, data->err, data->opt, data->expand);\n \treturn 0;\n }\n \n@@ -364,6 +366,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio\n {\n \tstruct strbuf input = STRBUF_INIT;\n \tstruct strbuf output = STRBUF_INIT;\n+\tstruct strbuf err = STRBUF_INIT;\n \tstruct strbuf format = STRBUF_INIT;\n \tstruct expand_data data;\n \tint save_warning;\n@@ -392,6 +395,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio\n \t\tcb.opt = batch;\n \t\tcb.expand = &data;\n \t\tcb.scratch = &output;\n+\t\tcb.err = &err;\n \n \t\tif (batch->unordered) {\n \t\t\tstruct oidset seen = OIDSET_INIT;\n@@ -416,6 +420,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio\n \t\tstrbuf_release(&format);\n \t\tstrbuf_release(&output);\n \t\tfree_global_resource();\n+\t\tstrbuf_release(&err);\n \t\treturn 0;\n \t}\n \n@@ -444,13 +449,14 @@ static int batch_objects(struct batch_options *batch, const struct option *optio\n \t\t\tdata.rest = p;\n \t\t}\n \n-\t\tbatch_one_object(input.buf, &output, batch, &data);\n+\t\tbatch_one_object(input.buf, &output, &err, batch, &data);\n \t}\n \n \tstrbuf_release(&format);\n \tstrbuf_release(&input);\n \tstrbuf_release(&output);\n \tfree_global_resource();\n+\tstrbuf_release(&err);\n \twarn_on_object_refname_ambiguity = save_warning;\n \treturn retval;\n }\n-- \ngitgitgadget\n\n"},{"id":"432648","messageId":"898e36a92b778a7ec9dfb4f7780dc2dcf95057c1.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 15/27] [GSOC] cat-file: re-implement --textconv, --filters options","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:58Z","receivedAt":"2021-08-13T08:23:38Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nAfter cat-file reuses the ref-filter logic, we re-implement the\nfunctions of --textconv and --filters options.\n\nAdd members `cat_file_cmdmode` to struct `ref_array_item`,\nso that struct `batch_option` member `cmdmode` will be passed\nto ref-filter, and then ref-filter will take use of it to filter\nthe content of the object in get_object().\n\nUse `actual_oi` to record the real expand_data: it may point to the\noriginal `oi` or the `act_oi` processed by `textconv_object()` or\n`convert_to_working_tree()`. `grab_values()` will grab the contents\nof `actual_oi` and `grab_common_values()` to grab the contents of origin\n`oi`, this ensures that `%(objectsize)` still uses the size of the\nunfiltered data.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n builtin/cat-file.c | 2 +-\n ref-filter.c | 37 +++++++++++++++++++++++++++++++++++--\n ref-filter.h | 1 +\n 3 files changed, 37 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex 034094dc640..77b470687e1 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -217,7 +217,7 @@ static void batch_object_write(const char *obj_name,\n \t\t\t struct expand_data *data)\n {\n \tint ret;\n-\tstruct ref_array_item item = { data->oid, data->rest };\n+\tstruct ref_array_item item = { data->oid, data->rest, opt->cmdmode };\n \n \tstrbuf_reset(scratch);\n \tstrbuf_reset(err);\ndiff --git a/ref-filter.c b/ref-filter.c\nindex f83b6f83494..f7c448f6fa1 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1,3 +1,4 @@\n+#define USE_THE_INDEX_COMPATIBILITY_MACROS\n #include \"builtin.h\"\n #include \"cache.h\"\n #include \"parse-options.h\"\n@@ -1759,6 +1760,9 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj\n {\n \t/* parse_object_buffer() will set eaten to 0 if free() will be needed */\n \tint eaten = 1;\n+\tstruct expand_data *actual_oi = oi;\n+\tstruct expand_data act_oi = {0};\n+\n \tif (oi->info.contentp) {\n \t\t/* We need to know that to use parse_object_buffer properly */\n \t\toi->info.sizep = &oi->size;\n@@ -1772,19 +1776,47 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj\n \t\tBUG(\"Object size is less than zero.\");\n \n \tif (oi->info.contentp) {\n-\t\t*obj = parse_object_buffer(the_repository, &oi->oid, oi->type, oi->size, oi->content, &eaten);\n+\t\tif ((ref->cat_file_cmdmode == 'c' || ref->cat_file_cmdmode == 'w') && !ref->rest)\n+\t\t\treturn strbuf_addf_ret(err, -1, _(\"missing path for '%s'\"),\n+\t\t\t\t\t oid_to_hex(&act_oi.oid));\n+\t\tif (oi->type == OBJ_BLOB) {\n+\t\t\tif (ref->cat_file_cmdmode == 'c') {\n+\t\t\t\tact_oi = *oi;\n+\t\t\t\tif (textconv_object(the_repository,\n+\t\t\t\t\t\t ref->rest, 0100644, &act_oi.oid,\n+\t\t\t\t\t\t 1, (char **)(&act_oi.content), &act_oi.size))\n+\t\t\t\t\tactual_oi = &act_oi;\n+\t\t\t} else if (ref->cat_file_cmdmode == 'w') {\n+\t\t\t\tstruct strbuf strbuf = STRBUF_INIT;\n+\t\t\t\tstruct checkout_metadata meta;\n+\t\t\t\tact_oi = *oi;\n+\n+\t\t\t\tinit_checkout_metadata(&meta, NULL, NULL, &act_oi.oid);\n+\t\t\t\tif (!convert_to_working_tree(&the_index, ref->rest, act_oi.content, act_oi.size, &strbuf, &meta))\n+\t\t\t\t\tdie(\"could not convert '%s' %s\",\n+\t\t\t\t\t oid_to_hex(&oi->oid), ref->rest);\n+\t\t\t\tact_oi.size = strbuf.len;\n+\t\t\t\tact_oi.content = strbuf_detach(&strbuf, NULL);\n+\t\t\t\tactual_oi = &act_oi;\n+\t\t\t}\n+\t\t}\n+\t\t*obj = parse_object_buffer(the_repository, &actual_oi->oid, actual_oi->type, actual_oi->size, actual_oi->content, &eaten);\n \t\tif (!*obj) {\n \t\t\tif (!eaten)\n+\t\t\t\tfree(actual_oi->content);\n+\t\t\tif (actual_oi != oi)\n \t\t\t\tfree(oi->content);\n \t\t\treturn strbuf_addf_ret(err, -1, _(\"parse_object_buffer failed on %s for %s\"),\n \t\t\t\t\t oid_to_hex(&oi->oid), ref->refname);\n \t\t}\n-\t\tgrab_values(ref->value, deref, *obj, oi);\n+\t\tgrab_values(ref->value, deref, *obj, actual_oi);\n \t}\n \n \tgrab_common_values(ref->value, deref, oi);\n \tif (!eaten)\n \t\tfree(oi->content);\n+\tif (actual_oi != oi)\n+\t\tfree(actual_oi->content);\n \treturn 0;\n }\n \n@@ -2193,6 +2225,7 @@ static struct ref_array_item *new_ref_array_item(const char *refname,\n \tFLEX_ALLOC_STR(ref, refname, refname);\n \toidcpy(&ref->objectname, oid);\n \tref->rest = NULL;\n+\tref->cat_file_cmdmode = 0;\n \n \treturn ref;\n }\ndiff --git a/ref-filter.h b/ref-filter.h\nindex ef3d89d73d4..853eb554a5b 100644\n--- a/ref-filter.h\n+++ b/ref-filter.h\n@@ -39,6 +39,7 @@ struct ref_sorting {\n struct ref_array_item {\n \tstruct object_id objectname;\n \tconst char *rest;\n+\tint cat_file_cmdmode;\n \tint flag;\n \tunsigned int kind;\n \tconst char *symref;\n-- \ngitgitgadget\n\n"},{"id":"432649","messageId":"ae2de77101497a7cc28867c56fad61d4cc1f2c49.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 16/27] [GSOC] ref-filter: remove grab_oid() function","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:22:59Z","receivedAt":"2021-08-13T08:23:43Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nBecause \"atom_type == ATOM_OBJECTNAME\" implies the condition\nof `starts_with(name, \"objectname\")`, \"atom_type == ATOM_TREE\"\nimplies the condition of `starts_with(name, \"tree\")`, so the\ncheck for `starts_with(name, field)` in grab_oid() is redundant.\n\nSo Remove the grab_oid() from ref-filter, to reduce repeated check.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 26 +++++++++-----------------\n 1 file changed, 9 insertions(+), 17 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex f7c448f6fa1..8664d9a6f4e 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1081,16 +1081,6 @@ static const char *do_grab_oid(const char *field, const struct object_id *oid,\n \t}\n }\n \n-static int grab_oid(const char *name, const char *field, const struct object_id *oid,\n-\t\t struct atom_value *v, struct used_atom *atom)\n-{\n-\tif (starts_with(name, field)) {\n-\t\tv->s = xstrdup(do_grab_oid(field, oid, atom));\n-\t\treturn 1;\n-\t}\n-\treturn 0;\n-}\n-\n /* See grab_values */\n static void grab_common_values(struct atom_value *val, int deref, struct expand_data *oi)\n {\n@@ -1116,8 +1106,9 @@ static void grab_common_values(struct atom_value *val, int deref, struct expand_\n \t\t\t}\n \t\t} else if (atom_type == ATOM_DELTABASE)\n \t\t\tv->s = xstrdup(oid_to_hex(&oi->delta_base_oid));\n-\t\telse if (atom_type == ATOM_OBJECTNAME && deref)\n-\t\t\tgrab_oid(name, \"objectname\", &oi->oid, v, &used_atom[i]);\n+\t\telse if (atom_type == ATOM_OBJECTNAME && deref) {\n+\t\t\tv->s = xstrdup(do_grab_oid(\"objectname\", &oi->oid, &used_atom[i]));\n+\t\t}\n \t}\n }\n \n@@ -1158,9 +1149,10 @@ static void grab_commit_values(struct atom_value *val, int deref, struct object\n \t\t\tcontinue;\n \t\tif (deref)\n \t\t\tname++;\n-\t\tif (atom_type == ATOM_TREE &&\n-\t\t grab_oid(name, \"tree\", get_commit_tree_oid(commit), v, &used_atom[i]))\n+\t\tif (atom_type == ATOM_TREE) {\n+\t\t\tv->s = xstrdup(do_grab_oid(\"tree\", get_commit_tree_oid(commit), &used_atom[i]));\n \t\t\tcontinue;\n+\t\t}\n \t\tif (atom_type == ATOM_NUMPARENT) {\n \t\t\tv->value = commit_list_count(commit->parents);\n \t\t\tv->s = xstrfmt(\"%lu\", (unsigned long)v->value);\n@@ -1965,9 +1957,9 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\t\t\tv->s = xstrdup(buf + 1);\n \t\t\t}\n \t\t\tcontinue;\n-\t\t} else if (!deref && atom_type == ATOM_OBJECTNAME &&\n-\t\t\t grab_oid(name, \"objectname\", &ref->objectname, v, atom)) {\n-\t\t\t\tcontinue;\n+\t\t} else if (!deref && atom_type == ATOM_OBJECTNAME) {\n+\t\t\t v->s = xstrdup(do_grab_oid(\"objectname\", &ref->objectname, atom));\n+\t\t\t continue;\n \t\t} else if (atom_type == ATOM_HEAD) {\n \t\t\tif (atom->u.head && !strcmp(ref->refname, atom->u.head))\n \t\t\t\tv->s = xstrdup(\"*\");\n-- \ngitgitgadget\n\n"},{"id":"432650","messageId":"2c6ce95c8278710b43bf9b7a2fe09732a3172b09.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 17/27] [GSOC] ref-filter: performance optimization by skip parse_object_buffer","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:23:00Z","receivedAt":"2021-08-13T08:23:43Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nWhen we are using some atoms such as %(raw), %(objectname),\n%(objecttype), we don't need to parse the content of the object,\nso we can skip parse_object_buffer() for performance optimization.\n\nIt is worth noting that in these cases, we still need to call\nparse_object_buffer() for parsing:\n\n1. The atom type is one of %(tag), %(type), %(object), %(tree),\n%(numparent) or %(parent).\n2. The type of the object is tag and the atom need to be\ndereferenced e.g. %(*objecttype).\n\nWith this patch, `git cat-file --batch` has a 12.5% performance\nimprovement. But there is still a certain gap with\n`git cat-file --batch` before using the logic of ref-filter.\n`git cat-file --batch` is 10% worse, `git cat-file --batch-check`\nis 30% worse.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 53 ++++++++++++++++++++++++++++++++++++++--------------\n ref-filter.h | 5 +++--\n 2 files changed, 42 insertions(+), 16 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 8664d9a6f4e..1251e062ff8 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1025,6 +1025,20 @@ static int reject_atom(int cat_file_mode, enum atom_type atom_type)\n \t}\n }\n \n+static int need_parse_buffer(enum atom_type atom_type) {\n+\tswitch (atom_type) {\n+\tcase ATOM_TAG:\n+\tcase ATOM_TYPE:\n+\tcase ATOM_OBJECT:\n+\tcase ATOM_TREE:\n+\tcase ATOM_NUMPARENT:\n+\tcase ATOM_PARENT:\n+\t\treturn 1;\n+\tdefault:\n+\t\treturn 0;\n+\t}\n+}\n+\n /*\n * Make sure the format string is well formed, and parse out\n * the used atoms.\n@@ -1045,6 +1059,8 @@ int verify_ref_format(struct ref_format *format)\n \t\tat = parse_ref_filter_atom(format, sp + 2, ep, &err);\n \t\tif (at < 0)\n \t\t\tdie(\"%s\", err.buf);\n+\t\tif (need_parse_buffer(used_atom[at].atom_type))\n+\t\t\tformat->can_skip_parse_buffer = 0;\n \t\tif (reject_atom(format->cat_file_mode, used_atom[at].atom_type))\n \t\t\tdie(_(\"this command reject atom %%(%.*s)\"), (int)(ep - sp - 2), sp + 2);\n \n@@ -1532,14 +1548,16 @@ static void grab_values(struct atom_value *val, int deref, struct object *obj, s\n {\n \tvoid *buf = data->content;\n \n-\tswitch (obj->type) {\n+\tswitch (data->type) {\n \tcase OBJ_TAG:\n-\t\tgrab_tag_values(val, deref, obj);\n+\t\tif (obj)\n+\t\t\tgrab_tag_values(val, deref, obj);\n \t\tgrab_sub_body_contents(val, deref, data);\n \t\tgrab_person(\"tagger\", val, deref, buf);\n \t\tbreak;\n \tcase OBJ_COMMIT:\n-\t\tgrab_commit_values(val, deref, obj);\n+\t\tif (obj)\n+\t\t\tgrab_commit_values(val, deref, obj);\n \t\tgrab_sub_body_contents(val, deref, data);\n \t\tgrab_person(\"author\", val, deref, buf);\n \t\tgrab_person(\"committer\", val, deref, buf);\n@@ -1792,16 +1810,23 @@ static int get_object(struct ref_array_item *ref, int deref, struct object **obj\n \t\t\t\tactual_oi = &act_oi;\n \t\t\t}\n \t\t}\n-\t\t*obj = parse_object_buffer(the_repository, &actual_oi->oid, actual_oi->type, actual_oi->size, actual_oi->content, &eaten);\n-\t\tif (!*obj) {\n-\t\t\tif (!eaten)\n-\t\t\t\tfree(actual_oi->content);\n-\t\t\tif (actual_oi != oi)\n-\t\t\t\tfree(oi->content);\n-\t\t\treturn strbuf_addf_ret(err, -1, _(\"parse_object_buffer failed on %s for %s\"),\n-\t\t\t\t\t oid_to_hex(&oi->oid), ref->refname);\n+\t\tif (ref->can_skip_parse_buffer &&\n+\t\t ((!deref &&\n+\t\t (!need_tagged || oi->type != OBJ_TAG)) ||\n+\t\t deref)) {\n+\t\t\tgrab_values(ref->value, deref, NULL, actual_oi);\n+\t\t} else {\n+\t\t\t*obj = parse_object_buffer(the_repository, &actual_oi->oid, actual_oi->type, actual_oi->size, actual_oi->content, &eaten);\n+\t\t\tif (!*obj) {\n+\t\t\t\tif (!eaten)\n+\t\t\t\t\tfree(actual_oi->content);\n+\t\t\t\tif (actual_oi != oi)\n+\t\t\t\t\tfree(oi->content);\n+\t\t\t\treturn strbuf_addf_ret(err, -1, _(\"parse_object_buffer failed on %s for %s\"),\n+\t\t\t\t\t\t oid_to_hex(&oi->oid), ref->refname);\n+\t\t\t}\n+\t\t\tgrab_values(ref->value, deref, *obj, actual_oi);\n \t\t}\n-\t\tgrab_values(ref->value, deref, *obj, actual_oi);\n \t}\n \n \tgrab_common_values(ref->value, deref, oi);\n@@ -2029,7 +2054,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t * If there is no atom that wants to know about tagged\n \t * object, we are done.\n \t */\n-\tif (!need_tagged || (obj->type != OBJ_TAG))\n+\tif (!need_tagged || (oi.type != OBJ_TAG))\n \t\treturn 0;\n \n \t/*\n@@ -2651,7 +2676,7 @@ int format_ref_array_item(struct ref_array_item *info,\n \n \tstate.quote_style = format->quote_style;\n \tpush_stack_element(&state.stack);\n-\n+\tinfo->can_skip_parse_buffer = format->can_skip_parse_buffer;\n \tfor (cp = format->format; *cp && (sp = find_next(cp)); cp = ep + 1) {\n \t\tstruct atom_value *atomv;\n \t\tint pos;\ndiff --git a/ref-filter.h b/ref-filter.h\nindex 853eb554a5b..8cae9f576d5 100644\n--- a/ref-filter.h\n+++ b/ref-filter.h\n@@ -41,6 +41,7 @@ struct ref_array_item {\n \tconst char *rest;\n \tint cat_file_cmdmode;\n \tint flag;\n+\tint can_skip_parse_buffer;\n \tunsigned int kind;\n \tconst char *symref;\n \tstruct commit *commit;\n@@ -83,12 +84,12 @@ struct ref_format {\n \tint quote_style;\n \tint use_rest;\n \tint use_color;\n-\n+\tint can_skip_parse_buffer;\n \t/* Internal state to ref-filter */\n \tint need_color_reset_at_eol;\n };\n \n-#define REF_FORMAT_INIT { .use_color = -1 }\n+#define REF_FORMAT_INIT { .use_color = -1, .can_skip_parse_buffer = 1 }\n \n /* Macros for checking --merged and --no-merged options */\n #define _OPT_MERGED_NO_MERGED(option, filter, h) \\\n-- \ngitgitgadget\n\n"},{"id":"432652","messageId":"a908402fb520c88d5b99294926b8687a1a53015a.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 18/27] [GSOC] ref-filter: use atom_type and merge two for loop in grab_person","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:23:01Z","receivedAt":"2021-08-13T08:23:43Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\ngrab_person() uses string for atom type comparison, this is very\ninefficient, we can use atom_type to replace it. At the same time,\nwe can merge the two for loops in grab_person() into one, this can\nreduce unnecessary traversal of the used_atom list.\n\nWith this patch, `git cat-file --batch` has a 2% performance\nimprovement.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 57 +++++++++++++++++-----------------------------------\n 1 file changed, 18 insertions(+), 39 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 1251e062ff8..3ccb531e073 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1307,63 +1307,42 @@ static void grab_date(const char *buf, struct atom_value *v, const char *atomnam\n }\n \n /* See grab_values */\n-static void grab_person(const char *who, struct atom_value *val, int deref, void *buf)\n+static void grab_person(enum atom_type type, struct atom_value *val, int deref, void *buf)\n {\n \tint i;\n+\tconst char *who = valid_atom[type].name;\n \tint wholen = strlen(who);\n \tconst char *wholine = NULL;\n \n \tfor (i = 0; i < used_atom_cnt; i++) {\n \t\tconst char *name = used_atom[i].name;\n+\t\tenum atom_type atom_type = used_atom[i].atom_type;\n \t\tstruct atom_value *v = &val[i];\n \t\tif (!!deref != (*name == '*'))\n \t\t\tcontinue;\n \t\tif (deref)\n \t\t\tname++;\n-\t\tif (strncmp(who, name, wholen))\n-\t\t\tcontinue;\n-\t\tif (name[wholen] != 0 &&\n-\t\t strcmp(name + wholen, \"name\") &&\n-\t\t !starts_with(name + wholen, \"email\") &&\n-\t\t !starts_with(name + wholen, \"date\"))\n+\t\tif ((atom_type < type || atom_type > type + 3) &&\n+\t\t /*\n+\t\t * For a tag or a commit object, if \"creator\" or \"creatordate\" is\n+\t\t * requested, do something special.\n+\t\t */\n+\t\t ((atom_type != ATOM_CREATOR && atom_type != ATOM_CREATORDATE) ||\n+\t\t ((atom_type == ATOM_CREATOR || atom_type == ATOM_CREATORDATE) &&\n+\t\t type != ATOM_TAGGER && type != ATOM_COMMITTER)))\n \t\t\tcontinue;\n \t\tif (!wholine)\n \t\t\twholine = find_wholine(who, wholen, buf);\n \t\tif (!wholine)\n \t\t\treturn; /* no point looking for it */\n-\t\tif (name[wholen] == 0)\n+\t\tif (atom_type == type || atom_type == ATOM_CREATOR)\n \t\t\tv->s = copy_line(wholine);\n-\t\telse if (!strcmp(name + wholen, \"name\"))\n+\t\telse if (atom_type == type + 1)\n \t\t\tv->s = copy_name(wholine);\n-\t\telse if (starts_with(name + wholen, \"email\"))\n+\t\telse if (atom_type == type + 2)\n \t\t\tv->s = copy_email(wholine, &used_atom[i]);\n-\t\telse if (starts_with(name + wholen, \"date\"))\n-\t\t\tgrab_date(wholine, v, name);\n-\t}\n-\n-\t/*\n-\t * For a tag or a commit object, if \"creator\" or \"creatordate\" is\n-\t * requested, do something special.\n-\t */\n-\tif (strcmp(who, \"tagger\") && strcmp(who, \"committer\"))\n-\t\treturn; /* \"author\" for commit object is not wanted */\n-\tif (!wholine)\n-\t\twholine = find_wholine(who, wholen, buf);\n-\tif (!wholine)\n-\t\treturn;\n-\tfor (i = 0; i < used_atom_cnt; i++) {\n-\t\tconst char *name = used_atom[i].name;\n-\t\tenum atom_type atom_type = used_atom[i].atom_type;\n-\t\tstruct atom_value *v = &val[i];\n-\t\tif (!!deref != (*name == '*'))\n-\t\t\tcontinue;\n-\t\tif (deref)\n-\t\t\tname++;\n-\n-\t\tif (atom_type == ATOM_CREATORDATE)\n+\t\telse if (atom_type == type + 3 || atom_type == ATOM_CREATORDATE)\n \t\t\tgrab_date(wholine, v, name);\n-\t\telse if (atom_type == ATOM_CREATOR)\n-\t\t\tv->s = copy_line(wholine);\n \t}\n }\n \n@@ -1553,14 +1532,14 @@ static void grab_values(struct atom_value *val, int deref, struct object *obj, s\n \t\tif (obj)\n \t\t\tgrab_tag_values(val, deref, obj);\n \t\tgrab_sub_body_contents(val, deref, data);\n-\t\tgrab_person(\"tagger\", val, deref, buf);\n+\t\tgrab_person(ATOM_TAGGER, val, deref, buf);\n \t\tbreak;\n \tcase OBJ_COMMIT:\n \t\tif (obj)\n \t\t\tgrab_commit_values(val, deref, obj);\n \t\tgrab_sub_body_contents(val, deref, data);\n-\t\tgrab_person(\"author\", val, deref, buf);\n-\t\tgrab_person(\"committer\", val, deref, buf);\n+\t\tgrab_person(ATOM_AUTHOR, val, deref, buf);\n+\t\tgrab_person(ATOM_COMMITTER, val, deref, buf);\n \t\tbreak;\n \tcase OBJ_TREE:\n \t\t/* grab_tree_values(val, deref, obj, buf, sz); */\n-- \ngitgitgadget\n\n"},{"id":"432651","messageId":"20cbc3645f14a2090195abeb880cca1541aa1206.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 19/27] [GSOC] ref-filter: remove strlen from find_subpos","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:23:02Z","receivedAt":"2021-08-13T08:23:44Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nfind_subpos() needs to use strlen() to get the length of the object\ncontent, but in the caller of find_subpos(): grab_sub_body_contents(),\nwe can already get the length of the object content through data->size.\nSo add a `size_t buf_size` in find_subpos(), and pass data->size to\nit. This can lead to weak performance optimizations.\n\nSigned-off-by: ZheNing Hu \nMentored-by: Christian Couder \nMentored-by: Hariom Verma \n---\n ref-filter.c | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 3ccb531e073..be4c4405b12 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1346,7 +1346,7 @@ static void grab_person(enum atom_type type, struct atom_value *val, int deref,\n \t}\n }\n \n-static void find_subpos(const char *buf,\n+static void find_subpos(const char *buf, size_t buf_size,\n \t\t\tconst char **sub, size_t *sublen,\n \t\t\tconst char **body, size_t *bodylen,\n \t\t\tsize_t *nonsiglen,\n@@ -1354,12 +1354,12 @@ static void find_subpos(const char *buf,\n {\n \tstruct strbuf payload = STRBUF_INIT;\n \tstruct strbuf signature = STRBUF_INIT;\n+\tconst char *begin = buf;\n \tconst char *eol;\n-\tconst char *end = buf + strlen(buf);\n \tconst char *sigstart;\n \n \t/* parse signature first; we might not even have a subject line */\n-\tparse_signature(buf, end - buf, &payload, &signature);\n+\tparse_signature(buf, buf_size, &payload, &signature);\n \n \t/* skip past header until we hit empty line */\n \twhile (*buf && *buf != '\\n') {\n@@ -1372,7 +1372,7 @@ static void find_subpos(const char *buf,\n \twhile (*buf == '\\n')\n \t\tbuf++;\n \t*sig = strbuf_detach(&signature, siglen);\n-\tsigstart = buf + parse_signed_buffer(buf, strlen(buf));\n+\tsigstart = buf + parse_signed_buffer(buf, buf_size - (buf - begin));\n \n \t/* subject is first non-empty line */\n \t*sub = buf;\n@@ -1395,7 +1395,7 @@ static void find_subpos(const char *buf,\n \twhile (*buf == '\\n' || *buf == '\\r')\n \t\tbuf++;\n \t*body = buf;\n-\t*bodylen = strlen(buf);\n+\t*bodylen = buf_size - (buf - begin);\n \t*nonsiglen = sigstart - buf;\n }\n \n@@ -1462,7 +1462,7 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp\n \t\t !starts_with(name, \"contents\")))\n \t\t\tcontinue;\n \t\tif (!subpos)\n-\t\t\tfind_subpos(buf,\n+\t\t\tfind_subpos(buf, data->size,\n \t\t\t\t &subpos, &sublen,\n \t\t\t\t &bodypos, &bodylen, &nonsiglen,\n \t\t\t\t &sigpos, &siglen);\n-- \ngitgitgadget\n\n"},{"id":"432653","messageId":"5afac0d412869d68f8e5a231e3967ab7d62e8707.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 20/27] [GSOC] ref-filter: introducing xstrvfmt_len() and xstrfmt_len()","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:23:03Z","receivedAt":"2021-08-13T08:23:45Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nxstrfmt(), xstrvfmt(), they are functions used to add formatt strings\nto a dynamic memory. Because their implementation uses strbuf to store\nits buffer and its length, so we can easily pass its length to the caller,\nso create xstrvfmt_len() and xstrfmt_len() which can fetch the length\nof the buffer through the parameter `len`. This can reduce the cost of\ncalculating the length of the string later through strlen().\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 50 +++++++++++++++++++++++++-------------------------\n strbuf.c | 21 +++++++++++++++++++++\n strbuf.h | 6 ++++++\n 3 files changed, 52 insertions(+), 25 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex be4c4405b12..6e904389d6b 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1115,10 +1115,10 @@ static void grab_common_values(struct atom_value *val, int deref, struct expand_\n \t\telse if (atom_type == ATOM_OBJECTSIZE) {\n \t\t\tif (used_atom[i].u.objectsize.option == O_SIZE_DISK) {\n \t\t\t\tv->value = oi->disk_size;\n-\t\t\t\tv->s = xstrfmt(\"%\"PRIuMAX, (uintmax_t)oi->disk_size);\n+\t\t\t\tv->s = xstrfmt_len(&v->s_size, \"%\"PRIuMAX, (uintmax_t)oi->disk_size);\n \t\t\t} else if (used_atom[i].u.objectsize.option == O_SIZE) {\n \t\t\t\tv->value = oi->size;\n-\t\t\t\tv->s = xstrfmt(\"%\"PRIuMAX , (uintmax_t)oi->size);\n+\t\t\t\tv->s = xstrfmt_len(&v->s_size, \"%\"PRIuMAX , (uintmax_t)oi->size);\n \t\t\t}\n \t\t} else if (atom_type == ATOM_DELTABASE)\n \t\t\tv->s = xstrdup(oid_to_hex(&oi->delta_base_oid));\n@@ -1171,7 +1171,7 @@ static void grab_commit_values(struct atom_value *val, int deref, struct object\n \t\t}\n \t\tif (atom_type == ATOM_NUMPARENT) {\n \t\t\tv->value = commit_list_count(commit->parents);\n-\t\t\tv->s = xstrfmt(\"%lu\", (unsigned long)v->value);\n+\t\t\tv->s = xstrfmt_len(&v->s_size, \"%lu\", (unsigned long)v->value);\n \t\t}\n \t\telse if (atom_type == ATOM_PARENT) {\n \t\t\tstruct commit_list *parents;\n@@ -1449,7 +1449,7 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp\n \t\t\t\tv->s = xmemdupz(buf, buf_size);\n \t\t\t\tv->s_size = buf_size;\n \t\t\t} else if (atom->u.raw_data.option == RAW_LENGTH) {\n-\t\t\t\tv->s = xstrfmt(\"%\"PRIuMAX, (uintmax_t)buf_size);\n+\t\t\t\tv->s = xstrfmt_len(&v->s_size, \"%\"PRIuMAX, (uintmax_t)buf_size);\n \t\t\t}\n \t\t\tcontinue;\n \t\t}\n@@ -1476,7 +1476,7 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp\n \t\t} else if (atom->u.contents.option == C_BODY_DEP)\n \t\t\tv->s = xmemdupz(bodypos, bodylen);\n \t\telse if (atom->u.contents.option == C_LENGTH)\n-\t\t\tv->s = xstrfmt(\"%\"PRIuMAX, (uintmax_t)strlen(subpos));\n+\t\t\tv->s = xstrfmt_len(&v->s_size, \"%\"PRIuMAX, (uintmax_t)strlen(subpos));\n \t\telse if (atom->u.contents.option == C_BODY)\n \t\t\tv->s = xmemdupz(bodypos, nonsiglen);\n \t\telse if (atom->u.contents.option == C_SIG)\n@@ -1645,56 +1645,56 @@ static const char *show_ref(struct refname_atom *atom, const char *refname)\n }\n \n static void fill_remote_ref_details(struct used_atom *atom, const char *refname,\n-\t\t\t\t struct branch *branch, const char **s)\n+\t\t\t\t struct branch *branch, struct atom_value *v)\n {\n \tint num_ours, num_theirs;\n \tif (atom->u.remote_ref.option == RR_REF)\n-\t\t*s = show_ref(&atom->u.remote_ref.refname, refname);\n+\t\tv->s = show_ref(&atom->u.remote_ref.refname, refname);\n \telse if (atom->u.remote_ref.option == RR_TRACK) {\n \t\tif (stat_tracking_info(branch, &num_ours, &num_theirs,\n \t\t\t\t NULL, atom->u.remote_ref.push,\n \t\t\t\t AHEAD_BEHIND_FULL) < 0) {\n-\t\t\t*s = xstrdup(msgs.gone);\n+\t\t\tv->s = xstrdup(msgs.gone);\n \t\t} else if (!num_ours && !num_theirs)\n-\t\t\t*s = xstrdup(\"\");\n+\t\t\tv->s = xstrdup(\"\");\n \t\telse if (!num_ours)\n-\t\t\t*s = xstrfmt(msgs.behind, num_theirs);\n+\t\t\tv->s = xstrfmt_len(&v->s_size, msgs.behind, num_theirs);\n \t\telse if (!num_theirs)\n-\t\t\t*s = xstrfmt(msgs.ahead, num_ours);\n+\t\t\tv->s = xstrfmt_len(&v->s_size, msgs.ahead, num_ours);\n \t\telse\n-\t\t\t*s = xstrfmt(msgs.ahead_behind,\n+\t\t\tv->s= xstrfmt_len(&v->s_size, msgs.ahead_behind,\n \t\t\t\t num_ours, num_theirs);\n-\t\tif (!atom->u.remote_ref.nobracket && *s[0]) {\n-\t\t\tconst char *to_free = *s;\n-\t\t\t*s = xstrfmt(\"[%s]\", *s);\n+\t\tif (!atom->u.remote_ref.nobracket && v->s[0]) {\n+\t\t\tconst char *to_free = v->s;\n+\t\t\tv->s = xstrfmt_len(&v->s_size, \"[%s]\", v->s);\n \t\t\tfree((void *)to_free);\n \t\t}\n \t} else if (atom->u.remote_ref.option == RR_TRACKSHORT) {\n \t\tif (stat_tracking_info(branch, &num_ours, &num_theirs,\n \t\t\t\t NULL, atom->u.remote_ref.push,\n \t\t\t\t AHEAD_BEHIND_FULL) < 0) {\n-\t\t\t*s = xstrdup(\"\");\n+\t\t\tv->s = xstrdup(\"\");\n \t\t\treturn;\n \t\t}\n \t\tif (!num_ours && !num_theirs)\n-\t\t\t*s = xstrdup(\"=\");\n+\t\t\tv->s = xstrdup(\"=\");\n \t\telse if (!num_ours)\n-\t\t\t*s = xstrdup(\"<\");\n+\t\t\tv->s = xstrdup(\"<\");\n \t\telse if (!num_theirs)\n-\t\t\t*s = xstrdup(\">\");\n+\t\t\tv->s = xstrdup(\">\");\n \t\telse\n-\t\t\t*s = xstrdup(\"<>\");\n+\t\t\tv->s = xstrdup(\"<>\");\n \t} else if (atom->u.remote_ref.option == RR_REMOTE_NAME) {\n \t\tint explicit;\n \t\tconst char *remote = atom->u.remote_ref.push ?\n \t\t\tpushremote_for_branch(branch, &explicit) :\n \t\t\tremote_for_branch(branch, &explicit);\n-\t\t*s = xstrdup(explicit ? remote : \"\");\n+\t\tv->s = xstrdup(explicit ? remote : \"\");\n \t} else if (atom->u.remote_ref.option == RR_REMOTE_REF) {\n \t\tconst char *merge;\n \n \t\tmerge = remote_ref_for_branch(branch, atom->u.remote_ref.push);\n-\t\t*s = xstrdup(merge ? merge : \"\");\n+\t\tv->s = xstrdup(merge ? merge : \"\");\n \t} else\n \t\tBUG(\"unhandled RR_* enum\");\n }\n@@ -1922,7 +1922,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \n \t\t\trefname = branch_get_upstream(branch, NULL);\n \t\t\tif (refname)\n-\t\t\t\tfill_remote_ref_details(atom, refname, branch, &v->s);\n+\t\t\t\tfill_remote_ref_details(atom, refname, branch, v);\n \t\t\telse\n \t\t\t\tv->s = xstrdup(\"\");\n \t\t\tcontinue;\n@@ -1943,7 +1943,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\t\t}\n \t\t\t/* We will definitely re-init v->s on the next line. */\n \t\t\tfree((char *)v->s);\n-\t\t\tfill_remote_ref_details(atom, refname, branch, &v->s);\n+\t\t\tfill_remote_ref_details(atom, refname, branch, v);\n \t\t\tcontinue;\n \t\t} else if (atom_type == ATOM_COLOR) {\n \t\t\tv->s = xstrdup(atom->u.color);\n@@ -2006,7 +2006,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\tif (!deref)\n \t\t\tv->s = xstrdup(refname);\n \t\telse\n-\t\t\tv->s = xstrfmt(\"%s^{}\", refname);\n+\t\t\tv->s = xstrfmt_len(&v->s_size, \"%s^{}\", refname);\n \t\tfree((char *)refname);\n \t}\n \ndiff --git a/strbuf.c b/strbuf.c\nindex c8a5789694c..8358d9c3f86 100644\n--- a/strbuf.c\n+++ b/strbuf.c\n@@ -982,6 +982,15 @@ char *xstrvfmt(const char *fmt, va_list ap)\n \treturn strbuf_detach(&buf, NULL);\n }\n \n+char *xstrvfmt_len(ssize_t *len, const char *fmt, va_list ap)\n+{\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstrbuf_vaddf(&buf, fmt, ap);\n+\tif (len)\n+\t\t*len = buf.len;\n+\treturn strbuf_detach(&buf, NULL);\n+}\n+\n char *xstrfmt(const char *fmt, ...)\n {\n \tva_list ap;\n@@ -994,6 +1003,18 @@ char *xstrfmt(const char *fmt, ...)\n \treturn ret;\n }\n \n+char *xstrfmt_len(ssize_t *len, const char *fmt, ...)\n+{\n+\tva_list ap;\n+\tchar *ret;\n+\n+\tva_start(ap, fmt);\n+\tret = xstrvfmt_len(len ,fmt, ap);\n+\tva_end(ap);\n+\n+\treturn ret;\n+}\n+\n void strbuf_addftime(struct strbuf *sb, const char *fmt, const struct tm *tm,\n \t\t int tz_offset, int suppress_tz_name)\n {\ndiff --git a/strbuf.h b/strbuf.h\nindex 5b1113abf8f..5211e0b714f 100644\n--- a/strbuf.h\n+++ b/strbuf.h\n@@ -738,4 +738,10 @@ char *xstrvfmt(const char *fmt, va_list ap);\n __attribute__((format (printf, 1, 2)))\n char *xstrfmt(const char *fmt, ...);\n \n+\n+__attribute__((format (printf, 2, 0)))\n+char *xstrvfmt_len(ssize_t *len, const char *fmt, va_list ap);\n+__attribute__((format (printf, 2, 3)))\n+char *xstrfmt_len(ssize_t *len, const char *fmt, ...);\n+\n #endif /* STRBUF_H */\n-- \ngitgitgadget\n\n"},{"id":"432654","messageId":"31acac9fde7d7239303255ccee2a3f066b2b5f6b.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 21/27] [GSOC] ref-filter: remove second parsing in format_ref_array_item","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:23:04Z","receivedAt":"2021-08-13T08:23:46Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nWe parsed format string in verify_ref_format() and stored the parsed\natom in used_atom array. But in format_ref_array_item() we have\nanother round of parsing format string. This affects performance.\n\nIntroducing the struct parsed_atom_list which can save the current\natom's start and end address in format string and its index in\nused_atom. All parsed_atom_list entry are linked together in the\nform of linked list, and the head node of the linked list is stored\nin struct ref_format. We can use clear_parsed_atom_list() to clear\nthe nodes on the linked list.\n\nThis can bring 1.9% performance improvement for git cat-file --batch.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n builtin/branch.c | 2 ++\n builtin/cat-file.c | 3 +++\n builtin/for-each-ref.c | 3 ++-\n builtin/tag.c | 2 ++\n builtin/verify-tag.c | 2 ++\n ref-filter.c | 45 +++++++++++++++++++++++++++++++++---------\n ref-filter.h | 11 +++++++++++\n 7 files changed, 58 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/branch.c b/builtin/branch.c\nindex b23b1d1752a..e361f8cc661 100644\n--- a/builtin/branch.c\n+++ b/builtin/branch.c\n@@ -459,6 +459,7 @@ static void print_ref_list(struct ref_filter *filter, struct ref_sorting *sortin\n \tstrbuf_release(&err);\n \tstrbuf_release(&out);\n \tref_array_clear(&array);\n+\tclear_parsed_atom_list(&format->parsed_atom_head);\n \tfree(to_free);\n }\n \n@@ -678,6 +679,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)\n \tmemset(&filter, 0, sizeof(filter));\n \tfilter.kind = FILTER_REFS_BRANCHES;\n \tfilter.abbrev = -1;\n+\tINIT_LIST_HEAD(&format.parsed_atom_head);\n \n \tif (argc == 2 && !strcmp(argv[1], \"-h\"))\n \t\tusage_with_options(builtin_branch_usage, options);\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex 77b470687e1..7205b21fb29 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -421,6 +421,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio\n \t\tstrbuf_release(&output);\n \t\tfree_global_resource();\n \t\tstrbuf_release(&err);\n+\t\tclear_parsed_atom_list(&batch->format.parsed_atom_head);\n \t\treturn 0;\n \t}\n \n@@ -457,6 +458,7 @@ static int batch_objects(struct batch_options *batch, const struct option *optio\n \tstrbuf_release(&output);\n \tfree_global_resource();\n \tstrbuf_release(&err);\n+\tclear_parsed_atom_list(&batch->format.parsed_atom_head);\n \twarn_on_object_refname_ambiguity = save_warning;\n \treturn retval;\n }\n@@ -534,6 +536,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix)\n \n \tbatch.buffer_output = -1;\n \tbatch.format.cat_file_mode = 1;\n+\tINIT_LIST_HEAD(&batch.format.parsed_atom_head);\n \targc = parse_options(argc, argv, prefix, options, cat_file_usage, 0);\n \n \tif (opt) {\ndiff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c\nindex 89cb6307d46..6e22d80d5b5 100644\n--- a/builtin/for-each-ref.c\n+++ b/builtin/for-each-ref.c\n@@ -53,8 +53,8 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix)\n \n \tmemset(&array, 0, sizeof(array));\n \tmemset(&filter, 0, sizeof(filter));\n-\n \tformat.format = \"%(objectname) %(objecttype)\\t%(refname)\";\n+\tINIT_LIST_HEAD(&format.parsed_atom_head);\n \n \tgit_config(git_default_config, NULL);\n \n@@ -96,6 +96,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix)\n \tref_array_clear(&array);\n \tfree_commit_list(filter.with_commit);\n \tfree_commit_list(filter.no_commit);\n+\tclear_parsed_atom_list(&format.parsed_atom_head);\n \tUNLEAK(sorting);\n \treturn 0;\n }\ndiff --git a/builtin/tag.c b/builtin/tag.c\nindex 452558ec957..549339cbbe4 100644\n--- a/builtin/tag.c\n+++ b/builtin/tag.c\n@@ -78,6 +78,7 @@ static int list_tags(struct ref_filter *filter, struct ref_sorting *sorting,\n \tstrbuf_release(&output);\n \tref_array_clear(&array);\n \tfree(to_free);\n+\tclear_parsed_atom_list(&format->parsed_atom_head);\n \n \treturn 0;\n }\n@@ -493,6 +494,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)\n \tmemset(&filter, 0, sizeof(filter));\n \tfilter.lines = -1;\n \topt.sign = -1;\n+\tINIT_LIST_HEAD(&format.parsed_atom_head);\n \n \targc = parse_options(argc, argv, prefix, options, git_tag_usage, 0);\n \ndiff --git a/builtin/verify-tag.c b/builtin/verify-tag.c\nindex f45136a06ba..8b0a2b2587c 100644\n--- a/builtin/verify-tag.c\n+++ b/builtin/verify-tag.c\n@@ -39,6 +39,7 @@ int cmd_verify_tag(int argc, const char **argv, const char *prefix)\n \t\tOPT_END()\n \t};\n \n+\tINIT_LIST_HEAD(&format.parsed_atom_head);\n \tgit_config(git_verify_tag_config, NULL);\n \n \targc = parse_options(argc, argv, prefix, verify_tag_options,\n@@ -73,5 +74,6 @@ int cmd_verify_tag(int argc, const char **argv, const char *prefix)\n \t\tif (format.format)\n \t\t\tpretty_print_ref(name, &oid, &format);\n \t}\n+\tclear_parsed_atom_list(&format.parsed_atom_head);\n \treturn had_error;\n }\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 6e904389d6b..771804c06f0 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1051,6 +1051,7 @@ int verify_ref_format(struct ref_format *format)\n \tfor (cp = format->format; *cp && (sp = find_next(cp)); ) {\n \t\tstruct strbuf err = STRBUF_INIT;\n \t\tconst char *color, *ep = strchr(sp, ')');\n+\t\tstruct parsed_atom_list *e;\n \t\tint at;\n \n \t\tif (!ep)\n@@ -1059,6 +1060,12 @@ int verify_ref_format(struct ref_format *format)\n \t\tat = parse_ref_filter_atom(format, sp + 2, ep, &err);\n \t\tif (at < 0)\n \t\t\tdie(\"%s\", err.buf);\n+\t\te = xmalloc(sizeof(*e));\n+\t\te->beg = sp + 2;\n+\t\te->end = ep;\n+\t\te->at = at;\n+\t\tlist_add_tail(&e->list, &format->parsed_atom_head);\n+\n \t\tif (need_parse_buffer(used_atom[at].atom_type))\n \t\t\tformat->can_skip_parse_buffer = 0;\n \t\tif (reject_atom(format->cat_file_mode, used_atom[at].atom_type))\n@@ -2649,26 +2656,32 @@ int format_ref_array_item(struct ref_array_item *info,\n \t\t\t struct strbuf *final_buf,\n \t\t\t struct strbuf *error_buf)\n {\n-\tconst char *cp, *sp, *ep;\n+\tconst char *cp, *sp;\n+\tstruct list_head *item;\n \tstruct ref_formatting_state state = REF_FORMATTING_STATE_INIT;\n \tint ret;\n \n \tstate.quote_style = format->quote_style;\n \tpush_stack_element(&state.stack);\n \tinfo->can_skip_parse_buffer = format->can_skip_parse_buffer;\n-\tfor (cp = format->format; *cp && (sp = find_next(cp)); cp = ep + 1) {\n+\n+\tcp = format->format;\n+\n+\tlist_for_each(item, &format->parsed_atom_head) {\n \t\tstruct atom_value *atomv;\n-\t\tint pos;\n+\t\tstruct parsed_atom_list *e =\n+\t\t\tlist_entry(item, struct parsed_atom_list, list);\n \n-\t\tep = strchr(sp, ')');\n-\t\tif (cp < sp)\n-\t\t\tappend_literal(cp, sp, &state);\n-\t\tpos = parse_ref_filter_atom(format, sp + 2, ep, error_buf);\n-\t\tif (pos < 0 || (ret = get_ref_atom_value(info, pos, &atomv, error_buf)) ||\n+\t\tif (cp < e->beg - 2)\n+\t\t\tappend_literal(cp, e->beg - 2, &state);\n+\t\tif ((ret = get_ref_atom_value(info, e->at, &atomv, error_buf)) ||\n \t\t atomv->handler(atomv, &state, error_buf)) {\n \t\t\tpop_stack_element(&state.stack);\n \t\t\treturn ret ? ret : -1;\n \t\t}\n+\t\tcp = e->end + 1;\n+\t\tif (!*cp)\n+\t\t\tbreak;\n \t}\n \tif (*cp) {\n \t\tsp = cp + strlen(cp);\n@@ -2716,10 +2729,12 @@ static int parse_sorting_atom(const char *atom)\n \t * This parses an atom using a dummy ref_format, since we don't\n \t * actually care about the formatting details.\n \t */\n+\tint res;\n \tstruct ref_format dummy = REF_FORMAT_INIT;\n \tconst char *end = atom + strlen(atom);\n \tstruct strbuf err = STRBUF_INIT;\n-\tint res = parse_ref_filter_atom(&dummy, atom, end, &err);\n+\n+\tres = parse_ref_filter_atom(&dummy, atom, end, &err);\n \tif (res < 0)\n \t\tdie(\"%s\", err.buf);\n \tstrbuf_release(&err);\n@@ -2792,3 +2807,15 @@ int parse_opt_merge_filter(const struct option *opt, const char *arg, int unset)\n \n \treturn 0;\n }\n+\n+void clear_parsed_atom_list(struct list_head *parsed_atom_head)\n+{\n+\tstruct list_head *pos, *tmp;\n+\tstruct parsed_atom_list *item;\n+\n+\tlist_for_each_safe(pos, tmp, parsed_atom_head) {\n+\t\titem = list_entry(pos, struct parsed_atom_list, list);\n+\t\tlist_del(pos);\n+\t\tfree(item);\n+\t}\n+}\ndiff --git a/ref-filter.h b/ref-filter.h\nindex 8cae9f576d5..2b871f9ed4f 100644\n--- a/ref-filter.h\n+++ b/ref-filter.h\n@@ -73,6 +73,13 @@ struct ref_filter {\n \t\tverbose;\n };\n \n+struct parsed_atom_list {\n+\tconst char *beg;\n+\tconst char *end;\n+\tint at;\n+\tstruct list_head list;\n+};\n+\n struct ref_format {\n \t/*\n \t * Set these to define the format; make sure you call\n@@ -87,6 +94,7 @@ struct ref_format {\n \tint can_skip_parse_buffer;\n \t/* Internal state to ref-filter */\n \tint need_color_reset_at_eol;\n+\tstruct list_head parsed_atom_head;\n };\n \n #define REF_FORMAT_INIT { .use_color = -1, .can_skip_parse_buffer = 1 }\n@@ -118,6 +126,9 @@ void ref_array_clear(struct ref_array *array);\n void free_global_resource(void);\n /* Free ref_array_item's value */\n void free_ref_array_item_value(struct ref_array_item *item);\n+/* Clear the parsed_atom_list in ref_format*/\n+void clear_parsed_atom_list(struct list_head *parsed_atom_head);\n+\n /* Used to verify if the given format is correct and to parse out the used atoms */\n int verify_ref_format(struct ref_format *format);\n /* Sort the given ref_array as per the ref_sorting provided */\n-- \ngitgitgadget\n\n"},{"id":"432655","messageId":"b68c00dfa5de5fae77024f4c2949a36ae2c14abd.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 22/27] [GSOC] ref-filter: introduction ref_filter_slopbuf","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:23:05Z","receivedAt":"2021-08-13T08:23:47Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nxstrdup(\"\") calls strdup(\"\"), which in glibc will\ncall malloc(1), this means that there will have unwanted\nmemory allocations.\n\nIntroduce the global one-byte array ref_filter_slopbuf,\nwhich learn from strbuf_slopbuf. let those places which\nneed fill strdup(\"\") now fill ref_filter_slopbuf. This\ncan reduce some unnecessary memory allocation.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 53 ++++++++++++++++++++++++++--------------------------\n 1 file changed, 27 insertions(+), 26 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 771804c06f0..5e00dbd993a 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -25,6 +25,8 @@\n #include \"hashmap.h\"\n #include \"strvec.h\"\n \n+char ref_filter_slopbuf[1];\n+\n static struct ref_msg {\n \tconst char *gone;\n \tconst char *ahead;\n@@ -1225,7 +1227,7 @@ static const char *copy_name(const char *buf)\n \t\tif (!strncmp(cp, \" <\", 2))\n \t\t\treturn xmemdupz(buf, cp - buf);\n \t}\n-\treturn xstrdup(\"\");\n+\treturn ref_filter_slopbuf;\n }\n \n static const char *copy_email(const char *buf, struct used_atom *atom)\n@@ -1233,7 +1235,7 @@ static const char *copy_email(const char *buf, struct used_atom *atom)\n \tconst char *email = strchr(buf, '<');\n \tconst char *eoemail;\n \tif (!email)\n-\t\treturn xstrdup(\"\");\n+\t\treturn ref_filter_slopbuf;\n \tswitch (atom->u.email_option.option) {\n \tcase EO_RAW:\n \t\teoemail = strchr(email, '>');\n@@ -1255,7 +1257,7 @@ static const char *copy_email(const char *buf, struct used_atom *atom)\n \t}\n \n \tif (!eoemail)\n-\t\treturn xstrdup(\"\");\n+\t\treturn ref_filter_slopbuf;\n \treturn xmemdupz(email, eoemail - email);\n }\n \n@@ -1309,7 +1311,7 @@ static void grab_date(const char *buf, struct atom_value *v, const char *atomnam\n \tv->value = timestamp;\n \treturn;\n bad:\n-\tv->s = xstrdup(\"\");\n+\tv->s = ref_filter_slopbuf;\n \tv->value = 0;\n }\n \n@@ -1519,7 +1521,7 @@ static void fill_missing_values(struct atom_value *val)\n \tfor (i = 0; i < used_atom_cnt; i++) {\n \t\tstruct atom_value *v = &val[i];\n \t\tif (v->s == NULL)\n-\t\t\tv->s = xstrdup(\"\");\n+\t\t\tv->s = ref_filter_slopbuf;\n \t}\n }\n \n@@ -1594,7 +1596,7 @@ static const char *lstrip_ref_components(const char *refname, int len)\n \t\tswitch (*start++) {\n \t\tcase '\\0':\n \t\t\tfree((char *)to_free);\n-\t\t\treturn xstrdup(\"\");\n+\t\t\treturn ref_filter_slopbuf;\n \t\tcase '/':\n \t\t\tremaining--;\n \t\t\tbreak;\n@@ -1632,7 +1634,7 @@ static const char *rstrip_ref_components(const char *refname, int len)\n \t\tchar *p = strrchr(start, '/');\n \t\tif (p == NULL) {\n \t\t\tfree((char *)to_free);\n-\t\t\treturn xstrdup(\"\");\n+\t\t\treturn ref_filter_slopbuf;\n \t\t} else\n \t\t\tp[0] = '\\0';\n \t}\n@@ -1663,7 +1665,7 @@ static void fill_remote_ref_details(struct used_atom *atom, const char *refname,\n \t\t\t\t AHEAD_BEHIND_FULL) < 0) {\n \t\t\tv->s = xstrdup(msgs.gone);\n \t\t} else if (!num_ours && !num_theirs)\n-\t\t\tv->s = xstrdup(\"\");\n+\t\t\tv->s = ref_filter_slopbuf;\n \t\telse if (!num_ours)\n \t\t\tv->s = xstrfmt_len(&v->s_size, msgs.behind, num_theirs);\n \t\telse if (!num_theirs)\n@@ -1680,7 +1682,7 @@ static void fill_remote_ref_details(struct used_atom *atom, const char *refname,\n \t\tif (stat_tracking_info(branch, &num_ours, &num_theirs,\n \t\t\t\t NULL, atom->u.remote_ref.push,\n \t\t\t\t AHEAD_BEHIND_FULL) < 0) {\n-\t\t\tv->s = xstrdup(\"\");\n+\t\t\tv->s = ref_filter_slopbuf;\n \t\t\treturn;\n \t\t}\n \t\tif (!num_ours && !num_theirs)\n@@ -1739,7 +1741,7 @@ char *get_head_description(void)\n static const char *get_symref(struct used_atom *atom, struct ref_array_item *ref)\n {\n \tif (!ref->symref)\n-\t\treturn xstrdup(\"\");\n+\t\treturn ref_filter_slopbuf;\n \telse\n \t\treturn show_ref(&atom->u.refname, ref->symref);\n }\n@@ -1861,7 +1863,7 @@ static char *get_worktree_path(const struct used_atom *atom, const struct ref_ar\n \te = hashmap_get(&(ref_to_worktree_map.map), &entry, ref->refname);\n \n \tif (!e)\n-\t\treturn xstrdup(\"\");\n+\t\treturn ref_filter_slopbuf;\n \n \tlookup_result = container_of(e, struct ref_to_worktree_entry, ent);\n \n@@ -1884,7 +1886,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\tref->symref = resolve_refdup(ref->refname, RESOLVE_REF_READING,\n \t\t\t\t\t NULL, NULL);\n \t\tif (!ref->symref)\n-\t\t\tref->symref = xstrdup(\"\");\n+\t\t\tref->symref = ref_filter_slopbuf;\n \t}\n \n \t/* Fill in specials first */\n@@ -1912,7 +1914,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\t\tif (ref->kind == FILTER_REFS_BRANCHES)\n \t\t\t\tv->s = get_worktree_path(atom, ref);\n \t\t\telse\n-\t\t\t\tv->s = xstrdup(\"\");\n+\t\t\t\tv->s = ref_filter_slopbuf;\n \t\t\tcontinue;\n \t\t}\n \t\telse if (atom_type == ATOM_SYMREF)\n@@ -1922,7 +1924,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\t\t/* only local branches may have an upstream */\n \t\t\tif (!skip_prefix(ref->refname, \"refs/heads/\",\n \t\t\t\t\t &branch_name)) {\n-\t\t\t\tv->s = xstrdup(\"\");\n+\t\t\t\tv->s = ref_filter_slopbuf;\n \t\t\t\tcontinue;\n \t\t\t}\n \t\t\tbranch = branch_get(branch_name);\n@@ -1931,11 +1933,11 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\t\tif (refname)\n \t\t\t\tfill_remote_ref_details(atom, refname, branch, v);\n \t\t\telse\n-\t\t\t\tv->s = xstrdup(\"\");\n+\t\t\t\tv->s = ref_filter_slopbuf;\n \t\t\tcontinue;\n \t\t} else if (atom_type == ATOM_PUSH && atom->u.remote_ref.push) {\n \t\t\tconst char *branch_name;\n-\t\t\tv->s = xstrdup(\"\");\n+\t\t\tv->s = ref_filter_slopbuf;\n \t\t\tif (!skip_prefix(ref->refname, \"refs/heads/\",\n \t\t\t\t\t &branch_name))\n \t\t\t\tcontinue;\n@@ -1948,8 +1950,6 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\t\t\tif (!refname)\n \t\t\t\t\tcontinue;\n \t\t\t}\n-\t\t\t/* We will definitely re-init v->s on the next line. */\n-\t\t\tfree((char *)v->s);\n \t\t\tfill_remote_ref_details(atom, refname, branch, v);\n \t\t\tcontinue;\n \t\t} else if (atom_type == ATOM_COLOR) {\n@@ -1962,7 +1962,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\t\tif (ref->flag & REF_ISPACKED)\n \t\t\t\tcp = copy_advance(cp, \",packed\");\n \t\t\tif (cp == buf)\n-\t\t\t\tv->s = xstrdup(\"\");\n+\t\t\t\tv->s = ref_filter_slopbuf;\n \t\t\telse {\n \t\t\t\t*cp = '\\0';\n \t\t\t\tv->s = xstrdup(buf + 1);\n@@ -1979,33 +1979,33 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\t\tcontinue;\n \t\t} else if (atom_type == ATOM_ALIGN) {\n \t\t\tv->handler = align_atom_handler;\n-\t\t\tv->s = xstrdup(\"\");\n+\t\t\tv->s = ref_filter_slopbuf;\n \t\t\tcontinue;\n \t\t} else if (atom_type == ATOM_END) {\n \t\t\tv->handler = end_atom_handler;\n-\t\t\tv->s = xstrdup(\"\");\n+\t\t\tv->s = ref_filter_slopbuf;\n \t\t\tcontinue;\n \t\t} else if (atom_type == ATOM_IF) {\n \t\t\tconst char *s;\n \t\t\tif (skip_prefix(name, \"if:\", &s))\n \t\t\t\tv->s = xstrdup(s);\n \t\t\telse\n-\t\t\t\tv->s = xstrdup(\"\");\n+\t\t\t\tv->s = ref_filter_slopbuf;\n \t\t\tv->handler = if_atom_handler;\n \t\t\tcontinue;\n \t\t} else if (atom_type == ATOM_THEN) {\n \t\t\tv->handler = then_atom_handler;\n-\t\t\tv->s = xstrdup(\"\");\n+\t\t\tv->s = ref_filter_slopbuf;\n \t\t\tcontinue;\n \t\t} else if (atom_type == ATOM_ELSE) {\n \t\t\tv->handler = else_atom_handler;\n-\t\t\tv->s = xstrdup(\"\");\n+\t\t\tv->s = ref_filter_slopbuf;\n \t\t\tcontinue;\n \t\t} else if (atom_type == ATOM_REST) {\n \t\t\tif (ref->rest)\n \t\t\t\tv->s = xstrdup(ref->rest);\n \t\t\telse\n-\t\t\t\tv->s = xstrdup(\"\");\n+\t\t\t\tv->s = ref_filter_slopbuf;\n \t\t\tcontinue;\n \t\t} else\n \t\t\tcontinue;\n@@ -2356,7 +2356,8 @@ void free_ref_array_item_value(struct ref_array_item *item)\n \tif (item->value) {\n \t\tint i;\n \t\tfor (i = 0; i < used_atom_cnt; i++)\n-\t\t\tfree((char *)item->value[i].s);\n+\t\t\tif (item->value[i].s != ref_filter_slopbuf)\n+\t\t\t\tfree((char *)item->value[i].s);\n \t\tfree(item->value);\n \t}\n }\n-- \ngitgitgadget\n\n"},{"id":"432656","messageId":"5b21b04683429fdf85b9df9723777fb6fad8ea7a.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 23/27] [GSOC] ref-filter: add deref member to struct used_atom","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:23:06Z","receivedAt":"2021-08-13T08:23:48Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nIn the original logic, the dereference flag '*' is stored\nat the beginning of used_atom[i].name. (e.g. '*objectname:short')\nThis leads us to make some check on used_atom[i].name[0] to\nknow if we the atom needs to be dereferenced.\n\nSo add `deref` flag to struct used_atom used to indicate\nwhether the atom needs to be dereferenced and let used_atom\nrecord only the remaining part (i.e. 'objectname:short')\n\nThis can simplify the logic of the program.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 55 ++++++++++++++++++++--------------------------------\n 1 file changed, 21 insertions(+), 34 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 5e00dbd993a..bde1f3de9eb 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -174,6 +174,7 @@ enum atom_type {\n * array.\n */\n static struct used_atom {\n+\tunsigned int deref : 1;\n \tenum atom_type atom_type;\n \tconst char *name;\n \tcmp_type type;\n@@ -321,7 +322,7 @@ static int objecttype_atom_parser(struct ref_format *format, struct used_atom *a\n {\n \tif (arg)\n \t\treturn strbuf_addf_ret(err, -1, _(\"%%(objecttype) does not take arguments\"));\n-\tif (*atom->name == '*')\n+\tif (atom->deref)\n \t\toi_deref.info.typep = &oi_deref.type;\n \telse\n \t\toi.info.typep = &oi.type;\n@@ -333,13 +334,13 @@ static int objectsize_atom_parser(struct ref_format *format, struct used_atom *a\n {\n \tif (!arg) {\n \t\tatom->u.objectsize.option = O_SIZE;\n-\t\tif (*atom->name == '*')\n+\t\tif (atom->deref)\n \t\t\toi_deref.info.sizep = &oi_deref.size;\n \t\telse\n \t\t\toi.info.sizep = &oi.size;\n \t} else if (!strcmp(arg, \"disk\")) {\n \t\tatom->u.objectsize.option = O_SIZE_DISK;\n-\t\tif (*atom->name == '*')\n+\t\tif (atom->deref)\n \t\t\toi_deref.info.disk_sizep = &oi_deref.disk_size;\n \t\telse\n \t\t\toi.info.disk_sizep = &oi.disk_size;\n@@ -353,7 +354,7 @@ static int deltabase_atom_parser(struct ref_format *format, struct used_atom *at\n {\n \tif (arg)\n \t\treturn strbuf_addf_ret(err, -1, _(\"%%(deltabase) does not take arguments\"));\n-\tif (*atom->name == '*')\n+\tif (atom->deref)\n \t\toi_deref.info.delta_base_oid = &oi_deref.delta_base_oid;\n \telse\n \t\toi.info.delta_base_oid = &oi.delta_base_oid;\n@@ -675,10 +676,13 @@ static int parse_ref_filter_atom(struct ref_format *format,\n \tconst char *sp;\n \tconst char *arg;\n \tint i, at, atom_len;\n+\tint deref = 0;\n \n \tsp = atom;\n-\tif (*sp == '*' && sp < ep)\n+\tif (*sp == '*' && sp < ep) {\n \t\tsp++; /* deref */\n+\t\tderef = 1;\n+\t}\n \tif (ep <= sp)\n \t\treturn strbuf_addf_ret(err, -1, _(\"malformed field name: %.*s\"),\n \t\t\t\t (int)(ep-atom), atom);\n@@ -695,7 +699,7 @@ static int parse_ref_filter_atom(struct ref_format *format,\n \t/* Do we have the atom already used elsewhere? */\n \tfor (i = 0; i < used_atom_cnt; i++) {\n \t\tint len = strlen(used_atom[i].name);\n-\t\tif (len == ep - atom && !memcmp(used_atom[i].name, atom, len))\n+\t\tif (len == ep - sp && !memcmp(used_atom[i].name, sp, len))\n \t\t\treturn i;\n \t}\n \n@@ -719,17 +723,18 @@ static int parse_ref_filter_atom(struct ref_format *format,\n \tused_atom_cnt++;\n \tREALLOC_ARRAY(used_atom, used_atom_cnt);\n \tused_atom[at].atom_type = i;\n-\tused_atom[at].name = xmemdupz(atom, ep - atom);\n+\tused_atom[at].deref = deref;\n+\tused_atom[at].name = xmemdupz(sp, ep - sp);\n \tused_atom[at].type = valid_atom[i].cmp_type;\n \tused_atom[at].source = valid_atom[i].source;\n \tif (used_atom[at].source == SOURCE_OBJ) {\n-\t\tif (*atom == '*')\n+\t\tif (deref)\n \t\t\toi_deref.info.contentp = &oi_deref.content;\n \t\telse\n \t\t\toi.info.contentp = &oi.content;\n \t}\n \tif (arg) {\n-\t\targ = used_atom[at].name + (arg - atom) + 1;\n+\t\targ = used_atom[at].name + (arg - sp) + 1;\n \t\tif (!*arg) {\n \t\t\t/*\n \t\t\t * Treat empty sub-arguments list as NULL (i.e.,\n@@ -741,7 +746,7 @@ static int parse_ref_filter_atom(struct ref_format *format,\n \tmemset(&used_atom[at].u, 0, sizeof(used_atom[at].u));\n \tif (valid_atom[i].parser && valid_atom[i].parser(format, &used_atom[at], arg, err))\n \t\treturn -1;\n-\tif (*atom == '*')\n+\tif (deref)\n \t\tneed_tagged = 1;\n \tif (i == ATOM_SYMREF)\n \t\tneed_symref = 1;\n@@ -1112,13 +1117,10 @@ static void grab_common_values(struct atom_value *val, int deref, struct expand_\n \tint i;\n \n \tfor (i = 0; i < used_atom_cnt; i++) {\n-\t\tconst char *name = used_atom[i].name;\n \t\tenum atom_type atom_type = used_atom[i].atom_type;\n \t\tstruct atom_value *v = &val[i];\n-\t\tif (!!deref != (*name == '*'))\n+\t\tif (!!deref != used_atom[i].deref)\n \t\t\tcontinue;\n-\t\tif (deref)\n-\t\t\tname++;\n \t\tif (atom_type == ATOM_OBJECTTYPE)\n \t\t\tv->s = xstrdup(type_name(oi->type));\n \t\telse if (atom_type == ATOM_OBJECTSIZE) {\n@@ -1144,13 +1146,10 @@ static void grab_tag_values(struct atom_value *val, int deref, struct object *ob\n \tstruct tag *tag = (struct tag *) obj;\n \n \tfor (i = 0; i < used_atom_cnt; i++) {\n-\t\tconst char *name = used_atom[i].name;\n \t\tenum atom_type atom_type = used_atom[i].atom_type;\n \t\tstruct atom_value *v = &val[i];\n-\t\tif (!!deref != (*name == '*'))\n+\t\tif (!!deref != used_atom[i].deref)\n \t\t\tcontinue;\n-\t\tif (deref)\n-\t\t\tname++;\n \t\tif (atom_type == ATOM_TAG)\n \t\t\tv->s = xstrdup(tag->tag);\n \t\telse if (atom_type == ATOM_TYPE && tag->tagged)\n@@ -1167,13 +1166,10 @@ static void grab_commit_values(struct atom_value *val, int deref, struct object\n \tstruct commit *commit = (struct commit *) obj;\n \n \tfor (i = 0; i < used_atom_cnt; i++) {\n-\t\tconst char *name = used_atom[i].name;\n \t\tenum atom_type atom_type = used_atom[i].atom_type;\n \t\tstruct atom_value *v = &val[i];\n-\t\tif (!!deref != (*name == '*'))\n+\t\tif (!!deref != used_atom[i].deref)\n \t\t\tcontinue;\n-\t\tif (deref)\n-\t\t\tname++;\n \t\tif (atom_type == ATOM_TREE) {\n \t\t\tv->s = xstrdup(do_grab_oid(\"tree\", get_commit_tree_oid(commit), &used_atom[i]));\n \t\t\tcontinue;\n@@ -1327,10 +1323,8 @@ static void grab_person(enum atom_type type, struct atom_value *val, int deref,\n \t\tconst char *name = used_atom[i].name;\n \t\tenum atom_type atom_type = used_atom[i].atom_type;\n \t\tstruct atom_value *v = &val[i];\n-\t\tif (!!deref != (*name == '*'))\n+\t\tif (!!deref != used_atom[i].deref)\n \t\t\tcontinue;\n-\t\tif (deref)\n-\t\t\tname++;\n \t\tif ((atom_type < type || atom_type > type + 3) &&\n \t\t /*\n \t\t * For a tag or a commit object, if \"creator\" or \"creatordate\" is\n@@ -1446,10 +1440,8 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp\n \t\tstruct atom_value *v = &val[i];\n \t\tenum atom_type atom_type = atom->atom_type;\n \n-\t\tif (!!deref != (*name == '*'))\n+\t\tif (!!deref != used_atom[i].deref)\n \t\t\tcontinue;\n-\t\tif (deref)\n-\t\t\tname++;\n \n \t\tif (atom_type == ATOM_RAW) {\n \t\t\tunsigned long buf_size = data->size;\n@@ -1895,7 +1887,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\tenum atom_type atom_type = atom->atom_type;\n \t\tconst char *name = used_atom[i].name;\n \t\tstruct atom_value *v = &ref->value[i];\n-\t\tint deref = 0;\n+\t\tint deref = atom->deref;\n \t\tconst char *refname;\n \t\tstruct branch *branch = NULL;\n \n@@ -1903,11 +1895,6 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\tv->handler = append_atom;\n \t\tv->atom = atom;\n \n-\t\tif (*name == '*') {\n-\t\t\tderef = 1;\n-\t\t\tname++;\n-\t\t}\n-\n \t\tif (atom_type == ATOM_REFNAME)\n \t\t\trefname = get_refname(atom, ref);\n \t\telse if (atom_type == ATOM_WORKTREEPATH) {\n-- \ngitgitgadget\n\n"},{"id":"432657","messageId":"bd568c73a2df0d6ac28fcaeb10d706073f73fe0a.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 24/27] [GSOC] ref-filter: introduce symref_atom_parser()","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:23:07Z","receivedAt":"2021-08-13T08:23:51Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nparse_ref_filter_atom() checks whether the type of the atom\nis ATOM_SYMREF, this is not necessary for other atoms. So\nadd the symref_atom_parser() specifically to parse the atom,\nand set need_symref in it. This can make the program logic\nmore concise.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 11 ++++++++---\n 1 file changed, 8 insertions(+), 3 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex bde1f3de9eb..a0de5f21ff5 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -481,6 +481,13 @@ static int person_email_atom_parser(struct ref_format *format, struct used_atom\n \treturn 0;\n }\n \n+static int symref_atom_parser(struct ref_format *format, struct used_atom *atom,\n+\t\t\t const char *arg, struct strbuf *err)\n+{\n+\tneed_symref = 1;\n+\treturn refname_atom_parser_internal(&atom->u.refname, arg, atom->name, err);\n+}\n+\n static int refname_atom_parser(struct ref_format *format, struct used_atom *atom,\n \t\t\t const char *arg, struct strbuf *err)\n {\n@@ -620,7 +627,7 @@ static struct {\n \t[ATOM_RAW] = { \"raw\", SOURCE_OBJ, FIELD_STR, raw_atom_parser },\n \t[ATOM_UPSTREAM] = { \"upstream\", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser },\n \t[ATOM_PUSH] = { \"push\", SOURCE_NONE, FIELD_STR, remote_ref_atom_parser },\n-\t[ATOM_SYMREF] = { \"symref\", SOURCE_NONE, FIELD_STR, refname_atom_parser },\n+\t[ATOM_SYMREF] = { \"symref\", SOURCE_NONE, FIELD_STR, symref_atom_parser },\n \t[ATOM_FLAG] = { \"flag\", SOURCE_NONE },\n \t[ATOM_HEAD] = { \"HEAD\", SOURCE_NONE, FIELD_STR, head_atom_parser },\n \t[ATOM_COLOR] = { \"color\", SOURCE_NONE, FIELD_STR, color_atom_parser },\n@@ -748,8 +755,6 @@ static int parse_ref_filter_atom(struct ref_format *format,\n \t\treturn -1;\n \tif (deref)\n \t\tneed_tagged = 1;\n-\tif (i == ATOM_SYMREF)\n-\t\tneed_symref = 1;\n \treturn at;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"432658","messageId":"fc87a6c84a36c0da39a419aec0edf0e40dc54474.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 25/27] [GSOC] ref-filter: use switch case instread of if else","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:23:08Z","receivedAt":"2021-08-13T08:23:51Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nIn the original code, the check of the atom type is through\nif/else conditional check, using switch/case can allow the\ncompiler to reduce unnecessary branch checks and increase\nthe readability of the code.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 163 ++++++++++++++++++++++++++++++++++-----------------\n 1 file changed, 109 insertions(+), 54 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex a0de5f21ff5..6fbf38f1375 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1126,9 +1126,11 @@ static void grab_common_values(struct atom_value *val, int deref, struct expand_\n \t\tstruct atom_value *v = &val[i];\n \t\tif (!!deref != used_atom[i].deref)\n \t\t\tcontinue;\n-\t\tif (atom_type == ATOM_OBJECTTYPE)\n+\t\tswitch (atom_type) {\n+\t\tcase ATOM_OBJECTTYPE : {\n \t\t\tv->s = xstrdup(type_name(oi->type));\n-\t\telse if (atom_type == ATOM_OBJECTSIZE) {\n+\t\t\tbreak;\n+\t\t} case ATOM_OBJECTSIZE : {\n \t\t\tif (used_atom[i].u.objectsize.option == O_SIZE_DISK) {\n \t\t\t\tv->value = oi->disk_size;\n \t\t\t\tv->s = xstrfmt_len(&v->s_size, \"%\"PRIuMAX, (uintmax_t)oi->disk_size);\n@@ -1136,10 +1138,20 @@ static void grab_common_values(struct atom_value *val, int deref, struct expand_\n \t\t\t\tv->value = oi->size;\n \t\t\t\tv->s = xstrfmt_len(&v->s_size, \"%\"PRIuMAX , (uintmax_t)oi->size);\n \t\t\t}\n-\t\t} else if (atom_type == ATOM_DELTABASE)\n+\t\t\tbreak;\n+\t\t}\n+\t\tcase ATOM_DELTABASE: {\n \t\t\tv->s = xstrdup(oid_to_hex(&oi->delta_base_oid));\n-\t\telse if (atom_type == ATOM_OBJECTNAME && deref) {\n-\t\t\tv->s = xstrdup(do_grab_oid(\"objectname\", &oi->oid, &used_atom[i]));\n+\t\t\tbreak;\n+\t\t}\n+\t\tcase ATOM_OBJECTNAME: {\n+\t\t\tif (deref)\n+\t\t\t\tv->s = xstrdup(do_grab_oid(\"objectname\", &oi->oid, &used_atom[i]));\n+\t\t\tbreak;\n+\t\t}\n+\t\tdefault: {\n+\t\t\tbreak;\n+\t\t}\n \t\t}\n \t}\n }\n@@ -1154,13 +1166,22 @@ static void grab_tag_values(struct atom_value *val, int deref, struct object *ob\n \t\tenum atom_type atom_type = used_atom[i].atom_type;\n \t\tstruct atom_value *v = &val[i];\n \t\tif (!!deref != used_atom[i].deref)\n-\t\t\tcontinue;\n-\t\tif (atom_type == ATOM_TAG)\n+\t\t\tbreak;\n+\t\tswitch (atom_type) {\n+\t\tcase ATOM_TAG:\n \t\t\tv->s = xstrdup(tag->tag);\n-\t\telse if (atom_type == ATOM_TYPE && tag->tagged)\n-\t\t\tv->s = xstrdup(type_name(tag->tagged->type));\n-\t\telse if (atom_type == ATOM_OBJECT && tag->tagged)\n-\t\t\tv->s = xstrdup(oid_to_hex(&tag->tagged->oid));\n+\t\t\tbreak;\n+\t\tcase ATOM_TYPE:\n+\t\t\tif (tag->tagged)\n+\t\t\t\tv->s = xstrdup(type_name(tag->tagged->type));\n+\t\t\tbreak;\n+\t\tcase ATOM_OBJECT:\n+\t\t\tif (tag->tagged)\n+\t\t\t\tv->s = xstrdup(oid_to_hex(&tag->tagged->oid));\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tbreak;\n+\t\t}\n \t}\n }\n \n@@ -1175,15 +1196,17 @@ static void grab_commit_values(struct atom_value *val, int deref, struct object\n \t\tstruct atom_value *v = &val[i];\n \t\tif (!!deref != used_atom[i].deref)\n \t\t\tcontinue;\n-\t\tif (atom_type == ATOM_TREE) {\n+\t\tswitch (atom_type) {\n+\t\tcase ATOM_TREE: {\n \t\t\tv->s = xstrdup(do_grab_oid(\"tree\", get_commit_tree_oid(commit), &used_atom[i]));\n-\t\t\tcontinue;\n+\t\t\tbreak;\n \t\t}\n-\t\tif (atom_type == ATOM_NUMPARENT) {\n+\t\tcase ATOM_NUMPARENT: {\n \t\t\tv->value = commit_list_count(commit->parents);\n \t\t\tv->s = xstrfmt_len(&v->s_size, \"%lu\", (unsigned long)v->value);\n+\t\t\tbreak;\n \t\t}\n-\t\telse if (atom_type == ATOM_PARENT) {\n+\t\tcase ATOM_PARENT: {\n \t\t\tstruct commit_list *parents;\n \t\t\tstruct strbuf s = STRBUF_INIT;\n \t\t\tfor (parents = commit->parents; parents; parents = parents->next) {\n@@ -1193,6 +1216,11 @@ static void grab_commit_values(struct atom_value *val, int deref, struct object\n \t\t\t\tstrbuf_addstr(&s, do_grab_oid(\"parent\", oid, &used_atom[i]));\n \t\t\t}\n \t\t\tv->s = strbuf_detach(&s, NULL);\n+\t\t\tbreak;\n+\t\t}\n+\t\tdefault: {\n+\t\t\tbreak;\n+\t\t}\n \t\t}\n \t}\n }\n@@ -1900,24 +1928,40 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\tv->handler = append_atom;\n \t\tv->atom = atom;\n \n-\t\tif (atom_type == ATOM_REFNAME)\n+\t\tswitch (atom_type) {\n+\t\tcase ATOM_REFNAME: {\n \t\t\trefname = get_refname(atom, ref);\n-\t\telse if (atom_type == ATOM_WORKTREEPATH) {\n+\t\t\tif (!deref)\n+\t\t\t\tv->s = xstrdup(refname);\n+\t\t\telse\n+\t\t\t\tv->s = xstrfmt_len(&v->s_size, \"%s^{}\", refname);\n+\t\t\tfree((char *)refname);\n+\t\t\tbreak;\n+\t\t}\n+\t\tcase ATOM_WORKTREEPATH: {\n \t\t\tif (ref->kind == FILTER_REFS_BRANCHES)\n \t\t\t\tv->s = get_worktree_path(atom, ref);\n \t\t\telse\n \t\t\t\tv->s = ref_filter_slopbuf;\n-\t\t\tcontinue;\n+\t\t\tbreak;\n \t\t}\n-\t\telse if (atom_type == ATOM_SYMREF)\n+\t\tcase ATOM_SYMREF: {\n \t\t\trefname = get_symref(atom, ref);\n-\t\telse if (atom_type == ATOM_UPSTREAM) {\n+\t\t\tif (!deref)\n+\t\t\t\tv->s = xstrdup(refname);\n+\t\t\telse\n+\t\t\t\tv->s = xstrfmt_len(&v->s_size, \"%s^{}\", refname);\n+\t\t\tif (refname != ref_filter_slopbuf)\n+\t\t\t\tfree((char *)refname);\n+\t\t\tbreak;\n+\t\t}\n+\t\tcase ATOM_UPSTREAM: {\n \t\t\tconst char *branch_name;\n \t\t\t/* only local branches may have an upstream */\n \t\t\tif (!skip_prefix(ref->refname, \"refs/heads/\",\n \t\t\t\t\t &branch_name)) {\n \t\t\t\tv->s = ref_filter_slopbuf;\n-\t\t\t\tcontinue;\n+\t\t\tbreak;\n \t\t\t}\n \t\t\tbranch = branch_get(branch_name);\n \n@@ -1926,13 +1970,17 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\t\t\tfill_remote_ref_details(atom, refname, branch, v);\n \t\t\telse\n \t\t\t\tv->s = ref_filter_slopbuf;\n-\t\t\tcontinue;\n-\t\t} else if (atom_type == ATOM_PUSH && atom->u.remote_ref.push) {\n+\t\t\tbreak;\n+\t\t}\n+\t\tcase ATOM_PUSH: {\n \t\t\tconst char *branch_name;\n+\n+\t\t\tif (!atom->u.remote_ref.push)\n+\t\t\t\tbreak;\n \t\t\tv->s = ref_filter_slopbuf;\n \t\t\tif (!skip_prefix(ref->refname, \"refs/heads/\",\n \t\t\t\t\t &branch_name))\n-\t\t\t\tcontinue;\n+\t\t\t\tbreak;\n \t\t\tbranch = branch_get(branch_name);\n \n \t\t\tif (atom->u.remote_ref.push_remote)\n@@ -1940,14 +1988,16 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\t\telse {\n \t\t\t\trefname = branch_get_push(branch, NULL);\n \t\t\t\tif (!refname)\n-\t\t\t\t\tcontinue;\n+\t\t\t\t\tbreak;\n \t\t\t}\n \t\t\tfill_remote_ref_details(atom, refname, branch, v);\n-\t\t\tcontinue;\n-\t\t} else if (atom_type == ATOM_COLOR) {\n+\t\t\tbreak;\n+\t\t}\n+\t\tcase ATOM_COLOR: {\n \t\t\tv->s = xstrdup(atom->u.color);\n \t\t\tcontinue;\n-\t\t} else if (atom_type == ATOM_FLAG) {\n+\t\t}\n+\t\tcase ATOM_FLAG: {\n \t\t\tchar buf[256], *cp = buf;\n \t\t\tif (ref->flag & REF_ISSYMREF)\n \t\t\t\tcp = copy_advance(cp, \",symref\");\n@@ -1959,54 +2009,59 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\t\t\t*cp = '\\0';\n \t\t\t\tv->s = xstrdup(buf + 1);\n \t\t\t}\n-\t\t\tcontinue;\n-\t\t} else if (!deref && atom_type == ATOM_OBJECTNAME) {\n-\t\t\t v->s = xstrdup(do_grab_oid(\"objectname\", &ref->objectname, atom));\n-\t\t\t continue;\n-\t\t} else if (atom_type == ATOM_HEAD) {\n+\t\t\tbreak;\n+\t\t}\n+\t\tcase ATOM_OBJECTNAME: {\n+\t\t\tif (!deref)\n+\t\t\t\tv->s = xstrdup(do_grab_oid(\"objectname\", &ref->objectname, atom));\n+\t\t\tbreak;\n+\t\t}\n+\t\tcase ATOM_HEAD: {\n \t\t\tif (atom->u.head && !strcmp(ref->refname, atom->u.head))\n \t\t\t\tv->s = xstrdup(\"*\");\n \t\t\telse\n \t\t\t\tv->s = xstrdup(\" \");\n-\t\t\tcontinue;\n-\t\t} else if (atom_type == ATOM_ALIGN) {\n+\t\t\tbreak;\n+\t\t}\n+\t\tcase ATOM_ALIGN: {\n \t\t\tv->handler = align_atom_handler;\n \t\t\tv->s = ref_filter_slopbuf;\n-\t\t\tcontinue;\n-\t\t} else if (atom_type == ATOM_END) {\n+\t\t\tbreak;\n+\t\t}\n+\t\tcase ATOM_END: {\n \t\t\tv->handler = end_atom_handler;\n \t\t\tv->s = ref_filter_slopbuf;\n-\t\t\tcontinue;\n-\t\t} else if (atom_type == ATOM_IF) {\n+\t\t\tbreak;\n+\t\t}\n+\t\tcase ATOM_IF: {\n \t\t\tconst char *s;\n \t\t\tif (skip_prefix(name, \"if:\", &s))\n \t\t\t\tv->s = xstrdup(s);\n \t\t\telse\n \t\t\t\tv->s = ref_filter_slopbuf;\n \t\t\tv->handler = if_atom_handler;\n-\t\t\tcontinue;\n-\t\t} else if (atom_type == ATOM_THEN) {\n+\t\t\tbreak;\n+\t\t}\n+\t\tcase ATOM_THEN: {\n \t\t\tv->handler = then_atom_handler;\n \t\t\tv->s = ref_filter_slopbuf;\n-\t\t\tcontinue;\n-\t\t} else if (atom_type == ATOM_ELSE) {\n+\t\t\tbreak;\n+\t\t}\n+\t\tcase ATOM_ELSE: {\n \t\t\tv->handler = else_atom_handler;\n \t\t\tv->s = ref_filter_slopbuf;\n-\t\t\tcontinue;\n-\t\t} else if (atom_type == ATOM_REST) {\n+\t\t\tbreak;\n+\t\t}\n+\t\tcase ATOM_REST: {\n \t\t\tif (ref->rest)\n \t\t\t\tv->s = xstrdup(ref->rest);\n \t\t\telse\n \t\t\t\tv->s = ref_filter_slopbuf;\n-\t\t\tcontinue;\n-\t\t} else\n-\t\t\tcontinue;\n-\n-\t\tif (!deref)\n-\t\t\tv->s = xstrdup(refname);\n-\t\telse\n-\t\t\tv->s = xstrfmt_len(&v->s_size, \"%s^{}\", refname);\n-\t\tfree((char *)refname);\n+\t\t\tbreak;\n+\t\t}\n+\t\tdefault:\n+\t\t\tbreak;\n+\t\t}\n \t}\n \n \tfor (i = 0; i < used_atom_cnt; i++) {\n-- \ngitgitgadget\n\n"},{"id":"432659","messageId":"8591897fbce05e9987a6ee9af157471a3fc9359f.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 27/27] [GSOC] ref-filter: add need_get_object_info flag to struct expand_data","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:23:10Z","receivedAt":"2021-08-13T08:23:52Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nIn populate_value() we use an empty struct object_info\ncompare with oi and oi_deref, which determine if we can\nreturn early, this is actually very inefficient.\n\nSo add a `need_get_object_info` flag to struct expand_data,\nwhich used to indicate whether we need call get_object()\nor we can return early.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 46 ++++++++++++++++++++++++++++++++--------------\n 1 file changed, 32 insertions(+), 14 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 4d7aa60da0f..edcb5d04cf9 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -77,6 +77,7 @@ static struct ref_trailer_buf {\n } ref_trailer_buf = {STRING_LIST_INIT_NODUP, STRBUF_INIT, STRBUF_INIT};\n \n static struct expand_data {\n+\tunsigned int need_get_object_info : 1;\n \tstruct object_id oid;\n \tenum object_type type;\n \tunsigned long size;\n@@ -322,10 +323,13 @@ static int objecttype_atom_parser(struct ref_format *format, struct used_atom *a\n {\n \tif (arg)\n \t\treturn strbuf_addf_ret(err, -1, _(\"%%(objecttype) does not take arguments\"));\n-\tif (atom->deref)\n+\tif (atom->deref) {\n \t\toi_deref.info.typep = &oi_deref.type;\n-\telse\n+\t\toi_deref.need_get_object_info = 1;\n+\t} else {\n \t\toi.info.typep = &oi.type;\n+\t\toi.need_get_object_info = 1;\n+\t}\n \treturn 0;\n }\n \n@@ -334,16 +338,23 @@ static int objectsize_atom_parser(struct ref_format *format, struct used_atom *a\n {\n \tif (!arg) {\n \t\tatom->u.objectsize.option = O_SIZE;\n-\t\tif (atom->deref)\n+\t\tif (atom->deref) {\n \t\t\toi_deref.info.sizep = &oi_deref.size;\n-\t\telse\n+\t\t\toi_deref.need_get_object_info = 1;\n+\t\t} else {\n \t\t\toi.info.sizep = &oi.size;\n+\t\t\toi.need_get_object_info = 1;\n+\t\t}\n \t} else if (!strcmp(arg, \"disk\")) {\n \t\tatom->u.objectsize.option = O_SIZE_DISK;\n-\t\tif (atom->deref)\n+\t\tif (atom->deref) {\n \t\t\toi_deref.info.disk_sizep = &oi_deref.disk_size;\n-\t\telse\n+\t\t\toi_deref.need_get_object_info = 1;\n+\t\t}\n+\t\telse {\n \t\t\toi.info.disk_sizep = &oi.disk_size;\n+\t\t\toi.need_get_object_info = 1;\n+\t\t}\n \t} else\n \t\treturn strbuf_addf_ret(err, -1, _(\"unrecognized %%(objectsize) argument: %s\"), arg);\n \treturn 0;\n@@ -354,10 +365,14 @@ static int deltabase_atom_parser(struct ref_format *format, struct used_atom *at\n {\n \tif (arg)\n \t\treturn strbuf_addf_ret(err, -1, _(\"%%(deltabase) does not take arguments\"));\n-\tif (atom->deref)\n+\tif (atom->deref) {\n \t\toi_deref.info.delta_base_oid = &oi_deref.delta_base_oid;\n-\telse\n+\t\toi_deref.need_get_object_info = 1;\n+\n+\t} else {\n \t\toi.info.delta_base_oid = &oi.delta_base_oid;\n+\t\toi.need_get_object_info = 1;\n+\t}\n \treturn 0;\n }\n \n@@ -735,10 +750,13 @@ static int parse_ref_filter_atom(struct ref_format *format,\n \tused_atom[at].type = valid_atom[i].cmp_type;\n \tused_atom[at].source = valid_atom[i].source;\n \tif (used_atom[at].source == SOURCE_OBJ) {\n-\t\tif (deref)\n+\t\tif (deref) {\n \t\t\toi_deref.info.contentp = &oi_deref.content;\n-\t\telse\n+\t\t\toi_deref.need_get_object_info = 1;\n+\t\t} else {\n \t\t\toi.info.contentp = &oi.content;\n+\t\t\toi.need_get_object_info = 1;\n+\t\t}\n \t}\n \tif (arg) {\n \t\targ = used_atom[at].name + (arg - sp) + 1;\n@@ -1910,7 +1928,6 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \tstruct object *obj;\n \tint i;\n \tint ret;\n-\tstruct object_info empty = OBJECT_INFO_INIT;\n \n \tCALLOC_ARRAY(ref->value, used_atom_cnt);\n \n@@ -2078,10 +2095,11 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)\n \t\t\t\t\t oid_to_hex(&ref->objectname), ref->refname);\n \t}\n \n-\tif (need_tagged)\n+\tif (need_tagged) {\n \t\toi.info.contentp = &oi.content;\n-\tif (!memcmp(&oi.info, &empty, sizeof(empty)) &&\n-\t !memcmp(&oi_deref.info, &empty, sizeof(empty)))\n+\t\toi.need_get_object_info = 1;\n+\t}\n+\tif (!oi.need_get_object_info && !oi_deref.need_get_object_info)\n \t\treturn 0;\n \n \n-- \ngitgitgadget\n"},{"id":"432660","messageId":"d7af20a228b91d7990214ebe4fe7a5d5ee44d02b.1628842990.git.gitgitgadget@gmail.com","threadId":"56274","inReplyTo":"pull.1016.git.1628842990.gitgitgadget@gmail.com","subject":"[PATCH 26/27] [GSOC] ref-filter: reuse finnal buffer if no stack need","fromName":"ZheNing Hu via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-13T08:23:09Z","receivedAt":"2021-08-13T08:23:53Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"From: ZheNing Hu \n\nBecause finnal_buf will be cleared before every time\nwe call format_ref_array_item(). So we actually add\ncontent to an empty strbuf, We can add the object's\ndata directly to this finnal_buffer instead of adding\nobjects' data to state.stack->output first, then\ncopying to finnal_buf.\n\nSo add a can_reuse_final_buffer flag to struct\nref_format to check if we are use %(align), %(end),\n%(if), %(then), %(else). if not, we can reuse the buf\nof finnal_buf.\n\nThis can bring 2% performance improvement for\ngit cat-file --batch.\n\nMentored-by: Christian Couder \nMentored-by: Hariom Verma \nSigned-off-by: ZheNing Hu \n---\n ref-filter.c | 36 +++++++++++++++++++++++++++++-------\n ref-filter.h | 3 ++-\n 2 files changed, 31 insertions(+), 8 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 6fbf38f1375..4d7aa60da0f 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1083,6 +1083,13 @@ int verify_ref_format(struct ref_format *format)\n \t\tif (reject_atom(format->cat_file_mode, used_atom[at].atom_type))\n \t\t\tdie(_(\"this command reject atom %%(%.*s)\"), (int)(ep - sp - 2), sp + 2);\n \n+\t\tif (used_atom[at].atom_type == ATOM_ALIGN ||\n+\t\t used_atom[at].atom_type == ATOM_END ||\n+\t\t used_atom[at].atom_type == ATOM_IF ||\n+\t\t used_atom[at].atom_type == ATOM_THEN ||\n+\t\t used_atom[at].atom_type == ATOM_ELSE)\n+\t\t\tformat->can_reuse_final_buffer = 0;\n+\n \t\tif ((format->quote_style == QUOTE_PYTHON ||\n \t\t format->quote_style == QUOTE_SHELL ||\n \t\t format->quote_style == QUOTE_TCL) &&\n@@ -2710,7 +2717,14 @@ int format_ref_array_item(struct ref_array_item *info,\n \tint ret;\n \n \tstate.quote_style = format->quote_style;\n-\tpush_stack_element(&state.stack);\n+\tif (format->can_reuse_final_buffer) {\n+\t\tstruct ref_formatting_stack *s = xmalloc(sizeof(struct ref_formatting_stack));\n+\t\ts->output = *final_buf;\n+\t\ts->prev = state.stack;\n+\t\tstate.stack = s;\n+\t} else {\n+\t\tpush_stack_element(&state.stack);\n+\t}\n \tinfo->can_skip_parse_buffer = format->can_skip_parse_buffer;\n \n \tcp = format->format;\n@@ -2722,9 +2736,10 @@ int format_ref_array_item(struct ref_array_item *info,\n \n \t\tif (cp < e->beg - 2)\n \t\t\tappend_literal(cp, e->beg - 2, &state);\n-\t\tif ((ret = get_ref_atom_value(info, e->at, &atomv, error_buf)) ||\n-\t\t atomv->handler(atomv, &state, error_buf)) {\n-\t\t\tpop_stack_element(&state.stack);\n+\t\tif (((ret = get_ref_atom_value(info, e->at, &atomv, error_buf)) ||\n+\t\t atomv->handler(atomv, &state, error_buf))) {\n+\t\t\tif (!format->can_reuse_final_buffer)\n+\t\t\t\tpop_stack_element(&state.stack);\n \t\t\treturn ret ? ret : -1;\n \t\t}\n \t\tcp = e->end + 1;\n@@ -2739,16 +2754,23 @@ int format_ref_array_item(struct ref_array_item *info,\n \t\tstruct atom_value resetv = ATOM_VALUE_INIT;\n \t\tresetv.s = GIT_COLOR_RESET;\n \t\tif (append_atom(&resetv, &state, error_buf)) {\n-\t\t\tpop_stack_element(&state.stack);\n+\t\t\tif (!format->can_reuse_final_buffer)\n+\t\t\t\tpop_stack_element(&state.stack);\n \t\t\treturn -1;\n \t\t}\n \t}\n \tif (state.stack->prev) {\n+\t\tassert(!format->can_reuse_final_buffer);\n \t\tpop_stack_element(&state.stack);\n \t\treturn strbuf_addf_ret(error_buf, -1, _(\"format: %%(end) atom missing\"));\n \t}\n-\tstrbuf_addbuf(final_buf, &state.stack->output);\n-\tpop_stack_element(&state.stack);\n+\tif(format->can_reuse_final_buffer) {\n+\t\t*final_buf = state.stack->output;\n+\t\tfree(state.stack);\n+\t} else {\n+\t\tstrbuf_addbuf(final_buf, &state.stack->output);\n+\t\tpop_stack_element(&state.stack);\n+\t}\n \treturn 0;\n }\n \ndiff --git a/ref-filter.h b/ref-filter.h\nindex 2b871f9ed4f..bf1d2fa18c6 100644\n--- a/ref-filter.h\n+++ b/ref-filter.h\n@@ -94,10 +94,11 @@ struct ref_format {\n \tint can_skip_parse_buffer;\n \t/* Internal state to ref-filter */\n \tint need_color_reset_at_eol;\n+\tint can_reuse_final_buffer;\n \tstruct list_head parsed_atom_head;\n };\n \n-#define REF_FORMAT_INIT { .use_color = -1, .can_skip_parse_buffer = 1 }\n+#define REF_FORMAT_INIT { .use_color = -1, .can_skip_parse_buffer = 1, .can_reuse_final_buffer = 1 }\n \n /* Macros for checking --merged and --no-merged options */\n #define _OPT_MERGED_NO_MERGED(option, filter, h) \\\n-- \ngitgitgadget\n\n"}]}