{"thread":{"id":"51911","subject":"[PATCH 0/8] fast export/import: handle nested tags, improve incremental exports","startedAt":"2019-09-25T01:40:14Z","lastAt":"2019-10-04T05:52:42Z","messageCount":33,"participants":["Elijah Newren","Junio C Hamano","René Scharfe"],"isPatch":true,"patchVersion":1,"patchTotal":8},"messages":[{"id":"382882","messageId":"20190925014005.17056-1-newren@gmail.com","threadId":"51911","inReplyTo":null,"subject":"[PATCH 0/8] fast export/import: handle nested tags, improve incremental exports","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-09-25T01:39:57Z","receivedAt":"2019-09-25T01:40:14Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"This series improves the incremental export story for fast-export and\nfast-import (--export-marks and --import-marks fell a bit short),\nfixes a couple small export/import bugs, and enables handling nested\ntags. In particular, the nested tags handling makes it so that\nfast-export and fast-import can finally handle the git.git repo.\n\nElijah Newren (8):\n fast-export: fix exporting a tag and nothing else\n fast-import: fix handling of deleted tags\n fast-import: allow tags to be identified by mark labels\n fast-import: add support for new 'alias' command\n fast-export: add support for --import-marks-if-exists\n fast-export: allow user to request tags be marked with --mark-tags\n t9350: add tests for tags of things other than a commit\n fast-export: handle nested tags\n\n Documentation/git-fast-export.txt | 17 ++++--\n Documentation/git-fast-import.txt | 23 ++++++++\n builtin/fast-export.c | 67 ++++++++++++++++------\n fast-import.c | 94 +++++++++++++++++++++++++++----\n t/t9300-fast-import.sh | 37 ++++++++++++\n t/t9350-fast-export.sh | 68 ++++++++++++++++++++--\n 6 files changed, 268 insertions(+), 38 deletions(-)\n\n-- \n2.23.0.177.g8af0b3ca64\n"},{"id":"382883","messageId":"20190925014005.17056-2-newren@gmail.com","threadId":"51911","inReplyTo":"20190925014005.17056-1-newren@gmail.com","subject":"[PATCH 1/8] fast-export: fix exporting a tag and nothing else","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-09-25T01:39:58Z","receivedAt":"2019-09-25T01:40:24Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"fast-export allows specifying revision ranges, which can be used to\nexport a tag without exporting the commit it tags. fast-export handled\nthis rather poorly: it would emit a \"from :0\" directive. Since marks\nstart at 1 and increase, this means it refers to an unknown commit and\nfast-import will choke on the input.\n\nWhen we are unable to look up a mark for the object being tagged, use a\n\"from $HASH\" directive instead to fix this problem.\n\nNote that this is quite similar to the behavior fast-export exhibits\nwith commits and parents when --reference-excluded-parents is passed\nalong with an excluded commit range. For tags of excluded commits we do\nnot require the --reference-excluded-parents flag because we always have\nto tag something. By contrast, when dealing with commits, pruning a\nparent is always a viable option, so we need the flag to specify that\nparent pruning is not wanted. (It is slightly weird that\n--reference-excluded-parents isn't the default with a separate\n--prune-excluded-parents flag, but backward compatibility concerns\nresulted in the current defaults.)\n\nSigned-off-by: Elijah Newren \n---\n builtin/fast-export.c | 7 ++++++-\n t/t9350-fast-export.sh | 13 +++++++++++++\n 2 files changed, 19 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/fast-export.c b/builtin/fast-export.c\nindex f541f55d33..5822271c6b 100644\n--- a/builtin/fast-export.c\n+++ b/builtin/fast-export.c\n@@ -860,7 +860,12 @@ static void handle_tag(const char *name, struct tag *tag)\n \n \tif (starts_with(name, \"refs/tags/\"))\n \t\tname += 10;\n-\tprintf(\"tag %s\\nfrom :%d\\n\", name, tagged_mark);\n+\tprintf(\"tag %s\\n\", name);\n+\tif (tagged_mark)\n+\t\tprintf(\"from :%d\\n\", tagged_mark);\n+\telse\n+\t\tprintf(\"from %s\\n\", oid_to_hex(&tagged->oid));\n+\n \tif (show_original_ids)\n \t\tprintf(\"original-oid %s\\n\", oid_to_hex(&tag->object.oid));\n \tprintf(\"%.*s%sdata %d\\n%.*s\\n\",\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex b4004e05c2..d32ff41859 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -53,6 +53,19 @@ test_expect_success 'fast-export | fast-import' '\n \n '\n \n+test_expect_success 'fast-export ^muss^{commit} muss' '\n+\tgit fast-export --tag-of-filtered-object=rewrite ^muss^{commit} muss >actual &&\n+\tcat >expected <<-EOF &&\n+\ttag muss\n+\tfrom $(git rev-parse --verify muss^{commit})\n+\t$(git cat-file tag muss | grep tagger)\n+\tdata 9\n+\tvalentin\n+\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'fast-export master~2..master' '\n \n \tgit fast-export master~2..master >actual &&\n-- \n2.23.0.177.g8af0b3ca64\n\n"},{"id":"382884","messageId":"20190925014005.17056-3-newren@gmail.com","threadId":"51911","inReplyTo":"20190925014005.17056-1-newren@gmail.com","subject":"[PATCH 2/8] fast-import: fix handling of deleted tags","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-09-25T01:39:59Z","receivedAt":"2019-09-25T01:40:24Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"If our input stream includes a tag which is later deleted, we were not\nproperly deleting it. We did have a step which would delete it, but we\nleft a tag in the tag list noting that it needed to be updated, and the\nupdating of annotated tags occurred AFTER ref deletion. So, when we\nrecord that a tag needs to be deleted, also remove it from the list of\nannotated tags to update.\n\nWhile this has likely been something that has not happened in practice,\nit will come up more in order to support nested tags. For nested tags,\nwe either need to give temporary names to the intermediate tags and then\ndelete them, or else we need to use the final name for the intermediate\ntags. If we use the final name for the intermediate tags, then in order\nto keep the sanity check that someone doesn't try to update the same tag\ntwice, we need to delete the ref after creating the intermediate tag.\nSo, either way nested tags imply the need to delete temporary inner tag\nreferences.\n\nSigned-off-by: Elijah Newren \n---\n fast-import.c | 29 +++++++++++++++++++++++++++++\n t/t9300-fast-import.sh | 13 +++++++++++++\n 2 files changed, 42 insertions(+)\n\ndiff --git a/fast-import.c b/fast-import.c\nindex b44d6a467e..dab905d667 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -2793,6 +2793,35 @@ static void parse_reset_branch(const char *arg)\n \t\tb = new_branch(arg);\n \tread_next_command();\n \tparse_from(b);\n+\tif (b->delete && !strncmp(arg, \"refs/tags/\", 10)) {\n+\t\t/*\n+\t\t * Elsewhere, we call dump_branches() before dump_tags(),\n+\t\t * and dump_branches() will handle ref deletions first, so\n+\t\t * in order to make sure the deletion actually takes effect,\n+\t\t * we need to remove the tag from our list of tags to update.\n+\t\t *\n+\t\t * NEEDSWORK: replace list of tags with hashmap for faster\n+\t\t * deletion?\n+\t\t */\n+\t\tstruct strbuf tag_name = STRBUF_INIT;\n+\t\tstruct tag *t, *prev = NULL;\n+\t\tfor (t = first_tag; t; t = t->next_tag) {\n+\t\t\tstrbuf_reset(&tag_name);\n+\t\t\tstrbuf_addf(&tag_name, \"refs/tags/%s\", t->name);\n+\t\t\tif (!strcmp(arg, tag_name.buf))\n+\t\t\t\tbreak;\n+\t\t\tprev = t;\n+\t\t}\n+\t\tif (t) {\n+\t\t\tif (prev)\n+\t\t\t\tprev->next_tag = t->next_tag;\n+\t\t\telse\n+\t\t\t\tfirst_tag = t->next_tag;\n+\t\t\tif (!t->next_tag)\n+\t\t\t\tlast_tag = prev;\n+\t\t\t/* There is no mem_pool_free(t) function to call. */\n+\t\t}\n+\t}\n \tif (command_buf.len > 0)\n \t\tunread_command_buf = 1;\n }\ndiff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh\nindex 141b7fa35e..74bc41333b 100755\n--- a/t/t9300-fast-import.sh\n+++ b/t/t9300-fast-import.sh\n@@ -85,6 +85,15 @@ test_expect_success 'A: create pack from stdin' '\n \tAn annotated tag that annotates a blob.\n \tEOF\n \n+\ttag to-be-deleted\n+\tfrom :3\n+\tdata <expect <<-EOF &&\n \t:2 $(git rev-parse --verify master:file2)\n-- \n2.23.0.177.g8af0b3ca64\n\n"},{"id":"382885","messageId":"20190925014005.17056-4-newren@gmail.com","threadId":"51911","inReplyTo":"20190925014005.17056-1-newren@gmail.com","subject":"[PATCH 3/8] fast-import: allow tags to be identified by mark labels","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-09-25T01:40:00Z","receivedAt":"2019-09-25T01:40:24Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Mark identifiers are used in fast-export and fast-import to provide a\nlabel to refer to earlier content. Blobs are given labels because they\nneed to be referenced in the commits where they first appear with a\ngiven filename, and commits are given labels because they can be the\nparents of other commits. Tags were never given labels, probably\nbecause they were viewed as unnecessary, but that presents two problems:\n\n 1. It leaves us without a way of referring to previous tags if we\n want to create a tag of a tag (or higher nestings).\n 2. It leaves us with no way of recording that a tag has already been\n imported when using --export-marks and --import-marks.\n\nFix these problems by allowing an optional mark label for tags.\n\nSigned-off-by: Elijah Newren \n---\n Documentation/git-fast-import.txt | 1 +\n fast-import.c | 3 ++-\n t/t9300-fast-import.sh | 19 +++++++++++++++++++\n 3 files changed, 22 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt\nindex 0bb276269e..4977869465 100644\n--- a/Documentation/git-fast-import.txt\n+++ b/Documentation/git-fast-import.txt\n@@ -774,6 +774,7 @@ lightweight (non-annotated) tags see the `reset` command below.\n \n ....\n \t'tag' SP LF\n+\tmark?\n \t'from' SP LF\n \toriginal-oid?\n \t'tagger' (SP )? SP LT GT SP LF\ndiff --git a/fast-import.c b/fast-import.c\nindex dab905d667..0271d81d0d 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -2713,6 +2713,7 @@ static void parse_new_tag(const char *arg)\n \t\tfirst_tag = t;\n \tlast_tag = t;\n \tread_next_command();\n+\tparse_mark();\n \n \t/* from ... */\n \tif (!skip_prefix(command_buf.buf, \"from \", &from))\n@@ -2769,7 +2770,7 @@ static void parse_new_tag(const char *arg)\n \tstrbuf_addbuf(&new_data, &msg);\n \tfree(tagger);\n \n-\tif (store_object(OBJ_TAG, &new_data, NULL, &t->oid, 0))\n+\tif (store_object(OBJ_TAG, &new_data, NULL, &t->oid, next_mark))\n \t\tt->pack_id = MAX_PACK_ID;\n \telse\n \t\tt->pack_id = pack_id;\ndiff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh\nindex 74bc41333b..3ad2b2f1ba 100755\n--- a/t/t9300-fast-import.sh\n+++ b/t/t9300-fast-import.sh\n@@ -94,6 +94,23 @@ test_expect_success 'A: create pack from stdin' '\n \treset refs/tags/to-be-deleted\n \tfrom 0000000000000000000000000000000000000000\n \n+\ttag nested\n+\tmark :6\n+\tfrom :4\n+\tdata <\n---\n Documentation/git-fast-import.txt | 22 +++++++++++\n fast-import.c | 62 ++++++++++++++++++++++++++-----\n t/t9300-fast-import.sh | 5 +++\n 3 files changed, 79 insertions(+), 10 deletions(-)\n\ndiff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt\nindex 4977869465..a3f1e0c5e4 100644\n--- a/Documentation/git-fast-import.txt\n+++ b/Documentation/git-fast-import.txt\n@@ -337,6 +337,13 @@ and control the current import process. More detailed discussion\n \t`commit` command. This command is optional and is not\n \tneeded to perform an import.\n \n+`alias`::\n+\tRecord that a mark refers to a given object without first\n+\tcreating any new object. Using --import-marks and referring\n+\tto missing marks will cause fast-import to fail, so aliases\n+\tcan provide a way to set otherwise pruned commits to a valid\n+\tvalue (e.g. the nearest non-pruned ancestor).\n+\n `checkpoint`::\n \tForces fast-import to close the current packfile, generate its\n \tunique SHA-1 checksum and index, and start a new packfile.\n@@ -914,6 +921,21 @@ a data chunk which does not have an LF as its last byte.\n +\n The `LF` after ` LF` is optional (it used to be required).\n \n+`alias`\n+~~~~~~~\n+Record that a mark refers to a given object without first creating any\n+new object.\n+\n+....\n+\t'alias' LF\n+\tmark\n+\t'to' SP LF\n+\tLF?\n+....\n+\n+For a detailed description of `` see above under `from`.\n+\n+\n `checkpoint`\n ~~~~~~~~~~~~\n Forces fast-import to close the current packfile, start a new one, and to\ndiff --git a/fast-import.c b/fast-import.c\nindex 0271d81d0d..8228cde759 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -2491,18 +2491,14 @@ static void parse_from_existing(struct branch *b)\n \t}\n }\n \n-static int parse_from(struct branch *b)\n+static int parse_objectish(struct branch *b, const char *objectish)\n {\n-\tconst char *from;\n \tstruct branch *s;\n \tstruct object_id oid;\n \n-\tif (!skip_prefix(command_buf.buf, \"from \", &from))\n-\t\treturn 0;\n-\n \toidcpy(&oid, &b->branch_tree.versions[1].oid);\n \n-\ts = lookup_branch(from);\n+\ts = lookup_branch(objectish);\n \tif (b == s)\n \t\tdie(\"Can't create a branch from itself: %s\", b->name);\n \telse if (s) {\n@@ -2510,8 +2506,8 @@ static int parse_from(struct branch *b)\n \t\toidcpy(&b->oid, &s->oid);\n \t\toidcpy(&b->branch_tree.versions[0].oid, t);\n \t\toidcpy(&b->branch_tree.versions[1].oid, t);\n-\t} else if (*from == ':') {\n-\t\tuintmax_t idnum = parse_mark_ref_eol(from);\n+\t} else if (*objectish == ':') {\n+\t\tuintmax_t idnum = parse_mark_ref_eol(objectish);\n \t\tstruct object_entry *oe = find_mark(idnum);\n \t\tif (oe->type != OBJ_COMMIT)\n \t\t\tdie(\"Mark :%\" PRIuMAX \" not a commit\", idnum);\n@@ -2525,13 +2521,13 @@ static int parse_from(struct branch *b)\n \t\t\t} else\n \t\t\t\tparse_from_existing(b);\n \t\t}\n-\t} else if (!get_oid(from, &b->oid)) {\n+\t} else if (!get_oid(objectish, &b->oid)) {\n \t\tparse_from_existing(b);\n \t\tif (is_null_oid(&b->oid))\n \t\t\tb->delete = 1;\n \t}\n \telse\n-\t\tdie(\"Invalid ref name or SHA1 expression: %s\", from);\n+\t\tdie(\"Invalid ref name or SHA1 expression: %s\", objectish);\n \n \tif (b->branch_tree.tree && !oideq(&oid, &b->branch_tree.versions[1].oid)) {\n \t\trelease_tree_content_recursive(b->branch_tree.tree);\n@@ -2542,6 +2538,26 @@ static int parse_from(struct branch *b)\n \treturn 1;\n }\n \n+static int parse_from(struct branch *b)\n+{\n+\tconst char *from;\n+\n+\tif (!skip_prefix(command_buf.buf, \"from \", &from))\n+\t\treturn 0;\n+\n+\treturn parse_objectish(b, from);\n+}\n+\n+static int parse_objectish_with_prefix(struct branch *b, const char *prefix)\n+{\n+\tconst char *base;\n+\n+\tif (!skip_prefix(command_buf.buf, prefix, &base))\n+\t\treturn 0;\n+\n+\treturn parse_objectish(b, base);\n+}\n+\n static struct hash_list *parse_merge(unsigned int *count)\n {\n \tstruct hash_list *list = NULL, **tail = &list, *n;\n@@ -3089,6 +3105,28 @@ static void parse_progress(void)\n \tskip_optional_lf();\n }\n \n+static void parse_alias(void)\n+{\n+\tstruct object_entry *e;\n+\tstruct branch b;\n+\n+\tskip_optional_lf();\n+\tread_next_command();\n+\n+\t/* mark ... */\n+\tparse_mark();\n+\tif (!next_mark)\n+\t\tdie(_(\"Expected 'mark' command, got %s\"), command_buf.buf);\n+\n+\t/* to ... */\n+\tmemset(&b, 0, sizeof(b));\n+\tif (!parse_objectish_with_prefix(&b, \"to \"))\n+\t\tdie(_(\"Expected 'to' command, got %s\"), command_buf.buf);\n+\te = find_object(&b.oid);\n+\tassert(e);\n+\tinsert_mark(next_mark, e);\n+}\n+\n static char* make_fast_import_path(const char *path)\n {\n \tif (!relative_marks_paths || is_absolute_path(path))\n@@ -3216,6 +3254,8 @@ static int parse_one_feature(const char *feature, int from_stream)\n \t\toption_import_marks(arg, from_stream, 1);\n \t} else if (skip_prefix(feature, \"export-marks=\", &arg)) {\n \t\toption_export_marks(arg);\n+\t} else if (!strcmp(feature, \"alias\")) {\n+\t\t; /* Don't die - this feature is supported */\n \t} else if (!strcmp(feature, \"get-mark\")) {\n \t\t; /* Don't die - this feature is supported */\n \t} else if (!strcmp(feature, \"cat-blob\")) {\n@@ -3372,6 +3412,8 @@ int cmd_main(int argc, const char **argv)\n \t\t\tparse_checkpoint();\n \t\telse if (!strcmp(\"done\", command_buf.buf))\n \t\t\tbreak;\n+\t\telse if (!strcmp(\"alias\", command_buf.buf))\n+\t\t\tparse_alias();\n \t\telse if (starts_with(command_buf.buf, \"progress \"))\n \t\t\tparse_progress();\n \t\telse if (skip_prefix(command_buf.buf, \"feature \", &v))\ndiff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh\nindex 3ad2b2f1ba..41f2a1dad9 100755\n--- a/t/t9300-fast-import.sh\n+++ b/t/t9300-fast-import.sh\n@@ -111,6 +111,10 @@ test_expect_success 'A: create pack from stdin' '\n \tTag of tag of our lovely commit\n \tEOF\n \n+\talias\n+\tmark :8\n+\tto :5\n+\n \tINPUT_END\n \tgit fast-import --export-marks=marks.out \n---\n Documentation/git-fast-export.txt | 17 +++++++++++++----\n builtin/fast-export.c | 7 +++++++\n t/t9350-fast-export.sh | 14 ++++++++++++++\n 3 files changed, 34 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt\nindex cc940eb9ad..c522b34f7b 100644\n--- a/Documentation/git-fast-export.txt\n+++ b/Documentation/git-fast-export.txt\n@@ -75,11 +75,20 @@ produced incorrect results if you gave these options.\n \tBefore processing any input, load the marks specified in\n \t. The input file must exist, must be readable, and\n \tmust use the same format as produced by --export-marks.\n+\n+--mark-tags::\n+\tIn addition to labelling blobs and commits with mark ids, also\n+\tlabel tags. This is useful in conjunction with\n+\t`--export-marks` and `--import-marks`, and is also useful (and\n+\tnecessary) for exporting of nested tags. It does not hurt\n+\tother cases and would be the default, but many fast-import\n+\tfrontends are not prepared to accept tags with mark\n+\tidentifiers.\n +\n-Any commits that have already been marked will not be exported again.\n-If the backend uses a similar --import-marks file, this allows for\n-incremental bidirectional exporting of the repository by keeping the\n-marks the same across runs.\n+Any commits (or tags) that have already been marked will not be\n+exported again. If the backend uses a similar --import-marks file,\n+this allows for incremental bidirectional exporting of the repository\n+by keeping the marks the same across runs.\n \n --fake-missing-tagger::\n \tSome old repositories have tags without a tagger. The\ndiff --git a/builtin/fast-export.c b/builtin/fast-export.c\nindex 575e47833b..d32e1e9327 100644\n--- a/builtin/fast-export.c\n+++ b/builtin/fast-export.c\n@@ -40,6 +40,7 @@ static int no_data;\n static int full_tree;\n static int reference_excluded_commits;\n static int show_original_ids;\n+static int mark_tags;\n static struct string_list extra_refs = STRING_LIST_INIT_NODUP;\n static struct string_list tag_refs = STRING_LIST_INIT_NODUP;\n static struct refspec refspecs = REFSPEC_INIT_FETCH;\n@@ -861,6 +862,10 @@ static void handle_tag(const char *name, struct tag *tag)\n \tif (starts_with(name, \"refs/tags/\"))\n \t\tname += 10;\n \tprintf(\"tag %s\\n\", name);\n+\tif (mark_tags) {\n+\t\tmark_next_object(&tag->object);\n+\t\tprintf(\"mark :%\"PRIu32\"\\n\", last_idnum);\n+\t}\n \tif (tagged_mark)\n \t\tprintf(\"from :%d\\n\", tagged_mark);\n \telse\n@@ -1165,6 +1170,8 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \t\t\t &reference_excluded_commits, N_(\"Reference parents which are not in fast-export stream by object id\")),\n \t\tOPT_BOOL(0, \"show-original-ids\", &show_original_ids,\n \t\t\t N_(\"Show original object ids of blobs/commits\")),\n+\t\tOPT_BOOL(0, \"mark-tags\", &mark_tags,\n+\t\t\t N_(\"Label tags with mark ids\")),\n \n \t\tOPT_END()\n \t};\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex ea84e2f173..b3fca6ffba 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -66,6 +66,20 @@ test_expect_success 'fast-export ^muss^{commit} muss' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'fast-export --mark-tags ^muss^{commit} muss' '\n+\tgit fast-export --mark-tags --tag-of-filtered-object=rewrite ^muss^{commit} muss >actual &&\n+\tcat >expected <<-EOF &&\n+\ttag muss\n+\tmark :1\n+\tfrom $(git rev-parse --verify muss^{commit})\n+\t$(git cat-file tag muss | grep tagger)\n+\tdata 9\n+\tvalentin\n+\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'fast-export master~2..master' '\n \n \tgit fast-export master~2..master >actual &&\n-- \n2.23.0.177.g8af0b3ca64\n\n"},{"id":"382888","messageId":"20190925014005.17056-6-newren@gmail.com","threadId":"51911","inReplyTo":"20190925014005.17056-1-newren@gmail.com","subject":"[PATCH 5/8] fast-export: add support for --import-marks-if-exists","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-09-25T01:40:02Z","receivedAt":"2019-09-25T01:40:25Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"fast-import has support for both an --import-marks flag and an\n--import-marks-if-exists flag; the latter of which will not die() if the\nfile does not exist. fast-export only had support for an --import-marks\nflag; add an --import-marks-if-exists flag for consistency.\n\nSigned-off-by: Elijah Newren \n---\n builtin/fast-export.c | 23 +++++++++++++++++++----\n t/t9350-fast-export.sh | 10 ++++------\n 2 files changed, 23 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/fast-export.c b/builtin/fast-export.c\nindex 5822271c6b..575e47833b 100644\n--- a/builtin/fast-export.c\n+++ b/builtin/fast-export.c\n@@ -1052,11 +1052,16 @@ static void export_marks(char *file)\n \t\terror(\"Unable to write marks file %s.\", file);\n }\n \n-static void import_marks(char *input_file)\n+static void import_marks(char *input_file, int check_exists)\n {\n \tchar line[512];\n-\tFILE *f = xfopen(input_file, \"r\");\n+\tFILE *f;\n+\tstruct stat sb;\n+\n+\tif (check_exists && stat(input_file, &sb))\n+\t\treturn;\n \n+\tf = xfopen(input_file, \"r\");\n \twhile (fgets(line, sizeof(line), f)) {\n \t\tuint32_t mark;\n \t\tchar *line_end, *mark_end;\n@@ -1120,7 +1125,9 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \tstruct rev_info revs;\n \tstruct object_array commits = OBJECT_ARRAY_INIT;\n \tstruct commit *commit;\n-\tchar *export_filename = NULL, *import_filename = NULL;\n+\tchar *export_filename = NULL,\n+\t *import_filename = NULL,\n+\t *import_filename_if_exists = NULL;\n \tuint32_t lastimportid;\n \tstruct string_list refspecs_list = STRING_LIST_INIT_NODUP;\n \tstruct string_list paths_of_changed_objects = STRING_LIST_INIT_DUP;\n@@ -1140,6 +1147,10 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \t\t\t N_(\"Dump marks to this file\")),\n \t\tOPT_STRING(0, \"import-marks\", &import_filename, N_(\"file\"),\n \t\t\t N_(\"Import marks from this file\")),\n+\t\tOPT_STRING(0, \"import-marks-if-exists\",\n+\t\t\t &import_filename_if_exists,\n+\t\t\t N_(\"file\"),\n+\t\t\t N_(\"Import marks from this file if it exists\")),\n \t\tOPT_BOOL(0, \"fake-missing-tagger\", &fake_missing_tagger,\n \t\t\t N_(\"Fake a tagger when tags lack one\")),\n \t\tOPT_BOOL(0, \"full-tree\", &full_tree,\n@@ -1187,8 +1198,12 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \tif (use_done_feature)\n \t\tprintf(\"feature done\\n\");\n \n+\tif (import_filename && import_filename_if_exists)\n+\t\tdie(_(\"Cannot pass both --import-marks and --import-marks-if-exists\"));\n \tif (import_filename)\n-\t\timport_marks(import_filename);\n+\t\timport_marks(import_filename, 0);\n+\telse if (import_filename_if_exists)\n+\t\timport_marks(import_filename_if_exists, 1);\n \tlastimportid = last_idnum;\n \n \tif (import_filename && revs.prune_data.nr)\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex d32ff41859..ea84e2f173 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -580,17 +580,15 @@ test_expect_success 'fast-export quotes pathnames' '\n '\n \n test_expect_success 'test bidirectionality' '\n-\t>marks-cur &&\n-\t>marks-new &&\n \tgit init marks-test &&\n-\tgit fast-export --export-marks=marks-cur --import-marks=marks-cur --branches | \\\n-\tgit --git-dir=marks-test/.git fast-import --export-marks=marks-new --import-marks=marks-new &&\n+\tgit fast-export --export-marks=marks-cur --import-marks-if-exists=marks-cur --branches | \\\n+\tgit --git-dir=marks-test/.git fast-import --export-marks=marks-new --import-marks-if-exists=marks-new &&\n \t(cd marks-test &&\n \tgit reset --hard &&\n \techo Wohlauf > file &&\n \tgit commit -a -m \"back in time\") &&\n-\tgit --git-dir=marks-test/.git fast-export --export-marks=marks-new --import-marks=marks-new --branches | \\\n-\tgit fast-import --export-marks=marks-cur --import-marks=marks-cur\n+\tgit --git-dir=marks-test/.git fast-export --export-marks=marks-new --import-marks-if-exists=marks-new --branches | \\\n+\tgit fast-import --export-marks=marks-cur --import-marks-if-exists=marks-cur\n '\n \n cat > expected << EOF\n-- \n2.23.0.177.g8af0b3ca64\n\n"},{"id":"382889","messageId":"20190925014005.17056-8-newren@gmail.com","threadId":"51911","inReplyTo":"20190925014005.17056-1-newren@gmail.com","subject":"[PATCH 7/8] t9350: add tests for tags of things other than a commit","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-09-25T01:40:04Z","receivedAt":"2019-09-25T01:40:25Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Multiple changes here:\n * add a test for a tag of a blob\n * add a test for a tag of a tag of a commit\n * add a comment to the tests for (possibly nested) tags of trees,\n making it clear that these tests are doing much less than you might\n expect\n\nSigned-off-by: Elijah Newren \n---\n t/t9350-fast-export.sh | 31 +++++++++++++++++++++++++++++++\n 1 file changed, 31 insertions(+)\n\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex b3fca6ffba..9ab281e4b9 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -540,10 +540,41 @@ test_expect_success 'tree_tag' '\n '\n \n # NEEDSWORK: not just check return status, but validate the output\n+# Note that these tests DO NOTHING other than print a warning that\n+# they are ommitting the one tag we asked them to export (because the\n+# tags resolve to a tree). They exist just to make sure we do not\n+# abort but instead just warn.\n test_expect_success 'tree_tag-obj' 'git fast-export tree_tag-obj'\n test_expect_success 'tag-obj_tag' 'git fast-export tag-obj_tag'\n test_expect_success 'tag-obj_tag-obj' 'git fast-export tag-obj_tag-obj'\n \n+test_expect_success 'handling tags of blobs' '\n+\tgit tag -a -m \"Tag of a blob\" blobtag $(git rev-parse master:file) &&\n+\tgit fast-export blobtag >actual &&\n+\tcat >expect <<-EOF &&\n+\tblob\n+\tmark :1\n+\tdata 9\n+\tdie Luft\n+\n+\ttag blobtag\n+\tfrom :1\n+\ttagger $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+\tdata 14\n+\tTag of a blob\n+\n+\tEOF\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_failure 'handling nested tags' '\n+\tgit tag -a -m \"This is a nested tag\" nested muss &&\n+\tgit fast-export --mark-tags nested >output &&\n+\tgrep \"^from $ZERO_OID$\" output &&\n+\tgrep \"^tag nested$\" output >tag_lines &&\n+\ttest_line_count = 2 tag_lines\n+'\n+\n test_expect_success 'directory becomes symlink' '\n \tgit init dirtosymlink &&\n \tgit init result &&\n-- \n2.23.0.177.g8af0b3ca64\n\n"},{"id":"382890","messageId":"20190925014005.17056-9-newren@gmail.com","threadId":"51911","inReplyTo":"20190925014005.17056-1-newren@gmail.com","subject":"[PATCH 8/8] fast-export: handle nested tags","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-09-25T01:40:05Z","receivedAt":"2019-09-25T01:40:25Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Signed-off-by: Elijah Newren \n---\n builtin/fast-export.c | 30 ++++++++++++++++++------------\n t/t9350-fast-export.sh | 2 +-\n 2 files changed, 19 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/fast-export.c b/builtin/fast-export.c\nindex d32e1e9327..58a74de42a 100644\n--- a/builtin/fast-export.c\n+++ b/builtin/fast-export.c\n@@ -843,22 +843,28 @@ static void handle_tag(const char *name, struct tag *tag)\n \t\t\tfree(buf);\n \t\t\treturn;\n \t\tcase REWRITE:\n-\t\t\tif (tagged->type != OBJ_COMMIT) {\n-\t\t\t\tdie(\"tag %s tags unexported %s!\",\n-\t\t\t\t oid_to_hex(&tag->object.oid),\n-\t\t\t\t type_name(tagged->type));\n-\t\t\t}\n-\t\t\tp = rewrite_commit((struct commit *)tagged);\n-\t\t\tif (!p) {\n-\t\t\t\tprintf(\"reset %s\\nfrom %s\\n\\n\",\n-\t\t\t\t name, oid_to_hex(&null_oid));\n-\t\t\t\tfree(buf);\n-\t\t\t\treturn;\n+\t\t\tif (tagged->type == OBJ_TAG && !mark_tags) {\n+\t\t\t\tdie(_(\"Error: Cannot export nested tags unless --mark-tags is specified.\"));\n+\t\t\t} else if (tagged->type == OBJ_COMMIT) {\n+\t\t\t\tp = rewrite_commit((struct commit *)tagged);\n+\t\t\t\tif (!p) {\n+\t\t\t\t\tprintf(\"reset %s\\nfrom %s\\n\\n\",\n+\t\t\t\t\t name, oid_to_hex(&null_oid));\n+\t\t\t\t\tfree(buf);\n+\t\t\t\t\treturn;\n+\t\t\t\t}\n+\t\t\t\ttagged_mark = get_object_mark(&p->object);\n+\t\t\t} else {\n+\t\t\t\t/* tagged->type is either OBJ_BLOB or OBJ_TAG */\n+\t\t\t\ttagged_mark = get_object_mark(tagged);\n \t\t\t}\n-\t\t\ttagged_mark = get_object_mark(&p->object);\n \t\t}\n \t}\n \n+\tif (tagged->type == OBJ_TAG) {\n+\t\tprintf(\"reset %s\\nfrom %s\\n\\n\",\n+\t\t name, oid_to_hex(&null_oid));\n+\t}\n \tif (starts_with(name, \"refs/tags/\"))\n \t\tname += 10;\n \tprintf(\"tag %s\\n\", name);\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex 9ab281e4b9..2e4e214815 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -567,7 +567,7 @@ test_expect_success 'handling tags of blobs' '\n \ttest_cmp expect actual\n '\n \n-test_expect_failure 'handling nested tags' '\n+test_expect_success 'handling nested tags' '\n \tgit tag -a -m \"This is a nested tag\" nested muss &&\n \tgit fast-export --mark-tags nested >output &&\n \tgrep \"^from $ZERO_OID$\" output &&\n-- \n2.23.0.177.g8af0b3ca64\n\n"},{"id":"383167","messageId":"20190930211018.23633-1-newren@gmail.com","threadId":"51911","inReplyTo":"20190925014005.17056-1-newren@gmail.com","subject":"[PATCH v2 0/8] fast export/import: handle nested tags, improve incremental exports","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-09-30T21:10:10Z","receivedAt":"2019-09-30T21:10:25Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"This series improves the incremental export story for fast-export and\nfast-import (--export-marks and --import-marks fell a bit short),\nfixes a couple small export/import bugs, and enables handling nested\ntags. In particular, the nested tags handling makes it so that\nfast-export and fast-import can finally handle the git.git repo.\n\nChanges since v1 (full range-diff below):\n - Fixed an issue integrating with next/pu (in particular, with\n jk/fast-import-history-bugfix)\n\nElijah Newren (8):\n fast-export: fix exporting a tag and nothing else\n fast-import: fix handling of deleted tags\n fast-import: allow tags to be identified by mark labels\n fast-import: add support for new 'alias' command\n fast-export: add support for --import-marks-if-exists\n fast-export: allow user to request tags be marked with --mark-tags\n t9350: add tests for tags of things other than a commit\n fast-export: handle nested tags\n\n Documentation/git-fast-export.txt | 17 ++++--\n Documentation/git-fast-import.txt | 23 ++++++++\n builtin/fast-export.c | 67 ++++++++++++++++------\n fast-import.c | 94 +++++++++++++++++++++++++++----\n t/t9300-fast-import.sh | 37 ++++++++++++\n t/t9350-fast-export.sh | 68 ++++++++++++++++++++--\n 6 files changed, 268 insertions(+), 38 deletions(-)\n\nRange-diff:\n1: b751d6c2d6 ! 1: 1d19498bc6 fast-import: fix handling of deleted tags\n @@ fast-import.c: static void parse_reset_branch(const char *arg)\n \t\tb = new_branch(arg);\n \tread_next_command();\n \tparse_from(b);\n -+\tif (b->delete && !strncmp(arg, \"refs/tags/\", 10)) {\n ++\tif (b->delete && !strncmp(b->name, \"refs/tags/\", 10)) {\n +\t\t/*\n +\t\t * Elsewhere, we call dump_branches() before dump_tags(),\n +\t\t * and dump_branches() will handle ref deletions first, so\n @@ fast-import.c: static void parse_reset_branch(const char *arg)\n +\t\tfor (t = first_tag; t; t = t->next_tag) {\n +\t\t\tstrbuf_reset(&tag_name);\n +\t\t\tstrbuf_addf(&tag_name, \"refs/tags/%s\", t->name);\n -+\t\t\tif (!strcmp(arg, tag_name.buf))\n ++\t\t\tif (!strcmp(b->name, tag_name.buf))\n +\t\t\t\tbreak;\n +\t\t\tprev = t;\n +\t\t}\n2: 26b77dde15 = 2: e1fd888e4a fast-import: allow tags to be identified by mark labels\n3: e0d1a1d7aa = 3: 93175f28d9 fast-import: add support for new 'alias' command\n4: edea892661 = 4: 8c8743395c fast-export: add support for --import-marks-if-exists\n5: 6af7e1fdd0 = 5: eebc40df33 fast-export: allow user to request tags be marked with --mark-tags\n6: 631ae9a63e = 6: de39f703c6 t9350: add tests for tags of things other than a commit\n7: c0e932e4da = 7: ac739dbb79 fast-export: handle nested tags\n-- \n2.23.0.264.gac739dbb79\n\n"},{"id":"383168","messageId":"20190930211018.23633-2-newren@gmail.com","threadId":"51911","inReplyTo":"20190930211018.23633-1-newren@gmail.com","subject":"[PATCH v2 1/8] fast-export: fix exporting a tag and nothing else","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-09-30T21:10:11Z","receivedAt":"2019-09-30T21:10:26Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"fast-export allows specifying revision ranges, which can be used to\nexport a tag without exporting the commit it tags. fast-export handled\nthis rather poorly: it would emit a \"from :0\" directive. Since marks\nstart at 1 and increase, this means it refers to an unknown commit and\nfast-import will choke on the input.\n\nWhen we are unable to look up a mark for the object being tagged, use a\n\"from $HASH\" directive instead to fix this problem.\n\nNote that this is quite similar to the behavior fast-export exhibits\nwith commits and parents when --reference-excluded-parents is passed\nalong with an excluded commit range. For tags of excluded commits we do\nnot require the --reference-excluded-parents flag because we always have\nto tag something. By contrast, when dealing with commits, pruning a\nparent is always a viable option, so we need the flag to specify that\nparent pruning is not wanted. (It is slightly weird that\n--reference-excluded-parents isn't the default with a separate\n--prune-excluded-parents flag, but backward compatibility concerns\nresulted in the current defaults.)\n\nSigned-off-by: Elijah Newren \n---\n builtin/fast-export.c | 7 ++++++-\n t/t9350-fast-export.sh | 13 +++++++++++++\n 2 files changed, 19 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/fast-export.c b/builtin/fast-export.c\nindex f541f55d33..5822271c6b 100644\n--- a/builtin/fast-export.c\n+++ b/builtin/fast-export.c\n@@ -860,7 +860,12 @@ static void handle_tag(const char *name, struct tag *tag)\n \n \tif (starts_with(name, \"refs/tags/\"))\n \t\tname += 10;\n-\tprintf(\"tag %s\\nfrom :%d\\n\", name, tagged_mark);\n+\tprintf(\"tag %s\\n\", name);\n+\tif (tagged_mark)\n+\t\tprintf(\"from :%d\\n\", tagged_mark);\n+\telse\n+\t\tprintf(\"from %s\\n\", oid_to_hex(&tagged->oid));\n+\n \tif (show_original_ids)\n \t\tprintf(\"original-oid %s\\n\", oid_to_hex(&tag->object.oid));\n \tprintf(\"%.*s%sdata %d\\n%.*s\\n\",\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex b4004e05c2..d32ff41859 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -53,6 +53,19 @@ test_expect_success 'fast-export | fast-import' '\n \n '\n \n+test_expect_success 'fast-export ^muss^{commit} muss' '\n+\tgit fast-export --tag-of-filtered-object=rewrite ^muss^{commit} muss >actual &&\n+\tcat >expected <<-EOF &&\n+\ttag muss\n+\tfrom $(git rev-parse --verify muss^{commit})\n+\t$(git cat-file tag muss | grep tagger)\n+\tdata 9\n+\tvalentin\n+\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'fast-export master~2..master' '\n \n \tgit fast-export master~2..master >actual &&\n-- \n2.23.0.264.gac739dbb79\n\n"},{"id":"383169","messageId":"20190930211018.23633-3-newren@gmail.com","threadId":"51911","inReplyTo":"20190930211018.23633-1-newren@gmail.com","subject":"[PATCH v2 2/8] fast-import: fix handling of deleted tags","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-09-30T21:10:12Z","receivedAt":"2019-09-30T21:10:27Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"If our input stream includes a tag which is later deleted, we were not\nproperly deleting it. We did have a step which would delete it, but we\nleft a tag in the tag list noting that it needed to be updated, and the\nupdating of annotated tags occurred AFTER ref deletion. So, when we\nrecord that a tag needs to be deleted, also remove it from the list of\nannotated tags to update.\n\nWhile this has likely been something that has not happened in practice,\nit will come up more in order to support nested tags. For nested tags,\nwe either need to give temporary names to the intermediate tags and then\ndelete them, or else we need to use the final name for the intermediate\ntags. If we use the final name for the intermediate tags, then in order\nto keep the sanity check that someone doesn't try to update the same tag\ntwice, we need to delete the ref after creating the intermediate tag.\nSo, either way nested tags imply the need to delete temporary inner tag\nreferences.\n\nSigned-off-by: Elijah Newren \n---\n fast-import.c | 29 +++++++++++++++++++++++++++++\n t/t9300-fast-import.sh | 13 +++++++++++++\n 2 files changed, 42 insertions(+)\n\ndiff --git a/fast-import.c b/fast-import.c\nindex b44d6a467e..546da3a938 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -2793,6 +2793,35 @@ static void parse_reset_branch(const char *arg)\n \t\tb = new_branch(arg);\n \tread_next_command();\n \tparse_from(b);\n+\tif (b->delete && !strncmp(b->name, \"refs/tags/\", 10)) {\n+\t\t/*\n+\t\t * Elsewhere, we call dump_branches() before dump_tags(),\n+\t\t * and dump_branches() will handle ref deletions first, so\n+\t\t * in order to make sure the deletion actually takes effect,\n+\t\t * we need to remove the tag from our list of tags to update.\n+\t\t *\n+\t\t * NEEDSWORK: replace list of tags with hashmap for faster\n+\t\t * deletion?\n+\t\t */\n+\t\tstruct strbuf tag_name = STRBUF_INIT;\n+\t\tstruct tag *t, *prev = NULL;\n+\t\tfor (t = first_tag; t; t = t->next_tag) {\n+\t\t\tstrbuf_reset(&tag_name);\n+\t\t\tstrbuf_addf(&tag_name, \"refs/tags/%s\", t->name);\n+\t\t\tif (!strcmp(b->name, tag_name.buf))\n+\t\t\t\tbreak;\n+\t\t\tprev = t;\n+\t\t}\n+\t\tif (t) {\n+\t\t\tif (prev)\n+\t\t\t\tprev->next_tag = t->next_tag;\n+\t\t\telse\n+\t\t\t\tfirst_tag = t->next_tag;\n+\t\t\tif (!t->next_tag)\n+\t\t\t\tlast_tag = prev;\n+\t\t\t/* There is no mem_pool_free(t) function to call. */\n+\t\t}\n+\t}\n \tif (command_buf.len > 0)\n \t\tunread_command_buf = 1;\n }\ndiff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh\nindex 141b7fa35e..74bc41333b 100755\n--- a/t/t9300-fast-import.sh\n+++ b/t/t9300-fast-import.sh\n@@ -85,6 +85,15 @@ test_expect_success 'A: create pack from stdin' '\n \tAn annotated tag that annotates a blob.\n \tEOF\n \n+\ttag to-be-deleted\n+\tfrom :3\n+\tdata <expect <<-EOF &&\n \t:2 $(git rev-parse --verify master:file2)\n-- \n2.23.0.264.gac739dbb79\n\n"},{"id":"383170","messageId":"20190930211018.23633-4-newren@gmail.com","threadId":"51911","inReplyTo":"20190930211018.23633-1-newren@gmail.com","subject":"[PATCH v2 3/8] fast-import: allow tags to be identified by mark labels","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-09-30T21:10:13Z","receivedAt":"2019-09-30T21:10:28Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Mark identifiers are used in fast-export and fast-import to provide a\nlabel to refer to earlier content. Blobs are given labels because they\nneed to be referenced in the commits where they first appear with a\ngiven filename, and commits are given labels because they can be the\nparents of other commits. Tags were never given labels, probably\nbecause they were viewed as unnecessary, but that presents two problems:\n\n 1. It leaves us without a way of referring to previous tags if we\n want to create a tag of a tag (or higher nestings).\n 2. It leaves us with no way of recording that a tag has already been\n imported when using --export-marks and --import-marks.\n\nFix these problems by allowing an optional mark label for tags.\n\nSigned-off-by: Elijah Newren \n---\n Documentation/git-fast-import.txt | 1 +\n fast-import.c | 3 ++-\n t/t9300-fast-import.sh | 19 +++++++++++++++++++\n 3 files changed, 22 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt\nindex 0bb276269e..4977869465 100644\n--- a/Documentation/git-fast-import.txt\n+++ b/Documentation/git-fast-import.txt\n@@ -774,6 +774,7 @@ lightweight (non-annotated) tags see the `reset` command below.\n \n ....\n \t'tag' SP LF\n+\tmark?\n \t'from' SP LF\n \toriginal-oid?\n \t'tagger' (SP )? SP LT GT SP LF\ndiff --git a/fast-import.c b/fast-import.c\nindex 546da3a938..0042440487 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -2713,6 +2713,7 @@ static void parse_new_tag(const char *arg)\n \t\tfirst_tag = t;\n \tlast_tag = t;\n \tread_next_command();\n+\tparse_mark();\n \n \t/* from ... */\n \tif (!skip_prefix(command_buf.buf, \"from \", &from))\n@@ -2769,7 +2770,7 @@ static void parse_new_tag(const char *arg)\n \tstrbuf_addbuf(&new_data, &msg);\n \tfree(tagger);\n \n-\tif (store_object(OBJ_TAG, &new_data, NULL, &t->oid, 0))\n+\tif (store_object(OBJ_TAG, &new_data, NULL, &t->oid, next_mark))\n \t\tt->pack_id = MAX_PACK_ID;\n \telse\n \t\tt->pack_id = pack_id;\ndiff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh\nindex 74bc41333b..3ad2b2f1ba 100755\n--- a/t/t9300-fast-import.sh\n+++ b/t/t9300-fast-import.sh\n@@ -94,6 +94,23 @@ test_expect_success 'A: create pack from stdin' '\n \treset refs/tags/to-be-deleted\n \tfrom 0000000000000000000000000000000000000000\n \n+\ttag nested\n+\tmark :6\n+\tfrom :4\n+\tdata <\n---\n Documentation/git-fast-import.txt | 22 +++++++++++\n fast-import.c | 62 ++++++++++++++++++++++++++-----\n t/t9300-fast-import.sh | 5 +++\n 3 files changed, 79 insertions(+), 10 deletions(-)\n\ndiff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt\nindex 4977869465..a3f1e0c5e4 100644\n--- a/Documentation/git-fast-import.txt\n+++ b/Documentation/git-fast-import.txt\n@@ -337,6 +337,13 @@ and control the current import process. More detailed discussion\n \t`commit` command. This command is optional and is not\n \tneeded to perform an import.\n \n+`alias`::\n+\tRecord that a mark refers to a given object without first\n+\tcreating any new object. Using --import-marks and referring\n+\tto missing marks will cause fast-import to fail, so aliases\n+\tcan provide a way to set otherwise pruned commits to a valid\n+\tvalue (e.g. the nearest non-pruned ancestor).\n+\n `checkpoint`::\n \tForces fast-import to close the current packfile, generate its\n \tunique SHA-1 checksum and index, and start a new packfile.\n@@ -914,6 +921,21 @@ a data chunk which does not have an LF as its last byte.\n +\n The `LF` after ` LF` is optional (it used to be required).\n \n+`alias`\n+~~~~~~~\n+Record that a mark refers to a given object without first creating any\n+new object.\n+\n+....\n+\t'alias' LF\n+\tmark\n+\t'to' SP LF\n+\tLF?\n+....\n+\n+For a detailed description of `` see above under `from`.\n+\n+\n `checkpoint`\n ~~~~~~~~~~~~\n Forces fast-import to close the current packfile, start a new one, and to\ndiff --git a/fast-import.c b/fast-import.c\nindex 0042440487..9a12850d16 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -2491,18 +2491,14 @@ static void parse_from_existing(struct branch *b)\n \t}\n }\n \n-static int parse_from(struct branch *b)\n+static int parse_objectish(struct branch *b, const char *objectish)\n {\n-\tconst char *from;\n \tstruct branch *s;\n \tstruct object_id oid;\n \n-\tif (!skip_prefix(command_buf.buf, \"from \", &from))\n-\t\treturn 0;\n-\n \toidcpy(&oid, &b->branch_tree.versions[1].oid);\n \n-\ts = lookup_branch(from);\n+\ts = lookup_branch(objectish);\n \tif (b == s)\n \t\tdie(\"Can't create a branch from itself: %s\", b->name);\n \telse if (s) {\n@@ -2510,8 +2506,8 @@ static int parse_from(struct branch *b)\n \t\toidcpy(&b->oid, &s->oid);\n \t\toidcpy(&b->branch_tree.versions[0].oid, t);\n \t\toidcpy(&b->branch_tree.versions[1].oid, t);\n-\t} else if (*from == ':') {\n-\t\tuintmax_t idnum = parse_mark_ref_eol(from);\n+\t} else if (*objectish == ':') {\n+\t\tuintmax_t idnum = parse_mark_ref_eol(objectish);\n \t\tstruct object_entry *oe = find_mark(idnum);\n \t\tif (oe->type != OBJ_COMMIT)\n \t\t\tdie(\"Mark :%\" PRIuMAX \" not a commit\", idnum);\n@@ -2525,13 +2521,13 @@ static int parse_from(struct branch *b)\n \t\t\t} else\n \t\t\t\tparse_from_existing(b);\n \t\t}\n-\t} else if (!get_oid(from, &b->oid)) {\n+\t} else if (!get_oid(objectish, &b->oid)) {\n \t\tparse_from_existing(b);\n \t\tif (is_null_oid(&b->oid))\n \t\t\tb->delete = 1;\n \t}\n \telse\n-\t\tdie(\"Invalid ref name or SHA1 expression: %s\", from);\n+\t\tdie(\"Invalid ref name or SHA1 expression: %s\", objectish);\n \n \tif (b->branch_tree.tree && !oideq(&oid, &b->branch_tree.versions[1].oid)) {\n \t\trelease_tree_content_recursive(b->branch_tree.tree);\n@@ -2542,6 +2538,26 @@ static int parse_from(struct branch *b)\n \treturn 1;\n }\n \n+static int parse_from(struct branch *b)\n+{\n+\tconst char *from;\n+\n+\tif (!skip_prefix(command_buf.buf, \"from \", &from))\n+\t\treturn 0;\n+\n+\treturn parse_objectish(b, from);\n+}\n+\n+static int parse_objectish_with_prefix(struct branch *b, const char *prefix)\n+{\n+\tconst char *base;\n+\n+\tif (!skip_prefix(command_buf.buf, prefix, &base))\n+\t\treturn 0;\n+\n+\treturn parse_objectish(b, base);\n+}\n+\n static struct hash_list *parse_merge(unsigned int *count)\n {\n \tstruct hash_list *list = NULL, **tail = &list, *n;\n@@ -3089,6 +3105,28 @@ static void parse_progress(void)\n \tskip_optional_lf();\n }\n \n+static void parse_alias(void)\n+{\n+\tstruct object_entry *e;\n+\tstruct branch b;\n+\n+\tskip_optional_lf();\n+\tread_next_command();\n+\n+\t/* mark ... */\n+\tparse_mark();\n+\tif (!next_mark)\n+\t\tdie(_(\"Expected 'mark' command, got %s\"), command_buf.buf);\n+\n+\t/* to ... */\n+\tmemset(&b, 0, sizeof(b));\n+\tif (!parse_objectish_with_prefix(&b, \"to \"))\n+\t\tdie(_(\"Expected 'to' command, got %s\"), command_buf.buf);\n+\te = find_object(&b.oid);\n+\tassert(e);\n+\tinsert_mark(next_mark, e);\n+}\n+\n static char* make_fast_import_path(const char *path)\n {\n \tif (!relative_marks_paths || is_absolute_path(path))\n@@ -3216,6 +3254,8 @@ static int parse_one_feature(const char *feature, int from_stream)\n \t\toption_import_marks(arg, from_stream, 1);\n \t} else if (skip_prefix(feature, \"export-marks=\", &arg)) {\n \t\toption_export_marks(arg);\n+\t} else if (!strcmp(feature, \"alias\")) {\n+\t\t; /* Don't die - this feature is supported */\n \t} else if (!strcmp(feature, \"get-mark\")) {\n \t\t; /* Don't die - this feature is supported */\n \t} else if (!strcmp(feature, \"cat-blob\")) {\n@@ -3372,6 +3412,8 @@ int cmd_main(int argc, const char **argv)\n \t\t\tparse_checkpoint();\n \t\telse if (!strcmp(\"done\", command_buf.buf))\n \t\t\tbreak;\n+\t\telse if (!strcmp(\"alias\", command_buf.buf))\n+\t\t\tparse_alias();\n \t\telse if (starts_with(command_buf.buf, \"progress \"))\n \t\t\tparse_progress();\n \t\telse if (skip_prefix(command_buf.buf, \"feature \", &v))\ndiff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh\nindex 3ad2b2f1ba..41f2a1dad9 100755\n--- a/t/t9300-fast-import.sh\n+++ b/t/t9300-fast-import.sh\n@@ -111,6 +111,10 @@ test_expect_success 'A: create pack from stdin' '\n \tTag of tag of our lovely commit\n \tEOF\n \n+\talias\n+\tmark :8\n+\tto :5\n+\n \tINPUT_END\n \tgit fast-import --export-marks=marks.out \n---\n builtin/fast-export.c | 23 +++++++++++++++++++----\n t/t9350-fast-export.sh | 10 ++++------\n 2 files changed, 23 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/fast-export.c b/builtin/fast-export.c\nindex 5822271c6b..575e47833b 100644\n--- a/builtin/fast-export.c\n+++ b/builtin/fast-export.c\n@@ -1052,11 +1052,16 @@ static void export_marks(char *file)\n \t\terror(\"Unable to write marks file %s.\", file);\n }\n \n-static void import_marks(char *input_file)\n+static void import_marks(char *input_file, int check_exists)\n {\n \tchar line[512];\n-\tFILE *f = xfopen(input_file, \"r\");\n+\tFILE *f;\n+\tstruct stat sb;\n+\n+\tif (check_exists && stat(input_file, &sb))\n+\t\treturn;\n \n+\tf = xfopen(input_file, \"r\");\n \twhile (fgets(line, sizeof(line), f)) {\n \t\tuint32_t mark;\n \t\tchar *line_end, *mark_end;\n@@ -1120,7 +1125,9 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \tstruct rev_info revs;\n \tstruct object_array commits = OBJECT_ARRAY_INIT;\n \tstruct commit *commit;\n-\tchar *export_filename = NULL, *import_filename = NULL;\n+\tchar *export_filename = NULL,\n+\t *import_filename = NULL,\n+\t *import_filename_if_exists = NULL;\n \tuint32_t lastimportid;\n \tstruct string_list refspecs_list = STRING_LIST_INIT_NODUP;\n \tstruct string_list paths_of_changed_objects = STRING_LIST_INIT_DUP;\n@@ -1140,6 +1147,10 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \t\t\t N_(\"Dump marks to this file\")),\n \t\tOPT_STRING(0, \"import-marks\", &import_filename, N_(\"file\"),\n \t\t\t N_(\"Import marks from this file\")),\n+\t\tOPT_STRING(0, \"import-marks-if-exists\",\n+\t\t\t &import_filename_if_exists,\n+\t\t\t N_(\"file\"),\n+\t\t\t N_(\"Import marks from this file if it exists\")),\n \t\tOPT_BOOL(0, \"fake-missing-tagger\", &fake_missing_tagger,\n \t\t\t N_(\"Fake a tagger when tags lack one\")),\n \t\tOPT_BOOL(0, \"full-tree\", &full_tree,\n@@ -1187,8 +1198,12 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \tif (use_done_feature)\n \t\tprintf(\"feature done\\n\");\n \n+\tif (import_filename && import_filename_if_exists)\n+\t\tdie(_(\"Cannot pass both --import-marks and --import-marks-if-exists\"));\n \tif (import_filename)\n-\t\timport_marks(import_filename);\n+\t\timport_marks(import_filename, 0);\n+\telse if (import_filename_if_exists)\n+\t\timport_marks(import_filename_if_exists, 1);\n \tlastimportid = last_idnum;\n \n \tif (import_filename && revs.prune_data.nr)\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex d32ff41859..ea84e2f173 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -580,17 +580,15 @@ test_expect_success 'fast-export quotes pathnames' '\n '\n \n test_expect_success 'test bidirectionality' '\n-\t>marks-cur &&\n-\t>marks-new &&\n \tgit init marks-test &&\n-\tgit fast-export --export-marks=marks-cur --import-marks=marks-cur --branches | \\\n-\tgit --git-dir=marks-test/.git fast-import --export-marks=marks-new --import-marks=marks-new &&\n+\tgit fast-export --export-marks=marks-cur --import-marks-if-exists=marks-cur --branches | \\\n+\tgit --git-dir=marks-test/.git fast-import --export-marks=marks-new --import-marks-if-exists=marks-new &&\n \t(cd marks-test &&\n \tgit reset --hard &&\n \techo Wohlauf > file &&\n \tgit commit -a -m \"back in time\") &&\n-\tgit --git-dir=marks-test/.git fast-export --export-marks=marks-new --import-marks=marks-new --branches | \\\n-\tgit fast-import --export-marks=marks-cur --import-marks=marks-cur\n+\tgit --git-dir=marks-test/.git fast-export --export-marks=marks-new --import-marks-if-exists=marks-new --branches | \\\n+\tgit fast-import --export-marks=marks-cur --import-marks-if-exists=marks-cur\n '\n \n cat > expected << EOF\n-- \n2.23.0.264.gac739dbb79\n\n"},{"id":"383173","messageId":"20190930211018.23633-7-newren@gmail.com","threadId":"51911","inReplyTo":"20190930211018.23633-1-newren@gmail.com","subject":"[PATCH v2 6/8] fast-export: allow user to request tags be marked with --mark-tags","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-09-30T21:10:16Z","receivedAt":"2019-09-30T21:10:32Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Add a new option, --mark-tags, which will output mark identifiers with\neach tag object. This improves the incremental export story with\n--export-marks since it will allow us to record that annotated tags have\nbeen exported, and it is also needed as a step towards supporting nested\ntags.\n\nSigned-off-by: Elijah Newren \n---\n Documentation/git-fast-export.txt | 17 +++++++++++++----\n builtin/fast-export.c | 7 +++++++\n t/t9350-fast-export.sh | 14 ++++++++++++++\n 3 files changed, 34 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt\nindex cc940eb9ad..c522b34f7b 100644\n--- a/Documentation/git-fast-export.txt\n+++ b/Documentation/git-fast-export.txt\n@@ -75,11 +75,20 @@ produced incorrect results if you gave these options.\n \tBefore processing any input, load the marks specified in\n \t. The input file must exist, must be readable, and\n \tmust use the same format as produced by --export-marks.\n+\n+--mark-tags::\n+\tIn addition to labelling blobs and commits with mark ids, also\n+\tlabel tags. This is useful in conjunction with\n+\t`--export-marks` and `--import-marks`, and is also useful (and\n+\tnecessary) for exporting of nested tags. It does not hurt\n+\tother cases and would be the default, but many fast-import\n+\tfrontends are not prepared to accept tags with mark\n+\tidentifiers.\n +\n-Any commits that have already been marked will not be exported again.\n-If the backend uses a similar --import-marks file, this allows for\n-incremental bidirectional exporting of the repository by keeping the\n-marks the same across runs.\n+Any commits (or tags) that have already been marked will not be\n+exported again. If the backend uses a similar --import-marks file,\n+this allows for incremental bidirectional exporting of the repository\n+by keeping the marks the same across runs.\n \n --fake-missing-tagger::\n \tSome old repositories have tags without a tagger. The\ndiff --git a/builtin/fast-export.c b/builtin/fast-export.c\nindex 575e47833b..d32e1e9327 100644\n--- a/builtin/fast-export.c\n+++ b/builtin/fast-export.c\n@@ -40,6 +40,7 @@ static int no_data;\n static int full_tree;\n static int reference_excluded_commits;\n static int show_original_ids;\n+static int mark_tags;\n static struct string_list extra_refs = STRING_LIST_INIT_NODUP;\n static struct string_list tag_refs = STRING_LIST_INIT_NODUP;\n static struct refspec refspecs = REFSPEC_INIT_FETCH;\n@@ -861,6 +862,10 @@ static void handle_tag(const char *name, struct tag *tag)\n \tif (starts_with(name, \"refs/tags/\"))\n \t\tname += 10;\n \tprintf(\"tag %s\\n\", name);\n+\tif (mark_tags) {\n+\t\tmark_next_object(&tag->object);\n+\t\tprintf(\"mark :%\"PRIu32\"\\n\", last_idnum);\n+\t}\n \tif (tagged_mark)\n \t\tprintf(\"from :%d\\n\", tagged_mark);\n \telse\n@@ -1165,6 +1170,8 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \t\t\t &reference_excluded_commits, N_(\"Reference parents which are not in fast-export stream by object id\")),\n \t\tOPT_BOOL(0, \"show-original-ids\", &show_original_ids,\n \t\t\t N_(\"Show original object ids of blobs/commits\")),\n+\t\tOPT_BOOL(0, \"mark-tags\", &mark_tags,\n+\t\t\t N_(\"Label tags with mark ids\")),\n \n \t\tOPT_END()\n \t};\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex ea84e2f173..b3fca6ffba 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -66,6 +66,20 @@ test_expect_success 'fast-export ^muss^{commit} muss' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'fast-export --mark-tags ^muss^{commit} muss' '\n+\tgit fast-export --mark-tags --tag-of-filtered-object=rewrite ^muss^{commit} muss >actual &&\n+\tcat >expected <<-EOF &&\n+\ttag muss\n+\tmark :1\n+\tfrom $(git rev-parse --verify muss^{commit})\n+\t$(git cat-file tag muss | grep tagger)\n+\tdata 9\n+\tvalentin\n+\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'fast-export master~2..master' '\n \n \tgit fast-export master~2..master >actual &&\n-- \n2.23.0.264.gac739dbb79\n\n"},{"id":"383174","messageId":"20190930211018.23633-8-newren@gmail.com","threadId":"51911","inReplyTo":"20190930211018.23633-1-newren@gmail.com","subject":"[PATCH v2 7/8] t9350: add tests for tags of things other than a commit","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-09-30T21:10:17Z","receivedAt":"2019-09-30T21:10:33Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Multiple changes here:\n * add a test for a tag of a blob\n * add a test for a tag of a tag of a commit\n * add a comment to the tests for (possibly nested) tags of trees,\n making it clear that these tests are doing much less than you might\n expect\n\nSigned-off-by: Elijah Newren \n---\n t/t9350-fast-export.sh | 31 +++++++++++++++++++++++++++++++\n 1 file changed, 31 insertions(+)\n\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex b3fca6ffba..9ab281e4b9 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -540,10 +540,41 @@ test_expect_success 'tree_tag' '\n '\n \n # NEEDSWORK: not just check return status, but validate the output\n+# Note that these tests DO NOTHING other than print a warning that\n+# they are ommitting the one tag we asked them to export (because the\n+# tags resolve to a tree). They exist just to make sure we do not\n+# abort but instead just warn.\n test_expect_success 'tree_tag-obj' 'git fast-export tree_tag-obj'\n test_expect_success 'tag-obj_tag' 'git fast-export tag-obj_tag'\n test_expect_success 'tag-obj_tag-obj' 'git fast-export tag-obj_tag-obj'\n \n+test_expect_success 'handling tags of blobs' '\n+\tgit tag -a -m \"Tag of a blob\" blobtag $(git rev-parse master:file) &&\n+\tgit fast-export blobtag >actual &&\n+\tcat >expect <<-EOF &&\n+\tblob\n+\tmark :1\n+\tdata 9\n+\tdie Luft\n+\n+\ttag blobtag\n+\tfrom :1\n+\ttagger $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+\tdata 14\n+\tTag of a blob\n+\n+\tEOF\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_failure 'handling nested tags' '\n+\tgit tag -a -m \"This is a nested tag\" nested muss &&\n+\tgit fast-export --mark-tags nested >output &&\n+\tgrep \"^from $ZERO_OID$\" output &&\n+\tgrep \"^tag nested$\" output >tag_lines &&\n+\ttest_line_count = 2 tag_lines\n+'\n+\n test_expect_success 'directory becomes symlink' '\n \tgit init dirtosymlink &&\n \tgit init result &&\n-- \n2.23.0.264.gac739dbb79\n\n"},{"id":"383175","messageId":"20190930211018.23633-9-newren@gmail.com","threadId":"51911","inReplyTo":"20190930211018.23633-1-newren@gmail.com","subject":"[PATCH v2 8/8] fast-export: handle nested tags","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-09-30T21:10:18Z","receivedAt":"2019-09-30T21:10:34Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Signed-off-by: Elijah Newren \n---\n builtin/fast-export.c | 30 ++++++++++++++++++------------\n t/t9350-fast-export.sh | 2 +-\n 2 files changed, 19 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/fast-export.c b/builtin/fast-export.c\nindex d32e1e9327..58a74de42a 100644\n--- a/builtin/fast-export.c\n+++ b/builtin/fast-export.c\n@@ -843,22 +843,28 @@ static void handle_tag(const char *name, struct tag *tag)\n \t\t\tfree(buf);\n \t\t\treturn;\n \t\tcase REWRITE:\n-\t\t\tif (tagged->type != OBJ_COMMIT) {\n-\t\t\t\tdie(\"tag %s tags unexported %s!\",\n-\t\t\t\t oid_to_hex(&tag->object.oid),\n-\t\t\t\t type_name(tagged->type));\n-\t\t\t}\n-\t\t\tp = rewrite_commit((struct commit *)tagged);\n-\t\t\tif (!p) {\n-\t\t\t\tprintf(\"reset %s\\nfrom %s\\n\\n\",\n-\t\t\t\t name, oid_to_hex(&null_oid));\n-\t\t\t\tfree(buf);\n-\t\t\t\treturn;\n+\t\t\tif (tagged->type == OBJ_TAG && !mark_tags) {\n+\t\t\t\tdie(_(\"Error: Cannot export nested tags unless --mark-tags is specified.\"));\n+\t\t\t} else if (tagged->type == OBJ_COMMIT) {\n+\t\t\t\tp = rewrite_commit((struct commit *)tagged);\n+\t\t\t\tif (!p) {\n+\t\t\t\t\tprintf(\"reset %s\\nfrom %s\\n\\n\",\n+\t\t\t\t\t name, oid_to_hex(&null_oid));\n+\t\t\t\t\tfree(buf);\n+\t\t\t\t\treturn;\n+\t\t\t\t}\n+\t\t\t\ttagged_mark = get_object_mark(&p->object);\n+\t\t\t} else {\n+\t\t\t\t/* tagged->type is either OBJ_BLOB or OBJ_TAG */\n+\t\t\t\ttagged_mark = get_object_mark(tagged);\n \t\t\t}\n-\t\t\ttagged_mark = get_object_mark(&p->object);\n \t\t}\n \t}\n \n+\tif (tagged->type == OBJ_TAG) {\n+\t\tprintf(\"reset %s\\nfrom %s\\n\\n\",\n+\t\t name, oid_to_hex(&null_oid));\n+\t}\n \tif (starts_with(name, \"refs/tags/\"))\n \t\tname += 10;\n \tprintf(\"tag %s\\n\", name);\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex 9ab281e4b9..2e4e214815 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -567,7 +567,7 @@ test_expect_success 'handling tags of blobs' '\n \ttest_cmp expect actual\n '\n \n-test_expect_failure 'handling nested tags' '\n+test_expect_success 'handling nested tags' '\n \tgit tag -a -m \"This is a nested tag\" nested muss &&\n \tgit fast-export --mark-tags nested >output &&\n \tgrep \"^from $ZERO_OID$\" output &&\n-- \n2.23.0.264.gac739dbb79\n\n"},{"id":"383281","messageId":"CABPp-BFckDZL4iAqhHmrXDpFi3eQgvOkbgjJdGRZE4ePUwNDuw@mail.gmail.com","threadId":"51911","inReplyTo":"20190930211018.23633-1-newren@gmail.com","subject":"Re: [PATCH v2 0/8] fast export/import: handle nested tags, improve incremental exports","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-10-02T15:54:02Z","receivedAt":"2019-10-02T15:54:16Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi Junio,\n\nOn Mon, Sep 30, 2019 at 2:10 PM Elijah Newren wrote:\n>\n> This series improves the incremental export story for fast-export and\n> fast-import (--export-marks and --import-marks fell a bit short),\n> fixes a couple small export/import bugs, and enables handling nested\n> tags. In particular, the nested tags handling makes it so that\n> fast-export and fast-import can finally handle the git.git repo.\n\nI see you picked up this corrected series in pu; thanks. However,\nyour merge commit, 2a99d6b6ff7c (\"Merge branch\n'en/fast-imexport-nested-tags' into pu\", 2019-10-02), claims \"Seems to\nbreak t9300 when merged to 'pu'.\". I know v1 did that and I could\nreproduce, but I can't reproduce any failures here. Was this message\njust left over or is there some problem you are seeing?\n\nThanks,\nElijah\n\n>\n> Changes since v1 (full range-diff below):\n> - Fixed an issue integrating with next/pu (in particular, with\n> jk/fast-import-history-bugfix)\n>\n> Elijah Newren (8):\n> fast-export: fix exporting a tag and nothing else\n> fast-import: fix handling of deleted tags\n> fast-import: allow tags to be identified by mark labels\n> fast-import: add support for new 'alias' command\n> fast-export: add support for --import-marks-if-exists\n> fast-export: allow user to request tags be marked with --mark-tags\n> t9350: add tests for tags of things other than a commit\n> fast-export: handle nested tags\n>\n> Documentation/git-fast-export.txt | 17 ++++--\n> Documentation/git-fast-import.txt | 23 ++++++++\n> builtin/fast-export.c | 67 ++++++++++++++++------\n> fast-import.c | 94 +++++++++++++++++++++++++++----\n> t/t9300-fast-import.sh | 37 ++++++++++++\n> t/t9350-fast-export.sh | 68 ++++++++++++++++++++--\n> 6 files changed, 268 insertions(+), 38 deletions(-)\n>\n> Range-diff:\n> 1: b751d6c2d6 ! 1: 1d19498bc6 fast-import: fix handling of deleted tags\n> @@ fast-import.c: static void parse_reset_branch(const char *arg)\n> b = new_branch(arg);\n> read_next_command();\n> parse_from(b);\n> -+ if (b->delete && !strncmp(arg, \"refs/tags/\", 10)) {\n> ++ if (b->delete && !strncmp(b->name, \"refs/tags/\", 10)) {\n> + /*\n> + * Elsewhere, we call dump_branches() before dump_tags(),\n> + * and dump_branches() will handle ref deletions first, so\n> @@ fast-import.c: static void parse_reset_branch(const char *arg)\n> + for (t = first_tag; t; t = t->next_tag) {\n> + strbuf_reset(&tag_name);\n> + strbuf_addf(&tag_name, \"refs/tags/%s\", t->name);\n> -+ if (!strcmp(arg, tag_name.buf))\n> ++ if (!strcmp(b->name, tag_name.buf))\n> + break;\n> + prev = t;\n> + }\n> 2: 26b77dde15 = 2: e1fd888e4a fast-import: allow tags to be identified by mark labels\n> 3: e0d1a1d7aa = 3: 93175f28d9 fast-import: add support for new 'alias' command\n> 4: edea892661 = 4: 8c8743395c fast-export: add support for --import-marks-if-exists\n> 5: 6af7e1fdd0 = 5: eebc40df33 fast-export: allow user to request tags be marked with --mark-tags\n> 6: 631ae9a63e = 6: de39f703c6 t9350: add tests for tags of things other than a commit\n> 7: c0e932e4da = 7: ac739dbb79 fast-export: handle nested tags\n> --\n> 2.23.0.264.gac739dbb79\n>\n"},{"id":"383301","messageId":"xmqqzhiidi2u.fsf@gitster-ct.c.googlers.com","threadId":"51911","inReplyTo":"CABPp-BFckDZL4iAqhHmrXDpFi3eQgvOkbgjJdGRZE4ePUwNDuw@mail.gmail.com","subject":"Re: [PATCH v2 0/8] fast export/import: handle nested tags, improve incremental exports","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-10-02T20:10:33Z","receivedAt":"2019-10-02T20:10:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Elijah Newren writes:\n\n> I see you picked up this corrected series in pu; thanks. However,\n> your merge commit, 2a99d6b6ff7c (\"Merge branch\n> 'en/fast-imexport-nested-tags' into pu\", 2019-10-02), claims \"Seems to\n> break t9300 when merged to 'pu'.\". I know v1 did that and I could\n> reproduce, but I can't reproduce any failures here. Was this message\n> just left over or is there some problem you are seeing?\n\nI thought that the latest What's cooking written after you sent the\ncorrected version hasn't been sent yet.\n\nAnd the draft copy I locally have for the next issue of what's cooking\nhas the comment removed already.\n\nAnything I missed?\n\n"},{"id":"383305","messageId":"CABPp-BEJ47MonV2WmD+UornSG5LPmpGAyjWQa_4DTkAfjMjRmg@mail.gmail.com","threadId":"51911","inReplyTo":"xmqqzhiidi2u.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v2 0/8] fast export/import: handle nested tags, improve incremental exports","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-10-02T21:05:55Z","receivedAt":"2019-10-02T21:06:12Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Oct 2, 2019 at 1:10 PM Junio C Hamano wrote:\n>\n> Elijah Newren writes:\n>\n> > I see you picked up this corrected series in pu; thanks. However,\n> > your merge commit, 2a99d6b6ff7c (\"Merge branch\n> > 'en/fast-imexport-nested-tags' into pu\", 2019-10-02), claims \"Seems to\n> > break t9300 when merged to 'pu'.\". I know v1 did that and I could\n> > reproduce, but I can't reproduce any failures here. Was this message\n> > just left over or is there some problem you are seeing?\n>\n> I thought that the latest What's cooking written after you sent the\n> corrected version hasn't been sent yet.\n>\n> And the draft copy I locally have for the next issue of what's cooking\n> has the comment removed already.\n>\n> Anything I missed?\n\nI was looking at the commit message for the merge commit where you\nmerged in v2 of the series, and was surprised to see the commit\nmessage claim there was still breakage with the new version of the\nseries:\n\n$ git log -1 2a99d6b6ff7c\ncommit 2a99d6b6ff7cd29cc41d587ae737ffb8e7babea4\nMerge: 7d81b61dd0 0e2eeb4cb4\nAuthor: Junio C Hamano \nDate: Wed Oct 2 16:08:06 2019 +0900\n\n Merge branch 'en/fast-imexport-nested-tags' into pu\n\n Updates to fast-import/export.\n\n Seems to break t9300 when merged to 'pu'.\n\n * en/fast-imexport-nested-tags:\n fast-export: handle nested tags\n t9350: add tests for tags of things other than a commit\n fast-export: allow user to request tags be marked with --mark-tags\n fast-export: add support for --import-marks-if-exists\n fast-import: add support for new 'alias' command\n fast-import: allow tags to be identified by mark labels\n fast-import: fix handling of deleted tags\n fast-export: fix exporting a tag and nothing else\n"},{"id":"383321","messageId":"xmqqa7aid4bl.fsf@gitster-ct.c.googlers.com","threadId":"51911","inReplyTo":"CABPp-BEJ47MonV2WmD+UornSG5LPmpGAyjWQa_4DTkAfjMjRmg@mail.gmail.com","subject":"Re: [PATCH v2 0/8] fast export/import: handle nested tags, improve incremental exports","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-10-03T01:07:42Z","receivedAt":"2019-10-03T01:10:16Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Elijah Newren writes:\n\n>> And the draft copy I locally have for the next issue of what's cooking\n>> has the comment removed already.\n>>\n>> Anything I missed?\n>\n> I was looking at the commit message for the merge commit where you\n> merged in v2 of the series...\n\nAh, sorry, yes you are right, the merge messages can lag behind,\nespecially until the topic hits 'next'. It is not a designed goal\nto work that way, but just a consequene of how my workflow is\nstructured.\n\n * A merge grabs the message from the then-latest \"what's cooking\" draft.\n\n * The \"what's cooking\" draft is then updated. This is a two-part\n process:\n\n - \"git log --first-parent master..pu\" is scanned, to see which\n parts of what topic are and are not yet in 'next' (this flips\n '-' and '+' prefix in the \"what's cooking\" report) and marks\n the changed parts for the next part.\n\n - An editor session looks for these topics with updated status\n and updates the comment in the \"what's cooking\" draft..\n\nSo, the topic with the new fix-up commit was merged to 'pu' with the\nmessage taken from the \"what's cooking\" draft before the draft gets\nupdated with that fix-up commit. When I have sufficient time, I\nrebuild 'pu' twice (i.e. do the integration once, and then update\nthe \"what's cooking\" draft, rewind 'pu' to 'master' and do the\nintegration again to pick up messages from the updated \"what's\ncooking\" draft), but yesterday was not one of those days X-<.\n\n"},{"id":"383330","messageId":"f5b70fc3-8aff-7dc6-66c4-df3cc85df312@web.de","threadId":"51911","inReplyTo":"20190930211018.23633-3-newren@gmail.com","subject":"Re: [PATCH v2 2/8] fast-import: fix handling of deleted tags","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2019-10-03T11:53:23Z","receivedAt":"2019-10-03T11:53:33Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 30.09.19 um 23:10 schrieb Elijah Newren:\n> If our input stream includes a tag which is later deleted, we were not\n> properly deleting it. We did have a step which would delete it, but we\n> left a tag in the tag list noting that it needed to be updated, and the\n> updating of annotated tags occurred AFTER ref deletion. So, when we\n> record that a tag needs to be deleted, also remove it from the list of\n> annotated tags to update.\n>\n> While this has likely been something that has not happened in practice,\n> it will come up more in order to support nested tags. For nested tags,\n> we either need to give temporary names to the intermediate tags and then\n> delete them, or else we need to use the final name for the intermediate\n> tags. If we use the final name for the intermediate tags, then in order\n> to keep the sanity check that someone doesn't try to update the same tag\n> twice, we need to delete the ref after creating the intermediate tag.\n> So, either way nested tags imply the need to delete temporary inner tag\n> references.\n>\n> Signed-off-by: Elijah Newren \n> ---\n> fast-import.c | 29 +++++++++++++++++++++++++++++\n> t/t9300-fast-import.sh | 13 +++++++++++++\n> 2 files changed, 42 insertions(+)\n>\n> diff --git a/fast-import.c b/fast-import.c\n> index b44d6a467e..546da3a938 100644\n> --- a/fast-import.c\n> +++ b/fast-import.c\n> @@ -2793,6 +2793,35 @@ static void parse_reset_branch(const char *arg)\n> \t\tb = new_branch(arg);\n> \tread_next_command();\n> \tparse_from(b);\n> +\tif (b->delete && !strncmp(b->name, \"refs/tags/\", 10)) {\n\nb->name is a NUL-terminated string; starts_with() could be used to avoid\nthe magic number 10.\n\n> +\t\t/*\n> +\t\t * Elsewhere, we call dump_branches() before dump_tags(),\n> +\t\t * and dump_branches() will handle ref deletions first, so\n> +\t\t * in order to make sure the deletion actually takes effect,\n> +\t\t * we need to remove the tag from our list of tags to update.\n> +\t\t *\n> +\t\t * NEEDSWORK: replace list of tags with hashmap for faster\n> +\t\t * deletion?\n> +\t\t */\n> +\t\tstruct strbuf tag_name = STRBUF_INIT;\n\nThis adds a small memory leak.\n\n> +\t\tstruct tag *t, *prev = NULL;\n> +\t\tfor (t = first_tag; t; t = t->next_tag) {\n> +\t\t\tstrbuf_reset(&tag_name);\n> +\t\t\tstrbuf_addf(&tag_name, \"refs/tags/%s\", t->name);\n> +\t\t\tif (!strcmp(b->name, tag_name.buf))\n\nSo the strbuf is used to prefix t->name with \"refs/tags/\", which we know\nb->name starts with, and to compare the result with b->name. Removing\nthe \"refs/tags/\" prefix from b->name using skip_prefix() and comparing\nthe result with t->name would be easier.\n\n> +\t\t\t\tbreak;\n> +\t\t\tprev = t;\n> +\t\t}\n> +\t\tif (t) {\n> +\t\t\tif (prev)\n> +\t\t\t\tprev->next_tag = t->next_tag;\n> +\t\t\telse\n> +\t\t\t\tfirst_tag = t->next_tag;\n> +\t\t\tif (!t->next_tag)\n> +\t\t\t\tlast_tag = prev;\n> +\t\t\t/* There is no mem_pool_free(t) function to call. */\n> +\t\t}\n> +\t}\n> \tif (command_buf.len > 0)\n> \t\tunread_command_buf = 1;\n> }\n\nHere's a squashable patch for that:\n\n---\n fast-import.c | 8 +++-----\n 1 file changed, 3 insertions(+), 5 deletions(-)\n\ndiff --git a/fast-import.c b/fast-import.c\nindex 70cd3f0ff4..a109591406 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -2779,6 +2779,7 @@ static void parse_new_tag(const char *arg)\n static void parse_reset_branch(const char *arg)\n {\n \tstruct branch *b;\n+\tconst char *tag_name;\n\n \tb = lookup_branch(arg);\n \tif (b) {\n@@ -2794,7 +2795,7 @@ static void parse_reset_branch(const char *arg)\n \t\tb = new_branch(arg);\n \tread_next_command();\n \tparse_from(b);\n-\tif (b->delete && !strncmp(b->name, \"refs/tags/\", 10)) {\n+\tif (b->delete && skip_prefix(b->name, \"refs/tags/\", &tag_name)) {\n \t\t/*\n \t\t * Elsewhere, we call dump_branches() before dump_tags(),\n \t\t * and dump_branches() will handle ref deletions first, so\n@@ -2804,12 +2805,9 @@ static void parse_reset_branch(const char *arg)\n \t\t * NEEDSWORK: replace list of tags with hashmap for faster\n \t\t * deletion?\n \t\t */\n-\t\tstruct strbuf tag_name = STRBUF_INIT;\n \t\tstruct tag *t, *prev = NULL;\n \t\tfor (t = first_tag; t; t = t->next_tag) {\n-\t\t\tstrbuf_reset(&tag_name);\n-\t\t\tstrbuf_addf(&tag_name, \"refs/tags/%s\", t->name);\n-\t\t\tif (!strcmp(b->name, tag_name.buf))\n+\t\t\tif (!strcmp(t->name, tag_name))\n \t\t\t\tbreak;\n \t\t\tprev = t;\n \t\t}\n--\n2.23.0\n"},{"id":"383348","messageId":"20191003202709.26279-2-newren@gmail.com","threadId":"51911","inReplyTo":"20191003202709.26279-1-newren@gmail.com","subject":"[PATCH -v3 1/8] fast-export: fix exporting a tag and nothing else","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-10-03T20:27:02Z","receivedAt":"2019-10-03T20:27:18Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"fast-export allows specifying revision ranges, which can be used to\nexport a tag without exporting the commit it tags. fast-export handled\nthis rather poorly: it would emit a \"from :0\" directive. Since marks\nstart at 1 and increase, this means it refers to an unknown commit and\nfast-import will choke on the input.\n\nWhen we are unable to look up a mark for the object being tagged, use a\n\"from $HASH\" directive instead to fix this problem.\n\nNote that this is quite similar to the behavior fast-export exhibits\nwith commits and parents when --reference-excluded-parents is passed\nalong with an excluded commit range. For tags of excluded commits we do\nnot require the --reference-excluded-parents flag because we always have\nto tag something. By contrast, when dealing with commits, pruning a\nparent is always a viable option, so we need the flag to specify that\nparent pruning is not wanted. (It is slightly weird that\n--reference-excluded-parents isn't the default with a separate\n--prune-excluded-parents flag, but backward compatibility concerns\nresulted in the current defaults.)\n\nSigned-off-by: Elijah Newren \n---\n builtin/fast-export.c | 7 ++++++-\n t/t9350-fast-export.sh | 13 +++++++++++++\n 2 files changed, 19 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/fast-export.c b/builtin/fast-export.c\nindex f541f55d33..5822271c6b 100644\n--- a/builtin/fast-export.c\n+++ b/builtin/fast-export.c\n@@ -860,7 +860,12 @@ static void handle_tag(const char *name, struct tag *tag)\n \n \tif (starts_with(name, \"refs/tags/\"))\n \t\tname += 10;\n-\tprintf(\"tag %s\\nfrom :%d\\n\", name, tagged_mark);\n+\tprintf(\"tag %s\\n\", name);\n+\tif (tagged_mark)\n+\t\tprintf(\"from :%d\\n\", tagged_mark);\n+\telse\n+\t\tprintf(\"from %s\\n\", oid_to_hex(&tagged->oid));\n+\n \tif (show_original_ids)\n \t\tprintf(\"original-oid %s\\n\", oid_to_hex(&tag->object.oid));\n \tprintf(\"%.*s%sdata %d\\n%.*s\\n\",\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex b4004e05c2..d32ff41859 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -53,6 +53,19 @@ test_expect_success 'fast-export | fast-import' '\n \n '\n \n+test_expect_success 'fast-export ^muss^{commit} muss' '\n+\tgit fast-export --tag-of-filtered-object=rewrite ^muss^{commit} muss >actual &&\n+\tcat >expected <<-EOF &&\n+\ttag muss\n+\tfrom $(git rev-parse --verify muss^{commit})\n+\t$(git cat-file tag muss | grep tagger)\n+\tdata 9\n+\tvalentin\n+\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'fast-export master~2..master' '\n \n \tgit fast-export master~2..master >actual &&\n-- \n2.23.0.264.g3b9f7f2fc6\n\n"},{"id":"383349","messageId":"20191003202709.26279-1-newren@gmail.com","threadId":"51911","inReplyTo":"20190930211018.23633-1-newren@gmail.com","subject":"[PATCH -v3 0/8] fast export/import: handle nested tags, improve incremental exports","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-10-03T20:27:01Z","receivedAt":"2019-10-03T20:27:19Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"This series improves the incremental export story for fast-export and\nfast-import (--export-marks and --import-marks fell a bit short),\nfixes a couple small export/import bugs, and enables handling nested\ntags. In particular, the nested tags handling makes it so that\nfast-export and fast-import can finally handle the git.git repo.\n\nChanges since v2 (full range-diff below):\n - Code cleanup of patch 2 suggested by René\n\nElijah Newren (8):\n fast-export: fix exporting a tag and nothing else\n fast-import: fix handling of deleted tags\n fast-import: allow tags to be identified by mark labels\n fast-import: add support for new 'alias' command\n fast-export: add support for --import-marks-if-exists\n fast-export: allow user to request tags be marked with --mark-tags\n t9350: add tests for tags of things other than a commit\n fast-export: handle nested tags\n\n Documentation/git-fast-export.txt | 17 ++++--\n Documentation/git-fast-import.txt | 23 ++++++++\n builtin/fast-export.c | 67 ++++++++++++++++------\n fast-import.c | 92 +++++++++++++++++++++++++++----\n t/t9300-fast-import.sh | 37 +++++++++++++\n t/t9350-fast-export.sh | 68 +++++++++++++++++++++--\n 6 files changed, 266 insertions(+), 38 deletions(-)\n\nRange-diff:\n1: a30cfbbb50 = 1: a30cfbbb50 fast-export: fix exporting a tag and nothing else\n2: 1d19498bc6 ! 2: 36fbf15134 fast-import: fix handling of deleted tags\n @@ Commit message\n So, either way nested tags imply the need to delete temporary inner tag\n references.\n \n + Helped-by: René Scharfe \n Signed-off-by: Elijah Newren \n \n ## fast-import.c ##\n +@@ fast-import.c: static void parse_new_tag(const char *arg)\n + static void parse_reset_branch(const char *arg)\n + {\n + \tstruct branch *b;\n ++\tconst char *tag_name;\n + \n + \tb = lookup_branch(arg);\n + \tif (b) {\n @@ fast-import.c: static void parse_reset_branch(const char *arg)\n \t\tb = new_branch(arg);\n \tread_next_command();\n \tparse_from(b);\n -+\tif (b->delete && !strncmp(b->name, \"refs/tags/\", 10)) {\n ++\tif (b->delete && skip_prefix(b->name, \"refs/tags/\", &tag_name)) {\n +\t\t/*\n +\t\t * Elsewhere, we call dump_branches() before dump_tags(),\n +\t\t * and dump_branches() will handle ref deletions first, so\n @@ fast-import.c: static void parse_reset_branch(const char *arg)\n +\t\t * NEEDSWORK: replace list of tags with hashmap for faster\n +\t\t * deletion?\n +\t\t */\n -+\t\tstruct strbuf tag_name = STRBUF_INIT;\n +\t\tstruct tag *t, *prev = NULL;\n +\t\tfor (t = first_tag; t; t = t->next_tag) {\n -+\t\t\tstrbuf_reset(&tag_name);\n -+\t\t\tstrbuf_addf(&tag_name, \"refs/tags/%s\", t->name);\n -+\t\t\tif (!strcmp(b->name, tag_name.buf))\n ++\t\t\tif (!strcmp(t->name, tag_name))\n +\t\t\t\tbreak;\n +\t\t\tprev = t;\n +\t\t}\n3: e1fd888e4a = 3: 3b5f4270f8 fast-import: allow tags to be identified by mark labels\n4: 93175f28d9 = 4: 489c7fd854 fast-import: add support for new 'alias' command\n5: 8c8743395c = 5: 38fd19caee fast-export: add support for --import-marks-if-exists\n6: eebc40df33 = 6: 2017b8d9f9 fast-export: allow user to request tags be marked with --mark-tags\n7: de39f703c6 = 7: 0efdbb81b1 t9350: add tests for tags of things other than a commit\n8: ac739dbb79 = 8: fe7c27d786 fast-export: handle nested tags\n-- \n2.23.0.264.g3b9f7f2fc6\n\n"},{"id":"383350","messageId":"20191003202709.26279-3-newren@gmail.com","threadId":"51911","inReplyTo":"20191003202709.26279-1-newren@gmail.com","subject":"[PATCH -v3 2/8] fast-import: fix handling of deleted tags","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-10-03T20:27:03Z","receivedAt":"2019-10-03T20:27:19Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"If our input stream includes a tag which is later deleted, we were not\nproperly deleting it. We did have a step which would delete it, but we\nleft a tag in the tag list noting that it needed to be updated, and the\nupdating of annotated tags occurred AFTER ref deletion. So, when we\nrecord that a tag needs to be deleted, also remove it from the list of\nannotated tags to update.\n\nWhile this has likely been something that has not happened in practice,\nit will come up more in order to support nested tags. For nested tags,\nwe either need to give temporary names to the intermediate tags and then\ndelete them, or else we need to use the final name for the intermediate\ntags. If we use the final name for the intermediate tags, then in order\nto keep the sanity check that someone doesn't try to update the same tag\ntwice, we need to delete the ref after creating the intermediate tag.\nSo, either way nested tags imply the need to delete temporary inner tag\nreferences.\n\nHelped-by: René Scharfe \nSigned-off-by: Elijah Newren \n---\n fast-import.c | 27 +++++++++++++++++++++++++++\n t/t9300-fast-import.sh | 13 +++++++++++++\n 2 files changed, 40 insertions(+)\n\ndiff --git a/fast-import.c b/fast-import.c\nindex b44d6a467e..caae0819f5 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -2778,6 +2778,7 @@ static void parse_new_tag(const char *arg)\n static void parse_reset_branch(const char *arg)\n {\n \tstruct branch *b;\n+\tconst char *tag_name;\n \n \tb = lookup_branch(arg);\n \tif (b) {\n@@ -2793,6 +2794,32 @@ static void parse_reset_branch(const char *arg)\n \t\tb = new_branch(arg);\n \tread_next_command();\n \tparse_from(b);\n+\tif (b->delete && skip_prefix(b->name, \"refs/tags/\", &tag_name)) {\n+\t\t/*\n+\t\t * Elsewhere, we call dump_branches() before dump_tags(),\n+\t\t * and dump_branches() will handle ref deletions first, so\n+\t\t * in order to make sure the deletion actually takes effect,\n+\t\t * we need to remove the tag from our list of tags to update.\n+\t\t *\n+\t\t * NEEDSWORK: replace list of tags with hashmap for faster\n+\t\t * deletion?\n+\t\t */\n+\t\tstruct tag *t, *prev = NULL;\n+\t\tfor (t = first_tag; t; t = t->next_tag) {\n+\t\t\tif (!strcmp(t->name, tag_name))\n+\t\t\t\tbreak;\n+\t\t\tprev = t;\n+\t\t}\n+\t\tif (t) {\n+\t\t\tif (prev)\n+\t\t\t\tprev->next_tag = t->next_tag;\n+\t\t\telse\n+\t\t\t\tfirst_tag = t->next_tag;\n+\t\t\tif (!t->next_tag)\n+\t\t\t\tlast_tag = prev;\n+\t\t\t/* There is no mem_pool_free(t) function to call. */\n+\t\t}\n+\t}\n \tif (command_buf.len > 0)\n \t\tunread_command_buf = 1;\n }\ndiff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh\nindex 141b7fa35e..74bc41333b 100755\n--- a/t/t9300-fast-import.sh\n+++ b/t/t9300-fast-import.sh\n@@ -85,6 +85,15 @@ test_expect_success 'A: create pack from stdin' '\n \tAn annotated tag that annotates a blob.\n \tEOF\n \n+\ttag to-be-deleted\n+\tfrom :3\n+\tdata <expect <<-EOF &&\n \t:2 $(git rev-parse --verify master:file2)\n-- \n2.23.0.264.g3b9f7f2fc6\n\n"},{"id":"383351","messageId":"20191003202709.26279-4-newren@gmail.com","threadId":"51911","inReplyTo":"20191003202709.26279-1-newren@gmail.com","subject":"[PATCH -v3 3/8] fast-import: allow tags to be identified by mark labels","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-10-03T20:27:04Z","receivedAt":"2019-10-03T20:27:22Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Mark identifiers are used in fast-export and fast-import to provide a\nlabel to refer to earlier content. Blobs are given labels because they\nneed to be referenced in the commits where they first appear with a\ngiven filename, and commits are given labels because they can be the\nparents of other commits. Tags were never given labels, probably\nbecause they were viewed as unnecessary, but that presents two problems:\n\n 1. It leaves us without a way of referring to previous tags if we\n want to create a tag of a tag (or higher nestings).\n 2. It leaves us with no way of recording that a tag has already been\n imported when using --export-marks and --import-marks.\n\nFix these problems by allowing an optional mark label for tags.\n\nSigned-off-by: Elijah Newren \n---\n Documentation/git-fast-import.txt | 1 +\n fast-import.c | 3 ++-\n t/t9300-fast-import.sh | 19 +++++++++++++++++++\n 3 files changed, 22 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt\nindex 0bb276269e..4977869465 100644\n--- a/Documentation/git-fast-import.txt\n+++ b/Documentation/git-fast-import.txt\n@@ -774,6 +774,7 @@ lightweight (non-annotated) tags see the `reset` command below.\n \n ....\n \t'tag' SP LF\n+\tmark?\n \t'from' SP LF\n \toriginal-oid?\n \t'tagger' (SP )? SP LT GT SP LF\ndiff --git a/fast-import.c b/fast-import.c\nindex caae0819f5..5b9e9e3b02 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -2713,6 +2713,7 @@ static void parse_new_tag(const char *arg)\n \t\tfirst_tag = t;\n \tlast_tag = t;\n \tread_next_command();\n+\tparse_mark();\n \n \t/* from ... */\n \tif (!skip_prefix(command_buf.buf, \"from \", &from))\n@@ -2769,7 +2770,7 @@ static void parse_new_tag(const char *arg)\n \tstrbuf_addbuf(&new_data, &msg);\n \tfree(tagger);\n \n-\tif (store_object(OBJ_TAG, &new_data, NULL, &t->oid, 0))\n+\tif (store_object(OBJ_TAG, &new_data, NULL, &t->oid, next_mark))\n \t\tt->pack_id = MAX_PACK_ID;\n \telse\n \t\tt->pack_id = pack_id;\ndiff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh\nindex 74bc41333b..3ad2b2f1ba 100755\n--- a/t/t9300-fast-import.sh\n+++ b/t/t9300-fast-import.sh\n@@ -94,6 +94,23 @@ test_expect_success 'A: create pack from stdin' '\n \treset refs/tags/to-be-deleted\n \tfrom 0000000000000000000000000000000000000000\n \n+\ttag nested\n+\tmark :6\n+\tfrom :4\n+\tdata <\n---\n Documentation/git-fast-import.txt | 22 +++++++++++\n fast-import.c | 62 ++++++++++++++++++++++++++-----\n t/t9300-fast-import.sh | 5 +++\n 3 files changed, 79 insertions(+), 10 deletions(-)\n\ndiff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt\nindex 4977869465..a3f1e0c5e4 100644\n--- a/Documentation/git-fast-import.txt\n+++ b/Documentation/git-fast-import.txt\n@@ -337,6 +337,13 @@ and control the current import process. More detailed discussion\n \t`commit` command. This command is optional and is not\n \tneeded to perform an import.\n \n+`alias`::\n+\tRecord that a mark refers to a given object without first\n+\tcreating any new object. Using --import-marks and referring\n+\tto missing marks will cause fast-import to fail, so aliases\n+\tcan provide a way to set otherwise pruned commits to a valid\n+\tvalue (e.g. the nearest non-pruned ancestor).\n+\n `checkpoint`::\n \tForces fast-import to close the current packfile, generate its\n \tunique SHA-1 checksum and index, and start a new packfile.\n@@ -914,6 +921,21 @@ a data chunk which does not have an LF as its last byte.\n +\n The `LF` after ` LF` is optional (it used to be required).\n \n+`alias`\n+~~~~~~~\n+Record that a mark refers to a given object without first creating any\n+new object.\n+\n+....\n+\t'alias' LF\n+\tmark\n+\t'to' SP LF\n+\tLF?\n+....\n+\n+For a detailed description of `` see above under `from`.\n+\n+\n `checkpoint`\n ~~~~~~~~~~~~\n Forces fast-import to close the current packfile, start a new one, and to\ndiff --git a/fast-import.c b/fast-import.c\nindex 5b9e9e3b02..ac368b3e2b 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -2491,18 +2491,14 @@ static void parse_from_existing(struct branch *b)\n \t}\n }\n \n-static int parse_from(struct branch *b)\n+static int parse_objectish(struct branch *b, const char *objectish)\n {\n-\tconst char *from;\n \tstruct branch *s;\n \tstruct object_id oid;\n \n-\tif (!skip_prefix(command_buf.buf, \"from \", &from))\n-\t\treturn 0;\n-\n \toidcpy(&oid, &b->branch_tree.versions[1].oid);\n \n-\ts = lookup_branch(from);\n+\ts = lookup_branch(objectish);\n \tif (b == s)\n \t\tdie(\"Can't create a branch from itself: %s\", b->name);\n \telse if (s) {\n@@ -2510,8 +2506,8 @@ static int parse_from(struct branch *b)\n \t\toidcpy(&b->oid, &s->oid);\n \t\toidcpy(&b->branch_tree.versions[0].oid, t);\n \t\toidcpy(&b->branch_tree.versions[1].oid, t);\n-\t} else if (*from == ':') {\n-\t\tuintmax_t idnum = parse_mark_ref_eol(from);\n+\t} else if (*objectish == ':') {\n+\t\tuintmax_t idnum = parse_mark_ref_eol(objectish);\n \t\tstruct object_entry *oe = find_mark(idnum);\n \t\tif (oe->type != OBJ_COMMIT)\n \t\t\tdie(\"Mark :%\" PRIuMAX \" not a commit\", idnum);\n@@ -2525,13 +2521,13 @@ static int parse_from(struct branch *b)\n \t\t\t} else\n \t\t\t\tparse_from_existing(b);\n \t\t}\n-\t} else if (!get_oid(from, &b->oid)) {\n+\t} else if (!get_oid(objectish, &b->oid)) {\n \t\tparse_from_existing(b);\n \t\tif (is_null_oid(&b->oid))\n \t\t\tb->delete = 1;\n \t}\n \telse\n-\t\tdie(\"Invalid ref name or SHA1 expression: %s\", from);\n+\t\tdie(\"Invalid ref name or SHA1 expression: %s\", objectish);\n \n \tif (b->branch_tree.tree && !oideq(&oid, &b->branch_tree.versions[1].oid)) {\n \t\trelease_tree_content_recursive(b->branch_tree.tree);\n@@ -2542,6 +2538,26 @@ static int parse_from(struct branch *b)\n \treturn 1;\n }\n \n+static int parse_from(struct branch *b)\n+{\n+\tconst char *from;\n+\n+\tif (!skip_prefix(command_buf.buf, \"from \", &from))\n+\t\treturn 0;\n+\n+\treturn parse_objectish(b, from);\n+}\n+\n+static int parse_objectish_with_prefix(struct branch *b, const char *prefix)\n+{\n+\tconst char *base;\n+\n+\tif (!skip_prefix(command_buf.buf, prefix, &base))\n+\t\treturn 0;\n+\n+\treturn parse_objectish(b, base);\n+}\n+\n static struct hash_list *parse_merge(unsigned int *count)\n {\n \tstruct hash_list *list = NULL, **tail = &list, *n;\n@@ -3087,6 +3103,28 @@ static void parse_progress(void)\n \tskip_optional_lf();\n }\n \n+static void parse_alias(void)\n+{\n+\tstruct object_entry *e;\n+\tstruct branch b;\n+\n+\tskip_optional_lf();\n+\tread_next_command();\n+\n+\t/* mark ... */\n+\tparse_mark();\n+\tif (!next_mark)\n+\t\tdie(_(\"Expected 'mark' command, got %s\"), command_buf.buf);\n+\n+\t/* to ... */\n+\tmemset(&b, 0, sizeof(b));\n+\tif (!parse_objectish_with_prefix(&b, \"to \"))\n+\t\tdie(_(\"Expected 'to' command, got %s\"), command_buf.buf);\n+\te = find_object(&b.oid);\n+\tassert(e);\n+\tinsert_mark(next_mark, e);\n+}\n+\n static char* make_fast_import_path(const char *path)\n {\n \tif (!relative_marks_paths || is_absolute_path(path))\n@@ -3214,6 +3252,8 @@ static int parse_one_feature(const char *feature, int from_stream)\n \t\toption_import_marks(arg, from_stream, 1);\n \t} else if (skip_prefix(feature, \"export-marks=\", &arg)) {\n \t\toption_export_marks(arg);\n+\t} else if (!strcmp(feature, \"alias\")) {\n+\t\t; /* Don't die - this feature is supported */\n \t} else if (!strcmp(feature, \"get-mark\")) {\n \t\t; /* Don't die - this feature is supported */\n \t} else if (!strcmp(feature, \"cat-blob\")) {\n@@ -3370,6 +3410,8 @@ int cmd_main(int argc, const char **argv)\n \t\t\tparse_checkpoint();\n \t\telse if (!strcmp(\"done\", command_buf.buf))\n \t\t\tbreak;\n+\t\telse if (!strcmp(\"alias\", command_buf.buf))\n+\t\t\tparse_alias();\n \t\telse if (starts_with(command_buf.buf, \"progress \"))\n \t\t\tparse_progress();\n \t\telse if (skip_prefix(command_buf.buf, \"feature \", &v))\ndiff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh\nindex 3ad2b2f1ba..41f2a1dad9 100755\n--- a/t/t9300-fast-import.sh\n+++ b/t/t9300-fast-import.sh\n@@ -111,6 +111,10 @@ test_expect_success 'A: create pack from stdin' '\n \tTag of tag of our lovely commit\n \tEOF\n \n+\talias\n+\tmark :8\n+\tto :5\n+\n \tINPUT_END\n \tgit fast-import --export-marks=marks.out \n---\n builtin/fast-export.c | 23 +++++++++++++++++++----\n t/t9350-fast-export.sh | 10 ++++------\n 2 files changed, 23 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/fast-export.c b/builtin/fast-export.c\nindex 5822271c6b..575e47833b 100644\n--- a/builtin/fast-export.c\n+++ b/builtin/fast-export.c\n@@ -1052,11 +1052,16 @@ static void export_marks(char *file)\n \t\terror(\"Unable to write marks file %s.\", file);\n }\n \n-static void import_marks(char *input_file)\n+static void import_marks(char *input_file, int check_exists)\n {\n \tchar line[512];\n-\tFILE *f = xfopen(input_file, \"r\");\n+\tFILE *f;\n+\tstruct stat sb;\n+\n+\tif (check_exists && stat(input_file, &sb))\n+\t\treturn;\n \n+\tf = xfopen(input_file, \"r\");\n \twhile (fgets(line, sizeof(line), f)) {\n \t\tuint32_t mark;\n \t\tchar *line_end, *mark_end;\n@@ -1120,7 +1125,9 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \tstruct rev_info revs;\n \tstruct object_array commits = OBJECT_ARRAY_INIT;\n \tstruct commit *commit;\n-\tchar *export_filename = NULL, *import_filename = NULL;\n+\tchar *export_filename = NULL,\n+\t *import_filename = NULL,\n+\t *import_filename_if_exists = NULL;\n \tuint32_t lastimportid;\n \tstruct string_list refspecs_list = STRING_LIST_INIT_NODUP;\n \tstruct string_list paths_of_changed_objects = STRING_LIST_INIT_DUP;\n@@ -1140,6 +1147,10 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \t\t\t N_(\"Dump marks to this file\")),\n \t\tOPT_STRING(0, \"import-marks\", &import_filename, N_(\"file\"),\n \t\t\t N_(\"Import marks from this file\")),\n+\t\tOPT_STRING(0, \"import-marks-if-exists\",\n+\t\t\t &import_filename_if_exists,\n+\t\t\t N_(\"file\"),\n+\t\t\t N_(\"Import marks from this file if it exists\")),\n \t\tOPT_BOOL(0, \"fake-missing-tagger\", &fake_missing_tagger,\n \t\t\t N_(\"Fake a tagger when tags lack one\")),\n \t\tOPT_BOOL(0, \"full-tree\", &full_tree,\n@@ -1187,8 +1198,12 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \tif (use_done_feature)\n \t\tprintf(\"feature done\\n\");\n \n+\tif (import_filename && import_filename_if_exists)\n+\t\tdie(_(\"Cannot pass both --import-marks and --import-marks-if-exists\"));\n \tif (import_filename)\n-\t\timport_marks(import_filename);\n+\t\timport_marks(import_filename, 0);\n+\telse if (import_filename_if_exists)\n+\t\timport_marks(import_filename_if_exists, 1);\n \tlastimportid = last_idnum;\n \n \tif (import_filename && revs.prune_data.nr)\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex d32ff41859..ea84e2f173 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -580,17 +580,15 @@ test_expect_success 'fast-export quotes pathnames' '\n '\n \n test_expect_success 'test bidirectionality' '\n-\t>marks-cur &&\n-\t>marks-new &&\n \tgit init marks-test &&\n-\tgit fast-export --export-marks=marks-cur --import-marks=marks-cur --branches | \\\n-\tgit --git-dir=marks-test/.git fast-import --export-marks=marks-new --import-marks=marks-new &&\n+\tgit fast-export --export-marks=marks-cur --import-marks-if-exists=marks-cur --branches | \\\n+\tgit --git-dir=marks-test/.git fast-import --export-marks=marks-new --import-marks-if-exists=marks-new &&\n \t(cd marks-test &&\n \tgit reset --hard &&\n \techo Wohlauf > file &&\n \tgit commit -a -m \"back in time\") &&\n-\tgit --git-dir=marks-test/.git fast-export --export-marks=marks-new --import-marks=marks-new --branches | \\\n-\tgit fast-import --export-marks=marks-cur --import-marks=marks-cur\n+\tgit --git-dir=marks-test/.git fast-export --export-marks=marks-new --import-marks-if-exists=marks-new --branches | \\\n+\tgit fast-import --export-marks=marks-cur --import-marks-if-exists=marks-cur\n '\n \n cat > expected << EOF\n-- \n2.23.0.264.g3b9f7f2fc6\n\n"},{"id":"383354","messageId":"20191003202709.26279-8-newren@gmail.com","threadId":"51911","inReplyTo":"20191003202709.26279-1-newren@gmail.com","subject":"[PATCH -v3 7/8] t9350: add tests for tags of things other than a commit","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-10-03T20:27:08Z","receivedAt":"2019-10-03T20:27:27Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Multiple changes here:\n * add a test for a tag of a blob\n * add a test for a tag of a tag of a commit\n * add a comment to the tests for (possibly nested) tags of trees,\n making it clear that these tests are doing much less than you might\n expect\n\nSigned-off-by: Elijah Newren \n---\n t/t9350-fast-export.sh | 31 +++++++++++++++++++++++++++++++\n 1 file changed, 31 insertions(+)\n\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex b3fca6ffba..9ab281e4b9 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -540,10 +540,41 @@ test_expect_success 'tree_tag' '\n '\n \n # NEEDSWORK: not just check return status, but validate the output\n+# Note that these tests DO NOTHING other than print a warning that\n+# they are ommitting the one tag we asked them to export (because the\n+# tags resolve to a tree). They exist just to make sure we do not\n+# abort but instead just warn.\n test_expect_success 'tree_tag-obj' 'git fast-export tree_tag-obj'\n test_expect_success 'tag-obj_tag' 'git fast-export tag-obj_tag'\n test_expect_success 'tag-obj_tag-obj' 'git fast-export tag-obj_tag-obj'\n \n+test_expect_success 'handling tags of blobs' '\n+\tgit tag -a -m \"Tag of a blob\" blobtag $(git rev-parse master:file) &&\n+\tgit fast-export blobtag >actual &&\n+\tcat >expect <<-EOF &&\n+\tblob\n+\tmark :1\n+\tdata 9\n+\tdie Luft\n+\n+\ttag blobtag\n+\tfrom :1\n+\ttagger $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+\tdata 14\n+\tTag of a blob\n+\n+\tEOF\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_failure 'handling nested tags' '\n+\tgit tag -a -m \"This is a nested tag\" nested muss &&\n+\tgit fast-export --mark-tags nested >output &&\n+\tgrep \"^from $ZERO_OID$\" output &&\n+\tgrep \"^tag nested$\" output >tag_lines &&\n+\ttest_line_count = 2 tag_lines\n+'\n+\n test_expect_success 'directory becomes symlink' '\n \tgit init dirtosymlink &&\n \tgit init result &&\n-- \n2.23.0.264.g3b9f7f2fc6\n\n"},{"id":"383355","messageId":"20191003202709.26279-9-newren@gmail.com","threadId":"51911","inReplyTo":"20191003202709.26279-1-newren@gmail.com","subject":"[PATCH -v3 8/8] fast-export: handle nested tags","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-10-03T20:27:09Z","receivedAt":"2019-10-03T20:27:28Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Signed-off-by: Elijah Newren \n---\n builtin/fast-export.c | 30 ++++++++++++++++++------------\n t/t9350-fast-export.sh | 2 +-\n 2 files changed, 19 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/fast-export.c b/builtin/fast-export.c\nindex d32e1e9327..58a74de42a 100644\n--- a/builtin/fast-export.c\n+++ b/builtin/fast-export.c\n@@ -843,22 +843,28 @@ static void handle_tag(const char *name, struct tag *tag)\n \t\t\tfree(buf);\n \t\t\treturn;\n \t\tcase REWRITE:\n-\t\t\tif (tagged->type != OBJ_COMMIT) {\n-\t\t\t\tdie(\"tag %s tags unexported %s!\",\n-\t\t\t\t oid_to_hex(&tag->object.oid),\n-\t\t\t\t type_name(tagged->type));\n-\t\t\t}\n-\t\t\tp = rewrite_commit((struct commit *)tagged);\n-\t\t\tif (!p) {\n-\t\t\t\tprintf(\"reset %s\\nfrom %s\\n\\n\",\n-\t\t\t\t name, oid_to_hex(&null_oid));\n-\t\t\t\tfree(buf);\n-\t\t\t\treturn;\n+\t\t\tif (tagged->type == OBJ_TAG && !mark_tags) {\n+\t\t\t\tdie(_(\"Error: Cannot export nested tags unless --mark-tags is specified.\"));\n+\t\t\t} else if (tagged->type == OBJ_COMMIT) {\n+\t\t\t\tp = rewrite_commit((struct commit *)tagged);\n+\t\t\t\tif (!p) {\n+\t\t\t\t\tprintf(\"reset %s\\nfrom %s\\n\\n\",\n+\t\t\t\t\t name, oid_to_hex(&null_oid));\n+\t\t\t\t\tfree(buf);\n+\t\t\t\t\treturn;\n+\t\t\t\t}\n+\t\t\t\ttagged_mark = get_object_mark(&p->object);\n+\t\t\t} else {\n+\t\t\t\t/* tagged->type is either OBJ_BLOB or OBJ_TAG */\n+\t\t\t\ttagged_mark = get_object_mark(tagged);\n \t\t\t}\n-\t\t\ttagged_mark = get_object_mark(&p->object);\n \t\t}\n \t}\n \n+\tif (tagged->type == OBJ_TAG) {\n+\t\tprintf(\"reset %s\\nfrom %s\\n\\n\",\n+\t\t name, oid_to_hex(&null_oid));\n+\t}\n \tif (starts_with(name, \"refs/tags/\"))\n \t\tname += 10;\n \tprintf(\"tag %s\\n\", name);\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex 9ab281e4b9..2e4e214815 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -567,7 +567,7 @@ test_expect_success 'handling tags of blobs' '\n \ttest_cmp expect actual\n '\n \n-test_expect_failure 'handling nested tags' '\n+test_expect_success 'handling nested tags' '\n \tgit tag -a -m \"This is a nested tag\" nested muss &&\n \tgit fast-export --mark-tags nested >output &&\n \tgrep \"^from $ZERO_OID$\" output &&\n-- \n2.23.0.264.g3b9f7f2fc6\n\n"},{"id":"383356","messageId":"20191003202709.26279-7-newren@gmail.com","threadId":"51911","inReplyTo":"20191003202709.26279-1-newren@gmail.com","subject":"[PATCH -v3 6/8] fast-export: allow user to request tags be marked with --mark-tags","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-10-03T20:27:07Z","receivedAt":"2019-10-03T20:27:28Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Add a new option, --mark-tags, which will output mark identifiers with\neach tag object. This improves the incremental export story with\n--export-marks since it will allow us to record that annotated tags have\nbeen exported, and it is also needed as a step towards supporting nested\ntags.\n\nSigned-off-by: Elijah Newren \n---\n Documentation/git-fast-export.txt | 17 +++++++++++++----\n builtin/fast-export.c | 7 +++++++\n t/t9350-fast-export.sh | 14 ++++++++++++++\n 3 files changed, 34 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt\nindex cc940eb9ad..c522b34f7b 100644\n--- a/Documentation/git-fast-export.txt\n+++ b/Documentation/git-fast-export.txt\n@@ -75,11 +75,20 @@ produced incorrect results if you gave these options.\n \tBefore processing any input, load the marks specified in\n \t. The input file must exist, must be readable, and\n \tmust use the same format as produced by --export-marks.\n+\n+--mark-tags::\n+\tIn addition to labelling blobs and commits with mark ids, also\n+\tlabel tags. This is useful in conjunction with\n+\t`--export-marks` and `--import-marks`, and is also useful (and\n+\tnecessary) for exporting of nested tags. It does not hurt\n+\tother cases and would be the default, but many fast-import\n+\tfrontends are not prepared to accept tags with mark\n+\tidentifiers.\n +\n-Any commits that have already been marked will not be exported again.\n-If the backend uses a similar --import-marks file, this allows for\n-incremental bidirectional exporting of the repository by keeping the\n-marks the same across runs.\n+Any commits (or tags) that have already been marked will not be\n+exported again. If the backend uses a similar --import-marks file,\n+this allows for incremental bidirectional exporting of the repository\n+by keeping the marks the same across runs.\n \n --fake-missing-tagger::\n \tSome old repositories have tags without a tagger. The\ndiff --git a/builtin/fast-export.c b/builtin/fast-export.c\nindex 575e47833b..d32e1e9327 100644\n--- a/builtin/fast-export.c\n+++ b/builtin/fast-export.c\n@@ -40,6 +40,7 @@ static int no_data;\n static int full_tree;\n static int reference_excluded_commits;\n static int show_original_ids;\n+static int mark_tags;\n static struct string_list extra_refs = STRING_LIST_INIT_NODUP;\n static struct string_list tag_refs = STRING_LIST_INIT_NODUP;\n static struct refspec refspecs = REFSPEC_INIT_FETCH;\n@@ -861,6 +862,10 @@ static void handle_tag(const char *name, struct tag *tag)\n \tif (starts_with(name, \"refs/tags/\"))\n \t\tname += 10;\n \tprintf(\"tag %s\\n\", name);\n+\tif (mark_tags) {\n+\t\tmark_next_object(&tag->object);\n+\t\tprintf(\"mark :%\"PRIu32\"\\n\", last_idnum);\n+\t}\n \tif (tagged_mark)\n \t\tprintf(\"from :%d\\n\", tagged_mark);\n \telse\n@@ -1165,6 +1170,8 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \t\t\t &reference_excluded_commits, N_(\"Reference parents which are not in fast-export stream by object id\")),\n \t\tOPT_BOOL(0, \"show-original-ids\", &show_original_ids,\n \t\t\t N_(\"Show original object ids of blobs/commits\")),\n+\t\tOPT_BOOL(0, \"mark-tags\", &mark_tags,\n+\t\t\t N_(\"Label tags with mark ids\")),\n \n \t\tOPT_END()\n \t};\ndiff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh\nindex ea84e2f173..b3fca6ffba 100755\n--- a/t/t9350-fast-export.sh\n+++ b/t/t9350-fast-export.sh\n@@ -66,6 +66,20 @@ test_expect_success 'fast-export ^muss^{commit} muss' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'fast-export --mark-tags ^muss^{commit} muss' '\n+\tgit fast-export --mark-tags --tag-of-filtered-object=rewrite ^muss^{commit} muss >actual &&\n+\tcat >expected <<-EOF &&\n+\ttag muss\n+\tmark :1\n+\tfrom $(git rev-parse --verify muss^{commit})\n+\t$(git cat-file tag muss | grep tagger)\n+\tdata 9\n+\tvalentin\n+\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'fast-export master~2..master' '\n \n \tgit fast-export master~2..master >actual &&\n-- \n2.23.0.264.g3b9f7f2fc6\n\n"},{"id":"383398","messageId":"xmqq7e5l83e4.fsf@gitster-ct.c.googlers.com","threadId":"51911","inReplyTo":"20191003202709.26279-1-newren@gmail.com","subject":"Re: [PATCH -v3 0/8] fast export/import: handle nested tags, improve incremental exports","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-10-04T05:51:15Z","receivedAt":"2019-10-04T05:52:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Elijah Newren writes:\n\n> This series improves the incremental export story for fast-export and\n> fast-import (--export-marks and --import-marks fell a bit short),\n> fixes a couple small export/import bugs, and enables handling nested\n> tags. In particular, the nested tags handling makes it so that\n> fast-export and fast-import can finally handle the git.git repo.\n>\n> Changes since v2 (full range-diff below):\n> - Code cleanup of patch 2 suggested by René\n>\n> Elijah Newren (8):\n> fast-export: fix exporting a tag and nothing else\n> fast-import: fix handling of deleted tags\n> fast-import: allow tags to be identified by mark labels\n> fast-import: add support for new 'alias' command\n> fast-export: add support for --import-marks-if-exists\n> fast-export: allow user to request tags be marked with --mark-tags\n> t9350: add tests for tags of things other than a commit\n> fast-export: handle nested tags\n\nThanks, both. Replaced.\n"}]}