{"thread":{"id":"14534","subject":"[RFC PATCH v2] Support gitlinks in fast-import.","startedAt":"2008-07-19T12:21:24Z","lastAt":"2008-07-20T12:07:49Z","messageCount":2,"participants":["Alexander Gavrilov","Johannes Schindelin"],"isPatch":true,"patchVersion":2,"patchTotal":null},"messages":[{"id":"83944","messageId":"200807191621.25654.angavrilov@gmail.com","threadId":"14534","inReplyTo":null,"subject":"[RFC PATCH v2] Support gitlinks in fast-import.","fromName":"Alexander Gavrilov","fromEmail":"angavrilov@gmail.com","sentAt":"2008-07-19T12:21:24Z","receivedAt":"2008-07-19T12:21:24Z","isPatch":true,"sender":{"key":"angavrilov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/42666?v=4"},"body":"Currently fast-import/export cannot be used for\nrepositories with submodules. This patch extends\nthe relevant programs to make them correctly\nprocess gitlinks.\n\nLinks can be represented by two forms of the\nModify command:\n\nM 160000 SHA1 some/path\n\nwhich sets the link target explicitly, or\n\nM 160000 :mark some/path\n\nwhere the mark refers to a commit. The latter\nform can be used by importing tools to build\nall submodules simultaneously in one physical\nrepository, and then simply fetch them apart.\n\nSigned-off-by: Alexander Gavrilov <angavrilov@gmail.com>\n---\n\n\tChanges:\n\t1) Fixed the typo in the documentation.\n\t2) Gitlinks are always exported as SHA.\n\t3) Added some comments.\n\t4) Added tests.\n\n\t-- Alexander\n\n Documentation/git-fast-import.txt |    3 +\n builtin-fast-export.c             |   17 +++-\n fast-import.c                     |   14 +++-\n t/t9300-fast-import.sh            |  152 +++++++++++++++++++++++++++++++++++++\n t/t9301-fast-export.sh            |   42 ++++++++++\n 5 files changed, 223 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt\nindex 395c055..d510ddb 100644\n--- a/Documentation/git-fast-import.txt\n+++ b/Documentation/git-fast-import.txt\n@@ -481,6 +481,9 @@ in octal.  Git only supports the following modes:\n   what you want.\n * `100755` or `755`: A normal, but executable, file.\n * `120000`: A symlink, the content of the file will be the link target.\n+* `160000`: A gitlink, SHA-1 of the object refers to a commit in\n+  another repository. Git links can only be specified by SHA or through\n+  a commit mark. They are used to implement submodules.\n \n In both formats `<path>` is the complete path of the file to be added\n (if not already existing) or modified (if already existing).\ndiff --git a/builtin-fast-export.c b/builtin-fast-export.c\nindex d0a462f..30ccbd5 100644\n--- a/builtin-fast-export.c\n+++ b/builtin-fast-export.c\n@@ -122,9 +122,16 @@ static void show_filemodify(struct diff_queue_struct *q,\n \t\tif (is_null_sha1(spec->sha1))\n \t\t\tprintf(\"D %s\\n\", spec->path);\n \t\telse {\n-\t\t\tstruct object *object = lookup_object(spec->sha1);\n-\t\t\tprintf(\"M %06o :%d %s\\n\", spec->mode,\n-\t\t\t       get_object_mark(object), spec->path);\n+\t\t\t/* Links refer to objects in another repositories, so\n+\t\t\t   output the SHA-1 verbatim */\n+\t\t\tif (S_ISGITLINK(spec->mode))\n+\t\t\t\tprintf(\"M %06o %s %s\\n\", spec->mode,\n+\t\t\t\t       sha1_to_hex(spec->sha1), spec->path);\n+\t\t\telse {\n+\t\t\t\tstruct object *object = lookup_object(spec->sha1);\n+\t\t\t\tprintf(\"M %06o :%d %s\\n\", spec->mode,\n+\t\t\t\t       get_object_mark(object), spec->path);\n+\t\t\t}\n \t\t}\n \t}\n }\n@@ -182,8 +189,10 @@ static void handle_commit(struct commit *commit, struct rev_info *rev)\n \t\tdiff_root_tree_sha1(commit->tree->object.sha1,\n \t\t\t\t    \"\", &rev->diffopt);\n \n+\t/* Export the referenced blobs, and remember the marks */\n \tfor (i = 0; i < diff_queued_diff.nr; i++)\n-\t\thandle_object(diff_queued_diff.queue[i]->two->sha1);\n+\t\tif (!S_ISGITLINK(diff_queued_diff.queue[i]->two->mode))\n+\t\t\thandle_object(diff_queued_diff.queue[i]->two->sha1);\n \n \tmark_object(&commit->object);\n \tif (!is_encoding_utf8(encoding))\ndiff --git a/fast-import.c b/fast-import.c\nindex e72b286..cb6d1ee 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -1868,6 +1868,7 @@ static void file_change_m(struct branch *b)\n \tcase S_IFREG | 0644:\n \tcase S_IFREG | 0755:\n \tcase S_IFLNK:\n+\tcase S_IFGITLINK:\n \tcase 0644:\n \tcase 0755:\n \t\t/* ok */\n@@ -1900,7 +1901,18 @@ static void file_change_m(struct branch *b)\n \t\tp = uq.buf;\n \t}\n \n-\tif (inline_data) {\n+\tif (S_ISGITLINK(mode)) {\n+\t\tif (inline_data)\n+\t\t\tdie(\"Git links cannot be specified 'inline': %s\",\n+\t\t\t\tcommand_buf.buf);\n+\t\telse if (oe) {\n+\t\t\tif (oe->type != OBJ_COMMIT)\n+\t\t\t\tdie(\"Not a commit (actually a %s): %s\",\n+\t\t\t\t\ttypename(oe->type), command_buf.buf);\n+\t\t}\n+\t\t/* else: accept the sha1 without check, as its object\n+\t\t         is expected to be in another repository */\n+\t} else if (inline_data) {\n \t\tstatic struct strbuf buf = STRBUF_INIT;\n \n \t\tif (p != uq.buf) {\ndiff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh\nindex 5edf56f..235a8a6 100755\n--- a/t/t9300-fast-import.sh\n+++ b/t/t9300-fast-import.sh\n@@ -918,4 +918,156 @@ test_expect_success \\\n \t grep \"progress \" <input >expect &&\n \t test_cmp expect actual'\n \n+###\n+### series P (gitlinks)\n+###\n+\n+cat >input <<INPUT_END\n+blob\n+mark :1\n+data 10\n+test file\n+\n+reset refs/heads/sub\n+commit refs/heads/sub\n+mark :2\n+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+data 12\n+sub_initial\n+M 100644 :1 file\n+\n+blob\n+mark :3\n+data <<DATAEND\n+[submodule \"sub\"]\n+\tpath = sub\n+\turl = \"`pwd`/sub\"\n+DATAEND\n+\n+commit refs/heads/subuse1\n+mark :4\n+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+data 8\n+initial\n+from refs/heads/master\n+M 100644 :3 .gitmodules\n+M 160000 :2 sub\n+\n+blob\n+mark :5\n+data 20\n+test file\n+more data\n+\n+commit refs/heads/sub\n+mark :6\n+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+data 11\n+sub_second\n+from :2\n+M 100644 :5 file\n+\n+commit refs/heads/subuse1\n+mark :7\n+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+data 7\n+second\n+from :4\n+M 160000 :6 sub\n+\n+INPUT_END\n+\n+test_expect_success \\\n+\t'P: supermodule & submodule mix' \\\n+\t'git-fast-import <input &&\n+\t git checkout subuse1 &&\n+\t rm -rf sub && mkdir sub && cd sub &&\n+\t git init &&\n+\t git fetch .. refs/heads/sub:refs/heads/master &&\n+\t git checkout master &&\n+\t cd .. &&\n+\t git submodule init &&\n+\t git submodule update'\n+\n+SUBLAST=$(git-rev-parse --verify sub)\n+SUBPREV=$(git-rev-parse --verify sub^)\n+\n+cat >input <<INPUT_END\n+blob\n+mark :1\n+data <<DATAEND\n+[submodule \"sub\"]\n+\tpath = sub\n+\turl = \"`pwd`/sub\"\n+DATAEND\n+\n+commit refs/heads/subuse2\n+mark :2\n+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+data 8\n+initial\n+from refs/heads/master\n+M 100644 :1 .gitmodules\n+M 160000 $SUBPREV sub\n+\n+commit refs/heads/subuse2\n+mark :3\n+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+data 7\n+second\n+from :2\n+M 160000 $SUBLAST sub\n+\n+INPUT_END\n+\n+test_expect_success \\\n+\t'P: verbatim SHA gitlinks' \\\n+\t'git branch -D sub &&\n+\t git gc && git prune &&\n+\t git-fast-import <input &&\n+\t test $(git-rev-parse --verify subuse2) = $(git-rev-parse --verify subuse1)'\n+\n+test_tick\n+cat >input <<INPUT_END\n+commit refs/heads/subuse3\n+mark :1\n+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+data <<COMMIT\n+corrupt\n+COMMIT\n+\n+from refs/heads/subuse2\n+M 160000 inline sub\n+data <<DATA\n+$SUBPREV\n+DATA\n+\n+INPUT_END\n+\n+test_expect_success 'P: fail on inline gitlink' '\n+    ! git-fast-import <input'\n+\n+test_tick\n+cat >input <<INPUT_END\n+blob\n+mark :1\n+data <<DATA\n+$SUBPREV\n+DATA\n+\n+commit refs/heads/subuse3\n+mark :2\n+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE\n+data <<COMMIT\n+corrupt\n+COMMIT\n+\n+from refs/heads/subuse2\n+M 160000 :1 sub\n+\n+INPUT_END\n+\n+test_expect_success 'P: fail on blob mark in gitlink' '\n+    ! git-fast-import <input'\n+\n test_done\ndiff --git a/t/t9301-fast-export.sh b/t/t9301-fast-export.sh\nindex f09bfb1..f18eec9 100755\n--- a/t/t9301-fast-export.sh\n+++ b/t/t9301-fast-export.sh\n@@ -120,4 +120,46 @@ test_expect_success 'signed-tags=strip' '\n \n '\n \n+test_expect_success 'setup submodule' '\n+\n+\tgit checkout -f master &&\n+\tmkdir sub &&\n+\tcd sub &&\n+\tgit init  &&\n+\techo test file > file &&\n+\tgit add file &&\n+\tgit commit -m sub_initial &&\n+\tcd .. &&\n+\tgit submodule add \"`pwd`/sub\" sub &&\n+\tgit commit -m initial &&\n+\ttest_tick &&\n+\tcd sub &&\n+\techo more data >> file &&\n+\tgit add file &&\n+\tgit commit -m sub_second &&\n+\tcd .. &&\n+\tgit add sub &&\n+\tgit commit -m second\n+\n+'\n+\n+test_expect_success 'submodule fast-export | fast-import' '\n+\n+\tSUBENT1=$(git ls-tree master^ sub) &&\n+\tSUBENT2=$(git ls-tree master sub) &&\n+\trm -rf new &&\n+\tmkdir new &&\n+\tgit --git-dir=new/.git init &&\n+\tgit fast-export --signed-tags=strip --all |\n+\t(cd new &&\n+\t git fast-import &&\n+\t test \"$SUBENT1\" = \"$(git ls-tree refs/heads/master^ sub)\" &&\n+\t test \"$SUBENT2\" = \"$(git ls-tree refs/heads/master sub)\" &&\n+\t git checkout master &&\n+\t git submodule init &&\n+\t git submodule update &&\n+\t cmp sub/file ../sub/file)\n+\n+'\n+\n test_done\n-- \n1.5.6.3.18.gfe82\n"},{"id":"84025","messageId":"alpine.DEB.1.00.0807201404280.3305@eeepc-johanness","threadId":"14534","inReplyTo":"200807191621.25654.angavrilov@gmail.com","subject":"Re: [RFC PATCH v2] Support gitlinks in fast-import.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-07-20T12:07:49Z","receivedAt":"2008-07-20T12:07:49Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sat, 19 Jul 2008, Alexander Gavrilov wrote:\n\n> Links can be represented by two forms of the\n> Modify command:\n> \n> M 160000 SHA1 some/path\n> \n> which sets the link target explicitly, or\n> \n> M 160000 :mark some/path\n> \n> where the mark refers to a commit. The latter\n> form can be used by importing tools to build\n> all submodules simultaneously in one physical\n> repository, and then simply fetch them apart.\n\nIt seems that people like to do this, have their submodule branches in the \nsuperproject.  I wonder if we should bless this by installing an alternate \nin the \"git submodule init\"ed submodules, or even do it during a \"git \nsubmodule update\".\n\n> +\t\t\t/* Links refer to objects in another repositories, so\n\ns/another/other/\n\nThanks, especially for the tests,\nDscho\n"}]}