{"thread":{"id":"46355","subject":"[PATCH] RFC: A new type of symbolic refs","startedAt":"2017-07-11T01:06:52Z","lastAt":"2017-07-18T19:04:06Z","messageCount":7,"participants":["Stefan Beller","Philip Oakley","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"324144","messageId":"20170711010639.31398-1-sbeller@google.com","threadId":"46355","inReplyTo":null,"subject":"[PATCH] RFC: A new type of symbolic refs","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-07-11T01:06:39Z","receivedAt":"2017-07-11T01:06:52Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"A symbolic ref can currently only point at (another symbolic) ref.\nHowever are use cases for symbolic refs to point at other things\nrepresenting a commit-ish, such as gitlink entries in other\nrepositories.  In this use case we can use such a symbolic link\nto strengthen the relationship between a submodule and a superproject.\nExamples:\n\n1) It makes it easier to explain when we recurse into submodules and\n   to which sha1 the submodule is updated.\n\n   Currently a submodule (or any repo) is either on a branch (i.e.\n   HEAD is a symbolic ref) or is in 'detached HEAD' state (HEAD is\n   a direct ref).\n   For submodules it is wrong to be on its own branch if it wants to be\n   controlled by the superproject as being on its own branch signals that\n   the submodule is independent and can move the HEAD freely.\n   Being in 'detached HEAD' state is the alternative to that and was\n   chosen when git-submodule was implemented, but it is equally wrong;\n   the lesser of two evils.\n\n   Semantically the correct way to state a submodule is part of the\n   superproject is by pointing its HEAD to the superproject.\n\n   We do have \"reset/checkout --recurse-submodules\" now, but it is\n   hard to explain what actually happens there (Which submodules are\n   updated? All of them! -- But I want a subset only!)\n\n   With this new mode of symbolic refs, any submodule that tracks the\n   superproject, would 'automatically follow' the superproject as the\n   submodules HEAD moves when the superproject changes.\n\n2) \"git -C submodule commit\" would behave the same as it would on branch\n   nowadays: It would make the commit in the submodule and then change\n   the target of the symbolic ref which would be the index of the\n   superproject! That implies that you do not need to 'git add' the\n   submodule to the superproject, but have it done automatically.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n cache.h              |  2 ++\n refs/files-backend.c | 10 ++++++++++\n submodule.c          | 40 ++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 52 insertions(+)\n\ndiff --git a/cache.h b/cache.h\nindex 71fe092644..4f79d23202 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -2029,4 +2029,6 @@ void sleep_millisec(int millisec);\n  */\n void safe_create_dir(const char *dir, int share);\n \n+extern int read_external_symref(struct strbuf *from, struct strbuf *out);\n+\n #endif /* CACHE_H */\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 0404f2c233..f56f7b87ce 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -713,6 +713,16 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \t\tgoto out;\n \t}\n \n+\tif (starts_with(buf, \"repo:\")) {\n+\t\tif (read_external_symref(&sb_contents, referent)) {\n+\t\t\t*type |= REF_ISBROKEN;\n+\t\t\tret = -1;\n+\t\t\tgoto out;\n+\t\t}\n+\t\t*type |= REF_ISSYMREF;\n+\t\tret = 0;\n+\t}\n+\n \t/*\n \t * Please note that FETCH_HEAD has additional\n \t * data after the sha.\ndiff --git a/submodule.c b/submodule.c\nindex da2b484879..7297f90485 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -2037,3 +2037,43 @@ int submodule_to_gitdir(struct strbuf *buf, const char *submodule)\n cleanup:\n \treturn ret;\n }\n+\n+int read_external_symref(struct strbuf *from, struct strbuf *out)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tconst char *repo, *gitlink;\n+\tint hint, code;\n+\tstruct strbuf **split = strbuf_split(from, 0);\n+\tstruct strbuf cmd_out = STRBUF_INIT;\n+\n+\tif (!split[0] || !split[1])\n+\t\treturn -1;\n+\n+\trepo = split[0]->buf + 5; /* skip 'repo:' */\n+\tgitlink = split[1]->buf;\n+\n+\targv_array_pushl(&cp.args,\n+\t\t\t\"ignored-first-arg\",\n+\t\t\t\"-C\", repo,\n+\t\t\t\"ls-tree\", \"-z\", \"HEAD\", \"--\", gitlink, NULL);\n+\n+\t/*\n+\t * 17 accounts for '160000 commit ',\n+\t * the \\t before path and trailing \\0.\n+\t */\n+\thint = 17 + GIT_SHA1_HEXSZ + split[1]->len;\n+\tcode = capture_command(&cp, &cmd_out, hint);\n+\n+\tstrbuf_release(split[0]);\n+\tstrbuf_release(split[1]);\n+\n+\tif (!code) {\n+\t\tstrbuf_reset(out);\n+\t\tstrbuf_add(out, cmd_out.buf + strlen(\"160000 commit \"),\n+\t\t\t   GIT_SHA1_HEXSZ);\n+\t} else\n+\t\treturn -1;\n+\n+\treturn 0;\n+}\n+\n-- \n2.13.2.695.g117ddefdb4\n\n"},{"id":"324621","messageId":"B998F81875C940B1B7D4D26764B44B04@PhilipOakley","threadId":"46355","inReplyTo":"20170711010639.31398-1-sbeller@google.com","subject":"Re: [PATCH] RFC: A new type of symbolic refs","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":"2017-07-16T13:04:13Z","receivedAt":"2017-07-16T13:04:23Z","isPatch":true,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"Stefan Beller\" <sbeller@google.com>\nSent: Tuesday, July 11, 2017 2:06 AM\n>A symbolic ref can currently only point at (another symbolic) ref.\n> However are use cases for symbolic refs to point at other things\n> representing a commit-ish, such as gitlink entries in other\n> repositories.  In this use case we can use such a symbolic link\n> to strengthen the relationship between a submodule and a superproject.\n> Examples:\n\nIf I understand this correctly, the new type is the 'starts_with(buf, ))'.\n\nIt just wasn't obvious from the text that the new type is \"repo:\" as you \nnever spell it out in the commit message. Should it be included in the \nmessage?\n\nHave I understood correctly?\n\n--\nPhilip\n[I'll be off-line till Friday, so will pick up then]\n\n>\n> 1) It makes it easier to explain when we recurse into submodules and\n>   to which sha1 the submodule is updated.\n>\n>   Currently a submodule (or any repo) is either on a branch (i.e.\n>   HEAD is a symbolic ref) or is in 'detached HEAD' state (HEAD is\n>   a direct ref).\n>   For submodules it is wrong to be on its own branch if it wants to be\n>   controlled by the superproject as being on its own branch signals that\n>   the submodule is independent and can move the HEAD freely.\n>   Being in 'detached HEAD' state is the alternative to that and was\n>   chosen when git-submodule was implemented, but it is equally wrong;\n>   the lesser of two evils.\n>\n>   Semantically the correct way to state a submodule is part of the\n>   superproject is by pointing its HEAD to the superproject.\n>\n>   We do have \"reset/checkout --recurse-submodules\" now, but it is\n>   hard to explain what actually happens there (Which submodules are\n>   updated? All of them! -- But I want a subset only!)\n>\n>   With this new mode of symbolic refs, any submodule that tracks the\n>   superproject, would 'automatically follow' the superproject as the\n>   submodules HEAD moves when the superproject changes.\n>\n> 2) \"git -C submodule commit\" would behave the same as it would on branch\n>   nowadays: It would make the commit in the submodule and then change\n>   the target of the symbolic ref which would be the index of the\n>   superproject! That implies that you do not need to 'git add' the\n>   submodule to the superproject, but have it done automatically.\n>\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n> cache.h              |  2 ++\n> refs/files-backend.c | 10 ++++++++++\n> submodule.c          | 40 ++++++++++++++++++++++++++++++++++++++++\n> 3 files changed, 52 insertions(+)\n>\n> diff --git a/cache.h b/cache.h\n> index 71fe092644..4f79d23202 100644\n> --- a/cache.h\n> +++ b/cache.h\n> @@ -2029,4 +2029,6 @@ void sleep_millisec(int millisec);\n>  */\n> void safe_create_dir(const char *dir, int share);\n>\n> +extern int read_external_symref(struct strbuf *from, struct strbuf *out);\n> +\n> #endif /* CACHE_H */\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 0404f2c233..f56f7b87ce 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -713,6 +713,16 @@ static int files_read_raw_ref(struct ref_store \n> *ref_store,\n>  goto out;\n>  }\n>\n> + if (starts_with(buf, \"repo:\")) {\n> + if (read_external_symref(&sb_contents, referent)) {\n> + *type |= REF_ISBROKEN;\n> + ret = -1;\n> + goto out;\n> + }\n> + *type |= REF_ISSYMREF;\n> + ret = 0;\n> + }\n> +\n>  /*\n>  * Please note that FETCH_HEAD has additional\n>  * data after the sha.\n> diff --git a/submodule.c b/submodule.c\n> index da2b484879..7297f90485 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -2037,3 +2037,43 @@ int submodule_to_gitdir(struct strbuf *buf, const \n> char *submodule)\n> cleanup:\n>  return ret;\n> }\n> +\n> +int read_external_symref(struct strbuf *from, struct strbuf *out)\n> +{\n> + struct child_process cp = CHILD_PROCESS_INIT;\n> + const char *repo, *gitlink;\n> + int hint, code;\n> + struct strbuf **split = strbuf_split(from, 0);\n> + struct strbuf cmd_out = STRBUF_INIT;\n> +\n> + if (!split[0] || !split[1])\n> + return -1;\n> +\n> + repo = split[0]->buf + 5; /* skip 'repo:' */\n> + gitlink = split[1]->buf;\n> +\n> + argv_array_pushl(&cp.args,\n> + \"ignored-first-arg\",\n> + \"-C\", repo,\n> + \"ls-tree\", \"-z\", \"HEAD\", \"--\", gitlink, NULL);\n> +\n> + /*\n> + * 17 accounts for '160000 commit ',\n> + * the \\t before path and trailing \\0.\n> + */\n> + hint = 17 + GIT_SHA1_HEXSZ + split[1]->len;\n> + code = capture_command(&cp, &cmd_out, hint);\n> +\n> + strbuf_release(split[0]);\n> + strbuf_release(split[1]);\n> +\n> + if (!code) {\n> + strbuf_reset(out);\n> + strbuf_add(out, cmd_out.buf + strlen(\"160000 commit \"),\n> +    GIT_SHA1_HEXSZ);\n> + } else\n> + return -1;\n> +\n> + return 0;\n> +}\n> +\n> -- \n> 2.13.2.695.g117ddefdb4\n> \n\n"},{"id":"324624","messageId":"B205A43E908A4EE3873F81CFEB7B58DA@PhilipOakley","threadId":"46355","inReplyTo":"B998F81875C940B1B7D4D26764B44B04@PhilipOakley","subject":"Re: [PATCH] RFC: A new type of symbolic refs","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":"2017-07-16T14:20:49Z","receivedAt":"2017-07-16T14:21:00Z","isPatch":true,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"Philip Oakley\" <philipoakley@iee.org>\n> From: \"Stefan Beller\" <sbeller@google.com>\n> Sent: Tuesday, July 11, 2017 2:06 AM\n>>A symbolic ref can currently only point at (another symbolic) ref.\n>> However are use cases for symbolic refs to point at other things\n>> representing a commit-ish, such as gitlink entries in other\n>> repositories.  In this use case we can use such a symbolic link\n>> to strengthen the relationship between a submodule and a superproject.\n>> Examples:\n>\n> If I understand this correctly, the new type is the 'starts_with(buf, ))'.\n\nOops, my mistaken message prep accidently removed the \"repo:\" from the \nabove.\n\n>\n> It just wasn't obvious from the text that the new type is \"repo:\" as you \n> never spell it out in the commit message. Should it be included in the \n> message?\n>\n> Have I understood correctly?\n>\n> --\n> Philip\n> [I'll be off-line till Friday, so will pick up then]\n>\n>>\n>> 1) It makes it easier to explain when we recurse into submodules and\n>>   to which sha1 the submodule is updated.\n>>\n>>   Currently a submodule (or any repo) is either on a branch (i.e.\n>>   HEAD is a symbolic ref) or is in 'detached HEAD' state (HEAD is\n>>   a direct ref).\n>>   For submodules it is wrong to be on its own branch if it wants to be\n>>   controlled by the superproject as being on its own branch signals that\n>>   the submodule is independent and can move the HEAD freely.\n>>   Being in 'detached HEAD' state is the alternative to that and was\n>>   chosen when git-submodule was implemented, but it is equally wrong;\n>>   the lesser of two evils.\n>>\n>>   Semantically the correct way to state a submodule is part of the\n>>   superproject is by pointing its HEAD to the superproject.\n>>\n>>   We do have \"reset/checkout --recurse-submodules\" now, but it is\n>>   hard to explain what actually happens there (Which submodules are\n>>   updated? All of them! -- But I want a subset only!)\n>>\n>>   With this new mode of symbolic refs, any submodule that tracks the\n>>   superproject, would 'automatically follow' the superproject as the\n>>   submodules HEAD moves when the superproject changes.\n>>\n>> 2) \"git -C submodule commit\" would behave the same as it would on branch\n>>   nowadays: It would make the commit in the submodule and then change\n>>   the target of the symbolic ref which would be the index of the\n>>   superproject! That implies that you do not need to 'git add' the\n>>   submodule to the superproject, but have it done automatically.\n>>\n>> Signed-off-by: Stefan Beller <sbeller@google.com>\n>> ---\n>> cache.h              |  2 ++\n>> refs/files-backend.c | 10 ++++++++++\n>> submodule.c          | 40 ++++++++++++++++++++++++++++++++++++++++\n>> 3 files changed, 52 insertions(+)\n>>\n>> diff --git a/cache.h b/cache.h\n>> index 71fe092644..4f79d23202 100644\n>> --- a/cache.h\n>> +++ b/cache.h\n>> @@ -2029,4 +2029,6 @@ void sleep_millisec(int millisec);\n>>  */\n>> void safe_create_dir(const char *dir, int share);\n>>\n>> +extern int read_external_symref(struct strbuf *from, struct strbuf \n>> *out);\n>> +\n>> #endif /* CACHE_H */\n>> diff --git a/refs/files-backend.c b/refs/files-backend.c\n>> index 0404f2c233..f56f7b87ce 100644\n>> --- a/refs/files-backend.c\n>> +++ b/refs/files-backend.c\n>> @@ -713,6 +713,16 @@ static int files_read_raw_ref(struct ref_store \n>> *ref_store,\n>>  goto out;\n>>  }\n>>\n>> + if (starts_with(buf, \"repo:\")) {\n>> + if (read_external_symref(&sb_contents, referent)) {\n>> + *type |= REF_ISBROKEN;\n>> + ret = -1;\n>> + goto out;\n>> + }\n>> + *type |= REF_ISSYMREF;\n>> + ret = 0;\n>> + }\n>> +\n>>  /*\n>>  * Please note that FETCH_HEAD has additional\n>>  * data after the sha.\n>> diff --git a/submodule.c b/submodule.c\n>> index da2b484879..7297f90485 100644\n>> --- a/submodule.c\n>> +++ b/submodule.c\n>> @@ -2037,3 +2037,43 @@ int submodule_to_gitdir(struct strbuf *buf, const \n>> char *submodule)\n>> cleanup:\n>>  return ret;\n>> }\n>> +\n>> +int read_external_symref(struct strbuf *from, struct strbuf *out)\n>> +{\n>> + struct child_process cp = CHILD_PROCESS_INIT;\n>> + const char *repo, *gitlink;\n>> + int hint, code;\n>> + struct strbuf **split = strbuf_split(from, 0);\n>> + struct strbuf cmd_out = STRBUF_INIT;\n>> +\n>> + if (!split[0] || !split[1])\n>> + return -1;\n>> +\n>> + repo = split[0]->buf + 5; /* skip 'repo:' */\n>> + gitlink = split[1]->buf;\n>> +\n>> + argv_array_pushl(&cp.args,\n>> + \"ignored-first-arg\",\n>> + \"-C\", repo,\n>> + \"ls-tree\", \"-z\", \"HEAD\", \"--\", gitlink, NULL);\n>> +\n>> + /*\n>> + * 17 accounts for '160000 commit ',\n>> + * the \\t before path and trailing \\0.\n>> + */\n>> + hint = 17 + GIT_SHA1_HEXSZ + split[1]->len;\n>> + code = capture_command(&cp, &cmd_out, hint);\n>> +\n>> + strbuf_release(split[0]);\n>> + strbuf_release(split[1]);\n>> +\n>> + if (!code) {\n>> + strbuf_reset(out);\n>> + strbuf_add(out, cmd_out.buf + strlen(\"160000 commit \"),\n>> +    GIT_SHA1_HEXSZ);\n>> + } else\n>> + return -1;\n>> +\n>> + return 0;\n>> +}\n>> +\n>> -- \n>> 2.13.2.695.g117ddefdb4\n>>\n> \n\n"},{"id":"324663","messageId":"CAGZ79kYjGBg6ep07OCbdGkwZkauZ8T07d-Z7L4_wym02zaOsMA@mail.gmail.com","threadId":"46355","inReplyTo":"B998F81875C940B1B7D4D26764B44B04@PhilipOakley","subject":"Re: [PATCH] RFC: A new type of symbolic refs","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-07-17T19:21:11Z","receivedAt":"2017-07-17T19:21:17Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Sun, Jul 16, 2017 at 6:04 AM, Philip Oakley <philipoakley@iee.org> wrote:\n\n>\n> If I understand this correctly, the new type is the 'starts_with(buf, \"repo:\"))'.\n>\n> It just wasn't obvious from the text that the new type is \"repo:\" as you\n> never spell it out in the commit message. Should it be included in the\n> message?\n>\n> Have I understood correctly?\n\nYes, you understood correctly.\n\nThe idea is to allow submodules to \"borrow\" its refs from the superproject.\nMy original idea was to only borrow HEAD from the superproject, but this\ncan be extended to more branches.\n\nBy having a more generic way of borrowing refs from another repo, this\ncan also be a different repo than the superproject (I don't know the use\ncase for this yet, but let's not have a to specific proposal).\n\nLet's see how this RFC fares in further interest from the community.\n"},{"id":"324686","messageId":"xmqqvamqg2fy.fsf@gitster.mtv.corp.google.com","threadId":"46355","inReplyTo":"20170711010639.31398-1-sbeller@google.com","subject":"Re: [PATCH] RFC: A new type of symbolic refs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-07-17T21:48:17Z","receivedAt":"2017-07-17T21:48:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> +int read_external_symref(struct strbuf *from, struct strbuf *out)\n> +{\n> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> +\tconst char *repo, *gitlink;\n> +\tint hint, code;\n> +\tstruct strbuf **split = strbuf_split(from, 0);\n> +\tstruct strbuf cmd_out = STRBUF_INIT;\n> +\n> +\tif (!split[0] || !split[1])\n> +\t\treturn -1;\n> +\n> +\trepo = split[0]->buf + 5; /* skip 'repo:' */\n> +\tgitlink = split[1]->buf;\n> +\n> +\targv_array_pushl(&cp.args,\n> +\t\t\t\"ignored-first-arg\",\n> +\t\t\t\"-C\", repo,\n> +\t\t\t\"ls-tree\", \"-z\", \"HEAD\", \"--\", gitlink, NULL);\n> +\n> +\t/*\n> +\t * 17 accounts for '160000 commit ',\n> +\t * the \\t before path and trailing \\0.\n> +\t */\n> +\thint = 17 + GIT_SHA1_HEXSZ + split[1]->len;\n> +\tcode = capture_command(&cp, &cmd_out, hint);\n> +\n> +\tstrbuf_release(split[0]);\n> +\tstrbuf_release(split[1]);\n> +\n> +\tif (!code) {\n> +\t\tstrbuf_reset(out);\n> +\t\tstrbuf_add(out, cmd_out.buf + strlen(\"160000 commit \"),\n> +\t\t\t   GIT_SHA1_HEXSZ);\n> +\t} else\n> +\t\treturn -1;\n> +\n> +\treturn 0;\n> +}\n\nThis may help the initial checkout, but to be useful after that, we\nneed to define what happens when an equivalent of \"git update-ref\nHEAD\" is done in the submodule repository, when HEAD is pointing\nelsewhere.  The above only shows read-only operation, which is not\nall that interesting.\n\nI _think_ a symbolic HEAD that points upwards to the gitlink entry in\nthe superproject's index is the easiest to understand and it is\nsomething we can define a clear and useful semantics for.\n\nWhen a recursive checkout of a branch 'foo' is made in the\nsuperproject, the index in the superproject would name the commit in\nthe submodule to be checked out.  We traditionally detech the HEAD\nat the submodule to that commit, but instead we could say \"check the\nindex of the superproject to see where the HEAD should be pointing\nat\" in the submodule.  Either way, immediately after such a\nrecursive checkout, \"git status\" inside the submodule would find\nthat the HEAD points at the commit recorded in the 'foo' branch of\nthe superproject and things are clean.  \n\nAfter you work in the submodule and make a commit, an equivalent of\n\"git update-ref HEAD\" is done behind the scene to update HEAD in the\nsubmodule.  In the traditional world, that is done to detached HEAD\nand nothing else changes, but if the symref HEAD points upwards into\nthe index of the superproject, what needs to be done is very obvious;\nwe do \"git add submodule\" in the superproject.  And this is not just\nlimited to creating a commit in the submodule.  \"reset --hard HEAD~2\"\nin the submodule to rewind the HEAD by two commits would also be an\nupdate to HEAD and through the symref-ness of the HEAD should result\nin an update to the index of the superproject.\n\nHowever, I do not think a good explanation of what should mean when\nthis new-style symbolic HEAD points at a commit in the superproject,\nwhether its limited to its HEAD or a tip of an arbitrary branch that\nmay not even be checked out.  These are not something we can easily\nchange without affecting wider context.  Our submodule, when we make\na new commit, may be ready to advance, but our superproject and\nother submodules may not be ready to be included in a new commit in\nthe superproject.\n\nSo I think the idea this patch illustrates is on to something\ninteresting and potentially useful, but I am not sure if it makes\nsense to tie it to anything but the index of the superproject.\n\nEven if we limit ourselves to pointing at the index of the\nsuperproject, there probably are a handful of interesting issues\nthat need to be clarified (not in the sense of \"this and that issues\nexist, so this won't be a useful feature\", but in the sense of \"we'd\nbe able to do these useful things using this feature, and we need to\nfill in more details\"), such as:\n\n - Making new commits in the submodule available to the upstream.\n   Just like a detached HEAD in the submodule, this is not tied to\n   any concrete branch, and it is unclear how a recursive \"push\"\n   from the superproject should propagate the changes to the\n   upstream of the submodule;\n\n - Switching between branches that bind the same commit for the\n   submodule in the superproject would work just like switching\n   between branches that record the same blob for a path, i.e. it\n   will carry forward a local modification.\n\n - The index entry in the superproject may now have to get involved\n   in fsck and reachability study in the submodule as reachability\n   root.  A corollary to this is that submodules behave more\n   similarly to regular blobs wrt \"git reset --hard\" in the\n   superproject, which is a good thing.  \"git -C submodule commit &&\n   git reset --hard\" will create a new commit in the submodule, add\n   it to the index of the superproject, and then lose that change\n   from the index of the superproject, making the commit\n   unreachable, just like \"edit file && git add file && git reset\n   --hard\" in the superproject will make the blob that records the\n   updated content of the file unreachable.\n\nThanks.\n"},{"id":"324691","messageId":"CAGZ79kY=j-sgrAFwAhP-_9EKBsa6FXARaVuCxM6gPHV7M+1OAA@mail.gmail.com","threadId":"46355","inReplyTo":"xmqqvamqg2fy.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] RFC: A new type of symbolic refs","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-07-17T23:22:36Z","receivedAt":"2017-07-17T23:22:43Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Jul 17, 2017 at 2:48 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> I _think_ a symbolic HEAD that points upwards to the gitlink entry in\n> the superproject's index is the easiest to understand and it is\n> something we can define a clear and useful semantics for.\n\nok, let's start with that. Once we have that we can see if that also\nmakes sense for the non-submodule/superproject world.\n\n> When a recursive checkout of a branch 'foo' is made in the\n> superproject, the index in the superproject would name the commit in\n> the submodule to be checked out.  We traditionally detech the HEAD\n> at the submodule to that commit, but instead we could say \"check the\n> index of the superproject to see where the HEAD should be pointing\n> at\" in the submodule.  Either way, immediately after such a\n> recursive checkout, \"git status\" inside the submodule would find\n> that the HEAD points at the commit recorded in the 'foo' branch of\n> the superproject and things are clean.\n\nRight.\n\n> After you work in the submodule and make a commit, an equivalent of\n> \"git update-ref HEAD\" is done behind the scene to update HEAD in the\n> submodule.  In the traditional world, that is done to detached HEAD\n> and nothing else changes, but if the symref HEAD points upwards into\n> the index of the superproject, what needs to be done is very obvious;\n> we do \"git add submodule\" in the superproject.  And this is not just\n> limited to creating a commit in the submodule.  \"reset --hard HEAD~2\"\n> in the submodule to rewind the HEAD by two commits would also be an\n> update to HEAD and through the symref-ness of the HEAD should result\n> in an update to the index of the superproject.\n\nagreed. we'd find out what HEAD~2 is and then write the gitlink in the\nsuperproject index to be that commit.\n\n> However, I do not think a good explanation of what should mean when\n> this new-style symbolic HEAD points at a commit in the superproject,\n> whether its limited to its HEAD or a tip of an arbitrary branch that\n> may not even be checked out.\n\nThe proposal only inspects the currently staged value of a gitlink, there\nis no way to reference a specific commit or (non-checked-out) branch,\nso it is not possible to point at them with the given read syntax.\nSo in case the superproject changes its HEAD (such as checking\nout a different branch, or detaching the HEAD), then the submodules\nHEAD changes automatically as the superprojects tree may change.\n\n\n>  These are not something we can easily\n> change without affecting wider context.  Our submodule, when we make\n> a new commit, may be ready to advance, but our superproject and\n> other submodules may not be ready to be included in a new commit in\n> the superproject.\n\nThat is an interesting problem, as un-staging the submodule\nis not as easy as un-staging a change in a file. Assume we\nhave a layout as\n\n  git init super\n  git -C super init sub\n  echo change >>super/sub/file\n  git -C super/sub commit -a -m \"submodule changes\"\n\nthen the following behaviors seem reasonable:\n\n    git -C super reset --soft sub\n      As the superprojects index is not touched, so is the\n      submodules HEAD and working tree. The submodule\n      stays as is.\n\n    git -C super reset --mixed sub\n      As this resets the index of the superproject, but not the working\n      tree, I'd expect the submodules HEAD to be detached keeping\n      the new content alive, such that it can be added as\n        git -C super add --re-attach-with-new-symref-mode sub\n\n    git -C super reset --hard sub\n      This resets the index and the working tree in the superproject,\n      so the submodules commit with the new content would be lost,\n      the submodules HEAD is still the new symbolic ref.  The subs\n      working tree is dirty and contains the change unless\n      --recurse-submodules is given in addition, which would wipe\n      the subs working directory in addition.\n\n> So I think the idea this patch illustrates is on to something\n> interesting and potentially useful, but I am not sure if it makes\n> sense to tie it to anything but the index of the superproject.\n\nHypothetical use case:\n\n  Your project targets multiple platforms and you are interested\n  in not producing the perfect history like we do, but you never\n  change a commit once created (for policy legal reasons).\n  Naturally this produces a lot of duds.\n  Create a a gitlink for each platform that points to the latest\n  commit known to work for the platform. Point HEAD to the\n  repository *itself*, and gitlink of the platform that you want to\n  follow. After each fetch, checking out the latest HEAD is known\n  to work for your platform.  (This hypothetical HEAD reference\n  to itself is entertaining for sure.)\n\n> Even if we limit ourselves to pointing at the index of the\n> superproject, there probably are a handful of interesting issues\n> that need to be clarified (not in the sense of \"this and that issues\n> exist, so this won't be a useful feature\", but in the sense of \"we'd\n> be able to do these useful things using this feature, and we need to\n> fill in more details\"), such as:\n>\n>  - Making new commits in the submodule available to the upstream.\n>    Just like a detached HEAD in the submodule, this is not tied to\n>    any concrete branch, and it is unclear how a recursive \"push\"\n>    from the superproject should propagate the changes to the\n>    upstream of the submodule;\n\nIn Gerrit land people are trained to use\n  git push HEAD:refs/for/master\nwhich would work fine when propagated to the submodule as-is.\n\nThere are two answers for the life without Gerrit in a dumb git remote.\n(a) In the far future in the ideal world, where version control is a\nsolved problem, submodules can reside inside the superproject\non the remote as well, i.e. in $GIT_DIR/modules/name, the URL\nmay be twisted to not expose the \"modules/name\" part, though such that\n\n    example.org/superproject\n    example.org/superproject/submodule\n\nis possible. In this world a push of the superproject would also transfer\nthe relevant submodule commits, such that the remote has all of the\nsubmodules history that contains all gitlink entries.\n\nPushing from inside the submodule may either push the superproject\n(as you can push from a directory of the superproject) or it would\nabort to not confuse a user.\n\n(b) In the near future, I have no idea how to best solve this.\nMaybe we can push it as a detached HEAD to the remote?\n\n>  - Switching between branches that bind the same commit for the\n>    submodule in the superproject would work just like switching\n>    between branches that record the same blob for a path, i.e. it\n>    will carry forward a local modification.\n\nRight.\n\n>\n>  - The index entry in the superproject may now have to get involved\n>    in fsck and reachability study in the submodule as reachability\n>    root.  A corollary to this is that submodules behave more\n>    similarly to regular blobs wrt \"git reset --hard\" in the\n>    superproject, which is a good thing.  \"git -C submodule commit &&\n>    git reset --hard\" will create a new commit in the submodule, add\n>    it to the index of the superproject, and then lose that change\n>    from the index of the superproject, making the commit\n>    unreachable, just like \"edit file && git add file && git reset\n>    --hard\" in the superproject will make the blob that records the\n>    updated content of the file unreachable.\n\nThis is in sync with my understanding above.\n\nThanks,\nStefan\n"},{"id":"324715","messageId":"xmqq60eppnxk.fsf@gitster.mtv.corp.google.com","threadId":"46355","inReplyTo":"CAGZ79kY=j-sgrAFwAhP-_9EKBsa6FXARaVuCxM6gPHV7M+1OAA@mail.gmail.com","subject":"Re: [PATCH] RFC: A new type of symbolic refs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-07-18T19:03:51Z","receivedAt":"2017-07-18T19:04:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n>> Even if we limit ourselves to pointing at the index of the\n>> superproject, there probably are a handful of interesting issues\n>> that need to be clarified (not in the sense of \"this and that issues\n>> exist, so this won't be a useful feature\", but in the sense of \"we'd\n>> be able to do these useful things using this feature, and we need to\n>> fill in more details\"), such as:\n>>\n>>  - Making new commits in the submodule available to the upstream.\n>>    Just like a detached HEAD in the submodule, this is not tied to\n>>    any concrete branch, and it is unclear how a recursive \"push\"\n>>    from the superproject should propagate the changes to the\n>>    upstream of the submodule;\n>\n> In Gerrit land people are trained to use\n>   git push HEAD:refs/for/master\n> which would work fine when propagated to the submodule as-is.\n\nWhat I was alluding to is how you find 'master' in 'for/master',\ngiven the submodule HEAD which only says \"I point at the index of\nthe superproject\".  Note that 'master' here refers to a branch\nin the submodule.  If we further limit ourselves and declare that\na submodule that uses this external-reference as HEAD must always\nbe on the same branch as the superproject is on, then we can find\nwhat branch the superproject is on, and use that.\n"}]}