{"thread":{"id":"47140","subject":"[RFC PATCH 0/4] git-status reports relation to superproject","startedAt":"2017-11-08T19:55:30Z","lastAt":"2017-11-11T05:34:36Z","messageCount":16,"participants":["Stefan Beller","Jonathan Tan","Junio C Hamano","Jacob Keller"],"isPatch":true,"patchVersion":1,"patchTotal":4},"messages":[{"id":"332088","messageId":"20171108195509.7839-1-sbeller@google.com","threadId":"47140","inReplyTo":null,"subject":"[RFC PATCH 0/4] git-status reports relation to superproject","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-11-08T19:55:05Z","receivedAt":"2017-11-08T19:55:30Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"  $ git -c status.superprojectinfo status\n  HEAD detached at v2.15-rc2\n  superproject is 6 commits behind HEAD 7070ce2..5e6d0fb\n  nothing to commit, working tree clean\n\nHow cool is that?\n\nThis series side steps the questions raised in\nhttps://public-inbox.org/git/xmqq4lq6hmp2.fsf_-_@gitster.mtv.corp.google.com/\nwhich I am also putting together albeit slowly.\n\nThis series just reports the relationship between the superprojects gitlink\n(if any) to HEAD. I think that is useful information in the current\nworld of submodules.\n\nStefan\n\nStefan Beller (4):\n  remote, revision: factor out exclusive counting between two commits\n  submodule.c: factor start_ls_files_dot_dot out of\n    get_superproject_working_tree\n  submodule.c: get superprojects gitlink value\n  git-status: report reference to superproject\n\n Documentation/config.txt |  5 +++\n builtin/commit.c         |  2 ++\n remote.c                 | 40 +-----------------------\n revision.c               | 45 +++++++++++++++++++++++++++\n revision.h               |  7 +++++\n submodule.c              | 80 ++++++++++++++++++++++++++++++++++++------------\n submodule.h              |  6 ++++\n t/t7519-superproject.sh  | 57 ++++++++++++++++++++++++++++++++++\n wt-status.c              | 37 ++++++++++++++++++++++\n wt-status.h              |  1 +\n 10 files changed, 222 insertions(+), 58 deletions(-)\n create mode 100755 t/t7519-superproject.sh\n\n-- \n2.15.0.128.g40905b34bf.dirty\n\n"},{"id":"332089","messageId":"20171108195509.7839-2-sbeller@google.com","threadId":"47140","inReplyTo":"20171108195509.7839-1-sbeller@google.com","subject":"[PATCH 1/4] remote, revision: factor out exclusive counting between two commits","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-11-08T19:55:06Z","receivedAt":"2017-11-08T19:55:32Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n remote.c   | 40 +---------------------------------------\n revision.c | 45 +++++++++++++++++++++++++++++++++++++++++++++\n revision.h |  7 +++++++\n 3 files changed, 53 insertions(+), 39 deletions(-)\n\ndiff --git a/remote.c b/remote.c\nindex 685e776a65..60c689383a 100644\n--- a/remote.c\n+++ b/remote.c\n@@ -1990,9 +1990,7 @@ int stat_tracking_info(struct branch *branch, int *num_ours, int *num_theirs,\n {\n \tstruct object_id oid;\n \tstruct commit *ours, *theirs;\n-\tstruct rev_info revs;\n \tconst char *base;\n-\tstruct argv_array argv = ARGV_ARRAY_INIT;\n \n \t/* Cannot stat unless we are marked to build on top of somebody else. */\n \tbase = branch_get_upstream(branch, NULL);\n@@ -2014,43 +2012,7 @@ int stat_tracking_info(struct branch *branch, int *num_ours, int *num_theirs,\n \tif (!ours)\n \t\treturn -1;\n \n-\t/* are we the same? */\n-\tif (theirs == ours) {\n-\t\t*num_theirs = *num_ours = 0;\n-\t\treturn 0;\n-\t}\n-\n-\t/* Run \"rev-list --left-right ours...theirs\" internally... */\n-\targv_array_push(&argv, \"\"); /* ignored */\n-\targv_array_push(&argv, \"--left-right\");\n-\targv_array_pushf(&argv, \"%s...%s\",\n-\t\t\t oid_to_hex(&ours->object.oid),\n-\t\t\t oid_to_hex(&theirs->object.oid));\n-\targv_array_push(&argv, \"--\");\n-\n-\tinit_revisions(&revs, NULL);\n-\tsetup_revisions(argv.argc, argv.argv, &revs, NULL);\n-\tif (prepare_revision_walk(&revs))\n-\t\tdie(\"revision walk setup failed\");\n-\n-\t/* ... and count the commits on each side. */\n-\t*num_ours = 0;\n-\t*num_theirs = 0;\n-\twhile (1) {\n-\t\tstruct commit *c = get_revision(&revs);\n-\t\tif (!c)\n-\t\t\tbreak;\n-\t\tif (c->object.flags & SYMMETRIC_LEFT)\n-\t\t\t(*num_ours)++;\n-\t\telse\n-\t\t\t(*num_theirs)++;\n-\t}\n-\n-\t/* clear object flags smudged by the above traversal */\n-\tclear_commit_marks(ours, ALL_REV_FLAGS);\n-\tclear_commit_marks(theirs, ALL_REV_FLAGS);\n-\n-\targv_array_clear(&argv);\n+\tcompare_commits(ours, theirs, num_ours, num_theirs);\n \treturn 0;\n }\n \ndiff --git a/revision.c b/revision.c\nindex 99c95c19b0..fe1faf2628 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -1159,6 +1159,51 @@ int ref_excluded(struct string_list *ref_excludes, const char *path)\n \treturn 0;\n }\n \n+void compare_commits(struct commit *ours, struct commit *theirs,\n+\t\t    int *num_ours, int *num_theirs)\n+{\n+\tstruct rev_info revs;\n+\tstruct argv_array argv = ARGV_ARRAY_INIT;\n+\n+\t/* are we the same? */\n+\tif (theirs == ours) {\n+\t\t*num_theirs = *num_ours = 0;\n+\t\treturn;\n+\t}\n+\n+\t/* Run \"rev-list --left-right ours...theirs\" internally... */\n+\targv_array_push(&argv, \"\"); /* ignored */\n+\targv_array_push(&argv, \"--left-right\");\n+\targv_array_pushf(&argv, \"%s...%s\",\n+\t\t\t oid_to_hex(&ours->object.oid),\n+\t\t\t oid_to_hex(&theirs->object.oid));\n+\targv_array_push(&argv, \"--\");\n+\n+\tinit_revisions(&revs, NULL);\n+\tsetup_revisions(argv.argc, argv.argv, &revs, NULL);\n+\tif (prepare_revision_walk(&revs))\n+\t\tdie(\"revision walk setup failed\");\n+\n+\t/* ... and count the commits on each side. */\n+\t*num_ours = 0;\n+\t*num_theirs = 0;\n+\twhile (1) {\n+\t\tstruct commit *c = get_revision(&revs);\n+\t\tif (!c)\n+\t\t\tbreak;\n+\t\tif (c->object.flags & SYMMETRIC_LEFT)\n+\t\t\t(*num_ours)++;\n+\t\telse\n+\t\t\t(*num_theirs)++;\n+\t}\n+\n+\t/* clear object flags smudged by the above traversal */\n+\tclear_commit_marks(ours, ALL_REV_FLAGS);\n+\tclear_commit_marks(theirs, ALL_REV_FLAGS);\n+\n+\targv_array_clear(&argv);\n+}\n+\n static int handle_one_ref(const char *path, const struct object_id *oid,\n \t\t\t  int flag, void *cb_data)\n {\ndiff --git a/revision.h b/revision.h\nindex 54761200ad..3ff6a5190b 100644\n--- a/revision.h\n+++ b/revision.h\n@@ -324,4 +324,11 @@ extern int rewrite_parents(struct rev_info *revs, struct commit *commit,\n  */\n extern struct commit_list *get_saved_parents(struct rev_info *revs, const struct commit *commit);\n \n+/*\n+ * Compute the number of commits between 'one' and 'two' storing the number\n+ * of commits in their parent DAG  ncluded in each but not the other.\n+ */\n+extern void compare_commits(struct commit *one, struct commit *two,\n+\t\t\t    int *num_one, int *num_two);\n+\n #endif\n-- \n2.15.0.128.g40905b34bf.dirty\n\n"},{"id":"332090","messageId":"20171108195509.7839-4-sbeller@google.com","threadId":"47140","inReplyTo":"20171108195509.7839-1-sbeller@google.com","subject":"[PATCH 3/4] submodule.c: get superprojects gitlink value","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-11-08T19:55:08Z","receivedAt":"2017-11-08T19:55:35Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule.c | 27 +++++++++++++++++++++++++++\n submodule.h |  6 ++++++\n 2 files changed, 33 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 4fcb64469e..68b123eb13 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -2074,6 +2074,33 @@ const char *get_superproject_working_tree(void)\n \treturn ret;\n }\n \n+/*\n+ * Returns 0 when the gitlink is found in the superprojects index,\n+ * the value will be found in `oid`. Otherwise return -1.\n+ */\n+int get_superproject_gitlink(struct object_id *oid)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tconst char *hash;\n+\n+\tif (start_ls_files_dot_dot(&cp, &sb) < 0)\n+\t\treturn -1;\n+\n+\tif (!skip_prefix(sb.buf, \"160000 \", &hash))\n+\t\t/*\n+\t\t * superproject doesn't have a gitlink at submodule position or\n+\t\t * output is gibberish\n+\t\t */\n+\t\treturn -1;\n+\n+\tif (get_oid_hex(hash, oid))\n+\t\t/* could not parse the object name */\n+\t\treturn -1;\n+\n+\treturn 0;\n+}\n+\n /*\n  * Put the gitdir for a submodule (given relative to the main\n  * repository worktree) into `buf`, or return -1 on error.\ndiff --git a/submodule.h b/submodule.h\nindex f0da0277a4..5fc602f0c7 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -137,4 +137,10 @@ extern void absorb_git_dir_into_superproject(const char *prefix,\n  */\n extern const char *get_superproject_working_tree(void);\n \n+/*\n+ * Returns 0 when the gitlink is found in the superprojects index,\n+ * the value will be found in `oid`. Otherwise return -1.\n+ */\n+extern int get_superproject_gitlink(struct object_id *oid);\n+\n #endif\n-- \n2.15.0.128.g40905b34bf.dirty\n\n"},{"id":"332091","messageId":"20171108195509.7839-5-sbeller@google.com","threadId":"47140","inReplyTo":"20171108195509.7839-1-sbeller@google.com","subject":"[PATCH 4/4] git-status: report reference to superproject","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-11-08T19:55:09Z","receivedAt":"2017-11-08T19:55:41Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a submodule the position of HEAD in relation to the gitlink pointer\nin the superproject may be of interest.\n\nIntroduce a config option `status.superprojectInfo` that when enabled\nwill report the relation between HEAD and the commit pointed to by the\ngitlink in the index of the superproject.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n Documentation/config.txt |  5 +++++\n builtin/commit.c         |  2 ++\n t/t7519-superproject.sh  | 57 ++++++++++++++++++++++++++++++++++++++++++++++++\n wt-status.c              | 37 +++++++++++++++++++++++++++++++\n wt-status.h              |  1 +\n 5 files changed, 102 insertions(+)\n create mode 100755 t/t7519-superproject.sh\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 5f0d62753d..7825a1a7be 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -3097,6 +3097,11 @@ status.submoduleSummary::\n \tsubmodule summary' command, which shows a similar output but does\n \tnot honor these settings.\n \n+status.superprojectInfo\n+\tDefaults to false.\n+\tThis shows the relation of the current HEAD to the commit pointed\n+\tto be the gitlink entry in the superprojects index.\n+\n stash.showPatch::\n \tIf this is set to true, the `git stash show` command without an\n \toption will show the stash entry in patch form.  Defaults to false.\ndiff --git a/builtin/commit.c b/builtin/commit.c\nindex c38542ee46..f937f6c6cf 100644\n--- a/builtin/commit.c\n+++ b/builtin/commit.c\n@@ -1286,6 +1286,8 @@ static int git_status_config(const char *k, const char *v, void *cb)\n \t\t\ts->submodule_summary = -1;\n \t\treturn 0;\n \t}\n+\tif (!strcmp(k, \"status.superprojectinfo\"))\n+\t\ts->superproject_info = git_config_bool(k, v);\n \tif (!strcmp(k, \"status.short\")) {\n \t\tif (git_config_bool(k, v))\n \t\t\tstatus_deferred_config.status_format = STATUS_FORMAT_SHORT;\ndiff --git a/t/t7519-superproject.sh b/t/t7519-superproject.sh\nnew file mode 100755\nindex 0000000000..ade2379a59\n--- /dev/null\n+++ b/t/t7519-superproject.sh\n@@ -0,0 +1,57 @@\n+\n+\n+test_description='git status for superproject relations'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'setup' '\n+\ttest_commit initial &&\n+\ttest_create_repo sub &&\n+\t# this whole file tests superproject reporting, so set this config here\n+\tgit -C sub config status.superprojectInfo true\n+'\n+\n+test_expect_success 'repo on initial commit does not mention superproject' '\n+\tgit -C sub status > actual &&\n+\ttest_i18ngrep \"No commits yet\" actual &&\n+\ttest_i18ngrep -v superproject actual\n+'\n+\n+test_expect_success 'setup submodule' '\n+\ttest_commit -C sub initial &&\n+\tgit submodule add ./sub sub\n+'\n+\n+test_expect_success 'submodule in sync with superproject index' '\n+\tgit -C sub status >actual &&\n+\ttest_i18ngrep \"superproject points at HEAD\" actual\n+'\n+\n+test_expect_success 'submodule in sync with superproject' '\n+\tgit commit -a -m \"superproject adds submodule\" &&\n+\tgit -C sub status >actual &&\n+\ttest_i18ngrep \"superproject points at HEAD\" actual\n+'\n+\n+test_expect_success 'submodule advances two commits' '\n+\tgit -C sub commit --allow-empty -m \"test\" &&\n+\tgit -C sub commit --allow-empty -m \"test2\" &&\n+\tgit -C sub status >actual &&\n+\ttest_i18ngrep \"superproject is 2 commits behind HEAD\" actual\n+'\n+\n+test_expect_success 'submodule behind superproject' '\n+\tgit add sub &&\n+\tgit commit -m \"update sub\" &&\n+\tgit -C sub reset --hard HEAD^ &&\n+\tgit -C sub status >actual &&\n+\ttest_i18ngrep \"superproject is ahead of HEAD by 1 commits\" actual\n+'\n+\n+test_expect_success 'submodule and superproject differ' '\n+\tgit -C sub commit --allow-empty -m \"test2b\" &&\n+\tgit -C sub status >actual &&\n+\ttest_i18ngrep \"superproject and HEAD differ by +1, -1 commits\" actual\n+'\n+\n+test_done\ndiff --git a/wt-status.c b/wt-status.c\nindex bedef256ce..3e8e27a550 100644\n--- a/wt-status.c\n+++ b/wt-status.c\n@@ -1027,6 +1027,41 @@ static void wt_longstatus_print_tracking(struct wt_status *s)\n \tstrbuf_release(&sb);\n }\n \n+static void wt_longstatus_print_superproject_relation(struct wt_status *s)\n+{\n+\tstruct object_id oid;\n+\tstruct commit *in_gitlink, *head;\n+\tint head_nr, gitlink_nr;\n+\n+\tif (get_superproject_gitlink(&oid))\n+\t\treturn;\n+\n+\tin_gitlink = lookup_commit(&oid);\n+\n+\tread_ref(\"HEAD\", &oid);\n+\thead = lookup_commit(&oid);\n+\n+\tcompare_commits(head, in_gitlink, &head_nr, &gitlink_nr);\n+\n+\tif (!head_nr && !gitlink_nr)\n+\t\tprintf(_(\"superproject points at HEAD\\n\"));\n+\telse if (!head_nr)\n+\t\tprintf(_(\"superproject is ahead of HEAD by %d commits %s..%s\\n\"),\n+\t\t\t gitlink_nr,\n+\t\t\t find_unique_abbrev(head->object.oid.hash, DEFAULT_ABBREV),\n+\t\t\t find_unique_abbrev(in_gitlink->object.oid.hash, DEFAULT_ABBREV));\n+\telse if (!gitlink_nr)\n+\t\tprintf(_(\"superproject is %d commits behind HEAD %s..%s\\n\"),\n+\t\t\t head_nr,\n+\t\t\t find_unique_abbrev(in_gitlink->object.oid.hash, DEFAULT_ABBREV),\n+\t\t\t find_unique_abbrev(head->object.oid.hash, DEFAULT_ABBREV));\n+\telse\n+\t\tprintf(_(\"superproject and HEAD differ by +%d, -%d commits %s...%s\\n\"),\n+\t\t\t gitlink_nr, head_nr,\n+\t\t\t find_unique_abbrev(head->object.oid.hash, DEFAULT_ABBREV),\n+\t\t\t find_unique_abbrev(in_gitlink->object.oid.hash, DEFAULT_ABBREV));\n+}\n+\n static int has_unmerged(struct wt_status *s)\n {\n \tint i;\n@@ -1593,6 +1628,8 @@ static void wt_longstatus_print(struct wt_status *s)\n \t\tif (!s->is_initial)\n \t\t\twt_longstatus_print_tracking(s);\n \t}\n+\tif (!s->is_initial && s->superproject_info)\n+\t\twt_longstatus_print_superproject_relation(s);\n \n \twt_longstatus_print_state(s, &state);\n \tfree(state.branch);\ndiff --git a/wt-status.h b/wt-status.h\nindex 64f4d33ea1..75700980d2 100644\n--- a/wt-status.h\n+++ b/wt-status.h\n@@ -70,6 +70,7 @@ struct wt_status {\n \tint display_comment_prefix;\n \tint relative_paths;\n \tint submodule_summary;\n+\tint superproject_info;\n \tint show_ignored_files;\n \tenum untracked_status_type show_untracked_files;\n \tconst char *ignore_submodule_arg;\n-- \n2.15.0.128.g40905b34bf.dirty\n\n"},{"id":"332092","messageId":"20171108195509.7839-3-sbeller@google.com","threadId":"47140","inReplyTo":"20171108195509.7839-1-sbeller@google.com","subject":"[PATCH 2/4] submodule.c: factor start_ls_files_dot_dot out of get_superproject_working_tree","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-11-08T19:55:07Z","receivedAt":"2017-11-08T19:55:44Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We'll reuse the code of the factored out function shortly, when exploring\nthe superproject for another aspect. Instead of knowing the root of the\nsuperproject we'll find out about the gitlink value.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule.c | 53 ++++++++++++++++++++++++++++++++++-------------------\n 1 file changed, 34 insertions(+), 19 deletions(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex 239d94d539..4fcb64469e 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1977,15 +1977,13 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \t}\n }\n \n-const char *get_superproject_working_tree(void)\n+/* Starts a child process `ls-files` one directory above the root of the repo. */\n+static int start_ls_files_dot_dot(struct child_process *cp, struct strbuf *out)\n {\n-\tstruct child_process cp = CHILD_PROCESS_INIT;\n-\tstruct strbuf sb = STRBUF_INIT;\n \tconst char *one_up = real_path_if_valid(\"../\");\n-\tconst char *cwd = xgetcwd();\n-\tconst char *ret = NULL;\n \tconst char *subpath;\n-\tint code;\n+\tchar *cwd = xgetcwd();\n+\tstruct strbuf sb = STRBUF_INIT;\n \tssize_t len;\n \n \tif (!is_inside_work_tree())\n@@ -1994,31 +1992,48 @@ const char *get_superproject_working_tree(void)\n \t\t * We might have a superproject, but it is harder\n \t\t * to determine.\n \t\t */\n-\t\treturn NULL;\n+\t\treturn -1;\n \n \tif (!one_up)\n-\t\treturn NULL;\n+\t\treturn -1;\n \n \tsubpath = relative_path(cwd, one_up, &sb);\n \n-\tprepare_submodule_repo_env(&cp.env_array);\n-\targv_array_pop(&cp.env_array);\n+\tprepare_submodule_repo_env(&cp->env_array);\n+\targv_array_pop(&cp->env_array);\n \n-\targv_array_pushl(&cp.args, \"--literal-pathspecs\", \"-C\", \"..\",\n+\targv_array_pushl(&cp->args, \"--literal-pathspecs\", \"-C\", \"..\",\n \t\t\t\"ls-files\", \"-z\", \"--stage\", \"--full-name\", \"--\",\n \t\t\tsubpath, NULL);\n-\tstrbuf_reset(&sb);\n \n-\tcp.no_stdin = 1;\n-\tcp.no_stderr = 1;\n-\tcp.out = -1;\n-\tcp.git_cmd = 1;\n+\tcp->no_stdin = 1;\n+\tcp->no_stderr = 1;\n+\tcp->out = -1;\n+\tcp->git_cmd = 1;\n \n-\tif (start_command(&cp))\n+\tif (start_command(cp))\n \t\tdie(_(\"could not start ls-files in ..\"));\n \n-\tlen = strbuf_read(&sb, cp.out, PATH_MAX);\n-\tclose(cp.out);\n+\tlen = strbuf_read(out, cp->out, PATH_MAX);\n+\tclose(cp->out);\n+\n+\tstrbuf_release(&sb);\n+\tfree(cwd);\n+\treturn len;\n+}\n+\n+const char *get_superproject_working_tree(void)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tconst char *cwd = xgetcwd();\n+\tconst char *ret = NULL;\n+\tint code;\n+\tssize_t len;\n+\n+\tlen = start_ls_files_dot_dot(&cp, &sb);\n+\tif (len < 0)\n+\t\treturn NULL;\n \n \tif (starts_with(sb.buf, \"160000\")) {\n \t\tint super_sub_len;\n-- \n2.15.0.128.g40905b34bf.dirty\n\n"},{"id":"332100","messageId":"20171108143646.2e1870a82360072557094e01@google.com","threadId":"47140","inReplyTo":"20171108195509.7839-1-sbeller@google.com","subject":"Re: [RFC PATCH 0/4] git-status reports relation to superproject","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2017-11-08T22:36:46Z","receivedAt":"2017-11-08T22:36:53Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Wed,  8 Nov 2017 11:55:05 -0800\nStefan Beller <sbeller@google.com> wrote:\n\n>   $ git -c status.superprojectinfo status\n>   HEAD detached at v2.15-rc2\n>   superproject is 6 commits behind HEAD 7070ce2..5e6d0fb\n>   nothing to commit, working tree clean\n> \n> How cool is that?\n> \n> This series side steps the questions raised in\n> https://public-inbox.org/git/xmqq4lq6hmp2.fsf_-_@gitster.mtv.corp.google.com/\n> which I am also putting together albeit slowly.\n> \n> This series just reports the relationship between the superprojects gitlink\n> (if any) to HEAD. I think that is useful information in the current\n> world of submodules.\n\nThe relationship is indeed currently useful, but if the long term plan\nis to strongly discourage detached submodule HEAD, then I would think\nthat these patches are in the wrong direction. (If the long term plan is\nto end up supporting both detached and linked submodule HEAD, then these\npatches are fine, of course.) So I think that the plan referenced in\nJunio's email (that you linked above) still needs to be discussed.\n\nAbout the patches themselves, they look OK to me. Some minor things off\nthe top of my head are to retain the \"ours\" and \"theirs\" (instead of\n\"one\" and \"two\"), and to replicate the language in remote.c more closely\n(\"This submodule is (ahead of/behind) the superproject by %d commit(s)\")\ninstead of inventing your own.\n"},{"id":"332102","messageId":"20171109001007.11894-1-sbeller@google.com","threadId":"47140","inReplyTo":"20171108143646.2e1870a82360072557094e01@google.com","subject":"[RFD] Long term plan with submodule refs?","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-11-09T00:10:07Z","receivedAt":"2017-11-09T00:10:21Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"> The relationship is indeed currently useful, but if the long term plan\n> is to strongly discourage detached submodule HEAD, then I would think\n> that these patches are in the wrong direction. (If the long term plan is\n> to end up supporting both detached and linked submodule HEAD, then these\n> patches are fine, of course.) So I think that the plan referenced in\n> Junio's email (that you linked above) still needs to be discussed.\n\nThis email presents different approaches.\n\nObjective\n=========\nThis document should summarize the current situation of Git submodules\nand start a discussion of where it can be headed long term.\nShow different ways in which submodule refs could evolve.\n\nBackground\n==========\nSubmodules in Git are considered as an independet repository currently.\nThis is okay for current workflows, such as utilizing a library that is\nrarely updated. Other workflows that require a tighter integration between\nsubmodule and superproject are possible, but cumbersome as there is an\nadditional step that has to be performed, which is the update of the gitlink\npointer in the superproject.\n\nOther discussions of the past:\n\"Re-attach HEAD?\"\n  https://public-inbox.org/git/20170501180058.8063-1-sbeller@google.com/\n\"Semantics of checkout --recursive for submodules on a branch\"\n  https://public-inbox.org/git/20170630003851.17288-1-sbeller@google.com/\n\"A new type of symref?\"\n  https://public-inbox.org/git/xmqqvamqg2fy.fsf@gitster.mtv.corp.google.com/\n\nWorkflows\n=========\n* Obtaining a copy of the Superproject tightly coupled with submodules\n  solved via git clone --recurse-submodules=<pathspec>\n* Changing the submodule selection\n  solved via submodule.active flags\n* Changing the remote / Interacting with a different remote for all submodules\n  -> need to be solved, not core issue of this discussion\n* Syncing to the latest upstream\n  solved via git pull --recurse  \n* Working on a local feature in one submodule\n  -> How do refs work spanning superproject/submodule?\n* Working on a feature spanning multiple submodules\n  -> How do refs work spanning multiple repos?\n* Working on a bug fix (Changing the feature that you currently work on, branches)\n  -> How does switching branches in the superproject affect submodules\n\nThis discussion should resolve around refs are handled in submodules in\nrelation to a superproject.\n\nPossible data models and workflow implications\n==============================================\nIn the following different data models are presented, which aid a submodule\nheavy workflow each giving pros and cons.\n\nKeep everything as is, superproject and submodule have their own refs\n---------------------------------------------------------------------\nIn this alternative we'd just make existing commands nicer, e.g.\ngit-status, git-log would give information about the superprojects\ngitlink similar as they give information about a remote branch.\n\nWe might want to introduce an option that triggers adding the submodule\nto the superproject once a commit is done in the submodule.\n\nPros:\n * easiest to implement\n * easy to understand when having a git background already\n \nCons:\n * Current tools that manage multiple repositories (e.g. repo, git-slave)\n   have \"branches in parallel\", i.e. each repo has a branch of the same\n   name, instead of using a superproject to manage the state of all repos\n   involved. So users of such tools may be confused by submodules.\n * when using a detached HEAD in the submodule, we may run into git-gc issues.\n \n\nUse replicate refs in submodules\n--------------------------------\nThis approach will replicate the superproject refs into the submodule\nref namespace, e.g. git-branch learns about --recurse-submodules, which\ncreates a branch of a given name in all submodules. These (topic) branches\nshould be kept in sync with the superproject\n\nPros:\n * This seemed intuitive to Gerrit users\n * 'quick' to implement, most of the commands are already there,\n   just git-branch is needed to have the workflows mentioned above complete.\nCons:\n * What does \"git checkout -b A B\" mean? (special case: B == HEAD)\n   Is the branch name replicated as a string into the submodule operation,\n   or do we dereference the superprojects gitlink and walk from there?\n   When taking the superprojects gitlink, then why do we have the branches\n   in the submodule in the first place? When taking the string as-is,\n   then it might confuse users.\n * non-atomic of refs between superproject and submodule by design;\n   This relies on superproject and submodule to stay in sync via hope.\n\nNo submodule refstore at all\n----------------------------\nUse refs and commits in the superproject to stitch submodule changes\ntogether. Disallow branches in the submodule. This is only restricted\nto the working tree inside the superproject, such that the output of git-branch\nchanges depending whether the working tree is in- or outside the superproject\nworking tree.\n\nThe messages of git-status inside the superproject working tree are changed\nas \"detached HEAD\"s are common in submodule and sound scary.  Maybe\n\"following the superproject\"\n\nPros:\n * solves the atomicity issue from the prior proposal\nCons:\n * In a submodule one must use a worktree outside the superproject\n   to do upstream work.\n * As the detached HEAD is not referenced, we have git-gc issues.\n\n\nNew type of symbolic refs\n=========================\nA symbolic ref can currently only point at a ref or another symbolic ref.\nThis proposal showcases different scenarios on how this could change in the\nfuture.\n\nHEAD pointing at the superprojects index\n----------------------------------------\nIntroduce a new symbolic ref that points at the superprojects\nindex of the gitlink. The format is\n\n  \"repo:\" <superprojects gitdir> '\\0' <gitlink-path> '\\0'\n\nJust like existing symrefs, the content of the ref will be read and followed.\nOn reading \"repo:\", the sha1 will be obtained equivalent to:\n\n    git -C <superproject> ls-files -s <gitlink-path> | awk '{ print $2}'\n\nRef write operations driven by the submodule, affecting symrefs\n  e.g. git checkout <other branch> (in the submodule)\n\nIn this scenario only the HEAD is optionally attached to the superproject,\nso we can rewrite the HEAD to be anything else, such as a branch just fine.\nOnce the HEAD is not pointing at the superproject any more, we'll leave the\nsubmodule alone in operations driven by the superproject.\nTo get back on the superproject branch, we’d need to invent new UX, such as\n   git checkout --attach-superproject\nas that is similar to --detach\n\nRef write operations driven by the submodule, affecting target ref\n  e.g. git commit, reset --hard, update-ref (in the submodule)\n\nThe HEAD stays the same, pointing at the superproject.\nThe gitlink is changed to the target sha1, using\n\n  git -C <superproject> update-index --add \\\n      --cacheinfo 160000,$SHA1,<gitlink-path>\n\nThis will affect the superprojects index, such that then a commit in\nthe superproject is needed.\n\nRef write operations driven by the superproject, changing the gitlink\n  e.g. git checkout <tree-ish>, git reset --hard (in the superproject)\n\nThis will change the gitlink in the superprojects index, such that the HEAD\nin the submodule changes, which would trigger an update of the\nsubmodules working tree.\n\nSuperproject operations spanning index and worktree\n  E.g. git reset --mixed\nAs the submodules HEAD is defined in the index, we would reset it to the\nversion in the last commit. As --mixed promises to not touch the working tree,\nthe submodules worktree would not be touched. git reset --mixed in the\nsuperproject is the same as --soft in the submodule.\n\nConsistency considerations (gc)\n  e.g. git gc --aggressive --prune=now\n\nThe repacking logic is already aware of a detached HEAD, such that\nusing this new symref mechanism would not generate problems as long as\nwe keep the HEAD attached to the superproject. However when commits/objects\nare created while the HEAD is attached to the superproject and then HEAD\nswitches to a local branch, there are problems with the created objects\nas they seem unreachable now.\n\nThis problem is not new as a superproject may record submodule objects\nthat are not reachable from any of the submodule branches. Such objects\nfall prey to overzealous packing in the submodule.\n\nThis proposal however exposes this problem a lot more, as the submodule\nhas fewer needs for branches.\n\nPros\n * easy to tell if a submodule is attached to the superproject,\n * no atomicity issues\n * once enough commands implement this behavior, it may be easier to understand\n   than previous alternatives and feel more intuitive\nCons:\n * gc issues for now\n * lots of work as it revamps submodules alot.\n \n This last proposal might be differentiated further, e.g. the submodule HEAD\n pointing at the superprojects gitlink in the index, in its HEAD or other\n branch.\n\nAny feedback welcome!\nStefan\n\n\n\n\n\n\n\n\n"},{"id":"332103","messageId":"20171108172945.33c42a0e91b4ac494217b788@google.com","threadId":"47140","inReplyTo":"20171109001007.11894-1-sbeller@google.com","subject":"Re: [RFD] Long term plan with submodule refs?","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2017-11-09T01:29:45Z","receivedAt":"2017-11-09T01:29:54Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Wed,  8 Nov 2017 16:10:07 -0800\nStefan Beller <sbeller@google.com> wrote:\n\nI thought of a possible alternative and how it would work.\n\n> Possible data models and workflow implications\n> ==============================================\n> In the following different data models are presented, which aid a submodule\n> heavy workflow each giving pros and cons.\n\nWhat if, in the submodule, we have a new ref backend that mirrors the\nsuperproject? When initializing the submodule, its original refs are not\ncloned at all, but instead virtual refs are used.\n\nCreation of brand-new refs is forbidden in the submodule.\n\nWhen reading a ref in the submodule, if that ref is the current branch\nin the superproject, read the corresponding gitlink entry in the index\n(which may be dirty); otherwise read the gitlink in the tree of the tip\ncommit.\n\nWhen updating a ref in the submodule, if that ref is the current branch\nin the superproject, update the index; otherwise, create a commit on top\nof the tip and update the ref to point to the new tip.\n\nNo synchronicity is enforced between superproject and submodule in terms\nof HEAD, though: If a submodule is currently checked out to a branch,\nand the gitlink for that branch is updated through whatever means, that\nis equivalent to a \"git reset --soft\" in the submodule.\n\nThese rules seem straightforward to me (although I have been working\nwith Git for a while, so perhaps I'm not the best judge), and I think\nleads to a good workflow, as discussed below.\n\n> Workflows\n> =========\n> * Obtaining a copy of the Superproject tightly coupled with submodules\n>   solved via git clone --recurse-submodules=<pathspec>\n> * Changing the submodule selection\n>   solved via submodule.active flags\n> * Changing the remote / Interacting with a different remote for all submodules\n>   -> need to be solved, not core issue of this discussion\n> * Syncing to the latest upstream\n>   solved via git pull --recurse  \n\n(skipping the above, since they are either solved or not a core issue)\n\n> * Working on a local feature in one submodule\n>   -> How do refs work spanning superproject/submodule?\n\nThis is perhaps one weak point of my proposal - you can't work on a\nsubmodule as if it were independent. You can checkout a branch and make\ncommits, but (i) they will automatically affect the superproject, and\n(ii) the \"origin/foo\" etc. branches are those of the superproject. (But\nif you checkout a detached HEAD, everything should still work.)\n\n> * Working on a feature spanning multiple submodules\n>   -> How do refs work spanning multiple repos?\n\nThe above rules allow the following workflow:\n - \"checkout --recurse-submodules\" the branch you want on the\n   superproject\n - make whatever changes you want in each submodule\n - commit each individual submodule (which updates the index of the\n   superproject), then commit the superproject (we can introduce a\n   commit --recurse-submodules to make this more convenient)\n - a \"push --recurse-submodules\" can be implemented to push the\n   superproject and its submodules independently (and the same refspec\n   can be legitimately used both when pushing the superproject and when\n   pushing a submodule, since the ref names are the same, and not by\n   coincidence)\n\nIf the user insists on making changes on a non-current branch (i.e. by\ncreating commits in submodules then using \"git update-ref\" or\nequivalent), possibly multiple commits would be created in the\nsuperproject, but the user can still squash them later if desired.\n\n> * Working on a bug fix (Changing the feature that you currently work on, branches)\n>   -> How does switching branches in the superproject affect submodules\n\nYou will have to stash or commit your changes. (Which reminds me...GC in\nthe subproject will need to consult the revlog of the superproject too.)\n\n> New type of symbolic refs\n> =========================\n> A symbolic ref can currently only point at a ref or another symbolic ref.\n> This proposal showcases different scenarios on how this could change in the\n> future.\n> \n> HEAD pointing at the superprojects index\n> ----------------------------------------\n\nAssuming we don't need synchronicity, the existing HEAD format can be\nretained. To clarify what happens during ref writes, I'll reuse the\nscenarios Stefan described:\n\n> Ref write operations driven by the submodule, affecting target ref\n>   e.g. git commit, reset --hard, update-ref (in the submodule)\n> \n> The HEAD stays the same, pointing at the superproject.\n> The gitlink is changed to the target sha1, using\n> \n>   git -C <superproject> update-index --add \\\n>       --cacheinfo 160000,$SHA1,<gitlink-path>\n> \n> This will affect the superprojects index, such that then a commit in\n> the superproject is needed.\n\nIn this proposal, the HEAD also stays the same (pointing at the branch).\n\nEither the index is updated or a commit is needed. If a commit is\nneeded, it is automatically performed.\n\n> Ref write operations driven by the superproject, changing the gitlink\n>   e.g. git checkout <tree-ish>, git reset --hard (in the superproject)\n> \n> This will change the gitlink in the superprojects index, such that the HEAD\n> in the submodule changes, which would trigger an update of the\n> submodules working tree.\n\nThe HEAD in the submodule is unchanged. If the value of a ref has\nchanged \"from underneath\", this is as if a \"git reset --soft\" was done.\n\n> Superproject operations spanning index and worktree\n>   E.g. git reset --mixed\n> As the submodules HEAD is defined in the index, we would reset it to the\n> version in the last commit. As --mixed promises to not touch the working tree,\n> the submodules worktree would not be touched. git reset --mixed in the\n> superproject is the same as --soft in the submodule.\n\nSame.\n\n> Consistency considerations (gc)\n>   e.g. git gc --aggressive --prune=now\n> \n> The repacking logic is already aware of a detached HEAD, such that\n> using this new symref mechanism would not generate problems as long as\n> we keep the HEAD attached to the superproject. However when commits/objects\n> are created while the HEAD is attached to the superproject and then HEAD\n> switches to a local branch, there are problems with the created objects\n> as they seem unreachable now.\n> \n> This problem is not new as a superproject may record submodule objects\n> that are not reachable from any of the submodule branches. Such objects\n> fall prey to overzealous packing in the submodule.\n\nThe scenario Stefan describes will work OK - if a commit is created\nwhile the HEAD is pointing to a branch, then either the superproject's\nindex will be updated or commits will be created in the superproject.\nWhen GC reads the list of refs in the submodule, the new submodule\ncommit will be included. (Remember that if the superproject's current\nbranch is \"foo\", \"refs/heads/foo\" in the submodule reflects the\nsuperproject's index, so any changes to the index, though uncommitted,\nwill appear as a ref.)\n\nThe problem still exists (e.g. stashes in the superproject) but is\nreduced, I think.\n"},{"id":"332108","messageId":"xmqqbmkcjaxo.fsf@gitster.mtv.corp.google.com","threadId":"47140","inReplyTo":"20171109001007.11894-1-sbeller@google.com","subject":"Re: [RFD] Long term plan with submodule refs?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-11-09T05:08:51Z","receivedAt":"2017-11-09T05:08:59Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n>> The relationship is indeed currently useful, but if the long term plan\n>> is to strongly discourage detached submodule HEAD, then I would think\n>> that these patches are in the wrong direction. (If the long term plan is\n>> to end up supporting both detached and linked submodule HEAD, then these\n>> patches are fine, of course.) So I think that the plan referenced in\n>> Junio's email (that you linked above) still needs to be discussed.\n>\n> This email presents different approaches.\n>\n> Objective\n> =========\n> This document should summarize the current situation of Git submodules\n> and start a discussion of where it can be headed long term.\n> Show different ways in which submodule refs could evolve.\n>\n> Background\n> ==========\n> Submodules in Git are considered as an independet repository currently.\n> This is okay for current workflows, such as utilizing a library that is\n> rarely updated. Other workflows that require a tighter integration between\n> submodule and superproject are possible, but cumbersome as there is an\n> additional step that has to be performed, which is the update of the gitlink\n> pointer in the superproject.\n\nI do not think \"rarely updaed\" is an issue.\n\nThe problem is that we may want to make it easier to use a\nsuperproject and its submodules as if the combined whole were a\nsingle project, which currently is not easy, primarily because\nsubmodules are separate entities with different set of branches that\ncan be checked out independently from what branch the superproject\nis working on.\n\n> Workflows\n> =========\n> * Obtaining a copy of the Superproject tightly coupled with submodules\n>   solved via git clone --recurse-submodules=<pathspec>\n> * Changing the submodule selection\n>   solved via submodule.active flags\n> * Changing the remote / Interacting with a different remote for all submodules\n>   -> need to be solved, not core issue of this discussion\n> * Syncing to the latest upstream\n>   solved via git pull --recurse  \n> * Working on a local feature in one submodule\n>   -> How do refs work spanning superproject/submodule?\n> * Working on a feature spanning multiple submodules\n>   -> How do refs work spanning multiple repos?\n> * Working on a bug fix (Changing the feature that you currently work on, branches)\n>   -> How does switching branches in the superproject affect submodules\n\nThese are good starting points for copying such a combined whole to\nyour local machine and start working on it.  The more interesting,\nimportant, and potentially difficult part is how the result of such\nwork is shared back to where you started from.  \"push --recursive\"\nmay be a simple phrase, but a sensible definition of how it should\nwork won't be that simple.\n\n> Possible data models and workflow implications\n> ==============================================\n> In the following different data models are presented, which aid a submodule\n> heavy workflow each giving pros and cons.\n>\n> Keep everything as is, superproject and submodule have their own refs\n> ---------------------------------------------------------------------\n> ...\n> Cons:\n>  * Current tools that manage multiple repositories (e.g. repo, git-slave)\n>    have \"branches in parallel\", i.e. each repo has a branch of the same\n>    name, instead of using a superproject to manage the state of all repos\n>    involved. So users of such tools may be confused by submodules.\n>  * when using a detached HEAD in the submodule, we may run into git-gc issues.\n\nWe should make detached HEAD safe against gc if it is not,\nregardless of the use of submodules.  I thought it already was made\nsafe long time ago.\n\n> Use replicate refs in submodules\n> --------------------------------\n> This approach will replicate the superproject refs into the submodule\n> ref namespace, e.g. git-branch learns about --recurse-submodules, which\n> creates a branch of a given name in all submodules. These (topic) branches\n> should be kept in sync with the superproject\n>\n> Pros:\n>  * This seemed intuitive to Gerrit users\n>  * 'quick' to implement, most of the commands are already there,\n>    just git-branch is needed to have the workflows mentioned above complete.\n> Cons:\n>  * What does \"git checkout -b A B\" mean? (special case: B == HEAD)\n\nThe command ran at which level?  In the superproject, or in a single\nsubmodule?\n\n>    Is the branch name replicated as a string into the submodule operation,\n>    or do we dereference the superprojects gitlink and walk from there?\n\nIf they are \"kept in sync with the superproject\", then there should\nbe no difference between the two, so I do not see any room for\nwondering about that.  In other words, if there is need to worry\nabout the differences between the above two, then it probably is\nfundamentally impossible to keep these in sync, and a design that\nassumes it is possible would have to expose glitches to the end-user\nexperience.\n\nI do not know if glitches resulting from there would be so severe to\nbe show-stoppers, though.  It might be possible to paper them over.\n\n> No submodule refstore at all\n> ----------------------------\n> Use refs and commits in the superproject to stitch submodule changes\n> together. Disallow branches in the submodule. This is only restricted\n> to the working tree inside the superproject, such that the output of git-branch\n> changes depending whether the working tree is in- or outside the superproject\n> working tree.\n\nThis would need enhancement for reachability code, but it feels the\ncleanest from the philosophical standpoint---if you want to treat a\nsuperproject and its submodules as if it were a single project,\nability to check out a branch in a submodule that does not match\nthat of the superproject would only get in the way of preserving the\nillusion of \"single project\"-ness.\n\n> New type of symbolic refs\n> =========================\n> A symbolic ref can currently only point at a ref or another symbolic ref.\n> This proposal showcases different scenarios on how this could change in the\n> future.\n>\n> HEAD pointing at the superprojects index\n> ----------------------------------------\n\nThis looks to me a mere implementation detail for a (part of)\nnecessary component to realize the above \"No submodule refstore\".\n\n> Superproject operations spanning index and worktree\n>   E.g. git reset --mixed\n> As the submodules HEAD is defined in the index, we would reset it to the\n> version in the last commit. As --mixed promises to not touch the working tree,\n> the submodules worktree would not be touched. git reset --mixed in the\n> superproject is the same as --soft in the submodule.\n\nI am not sure if you want to take these promises low-level \"single\nrepository\" plumbing operations make too literally.  \"reset --mixed\"\nmay promise not to touch the working tree, but it also promises not\nto touch submodules at all.  If you are breaking the latter anyway,\nit would make more sense not to be afraid of breaking the former if\nit makes sense in the context of allowing the command to do more by\nbreaking the latter.\n\n"},{"id":"332109","messageId":"xmqq7ev0j95t.fsf@gitster.mtv.corp.google.com","threadId":"47140","inReplyTo":"20171108172945.33c42a0e91b4ac494217b788@google.com","subject":"Re: [RFD] Long term plan with submodule refs?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-11-09T05:47:10Z","receivedAt":"2017-11-09T05:47:18Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan <jonathantanmy@google.com> writes:\n\n> What if, in the submodule, we have a new ref backend that mirrors the\n> superproject? When initializing the submodule, its original refs are not\n> cloned at all, but instead virtual refs are used.\n> ...\n> These rules seem straightforward to me (although I have been working\n> with Git for a while, so perhaps I'm not the best judge), and I think\n> leads to a good workflow, as discussed below.\n\nIndeed this is intriguing.\n\n> The above rules allow the following workflow:\n>  - \"checkout --recurse-submodules\" the branch you want on the\n>    superproject\n>  - make whatever changes you want in each submodule\n>  - commit each individual submodule (which updates the index of the\n>    superproject), then commit the superproject (we can introduce a\n>    commit --recurse-submodules to make this more convenient)\n\nThe \"recurse\" option would also give users an extra atomicity, and\nwould not be merely for convenience; when a user wants to treat a\nsuperproject and its two submodules as if the combined whole were a\nsingle repository, there shouldn't be two separate commits in the\nhistory of the superproject only because two submodules made one\ncommit each to work on a single theme that spans all of them.\n\n"},{"id":"332110","messageId":"CA+P7+xpTCHHNG1tAhysOFV5jFHpXdzRk5zfYNHUUVdYiGY55cA@mail.gmail.com","threadId":"47140","inReplyTo":"20171109001007.11894-1-sbeller@google.com","subject":"Re: [RFD] Long term plan with submodule refs?","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-11-09T06:54:25Z","receivedAt":"2017-11-09T06:54:52Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Wed, Nov 8, 2017 at 4:10 PM, Stefan Beller <sbeller@google.com> wrote:\n>> The relationship is indeed currently useful, but if the long term plan\n>> is to strongly discourage detached submodule HEAD, then I would think\n>> that these patches are in the wrong direction. (If the long term plan is\n>> to end up supporting both detached and linked submodule HEAD, then these\n>> patches are fine, of course.) So I think that the plan referenced in\n>> Junio's email (that you linked above) still needs to be discussed.\n>\n\n> New type of symbolic refs\n> =========================\n> A symbolic ref can currently only point at a ref or another symbolic ref.\n> This proposal showcases different scenarios on how this could change in the\n> future.\n>\n> HEAD pointing at the superprojects index\n> ----------------------------------------\n> Introduce a new symbolic ref that points at the superprojects\n> index of the gitlink. The format is\n>\n>   \"repo:\" <superprojects gitdir> '\\0' <gitlink-path> '\\0'\n>\n> Just like existing symrefs, the content of the ref will be read and followed.\n> On reading \"repo:\", the sha1 will be obtained equivalent to:\n>\n>     git -C <superproject> ls-files -s <gitlink-path> | awk '{ print $2}'\n>\n> Ref write operations driven by the submodule, affecting symrefs\n>   e.g. git checkout <other branch> (in the submodule)\n>\n> In this scenario only the HEAD is optionally attached to the superproject,\n> so we can rewrite the HEAD to be anything else, such as a branch just fine.\n> Once the HEAD is not pointing at the superproject any more, we'll leave the\n> submodule alone in operations driven by the superproject.\n> To get back on the superproject branch, we’d need to invent new UX, such as\n>    git checkout --attach-superproject\n> as that is similar to --detach\n>\n\nSome of the idea trimmed for brevity, but I like this aspect the most.\nCurrently, I work on several projects which have multiple\nrepositories, which are essentially submodules.\n\nHowever, historically, we kept them separate. 99% of the time, you can\nuse all 3 projects on \"master\" and everything works. But if you go\nback in time, there's no correlation to \"what did the parent project\nwant this \"COMMON\" folder to be at?\n\nI started promoting using submodules for this, since it seemed quite natural.\n\nThe core problem, is that several developers never quite understood or\ngrasped how submodules worked. There's problems like \"but what if I\nwanna work on master?\" or people assume submodules need to be checked\nout at master instead of in a detached HEAD state.\n\nSo we often get people who don't run git submodule update and thus are\nconfused about why their submodules are often out of date. (This can\nbe solved by recursive options to commands to more often recurse into\nsubmodules and checkout and update them).\n\nWe also often get people who accidentally commit the old version of\nthe repository, or commit an update to the parent project pointing the\nsubmodule at some commit which isn't yet in the upstream of the common\nrepository.\n\nThe proposal here seems to match the intuition about how submodules\nshould work, with the ability to \"attach\" or \"detach\" the submodule\nwhen working on the submodule directly.\n\nIdeally, I'd like for more ways to say \"ignore what my submodule is\nchecked out at, since I will have something else checked out, and\ndon't intend to commit just yet.\"\n\nBasically, a workflow where it's easier to have each submodule checked\nout at master, and we can still keep track of historical relationship\nof what commit was the submodule at some time ago, but without causing\nsome of these headaches.\n\nI've often tried to use the \"--skip-worktree\" bit to have people set\ntheir repository to ignore the submodule. Unfortunately, this is\npretty complex, and most of the time, developers never remember to do\nthis again on a fresh clone.\n\nThanks,\nJake\n"},{"id":"332128","messageId":"CAGZ79kZL2_v5S5OJ_FnuZbHrKmPX9gXwoyX36G0br+5i87JQhw@mail.gmail.com","threadId":"47140","inReplyTo":"xmqqbmkcjaxo.fsf@gitster.mtv.corp.google.com","subject":"Re: [RFD] Long term plan with submodule refs?","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-11-09T19:57:01Z","receivedAt":"2017-11-09T19:57:09Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Nov 8, 2017 at 9:08 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Stefan Beller <sbeller@google.com> writes:\n>\n>>> The relationship is indeed currently useful, but if the long term plan\n>>> is to strongly discourage detached submodule HEAD, then I would think\n>>> that these patches are in the wrong direction. (If the long term plan is\n>>> to end up supporting both detached and linked submodule HEAD, then these\n>>> patches are fine, of course.) So I think that the plan referenced in\n>>> Junio's email (that you linked above) still needs to be discussed.\n>>\n>> This email presents different approaches.\n>>\n>> Objective\n>> =========\n>> This document should summarize the current situation of Git submodules\n>> and start a discussion of where it can be headed long term.\n>> Show different ways in which submodule refs could evolve.\n>>\n>> Background\n>> ==========\n>> Submodules in Git are considered as an independet repository currently.\n>> This is okay for current workflows, such as utilizing a library that is\n>> rarely updated. Other workflows that require a tighter integration between\n>> submodule and superproject are possible, but cumbersome as there is an\n>> additional step that has to be performed, which is the update of the gitlink\n>> pointer in the superproject.\n>\n> I do not think \"rarely updaed\" is an issue.\n>\n> The problem is that we may want to make it easier to use a\n> superproject and its submodules as if the combined whole were a\n> single project, which currently is not easy, primarily because\n> submodules are separate entities with different set of branches that\n> can be checked out independently from what branch the superproject\n> is working on.\n\nWell and this fact seems to be not a problem in the current use of submodules,\nprecisely because the workflow either (a) is not too cumbersome or (b)\nis executed\nnot too often to bother enough.\n\n> These are good starting points for copying such a combined whole to\n> your local machine and start working on it.  The more interesting,\n> important, and potentially difficult part is how the result of such\n> work is shared back to where you started from.  \"push --recursive\"\n> may be a simple phrase, but a sensible definition of how it should\n> work won't be that simple.\n...\n>\n> We should make detached HEAD safe against gc if it is not,\n> regardless of the use of submodules.  I thought it already was made\n> safe long time ago.\n\nThe detached HEAD itself is protected via its reflog (which is around\nfor say 2 weeks?)\n\nIf I were to develop using detached HEAD only in todays world of\nsubmodules using different branches in the superproject, I run the risk\nof loosing some commits in the submodule, as they are not the detached\nHEAD all the time, but might even be loose tips.\n\nThis combined with the previous paragraph brings in another important\nconcern:\nSome projects would have a very different history when used as a\nsubmodule compared to when used as a stand alone project.\nOther projects may be closely aligned between their branches and\nwhat the superproject records.\n\nSo the more we deviate from the traditional branch model, the easier\nwe make it to have the submodule tips be very different from the\nstandalone tips, which may overexpose us to the gc issues as well as\nthe general question how much these projects have in common.\n\n>> Use replicate refs in submodules\n>> --------------------------------\n>> This approach will replicate the superproject refs into the submodule\n>> ref namespace, e.g. git-branch learns about --recurse-submodules, which\n>> creates a branch of a given name in all submodules. These (topic) branches\n>> should be kept in sync with the superproject\n>>\n>> Pros:\n>>  * This seemed intuitive to Gerrit users\n>>  * 'quick' to implement, most of the commands are already there,\n>>    just git-branch is needed to have the workflows mentioned above complete.\n>> Cons:\n>>  * What does \"git checkout -b A B\" mean? (special case: B == HEAD)\n>\n> The command ran at which level?  In the superproject, or in a single\n> submodule?\n\nIn the superproject, with --recurse-submodules, as the A and B would recurse\nas strings, and not change meaning depending on the gitlink value.\n\n>\n>>    Is the branch name replicated as a string into the submodule operation,\n>>    or do we dereference the superprojects gitlink and walk from there?\n>\n> If they are \"kept in sync with the superproject\", then there should\n> be no difference between the two, so I do not see any room for\n> wondering about that.\n\nExcept you can still break out by issuing commands in the submodule\nto change the submodule refs to be different from the superproject.\n\nThis was also more along the lines of thinking about the (Gerrit) remote,\nwhich does and okay, but not stellar job in keeping the remote branches\nfor superproject and submodule in sync. I'd expect glitches there.\n\n> In other words, if there is need to worry\n> about the differences between the above two, then it probably is\n> fundamentally impossible to keep these in sync, and a design that\n> assumes it is possible would have to expose glitches to the end-user\n> experience.\n\nyup. And by exposing you probably mean a patch series as presented?\n(git status/log/diff making noise about the glitch?)\n\n> I do not know if glitches resulting from there would be so severe to\n> be show-stoppers, though.  It might be possible to paper them over.\n\nI think so, too, as long as the user is pointed at the glitch to correct them.\n\n>\n>> No submodule refstore at all\n>> ----------------------------\n>> Use refs and commits in the superproject to stitch submodule changes\n>> together. Disallow branches in the submodule. This is only restricted\n>> to the working tree inside the superproject, such that the output of git-branch\n>> changes depending whether the working tree is in- or outside the superproject\n>> working tree.\n>\n> This would need enhancement for reachability code, but it feels the\n> cleanest from the philosophical standpoint---if you want to treat a\n> superproject and its submodules as if it were a single project,\n> ability to check out a branch in a submodule that does not match\n> that of the superproject would only get in the way of preserving the\n> illusion of \"single project\"-ness.\n\nI wonder if we can combine this with the approach Jonathan gave above.\nIn the worktree (of the submodule inside the superproject) you are allowed\nto use these \"mirrored\" refs, whereas in any other worktree you have full\naccess to the normal refs of the project.\n\n>\n>> New type of symbolic refs\n>> =========================\n>> A symbolic ref can currently only point at a ref or another symbolic ref.\n>> This proposal showcases different scenarios on how this could change in the\n>> future.\n>>\n>> HEAD pointing at the superprojects index\n>> ----------------------------------------\n>\n> This looks to me a mere implementation detail for a (part of)\n> necessary component to realize the above \"No submodule refstore\".\n\nAh ok.\n\nIf all branches would use this new symref type, the handling would\nseem to be very similar to what Jonathan described with a new type\nof refstore instead.\n\n>> Superproject operations spanning index and worktree\n>>   E.g. git reset --mixed\n>> As the submodules HEAD is defined in the index, we would reset it to the\n>> version in the last commit. As --mixed promises to not touch the working tree,\n>> the submodules worktree would not be touched. git reset --mixed in the\n>> superproject is the same as --soft in the submodule.\n>\n> I am not sure if you want to take these promises low-level \"single\n> repository\" plumbing operations make too literally.  \"reset --mixed\"\n> may promise not to touch the working tree, but it also promises not\n> to touch submodules at all.  If you are breaking the latter anyway,\n> it would make more sense not to be afraid of breaking the former if\n> it makes sense in the context of allowing the command to do more by\n> breaking the latter.\n\nok.\n\nThanks,\nStefan\n"},{"id":"332132","messageId":"CAGZ79kZAvMKQUjbqWZkhy39sE5e9k1DmkiA42ywiw2NgY1+Xig@mail.gmail.com","threadId":"47140","inReplyTo":"CA+P7+xpTCHHNG1tAhysOFV5jFHpXdzRk5zfYNHUUVdYiGY55cA@mail.gmail.com","subject":"Re: [RFD] Long term plan with submodule refs?","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-11-09T20:16:41Z","receivedAt":"2017-11-09T20:16:48Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Nov 8, 2017 at 10:54 PM, Jacob Keller <jacob.keller@gmail.com> wrote:\n> On Wed, Nov 8, 2017 at 4:10 PM, Stefan Beller <sbeller@google.com> wrote:\n>>> The relationship is indeed currently useful, but if the long term plan\n>>> is to strongly discourage detached submodule HEAD, then I would think\n>>> that these patches are in the wrong direction. (If the long term plan is\n>>> to end up supporting both detached and linked submodule HEAD, then these\n>>> patches are fine, of course.) So I think that the plan referenced in\n>>> Junio's email (that you linked above) still needs to be discussed.\n>>\n>\n>> New type of symbolic refs\n>> =========================\n>> A symbolic ref can currently only point at a ref or another symbolic ref.\n>> This proposal showcases different scenarios on how this could change in the\n>> future.\n>>\n>> HEAD pointing at the superprojects index\n>> ----------------------------------------\n>> Introduce a new symbolic ref that points at the superprojects\n>> index of the gitlink. The format is\n>>\n>>   \"repo:\" <superprojects gitdir> '\\0' <gitlink-path> '\\0'\n>>\n>> Just like existing symrefs, the content of the ref will be read and followed.\n>> On reading \"repo:\", the sha1 will be obtained equivalent to:\n>>\n>>     git -C <superproject> ls-files -s <gitlink-path> | awk '{ print $2}'\n>>\n>> Ref write operations driven by the submodule, affecting symrefs\n>>   e.g. git checkout <other branch> (in the submodule)\n>>\n>> In this scenario only the HEAD is optionally attached to the superproject,\n>> so we can rewrite the HEAD to be anything else, such as a branch just fine.\n>> Once the HEAD is not pointing at the superproject any more, we'll leave the\n>> submodule alone in operations driven by the superproject.\n>> To get back on the superproject branch, we’d need to invent new UX, such as\n>>    git checkout --attach-superproject\n>> as that is similar to --detach\n>>\n>\n> Some of the idea trimmed for brevity, but I like this aspect the most.\n> Currently, I work on several projects which have multiple\n> repositories, which are essentially submodules.\n>\n> However, historically, we kept them separate. 99% of the time, you can\n> use all 3 projects on \"master\" and everything works. But if you go\n> back in time, there's no correlation to \"what did the parent project\n> want this \"COMMON\" folder to be at?\n\nSo an environment where \"git submodule update --remote\" is not that\nharmful, but rather brings the joy of being up to date in each project?\n\n> I started promoting using submodules for this, since it seemed quite natural.\n>\n> The core problem, is that several developers never quite understood or\n> grasped how submodules worked. There's problems like \"but what if I\n> wanna work on master?\" or people assume submodules need to be checked\n> out at master instead of in a detached HEAD state.\n\nSo the documentation sucks?\n\nIt is intentional that from the superprojects perspective the gitlink\nmust be one\nexact value, and rely on the submodule to get to and keep that state.\n\n(I think we once discussed if setting the gitlink value to 00...00 or otherwise\nsignal that we actually want \"the most recent tip of the X branch\" would be\na good idea, but I do not think it as it misses the point of versioning)\n\n> So we often get people who don't run git submodule update and thus are\n> confused about why their submodules are often out of date. (This can\n> be solved by recursive options to commands to more often recurse into\n> submodules and checkout and update them).\n>\n> We also often get people who accidentally commit the old version of\n> the repository, or commit an update to the parent project pointing the\n> submodule at some commit which isn't yet in the upstream of the common\n> repository.\n\nWould an upstream prereceive hook (maybe even builtin and accessible via\n'receive.denyUnreachableSubmodules') help? (It would require submodules\nto be defined with relative URLs in the .gitmodules file and then the receive\ncommand can check for the gitlink value present in this other repository)\n\n> The proposal here seems to match the intuition about how submodules\n> should work, with the ability to \"attach\" or \"detach\" the submodule\n> when working on the submodule directly.\n\nWell I think the big picture discussion is how easy this attaching or\ndetaching is. Whether only the HEAD is attached or detached, or if we\ninvent a new refstore that is a complete new submodule thing, which\ncannot be detached from the superproject at all.\n\n> Ideally, I'd like for more ways to say \"ignore what my submodule is\n> checked out at, since I will have something else checked out, and\n> don't intend to commit just yet.\"\n\nThis is in the superproject, when doing a git add . ?\n\n> Basically, a workflow where it's easier to have each submodule checked\n> out at master, and we can still keep track of historical relationship\n> of what commit was the submodule at some time ago, but without causing\n> some of these headaches.\n\nSo essentially a repo or otherwise parallel workflow just with the versioning\nhappening magically behind your back?\n\n> I've often tried to use the \"--skip-worktree\" bit to have people set\n> their repository to ignore the submodule. Unfortunately, this is\n> pretty complex, and most of the time, developers never remember to do\n> this again on a fresh clone.\n\nThat sounds interesting.\n\nThanks,\nStefan\n"},{"id":"332146","messageId":"CA+P7+xoA5KNW7PEiW0wZp-aohsKbnH8AGAvkk3Y3Ghv=Jhp+hw@mail.gmail.com","threadId":"47140","inReplyTo":"CAGZ79kZAvMKQUjbqWZkhy39sE5e9k1DmkiA42ywiw2NgY1+Xig@mail.gmail.com","subject":"Re: [RFD] Long term plan with submodule refs?","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-11-10T03:37:11Z","receivedAt":"2017-11-10T03:37:39Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Thu, Nov 9, 2017 at 12:16 PM, Stefan Beller <sbeller@google.com> wrote:\n> On Wed, Nov 8, 2017 at 10:54 PM, Jacob Keller <jacob.keller@gmail.com> wrote:\n>> On Wed, Nov 8, 2017 at 4:10 PM, Stefan Beller <sbeller@google.com> wrote:\n>>>> The relationship is indeed currently useful, but if the long term plan\n>>>> is to strongly discourage detached submodule HEAD, then I would think\n>>>> that these patches are in the wrong direction. (If the long term plan is\n>>>> to end up supporting both detached and linked submodule HEAD, then these\n>>>> patches are fine, of course.) So I think that the plan referenced in\n>>>> Junio's email (that you linked above) still needs to be discussed.\n>>>\n>>\n>>> New type of symbolic refs\n>>> =========================\n>>> A symbolic ref can currently only point at a ref or another symbolic ref.\n>>> This proposal showcases different scenarios on how this could change in the\n>>> future.\n>>>\n>>> HEAD pointing at the superprojects index\n>>> ----------------------------------------\n>>> Introduce a new symbolic ref that points at the superprojects\n>>> index of the gitlink. The format is\n>>>\n>>>   \"repo:\" <superprojects gitdir> '\\0' <gitlink-path> '\\0'\n>>>\n>>> Just like existing symrefs, the content of the ref will be read and followed.\n>>> On reading \"repo:\", the sha1 will be obtained equivalent to:\n>>>\n>>>     git -C <superproject> ls-files -s <gitlink-path> | awk '{ print $2}'\n>>>\n>>> Ref write operations driven by the submodule, affecting symrefs\n>>>   e.g. git checkout <other branch> (in the submodule)\n>>>\n>>> In this scenario only the HEAD is optionally attached to the superproject,\n>>> so we can rewrite the HEAD to be anything else, such as a branch just fine.\n>>> Once the HEAD is not pointing at the superproject any more, we'll leave the\n>>> submodule alone in operations driven by the superproject.\n>>> To get back on the superproject branch, we’d need to invent new UX, such as\n>>>    git checkout --attach-superproject\n>>> as that is similar to --detach\n>>>\n>>\n>> Some of the idea trimmed for brevity, but I like this aspect the most.\n>> Currently, I work on several projects which have multiple\n>> repositories, which are essentially submodules.\n>>\n>> However, historically, we kept them separate. 99% of the time, you can\n>> use all 3 projects on \"master\" and everything works. But if you go\n>> back in time, there's no correlation to \"what did the parent project\n>> want this \"COMMON\" folder to be at?\n>\n> So an environment where \"git submodule update --remote\" is not that\n> harmful, but rather brings the joy of being up to date in each project?\n>\n>> I started promoting using submodules for this, since it seemed quite natural.\n>>\n>> The core problem, is that several developers never quite understood or\n>> grasped how submodules worked. There's problems like \"but what if I\n>> wanna work on master?\" or people assume submodules need to be checked\n>> out at master instead of in a detached HEAD state.\n>\n> So the documentation sucks?\n>\n> It is intentional that from the superprojects perspective the gitlink\n> must be one\n> exact value, and rely on the submodule to get to and keep that state.\n>\n> (I think we once discussed if setting the gitlink value to 00...00 or otherwise\n> signal that we actually want \"the most recent tip of the X branch\" would be\n> a good idea, but I do not think it as it misses the point of versioning)\n>\n>> So we often get people who don't run git submodule update and thus are\n>> confused about why their submodules are often out of date. (This can\n>> be solved by recursive options to commands to more often recurse into\n>> submodules and checkout and update them).\n>>\n>> We also often get people who accidentally commit the old version of\n>> the repository, or commit an update to the parent project pointing the\n>> submodule at some commit which isn't yet in the upstream of the common\n>> repository.\n>\n> Would an upstream prereceive hook (maybe even builtin and accessible via\n> 'receive.denyUnreachableSubmodules') help? (It would require submodules\n> to be defined with relative URLs in the .gitmodules file and then the receive\n> command can check for the gitlink value present in this other repository)\n>\n>> The proposal here seems to match the intuition about how submodules\n>> should work, with the ability to \"attach\" or \"detach\" the submodule\n>> when working on the submodule directly.\n>\n> Well I think the big picture discussion is how easy this attaching or\n> detaching is. Whether only the HEAD is attached or detached, or if we\n> invent a new refstore that is a complete new submodule thing, which\n> cannot be detached from the superproject at all.\n>\n>> Ideally, I'd like for more ways to say \"ignore what my submodule is\n>> checked out at, since I will have something else checked out, and\n>> don't intend to commit just yet.\"\n>\n> This is in the superproject, when doing a git add . ?\n>\n\nYes.\n\n>> Basically, a workflow where it's easier to have each submodule checked\n>> out at master, and we can still keep track of historical relationship\n>> of what commit was the submodule at some time ago, but without causing\n>> some of these headaches.\n>\n> So essentially a repo or otherwise parallel workflow just with the versioning\n> happening magically behind your back?\n\nIdeally, my developers would like to just have each submodule checked\nout at master.\n\nIdeally, I'd like to be able to checkout an old version of the parent\nproject and have it recorded what version of the shared submodule was\nat at the time.\n\nIdeally, my developers don't want to have to worry about knowing that\nthey shouldn't \"git add -a\" or \"git commit -a\" when they have a\nsubmodule checked out at a different location from the parent projects\ngitlink.\n\nThanks,\nJake\n\n>\n>> I've often tried to use the \"--skip-worktree\" bit to have people set\n>> their repository to ignore the submodule. Unfortunately, this is\n>> pretty complex, and most of the time, developers never remember to do\n>> this again on a fresh clone.\n>\n> That sounds interesting.\n>\n> Thanks,\n> Stefan\n"},{"id":"332245","messageId":"CAGZ79kbBypJfDn28j2Z_SCFG9iCZnx-sWmH6D2vqnS+tNKpnBw@mail.gmail.com","threadId":"47140","inReplyTo":"CA+P7+xoA5KNW7PEiW0wZp-aohsKbnH8AGAvkk3Y3Ghv=Jhp+hw@mail.gmail.com","subject":"Re: [RFD] Long term plan with submodule refs?","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-11-10T20:01:39Z","receivedAt":"2017-11-10T20:01:45Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":">\n>>> Basically, a workflow where it's easier to have each submodule checked\n>>> out at master, and we can still keep track of historical relationship\n>>> of what commit was the submodule at some time ago, but without causing\n>>> some of these headaches.\n>>\n>> So essentially a repo or otherwise parallel workflow just with the versioning\n>> happening magically behind your back?\n>\n> Ideally, my developers would like to just have each submodule checked\n> out at master.\n>\n> Ideally, I'd like to be able to checkout an old version of the parent\n> project and have it recorded what version of the shared submodule was\n> at at the time.\n\nThis sounds as if a \"passive superproject\" would work best for you, i.e.\neach commit in a submodule is bubbled up into the superproject,\nmaking a commit potentially even behind the scenes, such that the\nuser interaction with the superproject would be none.\n\nHowever this approach also sounds careless, as there is no precondition\nthat e.g. the superproject builds with all the submodules as is; it is a mere\ntracking of \"at this time we have the submodules arranged as such\",\nwhereas for the versioning aspect, you would want to have commit messages\nin the superproject saying *why* you bumped up a specific submodule.\nThe user may not like to give such an explanation as they already wrote\na commit message for the individual project.\n\nAlso this approach sounds like a local approach, as it is not clear to me,\nwhy you'd want to share the superproject history.\n\n> Ideally, my developers don't want to have to worry about knowing that\n> they shouldn't \"git add -a\" or \"git commit -a\" when they have a\n> submodule checked out at a different location from the parent projects\n> gitlink.\n>\n> Thanks,\n> Jake\n>\n"},{"id":"332276","messageId":"CA+P7+xp64ESHnQgt-9odhGKeCA6VurOozbFa6Qi2sQKmcQtZwQ@mail.gmail.com","threadId":"47140","inReplyTo":"CAGZ79kbBypJfDn28j2Z_SCFG9iCZnx-sWmH6D2vqnS+tNKpnBw@mail.gmail.com","subject":"Re: [RFD] Long term plan with submodule refs?","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-11-11T05:25:11Z","receivedAt":"2017-11-11T05:34:36Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Fri, Nov 10, 2017 at 12:01 PM, Stefan Beller <sbeller@google.com> wrote:\n>>\n>>>> Basically, a workflow where it's easier to have each submodule checked\n>>>> out at master, and we can still keep track of historical relationship\n>>>> of what commit was the submodule at some time ago, but without causing\n>>>> some of these headaches.\n>>>\n>>> So essentially a repo or otherwise parallel workflow just with the versioning\n>>> happening magically behind your back?\n>>\n>> Ideally, my developers would like to just have each submodule checked\n>> out at master.\n>>\n>> Ideally, I'd like to be able to checkout an old version of the parent\n>> project and have it recorded what version of the shared submodule was\n>> at at the time.\n>\n> This sounds as if a \"passive superproject\" would work best for you, i.e.\n> each commit in a submodule is bubbled up into the superproject,\n> making a commit potentially even behind the scenes, such that the\n> user interaction with the superproject would be none.\n>\n> However this approach also sounds careless, as there is no precondition\n> that e.g. the superproject builds with all the submodules as is; it is a mere\n> tracking of \"at this time we have the submodules arranged as such\",\n> whereas for the versioning aspect, you would want to have commit messages\n> in the superproject saying *why* you bumped up a specific submodule.\n> The user may not like to give such an explanation as they already wrote\n> a commit message for the individual project.\n>\n> Also this approach sounds like a local approach, as it is not clear to me,\n> why you'd want to share the superproject history.\n>\n>> Ideally, my developers don't want to have to worry about knowing that\n>> they shouldn't \"git add -a\" or \"git commit -a\" when they have a\n>> submodule checked out at a different location from the parent projects\n>> gitlink.\n>>\n>> Thanks,\n>> Jake\n>>\n\n\nIt doesn't need to be totally passive, in that some (or one\nmaintainer) can manage when the submodule pointer is actually updated,\nbut ideally other users don't have to worry about that and can\n\"pretend\" to always keep each submodule at master, as they have always\ndone in the past.\n\nThanks,\nJake\n"}]}