{"thread":{"id":"35608","subject":"[PATCH 1/2] git-submodule.sh: Support 'checkout' as a valid update command","startedAt":"2014-01-05T02:50:48Z","lastAt":"2014-01-15T01:02:08Z","messageCount":40,"participants":["Francesco Pretto","Heiko Voigt","W. Trevor King","Junio C Hamano","David Engster","Jens Lehmann"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"232657","messageId":"1388890249-3577-1-git-send-email-ceztko@gmail.com","threadId":"35608","inReplyTo":null,"subject":"[PATCH 1/2] git-submodule.sh: Support 'checkout' as a valid update command","fromName":"Francesco Pretto","fromEmail":"ceztko@gmail.com","sentAt":"2014-01-05T02:50:48Z","receivedAt":"2014-01-05T02:50:48Z","isPatch":true,"sender":{"key":"ceztko@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3037449?v=4"},"body":"According to \"Documentation/gitmodules.txt\", 'checkout' is a valid\n'submodule..update' command. Also \"git-submodule.sh\" refers to\nit and processes it correctly. Reflect commit 'ac1fbb' to support this\nsyntax and also validates property values during 'update' command,\nissuing a warning if the value found is unknwon.\n---\n git-submodule.sh | 14 +++++++++++++-\n 1 file changed, 13 insertions(+), 1 deletion(-)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 2677f2e..1d041a7 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -622,7 +622,7 @@ cmd_init()\n \t\t test -z \"$(git config submodule.\"$name\".update)\"\n \t\tthen\n \t\t\tcase \"$upd\" in\n-\t\t\trebase | merge | none)\n+\t\t\tcheckout | rebase | merge | none)\n \t\t\t\t;; # known modes of updating\n \t\t\t*)\n \t\t\t\techo >&2 \"warning: unknown update mode '$upd' suggested for submodule '$name'\"\n@@ -805,6 +805,18 @@ cmd_update()\n \t\t\tupdate_module=$update\n \t\telse\n \t\t\tupdate_module=$(git config submodule.\"$name\".update)\n+\t\t\tcase \"$update_module\" in\n+\t\t\t'')\n+\t\t\t\t;; # Unset update mode\n+\t\t\tcheckout | rebase | merge | none)\n+\t\t\t\t;; # Known update modes\n+\t\t\t!*)\n+\t\t\t\t;; # Custom update command\n+\t\t\t*)\n+\t\t\t\tupdate_module=\n+\t\t\t\techo >&2 \"warning: invalid update mode for submodule '$name'\"\n+\t\t\t\t;;\n+\t\t\tesac\n \t\tfi\n \n \t\tdisplaypath=$(relative_path \"$prefix$sm_path\")\n-- \n1.8.5.2.230.g032cd47.dirty\n"},{"id":"232658","messageId":"1388890249-3577-2-git-send-email-ceztko@gmail.com","threadId":"35608","inReplyTo":"1388890249-3577-1-git-send-email-ceztko@gmail.com","subject":"[PATCH 2/2] Introduce git submodule attached update","fromName":"Francesco Pretto","fromEmail":"ceztko@gmail.com","sentAt":"2014-01-05T02:50:49Z","receivedAt":"2014-01-05T02:50:49Z","isPatch":true,"sender":{"key":"ceztko@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3037449?v=4"},"body":"At the current state, the following use-case is not supported very\nwell in git:\n- a maintainer adds a submodule, checking out a specific branch of\nthe repository. He doesn't track the upstream submodule revision sha1;\n- a developer checkout the repository branch decided by the maintainer.\nSubsequent \"merge\" or \"rebase\" update operations don't detach the HEAD.\n\nTo ease the above use-case this patch:\n- introduces a \"submodule..attached\" property that, when set\n to \"true\", ensures that the \"update\" operation will result in\n the HEAD attached to a branch;\n- introduces \"--attach|--dettach\" switches to the submodule \"update\"\n command: they attach/detach the HEAD, overriding\n \"submodule..attached\" property value;\n- introduces \"--attached-update\" switch to the \"add\" operation. It:\n * sets \"submodule..attached\" to true;\n * sets \"submodule..ignore\" to all.\n\nUsing the '--attach' switch or operating in a repository with\n'submodule..attached' set to 'true' during \"update\" will:\n- checkout a branch with an attached HEAD if the repository was just\ncloned;\n- perform a fast-forward only merge of changes if it's a 'checkout'\nupdate operation;\n- reattach the HEAD prior performing a 'merge', 'rebase' or '!command'\nupdate operation if the HEAD was found detached. Orphaned commits\nwill also be merged back in the branch.\n\n'--attach' or 'submodule..attached' set to true also implies '--remote'.\n\nUsing the '--detach' switch or operating in a repository with\n'submodule..attached' set to 'false' during \"update\" will:\n- checkout a detached HEAD if the repository was just cloned;\n- detach the HEAD prior performing a 'merge', 'rebase' or '!command'\nupdate operation if the HEAD was found attached.\n\n'submodule..attached' works similarly to 'submodule..update'\nproperty: git copies the values found in \".gitmodules\" in \".git/config\" when\nperforming an \"init\" command. \"update\" looks for values in \".git/config\"\nonly.\n\n'--attach' and '--detach' switches override an opposite behaviour\nof 'submodule..attached' properties.\n\nThe patch is strongly additive and doesn't break any submodule specific\ntest. It also adds some tests specific to the added feature.\n---\n Documentation/git-submodule.txt | 48 +++++--\n Documentation/gitmodules.txt | 10 +-\n git-submodule.sh | 154 +++++++++++++++++++--\n t/t7410-submodule-attached-head.sh | 268 +++++++++++++++++++++++++++++++++++++\n 4 files changed, 457 insertions(+), 23 deletions(-)\n create mode 100755 t/t7410-submodule-attached-head.sh\n\ndiff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt\nindex bfef8a0..b97eefb 100644\n--- a/Documentation/git-submodule.txt\n+++ b/Documentation/git-submodule.txt\n@@ -10,13 +10,14 @@ SYNOPSIS\n --------\n [verse]\n 'git submodule' [--quiet] add [-b ] [-f|--force] [--name ]\n-\t [--reference ] [--depth ] [--] []\n+\t [--reference ] [--attached-update] [--depth ]\n+\t [--] []\n 'git submodule' [--quiet] status [--cached] [--recursive] [--] [...]\n 'git submodule' [--quiet] init [--] [...]\n 'git submodule' [--quiet] deinit [-f|--force] [--] ...\n 'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch]\n-\t [-f|--force] [--rebase] [--reference ] [--depth ]\n-\t [--merge] [--recursive] [--] [...]\n+\t [-f|--force] [--rebase] [--reference ] [--attach | --detach]\n+\t [--depth ] [--merge] [--recursive] [--] [...]\n 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) ]\n \t [commit] [--] [...]\n 'git submodule' [--quiet] foreach [--recursive] \n@@ -107,6 +108,9 @@ is the superproject and submodule repositories will be kept\n together in the same relative location, and only the\n superproject's URL needs to be provided: git-submodule will correctly\n locate the submodule using the relative URL in .gitmodules.\n++\n+If `--attached-update` is specified, the property `submodule..attached`\n+will be set to `true` and `submodule..ignore` will be set to `all`.\n \n status::\n \tShow the status of the submodules. This will print the SHA-1 of the\n@@ -156,12 +160,15 @@ it contains local modifications.\n update::\n \tUpdate the registered submodules, i.e. clone missing submodules and\n \tcheckout the commit specified in the index of the containing repository.\n-\tThis will make the submodules HEAD be detached unless `--rebase` or\n-\t`--merge` is specified or the key `submodule.$name.update` is set to\n-\t`rebase`, `merge` or `none`. `none` can be overridden by specifying\n-\t`--checkout`. Setting the key `submodule.$name.update` to `!command`\n-\twill cause `command` to be run. `command` can be any arbitrary shell\n-\tcommand that takes a single argument, namely the sha1 to update to.\n+\tThis will make the submodules HEAD be detached unless `--attach` is\n+\tspecified or `submodule.$name.attached` is set to `true`. The last setting\n+\tcan always be overridden specifying `--detach`. Update mode can be\n+\tselected specifying `--checkout`, `--rebase` or `--merge` switches\n+\tor setting the key `submodule.$name.update` to `checkout`, `rebase`,\n+\t`merge` or `none`. `none` will cause the submodule to be skipped during\n+\tthe update. Setting the key `submodule.$name.update` to `!command` will\n+\tcause `command` to be run. `command` can be any arbitrary shell command\n+\tthat takes a single argument, namely the sha1 to update to.\n +\n If the submodule is not yet initialized, and you just want to use the\n setting as stored in .gitmodules, you can automatically initialize the\n@@ -270,6 +277,23 @@ OPTIONS\n \tbe overridden by setting the `submodule..branch` option in\n \teither `.gitmodules` or `.git/config` (with `.git/config` taking\n \tprecedence).\n+\n+--attached-update::\n+\tThis option is only valid for the add command. Causes the add command\n+\talso to set the property `submodule..attached` to `true` and\n+\tthe property `submodule..ignore` to `all`.\n+\n+--attach::\n+\tThis option is only valid for the update commands. Causes the result\n+\tof an update operation to be an attached HEAD. In the update operation,\n+\tthe branch named by 'submodule..branch' is checked out as the new\n+\tHEAD of the submodule repository. If 'submodule..branch' is not\n+\tset, the 'master' branch is checked out as the new HEAD of the\n+\tsubmodule. Note: `--attach` also implies `--remote`.\n+\n+--detach::\n+\tThis option is only valid for the update command. Forces the result\n+\tof the update operation to be a detached HEAD in the submodule.\n +\n This works for any of the supported update procedures (`--checkout`,\n `--rebase`, etc.). The only change is the source of the target SHA-1.\n@@ -290,8 +314,7 @@ SHA-1. If you don't want to fetch, you should use `submodule update\n --merge::\n \tThis option is only valid for the update command.\n \tMerge the commit recorded in the superproject into the current branch\n-\tof the submodule. If this option is given, the submodule's HEAD will\n-\tnot be detached. If a merge failure prevents this process, you will\n+\tof the submodule. If a merge failure prevents this process, you will\n \thave to resolve the resulting conflicts within the submodule with the\n \tusual conflict resolution tools.\n \tIf the key `submodule.$name.update` is set to `merge`, this option is\n@@ -300,8 +323,7 @@ SHA-1. If you don't want to fetch, you should use `submodule update\n --rebase::\n \tThis option is only valid for the update command.\n \tRebase the current branch onto the commit recorded in the\n-\tsuperproject. If this option is given, the submodule's HEAD will not\n-\tbe detached. If a merge failure prevents this process, you will have\n+\tsuperproject. If a merge failure prevents this process, you will have\n \tto resolve these failures with linkgit:git-rebase[1].\n \tIf the key `submodule.$name.update` is set to `rebase`, this option is\n \timplicit.\ndiff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt\nindex f7be93f..9c436db 100644\n--- a/Documentation/gitmodules.txt\n+++ b/Documentation/gitmodules.txt\n@@ -38,7 +38,8 @@ submodule..url::\n submodule..update::\n \tDefines what to do when the submodule is updated by the superproject.\n \tIf 'checkout' (the default), the new commit specified in the\n-\tsuperproject will be checked out in the submodule on a detached HEAD.\n+\tsuperproject (or branch, with '--attach') will be checked out in\n+\tthe submodule.\n \tIf 'rebase', the current branch of the submodule will be rebased onto\n \tthe commit specified in the superproject. If 'merge', the commit\n \tspecified in the superproject will be merged into the current branch\n@@ -54,6 +55,13 @@ submodule..branch::\n \tIf the option is not specified, it defaults to 'master'. See the\n \t`--remote` documentation in linkgit:git-submodule[1] for details.\n \n+submodule..attached::\n+\tDetermine if the update operation will produce a detached HEAD or not.\n+\tValid values are `true` or `false`. If the property is set to `true`\n+\tand `submodule..branch` is not set, the branch `master` will\n+\tbe checked out. If `submodule..branch` is set the branch\n+\tspecified will be checked out instead.\n+\n submodule..fetchRecurseSubmodules::\n \tThis option can be used to control recursive fetching of this\n \tsubmodule. If this option is also present in the submodules entry in\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 1d041a7..bc6df2b 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -5,11 +5,11 @@\n # Copyright (c) 2007 Lars Hjemli\n \n dashless=$(basename \"$0\" | sed -e 's/-/ /')\n-USAGE=\"[--quiet] add [-b ] [-f|--force] [--name ] [--reference ] [--] []\n+USAGE=\"[--quiet] add [-b ] [-f|--force] [--name ] [--reference ] [--attached-update] [--] []\n or: $dashless [--quiet] status [--cached] [--recursive] [--] [...]\n or: $dashless [--quiet] init [--] [...]\n or: $dashless [--quiet] deinit [-f|--force] [--] ...\n- or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...]\n+ or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--attach | --detach] [--merge] [--recursive] [--] [...]\n or: $dashless [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...]\n or: $dashless [--quiet] foreach [--recursive] \n or: $dashless [--quiet] sync [--recursive] [--] [...]\"\n@@ -36,6 +36,9 @@ update=\n prefix=\n custom_name=\n depth=\n+attach=\n+detach=\n+attached_update=\n \n # The function takes at most 2 arguments. The first argument is the\n # URL that navigates to the submodule origin repo. When relative, this URL\n@@ -352,6 +355,9 @@ cmd_add()\n \t\t\tcustom_name=$2\n \t\t\tshift\n \t\t\t;;\n+\t\t--attached-update)\n+\t\t\tattached_update=yes\n+\t\t\t;;\n \t\t--depth)\n \t\t\tcase \"$2\" in '') usage ;; esac\n \t\t\tdepth=\"--depth=$2\"\n@@ -491,6 +497,12 @@ Use -f if you really want to add it.\" >&2\n \tthen\n \t\tgit config -f .gitmodules submodule.\"$sm_name\".branch \"$branch\"\n \tfi &&\n+\tif test -n \"$attached_update\"\n+\tthen\n+\t\t# We'll stay stick to the HEAD, no need to track revision sha1\n+\t\tgit config -f .gitmodules submodule.\"$sm_name\".attached \"true\"\n+\t\tgit config -f .gitmodules submodule.\"$sm_name\".ignore \"all\"\n+\tfi &&\n \tgit add --force .gitmodules ||\n \tdie \"$(eval_gettext \"Failed to register submodule '\\$sm_path'\")\"\n }\n@@ -632,6 +644,22 @@ cmd_init()\n \t\t\tgit config submodule.\"$name\".update \"$upd\" ||\n \t\t\tdie \"$(eval_gettext \"Failed to register update mode for submodule path '\\$displaypath'\")\"\n \t\tfi\n+\n+\t\t# Copy \"attached\" setting when it is not set yet\n+\t\tif attached=\"$(git config -f .gitmodules submodule.\"$name\".attached)\" &&\n+\t\t test -n \"$attached\" &&\n+\t\t test -z \"$(git config submodule.\"$name\".attached)\"\n+\t\tthen\n+\t\t\tcase \"$attached\" in\n+\t\t\ttrue | false)\n+\t\t\t\t;; # Valid attach flag values\n+\t\t\t*)\n+\t\t\t\techo >&2 \"warning: invalid attach flag value for submodule '$name'\"\n+\t\t\t\t;;\n+\t\t\tesac\n+\t\t\tgit config submodule.\"$name\".attached \"$attached\" ||\n+\t\t\tdie \"$(eval_gettext \"Failed to register attach option for submodule path '\\$displaypath'\")\"\n+\t\tfi\n \tdone\n }\n \n@@ -750,6 +778,14 @@ cmd_update()\n \t\t--reference=*)\n \t\t\treference=\"$1\"\n \t\t\t;;\n+\t\t--attach)\n+\t\t\tif test -n \"$detach\" ; then usage ; fi\n+\t\t\tattach=1\n+\t\t\t;;\n+\t\t--detach)\n+\t\t\tif test -n \"$attach\" ; then usage ; fi\n+\t\t\tdetach=1\n+\t\t\t;;\n \t\t-m|--merge)\n \t\t\tupdate=\"merge\"\n \t\t\t;;\n@@ -800,6 +836,28 @@ cmd_update()\n \t\tname=$(module_name \"$sm_path\") || exit\n \t\turl=$(git config submodule.\"$name\".url)\n \t\tbranch=$(get_submodule_config \"$name\" branch master)\n+\t\tattach_module=\n+\t\tdetach_module=\n+\t\tif test -n \"$attach\" -o -n \"$detach\"\n+\t\tthen\n+\t\t\tattach_module=$attach\n+\t\t\tdetach_module=$detach\n+\t\telse\n+\t\t\tattached=$(git config submodule.\"$name\".attached)\n+\t\t\tcase \"$attached\" in\n+\t\t\t'')\n+\t\t\t\t;; # Unset attach flag\n+\t\t\ttrue)\n+\t\t\t\tattach_module=1\n+\t\t\t\t;;\n+\t\t\tfalse)\n+\t\t\t\tdetach_module=1\n+\t\t\t\t;;\n+\t\t\t*)\n+\t\t\t\techo >&2 \"warning: invalid attach flag value for submodule '$name'\"\n+\t\t\t\t;;\n+\t\t\tesac\n+\t\tfi\n \t\tif ! test -z \"$update\"\n \t\tthen\n \t\t\tupdate_module=$update\n@@ -848,7 +906,16 @@ Maybe you want to use 'update --init'?\")\"\n \t\t\tdie \"$(eval_gettext \"Unable to find current revision in submodule path '\\$displaypath'\")\"\n \t\tfi\n \n-\t\tif test -n \"$remote\"\n+\t\thead_rev_ref=$(clear_local_git_env; cd \"$sm_path\" && git rev-parse --abbrev-ref HEAD) ||\n+\t\tdie \"$(eval_gettext \"Unable to determine revision ref in submodule path '\\$sm_path'\")\"\n+\t\thead_detached=\n+\t\tif test \"$head_rev_ref\" = \"HEAD\"\n+\t\tthen\n+\t\t\t# Determine if the HEAD is detached\n+\t\t\thead_detached=\"true\"\n+\t\tfi\n+\n+\t\tif test -n \"$remote\" -o -n \"$attach_module\"\n \t\tthen\n \t\t\tif test -z \"$nofetch\"\n \t\t\tthen\n@@ -862,7 +929,8 @@ Maybe you want to use 'update --init'?\")\"\n \t\t\tdie \"$(eval_gettext \"Unable to find current ${remote_name}/${branch} revision in submodule path '\\$sm_path'\")\"\n \t\tfi\n \n-\t\tif test \"$subsha1\" != \"$sha1\" -o -n \"$force\"\n+\t\tif test \"$subsha1\" != \"$sha1\" || test -n \"$attach_module\" -a -n \"$head_detached\" ||\n+\t\t\ttest -n \"$detach_module\" -a -z \"$head_detached\" || test -n \"$force\"\n \t\tthen\n \t\t\tsubforce=$force\n \t\t\t# If we don't already have a -f flag and the submodule has never been checked out\n@@ -882,40 +950,108 @@ Maybe you want to use 'update --init'?\")\"\n \t\t\tfi\n \n \t\t\t# Is this something we just cloned?\n+\t\t\tjust_cloned=\n \t\t\tcase \";$cloned_modules;\" in\n \t\t\t*\";$name;\"*)\n \t\t\t\t# then there is no local change to integrate\n-\t\t\t\tupdate_module= ;;\n+\t\t\t\tupdate_module=\"checkout\"\n+\t\t\t\tjust_cloned=yes\n+\t\t\t\t;;\n \t\t\tesac\n \n+\t\t\tif test -z \"$update_module\"\n+\t\t\tthen\n+\t\t\t\t# Fallback to checkout\n+\t\t\t\tupdate_module=\"checkout\"\n+\t\t\tfi\n+\n+\t\t\tcommand_attach=:\n+\t\t\tsuffix_attach=\n+\t\t\tif test \"$update_module\" != \"checkout\"\n+\t\t\tthen\n+\t\t\t\tif test -n \"$attach_module\" -a -n \"$head_detached\"\n+\t\t\t\tthen\n+\t\t\t\t\t# We need to reattach to the branch\n+\t\t\t\t\tcommand_attach=\"git checkout $subforce -q\"\n+\t\t\t\t\tsuffix_attach=$branch\n+\t\t\t\telif test -n \"$detach_module\" -a -z \"$head_detached\"\n+\t\t\t\tthen\n+\t\t\t\t\t# We need to detach from the branch\n+\t\t\t\t\tcommand_attach=\"git checkout $subforce -q\"\n+\t\t\t\t\tsuffix_attach=$sha1\n+\t\t\t\tfi\n+\t\t\tfi\n+\n+\t\t\tcommand_pre=:\n+\t\t\tsuffix_pre=\n+\t\t\tcommand_post=:\n+\t\t\tsuffix_pre=\n+\t\t\tsuffix=\n \t\t\tmust_die_on_failure=\n+\t\t\tcustom_update=\n \t\t\tcase \"$update_module\" in\n \t\t\trebase)\n \t\t\t\tcommand=\"git rebase\"\n+\t\t\t\tsuffix=$sha1\n \t\t\t\tdie_msg=\"$(eval_gettext \"Unable to rebase '\\$sha1' in submodule path '\\$displaypath'\")\"\n \t\t\t\tsay_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': rebased into '\\$sha1'\")\"\n \t\t\t\tmust_die_on_failure=yes\n+\t\t\t\tif test -n \"$attach_module\" -a -n \"$head_detached\" && test \"$subsha1\" != \"$sha1\"\n+\t\t\t\tthen\n+\t\t\t\t\t# After the rebase, we merge orphaned commits in the branch\n+\t\t\t\t\tcommand_post=\"git merge\"\n+\t\t\t\t\tsuffix_post=$subsha1\n+\t\t\t\tfi\n \t\t\t\t;;\n \t\t\tmerge)\n+\t\t\t\tif test -n \"$attach_module\" -a -n \"$head_detached\" && test \"$subsha1\" != \"$sha1\"\n+\t\t\t\tthen\n+\t\t\t\t\t# Prior the rebase, we merge orphaned commits in in the branch\n+\t\t\t\t\tcommand_pre=\"git merge\"\n+\t\t\t\t\tsuffix_pre=$subsha1\n+\t\t\t\tfi\n \t\t\t\tcommand=\"git merge\"\n+\t\t\t\tsuffix=$sha1\n \t\t\t\tdie_msg=\"$(eval_gettext \"Unable to merge '\\$sha1' in submodule path '\\$displaypath'\")\"\n \t\t\t\tsay_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': merged in '\\$sha1'\")\"\n \t\t\t\tmust_die_on_failure=yes\n \t\t\t\t;;\n+\t\t\tcheckout)\n+\t\t\t\tif test -n \"$attach_module\"\n+\t\t\t\tthen\n+\t\t\t\t\tcommand=\"git checkout $subforce -q\"\n+\t\t\t\t\tsuffix=$branch\n+\t\t\t\t\tdie_msg=\"$(eval_gettext \"Unable to checkout banch '\\$branch' in submodule path '\\$displaypath'\")\"\n+\t\t\t\t\tsay_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': checked out branch '\\$branch'\")\"\n+\t\t\t\t\tif test -z \"$just_cloned\" -a && test \"$subsha1\" != \"$sha1\"\n+\t\t\t\t\tthen\n+\t\t\t\t\t\t# Perform a fast-forward only merge of the origin\n+\t\t\t\t\t\tcommand_post=\"git merge $subforce --ff-only\"\n+\t\t\t\t\t\tsuffix_post=\"origin/$branch\"\n+\t\t\t\t\tfi\n+\t\t\t\telse\n+\t\t\t\t\tcommand=\"git checkout $subforce -q\"\n+\t\t\t\t\tsuffix=$sha1\n+\t\t\t\t\tdie_msg=\"$(eval_gettext \"Unable to checkout '\\$sha1' in submodule path '\\$displaypath'\")\"\n+\t\t\t\t\tsay_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': checked out '\\$sha1'\")\"\n+\t\t\t\tfi\n+\t\t\t\t;;\n \t\t\t!*)\n \t\t\t\tcommand=\"${update_module#!}\"\n+\t\t\t\tsuffix=$sha1\n \t\t\t\tdie_msg=\"$(eval_gettext \"Execution of '\\$command \\$sha1' failed in submodule path '\\$prefix\\$sm_path'\")\"\n \t\t\t\tsay_msg=\"$(eval_gettext \"Submodule path '\\$prefix\\$sm_path': '\\$command \\$sha1'\")\"\n \t\t\t\tmust_die_on_failure=yes\n+\t\t\t\tcustom_update=yes\n \t\t\t\t;;\n \t\t\t*)\n-\t\t\t\tcommand=\"git checkout $subforce -q\"\n-\t\t\t\tdie_msg=\"$(eval_gettext \"Unable to checkout '\\$sha1' in submodule path '\\$displaypath'\")\"\n-\t\t\t\tsay_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': checked out '\\$sha1'\")\"\n+\t\t\t\t# Valid user configurable update modes are already filtered above\n+\t\t\t\tdie \"$(eval_gettext \"Unexpected update mode in the current flow\")\"\n \t\t\t\t;;\n \t\t\tesac\n \n-\t\t\tif (clear_local_git_env; cd \"$sm_path\" && $command \"$sha1\")\n+\t\t\tif (clear_local_git_env; cd \"$sm_path\" && $command_attach \"$suffix_attach\" &&\n+\t\t\t\t$command_pre \"$suffix_pre\" && $command \"$suffix\" && $command_post \"$suffix_pro\")\n \t\t\tthen\n \t\t\t\tsay \"$say_msg\"\n \t\t\telif test -n \"$must_die_on_failure\"\ndiff --git a/t/t7410-submodule-attached-head.sh b/t/t7410-submodule-attached-head.sh\nnew file mode 100755\nindex 0000000..04b3018\n--- /dev/null\n+++ b/t/t7410-submodule-attached-head.sh\n@@ -0,0 +1,268 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2014 Francesco Pretto\n+#\n+\n+test_description='Support for submodules with attached head\n+\n+This test verifies the sanity of the add and update git submodule commands with\n+or without the --attached-update, --attach, --detach switches or the\n+submoudule..attach property set\n+'\n+\n+TEST_NO_CREATE_REPO=true\n+. ./test-lib.sh\n+\n+submodurl1=$(pwd -P)/repo1\n+submodurl2=$(pwd -P)/repo2\n+repourl=$(pwd -P)/repo\n+\n+test_expect_success 'setup - create repository \"repo1\" to be used as submodule' '\n+\tmkdir repo1 &&\n+\t(\n+\t\tcd repo1 &&\n+\t\tgit init &&\n+\t\tgit config receive.denyCurrentBranch ignore &&\n+\t\techo a >a &&\n+\t\tgit add a &&\n+\t\tgit commit -m \"repo1 commit 1\"\n+\t)\n+'\n+\n+test_expect_success 'setup - reate repository \"repo2\" to be used as submodule' '\n+\tmkdir repo2 &&\n+\t(\n+\t\tcd repo2 &&\n+\t\tgit init &&\n+\t\tgit config receive.denyCurrentBranch ignore &&\n+\t\techo a >a &&\n+\t\tgit add a &&\n+\t\tgit commit -m \"repo2 commit 1\"\n+\t)\n+'\n+\n+test_expect_success 'setup - create repository \"repo\" to be added with sumodules' '\n+\tmkdir repo &&\n+\t(\n+\t\tcd repo &&\n+\t\tgit init &&\n+\t\tgit config receive.denyCurrentBranch ignore &&\n+\t\techo a >a &&\n+\t\tgit add a &&\n+\t\tgit commit -m \"repo commit 1\"\n+\t)\n+'\n+\n+test_expect_success 'setup - clone repository \"repo\" in \"repoclone\"' '\n+\tgit clone \"$repourl\" repoclone\n+'\n+\n+test_expect_success 'setup - add \"mod1\" as regular submodule of \"repo\"' '\n+\t(\n+\t\tcd repo &&\n+\t\tgit submodule add \"$submodurl1\" submod1\n+\t)\n+'\n+\n+test_expect_success 'setup - add \"mod2\" as update attached HEAD submodule of \"repo\"' '\n+\t(\n+\t\tcd repo &&\n+\t\tgit submodule add --attached-update \"$submodurl2\" submod2\n+\t)\n+'\n+\n+test_expect_success 'setup - commit submodules in repo' '\n+\t(\n+\t\tcd repo &&\n+\t\tgit add . &&\n+\t\tgit commit -m \"Added submodules\"\n+\t)\n+'\n+\n+test_expect_success 'init submodules in cloned repo' '\n+\t(\n+\t\tcd repoclone &&\n+\t\tgit pull &&\n+\t\tgit submodule init\n+\t)\n+'\n+\n+test_expect_success 'update submodules in cloned repo' '\n+\t(\n+\t\tcd repoclone &&\n+\t\tgit submodule update\n+\t)\n+'\n+\n+test_expect_success 'assert submod1 HEAD is detached in cloned repo' '\n+\t(\n+\t\tcd repoclone/submod1 &&\n+\t\ttest \"$(git rev-parse --abbrev-ref HEAD)\" = \"HEAD\"\n+\t)\n+'\n+\n+test_expect_success 'assert submod2 HEAD is attached in cloned repo' '\n+\t(\n+\t\tcd repoclone/submod2 &&\n+\t\ttest \"$(git rev-parse --abbrev-ref HEAD)\" != \"HEAD\"\n+\t)\n+'\n+\n+test_expect_success 'update submodules with --attach in cloned repo' '\n+\t(\n+\t\tcd repoclone &&\n+\t\tgit submodule update --attach\n+\t)\n+'\n+\n+test_expect_success 'assert submod1 HEAD is attached in cloned repo' '\n+\t(\n+\t\tcd repoclone/submod1 &&\n+\t\ttest \"$(git rev-parse --abbrev-ref HEAD)\" != \"HEAD\"\n+\t)\n+'\n+\n+test_expect_success 'update submodules with --detach in cloned repo' '\n+\t(\n+\t\tcd repoclone &&\n+\t\tgit submodule update --detach\n+\t)\n+'\n+\n+test_expect_success 'assert submod1 HEAD is detached in cloned repo' '\n+\t(\n+\t\tcd repoclone/submod1 &&\n+\t\ttest \"$(git rev-parse --abbrev-ref HEAD)\" = \"HEAD\"\n+\t)\n+'\n+\n+test_expect_success 'assert submod2 HEAD is detached in cloned repo' '\n+\t(\n+\t\tcd repoclone/submod2 &&\n+\t\ttest \"$(git rev-parse --abbrev-ref HEAD)\" = \"HEAD\"\n+\t)\n+'\n+\n+test_expect_success 'update submodules in cloned repo (will restore HEAD states)' '\n+\t(\n+\t\tcd repoclone &&\n+\t\tgit submodule update\n+\t)\n+'\n+\n+test_expect_success 'assert submod1 HEAD is detached in cloned repo' '\n+\t(\n+\t\tcd repoclone/submod1 &&\n+\t\ttest \"$(git rev-parse --abbrev-ref HEAD)\" = \"HEAD\"\n+\t)\n+'\n+\n+test_expect_success 'assert submod2 HEAD is attached in cloned repo' '\n+\t(\n+\t\tcd repoclone/submod2 &&\n+\t\ttest \"$(git rev-parse --abbrev-ref HEAD)\" != \"HEAD\"\n+\t)\n+'\n+\n+test_expect_success 'setup - add update operation to submodules' '\n+\t(\n+\t\tcd repo &&\n+\t\tgit config -f .gitmodules submodule.submod1.update merge &&\n+\t\tgit config -f .gitmodules submodule.submod2.update rebase &&\n+\t\tgit add . &&\n+\t\tgit commit -m \"updated submodules\"\n+\t)\n+'\n+\n+test_expect_success 'setup - update cloned repo and reinitialize submodules' '\n+\t(\n+\t\tcd repoclone &&\n+\t\tgit pull &&\n+\t\tgit submodule init\n+\t)\n+'\n+\n+test_expect_success 'add some content to repo2' '\n+\t(\n+\t\tcd repo2 &&\n+\t\techo b >b &&\n+\t\tgit add b &&\n+\t\tgit commit -m \"repo2 commit 2\"\n+\t)\n+'\n+\n+test_expect_success 'update sumodules in cloned repo and verify that submod2 matches repo2' '\n+\t(\n+\t\tcd repoclone &&\n+\t\tgit submodule update &&\n+\t\ttest -e submod2/b\n+\t)\n+'\n+\n+test_expect_success 'prepend some content to repo1/a' '\n+\t(\n+\t\tcd repo1 &&\n+\t\techo -e \"b\\na\" >a &&\n+\t\tgit add a &&\n+\t\tgit commit -m \"repo1 commit 2\"\n+\t)\n+'\n+\n+test_expect_success 'append some content in repoclone/submod1 and commit' '\n+\t(\n+\t\tcd repoclone/submod1 &&\n+\t\techo c >>a &&\n+\t\tgit add a &&\n+\t\tgit commit -m \"submod1 commit 1\"\n+\t)\n+'\n+\n+test_expect_success 'update repoclone submodules with --attach' '\n+\t(\n+\t\tcd repoclone &&\n+\t\tgit submodule update --attach\n+\t)\n+'\n+\n+test_expect_success 'verify repoclone submod1 merge with reattached orphaned commits was correct' '\n+\t(\n+\t\tcd repoclone/submod1 &&\n+\t\ttest \"$(b &&\n+\t\tgit add b &&\n+\t\tgit commit -m \"repo1 commit 3\"\n+\t)\n+'\n+\n+test_expect_success 'update submodule submod2 (merge ff-only) and verify it matches repo2' '\n+\t(\n+\t\tcd repoclone &&\n+\t\tgit submodule update &&\n+\t\ttest -e submod2/b\n+\t)\n+'\n+\n+test_done\n-- \n1.8.5.2.230.g032cd47.dirty\n"},{"id":"232671","messageId":"CALas-ihrHM-vsqDmJD5VssQKhW-9+3Y5BDNr6pRe6ako=WD0og@mail.gmail.com","threadId":"35608","inReplyTo":"1388890249-3577-2-git-send-email-ceztko@gmail.com","subject":"Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"Francesco Pretto","fromEmail":"ceztko@gmail.com","sentAt":"2014-01-05T19:55:34Z","receivedAt":"2014-01-05T19:55:34Z","isPatch":true,"sender":{"key":"ceztko@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3037449?v=4"},"body":"(Hmmpth, forgot signoff...)\n\nTo whom it may interest, added some CC.\n\n2014/1/5 Francesco Pretto :\n> At the current state, the following use-case is not supported very\n> well in git:\n> - a maintainer adds a submodule, checking out a specific branch of\n> the repository. He doesn't track the upstream submodule revision sha1;\n> - a developer checkout the repository branch decided by the maintainer.\n> Subsequent \"merge\" or \"rebase\" update operations don't detach the HEAD.\n>\n> To ease the above use-case this patch:\n> - introduces a \"submodule..attached\" property that, when set\n> to \"true\", ensures that the \"update\" operation will result in\n> the HEAD attached to a branch;\n> - introduces \"--attach|--dettach\" switches to the submodule \"update\"\n> command: they attach/detach the HEAD, overriding\n> \"submodule..attached\" property value;\n> - introduces \"--attached-update\" switch to the \"add\" operation. It:\n> * sets \"submodule..attached\" to true;\n> * sets \"submodule..ignore\" to all.\n>\n> Using the '--attach' switch or operating in a repository with\n> 'submodule..attached' set to 'true' during \"update\" will:\n> - checkout a branch with an attached HEAD if the repository was just\n> cloned;\n> - perform a fast-forward only merge of changes if it's a 'checkout'\n> update operation;\n> - reattach the HEAD prior performing a 'merge', 'rebase' or '!command'\n> update operation if the HEAD was found detached. Orphaned commits\n> will also be merged back in the branch.\n>\n> '--attach' or 'submodule..attached' set to true also implies '--remote'.\n>\n> Using the '--detach' switch or operating in a repository with\n> 'submodule..attached' set to 'false' during \"update\" will:\n> - checkout a detached HEAD if the repository was just cloned;\n> - detach the HEAD prior performing a 'merge', 'rebase' or '!command'\n> update operation if the HEAD was found attached.\n>\n> 'submodule..attached' works similarly to 'submodule..update'\n> property: git copies the values found in \".gitmodules\" in \".git/config\" when\n> performing an \"init\" command. \"update\" looks for values in \".git/config\"\n> only.\n>\n> '--attach' and '--detach' switches override an opposite behaviour\n> of 'submodule..attached' properties.\n>\n> The patch is strongly additive and doesn't break any submodule specific\n> test. It also adds some tests specific to the added feature.\n> ---\n> Documentation/git-submodule.txt | 48 +++++--\n> Documentation/gitmodules.txt | 10 +-\n> git-submodule.sh | 154 +++++++++++++++++++--\n> t/t7410-submodule-attached-head.sh | 268 +++++++++++++++++++++++++++++++++++++\n> 4 files changed, 457 insertions(+), 23 deletions(-)\n> create mode 100755 t/t7410-submodule-attached-head.sh\n>\n> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt\n> index bfef8a0..b97eefb 100644\n> --- a/Documentation/git-submodule.txt\n> +++ b/Documentation/git-submodule.txt\n> @@ -10,13 +10,14 @@ SYNOPSIS\n> --------\n> [verse]\n> 'git submodule' [--quiet] add [-b ] [-f|--force] [--name ]\n> - [--reference ] [--depth ] [--] []\n> + [--reference ] [--attached-update] [--depth ]\n> + [--] []\n> 'git submodule' [--quiet] status [--cached] [--recursive] [--] [...]\n> 'git submodule' [--quiet] init [--] [...]\n> 'git submodule' [--quiet] deinit [-f|--force] [--] ...\n> 'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch]\n> - [-f|--force] [--rebase] [--reference ] [--depth ]\n> - [--merge] [--recursive] [--] [...]\n> + [-f|--force] [--rebase] [--reference ] [--attach | --detach]\n> + [--depth ] [--merge] [--recursive] [--] [...]\n> 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) ]\n> [commit] [--] [...]\n> 'git submodule' [--quiet] foreach [--recursive] \n> @@ -107,6 +108,9 @@ is the superproject and submodule repositories will be kept\n> together in the same relative location, and only the\n> superproject's URL needs to be provided: git-submodule will correctly\n> locate the submodule using the relative URL in .gitmodules.\n> ++\n> +If `--attached-update` is specified, the property `submodule..attached`\n> +will be set to `true` and `submodule..ignore` will be set to `all`.\n>\n> status::\n> Show the status of the submodules. This will print the SHA-1 of the\n> @@ -156,12 +160,15 @@ it contains local modifications.\n> update::\n> Update the registered submodules, i.e. clone missing submodules and\n> checkout the commit specified in the index of the containing repository.\n> - This will make the submodules HEAD be detached unless `--rebase` or\n> - `--merge` is specified or the key `submodule.$name.update` is set to\n> - `rebase`, `merge` or `none`. `none` can be overridden by specifying\n> - `--checkout`. Setting the key `submodule.$name.update` to `!command`\n> - will cause `command` to be run. `command` can be any arbitrary shell\n> - command that takes a single argument, namely the sha1 to update to.\n> + This will make the submodules HEAD be detached unless `--attach` is\n> + specified or `submodule.$name.attached` is set to `true`. The last setting\n> + can always be overridden specifying `--detach`. Update mode can be\n> + selected specifying `--checkout`, `--rebase` or `--merge` switches\n> + or setting the key `submodule.$name.update` to `checkout`, `rebase`,\n> + `merge` or `none`. `none` will cause the submodule to be skipped during\n> + the update. Setting the key `submodule.$name.update` to `!command` will\n> + cause `command` to be run. `command` can be any arbitrary shell command\n> + that takes a single argument, namely the sha1 to update to.\n> +\n> If the submodule is not yet initialized, and you just want to use the\n> setting as stored in .gitmodules, you can automatically initialize the\n> @@ -270,6 +277,23 @@ OPTIONS\n> be overridden by setting the `submodule..branch` option in\n> either `.gitmodules` or `.git/config` (with `.git/config` taking\n> precedence).\n> +\n> +--attached-update::\n> + This option is only valid for the add command. Causes the add command\n> + also to set the property `submodule..attached` to `true` and\n> + the property `submodule..ignore` to `all`.\n> +\n> +--attach::\n> + This option is only valid for the update commands. Causes the result\n> + of an update operation to be an attached HEAD. In the update operation,\n> + the branch named by 'submodule..branch' is checked out as the new\n> + HEAD of the submodule repository. If 'submodule..branch' is not\n> + set, the 'master' branch is checked out as the new HEAD of the\n> + submodule. Note: `--attach` also implies `--remote`.\n> +\n> +--detach::\n> + This option is only valid for the update command. Forces the result\n> + of the update operation to be a detached HEAD in the submodule.\n> +\n> This works for any of the supported update procedures (`--checkout`,\n> `--rebase`, etc.). The only change is the source of the target SHA-1.\n> @@ -290,8 +314,7 @@ SHA-1. If you don't want to fetch, you should use `submodule update\n> --merge::\n> This option is only valid for the update command.\n> Merge the commit recorded in the superproject into the current branch\n> - of the submodule. If this option is given, the submodule's HEAD will\n> - not be detached. If a merge failure prevents this process, you will\n> + of the submodule. If a merge failure prevents this process, you will\n> have to resolve the resulting conflicts within the submodule with the\n> usual conflict resolution tools.\n> If the key `submodule.$name.update` is set to `merge`, this option is\n> @@ -300,8 +323,7 @@ SHA-1. If you don't want to fetch, you should use `submodule update\n> --rebase::\n> This option is only valid for the update command.\n> Rebase the current branch onto the commit recorded in the\n> - superproject. If this option is given, the submodule's HEAD will not\n> - be detached. If a merge failure prevents this process, you will have\n> + superproject. If a merge failure prevents this process, you will have\n> to resolve these failures with linkgit:git-rebase[1].\n> If the key `submodule.$name.update` is set to `rebase`, this option is\n> implicit.\n> diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt\n> index f7be93f..9c436db 100644\n> --- a/Documentation/gitmodules.txt\n> +++ b/Documentation/gitmodules.txt\n> @@ -38,7 +38,8 @@ submodule..url::\n> submodule..update::\n> Defines what to do when the submodule is updated by the superproject.\n> If 'checkout' (the default), the new commit specified in the\n> - superproject will be checked out in the submodule on a detached HEAD.\n> + superproject (or branch, with '--attach') will be checked out in\n> + the submodule.\n> If 'rebase', the current branch of the submodule will be rebased onto\n> the commit specified in the superproject. If 'merge', the commit\n> specified in the superproject will be merged into the current branch\n> @@ -54,6 +55,13 @@ submodule..branch::\n> If the option is not specified, it defaults to 'master'. See the\n> `--remote` documentation in linkgit:git-submodule[1] for details.\n>\n> +submodule..attached::\n> + Determine if the update operation will produce a detached HEAD or not.\n> + Valid values are `true` or `false`. If the property is set to `true`\n> + and `submodule..branch` is not set, the branch `master` will\n> + be checked out. If `submodule..branch` is set the branch\n> + specified will be checked out instead.\n> +\n> submodule..fetchRecurseSubmodules::\n> This option can be used to control recursive fetching of this\n> submodule. If this option is also present in the submodules entry in\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index 1d041a7..bc6df2b 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -5,11 +5,11 @@\n> # Copyright (c) 2007 Lars Hjemli\n>\n> dashless=$(basename \"$0\" | sed -e 's/-/ /')\n> -USAGE=\"[--quiet] add [-b ] [-f|--force] [--name ] [--reference ] [--] []\n> +USAGE=\"[--quiet] add [-b ] [-f|--force] [--name ] [--reference ] [--attached-update] [--] []\n> or: $dashless [--quiet] status [--cached] [--recursive] [--] [...]\n> or: $dashless [--quiet] init [--] [...]\n> or: $dashless [--quiet] deinit [-f|--force] [--] ...\n> - or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...]\n> + or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--attach | --detach] [--merge] [--recursive] [--] [...]\n> or: $dashless [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...]\n> or: $dashless [--quiet] foreach [--recursive] \n> or: $dashless [--quiet] sync [--recursive] [--] [...]\"\n> @@ -36,6 +36,9 @@ update=\n> prefix=\n> custom_name=\n> depth=\n> +attach=\n> +detach=\n> +attached_update=\n>\n> # The function takes at most 2 arguments. The first argument is the\n> # URL that navigates to the submodule origin repo. When relative, this URL\n> @@ -352,6 +355,9 @@ cmd_add()\n> custom_name=$2\n> shift\n> ;;\n> + --attached-update)\n> + attached_update=yes\n> + ;;\n> --depth)\n> case \"$2\" in '') usage ;; esac\n> depth=\"--depth=$2\"\n> @@ -491,6 +497,12 @@ Use -f if you really want to add it.\" >&2\n> then\n> git config -f .gitmodules submodule.\"$sm_name\".branch \"$branch\"\n> fi &&\n> + if test -n \"$attached_update\"\n> + then\n> + # We'll stay stick to the HEAD, no need to track revision sha1\n> + git config -f .gitmodules submodule.\"$sm_name\".attached \"true\"\n> + git config -f .gitmodules submodule.\"$sm_name\".ignore \"all\"\n> + fi &&\n> git add --force .gitmodules ||\n> die \"$(eval_gettext \"Failed to register submodule '\\$sm_path'\")\"\n> }\n> @@ -632,6 +644,22 @@ cmd_init()\n> git config submodule.\"$name\".update \"$upd\" ||\n> die \"$(eval_gettext \"Failed to register update mode for submodule path '\\$displaypath'\")\"\n> fi\n> +\n> + # Copy \"attached\" setting when it is not set yet\n> + if attached=\"$(git config -f .gitmodules submodule.\"$name\".attached)\" &&\n> + test -n \"$attached\" &&\n> + test -z \"$(git config submodule.\"$name\".attached)\"\n> + then\n> + case \"$attached\" in\n> + true | false)\n> + ;; # Valid attach flag values\n> + *)\n> + echo >&2 \"warning: invalid attach flag value for submodule '$name'\"\n> + ;;\n> + esac\n> + git config submodule.\"$name\".attached \"$attached\" ||\n> + die \"$(eval_gettext \"Failed to register attach option for submodule path '\\$displaypath'\")\"\n> + fi\n> done\n> }\n>\n> @@ -750,6 +778,14 @@ cmd_update()\n> --reference=*)\n> reference=\"$1\"\n> ;;\n> + --attach)\n> + if test -n \"$detach\" ; then usage ; fi\n> + attach=1\n> + ;;\n> + --detach)\n> + if test -n \"$attach\" ; then usage ; fi\n> + detach=1\n> + ;;\n> -m|--merge)\n> update=\"merge\"\n> ;;\n> @@ -800,6 +836,28 @@ cmd_update()\n> name=$(module_name \"$sm_path\") || exit\n> url=$(git config submodule.\"$name\".url)\n> branch=$(get_submodule_config \"$name\" branch master)\n> + attach_module=\n> + detach_module=\n> + if test -n \"$attach\" -o -n \"$detach\"\n> + then\n> + attach_module=$attach\n> + detach_module=$detach\n> + else\n> + attached=$(git config submodule.\"$name\".attached)\n> + case \"$attached\" in\n> + '')\n> + ;; # Unset attach flag\n> + true)\n> + attach_module=1\n> + ;;\n> + false)\n> + detach_module=1\n> + ;;\n> + *)\n> + echo >&2 \"warning: invalid attach flag value for submodule '$name'\"\n> + ;;\n> + esac\n> + fi\n> if ! test -z \"$update\"\n> then\n> update_module=$update\n> @@ -848,7 +906,16 @@ Maybe you want to use 'update --init'?\")\"\n> die \"$(eval_gettext \"Unable to find current revision in submodule path '\\$displaypath'\")\"\n> fi\n>\n> - if test -n \"$remote\"\n> + head_rev_ref=$(clear_local_git_env; cd \"$sm_path\" && git rev-parse --abbrev-ref HEAD) ||\n> + die \"$(eval_gettext \"Unable to determine revision ref in submodule path '\\$sm_path'\")\"\n> + head_detached=\n> + if test \"$head_rev_ref\" = \"HEAD\"\n> + then\n> + # Determine if the HEAD is detached\n> + head_detached=\"true\"\n> + fi\n> +\n> + if test -n \"$remote\" -o -n \"$attach_module\"\n> then\n> if test -z \"$nofetch\"\n> then\n> @@ -862,7 +929,8 @@ Maybe you want to use 'update --init'?\")\"\n> die \"$(eval_gettext \"Unable to find current ${remote_name}/${branch} revision in submodule path '\\$sm_path'\")\"\n> fi\n>\n> - if test \"$subsha1\" != \"$sha1\" -o -n \"$force\"\n> + if test \"$subsha1\" != \"$sha1\" || test -n \"$attach_module\" -a -n \"$head_detached\" ||\n> + test -n \"$detach_module\" -a -z \"$head_detached\" || test -n \"$force\"\n> then\n> subforce=$force\n> # If we don't already have a -f flag and the submodule has never been checked out\n> @@ -882,40 +950,108 @@ Maybe you want to use 'update --init'?\")\"\n> fi\n>\n> # Is this something we just cloned?\n> + just_cloned=\n> case \";$cloned_modules;\" in\n> *\";$name;\"*)\n> # then there is no local change to integrate\n> - update_module= ;;\n> + update_module=\"checkout\"\n> + just_cloned=yes\n> + ;;\n> esac\n>\n> + if test -z \"$update_module\"\n> + then\n> + # Fallback to checkout\n> + update_module=\"checkout\"\n> + fi\n> +\n> + command_attach=:\n> + suffix_attach=\n> + if test \"$update_module\" != \"checkout\"\n> + then\n> + if test -n \"$attach_module\" -a -n \"$head_detached\"\n> + then\n> + # We need to reattach to the branch\n> + command_attach=\"git checkout $subforce -q\"\n> + suffix_attach=$branch\n> + elif test -n \"$detach_module\" -a -z \"$head_detached\"\n> + then\n> + # We need to detach from the branch\n> + command_attach=\"git checkout $subforce -q\"\n> + suffix_attach=$sha1\n> + fi\n> + fi\n> +\n> + command_pre=:\n> + suffix_pre=\n> + command_post=:\n> + suffix_pre=\n> + suffix=\n> must_die_on_failure=\n> + custom_update=\n> case \"$update_module\" in\n> rebase)\n> command=\"git rebase\"\n> + suffix=$sha1\n> die_msg=\"$(eval_gettext \"Unable to rebase '\\$sha1' in submodule path '\\$displaypath'\")\"\n> say_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': rebased into '\\$sha1'\")\"\n> must_die_on_failure=yes\n> + if test -n \"$attach_module\" -a -n \"$head_detached\" && test \"$subsha1\" != \"$sha1\"\n> + then\n> + # After the rebase, we merge orphaned commits in the branch\n> + command_post=\"git merge\"\n> + suffix_post=$subsha1\n> + fi\n> ;;\n> merge)\n> + if test -n \"$attach_module\" -a -n \"$head_detached\" && test \"$subsha1\" != \"$sha1\"\n> + then\n> + # Prior the rebase, we merge orphaned commits in in the branch\n> + command_pre=\"git merge\"\n> + suffix_pre=$subsha1\n> + fi\n> command=\"git merge\"\n> + suffix=$sha1\n> die_msg=\"$(eval_gettext \"Unable to merge '\\$sha1' in submodule path '\\$displaypath'\")\"\n> say_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': merged in '\\$sha1'\")\"\n> must_die_on_failure=yes\n> ;;\n> + checkout)\n> + if test -n \"$attach_module\"\n> + then\n> + command=\"git checkout $subforce -q\"\n> + suffix=$branch\n> + die_msg=\"$(eval_gettext \"Unable to checkout banch '\\$branch' in submodule path '\\$displaypath'\")\"\n> + say_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': checked out branch '\\$branch'\")\"\n> + if test -z \"$just_cloned\" -a && test \"$subsha1\" != \"$sha1\"\n> + then\n> + # Perform a fast-forward only merge of the origin\n> + command_post=\"git merge $subforce --ff-only\"\n> + suffix_post=\"origin/$branch\"\n> + fi\n> + else\n> + command=\"git checkout $subforce -q\"\n> + suffix=$sha1\n> + die_msg=\"$(eval_gettext \"Unable to checkout '\\$sha1' in submodule path '\\$displaypath'\")\"\n> + say_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': checked out '\\$sha1'\")\"\n> + fi\n> + ;;\n> !*)\n> command=\"${update_module#!}\"\n> + suffix=$sha1\n> die_msg=\"$(eval_gettext \"Execution of '\\$command \\$sha1' failed in submodule path '\\$prefix\\$sm_path'\")\"\n> say_msg=\"$(eval_gettext \"Submodule path '\\$prefix\\$sm_path': '\\$command \\$sha1'\")\"\n> must_die_on_failure=yes\n> + custom_update=yes\n> ;;\n> *)\n> - command=\"git checkout $subforce -q\"\n> - die_msg=\"$(eval_gettext \"Unable to checkout '\\$sha1' in submodule path '\\$displaypath'\")\"\n> - say_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': checked out '\\$sha1'\")\"\n> + # Valid user configurable update modes are already filtered above\n> + die \"$(eval_gettext \"Unexpected update mode in the current flow\")\"\n> ;;\n> esac\n>\n> - if (clear_local_git_env; cd \"$sm_path\" && $command \"$sha1\")\n> + if (clear_local_git_env; cd \"$sm_path\" && $command_attach \"$suffix_attach\" &&\n> + $command_pre \"$suffix_pre\" && $command \"$suffix\" && $command_post \"$suffix_pro\")\n> then\n> say \"$say_msg\"\n> elif test -n \"$must_die_on_failure\"\n> diff --git a/t/t7410-submodule-attached-head.sh b/t/t7410-submodule-attached-head.sh\n> new file mode 100755\n> index 0000000..04b3018\n> --- /dev/null\n> +++ b/t/t7410-submodule-attached-head.sh\n> @@ -0,0 +1,268 @@\n> +#!/bin/sh\n> +#\n> +# Copyright (c) 2014 Francesco Pretto\n> +#\n> +\n> +test_description='Support for submodules with attached head\n> +\n> +This test verifies the sanity of the add and update git submodule commands with\n> +or without the --attached-update, --attach, --detach switches or the\n> +submoudule..attach property set\n> +'\n> +\n> +TEST_NO_CREATE_REPO=true\n> +. ./test-lib.sh\n> +\n> +submodurl1=$(pwd -P)/repo1\n> +submodurl2=$(pwd -P)/repo2\n> +repourl=$(pwd -P)/repo\n> +\n> +test_expect_success 'setup - create repository \"repo1\" to be used as submodule' '\n> + mkdir repo1 &&\n> + (\n> + cd repo1 &&\n> + git init &&\n> + git config receive.denyCurrentBranch ignore &&\n> + echo a >a &&\n> + git add a &&\n> + git commit -m \"repo1 commit 1\"\n> + )\n> +'\n> +\n> +test_expect_success 'setup - reate repository \"repo2\" to be used as submodule' '\n> + mkdir repo2 &&\n> + (\n> + cd repo2 &&\n> + git init &&\n> + git config receive.denyCurrentBranch ignore &&\n> + echo a >a &&\n> + git add a &&\n> + git commit -m \"repo2 commit 1\"\n> + )\n> +'\n> +\n> +test_expect_success 'setup - create repository \"repo\" to be added with sumodules' '\n> + mkdir repo &&\n> + (\n> + cd repo &&\n> + git init &&\n> + git config receive.denyCurrentBranch ignore &&\n> + echo a >a &&\n> + git add a &&\n> + git commit -m \"repo commit 1\"\n> + )\n> +'\n> +\n> +test_expect_success 'setup - clone repository \"repo\" in \"repoclone\"' '\n> + git clone \"$repourl\" repoclone\n> +'\n> +\n> +test_expect_success 'setup - add \"mod1\" as regular submodule of \"repo\"' '\n> + (\n> + cd repo &&\n> + git submodule add \"$submodurl1\" submod1\n> + )\n> +'\n> +\n> +test_expect_success 'setup - add \"mod2\" as update attached HEAD submodule of \"repo\"' '\n> + (\n> + cd repo &&\n> + git submodule add --attached-update \"$submodurl2\" submod2\n> + )\n> +'\n> +\n> +test_expect_success 'setup - commit submodules in repo' '\n> + (\n> + cd repo &&\n> + git add . &&\n> + git commit -m \"Added submodules\"\n> + )\n> +'\n> +\n> +test_expect_success 'init submodules in cloned repo' '\n> + (\n> + cd repoclone &&\n> + git pull &&\n> + git submodule init\n> + )\n> +'\n> +\n> +test_expect_success 'update submodules in cloned repo' '\n> + (\n> + cd repoclone &&\n> + git submodule update\n> + )\n> +'\n> +\n> +test_expect_success 'assert submod1 HEAD is detached in cloned repo' '\n> + (\n> + cd repoclone/submod1 &&\n> + test \"$(git rev-parse --abbrev-ref HEAD)\" = \"HEAD\"\n> + )\n> +'\n> +\n> +test_expect_success 'assert submod2 HEAD is attached in cloned repo' '\n> + (\n> + cd repoclone/submod2 &&\n> + test \"$(git rev-parse --abbrev-ref HEAD)\" != \"HEAD\"\n> + )\n> +'\n> +\n> +test_expect_success 'update submodules with --attach in cloned repo' '\n> + (\n> + cd repoclone &&\n> + git submodule update --attach\n> + )\n> +'\n> +\n> +test_expect_success 'assert submod1 HEAD is attached in cloned repo' '\n> + (\n> + cd repoclone/submod1 &&\n> + test \"$(git rev-parse --abbrev-ref HEAD)\" != \"HEAD\"\n> + )\n> +'\n> +\n> +test_expect_success 'update submodules with --detach in cloned repo' '\n> + (\n> + cd repoclone &&\n> + git submodule update --detach\n> + )\n> +'\n> +\n> +test_expect_success 'assert submod1 HEAD is detached in cloned repo' '\n> + (\n> + cd repoclone/submod1 &&\n> + test \"$(git rev-parse --abbrev-ref HEAD)\" = \"HEAD\"\n> + )\n> +'\n> +\n> +test_expect_success 'assert submod2 HEAD is detached in cloned repo' '\n> + (\n> + cd repoclone/submod2 &&\n> + test \"$(git rev-parse --abbrev-ref HEAD)\" = \"HEAD\"\n> + )\n> +'\n> +\n> +test_expect_success 'update submodules in cloned repo (will restore HEAD states)' '\n> + (\n> + cd repoclone &&\n> + git submodule update\n> + )\n> +'\n> +\n> +test_expect_success 'assert submod1 HEAD is detached in cloned repo' '\n> + (\n> + cd repoclone/submod1 &&\n> + test \"$(git rev-parse --abbrev-ref HEAD)\" = \"HEAD\"\n> + )\n> +'\n> +\n> +test_expect_success 'assert submod2 HEAD is attached in cloned repo' '\n> + (\n> + cd repoclone/submod2 &&\n> + test \"$(git rev-parse --abbrev-ref HEAD)\" != \"HEAD\"\n> + )\n> +'\n> +\n> +test_expect_success 'setup - add update operation to submodules' '\n> + (\n> + cd repo &&\n> + git config -f .gitmodules submodule.submod1.update merge &&\n> + git config -f .gitmodules submodule.submod2.update rebase &&\n> + git add . &&\n> + git commit -m \"updated submodules\"\n> + )\n> +'\n> +\n> +test_expect_success 'setup - update cloned repo and reinitialize submodules' '\n> + (\n> + cd repoclone &&\n> + git pull &&\n> + git submodule init\n> + )\n> +'\n> +\n> +test_expect_success 'add some content to repo2' '\n> + (\n> + cd repo2 &&\n> + echo b >b &&\n> + git add b &&\n> + git commit -m \"repo2 commit 2\"\n> + )\n> +'\n> +\n> +test_expect_success 'update sumodules in cloned repo and verify that submod2 matches repo2' '\n> + (\n> + cd repoclone &&\n> + git submodule update &&\n> + test -e submod2/b\n> + )\n> +'\n> +\n> +test_expect_success 'prepend some content to repo1/a' '\n> + (\n> + cd repo1 &&\n> + echo -e \"b\\na\" >a &&\n> + git add a &&\n> + git commit -m \"repo1 commit 2\"\n> + )\n> +'\n> +\n> +test_expect_success 'append some content in repoclone/submod1 and commit' '\n> + (\n> + cd repoclone/submod1 &&\n> + echo c >>a &&\n> + git add a &&\n> + git commit -m \"submod1 commit 1\"\n> + )\n> +'\n> +\n> +test_expect_success 'update repoclone submodules with --attach' '\n> + (\n> + cd repoclone &&\n> + git submodule update --attach\n> + )\n> +'\n> +\n> +test_expect_success 'verify repoclone submod1 merge with reattached orphaned commits was correct' '\n> + (\n> + cd repoclone/submod1 &&\n> + test \"$( + )\n> +'\n> +\n> +test_expect_success 'setup - set operation checkout to submodule sumod1 in repo' '\n> + (\n> + cd repo &&\n> + git config -f .gitmodules submodule.submod1.update checkout &&\n> + git add . &&\n> + git commit -m \"updated submodules\"\n> + )\n> +'\n> +\n> +test_expect_success 'setup - update cloned repo and reinitialize submodules' '\n> + (\n> + cd repoclone &&\n> + git pull &&\n> + git submodule init\n> + )\n> +'\n> +\n> +test_expect_success 'add some content to repo1' '\n> + (\n> + cd repo1 &&\n> + echo b >b &&\n> + git add b &&\n> + git commit -m \"repo1 commit 3\"\n> + )\n> +'\n> +\n> +test_expect_success 'update submodule submod2 (merge ff-only) and verify it matches repo2' '\n> + (\n> + cd repoclone &&\n> + git submodule update &&\n> + test -e submod2/b\n> + )\n> +'\n> +\n> +test_done\n> --\n> 1.8.5.2.230.g032cd47.dirty\n>\n"},{"id":"232672","messageId":"20140105202009.GA3737@book.hvoigt.net","threadId":"35608","inReplyTo":"1388890249-3577-1-git-send-email-ceztko@gmail.com","subject":"Re: [PATCH 1/2] git-submodule.sh: Support 'checkout' as a valid update command","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2014-01-05T20:20:10Z","receivedAt":"2014-01-05T20:20:10Z","isPatch":true,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"On Sun, Jan 05, 2014 at 03:50:48AM +0100, Francesco Pretto wrote:\n> According to \"Documentation/gitmodules.txt\", 'checkout' is a valid\n> 'submodule..update' command. Also \"git-submodule.sh\" refers to\n> it and processes it correctly. Reflect commit 'ac1fbb' to support this\n> syntax and also validates property values during 'update' command,\n> issuing a warning if the value found is unknwon.\n\ns/unknwon/unknown/\n\n> ---\n> git-submodule.sh | 14 +++++++++++++-\n> 1 file changed, 13 insertions(+), 1 deletion(-)\n> \n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index 2677f2e..1d041a7 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -622,7 +622,7 @@ cmd_init()\n> \t\t test -z \"$(git config submodule.\"$name\".update)\"\n> \t\tthen\n> \t\t\tcase \"$upd\" in\n> -\t\t\trebase | merge | none)\n> +\t\t\tcheckout | rebase | merge | none)\n> \t\t\t\t;; # known modes of updating\n> \t\t\t*)\n> \t\t\t\techo >&2 \"warning: unknown update mode '$upd' suggested for submodule '$name'\"\n> @@ -805,6 +805,18 @@ cmd_update()\n> \t\t\tupdate_module=$update\n> \t\telse\n> \t\t\tupdate_module=$(git config submodule.\"$name\".update)\n> +\t\t\tcase \"$update_module\" in\n> +\t\t\t'')\n> +\t\t\t\t;; # Unset update mode\n> +\t\t\tcheckout | rebase | merge | none)\n> +\t\t\t\t;; # Known update modes\n> +\t\t\t!*)\n> +\t\t\t\t;; # Custom update command\n> +\t\t\t*)\n> +\t\t\t\tupdate_module=\n> +\t\t\t\techo >&2 \"warning: invalid update mode for submodule '$name'\"\n\nHow about additionally telling the user the current value that is wrong\nlike this:\n\n\techo >&2 \"warning: invalid update mode '$update_module' for submodule '$name'\"\n\n?\n\nBut apart from those minor nits the patch looks good to me.\n\nCheers Heiko\n"},{"id":"232673","messageId":"20140105203349.GB3737@book.hvoigt.net","threadId":"35608","inReplyTo":"1388890249-3577-2-git-send-email-ceztko@gmail.com","subject":"Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2014-01-05T20:33:49Z","receivedAt":"2014-01-05T20:33:49Z","isPatch":true,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"On Sun, Jan 05, 2014 at 03:50:49AM +0100, Francesco Pretto wrote:\n> At the current state, the following use-case is not supported very\n> well in git:\n> - a maintainer adds a submodule, checking out a specific branch of\n> the repository. He doesn't track the upstream submodule revision sha1;\n> - a developer checkout the repository branch decided by the maintainer.\n> Subsequent \"merge\" or \"rebase\" update operations don't detach the HEAD.\n\nCould you please extend the description of your use-case so we can\nunderstand your goal better?\n\nThe following questions directly pop into my mind:\n\n - What means the maintainer does not track the submodules sha1? Does\n that mean the superproject always refers to submodule commits using\n branches?\n - What happens if you want to go back to an earlier revision? Lets say\n a tagged release? How is ensured that you get the correct revision in\n the submodules?\n - In which situations does the developer or maintainer switch between\n your attached/detached mode?\n - What is the \"repository branch\" which is given to the developer by\n the maintainer used for? Who creates this branch and who merges into\n it?\n - What are these subsequent \"merge\" or \"rebase\" update operations? Do\n you mean everyone has submodule.name.update configured to merge or\n rebase?\n\nStill puzzled.\n\nCheers Heiko\n"},{"id":"232674","messageId":"20140105204423.GF3156@odin.tremily.us","threadId":"35608","inReplyTo":"1388890249-3577-1-git-send-email-ceztko@gmail.com","subject":"Re: [PATCH 1/2] git-submodule.sh: Support 'checkout' as a valid update command","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-05T20:44:23Z","receivedAt":"2014-01-05T20:44:23Z","isPatch":true,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"On Sun, Jan 05, 2014 at 03:50:48AM +0100, Francesco Pretto wrote:\n> +\t\t\tcase \"$update_module\" in\n> +\t\t\t'')\n> +\t\t\t\t;; # Unset update mode\n> +\t\t\tcheckout | rebase | merge | none)\n> +\t\t\t\t;; # Known update modes\n> +\t\t\t!*)\n> +\t\t\t\t;; # Custom update command\n> +\t\t\t*)\n> +\t\t\t\tupdate_module=\n> +\t\t\t\techo >&2 \"warning: invalid update mode for submodule '$name'\"\n> +\t\t\t\t;;\n> +\t\t\tesac\n\nI'd prefer `die \"…\"` to `echo >&2 \"…\"`. It's hard to know if mapping\nthe user's preferred (unknown) update mechanism to 'checkout' is\nserious or not.\n\nThis commit also makes me think that --rebase, --merge, and --checkout\nshould be replaced with a single --update={rebase|merge|checkout|!…}\noption, but that's probably food for another commit (and a long\nfinger-breaking deprecation period).\n\nCheers,\nTrevor\n\n-- \nThis email may be signed or encrypted with GnuPG (http://www.gnupg.org).\nFor more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\n"},{"id":"232677","messageId":"CALas-ijjzyRVuc0NaAS5QS98pX2198mv4HoHDacgYFYNLXbXFw@mail.gmail.com","threadId":"35608","inReplyTo":"20140105203349.GB3737@book.hvoigt.net","subject":"Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"Francesco Pretto","fromEmail":"ceztko@gmail.com","sentAt":"2014-01-05T21:46:11Z","receivedAt":"2014-01-05T21:46:11Z","isPatch":true,"sender":{"key":"ceztko@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3037449?v=4"},"body":"2014/1/5 Heiko Voigt :\n>\n> Could you please extend the description of your use-case so we can\n> understand your goal better?\n>\n\nJust in case you missed the first patch iteration[1].\n\n> The following questions directly pop into my mind:\n>\n> - What means the maintainer does not track the submodules sha1? Does\n> that mean the superproject always refers to submodule commits using\n> branches?\n\nIt means he doesn't need to control other developers commit to be\nchecked out so he sets \"submodule..ignore\" to \"all\". In this way\nhe and the developers can work actively in their submodule copy.\n\n> - What happens if you want to go back to an earlier revision? Lets say\n> a tagged release? How is ensured that you get the correct revision in\n> the submodules?\n\n\"submodule..branch\" is one setting that is not copied in\n\".git/config\" by \"git submodule init\". \"git submodule update\" will use\nthe setting in \".gitmodules\" if not overridden voluntarily by the\ndeveloper in \".git/config\". The maintainer can change that setting in\n\".gitmodules\" and commit the change. Modifies will be propagated by\nthe next \"git pull && git submodule update\" of the developer in the\nsuperproject.\n\n> - In which situations does the developer or maintainer switch between\n> your attached/detached mode?\n\nThe developer/maintainer does so optionally and voluntarily and it\neffects only its private working tree.\n\n> - What is the \"repository branch\" which is given to the developer by\n> the maintainer used for? Who creates this branch and who merges into\n> it?\n\nThe branch of course must exist prior submodule adding. In this\nuse-case it does not really matter who creates it and who merges into\nit. Everyone with the right to merge into it has to work in the\nsubmodule seamlessly, as it was working on separate clone of the same\nrepository used as the submodule.\n\n> - What are these subsequent \"merge\" or \"rebase\" update operations? Do\n> you mean everyone has submodule.name.update configured to merge or\n> rebase?\n>\n\nsubsequent \"merge\" or \"rebase\" update operations are just the ones\nafter the initial clone/checkout, nothing particular.\n\nGreetings,\nFrancesco\n\n[1] http://marc.info/?l=git&m=138836829531511&w=2\n"},{"id":"232681","messageId":"CALas-ijydCqhx5mgmMkcBE73TqNVckRooZ5x22uSq1Ldm4CGDA@mail.gmail.com","threadId":"35608","inReplyTo":"20140105203349.GB3737@book.hvoigt.net","subject":"Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"Francesco Pretto","fromEmail":"ceztko@gmail.com","sentAt":"2014-01-05T23:22:23Z","receivedAt":"2014-01-05T23:22:23Z","isPatch":true,"sender":{"key":"ceztko@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3037449?v=4"},"body":"2014/1/5 Heiko Voigt :\n> Could you please extend the description of your use-case so we can\n> understand your goal better?\n>\n\nMaybe I found better words to explain you my goal: the current git\nsubmodule use-case threats the submodule as a project independent\ndependency. My use case threats the submodule as part of the\nsuperproject repository. It could be easier to say that in this way\nsubmodules would behave very similarly to \"svn:externals\", something\nthat is actually missing in git. My goal is obtain this without\naltering git behavior for the existing use case.\n\n> - In which situations does the developer or maintainer switch between\n> your attached/detached mode?\n\nAs I told you in the other answer this is voluntary done by the\ndeveloper, as he prefers. I came to the conclusion that the\n\"--attach|--detach\" switches for the \"update\" command are not that\nuseful and can be removed. It's still possible to obtain the switch\nbetween detached/attached very easily in this way:\n\n# Attach submodule\n$ git config submodule..attached \"true\"\n$ git submodule update\n\n# Detach submodule\n$ git config submodule..attached \"false\"\n$ git submodule update\n\n# Unset property in both \".gitmodules\" and \".git/config\" means -> do nothing\n$ git config --unset submodule..attached\n$ git submodule update\n\nAlso my \"submodule..attached\" property at the moment behaves\nlike \"submodule..update\": it is copied in \".git/config\" by \"git\nsubmodule init\". This is probably a mistake: the overridden value\nshould be stored in \".git/config\" only at the developer will, so the\nmaintainer has still a chance to modify it in \".gitmodules\" and\npropagate the behavior.\n\nI would send an updated patch but at this point I prefer to wait for a\nfull review.\n\nThank you,\nFrancesco\n"},{"id":"232712","messageId":"20140106140627.GA27265@t2784.greatnet.de","threadId":"35608","inReplyTo":"CALas-ijjzyRVuc0NaAS5QS98pX2198mv4HoHDacgYFYNLXbXFw@mail.gmail.com","subject":"Re: Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2014-01-06T14:06:27Z","receivedAt":"2014-01-06T14:06:27Z","isPatch":true,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"On Sun, Jan 05, 2014 at 10:46:11PM +0100, Francesco Pretto wrote:\n> 2014/1/5 Heiko Voigt :\n> > The following questions directly pop into my mind:\n> >\n> > - What means the maintainer does not track the submodules sha1? Does\n> > that mean the superproject always refers to submodule commits using\n> > branches?\n> \n> It means he doesn't need to control other developers commit to be\n> checked out so he sets \"submodule..ignore\" to \"all\". In this way\n> he and the developers can work actively in their submodule copy.\n\nSo practically speaking: You mean that the value of\nsubmodule..ignore is set to \"all\" in the master branch of the\nsuperproject? From your other email referring to svn:externals I figure\nthat.\n\n> > - What happens if you want to go back to an earlier revision? Lets say\n> > a tagged release? How is ensured that you get the correct revision in\n> > the submodules?\n> \n> \"submodule..branch\" is one setting that is not copied in\n> \".git/config\" by \"git submodule init\". \"git submodule update\" will use\n> the setting in \".gitmodules\" if not overridden voluntarily by the\n> developer in \".git/config\". The maintainer can change that setting in\n> \".gitmodules\" and commit the change. Modifies will be propagated by\n> the next \"git pull && git submodule update\" of the developer in the\n> superproject.\n\nI do not understand how does that ensure you get the correct submodule\nrevision when checking out a tagged release? To get a precise revision\nthe superproject needs to track a sha1 of a submodule commit. I do not\nsee how that has anything to do with submodule..branch?\n\n> > - In which situations does the developer or maintainer switch between\n> > your attached/detached mode?\n> \n> The developer/maintainer does so optionally and voluntarily and it\n> effects only its private working tree.\n\nThis does not answer my question. I would like to find out the reason\nwhy one would do the switch.\n\n> > - What is the \"repository branch\" which is given to the developer by\n> > the maintainer used for? Who creates this branch and who merges into\n> > it?\n> \n> The branch of course must exist prior submodule adding. In this\n> use-case it does not really matter who creates it and who merges into\n> it. Everyone with the right to merge into it has to work in the\n> submodule seamlessly, as it was working on separate clone of the same\n> repository used as the submodule.\no\nHere is the same. I am searching for a description like:\n\nIf the developer works on a feature that needs a submodule change he:\n - creates a submodule branch\n - configures that submodule branch in the superproject:\n \tgit config -f .gitmodules submodule.common.branch dev/some-feature\n\tgit commit -am \"TEMP: track submodule common on branch\"\n - and pushes out his superproject branch\n\nThe submodule branch is then posted for review and continued to work on.\n\nOnce everyone involved is happy with the submodule change the branch in\nthere gets merged to master.\n\nNow the branch in the superproject is modified to drop the change in\n.gitmodules and the sha1 reference in the superproject is updated to the\ncurrent master of the superproject.\n\nThe superproject branch is posted for review.\n\n...\n\nCould you describe something like this for your workflow? A complete\nchange lifecycle when a developer works, as you call it, \"actively\" in a\nsubmodule?\n\n> > - What are these subsequent \"merge\" or \"rebase\" update operations? Do\n> > you mean everyone has submodule.name.update configured to merge or\n> > rebase?\n> >\n> \n> subsequent \"merge\" or \"rebase\" update operations are just the ones\n> after the initial clone/checkout, nothing particular.\n\nTo clarify you are talking about issuing \"git merge\" or \"git rebase\"\ncommands in the superproject?\n\nCheers Heiko\n"},{"id":"232713","messageId":"20140106141805.GB27265@t2784.greatnet.de","threadId":"35608","inReplyTo":"CALas-ijydCqhx5mgmMkcBE73TqNVckRooZ5x22uSq1Ldm4CGDA@mail.gmail.com","subject":"Re: Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2014-01-06T14:18:05Z","receivedAt":"2014-01-06T14:18:05Z","isPatch":true,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"On Mon, Jan 06, 2014 at 12:22:23AM +0100, Francesco Pretto wrote:\n> 2014/1/5 Heiko Voigt :\n> > Could you please extend the description of your use-case so we can\n> > understand your goal better?\n> >\n> \n> Maybe I found better words to explain you my goal: the current git\n> submodule use-case threats the submodule as a project independent\n> dependency. My use case threats the submodule as part of the\n> superproject repository. It could be easier to say that in this way\n> submodules would behave very similarly to \"svn:externals\", something\n> that is actually missing in git. My goal is obtain this without\n> altering git behavior for the existing use case.\n\nI am not so sure. svn:externals was IMO a hack in SVN to bind projects\ntogether. It does not record the revision and so has nothing to do\nwith version control. If you simply want to always checkout the\ndevelopment tip of some project you could do something like this:\n\n\tgit submodule foreach 'git fetch && git checkout origin/master'\n\nThe demand for this 'missing feature' which we call the 'floating\nsubmodules' model has been around for some time but until now we could\nconvince people that its not a feature but you are actually loosing\nhistory information.\n\nThe workflow could always be changed to allow recording revisions. Which\nis why you use git in the first place right? If you discard revisions\nfor submodules tracking down regression bugs can become a big problem or\ncompletely impossible. Try using git bisect on such a history.\n\n> > - In which situations does the developer or maintainer switch between\n> > your attached/detached mode?\n> \n> As I told you in the other answer this is voluntary done by the\n> developer, as he prefers.\n\nCould you tell me a typical reason?\n\n\n> I came to the conclusion that the\n> \"--attach|--detach\" switches for the \"update\" command are not that\n> useful and can be removed. It's still possible to obtain the switch\n> between detached/attached very easily in this way:\n> \n> # Attach submodule\n> $ git config submodule..attached \"true\"\n> $ git submodule update\n> \n> # Detach submodule\n> $ git config submodule..attached \"false\"\n> $ git submodule update\n> \n> # Unset property in both \".gitmodules\" and \".git/config\" means -> do nothing\n> $ git config --unset submodule..attached\n> $ git submodule update\n> \n> Also my \"submodule..attached\" property at the moment behaves\n> like \"submodule..update\": it is copied in \".git/config\" by \"git\n> submodule init\". This is probably a mistake: the overridden value\n> should be stored in \".git/config\" only at the developer will, so the\n> maintainer has still a chance to modify it in \".gitmodules\" and\n> propagate the behavior.\n> \n> I would send an updated patch but at this point I prefer to wait for a\n> full review.\n\nLets first discuss and figure out what is the real missing feature here\nand what should be implemented before working further on the code.\n\nCheers Heiko\n"},{"id":"232720","messageId":"20140106155849.GS3156@odin.tremily.us","threadId":"35608","inReplyTo":"20140106141805.GB27265@t2784.greatnet.de","subject":"Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-06T15:58:49Z","receivedAt":"2014-01-06T15:58:49Z","isPatch":true,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"On Mon, Jan 06, 2014 at 03:18:05PM +0100, Heiko Voigt wrote:\n> If you simply want to always checkout the development tip of some\n> project you could do something like this:\n> \n> \tgit submodule foreach 'git fetch && git checkout origin/master'\n\nOr (respecting submodule..branch):\n\n $ git submodule update --remote\n\nYou can even:\n\n $ git submodule update --remote --recursive\n\nwhenever you get an itch to upgrade everything everything in one\nsweeping, hard-to-debug move ;).\n\nCheers,\nTrevor\n\n-- \nThis email may be signed or encrypted with GnuPG (http://www.gnupg.org).\nFor more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\n"},{"id":"232724","messageId":"xmqq7gadkroa.fsf@gitster.dls.corp.google.com","threadId":"35608","inReplyTo":"20140105204423.GF3156@odin.tremily.us","subject":"Re: [PATCH 1/2] git-submodule.sh: Support 'checkout' as a valid update command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-01-06T16:20:53Z","receivedAt":"2014-01-06T16:20:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"W. Trevor King\" writes:\n\n> On Sun, Jan 05, 2014 at 03:50:48AM +0100, Francesco Pretto wrote:\n>> +\t\t\tcase \"$update_module\" in\n>> +\t\t\t'')\n>> +\t\t\t\t;; # Unset update mode\n>> +\t\t\tcheckout | rebase | merge | none)\n>> +\t\t\t\t;; # Known update modes\n>> +\t\t\t!*)\n>> +\t\t\t\t;; # Custom update command\n>> +\t\t\t*)\n>> +\t\t\t\tupdate_module=\n>> +\t\t\t\techo >&2 \"warning: invalid update mode for submodule '$name'\"\n>> +\t\t\t\t;;\n>> +\t\t\tesac\n>\n> I'd prefer `die \"…\"` to `echo >&2 \"…\"`. It's hard to know if mapping\n> the user's preferred (unknown) update mechanism to 'checkout' is\n> serious or not.\n>\n> This commit also makes me think that --rebase, --merge, and --checkout\n> should be replaced with a single --update={rebase|merge|checkout|!…}\n> option, but that's probably food for another commit (and a long\n> finger-breaking deprecation period).\n\nAll of the above points sound sensible to me.\n"},{"id":"232737","messageId":"xmqq8uutj9c9.fsf@gitster.dls.corp.google.com","threadId":"35608","inReplyTo":"xmqq7gadkroa.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH 1/2] git-submodule.sh: Support 'checkout' as a valid update command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-01-06T17:42:14Z","receivedAt":"2014-01-06T17:42:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> \"W. Trevor King\" writes:\n>\n>> On Sun, Jan 05, 2014 at 03:50:48AM +0100, Francesco Pretto wrote:\n>>> +\t\t\tcase \"$update_module\" in\n>>> +\t\t\t'')\n>>> +\t\t\t\t;; # Unset update mode\n>>> +\t\t\tcheckout | rebase | merge | none)\n>>> +\t\t\t\t;; # Known update modes\n>>> +\t\t\t!*)\n>>> +\t\t\t\t;; # Custom update command\n>>> +\t\t\t*)\n>>> +\t\t\t\tupdate_module=\n>>> +\t\t\t\techo >&2 \"warning: invalid update mode for submodule '$name'\"\n>>> +\t\t\t\t;;\n>>> +\t\t\tesac\n>>\n>> I'd prefer `die \"…\"` to `echo >&2 \"…\"`. It's hard to know if mapping\n>> the user's preferred (unknown) update mechanism to 'checkout' is\n>> serious or not.\n>>\n>> This commit also makes me think that --rebase, --merge, and --checkout\n>> should be replaced with a single --update={rebase|merge|checkout|!…}\n>> option, but that's probably food for another commit (and a long\n>> finger-breaking deprecation period).\n>\n> All of the above points sound sensible to me.\n\nI'll tentatively queue this on 'pu' (with the suggested \"die\"\nupdate), with some rewording of the log message. The patch needs to\nbe signed-off, though.\n\nThanks.\n"},{"id":"232739","messageId":"CALas-ihHD_eJOXLUrhCVZjidQDmrCN=QpdfMKoN1i9A7FAo3RQ@mail.gmail.com","threadId":"35608","inReplyTo":"20140106140627.GA27265@t2784.greatnet.de","subject":"Re: Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"Francesco Pretto","fromEmail":"ceztko@gmail.com","sentAt":"2014-01-06T17:47:58Z","receivedAt":"2014-01-06T17:47:58Z","isPatch":true,"sender":{"key":"ceztko@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3037449?v=4"},"body":"Dear Heiko, my replies below. I also take a couple excerpts from other\nemails, as I prefer to not flame on different threads :) .\n\n2014/1/6 Heiko Voigt :\n> On Sun, Jan 05, 2014 at 10:46:11PM +0100, Francesco Pretto wrote:\n>> It means he doesn't need to control other developers commit to be\n>> checked out so he sets \"submodule..ignore\" to \"all\". In this way\n>> he and the developers can work actively in their submodule copy.\n>\n> So practically speaking: You mean that the value of\n> submodule..ignore is set to \"all\" in the master branch of the\n> superproject? From your other email referring to svn:externals I figure\n> that.\n>\n\nCorrect, but this works also if the branch of the superproject is a\ndifferent branch than \"master\". I think you are right in a point, see\nthe next reply.\n\n> The workflow could always be changed to allow recording revisions. Which\n> is why you use git in the first place right? If you discard revisions\n> for submodules tracking down regression bugs can become a big problem or\n> completely impossible. Try using git bisect on such a history.\n>\n\nOk, you are right: setting \"submodule..ignore\" to \"all\" by\ndefault with a switch \"--attached\" in \"git submodule add\" is too much.\nMy point is just make sure users will checkout an attached HEAD.\n\n>> \"submodule..branch\" is one setting that is not copied in\n>> \".git/config\" by \"git submodule init\". \"git submodule update\" will use\n>> the setting in \".gitmodules\" if not overridden voluntarily by the\n>> developer in \".git/config\". The maintainer can change that setting in\n>> \".gitmodules\" and commit the change. Modifies will be propagated by\n>> the next \"git pull && git submodule update\" of the developer in the\n>> superproject.\n>\n> I do not understand how does that ensure you get the correct submodule\n> revision when checking out a tagged release? To get a precise revision\n> the superproject needs to track a sha1 of a submodule commit. I do not\n> see how that has anything to do with submodule..branch?\n>\n\n\"submodule..attacched\" set to true implies \"--remote\". sha1 of\nthe latest commit is taken from \"origin/$branch\". In this way you get\nthe latest commit of that branch, and you do a 'merge', 'rebase',\n'checkout' or '!command' according to the configured ''. This\nmechanism is already in the patch at the current state.\n\n>> > - In which situations does the developer or maintainer switch between\n>> > your attached/detached mode?\n>>\n>> The developer/maintainer does so optionally and voluntarily and it\n>> effects only its private working tree.\n>\n> This does not answer my question. I would like to find out the reason\n> why one would do the switch.\n>\n\nThe developer does it voluntarily, at his responsibility, because he\nmay decide to partecipate more actively to the development of the\nsubmodule and still want to use a simple \"git submodule update\" to\nupdates his submodules, overriding its configuration as it can be done\nfor other properties like, for example, \"branch\". This ability is of\ncourse already possible reattaching the HEAD manually but you loose\nthe convenient ability to use \"git submodule update\".\n\n>> The branch of course must exist prior submodule adding. In this\n>> use-case it does not really matter who creates it and who merges into\n>> it. Everyone with the right to merge into it has to work in the\n>> submodule seamlessly, as it was working on separate clone of the same\n>> repository used as the submodule.\n> o\n> Here is the same. I am searching for a description like:\n>\n> If the developer works on a feature that needs a submodule change he:\n> - creates a submodule branch\n> - configures that submodule branch in the superproject:\n> git config -f .gitmodules submodule.common.branch dev/some-feature\n> git commit -am \"TEMP: track submodule common on branch\"\n> - and pushes out his superproject branch\n>\n> The submodule branch is then posted for review and continued to work on.\n>\n> Once everyone involved is happy with the submodule change the branch in\n> there gets merged to master.\n>\n> Now the branch in the superproject is modified to drop the change in\n> .gitmodules and the sha1 reference in the superproject is updated to the\n> current master of the superproject.\n>\n> The superproject branch is posted for review.\n>\n> ...\n>\n> Could you describe something like this for your workflow? A complete\n> change lifecycle when a developer works, as you call it, \"actively\" in a\n> submodule?\n>\n\nI'm really sorry, I thought this was already clear from the first\npatch iteration. I will go more in depth:\n\nSay we have our actual projects \"project1\" and \"project2\". Say we have\na project \"common\". This \"common\" project is *not* independent: it\nexists only to serve \"project1\" and \"project2\" and it's tested only in\nthe scope of \"project1\" and \"project2\". Also \"common\" is very actively\ndeveloped and follows the same lifecyle of \"project1\" and \"project2\"\non separate branches. I think that it's important that you get this\npoint: developers of \"common\" don't clone it separately, as it would\nbe impossible to test it, they clone \"project1\" and \"project2\" and\nexpect to find it inside one of these.\n\nLet say now how \"common\" can be shared between \"project1\" and\n\"project2\" so developers of \"project1\" don't break \"project2\" working\non \"common\" and the other way around. \"common\" has the following\nstable branches:\n- master;\n- master-project1;\n- master-project2;\n\nThe maintainer of \"project1\" at master branch sets \"common\" as\nsubmodule at branch \"master-project1\". The maintainer of \"project2\" at\nmaster branch sets \"common\" as submodule at branch \"master-project2\".\nPeriodically a maintainer of \"common\" will pull both master-project1\nand master-project1 on a branch \"next\". Maintainers of both \"project1\"\nand \"project1\", in coordination, are responsible to make next to be\nmerged in master, so they can both sync their master-project1 and\nmaster-project2 in the \"common\" submodule.\n\nLet say now how for example {\"project1\" and \"common\"} can evolve. As\ntold, these can't really evolve separately. Maintainer of \"project1\"\nprepares a branch called \"staging-featureA\". \"featureA\" is big and\nwill require more people to work on the same feature for days/weeks.\nMaintainer of \"project1\" also prepares a branch\n\"project1-staging-featureA\" on \"common\" and set \".gitmodules\" of\n\"project1\" to point to \"project1-staging-featureA\". Developers of\nfeatureA would like to do this:\n\n$ git pull\n$ git checkout staging-featureA\n$ git submodule update # clones an attached HEAD of common on the branch\n #\n'submodule.common.project1.staging-featureA'\n$ .... start coding in common seamlessly as they where in project1 ....\n\nAlso developers do frequently rebase:\n$ git pull --rebase\n$ git submodule update\n\nOr maybe a shortcut of this: \"git submodule update\" should be given\nthe possibility to go \"--remote\" by default. Of course if \"common\" of\nthe developer is in a branch different that 'submodule..branch'\n\"git submodule update\" has not to switch the branch.\n\n>> Maybe who coded submodules at first was thinking that the best way to\n>> contribute to a project is to checkout that repository, and not work\n>> in the submodule. As said, this works well when the submodule\n>> repository is a full project, and not a bunch of shared code.\n>\n>Why not work in the submodule? See explanation above.\n>\n\nBecause, as said above, the submodule is not independent. It does not\nhave proper code that test it and the best test case is using the\nsubmodule in the scope of the superproject.\n\n>> > - What are these subsequent \"merge\" or \"rebase\" update operations? Do\n>> > you mean everyone has submodule.name.update configured to merge or\n>> > rebase?\n>> >\n>>\n>> subsequent \"merge\" or \"rebase\" update operations are just the ones\n>> after the initial clone/checkout, nothing particular.\n>\n> To clarify you are talking about issuing \"git merge\" or \"git rebase\"\n> commands in the superproject?\n>\n\nNo, I'm talking about \"git submoudule update\" with \"--merge\" or\n\"--rebase switches or \"submodule.name.update\" configured.\n\n2014/1/6 Heiko Voigt :\n> I am not so sure. svn:externals was IMO a hack in SVN to bind projects\n> together. It does not record the revision and so has nothing to do\n> with version control. If you simply want to always checkout the\n> development tip of some project you could do something like this:\n>\n> git submodule foreach 'git fetch && git checkout origin/master'\n\nThis can be very unconvenient if the reccomended *starting* branch to\nwhere attach the HEAD is not \"master\":\n\ngit submodule foreach 'branch=\"$(git config -f $toplevel/.gitmodules\nsubmodule.$name.branch)\"; git checkout origin/$branch\n\nOf course the developer after they may want to move to a local branch\n(or may not: please don't forget shared repositories), but the above\ncommand may difficult to be taught. Also with the comit[1] that blocks\ncopying of !command to \".git/config\" and sets default \"none\", you made\nit harder to offer a mantainer decided default update behavior like\nthe one I described.\n\n> The demand for this 'missing feature' which we call the 'floating\n> submodules' model has been around for some time but until now we could\n> convince people that its not a feature but you are actually loosing\n> history information.\n>\n\nPutting your reasoning to the extreme consequences: why does not \"git\nclone\" clone a dettached HEAD? In your workflow I should probably\nnever start coding from \"master\", but people frequently do instead.\nAlso, as you observed, it's possible to track submodule revision sha1\nalso on an attached HEAD. If I don't want anyone to track the revision\nI can already set 'ignore' property.\n\nMy bottom line:\n- For what I understand, detached HEAD it's a way to say \"hey, you\nhave to stay on this commit. Also don't even think you can push to the\nupstream branch\". This sometimes can't be spurious, as in the use case\nI wrote above: access control on the remote repositories should be\nenough. I think maintainers should have the option to make developers\nto clone a repository starting with an attached HEAD on the branch\nsuggested in submodule.$name.branch;\n- \"git submodule update\" is missing a property to do automatically\n\"--remote\". I think in the use case I wrote it's really handy to have\na \"git submodule update\" to act like this.\n\nThank you for reading and sorry for the long email :)\n\nGreetings,\nFrancesco\n\n[1] http://marc.info/?l=git&m=138610752125816&w=2\n"},{"id":"232740","messageId":"CALas-iiQO8OwhS_W9u3sNDYuWf_3XnFsF11NZGLnnt_+pDTtVA@mail.gmail.com","threadId":"35608","inReplyTo":"xmqq8uutj9c9.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH 1/2] git-submodule.sh: Support 'checkout' as a valid update command","fromName":"Francesco Pretto","fromEmail":"ceztko@gmail.com","sentAt":"2014-01-06T17:52:43Z","receivedAt":"2014-01-06T17:52:43Z","isPatch":true,"sender":{"key":"ceztko@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3037449?v=4"},"body":"Ok, applying the suggested modifications and resending shortly.\n\nThank you,\nFrancesco\n\n2014/1/6 Junio C Hamano :\n> Junio C Hamano writes:\n>\n>> \"W. Trevor King\" writes:\n>>\n>>> On Sun, Jan 05, 2014 at 03:50:48AM +0100, Francesco Pretto wrote:\n>>>> + case \"$update_module\" in\n>>>> + '')\n>>>> + ;; # Unset update mode\n>>>> + checkout | rebase | merge | none)\n>>>> + ;; # Known update modes\n>>>> + !*)\n>>>> + ;; # Custom update command\n>>>> + *)\n>>>> + update_module=\n>>>> + echo >&2 \"warning: invalid update mode for submodule '$name'\"\n>>>> + ;;\n>>>> + esac\n>>>\n>>> I'd prefer `die \"…\"` to `echo >&2 \"…\"`. It's hard to know if mapping\n>>> the user's preferred (unknown) update mechanism to 'checkout' is\n>>> serious or not.\n>>>\n>>> This commit also makes me think that --rebase, --merge, and --checkout\n>>> should be replaced with a single --update={rebase|merge|checkout|!…}\n>>> option, but that's probably food for another commit (and a long\n>>> finger-breaking deprecation period).\n>>\n>> All of the above points sound sensible to me.\n>\n> I'll tentatively queue this on 'pu' (with the suggested \"die\"\n> update), with some rewording of the log message. The patch needs to\n> be signed-off, though.\n>\n> Thanks.\n"},{"id":"232755","messageId":"87ppo4zzkb.fsf@engster.org","threadId":"35608","inReplyTo":"CALas-ihHD_eJOXLUrhCVZjidQDmrCN=QpdfMKoN1i9A7FAo3RQ@mail.gmail.com","subject":"Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"David Engster","fromEmail":"deng@randomsample.de","sentAt":"2014-01-06T19:21:24Z","receivedAt":"2014-01-06T19:21:24Z","isPatch":true,"sender":{"key":"deng@randomsample.de","avatar":null},"body":"Francesco Pretto writes:\n> 2014/1/6 Heiko Voigt :\n>> Could you describe something like this for your workflow? A complete\n>> change lifecycle when a developer works, as you call it, \"actively\" in a\n>> submodule?\n>>\n>\n> I'm really sorry, I thought this was already clear from the first\n> patch iteration. I will go more in depth:\n\nWhile I have some trouble understanding all the details of Francesco's\ndescription, I find the idea of \"attaching submodules to branches\" very\nuseful. I think I could well use that to simplify my merging grunt work\nfor GNU Emacs (which, in case you're wondering, is probably switching to\ngit as its VCS). I don't mean to hijack this thread, and I guess my use\ncase is a bit different than what Francesco has in mind; still, I think\nit is similar enough that my use case could help in talking about the\ndetails of his patch; if not, please feel free to ignore it.\n\nGNU Emacs ships with some pretty large packages, namely Gnus, Org and\nCEDET, which are also available as \"stand-alone\" versions for manual\ninstallation, and their development happens in separate upstream\nrepositories. Since I'm a CEDET developer, I'll use it as an example in\nthe following.\n\nFirst off, it is important to note that merges are always\nbi-directional: not only is new CEDET code pulled into Emacs, but Emacs\ndevelopers also change things which have to be merged back upstream. So\nfar, merging between the two repositories was done manually by me, which\nis error-prone (and boring). I think that by pulling in CEDET directly\nas a submodule, this merging could be made easier. Most importantly, my\nhope is that more people than me could do it. :-)\n\nHere's how I would like this to work; first the CEDET -> Emacs part,\nwhich is rather straight-forward:\n\n- The CEDET repository has two branches: 'master' and 'stable'.\n\n- The Emacs repository imports CEDET's 'stable' branch as a submodule.\n\n- CEDET's main development happens in 'master', and the CEDET developers\n are responsible for merging stable code to 'stable'. They will then\n make a new commit for the submodule in Emacs accordingly.\n\nThe Emacs -> CEDET part is more hairy. Most of the time, the fixes\nhappening in the Emacs repository for CEDET are very small and/or\ntrivial and can usually be considered \"always stable\": fixes for\nspelling, compiler warnings, or small refactorings like renames,\netc. This kind of \"merging back to CEDET upstream\" should hence be as\neasy as possible for Emacs developers:\n\n- When an Emacs developer changes something in the CEDET submodule, the\n changes they commit should by default automatically land in CEDET's\n 'stable' branch. That means that when they enter the submodule, they\n should be in the branch 'stable' instead of being detached, and a push\n should update the 'stable' branch in CEDET accordingly. The submodule\n must then be committed as well.\n\n- It is then up to the CEDET developers to merge these changes into the\n 'master' branch of the CEDET repo.\n\nI know that the \"correct\" workflow would be to always use feature\nbranches, but it'd be nice if that could be avoided if one so chooses.\n\nA little picture in the hope that it makes things clearer:\n\n +-----------+\n | master | <--\n+-------+ +-----------+ | Merges to/from master\n| CEDET | | done only by CEDET developers\n+-------+ | \n +-----------+ |\n | stable | <-- <--------\n +-----------+ |\n |\n |\n | Any Emacs developer\n | can push and commit\n | submodule\n+--------+ +----------------------+ |\n| Emacs | -- | lisp/cedet submodule | <-\n+--------+ +----------------------+\n\nAFAICS the main problem with this approach is that one always has to\nthink of committing the new SHA1 of the submodule. If I understand\nFrancesco correctly, he wants to eliminate the need for that by simply\nalways taking the head of the attached branch. I also think that would\nbe a nice feature, since in the above drawing, the lisp/cedet submodule\nshould always follow the 'stable' branch in CEDET upstream. However, as\nHeiko notes, the history must be preserved to be able to go back to\nearlier revisions, so there must be some kind of commit for the\nsubmodule when 'stable' changes; maybe that could be automated somehow?\n\n-David\n"},{"id":"232798","messageId":"20140107041004.GA11060@odin.tremily.us","threadId":"35608","inReplyTo":"CALas-ihHD_eJOXLUrhCVZjidQDmrCN=QpdfMKoN1i9A7FAo3RQ@mail.gmail.com","subject":"Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-07T04:10:04Z","receivedAt":"2014-01-07T04:10:04Z","isPatch":true,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"On Mon, Jan 06, 2014 at 06:47:58PM +0100, Francesco Pretto wrote:\n> I'm really sorry, I thought this was already clear from the first\n> patch iteration. I will go more in depth:\n\nFor me anyway, this extra detail is very helpful. Thanks :).\n\n> Maintainer of \"project1\" also prepares a branch\n> \"project1-staging-featureA\" on \"common\" and set \".gitmodules\" of\n> \"project1\" to point to \"project1-staging-featureA\". Developers of\n> featureA would like to do this:\n> \n> $ git pull\n> $ git checkout staging-featureA\n> $ git submodule update # clones an attached HEAD of common on the branch\n> # 'submodule.common.project1.staging-featureA'\n> $ .... start coding in common seamlessly as they where in project1 ....\n\nSo the checked-out branch switches depending on the local superproject\nbranch. That sounds nice, but I'm not sure where the\nsuperproject-branch-to-local-submodule-branch mapping would be stored.\nWe currently do this for remote-tracking submodule branches with an\nin-tree .gitmodules (which can differ between submodule branches) with\nlocal overides in a single out-of-tree .git/config (which is\nindependent of the checked out branch). Ideally we'd have a way to\nadd local overrides on a per-superproject-branch basis, but I don't\nknow what that would look like.\n\n> Also developers do frequently rebase:\n> $ git pull --rebase\n> $ git submodule update\n> \n> Or maybe a shortcut of this: \"git submodule update\" should be given\n> the possibility to go \"--remote\" by default.\n\nRebasing the superproject and then updating the submodules (to the\nsuperproject's gitlinked commits) is not the same as a --remote update\n(to the subproject's upstream branch tip).\n\n> Of course if \"common\" of the developer is in a branch different that\n> 'submodule..branch' \"git submodule update\" has not to switch\n> the branch.\n\nI don't understand what you're saying here.\n\n> >> Maybe who coded submodules at first was thinking that the best\n> >> way to contribute to a project is to checkout that repository,\n> >> and not work in the submodule. As said, this works well when the\n> >> submodule repository is a full project, and not a bunch of shared\n> >> code.\n> >\n> >Why not work in the submodule? See explanation above.\n> \n> Because, as said above, the submodule is not independent. It does\n> not have proper code that test it and the best test case is using\n> the submodule in the scope of the superproject.\n\nYou can cd into the submodule, and develop it as an independent\nrepository. When you want to test your changes, just cd back into the\nsuperproject and run your test suite.\n\n> 2014/1/6 Heiko Voigt :\n> > I am not so sure. svn:externals was IMO a hack in SVN to bind projects\n> > together. It does not record the revision and so has nothing to do\n> > with version control. If you simply want to always checkout the\n> > development tip of some project you could do something like this:\n> >\n> > git submodule foreach 'git fetch && git checkout origin/master'\n> \n> This can be very unconvenient if the reccomended *starting* branch to\n> where attach the HEAD is not \"master\":\n> git submodule foreach 'branch=\"$(git config -f $toplevel/.gitmodules\n> submodule.$name.branch)\"; git checkout origin/$branch\n\nWhich is equivalent to:\n\n $ git submodule update --remote --checkout\n\nexcept for branch-vs-detached-HEAD. If you are doing local\ndevelopment, I'd recommend setting up submodule..update to a\nnon-checkout strategy and using:\n\n $ git submodule update --remote\n\nwhich will integrate the upstream changes with any local changes\n(updating whichever local submodule branch you had checked out).\n\n> Also with the comit[1] that blocks copying of !command to\n> \".git/config\" and sets default \"none\", you made it harder to offer a\n> mantainer decided default update behavior like the one I described.\n\nThe maintainer can still suggest checkout/pull/rebase, and the\ndeveloper can still clear remove the none from .git/config after\ninitializing the submodule. You only need to do this once per\nsubmodule.\n\n> I think maintainers should have the option to make developers to\n> clone a repository starting with an attached HEAD on the branch\n> suggested in submodule.$name.branch;\n\nI agree, and want to use a non-checkout submodule..update mode\nto identify developers who would want this. My v2 patch switches on\nsubmodule..branch, but I'll update it in v3 to switch on\nsubmodule..update. There's no need to confuse this with\nadditional attach/detach functionality.\n\n> - \"git submodule update\" is missing a property to do automatically\n> \"--remote\". I think in the use case I wrote it's really handy to have\n> a \"git submodule update\" to act like this.\n\nYou can already add aliases, but a remote/local-gitlink config\nvariable would be nice too.\n\nHere's an attempted summary of our desires, and my ideal route\nforward:\n\n* Preferred local submodule branches for each superproject branch.\n * Not currently supported by Git.\n * Requires some sort of per-superproject-branch .git/config.\n * Fall back to the remote-tracking submodule..branch?\n\n* Auto checkout of the preferred branch\n * Can do this at clone-update time with my patch.\n * For later submodule branch switches, maybe we want:\n\n git submodule checkout [-b ] […]\n\n Then if a user blows off their detached HEAD, at least they'll\n feel a bit sheepish afterwards.\n\n* Configurable (remote or local) default update source (so folks who\n primarily update --remote don't have to have long command lines).\n * New submodule..source = {remote|local} config\n * New 'update [--source={local|remote}]' option\n * Deprecate 'update --remote' with a long phase out.\n However:\n * Maybe they should just setup an alias instead?\n\nCheers,\nTrevor\n\n-- \nThis email may be signed or encrypted with GnuPG (http://www.gnupg.org).\nFor more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\n"},{"id":"232828","messageId":"xmqq38kzei3d.fsf@gitster.dls.corp.google.com","threadId":"35608","inReplyTo":"CALas-ihHD_eJOXLUrhCVZjidQDmrCN=QpdfMKoN1i9A7FAo3RQ@mail.gmail.com","subject":"Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-01-07T18:56:38Z","receivedAt":"2014-01-07T18:56:38Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Francesco Pretto writes:\n\n>>> > - In which situations does the developer or maintainer switch between\n>>> > your attached/detached mode?\n>>>\n>>> The developer/maintainer does so optionally and voluntarily and it\n>>> effects only its private working tree.\n>>\n>> This does not answer my question. I would like to find out the reason\n>> why one would do the switch.\n>\n> The developer does it voluntarily, at his responsibility, because he\n> may decide to partecipate more actively to the development of the\n> submodule and still want to use a simple \"git submodule update\" to\n> updates his submodules, overriding its configuration as it can be done\n> for other properties like, for example, \"branch\".\n\nIt is still unclear to me why we need attached/detached mode for\nthat. The developer may want to do an exploratory development,\nwhose result is unknown to deserve to be committed on the specified\nbranch at the beginning, and choose to build on a detached HEAD,\nwhich is a perfectly normal thing to do. But the standard way to do\nso, whether the developer is working in the top-level superproject\nor in a submodule, would be to just do:\n\n\tcd $there && git checkout HEAD^0\n\nor use whatever commit the state to be detached is at instead of\n\"HEAD\" in the above example, no?\n"},{"id":"232829","messageId":"xmqqy52rd30b.fsf@gitster.dls.corp.google.com","threadId":"35608","inReplyTo":"CALas-ihHD_eJOXLUrhCVZjidQDmrCN=QpdfMKoN1i9A7FAo3RQ@mail.gmail.com","subject":"Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-01-07T19:07:48Z","receivedAt":"2014-01-07T19:07:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Francesco Pretto writes:\n\n> My bottom line:\n> - For what I understand, detached HEAD it's a way to say \"hey, you\n> have to stay on this commit. Also don't even think you can push to the\n> upstream branch\". This sometimes can't be spurious, as in the use case\n> I wrote above: access control on the remote repositories should be\n> enough. I think maintainers should have the option to make developers\n> to clone a repository starting with an attached HEAD on the branch\n> suggested in submodule.$name.branch;\n> - \"git submodule update\" is missing a property to do automatically\n> \"--remote\". I think in the use case I wrote it's really handy to have\n> a \"git submodule update\" to act like this.\n\nThe short version I read in the message is that your workflow, in\nwhich partipants want to work on a branch, gets frustrating with the\ncurrent system only because the default update/initial cloning\ndetaches HEAD and will stay in that state until the user gets out of\nthe detached state manually. Once that initial detachment is fixed,\nthere is no more major issue, as update will stay on that branch.\n\nAm I reading you correctly?\n"},{"id":"232833","messageId":"CALas-igZCvqybsw6honUTddnJYFkOYfieyfvW0yfT2oUVrPhVg@mail.gmail.com","threadId":"35608","inReplyTo":"xmqqy52rd30b.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"Francesco Pretto","fromEmail":"ceztko@gmail.com","sentAt":"2014-01-07T19:25:33Z","receivedAt":"2014-01-07T19:25:33Z","isPatch":true,"sender":{"key":"ceztko@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3037449?v=4"},"body":"2014/1/7 Junio C Hamano :\n> Francesco Pretto writes:\n>\n>> My bottom line:\n>> - For what I understand, detached HEAD it's a way to say \"hey, you\n>> have to stay on this commit. Also don't even think you can push to the\n>> upstream branch\". This sometimes can't be spurious, as in the use case\n>> I wrote above: access control on the remote repositories should be\n>> enough. I think maintainers should have the option to make developers\n>> to clone a repository starting with an attached HEAD on the branch\n>> suggested in submodule.$name.branch;\n>> - \"git submodule update\" is missing a property to do automatically\n>> \"--remote\". I think in the use case I wrote it's really handy to have\n>> a \"git submodule update\" to act like this.\n>\n> The short version I read in the message is that your workflow, in\n> which partipants want to work on a branch, gets frustrating with the\n> current system only because the default update/initial cloning\n> detaches HEAD and will stay in that state until the user gets out of\n> the detached state manually. Once that initial detachment is fixed,\n> there is no more major issue, as update will stay on that branch.\n>\n> Am I reading you correctly?\n>\n\nYep, you got it correctly.\n"},{"id":"232834","messageId":"20140107192713.GG11060@odin.tremily.us","threadId":"35608","inReplyTo":"87ppo4zzkb.fsf@engster.org","subject":"Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-07T19:27:13Z","receivedAt":"2014-01-07T19:27:13Z","isPatch":true,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"On Mon, Jan 06, 2014 at 08:21:24PM +0100, David Engster wrote:\n> +-----------+\n> | master | <--\n> +-------+ +-----------+ | Merges to/from master\n> | CEDET | | done only by CEDET developers\n> +-------+ | \n> +-----------+ |\n> | stable | <-- <--------\n> +-----------+ |\n> |\n> |\n> | Any Emacs developer\n> | can push and commit\n> | submodule\n> +--------+ +----------------------+ |\n> | Emacs | -- | lisp/cedet submodule | <-\n> +--------+ +----------------------+\n\nThis looks reasonable, and except for the detached-HEAD after the\ninitial update-clone, I think Git already supports everything you\nneed. If you set submodule.cedet.update to 'rebase' (or 'merge') you\ncan easily integrate your local master changes with cedet/master\n(e.g. if a CEDET dev updates cedet/master before the Emacs dev has a\nchance to push their fix). With the non-checkout update mode, you'll\nalso stay on your checked-out master branch during 'submodule update'\ncalls.\n\n> AFAICS the main problem with this approach is that one always has to\n> think of committing the new SHA1 of the submodule.\n> …\n> However, as Heiko notes, the history must be preserved to be able to\n> go back to earlier revisions, so there must be some kind of commit\n> for the submodule when 'stable' changes; maybe that could be\n> automated somehow?\n\nIf an Emacs dev in the submodule makes the CEDET change, you could use\na post-commit hook (in the CEDET submodule) to also commit the change\nto the Emacs superproject). However, commiting only the submodule\nbump may not be what you want. Maybe there are other superproject\nchanges that should be committed alongside the submodule bump. Maybe\nthere is stuff in the superprojects's staging area that should *not*\nbe committed alongside the submodule bump. This ambiguity makes it\ntricky for Git to automatically do “the right thing”.\n\nIf cedet/master is updated independently by the CEDET devs, there's no\nway for the local Emacs repo to know about the change, so it's\nimpossible to automatically update Emacs (without polling for CEDET\nupdates or some other transgression ;).\n\nCheers,\nTrevor\n\n-- \nThis email may be signed or encrypted with GnuPG (http://www.gnupg.org).\nFor more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\n"},{"id":"232838","messageId":"CALas-ijD977divDQtY0ZhDAQiA60aGLr0KzN+QvoL=zTb1z=6A@mail.gmail.com","threadId":"35608","inReplyTo":"xmqq38kzei3d.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"Francesco Pretto","fromEmail":"ceztko@gmail.com","sentAt":"2014-01-07T19:44:51Z","receivedAt":"2014-01-07T19:44:51Z","isPatch":true,"sender":{"key":"ceztko@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3037449?v=4"},"body":"2014/1/7 Junio C Hamano :\n> Francesco Pretto writes:\n>> The developer does it voluntarily, at his responsibility, because he\n>> may decide to partecipate more actively to the development of the\n>> submodule and still want to use a simple \"git submodule update\" to\n>> updates his submodules, overriding its configuration as it can be done\n>> for other properties like, for example, \"branch\".\n>\n> It is still unclear to me why we need attached/detached mode for\n> that. The developer may want to do an exploratory development,\n> whose result is unknown to deserve to be committed on the specified\n> branch at the beginning, and choose to build on a detached HEAD,\n> which is a perfectly normal thing to do. But the standard way to do\n> so, whether the developer is working in the top-level superproject\n> or in a submodule, would be to just do:\n>\n> cd $there && git checkout HEAD^0\n>\n> or use whatever commit the state to be detached is at instead of\n> \"HEAD\" in the above example, no?\n>\n\nBecause of the overlapping change with the the other patch proposed by\nTrevor, and to not generate confusion, I will stop for now pursuing\nfor an \"attach|detach\" command/switch specific for submodules, waiting\nfor Trevors's patch possible acceptance. After that I will see it\nstill makes sense or not.\n"},{"id":"232877","messageId":"20140107223625.GB29954@odin.tremily.us","threadId":"35608","inReplyTo":"20140107041004.GA11060@odin.tremily.us","subject":"Preferred local submodule branches (was: Introduce git submodule attached update)","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-07T22:36:25Z","receivedAt":"2014-01-07T22:36:25Z","isPatch":false,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"On Tue, Jan 07, 2014 at 10:51:34PM +0100, Francesco Pretto wrote:\n> 2014/1/7 W. Trevor King :\n> >\n> > I'd be happy to hear ideas about superproject-branch-specific local\n> > overrides to a hypothetical submodule..local-branch, in the\n> > event that a developer doesn't like a default set in .gitmodules. If\n> > I could think of a way to do that, we could avoid this heuristic\n> > approach, and make the local submodule..local-branch\n> > vs. remote-tracking submodule..branch distinction more obvious.\n> \n> Uh, I think you got it wrong in the other thread:\n\nI'm grafting this discussion back on to the thread where I proposed\nsubmodule..local-branch.\n\n> I didn't proposed such feature.\n\nRight. I proposed this feature after reading your proposed workflow.\n\n> I just wanted the attached submodule use case to be supported and of\n> course \"--branch means attached\" is even easier to get this.\n\nAs I understood it, the '--branch means attached' stuff was tied up\nwith automatic --remote updates.\n\nThere are three branches that submodule folks usually care about:\n\n1. The linked $sha1 in the superproject (set explicitly for every\n superproject commit, and thus for every superproject branch).\n2. The remote-tracking submodule..branch that lives in the\n upstream submodule..url repository.\n3. The submodule's locally checked out branch, which we currently let\n the developer setup by hand, which is used integrated with one of\n the other two branches during non-checkout updates.\n\nGit is currently a bit weak on conveniently handling type-3 branches.\n“Just use what the developer has setup” works well for many basic\nworkflows, but falls short for:\n\n* Cloning-updates, where we currently always setup a detached HEAD.\n* Workflows where the preferred type-3 branch depends on the\n superproject branch.\n\nThe former is easy to fix [1] if you accept submodule..branch as\na guess, but this conflates the type-2 and type-3 branches.\n\nFor the latter, you'd want something like:\n\nOn Mon, Jan 06, 2014 at 08:10:04PM -0800, W. Trevor King wrote:\n> * Auto checkout of the preferred branch\n> * Can do this at clone-update time with my patch.\n> * For later submodule branch switches, maybe we want:\n> \n> git submodule checkout [-b ] […]\n> \n> Then if a user blows off their detached HEAD, at least they'll\n> feel a bit sheepish afterwards.\n\nwhich would likely need some of Jens' new core checkout handling [2].\n\nCheers,\nTrevor\n\n[1]: Using something along the lines of my\n http://article.gmane.org/gmane.comp.version-control.git/239967\n[2]: http://article.gmane.org/gmane.comp.version-control.git/240117\n\n-- \nThis email may be signed or encrypted with GnuPG (http://www.gnupg.org).\nFor more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\n"},{"id":"232879","messageId":"20140107225128.GC10782@sandbox-ub","threadId":"35608","inReplyTo":"20140107041004.GA11060@odin.tremily.us","subject":"Re: Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2014-01-07T22:51:28Z","receivedAt":"2014-01-07T22:51:28Z","isPatch":true,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA1\n\nOn Mon, Jan 06, 2014 at 08:10:04PM -0800, W. Trevor King wrote:\n> Here's an attempted summary of our desires, and my ideal route\n> forward:\n> \n> * Preferred local submodule branches for each superproject branch.\n> * Not currently supported by Git.\n> * Requires some sort of per-superproject-branch .git/config.\n> * Fall back to the remote-tracking submodule..branch?\n> \n> * Auto checkout of the preferred branch\n> * Can do this at clone-update time with my patch.\n> * For later submodule branch switches, maybe we want:\n> \n> git submodule checkout [-b ] […]\n> \n> Then if a user blows off their detached HEAD, at least they'll\n> feel a bit sheepish afterwards.\n\nWell, for development on a detached HEAD in a submodule we are currently\nnot very careful anyway. A simple\n\n\tgit submodule update\n\nwill already blow away any detached HEAD work. But AFAIK it should\ntrigger the \"you are leaving commits from a detached HEAD behind\"\nwarning, so there is some safeguard and recovery.\n\nCheers Heiko\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1.4.14 (GNU/Linux)\n\niEYEARECAAYFAlLMhPAACgkQjLR3Aoip+rqP6wCeIhtpWLJC3XVO3nu2ViQTbHPg\nT5wAoLLEZ256GOOjBxoTKo2/FmfvQGLp\n=+bqm\n-----END PGP SIGNATURE-----\n"},{"id":"232880","messageId":"20140107231404.GD26583@odin.tremily.us","threadId":"35608","inReplyTo":"20140107225128.GC10782@sandbox-ub","subject":"Re: [PATCH 2/2] Introduce git submodule attached update","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-07T23:14:04Z","receivedAt":"2014-01-07T23:14:04Z","isPatch":true,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"On Tue, Jan 07, 2014 at 11:51:28PM +0100, Heiko Voigt wrote:\n> On Mon, Jan 06, 2014 at 08:10:04PM -0800, W. Trevor King wrote:\n> > Here's an attempted summary of our desires, and my ideal route\n> > forward:\n> > \n> > * Preferred local submodule branches for each superproject branch.\n> > * Not currently supported by Git.\n> > * Requires some sort of per-superproject-branch .git/config.\n> > * Fall back to the remote-tracking submodule..branch?\n> > \n> > * Auto checkout of the preferred branch\n> > * Can do this at clone-update time with my patch.\n> > * For later submodule branch switches, maybe we want:\n> > \n> > git submodule checkout [-b ] […]\n> > \n> > Then if a user blows off their detached HEAD, at least they'll\n> > feel a bit sheepish afterwards.\n> \n> Well, for development on a detached HEAD in a submodule we are currently\n> not very careful anyway. A simple\n> \n> \tgit submodule update\n> \n> will already blow away any detached HEAD work.\n\nOnly if you use the checkout strategy. With --merge or --rebase,\nyou'll have the $sha1 (or upstream remote with --remote) integrated\nwith your detached HEAD work. You end up with a new detached HEAD\ncontaining the result of the integration (just confirmed with tests\nusing Git v1.8.3.2). That seems reasonable to me, so I'm happy with\nthe integration logic.\n\n> But AFAIK it should trigger the \"you are leaving commits from a\n> detached HEAD behind\" warning, so there is some safeguard and\n> recovery.\n\nI did not see those in testing with Git v1.8.3.2, likely because of\nthe '-f -q' we pass to 'git checkout' for checkout-mode updates.\n\nRegardless of branch integration issues, I think a\nper-superproject-branch preferred submodule branch is important for\n'git checkout' to work in the superproject. If you want:\n\n* submodule branch master for superproject branch master, and\n* submodule branch my-feature for superproject branch my-feature,\n\n $ git checkout my-feature\n\nin the superproject is currently going to leave you with the submodule\non master, which is not convenient ;). I think we should come up with\na better solution to the superproject checkout problem before adding\nin additional complications due to branch integration ;).\n\nCheers,\nTrevor\n\n-- \nThis email may be signed or encrypted with GnuPG (http://www.gnupg.org).\nFor more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\n"},{"id":"232881","messageId":"20140107235208.GC29954@odin.tremily.us","threadId":"35608","inReplyTo":"20140107223625.GB29954@odin.tremily.us","subject":"Re: Preferred local submodule branches (was: Introduce git submodule attached update)","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-07T23:52:08Z","receivedAt":"2014-01-07T23:52:08Z","isPatch":false,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"On Tue, Jan 07, 2014 at 02:36:25PM -0800, W. Trevor King wrote:\n> There are three branches that submodule folks usually care about:\n> \n> 1. The linked $sha1 in the superproject (set explicitly for every\n> superproject commit, and thus for every superproject branch).\n> 2. The remote-tracking submodule..branch that lives in the\n> upstream submodule..url repository.\n> 3. The submodule's locally checked out branch, which we currently let\n> the developer setup by hand, which is used integrated with one of\n> the other two branches during non-checkout updates.\n> \n> Git is currently a bit weak on conveniently handling type-3 branches.\n> “Just use what the developer has setup” works well for many basic\n> workflows, but falls short for:\n> \n> * Cloning-updates, where we currently always setup a detached HEAD.\n> * Workflows where the preferred type-3 branch depends on the\n> superproject branch.\n> \n> The former is easy to fix [1] if you accept submodule..branch as\n> a guess, but this conflates the type-2 and type-3 branches.\n> \n> For the latter, you'd want something like:\n> \n> On Mon, Jan 06, 2014 at 08:10:04PM -0800, W. Trevor King wrote:\n> > * Auto checkout of the preferred branch\n> > * Can do this at clone-update time with my patch.\n> > * For later submodule branch switches, maybe we want:\n> > \n> > git submodule checkout [-b ] […]\n> > \n> > Then if a user blows off their detached HEAD, at least they'll\n> > feel a bit sheepish afterwards.\n> \n> which would likely need some of Jens' new core checkout handling [2].\n> \n> [1]: Using something along the lines of my\n> http://article.gmane.org/gmane.comp.version-control.git/239967\n> [2]: http://article.gmane.org/gmane.comp.version-control.git/240117\n\nFor example, in Jonathan's recent version of Jens' series, the\ninitial-setup and update functionality are moving into C. See:\n\n* populate_submodule() [1] for the initial-clone setup (calling\n 'read-tree'), and\n* update_submodule() [2] for subsequent updates (calling 'checkout -q'\n with an optional '-f')\n\nthis is where any submodule..local-branch would come into play,\nif we decide to go down that route. It doesn't look like the C\nupdates have the auto-clone functionality that the Bash updates have.\nI'm not sure if that's in the pipe or not. I'm not as familiar with\nthe C implementation though, so maybe I'm missing the mark here.\n\nCheers,\nTrevor\n\n[1]: http://article.gmane.org/gmane.comp.version-control.git/239698\n[2]: http://article.gmane.org/gmane.comp.version-control.git/239699\n\n-- \nThis email may be signed or encrypted with GnuPG (http://www.gnupg.org).\nFor more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\n"},{"id":"232892","messageId":"20140108034708.GG26583@odin.tremily.us","threadId":"35608","inReplyTo":"20140107235208.GC29954@odin.tremily.us","subject":"Re: Preferred local submodule branches","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-08T03:47:08Z","receivedAt":"2014-01-08T03:47:08Z","isPatch":false,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"On Wed, Jan 08, 2014 at 03:12:44AM +0100, Francesco Pretto wrote:\n> 2014/1/8 W. Trevor King :\n> > Note that I've moved away from “submodule..branch\n> > set means attached” towards “we should set per-superproject-branch\n> > submodule..local-branch explicitly” [1].\n> \n> Honestly, I'm having an hard time to follow this thread.\n\nI tried to refocus things (with a new subject) in this sub-thread.\nHopefully that helps make the discussion more linear ;).\n\n> Also, you didn't update the patch.\n\nI'm waiting [1] to see how the C-level checkout by Jens and Jonathan\nprogresses [2,3] before writing more code.\n\n> If you were endorsed by someone (Junio, Heiko, ...) for the\n> \"submodule..local-branch\" feature please show me where.\n\nAs far as I know, no-one else has endorsed this idea (yet :). Heiko\nhas expressed concern [4], but not convincingly enough (yet :) to win\nme over ;).\n\n> I somehow understand the point of the\n> \"submodule..local-branch\" property, but I can't \"see\" the the\n> workflow. Please, show me some hypothetical scripting example with\n> as much complete as possible workflow (creation, developer update,\n> mantainers creates feature branch, developer update, developer\n> attach to another branch).\n\nI've put this at the bottom of the message to avoid bothering the\ntl;dr crowd, although they have probably long since tuned us out ;).\n\n> Also, consider I proposed to support the attached HEAD path to\n> reduce complexity and support a simpler use case for git\n> submodules. I would be disappointed if the complexity is reduced in\n> a way and augmented in another.\n\nAgreed. I think we're all looking for the least-complex solution that\ncovers all (or most) reasonable workflows.\n\n> > On Wed, Jan 08, 2014 at 01:17:49AM +0100, Francesco Pretto wrote:\n> >> # Attach the submodule HEAD to .\n> >> # Also set \".git/config\" 'submodule..branch' to \n> >> $ git submodule head -b --attach \n> > [...]\n> > I also prefer 'checkout' to 'head', because 'checkout'\n> > already exists in non-submodule Git for switching between local\n> > branches.\n> \n> I can agree with similarity to other git commands, but 'checkout'\n> does not give me the idea of something that writes to \".git/config\"\n> or \".gitmodules\".\n\nNeither does 'head'. We have precedence in 'git submodule add' for\nembracing and extending a core git command with additional .gitmodules\nmanipulation. I think it's easier to pick up the submodule jargon\nwhen we add submodule-specific side-effects to submodule-specific\ncommands named after their core analogs than it would be if we pick\nunique names for the submodule-specific commands.\n\n> >> # Unset \".git/config\" 'submodule..branch'\n> >> # Also attach or detach the HEAD according to what is in \".gitmodules\":\n> >> # with Trevor's patch 'submodule..branch' set means attached,\n> >> # unset means detached\n> >> $ git submodule head --reset \n> >\n> > To me this reads “always detach HEAD” (because it unsets\n> > submodule..branch, and submodule..branch unset means\n> > detached).\n> \n> I disagree: this would remove only the value in \".git/config\". If the\n> value is till present in \".gitmodules\", as I wrote above, the behavior\n> of what is in the index should be respected as for the other\n> properties. Also it gives a nice meaning to a switch like --reset :\n> return to how it was before.\n\nAh, that makes more sense. I had confused .git/config with\n“.gitmodules and .git/config”.\n\n> >> NOTE: feature branch part!\n> >>\n> >> # Set \".gitmodules\" 'submodule..branch' to \n> >> $ git submodule head -b --attach --index \n> >>\n> >> # Unset \".gitmodules\" 'submodule..branch'\n> >> $ git submodule head --reset --index \n> >> ---------------------------------------------------------------------\n> >\n> > These are just manipulating .gitmodules. I think we also need\n> > per-superproject-branch configs under the superproject's .git/ for\n> > developer overrides.\n> \n> I disagree: in my idea the --index switch is a maintainer only command\n> to modify the behavior of the developers and touch only indexed files\n> (.gitmodules, or create a new submodule branch). It expressly don't\n> touch .git/config.\n\nSomething that just touches the config files is syntactic sugar, so I\navoided a more detailed review and moved on to address what I saw as a\nmore fundamental issue (preferred submodule local branches on a\nper-superproject-branch level).\n\nHere's a detailed workflow for the {my-feature, my-feature, master}\nexample I roughed out before [5].\n\n # create the subproject\n mkdir subproject &&\n (\n cd subproject &&\n git init &&\n echo 'Hello, world' > README &&\n git add README &&\n git commit -m 'Subproject v1'\n ) &&\n # create the superproject\n mkdir superproject\n (\n cd superproject &&\n git init &&\n git submodule add ../subproject submod &&\n git config -f .gitmodules submodule.submod.update merge &&\n git commit -am 'Superproject v1' &&\n ( # 'submodule update' doesn't look in .gitmodules (yet [6]) for a\n # default update mode. Copy submodule.submod.update over to\n # .git/config\n git submodule init\n )\n ) &&\n # start a feature branch on the superproject\n (\n cd superproject &&\n #git checkout -b my-feature --recurse-submodules &&\n ( # 'git submodule checkout --recurse-submodules' doesn't exist yet, so...\n git checkout -b my-feature &&\n git config -f .gitmodules submodule.submod.local-branch my-feature &&\n cd submod &&\n git checkout -b my-feature\n ) &&\n (\n cd submod &&\n echo 'Add the subproject side of this feature' > my-feature &&\n git add my-feature &&\n git commit -m 'Add my feature to the subproject'\n ) &&\n echo 'Add the superproject side of this feature' > my-feature &&\n git add my-feature &&\n git commit -m 'Add the feature to the superproject'\n ) &&\n # meanwhile, the subproject has been advancing\n (\n cd subproject &&\n echo 'Goodbye, world' >> README &&\n git commit -am 'Subproject v2'\n ) &&\n # we need to get that critical advance into the superproject quick!\n (\n cd superproject &&\n # update the master branch\n #git checkout --recurse-submodules master\n ( # 'git checkout --recurse-submodules' doesn't exist yet [2,3].\n # Even with that patch, 'git checkout' won't respect\n # submodule..local-branch without further work.\n git checkout master &&\n cd submod &&\n git checkout master # don't pull in our my-feature work\n )\n git submodule update --remote &&\n git commit -am 'Catch submod up with Subproject v2' &&\n # update the my-feature branch\n git checkout my-feature\n ( # 'git checkout' doesn't mess with submodules\n cd submod &&\n git checkout my-feature\n )\n git submodule update --remote &&\n git commit -am 'Catch submod up with Subproject v2' &&\n # what does the history look like?\n (\n cd submod &&\n git --no-pager log --graph --date-order --oneline --decorate --all\n # * 3a22cef (HEAD, my-feature) Merge commit 'd53958b18277ce5bd6c734e9597a69bb878b31e1' into my-feature\n # |\\ \n # * | 8322dcc Add my feature to the subproject\n # | * d53958b (origin/master, origin/HEAD, master) Subproject v2\n # |/ \n # * 9813010 Subproject v1\n ) &&\n git ls-tree master submod &&\n # 160000 commit d53958b18277ce5bd6c734e9597a69bb878b31e1 submod\n git ls-tree my-feature submod\n # 160000 commit 3a22cef30db57f1b89251f3e434fa0bd0f1b99a2 submod\n )\n git --version\n # git version 1.8.3.2\n\nThe currently-ugly bits could be fixed with:\n\n* 'git submodule update' falling back on .gitmodules for\n submodule..update [6].\n* 'git submodule checkout -b my-feature --recurse-submodules' should\n checkout the submodule..local-branch configured for the\n super-project's my-feature branch (but only if that wouldn't destroy\n some current submodule information). This would build on work in\n Jens and Jonathans' branch [2,3].\n\nCheers,\nTrevor\n\n[1]: http://article.gmane.org/gmane.comp.version-control.git/240127\n[2]: http://article.gmane.org/gmane.comp.version-control.git/240117\n[3]: http://thread.gmane.org/gmane.comp.version-control.git/239695\n[4]: http://article.gmane.org/gmane.comp.version-control.git/240178\n[5]: http://article.gmane.org/gmane.comp.version-control.git/240190\n[6]: http://article.gmane.org/gmane.comp.version-control.git/239246\n\n-- \nThis email may be signed or encrypted with GnuPG (http://www.gnupg.org).\nFor more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\n"},{"id":"232894","messageId":"20140108040627.GD29954@odin.tremily.us","threadId":"35608","inReplyTo":"20140108034708.GG26583@odin.tremily.us","subject":"Re: Preferred local submodule branches","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-08T04:06:27Z","receivedAt":"2014-01-08T04:06:27Z","isPatch":false,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"On Tue, Jan 07, 2014 at 07:47:08PM -0800, W. Trevor King wrote:\n> #git checkout --recurse-submodules master\n> ( # 'git checkout --recurse-submodules' doesn't exist yet [2,3].\n> # Even with that patch, 'git checkout' won't respect\n> # submodule..local-branch without further work.\n> git checkout master &&\n> cd submod &&\n> git checkout master # don't pull in our my-feature work\n> )\n> git submodule update --remote &&\n> git commit -am 'Catch submod up with Subproject v2' &&\n> # update the my-feature branch\n> git checkout my-feature\n> ( # 'git checkout' doesn't mess with submodules\n> cd submod &&\n> git checkout my-feature\n> )\n\nOops, the my-feature checkout block should have been:\n\n #git checkout --recurse-submodules my-feature\n ( # 'git checkout --recurse-submodules' doesn't exist yet...\n git checkout my-feature &&\n cd submod &&\n git checkout my-feature\n )\n\nmirroring the earlier master checkout block. Sorry for the sloppy\nediting.\n\nCheers,\nTrevor\n\n-- \nThis email may be signed or encrypted with GnuPG (http://www.gnupg.org).\nFor more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\n"},{"id":"232943","messageId":"cover.1389247320.git.wking@tremily.us","threadId":"35608","inReplyTo":"20140108040627.GD29954@odin.tremily.us","subject":"[RFC v3 0/4] Preferred local submodule branches","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-09T06:17:51Z","receivedAt":"2014-01-09T06:17:51Z","isPatch":false,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"From: \"W. Trevor King\" \n\nIn another branch of the submodule thread Francesco kicked off, I\nmentioned that we could store the preferred local submodule branch on\na per-superbranch level if we used the\n.git/modules//config for local overrides [1]. Here's\na patch series that greatly extends my v2 \"submodule: Respect\nrequested branch on all clones\" series [2] to also support automatic,\nrecursive submodule checkouts, as I outlined here [3]. After this\nseries, I can get through:\n\n # create the subproject\n mkdir subproject &&\n (\n cd subproject &&\n git init &&\n echo 'Hello, world' > README &&\n git add README &&\n git commit -m 'Subproject v1'\n ) &&\n # create the superproject\n mkdir superproject\n (\n cd superproject &&\n git init &&\n git submodule add ../subproject submod &&\n git config -f .gitmodules submodule.submod.update merge &&\n git commit -am 'Superproject v1' &&\n ( # 'submodule update' doesn't look in .gitmodules (yet [4]) for a\n # default update mode. Copy submodule.submod.update over to\n # .git/config\n git submodule init\n )\n ) &&\n # start a feature branch on the superproject\n (\n cd superproject &&\n #git checkout -b my-feature --recurse-submodules &&\n ( # 'git submodule checkout --recurse-submodules' doesn't exist yet, so...\n git checkout -b my-feature &&\n git submodule checkout -b --gitmodules\n ) &&\n (\n cd submod &&\n echo 'Add the subproject side of this feature' > my-feature &&\n git add my-feature &&\n git commit -m 'Add my feature to the subproject'\n ) &&\n echo 'Add the superproject side of this feature' > my-feature &&\n git add my-feature &&\n git commit -am 'Add the feature to the superproject'\n ) &&\n # meanwhile, the subproject has been advancing\n (\n cd subproject &&\n echo 'Goodbye, world' >> README &&\n git commit -am 'Subproject v2'\n ) &&\n # we need to get that critical advance into the superproject quick!\n (\n cd superproject &&\n # update the master branch\n #git checkout --recurse-submodules master\n ( # 'git checkout --recurse-submodules' doesn't exist yet [5,6].\n # Even with that patch, 'git checkout' won't respect\n # submodule..local-branch without further work.\n git checkout master &&\n git submodule checkout\n ) &&\n git submodule update --remote &&\n git commit -am 'Catch submod up with Subproject v2' &&\n # update the my-feature branch\n #git checkout --recurse-submodules my-feature &&\n ( # 'git checkout --recurse-submodules' doesn't exist yet [5,6].\n git checkout my-feature &&\n git submodule checkout\n ) &&\n git submodule update --remote &&\n git commit -am 'Catch submod up with Subproject v2' &&\n # what does the history look like?\n (\n cd submod &&\n git --no-pager log --graph --date-order --oneline --decorate --all\n # * 16d9e3e (HEAD, my-feature) Merge commit 'f5e134d5747ee4a206e96d8c017f92f5b29a07f3' into my-feature\n # |\\ \n # | * f5e134d (origin/master, origin/HEAD, master) Subproject v2\n # * | 0a1cd07 Add my feature to the subproject\n # |/ \n # * c2d32ba Subproject v1\n ) &&\n printf 'master: ' &&\n git ls-tree master submod &&\n # master: 160000 commit f5e134d5747ee4a206e96d8c017f92f5b29a07f3 submod\n printf 'my-feature: ' &&\n git ls-tree my-feature submod\n # my-feature: 160000 commit 16d9e3ea2fb57e7a166587203abdb328f90895d1 submod\n )\n git --version\n # git version 1.8.5.2.237.g01c62c6\n\nI think the first three patches are fairly solid. The last one gets\nthrough the above script, but I'd need a more thorough test suite\nbefore I trusted it. I tried to be detailed in the commit messages,\nbut of course, we'd want some user-facing documentation if we actually\nmerged something like this series. I'm sending it to the list mostly\nto explain my current views and re-focus debate [1].\n\n[1]: http://article.gmane.org/gmane.comp.version-control.git/240240\n[2]: http://article.gmane.org/gmane.comp.version-control.git/239967\n[3]: http://article.gmane.org/gmane.comp.version-control.git/240192\n[4]: http://article.gmane.org/gmane.comp.version-control.git/239246\n[5]: http://thread.gmane.org/gmane.comp.version-control.git/239695\n[6]: http://article.gmane.org/gmane.comp.version-control.git/240117\n\nCheers,\nTrevor\n\nW. Trevor King (4):\n submodule: Add helpers for configurable local branches\n submodule: Teach 'update' to preserve local branches\n submodule: Teach 'add' about a configurable local-branch\n submodule: Add a new 'checkout' command\n\n git-submodule.sh | 152 ++++++++++++++++++++++++++++++++++++++++++++++++++-----\n 1 file changed, 138 insertions(+), 14 deletions(-)\n\n-- \n1.8.5.2.237.g01c62c6\n"},{"id":"232946","messageId":"684f061e58bd16e617f438be9d6ed7b8d913463b.1389247320.git.wking@tremily.us","threadId":"35608","inReplyTo":"cover.1389247320.git.wking@tremily.us","subject":"[RFC v3 1/4] submodule: Add helpers for configurable local branches","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-09T06:17:52Z","receivedAt":"2014-01-09T06:17:52Z","isPatch":false,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"From: \"W. Trevor King\" \n\nThere are three branches that submodule folks usually care about:\n\n1. The linked $sha1 in the superproject (set explicitly for every\n superproject commit, and thus for every superproject branch).\n2. The remote-tracking submodule..branch that tracks a branch in\n upstream submodule..url repository.\n3. The submodule's locally checked out branch, which we currently let\n the developer setup by hand, which is integrated with one of the\n other two branches by non-checkout update modes.\n\nGit is currently a bit weak on conveniently handling branch #3. \"Just\nuse what the developer has setup\" works well for many basic workflows,\nbut falls short for:\n\n* Cloning-updates, where we currently always setup a detached HEAD.\n This is easy to fix if you accept submodule..branch or the\n branch pointed to by the cloned repository's HEAD as a guess, but\n this conflates branch #2 and branch #3, which may confuse users.\n\n* Workflows where the preferred #3 branch depends on the superproject\n branch. For example, if the remote subproject has only a master\n branch, but the local superproject needs to develop several\n submodule feature branches simultaneously, you can have a situation\n like this:\n\n Superproject branch Submodule branch Subproject branch\n =================== ================ =================\n master master master\n feature-1 feature-1 master\n feature-2 feature-2 master\n feature-3 feature-2 master\n\nIn order to checkout the appropriate submodule branch for a given\nsuperproject branch, we need a way to specify the preferred submodule\nbranch for a given superproject branch. This commit adds two helper\nfunctions:\n\n* get_current_branch, to determine which superproject branch you're\n on, and\n* get_local_branch, to determine the preferred submodule branch for\n that superproject branch.\n\nThe lookup chain for the local-branch is:\n\n1. superproject..local-branch in the submodule's\n config (superproject/.git/modules//config). This\n is where the developer can store local per-superproject-branch\n overrides (e.g. if they wanted to use submodule branch feature-1\n with superproject branch feature-3).\n2. submodule..local-branch in the superproject's\n config. This is where the developer can store local\n cross-superproject-branch overrides (e.g. if they wanted to use\n submodule branch master for any superproject branch that didn't\n have a per-superproject-branch override).\n3. submodule..local-branch in the superproject's\n .gitmodules file. Because the gitmodules file is stored in the\n superproject's versioned tree, it is automatically\n superproject-branch-specific. For example:\n\n $ git cat-file -p feature-1:.gitmodules\n ...\n [submodule \"submod\"]\n ...\n local-branch = feature-1\n $ git cat-file -p feature-3:.gitmodules\n ...\n [submodule \"submod\"]\n ...\n local-branch = feature-2\n\n this is where the project-wide defaults are setup and shared\n between developers.\n4. The default local-branch is 'master'.\n\nThe new get_local_branch function handles the first step in this\nchain. The next two steps are already covered by the existing\nget_submodule_config.\n---\n git-submodule.sh | 33 +++++++++++++++++++++++++++++++++\n 1 file changed, 33 insertions(+)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 2677f2e..56fc3f1 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -220,6 +220,39 @@ get_submodule_config () {\n \tprintf '%s' \"${value:-$default}\"\n }\n \n+#\n+# Print a submodule's configured local branch name\n+#\n+# $1 = superproject branch\n+# $2 = default (from the superproject's .gitmodules)\n+#\n+# To be called from the submodule root directory.\n+#\n+get_local_branch ()\n+{\n+\tsuperproject_branch=\"$1\"\n+\tdefault=\"${2:-master}\"\n+\tif test -z \"${superproject_branch}\"\n+\tthen\n+\t\tvalue=\"\"\n+\telse\n+\t\tvalue=$(git config superproject.\"$superproject_branch\".local-branch)\n+\tfi\n+\tprintf '%s' \"${value:-$default}\"\n+}\n+\n+#\n+# Print the currently checked out branch of the current repository\n+#\n+# $1 = default\n+#\n+get_current_branch ()\n+{\n+\tdefault=\"$1\"\n+\tbranch=$(git rev-parse --abbrev-ref HEAD 2>/dev/null) ||\n+\tbranch=\"\"\n+\tprintf '%s' \"${branch:-$default}\"\n+}\n \n #\n # Map submodule path to submodule name\n-- \n1.8.5.2.237.g01c62c6\n"},{"id":"232945","messageId":"34c874fbdb0c472e1fadc068baaf7ed00b7b696d.1389247320.git.wking@tremily.us","threadId":"35608","inReplyTo":"cover.1389247320.git.wking@tremily.us","subject":"[RFC v3 2/4] submodule: Teach 'update' to preserve local branches","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-09T06:17:53Z","receivedAt":"2014-01-09T06:17:53Z","isPatch":false,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"From: \"W. Trevor King\" \n\nThere's no sense in setting up a local branch if we're just going to\ngo back to a detached HEAD with every checkout-mode update. This\ncommit replaces the checkout with a reset, updating whatever the\nlocally checked out branch (or detached HEAD) happens to be. While it\nis tempting to checkout a new local-branch here (as we did after the\nclone), it's more consistent to follow the lead of the other update\nmodes and just use the currently checked out branch.\n---\n git-submodule.sh | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 56fc3f1..c5ea7bd 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -930,9 +930,9 @@ Maybe you want to use 'update --init'?\")\"\n \t\t\t\tmust_die_on_failure=yes\n \t\t\t\t;;\n \t\t\t*)\n-\t\t\t\tcommand=\"git checkout $subforce -q\"\n-\t\t\t\tdie_msg=\"$(eval_gettext \"Unable to checkout '\\$sha1' in submodule path '\\$displaypath'\")\"\n-\t\t\t\tsay_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': checked out '\\$sha1'\")\"\n+\t\t\t\tcommand=\"git reset --hard -q\"\n+\t\t\t\tdie_msg=\"$(eval_gettext \"Unable to reset branch to '\\$sha1' in submodule path '\\$displaypath'\")\"\n+\t\t\t\tsay_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': reset branch to '\\$sha1'\")\"\n \t\t\t\t;;\n \t\t\tesac\n \n-- \n1.8.5.2.237.g01c62c6\n"},{"id":"232942","messageId":"75e8c98df73273c2c8174e726e3fc961fbebd6a7.1389247320.git.wking@tremily.us","threadId":"35608","inReplyTo":"cover.1389247320.git.wking@tremily.us","subject":"[RFC v3 3/4] submodule: Teach 'add' about a configurable local-branch","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-09T06:17:54Z","receivedAt":"2014-01-09T06:17:54Z","isPatch":false,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"From: \"W. Trevor King\" \n\nThis patch teaches 'git submodule add' to look for a preferred\nlocal-branch, and to checkout that branch after the initial clone.\nThe local branch will always point at the commit checked out by the\ninternal 'git clone' operation. For example:\n\n $ git submodule add git://example.com/subproject.git submod\n\nwill checkout the branch pointed to by the cloned repository's HEAD,\nand call the local branch 'master'.\n\n $ git submodule add -b my-feature git://example.com/subproject.git submod\n\nwill checkout the branch pointed to by the cloned repository's\nmy-feature, and *still* call the local branch 'master'.\n\n'git submodule add' does not always make an initial clone (e.g. if a\ngit repository already exists at the target path). In cases where\n'git submodule add' does not clone a repository, we just leave the\nlocal branch alone.\n\nThis commit also shifts the post-clone branch checkout logic from\ncmd_add to module_clone, so it can be shared with cmd_update. The\nprevious code only checked out the requested branch in cmd_add but not\nin cmd_update; this left the user on a detached HEAD after an update\ninitially cloned, and subsequent updates kept the HEAD detached,\nunless the user moved to the desired branch himself. Now, unless the\nuser explicitly asks to work on a detached HEAD, subsequent updates\nall happen on the specified branch, which matches the end-user\nexpectation much better.\n---\n git-submodule.sh | 23 +++++++++++++----------\n 1 file changed, 13 insertions(+), 10 deletions(-)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex c5ea7bd..7cee0bf 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -339,7 +339,19 @@ module_clone()\n \techo \"gitdir: $rel/$a\" >\"$sm_path/.git\"\n \n \trel=$(echo $a | sed -e 's|[^/][^/]*|..|g')\n-\t(clear_local_git_env; cd \"$sm_path\" && GIT_WORK_TREE=. git config core.worktree \"$rel/$b\")\n+\tsuperproject_branch=$(get_current_branch)\n+\tdefault_local_branch=$(get_submodule_config \"$sm_name\" local-branch)\n+\t(\n+\t\tclear_local_git_env\n+\t\tcd \"$sm_path\" &&\n+\t\tGIT_WORK_TREE=. git config core.worktree \"$rel/$b\" &&\n+\t\tlocal_branch=$(get_local_branch \"${superproject_branch}\" \"${default_local_branch}\") &&\n+\t\t# ash fails to wordsplit ${branch:+-b \"$branch\"...}\n+\t\tcase \"$branch\" in\n+\t\t'') git checkout -f -q -B \"$local_branch\" ;;\n+\t\t?*) git checkout -f -q -B \"$local_branch\" \"origin/$branch\" ;;\n+\t\tesac\n+\t) || die \"$(eval_gettext \"Unable to checkout submodule '\\$sm_path'\")\"\n }\n \n isnumber()\n@@ -503,15 +515,6 @@ Use -f if you really want to add it.\" >&2\n \t\t\tfi\n \t\tfi\n \t\tmodule_clone \"$sm_path\" \"$sm_name\" \"$realrepo\" \"$reference\" \"$depth\" || exit\n-\t\t(\n-\t\t\tclear_local_git_env\n-\t\t\tcd \"$sm_path\" &&\n-\t\t\t# ash fails to wordsplit ${branch:+-b \"$branch\"...}\n-\t\t\tcase \"$branch\" in\n-\t\t\t'') git checkout -f -q ;;\n-\t\t\t?*) git checkout -f -q -B \"$branch\" \"origin/$branch\" ;;\n-\t\t\tesac\n-\t\t) || die \"$(eval_gettext \"Unable to checkout submodule '\\$sm_path'\")\"\n \tfi\n \tgit config submodule.\"$sm_name\".url \"$realrepo\"\n \n-- \n1.8.5.2.237.g01c62c6\n"},{"id":"232944","messageId":"01c62c6fea978bc7938601ed3b00746c479a00f7.1389247320.git.wking@tremily.us","threadId":"35608","inReplyTo":"cover.1389247320.git.wking@tremily.us","subject":"[RFC v3 4/4] submodule: Add a new 'checkout' command","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-09T06:17:55Z","receivedAt":"2014-01-09T06:17:55Z","isPatch":false,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"From: \"W. Trevor King\" \n\nThis borrows a good deal of the cmd_foreach logic to iterate through\nsubmodules (potentially recursively), checking out the preferred local\nbranch for each submodule (as appropriate for the current superproject\nbranch). Ideally, this logic would be bundled into the forthcoming:\n\n $ git checkout --recurse-submodules\n\nlogic that Jens Lehmann and Jonathan Nieder are working up in C.\nUntil that happens, you can simulate that checkout behaviour with:\n\n $ git checkout some-branch\n $ git submodule checkout --recursive\n\nThe relevant superproject branch used to determine the preferred\nsubmodule branch is the submodules immediate parent, not the top-level\nsuperproject. For example, with the following submodule inheritance:\n\n superproject (branch super-1)\n `-- midproject (branch mid-1)\n `-- subproject (branch sub-1)\n\nThe .gitmodules configs should look like (assuming there are no local\noverrides):\n\n $ git cat-file -p super-1:.gitmodules\n ...\n [submodule \"midproject\"]\n ...\n local-branch = mid-1\n $ cd midproject\n $ git cat-file -p mid-1:.gitmodules\n ...\n [submodule \"subproject\"]\n ...\n local-branch = sub-1\n\nThe super-1 branch need not even exist in the midproject repository.\n\nThis commit handles branch switches inside existing submodules.\nHandling (or even detecting) submodules that are created and destroyed\nor moving submodules that change path between the initial and final\nsuperproject branch is put off to future patches.\n\nI also added minimal support for initial branch creation. Create your\ninitial branch with:\n\n $ git checkout -b my-feature\n $ git submodule checkout --recursive -b --gitmodules\n\nwhich will create new 'my-feature' branches in each submodule (or die\ntrying). It will also save 'my-feature' to the superproject's\n.gitmodules' submodule..local-branch for future checkouts.\nAfter setting up a branch like this, future checkouts (from some other\nbranch) will look like:\n\n $ git checkout my-feature\n $ git submodule checkout --recursive\n---\n git-submodule.sh | 90 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 89 insertions(+), 1 deletion(-)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 7cee0bf..16cebb1 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -6,6 +6,7 @@\n \n dashless=$(basename \"$0\" | sed -e 's/-/ /')\n USAGE=\"[--quiet] add [-b ] [-f|--force] [--name ] [--reference ] [--] []\n+ or: $dashless [--quiet] checkout [--recursive] [-b|-B] [--gitmodules] [--] [...]\n or: $dashless [--quiet] status [--cached] [--recursive] [--] [...]\n or: $dashless [--quiet] init [--] [...]\n or: $dashless [--quiet] deinit [-f|--force] [--] ...\n@@ -36,6 +37,10 @@ update=\n prefix=\n custom_name=\n depth=\n+create_new_branch=\n+checkout_options=\n+save_in_gitmodules=\n+default_local_branch=\"master\"\n \n # The function takes at most 2 arguments. The first argument is the\n # URL that navigates to the submodule origin repo. When relative, this URL\n@@ -532,6 +537,89 @@ Use -f if you really want to add it.\" >&2\n }\n \n #\n+# Checkout the appropriate local branch for each submodule\n+#\n+cmd_checkout()\n+{\n+\t# parse $args after \"submodule ... checkout\".\n+\twhile test $# -ne 0\n+\tdo\n+\t\tcase \"$1\" in\n+\t\t-q|--quiet)\n+\t\t\tGIT_QUIET=1\n+\t\t\t;;\n+\t\t--recursive)\n+\t\t\trecursive=1\n+\t\t\t;;\n+\t\t-b|-B)\n+\t\t\tcheckout_options=\"${checkout_options} $1\"\n+\t\t\tcreate_new_branch=1\n+\t\t\t;;\n+\t\t--gitmodules)\n+\t\t\tsave_in_gitmodules=1\n+\t\t\t;;\n+\t\t--)\n+\t\t\tshift\n+\t\t\tbreak\n+\t\t\t;;\n+\t\t-*)\n+\t\t\tusage\n+\t\t\t;;\n+\t\t*)\n+\t\t\tbreak\n+\t\t\t;;\n+\t\tesac\n+\t\tshift\n+\tdone\n+\n+\ttoplevel=$(pwd)\n+\n+\tsuperproject_branch=$(get_current_branch)\n+\tif test -n \"$create_new_branch\"\n+\tthen\n+\t\tdefault_local_branch=\"${superproject_branch}\"\n+\tfi\n+\n+\t# dup stdin so that it can be restored when running the external\n+\t# command in the subshell (and a recursive call to this function)\n+\texec 3<&0\n+\n+\tmodule_list \"$@\" |\n+\twhile read mode sha1 stage sm_path\n+\tdo\n+\t\tdie_if_unmatched \"$mode\"\n+\t\tname=$(module_name \"$sm_path\") || exit\n+\t\tdisplaypath=$(relative_path \"$prefix$sm_path\")\n+\t\tif test -e \"$sm_path\"/.git\n+\t\tthen\n+\t\t\tsay \"$(eval_gettext \"Entering '\\$displaypath'\")\"\n+\t\t\tname=$(module_name \"$sm_path\")\n+\t\t\tsuper_local_branch=$(get_submodule_config \"$name\" local-branch \"${default_local_branch}\")\n+\t\t\t(\n+\t\t\t\tprefix=\"$prefix$sm_path/\"\n+\t\t\t\tclear_local_git_env\n+\t\t\t\tcd \"$sm_path\" &&\n+\t\t\t\tlocal_branch=$(get_local_branch \"${superproject_branch}\" \"${super_local_branch}\") &&\n+\t\t\t\tgit checkout ${checkout_options} \"${local_branch}\" &&\n+\t\t\t\tif test -n \"$recursive\"\n+\t\t\t\tthen\n+\t\t\t\t\tcmd_checkout\n+\t\t\t\tfi\n+\t\t\t) <&3 3<&- ||\n+\t\t\tdie \"$(eval_gettext \"Stopping at '\\$displaypath'; script returned non-zero status.\")\"\n+\t\t\tif test -n \"${save_in_gitmodules}\"\n+\t\t\tthen\n+\t\t\t\t(\n+\t\t\t\t\tlocal_branch=$(clear_local_git_env && cd \"$sm_path\" && get_current_branch) &&\n+\t\t\t\t\tgit config -f .gitmodules submodule.\"${name}\".local-branch \"${local_branch}\"\n+\t\t\t\t) ||\n+\t\t\t\tdie \"$(eval_gettext \"Could not save local-branch for '\\$displaypath'\")\"\n+\t\t\tfi\n+\t\tfi\n+\tdone\n+}\n+\n+#\n # Execute an arbitrary command sequence in each checked out\n # submodule\n #\n@@ -1378,7 +1466,7 @@ cmd_sync()\n while test $# != 0 && test -z \"$command\"\n do\n \tcase \"$1\" in\n-\tadd | foreach | init | deinit | update | status | summary | sync)\n+\tadd | checkout | foreach | init | deinit | update | status | summary | sync)\n \t\tcommand=$1\n \t\t;;\n \t-q|--quiet)\n-- \n1.8.5.2.237.g01c62c6\n"},{"id":"233028","messageId":"20140112010847.GJ29954@odin.tremily.us","threadId":"35608","inReplyTo":"cover.1389247320.git.wking@tremily.us","subject":"Tight submodule bindings (was: Preferred local submodule branches)","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-12T01:08:47Z","receivedAt":"2014-01-12T01:08:47Z","isPatch":false,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"On Wed, Jan 08, 2014 at 10:17:51PM -0800, W. Trevor King wrote:\n> In another branch of the submodule thread Francesco kicked off, I\n> mentioned that we could store the preferred local submodule branch on\n> a per-superbranch level if we used the\n> .git/modules//config for local overrides [1]. Here's\n> a patch series that greatly extends my v2 \"submodule: Respect\n> requested branch on all clones\" series [2] to also support automatic,\n> recursive submodule checkouts, as I outlined here [3].\n> \n> [1]: http://article.gmane.org/gmane.comp.version-control.git/240240\n> [2]: http://article.gmane.org/gmane.comp.version-control.git/239967\n> [3]: http://article.gmane.org/gmane.comp.version-control.git/240192\n\nWhile mulling over better ways to explain my local-branch idea, I've\ncome up with a more tightly bound model that may help break the\nsilence that has greeted the “Preferred local submodule branches”\nseries ;). That series doesn't have strong options on update\nmechanics, which leads to wishy-washy exchanges where nobody has a\nclear mental picture:\n\nOn Thu, Jan 09, 2014 at 10:40:52PM +0100, Jens Lehmann wrote:\n> Am 09.01.2014 20:55, schrieb W. Trevor King:\n> > On Thu, Jan 09, 2014 at 08:23:07PM +0100, Jens Lehmann wrote:\n> >> Am 09.01.2014 18:32, schrieb W. Trevor King:\n> >>>> when superproject branches are merged (with and without conflicts),\n> >>>\n> >>> I don't think this currently does anything to the submodule itself,\n> >>> and that makes sense to me (use 'submodule update' or my 'submodule\n> >>> checkout' if you want such effects). We should keep the current logic\n> >>> for updating the gitlinked $sha. In the case that the\n> >>> .gitmodule-configured local-branches disagree, we should give the\n> >>> usual conflict warning (and <<<===>>> markup) and let the user resolve\n> >>> the conflict in the usual way.\n> >>\n> >> For me it makes lots of sense that in recursive checkout mode the\n> >> merged submodules are already checked out (if possible) right after\n> >> a superproject merge, making another \"submodule update\" unnecessary\n> >> (the whole point of recursive update is to make \"submodule update\"\n> >> obsolete, except for \"--remote\").\n> > \n> > If you force the user to have the configured local-branch checked out\n> > before a non-checkout operations with checkout side-effects (as we\n> > currently do for other kinds of dirty trees), I think you'll avoid\n> > most (all?) of the branch-clobbering problems.\n> \n> I'm thinking that a local branch works in two directions: It should\n> make it easy to follow an upstream branch and also make changes to it\n> (and publish those) if necessary. But neither local nor upstream\n> changes take precedence, so the user should either use \"merge\" or\n> \"rebase\" as update strategy or be asked to resolve the conflict\n> manually when \"checkout\" is configured and the branches diverged.\n> Does that make sense?\n\nThe current series is only weakly bound (you can explicitly call git\nsubmodule checkout' to change to the preferred local submodule\nbranch), and the current Git is extremely weakly bound (you have to cd\ninto the submodule and change branches by hand). The following\nextrapolates the “Preferred local submodule branches” series to a\ntightly-bound ideal.\n\nGitlinked commit hash\n---------------------\n\nThe submodule model revolves around links to commits (“gitlinks”):\n\n $ git ls-tree HEAD\n 100644 blob 189fc359d3dc1ed5019b9834b93f0dfb49c5851f .gitmodules\n 160000 commit fbfa124c29362f180026bf0074630e8bd0ff4550 submod\n\nThese are effectively switchable trees. The tree referenced by commit\nfbfa124 is 492781c:\n\n $ (cd submod/ && git cat-file commit fbfa124)\n tree 492781c581d4dec380a61ef5ec69a104de448a74\n …\n\nIf you init the submodule, subsequent checkouts will check out that\ntree, just like 'git checkout' would do if you'd had a superproject\ntree like:\n\n $ git ls-tree HEAD\n 100644 blob 189fc359d3dc1ed5019b9834b93f0dfb49c5851f .gitmodules\n 040000 tree 492781c581d4dec380a61ef5ec69a104de448a74 submod\n\nFor folks who treat the submodule as a black box (and do no local\ndevelopment), switchable trees are all they care about. They can\neasily checkout (or not, with deinit), the submodule tree at a\ngitlinked hash, and everything is nice and reproducible. The fact\nthat 'submod' is stored as a commit object and not a tree, is just a\nconvenient marker for optional init/deinit/remote-update-integration\nfunctionality.\n\nAdditional metadata, the initial checkout, and syncing down\n-----------------------------------------------------------\n\nHowever, folks who do local submodule development will care about\nwhich submodule commit is responsible for that tree, because that's\ngoing to be the base of their local development. They also care about\nadditional out-of-tree information, including the branch that commit\nis on. For already-initialized submodules, there are existing places\nin the submodule config to store this configuration:\n\n1. HEAD for the checked-out branch,\n2. branch..remote → remote..url for the upstream\n subproject URL,\n4. branch..rebase (or pull.rebase) to prefer rebase over merge\n for integration,\n5. …\n\nYou need somewhere in-tree to store this destined-to-be-out-of-tree\ninformation, so that superproject developers that have not yet\ninitialized the submodule will know what values are suggested by the\nsuperproject maintainers. That's where .gitmodules comes in, because\nstoring all of this fairly static, locally overridable information in\nthe gitlink itself would be nonsensical (said Linus in 2007 [1]).\nWhen you checkout a submodule for the first time, Git should take the\ndefault information from .gitmodules and file it away in the\nsubmodule's appropriate out-of-tree config locations. The out-of-tree\ndata listed above should be stored in:\n\n1. submodule..local-branch\n2. submodule..url\n4. submodule..update\n5. …\n\nOnce you have an in-tree way to specify defaults for this out-of-tree\ninformation, you're going to have developers like me that just want to\nstick with the defaults, following them through changes. That means\nyou'd like to have the “copy .gitmodules defaults into your\nsubmodule's config” functionality that usually happens on the initial\nsubmodule checkout happen on *every superproject-initiated checkout*.\nIn fact, I think life is easier for everyone if this is the default,\nand we add a new option (submodule..sync = false) that says\n“don't overwrite optional settings in my submodule's out-of-tree\nconfig on checkout” for for folks who want to opt out. Don't worry,\nthis is not going to clobber people, because we'll be syncing the\nother way too.\n\nSyncing up\n----------\n\nIn the previous section I explained how data should flow from\n.gitmodules into out-of-tree configs. What about the other direction?\nWe currently let folks handle this by hand, but I'd prefer a tighter\nintegration between the submodule config and the superproject tree to\navoid losing work. That means changes to tracked submodule status\n(checked-out hash, checked-out branch, upstream URL, upstream branch,\ndefault integration strategy, …) should trigger dirty-tree status just\nlike uncommitted changes to in-tree files. 'git add' (or stash) on\nthe dirty submodule would store changed commit hashes in the index,\npull changed out-of-tree configs back into the in-tree .gitmodules,\nand add the new .gitmodules to the index. If the working .gitmodules\nwas already dirty (vs. the index), the add/stash should die without\nmaking any changes. If the user has disabled syncing between\n.gitmodules and the submodule's out-of-tree configs, then don't worry\nabout optional settings. Always sync the required settings, which at\nthis point would just be submodule..local-branch.\n\nPurely local metadata\n---------------------\n\nSome metadata does not make sense in the superproject tree. For\nexample, whether a submodule is interesting enough to checkout\n(init/deinit) or whether you want to auto-sync optional metadata\n.gitmodules defaults. This metadata should live in the superproject's\nout-of-tree config, and should not be stored in the in-tree\n.gitmodules. Since you *will* want to share the upstream URL, I\nproposed using an explicit submodule..active setting to store\nthe “do I care” information [2], instead of overloading\nsubmodule..url (I'd auto-sync the .gitmodule's\nsubmodule..url with the subproject's remote.origin.url unless\nthe user opted out of .gitmodules syncing).\n\nSubsequent checkouts\n--------------------\n\nNow that we have strict linking between the submodule state (both\nin-tree and out-of-tree configs) and the superproject tree (gitlink\nand .gitmodules), changing between superproject branches is really\neasy:\n\n1. Make sure the working tree is not dirty. If it is, ask the user to\n either add-and-commit or stash, and then die to let them do so.\n\n2. Checkout the new superproject branch.\n\n 2.1. For each old submodule that doesn't exist in the new branch,\n blow away the submodule directory (assuming a new-style\n .git/modules/… layout, and not an old-style submod/.git/…\n layout).\n\n 2.2. For each gitlinked submodule that didn't exist in the old\n branch, setup the submodule as if you were doing the initial\n cloning checkout (forcing a new local-branch to point at the\n gitlinked commit). If you find local out-of-tree\n *superproject* configs that conflict with the .gitmodules\n values, prefer the superproject configs. Clobber submodule\n configs and local branches at will (modulo\n submodule..sync), because any submodule configs that the\n user wanted to keep should have been added to the superproject\n branch earlier (or stashed).\n\nIntegrating other branches\n--------------------------\n\nMerges and rebases can alter the submodule's in-tree configs (and\ncreate and remove submodules). The existing logic for merging\n.gitmodules and gitlinks works well, so stick with that. In the event\nthat there are unresolvable conflicts, bail out and let the user\nresolve the conflicts and use 'git commit' to finish checking out the\nresolved state.\n\nIssues\n------\n\nI like the current submodule integration configuration:\n\n* submodule..branch (specify the remote branch to integrate, but\n I'd prefer submodule..integration-ref for clarity).\n* submodule..update (specify how to integrate it, but I'd prefer\n submodule..integration-mode for clarity).\n\nmore than the current core integration configuration:\n\n* branch..merge (with branch..remote, the branch to remote\n branch to integrate via merging).\n* branch..rebase (override branch..merge to integrate via\n rebasing).\n\nThese seem to mix the orthogonal concepts of integration target and\nintegration mode, and the divergence from the .gitmodules\nrepresentation makes syncing awkward.\n\nSummary\n-------\n\nNew .gitmodules options:\n\n* submodule..local-branch, store the submodule's HEAD, must stay\n in sync for checkouts.\n\nNew .git/config options:\n\n* submodule..active, for init/deinit.\n\n* submodule..sync, for whether you want to automatically sync\n the submodule's out-of-tree configs up to .gitmodules before\n checkout operations, and sync back from .gitmodules (possibly\n altered on the new branch) into the submodule's out-of-tree configs\n during checkout.\n\nWith this tighter binding, submodule information is either tracked in\nthe superproject, or explicitly not touched by the superproject. That\nmakes it much harder to break things or clobber a user's work, and\nalso much easier to keep submodules up to date with superproject\nchanges. Users shouldn't have to explicitly manage their submodules\nto carry out routine core tasks like checking out other branches.\n\nI see no reason to add --recurse-submodule flags to 'git checkout'\n(and merge, …). Anything that happens post-clone should recurse\nthrough submodules automatically, and use the submodule..active\nsetting to decide when recursion is desired.\n\nI think the ideal submodule-specific interface would be just:\n\n* git submodule [--quiet] add [-b ] [-f|--force] [--name ]\n [--reference ] [--] []\n* git submodule [--quiet] init [--] [...]\n* git submodule [--quiet] deinit [-f|--force] [--] ...\n* git submodule [--quiet] foreach [--recursive] \n\nThe current 'git submodule update --remote' would just be:\n\n $ git submodule foreach 'git pull'\n\nbecause all of the local-branch checkouts would have already been\nhandled. Similarly, a global push would be just:\n\n $ git submodule foreach 'git push'\n\nYou get all the per-submodule configuration (for triangular workflows,\netc.) for free, with no submodule-specific confusion.\n\nSo, is this:\n\n* Interesting enough to be worth pursuing?\n* Simple enough to be easily understood?\n\nI'd be happy to mock this up in shell, but only if anyone else would\nbe interested enough to review the implementation ;). Then I'll look\ninto integrating the preferred model (this tightly bound proposal, or\nv3's looser bindings, or ) in C, building on Jens and\nJonathan's work.\n\nCheers,\nTrevor\n\n[1]: http://article.gmane.org/gmane.comp.version-control.git/44162\n[2]: http://article.gmane.org/gmane.comp.version-control.git/211042\n\n-- \nThis email may be signed or encrypted with GnuPG (http://www.gnupg.org).\nFor more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\n"},{"id":"233059","messageId":"52D44081.7070504@web.de","threadId":"35608","inReplyTo":"20140112010847.GJ29954@odin.tremily.us","subject":"Re: Tight submodule bindings","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2014-01-13T19:37:37Z","receivedAt":"2014-01-13T19:37:37Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Thanks for the writeup, comments below.\n\nAm 12.01.2014 02:08, schrieb W. Trevor King:\n> Gitlinked commit hash\n> ---------------------\n> \n> The submodule model revolves around links to commits (“gitlinks”):\n> \n> $ git ls-tree HEAD\n> 100644 blob 189fc359d3dc1ed5019b9834b93f0dfb49c5851f .gitmodules\n> 160000 commit fbfa124c29362f180026bf0074630e8bd0ff4550 submod\n> \n> These are effectively switchable trees. The tree referenced by commit\n> fbfa124 is 492781c:\n> \n> $ (cd submod/ && git cat-file commit fbfa124)\n> tree 492781c581d4dec380a61ef5ec69a104de448a74\n> …\n> \n> If you init the submodule, subsequent checkouts will check out that\n> tree, just like 'git checkout' would do if you'd had a superproject\n> tree like:\n> \n> $ git ls-tree HEAD\n> 100644 blob 189fc359d3dc1ed5019b9834b93f0dfb49c5851f .gitmodules\n> 040000 tree 492781c581d4dec380a61ef5ec69a104de448a74 submod\n> \n> For folks who treat the submodule as a black box (and do no local\n> development), switchable trees are all they care about. They can\n> easily checkout (or not, with deinit), the submodule tree at a\n> gitlinked hash, and everything is nice and reproducible. The fact\n> that 'submod' is stored as a commit object and not a tree, is just a\n> convenient marker for optional init/deinit/remote-update-integration\n> functionality.\n\nBut there are users (like me) who do not treat submodules as\nblack boxes and nonetheless do development in them with update\nset to checkout (after creating a feature branch of course ;-).\n\n> Additional metadata, the initial checkout, and syncing down\n> -----------------------------------------------------------\n> \n> However, folks who do local submodule development will care about\n> which submodule commit is responsible for that tree, because that's\n> going to be the base of their local development. They also care about\n> additional out-of-tree information, including the branch that commit\n> is on. For already-initialized submodules, there are existing places\n> in the submodule config to store this configuration:\n> \n> 1. HEAD for the checked-out branch,\n> 2. branch..remote → remote..url for the upstream\n> subproject URL,\n> 4. branch..rebase (or pull.rebase) to prefer rebase over merge\n> for integration,\n> 5. …\n> \n> You need somewhere in-tree to store this destined-to-be-out-of-tree\n> information, so that superproject developers that have not yet\n> initialized the submodule will know what values are suggested by the\n> superproject maintainers. That's where .gitmodules comes in, because\n> storing all of this fairly static, locally overridable information in\n> the gitlink itself would be nonsensical (said Linus in 2007 [1]).\n> When you checkout a submodule for the first time, Git should take the\n> default information from .gitmodules and file it away in the\n> submodule's appropriate out-of-tree config locations.\n\nI disagree, that only makes sense for the URL setting (and this\ncurrently only happens with the update setting, which I intend to\nchange). Everything else should be taken from .gitmodules unless\nthe user wants to override it. The only setting I'm not so sure\nabout is the local branch setting, as that might have to propagate\ninto the submodule.\n\n> The out-of-tree\n> data listed above should be stored in:\n> \n> 1. submodule..local-branch\n> 2. submodule..url\n> 4. submodule..update\n> 5. …\n> \n> Once you have an in-tree way to specify defaults for this out-of-tree\n> information, you're going to have developers like me that just want to\n> stick with the defaults, following them through changes. That means\n> you'd like to have the “copy .gitmodules defaults into your\n> submodule's config” functionality that usually happens on the initial\n> submodule checkout happen on *every superproject-initiated checkout*.\n\nYou don't need to copy it every time when you simply use the\n.gitmodules file as fallback, no?\n\n> In fact, I think life is easier for everyone if this is the default,\n> and we add a new option (submodule..sync = false) that says\n> “don't overwrite optional settings in my submodule's out-of-tree\n> config on checkout” for for folks who want to opt out. Don't worry,\n> this is not going to clobber people, because we'll be syncing the\n> other way too.\n\nYet another flag to make peoples life easier? I don't think so ;-)\n\n> Syncing up\n> ----------\n> \n> In the previous section I explained how data should flow from\n> .gitmodules into out-of-tree configs. What about the other direction?\n> We currently let folks handle this by hand, but I'd prefer a tighter\n> integration between the submodule config and the superproject tree to\n> avoid losing work. That means changes to tracked submodule status\n> (checked-out hash, checked-out branch, upstream URL, upstream branch,\n> default integration strategy, …) should trigger dirty-tree status just\n> like uncommitted changes to in-tree files. 'git add' (or stash) on\n> the dirty submodule would store changed commit hashes in the index,\n> pull changed out-of-tree configs back into the in-tree .gitmodules,\n> and add the new .gitmodules to the index. If the working .gitmodules\n> was already dirty (vs. the index), the add/stash should die without\n> making any changes. If the user has disabled syncing between\n> .gitmodules and the submodule's out-of-tree configs, then don't worry\n> about optional settings. Always sync the required settings, which at\n> this point would just be submodule..local-branch.\n\nSuch a logic might make sense. And without copying stuff from\n.gitmodules someplace else it becomes even easier ;-)\n\n> Purely local metadata\n> ---------------------\n> \n> Some metadata does not make sense in the superproject tree. For\n> example, whether a submodule is interesting enough to checkout\n> (init/deinit) or whether you want to auto-sync optional metadata\n> .gitmodules defaults. This metadata should live in the superproject's\n> out-of-tree config, and should not be stored in the in-tree\n> .gitmodules.\n\nNot always. It makes a lot of sense to let upstream mark a\nsubmodule as \"too big and you won't need it anyway\" in the\n.gitmodules file.\n\n> Since you *will* want to share the upstream URL, I\n> proposed using an explicit submodule..active setting to store\n> the “do I care” information [2], instead of overloading\n> submodule..url (I'd auto-sync the .gitmodule's\n> submodule..url with the subproject's remote.origin.url unless\n> the user opted out of .gitmodules syncing).\n\nThat is wrong as it would break horribly when you check out an\nold commit with a now dead submodule URL and that gets automatically\nsynced.\n\n> Subsequent checkouts\n> --------------------\n> \n> Now that we have strict linking between the submodule state (both\n> in-tree and out-of-tree configs) and the superproject tree (gitlink\n> and .gitmodules), changing between superproject branches is really\n> easy:\n> \n> 1. Make sure the working tree is not dirty. If it is, ask the user to\n> either add-and-commit or stash, and then die to let them do so.\n\nThis condition is too hard, relax that to \"a trivial merge can\nswitch from current state to target state\" and make it behave just\nlike branch switching in the superproject. After all submodules\nshould behave as much as possible like content of the superproject.\n\n> 2. Checkout the new superproject branch.\n> \n> 2.1. For each old submodule that doesn't exist in the new branch,\n> blow away the submodule directory (assuming a new-style\n> .git/modules/… layout, and not an old-style submod/.git/…\n> layout).\n\nYep.\n\n> 2.2. For each gitlinked submodule that didn't exist in the old\n> branch, setup the submodule as if you were doing the initial\n> cloning checkout (forcing a new local-branch to point at the\n> gitlinked commit). If you find local out-of-tree\n> *superproject* configs that conflict with the .gitmodules\n> values, prefer the superproject configs.\n\nYup, our working title for that is \"autoinit\".\n\n> Clobber submodule\n> configs and local branches at will (modulo\n> submodule..sync), because any submodule configs that the\n> user wanted to keep should have been added to the superproject\n> branch earlier (or stashed).\n\nI don't think I like this part, but I admit I do not fully understand\nwhat you mean here. Clobbering stuff the user did doesn't sound very\nnice.\n\n> Integrating other branches\n> --------------------------\n> \n> Merges and rebases can alter the submodule's in-tree configs (and\n> create and remove submodules). The existing logic for merging\n> .gitmodules and gitlinks works well, so stick with that. In the event\n> that there are unresolvable conflicts, bail out and let the user\n> resolve the conflicts and use 'git commit' to finish checking out the\n> resolved state.\n\nAgreed.\n\n> Issues\n> ------\n> \n> I like the current submodule integration configuration:\n> \n> * submodule..branch (specify the remote branch to integrate, but\n> I'd prefer submodule..integration-ref for clarity).\n> * submodule..update (specify how to integrate it, but I'd prefer\n> submodule..integration-mode for clarity).\n\nBut we won't rename those now.\n\n> more than the current core integration configuration:\n> \n> * branch..merge (with branch..remote, the branch to remote\n> branch to integrate via merging).\n> * branch..rebase (override branch..merge to integrate via\n> rebasing).\n> \n> These seem to mix the orthogonal concepts of integration target and\n> integration mode, and the divergence from the .gitmodules\n> representation makes syncing awkward.\n\nI'm still hoping we might come up with a solution that doesn't need\nthe syncing.\n\n> Summary\n> -------\n> \n> New .gitmodules options:\n> \n> * submodule..local-branch, store the submodule's HEAD, must stay\n> in sync for checkouts.\n\nI'm still not convinced that the current branch setting couldn't be\nextended to carry that information, but no objections against\nconfiguring such a branch. But what do you mean with \"must stay in\nsync for checkouts\"?\n\n> New .git/config options:\n> \n> * submodule..active, for init/deinit.\n\nI understand an option for automatic init (autoinit), but not for\nautomatic deinit. Is the latter really useful?\n\n> * submodule..sync, for whether you want to automatically sync\n> the submodule's out-of-tree configs up to .gitmodules before\n> checkout operations, and sync back from .gitmodules (possibly\n> altered on the new branch) into the submodule's out-of-tree configs\n> during checkout.\n\nNot needed if you use .gitmodules as fallback.\n\n> With this tighter binding, submodule information is either tracked in\n> the superproject, or explicitly not touched by the superproject. That\n> makes it much harder to break things or clobber a user's work, and\n> also much easier to keep submodules up to date with superproject\n> changes. Users shouldn't have to explicitly manage their submodules\n> to carry out routine core tasks like checking out other branches.\n\nAgreed.\n\n> I see no reason to add --recurse-submodule flags to 'git checkout'\n> (and merge, …). Anything that happens post-clone should recurse\n> through submodules automatically, and use the submodule..active\n> setting to decide when recursion is desired.\n\nBackwards compatibility and testing. Let's first implement that and\nprovide a config option to enable it for real world testing, and then\nlet's discuss changing the default later.\n\n> I think the ideal submodule-specific interface would be just:\n> \n> * git submodule [--quiet] add [-b ] [-f|--force] [--name ]\n> [--reference ] [--] []\n> * git submodule [--quiet] init [--] [...]\n> * git submodule [--quiet] deinit [-f|--force] [--] ...\n> * git submodule [--quiet] foreach [--recursive] \n\nOk.\n\n> The current 'git submodule update --remote' would just be:\n> \n> $ git submodule foreach 'git pull'\n> \n> because all of the local-branch checkouts would have already been\n> handled.\n\nNope, that does different things to submodules where \"branch\" isn't\nconfigured, right?\n\n> Similarly, a global push would be just:\n> \n> $ git submodule foreach 'git push'\n\nWhat's wrong with:\n\n$ git push --recurse-submodules=on-demand\n\nAnd it'll push the superproject at the same time. Extra points for\nalready being implemented ;-)\n\n> You get all the per-submodule configuration (for triangular workflows,\n> etc.) for free, with no submodule-specific confusion.\n> \n> So, is this:\n> \n> * Interesting enough to be worth pursuing?\n> * Simple enough to be easily understood?\n\nThe thoughts about the branch workflow are really interesting. Some\nother proposals overshoot a bit in my opinion ;-)\n\n> I'd be happy to mock this up in shell, but only if anyone else would\n> be interested enough to review the implementation ;). Then I'll look\n> into integrating the preferred model (this tightly bound proposal, or\n> v3's looser bindings, or ) in C, building on Jens and\n> Jonathan's work.\n\nThe update modes (cleaning removed submodules and creating new ones)\nare better handled in my recursive checkout series. But I believe we\ncan at least prototype the branch handling in shell.\n"},{"id":"233061","messageId":"20140113200724.GP10613@odin.tremily.us","threadId":"35608","inReplyTo":"52D44081.7070504@web.de","subject":"Re: Tight submodule bindings","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-13T20:07:24Z","receivedAt":"2014-01-13T20:07:24Z","isPatch":false,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"On Mon, Jan 13, 2014 at 08:37:37PM +0100, Jens Lehmann wrote:\n> Am 12.01.2014 02:08, schrieb W. Trevor King:\n> > For folks who treat the submodule as a black box (and do no local\n> > development), switchable trees are all they care about. They can\n> > easily checkout (or not, with deinit), the submodule tree at a\n> > gitlinked hash, and everything is nice and reproducible. The fact\n> > that 'submod' is stored as a commit object and not a tree, is just\n> > a convenient marker for optional\n> > init/deinit/remote-update-integration functionality.\n> \n> But there are users (like me) who do not treat submodules as\n> black boxes and nonetheless do development in them with update\n> set to checkout (after creating a feature branch of course ;-).\n\nI'm still not clear on how this works for you ;). Can you sketch out\nan example shell history showing how you use checkout updates to do\nthis?\n\n> > When you checkout a submodule for the first time, Git should take\n> > the default information from .gitmodules and file it away in the\n> > submodule's appropriate out-of-tree config locations.\n> \n> I disagree, that only makes sense for the URL setting (and this\n> currently only happens with the update setting, which I intend to\n> change). Everything else should be taken from .gitmodules unless\n> the user wants to override it. The only setting I'm not so sure\n> about is the local branch setting, as that might have to propagate\n> into the submodule.\n\nI think copying into the submodule's out-of-tree config is the way to\ngo, because users won't always be driving the submodule from the\nsuperproject. If the settings are in the submodule's out-of-tree\nconfig, everything will be consistent betwee stuff run from the\nsuperproject and stuff run from submodule itself. It also allows us\nto use familiar configuration commands inside the submodule, and have\nthose automatically mapped back into the .gitmodules file for us.\n\n> > In fact, I think life is easier for everyone if this is the\n> > default, and we add a new option (submodule..sync = false)\n> > that says “don't overwrite optional settings in my submodule's\n> > out-of-tree config on checkout” for for folks who want to opt out.\n> > Don't worry, this is not going to clobber people, because we'll be\n> > syncing the other way too.\n> \n> Yet another flag to make peoples life easier? I don't think so ;-)\n\nI'm fine if there is no opt-out, and the syncing is mandatory, but I\nimagine that folks who want a local (unshared, not in .gitmodules) URL\nwould complain.\n\n> > Purely local metadata\n> > ---------------------\n> > \n> > Some metadata does not make sense in the superproject tree. For\n> > example, whether a submodule is interesting enough to checkout\n> > (init/deinit) or whether you want to auto-sync optional metadata\n> > .gitmodules defaults. This metadata should live in the\n> > superproject's out-of-tree config, and should not be stored in the\n> > in-tree .gitmodules.\n> \n> Not always. It makes a lot of sense to let upstream mark a\n> submodule as \"too big and you won't need it anyway\" in the\n> .gitmodules file.\n\nGood. Then there's no need for this special class of settings.\n\n> > Since you *will* want to share the upstream URL, I proposed using\n> > an explicit submodule..active setting to store the “do I\n> > care” information [2], instead of overloading submodule..url\n> > (I'd auto-sync the .gitmodule's submodule..url with the\n> > subproject's remote.origin.url unless the user opted out of\n> > .gitmodules syncing).\n> \n> That is wrong as it would break horribly when you check out an old\n> commit with a now dead submodule URL and that gets automatically\n> synced.\n\nIf you've already checked out the submodule with a current URL, you\nshould already have the old commit locally, and Git will use it\nwithout trying to re-fetch from the broken old URL.\n\n> > Subsequent checkouts\n> > --------------------\n> > \n> > Now that we have strict linking between the submodule state (both\n> > in-tree and out-of-tree configs) and the superproject tree (gitlink\n> > and .gitmodules), changing between superproject branches is really\n> > easy:\n> > \n> > 1. Make sure the working tree is not dirty. If it is, ask the user to\n> > either add-and-commit or stash, and then die to let them do so.\n> \n> This condition is too hard, relax that to \"a trivial merge can\n> switch from current state to target state\" and make it behave just\n> like branch switching in the superproject. After all submodules\n> should behave as much as possible like content of the superproject.\n\nSounds good to me.\n\n> > Clobber submodule\n> > configs and local branches at will (modulo\n> > submodule..sync), because any submodule configs that\n> > the user wanted to keep should have been added to the\n> > superproject branch earlier (or stashed).\n> \n> I don't think I like this part, but I admit I do not fully\n> understand what you mean here. Clobbering stuff the user did doesn't\n> sound very nice.\n\nIt's fine because we forced them to commit or stash any (not trivially\nmergable) changes before starting the checkout command.\n\n> > Summary\n> > -------\n> > \n> > New .gitmodules options:\n> > \n> > * submodule..local-branch, store the submodule's HEAD, must\n> > stay in sync for checkouts.\n> \n> I'm still not convinced that the current branch setting couldn't be\n> extended to carry that information, but no objections against\n> configuring such a branch. But what do you mean with \"must stay in\n> sync for checkouts\"?\n\nThat checkout-inducing commands should die if the .gitmodule's\nlocal-branch and the submodule's HEAD don't match.\n\n> > New .git/config options:\n> > \n> > * submodule..active, for init/deinit.\n> \n> I understand an option for automatic init (autoinit), but not for\n> automatic deinit. Is the latter really useful?\n\nThis isn't auto-anything. This is just “I think the submodule is\ninteresting, please turn it on” (i.e. I ran “git submodule init\n”).\n\nOnly active submodules should get all the syncing, setup, and teardown\nlogic that goes along with submodule checkout. Inactive submodules\nare ignored.\n\n> > I see no reason to add --recurse-submodule flags to 'git checkout'\n> > (and merge, …). Anything that happens post-clone should recurse\n> > through submodules automatically, and use the\n> > submodule..active setting to decide when recursion is\n> > desired.\n> \n> Backwards compatibility and testing. Let's first implement that and\n> provide a config option to enable it for real world testing, and\n> then let's discuss changing the default later.\n\nOk.\n\n> > The current 'git submodule update --remote' would just be:\n> > \n> > $ git submodule foreach 'git pull'\n> > \n> > because all of the local-branch checkouts would have already been\n> > handled.\n> \n> Nope, that does different things to submodules where \"branch\" isn't\n> configured, right?\n\nIt does the same thing. Without submodule..branch configured,\nyou just integrate the subproject's master.\n\n> > Similarly, a global push would be just:\n> > \n> > $ git submodule foreach 'git push'\n> \n> What's wrong with:\n> \n> $ git push --recurse-submodules=on-demand\n> \n> And it'll push the superproject at the same time. Extra points for\n> already being implemented ;-)\n\nThat's a strong argument ;). I still don't think the new-in-1.7.4 UI\nchange will add value. The new-in-1.7.7 --recurse-submodules=check\nwould still be useful.\n\n> > I'd be happy to mock this up in shell, but only if anyone else\n> > would be interested enough to review the implementation ;). Then\n> > I'll look into integrating the preferred model (this tightly bound\n> > proposal, or v3's looser bindings, or ) in C,\n> > building on Jens and Jonathan's work.\n> \n> The update modes (cleaning removed submodules and creating new ones)\n> are better handled in my recursive checkout series. But I believe we\n> can at least prototype the branch handling in shell.\n\nI'll prototype it, and keep trying to convince you about the syncing\n;). I think the main arguments for syncing are:\n\n* No divergent configs between superproject-initiated actions and\n submodule-initiated actions.\n* No work clobbered, or accidentally uncommitted, due to syncing\n submodule -> superproject before checkout-inducing commands.\n\nCheers,\nTrevor\n\n-- \nThis email may be signed or encrypted with GnuPG (http://www.gnupg.org).\nFor more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\n"},{"id":"233066","messageId":"xmqqk3e35y3p.fsf@gitster.dls.corp.google.com","threadId":"35608","inReplyTo":"20140112010847.GJ29954@odin.tremily.us","subject":"Re: Tight submodule bindings","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-01-13T22:13:46Z","receivedAt":"2014-01-13T22:13:46Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"W. Trevor King\" writes:\n\n> Additional metadata, the initial checkout, and syncing down\n> -----------------------------------------------------------\n>\n> However, folks who do local submodule development will care about\n> which submodule commit is responsible for that tree, because that's\n> going to be the base of their local development. They also care about\n> additional out-of-tree information, including the branch that commit\n> is on.\n\nWell, please step back a bit.\n\nThey do not have to care about what local branch they use to build\nfollow-up work based on that commit. In fact, they would want to be\nable to develop more than one histories on top, which means more\nthan one branches they can name themselves.\n\nThe only thing they care about is where the result of their\ndevelopment _goes_, that is the URL and the branch of the remote\nthey are pushing back to.\n\nI have a feeling that this is not specific for submodules---if you\ndid this:\n\n\tgit init here\n cd here\n git fetch $there master\n git reset --hard FETCH_HEAD\n\nand are given the resulting working tree to start hacking on, you\nwould not know where the history came from, or where your result\nwants to go. \n\nSo \"the branch that commit is on\" is a wrong thing to focus on.\n\"The branch the history built on top of the commit wants to go\" may\nbe closer and these two are different.\n\n> For already-initialized submodules, there are existing places\n> in the submodule config to store this configuration:\n>\n> 1. HEAD for the checked-out branch,\n> 2. branch..remote → remote..url for the upstream\n> subproject URL,\n> 4. branch..rebase (or pull.rebase) to prefer rebase over merge\n> for integration,\n> 5. …\n\nWhat happened to 3 ;-)?\n\nAnd also branch..merge may say on which of _their_ branch the\ncommit you learn in the superproject tree would be found. If you\nare using centralized workflow, that would be the branch at your\ncentral repository to update with your push, too.\n\nIn any case, \"local-branch\" is wrong from two aspects:\n\n 1. (obvious) It does not follow our naming convention not to use\n dashed-names for configuration variables.\n\n 2. You do not care about the names you use locally. The only thing\n you care about is where people meet at the central repository,\n i.e. where your result is pushed to.\n\n\n> Syncing up\n> ----------\n>\n> In the previous section I explained how data should flow from\n> .gitmodules into out-of-tree configs.\n\ns/should/you think should/, I think, but another way may be not to\ncopy and read from there, which may be a lot simpler. Then upon\nswitching branches of top-level superproject (which would update\n.gitmodules to the version on the new branch), you may get different\nsettings automatically. But see below.\n\n> ... Since you *will* want to share the upstream URL, I\n> proposed using an explicit submodule..active setting to store\n> the “do I care” information [2], instead of overloading\n> submodule..url (I'd auto-sync the .gitmodule's\n> submodule..url with the subproject's remote.origin.url unless\n> the user opted out of .gitmodules syncing).\n\nIt may not be a good idea to blindly update to whatever happens to\nbe in .gitmodules, especially once submodule.*.url is initialized.\n\nI think we would need a bit more sophisticated mechanism than \"use\nfrom .git/config if set, otherwise use from .gitmodules\", at least\nfor the URL. It may not be limited to the URL, and other pieces\nof metainformation about submodules may need similar handling, but\nI'd refrain from extending the scope of discussion needlessly at\nthis point.\n\nImagine that your embedded appliance project used to use a submodule\nfrom git://k.org/linux-2.6 as its kernel component and now the\nupstream of it is instead called just git://k.org/linux; the URL\nspecified by submodule.kernel.url in .gitmodules for the entry\nsubmodule.kernel.path=kernel would have changed from the former to\nthe latter sometime in the superproject's history. Switching back\nto an old version in the superproject to fix an old bug in the\nmaintenance track of the superproject would still want to push\nassociated fixes to the kernel to k.org/linux, not linux-2.6, the\nlatter of which may now be defunct [*1*]. One way to make it work\nsemi-automatically is to keep track of what the user has seen in\n.gitmodules and offer chances to update entries in .git/config. If\nyou cloned the superproject recently, you would only know about the\nnew git://k.org/linux URL and that would be copied to .git/config\n(which the current code does). In addition, you would remember that\nwe saw git://k.org/linux URL (which the current code does not).\nUpon switching back to an old version, we could notice that the URL\nin .gitmodules, which is git://k.org/linux-2.6, is not something the\nuser has seen, and at that point we could ask the user to tell us\nwhat URL should be used, record the answer _and_ the fact that we\nsaw that old URL as well. Then until the superproject updates the\nURL the next time to a value that we have never seen, the user can\nkeep using the right URL without being asked [*2*].\n\n\n> 2. Checkout the new superproject branch.\n>\n> 2.1. For each old submodule that doesn't exist in the new branch,\n> blow away the submodule directory (assuming a new-style\n> .git/modules/… layout, and not an old-style submod/.git/…\n> layout).\n\nSure.\n\n> 2.2. For each gitlinked submodule that didn't exist in the old\n> branch, setup the submodule as if you were doing the initial\n> cloning checkout (forcing a new local-branch to point at the\n> gitlinked commit). If you find local out-of-tree\n> *superproject* configs that conflict with the .gitmodules\n> values, prefer the superproject configs. Clobber submodule\n> configs and local branches at will (modulo\n> submodule..sync), because any submodule configs that the\n> user wanted to keep should have been added to the superproject\n> branch earlier (or stashed).\n\nSee above.\n\n\n[Footnote]\n\n*1* On the other hand, the switch of the submodule URL in the\nsuperproject may have been between two separate projects (e.g. you\nused to build your embedded appliance using BSD kernel but recent\nversions use Linux kernel)---in such a project, you would want the\nsubmodule URL to follow what is in .gitmodules when you switch\nbetween old and new versions in the superproject. But our\nrecommendation in such a case is to use different names for\nsubmodules that is bound at the same path in the superproject so\nthat we can keep them as two separate repositories in .git/mdoules/\nof the superproject. So at least for the URL, there is no reason to\nuse the old version that appears in .gitmodules of the superproject\neven when you checkout an old version of it.\n\n*2* This \"remembering\" may have to be more than \"have we seen this\"\none-bit per different values. For URL, I think the one-bit is\nenough, but for other things, it might make sense to keep track of\n\"In the version of superproject with X in .gitmodules, the user\nwants to use value Y\" for each values X the user has seen.\n"},{"id":"233078","messageId":"20140114024426.GB23617@odin.tremily.us","threadId":"35608","inReplyTo":"xmqqk3e35y3p.fsf@gitster.dls.corp.google.com","subject":"Re: Tight submodule bindings","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-14T02:44:26Z","receivedAt":"2014-01-14T02:44:26Z","isPatch":false,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"On Mon, Jan 13, 2014 at 02:13:46PM -0800, Junio C Hamano wrote:\n> \"W. Trevor King\" writes:\n> \n> > Additional metadata, the initial checkout, and syncing down\n> > -----------------------------------------------------------\n> >\n> > However, folks who do local submodule development will care about\n> > which submodule commit is responsible for that tree, because\n> > that's going to be the base of their local development. They also\n> > care about additional out-of-tree information, including the\n> > branch that commit is on.\n> \n> Well, please step back a bit.\n> \n> They do not have to care about what local branch they use to build\n> follow-up work based on that commit.\n\nThey do if they want to checkout the banch out again later, before\npushing it somewhere public.\n\n> In fact, they would want to be able to develop more than one\n> histories on top, which means more than one branches they can name\n> themselves.\n\nAgreed, bug for each superproject branch they will still have a single\nsubmodule branch that should be checked out by default when they\ncheckout that superproject branch.\n\n> The only thing they care about is where the result of their\n> development _goes_, that is the URL and the branch of the remote\n> they are pushing back to.\n\nMaybe they're just doing local development? I think the remote\nbranch(es) you pull from and push to are important, but not the only\nthing you might care about.\n\n> I have a feeling that this is not specific for submodules---if you\n> did this:\n> \n> \tgit init here\n> cd here\n> git fetch $there master\n> git reset --hard FETCH_HEAD\n> \n> and are given the resulting working tree to start hacking on, you\n> would not know where the history came from, or where your result\n> wants to go. \n> \n> So \"the branch that commit is on\" is a wrong thing to focus on.\n> \"The branch the history built on top of the commit wants to go\" may\n> be closer and these two are different.\n\nThat makes sense. I don't think the former (as distinct from the\nlatter) is of any interest to anybody. I don't care what the branch\nname was when the past history was developed. I don't even really\ncare about the new branch name. I do care that checking out a\nsuperproject branch gives me the same branch (with pull/push configs,\netc.) that I had the last time I was on that superproject branch.\n\n> > For already-initialized submodules, there are existing places\n> > in the submodule config to store this configuration:\n> >\n> > 1. HEAD for the checked-out branch,\n> > 2. branch..remote → remote..url for the upstream\n> > subproject URL,\n> > 4. branch..rebase (or pull.rebase) to prefer rebase over merge\n> > for integration,\n> > 5. …\n> \n> What happened to 3 ;-)?\n\nI can't count? :p\n\n> In any case, \"local-branch\" is wrong from two aspects:\n> \n> 1. (obvious) It does not follow our naming convention not to use\n> dashed-names for configuration variables.\n\nI'll use localBranch in my mockup ;). Although skimming through\nconfig.txt shows a number of alllowercase settings as well as\ncamelCase.\n\n> 2. You do not care about the names you use locally. The only thing\n> you care about is where people meet at the central repository,\n> i.e. where your result is pushed to.\n\nI also care about local-checkout consistency, as described above.\n\n> > Syncing up\n> > ----------\n> >\n> > In the previous section I explained how data should flow from\n> > .gitmodules into out-of-tree configs.\n> \n> s/should/you think should/, I think, but another way may be not to\n> copy and read from there, which may be a lot simpler. Then upon\n> switching branches of top-level superproject (which would update\n> .gitmodules to the version on the new branch), you may get different\n> settings automatically.\n\nThat only works for superproject-level commands that know about the\n.gitmodules file. If you cd into the submodule and work there\ndirectly, your actions will be using the submodule's out-of-tree\nconfig. I think most of the time folks will want those out-of-tree\nconfigs to match the settings in the superproject's .gitmodules, hence\nthe submodule..sync defaulting to true.\n\n> > ... Since you *will* want to share the upstream URL, I proposed\n> > using an explicit submodule..active setting to store the “do\n> > I care” information [2], instead of overloading\n> > submodule..url (I'd auto-sync the .gitmodule's\n> > submodule..url with the subproject's remote.origin.url\n> > unless the user opted out of .gitmodules syncing).\n> \n> It may not be a good idea to blindly update to whatever happens to\n> be in .gitmodules, especially once submodule.*.url is initialized.\n\nWhy not? We're blindly updating it to the value that was previously\npulled out of the submodule's out-of-tree config. If the user doesn't\nlike what's happening to .gitmodules upstream and doesn't want to keep\na patched version locally, they can always turn off\nsubmodule..sync.\n\n> Imagine that your embedded appliance project used to use a submodule\n> from git://k.org/linux-2.6 as its kernel component and now the\n> upstream of it is instead called just git://k.org/linux; the URL\n> specified by submodule.kernel.url in .gitmodules for the entry\n> submodule.kernel.path=kernel would have changed from the former to\n> the latter sometime in the superproject's history. Switching back\n> to an old version in the superproject to fix an old bug in the\n> maintenance track of the superproject would still want to push\n> associated fixes to the kernel to k.org/linux, not linux-2.6, the\n> latter of which may now be defunct [*1*].\n\nThe checkout would work (because the old gitlinked commit is already\nin the local repository), but the push would not. I don't think it\nwould be difficult to recover from that manually (and just specify the\nfull URL when pushing). You could also:\n\n1. Commit your fix.\n2. Checkout a more modern superproject branch (which will load the\n current URL into the submodule's config).\n3. Push the fix.\n4. Continue to work on the modern branch.\n\nThat doesn't sound much more difficult than the ideal:\n\n1. Commit your fix.\n2. Push the fix.\n3. Checkout a more modern superproject branch (which will load the\n current URL into the submodule's config).\n4. Continue to work on the modern branch.\n\nIf you expect to be back making more superproject/subproject joint\nbugfixes in future, I think it makes sense to start a maintenance\nbranch of the superproject that updates the .gitmodules URL to point\nat the modern location.\n\nCheers,\nTrevor\n\n-- \nThis email may be signed or encrypted with GnuPG (http://www.gnupg.org).\nFor more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\n"},{"id":"233118","messageId":"CALas-iiLQHVpH9-KbWHVJzYSho3cV-ELmG4+R_8XGT7Pb+=gWQ@mail.gmail.com","threadId":"35608","inReplyTo":"75e8c98df73273c2c8174e726e3fc961fbebd6a7.1389247320.git.wking@tremily.us","subject":"Re: [RFC v3 3/4] submodule: Teach 'add' about a configurable local-branch","fromName":"Francesco Pretto","fromEmail":"ceztko@gmail.com","sentAt":"2014-01-15T00:18:12Z","receivedAt":"2014-01-15T00:18:12Z","isPatch":false,"sender":{"key":"ceztko@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3037449?v=4"},"body":"I've matured this opinion about \"local-branch\" some days ago, but I\ncouldn't join the discussion because I was extremely busy. Hope it's\nis still current (and correct).\n\n2014/1/9 W. Trevor King \n>\n> @@ -339,7 +339,19 @@ module_clone()\n> echo \"gitdir: $rel/$a\" >\"$sm_path/.git\"\n>\n> rel=$(echo $a | sed -e 's|[^/][^/]*|..|g')\n> - (clear_local_git_env; cd \"$sm_path\" && GIT_WORK_TREE=. git config core.worktree \"$rel/$b\")\n> + superproject_branch=$(get_current_branch)\n> + default_local_branch=$(get_submodule_config \"$sm_name\" local-branch)\n> + (\n> + clear_local_git_env\n> + cd \"$sm_path\" &&\n> + GIT_WORK_TREE=. git config core.worktree \"$rel/$b\" &&\n> + local_branch=$(get_local_branch \"${superproject_branch}\" \"${default_local_branch}\") &&\n> + # ash fails to wordsplit ${branch:+-b \"$branch\"...}\n> + case \"$branch\" in\n> + '') git checkout -f -q -B \"$local_branch\" ;;\n> + ?*) git checkout -f -q -B \"$local_branch\" \"origin/$branch\" ;;\n> + esac\n> + ) || die \"$(eval_gettext \"Unable to checkout submodule '\\$sm_path'\")\"\n> }\n>\n\nalso\n\n2014/1/8 W. Trevor King :\n> To elaborate the idea I sketched out here [2], say\n> you want:\n>\n> Superproject branch Submodule branch Upstream branch\n> =================== ================ ===============\n> master master master\n> super-feature master master\n> my-feature my-feature master\n> other-feature other-feature other-feature\n>\n> That's only going to work with per-superproject-branch configs for\n> both the local and remote branches. Using the same name for both\n> local and remote branches does not work.\n>\n> Let me motivate each of the combinations in the above table:\n>\n> * master, master, master: The stable trunk.\n> * super-feature, master, master: A superproject feature that works\n> with the stock submodule.\n> * my-feature, my-feature, master: A superproject feature that needs an\n> improved submodule, but wants to integrate upstream master changes\n> during development.\n> * other-feature, other-feature, other-feature: A superproject feature\n> that needs an improved submodule, and wants to integrate\n> other-feature changes that are also being developed upstream\n\nThe \"local-branch\" feature means to my brain the following: I,\nmaintainer, decide for you, developer, what name should be the branch\nyou are checking out. While, in general, it makes sense for a\ndeveloper to switch to a differently named \"feature branch\" that can\npull the original remote branch if he's actively developing (on any\nrepository, not only a submodule), this leads me to the following\nquestions: would it be good to introduce such enforcement? Do we allow\nsomething similar on regular repositories? In short I believe this\nworkflow may reflect a personal attitude. In that case I'm unsure if\ngit should ease it so specifically.\n"},{"id":"233119","messageId":"20140115010208.GF2647@odin.tremily.us","threadId":"35608","inReplyTo":"CALas-iiLQHVpH9-KbWHVJzYSho3cV-ELmG4+R_8XGT7Pb+=gWQ@mail.gmail.com","subject":"Re: [RFC v3 3/4] submodule: Teach 'add' about a configurable local-branch","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2014-01-15T01:02:08Z","receivedAt":"2014-01-15T01:02:08Z","isPatch":false,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"On Wed, Jan 15, 2014 at 01:18:12AM +0100, Francesco Pretto wrote:\n> I've matured this opinion about \"local-branch\" some days ago, but I\n> couldn't join the discussion because I was extremely busy. Hope it's\n> is still current (and correct).\n\nI think the discussion is still open, but actions are postponed until\n'checkout --recurse-submodules' lands [1].\n\n> The \"local-branch\" feature means to my brain the following: I,\n> maintainer, decide for you, developer, what name should be the\n> branch you are checking out.\n\nThe goal is to have “I, your faithful Git command, check out for you,\noh wise developer, the superproject branch you requested, along with\nthe local submodule branch associated with that super project branch.”\n;) Maybe that would be more clear if the localBranch settings are\npurely local (stored only in out-of-tree configs), and not contained\nin the superproject's .gitmodules file? As this series stands, that\nwould just drop step 3 from the lookup chain [2]. That step is below\nall the local out-of-tree locations though, and I see no\nnon-psychological reason to keep folks from sharing reasonable default\nnames for local branches.\n\nCheers,\nTrevor\n\n[1]: http://article.gmane.org/gmane.comp.version-control.git/240420\n[2]: http://article.gmane.org/gmane.comp.version-control.git/240251\n\n-- \nThis email may be signed or encrypted with GnuPG (http://www.gnupg.org).\nFor more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\n"}]}