{"thread":{"id":"5796","subject":"[RFC] submodule prototype","startedAt":"2006-10-03T16:26:42Z","lastAt":"2006-10-03T16:26:42Z","messageCount":1,"participants":["Martin Waitz"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"28157","messageId":"20061003162642.GL2871@admingilde.org","threadId":"5796","inReplyTo":null,"subject":"[RFC] submodule prototype","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-10-03T16:26:42Z","receivedAt":"2006-10-03T16:26:42Z","isPatch":false,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nso, now my little submodule prototype did it's first pull correctly\nso I think I should share some ideas here.\n\nhttp://git.admingilde.org/tali/git.git/module\n\nA submodule here is implemented as a more-or-less normal GIT repository.\nThe object directory is shared with the parent and all refs are visible\nin both the parent and all submodules (via refs/parent and refs/module)\nso that fsck works.\n\nOn top of that is some special handling for commit and pull:\nWhen commiting in the parent project then all submodule refs are\nstored in the tree, too.\nFetching from a remote parent copies all submodule references into\nthe current repository, too.\nAfter a successful merge the submodules are updated to reflect\nthe new state.\n\nThe code below is obviously not finished and very fragile,\nit has only been tested with the litte test script so far,\nbut I think it's a nice basis for discussion.\n\nWhat do you think?\n\nSome specific questions:\n\nUser interface: like show here only one more command to set up\nsubmodules and then using standard commands for parent and submodule\nor does it make more sense to add special commands for updating\nsubmodules (and not recursing into them by default)?\n\nDoes it really make sense to have one shared object directory?\nThis allows to fetch the whole project in one go, but it\nmight be more robust to set up individual object databases.\n\ndiff --git a/.gitignore b/.gitignore\nindex a3d9c7a..2ef60fc 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -47,6 +47,7 @@ git-http-push\n git-imap-send\n git-index-pack\n git-init-db\n+git-init-module\n git-instaweb\n git-local-fetch\n git-log\ndiff --git a/Makefile b/Makefile\nindex 8467447..4493973 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -154,7 +154,7 @@ ### --- END CONFIGURATION SECTION ---\n SCRIPT_SH = \\\n \tgit-bisect.sh git-branch.sh git-checkout.sh \\\n \tgit-cherry.sh git-clean.sh git-clone.sh git-commit.sh \\\n-\tgit-fetch.sh \\\n+\tgit-fetch.sh git-init-module.sh \\\n \tgit-ls-remote.sh \\\n \tgit-merge-one-file.sh git-parse-remote.sh \\\n \tgit-pull.sh git-rebase.sh \\\ndiff --git a/git-checkout.sh b/git-checkout.sh\nindex 580a9e8..a8764fd 100755\n--- a/git-checkout.sh\n+++ b/git-checkout.sh\n@@ -211,3 +211,20 @@ if [ \"$?\" -eq 0 ]; then\n else\n \texit 1\n fi\n+\n+#\n+# Now checkout all the submodules recursively\n+# FIXME: handle branch updates\n+#\n+unset GIT_DIR\n+parent_dir=`pwd`\n+if test -r .gitmodules; then\n+\twhile read module; do\n+\t\tcd \"$parent_dir\"\n+\t\tHEAD=$(git-symbolic-ref \"../.gitmodule/$module/HEAD\")\n+\t\tmkdir -p \"$module\"\n+\t\tcd \"$module\"\n+\t\tgit-init-module >/dev/null\n+\t\tgit-checkout $HEAD\n+\tdone < .gitmodules\n+fi\ndiff --git a/git-clone.sh b/git-clone.sh\nindex e1b3bf3..cce21e9 100755\n--- a/git-clone.sh\n+++ b/git-clone.sh\n@@ -410,7 +410,7 @@ Pull: refs/heads/$head_points_at:$origin\n \n \tcase \"$no_checkout\" in\n \t'')\n-\t\tgit-read-tree -m -u -v HEAD HEAD\n+\t\tgit-checkout\n \tesac\n fi\n rm -f \"$GIT_DIR/CLONE_HEAD\" \"$GIT_DIR/REMOTE_HEAD\"\ndiff --git a/git-commit.sh b/git-commit.sh\nindex 5a4c659..1b79182 100755\n--- a/git-commit.sh\n+++ b/git-commit.sh\n@@ -342,6 +342,18 @@ then\n \tTOP=./\n fi\n \n+# update the .gitmodule/ contents\n+# FIXME: don't overwrite, test if submodule is ancestor of current value\n+if test -r .gitmodules; then\n+\twhile read module; do\n+\t\trm -rf \".gitmodule/$module\"\n+\t\tmkdir -p \".gitmodule/$module/refs\"\n+\t\tcp -a \"$module/.git/refs/heads\" \".gitmodule/$module/refs/\"\n+\t\tcp -a \"$module/.git/refs/tags\" \".gitmodule/$module/refs/\"\n+\t\tcp -a \"$module/.git/HEAD\" \".gitmodule/$module/\"\n+\tdone < .gitmodules\n+fi\n+\n case \"$all,$also\" in\n t,)\n \tsave_index &&\ndiff --git a/git-init-module.sh b/git-init-module.sh\nnew file mode 100644\nindex 0000000..b64f630\n--- /dev/null\n+++ b/git-init-module.sh\n@@ -0,0 +1,65 @@\n+#!/bin/sh -e\n+\n+USAGE=\"\"\n+LONG_USAGE=\"Sets up a GIT sub module in the current directory\"\n+SUBDIRECTORY_OK=yes\n+\n+# get directory of parent GIT repository\n+GIT_MODULE_DIR=`pwd`\n+cd ..\n+. git-sh-setup\n+GIT_PARENT_DIR=`cd \"$GIT_DIR\" && pwd`\n+GIT_PARENT_DIR=\"${GIT_PARENT_DIR%.git}\"\n+\n+# make relative path\n+GIT_MODULE_REL_DIR=\"${GIT_MODULE_DIR#$GIT_PARENT_DIR}\"\n+GIT_PARENT_REL_DIR=\"$GIT_PARENT_DIR\" # FIXME should be made relative\n+\n+populate_dirs() {\n+\t(cd \"$1\" && find . -type d) | (cd \"$2\" && xargs mkdir -p)\n+}\n+\n+# a mv which can merge directory trees\n+move() {\n+\tpopulate_dirs \"$@\"\n+\t(cd \"$1\" && find . ! -type d) | xargs -n 1 -i mv \"{}\" \"$2\"\n+\tfind \"$1\" -depth | xargs rmdir\n+}\n+\n+\n+# create a basic repository here\n+cd \"$GIT_MODULE_DIR\"\n+git-init-db\n+\n+# use parents object directory instead\n+if test ! -L .git/objects; then\n+\tmove .git/objects \"$GIT_PARENT_DIR/.git/objects/\"\n+\tln -s \"$GIT_PARENT_REL_DIR/.git/objects\" .git/objects\n+fi\n+\n+# make all references known everywhere, so that git-fsck-objects works.\n+mkdir -p \"$GIT_PARENT_DIR/.git/refs/module/$GIT_MODULE_REL_DIR\"\n+ln -s -f -n \"$GIT_PARENT_REL_DIR/.git/refs\" .git/refs/parent\n+for dir in heads tags remotes; do\n+\tln -s -f -n \"$GIT_MODULE_DIR/.git/refs/$dir\" \"$GIT_PARENT_DIR/.git/refs/module/$GIT_MODULE_REL_DIR/$dir\"\n+done\n+\n+\n+# copy heads from parent, but without overwriting anything\n+GIT_PARENT_REFS=\"$GIT_PARENT_DIR/.gitmodule/$GIT_MODULE_REL_DIR\"\n+mkdir -p \"$GIT_PARENT_REFS\"\n+populate_dirs \"$GIT_PARENT_REFS\" .git\n+(cd \"$GIT_PARENT_REFS\" && find ! -type d) | while read path; do\n+\tif test ! -e \".git/$path\"; then\n+\t\tcp \"$GIT_PARENT_REFS/$path\" \".git/$path\"\n+\tfi\n+done\n+\n+# register the module in the parent\n+(cat \"$GIT_PARENT_DIR/.gitmodules\" 2>/dev/null || true;\n+ echo \"$GIT_MODULE_REL_DIR\") |\n+\tsort -u > \"$GIT_PARENT_DIR/.gitmodules.new\"\n+mv \"$GIT_PARENT_DIR/.gitmodules.new\" \"$GIT_PARENT_DIR/.gitmodules\"\n+\n+grep \"^/$GIT_MODULE_REL_DIR$\" \"$GIT_PARENT_DIR/.gitignore\" >/dev/null 2>&1 ||\n+echo \"/$GIT_MODULE_REL_DIR\" >> \"$GIT_PARENT_DIR/.gitignore\"\ndiff --git a/git-merge.sh b/git-merge.sh\nindex d049e16..273d864 100755\n--- a/git-merge.sh\n+++ b/git-merge.sh\n@@ -97,8 +97,33 @@ finish () {\n \t\tesac\n \t\t;;\n \tesac\n+\n+\t# now take a look at changes in submodules\n+\tif test -e .gitmodules; then\n+\t\twhile read module; do\n+\t\t\tmerge_module \"$module\"\n+\t\tdone < .gitmodules\n+\tfi\n }\n \n+merge_module () {\n+\tmodule=\"$1\"\n+\tHEAD=$(git-symbolic-ref \"../.gitmodule/$module/HEAD\")\n+\tnew_head=$(cat \".gitmodule/$module/$HEAD\")\n+\t# FIXME: first step: merge HEAD only\n+\t(set -e\n+\t\tcd \"$module\"\n+\t\tunset GIT_DIR\n+\t\t# FIXME: sure there is a nicer way to do this...\n+\t\techo \"Updateing $module\"\n+\t\tgit-branch \"refs/update/$HEAD\" \"$new_head\"\n+\t\tgit-merge \"updated by parent\" \"$HEAD\" \"refs/update/$HEAD\"\n+\t\tgit-branch -d \"refs/update/$HEAD\"\n+\t\tgit-checkout \"$HEAD\"\n+\t)\n+}\n+\n+\n rloga=\n while case \"$#\" in 0) break ;; esac\n do\ndiff --git a/git-parse-remote.sh b/git-parse-remote.sh\nindex 187f088..230eb4d 100755\n--- a/git-parse-remote.sh\n+++ b/git-parse-remote.sh\n@@ -149,6 +149,17 @@ get_remote_default_refs_for_fetch () {\n \t*)\n \t\tdie \"internal error: get-remote-default-ref-for-push $1\" ;;\n \tesac\n+\t# also fetch all submodule data\n+\tif test -e .gitmodules; then\n+\t\twhile read module; do\n+\t\t\t(cd .gitmodule/$module/refs && find * ! -type d) |\n+\t\t\twhile read ref; do\n+\t\t\t\t# only fetch the ref, don't store it\n+\t\t\t\t# as it gets stored in the repository directly\n+\t\t\t\techo \".refs/module/$module/$ref:\"\n+\t\t\tdone\n+\t\tdone < .gitmodules\n+\tfi\n }\n \n get_remote_refs_for_push () {\ndiff --git a/t/t7500-submodule.sh b/t/t7500-submodule.sh\nnew file mode 100755\nindex 0000000..cb57fef\n--- /dev/null\n+++ b/t/t7500-submodule.sh\n@@ -0,0 +1,76 @@\n+#!/bin/sh\n+\n+test_description=\"Test submodule support.\"\n+\n+. ./test-lib.sh\n+\n+\n+test_expect_success \"Create parent\" \\\n+    'mkdir p &&\n+     cd p &&\n+     git-init-db &&\n+     echo \"Parent repository\" > README &&\n+     git-add README &&\n+     git-commit -a -m \"init parent\" &&\n+     cd ..'\n+\n+test_expect_success \"Create submodule\" \\\n+    'mkdir p/sub &&\n+     cd p/sub &&\n+     git-init-module &&\n+     echo \"Submodule repository\" > README &&\n+     git-add README &&\n+     git-commit -a -m \"init submodule\" &&\n+     cd ../..'\n+\n+test_expect_success \"Add submodule to parent\" \\\n+    'cd p &&\n+     git-add .gitmodules .gitignore &&\n+     git-status >/dev/null &&\n+     git-add .gitmodule &&\n+     git-commit -a -m \"add submodule\" &&\n+     cd ..'\n+\n+check_repos() {\n+     ( git-status > ../actual || true ) &&\n+     git-fsck-objects --full 2>&1 >> ../actual 2>&1 &&\n+     cd sub &&\n+     ( git-status >> ../../actual 2>&1 || true ) &&\n+     git-fsck-objects --full >> ../../actual 2>&1 &&\n+     cd ../.. &&\n+     echo \"nothing to commit\" > expected &&\n+     echo \"nothing to commit\" >> expected &&\n+     diff -u expected actual\n+}\n+\n+test_expect_success \"Check repositories\" \\\n+    'cd p && check_repos'\n+\n+test_expect_success \"Cloning parent\" \\\n+    'git-clone p p2'\n+\n+test_expect_success \"Check cloned repositories\" \\\n+    'cd p2 && check_repos'\n+\n+test_expect_success \"Commit to submodule\" \\\n+    'cd p/sub &&\n+     echo foo > bar &&\n+     git-add bar &&\n+     git-commit -a -m \"add bar\" &&\n+     cd ../..'\n+\n+test_expect_success \"Commit to parent\" \\\n+    'cd p &&\n+     git-commit -a -m \"update sub\" &&\n+     cd ..'\n+\n+test_expect_success \"Pull changes into clone\" \\\n+    'cd p2 &&\n+     git-pull ../p &&\n+     test -f sub/bar &&\n+     cd ..'\n+\n+test_expect_success \"Check updated repositories\" \\\n+    'cd p2 && check_repos'\n+\n+test_done\n\n-- \nMartin Waitz\n"}]}