{"thread":{"id":"43273","subject":"Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks","startedAt":"2006-12-20T13:09:01Z","lastAt":"2006-12-20T16:09:44Z","messageCount":14,"participants":["Andy Parkins","Jakub Narebski","Rogan Dawes","Johannes Schindelin","Martin Waitz","Johannes Sixt","Randal L. Schwartz"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"295106","messageId":"200612201309.02119.andyparkins@gmail.com","threadId":"43273","inReplyTo":null,"subject":"[RFC/PATCH] Implement poor-man's submodule support using commit hooks","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-20T13:09:01Z","receivedAt":"2006-12-20T13:09:01Z","isPatch":true,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"Make a file called .gitmodules.  In it, list the paths containing a\nsubmodule.  Add that file to the index.\n\nThis patch adds a check to the pre-commit hook finds that file and pulls\nthe HEAD hash out of each of the listed submodule repositories.  That\nhash is then listed to the .gitmodules file along with the submodule\nname and .gitmodules is added back to the repository.\n\nYou've now got poor-man's submodule support.  Any commits to the\nsubmodule will change the hash and hence the .gitmodules file will be\ndifferent and therefore will show up as \"modified\" to git.\n\nIt's not got any nice UI for checking out (obviously) or merging; but it\ndoes at least record the state of a project. With a bit of manual work you\ncan easily check out the right commit in the submodule.  If there were a\npost-checkout hook script, this could probably be automated.\n\nTo prevent git-prune in the submodule from removing references that the\nsupermodule refers to the post-commit hook reads the .gitmodules file\nand creates a file in submodule/.git/refs/superrefs/ that refers to the\nhash we've references.  git-prune in the submodule will find that\nreference and hence won't remove it from under us.\n\nProblems:\n - git-prune in the supermodule doesn't clean the supermodule refs in\n   the submodule\n - no checkout support\n - no reset support\n - no merge support (other than what git provides for the .gitmodule\n   file)\n - no check for dirty submodule before commit\n\nSigned-off-by: Andy Parkins <andyparkins@gmail.com>\n---\nThis is in replacement of the previous patch.\n\nI've used the --show-cdup option of git-rev-parse to make it work even in\nsubdirectories.  I've also added git-prune protection by adding a post-commit\nscript to reference the hash in the submodule.\n\nWhat'd you reckon?  Might be useful until real submodule support arrives.\nIf there is no .gitmodules file in the root, then git behaves as it always did.\n\nI suppose if this were actually found to be really useful, it should go in\ngit-commit.sh itself, rather than the hooks.\n\n\n templates/hooks--post-commit |   29 +++++++++++++++++++++++++-\n templates/hooks--pre-commit  |   47 +++++++++++++++++++++++++++++++++++++++++-\n 2 files changed, 74 insertions(+), 2 deletions(-)\n\ndiff --git a/templates/hooks--post-commit b/templates/hooks--post-commit\nindex 8be6f34..551d928 100644\n--- a/templates/hooks--post-commit\n+++ b/templates/hooks--post-commit\n@@ -5,4 +5,31 @@\n #\n # To enable this hook, make this file executable.\n \n-: Nothing\n+# Poor-man's Submodules\n+# ---------------------\n+# If we're here, then a commit has succeeded. If submodule support is enabled\n+# then we need a way of telling the submodule that we now reference a hash\n+# owned by it, so that it is not pruned.\n+WORKINGTOP=$(git-rev-parse --show-cdup)\n+GITMODULES=\"${WORKINGTOP}.gitmodules\"\n+if [ -f \"$GITMODULES\" ]; then\n+\tcat \"$GITMODULES\" |\n+\twhile read subdir hash\n+\tdo\n+\t\tSUBMODULEPATH=\"$WORKINGTOP$subdir/.git/refs/superrefs\"\n+\n+\t\t# XXX: check if the line is a comment\n+\n+\t\t# check if the subdir is a repository\n+\t\tif [ ! -d \"$WORKINGTOP$subdir/.git\" ]; then\n+\t\t\tcontinue;\n+\t\tfi\n+\n+\t\t# Write the hash to a file of the same name - this means that if we get\n+\t\t# multiple commits that refer to the submodule, we only get one file in\n+\t\t# the submodule, as the submodule hash is constant across supermodule\n+\t\t# commits\n+\t\tmkdir -p \"$SUBMODULEPATH\"\n+\t\techo $hash > \"$SUBMODULEPATH/$hash\"\n+\tdone\n+fi\ndiff --git a/templates/hooks--pre-commit b/templates/hooks--pre-commit\nindex 723a9ef..7718369 100644\n--- a/templates/hooks--pre-commit\n+++ b/templates/hooks--pre-commit\n@@ -67,5 +67,50 @@ perl -e '\n \t}\n     }\n     exit($found_bad);\n-'\n+' || exit 1\n+\n+# Poor-man's Submodules\n+# ---------------------\n+# Enable poor-man's submodule support when .gitmodules is present\n+# Simply create a .gitmodules file listing the paths in your repository\n+# that contain other git repositories; each line will be replaced with the\n+# path followed by the hash of the current HEAD of the submodule.\n+# When the submodule changes hash this file will be different from the\n+# version in the repository and a change will be noted - voila, submodules.\n+# Of course there is no checkout support, but at least the current state\n+# will be accurately recorded\n+WORKINGTOP=$(git-rev-parse --show-cdup)\n+GITMODULES=\"${WORKINGTOP}.gitmodules\"\n+if [ -f \"$GITMODULES\" ]; then\n+\tcat \"$GITMODULES\" |\n+\twhile read subdir hash\n+\tdo\n+\t\t# check if the line is a comment and output it anyway\n+\t\tif (expr \"x$subdir\" : \"x#\" >/dev/null) then\n+\t\t\techo \"$subdir $hash\"\n+\t\t\tcontinue;\n+\t\tfi\n+\n+\t\t# check if the subdir is a repository\n+\t\tif [ ! -d \"$WORKINGTOP$subdir/.git\" ]; then\n+\t\t\techo \"$subdir is not a git repository, so it can't be a submodule\"\n+\t\t\texit 1;\n+\t\tfi\n+\n+\t\t# XXX: really need a check here and quit if the submodule is\n+\t\t# dirty\n+\n+\t\techo \"$subdir $(GIT_DIR=$WORKINGTOP$subdir/.git git-rev-parse HEAD)\"\n+\tdone > newgitmodules\n+\t# Update\n+\tmv newgitmodules \"$GITMODULES\"\n+\n+\t# This relies on the .gitmodules file having already been added to\n+\t# the repository - perhaps this should be automated?\n+\tgit-update-index \"$GITMODULES\" ||\n+\t(\n+\techo \"FATAL: Submodule tracker file, $GITMODULES, is not tracked in this repository.\" >&2\n+\texit 1\n+\t)\n+fi\n \n-- \n1.4.4.2.g0d2a\n"},{"id":"297243","messageId":"45893AC6.910D5748@eudaptics.com","threadId":"43273","inReplyTo":"200612201309.02119.andyparkins@gmail.com","subject":"Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks","fromName":"Johannes Sixt","fromEmail":"j.sixt@eudaptics.com","sentAt":"2006-12-20T13:29:42Z","receivedAt":"2006-12-20T13:29:42Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Andy Parkins wrote:\n> +WORKINGTOP=$(git-rev-parse --show-cdup)\n> +GITMODULES=\"${WORKINGTOP}.gitmodules\"\n> +if [ -f \"$GITMODULES\" ]; then\n> +       cat \"$GITMODULES\" |\n\nuseless-use-of-cat-syndrome\n\n> +       while read subdir hash\n\nWouldn't it be better to have the order of subdir and hash swapped? That\nway subdir may contain blanks, and it gives nicer alignment in the file\nbecause of the constant length of the hashes.\n\n-- Hannes\n"},{"id":"298149","messageId":"20061220133648.GQ12411@admingilde.org","threadId":"43273","inReplyTo":"200612201309.02119.andyparkins@gmail.com","subject":"Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-12-20T13:36:48Z","receivedAt":"2006-12-20T13:36:48Z","isPatch":true,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Wed, Dec 20, 2006 at 01:09:01PM +0000, Andy Parkins wrote:\n> \n> You've now got poor-man's submodule support.  Any commits to the\n> submodule will change the hash and hence the .gitmodules file will be\n> different and therefore will show up as \"modified\" to git.\n\nI started with something similiar, too.\nYou can have a look at http://git.admingilde.org/tali/git.git/module\nwhich tries to implement submodules without changing the core.\n\n-- \nMartin Waitz\n"},{"id":"296440","messageId":"200612201347.13805.andyparkins@gmail.com","threadId":"43273","inReplyTo":"45893AC6.910D5748@eudaptics.com","subject":"Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-20T13:47:05Z","receivedAt":"2006-12-20T13:47:05Z","isPatch":true,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Wednesday 2006 December 20 13:29, Johannes Sixt wrote:\n\n> > +       cat \"$GITMODULES\" |\n>\n> useless-use-of-cat-syndrome\n\nI tried \n\n while\n do \n done < file1 > file1\n\nAnd that didn't work.  However, it was racy anyway using the same file, so I \nchanged to as it is now, but forgot to switch back to input redirection.\n\ndiff --git a/git-commit.sh b/git-commit.sh\ndiff --git a/templates/hooks--pre-commit b/templates/hooks--pre-commit\nindex 7718369..74edfe2 100644\n--- a/templates/hooks--pre-commit\n+++ b/templates/hooks--pre-commit\n@@ -82,7 +82,6 @@ perl -e '\n WORKINGTOP=$(git-rev-parse --show-cdup)\n GITMODULES=\"${WORKINGTOP}.gitmodules\"\n if [ -f \"$GITMODULES\" ]; then\n-   cat \"$GITMODULES\" |\n    while read subdir hash\n    do\n        # check if the line is a comment and output it anyway\n@@ -101,7 +100,7 @@ if [ -f \"$GITMODULES\" ]; then\n        # dirty\n \n        echo \"$subdir $(GIT_DIR=$WORKINGTOP$subdir/.git git-rev-parse HEAD)\"\n-   done > newgitmodules\n+   done < \"$GITMODULES\" > newgitmodules\n    # Update\n    mv newgitmodules \"$GITMODULES\"\n\n> > +       while read subdir hash\n>\n> Wouldn't it be better to have the order of subdir and hash swapped? That\n> way subdir may contain blanks, and it gives nicer alignment in the file\n> because of the constant length of the hashes.\n\nUnfortunately, it is the hash that is optional.  When you create the file, you \ndon't list the hashes, you list the subdirectories.  I suppose I could make \nit so you have to give \"000000\" or something first?\n\n\n\nAndy\n\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"298700","messageId":"200612201348.25897.andyparkins@gmail.com","threadId":"43273","inReplyTo":"20061220133648.GQ12411@admingilde.org","subject":"Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-20T13:48:24Z","receivedAt":"2006-12-20T13:48:24Z","isPatch":true,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Wednesday 2006 December 20 13:36, Martin Waitz wrote:\n\n> I started with something similiar, too.\n> You can have a look at http://git.admingilde.org/tali/git.git/module\n> which tries to implement submodules without changing the core.\n\nI wasn't really trying to do away with the need for core changes; I just \nwanted something simple that I could use today.\n\nIf this is old news - no problem.  Drop it.\n\n\nAndy\n\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"294295","messageId":"embfus$d50$1@sea.gmane.org","threadId":"43273","inReplyTo":"200612201347.13805.andyparkins@gmail.com","subject":"Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-12-20T14:07:56Z","receivedAt":"2006-12-20T14:07:56Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Andy Parkins wrote:\n\n> On Wednesday 2006 December 20 13:29, Johannes Sixt wrote:\n\n>>> +       while read subdir hash\n>>\n>> Wouldn't it be better to have the order of subdir and hash swapped? That\n>> way subdir may contain blanks, and it gives nicer alignment in the file\n>> because of the constant length of the hashes.\n> \n> Unfortunately, it is the hash that is optional.  When you create the file, you \n> don't list the hashes, you list the subdirectories.  I suppose I could make \n> it so you have to give \"000000\" or something first?\n\nThat's the convention git uses in git-diff, reflog, etc.\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n\n"},{"id":"297089","messageId":"45894640.2E0768B5@eudaptics.com","threadId":"43273","inReplyTo":"200612201309.02119.andyparkins@gmail.com","subject":"Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks","fromName":"Johannes Sixt","fromEmail":"j.sixt@eudaptics.com","sentAt":"2006-12-20T14:18:40Z","receivedAt":"2006-12-20T14:18:40Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Andy Parkins wrote:\n> +               # check if the subdir is a repository\n> +               if [ ! -d \"$WORKINGTOP$subdir/.git\" ]; then\n> +                       echo \"$subdir is not a git repository, so it can't be a submodule\"\n\nYou must redirect the message to >&2 otherwise it remains invisible (it\ngoes into file newgitmodules).\n\n-- Hannes\n"},{"id":"298454","messageId":"200612201420.58756.andyparkins@gmail.com","threadId":"43273","inReplyTo":"embfus$d50$1@sea.gmane.org","subject":"Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-20T14:20:52Z","receivedAt":"2006-12-20T14:20:52Z","isPatch":true,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Wednesday 2006 December 20 14:07, Jakub Narebski wrote:\n\n> > Unfortunately, it is the hash that is optional.  When you create the\n> > file, you don't list the hashes, you list the subdirectories.  I suppose\n> > I could make it so you have to give \"000000\" or something first?\n>\n> That's the convention git uses in git-diff, reflog, etc.\n\nThey are all outputs though; this is an input.  I did it that way round so \nthat I could say:\n\ncat .gitmodules <<EOF\nsubmodule1\nsubmodule2\nEOF\n\nAnd that was it.  Of course, this doesn't work when there are spaces in the \nfilename, so I guess it's got to be \n\ncat .gitmodules <<EOF\n000 submodule1\n000 submodule2\nEOF\n\nI'll wait to see if there are more objections before posting an updated patch.\n\n\nAndy\n\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"295669","messageId":"458949C4.1020207@dawes.za.net","threadId":"43273","inReplyTo":"200612201347.13805.andyparkins@gmail.com","subject":"Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks","fromName":"Rogan Dawes","fromEmail":"discard@dawes.za.net","sentAt":"2006-12-20T14:33:40Z","receivedAt":"2006-12-20T14:33:40Z","isPatch":true,"sender":{"key":"discard@dawes.za.net","avatar":null},"body":"Andy Parkins wrote:\n> On Wednesday 2006 December 20 13:29, Johannes Sixt wrote:\n> \n>>> +       cat \"$GITMODULES\" |\n>> useless-use-of-cat-syndrome\n> \n> I tried \n> \n>  while\n>  do \n>  done < file1 > file1\n> \n> And that didn't work. \n\nIt wouldn't, since the redirection overwrites the file at the same time \nas trying to read from it. A better way is to redirect to a temp file, \nand rename it if the previous operation was successful.\n\ne.g.\n\n0 $ echo true > t\n0 $ sed 's/true/false/g' < t > t~ && mv t~ t || rm t~\n0 $ cat t\nfalse\n\nand\n\n0 $ echo true > t\n0 $ (sed 's/true/false/g' < t > t~; false) && mv t~ t || rm t~\n0 $ cat t\ntrue\n0 $ ls t~\nls: t~: No such file or directory\n2 $\n\n"},{"id":"294193","messageId":"200612201435.49685.andyparkins@gmail.com","threadId":"43273","inReplyTo":"20061220133648.GQ12411@admingilde.org","subject":"Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-20T14:35:48Z","receivedAt":"2006-12-20T14:35:48Z","isPatch":true,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Wednesday 2006 December 20 13:36, Martin Waitz wrote:\n\n> I started with something similiar, too.\n> You can have a look at http://git.admingilde.org/tali/git.git/module\n> which tries to implement submodules without changing the core.\n\nI had a look at that; and my previous objection still applies: it blends two \nrepositories.  I'd rather they were separate.\n\nHaving them separate is what makes my little hook scripts really simple; there \nis no special initialisation script needed, and the neither the supermodule \nnor the submodule needs special treatment; and there is no need for any \nalternates or new object type.  If I want to forget about submodule support I \njust delete the .gitmodules file and it's all back to normal.\n\nI don't say you're wrong in your chosen method, and I don't say I'm right.  I \nwasn't really planning on going much further with these scripts, they \nactually represent all that I personally want from a submodule system, I am \nsure others would want far more sophistication.\n\n\n\nAndy\n\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"297608","messageId":"200612201440.41784.andyparkins@gmail.com","threadId":"43273","inReplyTo":"458949C4.1020207@dawes.za.net","subject":"Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks","fromName":"Andy Parkins","fromEmail":"andyparkins@gmail.com","sentAt":"2006-12-20T14:40:40Z","receivedAt":"2006-12-20T14:40:40Z","isPatch":true,"sender":{"key":"andyparkins@gmail.com","avatar":null},"body":"On Wednesday 2006 December 20 14:33, Rogan Dawes wrote:\n\n> It wouldn't, since the redirection overwrites the file at the same time\n\nOf course you are correct; however it wasn't completely crazy.  Before now, \nI've replaced files in C by doing\n\n fd_old = open( \"someexistingfile\" );\n unlink( \"someexistingfile\" );\n fd_new = open( \"someexistingfile\" );\n\nSo I was really just trying it in case bash did a similar thing internally.\n\n> as trying to read from it. A better way is to redirect to a temp file,\n> and rename it if the previous operation was successful.\n\nThat's what I ended up doing; as you say it's better anyway because the error \nconditions don't mean that the .gitmodules file ends up half written.\n\n\nAndy\n-- \nDr Andy Parkins, M Eng (hons), MIEE\n"},{"id":"297513","messageId":"86y7p21smq.fsf@blue.stonehenge.com","threadId":"43273","inReplyTo":"200612201440.41784.andyparkins@gmail.com","subject":"Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks","fromName":"Randal L. Schwartz","fromEmail":"merlyn@stonehenge.com","sentAt":"2006-12-20T15:42:05Z","receivedAt":"2006-12-20T15:42:05Z","isPatch":true,"sender":{"key":"merlyn@stonehenge.com","avatar":"https://gravatar.com/avatar/dc528d210743ff0333e6213f9ee7b33b23f1b7bc1f3c5a8c2d819074ecd7ab19?d=mp&s=160"},"body":">>>>> \"Andy\" == Andy Parkins <andyparkins@gmail.com> writes:\n\nAndy>  fd_old = open( \"someexistingfile\" );\nAndy>  unlink( \"someexistingfile\" );\nAndy>  fd_new = open( \"someexistingfile\" );\n\nThis is dangerous if the process aborts suddenly.\n\nMight be better to do this:\n\nmumble_command < thefile > thefile.$$\nmv thefile.$$ thefile\n\nThe second move will be atomic.  The only problem will be if the process\naborts, which will leave stale temp files around.\n\n-- \nRandal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095\n<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>\nPerl/Unix/security consulting, Technical writing, Comedy, etc. etc.\n"},{"id":"296495","messageId":"20061220154424.GS12411@admingilde.org","threadId":"43273","inReplyTo":"200612201435.49685.andyparkins@gmail.com","subject":"Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks","fromName":"Martin Waitz","fromEmail":"tali@admingilde.org","sentAt":"2006-12-20T15:44:25Z","receivedAt":"2006-12-20T15:44:25Z","isPatch":true,"sender":{"key":"tali@admingilde.org","avatar":"https://gravatar.com/avatar/3f89b03eee362187effabe257898735b475673a12265c398ea9161259ae91553?d=mp&s=160"},"body":"hoi :)\n\nOn Wed, Dec 20, 2006 at 02:35:48PM +0000, Andy Parkins wrote:\n> I had a look at that; and my previous objection still applies: it blends two \n> repositories.  I'd rather they were separate.\n\nyes, but one can see what is needed and where the real problems are.\n\nAnd I really think we should only start adding submodule code to\nmainline git when we are sure that we found an approach where we can see\nthat it works and that we don't have to change it later.\n\n-- \nMartin Waitz\n"},{"id":"296392","messageId":"Pine.LNX.4.63.0612201708420.19693@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"43273","inReplyTo":"200612201440.41784.andyparkins@gmail.com","subject":"Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-12-20T16:09:44Z","receivedAt":"2006-12-20T16:09:44Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 20 Dec 2006, Andy Parkins wrote:\n\n>  Before now, I've replaced files in C by doing\n> \n>  fd_old = open( \"someexistingfile\" );\n>  unlink( \"someexistingfile\" );\n>  fd_new = open( \"someexistingfile\" );\n\nLucky you! Obiously you are not stuck in the 10th circle called Windows.\n\nCiao,\n"}]}