{"thread":{"id":"36342","subject":"[RFC] git submodule split","startedAt":"2014-04-02T21:52:39Z","lastAt":"2014-04-07T19:04:38Z","messageCount":4,"participants":["Michal Sojka","Jens Lehmann"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"238322","messageId":"87sipviel4.fsf@steelpick.2x.cz","threadId":"36342","inReplyTo":null,"subject":"[RFC] git submodule split","fromName":"Michal Sojka","fromEmail":"sojkam1@fel.cvut.cz","sentAt":"2014-04-02T21:52:39Z","receivedAt":"2014-04-02T21:52:39Z","isPatch":false,"sender":{"key":"sojkam1@fel.cvut.cz","avatar":"https://avatars.githubusercontent.com/u/140542?v=4"},"body":"Hello,\n\nI needed to convert a subdirectory of a repo to a submodule and have the\nhistories of both repos linked together. I found that this was discussed\nfew years back [1], but the code seemed quite complicated and was not\nmerged.\n\n[1]: http://git.661346.n2.nabble.com/RFC-What-s-the-best-UI-for-git-submodule-split-tp2318127.html\n\nNow, the situation is better, because git subtree can already do most of\nthe work. Below is a script that I used to split a submodule from my\nrepo. It basically consist of a call to 'git subtree split' followed by\n'git filter-branch' to link the histories together.\n\nI'd like to get some initial feedback on it before attempting to\nintegrate it with git sources (i.e. writing tests and doc). What do you\nthink?\n\nThanks,\n-Michal\n\n\n#!/bin/sh\n\nset -e\n\n. git-sh-setup\n\nurl=$1\ndir=$2\n\ntest -d \"$dir\" || die \"$dir is not a directory\"\n\n# Create subtree corresponding to the directory\nsubtree=$(git subtree split --prefix=\"$dir\")\n\nsubtree_tag=tmp/submodule-split-$$\ngit tag $subtree_tag $subtree\nsuperproject=$PWD\nexport subtree subtree_tag superproject\n\n# Replace the directory with submodule reference in the whole history\ngit filter-branch -f --index-filter \"\n    set -e\n    # Check whether the $dir exists in this commit\n    if git ls-files --error-unmatch '$dir' > /dev/null 2>&1; then\n\n        # Find subtree commit corresponding to the commit in the\n        # superproject (this could be made faster by not running git log\n        # for every commit)\n        subcommit=\\$(git log --format='%T %H' $subtree |\n     \t    grep ^\\$(git ls-tree \\$GIT_COMMIT -- '$dir'|awk '{print \\$3}') |\n     \t    awk '{print \\$2}')\n\n        # filter-branch runs the filter in an empty work-tree - create the\n        # future submodule in it so that the 'git submodule add' below\n        # does not try to clone it.\n        if ! test -d '$dir'; then\n     \t    mkdir -p '$dir'\n     \t    ( cd '$dir' && clear_local_git_env && git init --quiet && git pull $superproject $subtree_tag )\n        fi\n\n        # Remove all files under $dir from index so that the 'git\n        # submodule add' below does not complain.\n        git ls-files '$dir'|git update-index --force-remove --stdin\n\n        # Add the submodule - the goal here is to create/update .gitmodules\n        git submodule add $url '$dir'\n\n        # Update the submodule commit hash to the correct value\n        echo \\\"160000 \\$subcommit\t$dir\\\"|git update-index --index-info\n    fi\n\"\n\n# Replace the directory in the working tree with the submodule\n( cd \"$dir\" && find -mindepth 1 -delete && git init && git pull $superproject $subtree_tag )\n\n# Clean up\ngit tag --delete $subtree_tag\n"},{"id":"238397","messageId":"53417C00.3090805@web.de","threadId":"36342","inReplyTo":"87sipviel4.fsf@steelpick.2x.cz","subject":"Re: [RFC] git submodule split","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2014-04-06T16:08:32Z","receivedAt":"2014-04-06T16:08:32Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 02.04.2014 23:52, schrieb Michal Sojka:\n> Hello,\n> \n> I needed to convert a subdirectory of a repo to a submodule and have the\n> histories of both repos linked together. I found that this was discussed\n> few years back [1], but the code seemed quite complicated and was not\n> merged.\n> \n> [1]: http://git.661346.n2.nabble.com/RFC-What-s-the-best-UI-for-git-submodule-split-tp2318127.html\n> \n> Now, the situation is better, because git subtree can already do most of\n> the work. Below is a script that I used to split a submodule from my\n> repo. It basically consist of a call to 'git subtree split' followed by\n> 'git filter-branch' to link the histories together.\n> \n> I'd like to get some initial feedback on it before attempting to\n> integrate it with git sources (i.e. writing tests and doc). What do you\n> think?\n\nWhy do want to rewrite the whole history of the superproject,\nwouldn't it suffice to turn a directory into a submodule with\nthe same content in a simple commit? Don't get me wrong, I'm\nnot against adding such a functionality to contrib, I'm just\ntrying to understand the motivation for your script.\n\n> Thanks,\n> -Michal\n> \n> \n> #!/bin/sh\n> \n> set -e\n> \n> . git-sh-setup\n> \n> url=$1\n> dir=$2\n> \n> test -d \"$dir\" || die \"$dir is not a directory\"\n> \n> # Create subtree corresponding to the directory\n> subtree=$(git subtree split --prefix=\"$dir\")\n> \n> subtree_tag=tmp/submodule-split-$$\n> git tag $subtree_tag $subtree\n> superproject=$PWD\n> export subtree subtree_tag superproject\n> \n> # Replace the directory with submodule reference in the whole history\n> git filter-branch -f --index-filter \"\n>     set -e\n>     # Check whether the $dir exists in this commit\n>     if git ls-files --error-unmatch '$dir' > /dev/null 2>&1; then\n> \n>         # Find subtree commit corresponding to the commit in the\n>         # superproject (this could be made faster by not running git log\n>         # for every commit)\n>         subcommit=\\$(git log --format='%T %H' $subtree |\n>      \t    grep ^\\$(git ls-tree \\$GIT_COMMIT -- '$dir'|awk '{print \\$3}') |\n>      \t    awk '{print \\$2}')\n> \n>         # filter-branch runs the filter in an empty work-tree - create the\n>         # future submodule in it so that the 'git submodule add' below\n>         # does not try to clone it.\n>         if ! test -d '$dir'; then\n>      \t    mkdir -p '$dir'\n>      \t    ( cd '$dir' && clear_local_git_env && git init --quiet && git pull $superproject $subtree_tag )\n>         fi\n> \n>         # Remove all files under $dir from index so that the 'git\n>         # submodule add' below does not complain.\n>         git ls-files '$dir'|git update-index --force-remove --stdin\n> \n>         # Add the submodule - the goal here is to create/update .gitmodules\n>         git submodule add $url '$dir'\n> \n>         # Update the submodule commit hash to the correct value\n>         echo \\\"160000 \\$subcommit\t$dir\\\"|git update-index --index-info\n>     fi\n> \"\n> \n> # Replace the directory in the working tree with the submodule\n> ( cd \"$dir\" && find -mindepth 1 -delete && git init && git pull $superproject $subtree_tag )\n> \n> # Clean up\n> git tag --delete $subtree_tag\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at  http://vger.kernel.org/majordomo-info.html\n> \n"},{"id":"238399","messageId":"87d2gukvhb.fsf@steelpick.2x.cz","threadId":"36342","inReplyTo":"53417C00.3090805@web.de","subject":"Re: [RFC] git submodule split","fromName":"Michal Sojka","fromEmail":"sojkam1@fel.cvut.cz","sentAt":"2014-04-06T21:18:24Z","receivedAt":"2014-04-06T21:18:24Z","isPatch":false,"sender":{"key":"sojkam1@fel.cvut.cz","avatar":"https://avatars.githubusercontent.com/u/140542?v=4"},"body":"On Sun, Apr 06 2014, Jens Lehmann wrote:\n> Am 02.04.2014 23:52, schrieb Michal Sojka:\n>> Hello,\n>> \n>> I needed to convert a subdirectory of a repo to a submodule and have the\n>> histories of both repos linked together. I found that this was discussed\n>> few years back [1], but the code seemed quite complicated and was not\n>> merged.\n>> \n>> [1]: http://git.661346.n2.nabble.com/RFC-What-s-the-best-UI-for-git-submodule-split-tp2318127.html\n>> \n>> Now, the situation is better, because git subtree can already do most of\n>> the work. Below is a script that I used to split a submodule from my\n>> repo. It basically consist of a call to 'git subtree split' followed by\n>> 'git filter-branch' to link the histories together.\n>> \n>> I'd like to get some initial feedback on it before attempting to\n>> integrate it with git sources (i.e. writing tests and doc). What do you\n>> think?\n>\n> Why do want to rewrite the whole history of the superproject,\n> wouldn't it suffice to turn a directory into a submodule with\n> the same content in a simple commit? \n\nI wanted to publish a project including its history but a part of that\nproject could not be made public due to legal reasons. Putting that part\ninto submodule seemed like best idea.\n\n-Michal\n"},{"id":"238498","messageId":"5342F6C6.3080703@web.de","threadId":"36342","inReplyTo":"87d2gukvhb.fsf@steelpick.2x.cz","subject":"Re: [RFC] git submodule split","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2014-04-07T19:04:38Z","receivedAt":"2014-04-07T19:04:38Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 06.04.2014 23:18, schrieb Michal Sojka:\n> On Sun, Apr 06 2014, Jens Lehmann wrote:\n>> Am 02.04.2014 23:52, schrieb Michal Sojka:\n>>> Hello,\n>>>\n>>> I needed to convert a subdirectory of a repo to a submodule and have the\n>>> histories of both repos linked together. I found that this was discussed\n>>> few years back [1], but the code seemed quite complicated and was not\n>>> merged.\n>>>\n>>> [1]: http://git.661346.n2.nabble.com/RFC-What-s-the-best-UI-for-git-submodule-split-tp2318127.html\n>>>\n>>> Now, the situation is better, because git subtree can already do most of\n>>> the work. Below is a script that I used to split a submodule from my\n>>> repo. It basically consist of a call to 'git subtree split' followed by\n>>> 'git filter-branch' to link the histories together.\n>>>\n>>> I'd like to get some initial feedback on it before attempting to\n>>> integrate it with git sources (i.e. writing tests and doc). What do you\n>>> think?\n>>\n>> Why do want to rewrite the whole history of the superproject,\n>> wouldn't it suffice to turn a directory into a submodule with\n>> the same content in a simple commit? \n> \n> I wanted to publish a project including its history but a part of that\n> project could not be made public due to legal reasons. Putting that part\n> into submodule seemed like best idea.\n\nYep, that makes lots of sense.\n\nI'm not sure yet if this functionality is needed often enough to\nput the script under contrib, but I won't object as long as you'd\nbe willing to maintain it (and help people on this list when they\nreport any issues).\n"}]}