{"thread":{"id":"63253","subject":"problem w/recursive, submodules?","startedAt":"2025-04-04T12:05:06Z","lastAt":"2025-04-05T11:44:13Z","messageCount":4,"participants":["Torpey, Bill","Lucas Seiki Oshiro"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"515663","messageId":"PH0PR20MB6183E53678E814C2B473BE8B92AE2@PH0PR20MB6183.namprd20.prod.outlook.com","threadId":"63253","inReplyTo":null,"subject":"problem w/recursive, submodules?","fromName":"Torpey, Bill","fromEmail":"bill.torpey@broadridge.com","sentAt":"2025-04-03T14:48:38Z","receivedAt":"2025-04-04T12:05:06Z","isPatch":false,"sender":{"key":"bill.torpey@broadridge.com","avatar":null},"body":"Thank you for filling out a Git bug report!\nPlease answer the following questions to help us understand your issue.\n\nWhat did you do before the bug happened? (Steps to reproduce your issue)\n\n(We've got a setup where our internal git repos provide wrapper scripts etc. for our \nenvironment and then pull in open-source projects, some of which we contribute to and others\nwhich we sponsor directly.  We've been using submodules to make that work and de-couple\nthe internal bits from the open-source bits). \n\nCloned a repo including a submodule:\n\n\t$ git clone -v -b master --recursive ssh://git@<internal>/nyfix/OpenMAMA-omnm.git\n\tCloning into 'OpenMAMA-omnm'...\n\tremote: Enumerating objects: 789, done.\n\tremote: Counting objects: 100% (789/789), done.\n\tremote: Compressing objects: 100% (461/461), done.\n\tremote: Total 789 (delta 498), reused 472 (delta 323), pack-reused 0\n\tReceiving objects: 100% (789/789), 492.75 KiB | 970.00 KiB/s, done.\n\tResolving deltas: 100% (498/498), done.\n\tSubmodule 'staging' (https://github.com/nyfix/OpenMAMA-omnm.git) registered for path 'src'\n\tCloning into '/Users/bill.torpey/temp/stash/OpenMAMA-omnm/src'...\n\tremote: Enumerating objects: 916, done.\n\tremote: Counting objects: 100% (127/127), done.\n\tremote: Compressing objects: 100% (58/58), done.\n\tremote: Total 916 (delta 63), reused 105 (delta 49), pack-reused 789 (from 1)\n\tReceiving objects: 100% (916/916), 563.50 KiB | 444.00 KiB/s, done.\n\tResolving deltas: 100% (594/594), done.\n\tSubmodule path 'src': checked out '7bb809da9746051b96c0448f8137f25eaf8c15d1'\n\nNoticed that the commit hash of the submodule does NOT match the commit hash when the submodule is cloned directly:\n\n\t$ git clone -v -b staging https://github.com/nyfix/OpenMAMA-omnm.git\n\tCloning into 'OpenMAMA-omnm'...\n\tPOST git-upload-pack (377 bytes)\n\tPOST git-upload-pack (gzip 1559 to 810 bytes)\n\tremote: Enumerating objects: 916, done.\n\tremote: Counting objects: 100% (127/127), done.\n\tremote: Compressing objects: 100% (58/58), done.\n\tremote: Total 916 (delta 63), reused 105 (delta 49), pack-reused 789 (from 1)\n\tReceiving objects: 100% (916/916), 563.50 KiB | 454.00 KiB/s, done.\n\tResolving deltas: 100% (594/594), done.\n\t$ cd OpenMAMA-omnm\n\t$ git status\n\tOn branch staging\n\tYour branch is up to date with 'origin/staging'.\n\n\tnothing to commit, working tree clean\n\t$ git rev-parse HEAD\n\ta092be3bc330f53f644b53d80881bf30cadd367f\n\nWhat did you expect to happen? (Expected behavior)\n\nI expected the commit hash to be correct (a092be3bc330f53f644b53d80881bf30cadd367f) in both \nthe recursive clone and the direct clone.\n\nWhat happened instead? (Actual behavior)\n\nThe commit hash from the recursive clone is the hash of a different (more recent) commit:\n\n\t$ git switch release/1.0.0b\n\tbranch 'release/1.0.0b' set up to track 'origin/release/1.0.0b'.\n\tSwitched to a new branch 'release/1.0.0b'\n\t$ git rev-parse HEAD\n\t7bb809da9746051b96c0448f8137f25eaf8c15d1\n\n\nWhat's different between what you expected and what actually happened?\n\nThe commit hash in the recursive clone is wrong.\n\nAnything else you want to add:\n\nNope - just, thanks for taking a look.  \n\nPlease review the rest of the bug report below.\nYou can delete any lines you don't wish to share.\n\n\n[System Info]\ngit version:\ngit version 2.49.0\ncpu: arm64\nno commit associated with this build\nsizeof-long: 8\nsizeof-size_t: 8\nshell-path: /bin/sh\nfeature: fsmonitor--daemon\nlibcurl: 8.6.0\nzlib: 1.2.12\nuname: Darwin 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:30 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6000 arm64\ncompiler info: clang: 16.0.0 (clang-1600.0.26.6)\nlibc info: no libc information available\n$SHELL (typically, interactive shell): /bin/bash\n\n\n[Enabled Hooks]\n\nThis message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system.\n"},{"id":"515672","messageId":"9274ED7A-0267-43E3-9AE1-DC0BBEC81E69@gmail.com","threadId":"63253","inReplyTo":"PH0PR20MB6183E53678E814C2B473BE8B92AE2@PH0PR20MB6183.namprd20.prod.outlook.com","subject":"Re: problem w/recursive, submodules?","fromName":"Lucas Seiki Oshiro","fromEmail":"lucasseikioshiro@gmail.com","sentAt":"2025-04-04T14:59:46Z","receivedAt":"2025-04-04T15:00:00Z","isPatch":false,"sender":{"key":"lucasseikioshiro@gmail.com","avatar":"https://avatars.githubusercontent.com/u/12701580?v=4"},"body":"\n> $ git clone -v -b master --recursive ssh://git@<internal>/nyfix/OpenMAMA-omnm.git\n\nIt's harder to understand the problem without being able to\nreproduce it. Do you have another example with public\nrepositories?\n\n> The commit hash from the recursive clone is the hash of a different (more recent) commit:\n\nIf I understood it correctly, it looks to me that it's\nanother case of one of the most common misconceptions about\nsubmodules: they always reference a specific commit instead of\nbranch or the latest commit.\n\nIf you want your repository to reference other commit of your\nsubmodule, you'll need to manually change this and commit this\nchange. One way you can do that:\n\ncd <submodule>\ngit checkout <new commit of the submodule>\ncd -\ngit add <submodule>\ngit commit\n\nNote that you'll need to do that every time you want you want\nyour repository to reference other commit of the submodule.\n\n"},{"id":"515687","messageId":"E5FA108F-0694-4614-8899-4913E40FDA4E@gmail.com","threadId":"63253","inReplyTo":"PH0PR20MB6183CD9646B42F0DD552166792A92@PH0PR20MB6183.namprd20.prod.outlook.com","subject":"Re: problem w/recursive, submodules?","fromName":"Lucas Seiki Oshiro","fromEmail":"lucasseikioshiro@gmail.com","sentAt":"2025-04-04T21:58:57Z","receivedAt":"2025-04-04T21:59:11Z","isPatch":false,"sender":{"key":"lucasseikioshiro@gmail.com","avatar":"https://avatars.githubusercontent.com/u/12701580?v=4"},"body":"\n> Shouldn't the recursive clone pick up the HEAD of the submodule branch?\n\nNo, because a submodule is a specific reference to a commit of another\nrepository.\n\n> If it sounds like we don't quite understand what git is doing here, that's a fair statement ;-)\n\nNo problem :-D, in fact, the idea of submodules is quite simple: there's\nthis .gitmodules files where the submodule metadata is stored, and there's\nan entry mapping the name of the submodule to its commit inside each\ncommit of the parent repository.\n\nAs a public example we have the Git source code, which has a submodule.\nYou can clone it with:\n\ngit clone https://github.com/git/git.git\n\nI'm working with the current master of that repository, which today is\nat the commit 5b97a56. As I said before, this repository has a\nsubmodule called sha1collisiondetection. Supposing that you're inside\nthe Git repository, if you run:\n\ngit rev-parse 5b97a56:sha1collisiondetection\n\nyou'll get this hash: 855827c. This is the hash commit of the\nsubmodule at the commit 5b97a56 of the parent repository. Note that I\nchecked this without touching the submodule itself, as this information\nis stored in the parent repository.\n\nYou can do that for any commit. For example, if you run for 23e37f8:\n\ngit rev-parse 23e37f8:sha1collisiondetection\n\nyou'll see that 23e37f8 references another submodule commit. You\ncan even see the log of submodule reference changes:\n\ngit log -- sha1collisiondetection\n\n> but it sounds like git internally stores the commit hash instead?\n\nYes!\n\n> - We've found that the problem is avoided if we execute the following command *after* doing the recursive > clone.  Not sure why this is necessary -- shouldn't the recursive clone handle that on its own?\n> \n> git submodule update --remote --recursive\n\nSorry for answering you this out of the order, but it makes\nmore sense to answer here.\n\nThis command will checkout the submodule to the latest commit of\nthe branch that is in the branch field of the .gitmodules (in\nyour case, `staging`, or in my case of the Git source code,\n`master`).\n\nGoing back to my example, let's first initialize the submodule,\nwhich I didn't before:\n\ngit submodule update --init --recursive\n\nThen, run your command:\n\ngit submodule update --remote --recursive\n\nThis will bring the latest commits of the branch `master` of\nthe submodule. Now, if you run:\n\ngit status\n\nyou'll see that the submodule has changed. This is because it is\nin a commit that is not the one referenced by the HEAD of the\nparent repository. Just like a file that was changed and not\ncommitted. And again, just like a file, if you want the\nrepository to point to that commit from now on you'll need to:\n\ngit add sha1collisiondetection\ngit commit -m \"Update the submodule\"\n\n> - It sounds like you're suggesting that we need to go into the submodule and manually checkout a specific commit?\n\nYes, or you can use git submodule update like you were doing,\nboth work. But keep in mind that it will leave that in a\nunstaged state. If you want to keep it, you'll need to commit\nthat submodule reference change.\n\nIn conclusion, think of submodules as dependencies with fixed\nversions, where each version change needs to be commited in\nthe parent repository.\n\n> Thanks again for your help!\n\nYou're welcome! I hope that it helps you!\n\n"},{"id":"515696","messageId":"PH0PR20MB6183CD9646B42F0DD552166792A92@PH0PR20MB6183.namprd20.prod.outlook.com","threadId":"63253","inReplyTo":"9274ED7A-0267-43E3-9AE1-DC0BBEC81E69@gmail.com","subject":"Re: problem w/recursive, submodules?","fromName":"Torpey, Bill","fromEmail":"bill.torpey@broadridge.com","sentAt":"2025-04-04T19:42:08Z","receivedAt":"2025-04-05T11:44:13Z","isPatch":false,"sender":{"key":"bill.torpey@broadridge.com","avatar":null},"body":"- I'm afraid that we're unable to expose the internal repo because of co. policies.  Other than wrapper scripts for our internal dev. environment, the only thing of note in the internal repo is the .gitmodules file, which consists of the following:\n\n[submodule \"staging\"]\n\tpath = src\n\tbranch = staging\n\turl = https://github.com/nyfix/OpenMAMA-omnm.git\n\n\n- We've found that the problem is avoided if we execute the following command *after* doing the recursive clone.  Not sure why this is necessary -- shouldn't the recursive clone handle that on its own?\n\ngit submodule update --remote --recursive\n\n\n- It sounds like you're suggesting that we need to go into the submodule and manually checkout a specific commit?  Shouldn't the recursive clone pick up the HEAD of the submodule branch?\n\n\nIf it sounds like we don't quite understand what git is doing here, that's a fair statement ;-)  In particular, only the submodule branch is exposed to us, but it sounds like git internally stores the commit hash instead?\n\nThanks again for your help!\n________________________________________\nFrom: Lucas Seiki Oshiro <lucasseikioshiro@gmail.com>\nSent: Friday, April 4, 2025 10:59 AM\nTo: Torpey, Bill <Bill.Torpey@broadridge.com>\nCc: git@vger.kernel.org <git@vger.kernel.org>\nSubject: Re: problem w/recursive, submodules?\n \nThis Message Is From an Unknown Sender\nYou have not previously corresponded with this sender.\n \n> $ git clone -v -b master --recursive ssh://git@<internal>/nyfix/OpenMAMA-omnm.git\n\nIt's harder to understand the problem without being able to\nreproduce it. Do you have another example with public\nrepositories?\n\n> The commit hash from the recursive clone is the hash of a different (more recent) commit:\n\nIf I understood it correctly, it looks to me that it's\nanother case of one of the most common misconceptions about\nsubmodules: they always reference a specific commit instead of\nbranch or the latest commit.\n\nIf you want your repository to reference other commit of your\nsubmodule, you'll need to manually change this and commit this\nchange. One way you can do that:\n\ncd <submodule>\ngit checkout <new commit of the submodule>\ncd -\ngit add <submodule>\ngit commit\n\nNote that you'll need to do that every time you want you want\nyour repository to reference other commit of the submodule.\n\nThis message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system.\n"}]}