# failed submodule update re-run results in no checked out files? 3 messages from 2016-02-17 to 2016-02-18. Participants: Jacob Keller, Stefan Beller. Thread: https://gitlist.dev/t/41436 ## Jacob Keller, 2016-02-17 23:54 Subject: failed submodule update re-run results in no checked out files? Message-ID: URL: https://gitlist.dev/e/CA%2BP7%2BxoF1Rz_43-v3NzGk1Hxp%3DyhF3toh1nFNQFMRVWr5q8STg%40mail.gmail.com ``` Hi, I am having an issue currently when using Git with a remote server which has a limited number of ssh connections. The ssh server sometimes closes connections due to too many concurrent connections. I will get the following output from git in this case when performing a submodule update of a submodule which is not yet currently cloned/checked out. stdout: Cloning into 'src/SHARED'... stderr: Total 10288 (delta 7577), reused 10190 (delta 7577) Received disconnect from 10.96.8.71: 7: Too many concurrent connections fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. Unable to fetch in submodule path 'src/SHARED' The submodule is not cloned successfully, and this occurs somewhere in the middle of the process. If I run the command a 2nd time, git submodule update --remote src/SHARED, I get a successful run, but the files are not actually checked out. I believe this is because the clone that failed did succeed in getting the repository into a state where all the files are "removed" so a further submodule update will do nothing since it's "already" checked out at the correct commit. Am I right in my understanding? Is this a bug? I believe I can fix this using --force. Note that i don't yet currently have a reliable reproduction of this for various reasons, not least of which is that simulating network error is difficult. Any thoughts on this? Should I just have my script that runs my continuous integration builds add a check to ensure files are checked out? Is "--force" enough to get the submodule to be re-checked out even if it's already checked out at the location? Thanks, Jake ``` ## Stefan Beller, 2016-02-18 00:15 Subject: Re: failed submodule update re-run results in no checked out files? Message-ID: URL: https://gitlist.dev/e/CAGZ79kaM5JREvfCmNMn8FQm1mutmDB3e8DYmtxMEzmUO5mqccw%40mail.gmail.com In-Reply-To: ``` On Wed, Feb 17, 2016 at 3:54 PM, Jacob Keller wrote: > Hi, > > I am having an issue currently when using Git with a remote server > which has a limited number of ssh connections. > > The ssh server sometimes closes connections due to too many concurrent > connections. I will get the following output from git in this case > when performing a submodule update of a submodule which is not yet > currently cloned/checked out. > > stdout: Cloning into 'src/SHARED'... Which version of Git are you using? (does it include origin/sb/submodule-parallel-update, which rewrites lots of the relevant code? Also it introduces parallelism, which may not be anticipated by the server admin) > > stderr: Total 10288 (delta 7577), reused 10190 (delta 7577) Seeing both stdout and stderr, I assume you're on master or even behind that, which doesn't include the rewrite. > Received disconnect from 10.96.8.71: 7: Too many concurrent connections > fatal: Could not read from remote repository. > > Please make sure you have the correct access rights > and the repository exists. > Unable to fetch in submodule path 'src/SHARED' > > The submodule is not cloned successfully, and this occurs somewhere in > the middle of the process. I wonder if the client should retry each submodule at least once, in case of transient errors such as this ssh config having too many connections. > > If I run the command a 2nd time, > > git submodule update --remote src/SHARED, > > I get a successful run, but the files are not actually checked out. The submodules are cloned via clone --no-checkout. This is because you may have a custom update strategy configured, (or a preset such as "none" which would clone but do nothing further) > I > believe this is because the clone that failed did succeed in getting > the repository into a state where all the files are "removed" so a > further submodule update will do nothing since it's "already" checked > out at the correct commit. you can pass --force into "git submodule update" which passes that flag along to the checkout. > > Am I right in my understanding? Is this a bug? I believe I can fix > this using --force. > > Note that i don't yet currently have a reliable reproduction of this > for various reasons, not least of which is that simulating network > error is difficult. I have a similar problem which I am debugging currently with the new code. (~1000 submodules which may fail randomly; This is why I wonder about either automated retries or ignoring the errors) > > Any thoughts on this? Should I just have my script that runs my > continuous integration builds add a check to ensure files are checked > out? Is "--force" enough to get the submodule to be re-checked out > even if it's already checked out at the location? I believe so. Stefan > > Thanks, > Jake > -- > To unsubscribe from this list: send the line "unsubscribe git" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ``` ## Jacob Keller, 2016-02-18 00:26 Subject: Re: failed submodule update re-run results in no checked out files? Message-ID: URL: https://gitlist.dev/e/CA%2BP7%2Bxqj2i7EFpOhsecRPniHL68sEcmnaCB%2Bm6VuO4U3dfnwog%40mail.gmail.com In-Reply-To: ``` On Wed, Feb 17, 2016 at 4:15 PM, Stefan Beller wrote: > On Wed, Feb 17, 2016 at 3:54 PM, Jacob Keller wrote: >> Hi, >> >> I am having an issue currently when using Git with a remote server >> which has a limited number of ssh connections. >> >> The ssh server sometimes closes connections due to too many concurrent >> connections. I will get the following output from git in this case >> when performing a submodule update of a submodule which is not yet >> currently cloned/checked out. >> >> stdout: Cloning into 'src/SHARED'... > > Which version of Git are you using? > (does it include origin/sb/submodule-parallel-update, which rewrites > lots of the relevant code? Also it introduces parallelism, which may not be > anticipated by the server admin) > I am not 100% sure, but multiple versions. I have multiple virtual machines that run automated tests, and I have not kept all the VMs up to date. >> >> stderr: Total 10288 (delta 7577), reused 10190 (delta 7577) > > Seeing both stdout and stderr, I assume you're on master or even behind that, > which doesn't include the rewrite. > I think I am using 2.4.x right now. >> Received disconnect from 10.96.8.71: 7: Too many concurrent connections >> fatal: Could not read from remote repository. >> >> Please make sure you have the correct access rights >> and the repository exists. >> Unable to fetch in submodule path 'src/SHARED' >> >> The submodule is not cloned successfully, and this occurs somewhere in >> the middle of the process. > > I wonder if the client should retry each submodule at least once, in case of > transient errors such as this ssh config having too many connections. > Well, in my case it wouldn't be "transient" and my process already does an automatic re-run when it fails. (Technically I am using the Java wrapper around CLI Git for Jenkins, which does this re-wrapping). It is possible something is wrong in the java wrapper, but I determined that it boils down to this sequence. I have yet to be able to reproduce a failure locally though without the Java wrapper (due to being difficult to generate enough simultaneous connections to cause the failure). >> >> If I run the command a 2nd time, >> >> git submodule update --remote src/SHARED, >> >> I get a successful run, but the files are not actually checked out. > > The submodules are cloned via clone --no-checkout. > This is because you may have a custom update strategy configured, > (or a preset such as "none" which would clone but do nothing further) > I see. >> I >> believe this is because the clone that failed did succeed in getting >> the repository into a state where all the files are "removed" so a >> further submodule update will do nothing since it's "already" checked >> out at the correct commit. > > you can pass --force into "git submodule update" which passes that > flag along to the checkout. > Right, this is what worked as a workaround for me. >> >> Am I right in my understanding? Is this a bug? I believe I can fix >> this using --force. >> >> Note that i don't yet currently have a reliable reproduction of this >> for various reasons, not least of which is that simulating network >> error is difficult. > > I have a similar problem which I am debugging currently with the > new code. (~1000 submodules which may fail randomly; This is > why I wonder about either automated retries or ignoring the errors) > >> >> Any thoughts on this? Should I just have my script that runs my >> continuous integration builds add a check to ensure files are checked >> out? Is "--force" enough to get the submodule to be re-checked out >> even if it's already checked out at the location? > > I believe so. > > Stefan > Ok, sounds like for now I need to extend my setup to use --force instead, which will resolve the issue for me. I like the idea of having git itself perform some retries incase of transient failures though. >> >> Thanks, >> Jake >> -- >> To unsubscribe from this list: send the line "unsubscribe git" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html ```