{"thread":{"id":"62045","subject":"Is it possible to resume download on single file.","startedAt":"2024-09-03T14:00:51Z","lastAt":"2024-09-06T09:30:52Z","messageCount":2,"participants":["tao lv","Johannes Schindelin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"502036","messageId":"CAFePT4wau_YG-3wOs94YrJ42im+x=Tp98Fu6vBJUG7pVDviO0Q@mail.gmail.com","threadId":"62045","inReplyTo":null,"subject":"Is it possible to resume download on single file.","fromName":"tao lv","fromEmail":"thebookofunknowable@gmail.com","sentAt":"2024-09-03T14:00:39Z","receivedAt":"2024-09-03T14:00:51Z","isPatch":false,"sender":{"key":"thebookofunknowable@gmail.com","avatar":null},"body":"Hi:\n\nIs it possible to resume a broken transfer on a per-file basis in Git?\nI am currently using git version 2.44.0.windows.1 (Git for Windows).\nDue to poor connection between my workplace and GitHub, cloning\nprojects often fails.\nTherefore, I often operate through the following process: (the\nrepository is chosen as git only for example)\n\n```\ngit clone --no-checkout --depth=1 --filter=blob:none https://github.com/git/git\ncd git\ngit checkout HEAD -- \"*\"\n```\n\nThis allows me to download Git files one by one.\nHowever, sometimes a single file in the remote repository may be too\nlarge, causing even this method to fail to download all repository\ncontents.\nSo, I want to know if it is possible to resume transfers when\ndownloading Git files? Or will this feature be added to git in the\nfuture?\n"},{"id":"502315","messageId":"b6e1ae0e-871d-334f-b97c-b8351daca1b0@gmx.de","threadId":"62045","inReplyTo":"CAFePT4wau_YG-3wOs94YrJ42im+x=Tp98Fu6vBJUG7pVDviO0Q@mail.gmail.com","subject":"Re: Is it possible to resume download on single file.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2024-09-06T09:30:49Z","receivedAt":"2024-09-06T09:30:52Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Tao,\n\nOn Tue, 3 Sep 2024, tao lv wrote:\n\n> Is it possible to resume a broken transfer on a per-file basis in Git?\n> I am currently using git version 2.44.0.windows.1 (Git for Windows).\n> Due to poor connection between my workplace and GitHub, cloning\n> projects often fails.\n> Therefore, I often operate through the following process: (the\n> repository is chosen as git only for example)\n>\n> ```\n> git clone --no-checkout --depth=1 --filter=blob:none https://github.com/git/git\n> cd git\n> git checkout HEAD -- \"*\"\n> ```\n>\n> This allows me to download Git files one by one.\n> However, sometimes a single file in the remote repository may be too\n> large, causing even this method to fail to download all repository\n> contents.\n> So, I want to know if it is possible to resume transfers when\n> downloading Git files? Or will this feature be added to git in the\n> future?\n\nGit's design does not allow for resumable clones at this time.\n\nThere are a few more or less hacky ways I can think of to address this\nproblem:\n\n1) Prime the clone with a local \"reference\" repository\n\nIf one of your colleagues already has cloned the repository in question\nsuccessfully (and _not_ partially), and if both of you can access a local\nnetwork drive, then this colleague could initialize a bare repository on\nsaid network drive that can then be used to accelerate future clones.\n\nThe reference repository would be initialized somewhat like this:\n\n\tcd /path/to/repository\n\tgit clone --mirror --bare . /path/to/network/drive/repository\n\nThe clone operation would look somewhat like this:\n\n\tgit clone --reference /path/to/network/drive/repository \\\n\t\t--dissociate https://github.com/the/repository\n\n2) Using bundle URIs\n\nThis assumes that your workplace has _some_ server that has both good\nconnection to GitHub as well as can be accessed reliably from your\nworkstation.\n\nThe idea is to maintain a set of static bundle files (for details, see\nhttps://git-scm.com/docs/bundle-uri) that can not only be distributed via\nContent Delivery Networks (\"CDN\") but also via servers that allow\nresumable downloads. These bundles would take the role of the \"reference\nrepository\" in the previous bullet point.\n\nOne tool to make such a setup relatively easy to initialize and maintain\nis this: https://github.com/git-ecosystem/git-bundle-server#readme\n\nThis does require a server that has reliable network connections to GitHub\n(to update the bundles) and also to your workstation, and it requires a\nbit of work to maintain.\n\n3) HACK! Downloading the raw blob objects' contents\n\nSince the problem seems to come from the blob objects, you could download\nthe raw file contents and feed them to `git hash-object -w --stdin` to\npopulate the local repository. That way, your partial clone would not need\nto fetch them via the (non-resumable) `git fetch` protocol.\n\nOne way to download those raw contents is via the \"Raw\" URLs like\nhttps://github.com/git/git/raw/v2.46.0/README.md. You can ensure that the\ncorresponding Git object is present in your local repository (and will\ntherefore not need to be fetched again) via something like this:\n\n\tcurl -Lfs https://github.com/git/git/raw/v2.46.0/README.md |\n\tgit hash-object -w --stdin\n\nThat can get a bit tedious if there are many blob objects that need to be\nimported into the local repository; To counter that, you could download a\ntar archive via something like https://github.com/git/git/tarball/v2.46.0\nand then import it via something like `import-tars.perl` (see\nhttps://github.com/git/git/blob/v2.46.0/contrib/fast-import/import-tars.perl).\nAccording to this public information, the tar archives are cached and\ntherefore the download should be resumable:\nhttps://docs.github.com/en/repositories/working-with-files/using-files/downloading-source-code-archives#stability-of-source-code-archives\n\nI hope this information is useful to you. If you manage to improve your\nsituation, I would be curious to learn what worked for you.\n\nCiao,\nJohannes\n"}]}