threads / discuss / 29001

git not resuming push

Subject: git not resuming push

## tl;dr

5 messages between Nov 23, 2011 and Nov 23, 2011.

replies: 4people: 3as markdown or json

Paul Brossier· Nov 23, 2011, 03:58 UTC · lore

Hi!

I'm trying to upload large amount of data from a connection that often cuts.

If the connection fails after uploading part of the data, it seems I need to start over from zero again. Is there a way to resume the upload instead?

Here is an example:

$ git push Counting objects: 504, done. Delta compression using up to 2 threads. Compressing objects: 100% (475/475), done. Write failed: Broken pipe1/476), 533.78 MiB | 60 KiB/s fatal: The remote end hung up unexpectedly error: pack-objects died of signal 13 error: failed to push some refs to 'myhost.org:/git/myrepo/'

$ git push Counting objects: 504, done. Delta compression using up to 2 threads. Compressing objects: 100% (475/475), done. Writing objects: 12% (61/476), 89.34 MiB | 23 KiB/s

Thanks, piem

Jeff King· Nov 23, 2011, 05:02 UTC · re: Paul Brossier · lore

Re: git not resuming push

On Tue, Nov 22, 2011 at 08:58:56PM -0700, Paul Brossier wrote:

> If the connection fails after uploading part of the data, it seems I
> need to start over from zero again. Is there a way to resume the upload
> instead?

No, there isn't a way to resume just using push.

If you have shell access on the server, one workaround you can do is to create a bundle with the commits in question, upload it via some resumable protocol (like sftp, http, rsync, etc), possibly taking many attempts, and then fetch the result on the server side from the bundle into the repository.

See "git help bundle" for some examples.

-Peff

Junio C Hamano· Nov 23, 2011, 06:24 UTC · re: Jeff King · lore

Re: git not resuming push

Jeff King <peff@peff.net> writes:

Show 15 quoted lines

> On Tue, Nov 22, 2011 at 08:58:56PM -0700, Paul Brossier wrote:
>
>> If the connection fails after uploading part of the data, it seems I
>> need to start over from zero again. Is there a way to resume the upload
>> instead?
>
> No, there isn't a way to resume just using push.
>
> If you have shell access on the server, one workaround you can do is to
> create a bundle with the commits in question, upload it via some
> resumable protocol (like sftp, http, rsync, etc), possibly taking many
> attempts, and then fetch the result on the server side from the bundle
> into the repository.
>
> See "git help bundle" for some examples.

Another possibility, if it is the connection between you and the other side that is the problem, is to chunk your push in smaller pieces. That is, if you are trying to push out v3.0, you first push only to v1.0, then to v2.0, and then finally to v3.0.

Peff, by the way, wouldn't this request reminds of us of a scenario we discussed recently, which I said I would imagine would be common while you dismissed as not likely to be common?

Paul Brossier· Nov 23, 2011, 17:25 UTC · re: Junio C Hamano · lore

Re: git not resuming push

ok, that, with a bit of patience, should help!

thank you both for your answers,

piem

On 22/11/2011 23:24, Junio C Hamano wrote:

Show 28 quoted lines

> Jeff King<peff@peff.net>  writes:
>
>> On Tue, Nov 22, 2011 at 08:58:56PM -0700, Paul Brossier wrote:
>>
>>> If the connection fails after uploading part of the data, it seems I
>>> need to start over from zero again. Is there a way to resume the upload
>>> instead?
>>
>> No, there isn't a way to resume just using push.
>>
>> If you have shell access on the server, one workaround you can do is to
>> create a bundle with the commits in question, upload it via some
>> resumable protocol (like sftp, http, rsync, etc), possibly taking many
>> attempts, and then fetch the result on the server side from the bundle
>> into the repository.
>>
>> See "git help bundle" for some examples.
>
> Another possibility, if it is the connection between you and the other
> side that is the problem, is to chunk your push in smaller pieces. That
> is, if you are trying to push out v3.0, you first push only to v1.0, then
> to v2.0, and then finally to v3.0.
>
> Peff, by the way, wouldn't this request reminds of us of a scenario we
> discussed recently, which I said I would imagine would be common while you
> dismissed as not likely to be common?
>
>

Jeff King· Nov 23, 2011, 18:46 UTC · re: Junio C Hamano · lore

Re: git not resuming push

On Tue, Nov 22, 2011 at 10:24:12PM -0800, Junio C Hamano wrote:

Show 8 quoted lines

> Another possibility, if it is the connection between you and the other
> side that is the problem, is to chunk your push in smaller pieces. That
> is, if you are trying to push out v3.0, you first push only to v1.0, then
> to v2.0, and then finally to v3.0.
> 
> Peff, by the way, wouldn't this request reminds of us of a scenario we
> discussed recently, which I said I would imagine would be common while you
> dismissed as not likely to be common?

I didn't say it wasn't common; I said I didn't think it was the majority case, and that I didn't want a solution that only helped this case without covering the other cases.

Also, it isn't clear to me that an immediate-reconnect scheme would work. When the connection gets dropped, is it simply that TCP connection that goes out, and another one could immediately be made to resume? Or is the network actually out (even if only for a few seconds)?

-Peff