threads / discuss / 50419

Proposal: Output should push to different servers in parallel

Subject: Proposal: Output should push to different servers in parallel

## tl;dr

3 messages between Feb 6, 2019 and Feb 7, 2019.

replies: 2people: 3as markdown or json

Victor Porton· Feb 6, 2019, 22:33 UTC · lore

I experienced a slowdown in Git pushing when I push to more than one server.

I propose:

Run push to several servers in parallel.

Not to mix the output, nevertheless serialize the output, that is for example cache the output of the second server push and start to output it immediately after the first server push is finished.

This approach combines the advantages of the current way (I suppose it is so) to serialize pushes: first push to the first server, then to the second, etc. and of my idea to push in parallel.

I think the best way would be use multithreading, but multiprocessing would be a good quick solution.

Ævar Arnfjörð Bjarmason· Feb 7, 2019, 08:37 UTC · re: Victor Porton · lore

Re: Proposal: Output should push to different servers in parallel

On Wed, Feb 06 2019, Victor Porton wrote:

Show 16 quoted lines

> I experienced a slowdown in Git pushing when I push to more than one server.
>
> I propose:
>
> Run push to several servers in parallel.
>
> Not to mix the output, nevertheless serialize the output, that is for
> example cache the output of the second server push and start to output
> it immediately after the first server push is finished.
>
> This approach combines the advantages of the current way (I suppose it
> is so) to serialize pushes: first push to the first server, then to
> the second, etc. and of my idea to push in parallel.
>
> I think the best way would be use multithreading, but multiprocessing
> would be a good quick solution.

This seems like a reasonable idea, until such time as someone submits patches to implement this in git you can do this with some invocation of GNU parallel -k, i.e. operate on N remotes in parallel, and use the -k option to buffer up all their output and present it in sequence.

Junio C Hamano· Feb 7, 2019, 18:26 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: Proposal: Output should push to different servers in parallel

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> This seems like a reasonable idea, until such time as someone submits
> patches to implement this in git you can do this with some invocation of
> GNU parallel -k, i.e. operate on N remotes in parallel, and use the -k
> option to buffer up all their output and present it in sequence.

Stopping the message there makes it sound like a polite way to say "a generic tool to allow you doing it on anything, not limited to Git, is already available, and a solution specific to Git is unwanted."

I wanted to follow up with something that says "The 'parallel' tool works in the meantime, but here are examples of very useful things that we would not be able to live without that 'parallel' wouldn't let us do, and we need a Git specific solution to obtain that", but I am coming up with empty, so perhaps indeed we do not want a Git specific solution ;-)