Re: [RFC] upload-pack deadlock

> In a pthread-enabled version of upload-pack, there's a race condition
> that can cause a deadlock on the fflush(NULL) we call from run-command.
>
> What happens is this:
>
>   1. Upload-pack is informed we are doing a shallow clone.
>
>   2. We call start_async() to spawn a thread that will generate rev-list
>      results to feed to pack-objects. It gets a file descriptor to a
>      pipe which will eventually hook to pack-objects.
>
>   3. The rev-list thread uses fdopen to create a new output stream
>      around the fd we gave it, called pack_pipe.
>
>   4. The thread writes results to pack_pipe. Outside of our control,
>      libc is doing locking on the stream. We keep writing until the OS
>      pipe buffer is full, and then we block in write(), still holding
>      the lock.
>
>   5. The main thread now uses start_command to spawn pack-objects.
>      Before forking, it calls fflush(NULL) to flush every stdio output
>      buffer. It blocks trying to get the lock on pack_pipe.
>
> And we have a deadlock. The thread will block until somebody starts
> reading from the pipe. But nobody will read from the pipe until we
> finish flushing to the pipe.

> There are a few possible solutions:
>
>   1. Flip the order of initialization, so that we don't start writing
>      anything until the pack-objects reader is already in place. That
>      works in this case, and the patch is at the end of this mail. The
>      biggest problem is that it doesn't fix the general case.