Re: Why is "git fetch --prune" so much slower than "git remote prune"?

From: Jeff King <peff@peff.net>
Date: Mar 19, 2015, 21:26 UTC
Message-ID: <20150319212640.GB8363@peff.net>
In-Reply-To: <xmqqpp84iye2.fsf@gitster.dls.corp.google.com>

On Thu, Mar 19, 2015 at 12:24:21PM -0700, Junio C Hamano wrote:

Show 17 quoted lines

> > For pruning, we can't use a ref_transaction as it is currently
> > implemented because it would fail if any of the reference deletions
> > failed. But in this case I think if any deletions fail, we would prefer
> > to emit a warning but keep going.
> 
> I am not quite sure what you mean here.  I agree with you if you
> meant "we shouldn't fail the fetch only because 'fetch --prune'
> failed to remove only one of the remote-tracking refs that are no
> longer there" but that can easily be solved by the pruning phase
> into a separate transaction.  If you meant "we would rather remove
> origin/{a,b} non-atomically when we noticed that origin/{a,b,c} are
> all gone than leaving all three intact only because we failed to
> remove origin/c for whatever reason", my knee-jerk reaction is "does
> it make practical difference to the end user between these two?"
> 
> What are the plausible cause of failing to prune unused
> remote-tracking refs?

I had assumed earlier that Michael meant to use a single ref_transaction for all updates. Thinking on it more, that is probably a bad idea, as it makes fetch atomic in a user-visible way, whereas currently the updates are always per-ref (i.e., some may fail, but we let others succeed). I don't know if people actually care or not (certainly the exit code of fetch represents all of the refs, so it is not like you could say "eh, git-fetch return 1, but it probably got the ref I wanted" without parsing the human-readable output).

If it is just a single atomic commit for all of the deletions, I agree it is less of a big deal. They are unlikely to fail, and when they do, you are not blocking the new refs from coming in.

I think the ref_transaction does have some smarts to handle a case where we are updating "refs/foo/bar" while "refs/foo" exists but is deleted in the transaction. We switched to pruning before updating in 10a6cc8 (fetch --prune: Run prune before fetching, 2014-01-02), so it is a non-issue, but what is there now is technically racy[1], and it would have been nice to let the ref-storage code handle it. I guess we still can if we introduce a "git fetch --atomic" option.

-Peff

[1] http://article.gmane.org/gmane.comp.version-control.git/239519