Re: [PATCH v2 2/3] fast-export: improve speed by skipping blobs

From: Jeff King <peff@peff.net>
Date: May 6, 2013, 16:20 UTC
Message-ID: <20130506162008.GB7992@sigill.intra.peff.net>
In-Reply-To: <7v7gjctabm.fsf@alter.siamese.dyndns.org>

On Mon, May 06, 2013 at 08:08:45AM -0700, Junio C Hamano wrote:

Show 10 quoted lines

> > I'm also not sure why your claim "we don't care about blobs" is true,
> > because naively we would want future runs of fast-export to avoid having
> > to write out the whole blob content when mentioning the blob again.
> 
> The existing documentation is fairly clear that marks for objects
> other than commits are not exported, and the import-marks codepath
> discards anything but commits, so there is no mechanism for the
> existing fast-export users to leave blob marks in the marks file for
> later runs of fast-export to take advantage of.  The second
> invocation cannot refer to such a blob in the first place.

OK. If the argument is "we do not write them, so do not bother reading them back in", I think that is reasonable. It could hurt anybody trying to run "fast-export" against a marks file created by somebody else, but that is also the same case that is being helped here (since otherwise, we would not be seeing blob entries at all).

I do not offhand know enough about the internals of import/export-style remote-helpers to say whether the "hurt" case even exists, let alone how common it is.

Show 7 quoted lines

> By discarding marks on blobs, we may be robbing some optimization
> possibilities, and by discarding marks on tags, we may be robbing
> some features, from users of fast-export; we might want to add an
> option "--use-object-marks={blob,commit,tag}" or something to both
> fast-export and fast-import, so that the former can optionally write
> marks for non-commits out, and the latter can omit non commit marks
> if the user do not need them. But that is a separate issue.

Yeah, that would allow the old behavior (and more) if anybody is hurt by this. It is nice if the order of implementation is "more features, then flip the default" because it provides an immediate escape hatch for anybody who is hurt by the change in default. But again, I do not know enough to say whether such hurt cases even exist.

-Peff