Re: [PATCH v2 2/3] fast-export: improve speed by skipping blobs

From: Junio C Hamano <gitster@pobox.com>
Date: May 6, 2013, 17:17 UTC
Message-ID: <7v8v3srpsa.fsf@alter.siamese.dyndns.org>
In-Reply-To: <20130506164046.GA20257@sigill.intra.peff.net>

Jeff King <peff@peff.net> writes:

> So yes, I think this is an obviously correct optimization. Thanks for
> clarifying, and sorry to be so slow.

No need to be sorry. It just shows that the log message could have been more helpful.

Here is what I tentatively queued.

commit 83582e91d22c66413b291d4d6d45bbeafddc2af9
Author: Felipe Contreras <felipe.contreras@gmail.com>
Date:   Sun May 5 17:38:53 2013 -0500

    fast-export: do not parse non-commit objects while reading marks file
    
    We read from the marks file and keep only marked commits, but in
    order to find the type of object, we are parsing the whole thing,
    which is slow, specially in big repositories with lots of big files.
    
    There's no need for that, we can query the object information with
    sha1_object_info().
    
    Before this, loading the objects of a fresh emacs import, with 260598
    blobs took 14 minutes, after this patch, it takes 3 seconds.
    
    This is the way fast-import does it. Also die if the object is not
    found (like fast-import).
    
    Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>