Re: What's cooking in git.git (Apr 2011, #04; Tue, 12)

From: Michael Haggerty <mhagger@alum.mit.edu>
Date: Apr 15, 2011, 08:31 UTC
Message-ID: <4DA80253.6010204@alum.mit.edu>
In-Reply-To: <BANLkTimeiH_ohJ6yGTU0Ei3t2xvUz0zCUA@mail.gmail.com>

On 04/15/2011 08:24 AM, Piotr Krukowiecki wrote:

Show 26 quoted lines

> On Thu, Apr 14, 2011 at 5:21 PM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
>> On 04/14/2011 03:38 PM, Piotr Krukowiecki wrote:
>>> On Wed, Apr 13, 2011 at 12:43 AM, Junio C Hamano <gitster@pobox.com> wrote:
>>>> * mh/git-svn-automkdirs (2011-04-01) 1 commit
>>>>  (merged to 'next' on 2011-04-03 at 7fa4978)
>>>>  + git-svn: add an option to skip the creation of empty directories
>>>>
>>>> Should be safe, but I'd like an Ack from git-svn folks.
>>>
>>> I wanted to test performance of this change - what's the best way to do it?
>>>
>>> I tried some ideas, but rebase was too fast for performance measurements.
>>> I did not have new commits, but just some old, already in trunk changes, which
>>> I tried to rebase - probably it was just fast forward?
>>
>> The unhandled.log.gz file for trunk of our main project is 14 Mb and
>> uncompresses to 233 Mb.  About 90% of it consists of svn:mergeinfo
>> properties for file that were copied or renamed within the repository;
>> most of the rest is other random SVN file properties.
>>
>> With such a huge unhandled.log file, "git svn mkdirs" took about 10s for
>> me.  I believe that "git svn rebase" should take at least as long, even
>> if it is a fast-forward.
> 
> That might be the reason - my unhandled.log is 17MB (unpacked) and mkdirs
> takes 0.5s

Yes, it is also my assumption that parsing so much text in Perl is what causes the slowdown. But as long as git-svn insists on plonking so much information in unhandled.log but then handling it anyway, it seems like a good policy to prevent it from reading this file any more than necessary. And for me, the creation of empty directories is not worth 10s. (Even your 0.5s is pretty slow by git standards :-) ).

An alternative might be to move the emptydir information from unhandled.log to a separate file. The "empty_dir" lines in my unhandled log are only about 0.1% of the file contents, and should be parseable in a negligible amount of time. But moving the data would presumably have implications for backwards compatibility.

[Are there any design documents for git-svn? I have had a hard time deciphering the code and understanding what data it stores and where.]

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/