Re: git-p4import.py robustness changes

From: Scott Lamb <slamb@slamb.org>
Date: Jun 3, 2007, 20:12 UTC
Message-ID: <839AEF71-ED29-4A79-BE97-C79EAFEDC466@slamb.org>
In-Reply-To: <200706031511.31157.simon@lst.de>

On Jun 3, 2007, at 6:11 AM, Simon Hausmann wrote:

Show 11 quoted lines

> On the topic of git integration with perforce, what are the chances  
> of getting
> git-p4 ( http://repo.or.cz/w/fast-export.git ) into git's contrib/ 
> fast-export
> area? :)
>
> git-p4 can do everything git-p4import can do plus a lot more (it  
> can track
> multiple branches, it's a hell of a lot faster, it can export back  
> to p4 and
> it also works on Windows!).

I missed that one...I just saw Tailor and the Perl script someone else had written.

Ergh. git-p4 imports both "subprocess" and "popen2" and also uses "system" and "os.popen". Why use four different modules to launch git and p4?

The branch support's interesting. Have you considered tracking integration history? I was pondering it and am not sure if it's feasible. Perforce doesn't seem to have an efficient way of displaying it (just "p4 integrates" that will fetch *all* revisions even if you want incremental results and "p4 filelog" which would have to be done on each file). Also, I think there's some mismatch between the Perforce and git models.

git-p4import.py should work fine on Windows, too - the binary mode on the pipe should be all handled by "subprocess", and git-p4's data.replace("\r\n", "\n") is not necessary if you use "LineEnd: unix" or "share" in the Perforce client specification.

As for performance...hmm. Looks like git-p4import.py runs these commands for each Perforce revision:

     realtime  operation
         3.4%  p4 describe -s N
        66.6%  p4 sync ...@N
    [*] 10.2%  git ls-files -m -d -o -z | git update-index --add -- 
remove -z --stdin
         2.6%  git rev-parse --verify HEAD
         4.2%  git write-tree
         2.8%  git commit-tree xxxxxx
         7.5%  git tag -f p4/N xxxxxx
         2.7%  git update-ref HEAD xxxxxx

That's with Perforce running over the network. Are you running locally?

git-p4 seems to use "git fast-import". I guess the big performance improvement there is removing the ls-files operation? So we're talking about a 0-10% speedup, right? Plus some fork()/exec() overhead.

[*] - Note that I just discovered a big performance regression in my patches. Reading the ls-files into Python, through a regexp, and back out through update-index was a horrible idea. The times above are with that fixed.

-- 
Scott Lamb <http://www.slamb.org/>