threads / discuss / 13632

Re: perforce import: git-p4 memory usage

Subject: Re: perforce import: git-p4 memory usage

## tl;dr

3 messages between May 23, 2008 and May 23, 2008.

replies: 2people: 3as markdown or json

Luke Diamand· May 23, 2008, 10:07 UTC · lore

perforce import: git-p4 memory usage

Hi!

I'm trying to import part of a perforce repo with git-p4.

However, git-p4 appears to try to read all the repo into memory (and moreover seems to need about twice as much memory as repo).

Once it runs out of swap, it dies (unsurprisingly).

I think it's failing in readP4Files(), where it appears to read the entire repository in one go with "p4 -G -x - print".

Can I just rework this function to do stuff one file at a time? Or is that dumb?

Thanks Luke Diamand

Andreas Ericsson· May 23, 2008, 11:25 UTC · re: Luke Diamand · lore

Luke Diamand wrote:

Show 15 quoted lines

> Hi!
> 
> I'm trying to import part of a perforce repo with git-p4.
> 
> However, git-p4 appears to try to read all the repo into memory (and
> moreover seems to need about twice as much memory as repo).
> 
> Once it runs out of swap, it dies (unsurprisingly).
> 
> I think it's failing in readP4Files(), where it appears to read the
> entire repository in one go with "p4 -G -x - print".
> 
> Can I just rework this function to do stuff one file at a time? Or is
> that dumb?
>

It's probably a lot slower, and since git works with changesets it probably won't work all that good unless you somehow construct some middle-stage which you can then feed to git-fastimport or some such.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Logan Kennelly· May 23, 2008, 22:27 UTC · re: Luke Diamand · lore

On May 23, 2008, at 3:07 AM, Luke Diamand wrote:

Show 10 quoted lines

> However, git-p4 appears to try to read all the repo into memory (and
> moreover seems to need about twice as much memory as repo).
>
> Once it runs out of swap, it dies (unsurprisingly).
>
> I think it's failing in readP4Files(), where it appears to read the
> entire repository in one go with "p4 -G -x - print".
>
> Can I just rework this function to do stuff one file at a time? Or is
> that dumb?

I don't know if they were ever accepted into the mainline, but see the series of patches submitted by Tommy Thorn.

http://article.gmane.org/gmane.comp.version-control.git/72342

-- 
                                                         Logan Kennelly
       ,,,
      (. .)
--ooO-(_)-Ooo--