From: Scott Lamb <slamb@slamb.org>
Date: Fri, 08 Jun 2007 21:31:32 GMT
Subject: Re: git-p4 fails when cloning a p4 depo.
Message-ID: <4669CAB4.5080507@slamb.org>
In-Reply-To: <1621f9fa0706081113w7bb765ebx74f03a7407b753cb@mail.gmail.com>

Benjamin Sergeant wrote:
> A perforce command with all the files in the repo is generated to get
> all the file content.
> Here is a patch to break it into multiple successive perforce command
> who uses 4K of parameter max, and collect the output for later.
> 
> It works, but not for big depos, because the whole perforce depo
> content is stored in memory in P4Sync.run(), and it looks like mine is
> bigger than 2 Gigs, so I had to kill the process.

Hmm. I tried git-p4 out on Sunday, and it definitely didn't do this 
then...this commit must have showed up since:

     commit 6460cf12df4556f889888b5f0b49e07040747e6f
     Author: Han-Wen Nienhuys <hanwen@google.com>
     Date:   Wed May 23 18:49:35 2007 -0300

         Read p4 files in one batch.

I believe HEAD at the time was this:

     commit 458e0545cb3dd03af9cd1a61480cbb764639043a
     Author: Simon Hausmann <simon@lst.de>
     Date:   Mon May 28 19:24:57 2007 +0200

         Fix typo in listExistingP4Branches that broke sync.

so you might try checking out an old version, and I'll go RTFM on 
reading merge history, because I can't figure out when this happened.

> 
> 
> diff --git a/git-p4 b/git-p4
> index 36fe69a..906b193 100755
> --- a/git-p4
> +++ b/git-p4
> @@ -703,9 +703,22 @@ class P4Sync(Command):
>         if not files:
>             return
> 
> -        filedata = p4CmdList('print %s' % ' '.join(['"%s#%s"' % 
> (f['path'],
> -                                                                 f['rev'])
> -                                                    for f in files]))
> +        # We cannot put all the files on the command line
> +        # OS have limitations on the max lenght of arguments
> +        # POSIX says it's 4096 bytes, default for Linux seems to be 130 K.
> +        # and all OS from the table below seems to be higher than POSIX.
> +        # See http://www.in-ulm.de/~mascheck/various/argmax/

No need to hardcode - from Python this is 
os.sysconf(os.sysconf_names['SC_ARG_MAX'])

> +        chunk = ''
> +        filedata = []
> +        for i in xrange(len(files)):
> +            f = files[i]
> +            chunk += '"%s#%s" ' % (f['path'], f['rev'])
> +            if len(chunk) > 4000 or i == len(files)-1:
> +                data = p4CmdList('print %s' % chunk)
> +                if "p4ExitCode" in data[0]:
> +                    die("Problems executing p4. Error: [%d]." %
> (data[0]['p4ExitCode']));
> +                filedata.extend(data)
> +                chunk = ''
> 
>         j = 0;
>         contents = {}
> @@ -1486,3 +1499,5 @@ def main():
> 
> if __name__ == '__main__':
>     main()
> +
> +# vim: set filetype=python sts=4 sw=4 et si :