From: Benjamin Sergeant <bsergean@gmail.com>
Date: Sun, 17 Jun 2007 16:09:00 GMT
Subject: Re: git-p4 fails when cloning a p4 depo.
Message-ID: <1621f9fa0706170909s779b471fl872c027b8e7b446c@mail.gmail.com>
In-Reply-To: <200706171011.52492.simon@lst.de>

On 6/17/07, Simon Hausmann <simon@lst.de> wrote:
> On Tuesday 12 June 2007 03:13:17 Han-Wen Nienhuys wrote:
> > Benjamin Sergeant escreveu:
> > > A perforce command with all the files in the repo is generated to get
> > > all the file content.
> > > Here is a patch to break it into multiple successive perforce command
> > > who uses 4K of parameter max, and collect the output for later.
> > >
> > > It works, but not for big depos, because the whole perforce depo
> > > content is stored in memory in P4Sync.run(), and it looks like mine is
> > > bigger than 2 Gigs, so I had to kill the process.
> >
> > General idea of the patch is ok.  some nits:
> > > +        chunk = ''
> > > +        filedata = []
> > > +        for i in xrange(len(files)):
> >
> > why not
> >
> >   for f in files:
> >
> > ?
>
> It seems 'i' is used a bit later. Is there a nicer way to express this in
> python?
>
> > > +            f = files[i]
> > > +            chunk += '"%s#%s" ' % (f['path'], f['rev'])
> > > +            if len(chunk) > 4000 or i == len(files)-1:
> >
> > 4k seems reasonable enough, but can you take the min() with
> > os.sysconf('SC_ARG_MAX') ?
> >
> > Can you address this and resend so we can apply the patch?
> > Thanks.
>
> Since I ran into the very problem of a too long commandline myself yesterday I
> took the liberty of adding the SC_ARG_MAX bit to Benjamin's patch and
> comitting it then.
>

Cool.

(probably useless but)
For what it's worth ; Here is a tar file with 2 patchs:
 - The original one
 - The second one that adds the SC_ARG_MAX

BTW, doing a += on a string is not supposed to be fast, appending elem
to a sequence and then using ' '.join on them to build the big string
is said to be faster (I did not timed it thought). (in the second
patch also)

Thanks,
Benjamin.


>
> Simon
>
>