threads / discuss / 33525

is git-p4 compatible with p4/linux?

Subject: is git-p4 compatible with p4/linux?

## tl;dr

8 messages between Apr 17, 2013 and Apr 20, 2013.

replies: 7people: 3as markdown or json

Alexander Tomlinson· Apr 17, 2013, 04:31 UTC · lore

git-p4.py (1.8.2.1.418.gaec3f77) has at least two behaviors that seem to be incompatible with the version of p4 that I recently downloaded from perforce.com (P4/LINUX26X86_64/2013.1/610569).

TLDR: Is git-p4 written for an old version of p4 CLI with different
behavior?  Or for a windows or mac release of p4?  Or am I missing
something?

First issue -----------

git-p4 assumes the output of 'p4 print' adds a newline to the target. To work around this, git-p4.py strips the last char from symlinks as shown in the following snippet:

    if type_base =3D=3D "symlink":
        git_mode =3D "120000"
        # p4 print on a symlink contains "target\n"; remove the newline
        data =3D ''.join(contents)
        contents =3D [data[:-1]]
But my 'p4 print' does not output the newline:
    $ ls -l pcre
    lrwxrwxrwx 1 user group 12 Apr 16 10:27 pcre -> ../libs/pcre
    $ p4 print -q pcre | od -t x1a
    0000000  2e  2e  2f  6c  69  62  73  2f  70  63  72  65
             .   .   /   l   i   b   s   /   p   c   r   e
    0000014

If I use 'git p4 clone' the above file shows up in git as a symlink to '../libs/pcr'. I had another symlink whose target had a strlen of 1 and the 'git p4 clone' failed b/c after stripping the last char the result was an empty string.

Second issue ------------

git-p4 uses 'p4 print -q -o o FILE' to print a file to stdout. At least that is how I interpret this snippet:

    text = p4_read_pipe(['print', '-q', '-o', '-', file['depotFile']])

However, p4/Linux prints to stdout by default and '-o -' will save the output in a file named '-'.

My git and p4 versions:
    $ git --version
    git version 1.8.2.1.418.gaec3f77
    $ p4 -V
    Perforce - The Fast Software Configuration Management System.
    Copyright 1995-2013 Perforce Software.  All rights reserved.
    This product includes software developed by the OpenSSL Project
    for use in the OpenSSL Toolkit (http://www.openssl.org/)
    See 'p4 help legal' for full OpenSSL license information
    Version of OpenSSL Libraries: OpenSSL 1.0.1c 10 May 2012
    Rev. P4/LINUX26X86_64/2013.1/610569 (2013/03/19).
--Alex
Alexander Tomlinson· Apr 17, 2013, 04:37 UTC · re: Alexander Tomlinson · lore

Re: is git-p4 compatible with p4/linux?

Sorry about the =3D escapes. My first attempt to submit bounced b/c it was html and I missed cleaning some escapes. The snippet should be:

    if type_base == "symlink":
        git_mode = "120000"
        # p4 print on a symlink contains "target\n"; remove the newline
        data = ''.join(contents)
        contents = [data[:-1]]
--Alex
Pete Wyckoff· Apr 19, 2013, 00:09 UTC · re: Alexander Tomlinson · lore

Re: is git-p4 compatible with p4/linux?

alex@aivor.com wrote on Tue, 16 Apr 2013 23:31 -0500:
Show 7 quoted lines
> git-p4.py (1.8.2.1.418.gaec3f77) has at least two behaviors that
> seem to be incompatible with the version of p4 that I recently
> downloaded from perforce.com (P4/LINUX26X86_64/2013.1/610569).
> 
> TLDR: Is git-p4 written for an old version of p4 CLI with different
> behavior?  Or for a windows or mac release of p4?  Or am I missing
> something?

I had not done any testing beyond p4 12.2 (linux). But running the unit tests through 13.1 just now, they all pass.

    $ p4 -V
    Perforce - The Fast Software Configuration Management System.
    Copyright 1995-2013 Perforce Software.  All rights reserved.
    This product includes software developed by the OpenSSL Project
    for use in the OpenSSL Toolkit (http://www.openssl.org/)
    See 'p4 help legal' for full OpenSSL license information
    Version of OpenSSL Libraries: OpenSSL 1.0.1c 10 May 2012
    Rev. P4/LINUX26X86_64/2013.1/610569 (2013/03/19).
I'm using python 2.7.3.
Show 27 quoted lines
> First issue
> -----------
> 
> git-p4 assumes the output of 'p4 print' adds a newline to the
> target.  To work around this, git-p4.py strips the last char from
> symlinks as shown in the following snippet:
> 
>     if type_base =3D=3D "symlink":
>         git_mode =3D "120000"
>         # p4 print on a symlink contains "target\n"; remove the newline
>         data =3D ''.join(contents)
>         contents =3D [data[:-1]]
> 
> But my 'p4 print' does not output the newline:
> 
>     $ ls -l pcre
>     lrwxrwxrwx 1 user group 12 Apr 16 10:27 pcre -> ../libs/pcre
> 
>     $ p4 print -q pcre | od -t x1a
>     0000000  2e  2e  2f  6c  69  62  73  2f  70  63  72  65
>              .   .   /   l   i   b   s   /   p   c   r   e
>     0000014
> 
> If I use 'git p4 clone' the above file shows up in git as a
> symlink to '../libs/pcr'.  I had another symlink whose target had
> a strlen of 1 and the 'git p4 clone' failed b/c after stripping
> the last char the result was an empty string.

There wasn't an explict test for symlinks, but I threw one together quickly and it seems to work. Can you show a bit more information about anything that potentially might be odd with your install?

    arf-git-test$ ls -l symlink
    lrwxrwxrwx 1 pw pw 14 Apr 18 20:02 symlink -> symlink-target
    $ p4 fstat symlink
    ... depotFile //depot/symlink
    ... clientFile /run/shm/trash directory.t9802-git-p4-filetype/cli/symlink
    ... isMapped 
    ... headAction add
    ... headType symlink
    ... headTime 1366329740
    ... headRev 1
    ... headChange 6
    ... headModTime 1366329740
    ... haveRev 1
    $ p4 print -q symlink | od -t x1a
    0000000  73  79  6d  6c  69  6e  6b  2d  74  61  72  67  65  74  0a
	      s   y   m   l   i   n   k   -   t   a   r   g   e   t  nl
    0000017

No idea why I get an "nl" but you do not. If you run _without_ the "| od ...", then the shell prompt ends up on the same line as the output? Any interesting shell or shell settings?

Show 24 quoted lines
> Second issue
> ------------
> 
> git-p4 uses 'p4 print -q -o o FILE' to print a file to stdout.
> At least that is how I interpret this snippet:
> 
>     text = p4_read_pipe(['print', '-q', '-o', '-', file['depotFile']])
> 
> However, p4/Linux prints to stdout by default and '-o -' will save
> the output in a file named '-'.
> 
> My git and p4 versions:
> 
>     $ git --version
>     git version 1.8.2.1.418.gaec3f77
> 
>     $ p4 -V
>     Perforce - The Fast Software Configuration Management System.
>     Copyright 1995-2013 Perforce Software.  All rights reserved.
>     This product includes software developed by the OpenSSL Project
>     for use in the OpenSSL Toolkit (http://www.openssl.org/)
>     See 'p4 help legal' for full OpenSSL license information
>     Version of OpenSSL Libraries: OpenSSL 1.0.1c 10 May 2012
>     Rev. P4/LINUX26X86_64/2013.1/610569 (2013/03/19).

This code only happens on utf16 files. But running it by hand, I cannot reproduce the different behavior:

    $ p4 print -q //depot/f-ascii
    three
    line
    text
    $ p4 print -o - -q //depot/f-ascii
    three
    line
    $ ls ./-
    ls: cannot access ./-: No such file or directory
I'm again confused.  Any hints you can give would be helpful.
		-- Pete
Alexander Tomlinson· Apr 19, 2013, 01:34 UTC · re: Pete Wyckoff · lore

Re: is git-p4 compatible with p4/linux?

On Apr 18, 2013, at 7:09 PM, Pete Wyckoff <pw@padd.com> wrote:
Show 80 quoted lines
> alex@aivor.com wrote on Tue, 16 Apr 2013 23:31 -0500:
>> git-p4.py (1.8.2.1.418.gaec3f77) has at least two behaviors that
>> seem to be incompatible with the version of p4 that I recently
>> downloaded from perforce.com (P4/LINUX26X86_64/2013.1/610569).
>> 
>> TLDR: Is git-p4 written for an old version of p4 CLI with different
>> behavior?  Or for a windows or mac release of p4?  Or am I missing
>> something?
> 
> I had not done any testing beyond p4 12.2 (linux).  But running
> the unit tests through 13.1 just now, they all pass.
> 
>    $ p4 -V
>    Perforce - The Fast Software Configuration Management System.
>    Copyright 1995-2013 Perforce Software.  All rights reserved.
>    This product includes software developed by the OpenSSL Project
>    for use in the OpenSSL Toolkit (http://www.openssl.org/)
>    See 'p4 help legal' for full OpenSSL license information
>    Version of OpenSSL Libraries: OpenSSL 1.0.1c 10 May 2012
>    Rev. P4/LINUX26X86_64/2013.1/610569 (2013/03/19).
> 
> I'm using python 2.7.3.
> 
>> First issue
>> -----------
>> 
>> git-p4 assumes the output of 'p4 print' adds a newline to the
>> target.  To work around this, git-p4.py strips the last char from
>> symlinks as shown in the following snippet:
>> 
>>    if type_base =3D=3D "symlink":
>>        git_mode =3D "120000"
>>        # p4 print on a symlink contains "target\n"; remove the newline
>>        data =3D ''.join(contents)
>>        contents =3D [data[:-1]]
>> 
>> But my 'p4 print' does not output the newline:
>> 
>>    $ ls -l pcre
>>    lrwxrwxrwx 1 user group 12 Apr 16 10:27 pcre -> ../libs/pcre
>> 
>>    $ p4 print -q pcre | od -t x1a
>>    0000000  2e  2e  2f  6c  69  62  73  2f  70  63  72  65
>>             .   .   /   l   i   b   s   /   p   c   r   e
>>    0000014
>> 
>> If I use 'git p4 clone' the above file shows up in git as a
>> symlink to '../libs/pcr'.  I had another symlink whose target had
>> a strlen of 1 and the 'git p4 clone' failed b/c after stripping
>> the last char the result was an empty string.
> 
> There wasn't an explict test for symlinks, but I threw
> one together quickly and it seems to work.  Can you show
> a bit more information about anything that potentially might
> be odd with your install?
> 
>    arf-git-test$ ls -l symlink
>    lrwxrwxrwx 1 pw pw 14 Apr 18 20:02 symlink -> symlink-target
> 
>    $ p4 fstat symlink
>    ... depotFile //depot/symlink
>    ... clientFile /run/shm/trash directory.t9802-git-p4-filetype/cli/symlink
>    ... isMapped 
>    ... headAction add
>    ... headType symlink
>    ... headTime 1366329740
>    ... headRev 1
>    ... headChange 6
>    ... headModTime 1366329740
>    ... haveRev 1
> 
>    $ p4 print -q symlink | od -t x1a
>    0000000  73  79  6d  6c  69  6e  6b  2d  74  61  72  67  65  74  0a
> 	      s   y   m   l   i   n   k   -   t   a   r   g   e   t  nl
>    0000017
> 
> No idea why I get an "nl" but you do not.  If you run _without_
> the "| od ...", then the shell prompt ends up on the same line
> as the output?  Any interesting shell or shell settings?
> 
Yes.  The shell prompt is on the same line as the output:

$ PS1="FOOBAR$" FOOBAR$p4 print -q pcre ../libs/pcreFOOBAR$

Perhaps it is a configuration item on the server and/or client. It seems we are running the same version of p4. But just to be sure, check yours against mine:

$ cksum $(which p4) 3254530484 2420552 /usr/bin/p4

If yours if different, can you email a copy of your p4 executable to me so I can check if it works differently than mine?

I will also check with coworkers here to see how their client behaves.
Show 41 quoted lines
>> Second issue
>> ------------
>> 
>> git-p4 uses 'p4 print -q -o o FILE' to print a file to stdout.
>> At least that is how I interpret this snippet:
>> 
>>    text = p4_read_pipe(['print', '-q', '-o', '-', file['depotFile']])
>> 
>> However, p4/Linux prints to stdout by default and '-o -' will save
>> the output in a file named '-'.
>> 
>> My git and p4 versions:
>> 
>>    $ git --version
>>    git version 1.8.2.1.418.gaec3f77
>> 
>>    $ p4 -V
>>    Perforce - The Fast Software Configuration Management System.
>>    Copyright 1995-2013 Perforce Software.  All rights reserved.
>>    This product includes software developed by the OpenSSL Project
>>    for use in the OpenSSL Toolkit (http://www.openssl.org/)
>>    See 'p4 help legal' for full OpenSSL license information
>>    Version of OpenSSL Libraries: OpenSSL 1.0.1c 10 May 2012
>>    Rev. P4/LINUX26X86_64/2013.1/610569 (2013/03/19).
> 
> This code only happens on utf16 files.  But running it by hand,
> I cannot reproduce the different behavior:
> 
>    $ p4 print -q //depot/f-ascii
>    three
>    line
>    text
> 
>    $ p4 print -o - -q //depot/f-ascii
>    three
>    line
> 
>    $ ls ./-
>    ls: cannot access ./-: No such file or directory
> 
> I'm again confused.  Any hints you can give would be helpful.

This "second issue" is a non-issue. It seems "-o -" does send to stdout for files. For symlinks, it creates a symlink named "-". Example:

$ ls -l pcre lrwxrwxrwx 1 atomlinson atomlinson 12 Apr 18 17:17 pcre -> ../libs/pcre/ $ ls -l ./- ./xxx /bin/ls: cannot access ./-: No such file or directory /bin/ls: cannot access ./xxx: No such file or directory $ p4 print -q -o - pcre $ p4 print -q -o xxx pcre $ ls -l ./- ./xxx lrwxrwxrwx 1 atomlinson atomlinson 12 Apr 18 20:25 ./- -> ../libs/pcre/ lrwxrwxrwx 1 atomlinson atomlinson 12 Apr 18 20:25 ./xxx -> ../libs/pcre/

Show 6 quoted lines
> 
> 		-- Pete
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Thanks, Alex

Pete Wyckoff· Apr 19, 2013, 11:55 UTC · re: Alexander Tomlinson · lore

Re: is git-p4 compatible with p4/linux?

alex@aivor.com wrote on Thu, 18 Apr 2013 20:34 -0500:
Show 9 quoted lines
> Perhaps it is a configuration item on the server and/or client.   It seems we
> are running the same version of p4.  But just to be sure, check yours against
> mine:
> 
> $ cksum $(which p4)
> 3254530484 2420552 /usr/bin/p4
> 
> If yours if different, can you email a copy of your p4 executable to me
> so I can check if it works differently than mine?
It is the same binary.  Assuming you're running p4d 2013.1 too?
    $ p4 info
    [..]
    Server version: P4D/LINUX26X86_64/2013.1/610569 (2013/03/19)
I'm using these p4 client settings:
    Options:        noallwrite noclobber nocompress unlocked nomodtime normdir
    SubmitOptions:  submitunchanged
    LineEnd:        unix

Running "strace -vf -s 2000 p4 print -q symlink" shows that the newline is embedded in the response from p4d:

    read(7, "...\0\0\0symlink-target\n\0func\0\23...", 4096) = 277

Also the file depot/symlink,v in the p4d depot area includes the \n in the RCS file too, somewhat surprisingly:

    $ cat depot/symlink,v
    head     1.6;
    [..]
    @@
    text
    @symlink-target
    @
Show 32 quoted lines
> I will also check with coworkers here to see how their client behaves.
> > This code only happens on utf16 files.  But running it by hand,
> > I cannot reproduce the different behavior:
> > 
> >    $ p4 print -q //depot/f-ascii
> >    three
> >    line
> >    text
> > 
> >    $ p4 print -o - -q //depot/f-ascii
> >    three
> >    line
> > 
> >    $ ls ./-
> >    ls: cannot access ./-: No such file or directory
> > 
> > I'm again confused.  Any hints you can give would be helpful.
> 
> This "second issue" is a non-issue.  It seems "-o -" does send to
> stdout for files.  For symlinks, it creates a symlink named "-".
> Example:
> 
> $ ls -l pcre
> lrwxrwxrwx 1 atomlinson atomlinson 12 Apr 18 17:17 pcre -> ../libs/pcre/
> $ ls -l ./- ./xxx
> /bin/ls: cannot access ./-: No such file or directory
> /bin/ls: cannot access ./xxx: No such file or directory
> $ p4 print -q -o - pcre
> $ p4 print -q -o xxx pcre
> $ ls -l ./- ./xxx
> lrwxrwxrwx 1 atomlinson atomlinson 12 Apr 18 20:25 ./- -> ../libs/pcre/
> lrwxrwxrwx 1 atomlinson atomlinson 12 Apr 18 20:25 ./xxx -> ../libs/pcre/

Me too. That's annoying behavior, but not used by git-p4 fortunately. The "-o -" option is only used for odd utf16 files where "p4 print" generates invalid output.

		-- Pete
Alex Tomlinson· Apr 19, 2013, 15:20 UTC · re: Pete Wyckoff · lore

Re: is git-p4 compatible with p4/linux?

I found some symlinks in our depot that have the newline and others that do not. It looks like p4 print just dumps whatever is in the depot file, which of course is what you would expect it to do. So the question (for me) is, how did some of my symlink targets end up without the newline? Note: that's not a question for git-p4 because git-p4 was not involved in the creation of this p4 depot (or the symlink).

git-p4 is off the hook since it appears to be an issue with the depot itself.

Thanks for your time! --Alex

David Aguilar· Apr 20, 2013, 10:50 UTC · re: Pete Wyckoff · lore

Re: is git-p4 compatible with p4/linux?

On Thu, Apr 18, 2013 at 5:09 PM, Pete Wyckoff <pw@padd.com> wrote:
Show 12 quoted lines
>> First issue
>> -----------
>>
>> git-p4 assumes the output of 'p4 print' adds a newline to the
>> target.  To work around this, git-p4.py strips the last char from
>> symlinks as shown in the following snippet:
>>
>>     if type_base == "symlink":
>>         git_mode = "120000"
>>         # p4 print on a symlink contains "target\n"; remove the newline
>>         data = ''.join(contents)
>>         contents = [data[:-1]]
This line could be made more robust by changing it to:
    contents = [data.rstrip('\n')]

That way it only strips off newlines if they exist, which essentially papers over these rogue depot files. Alternatively, it could use rstrip() with no arguments to cast a wider net and catch all whitespace.

-- 
David
Pete Wyckoff· Apr 20, 2013, 13:22 UTC · re: David Aguilar · lore

Re: is git-p4 compatible with p4/linux?

davvid@gmail.com wrote on Sat, 20 Apr 2013 03:50 -0700:
Show 22 quoted lines
> On Thu, Apr 18, 2013 at 5:09 PM, Pete Wyckoff <pw@padd.com> wrote:
> >> First issue
> >> -----------
> >>
> >> git-p4 assumes the output of 'p4 print' adds a newline to the
> >> target.  To work around this, git-p4.py strips the last char from
> >> symlinks as shown in the following snippet:
> >>
> >>     if type_base == "symlink":
> >>         git_mode = "120000"
> >>         # p4 print on a symlink contains "target\n"; remove the newline
> >>         data = ''.join(contents)
> >>         contents = [data[:-1]]
> 
> This line could be made more robust by changing it to:
> 
>     contents = [data.rstrip('\n')]
> 
> That way it only strips off newlines if they exist, which essentially
> papers over these rogue depot files.
> Alternatively, it could use rstrip() with no arguments to cast a wider
> net and catch all whitespace.

I was tempted to do that, but it is possible to put \n and other space characters in the target of symlinks. It's unfortunate that p4 always appears to tack on a newline itself.

We'll see if Alex comes up with a pattern that shows how he ended up with the odd symlinks.

		-- Pete

← back to recent threads