[BUG] git p4 clone fails when p4 sizes does not return 'fileSize' key

> Huh, I actually have a slightly different fix for the same issue.
> It doesn't suppress the corresponding verbose output completely, but just removes the size information from it.
>
> Also, I'd mention that the workaround is trivial -- simply omit the "--verbose" option.
>
> Andrey Mazo (1):
>   git-p4: fix `sync --verbose` traceback due to 'fileSize'
>
>  git-p4.py | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
>
>
> base-commit: 468165c1d8a442994a825f3684528361727cd8c0
> --
> 2.16.1
>

> *I* think keeping the filesize info is better with --verbose option as
> that gives some clue about the file we are working on. What do you
> think?
> Script has similar checks of key existence at other places where it is
> looking for fileSize.
> 
> On Tue, Apr 17, 2018 at 9:22 AM, Andrey Mazo <amazo@checkvideo.com> wrote:
>> Huh, I actually have a slightly different fix for the same issue.
>> It doesn't suppress the corresponding verbose output completely, but just removes the size information from it.
>>
>> Also, I'd mention that the workaround is trivial -- simply omit the "--verbose" option.
>>
>> Andrey Mazo (1):
>>   git-p4: fix `sync --verbose` traceback due to 'fileSize'
>>
>>  git-p4.py | 8 ++++++--
>>  1 file changed, 6 insertions(+), 2 deletions(-)
>>
>>
>> base-commit: 468165c1d8a442994a825f3684528361727cd8c0
>> --
>> 2.16.1
>>
> 
> -- 
> Thanks & Regards
> Thandesha VK | Cellphone +1 (703) 459-5386

> Sounds good. How about an enhanced version of fix from both of us.
> This will let us know that something is not right with the file but
> will not bark
> 
> $ git diff
> diff --git a/git-p4.py b/git-p4.py
> index 7bb9cadc6..df901976f 100755
> --- a/git-p4.py
> +++ b/git-p4.py
> @@ -2566,7 +2566,12 @@ class P4Sync(Command, P4UserMap):
>          relPath = self.stripRepoPath(file['depotFile'], self.branchPrefixes)
>          relPath = self.encodeWithUTF8(relPath)
>          if verbose:
> -            size = int(self.stream_file['fileSize'])
> +            if 'fileSize' not in self.stream_file:
> +               print "WARN: File size from perforce unknown. Please verify by p4 sizes %s" %(file['depotFile'])

> +               size = "-1"
> +            else:
> +               size = self.stream_file['fileSize']
> +            size = int(size)
>              sys.stdout.write('\r%s --> %s (%i MB)\n' %
> (file['depotFile'], relPath, size/1024/1024))
>              sys.stdout.flush()
> 
> 
> On Tue, Apr 17, 2018 at 10:33 AM, Mazo, Andrey <amazo@checkvideo.com> wrote:
>> Sure, I totally agree.
>> Sorry, I just wasn't clear enough in my previous email.
>> I meant that your patch suppresses "%s --> %s (%i MB)" line in case "fileSize" is not available,
>> while my patch suppresses just "(%i MB)" portion if the "fileSize" is not known.
>> In other words,
>>  * if "fileSize" is known:
>>  ** both yours and mine patches don't change existing behavior;
>>  * if "fileSize" is not known:
>>  ** your patch makes streamOneP4File() not print anything;
>>  ** my patch makes streamOneP4File() print "%s --> %s".
>>
>> Hope, I'm clearer this time.
>>
>> Thank you,
>> Andrey
>>
>> From: Thandesha VK <thanvk@gmail.com>
>>> *I* think keeping the filesize info is better with --verbose option as
>>> that gives some clue about the file we are working on. What do you
>>> think?
>>> Script has similar checks of key existence at other places where it is
>>> looking for fileSize.
>>>
>>> On Tue, Apr 17, 2018 at 9:22 AM, Andrey Mazo <amazo@checkvideo.com> wrote:
>>>> Huh, I actually have a slightly different fix for the same issue.
>>>> It doesn't suppress the corresponding verbose output completely, but just removes the size information from it.
>>>>
>>>> Also, I'd mention that the workaround is trivial -- simply omit the "--verbose" option.
>>>>
>>>> Andrey Mazo (1):
>>>>   git-p4: fix `sync --verbose` traceback due to 'fileSize'
>>>>
>>>>  git-p4.py | 8 ++++++--
>>>>  1 file changed, 6 insertions(+), 2 deletions(-)
>>>>
>>>>
>>>> base-commit: 468165c1d8a442994a825f3684528361727cd8c0
>>>> --
>>>> 2.16.1
>>>>
>>>
>>> --
>>> Thanks & Regards
>>> Thandesha VK | Cellphone +1 (703) 459-5386
>
>
>
> -- 
> Thanks & Regards
> Thandesha VK | Cellphone +1 (703) 459-5386

> Does a missing "fileSize" actually mean that there is something wrong with the file?
> Because, for me, `p4 -G print` doesn't print "fileSize" for _any_ file.
> (which I attribute to our rather ancient (2007.2) Perforce server)
> I'm not an expert in Perforce, so don't know for sure.
>
> However, `p4 -G sizes` works fine even with our p4 server.
> Should we then go one step further and use `p4 -G sizes` to obtain the "fileSize" when it's not returned by `p4 -G print`?
> Or is it an overkill for a simple verbose print out?
>
> Also, please, find one comment inline below.
>
> Thank you,
> Andrey
>
> From: Thandesha VK <thanvk@gmail.com>
>> Sounds good. How about an enhanced version of fix from both of us.
>> This will let us know that something is not right with the file but
>> will not bark
>>
>> $ git diff
>> diff --git a/git-p4.py b/git-p4.py
>> index 7bb9cadc6..df901976f 100755
>> --- a/git-p4.py
>> +++ b/git-p4.py
>> @@ -2566,7 +2566,12 @@ class P4Sync(Command, P4UserMap):
>>          relPath = self.stripRepoPath(file['depotFile'], self.branchPrefixes)
>>          relPath = self.encodeWithUTF8(relPath)
>>          if verbose:
>> -            size = int(self.stream_file['fileSize'])
>> +            if 'fileSize' not in self.stream_file:
>> +               print "WARN: File size from perforce unknown. Please verify by p4 sizes %s" %(file['depotFile'])
> For whatever reason, the code below uses sys.stdout.write() instead of print().
> Should it be used here for consistency as well?
>
>> +               size = "-1"
>> +            else:
>> +               size = self.stream_file['fileSize']
>> +            size = int(size)
>>              sys.stdout.write('\r%s --> %s (%i MB)\n' %
>> (file['depotFile'], relPath, size/1024/1024))
>>              sys.stdout.flush()
>>
>>
>> On Tue, Apr 17, 2018 at 10:33 AM, Mazo, Andrey <amazo@checkvideo.com> wrote:
>>> Sure, I totally agree.
>>> Sorry, I just wasn't clear enough in my previous email.
>>> I meant that your patch suppresses "%s --> %s (%i MB)" line in case "fileSize" is not available,
>>> while my patch suppresses just "(%i MB)" portion if the "fileSize" is not known.
>>> In other words,
>>>  * if "fileSize" is known:
>>>  ** both yours and mine patches don't change existing behavior;
>>>  * if "fileSize" is not known:
>>>  ** your patch makes streamOneP4File() not print anything;
>>>  ** my patch makes streamOneP4File() print "%s --> %s".
>>>
>>> Hope, I'm clearer this time.
>>>
>>> Thank you,
>>> Andrey
>>>
>>> From: Thandesha VK <thanvk@gmail.com>
>>>> *I* think keeping the filesize info is better with --verbose option as
>>>> that gives some clue about the file we are working on. What do you
>>>> think?
>>>> Script has similar checks of key existence at other places where it is
>>>> looking for fileSize.
>>>>
>>>> On Tue, Apr 17, 2018 at 9:22 AM, Andrey Mazo <amazo@checkvideo.com> wrote:
>>>>> Huh, I actually have a slightly different fix for the same issue.
>>>>> It doesn't suppress the corresponding verbose output completely, but just removes the size information from it.
>>>>>
>>>>> Also, I'd mention that the workaround is trivial -- simply omit the "--verbose" option.
>>>>>
>>>>> Andrey Mazo (1):
>>>>>   git-p4: fix `sync --verbose` traceback due to 'fileSize'
>>>>>
>>>>>  git-p4.py | 8 ++++++--
>>>>>  1 file changed, 6 insertions(+), 2 deletions(-)
>>>>>
>>>>>
>>>>> base-commit: 468165c1d8a442994a825f3684528361727cd8c0
>>>>> --
>>>>> 2.16.1
>>>>>
>>>>
>>>> --
>>>> Thanks & Regards
>>>> Thandesha VK | Cellphone +1 (703) 459-5386
>>
>>
>>
>> --
>> Thanks & Regards
>> Thandesha VK | Cellphone +1 (703) 459-5386

>
> On Tue, Apr 17, 2018 at 11:47 AM, Mazo, Andrey <amazo@checkvideo.com> wrote:
>> Does a missing "fileSize" actually mean that there is something wrong with the file?
>> Because, for me, `p4 -G print` doesn't print "fileSize" for _any_ file.
>> (which I attribute to our rather ancient (2007.2) Perforce server)
>> I'm not an expert in Perforce, so don't know for sure.

>>
>> Also, please, find one comment inline below.
>>
>> Thank you,
>> Andrey
>>
>> From: Thandesha VK <thanvk@gmail.com>
>>> Sounds good. How about an enhanced version of fix from both of us.
>>> This will let us know that something is not right with the file but
>>> will not bark
>>>
>>> $ git diff
>>> diff --git a/git-p4.py b/git-p4.py
>>> index 7bb9cadc6..df901976f 100755
>>> --- a/git-p4.py
>>> +++ b/git-p4.py
>>> @@ -2566,7 +2566,12 @@ class P4Sync(Command, P4UserMap):
>>>          relPath = self.stripRepoPath(file['depotFile'], self.branchPrefixes)
>>>          relPath = self.encodeWithUTF8(relPath)
>>>          if verbose:
>>> -            size = int(self.stream_file['fileSize'])
>>> +            if 'fileSize' not in self.stream_file:
>>> +               print "WARN: File size from perforce unknown. Please verify by p4 sizes %s" %(file['depotFile'])
>> For whatever reason, the code below uses sys.stdout.write() instead of print().
>> Should it be used here for consistency as well?
>>
>>> +               size = "-1"
>>> +            else:
>>> +               size = self.stream_file['fileSize']
>>> +            size = int(size)
>>>              sys.stdout.write('\r%s --> %s (%i MB)\n' %
>>> (file['depotFile'], relPath, size/1024/1024))
>>>              sys.stdout.flush()
>>>
>>>
>>> On Tue, Apr 17, 2018 at 10:33 AM, Mazo, Andrey <amazo@checkvideo.com> wrote:
>>>> Sure, I totally agree.
>>>> Sorry, I just wasn't clear enough in my previous email.
>>>> I meant that your patch suppresses "%s --> %s (%i MB)" line in case "fileSize" is not available,
>>>> while my patch suppresses just "(%i MB)" portion if the "fileSize" is not known.
>>>> In other words,
>>>>  * if "fileSize" is known:
>>>>  ** both yours and mine patches don't change existing behavior;
>>>>  * if "fileSize" is not known:
>>>>  ** your patch makes streamOneP4File() not print anything;
>>>>  ** my patch makes streamOneP4File() print "%s --> %s".
>>>>
>>>> Hope, I'm clearer this time.
>>>>
>>>> Thank you,
>>>> Andrey
>>>>
>>>> From: Thandesha VK <thanvk@gmail.com>
>>>>> *I* think keeping the filesize info is better with --verbose option as
>>>>> that gives some clue about the file we are working on. What do you
>>>>> think?
>>>>> Script has similar checks of key existence at other places where it is
>>>>> looking for fileSize.
>>>>>
>>>>> On Tue, Apr 17, 2018 at 9:22 AM, Andrey Mazo <amazo@checkvideo.com> wrote:
>>>>>> Huh, I actually have a slightly different fix for the same issue.
>>>>>> It doesn't suppress the corresponding verbose output completely, but just removes the size information from it.
>>>>>>
>>>>>> Also, I'd mention that the workaround is trivial -- simply omit the "--verbose" option.
>>>>>>
>>>>>> Andrey Mazo (1):
>>>>>>   git-p4: fix `sync --verbose` traceback due to 'fileSize'
>>>>>>
>>>>>>  git-p4.py | 8 ++++++--
>>>>>>  1 file changed, 6 insertions(+), 2 deletions(-)
>>>>>>

> On 17 April 2018 at 20:12, Thandesha VK <thanvk@gmail.com> wrote:
>> I have few cases where even p4 -G sizes (or p4 sizes) is not returning
>> the size value even with latest version of p4 (17.2). In that case, we
>> have to regenerate the digest for file save it - It mean something is
>> wrong with the file in perforce.
>
> Just to be clear - git-p4 does not use the p4 "sizes" command anywhere AFAIK.
>
> We are just talking about the output from "p4 print" and the
> "fileSize" key, right?
>
> Does that happen with the 17.2 version of p4?
>
>> Regarding, sys.stdout.write v/s print, I see script using both of them
>> without a common pattern. I can change it to whatever is more
>> appropriate.
>
> print() probably makes more sense; can we try to use the function form
> so that we don't deliberately make the path to python3 harder (albeit
> in a very tiny way).
>
>>
>> On Tue, Apr 17, 2018 at 11:47 AM, Mazo, Andrey <amazo@checkvideo.com> wrote:
>>> Does a missing "fileSize" actually mean that there is something wrong with the file?
>>> Because, for me, `p4 -G print` doesn't print "fileSize" for _any_ file.
>>> (which I attribute to our rather ancient (2007.2) Perforce server)
>>> I'm not an expert in Perforce, so don't know for sure.
>
> My 2015 version of p4d reports a fileSize.
>
>>>
>>> However, `p4 -G sizes` works fine even with our p4 server.
>>> Should we then go one step further and use `p4 -G sizes` to obtain the "fileSize" when it's not returned by `p4 -G print`?
>>> Or is it an overkill for a simple verbose print out?
>
> If your server isn't reporting "fileSize" then there are a few other
> places where I would expect git-p4 to also fail.
>
> If we're going to support this very ancient version of p4d, then
> gracefully handling a missing fileSize will be useful.
>
>>>
>>> Also, please, find one comment inline below.
>>>
>>> Thank you,
>>> Andrey
>>>
>>> From: Thandesha VK <thanvk@gmail.com>
>>>> Sounds good. How about an enhanced version of fix from both of us.
>>>> This will let us know that something is not right with the file but
>>>> will not bark
>>>>
>>>> $ git diff
>>>> diff --git a/git-p4.py b/git-p4.py
>>>> index 7bb9cadc6..df901976f 100755
>>>> --- a/git-p4.py
>>>> +++ b/git-p4.py
>>>> @@ -2566,7 +2566,12 @@ class P4Sync(Command, P4UserMap):
>>>>          relPath = self.stripRepoPath(file['depotFile'], self.branchPrefixes)
>>>>          relPath = self.encodeWithUTF8(relPath)
>>>>          if verbose:
>>>> -            size = int(self.stream_file['fileSize'])
>>>> +            if 'fileSize' not in self.stream_file:
>>>> +               print "WARN: File size from perforce unknown. Please verify by p4 sizes %s" %(file['depotFile'])
>>> For whatever reason, the code below uses sys.stdout.write() instead of print().
>>> Should it be used here for consistency as well?
>>>
>>>> +               size = "-1"
>>>> +            else:
>>>> +               size = self.stream_file['fileSize']
>>>> +            size = int(size)
>>>>              sys.stdout.write('\r%s --> %s (%i MB)\n' %
>>>> (file['depotFile'], relPath, size/1024/1024))
>>>>              sys.stdout.flush()
>>>>
>>>>
>>>> On Tue, Apr 17, 2018 at 10:33 AM, Mazo, Andrey <amazo@checkvideo.com> wrote:
>>>>> Sure, I totally agree.
>>>>> Sorry, I just wasn't clear enough in my previous email.
>>>>> I meant that your patch suppresses "%s --> %s (%i MB)" line in case "fileSize" is not available,
>>>>> while my patch suppresses just "(%i MB)" portion if the "fileSize" is not known.
>>>>> In other words,
>>>>>  * if "fileSize" is known:
>>>>>  ** both yours and mine patches don't change existing behavior;
>>>>>  * if "fileSize" is not known:
>>>>>  ** your patch makes streamOneP4File() not print anything;
>>>>>  ** my patch makes streamOneP4File() print "%s --> %s".
>>>>>
>>>>> Hope, I'm clearer this time.
>>>>>
>>>>> Thank you,
>>>>> Andrey
>>>>>
>>>>> From: Thandesha VK <thanvk@gmail.com>
>>>>>> *I* think keeping the filesize info is better with --verbose option as
>>>>>> that gives some clue about the file we are working on. What do you
>>>>>> think?
>>>>>> Script has similar checks of key existence at other places where it is
>>>>>> looking for fileSize.
>>>>>>
>>>>>> On Tue, Apr 17, 2018 at 9:22 AM, Andrey Mazo <amazo@checkvideo.com> wrote:
>>>>>>> Huh, I actually have a slightly different fix for the same issue.
>>>>>>> It doesn't suppress the corresponding verbose output completely, but just removes the size information from it.
>>>>>>>
>>>>>>> Also, I'd mention that the workaround is trivial -- simply omit the "--verbose" option.
>>>>>>>
>>>>>>> Andrey Mazo (1):
>>>>>>>   git-p4: fix `sync --verbose` traceback due to 'fileSize'
>>>>>>>
>>>>>>>  git-p4.py | 8 ++++++--
>>>>>>>  1 file changed, 6 insertions(+), 2 deletions(-)
>>>>>>>
>
> Thanks
> Luke

> On Tue, 17 Apr 2018, 09:22 Andrey Mazo, <amazo@checkvideo.com> wrote:
>>  Perforce server 2007.2 (and maybe others) doesn't return "fileSize"
>> attribute in its reply to `p4 -G print` command.
>> This causes the following traceback when running `git p4 sync --verbose`:
>> """
>>     Traceback (most recent call last):
>>       File "/usr/libexec/git-core/git-p4", line 3839, in <module>
>>         main()
>>       File "/usr/libexec/git-core/git-p4", line 3833, in main
>>         if not cmd.run(args):
>>       File "/usr/libexec/git-core/git-p4", line 3567, in run
>>         self.importChanges(changes)
>>       File "/usr/libexec/git-core/git-p4", line 3233, in importChanges
>>         self.commit(description, filesForCommit, branch, parent)
>>       File "/usr/libexec/git-core/git-p4", line 2855, in commit
>>         self.streamP4Files(files)
>>       File "/usr/libexec/git-core/git-p4", line 2747, in streamP4Files
>>         cb=streamP4FilesCbSelf)
>>       File "/usr/libexec/git-core/git-p4", line 552, in p4CmdList
>>         cb(entry)
>>       File "/usr/libexec/git-core/git-p4", line 2741, in streamP4FilesCbSelf
>>         self.streamP4FilesCb(entry)
>>       File "/usr/libexec/git-core/git-p4", line 2689, in streamP4FilesCb
>>         self.streamOneP4File(self.stream_file, self.stream_contents)
>>       File "/usr/libexec/git-core/git-p4", line 2566, in streamOneP4File
>>         size = int(self.stream_file['fileSize'])
>>     KeyError: 'fileSize'
>> """
>> 
>> Fix this by omitting the file size information from the verbose print out.
>> Also, don't use "self.stream_file" directly,
>> but rather use passed in "file" argument.
>> (which point to the same "self.stream_file" for all existing callers)
>> 
>> Signed-off-by: Andrey Mazo <amazo@checkvideo.com>
>> ---
>>  git -p4.py | 8 ++++++--
>>  1 file changed, 6 insertions(+), 2 deletions(-)
>> 
>> diff --git a/git-p4.py b/git-p4.py
>> index 7bb9cadc6..6f05f915a 100755
>> --- a/git-p4.py
>> +++ b/git-p4.py
>> @@ -2566,8 +2566,12 @@ class P4Sync(Command, P4UserMap):
>>          relPath = self.stripRepoPath(file['depotFile'], self.branchPrefixes)
>>          relPath = self.encodeWithUTF8(relPath)
>>          if verbose:
>> -            size = int(self.stream_file['fileSize'])
>> -            sys.stdout.write('\r%s --> %s (%i MB)\n' % (file['depotFile'], relPath, size/1024/1024))
>> +            size = file.get('fileSize', None)
>> +            if size is None:
>> +                sizeStr = ''
>> +            else:
>> +                sizeStr = ' (%i MB)' % (int(size)/1024/1024)
>> +            sys.stdout.write('\r%s --> %s%s\n' % (file['depotFile'], relPath, sizeStr))
>>              sys.stdout.flush()
>> 
>>          (type_base, type_mods) = split_p4_type(file["type"])
>> -- 
>> 2.16.1
>>
> Thanks, that looks like a good fix to me.  Ack.

> Luke,
>
> Thank you for reviewing and acking my patch!
> By the way, did you see Thandesha's proposed patch [1] to print a warning in case of the missing "fileSize" attribute?
> Should we go that route instead?
> Or should we try harder to get the size by running `p4 -G sizes`?
>
> [1] https://public-inbox.org/git/CAJJpmi-pLb4Qcka5aLKXA8B1VOZFFF+OAQ0fgUq9YviobRpYGg@mail.gmail.com/t/#m6053d2031020e08edd24ada6c9eb49721ebc4e27
>
> Thank you,
> Andrey
>
> From: Luke Diamand <luke@diamand.org>
>> On Tue, 17 Apr 2018, 09:22 Andrey Mazo, <amazo@checkvideo.com> wrote:
>>>  Perforce server 2007.2 (and maybe others) doesn't return "fileSize"
>>> attribute in its reply to `p4 -G print` command.
>>> This causes the following traceback when running `git p4 sync --verbose`:
>>> """
>>>     Traceback (most recent call last):
>>>       File "/usr/libexec/git-core/git-p4", line 3839, in <module>
>>>         main()
>>>       File "/usr/libexec/git-core/git-p4", line 3833, in main
>>>         if not cmd.run(args):
>>>       File "/usr/libexec/git-core/git-p4", line 3567, in run
>>>         self.importChanges(changes)
>>>       File "/usr/libexec/git-core/git-p4", line 3233, in importChanges
>>>         self.commit(description, filesForCommit, branch, parent)
>>>       File "/usr/libexec/git-core/git-p4", line 2855, in commit
>>>         self.streamP4Files(files)
>>>       File "/usr/libexec/git-core/git-p4", line 2747, in streamP4Files
>>>         cb=streamP4FilesCbSelf)
>>>       File "/usr/libexec/git-core/git-p4", line 552, in p4CmdList
>>>         cb(entry)
>>>       File "/usr/libexec/git-core/git-p4", line 2741, in streamP4FilesCbSelf
>>>         self.streamP4FilesCb(entry)
>>>       File "/usr/libexec/git-core/git-p4", line 2689, in streamP4FilesCb
>>>         self.streamOneP4File(self.stream_file, self.stream_contents)
>>>       File "/usr/libexec/git-core/git-p4", line 2566, in streamOneP4File
>>>         size = int(self.stream_file['fileSize'])
>>>     KeyError: 'fileSize'
>>> """
>>>
>>> Fix this by omitting the file size information from the verbose print out.
>>> Also, don't use "self.stream_file" directly,
>>> but rather use passed in "file" argument.
>>> (which point to the same "self.stream_file" for all existing callers)
>>>
>>> Signed-off-by: Andrey Mazo <amazo@checkvideo.com>
>>> ---
>>>  git -p4.py | 8 ++++++--
>>>  1 file changed, 6 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/git-p4.py b/git-p4.py
>>> index 7bb9cadc6..6f05f915a 100755
>>> --- a/git-p4.py
>>> +++ b/git-p4.py
>>> @@ -2566,8 +2566,12 @@ class P4Sync(Command, P4UserMap):
>>>          relPath = self.stripRepoPath(file['depotFile'], self.branchPrefixes)
>>>          relPath = self.encodeWithUTF8(relPath)
>>>          if verbose:
>>> -            size = int(self.stream_file['fileSize'])
>>> -            sys.stdout.write('\r%s --> %s (%i MB)\n' % (file['depotFile'], relPath, size/1024/1024))
>>> +            size = file.get('fileSize', None)
>>> +            if size is None:
>>> +                sizeStr = ''
>>> +            else:
>>> +                sizeStr = ' (%i MB)' % (int(size)/1024/1024)
>>> +            sys.stdout.write('\r%s --> %s%s\n' % (file['depotFile'], relPath, sizeStr))
>>>              sys.stdout.flush()
>>>
>>>          (type_base, type_mods) = split_p4_type(file["type"])
>>> --
>>> 2.16.1
>>>
>> Thanks, that looks like a good fix to me.  Ack.

> My fix is for the case where p4 -G sizes not returning the key and
> value for fileSize. This can happen in some cases. Only option at that
> point of time is to warn the user about the problematic file and keep
> moving (or should we abort??)
> 
> Thanks
> Thandesha
> 
> On Tue, Apr 17, 2018 at 2:18 PM, Mazo, Andrey <amazo@checkvideo.com> wrote:
>> Luke,
>>
>> Thank you for reviewing and acking my patch!
>> By the way, did you see Thandesha's proposed patch [1] to print a warning in case of the missing "fileSize" attribute?
>> Should we go that route instead?
>> Or should we try harder to get the size by running `p4 -G sizes`?
>>
>> [1]  https://public-inbox.org/git/CAJJpmi-pLb4Qcka5aLKXA8B1VOZFFF+OAQ0fgUq9YviobRpYGg@mail.gmail.com/t/#m6053d2031020e08edd24ada6c9eb49721ebc4e27
>>
>> Thank you,
>> Andrey
>>
>> From: Luke Diamand <luke@diamand.org>
>>> On Tue, 17 Apr 2018, 09:22 Andrey Mazo, <amazo@checkvideo.com> wrote:
>>>>  Perforce server 2007.2 (and maybe others) doesn't return "fileSize"
>>>> attribute in its reply to `p4 -G print` command.
>>>> This causes the following traceback when running `git p4 sync --verbose`:
>>>> """
>>>>     Traceback (most recent call last):
>>>>       File "/usr/libexec/git-core/git-p4", line 3839, in <module>
>>>>         main()
>>>>       File "/usr/libexec/git-core/git-p4", line 3833, in main
>>>>         if not cmd.run(args):
>>>>       File "/usr/libexec/git-core/git-p4", line 3567, in run
>>>>         self.importChanges(changes)
>>>>       File "/usr/libexec/git-core/git-p4", line 3233, in importChanges
>>>>         self.commit(description, filesForCommit, branch, parent)
>>>>       File "/usr/libexec/git-core/git-p4", line 2855, in commit
>>>>         self.streamP4Files(files)
>>>>       File "/usr/libexec/git-core/git-p4", line 2747, in streamP4Files
>>>>         cb=streamP4FilesCbSelf)
>>>>       File "/usr/libexec/git-core/git-p4", line 552, in p4CmdList
>>>>         cb(entry)
>>>>       File "/usr/libexec/git-core/git-p4", line 2741, in streamP4FilesCbSelf
>>>>         self.streamP4FilesCb(entry)
>>>>       File "/usr/libexec/git-core/git-p4", line 2689, in streamP4FilesCb
>>>>         self.streamOneP4File(self.stream_file, self.stream_contents)
>>>>       File "/usr/libexec/git-core/git-p4", line 2566, in streamOneP4File
>>>>         size = int(self.stream_file['fileSize'])
>>>>     KeyError: 'fileSize'
>>>> """
>>>>
>>>> Fix this by omitting the file size information from the verbose print out.
>>>> Also, don't use "self.stream_file" directly,
>>>> but rather use passed in "file" argument.
>>>> (which point to the same "self.stream_file" for all existing callers)
>>>>
>>>> Signed-off-by: Andrey Mazo <amazo@checkvideo.com>
>>>> ---
>>>>  git -p4.py | 8 ++++++--
>>>>  1 file changed, 6 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/git-p4.py b/git-p4.py
>>>> index 7bb9cadc6..6f05f915a 100755
>>>> --- a/git-p4.py
>>>> +++ b/git-p4.py
>>>> @@ -2566,8 +2566,12 @@ class P4Sync(Command, P4UserMap):
>>>>          relPath = self.stripRepoPath(file['depotFile'], self.branchPrefixes)
>>>>          relPath = self.encodeWithUTF8(relPath)
>>>>          if verbose:
>>>> -            size = int(self.stream_file['fileSize'])
>>>> -            sys.stdout.write('\r%s --> %s (%i MB)\n' % (file['depotFile'], relPath, size/1024/1024))
>>>> +            size = file.get('fileSize', None)
>>>> +            if size is None:
>>>> +                sizeStr = ''
>>>> +            else:
>>>> +                sizeStr = ' (%i MB)' % (int(size)/1024/1024)
>>>> +            sys.stdout.write('\r%s --> %s%s\n' % (file['depotFile'], relPath, sizeStr))
>>>>              sys.stdout.flush()
>>>>
>>>>          (type_base, type_mods) = split_p4_type(file["type"])
>>>> --
>>>> 2.16.1
>>>>
>>> Thanks, that looks like a good fix to me.  Ack.
>
> -- 
> Thanks & Regards
> Thandesha VK | Cellphone +1 (703) 459-5386

> Thandesha,
>
> If I read your patch correctly, it adds a warning in case `p4 -G print` doesn't return "fileSize" (not `p4 sizes`).
> I don't see `p4 sizes` being used by git-p4 at all.
> As I said earlier, for our ancient Perforce server, `p4 -G print` _never_ returns "fileSize".
> So, it's definitely not a reason to abort.
>
> Thank you,
> Andrey
>
> From: Thandesha VK <thanvk@gmail.com>
>> My fix is for the case where p4 -G sizes not returning the key and
>> value for fileSize. This can happen in some cases. Only option at that
>> point of time is to warn the user about the problematic file and keep
>> moving (or should we abort??)
>>
>> Thanks
>> Thandesha
>>
>> On Tue, Apr 17, 2018 at 2:18 PM, Mazo, Andrey <amazo@checkvideo.com> wrote:
>>> Luke,
>>>
>>> Thank you for reviewing and acking my patch!
>>> By the way, did you see Thandesha's proposed patch [1] to print a warning in case of the missing "fileSize" attribute?
>>> Should we go that route instead?
>>> Or should we try harder to get the size by running `p4 -G sizes`?
>>>
>>> [1]  https://public-inbox.org/git/CAJJpmi-pLb4Qcka5aLKXA8B1VOZFFF+OAQ0fgUq9YviobRpYGg@mail.gmail.com/t/#m6053d2031020e08edd24ada6c9eb49721ebc4e27
>>>
>>> Thank you,
>>> Andrey
>>>
>>> From: Luke Diamand <luke@diamand.org>
>>>> On Tue, 17 Apr 2018, 09:22 Andrey Mazo, <amazo@checkvideo.com> wrote:
>>>>>  Perforce server 2007.2 (and maybe others) doesn't return "fileSize"
>>>>> attribute in its reply to `p4 -G print` command.
>>>>> This causes the following traceback when running `git p4 sync --verbose`:
>>>>> """
>>>>>     Traceback (most recent call last):
>>>>>       File "/usr/libexec/git-core/git-p4", line 3839, in <module>
>>>>>         main()
>>>>>       File "/usr/libexec/git-core/git-p4", line 3833, in main
>>>>>         if not cmd.run(args):
>>>>>       File "/usr/libexec/git-core/git-p4", line 3567, in run
>>>>>         self.importChanges(changes)
>>>>>       File "/usr/libexec/git-core/git-p4", line 3233, in importChanges
>>>>>         self.commit(description, filesForCommit, branch, parent)
>>>>>       File "/usr/libexec/git-core/git-p4", line 2855, in commit
>>>>>         self.streamP4Files(files)
>>>>>       File "/usr/libexec/git-core/git-p4", line 2747, in streamP4Files
>>>>>         cb=streamP4FilesCbSelf)
>>>>>       File "/usr/libexec/git-core/git-p4", line 552, in p4CmdList
>>>>>         cb(entry)
>>>>>       File "/usr/libexec/git-core/git-p4", line 2741, in streamP4FilesCbSelf
>>>>>         self.streamP4FilesCb(entry)
>>>>>       File "/usr/libexec/git-core/git-p4", line 2689, in streamP4FilesCb
>>>>>         self.streamOneP4File(self.stream_file, self.stream_contents)
>>>>>       File "/usr/libexec/git-core/git-p4", line 2566, in streamOneP4File
>>>>>         size = int(self.stream_file['fileSize'])
>>>>>     KeyError: 'fileSize'
>>>>> """
>>>>>
>>>>> Fix this by omitting the file size information from the verbose print out.
>>>>> Also, don't use "self.stream_file" directly,
>>>>> but rather use passed in "file" argument.
>>>>> (which point to the same "self.stream_file" for all existing callers)
>>>>>
>>>>> Signed-off-by: Andrey Mazo <amazo@checkvideo.com>
>>>>> ---
>>>>>  git -p4.py | 8 ++++++--
>>>>>  1 file changed, 6 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/git-p4.py b/git-p4.py
>>>>> index 7bb9cadc6..6f05f915a 100755
>>>>> --- a/git-p4.py
>>>>> +++ b/git-p4.py
>>>>> @@ -2566,8 +2566,12 @@ class P4Sync(Command, P4UserMap):
>>>>>          relPath = self.stripRepoPath(file['depotFile'], self.branchPrefixes)
>>>>>          relPath = self.encodeWithUTF8(relPath)
>>>>>          if verbose:
>>>>> -            size = int(self.stream_file['fileSize'])
>>>>> -            sys.stdout.write('\r%s --> %s (%i MB)\n' % (file['depotFile'], relPath, size/1024/1024))
>>>>> +            size = file.get('fileSize', None)
>>>>> +            if size is None:
>>>>> +                sizeStr = ''
>>>>> +            else:
>>>>> +                sizeStr = ' (%i MB)' % (int(size)/1024/1024)
>>>>> +            sys.stdout.write('\r%s --> %s%s\n' % (file['depotFile'], relPath, sizeStr))
>>>>>              sys.stdout.flush()
>>>>>
>>>>>          (type_base, type_mods) = split_p4_type(file["type"])
>>>>> --
>>>>> 2.16.1
>>>>>
>>>> Thanks, that looks like a good fix to me.  Ack.
>>
>> --
>> Thanks & Regards
>> Thandesha VK | Cellphone +1 (703) 459-5386