threads / discuss / 34356

Re: git p4 clone not processing branches properly

Subject: Re: git p4 clone not processing branches properly

## tl;dr

12 messages between Jul 5, 2013 and Jul 13, 2013.

replies: 11people: 2as markdown or json

Matthieu Brucher· Jul 5, 2013, 13:16 UTC · lore
Hi,
I'm trying to convert a Perforce repository to git, knowing that:
- I use client specs to remove a bunch of folders containing binaires
(several GiB)
- branch mappings may not be properly set, and I can't change them
Now, the branches are layout like this:
- Branch/Main <- master
- Branch/Feature1
- ...
- Releases/2013
- ...
I would like to have these branches and releases branches inside by
cloned git repository, but this doesn't work. I keep on getting each
file with Project/Branch/Main as well as Project/Branch/Feature1 and
all others in my master branch.
I tried to add branchLists like this:
        branchList = Branch/Main:Releases/2013
        branchList = Releases/2013:Branch/Feature1
but it doesn't change a thing with the following command:
 git p4 clone --verbose --use-client-spec --detect-branches
//Depot/Project@specificrevision

I can see that branches are detected from the Perforce server, but none are actually detected for this specific project: p4-git branches: [] initial parents: {}

Can someone give a pointer to a tutorial or something for a complex case like this?

Regards,
Matthieu Brucher
--
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/
Vitor Antunes· Jul 5, 2013, 17:56 UTC · re: Matthieu Brucher · lore
Matthieu Brucher <matthieu.brucher <at> gmail.com> writes:
Show 32 quoted lines
> 
> Hi,
> 
> I'm trying to convert a Perforce repository to git, knowing that:
> - I use client specs to remove a bunch of folders containing binaires
> (several GiB)
> - branch mappings may not be properly set, and I can't change them
> 
> Now, the branches are layout like this:
> - Branch/Main <- master
> - Branch/Feature1
> - ...
> - Releases/2013
> - ...
> I would like to have these branches and releases branches inside by
> cloned git repository, but this doesn't work. I keep on getting each
> file with Project/Branch/Main as well as Project/Branch/Feature1 and
> all others in my master branch.
> I tried to add branchLists like this:
>         branchList = Branch/Main:Releases/2013
>         branchList = Releases/2013:Branch/Feature1
> but it doesn't change a thing with the following command:
>  git p4 clone --verbose --use-client-spec --detect-branches
> //Depot/Project <at> specificrevision
> 
> I can see that branches are detected from the Perforce server, but
> none are actually detected for this specific project:
> p4-git branches: []
> initial parents: {}
> 
> Can someone give a pointer to a tutorial or something for a complex
> case like this?
Hi Matthieu,

Could you please try using //Depot/Project<at>all instead of selecting a specific revision? Also, by using that command it means that the following depot paths must exist: //Depot/Project/Branch/Main //Depot/Project/Releases/2013 //Depot/Project/Branch/Feature1

I've never used the --use-client-spec, so I'm not sure if that will not break the branch detection code.

Cheers, Vitor

P.S. - Please keep me in CC because I'm not subscribed to the mailing list.

Matthieu Brucher· Jul 5, 2013, 18:11 UTC · re: Vitor Antunes · lore
> Hi Matthieu,
>
> Could you please try using //Depot/Project<at>all instead of selecting a
> specific revision?

I can try. Indeed, at this revision, the two other branches do not yet exist. But @all will get everything? Last time, I only got head (IIRC).

Show 5 quoted lines
> Also, by using that command it means that the following depot paths must
> exist:
> //Depot/Project/Branch/Main
> //Depot/Project/Releases/2013
> //Depot/Project/Branch/Feature1
Yes, they indeed do.
> I've never used the --use-client-spec, so I'm not sure if that will not
> break the branch detection code.

I need to do that because if I don't, the depot is clobbed with binaries. Or perhaps if I put some .gitignore stuff, I might not do this?

> Cheers,
> Vitor
Thanks for the tips, I will try tomorrow.
Cheers,
--
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/
Vitor Antunes· Jul 5, 2013, 18:36 UTC · re: Matthieu Brucher · lore

On Fri, Jul 5, 2013 at 7:11 PM, Matthieu Brucher <matthieu.brucher@gmail.com> wrote:

Show 8 quoted lines
>> Hi Matthieu,
>>
>> Could you please try using //Depot/Project<at>all instead of selecting a
>> specific revision?
>
> I can try. Indeed, at this revision, the two other branches do not yet
> exist. But @all will get everything? Last time, I only got head
> (IIRC).

Our P4 server has a limitation on the number of lines returned by "p4 changes" command, so sometimes I have to use @change_start,@change_stop instead of @all. You might want to use this range limitation to test git-p4 by limiting to a small number of changelists that allows you to check if at least one branch is correctly detected.

Show 7 quoted lines
>> Also, by using that command it means that the following depot paths must
>> exist:
>> //Depot/Project/Branch/Main
>> //Depot/Project/Releases/2013
>> //Depot/Project/Branch/Feature1
>
> Yes, they indeed do.
In this case the problem should not be in branchList configuration.
Show 6 quoted lines
>> I've never used the --use-client-spec, so I'm not sure if that will not
>> break the branch detection code.
>
> I need to do that because if I don't, the depot is clobbed with
> binaries. Or perhaps if I put some .gitignore stuff, I might not do
> this?

Keep using it, at least for now. If everything else fails we can look at this again.

Cheers, Vitor

Matthieu Brucher· Jul 5, 2013, 18:45 UTC · re: Vitor Antunes · lore
Show 9 quoted lines
>> I can try. Indeed, at this revision, the two other branches do not yet
>> exist. But @all will get everything? Last time, I only got head
>> (IIRC).
>
> Our P4 server has a limitation on the number of lines returned by "p4
> changes" command, so sometimes I have to use @change_start,@change_stop
> instead of @all. You might want to use this range limitation to test
> git-p4 by limiting to a small number of changelists that allows you to
> check if at least one branch is correctly detected.

I didn't know about this. I wanted to start the cloning at some point in the past, that's why I used the @123456789 notation.

Show 19 quoted lines
>>> Also, by using that command it means that the following depot paths must
>>> exist:
>>> //Depot/Project/Branch/Main
>>> //Depot/Project/Releases/2013
>>> //Depot/Project/Branch/Feature1
>>
>> Yes, they indeed do.
>
> In this case the problem should not be in branchList configuration.
>
>>> I've never used the --use-client-spec, so I'm not sure if that will not
>>> break the branch detection code.
>>
>> I need to do that because if I don't, the depot is clobbed with
>> binaries. Or perhaps if I put some .gitignore stuff, I might not do
>> this?
>
> Keep using it, at least for now. If everything else fails we can look at
> this again.
OK, I'll send a mail on Monday (forgot it was the week end tomorrow...)
Cheers,
Matthieu
--
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/
Matthieu Brucher· Jul 8, 2013, 10:09 UTC · re: Matthieu Brucher · lore
Hi again,
I tried with @all, but it didn'y work as expected. It imported a bunch
of revisions (but no files?) and ended with:
  Reading pipe: ['git', 'config', '--bool', 'git-p4.importLabels']
  Not checking out any branch, use "git checkout -q -b master <branch>"
  executing git config --bool git-p4.useclientspec true
And when I tried to checkout Branch/Main, it failed with
  fatal: Cannot update paths and switch to branch 'master' at the same time.
  Did you intend to checkout 'Branch/Main' which can not be resolved as commit?
Thanks,
Matthieu
2013/7/5 Matthieu Brucher <matthieu.brucher@gmail.com>:
Show 43 quoted lines
>>> I can try. Indeed, at this revision, the two other branches do not yet
>>> exist. But @all will get everything? Last time, I only got head
>>> (IIRC).
>>
>> Our P4 server has a limitation on the number of lines returned by "p4
>> changes" command, so sometimes I have to use @change_start,@change_stop
>> instead of @all. You might want to use this range limitation to test
>> git-p4 by limiting to a small number of changelists that allows you to
>> check if at least one branch is correctly detected.
>
> I didn't know about this. I wanted to start the cloning at some point
> in the past, that's why I used the @123456789 notation.
>
>>>> Also, by using that command it means that the following depot paths must
>>>> exist:
>>>> //Depot/Project/Branch/Main
>>>> //Depot/Project/Releases/2013
>>>> //Depot/Project/Branch/Feature1
>>>
>>> Yes, they indeed do.
>>
>> In this case the problem should not be in branchList configuration.
>>
>>>> I've never used the --use-client-spec, so I'm not sure if that will not
>>>> break the branch detection code.
>>>
>>> I need to do that because if I don't, the depot is clobbed with
>>> binaries. Or perhaps if I put some .gitignore stuff, I might not do
>>> this?
>>
>> Keep using it, at least for now. If everything else fails we can look at
>> this again.
>
> OK, I'll send a mail on Monday (forgot it was the week end tomorrow...)
>
> Cheers,
>
> Matthieu
> --
> Information System Engineer, Ph.D.
> Blog: http://matt.eifelle.com
> LinkedIn: http://www.linkedin.com/in/matthieubrucher
> Music band: http://liliejay.com/
-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/
Vitor Antunes· Jul 8, 2013, 10:24 UTC · re: Matthieu Brucher · lore

On Mon, Jul 8, 2013 at 11:09 AM, Matthieu Brucher <matthieu.brucher@gmail.com> wrote:

Show 11 quoted lines
> Hi again,
>
> I tried with @all, but it didn'y work as expected. It imported a bunch
> of revisions (but no files?) and ended with:
>   Reading pipe: ['git', 'config', '--bool', 'git-p4.importLabels']
>   Not checking out any branch, use "git checkout -q -b master <branch>"
>   executing git config --bool git-p4.useclientspec true
>
> And when I tried to checkout Branch/Main, it failed with
>   fatal: Cannot update paths and switch to branch 'master' at the same time.
>   Did you intend to checkout 'Branch/Main' which can not be resolved as commit?
Hi Matthieu,

Please run "git branch -a" in that repository and you should be able to see the various branches under /remotes/p4/ Then you just need to choose a branch and run "git checkout -b git_branch_name p4/p4_branch_name".

Cheers, Vitor

Matthieu Brucher· Jul 8, 2013, 10:51 UTC · re: Vitor Antunes · lore

Unfortunately, git branch -a returns nothing :/ I tried with the simple detect-branches as well as with the config values. Perhaps the spec-client? Although it is strange as it seems that the repository is completely empty.

Thanks,
Matthieu
2013/7/8 Vitor Antunes <vitor.hda@gmail.com>:
Show 23 quoted lines
> On Mon, Jul 8, 2013 at 11:09 AM, Matthieu Brucher
> <matthieu.brucher@gmail.com> wrote:
>> Hi again,
>>
>> I tried with @all, but it didn'y work as expected. It imported a bunch
>> of revisions (but no files?) and ended with:
>>   Reading pipe: ['git', 'config', '--bool', 'git-p4.importLabels']
>>   Not checking out any branch, use "git checkout -q -b master <branch>"
>>   executing git config --bool git-p4.useclientspec true
>>
>> And when I tried to checkout Branch/Main, it failed with
>>   fatal: Cannot update paths and switch to branch 'master' at the same time.
>>   Did you intend to checkout 'Branch/Main' which can not be resolved as commit?
>
> Hi Matthieu,
>
> Please run "git branch -a" in that repository and you should be able
> to see the various branches under /remotes/p4/
> Then you just need to choose a branch and run "git checkout -b
> git_branch_name p4/p4_branch_name".
>
> Cheers,
> Vitor
-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/
Matthieu Brucher· Jul 8, 2013, 11:10 UTC · re: Matthieu Brucher · lore

Without the spec client, it seems that the branches are recognized, but there are some many binary files that I need to remove them during the migration. I tried setting a .gitignore beforehand, but it is not respected (I tried to remove some folders with folder/ in .gitignore, but the folder are still imported). It there a switch for the import somewhere?

Thanks,
Matthieu
2013/7/8 Matthieu Brucher <matthieu.brucher@gmail.com>:
Show 41 quoted lines
> Unfortunately, git branch -a returns nothing :/
> I tried with the simple detect-branches as well as with the config values.
> Perhaps the spec-client? Although it is strange as it seems that the
> repository is completely empty.
>
> Thanks,
>
> Matthieu
>
> 2013/7/8 Vitor Antunes <vitor.hda@gmail.com>:
>> On Mon, Jul 8, 2013 at 11:09 AM, Matthieu Brucher
>> <matthieu.brucher@gmail.com> wrote:
>>> Hi again,
>>>
>>> I tried with @all, but it didn'y work as expected. It imported a bunch
>>> of revisions (but no files?) and ended with:
>>>   Reading pipe: ['git', 'config', '--bool', 'git-p4.importLabels']
>>>   Not checking out any branch, use "git checkout -q -b master <branch>"
>>>   executing git config --bool git-p4.useclientspec true
>>>
>>> And when I tried to checkout Branch/Main, it failed with
>>>   fatal: Cannot update paths and switch to branch 'master' at the same time.
>>>   Did you intend to checkout 'Branch/Main' which can not be resolved as commit?
>>
>> Hi Matthieu,
>>
>> Please run "git branch -a" in that repository and you should be able
>> to see the various branches under /remotes/p4/
>> Then you just need to choose a branch and run "git checkout -b
>> git_branch_name p4/p4_branch_name".
>>
>> Cheers,
>> Vitor
>
>
>
> --
> Information System Engineer, Ph.D.
> Blog: http://matt.eifelle.com
> LinkedIn: http://www.linkedin.com/in/matthieubrucher
> Music band: http://liliejay.com/
-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/
Vitor Antunes· Jul 8, 2013, 13:43 UTC · re: Matthieu Brucher · lore

On Mon, Jul 8, 2013 at 12:10 PM, Matthieu Brucher <matthieu.brucher@gmail.com> wrote:

Show 7 quoted lines
> Without the spec client, it seems that the branches are recognized,
> but there are some many binary files that I need to remove them during
> the migration.
> I tried setting a .gitignore beforehand, but it is not respected (I
> tried to remove some folders with folder/ in .gitignore, but the
> folder are still imported).
> It there a switch for the import somewhere?
Hi Matthieu,

Unfortunately I've never tested the branch detection together with spec configuration. But there is a test case for it in the code that refers to the following question in StackOverflow:

http://stackoverflow.com/questions/11893688
Could you also tell us which version of git you are using?
Pete, maybe you can help Matthieu further on this question?

Thanks, Vitor

Matthieu Brucher· Jul 8, 2013, 14:03 UTC · re: Vitor Antunes · lore
Hi,

I tried without spec, but then it tried importing everything, even though there was a .gitignore and a .git/config/exclude file. Then, it crashed during the importation because it could find an old branch (I don't have access to everything on the repository), so I tried importing just the recent past, but then it failed because it identified a branch names Branch/Main/src... It is starting to feel as if I will have to compromise between something that works but without branches and without the proper names (the files are named Project/Branch/Main/...) or having the proper names, but with all binaries, bogus branches... I know it is not due to git, it is mainly that Perforce and git have very different workflows. Or perhaps with any luck, the server is up to date, and I can find a way of using Perforce's bridge.

Thanks,
Matthieu
2013/7/8 Vitor Antunes <vitor.hda@gmail.com>:
Show 24 quoted lines
> On Mon, Jul 8, 2013 at 12:10 PM, Matthieu Brucher
> <matthieu.brucher@gmail.com> wrote:
>> Without the spec client, it seems that the branches are recognized,
>> but there are some many binary files that I need to remove them during
>> the migration.
>> I tried setting a .gitignore beforehand, but it is not respected (I
>> tried to remove some folders with folder/ in .gitignore, but the
>> folder are still imported).
>> It there a switch for the import somewhere?
>
> Hi Matthieu,
>
> Unfortunately I've never tested the branch detection together with spec
> configuration. But there is a test case for it in the code that refers
> to the following question in StackOverflow:
>
> http://stackoverflow.com/questions/11893688
>
> Could you also tell us which version of git you are using?
>
> Pete, maybe you can help Matthieu further on this question?
>
> Thanks,
> Vitor
-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/
Vitor Antunes· Jul 13, 2013, 00:28 UTC · re: Matthieu Brucher · lore

On Mon, Jul 8, 2013 at 3:03 PM, Matthieu Brucher <matthieu.brucher@gmail.com> wrote:

Show 15 quoted lines
> Hi,
>
> I tried without spec, but then it tried importing everything, even
> though there was a .gitignore and a .git/config/exclude file.
> Then, it crashed during the importation because it could find an old
> branch (I don't have access to everything on the repository), so I
> tried importing just the recent past, but then it failed because it
> identified a branch names Branch/Main/src...
> It is starting to feel as if I will have to compromise between
> something that works but without branches and without the proper names
> (the files are named Project/Branch/Main/...) or having the proper
> names, but with all binaries, bogus branches...
> I know it is not due to git, it is mainly that Perforce and git have
> very different workflows. Or perhaps with any luck, the server is up
> to date, and I can find a way of using Perforce's bridge.
Hi Matthieu,

I created a simple test case where I added a file to one of the branches in P4 but excluded it in the client spec. During synchronization git p4 correctly ignored it.

Please check which git version you are using. It is possible the issue you are experiencing was fixed in a more recent version.

Also, if possible, please try creating a simple test case that replicates your issue (p4 and p4d are free to use when the repo is only used by one person). If you record the commands you used and share them with me I will be able to better check this issue on my side.

Cheers, Vitor

← back to recent threads