threads / discuss / 27101

git-p4 issue

Subject: git-p4 issue

## tl;dr

13 messages between Apr 15, 2011 and Dec 17, 2011.

replies: 12people: 4as markdown or json

Michael Horowitz· Apr 15, 2011, 03:00 UTC · lore
Pete,

I was hoping you could help me out again.  After using git-p4 for a while without a problem, it has suddenly stopped working for me.  I am using the latest master.  I haven't seen any recent changes that I think could have caused this, but maybe you'll have some insight.

The issue is that when I do a git-p4 sync on my existing repository, it reports success, but seems to do nothing.  It does not download the latest changes from p4.  If I delete my repository and start over, it will download all the latest changes, even the ones it was not downloading previously, but if I try to sync again later, it does not do anything.  I tried running it with the "--verbose" mode, and I see it says it is loading each of the changes, but they are not ending up in the git repository, and it is not reporting any errors.

Any ideas of what this could be?  Is there anything else I can run to help debug this?

Thanks,
Mike
Tor Arvid Lund· Apr 15, 2011, 20:22 UTC · re: Michael Horowitz · lore

Re: git-p4 issue

On Fri, Apr 15, 2011 at 5:00 AM, Michael Horowitz <michael.horowitz@ieee.org> wrote:

Show 15 quoted lines
> Pete,
>
> I was hoping you could help me out again.  After using git-p4 for a
> while without a problem, it has suddenly stopped working for me.  I am
> using the latest master.  I haven't seen any recent changes that I
> think could have caused this, but maybe you'll have some insight.
>
> The issue is that when I do a git-p4 sync on my existing repository,
> it reports success, but seems to do nothing.  It does not download the
> latest changes from p4.  If I delete my repository and start over, it
> will download all the latest changes, even the ones it was not
> downloading previously, but if I try to sync again later, it does not
> do anything.  I tried running it with the "--verbose" mode, and I see
> it says it is loading each of the changes, but they are not ending up
> in the git repository, and it is not reporting any errors.
Hi, Michael.

Is it possible that you expect that 'git p4 sync' should update your working branch and/or working tree? Assuming a simple clone with a local master branch, running 'git-p4 sync' will update the branch remotes/p4/master, but it won't do anything on my working master branch...

Maybe you want to call 'git rebase p4/master' afterwards, or use the shorthand 'git p4 rebase' to do a sync+rebase.

Regards, Tor Arvid.

Show 11 quoted lines
> Any ideas of what this could be?  Is there anything else I can run to
> help debug this?
>
> Thanks,
>
> Mike
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
Michael Horowitz· Apr 15, 2011, 20:39 UTC · re: Tor Arvid Lund · lore

Re: git-p4 issue

I am sure that is a common mistake people make, but not in this case. I have been using it successfully for a while now, it just suddenly stopped working, not sure what changed. I am not seeing anything on remotes/p4/master either, and I was originally doing rebase and went back to sync so I could run "--verbose" and see if it was even downloading those changes. I can clearly see it says it is downloading them, but then they just don't end up in git.

Since I don't see an error message about it failing to sync, I am at a
loss to figure out why it says it succeeded, but it didn't.  Could
there be one step in the code that is not catching an error condition?
 I am not all that familiar with Python, but if someone could point me
where to put some debug messages, I can do some testing.
Thanks,
Mike
On Fri, Apr 15, 2011 at 4:22 PM, Tor Arvid Lund <torarvid@gmail.com> wrote:
Show 44 quoted lines
>
> On Fri, Apr 15, 2011 at 5:00 AM, Michael Horowitz
> <michael.horowitz@ieee.org> wrote:
> > Pete,
> >
> > I was hoping you could help me out again.  After using git-p4 for a
> > while without a problem, it has suddenly stopped working for me.  I am
> > using the latest master.  I haven't seen any recent changes that I
> > think could have caused this, but maybe you'll have some insight.
> >
> > The issue is that when I do a git-p4 sync on my existing repository,
> > it reports success, but seems to do nothing.  It does not download the
> > latest changes from p4.  If I delete my repository and start over, it
> > will download all the latest changes, even the ones it was not
> > downloading previously, but if I try to sync again later, it does not
> > do anything.  I tried running it with the "--verbose" mode, and I see
> > it says it is loading each of the changes, but they are not ending up
> > in the git repository, and it is not reporting any errors.
>
> Hi, Michael.
>
> Is it possible that you expect that 'git p4 sync' should update your
> working branch and/or working tree? Assuming a simple clone with a
> local master branch, running 'git-p4 sync' will update the branch
> remotes/p4/master, but it won't do anything on my working master
> branch...
>
> Maybe you want to call 'git rebase p4/master' afterwards, or use the
> shorthand 'git p4 rebase' to do a sync+rebase.
>
> Regards,
> Tor Arvid.
>
> > Any ideas of what this could be?  Is there anything else I can run to
> > help debug this?
> >
> > Thanks,
> >
> > Mike
> > --
> > To unsubscribe from this list: send the line "unsubscribe git" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
Pete Wyckoff· Apr 16, 2011, 16:01 UTC · re: Michael Horowitz · lore

Re: git-p4 issue

michael.horowitz@ieee.org wrote on Fri, 15 Apr 2011 16:39 -0400:
Show 13 quoted lines
> I am sure that is a common mistake people make, but not in this case.
> I have been using it successfully for a while now, it just suddenly
> stopped working, not sure what changed.  I am not seeing anything on
> remotes/p4/master either, and I was originally doing rebase and went
> back to sync so I could run "--verbose" and see if it was even
> downloading those changes.  I can clearly see it says it is
> downloading them, but then they just don't end up in git.
> 
> Since I don't see an error message about it failing to sync, I am at a
> loss to figure out why it says it succeeded, but it didn't.  Could
> there be one step in the code that is not catching an error condition?
>  I am not all that familiar with Python, but if someone could point me
> where to put some debug messages, I can do some testing.

It would say something if it failed. If git-p4 says it brought in changes, they would appear at the top of p4/master, or another branch in remotes/p4 if you use git-p4's auto-branch feature.

Do "git branch -av". You should see remotes/p4/master with the latest p4 change.

It is possible to create a branch "p4/master" that is not remotes/p4/master and possibly cause confusion. "tree -a .git/refs" to see all your branches.

You can try "git fsck" to see if something is terribly wrong. Or nose around .git, see if there are some recently-created objects and use "git cat-file -p" to look at them.

If you are willing to tar up your .git and leave it somewhere, I can take a look too.

		-- Pete
Show 52 quoted lines
> Thanks,
> 
> Mike
> 
> 
> 
> On Fri, Apr 15, 2011 at 4:22 PM, Tor Arvid Lund <torarvid@gmail.com> wrote:
> >
> > On Fri, Apr 15, 2011 at 5:00 AM, Michael Horowitz
> > <michael.horowitz@ieee.org> wrote:
> > > Pete,
> > >
> > > I was hoping you could help me out again.  After using git-p4 for a
> > > while without a problem, it has suddenly stopped working for me.  I am
> > > using the latest master.  I haven't seen any recent changes that I
> > > think could have caused this, but maybe you'll have some insight.
> > >
> > > The issue is that when I do a git-p4 sync on my existing repository,
> > > it reports success, but seems to do nothing.  It does not download the
> > > latest changes from p4.  If I delete my repository and start over, it
> > > will download all the latest changes, even the ones it was not
> > > downloading previously, but if I try to sync again later, it does not
> > > do anything.  I tried running it with the "--verbose" mode, and I see
> > > it says it is loading each of the changes, but they are not ending up
> > > in the git repository, and it is not reporting any errors.
> >
> > Hi, Michael.
> >
> > Is it possible that you expect that 'git p4 sync' should update your
> > working branch and/or working tree? Assuming a simple clone with a
> > local master branch, running 'git-p4 sync' will update the branch
> > remotes/p4/master, but it won't do anything on my working master
> > branch...
> >
> > Maybe you want to call 'git rebase p4/master' afterwards, or use the
> > shorthand 'git p4 rebase' to do a sync+rebase.
> >
> > Regards,
> > Tor Arvid.
> >
> > > Any ideas of what this could be?  Is there anything else I can run to
> > > help debug this?
> > >
> > > Thanks,
> > >
> > > Mike
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe git" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > >
> 
Vitor Antunes· Apr 18, 2011, 12:59 UTC · re: Pete Wyckoff · lore

Re: git-p4 issue

Hi Pete and Michael,
Pete Wyckoff <pw <at> padd.com> writes:
Show 9 quoted lines
> 
> michael.horowitz <at> ieee.org wrote on Fri, 15 Apr 2011 16:39 -0400:
> > I am sure that is a common mistake people make, but not in this case.
> > I have been using it successfully for a while now, it just suddenly
> > stopped working, not sure what changed.  I am not seeing anything on
> > remotes/p4/master either, and I was originally doing rebase and went
> > back to sync so I could run "--verbose" and see if it was even
> > downloading those changes.  I can clearly see it says it is
> > downloading them, but then they just don't end up in git.

I've seen similar behavior when branch detection is enabled. Could it be that the new changelists are not being imported because they are under a folder that is not part of the path provided to git-p4 sync (or automatically detected by it) during the original import? That would explain why --verbose shows the changelists being downloaded but then they are not imported.

You said that if you clone the depot again you see the new changes. Could you please confirm if "depot_paths" value is the same in git-log of both git repos?

Show 25 quoted lines
> > Since I don't see an error message about it failing to sync, I am at a
> > loss to figure out why it says it succeeded, but it didn't.  Could
> > there be one step in the code that is not catching an error condition?
> >  I am not all that familiar with Python, but if someone could point me
> > where to put some debug messages, I can do some testing.
> 
> It would say something if it failed.  If git-p4 says it brought
> in changes, they would appear at the top of p4/master, or another
> branch in remotes/p4 if you use git-p4's auto-branch feature.
> 
> Do "git branch -av".  You should see remotes/p4/master with the
> latest p4 change.
> 
> It is possible to create a branch "p4/master" that is not
> remotes/p4/master and possibly cause confusion.  "tree -a
> .git/refs" to see all your branches.
> 
> You can try "git fsck" to see if something is terribly wrong.
> Or nose around .git, see if there are some recently-created
> objects and use "git cat-file -p" to look at them.
> 
> If you are willing to tar up your .git and leave it somewhere,
> I can take a look too.
> 
> 		-- Pete
Vitor
Michael Horowitz· Apr 19, 2011, 03:57 UTC · re: Michael Horowitz · lore

Re: git-p4 issue

OK, after some digging, I think I have figured out what is going on, but I am not sure how to fix it, at least not safely.

There seem to be several different ways of detecting branches, and I am not exactly sure what they are all used for, or why there are so many, but the core of the issue I am having is that in importChanges when it calls splitFilesIntoBranches, it assumes "self.knownBranches" has all the branches, even though it already has the branches from "self.p4BranchesInGit". The exact reason I don't know, but the results is splitFilesIntoBranches returns an empty array, and so when the code loops over it, it silently does nothing.

The first fix that would be helpful is to at least report an error if it can't find the branches, rather than silently doing nothing. I am not exactly sure why it needs to look in "self.knownBranches", so I don't know what the error should report, maybe the error should be reported earlier?

The other issue is how "self.knownBranches" seems to be populated. It looks like form my code path, which decides to "Import from/to multiple branches", it tries to detect branches by using "p4 branches". Again, this is odd, since I can see it already has the names of the branches from "self.p4BranchesInGit". I am not familiar enough with the code (and figuring out Python as I go along) to know why. The problem with using "p4 branches" is those aren't really branches, they are aliases to a merge command (integrate in p4 lingo) which stores the from and to branch. The branch in Perforce is really just a directory. Interestingly enough, it seems this logic also attempts to detect new branches and automatically import them, but ironically this doesn't actually work for me.

So, the crux of the problem is that "p4 branches" are not necessary to have at all. The reason this suddenly stopped working for me is that someone had created one of these branch definitions and I didn't know, so it was accidentally working all this time, but only for 2 of the branches. Then the person removed the definition, and it stopped working. Now the workaround is to go and create these things for every branch, but considering these are unnecessary and cumbersome to create, and the code seems to be able to find the branches already from the "self.p4BranchesInGit" anyway, I would like to remove the dependency on that logic.

Now, I could go ahead and hack something that does things differently, but since I don't really know the intention of these structures or how it might impact elsewhere in the code, I could use some guidance from someone who knows this code well.

Thanks,
Mike

On Fri, Apr 15, 2011 at 4:39 PM, Michael Horowitz <michael.horowitz@ieee.org> wrote:

Show 66 quoted lines
> I am sure that is a common mistake people make, but not in this case.
> I have been using it successfully for a while now, it just suddenly
> stopped working, not sure what changed.  I am not seeing anything on
> remotes/p4/master either, and I was originally doing rebase and went
> back to sync so I could run "--verbose" and see if it was even
> downloading those changes.  I can clearly see it says it is
> downloading them, but then they just don't end up in git.
>
> Since I don't see an error message about it failing to sync, I am at a
> loss to figure out why it says it succeeded, but it didn't.  Could
> there be one step in the code that is not catching an error condition?
>  I am not all that familiar with Python, but if someone could point me
> where to put some debug messages, I can do some testing.
>
> Thanks,
>
> Mike
>
>
>
> On Fri, Apr 15, 2011 at 4:22 PM, Tor Arvid Lund <torarvid@gmail.com> wrote:
>>
>> On Fri, Apr 15, 2011 at 5:00 AM, Michael Horowitz
>> <michael.horowitz@ieee.org> wrote:
>> > Pete,
>> >
>> > I was hoping you could help me out again.  After using git-p4 for a
>> > while without a problem, it has suddenly stopped working for me.  I am
>> > using the latest master.  I haven't seen any recent changes that I
>> > think could have caused this, but maybe you'll have some insight.
>> >
>> > The issue is that when I do a git-p4 sync on my existing repository,
>> > it reports success, but seems to do nothing.  It does not download the
>> > latest changes from p4.  If I delete my repository and start over, it
>> > will download all the latest changes, even the ones it was not
>> > downloading previously, but if I try to sync again later, it does not
>> > do anything.  I tried running it with the "--verbose" mode, and I see
>> > it says it is loading each of the changes, but they are not ending up
>> > in the git repository, and it is not reporting any errors.
>>
>> Hi, Michael.
>>
>> Is it possible that you expect that 'git p4 sync' should update your
>> working branch and/or working tree? Assuming a simple clone with a
>> local master branch, running 'git-p4 sync' will update the branch
>> remotes/p4/master, but it won't do anything on my working master
>> branch...
>>
>> Maybe you want to call 'git rebase p4/master' afterwards, or use the
>> shorthand 'git p4 rebase' to do a sync+rebase.
>>
>> Regards,
>> Tor Arvid.
>>
>> > Any ideas of what this could be?  Is there anything else I can run to
>> > help debug this?
>> >
>> > Thanks,
>> >
>> > Mike
>> > --
>> > To unsubscribe from this list: send the line "unsubscribe git" in
>> > the body of a message to majordomo@vger.kernel.org
>> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> >
>
Vitor Antunes· Apr 19, 2011, 09:59 UTC · re: Michael Horowitz · lore

Re: git-p4 issue

Hi Mike,
Could you please search for the following set of patches in this mailing list?
[PATCH v2 0/3] git-p4: Improve branch support

I think I sent v2 twice somehow, so please make sure you pick the latest ;) In these patches I add the possibility to use a "git-p4.branchList" configuration to define the branches. The patch is still to be approved because most people in the mailing list do not use branch detection, but I use it daily and it is working in my side. Could you please test it?

Thanks, Vitor

P.S. - It's better that you do not apply "[PATCH v2 1/3] git-p4: Correct branch base depot path detection", as it may require you to clone the all depot again.

Pete Wyckoff· Apr 20, 2011, 00:31 UTC · re: Michael Horowitz · lore

Re: git-p4 issue

michael.horowitz@ieee.org wrote on Mon, 18 Apr 2011 23:57 -0400:
Show 11 quoted lines
> OK, after some digging, I think I have figured out what is going on,
> but I am not sure how to fix it, at least not safely.
> 
> There seem to be several different ways of detecting branches, and I
> am not exactly sure what they are all used for, or why there are so
> many, but the core of the issue I am having is that in importChanges
> when it calls splitFilesIntoBranches, it assumes "self.knownBranches"
> has all the branches, even though it already has the branches from
> "self.p4BranchesInGit".  The exact reason I don't know, but the
> results is splitFilesIntoBranches returns an empty array, and so when
> the code loops over it, it silently does nothing.

I too am confused by the branch handling in git-p4, and have never used it. I'd love to rip it all out, along with the confusion, but know that some people use it with success.

At least it all could use some overhaul and documentation so we can see what's going on.

Show 18 quoted lines
> The first fix that would be helpful is to at least report an error if
> it can't find the branches, rather than silently doing nothing.  I am
> not exactly sure why it needs to look in "self.knownBranches", so I
> don't know what the error should report, maybe the error should be
> reported earlier?
> 
> The other issue is how "self.knownBranches" seems to be populated.  It
> looks like form my code path, which decides to "Import from/to
> multiple branches", it tries to detect branches by using "p4
> branches".  Again, this is odd, since I can see it already has the
> names of the branches from "self.p4BranchesInGit".  I am not familiar
> enough with the code (and figuring out Python as I go along) to know
> why.  The problem with using "p4 branches" is those aren't really
> branches, they are aliases to a merge command (integrate in p4 lingo)
> which stores the from and to branch.  The branch in Perforce is really
> just a directory.  Interestingly enough, it seems this logic also
> attempts to detect new branches and automatically import them, but
> ironically this doesn't actually work for me.

This is my understanding of "p4 branch" as well. At our site, the list of p4 branches does not at all correspond to what we think of as code development branches in git. Again, maybe others use it differently?

We do maintain p4 view lists, but that is kept out-of-band, not in any p4 mechanism. These map friendlier short names to a list of directories in p4 and how to assemble those into a workspace.

Show 15 quoted lines
> So, the crux of the problem is that "p4 branches" are not necessary to
> have at all.  The reason this suddenly stopped working for me is that
> someone had created one of these branch definitions and I didn't know,
> so it was accidentally working all this time, but only for 2 of the
> branches.  Then the person removed the definition, and it stopped
> working.  Now the workaround is to go and create these things for
> every branch, but considering these are unnecessary and cumbersome to
> create, and the code seems to be able to find the branches already
> from the "self.p4BranchesInGit" anyway, I would like to remove the
> dependency on that logic.
> 
> Now, I could go ahead and hack something that does things differently,
> but since I don't really know the intention of these structures or how
> it might impact elsewhere in the code, I could use some guidance from
> someone who knows this code well.

Vitor uses branches, and his patch that he recommends might be the work-around you are looking for.

I thought all this branch code was opt-in, so if you fail to say "--detect-branches", it won't try to auto-detect anything.

But there is maybe another use case in here, which is to import multiple directories of the depot into _different_ refs/remotes/p4/<branch>. (I've only ever done one at a time, and into the default p4/master.) And now that you have multiple git-p4 branches, you're stuck with them due to the login in p4BranchesInGit(). That feature should be handled independently of the "p4 branch" auto-detection one.

The branch handling needs rework. You might help by describing how you want it to work and we can see if this is the same as how Vitor uses branches.

		-- Pete
Michael Horowitz· Apr 20, 2011, 02:40 UTC · re: Pete Wyckoff · lore

Re: git-p4 issue

I don't have a problem with the branch detection if other people use it in ways I don't, but it would be nice to have more options and documentation around it.

The best I can do to describe what I want is for it to use what is returned by "git-p4 branches", at minimum.  If there is some optional additional "new branch detection" logic, I don't have a problem with that, but that should only be in addition to the branches it already knows about from "git-p4 branches".  So, when I do a "git-p4 sync" or "git-p4 rebase", and it is importing changes from/to multiple branches, then it should get that list of branches using the same method "git-p4 branches" uses.  Does that make sense?

Thanks,
Mike
On Tue, Apr 19, 2011 at 8:31 PM, Pete Wyckoff <pw@padd.com> wrote:
Show 84 quoted lines
>
> michael.horowitz@ieee.org wrote on Mon, 18 Apr 2011 23:57 -0400:
> > OK, after some digging, I think I have figured out what is going on,
> > but I am not sure how to fix it, at least not safely.
> >
> > There seem to be several different ways of detecting branches, and I
> > am not exactly sure what they are all used for, or why there are so
> > many, but the core of the issue I am having is that in importChanges
> > when it calls splitFilesIntoBranches, it assumes "self.knownBranches"
> > has all the branches, even though it already has the branches from
> > "self.p4BranchesInGit".  The exact reason I don't know, but the
> > results is splitFilesIntoBranches returns an empty array, and so when
> > the code loops over it, it silently does nothing.
>
> I too am confused by the branch handling in git-p4, and have never
> used it.  I'd love to rip it all out, along with the confusion,
> but know that some people use it with success.
>
> At least it all could use some overhaul and documentation so we
> can see what's going on.
>
> > The first fix that would be helpful is to at least report an error if
> > it can't find the branches, rather than silently doing nothing.  I am
> > not exactly sure why it needs to look in "self.knownBranches", so I
> > don't know what the error should report, maybe the error should be
> > reported earlier?
> >
> > The other issue is how "self.knownBranches" seems to be populated.  It
> > looks like form my code path, which decides to "Import from/to
> > multiple branches", it tries to detect branches by using "p4
> > branches".  Again, this is odd, since I can see it already has the
> > names of the branches from "self.p4BranchesInGit".  I am not familiar
> > enough with the code (and figuring out Python as I go along) to know
> > why.  The problem with using "p4 branches" is those aren't really
> > branches, they are aliases to a merge command (integrate in p4 lingo)
> > which stores the from and to branch.  The branch in Perforce is really
> > just a directory.  Interestingly enough, it seems this logic also
> > attempts to detect new branches and automatically import them, but
> > ironically this doesn't actually work for me.
>
> This is my understanding of "p4 branch" as well.  At our site,
> the list of p4 branches does not at all correspond to what we
> think of as code development branches in git.  Again, maybe
> others use it differently?
>
> We do maintain p4 view lists, but that is kept out-of-band, not
> in any p4 mechanism.  These map friendlier short names to a list
> of directories in p4 and how to assemble those into a workspace.
>
> > So, the crux of the problem is that "p4 branches" are not necessary to
> > have at all.  The reason this suddenly stopped working for me is that
> > someone had created one of these branch definitions and I didn't know,
> > so it was accidentally working all this time, but only for 2 of the
> > branches.  Then the person removed the definition, and it stopped
> > working.  Now the workaround is to go and create these things for
> > every branch, but considering these are unnecessary and cumbersome to
> > create, and the code seems to be able to find the branches already
> > from the "self.p4BranchesInGit" anyway, I would like to remove the
> > dependency on that logic.
> >
> > Now, I could go ahead and hack something that does things differently,
> > but since I don't really know the intention of these structures or how
> > it might impact elsewhere in the code, I could use some guidance from
> > someone who knows this code well.
>
> Vitor uses branches, and his patch that he recommends might be
> the work-around you are looking for.
>
> I thought all this branch code was opt-in, so if you fail to say
> "--detect-branches", it won't try to auto-detect anything.
>
> But there is maybe another use case in here, which is to
> import multiple directories of the depot into _different_
> refs/remotes/p4/<branch>.  (I've only ever done one at a
> time, and into the default p4/master.)  And now that you
> have multiple git-p4 branches, you're stuck with them due to the
> login in p4BranchesInGit().  That feature should be handled
> independently of the "p4 branch" auto-detection one.
>
> The branch handling needs rework.  You might help by describing
> how you want it to work and we can see if this is the same as how
> Vitor uses branches.
>
>                -- Pete
Vitor Antunes· Apr 20, 2011, 10:51 UTC · re: Michael Horowitz · lore

Re: git-p4 issue

Hi Mike,
Michael Horowitz <michael.horowitz <at> ieee.org> writes:
> I don't have a problem with the branch detection if other people use
> it in ways I don't, but it would be nice to have more options and
> documentation around it.

Yes, documentation is a bit scarce. And looking again at my patch the description text also does not seem quite good enough.

Show 8 quoted lines
> The best I can do to describe what I want is for it to use what is
> returned by "git-p4 branches", at minimum. If there is some optional
> additional "new branch detection" logic, I don't have a problem with
> that, but that should only be in addition to the branches it already
> knows about from "git-p4 branches". So, when I do a "git-p4 sync" or
> "git-p4 rebase", and it is importing changes from/to multiple
> branches, then it should get that list of branches using the same
> method "git-p4 branches" uses. Does that make sense?

I think I understand your point and it may make sense. Currently, "self.knownBranches" is the list used during import from P4. The idea behind making this list different from "self.p4BranchesInGit" might have been to allow stop following a given branch by removing its definition from P4. Of course, if you already imported it earlier and there is a commit into it I think it makes sense to import the new commit instead of ignoring it as it is being done now. With that said, I think it would be a good idea to somehow merge the two lists together. But, as Pete already pointed out, the branch code is too complex as it is now and it needs a deep review. So it might make sense to include this feature as part of that review.

The patch I directed you to ([1]) allows you to create a list of branch-origin to branch-destination pairs independent of "p4 branches" output. So you should be able to use this as a workaround for now.

> Thanks,
> 
> Mike

Regards, Vitor

[1] http://article.gmane.org/gmane.comp.version-control.git/168001
Michael Horowitz· May 6, 2011, 20:16 UTC · re: Michael Horowitz · lore

Re: git-p4 issue

Vitor,
Show 9 quoted lines
> Could you please search for the following set of patches in this mailing list?
>
> [PATCH v2 0/3] git-p4: Improve branch support
>
> I think I sent v2 twice somehow, so please make sure you pick the latest ;)
> In these patches I add the possibility to use a "git-p4.branchList"
> configuration to define the branches. The patch is still to be approved because
> most people in the mailing list do not use branch detection, but I use it daily
> and it is working in my side. Could you please test it?

Can you send this patch again? It looks like you had previously responded to the list only, so I never got this message, as I wasn't on the list at the time (I am now). I only saw this because I was searching the archive for something else. I searched for the patch, but the actual patch body doesn't seem to be in the archive.

Thanks,
Mike

On Tue, Apr 19, 2011 at 10:40 PM, Michael Horowitz <michael.horowitz@ieee.org> wrote:

Show 105 quoted lines
> I don't have a problem with the branch detection if other people use
> it in ways I don't, but it would be nice to have more options and
> documentation around it.
>
> The best I can do to describe what I want is for it to use what is
> returned by "git-p4 branches", at minimum.  If there is some optional
> additional "new branch detection" logic, I don't have a problem with
> that, but that should only be in addition to the branches it already
> knows about from "git-p4 branches".  So, when I do a "git-p4 sync" or
> "git-p4 rebase", and it is importing changes from/to multiple
> branches, then it should get that list of branches using the same
> method "git-p4 branches" uses.  Does that make sense?
>
> Thanks,
>
> Mike
>
>
>
> On Tue, Apr 19, 2011 at 8:31 PM, Pete Wyckoff <pw@padd.com> wrote:
>>
>> michael.horowitz@ieee.org wrote on Mon, 18 Apr 2011 23:57 -0400:
>> > OK, after some digging, I think I have figured out what is going on,
>> > but I am not sure how to fix it, at least not safely.
>> >
>> > There seem to be several different ways of detecting branches, and I
>> > am not exactly sure what they are all used for, or why there are so
>> > many, but the core of the issue I am having is that in importChanges
>> > when it calls splitFilesIntoBranches, it assumes "self.knownBranches"
>> > has all the branches, even though it already has the branches from
>> > "self.p4BranchesInGit".  The exact reason I don't know, but the
>> > results is splitFilesIntoBranches returns an empty array, and so when
>> > the code loops over it, it silently does nothing.
>>
>> I too am confused by the branch handling in git-p4, and have never
>> used it.  I'd love to rip it all out, along with the confusion,
>> but know that some people use it with success.
>>
>> At least it all could use some overhaul and documentation so we
>> can see what's going on.
>>
>> > The first fix that would be helpful is to at least report an error if
>> > it can't find the branches, rather than silently doing nothing.  I am
>> > not exactly sure why it needs to look in "self.knownBranches", so I
>> > don't know what the error should report, maybe the error should be
>> > reported earlier?
>> >
>> > The other issue is how "self.knownBranches" seems to be populated.  It
>> > looks like form my code path, which decides to "Import from/to
>> > multiple branches", it tries to detect branches by using "p4
>> > branches".  Again, this is odd, since I can see it already has the
>> > names of the branches from "self.p4BranchesInGit".  I am not familiar
>> > enough with the code (and figuring out Python as I go along) to know
>> > why.  The problem with using "p4 branches" is those aren't really
>> > branches, they are aliases to a merge command (integrate in p4 lingo)
>> > which stores the from and to branch.  The branch in Perforce is really
>> > just a directory.  Interestingly enough, it seems this logic also
>> > attempts to detect new branches and automatically import them, but
>> > ironically this doesn't actually work for me.
>>
>> This is my understanding of "p4 branch" as well.  At our site,
>> the list of p4 branches does not at all correspond to what we
>> think of as code development branches in git.  Again, maybe
>> others use it differently?
>>
>> We do maintain p4 view lists, but that is kept out-of-band, not
>> in any p4 mechanism.  These map friendlier short names to a list
>> of directories in p4 and how to assemble those into a workspace.
>>
>> > So, the crux of the problem is that "p4 branches" are not necessary to
>> > have at all.  The reason this suddenly stopped working for me is that
>> > someone had created one of these branch definitions and I didn't know,
>> > so it was accidentally working all this time, but only for 2 of the
>> > branches.  Then the person removed the definition, and it stopped
>> > working.  Now the workaround is to go and create these things for
>> > every branch, but considering these are unnecessary and cumbersome to
>> > create, and the code seems to be able to find the branches already
>> > from the "self.p4BranchesInGit" anyway, I would like to remove the
>> > dependency on that logic.
>> >
>> > Now, I could go ahead and hack something that does things differently,
>> > but since I don't really know the intention of these structures or how
>> > it might impact elsewhere in the code, I could use some guidance from
>> > someone who knows this code well.
>>
>> Vitor uses branches, and his patch that he recommends might be
>> the work-around you are looking for.
>>
>> I thought all this branch code was opt-in, so if you fail to say
>> "--detect-branches", it won't try to auto-detect anything.
>>
>> But there is maybe another use case in here, which is to
>> import multiple directories of the depot into _different_
>> refs/remotes/p4/<branch>.  (I've only ever done one at a
>> time, and into the default p4/master.)  And now that you
>> have multiple git-p4 branches, you're stuck with them due to the
>> login in p4BranchesInGit().  That feature should be handled
>> independently of the "p4 branch" auto-detection one.
>>
>> The branch handling needs rework.  You might help by describing
>> how you want it to work and we can see if this is the same as how
>> Vitor uses branches.
>>
>>                -- Pete
>
Vitor Antunes· May 13, 2011, 14:31 UTC · re: Michael Horowitz · lore

Re: git-p4 issue

Hi Michael,
Michael Horowitz <michael.horowitz <at> ieee.org> writes:
Show 21 quoted lines
> 
> Vitor,
> 
> > Could you please search for the following set of patches in this
> > mailing list?
> >
> > [PATCH v2 0/3] git-p4: Improve branch support
> >
> > I think I sent v2 twice somehow, so please make sure you pick the
> > latest ;)
> > In these patches I add the possibility to use a "git-p4.branchList"
> > configuration to define the branches. The patch is still to be
> > approved because most people in the mailing list do not use branch
> > detection, but I use it daily and it is working in my side. Could
> > you please test it?
> 
> Can you send this patch again?  It looks like you had previously
> responded to the list only, so I never got this message, as I wasn't
> on the list at the time (I am now).  I only saw this because I was
> searching the archive for something else.  I searched for the patch,
> but the actual patch body doesn't seem to be in the archive.

I am also not in the mailing list, I just follow its RSS and try to follow up on the git-p4 related topics ;) That is the reason why you were not included in the reply.

But I have been a bit busy and did not see this email passing by. Sorry. Luckily I had a tab opened in this thread, which I looked at today wondering what it was about! :P

Please follow the link to the thread [1] and you can open each of the entries. Take care not to apply patch 1/3 as it may require you to clone everything again.

> Thanks,
> 
> Mike
> 

Regards, Vitor

[1] http://thread.gmane.org/gmane.comp.version-control.git/167998/focus=168000
Michael Horowitz· Dec 17, 2011, 05:52 UTC · re: Vitor Antunes · lore

Re: git-p4 issue

Vitor,

I know it has been a long time, but I finally tried the below, and it works for me. I was doing things wrong at first, but I think I finally understand what is going on...

First, the branchList is great, that is what I needed. Now I can completely ignore the p4 branches. It would be great if there was an option to not have it query p4 branches at all, because we have so many branches here it takes forever, and I am just using branchList. What I did for now was to use your branchUser option, and pass it a bogus username, like "XXXXXX". At least this way it returns fast, but would be nice to not have it try to query at all.

I think I understand now why all this is needed if you want it to detect branches on its own. Since in Perforce branches are just directories, there is no way to tell if a directory is a branch or not, and people could organize branches in an arbitrary directory structure. I was thinking you could look at sibling directories under the same parent, which may be common, but there is no way to know for sure, so better to be explicit.

The other odd issue is that if you do not add the new branch to branchList (or p4 branches if you are using them) before you do the git-p4 sync, then since it goes through change numbers in order, it will never go back again and catch the branch in a change it already synced. This is why I could never get it to work, because I never had it in place before I synced the relevant change. What I realized though is that I can always do a "git-p4 sync --detect-branches //depot/foo/bar/@all" again, even though I already have the most recent change, and it will re-detect any branches I missed (given I have since added them to branchList).

What I had been thinking though is that if it had already detected branches once, it could at least update the remote p4 branches it already knew about to point at the latest revision, just not try to detect new ones, and not rely on branchList. Of course you would still need branchList to detect any new branches.

Anyway, works a lot better than it did before.
Thanks,
Mike
On Fri, May 13, 2011 at 10:31 AM, Vitor Antunes <vitor.hda@gmail.com> wrote:
Show 53 quoted lines
> Hi Michael,
>
> Michael Horowitz <michael.horowitz <at> ieee.org> writes:
>
>>
>> Vitor,
>>
>> > Could you please search for the following set of patches in this
>> > mailing list?
>> >
>> > [PATCH v2 0/3] git-p4: Improve branch support
>> >
>> > I think I sent v2 twice somehow, so please make sure you pick the
>> > latest ;)
>> > In these patches I add the possibility to use a "git-p4.branchList"
>> > configuration to define the branches. The patch is still to be
>> > approved because most people in the mailing list do not use branch
>> > detection, but I use it daily and it is working in my side. Could
>> > you please test it?
>>
>> Can you send this patch again?  It looks like you had previously
>> responded to the list only, so I never got this message, as I wasn't
>> on the list at the time (I am now).  I only saw this because I was
>> searching the archive for something else.  I searched for the patch,
>> but the actual patch body doesn't seem to be in the archive.
>
> I am also not in the mailing list, I just follow its RSS and try to
> follow up on the git-p4 related topics ;) That is the reason why you
> were not included in the reply.
>
> But I have been a bit busy and did not see this email passing by. Sorry.
> Luckily I had a tab opened in this thread, which I looked at today
> wondering what it was about! :P
>
> Please follow the link to the thread [1] and you can open each of the
> entries. Take care not to apply patch 1/3 as it may require you to clone
> everything again.
>
>> Thanks,
>>
>> Mike
>>
>
> Regards,
> Vitor
>
> [1] http://thread.gmane.org/gmane.comp.version-control.git/167998/focus=168000
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

← back to recent threads