threads / discuss / 37097

git p4 diff-tree ambiguous argument error

Subject: git p4 diff-tree ambiguous argument error

## tl;dr

7 messages between Jul 10, 2014 and Jul 12, 2014.

replies: 6people: 3as markdown or json

Bill Door· Jul 10, 2014, 17:30 UTC · lore

$ git p4 sync --detect-branches --import-labels //main@all ... Lots of useful information elided fatal: ambiguous argument 'git-p4-tmp/8031': unknown revision or path not in the working tree. Use '--' to separate paths from revisions, like this: 'git <command> [<revision>...] -- [<file>...]' Command failed: ['git', 'diff-tree', '6b3ef26a3e2635a5ff0170e15fdadb386672f8b9', 'git-p4-tmp/8031']

If I re-run the command, it works the second time. Of course there are 73000+ commits. This is gonna take a while.

I've done some debugging. It appears there is a timing problem between git-p4 and git.

The failure occurs in P4Sync.searchParent(). Even though a checkpoint is sent to git (for fast-import) just prior to the call to searchParent() in importChanges(), the file does not yet exist. I used pdb, paused the program just before the call to diff-tree and the file was missing. After the program exits due to the error the file exists (i.e. the OS flushed the file). This is why re-running continues to work, there is an "old" file with basically the same information laying around (dangerous).

How can I get git (fast-import) to flush the file at the right time?

$ git --version
git version 1.7.12.4
$ python --version
Python 2.6.6
OS: GNU/Linux 2.6.32-431.el6.x86_64

-- View this message in context: http://git.661346.n2.nabble.com/git-p4-diff-tree-ambiguous-argument-error-tp7614774.html Sent from the git mailing list archive at Nabble.com.

Luke Diamand· Jul 10, 2014, 18:07 UTC · re: Bill Door · lore

Re: git p4 diff-tree ambiguous argument error

Is this using NFS, or local storage?

On 10/07/14 18:30, Bill Door wrote:

Show 42 quoted lines

> $ git p4 sync --detect-branches --import-labels //main@all
> ... Lots of useful information elided
> fatal: ambiguous argument 'git-p4-tmp/8031': unknown revision or path not in
> the working tree.
> Use '--' to separate paths from revisions, like this:
> 'git <command> [<revision>...] -- [<file>...]'
> Command failed: ['git', 'diff-tree',
> '6b3ef26a3e2635a5ff0170e15fdadb386672f8b9', 'git-p4-tmp/8031']
>
> If I re-run the command, it works the second time. Of course there are
> 73000+ commits. This is gonna take a while.
>
> I've done some debugging. It appears there is a timing problem between
> git-p4 and git.
>
> The failure occurs in P4Sync.searchParent(). Even though a checkpoint is
> sent to git (for fast-import) just prior to the call to searchParent() in
> importChanges(), the file does not yet exist. I used pdb, paused the program
> just before the call to diff-tree and the file was missing. After the
> program exits due to the error the file exists (i.e. the OS flushed the
> file). This is why re-running continues to work, there is an "old" file with
> basically the same information laying around (dangerous).
>
> How can I get git (fast-import) to flush the file at the right time?
>
> $ git --version
> git version 1.7.12.4
> $ python --version
> Python 2.6.6
> OS: GNU/Linux 2.6.32-431.el6.x86_64
>
>
>
>
> --
> View this message in context: http://git.661346.n2.nabble.com/git-p4-diff-tree-ambiguous-argument-error-tp7614774.html
> Sent from the git mailing list archive at Nabble.com.
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Duane Murphy· Jul 10, 2014, 18:08 UTC · re: Luke Diamand · lore

Re: git p4 diff-tree ambiguous argument error

All local storage.

 …Duane

On Jul 10, 2014, at 11:07 AM, Luke Diamand <luke@diamand.org> wrote:

Show 48 quoted lines

> Is this using NFS, or local storage?
> 
> 
> 
> On 10/07/14 18:30, Bill Door wrote:
>> $ git p4 sync --detect-branches --import-labels //main@all
>> ... Lots of useful information elided
>> fatal: ambiguous argument 'git-p4-tmp/8031': unknown revision or path not in
>> the working tree.
>> Use '--' to separate paths from revisions, like this:
>> 'git <command> [<revision>...] -- [<file>...]'
>> Command failed: ['git', 'diff-tree',
>> '6b3ef26a3e2635a5ff0170e15fdadb386672f8b9', 'git-p4-tmp/8031']
>> 
>> If I re-run the command, it works the second time. Of course there are
>> 73000+ commits. This is gonna take a while.
>> 
>> I've done some debugging. It appears there is a timing problem between
>> git-p4 and git.
>> 
>> The failure occurs in P4Sync.searchParent(). Even though a checkpoint is
>> sent to git (for fast-import) just prior to the call to searchParent() in
>> importChanges(), the file does not yet exist. I used pdb, paused the program
>> just before the call to diff-tree and the file was missing. After the
>> program exits due to the error the file exists (i.e. the OS flushed the
>> file). This is why re-running continues to work, there is an "old" file with
>> basically the same information laying around (dangerous).
>> 
>> How can I get git (fast-import) to flush the file at the right time?
>> 
>> $ git --version
>> git version 1.7.12.4
>> $ python --version
>> Python 2.6.6
>> OS: GNU/Linux 2.6.32-431.el6.x86_64
>> 
>> 
>> 
>> 
>> --
>> View this message in context: http://git.661346.n2.nabble.com/git-p4-diff-tree-ambiguous-argument-error-tp7614774.html
>> Sent from the git mailing list archive at Nabble.com.
>> --
>> To unsubscribe from this list: send the line "unsubscribe git" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> 
>

Duane Murphy· Jul 10, 2014, 19:19 UTC · re: Duane Murphy · lore

Re: git p4 diff-tree ambiguous argument error

Some additional investigation.

I am working in a copy of a repository that was originally used to pull the data from Perforce. As part of my experiments to figure out this problem, I deleted the contents of .git/git-p4-tmp/.

I noticed that git-p4 would continue if those files were present. I have now copied the files that were in .git/git-p4-tmp/ from the other repository.

git-p4 is not crashing now, but I also noticed that none of the dates on these files have changed. These files should have been touched each time that a branch is taken, but these files have not changed while the sync is running.

That seems significant.

I expect git-p4 to crash again on a new commit that is not in .git/git-p4-tmp/. Then I have to start the 8-12 hour process over again (did I mention 70k commits?).

 …Duane

On Jul 10, 2014, at 11:08 AM, Duane Murphy <duanemurphy@mac.com> wrote:

Show 55 quoted lines

> All local storage.
> 
> …Duane
> 
> On Jul 10, 2014, at 11:07 AM, Luke Diamand <luke@diamand.org> wrote:
> 
>> Is this using NFS, or local storage?
>> 
>> 
>> 
>> On 10/07/14 18:30, Bill Door wrote:
>>> $ git p4 sync --detect-branches --import-labels //main@all
>>> ... Lots of useful information elided
>>> fatal: ambiguous argument 'git-p4-tmp/8031': unknown revision or path not in
>>> the working tree.
>>> Use '--' to separate paths from revisions, like this:
>>> 'git <command> [<revision>...] -- [<file>...]'
>>> Command failed: ['git', 'diff-tree',
>>> '6b3ef26a3e2635a5ff0170e15fdadb386672f8b9', 'git-p4-tmp/8031']
>>> 
>>> If I re-run the command, it works the second time. Of course there are
>>> 73000+ commits. This is gonna take a while.
>>> 
>>> I've done some debugging. It appears there is a timing problem between
>>> git-p4 and git.
>>> 
>>> The failure occurs in P4Sync.searchParent(). Even though a checkpoint is
>>> sent to git (for fast-import) just prior to the call to searchParent() in
>>> importChanges(), the file does not yet exist. I used pdb, paused the program
>>> just before the call to diff-tree and the file was missing. After the
>>> program exits due to the error the file exists (i.e. the OS flushed the
>>> file). This is why re-running continues to work, there is an "old" file with
>>> basically the same information laying around (dangerous).
>>> 
>>> How can I get git (fast-import) to flush the file at the right time?
>>> 
>>> $ git --version
>>> git version 1.7.12.4
>>> $ python --version
>>> Python 2.6.6
>>> OS: GNU/Linux 2.6.32-431.el6.x86_64
>>> 
>>> 
>>> 
>>> 
>>> --
>>> View this message in context: http://git.661346.n2.nabble.com/git-p4-diff-tree-ambiguous-argument-error-tp7614774.html
>>> Sent from the git mailing list archive at Nabble.com.
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe git" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> 
>> 
>

Pete Wyckoff· Jul 12, 2014, 18:10 UTC · re: Duane Murphy · lore

Re: git p4 diff-tree ambiguous argument error

duanemurphy@mac.com wrote on Thu, 10 Jul 2014 12:19 -0700:

Show 17 quoted lines

> Some additional investigation. 
> 
> I am working in a copy of a repository that was originally used to pull the data 
> from Perforce. As part of my experiments to figure out this problem, I deleted 
> the contents of .git/git-p4-tmp/. 
> 
> I noticed that git-p4 would continue if those files were present. I have now 
> copied the files that were in .git/git-p4-tmp/ from the other repository. 
> 
> git-p4 is not crashing now, but I also noticed that none of the dates on these files
> have changed. These files should have been touched each time that a branch is taken,
> but these files have not changed while the sync is running.
> 
> That seems significant. 
> 
> I expect git-p4 to crash again on a new commit that is not in .git/git-p4-tmp/. 
> Then I have to start the 8-12 hour process over again (did I mention 70k commits?).

Bizarre. That directory is really supposed to be temporary, and live only during a single git p4 invocation. It's just a bunch of branch heads for the temporary commits. I don't know why those branches, and the git-p4-tmp directory, hang around after you run git p4. Might be worth your investigation.

The second weirdness is why a new run doesn't create the branch. This maybe points to self.checkpoint() not really checkpointing. It does send a "checkpoint" down the git fast-import stream, which is supposed to make it write the branches out. You might consider grabbing the fast-import process in a debugger and see why it's not writing out the branch head.

There's lots of changes since v1.7.12.4, but nothing obvious I can see that would cause this. Sorry,

		-- Pete

Duane Murphy· Jul 12, 2014, 19:19 UTC · re: Pete Wyckoff · lore

Re: git p4 diff-tree ambiguous argument error

On Jul 12, 2014, at 11:10 AM, Pete Wyckoff <pw@padd.com> wrote:

Show 23 quoted lines

>> Some additional investigation. 
>> 
>> I am working in a copy of a repository that was originally used to pull the data 
>> from Perforce. As part of my experiments to figure out this problem, I deleted 
>> the contents of .git/git-p4-tmp/. 
>> 
>> I noticed that git-p4 would continue if those files were present. I have now 
>> copied the files that were in .git/git-p4-tmp/ from the other repository. 
>> 
>> git-p4 is not crashing now, but I also noticed that none of the dates on these files
>> have changed. These files should have been touched each time that a branch is taken,
>> but these files have not changed while the sync is running.
>> 
>> That seems significant. 
>> 
>> I expect git-p4 to crash again on a new commit that is not in .git/git-p4-tmp/. 
>> Then I have to start the 8-12 hour process over again (did I mention 70k commits?).
> 
> Bizarre.  That directory is really supposed to be temporary, and
> live only during a single git p4 invocation.  It's just a bunch
> of branch heads for the temporary commits.  I don't know why
> those branches, and the git-p4-tmp directory, hang around after
> you run git p4.  Might be worth your investigation.

The reason the files are still there is because git-p4 crashes (exits with an error) and doesn't clean them up.

When git-p4 exits cleanly, the directory is fine. My experience is that git-p4 exists abnormally often enough.

Show 6 quoted lines

> The second weirdness is why a new run doesn't create the branch.
> This maybe points to self.checkpoint() not really checkpointing.
> It does send a "checkpoint" down the git fast-import stream,
> which is supposed to make it write the branches out.  You might
> consider grabbing the fast-import process in a debugger and see
> why it's not writing out the branch head.

I started looking at the code for fast-import. There's a lot of code there, so didn't go down that path.

I've posted another message indicating that I sincerely believe the problem is in git-fast-import. My current working theory is that because the file matches another commit (that's the point of writing the commit) git-fast-import doesn't flush the file. I've clearly stopped git-p4 immediately after the checkpoint and there is no file. When the program exits (abnormally of course), the file appears (thanks to the file system). Seems like a flush problem somewhere.

I was hoping someone who knows git-fast-import would have a clue as to the problem.

> There's lots of changes since v1.7.12.4, but nothing obvious I
> can see that would cause this.  Sorry,

Thanks for taking a look.

Bill Door· Jul 11, 2014, 17:40 UTC · re: Bill Door · lore

Re: git p4 diff-tree ambiguous argument error

More data points. I have reproduced the problem on

$ git --version git version 1.8.5.2 (Apple Git-48) $ python --version Python 2.7.5 $ uname -a Darwin Kernel Version 13.3.0: Tue Jun 3 21:27:35 PDT 2014; root:xnu-2422.110.17~1/RELEASE_X86_64 x86_64

However, it the command is used on a new empty repository, there is no failure. Running a second time causes the failure.

Thanks for any help you can offer in tracking down this problem!

-- View this message in context: http://git.661346.n2.nabble.com/git-p4-diff-tree-ambiguous-argument-error-tp7614774p7614882.html Sent from the git mailing list archive at Nabble.com.