Yet another git perforce integration

Hi,

Integrations of Git with Perforce seem to be fashion these days, so I'd like to chip in and announce another one :)

I've written a Python script called git-p4 that allows (incremental) imports from Perforce as well as submitting changes back from git into a Perforce depot.

It is currently part of the fast-export repository that Chris Lee started at

        http://repo.or.cz/w/fast-export.git

I've been using it now for about a month for my everyday work (which requires Perforce), so it seems to work reasonably stable at least in my setup :)

The usage is pretty simple:

        git-p4 clone //depot/path/my/project

Syncing:

        git-p4 sync

or just

        git-p4 rebase

to sync and automatically rebase into the current branch (much like git-svn rebase :)

        git-p4 submit

is used to submit changes back into Perforce (use at your own risk! :)

The import itself uses git-fast-import, so it's very fast. There is also no need to have the imported Perforce projects in the Perforce client view as it just uses "p4 changes //depot/path" and "p4 print //depot/path/file#revision", which I find very convenient and still fast enough for use (since of course only changed files are printed).

It also doesn't require any additional meta-data. Instead every import commit has a line added to the log message that contains the Perforce path the changeset comes from as well as the change number. git-p4 sync/rebase "parses" this on the last commit in the "p4" git branch to find out where to continue importing for incremental imports.

What's still missing is a bit of cleanup. For example I'd like to put the import branch into refs/remotes instead of refs/heads, but I've had some problems with fast-import when trying that. Also the support for Perforce branches isn't quite working yet. I'd ideally like to for example map a structure like

//depot/project/main //depot/project/branch1 //depot/project/branch2

into

remotes/p4/main remotes/p4/branch1 remotes/p4/branch2

in git and still support incremental commits. But that isn't working yet ;(

Also I've never tried it on Windows and I expect problems as the script uses pipes, calls "patch", etc.

Nevertheless I hope that this tool may be interesting for others as well. Maybe there's a possibility of including it in git/contrib/fast-import?

Simon

On 5/8/07, Simon Hausmann <simon@lst.de> wrote:

right, because you have to know where (and when) your perforce repo is, and then where and when your copy (in Git) is, copy the p4 changes, rebase your changes on top of the just copied changes (p4 history is linear), try to submit, watch out for fails, because someone else could have submitted something, possibly resync and repeat. Cursing and swearing notwithstanding. I simplified the operation by promising to hit everyone who submits anything in my domain (and was supported by the local P4 sysadms, so it worked).

I confirm that, but could not use the same approach in git-p4-import I posted earlier: I don't control the sync operation, and the stupid prog that does the sync does not allow in-client operation (p4 sync -n). I just have to work somehow with the junk it left in working directory

How do you handle that patchwork of mappings synced to diverse revisions that P4 clients tend to become? Don't you have to save change number or revision for _each_ file?

AFACS, it is impossible: Perforce does not have branches (in Git's meaning of the word). It only has directories. Integration (it is something like "in-repo-copy") metadata are just duct-taped on it (that stupid branchspec).

...and case-sensitivity. BTW, how does your script handle filenames with special characters and spaces in them?

On Tuesday 08 May 2007 11:47:31 Alex Riesen wrote:

Yeah, it's a bit tricky. I'm using "p4 where" to locate the p4 checkout and submitting back to perforce is an interactive process for me. The user is presented with the entire diff for each change as well as the log message in an editor and can also abort/continue in-between. In real-life usage I've found it to be fairly smooth though (no or only few conflits), but then I guess it depends also on the number of people working on the depot as a whole at the same time :)

I'm not sure I understand the question. I don't really use the p4 client view at all. I know the last imported p4 change number from the top commit in the git "p4" branch. Then I use "p4 changes //depot/path/@lastchangenum,#head" to get the list of changes that need to be imported. For each change I look at the list of affected files using "p4 describe" and feed each file using "p4 print" into git-fast-import. None of these import operations require anything from the client view/mapping.

Only when submitting changes back to Perforce the corresponding depot path must be checked out /somewhere/ (!= git repository), so that "p4 where //depot/path" reports it. But then I just offer the user to run "p4 sync" before applying the changes from git.

I guess the feasibility may very well depend a bit on the development model used with p4. I don't know if there are many models, but for example we have a fairly linear model where it's normal to be at the #head revision, don't use a whole lot of branches in Perforce. So p4 users are used to running "p4 resolve" and I personally got used to git conflict resolving through "git rebase" :)

True, it probably depends a bit on the depot organization. For example we have //depot/qt/main and then branches like //depot/qt/4.3 or research branches like //depot/qt/research/somecoolfeature . That's the kind of structure I'd like to map to git, and merges /could/ be detected from integrations if after a commit that has integrations between these "known" branches all revisions of all files are integrated. But that's obviously a very slow operation and I currently don't think it's worth it. But of course detecting the initial branch point is easy and fast and that is worth it (to record the branch creation, there's no merge of course).

I don't really treat them special. I get the file name from the output of "p4 describe" (using the Guido option :) and pass it on to git-fast-import.

Simon

On 5/8/07, Simon Hausmann <simon@lst.de> wrote:

Ah, you're _that_ lucky...

Recreating it anywhere, not just in Git, would be stupid. As it is in Perforce.

You seem to assume that the names never contain double quotes and backslashes. For example in the names of directories.