{"thread":{"id":"9899","subject":"State of Perforce importing.","startedAt":"2007-09-17T19:30:28Z","lastAt":"2007-09-20T06:12:52Z","messageCount":19,"participants":["David Brown","Simon Hausmann","Sam Vilain","Reece Dunn","Dmitry Kakurin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"53348","messageId":"20070917193027.GA24282@old.davidb.org","threadId":"9899","inReplyTo":null,"subject":"State of Perforce importing.","fromName":"David Brown","fromEmail":"git@davidb.org","sentAt":"2007-09-17T19:30:28Z","receivedAt":"2007-09-17T19:30:28Z","isPatch":false,"sender":{"key":"git@davidb.org","avatar":"https://gravatar.com/avatar/94c86a2938470a74c2eac5e2b69afc0871f79a660295c02219597aba8cb101c1?d=mp&s=160"},"body":"I'd like to track a lot of code living in a Perforce repository, so I've\nbeen playing with 'git-p4.py'. Is the one in the contrib/fast-import\ndirectory the latest version, or is there a better place.\n\nSo far, it is having a couple of problems:\n\n - The commit comment is empty. It doesn't seem to grab the Perforce\n description, and the user seems to be .\n\n - Every revision seems to check every file out of Perforce. This means\n that for the directory I want, every revision is going to take about 20\n minutes.\n\nBefore I start working on it, I'd like to make sure I'm working on the\nlatest code, though.\n\nThanks,\nDavid\n"},{"id":"53400","messageId":"200709180858.25188.simon@lst.de","threadId":"9899","inReplyTo":"20070917193027.GA24282@old.davidb.org","subject":"Re: State of Perforce importing.","fromName":"Simon Hausmann","fromEmail":"simon@lst.de","sentAt":"2007-09-18T06:58:21Z","receivedAt":"2007-09-18T06:58:21Z","isPatch":false,"sender":{"key":"hausmann@kde.org","avatar":"https://gravatar.com/avatar/bc9aad4fb31dce17eb66e690e7b51fe980c62da3c225c785da35dd806b8da778?d=mp&s=160"},"body":"On Monday 17 September 2007 21:30:28 David Brown wrote:\n> I'd like to track a lot of code living in a Perforce repository, so I've\n> been playing with 'git-p4.py'. Is the one in the contrib/fast-import\n> directory the latest version, or is there a better place.\n\nThis is indeed the latest version (on contrib/fast-import).\n\n> So far, it is having a couple of problems:\n>\n> - The commit comment is empty. It doesn't seem to grab the Perforce\n> description, and the user seems to be .\n\nThis may be a problem with the python output of perforce. Can you run the \nfollowing command?\n\n\tgit-p4 debug change \n\nThat should print a dictionary that has a 'desc' field containing the commit \ncomment/log and a 'user' field that has the perforce user name.\n\n> - Every revision seems to check every file out of Perforce. This means\n> that for the directory I want, every revision is going to take about\n> 20 minutes.\n\nFor every revision only every _changed_ file is retrieved (using p4 \nprint //path/file#revision).\n\n\nSimon\n"},{"id":"53402","messageId":"46EF7DD1.9090301@vilain.net","threadId":"9899","inReplyTo":"20070917193027.GA24282@old.davidb.org","subject":"Re: State of Perforce importing.","fromName":"Sam Vilain","fromEmail":"sam@vilain.net","sentAt":"2007-09-18T07:27:13Z","receivedAt":"2007-09-18T07:27:13Z","isPatch":false,"sender":{"key":"sam@vilain.net","avatar":"https://gravatar.com/avatar/8fc840ca854dbf6f7065b4335e3b934951c1dca3b11db688e95e471901f8f4a8?d=mp&s=160"},"body":"David Brown wrote:\n> I'd like to track a lot of code living in a Perforce repository, so I've\n> been playing with 'git-p4.py'. Is the one in the contrib/fast-import\n> directory the latest version, or is there a better place.\n\nI'm pretty close to giving a newer one a spin, that actually imports\nfrom the raw perforce back-end files without needing the perforce\nserver. I am hoping that this should give a very clean import and will\nbe very fast and efficient, sending files that share ancestry to gfi in\nsequence so that the on-the-fly delta system works.\n\nIf you're interested, take a look at\nhttp://utsl.gen.nz/gitweb/?p=git-p4raw;a=summary. Expect the commands\nthat say \"WIP\" to be rebased :-). It requires Postgres - I haven't yet\nre-written the SQL queries that step outside of MySQL's little box.\n\nIt could possibly be adapted to use the p4 client (though I'd expect\nthat to be relatively slow per-revision), and possibly be extended to be\nbidirectional as all of the upstream change number information is\nrecorded, a la git-svn.\n\nSam.\n"},{"id":"53490","messageId":"20070918154918.GA19106@old.davidb.org","threadId":"9899","inReplyTo":"46EF7DD1.9090301@vilain.net","subject":"Re: State of Perforce importing.","fromName":"David Brown","fromEmail":"git@davidb.org","sentAt":"2007-09-18T15:49:18Z","receivedAt":"2007-09-18T15:49:18Z","isPatch":false,"sender":{"key":"git@davidb.org","avatar":"https://gravatar.com/avatar/94c86a2938470a74c2eac5e2b69afc0871f79a660295c02219597aba8cb101c1?d=mp&s=160"},"body":"On Tue, Sep 18, 2007 at 07:27:13PM +1200, Sam Vilain wrote:\n\n>I'm pretty close to giving a newer one a spin, that actually imports\n>from the raw perforce back-end files without needing the perforce\n>server. I am hoping that this should give a very clean import and will\n>be very fast and efficient, sending files that share ancestry to gfi in\n>sequence so that the on-the-fly delta system works.\n\nUnfortunately, this isn't something I'm going to be able to use. The\nPerforce server will remain live, and resides on a machine I don't have\naccess to.\n\n>It could possibly be adapted to use the p4 client (though I'd expect\n>that to be relatively slow per-revision), and possibly be extended to be\n>bidirectional as all of the upstream change number information is\n>recorded, a la git-svn.\n\nI was able to get 'git-p4' to work a lot better by using @all, but it still\nhas some problems, at least bad interactions with P4.\n\n - It doesn't use any client spec. Our P4 server space is a complete\n mismash and has to be fixed up to get a sane directory layout. For\n example, some revisions have hundred-MB tar files sitting in the root\n directory and I don't want that in the repo. I also need to exclude\n directories, and in some cases completely rearrange the directory\n layout.\n\n - Our P4 server is set to be case insensitive. 'git-p4' ignores paths\n that come back from the server that are specified using a different\n case. Unfortunately, this means that a handful of files just get\n randomly dropped from each revision.\n\n I tried importing a client path instead of a depot path, but the names\n that come back from 'p4 files' are depot based so none ever match. I\n end up with a nice revision history of entirely empty trees.\n\nI'm probably going to end up writing an importer that uses an actual client\nworkspace to let Perforce do the client mapping. I'm also going to have to\nput some work into some code to clean up the log messages, since most of\nour changes have as a first line \"New Features:\", which makes for a rather\nuninformative shortlog.\n\nBut, I did learn about 'p4 -G' from git-p4 so that will help in getting\ninformation from the repository.\n\nThanks,\nDavid\n"},{"id":"53497","messageId":"3f4fd2640709181053t70b7abcdi2c4eaf67e7b75338@mail.gmail.com","threadId":"9899","inReplyTo":"20070918154918.GA19106@old.davidb.org","subject":"Re: State of Perforce importing.","fromName":"Reece Dunn","fromEmail":"msclrhd@googlemail.com","sentAt":"2007-09-18T17:53:45Z","receivedAt":"2007-09-18T17:53:45Z","isPatch":false,"sender":{"key":"msclrhd@googlemail.com","avatar":null},"body":"On 18/09/2007, David Brown wrote:\n> On Tue, Sep 18, 2007 at 07:27:13PM +1200, Sam Vilain wrote:\n>\n> >I'm pretty close to giving a newer one a spin, that actually imports\n> >from the raw perforce back-end files without needing the perforce\n> >server. I am hoping that this should give a very clean import and will\n> >be very fast and efficient, sending files that share ancestry to gfi in\n> >sequence so that the on-the-fly delta system works.\n>\n> Unfortunately, this isn't something I'm going to be able to use. The\n> Perforce server will remain live, and resides on a machine I don't have\n> access to.\n\nI use git-p4 in the same way. The best approach would be to have both\ntools and to use whichever one best matches your needs.\n\n> >It could possibly be adapted to use the p4 client (though I'd expect\n> >that to be relatively slow per-revision), and possibly be extended to be\n> >bidirectional as all of the upstream change number information is\n> >recorded, a la git-svn.\n>\n> I was able to get 'git-p4' to work a lot better by using @all, but it still\n> has some problems, at least bad interactions with P4.\n\nI have also seen this.\n\n> - It doesn't use any client spec. Our P4 server space is a complete\n> mismash and has to be fixed up to get a sane directory layout. For\n> example, some revisions have hundred-MB tar files sitting in the root\n> directory and I don't want that in the repo. I also need to exclude\n> directories, and in some cases completely rearrange the directory\n> layout.\n\nThe directory exclusion you could do the other way, if git-p4\nsupported multiple directory paths.\n\nThe main issues with using client workspaces is that they require you\nto use `p4 sync`, whereas git-p4 uses `p4 print` and that they may\nchange as the repository changes, but Perforce does not track these\nchanges.\n\nThat said, something like what workspaces are doing (allowing you to\nspecify multiple paths and where they are to go) would be useful.\n\n> - Our P4 server is set to be case insensitive. 'git-p4' ignores paths\n> that come back from the server that are specified using a different\n> case. Unfortunately, this means that a handful of files just get\n> randomly dropped from each revision.\n\nIt is worse than this. If you have:\n\n p4 integrate foo Foo\n p4 delete foo\n\nthen git-p4 will completely remove the file foo from the repository! I\nreported this a while back, but did not get a reply.\n\nThis is something I want to fix, as doing an '@all', rebase or sync is\nbroken in this case when dealing with renamed files as above when\nimporting a repository on a case insensitive system.\n\nThe alternative is to do the importing from Perforce on a Linux\nmachine and then clone/pull/rebase from it on a Windows machine.\n\n> I tried importing a client path instead of a depot path, but the names\n> that come back from 'p4 files' are depot based so none ever match. I\n> end up with a nice revision history of entirely empty trees.\n\nIdeally, git-p4 should bail out here with an error about the path\nneeding to be specified as a depot path.\n\n> I'm probably going to end up writing an importer that uses an actual client\n> workspace to let Perforce do the client mapping.\n\nAs eluded to above, I want to extend git-p4 to support a\nworkspace-like importer map file to use instead of a specific depot\npath (with a single depot path being supported as well for backward\ncompatibility and simplicity if the repository you are importing from\nhas a simple layout).\n\nThere is no need to create yet another Perforce importing tool, git-p4\nworks well in most cases. If we focus on improving git-p4, extending\nit to support the functionality mentioned here, fix the issues that\nthere are with it, then that will be more beneficial to the community\nas they will not have to learn another tool with a different set of\nbugs and issues.\n\n> I'm also going to have to\n> put some work into some code to clean up the log messages, since most of\n> our changes have as a first line \"New Features:\", which makes for a rather\n> uninformative shortlog.\n\nI would not do that. It is a good idea to keep the original log\nmessages, even if it does make for an uninformative shortlog. Look at\nsome of the CVS/SVN imported logs!\n\n- Reece\n"},{"id":"53528","messageId":"20070918231921.GA17652@old.davidb.org","threadId":"9899","inReplyTo":"3f4fd2640709181053t70b7abcdi2c4eaf67e7b75338@mail.gmail.com","subject":"Re: State of Perforce importing.","fromName":"David Brown","fromEmail":"git@davidb.org","sentAt":"2007-09-18T23:19:21Z","receivedAt":"2007-09-18T23:19:21Z","isPatch":false,"sender":{"key":"git@davidb.org","avatar":"https://gravatar.com/avatar/94c86a2938470a74c2eac5e2b69afc0871f79a660295c02219597aba8cb101c1?d=mp&s=160"},"body":"On Tue, Sep 18, 2007 at 06:53:45PM +0100, Reece Dunn wrote:\n\n>The main issues with using client workspaces is that they require you\n>to use `p4 sync`, whereas git-p4 uses `p4 print` and that they may\n>change as the repository changes, but Perforce does not track these\n>changes.\n\nUnfortunately, we have one project that heavily abuses P4 client specs.\nFor every release, someone creates a >900 line client spec and labels the\nfiles in it. Those are the versions that need to get checked in, and\nwithout rewriting much of what P4 does, I'm going to have to let P4 do the\nsyncing and checking out.\n\n>I would not do that. It is a good idea to keep the original log\n>messages, even if it does make for an uninformative shortlog. Look at\n>some of the CVS/SVN imported logs!\n\nI think what I want then is something to filter between 'git log' and 'git\nshortlog' that would find a summary line in the commit message and copy it\nto the top. It wouldn't change the history, but clean it up for shortlog's\npurpose.\n\nDavid\n"},{"id":"53530","messageId":"20070918233749.GA19533@old.davidb.org","threadId":"9899","inReplyTo":"20070917193027.GA24282@old.davidb.org","subject":"Re: State of Perforce importing.","fromName":"David Brown","fromEmail":"git@davidb.org","sentAt":"2007-09-18T23:37:49Z","receivedAt":"2007-09-18T23:37:49Z","isPatch":false,"sender":{"key":"git@davidb.org","avatar":"https://gravatar.com/avatar/94c86a2938470a74c2eac5e2b69afc0871f79a660295c02219597aba8cb101c1?d=mp&s=160"},"body":"On Mon, Sep 17, 2007 at 12:30:28PM -0700, David Brown wrote:\n> I'd like to track a lot of code living in a Perforce repository, so I've\n> been playing with 'git-p4.py'. Is the one in the contrib/fast-import\n> directory the latest version, or is there a better place.\n>\n> So far, it is having a couple of problems:\n>\n> - The commit comment is empty. It doesn't seem to grab the Perforce\n> description, and the user seems to be .\n>\n> - Every revision seems to check every file out of Perforce. This means\n> that for the directory I want, every revision is going to take about 20\n> minutes.\n\nAn additional problem:\n\n - git-p4 doesn't preserve the execute permission bit from Perforce.\n\nDavid\n"},{"id":"53534","messageId":"46F06B5C.2050207@vilain.net","threadId":"9899","inReplyTo":"20070918231921.GA17652@old.davidb.org","subject":"Re: State of Perforce importing.","fromName":"Sam Vilain","fromEmail":"sam@vilain.net","sentAt":"2007-09-19T00:20:44Z","receivedAt":"2007-09-19T00:20:44Z","isPatch":false,"sender":{"key":"sam@vilain.net","avatar":"https://gravatar.com/avatar/8fc840ca854dbf6f7065b4335e3b934951c1dca3b11db688e95e471901f8f4a8?d=mp&s=160"},"body":"David Brown wrote:\n> On Tue, Sep 18, 2007 at 06:53:45PM +0100, Reece Dunn wrote:\n> \n>> The main issues with using client workspaces is that they require you\n>> to use `p4 sync`, whereas git-p4 uses `p4 print` and that they may\n>> change as the repository changes, but Perforce does not track these\n>> changes.\n> \n> Unfortunately, we have one project that heavily abuses P4 client specs.\n> For every release, someone creates a >900 line client spec and labels the\n> files in it. Those are the versions that need to get checked in, and\n> without rewriting much of what P4 does, I'm going to have to let P4 do the\n> syncing and checking out.\n\nIf you can get a hold of the \"checkpoint\" and \"journal\" files, you could\nprobably throw the client spec data into a few Pg tables, chuck a couple\nof constraints on it to confirm that it works the way you thought, and\nthen get the information on what's where using a SQL query. The file\nimages themselves can come from wherever, it doesn't really matter\nbecause there are MD5 hashes in the data tables you can use to confirm\nyou got the right file.\n\nI haven't looked at importing the client spec because it's not important\nfor the project I'm importing. But I'd be happy to provide pointers.\n\n>> I would not do that. It is a good idea to keep the original log\n>> messages, even if it does make for an uninformative shortlog. Look at\n>> some of the CVS/SVN imported logs!\n> I think what I want then is something to filter between 'git log' and 'git\n> shortlog' that would find a summary line in the commit message and copy it\n> to the top. It wouldn't change the history, but clean it up for shortlog's\n> purpose.\n\nSounds like a job for a templating git-log porcelain.\n\nSam.\n"},{"id":"53535","messageId":"46F06C0C.8090201@vilain.net","threadId":"9899","inReplyTo":"20070918233749.GA19533@old.davidb.org","subject":"Re: State of Perforce importing.","fromName":"Sam Vilain","fromEmail":"sam@vilain.net","sentAt":"2007-09-19T00:23:40Z","receivedAt":"2007-09-19T00:23:40Z","isPatch":false,"sender":{"key":"sam@vilain.net","avatar":"https://gravatar.com/avatar/8fc840ca854dbf6f7065b4335e3b934951c1dca3b11db688e95e471901f8f4a8?d=mp&s=160"},"body":"David Brown wrote:\n> \n> An additional problem:\n> \n> - git-p4 doesn't preserve the execute permission bit from Perforce.\n\nFWIW I found that bit on bit 9 of the 'file type' flag in the db, which\nis the third column in the \"db.rev\" table. It's used to come up with\nthe silly names like \"text\" vs \"xtext\" (difference? well, one's\nexecutable of course).\n\nSam.\n"},{"id":"53536","messageId":"46F06C0F.3040609@vilain.net","threadId":"9899","inReplyTo":"3f4fd2640709181053t70b7abcdi2c4eaf67e7b75338@mail.gmail.com","subject":"Re: State of Perforce importing.","fromName":"Sam Vilain","fromEmail":"sam@vilain.net","sentAt":"2007-09-19T00:23:43Z","receivedAt":"2007-09-19T00:23:43Z","isPatch":false,"sender":{"key":"sam@vilain.net","avatar":"https://gravatar.com/avatar/8fc840ca854dbf6f7065b4335e3b934951c1dca3b11db688e95e471901f8f4a8?d=mp&s=160"},"body":"Reece Dunn wrote:\n> There is no need to create yet another Perforce importing tool, git-p4\n> works well in most cases. If we focus on improving git-p4, extending\n> it to support the functionality mentioned here, fix the issues that\n> there are with it, then that will be more beneficial to the community\n> as they will not have to learn another tool with a different set of\n> bugs and issues.\n\nI like my approach; it's clean and I think shows a tasteful level of\ndistrust towards the sanity and integrity of the data held by Perforce.\n Actually it really helped me understand what was really going on;\nbecause the information as displayed by for instance \"p4 integrate\" is\na lot more confusing than the underlying tables (IMHO).\n\nSam.\n"},{"id":"53537","messageId":"20070919002617.GA22187@old.davidb.org","threadId":"9899","inReplyTo":"46F06B5C.2050207@vilain.net","subject":"Re: State of Perforce importing.","fromName":"David Brown","fromEmail":"git@davidb.org","sentAt":"2007-09-19T00:26:17Z","receivedAt":"2007-09-19T00:26:17Z","isPatch":false,"sender":{"key":"git@davidb.org","avatar":"https://gravatar.com/avatar/94c86a2938470a74c2eac5e2b69afc0871f79a660295c02219597aba8cb101c1?d=mp&s=160"},"body":"On Wed, Sep 19, 2007 at 12:20:44PM +1200, Sam Vilain wrote:\n\n>If you can get a hold of the \"checkpoint\" and \"journal\" files, you could\n>probably throw the client spec data into a few Pg tables, chuck a couple\n>of constraints on it to confirm that it works the way you thought, and\n>then get the information on what's where using a SQL query. The file\n>images themselves can come from wherever, it doesn't really matter\n>because there are MD5 hashes in the data tables you can use to confirm\n>you got the right file.\n\nIn my instance, I don't have an account on the P4 server, so I'm going to\nhave to deal with this through P4's normal client.\n\nI don't have much confidence that P4 really knows what it is doing when it\ncomes to integrate, so I'm not much worried about branches. But, I do want\nto accurately get the history of a particular branch.\n\nSo far, the only thing that isn't working is the execute bit doesn't get\nset on files that should have it.\n\nDave\n"},{"id":"53538","messageId":"20070919002722.GB22187@old.davidb.org","threadId":"9899","inReplyTo":"46F06C0C.8090201@vilain.net","subject":"Re: State of Perforce importing.","fromName":"David Brown","fromEmail":"git@davidb.org","sentAt":"2007-09-19T00:27:22Z","receivedAt":"2007-09-19T00:27:22Z","isPatch":false,"sender":{"key":"git@davidb.org","avatar":"https://gravatar.com/avatar/94c86a2938470a74c2eac5e2b69afc0871f79a660295c02219597aba8cb101c1?d=mp&s=160"},"body":"On Wed, Sep 19, 2007 at 12:23:40PM +1200, Sam Vilain wrote:\n>David Brown wrote:\n>> \n>> An additional problem:\n>> \n>> - git-p4 doesn't preserve the execute permission bit from Perforce.\n>\n>FWIW I found that bit on bit 9 of the 'file type' flag in the db, which\n>is the third column in the \"db.rev\" table. It's used to come up with\n>the silly names like \"text\" vs \"xtext\" (difference? well, one's\n>executable of course).\n\nIt does come back in the 'kind' field when it asks the client for the file\ntype. I'll look into using that information to set the execute bit in the\nmode it sends off.\n\nDave\n"},{"id":"53551","messageId":"200709190819.12188.simon@lst.de","threadId":"9899","inReplyTo":"20070918233749.GA19533@old.davidb.org","subject":"Re: State of Perforce importing.","fromName":"Simon Hausmann","fromEmail":"simon@lst.de","sentAt":"2007-09-19T06:19:11Z","receivedAt":"2007-09-19T06:19:11Z","isPatch":false,"sender":{"key":"hausmann@kde.org","avatar":"https://gravatar.com/avatar/bc9aad4fb31dce17eb66e690e7b51fe980c62da3c225c785da35dd806b8da778?d=mp&s=160"},"body":"On Wednesday 19 September 2007 01:37:49 David Brown wrote:\n> On Mon, Sep 17, 2007 at 12:30:28PM -0700, David Brown wrote:\n> > I'd like to track a lot of code living in a Perforce repository, so I've\n> > been playing with 'git-p4.py'. Is the one in the contrib/fast-import\n> > directory the latest version, or is there a better place.\n> >\n> > So far, it is having a couple of problems:\n> >\n> > - The commit comment is empty. It doesn't seem to grab the Perforce\n> > description, and the user seems to be .\n> >\n> > - Every revision seems to check every file out of Perforce. This means\n> > that for the directory I want, every revision is going to take about\n> > 20 minutes.\n>\n> An additional problem:\n>\n> - git-p4 doesn't preserve the execute permission bit from Perforce.\n\nHmm, can you paste the output of\n\n\tp4 fstat //path/in/depot/to/file/that/is/imported/incorrectly\n\n? I'm interested in the type of the file that p4 reports.\n\nFWIW it works for me ;-)\n\nThanks,\nSimon\n"},{"id":"53585","messageId":"20070919171243.GA23902@old.davidb.org","threadId":"9899","inReplyTo":"200709190819.12188.simon@lst.de","subject":"Re: State of Perforce importing.","fromName":"David Brown","fromEmail":"git@davidb.org","sentAt":"2007-09-19T17:12:43Z","receivedAt":"2007-09-19T17:12:43Z","isPatch":false,"sender":{"key":"git@davidb.org","avatar":"https://gravatar.com/avatar/94c86a2938470a74c2eac5e2b69afc0871f79a660295c02219597aba8cb101c1?d=mp&s=160"},"body":"On Wed, Sep 19, 2007 at 08:19:11AM +0200, Simon Hausmann wrote:\n\n>> An additional problem:\n>>\n>> - git-p4 doesn't preserve the execute permission bit from Perforce.\n>\n>Hmm, can you paste the output of\n>\n>\tp4 fstat //path/in/depot/to/file/that/is/imported/incorrectly\n>\n>? I'm interested in the type of the file that p4 reports.\n\n headType kxtext\n\nso the problem is that the git-p4 is only looking for an 'x' at the start.\nAccording to 'p4 help filetypes', we need to use execute for any of:\n\n cxtext, kxtext, uxbinary, and the others that start with 'x'.\n\nI think it would be sufficient to check the first or second character for\nan 'x'. I'll make a change and give it a try later today.\n\nDavid\n"},{"id":"53589","messageId":"3f4fd2640709191123j64b53878vc96d785c13c3bca2@mail.gmail.com","threadId":"9899","inReplyTo":"20070919171243.GA23902@old.davidb.org","subject":"Re: State of Perforce importing.","fromName":"Reece Dunn","fromEmail":"msclrhd@googlemail.com","sentAt":"2007-09-19T18:23:41Z","receivedAt":"2007-09-19T18:23:41Z","isPatch":false,"sender":{"key":"msclrhd@googlemail.com","avatar":null},"body":"On 19/09/2007, David Brown wrote:\n> On Wed, Sep 19, 2007 at 08:19:11AM +0200, Simon Hausmann wrote:\n>\n> >> An additional problem:\n> >>\n> >> - git-p4 doesn't preserve the execute permission bit from Perforce.\n> >\n> >Hmm, can you paste the output of\n> >\n> > p4 fstat //path/in/depot/to/file/that/is/imported/incorrectly\n> >\n> >? I'm interested in the type of the file that p4 reports.\n>\n> headType kxtext\n>\n> so the problem is that the git-p4 is only looking for an 'x' at the start.\n> According to 'p4 help filetypes', we need to use execute for any of:\n>\n> cxtext, kxtext, uxbinary, and the others that start with 'x'.\n>\n> I think it would be sufficient to check the first or second character for\n> an 'x'. I'll make a change and give it a try later today.\n\nThese are the old file types. If you read the output of `p4 help\nfiletypes`, the new way of specifying this is with file type\nmodifiers. Therefore, you also have things like text+x.\n\n- Reece\n"},{"id":"53590","messageId":"20070919182545.GA2266@old.davidb.org","threadId":"9899","inReplyTo":"3f4fd2640709191123j64b53878vc96d785c13c3bca2@mail.gmail.com","subject":"Re: State of Perforce importing.","fromName":"David Brown","fromEmail":"git@davidb.org","sentAt":"2007-09-19T18:25:45Z","receivedAt":"2007-09-19T18:25:45Z","isPatch":false,"sender":{"key":"git@davidb.org","avatar":"https://gravatar.com/avatar/94c86a2938470a74c2eac5e2b69afc0871f79a660295c02219597aba8cb101c1?d=mp&s=160"},"body":"On Wed, Sep 19, 2007 at 07:23:41PM +0100, Reece Dunn wrote:\n\n>> I think it would be sufficient to check the first or second character for\n>> an 'x'. I'll make a change and give it a try later today.\n>\n>These are the old file types. If you read the output of `p4 help\n>filetypes`, the new way of specifying this is with file type\n>modifiers. Therefore, you also have things like text+x.\n\nSo my patch I just sent may not be sufficient. Thing is, we set the file\ntype as 'text+x' and it comes back as xtext, so I'm not sure if P4 ever\ngives out text+x or if that is just available as a new way of specifying\nthem.\n\nDavid\n"},{"id":"53593","messageId":"3f4fd2640709191156q18d8eb1cg609f0ad209cf8144@mail.gmail.com","threadId":"9899","inReplyTo":"20070919182545.GA2266@old.davidb.org","subject":"Re: State of Perforce importing.","fromName":"Reece Dunn","fromEmail":"msclrhd@googlemail.com","sentAt":"2007-09-19T18:56:50Z","receivedAt":"2007-09-19T18:56:50Z","isPatch":false,"sender":{"key":"msclrhd@googlemail.com","avatar":null},"body":"On 19/09/2007, David Brown wrote:\n> On Wed, Sep 19, 2007 at 07:23:41PM +0100, Reece Dunn wrote:\n>\n> >> I think it would be sufficient to check the first or second character for\n> >> an 'x'. I'll make a change and give it a try later today.\n> >\n> >These are the old file types. If you read the output of `p4 help\n> >filetypes`, the new way of specifying this is with file type\n> >modifiers. Therefore, you also have things like text+x.\n>\n> So my patch I just sent may not be sufficient. Thing is, we set the file\n> type as 'text+x' and it comes back as xtext, so I'm not sure if P4 ever\n> gives out text+x or if that is just available as a new way of specifying\n> them.\n\nI'm not sure. The Perforce help says that xtext and its variants are\nthere for backward compatibility. If you are running an older server\nwith a new client, or the other way around, they may be doing a map\nfrom text+x to xtext so that the old version can work properly. This\nis just speculation, though.\n\nI don't know enough to say what Perforce is doing. I find it strange\nthat it is reporting xtext, when you specified text+x.\n\nHave you tried a combination that is not supported? Is xunicode\nsupported, in which case you could try unicode+x (if you have a file\nthat you can experiment with)?\n\n- Reece\n"},{"id":"53608","messageId":"3f4fd2640709191420w1245449cwb9646b6c6e0e1c82@mail.gmail.com","threadId":"9899","inReplyTo":"46F06C0F.3040609@vilain.net","subject":"Re: State of Perforce importing.","fromName":"Reece Dunn","fromEmail":"msclrhd@googlemail.com","sentAt":"2007-09-19T21:20:10Z","receivedAt":"2007-09-19T21:20:10Z","isPatch":false,"sender":{"key":"msclrhd@googlemail.com","avatar":null},"body":"On 19/09/2007, Sam Vilain wrote:\n> Reece Dunn wrote:\n> > There is no need to create yet another Perforce importing tool, git-p4\n> > works well in most cases. If we focus on improving git-p4, extending\n> > it to support the functionality mentioned here, fix the issues that\n> > there are with it, then that will be more beneficial to the community\n> > as they will not have to learn another tool with a different set of\n> > bugs and issues.\n>\n> I like my approach; it's clean and I think shows a tasteful level of\n> distrust towards the sanity and integrity of the data held by Perforce.\n> Actually it really helped me understand what was really going on;\n> because the information as displayed by for instance \"p4 integrate\" is\n> a lot more confusing than the underlying tables (IMHO).\n\nI agree. What I wasn't clear about in that paragraph, but had eluded\nto in other comments in that email, is that having both git-p4 and\ngit-p4raw is a good thing as they operate on two differing use cases.\nWhat I was referring to there is to have another equivalent of git-p4\nthat interfaced using the p4 client.\n\n- Reece\n"},{"id":"53627","messageId":"5BC36977390A4E61B826613630DF0BBC@ntdev.corp.microsoft.com","threadId":"9899","inReplyTo":"20070918154918.GA19106@old.davidb.org","subject":"Re: State of Perforce importing.","fromName":"Dmitry Kakurin","fromEmail":"dmitry.kakurin@gmail.com","sentAt":"2007-09-20T06:12:52Z","receivedAt":"2007-09-20T06:12:52Z","isPatch":false,"sender":{"key":"dmitry.kakurin@gmail.com","avatar":null},"body":"I had to import quite a big Perforce depot too. And after some struggle it \nwent fine.\nFor case mismatch: \nhttp://kb.perforce.com/AdminTasks/SuperuserTasks/CrossPlatfor..erMigration, \nbullet 10. It requires server access.\nFor a@b and for excluding one p4 path during migration you could use my \nquick-and-dirty fix (attached).\nYou can easily extend it to include and exclude multiple paths. Then it \nbecomes as flexible as p4 client mapping.\n\n- Dmitry\n----- Original Message ----- \nFrom: \"David Brown\" \nNewsgroups: gmane.comp.version-control.git\nTo: \"Sam Vilain\" \nCc: \"Git\" \nSent: Tuesday, 18 September 2007 8:49\nSubject: Re: State of Perforce importing.\n\n\n> On Tue, Sep 18, 2007 at 07:27:13PM +1200, Sam Vilain wrote:\n>\n>>I'm pretty close to giving a newer one a spin, that actually imports\n>>from the raw perforce back-end files without needing the perforce\n>>server. I am hoping that this should give a very clean import and will\n>>be very fast and efficient, sending files that share ancestry to gfi in\n>>sequence so that the on-the-fly delta system works.\n>\n> Unfortunately, this isn't something I'm going to be able to use. The\n> Perforce server will remain live, and resides on a machine I don't have\n> access to.\n>\n>>It could possibly be adapted to use the p4 client (though I'd expect\n>>that to be relatively slow per-revision), and possibly be extended to be\n>>bidirectional as all of the upstream change number information is\n>>recorded, a la git-svn.\n>\n> I was able to get 'git-p4' to work a lot better by using @all, but it \n> still\n> has some problems, at least bad interactions with P4.\n>\n> - It doesn't use any client spec. Our P4 server space is a complete\n> mismash and has to be fixed up to get a sane directory layout. For\n> example, some revisions have hundred-MB tar files sitting in the root\n> directory and I don't want that in the repo. I also need to exclude\n> directories, and in some cases completely rearrange the directory\n> layout.\n>\n> - Our P4 server is set to be case insensitive. 'git-p4' ignores paths\n> that come back from the server that are specified using a different\n> case. Unfortunately, this means that a handful of files just get\n> randomly dropped from each revision.\n>\n> I tried importing a client path instead of a depot path, but the names\n> that come back from 'p4 files' are depot based so none ever match. I\n> end up with a nice revision history of entirely empty trees.\n>\n> I'm probably going to end up writing an importer that uses an actual \n> client\n> workspace to let Perforce do the client mapping. I'm also going to have \n> to\n> put some work into some code to clean up the log messages, since most of\n> our changes have as a first line \"New Features:\", which makes for a rather\n> uninformative shortlog.\n>\n> But, I did learn about 'p4 -G' from git-p4 so that will help in getting\n> information from the repository.\n>\n> Thanks,\n> David \n\n\n>From a089b02239c3bc310956964d61075b093d26549f Mon Sep 17 00:00:00 2001\nFrom: Dmitry Kakurin \nDate: Sun, 9 Sep 2007 13:58:12 -0700\nSubject: [PATCH] git-p4: Added --exclude option and use P4 client name in commits\n\n---\n contrib/fast-import/git-p4 | 19 +++++++++++++++----\n 1 files changed, 15 insertions(+), 4 deletions(-)\n\ndiff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4\nindex adaaae6..337854f 100755\n--- a/contrib/fast-import/git-p4\n+++ b/contrib/fast-import/git-p4\n@@ -769,6 +769,7 @@ class P4Sync(Command):\n self.keepRepoPath = False\n self.depotPaths = None\n self.p4BranchesInGit = []\n+ self.cloneExclude = None\n \n if gitConfig(\"git-p4.syncFromOrigin\") == \"false\":\n self.syncWithOrigin = False\n@@ -779,8 +780,12 @@ class P4Sync(Command):\n while commit.has_key(\"depotFile%s\" % fnum):\n path = commit[\"depotFile%s\" % fnum]\n \n- found = [p for p in self.depotPaths\n- if path.startswith (p)]\n+ if self.cloneExclude and path.startswith( self.cloneExclude ):\n+ found = False\n+ else:\n+ found = [p for p in self.depotPaths\n+ if path.startswith (p)]\n+\n if not found:\n fnum = fnum + 1\n continue\n@@ -905,6 +910,8 @@ class P4Sync(Command):\n else:\n committer = \"%s %s %s\" % (author, epoch, self.tz)\n \n+ committer = \"%s <%s@%s> %s %s\" % (author, details[\"user\"], details[\"client\"], epoch, self.tz)\n+\n self.gitStream.write(\"committer %s\\n\" % committer)\n \n self.gitStream.write(\"data <