{"thread":{"id":"32587","subject":"[PATCH 0/3] fixup remaining cvsimport tests","startedAt":"2013-01-11T04:27:16Z","lastAt":"2013-01-24T03:15:50Z","messageCount":16,"participants":["Chris Rorvick","John Keeping","Junio C Hamano","Eric S. Raymond","Michael Haggerty"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"206491","messageId":"1357878439-27500-1-git-send-email-chris@rorvick.com","threadId":"32587","inReplyTo":null,"subject":"[PATCH 0/3] fixup remaining cvsimport tests","fromName":"Chris Rorvick","fromEmail":"chris@rorvick.com","sentAt":"2013-01-11T04:27:16Z","receivedAt":"2013-01-11T04:27:16Z","isPatch":true,"sender":{"key":"chris@rorvick.com","avatar":"https://avatars.githubusercontent.com/u/824726?v=4"},"body":"These patchs apply on top of of Eric Raymond's cvsimport patch. 7 of 15\ntests in t9600 fail, one of which is fixed w/ a cvsps patch I've sent\nto Eric (fixes revision map.) It no longer uses \"origin\" as the default\nbranch which I suspect is a problem for at least some of the remaining\ntests. Both of the t9604 tests pass.\n\nChris\n\nChris Rorvick (3):\n t/lib-cvs.sh: allow cvsps version 3.x.\n t9600: fixup for new cvsimport\n t9604: fixup for new cvsimport\n\n t/lib-cvs.sh | 2 +-\n t/t9600-cvsimport.sh | 10 ++++------\n t/t9604-cvsimport-timestamps.sh | 5 ++---\n 3 files changed, 7 insertions(+), 10 deletions(-)\n\n-- \n1.8.1.1.g220e17a\n"},{"id":"206492","messageId":"1357878439-27500-2-git-send-email-chris@rorvick.com","threadId":"32587","inReplyTo":"1357878439-27500-1-git-send-email-chris@rorvick.com","subject":"[PATCH 1/3] t/lib-cvs.sh: allow cvsps version 3.x.","fromName":"Chris Rorvick","fromEmail":"chris@rorvick.com","sentAt":"2013-01-11T04:27:17Z","receivedAt":"2013-01-11T04:27:17Z","isPatch":true,"sender":{"key":"chris@rorvick.com","avatar":"https://avatars.githubusercontent.com/u/824726?v=4"},"body":"---\n t/lib-cvs.sh | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/t/lib-cvs.sh b/t/lib-cvs.sh\nindex 44263ad..b55e861 100644\n--- a/t/lib-cvs.sh\n+++ b/t/lib-cvs.sh\n@@ -15,7 +15,7 @@ export CVS\n \n cvsps_version=`cvsps -h 2>&1 | sed -ne 's/cvsps version //p'`\n case \"$cvsps_version\" in\n-2.1 | 2.2*)\n+2.1 | 2.2* | 3.*)\n \t;;\n '')\n \tskip_all='skipping cvsimport tests, cvsps not found'\n-- \n1.8.1.1.g220e17a\n"},{"id":"206493","messageId":"1357878439-27500-3-git-send-email-chris@rorvick.com","threadId":"32587","inReplyTo":"1357878439-27500-1-git-send-email-chris@rorvick.com","subject":"[PATCH 2/3] t9600: fixup for new cvsimport","fromName":"Chris Rorvick","fromEmail":"chris@rorvick.com","sentAt":"2013-01-11T04:27:18Z","receivedAt":"2013-01-11T04:27:18Z","isPatch":true,"sender":{"key":"chris@rorvick.com","avatar":"https://avatars.githubusercontent.com/u/824726?v=4"},"body":"---\n t/t9600-cvsimport.sh | 10 ++++------\n 1 file changed, 4 insertions(+), 6 deletions(-)\n\ndiff --git a/t/t9600-cvsimport.sh b/t/t9600-cvsimport.sh\nindex 4c384ff..14f54d5 100755\n--- a/t/t9600-cvsimport.sh\n+++ b/t/t9600-cvsimport.sh\n@@ -44,7 +44,7 @@ EOF\n \n test_expect_success PERL 'import a trivial module' '\n \n-\tgit cvsimport -a -R -z 0 -C module-git module &&\n+\tgit cvsimport -R -z 0 -C module-git module &&\n \ttest_cmp module-cvs/o_fortuna module-git/o_fortuna\n \n '\n@@ -90,8 +90,7 @@ test_expect_success PERL 'update git module' '\n \n \t(cd module-git &&\n \tgit config cvsimport.trackRevisions true &&\n-\tgit cvsimport -a -z 0 module &&\n-\tgit merge origin\n+\tgit cvsimport -z 0 module\n \t) &&\n \ttest_cmp module-cvs/o_fortuna module-git/o_fortuna\n \n@@ -119,8 +118,7 @@ test_expect_success PERL 'cvsimport.module config works' '\n \t(cd module-git &&\n \t\tgit config cvsimport.module module &&\n \t\tgit config cvsimport.trackRevisions true &&\n-\t\tgit cvsimport -a -z0 &&\n-\t\tgit merge origin\n+\t\tgit cvsimport -z0\n \t) &&\n \ttest_cmp module-cvs/tick module-git/tick\n \n@@ -140,7 +138,7 @@ test_expect_success PERL 'import from a CVS working tree' '\n \t$CVS co -d import-from-wt module &&\n \t(cd import-from-wt &&\n \t\tgit config cvsimport.trackRevisions false &&\n-\t\tgit cvsimport -a -z0 &&\n+\t\tgit cvsimport -z0 &&\n \t\techo 1 >expect &&\n \t\tgit log -1 --pretty=format:%s%n >actual &&\n \t\ttest_cmp actual expect\n-- \n1.8.1.1.g220e17a\n"},{"id":"206494","messageId":"1357878439-27500-4-git-send-email-chris@rorvick.com","threadId":"32587","inReplyTo":"1357878439-27500-1-git-send-email-chris@rorvick.com","subject":"[PATCH 3/3] t9604: fixup for new cvsimport","fromName":"Chris Rorvick","fromEmail":"chris@rorvick.com","sentAt":"2013-01-11T04:27:19Z","receivedAt":"2013-01-11T04:27:19Z","isPatch":true,"sender":{"key":"chris@rorvick.com","avatar":"https://avatars.githubusercontent.com/u/824726?v=4"},"body":"---\n t/t9604-cvsimport-timestamps.sh | 5 ++---\n 1 file changed, 2 insertions(+), 3 deletions(-)\n\ndiff --git a/t/t9604-cvsimport-timestamps.sh b/t/t9604-cvsimport-timestamps.sh\nindex 1fd5142..b1629b6 100755\n--- a/t/t9604-cvsimport-timestamps.sh\n+++ b/t/t9604-cvsimport-timestamps.sh\n@@ -7,8 +7,7 @@ setup_cvs_test_repository t9604\n \n test_expect_success 'check timestamps are UTC (TZ=CST6CDT)' '\n \n-\tTZ=CST6CDT git cvsimport -p\"-x\" -C module-1 module &&\n-\tgit cvsimport -p\"-x\" -C module-1 module &&\n+\tTZ=CST6CDT git cvsimport -C module-1 module &&\n \t(\n \t\tcd module-1 &&\n \t\tgit log --format=\"%s %ai\"\n@@ -42,7 +41,7 @@ test_expect_success 'check timestamps with author-specific timezones' '\n \tuser3=User Three EST5EDT\n \tuser4=User Four MST7MDT\n \tEOF\n-\tgit cvsimport -p\"-x\" -A cvs-authors -C module-2 module &&\n+\tgit cvsimport -A cvs-authors -C module-2 module &&\n \t(\n \t\tcd module-2 &&\n \t\tgit log --format=\"%s %ai %an\"\n-- \n1.8.1.1.g220e17a\n"},{"id":"207294","messageId":"20130120125838.GK31172@serenity.lan","threadId":"32587","inReplyTo":"1357878439-27500-1-git-send-email-chris@rorvick.com","subject":"Re: [PATCH 0/3] fixup remaining cvsimport tests","fromName":"John Keeping","fromEmail":"john@keeping.me.uk","sentAt":"2013-01-20T12:58:38Z","receivedAt":"2013-01-20T12:58:38Z","isPatch":true,"sender":{"key":"john@keeping.me.uk","avatar":"https://avatars.githubusercontent.com/u/1702081?v=4"},"body":"Hi Chris,\n\nOn Thu, Jan 10, 2013 at 10:27:16PM -0600, Chris Rorvick wrote:\n> These patchs apply on top of of Eric Raymond's cvsimport patch. 7 of 15\n> tests in t9600 fail, one of which is fixed w/ a cvsps patch I've sent\n> to Eric (fixes revision map.)\n\nDid you post the fix for the revision map publicly anywhere? I'm hoping\nto publish some fixes to command handling but would like to have the\ntests passing first - and if you've already done the work...\n\nSorry if you have and I've missed it,\n\nJohn\n"},{"id":"207307","messageId":"CAEUsAPZKd+mw2iK7nd6rTtB8N+B99ud19FkuSx0HVitNxrxxZA@mail.gmail.com","threadId":"32587","inReplyTo":"20130120125838.GK31172@serenity.lan","subject":"Re: [PATCH 0/3] fixup remaining cvsimport tests","fromName":"Chris Rorvick","fromEmail":"chris@rorvick.com","sentAt":"2013-01-20T15:22:03Z","receivedAt":"2013-01-20T15:22:03Z","isPatch":true,"sender":{"key":"chris@rorvick.com","avatar":"https://avatars.githubusercontent.com/u/824726?v=4"},"body":"On Sun, Jan 20, 2013 at 6:58 AM, John Keeping wrote:\n> Hi Chris,\n>\n> On Thu, Jan 10, 2013 at 10:27:16PM -0600, Chris Rorvick wrote:\n>> These patchs apply on top of of Eric Raymond's cvsimport patch. 7 of 15\n>> tests in t9600 fail, one of which is fixed w/ a cvsps patch I've sent\n>> to Eric (fixes revision map.)\n>\n> Did you post the fix for the revision map publicly anywhere?\n\nIt's in Eric's repo and included in version 3.8:\n\nhttps://gitorious.org/cvsps/cvsps/commit/abe81e1775a8959291f629029513d1b7160bbde6\n\nChris\n"},{"id":"207308","messageId":"20130120152857.GM31172@serenity.lan","threadId":"32587","inReplyTo":"CAEUsAPZKd+mw2iK7nd6rTtB8N+B99ud19FkuSx0HVitNxrxxZA@mail.gmail.com","subject":"Re: [PATCH 0/3] fixup remaining cvsimport tests","fromName":"John Keeping","fromEmail":"john@keeping.me.uk","sentAt":"2013-01-20T15:28:57Z","receivedAt":"2013-01-20T15:28:57Z","isPatch":true,"sender":{"key":"john@keeping.me.uk","avatar":"https://avatars.githubusercontent.com/u/1702081?v=4"},"body":"On Sun, Jan 20, 2013 at 09:22:03AM -0600, Chris Rorvick wrote:\n> On Sun, Jan 20, 2013 at 6:58 AM, John Keeping wrote:\n>> On Thu, Jan 10, 2013 at 10:27:16PM -0600, Chris Rorvick wrote:\n>>> These patchs apply on top of of Eric Raymond's cvsimport patch. 7 of 15\n>>> tests in t9600 fail, one of which is fixed w/ a cvsps patch I've sent\n>>> to Eric (fixes revision map.)\n>>\n>> Did you post the fix for the revision map publicly anywhere?\n> \n> It's in Eric's repo and included in version 3.8:\n> \n> https://gitorious.org/cvsps/cvsps/commit/abe81e1775a8959291f629029513d1b7160bbde6\n\nThanks. For some reason I thought the fix would be to\ngit-cvsimport-3.py. Obviously I should have read more carefully.\n\nSorry for the noise.\n\n\nJohn\n"},{"id":"207318","messageId":"7vsj5vlm1d.fsf@alter.siamese.dyndns.org","threadId":"32587","inReplyTo":"20130120152857.GM31172@serenity.lan","subject":"Re: [PATCH 0/3] fixup remaining cvsimport tests","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-01-20T18:57:50Z","receivedAt":"2013-01-20T18:57:50Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"John Keeping writes:\n\n> On Sun, Jan 20, 2013 at 09:22:03AM -0600, Chris Rorvick wrote:\n>> On Sun, Jan 20, 2013 at 6:58 AM, John Keeping wrote:\n>>> On Thu, Jan 10, 2013 at 10:27:16PM -0600, Chris Rorvick wrote:\n>>>> These patchs apply on top of of Eric Raymond's cvsimport patch. 7 of 15\n>>>> tests in t9600 fail, one of which is fixed w/ a cvsps patch I've sent\n>>>> to Eric (fixes revision map.)\n>>>\n>>> Did you post the fix for the revision map publicly anywhere?\n>> \n>> It's in Eric's repo and included in version 3.8:\n>> \n>> https://gitorious.org/cvsps/cvsps/commit/abe81e1775a8959291f629029513d1b7160bbde6\n>\n> Thanks. For some reason I thought the fix would be to\n> git-cvsimport-3.py. Obviously I should have read more carefully.\n>\n> Sorry for the noise.\n\nThis is not a noise, though.\n\nChris, how would we want to proceed? I'd prefer at some point to\nsee cvsimport-3 to be in sync when the one patched and tested in\nEric's repository is proven enough. Will Eric be the gatekeeper, or\nwill you be sending patches this way as well?\n"},{"id":"207323","messageId":"20130120192412.GA7498@serenity.lan","threadId":"32587","inReplyTo":"7vsj5vlm1d.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 0/3] fixup remaining cvsimport tests","fromName":"John Keeping","fromEmail":"john@keeping.me.uk","sentAt":"2013-01-20T19:24:12Z","receivedAt":"2013-01-20T19:24:12Z","isPatch":true,"sender":{"key":"john@keeping.me.uk","avatar":"https://avatars.githubusercontent.com/u/1702081?v=4"},"body":"On Sun, Jan 20, 2013 at 10:57:50AM -0800, Junio C Hamano wrote:\n> John Keeping writes:\n>> On Sun, Jan 20, 2013 at 09:22:03AM -0600, Chris Rorvick wrote:\n>>> On Sun, Jan 20, 2013 at 6:58 AM, John Keeping wrote:\n>>>> On Thu, Jan 10, 2013 at 10:27:16PM -0600, Chris Rorvick wrote:\n>>>>> These patchs apply on top of of Eric Raymond's cvsimport patch. 7 of 15\n>>>>> tests in t9600 fail, one of which is fixed w/ a cvsps patch I've sent\n>>>>> to Eric (fixes revision map.)\n>>>>\n>>>> Did you post the fix for the revision map publicly anywhere?\n>>> \n>>> It's in Eric's repo and included in version 3.8:\n>>> \n>>> https://gitorious.org/cvsps/cvsps/commit/abe81e1775a8959291f629029513d1b7160bbde6\n>>\n>> Thanks. For some reason I thought the fix would be to\n>> git-cvsimport-3.py. Obviously I should have read more carefully.\n>>\n>> Sorry for the noise.\n> \n> This is not a noise, though.\n> \n> Chris, how would we want to proceed? I'd prefer at some point to\n> see cvsimport-3 to be in sync when the one patched and tested in\n> Eric's repository is proven enough. Will Eric be the gatekeeper, or\n> will you be sending patches this way as well?\n\nIn this case the patch was to the C portion of cvsps, not the Python\ncvs-import, so not relevant for this particular case.\n\nI currently have a set of patches on top of jc/cvsimport-upgrade, which\nis slightly out-of-sync with git-cvsimport.py in Eric's cvsps\nrepository, because I hadn't realised that the latter existed until\nabout an hour ago.\n\nI haven't decided yet whether to rebase those onto the git-cvsimport.py\nin the cvsps repository or send them here to apply on top of\njc/cvsimport-upgrade. Given that git-cvsimport is a command which has\nbeen around for a while and (although this is a complete re-write) the\naim of these changes is to keep it working as the upstream project\nchanges, I have a slight preference for keeping git-cvsimport here and\nrecommending that the copy in the cvsps repository is removed.\n\n\nJohn\n"},{"id":"207329","messageId":"CAEUsAPaw8EUcZFbODDj9Z-=3Ppd1CC=jvYDvuyntFkX_3V0ynQ@mail.gmail.com","threadId":"32587","inReplyTo":"7vsj5vlm1d.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 0/3] fixup remaining cvsimport tests","fromName":"Chris Rorvick","fromEmail":"chris@rorvick.com","sentAt":"2013-01-20T20:17:02Z","receivedAt":"2013-01-20T20:17:02Z","isPatch":true,"sender":{"key":"chris@rorvick.com","avatar":"https://avatars.githubusercontent.com/u/824726?v=4"},"body":"On Sun, Jan 20, 2013 at 12:57 PM, Junio C Hamano wrote:\n> John Keeping writes:\n>\n>> On Sun, Jan 20, 2013 at 09:22:03AM -0600, Chris Rorvick wrote:\n>>> On Sun, Jan 20, 2013 at 6:58 AM, John Keeping wrote:\n>>>> On Thu, Jan 10, 2013 at 10:27:16PM -0600, Chris Rorvick wrote:\n>>>>> These patchs apply on top of of Eric Raymond's cvsimport patch. 7 of 15\n>>>>> tests in t9600 fail, one of which is fixed w/ a cvsps patch I've sent\n>>>>> to Eric (fixes revision map.)\n>>>>\n>>>> Did you post the fix for the revision map publicly anywhere?\n>>>\n>>> It's in Eric's repo and included in version 3.8:\n>>>\n>>> https://gitorious.org/cvsps/cvsps/commit/abe81e1775a8959291f629029513d1b7160bbde6\n>>\n>> Thanks. For some reason I thought the fix would be to\n>> git-cvsimport-3.py. Obviously I should have read more carefully.\n>>\n>> Sorry for the noise.\n>\n> This is not a noise, though.\n>\n> Chris, how would we want to proceed? I'd prefer at some point to\n> see cvsimport-3 to be in sync when the one patched and tested in\n> Eric's repository is proven enough. Will Eric be the gatekeeper, or\n> will you be sending patches this way as well?\n\nI probably won't be sending any more patches on this. My hope was to\nget cvsimport-3 (w/ cvsps as the engine) in a state such that one\ncould transition from the previous version seamlessly. But the break\nin t9605 has convinced me this is not worth the effort--even in this\ntrivial case cvsps is broken. The fuzzing logic aggregates commits\ninto patch sets that have timestamps within a specified window and\notherwise matching attributes. This aggregation causes file-level\ncommit timestamps to be lost and we are left with a single timestamp\nfor the patch set: the minimum for all contained CVS commits. When\nall commits have been processed, the patch sets are ordered\nchronologically and printed.\n\nThe problem is that is that a CVS commit is rolled into a patch set\nregardless of whether the patch set's timestamp falls within the\nadjacent CVS file-level commits. Even worse, since the patch set\ntimestamp changes as subsequent commits are added (i.e., it's always\npicking the earliest) it is potentially indeterminate at the time a\ncommit is added. The result is that file revisions can be reordered\nin resulting Git import (see t9605.) I spent some time last week\ntrying to solve this but I coudln't think of anything that wasn't a\nsubstantial re-work of the code.\n\nI have never used cvs2git, but I suspect Eric's efforts in making it a\npotential backend for cvsimport are a better use of time.\n\nChris\n"},{"id":"207336","messageId":"CAEUsAPbDTUXhz2BoDOwKCjcLS6BQA=jZ6DME4ZDfTDXRL=ZMqA@mail.gmail.com","threadId":"32587","inReplyTo":"20130120192412.GA7498@serenity.lan","subject":"Re: [PATCH 0/3] fixup remaining cvsimport tests","fromName":"Chris Rorvick","fromEmail":"chris@rorvick.com","sentAt":"2013-01-20T21:17:53Z","receivedAt":"2013-01-20T21:17:53Z","isPatch":true,"sender":{"key":"chris@rorvick.com","avatar":"https://avatars.githubusercontent.com/u/824726?v=4"},"body":"On Sun, Jan 20, 2013 at 1:24 PM, John Keeping wrote:\n> On Sun, Jan 20, 2013 at 10:57:50AM -0800, Junio C Hamano wrote:\n>> This is not a noise, though.\n>>\n>> Chris, how would we want to proceed? I'd prefer at some point to\n>> see cvsimport-3 to be in sync when the one patched and tested in\n>> Eric's repository is proven enough. Will Eric be the gatekeeper, or\n>> will you be sending patches this way as well?\n>\n> In this case the patch was to the C portion of cvsps, not the Python\n> cvs-import, so not relevant for this particular case.\n\nOh, I think I misunderstood the question. The only time I passed a\npatch specifically for git-cvsimport.py directly to Eric was before\nthe his patch was in Junio's repository. Unless I'm mistaken, only\nthe second patch Eric sent was actually imported. Subsequent to this\nI would have submitted any patches for git-cvsimport.py directly to\nthe git list. I just didn't have any--cvsps had several problems that\nneeded to be worked out before it made sense to look at the importer.\n\nIn other words, I don't think Eric should be a gatekeeper of this code.\n\nChris\n"},{"id":"207352","messageId":"CAEUsAPYdpsbhCZfp-1w91ZiyqgEa=8TNf2MJihMViqVZmW3sRw@mail.gmail.com","threadId":"32587","inReplyTo":"CAEUsAPaw8EUcZFbODDj9Z-=3Ppd1CC=jvYDvuyntFkX_3V0ynQ@mail.gmail.com","subject":"Re: [PATCH 0/3] fixup remaining cvsimport tests","fromName":"Chris Rorvick","fromEmail":"chris@rorvick.com","sentAt":"2013-01-21T01:34:46Z","receivedAt":"2013-01-21T01:34:46Z","isPatch":true,"sender":{"key":"chris@rorvick.com","avatar":"https://avatars.githubusercontent.com/u/824726?v=4"},"body":"On Sun, Jan 20, 2013 at 2:17 PM, Chris Rorvick wrote:\n> On Sun, Jan 20, 2013 at 12:57 PM, Junio C Hamano wrote:\n>> John Keeping writes:\n>>\n>>> On Sun, Jan 20, 2013 at 09:22:03AM -0600, Chris Rorvick wrote:\n>>>> On Sun, Jan 20, 2013 at 6:58 AM, John Keeping wrote:\n>>>>> On Thu, Jan 10, 2013 at 10:27:16PM -0600, Chris Rorvick wrote:\n>>>>>> These patchs apply on top of of Eric Raymond's cvsimport patch. 7 of 15\n>>>>>> tests in t9600 fail, one of which is fixed w/ a cvsps patch I've sent\n>>>>>> to Eric (fixes revision map.)\n>>>>>\n>>>>> Did you post the fix for the revision map publicly anywhere?\n>>>>\n>>>> It's in Eric's repo and included in version 3.8:\n>>>>\n>>>> https://gitorious.org/cvsps/cvsps/commit/abe81e1775a8959291f629029513d1b7160bbde6\n>>>\n>>> Thanks. For some reason I thought the fix would be to\n>>> git-cvsimport-3.py. Obviously I should have read more carefully.\n>>>\n>>> Sorry for the noise.\n>>\n>> This is not a noise, though.\n>>\n>> Chris, how would we want to proceed? I'd prefer at some point to\n>> see cvsimport-3 to be in sync when the one patched and tested in\n>> Eric's repository is proven enough. Will Eric be the gatekeeper, or\n>> will you be sending patches this way as well?\n>\n> I probably won't be sending any more patches on this. My hope was to\n> get cvsimport-3 (w/ cvsps as the engine) in a state such that one\n> could transition from the previous version seamlessly. But the break\n> in t9605 has convinced me this is not worth the effort--even in this\n> trivial case cvsps is broken. The fuzzing logic aggregates commits\n> into patch sets that have timestamps within a specified window and\n> otherwise matching attributes. This aggregation causes file-level\n> commit timestamps to be lost and we are left with a single timestamp\n> for the patch set: the minimum for all contained CVS commits. When\n> all commits have been processed, the patch sets are ordered\n> chronologically and printed.\n>\n> The problem is that is that a CVS commit is rolled into a patch set\n> regardless of whether the patch set's timestamp falls within the\n> adjacent CVS file-level commits. Even worse, since the patch set\n> timestamp changes as subsequent commits are added (i.e., it's always\n> picking the earliest) it is potentially indeterminate at the time a\n> commit is added. The result is that file revisions can be reordered\n> in resulting Git import (see t9605.) I spent some time last week\n> trying to solve this but I coudln't think of anything that wasn't a\n> substantial re-work of the code.\n>\n> I have never used cvs2git, but I suspect Eric's efforts in making it a\n> potential backend for cvsimport are a better use of time.\n>\n> Chris\n\nHi Eric,\n\nI noticed you were taken off this thread. As I mention above, I\nlooked into the bug tested in the t9605 patch Junio applied on top of\nyour cvsimport patch. The test was actually written for master to\ntest the Perl/cvsps2 import, but with minor modification you can\nverify the problem still exists in the 3.x versions of cvsps.\n\nI think the email above explains the problem pretty well. It's not\nclear to me what all the nastiness is that you've resolved with cvsps\nsince taking over; I've been mostly concerned with importing an almost\nbranchless repository which I thought avoided the types of problems\nyou were addressing. But this bug can actually cause Git's main\nimport branch to become inconsistent with CVS HEAD and you don't have\nto do anything too weird to get hit by it.\n\nFixing this seemed like it would require splitting the processing out\ninto a couple phases and would be a fair amount of work, but maybe I'm\njust not looking at the problem right.\n\nChris\n"},{"id":"207360","messageId":"20130121024314.GA27799@thyrsus.com","threadId":"32587","inReplyTo":"CAEUsAPYdpsbhCZfp-1w91ZiyqgEa=8TNf2MJihMViqVZmW3sRw@mail.gmail.com","subject":"Re: [PATCH 0/3] fixup remaining cvsimport tests","fromName":"Eric S. Raymond","fromEmail":"esr@thyrsus.com","sentAt":"2013-01-21T02:43:14Z","receivedAt":"2013-01-21T02:43:14Z","isPatch":true,"sender":{"key":"esr@thyrsus.com","avatar":"https://avatars.githubusercontent.com/u/727961?v=4"},"body":"> > I probably won't be sending any more patches on this. My hope was to\n> > get cvsimport-3 (w/ cvsps as the engine) in a state such that one\n> > could transition from the previous version seamlessly. But the break\n> > in t9605 has convinced me this is not worth the effort--even in this\n> > trivial case cvsps is broken. The fuzzing logic aggregates commits\n> > into patch sets that have timestamps within a specified window and\n> > otherwise matching attributes. This aggregation causes file-level\n> > commit timestamps to be lost and we are left with a single timestamp\n> > for the patch set: the minimum for all contained CVS commits. When\n> > all commits have been processed, the patch sets are ordered\n> > chronologically and printed.\n> >\n> > The problem is that is that a CVS commit is rolled into a patch set\n> > regardless of whether the patch set's timestamp falls within the\n> > adjacent CVS file-level commits. Even worse, since the patch set\n> > timestamp changes as subsequent commits are added (i.e., it's always\n> > picking the earliest) it is potentially indeterminate at the time a\n> > commit is added. The result is that file revisions can be reordered\n> > in resulting Git import (see t9605.) I spent some time last week\n> > trying to solve this but I coudln't think of anything that wasn't a\n> > substantial re-work of the code.\n\nI've lost who was who in the comment thread, but I think it is rather likely\nthat the above diagnosis is correct in every respect.\n\nI won't know for certain until I finish the test suite and apply it to\nall three tools (cvsps, cvs2git, cvs-fast-export) but what I've seen\nof their code indicates that cvsps has the weakest changeset analysis of\nthe three, even after my fixes.\n\n> > I have never used cvs2git, but I suspect Eric's efforts in making it a\n> > potential backend for cvsimport are a better use of time.\n\nAgreed. I didn't add multiengine support to csvsimport at random or\njust because Heiko Vogt was bugging me about parsecvs. I was\nhalf-expecting cvsps to manifest a showstopper like this - hoping it\nwouldn't, but hedging against the possibility by making alternate\nengines easy to plug into git-cvsimport seemed like a *really good\nidea* from the beginning of my work on it. Sometimes being that kind\nof right really sucks.\n\nWhile I am going to have a try at modifying cvsps to make Chris's\nt9605 case work, I'm going to strictly limit the amount of time I\nspend on that effort since (as you imply) it is fairly likely this\nwould be throwing good money after bad.\n\n> Fixing this seemed like it would require splitting the processing out\n> into a couple phases and would be a fair amount of work, but maybe I'm\n> just not looking at the problem right.\n\nActually I think you've called it *exactly* right. The job has to be \ndone in multiple clique-spitting phases - that's why cvs2git has 7 passes\n(though a few of those, perhaps as many as 3, are artifactual).\n\nThis is why the next step in my current work plan for CVS-related stuff will\nbe unbundling my test suite from the cvsps tree and running it to see if\ncvs-fast-export dominates cvsps. \n\nI'm expecting that it will, in which case my plan will be to salvage\nthe CVS client code out of cvsps (*that* part is quite good - fast,\nclean, effective) gluing it to the better analysis stage in\ncvs-fast-export, and then shooting cvsps through the head and burying\nit behind the barn.\n-- \n\t\tEric S. Raymond\n"},{"id":"207579","messageId":"50FFB35C.7070809@alum.mit.edu","threadId":"32587","inReplyTo":"CAEUsAPaw8EUcZFbODDj9Z-=3Ppd1CC=jvYDvuyntFkX_3V0ynQ@mail.gmail.com","subject":"Re: [PATCH 0/3] fixup remaining cvsimport tests","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2013-01-23T09:54:36Z","receivedAt":"2013-01-23T09:54:36Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 01/20/2013 09:17 PM, Chris Rorvick wrote:\n> I probably won't be sending any more patches on this. My hope was to\n> get cvsimport-3 (w/ cvsps as the engine) in a state such that one\n> could transition from the previous version seamlessly. But the break\n> in t9605 has convinced me this is not worth the effort--even in this\n> trivial case cvsps is broken. The fuzzing logic aggregates commits\n> into patch sets that have timestamps within a specified window and\n> otherwise matching attributes. This aggregation causes file-level\n> commit timestamps to be lost and we are left with a single timestamp\n> for the patch set: the minimum for all contained CVS commits. When\n> all commits have been processed, the patch sets are ordered\n> chronologically and printed.\n> \n> The problem is that is that a CVS commit is rolled into a patch set\n> regardless of whether the patch set's timestamp falls within the\n> adjacent CVS file-level commits. Even worse, since the patch set\n> timestamp changes as subsequent commits are added (i.e., it's always\n> picking the earliest) it is potentially indeterminate at the time a\n> commit is added. The result is that file revisions can be reordered\n> in resulting Git import (see t9605.) I spent some time last week\n> trying to solve this but I coudln't think of anything that wasn't a\n> substantial re-work of the code.\n> \n> I have never used cvs2git, but I suspect Eric's efforts in making it a\n> potential backend for cvsimport are a better use of time.\n\nThanks for your explanation of how cvsps works.\n\nThis is roughly how cvs2svn used to work years ago, prior to release\n2.x. In addition it did a number of things to try to tweak the\ntimestamp ordering to avoid committing file-level commits in the wrong\norder. It never worked 100%; each tweak that was made to fix one\nproblem created another problem in another scenario.\n\ncvs2svn/cvs2git 2.x takes a very different approach. It uses a\ntimestamp threshold along with author and commit-message matching to\nfind the biggest set of file-level commits that might constitute a\nrepository-level commit. But then it checks the proto-commits to see if\nthey violate the ordering constraints imposed by the individual\nfile-level commits. For example, if the initial grouping gives the\nfollowing proto-commits:\n\nproto-commit 1: a.txt 1.1 b.txt 1.2\n\nproto-commit 2: a.txt 1.2 b.txt 1.1\n\nthen it is apparent that something is wrong, because a.txt 1.1\nnecessarily comes before a.txt 1.2 whereas b.txt 1.1 necessarily comes\nbefore b.txt 1.2 (CVS can at least be relied on to get this right!) and\ntherefore there is no consistent ordering of the two proto-commits.\nMore generally, the proto-commits have to form a directed acyclic graph,\nwhereas this graph has a cycle 1 -> 2 -> 1. When cvs2svn/cvs2git finds\na cycle, it uses heuristics to break up one or more of the proto-commits\nto break the cycle. In this case it might break proto-commit 1 into two\ncommits:\n\nproto-commit 1a: a.txt 1.1\n\nproto-commit 2: a.txt 1.2 b.txt 1.1\n\nproto-commit 1b: b.txt 1.2\n\nNow it is possible to commit them in the order 1a,2,1b. (Exactly this\nscenario is tested in t9603.)\n\nOf course a typical proto-commit graph often contains far more\ncomplicated cycles, but the approach remains the same: split\nproto-commits up as necessary until the graph is acyclic. One can\nquibble about the heuristics that cvs2svn/cvs2git uses to break up\nproto-commits. But the final result of the algorithm is *guaranteed* to\nbe consistent with the file-level CVS history and also self-consistent.\n\nI am skeptical that a simpler approach will ever work 100%.\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\nhttp://softwareswirl.blogspot.com/\n"},{"id":"207584","messageId":"20130123110312.GK7498@serenity.lan","threadId":"32587","inReplyTo":"50FFB35C.7070809@alum.mit.edu","subject":"Re: [PATCH 0/3] fixup remaining cvsimport tests","fromName":"John Keeping","fromEmail":"john@keeping.me.uk","sentAt":"2013-01-23T11:03:12Z","receivedAt":"2013-01-23T11:03:12Z","isPatch":true,"sender":{"key":"john@keeping.me.uk","avatar":"https://avatars.githubusercontent.com/u/1702081?v=4"},"body":"On Wed, Jan 23, 2013 at 10:54:36AM +0100, Michael Haggerty wrote:\n> On 01/20/2013 09:17 PM, Chris Rorvick wrote:\n>> I have never used cvs2git, but I suspect Eric's efforts in making it a\n>> potential backend for cvsimport are a better use of time.\n\nIs it possible to perform an incremental import with cvs2git? This\nseems to be the one use case where the old cvsimport script (with cvsps\n2.x) still performs the best.\n\nI suppose that just re-running the full import will do the right thing\nsince the commits in Git should be identical, but would it be possible\nto do better given the right information about a previous run?\n\n\nJohn\n"},{"id":"207660","messageId":"5100A766.8050902@alum.mit.edu","threadId":"32587","inReplyTo":"20130123110312.GK7498@serenity.lan","subject":"Re: [PATCH 0/3] fixup remaining cvsimport tests","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2013-01-24T03:15:50Z","receivedAt":"2013-01-24T03:15:50Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 01/23/2013 12:03 PM, John Keeping wrote:\n> On Wed, Jan 23, 2013 at 10:54:36AM +0100, Michael Haggerty wrote:\n>> On 01/20/2013 09:17 PM, Chris Rorvick wrote:\n>>> I have never used cvs2git, but I suspect Eric's efforts in making it a\n>>> potential backend for cvsimport are a better use of time.\n> \n> Is it possible to perform an incremental import with cvs2git? This\n> seems to be the one use case where the old cvsimport script (with cvsps\n> 2.x) still performs the best.\n> \n> I suppose that just re-running the full import will do the right thing\n> since the commits in Git should be identical, but would it be possible\n> to do better given the right information about a previous run?\n\nNo, cvs2git does not support incremental imports. One user has reported\nthat he *usually* gets identical commits for the overlapping history\nwhen he re-runs a full import, and last I heard he was using this as a\nkind of incremental import. We make an effort to make imports\nreproducible, at least when using a single version of cvs2git (for\nexample, we process things in deterministic order rather than the order\nthey happen come out of a file directory or Python hashmap). But the\ncycle-breaking heuristics in particular can give different results if\nhistory is added, not to mention the fact that CVS allows the user to\nmake changes with retroactive and non-timestamped effects (e.g., adding\nor removing files from an existing branch/tag, changing a file's default\nbranch from vendor to HEAD, changing the log messages of old revisions,\nobsoleting revisions). And if your repository is large, a full import\ncan take a while.\n\nIt would be possible to enhance cvs2git to handle incremental imports\n(well, at least if you rule out a few CVS commands that change deep\nhistory). But I don't believe anybody is working on this.\n\nMichael\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\nhttp://softwareswirl.blogspot.com/\n"}]}