{"thread":{"id":"18570","subject":"svn clone Checksum mismatch question","startedAt":"2009-03-26T10:31:53Z","lastAt":"2009-04-03T16:25:29Z","messageCount":28,"participants":["Gilbert Liddell","Björn Steinbrink","Sverre Rabbelier","Peter Harris","Anton Gyllenberg","Johannes Schindelin","Eric Wong","Junio C Hamano","Linus Torvalds"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"109521","messageId":"22719363.post@talk.nabble.com","threadId":"18570","inReplyTo":null,"subject":"svn clone Checksum mismatch question","fromName":"Gilbert Liddell","fromEmail":"gliddell@totalrepair.co.uk","sentAt":"2009-03-26T10:31:53Z","receivedAt":"2009-03-26T10:31:53Z","isPatch":false,"sender":{"key":"gliddell@totalrepair.co.uk","avatar":null},"body":"\nHi,\n\nI've just started using GIT this week, currently the project i'm working on\nis held in subversion. I tested git svn clone with a small test project\n(about 10 files) which worked a treat.\n\nThis morning i decided to test the clone with the full project i'm working\non (11,000 files) and I get the error message Checksum mismatch: vn2.sln\n0f7a82f1d38b819 expected: fde799e5ba0d1d07e6b539016bea3260\ngot: e71db1010a0da06ea76d4163c452df72\n\nCan someone help with why this error is happening? Is there an issue with\nthe GIT clone and large repositories?\n\nThanks in advance for your help,\nGilbert.\n-- \nView this message in context: http://www.nabble.com/svn-clone-Checksum-mismatch-question-tp22719363p22719363.html\nSent from the git mailing list archive at Nabble.com.\n"},{"id":"109527","messageId":"20090326130213.GC3114@atjola.homenet","threadId":"18570","inReplyTo":"22719363.post@talk.nabble.com","subject":"Re: svn clone Checksum mismatch question","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-03-26T13:02:13Z","receivedAt":"2009-03-26T13:02:13Z","isPatch":false,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2009.03.26 03:31:53 -0700, Gilbert Liddell wrote:\n> This morning i decided to test the clone with the full project i'm working\n> on (11,000 files) and I get the error message Checksum mismatch: vn2.sln\n> 0f7a82f1d38b819 expected: fde799e5ba0d1d07e6b539016bea3260\n> got: e71db1010a0da06ea76d4163c452df72\n> \n> Can someone help with why this error is happening? Is there an issue with\n> the GIT clone and large repositories?\n\nWhich git version is that? There was some bug in git-svn that caused it\nto fill the disk with temporary files, without noticing that those files\nget truncated when the disk is full. That was fixed in some 1.6.0.x\nrelease IIRC.\n\nBjörn\n"},{"id":"109529","messageId":"D92CD911394B11428C65AFB4222835AE01255586@mercury.totalrepair.co.uk","threadId":"18570","inReplyTo":"20090326130213.GC3114@atjola.homenet","subject":"RE: svn clone Checksum mismatch question","fromName":"Gilbert Liddell","fromEmail":"gliddell@totalrepair.co.uk","sentAt":null,"receivedAt":"2009-03-26T13:40:19Z","isPatch":false,"sender":{"key":"gliddell@totalrepair.co.uk","avatar":null},"body":"Hi Björn,\n\nThanks for the reply, i'm using git version 1.6.2.msysgit.0.186.gf7512\n\nGilbert.\n\n-----Original Message-----\nFrom: Björn Steinbrink [mailto:B.Steinbrink@gmx.de] \nSent: 26 March 2009 13:02\nTo: Gilbert Liddell\nCc: git@vger.kernel.org\nSubject: Re: svn clone Checksum mismatch question\n\nOn 2009.03.26 03:31:53 -0700, Gilbert Liddell wrote:\n> This morning i decided to test the clone with the full project i'm working\n> on (11,000 files) and I get the error message Checksum mismatch: vn2.sln\n> 0f7a82f1d38b819 expected: fde799e5ba0d1d07e6b539016bea3260\n> got: e71db1010a0da06ea76d4163c452df72\n> \n> Can someone help with why this error is happening? Is there an issue with\n> the GIT clone and large repositories?\n\nWhich git version is that? There was some bug in git-svn that caused it\nto fill the disk with temporary files, without noticing that those files\nget truncated when the disk is full. That was fixed in some 1.6.0.x\nrelease IIRC.\n\nBjörn\n\nRegistered in Scotland\n32 Fountain Drive\nInchinnan Business Park\nRenfrewshire,PA4 9RF.\nCompany number:SC112872\n\nThis e-mail and any files transmitted with it are confidential and\nintended solely for the use of the individual or entity to whom\nthey are addressed.\nIf you have received this e-mail in error please notify the\noriginator of the message. This footer also confirms that this\ne-mail message has been scanned for the presence of computer viruses.\n\nAny views expressed in this message are those of the individual\nsender, except where the sender specifies and with authority,\nstates them to be the views of Total Repair Solutions.\n\nScanning of this message and addition of this footer is performed\nby SurfControl E-mail Filter software in conjunction with \nvirus detection software.\n"},{"id":"109531","messageId":"fabb9a1e0903260654n5e682c49hbad3d2ece093af3f@mail.gmail.com","threadId":"18570","inReplyTo":"D92CD911394B11428C65AFB4222835AE01255586@mercury.totalrepair.co.uk","subject":"Re: svn clone Checksum mismatch question","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2009-03-26T13:54:50Z","receivedAt":"2009-03-26T13:54:50Z","isPatch":false,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\n[We do not top post on this list, instead it is customary to reply\ninline, as I and Björn have done]\n\n2009/3/26 Gilbert Liddell :\n>2009/3/26 Björn Steinbrink :\n>> On 2009.03.26 03:31:53 -0700, Gilbert Liddell wrote:\n>>> This morning i decided to test the clone with the full project i'm working\n>>> on (11,000 files) and I get the error message Checksum mismatch: vn2.sln\n>>> 0f7a82f1d38b819 expected: fde799e5ba0d1d07e6b539016bea3260\n>>> got: e71db1010a0da06ea76d4163c452df72\n>>>\n>>> Can someone help with why this error is happening? Is there an issue with\n>>> the GIT clone and large repositories?\n>>\n>> Which git version is that? There was some bug in git-svn that caused it\n>> to fill the disk with temporary files, without noticing that those files\n>> get truncated when the disk is full. That was fixed in some 1.6.0.x\n>> release IIRC.\n>\n> Thanks for the reply, i'm using git version 1.6.2.msysgit.0.186.gf7512\n\nSeems like it could be one of the known bugs of git-svn on windows?\n(ccing Dscho and J6t)\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"109532","messageId":"D92CD911394B11428C65AFB4222835AE01255587@mercury.totalrepair.co.uk","threadId":"18570","inReplyTo":"fabb9a1e0903260654n5e682c49hbad3d2ece093af3f@mail.gmail.com","subject":"RE: svn clone Checksum mismatch question","fromName":"Gilbert Liddell","fromEmail":"gliddell@totalrepair.co.uk","sentAt":null,"receivedAt":"2009-03-26T13:54:50Z","isPatch":false,"sender":{"key":"gliddell@totalrepair.co.uk","avatar":null},"body":">2009/3/26 Gilbert Liddell :\n>>2009/3/26 Björn Steinbrink :\n>>> On 2009.03.26 03:31:53 -0700, Gilbert Liddell wrote:\n>>>> This morning i decided to test the clone with the full project i'm working\n>>>> on (11,000 files) and I get the error message Checksum mismatch: vn2.sln\n>>>> 0f7a82f1d38b819 expected: fde799e5ba0d1d07e6b539016bea3260\n>>>> got: e71db1010a0da06ea76d4163c452df72\n>>>>\n>>>> Can someone help with why this error is happening? Is there an issue with\n>>>> the GIT clone and large repositories?\n>>>\n>>> Which git version is that? There was some bug in git-svn that caused it\n>>> to fill the disk with temporary files, without noticing that those files\n>>> get truncated when the disk is full. That was fixed in some 1.6.0.x\n>>> release IIRC.\n>>\n>> Thanks for the reply, i'm using git version 1.6.2.msysgit.0.186.gf7512\n>\n>Seems like it could be one of the known bugs of git-svn on windows?\n> (ccing Dscho and J6t)\n>\n>-- \n>Cheers,\n>\n>Sverre Rabbelier\n\nHi,\n\nApologies for the Top Posting.\nI've not been able to find any info about this being a but with git-svn on Windows. I stumbled across this post that appears to be the same/similar issue - \nhttp://lists-archives.org/git/668493-git-svn-checksum-mismatch-importing-large-file.html\n\nGilbert.\n\n\n\n\nRegistered in Scotland\n32 Fountain Drive\nInchinnan Business Park\nRenfrewshire,PA4 9RF.\nCompany number:SC112872\n\nThis e-mail and any files transmitted with it are confidential and\nintended solely for the use of the individual or entity to whom\nthey are addressed.\nIf you have received this e-mail in error please notify the\noriginator of the message. This footer also confirms that this\ne-mail message has been scanned for the presence of computer viruses.\n\nAny views expressed in this message are those of the individual\nsender, except where the sender specifies and with authority,\nstates them to be the views of Total Repair Solutions.\n\nScanning of this message and addition of this footer is performed\nby SurfControl E-mail Filter software in conjunction with \nvirus detection software.\n"},{"id":"109535","messageId":"eaa105840903260734p1a6b95ewb293974f49fc7f24@mail.gmail.com","threadId":"18570","inReplyTo":"22719363.post@talk.nabble.com","subject":"Re: svn clone Checksum mismatch question","fromName":"Peter Harris","fromEmail":"git@peter.is-a-geek.org","sentAt":"2009-03-26T14:34:29Z","receivedAt":"2009-03-26T14:34:29Z","isPatch":false,"sender":{"key":"git@peter.is-a-geek.org","avatar":null},"body":"On Thu, Mar 26, 2009 at 6:31 AM, Gilbert Liddell wrote:\n>\n> This morning i decided to test the clone with the full project i'm working\n> on (11,000 files) and I get the error message Checksum mismatch: vn2.sln\n> 0f7a82f1d38b819 expected: fde799e5ba0d1d07e6b539016bea3260\n> got: e71db1010a0da06ea76d4163c452df72\n>\n> Can someone help with why this error is happening? Is there an issue with\n> the GIT clone and large repositories?\n\n(since you mentioned msysgit in another reply) What is your\ncore.autocrlf setting? Did you default it to 'true' or 'input' when\nyou installed msysgit?\n\nTry \"git config core.autocrlf false\" and resume the import process\n(with \"git svn fetch\" or similar).\n\nImporting from svn with autocrlf on only works if every text file has\nsvn:eol-style=native set in every revision. *.sln files are even\nworse, since they look like text to git, but they're really binary (so\nnobody sets svn:eol-style on them).\n\nPeter Harris\n"},{"id":"109537","messageId":"alpine.DEB.1.00.0903261534150.12753@intel-tinevez-2-302","threadId":"18570","inReplyTo":"fabb9a1e0903260654n5e682c49hbad3d2ece093af3f@mail.gmail.com","subject":"Re: svn clone Checksum mismatch question","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-03-26T14:34:43Z","receivedAt":"2009-03-26T14:34:43Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 26 Mar 2009, Sverre Rabbelier wrote:\n\n> Seems like it could be one of the known bugs of git-svn on windows? \n> (ccing Dscho and J6t)\n\nEOUTOFGITTIME,\nDscho\n"},{"id":"109536","messageId":"83dfc36c0903260735q3231ce96h5949d1123858995f@mail.gmail.com","threadId":"18570","inReplyTo":"20090326130213.GC3114@atjola.homenet","subject":"Re: svn clone Checksum mismatch question","fromName":"Anton Gyllenberg","fromEmail":"anton@iki.fi","sentAt":"2009-03-26T14:35:00Z","receivedAt":"2009-03-26T14:35:00Z","isPatch":false,"sender":{"key":"anton@iki.fi","avatar":null},"body":"2009/3/26 Björn Steinbrink :\n> On 2009.03.26 03:31:53 -0700, Gilbert Liddell wrote:\n>> This morning i decided to test the clone with the full project i'm working\n>> on (11,000 files) and I get the error message Checksum mismatch: vn2.sln\n>> 0f7a82f1d38b819 expected: fde799e5ba0d1d07e6b539016bea3260\n>> got: e71db1010a0da06ea76d4163c452df72\n>>\n>> Can someone help with why this error is happening? Is there an issue with\n>> the GIT clone and large repositories?\n>\n> Which git version is that? There was some bug in git-svn that caused it\n> to fill the disk with temporary files, without noticing that those files\n> get truncated when the disk is full. That was fixed in some 1.6.0.x\n> release IIRC.\n\nI don't know if this is the same issue, but the I get a similar error\non the public twisted-python repository on both windows and linux,\nwith several different versions and plenty of free disk space. As this\nis a publicly accessible repository it should be easy to reproduce:\n\ngit svn init -s svn://svn.twistedmatrix.com/svn/Twisted twisted\ncd twisted\ngit svn fetch -r 13611:HEAD\n\nThis ultimately dies with the following error:\nW: +empty_dir: trunk/doc/core/howto/listings/finger/finger\nr13612 = f6d995ac255e3dfa08a517a6e72fbcfe63feaaa0 (trunk)\nChecksum mismatch:\nbranches/foom/--omg-optimized/twisted/internet/cdefer/cdefer.pyx\n264b0c5f7b3a00d401d1a5dcce67a3734f0eede3\nexpected: c7ccddd195f132926e20bab573da7ef3\n got: f006323ff4714ca52c0228ce6390d415\n\n\nI found this a long time ago but never got around to analyze or report it.\n\nAnton\n"},{"id":"109632","messageId":"83dfc36c0903270418q59a81290xcb8043b8c037be18@mail.gmail.com","threadId":"18570","inReplyTo":"83dfc36c0903260735q3231ce96h5949d1123858995f@mail.gmail.com","subject":"Re: svn clone Checksum mismatch question","fromName":"Anton Gyllenberg","fromEmail":"anton@iki.fi","sentAt":"2009-03-27T11:18:07Z","receivedAt":"2009-03-27T11:18:07Z","isPatch":false,"sender":{"key":"anton@iki.fi","avatar":null},"body":"I hope I didn't hijack the thread with an unrelated issue.\n\n2009/3/26 Anton Gyllenberg :\n> I don't know if this is the same issue, but the I get a similar error\n> on the public twisted-python repository on both windows and linux,\n> with several different versions and plenty of free disk space. As this\n> is a publicly accessible repository it should be easy to reproduce:\n>\n> git svn init -s svn://svn.twistedmatrix.com/svn/Twisted twisted\n> cd twisted\n> git svn fetch -r 13611:HEAD\n>\n> This ultimately dies with the following error:\n> W: +empty_dir: trunk/doc/core/howto/listings/finger/finger\n> r13612 = f6d995ac255e3dfa08a517a6e72fbcfe63feaaa0 (trunk)\n> Checksum mismatch:\n> branches/foom/--omg-optimized/twisted/internet/cdefer/cdefer.pyx\n> 264b0c5f7b3a00d401d1a5dcce67a3734f0eede3\n> expected: c7ccddd195f132926e20bab573da7ef3\n>     got: f006323ff4714ca52c0228ce6390d415\n\nLooking into this, the mentioned blob\n264b0c5f7b3a00d401d1a5dcce67a3734f0eede3 with md5sum\nf006323ff4714ca52c0228ce6390d415 is not at path\nbranches/foom/--omg-optimized/twisted/internet/cdefer/cdefer.pyx. The\ncontents of the blob is the seemingly totally unrelated LICENSE file\nthat is found at trunk/LICENSE and\nbranches/foom/--omg-optimized/LICENSE. cdefer.pyx does have the md5sum\nc7ccddd195f132926e20bab573da7ef3. Note that the branch root directory\nis branches/foom/--omg-optimized (like with the branch name being\nfoom/--omg-optimized), not just branches/foom. Is think git-svn relies\non the standard layout being branches directly under the branches/\ndirectory, but I don't see how this would get the paths mixed up like\nthis.\n\nLooking at what was done around this commit one finds odd stuff, like\ndeleting directories in trunk and then copying from a previous\nrevision of trunk to under the branch:\nhttp://twistedmatrix.com/trac/changeset/13611\n\nI created a local test svn repository and tried to do something\nsimilar but git-svn had no problem with my test.\n\nThis is issue is not critical for me in any way but if somebody wants\nto look into it I am happy to help out.\n\nAnton\n"},{"id":"109750","messageId":"20090329060858.GB15773@dcvr.yhbt.net","threadId":"18570","inReplyTo":"83dfc36c0903270418q59a81290xcb8043b8c037be18@mail.gmail.com","subject":"Re: svn clone Checksum mismatch question","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2009-03-29T06:08:58Z","receivedAt":"2009-03-29T06:08:58Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Anton Gyllenberg wrote:\n> I hope I didn't hijack the thread with an unrelated issue.\n\nNo worries if you did. You helped me find a stupid bug in git-svn\nthat's been there for 3 years. I couldn't help the original poster at\nall since he was working on a private repo and I can't support git-svn\nunder Windows.\n\n> 2009/3/26 Anton Gyllenberg :\n> > I don't know if this is the same issue, but the I get a similar error\n> > on the public twisted-python repository on both windows and linux,\n> > with several different versions and plenty of free disk space. As this\n> > is a publicly accessible repository it should be easy to reproduce:\n> >\n> > git svn init -s svn://svn.twistedmatrix.com/svn/Twisted twisted\n> > cd twisted\n> > git svn fetch -r 13611:HEAD\n> >\n> > This ultimately dies with the following error:\n> > W: +empty_dir: trunk/doc/core/howto/listings/finger/finger\n> > r13612 = f6d995ac255e3dfa08a517a6e72fbcfe63feaaa0 (trunk)\n> > Checksum mismatch:\n> > branches/foom/--omg-optimized/twisted/internet/cdefer/cdefer.pyx\n> > 264b0c5f7b3a00d401d1a5dcce67a3734f0eede3\n> > expected: c7ccddd195f132926e20bab573da7ef3\n> >     got: f006323ff4714ca52c0228ce6390d415\n> \n> is branches/foom/--omg-optimized (like with the branch name being\n> foom/--omg-optimized), not just branches/foom. Is think git-svn relies\n> on the standard layout being branches directly under the branches/\n> directory, but I don't see how this would get the paths mixed up like\n> this.\n\nRoot problem: I misused \"git ls-tree\" for 3 years and nobody noticed.\nAt least I'm glad the checksum verification every step of the way caught\nthis bug and prevented propagating it into repository corruption.\n\n> Looking at what was done around this commit one finds odd stuff, like\n> deleting directories in trunk and then copying from a previous\n> revision of trunk to under the branch:\n> http://twistedmatrix.com/trac/changeset/13611\n> \n> I created a local test svn repository and tried to do something\n> similar but git-svn had no problem with my test.\n\nI was fooled by the weird copy sequences, too.\n\n> This is issue is not critical for me in any way but if somebody wants\n> to look into it I am happy to help out.\n\nI guess few folks in the UNIX world are crazy enough to make pathnames\nprefixed with dashes :) But I do wonder how/if many repositories out\nthere failed and nobody bothered to report it...\n\nPatch in reply\n\n-- \nEric Wong\n"},{"id":"293762","messageId":"20090329061045.GA29721@dcvr.yhbt.net","threadId":"18570","inReplyTo":"20090329060858.GB15773@dcvr.yhbt.net","subject":"[PATCH] git-svn: fix ls-tree usage with dash-prefixed paths","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2009-03-29T06:10:45Z","receivedAt":"2009-03-29T06:10:45Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"To find the blob object name given a tree and pathname, we were\nincorrectly calling \"git ls-tree\" with a \"--\" argument followed\nby the pathname of the file we wanted to get.\n\n git ls-tree -- --dashed/path/name.c\n\nUnlike many command-line interfaces, the \"--\" alone does not\nsymbolize the end of non-option arguments on the command-line.\n\nls-tree interprets the \"--\" as a prefix to match against, thus\nthe entire contents of the --dashed/* hierarchy would be\nreturned because the \"--\" matches \"--dashed\" and every path\nunder it.\n\nThanks to Anton Gyllenberg for pointing me toward the\nTwisted repository as a real-world example of this case.\n\nSigned-off-by: Eric Wong \n---\n\n Junio: This can go to maint. Thanks\n\n git-svn.perl | 15 +++++++++------\n 1 files changed, 9 insertions(+), 6 deletions(-)\n\ndiff --git a/git-svn.perl b/git-svn.perl\nindex 8be6be0..f21cfb4 100755\n--- a/git-svn.perl\n+++ b/git-svn.perl\n@@ -3387,15 +3387,18 @@ sub delete_entry {\n \treturn undef if ($gpath eq '');\n \n \t# remove entire directories.\n-\tif (command('ls-tree', $self->{c}, '--', $gpath) =~ /^040000 tree/) {\n+\tmy ($tree) = (command('ls-tree', '-z', $self->{c}, \"./$gpath\")\n+\t =~ /\\A040000 tree ([a-f\\d]{40})\\t\\Q$gpath\\E\\0/);\n+\tif ($tree) {\n \t\tmy ($ls, $ctx) = command_output_pipe(qw/ls-tree\n \t\t -r --name-only -z/,\n-\t\t\t\t $self->{c}, '--', $gpath);\n+\t\t\t\t $tree);\n \t\tlocal $/ = \"\\0\";\n \t\twhile (<$ls>) {\n \t\t\tchomp;\n-\t\t\t$self->{gii}->remove($_);\n-\t\t\tprint \"\\tD\\t$_\\n\" unless $::_q;\n+\t\t\tmy $rmpath = \"$gpath/$_\";\n+\t\t\t$self->{gii}->remove($rmpath);\n+\t\t\tprint \"\\tD\\t$rmpath\\n\" unless $::_q;\n \t\t}\n \t\tprint \"\\tD\\t$gpath/\\n\" unless $::_q;\n \t\tcommand_close_pipe($ls, $ctx);\n@@ -3414,8 +3417,8 @@ sub open_file {\n \tgoto out if is_path_ignored($path);\n \n \tmy $gpath = $self->git_path($path);\n-\t($mode, $blob) = (command('ls-tree', $self->{c}, '--', $gpath)\n-\t =~ /^(\\d{6}) blob ([a-f\\d]{40})\\t/);\n+\t($mode, $blob) = (command('ls-tree', '-z', $self->{c}, \"./$gpath\")\n+\t =~ /\\A(\\d{6}) blob ([a-f\\d]{40})\\t\\Q$gpath\\E\\0/);\n \tunless (defined $mode && defined $blob) {\n \t\tdie \"$path was not found in commit $self->{c} (r$rev)\\n\";\n \t}\n-- \n"},{"id":"109786","messageId":"7v8wmoqdc1.fsf@gitster.siamese.dyndns.org","threadId":"18570","inReplyTo":"20090329061045.GA29721@dcvr.yhbt.net","subject":"Re: [PATCH] git-svn: fix ls-tree usage with dash-prefixed paths","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-03-29T20:33:02Z","receivedAt":"2009-03-29T20:33:02Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Wong writes:\n\n> To find the blob object name given a tree and pathname, we were\n> incorrectly calling \"git ls-tree\" with a \"--\" argument followed\n> by the pathname of the file we wanted to get.\n>\n> git ls-tree -- --dashed/path/name.c\n>\n> Unlike many command-line interfaces, the \"--\" alone does not\n> symbolize the end of non-option arguments on the command-line.\n>\n> ls-tree interprets the \"--\" as a prefix to match against, thus\n> the entire contents of the --dashed/* hierarchy would be\n> returned because the \"--\" matches \"--dashed\" and every path\n> under it.\n\nThe above makes only half a sense to me. In an empty directory:\n\n $ git init\n Initialized empty Git repository in /tmp/empty/.git\n $ mkdir -p ./--dashed/path\n $ >./--dashed/path/name\n $ git add .\n $ git ls-files\n --dashed/path/name\n $ git commit -a -m initial\n [master (root-commit) cd44284] initial\n 0 files changed, 0 insertions(+), 0 deletions(-)\n create mode 100644 --dashed/path/name\n $ git ls-tree HEAD^{tree} --\n $ git ls-tree HEAD^{tree} -- --dashed/path/name\n 100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\t--dashed/path/name\n $ mkdir ./--\n $ >./--/eman\n $ git add .\n $ git commit -m second\n [master 80f8ef9] second\n 0 files changed, 0 insertions(+), 0 deletions(-)\n create mode 100644 --/eman\n $ git ls-tree HEAD^{tree} -- --dashed/path\n 100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\t--/eman\n 040000 tree 23e59e0c91294c39ac7c5a2e39efb01d878de9a0\t--dashed/path\n $ exit\n\nPerhaps the problem repository had a pathname that is exactly -- (in\naddition to --dashed/), and ls-tree emitted everything under --/\nhierarchy? In other words, your fix to git-svn may be correct and I am\nreading your problem description above incorrectly?\n\nAs the command always takes exactly one tree, it could be argued that it\nis not a bug that it does not honour the usual -- convention, even though\nI am tempted to think it is of a very dark shade of gray. It is certainly\nsomething that we would have done differently if we were implementing the\ncommand today.\n\n\"Fixing\" ls-tree would be trivial to ignore the first \"--\" if it precedes\nother pathspecs (see below), but the command is a plumbing, and such a\nchange will break existing scripts that have relied on the existing\nbehaviour since 2005, so I do not think it is worth the risk of causing\nsuch silent breakages to them. Besides, with such a \"fix\", fixing of user\nscripts will become much more cumbersome, as they need to detect the\nversion of git and drive ls-tree differently.\n\n\n builtin-ls-tree.c | 6 ++++++\n 1 files changed, 6 insertions(+), 0 deletions(-)\n\ndiff --git a/builtin-ls-tree.c b/builtin-ls-tree.c\nindex 22008df..08c4307 100644\n--- a/builtin-ls-tree.c\n+++ b/builtin-ls-tree.c\n@@ -186,6 +186,12 @@ int cmd_ls_tree(int argc, const char **argv, const char *prefix)\n \tif (get_sha1(argv[1], sha1))\n \t\tdie(\"Not a valid object name %s\", argv[1]);\n \n+\tif (3 < argc && !strcmp(argv[2], \"--\")) {\n+\t\t/* ls-tree -- pathspec */\n+\t\targc--;\n+\t\targv++;\n+\t\twarning(\"ignoring -- in 'ls-tree -- '\");\n+\t}\n \tpathspec = get_pathspec(prefix, argv + 2);\n \ttree = parse_tree_indirect(sha1);\n \tif (!tree)\n"},{"id":"109790","messageId":"20090329215651.GA4355@dcvr.yhbt.net","threadId":"18570","inReplyTo":"7v8wmoqdc1.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] git-svn: fix ls-tree usage with dash-prefixed paths","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2009-03-29T21:56:51Z","receivedAt":"2009-03-29T21:56:51Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Junio C Hamano wrote:\n> Eric Wong writes:\n> \n> > To find the blob object name given a tree and pathname, we were\n> > incorrectly calling \"git ls-tree\" with a \"--\" argument followed\n> > by the pathname of the file we wanted to get.\n> >\n> > git ls-tree -- --dashed/path/name.c\n> >\n> > Unlike many command-line interfaces, the \"--\" alone does not\n> > symbolize the end of non-option arguments on the command-line.\n> >\n> > ls-tree interprets the \"--\" as a prefix to match against, thus\n> > the entire contents of the --dashed/* hierarchy would be\n> > returned because the \"--\" matches \"--dashed\" and every path\n> > under it.\n> \n> The above makes only half a sense to me. In an empty directory:\n\nAh, I think you missed this line:\n\n\"the entire contents of the --dashed/* hierarchy would be\"\n\n> $ git init\n> Initialized empty Git repository in /tmp/empty/.git\n> $ mkdir -p ./--dashed/path\n> $ >./--dashed/path/name\n\n# Add a second file\n\t>./--dashed/path/ame\n\n> $ git add .\n> $ git ls-files\n> --dashed/path/name\n> $ git commit -a -m initial\n> [master (root-commit) cd44284] initial\n> 0 files changed, 0 insertions(+), 0 deletions(-)\n> create mode 100644 --dashed/path/name\n> $ git ls-tree HEAD^{tree} --\n> $ git ls-tree HEAD^{tree} -- --dashed/path/name\n> 100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\t--dashed/path/name\n> $ mkdir ./--\n> $ >./--/eman\n> $ git add .\n> $ git commit -m second\n> [master 80f8ef9] second\n> 0 files changed, 0 insertions(+), 0 deletions(-)\n> create mode 100644 --/eman\n> $ git ls-tree HEAD^{tree} -- --dashed/path\n> 100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\t--/eman\n> 040000 tree 23e59e0c91294c39ac7c5a2e39efb01d878de9a0\t--dashed/path\n\nThis is similar to the problem I was experiencing.\n\n> $ exit\n> \n> Perhaps the problem repository had a pathname that is exactly -- (in\n> addition to --dashed/), and ls-tree emitted everything under --/\n> hierarchy? In other words, your fix to git-svn may be correct and I am\n> reading your problem description above incorrectly?\n\nI think so.\n\n> As the command always takes exactly one tree, it could be argued that it\n> is not a bug that it does not honour the usual -- convention, even though\n> I am tempted to think it is of a very dark shade of gray. It is certainly\n> something that we would have done differently if we were implementing the\n> command today.\n\nWell, if somebody had a path in their repo called \"--full-name\" then it\nwould certainly be ambiguous and respecting \"--\" would help. Something\nwe should definitely go back and fix if we have time travel[1]\n\n> \"Fixing\" ls-tree would be trivial to ignore the first \"--\" if it precedes\n> other pathspecs (see below), but the command is a plumbing, and such a\n> change will break existing scripts that have relied on the existing\n> behaviour since 2005, so I do not think it is worth the risk of causing\n> such silent breakages to them. Besides, with such a \"fix\", fixing of user\n> scripts will become much more cumbersome, as they need to detect the\n> version of git and drive ls-tree differently.\n\nI concur completely. I didn't propose a \"fix\" to ls-tree for exactly\nthe reasons you stated.\n\n\n[1] But if we had time travel we could just release git before any other\nSCM and hopefully not have to deal with SVN at all :)\n\n-- \nEric Wong\n"},{"id":"109832","messageId":"20090330052817.GB2681@atjola.homenet","threadId":"18570","inReplyTo":"7v8wmoqdc1.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] git-svn: fix ls-tree usage with dash-prefixed paths","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-03-30T05:28:17Z","receivedAt":"2009-03-30T05:28:17Z","isPatch":true,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2009.03.29 13:33:02 -0700, Junio C Hamano wrote:\n> Eric Wong writes:\n> \n> > To find the blob object name given a tree and pathname, we were\n> > incorrectly calling \"git ls-tree\" with a \"--\" argument followed\n> > by the pathname of the file we wanted to get.\n> >\n> > git ls-tree -- --dashed/path/name.c\n> >\n> > Unlike many command-line interfaces, the \"--\" alone does not\n> > symbolize the end of non-option arguments on the command-line.\n> >\n> > ls-tree interprets the \"--\" as a prefix to match against, thus\n> > the entire contents of the --dashed/* hierarchy would be\n> > returned because the \"--\" matches \"--dashed\" and every path\n> > under it.\n> \n> The above makes only half a sense to me. In an empty directory:\n> \n> $ git init\n> Initialized empty Git repository in /tmp/empty/.git\n> $ mkdir -p ./--dashed/path\n> $ >./--dashed/path/name\n> $ git add .\n> $ git ls-files\n> --dashed/path/name\n> $ git commit -a -m initial\n> [master (root-commit) cd44284] initial\n> 0 files changed, 0 insertions(+), 0 deletions(-)\n> create mode 100644 --dashed/path/name\n> $ git ls-tree HEAD^{tree} --\n> $ git ls-tree HEAD^{tree} -- --dashed/path/name\n> 100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\t--dashed/path/name\n> $ mkdir ./--\n> $ >./--/eman\n> $ git add .\n> $ git commit -m second\n> [master 80f8ef9] second\n> 0 files changed, 0 insertions(+), 0 deletions(-)\n> create mode 100644 --/eman\n> $ git ls-tree HEAD^{tree} -- --dashed/path\n> 100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\t--/eman\n> 040000 tree 23e59e0c91294c39ac7c5a2e39efb01d878de9a0\t--dashed/path\n> $ exit\n> \n> Perhaps the problem repository had a pathname that is exactly -- (in\n> addition to --dashed/), and ls-tree emitted everything under --/\n> hierarchy? In other words, your fix to git-svn may be correct and I am\n> reading your problem description above incorrectly?\n\nYour test case is flawed, because you only have a single path in\n--dashed/\n\nInitialized empty Git repository in /home/doener/test/.git/\n$ mkdir ./--dashed\n$ touch ./--dashed/{1,2}\n$ git add .\n$ git ls-files\n--dashed/1\n--dashed/2\n$ git commit -m init\n[master (root-commit) ae7cd83] init\n 0 files changed, 0 insertions(+), 0 deletions(-)\n create mode 100644 --dashed/1\n create mode 100644 --dashed/2\n$ git ls-tree HEAD^{tree}\n040000 tree f353b342b53872c6a510229524f819c4fe0d5c1b\t--dashed\n$ git ls-tree HEAD^{tree} --\n$ git ls-tree HEAD^{tree} -- --dashed\n040000 tree f353b342b53872c6a510229524f819c4fe0d5c1b\t--dashed\n$ git ls-tree HEAD^{tree} -- --dashed/\n100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\t--dashed/1\n100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\t--dashed/2\n$ git ls-tree HEAD^{tree} -- --dashed/1\n100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\t--dashed/1\n100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\t--dashed/2\n\nOr even more weird (at least to me):\n\nInitialized empty Git repository in /home/doener/test/.git/\n$ mkdir foo fab\n$ touch {foo,fab}/{1,2}\n$ git add .\n$ git commit -m init\n[master (root-commit) fdb7bb3] init\n 0 files changed, 0 insertions(+), 0 deletions(-)\n create mode 100644 fab/1\n create mode 100644 fab/2\n create mode 100644 foo/1\n create mode 100644 foo/2\n$ git ls-files foo/1 fab/1\nfab/1\nfoo/1\n$ git ls-files foo/1 fab/1 f\nfab/1\nfoo/1\n$ git ls-tree HEAD^{tree} foo/1 fab/1\n100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\tfab/1\n100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\tfoo/1\n$ git ls-tree HEAD^{tree} foo/1 fab/1 f\n100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\tfab/1\n100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\tfab/2\n100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\tfoo/1\n100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391\tfoo/2\n\nSo if you go into some tree, any additional pattern that is a prefix of\nthe tree name will match the tree and its contents.\n\nBjörn\n"},{"id":"109835","messageId":"7v3acvldc7.fsf@gitster.siamese.dyndns.org","threadId":"18570","inReplyTo":"20090329215651.GA4355@dcvr.yhbt.net","subject":"Re: [PATCH] git-svn: fix ls-tree usage with dash-prefixed paths","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-03-30T06:44:08Z","receivedAt":"2009-03-30T06:44:08Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Wong writes:\n\n> Junio C Hamano wrote:\n>> Eric Wong writes:\n>> \n>> > To find the blob object name given a tree and pathname, we were\n>> > incorrectly calling \"git ls-tree\" with a \"--\" argument followed\n>> > by the pathname of the file we wanted to get.\n>> >\n>> > git ls-tree -- --dashed/path/name.c\n>> >\n>> > Unlike many command-line interfaces, the \"--\" alone does not\n>> > symbolize the end of non-option arguments on the command-line.\n>> >\n>> > ls-tree interprets the \"--\" as a prefix to match against, thus\n>> > the entire contents of the --dashed/* hierarchy would be\n>> > returned because the \"--\" matches \"--dashed\" and every path\n>> > under it.\n>> \n>> The above makes only half a sense to me. In an empty directory:\n>\n> Ah, I think you missed this line:\n>\n> \"the entire contents of the --dashed/* hierarchy would be\"\n\nActually, that was what I was trying to demonstrate to be false. Notice\nthe empty output from the first ls-tree with only -- and no other pathspec\non the command line. \"--\" should not match \"--dashed/*\" anything (but\nalso notice that I said \"should\" here).\n\n>> $ git init\n>> Initialized empty Git repository in /tmp/empty/.git\n>> $ mkdir -p ./--dashed/path\n>> $ >./--dashed/path/name\n>\n> # Add a second file\n> \t>./--dashed/path/ame\n\nI think that is an independent bug. Not just \"--\" but it appears \"--d\"\nseems to hit it (and this is an ancient bug---even v1.0.0 seems to have\nit).\n\n> [1] But if we had time travel we could just release git before any other\n> SCM and hopefully not have to deal with SVN at all :)\n\n;-)\n\nI suspect that ls-tree needs a fix, not about \"--\" but about the pathspec\nfiltering. It appears that the part that decides if a subtree is worth\ntraversing into uses the correct \"is a pathspec pattern match leading path\ncomponents?\" semantics (i.e. \"--dashed\" matches but \"--\" doesn't), but\nafter traversing into subtrees, the part that emits the output uses a\nbroken semantics \"does the path have any pathspec patter as its prefix?\"\nIt shouldn't check for \"prefix\", but for \"leading path components\", in\nother words, the match must happen at directory boundaries.\n\nAnd I do not think *this* bug is too late to fix. We should fix it.\n"},{"id":"109845","messageId":"83dfc36c0903300026j2e3cb8acm86bb47b5ca7014e1@mail.gmail.com","threadId":"18570","inReplyTo":"20090329060858.GB15773@dcvr.yhbt.net","subject":"Re: svn clone Checksum mismatch question","fromName":"Anton Gyllenberg","fromEmail":"anton@iki.fi","sentAt":"2009-03-30T07:26:40Z","receivedAt":"2009-03-30T07:26:40Z","isPatch":false,"sender":{"key":"anton@iki.fi","avatar":null},"body":"Makes perfect sense now that you explain it and I see the patch. I\nactually tried debugging it a bit but got lost in the perl SVN code.\nThank you Eric for figuring this out and for all your work on git-svn\nin general!\n\n> I guess few folks in the UNIX world are crazy enough to make pathnames\n> prefixed with dashes :)\n\nI guess they wouldn't call the project Twisted without reason :-).\n\nAnton\n"},{"id":"109928","messageId":"20090330174151.GA32728@dcvr.yhbt.net","threadId":"18570","inReplyTo":"7v3acvldc7.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] git-svn: fix ls-tree usage with dash-prefixed paths","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2009-03-30T17:41:51Z","receivedAt":"2009-03-30T17:41:51Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Junio C Hamano wrote:\n> Eric Wong writes:\n> \n> > Junio C Hamano wrote:\n> >> Eric Wong writes:\n> >> \n> >> > To find the blob object name given a tree and pathname, we were\n> >> > incorrectly calling \"git ls-tree\" with a \"--\" argument followed\n> >> > by the pathname of the file we wanted to get.\n> >> >\n> >> > git ls-tree -- --dashed/path/name.c\n> >> >\n> >> > Unlike many command-line interfaces, the \"--\" alone does not\n> >> > symbolize the end of non-option arguments on the command-line.\n> >> >\n> >> > ls-tree interprets the \"--\" as a prefix to match against, thus\n> >> > the entire contents of the --dashed/* hierarchy would be\n> >> > returned because the \"--\" matches \"--dashed\" and every path\n> >> > under it.\n> >> \n> >> The above makes only half a sense to me. In an empty directory:\n> >\n> > Ah, I think you missed this line:\n> >\n> > \"the entire contents of the --dashed/* hierarchy would be\"\n> \n> Actually, that was what I was trying to demonstrate to be false. Notice\n> the empty output from the first ls-tree with only -- and no other pathspec\n> on the command line. \"--\" should not match \"--dashed/*\" anything (but\n> also notice that I said \"should\" here).\n> \n> >> $ git init\n> >> Initialized empty Git repository in /tmp/empty/.git\n> >> $ mkdir -p ./--dashed/path\n> >> $ >./--dashed/path/name\n> >\n> > # Add a second file\n> > \t>./--dashed/path/ame\n> \n> I think that is an independent bug. Not just \"--\" but it appears \"--d\"\n> seems to hit it (and this is an ancient bug---even v1.0.0 seems to have\n> it).\n\n> I suspect that ls-tree needs a fix, not about \"--\" but about the pathspec\n> filtering. It appears that the part that decides if a subtree is worth\n> traversing into uses the correct \"is a pathspec pattern match leading path\n> components?\" semantics (i.e. \"--dashed\" matches but \"--\" doesn't), but\n> after traversing into subtrees, the part that emits the output uses a\n> broken semantics \"does the path have any pathspec patter as its prefix?\"\n> It shouldn't check for \"prefix\", but for \"leading path components\", in\n> other words, the match must happen at directory boundaries.\n> \n> And I do not think *this* bug is too late to fix. We should fix it.\n\n>From the ls-tree documentation, I was under the impression that \"--\"\nmatching \"--dashed\" was intended:\n\n When paths are given, show them (note that this isn't really raw\n pathnames, but rather a list of patterns to match).\n\nIt doesn't make sense to me match like this, either; but I do think it\nwas intended and it will break things if people depend on the\nexisting behavior.\n\n-- \nEric Wong\n"},{"id":"109932","messageId":"7vy6umdgxq.fsf@gitster.siamese.dyndns.org","threadId":"18570","inReplyTo":"20090330174151.GA32728@dcvr.yhbt.net","subject":"Re: [PATCH] git-svn: fix ls-tree usage with dash-prefixed paths","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-03-30T18:05:53Z","receivedAt":"2009-03-30T18:05:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Wong writes:\n\n> Junio C Hamano wrote:\n>\n>> I think that is an independent bug. Not just \"--\" but it appears \"--d\"\n>> seems to hit it (and this is an ancient bug---even v1.0.0 seems to have\n>> it).\n>\n>> I suspect that ls-tree needs a fix, not about \"--\" but about the pathspec\n>> filtering. It appears that the part that decides if a subtree is worth\n>> traversing into uses the correct \"is a pathspec pattern match leading path\n>> components?\" semantics (i.e. \"--dashed\" matches but \"--\" doesn't), but\n>> after traversing into subtrees, the part that emits the output uses a\n>> broken semantics \"does the path have any pathspec patter as its prefix?\"\n>> It shouldn't check for \"prefix\", but for \"leading path components\", in\n>> other words, the match must happen at directory boundaries.\n>> \n>> And I do not think *this* bug is too late to fix. We should fix it.\n>\n> From the ls-tree documentation, I was under the impression that \"--\"\n> matching \"--dashed\" was intended:\n>\n> When paths are given, show them (note that this isn't really raw\n> pathnames, but rather a list of patterns to match).\n>\n> It doesn't make sense to me match like this, either; but I do think it\n> was intended and it will break things if people depend on the\n> existing behavior.\n\nOk, but then the decision to descend into --dashed should be consistent\nwith that policy, no? Right now, it appears that giving \"--\" alone says\n\"Anything under --dashed can never match that pattern, so I wouldn't\nbother recursing into it\".\n"},{"id":"109949","messageId":"20090330225834.GA24254@dcvr.yhbt.net","threadId":"18570","inReplyTo":"7vy6umdgxq.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] git-svn: fix ls-tree usage with dash-prefixed paths","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2009-03-30T22:58:34Z","receivedAt":"2009-03-30T22:58:34Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Junio C Hamano wrote:\n> Eric Wong writes:\n> \n> > Junio C Hamano wrote:\n> >\n> >> I think that is an independent bug. Not just \"--\" but it appears \"--d\"\n> >> seems to hit it (and this is an ancient bug---even v1.0.0 seems to have\n> >> it).\n> >\n> >> I suspect that ls-tree needs a fix, not about \"--\" but about the pathspec\n> >> filtering. It appears that the part that decides if a subtree is worth\n> >> traversing into uses the correct \"is a pathspec pattern match leading path\n> >> components?\" semantics (i.e. \"--dashed\" matches but \"--\" doesn't), but\n> >> after traversing into subtrees, the part that emits the output uses a\n> >> broken semantics \"does the path have any pathspec patter as its prefix?\"\n> >> It shouldn't check for \"prefix\", but for \"leading path components\", in\n> >> other words, the match must happen at directory boundaries.\n> >> \n> >> And I do not think *this* bug is too late to fix. We should fix it.\n> >\n> > From the ls-tree documentation, I was under the impression that \"--\"\n> > matching \"--dashed\" was intended:\n> >\n> > When paths are given, show them (note that this isn't really raw\n> > pathnames, but rather a list of patterns to match).\n> >\n> > It doesn't make sense to me match like this, either; but I do think it\n> > was intended and it will break things if people depend on the\n> > existing behavior.\n> \n> Ok, but then the decision to descend into --dashed should be consistent\n> with that policy, no? Right now, it appears that giving \"--\" alone says\n> \"Anything under --dashed can never match that pattern, so I wouldn't\n> bother recursing into it\".\n\nRight. Except in the case when there are multiple files inside --dashed/\nas Björn's email illustrated. So there seems to be a bug in the way\nthe number of files inside --dashed/ affects what \"--\" does when used\nwith \"--dashed/1\" (if --dashed/2 also exists). Very confusing :x\n\n-- \nEric Wong\n"},{"id":"109979","messageId":"20090331071100.GA3307@atjola.homenet","threadId":"18570","inReplyTo":"20090330225834.GA24254@dcvr.yhbt.net","subject":"Re: [PATCH] git-svn: fix ls-tree usage with dash-prefixed paths","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-03-31T07:11:00Z","receivedAt":"2009-03-31T07:11:00Z","isPatch":true,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2009.03.30 15:58:34 -0700, Eric Wong wrote:\n> Junio C Hamano wrote:\n> > Eric Wong writes:\n> > > From the ls-tree documentation, I was under the impression that \"--\"\n> > > matching \"--dashed\" was intended:\n> > >\n> > > When paths are given, show them (note that this isn't really raw\n> > > pathnames, but rather a list of patterns to match).\n> > >\n> > > It doesn't make sense to me match like this, either; but I do think it\n> > > was intended and it will break things if people depend on the\n> > > existing behavior.\n\nI guess that paragraph was meant to explain why \"git ls-tree HEAD\nDocumentation\" and \"git ls-tree HEAD Documentation/\" give different\nresults. The first one shows the entry for the tree object, while the\nsecond one shows the contents of the tree object. In contrast to \"ls\"\nwhich would descend into the directory in both cases.\n\n> > Ok, but then the decision to descend into --dashed should be consistent\n> > with that policy, no? Right now, it appears that giving \"--\" alone says\n> > \"Anything under --dashed can never match that pattern, so I wouldn't\n> > bother recursing into it\".\n> \n> Right. Except in the case when there are multiple files inside --dashed/\n> as Björn's email illustrated. So there seems to be a bug in the way\n> the number of files inside --dashed/ affects what \"--\" does when used\n> with \"--dashed/1\" (if --dashed/2 also exists). Very confusing :x\n\nIt's not the number of files that matters. With just one file, you just\ndon't notice the buggy behaviour, because showing all files is the same\nas showing the specified file.\n\nAnd interestingly, the problem doesn't seem to be in\nshow_tree/show_recursive, but in match_tree_entry.\n\nWith \"git ls-tree HEAD gitweb/git-favicon.png g\" we descend into gitweb/\nand at some point we get:\n\nmatch = \"g\"\nbase = \"gitweb/\"\n\nAnd we have:\nif (baselen >= matchlen) {\n\tif (strncmp(base, match, matchlen))\n\t\tcontinue;\n\t/* The base is a subdirectory of a path which was specified */\n\treturn 1;\n}\n\nSo we return 1 there. The code doesn't do what the comment says, so I\nguess we can be pretty sure that the behaviour is not intended.\n\nBjörn\n"},{"id":"109983","messageId":"20090331073147.GB3307@atjola.homenet","threadId":"18570","inReplyTo":"20090331071100.GA3307@atjola.homenet","subject":"Re: [PATCH] git-svn: fix ls-tree usage with dash-prefixed paths","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-03-31T07:31:47Z","receivedAt":"2009-03-31T07:31:47Z","isPatch":true,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2009.03.31 09:11:00 +0200, Björn Steinbrink wrote:\n> And interestingly, the problem doesn't seem to be in\n> show_tree/show_recursive, but in match_tree_entry.\n> \n> With \"git ls-tree HEAD gitweb/git-favicon.png g\" we descend into gitweb/\n> and at some point we get:\n> \n> match = \"g\"\n> base = \"gitweb/\"\n> \n> And we have:\n> if (baselen >= matchlen) {\n> \tif (strncmp(base, match, matchlen))\n> \t\tcontinue;\n> \t/* The base is a subdirectory of a path which was specified */\n> \treturn 1;\n> }\n> \n> So we return 1 there. The code doesn't do what the comment says, so I\n> guess we can be pretty sure that the behaviour is not intended.\n\nYup, it's in match_tree_entry, you get the same thing with git show.\nWith git.git, you can try with:\n\ngit show 4fa535a -- Documentation/git-merge.txt D\n\nI'll try to get a patch done, if noone beats me to it.\n\nBjörn\n"},{"id":"109988","messageId":"20090331094107.GC3307@atjola.homenet","threadId":"18570","inReplyTo":"20090331073147.GB3307@atjola.homenet","subject":"Re: [PATCH] git-svn: fix ls-tree usage with dash-prefixed paths","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-03-31T09:41:07Z","receivedAt":"2009-03-31T09:41:07Z","isPatch":true,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2009.03.31 09:31:47 +0200, Björn Steinbrink wrote:\n> On 2009.03.31 09:11:00 +0200, Björn Steinbrink wrote:\n> > And interestingly, the problem doesn't seem to be in\n> > show_tree/show_recursive, but in match_tree_entry.\n> > \n> > With \"git ls-tree HEAD gitweb/git-favicon.png g\" we descend into gitweb/\n> > and at some point we get:\n> > \n> > match = \"g\"\n> > base = \"gitweb/\"\n> > \n> > And we have:\n> > if (baselen >= matchlen) {\n> > \tif (strncmp(base, match, matchlen))\n> > \t\tcontinue;\n> > \t/* The base is a subdirectory of a path which was specified */\n> > \treturn 1;\n> > }\n> > \n> > So we return 1 there. The code doesn't do what the comment says, so I\n> > guess we can be pretty sure that the behaviour is not intended.\n> \n> Yup, it's in match_tree_entry, you get the same thing with git show.\n> With git.git, you can try with:\n> \n> git show 4fa535a -- Documentation/git-merge.txt D\n> \n> I'll try to get a patch done, if noone beats me to it.\n\nAh, crap, \"git show\" actually uses a different function,\ntree_entry_interesting, which happens to have the same problem, but\nneeds a slightly different fix.\n\nBjörn\n"},{"id":"110015","messageId":"20090331150501.GA11446@atjola.homenet","threadId":"18570","inReplyTo":"20090331094107.GC3307@atjola.homenet","subject":"[PATCH] tree_entry_interesting: Only recurse when the pathspec is a leading path component","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-03-31T15:05:01Z","receivedAt":"2009-03-31T15:05:01Z","isPatch":true,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"Previously the code did a simple prefix match, which means that it\ntreated for example \"foo/\" as a subdirectory of \"f\".\n\nSigned-off-by: Björn Steinbrink \n---\nI'm not exactly happy with the commit message, but that's the best I\ncould come up with. Probably shows how little I know about that code :-/\nThe test suite still passes and I'll try to provide a new testcase\ntonight or tommorow.\n\n tree-diff.c | 12 +++++++++---\n 1 files changed, 9 insertions(+), 3 deletions(-)\n\ndiff --git a/tree-diff.c b/tree-diff.c\nindex 9f67af6..b05d0f4 100644\n--- a/tree-diff.c\n+++ b/tree-diff.c\n@@ -118,10 +118,16 @@ static int tree_entry_interesting(struct tree_desc *desc, const char *base, int\n \t\t\t\tcontinue;\n \n \t\t\t/*\n-\t\t\t * The base is a subdirectory of a path which\n-\t\t\t * was specified, so all of them are interesting.\n+\t\t\t * If the base is a subdirectory of a path which\n+\t\t\t * was specified, all of them are interesting.\n \t\t\t */\n-\t\t\treturn 2;\n+\t\t\tif (!matchlen ||\n+\t\t\t base[matchlen] == '/' ||\n+\t\t\t match[matchlen - 1] == '/')\n+\t\t\t\treturn 2;\n+\n+\t\t\t/* Just a random prefix match */\n+\t\t\tcontinue;\n \t\t}\n \n \t\t/* Does the base match? */\n-- \n1.6.2.1.426.gf94cd\n"},{"id":"110189","messageId":"7vbprfn0ai.fsf@gitster.siamese.dyndns.org","threadId":"18570","inReplyTo":"20090331150501.GA11446@atjola.homenet","subject":"Re: [PATCH] tree_entry_interesting: Only recurse when the pathspec is a leading path component","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-04-02T04:32:05Z","receivedAt":"2009-04-02T04:32:05Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Björn Steinbrink writes:\n\n> Previously the code did a simple prefix match, which means that it\n> treated for example \"foo/\" as a subdirectory of \"f\".\n>\n> Signed-off-by: Björn Steinbrink \n> ---\n> I'm not exactly happy with the commit message, but that's the best I\n> could come up with. Probably shows how little I know about that code :-/\n> The test suite still passes and I'll try to provide a new testcase\n> tonight or tommorow.\n\nI'm planning to queue this.\n\nFrom: Björn Steinbrink \nDate: Tue, 31 Mar 2009 17:05:01 +0200\nSubject: [PATCH] tree_entry_interesting: a pathspec only matches at directory boundary\n\nPreviously the code did a simple prefix match, which means that a\npath in a directory \"frotz/\" would have matched with pathspec \"f\".\n\nSigned-off-by: Björn Steinbrink \nSigned-off-by: Junio C Hamano \n---\n t/t4010-diff-pathspec.sh | 8 ++++++++\n tree-diff.c | 12 +++++++++---\n 2 files changed, 17 insertions(+), 3 deletions(-)\n\ndiff --git a/t/t4010-diff-pathspec.sh b/t/t4010-diff-pathspec.sh\nindex ad3d9e4..4c4c8b1 100755\n--- a/t/t4010-diff-pathspec.sh\n+++ b/t/t4010-diff-pathspec.sh\n@@ -62,4 +62,12 @@ test_expect_success \\\n 'git diff-index --cached $tree -- file0/ >current &&\n compare_diff_raw current expected'\n \n+test_expect_success 'diff-tree pathspec' '\n+\ttree2=$(git write-tree) &&\n+\techo \"$tree2\" &&\n+\tgit diff-tree -r --name-only $tree $tree2 -- pa path1/a >current &&\n+\t>expected &&\n+\ttest_cmp expected current\n+'\n+\n test_done\ndiff --git a/tree-diff.c b/tree-diff.c\nindex 9f67af6..b05d0f4 100644\n--- a/tree-diff.c\n+++ b/tree-diff.c\n@@ -118,10 +118,16 @@ static int tree_entry_interesting(struct tree_desc *desc, const char *base, int\n \t\t\t\tcontinue;\n \n \t\t\t/*\n-\t\t\t * The base is a subdirectory of a path which\n-\t\t\t * was specified, so all of them are interesting.\n+\t\t\t * If the base is a subdirectory of a path which\n+\t\t\t * was specified, all of them are interesting.\n \t\t\t */\n-\t\t\treturn 2;\n+\t\t\tif (!matchlen ||\n+\t\t\t base[matchlen] == '/' ||\n+\t\t\t match[matchlen - 1] == '/')\n+\t\t\t\treturn 2;\n+\n+\t\t\t/* Just a random prefix match */\n+\t\t\tcontinue;\n \t\t}\n \n \t\t/* Does the base match? */\n"},{"id":"110190","messageId":"7vwsa3llac.fsf_-_@gitster.siamese.dyndns.org","threadId":"18570","inReplyTo":"7vbprfn0ai.fsf@gitster.siamese.dyndns.org","subject":"[PATCH] match_tree_entry(): a pathspec only matches at directory boundaries","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-04-02T04:41:31Z","receivedAt":"2009-04-02T04:41:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Previously the code did a simple prefix match, which means that a path in\na directory \"frotz/\" would have matched with pathspec \"f\".\n\nSigned-off-by: Junio C Hamano \n---\n\n * And this is a companion patch to fix ls-tree. The test case uses a\n tree that has path3/1.txt and path3/2.txt in it.\n\n The bug Eric diagnosed and worked around in git-svn makes the current\n code show these two paths when pathspec \"pa\" and \"path3/a\" are given.\n The presense of \"path3/a\" makes the tree walker traverse down to path3\n subtree (in case something that matches \"a\" is in there---this is a\n correct behaviour), but then in that subtree, \"pa\" incorrectly matches\n \"path3/1.txt\".\n\n This logic dates back to 0ca14a5 (Start adding interfaces to read in\n partial trees, 2005-07-14). I think it is just a simple oversight and\n we should fix it.\n\n t/t3101-ls-tree-dirname.sh | 6 ++++++\n tree.c | 8 ++++++--\n 2 files changed, 12 insertions(+), 2 deletions(-)\n\ndiff --git a/t/t3101-ls-tree-dirname.sh b/t/t3101-ls-tree-dirname.sh\nindex 4dd7d12..51cb4a3 100755\n--- a/t/t3101-ls-tree-dirname.sh\n+++ b/t/t3101-ls-tree-dirname.sh\n@@ -135,4 +135,10 @@ test_expect_success \\\n EOF\n test_output'\n \n+test_expect_success 'ls-tree filter is leading path match' '\n+\tgit ls-tree $tree pa path3/a >current &&\n+\t>expected &&\n+\ttest_output\n+'\n+\n test_done\ndiff --git a/tree.c b/tree.c\nindex 03e782a..d82a047 100644\n--- a/tree.c\n+++ b/tree.c\n@@ -60,8 +60,12 @@ static int match_tree_entry(const char *base, int baselen, const char *path, uns\n \t\t\t/* If it doesn't match, move along... */\n \t\t\tif (strncmp(base, match, matchlen))\n \t\t\t\tcontinue;\n-\t\t\t/* The base is a subdirectory of a path which was specified. */\n-\t\t\treturn 1;\n+\t\t\t/* pathspecs match only at the directory boundaries */\n+\t\t\tif (!matchlen ||\n+\t\t\t base[matchlen] == '/' ||\n+\t\t\t match[matchlen - 1] == '/')\n+\t\t\t\treturn 1;\n+\t\t\tcontinue;\n \t\t}\n \n \t\t/* Does the base match? */\n-- \n1.6.2.1.483.gcc994\n"},{"id":"110209","messageId":"20090402113851.GA12675@atjola.homenet","threadId":"18570","inReplyTo":"7vbprfn0ai.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] tree_entry_interesting: Only recurse when the pathspec is a leading path component","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-04-02T11:38:51Z","receivedAt":"2009-04-02T11:38:51Z","isPatch":true,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2009.04.01 21:32:05 -0700, Junio C Hamano wrote:\n> I'm planning to queue this.\n> [Improved version of my patch]\n\nAh, thanks. Got busy with other stuff, and tried to fix another related\nbug in ls-tree which made me forget to send the match_tree_entry fix :-/\n\nThat other ls-tree bug is with recursing into subdirectories, because\nmatch_tree_entry matches even when the base is a subdirectory, but\nls-tree doesn't actually want that behaviour, I think. For example:\n\n$ git ls-tree --abbrev HEAD git-gui/macosx/AppMain.tcl\n100644 blob ddbe633\tgit-gui/macosx/AppMain.tcl\n\n$ git ls-tree --abbrev HEAD git-gui/\n100644 blob f96112d\tgit-gui/.gitattributes\n100644 blob 6483b21\tgit-gui/.gitignore\n100755 blob b3f937e\tgit-gui/GIT-VERSION-GEN\n100644 blob 3ad8a21\tgit-gui/Makefile\n100755 blob 12e117e\tgit-gui/git-gui--askpass\n100755 blob e018e07\tgit-gui/git-gui.sh\n040000 tree f723285\tgit-gui/lib\n040000 tree 73f3c34\tgit-gui/macosx\n040000 tree 11cd1a0\tgit-gui/po\n040000 tree 144728d\tgit-gui/windows\n\n$ git ls-tree --abbrev HEAD git-gui/macosx/AppMain.tcl git-gui/\n100644 blob f96112d\tgit-gui/.gitattributes\n100644 blob 6483b21\tgit-gui/.gitignore\n100755 blob b3f937e\tgit-gui/GIT-VERSION-GEN\n100644 blob 3ad8a21\tgit-gui/Makefile\n100755 blob 12e117e\tgit-gui/git-gui--askpass\n100755 blob e018e07\tgit-gui/git-gui.sh\n040000 tree f723285\tgit-gui/lib\n100644 blob ddbe633\tgit-gui/macosx/AppMain.tcl\n100644 blob b3bf15f\tgit-gui/macosx/Info.plist\n100644 blob 77d88a7\tgit-gui/macosx/git-gui.icns\n040000 tree 11cd1a0\tgit-gui/po\n040000 tree 144728d\tgit-gui/windows\n\nThe last ls-tree shows all entries from git-gui/macosx/, beacuse the\nfirst pattern makes it descend into that tree and all entries are\nmatched by the git-gui/ prefix. So the combined ls-tree shows more than\nwhat the individual calls show. Seems wrong to me, but I'm unsure how to\ntackle that, assuming that match_tree_entry is right in allowing any\nbase that's a subdirectory of a specified pathspec.\n\nBjörn\n"},{"id":"110227","messageId":"alpine.LFD.2.00.0904020923180.4130@localhost.localdomain","threadId":"18570","inReplyTo":"7vwsa3llac.fsf_-_@gitster.siamese.dyndns.org","subject":"Re: [PATCH] match_tree_entry(): a pathspec only matches at directory boundaries","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2009-04-02T16:36:46Z","receivedAt":"2009-04-02T16:36:46Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Wed, 1 Apr 2009, Junio C Hamano wrote:\n>\n> Previously the code did a simple prefix match, which means that a path in\n> a directory \"frotz/\" would have matched with pathspec \"f\".\n> \n> Signed-off-by: Junio C Hamano \n\nI have this suspicion that we should be able to write it more readably, \nbut yes:\n\n\tAcked-by: Linus Torvalds \n\nbecause the current code is clearly buggy.\n\n\t\tLinus\n"},{"id":"110294","messageId":"7veiw9aeme.fsf@gitster.siamese.dyndns.org","threadId":"18570","inReplyTo":"20090402113851.GA12675@atjola.homenet","subject":"Re: [PATCH] tree_entry_interesting: Only recurse when the pathspec is a leading path component","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-04-03T16:25:29Z","receivedAt":"2009-04-03T16:25:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Björn Steinbrink writes:\n\n> On 2009.04.01 21:32:05 -0700, Junio C Hamano wrote:\n>> I'm planning to queue this.\n>> [Improved version of my patch]\n>\n> Ah, thanks. Got busy with other stuff, and tried to fix another related\n> bug in ls-tree which made me forget to send the match_tree_entry fix :-/\n>\n> That other ls-tree bug is with recursing into subdirectories, because\n> match_tree_entry matches even when the base is a subdirectory, but\n> ls-tree doesn't actually want that behaviour, I think. For example:\n>\n> $ git ls-tree --abbrev HEAD git-gui/macosx/AppMain.tcl\n> 100644 blob ddbe633\tgit-gui/macosx/AppMain.tcl\n>\n> $ git ls-tree --abbrev HEAD git-gui/\n> 100644 blob f96112d\tgit-gui/.gitattributes\n> 100644 blob 6483b21\tgit-gui/.gitignore\n> 100755 blob b3f937e\tgit-gui/GIT-VERSION-GEN\n> 100644 blob 3ad8a21\tgit-gui/Makefile\n> 100755 blob 12e117e\tgit-gui/git-gui--askpass\n> 100755 blob e018e07\tgit-gui/git-gui.sh\n> 040000 tree f723285\tgit-gui/lib\n> 040000 tree 73f3c34\tgit-gui/macosx\n> 040000 tree 11cd1a0\tgit-gui/po\n> 040000 tree 144728d\tgit-gui/windows\n>\n> $ git ls-tree --abbrev HEAD git-gui/macosx/AppMain.tcl git-gui/\n> 100644 blob f96112d\tgit-gui/.gitattributes\n> 100644 blob 6483b21\tgit-gui/.gitignore\n> 100755 blob b3f937e\tgit-gui/GIT-VERSION-GEN\n> 100644 blob 3ad8a21\tgit-gui/Makefile\n> 100755 blob 12e117e\tgit-gui/git-gui--askpass\n> 100755 blob e018e07\tgit-gui/git-gui.sh\n> 040000 tree f723285\tgit-gui/lib\n> 100644 blob ddbe633\tgit-gui/macosx/AppMain.tcl\n> 100644 blob b3bf15f\tgit-gui/macosx/Info.plist\n> 100644 blob 77d88a7\tgit-gui/macosx/git-gui.icns\n> 040000 tree 11cd1a0\tgit-gui/po\n> 040000 tree 144728d\tgit-gui/windows\n>\n> The last ls-tree shows all entries from git-gui/macosx/, beacuse the\n> first pattern makes it descend into that tree and all entries are\n> matched by the git-gui/ prefix. So the combined ls-tree shows more than\n> what the individual calls show. Seems wrong to me, but I'm unsure how to\n> tackle that, assuming that match_tree_entry is right in allowing any\n> base that's a subdirectory of a specified pathspec.\n\nWhat might be confusing is how ls-tree, when run without an explicit\noption -r (ecurse), comes up with candidates to show. With pathspecs, you\nare instructing it to recurse into subtrees as deep as needed to discover\nall the paths that can match them, and your first pathspec causes the\ncontents of macosx subtree to be looked at for this reason.\n\nPathspecs are always OR'ed together, never AND'ed, and your second\npathspec git-gui covers them, so I do not think it is a bug.\n\nIf we had a glob support in the \"leading path components\" pathspec matcher\nused by ls-tree/diff-tree, I would imagine you could do something like;\n\n\tls-tree --recurse-to='*/*/*' $other $pathspec\n\nto say \"recurse three levels, but do not use */*/* as a filter to decide\nwhat to show; the filters are only $other and $pathspec\", iow, using\n\n\t(AND \"*/*/*\" (OR $other $pathspec))\n\nsemantics. But I think this kind of complexity falls out of the \"useful\"\ncategory and into a \"mental exercise\" category.\n"}]}