{"thread":{"id":"29325","subject":"git grep doesn't follow symbolic link","startedAt":"2012-01-09T16:54:14Z","lastAt":"2012-01-17T06:44:51Z","messageCount":10,"participants":["Bertrand BENOIT","Ramkumar Ramachandra","Thomas Rast","Junio C Hamano","Nguyen Thai Ngoc Duy"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"182172","messageId":"CAPRVejc7xND_8Y=Pb5rYGEcaKYUaX7_WkSro-_EL8tTGxkfY3Q@mail.gmail.com","threadId":"29325","inReplyTo":null,"subject":"git grep doesn't follow symbolic link","fromName":"Bertrand BENOIT","fromEmail":"projettwk@users.sourceforge.net","sentAt":"2012-01-09T16:54:14Z","receivedAt":"2012-01-09T16:54:14Z","isPatch":false,"sender":{"key":"projettwk@users.sourceforge.net","avatar":null},"body":"Hi,\n\nI've not found information about that in documentation, so I do a report.\n\nWhen using git grep, symbolic links are not followed.\nIs it a wanted behavior ?\n\nI've tested with a symbolic link:\n - 'ignored'\n - NOT staged for commit\n - to be commited\n - commited\nAnytime -> no result when asking on symbolic link\n\nExample:\n# git grep foo mySrc\n-> OK answer [...]\n\n# ln -s mySrc test\n# git grep foo test\n-> KO: No answer\n\n# git add test\n# git grep foo test\n-> KO: No answer\n\n# git commit -m \"DO NOT PUSH\" test\n# git grep foo test\n-> KO: No answer\n\nBest Regards,\nBertrand\n"},{"id":"182211","messageId":"CALkWK0=-LZH4MYhX50v-RWpGA2r+6q50YxsKaOxc0mJ__yuK7g@mail.gmail.com","threadId":"29325","inReplyTo":"CAPRVejc7xND_8Y=Pb5rYGEcaKYUaX7_WkSro-_EL8tTGxkfY3Q@mail.gmail.com","subject":"Re: git grep doesn't follow symbolic link","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2012-01-10T05:56:36Z","receivedAt":"2012-01-10T05:56:36Z","isPatch":false,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Bertrand,\n\nBertrand BENOIT wrote:\n> When using git grep, symbolic links are not followed.\n> Is it a wanted behavior ?\n\nI'd imagine so: symbolic links are not portable across different file\nsystems; Git's internal representation of a symbolic link is a file\ncontaining the path of the file to be linked to.\n\n> I've not found information about that in documentation, so I do a report.\n\nHm, the description says:\n\nLook for specified patterns in the tracked files in the work tree, blobs\nregistered in the index file, or blobs in given tree objects.\n\nHm, \"tracked files in the work tree\" is definitely sub-optimal: \"blobs\ncorresponding to the tracked files in the work tree\" is probably\nbetter. Then again, I think the description is too cryptic for an\nend-user: do you have any suggestions? Have we mentioned how Git\nhandles symbolic links anywhere in the documentation? If not, where\nshould this information go?\n\n-- Ram\n"},{"id":"182232","messageId":"877h0zlvwd.fsf@thomas.inf.ethz.ch","threadId":"29325","inReplyTo":"CALkWK0=-LZH4MYhX50v-RWpGA2r+6q50YxsKaOxc0mJ__yuK7g@mail.gmail.com","subject":"Re: git grep doesn't follow symbolic link","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2012-01-10T10:00:34Z","receivedAt":"2012-01-10T10:00:34Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Ramkumar Ramachandra writes:\n\n> Hi Bertrand,\n>\n> Bertrand BENOIT wrote:\n>> When using git grep, symbolic links are not followed.\n>> Is it a wanted behavior ?\n>\n> I'd imagine so: symbolic links are not portable across different file\n> systems; Git's internal representation of a symbolic link is a file\n> containing the path of the file to be linked to.\n\nI'd actually welcome a fix to this general area, for an entirely\ndifferent reason. With bash and ordinary diff I can do things like\n\n diff -u <(ls) <(cd elsewhere && ls) | less\n\nBut I lose all the cute features of git-diff. I *could* say\n\n git diff --no-index <(ls) <(cd elsewhere && ls)\n\nand it helpfully tells me\n\n diff --git 1/dev/fd/63 2/dev/fd/62\n index 55ccbe5..d796c45 120000\n --- 1/dev/fd/63\n +++ 2/dev/fd/62\n @@ -1 +1 @@\n -pipe:[607341]\n \\ No newline at end of file\n +pipe:[607343]\n \\ No newline at end of file\n\nOf course that's diff and not grep, but I think they suffer from the\nsame flaw: they share the file-kind handling logic of the rest of git in\na case where it's not very helpful.\n\n-- \nThomas Rast\ntrast@{inf,student}.ethz.ch\n"},{"id":"182262","messageId":"7vwr8za04q.fsf@alter.siamese.dyndns.org","threadId":"29325","inReplyTo":"877h0zlvwd.fsf@thomas.inf.ethz.ch","subject":"Re: git grep doesn't follow symbolic link","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-01-10T18:22:13Z","receivedAt":"2012-01-10T18:22:13Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Thomas Rast writes:\n\n>> I'd imagine so: symbolic links are not portable across different file\n>> systems; Git's internal representation of a symbolic link is a file\n>> containing the path of the file to be linked to.\n>\n> I'd actually welcome a fix to this general area,...\n\nEven though some platforms may lack symbolic links, where they are\nsupported, they have a clear and defined meaning and that is what Git\ntracks as contents: where the link points at.\n\nSo we would want our \"git diff\" to tell us, even if you moved without\ncontent modification the symbolic link target that lives somewhere on your\nfilesystem but is outside the control of Git, and updated a symbolic link\nthat is tracked by Git to point to a new location, that you updated the\nlink. On the other hand, if you did not update a tracked symbolic link,\neven if the location the link points at that may or may not be under the\ncontrol of Git, we do not want \"git diff\" to show anything. As far as that\nlink is concerned, nothing has changed.\n\nChanging this would not be a fix; it would be butchering.\n\nHaving said that...\n\n> But I lose all the cute features of git-diff. I *could* say\n>\n> git diff --no-index <(ls) <(cd elsewhere && ls)\n\n... \"--no-index\" is specifically _not_ about tracked contents of Git, but\nwas bolted on as a poor-man's substitute for GNU diff (think of it as\nsomebody wanted to have the nifty \"git diff\" features like renames and\ncoloring, but did not want to bother to port them to GNU diff codebase,\nbut instead hacked Git codebase to work outside Git tracked contents). In\nthat context, I would agree that it _might_ make sense to treat special\nfiles and symbolic links in a way that is different from how tracked\ncontents are handled.\n"},{"id":"182542","messageId":"CACsJy8BfvhWxqBOj=+7AiF8dZBVEASAuxiOsjOvpmfE3uPrO3A@mail.gmail.com","threadId":"29325","inReplyTo":"7vwr8za04q.fsf@alter.siamese.dyndns.org","subject":"Re: git grep doesn't follow symbolic link","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-01-14T09:50:54Z","receivedAt":"2012-01-14T09:50:54Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"(Pang's patch [1] caught my attention so I returned to the original discussion)\n\n[1] http://thread.gmane.org/gmane.comp.version-control.git/188552\n\nOn Wed, Jan 11, 2012 at 1:22 AM, Junio C Hamano wrote:\n> Thomas Rast writes:\n>\n>>> I'd imagine so: symbolic links are not portable across different file\n>>> systems; Git's internal representation of a symbolic link is a file\n>>> containing the path of the file to be linked to.\n>>\n>> I'd actually welcome a fix to this general area,...\n>\n> Even though some platforms may lack symbolic links, where they are\n> supported, they have a clear and defined meaning and that is what Git\n> tracks as contents: where the link points at.\n>\n> So we would want our \"git diff\" to tell us, even if you moved without\n> content modification the symbolic link target that lives somewhere on your\n> filesystem but is outside the control of Git, and updated a symbolic link\n> that is tracked by Git to point to a new location, that you updated the\n> link. On the other hand, if you did not update a tracked symbolic link,\n> even if the location the link points at that may or may not be under the\n> control of Git, we do not want \"git diff\" to show anything. As far as that\n> link is concerned, nothing has changed.\n>\n> Changing this would not be a fix; it would be butchering.\n\nThat's a good default. But git should allow me to say \"diff the files\nthat symlinks point to\". Link target is content from git perspective,\nnot from user perspective.\n\nSo instead changing the default behavior specifically for git-grep as\nPang did, I think adding --follow-symlinks option, that could be\npassed to grep or any of diff family, would be a better approach.\n-- \nDuy\n"},{"id":"182567","messageId":"7v1ur1yazf.fsf@alter.siamese.dyndns.org","threadId":"29325","inReplyTo":"CACsJy8BfvhWxqBOj=+7AiF8dZBVEASAuxiOsjOvpmfE3uPrO3A@mail.gmail.com","subject":"Re: git grep doesn't follow symbolic link","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-01-15T02:07:32Z","receivedAt":"2012-01-15T02:07:32Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyen Thai Ngoc Duy writes:\n\n>> Even though some platforms may lack symbolic links, where they are\n>> supported, they have a clear and defined meaning and that is what Git\n>> tracks as contents: where the link points at.\n>>\n>> So we would want our \"git diff\" to tell us, even if you moved without\n>> content modification the symbolic link target that lives somewhere on your\n>> filesystem but is outside the control of Git, and updated a symbolic link\n>> that is tracked by Git to point to a new location, that you updated the\n>> link. On the other hand, if you did not update a tracked symbolic link,\n>> even if the location the link points at that may or may not be under the\n>> control of Git, we do not want \"git diff\" to show anything. As far as that\n>> link is concerned, nothing has changed.\n>>\n>> Changing this would not be a fix; it would be butchering.\n>\n> That's a good default. But git should allow me to say \"diff the files\n> that symlinks point to\". Link target is content from git perspective,\n> not from user perspective.\n>\n> So instead changing the default behavior specifically for git-grep as\n> Pang did, I think adding --follow-symlinks option, that could be\n> passed to grep or any of diff family, would be a better approach.\n\nStop and think what \"git diff --follow-symlinks v1.3.0 v1.7.0\" should do\nwhen these versions record a symbolic link, \"from user perspective\", if\nthe link points outside the tracked contents. Naturally, the users would\nexpect that the comparison is made between the contents of the file back\nwhen v1.3.0 was tagged and the contents of the file (which may or may not\nbe the same path depending on the target of that symbolic link) back when\nv1.7.0 was tagged.\n\nBut that is something that the user is *NOT* tracking with the system, and\nhence something we cannot give the right answer. Your \"--follow-symlinks\"\noption only encourages the *wrong* perception on the users' side, without\nsupporting what it appears to promise to the users. Why could it be an\nimprovement?\n\nCompared to that, limiting the optional support for following symlinks\nonly in \"--no-index\" case, where the user explicitly asks us to look at\nthe data that is not managed by Git at all, makes more sense. At the\ndesign level, I wouldn't be fundamentally opposed to a change to add an\noptional \"follow the symlink\" feature only when \"--no-index\" is asked for.\n\nI didn't look at the posted patch, so I do not know if it adds an optional\nfollowing or unconditionally makes us follow symbolic links, or if the\npatch sensibly implements the feature, though. That is a separate issue.\n"},{"id":"182578","messageId":"CACsJy8CaBAEJo_LuvjYhb2kfofH83cbR5DFDffmmCU3uJFqk+g@mail.gmail.com","threadId":"29325","inReplyTo":"7v1ur1yazf.fsf@alter.siamese.dyndns.org","subject":"Re: git grep doesn't follow symbolic link","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-01-15T09:47:31Z","receivedAt":"2012-01-15T09:47:31Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sun, Jan 15, 2012 at 9:07 AM, Junio C Hamano wrote:\n> Stop and think what \"git diff --follow-symlinks v1.3.0 v1.7.0\" should do\n> when these versions record a symbolic link, \"from user perspective\", if\n> the link points outside the tracked contents. Naturally, the users would\n> expect that the comparison is made between the contents of the file back\n> when v1.3.0 was tagged and the contents of the file (which may or may not\n> be the same path depending on the target of that symbolic link) back when\n> v1.7.0 was tagged.\n>\n> But that is something that the user is *NOT* tracking with the system, and\n> hence something we cannot give the right answer. Your \"--follow-symlinks\"\n> option only encourages the *wrong* perception on the users' side, without\n> supporting what it appears to promise to the users. Why could it be an\n> improvement?\n\nIt's not wrong per se. It's an implication that users have to take\nwhen they choose to use it. We may help make it clear that the\nsymlinks point to untracked files by putting some indication in the\ndiff.\n\nWhen I do \"git log -Sfoo -- '*.cxx'\" I don't really care if bar.cxx is\na symlink. Neither does my compiler. It may be a symlink's target\nchange that makes \"foo\" appear. Git could help me detect that quickly\ninstead of sticking with tracked contents only.\n\nEven if we decide --follow-symlinks=untracked is a bad idea,\n--follow-symlinks=tracked (i.e. follow symlinks to tracked files only)\nis still a good thing to support. And I suspect that's a more common\ncase as linking outside repository could is undeterministic.\n\nThe \"=tracked\" could be dropped if we have no other option value. I'm\nthinking of --follow-symlinks=submodule, which is currently covered by\na separate option name.\n-- \nDuy\n"},{"id":"182640","messageId":"7vwr8ruv1j.fsf@alter.siamese.dyndns.org","threadId":"29325","inReplyTo":"CACsJy8CaBAEJo_LuvjYhb2kfofH83cbR5DFDffmmCU3uJFqk+g@mail.gmail.com","subject":"Re: git grep doesn't follow symbolic link","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-01-16T22:44:40Z","receivedAt":"2012-01-16T22:44:40Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyen Thai Ngoc Duy writes:\n\n> It's not wrong per se. It's an implication that users have to take\n> when they choose to use it. We may help make it clear that the\n> symlinks point to untracked files by putting some indication in the\n> diff.\n>\n> When I do \"git log -Sfoo -- '*.cxx'\" I don't really care if bar.cxx is\n> a symlink. Neither does my compiler. It may be a symlink's target\n> change that makes \"foo\" appear. Git could help me detect that quickly\n> instead of sticking with tracked contents only.\n\nAs there is nothing in Git that tells that whatever is pointed at by\nbar.cxx that happens to be in your filesystem today had \"foo\" in it when\nthat historical version of the commit whose bar.cxx symlink was updated to\npoint to that file. It is *WRONG* to show the commit as something that\nchanges bar.cxx to contain \"foo\" (or more precisely, changes the count of\n\"foo\" in it).\n\nWhy is it so hard to understand?\n"},{"id":"182659","messageId":"CACsJy8B9AGuRSx_5P22TOsqrA1rTEjQb78NN7PcTuK53iUmP_w@mail.gmail.com","threadId":"29325","inReplyTo":"7vwr8ruv1j.fsf@alter.siamese.dyndns.org","subject":"Re: git grep doesn't follow symbolic link","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-01-17T01:55:28Z","receivedAt":"2012-01-17T01:55:28Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Jan 17, 2012 at 5:44 AM, Junio C Hamano wrote:\n> Nguyen Thai Ngoc Duy writes:\n>\n>> It's not wrong per se. It's an implication that users have to take\n>> when they choose to use it. We may help make it clear that the\n>> symlinks point to untracked files by putting some indication in the\n>> diff.\n>>\n>> When I do \"git log -Sfoo -- '*.cxx'\" I don't really care if bar.cxx is\n>> a symlink. Neither does my compiler. It may be a symlink's target\n>> change that makes \"foo\" appear. Git could help me detect that quickly\n>> instead of sticking with tracked contents only.\n>\n> As there is nothing in Git that tells that whatever is pointed at by\n> bar.cxx that happens to be in your filesystem today had \"foo\" in it when\n> that historical version of the commit whose bar.cxx symlink was updated to\n> point to that file. It is *WRONG* to show the commit as something that\n> changes bar.cxx to contain \"foo\" (or more precisely, changes the count of\n> \"foo\" in it).\n>\n> Why is it so hard to understand?\n\nOK resolving links to untracked contents is bad and should only be\nsupported in --no-index case, resolving links to tracked contents\nshould be ok in principal? (I'm not sure how messed up diff code could\nbe with these changes)\n-- \nDuy\n"},{"id":"182672","messageId":"7v7h0qu8t8.fsf@alter.siamese.dyndns.org","threadId":"29325","inReplyTo":"CACsJy8B9AGuRSx_5P22TOsqrA1rTEjQb78NN7PcTuK53iUmP_w@mail.gmail.com","subject":"Re: git grep doesn't follow symbolic link","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-01-17T06:44:51Z","receivedAt":"2012-01-17T06:44:51Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyen Thai Ngoc Duy writes:\n\n> OK resolving links to untracked contents is bad and should only be\n> supported in --no-index case, resolving links to tracked contents should\n> be ok in principal?\n\nConceptually it is not as bad, but I doubt it is still \"ok\".\n\nIt would defeat one of the fundamental properties of Git (or any content\nbased revision control scheme for that matter): a tree object records the\nhash of its contents, so if two subtrees agree at the content hash level,\nyou do not have to descend into them to compare what they contain.\n\nImagine that you have a symlink at a/b/c/d that points a file e at the\nroot level, and you are running \"git log a/b/c\". Even if the entire\nhierarchy a/ does not change in a commit since its parent, you may have to\nshow a/b/c/d only because \"e\" has changed.\n\nSo I suspect that the required change would involve a lot more than a\nnaïve \"when we reach the leaf level, if it is a symlink, read the link\ncontents and call get_tree_entry() to dereference the blob, or if the link\npoints outside the tree, use 0{40} to say 'contents undefined'\".\n\nAfter you compare 'a' of parent and child and find them to be identical,\nyou still need to anticipate that the hierarchy _might_ have a symbolic\nlink somewhere deep inside, and read _everything_ at least once in order\nto find symbolic links and where they point at (if you did that to parent\nalready, and if you know that the child agrees with it at 'a', then you\ncan obviously do not have to read everything in the child---you know the\nparent and the child have the same _contents_ in 'a' at that point). And\nthen grab the pointee out of parent tree and child tree to compare.\n\nI personally do not think it is worth it.\n"}]}