{"thread":{"id":"25424","subject":"git ls-files handles paths differently in Windows and Mac (probably Linux)","startedAt":"2010-10-12T12:14:10Z","lastAt":"2010-10-13T11:35:04Z","messageCount":13,"participants":["Kirill Likhodedov","Alex Riesen","Johannes Sixt","Jakub Narebski"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"153298","messageId":"3C33A244-2449-4A3C-A8B5-2060CE61EE04@jetbrains.com","threadId":"25424","inReplyTo":null,"subject":"git ls-files handles paths differently in Windows and Mac (probably Linux)","fromName":"Kirill Likhodedov","fromEmail":"kirill.likhodedov@jetbrains.com","sentAt":"2010-10-12T12:14:10Z","receivedAt":"2010-10-12T12:14:10Z","isPatch":false,"sender":{"key":"kirill.likhodedov@jetbrains.com","avatar":"https://gravatar.com/avatar/9fd55d2a110e8e96deb3ed7b503d7f3b9b5b1ed5ca9ef6683c15525b11b94e63?d=mp&s=160"},"body":"\nHi all,\n\nThe behavior of ls-files is inconsistent in Windows and Mac.\n\nI want to see which files were changed in the specific directory of my working tree, so I call ls-files:\n> git ls-files -douvm --exclude-standard -- MYDIR\n\nOn Mac (and probably on Linux) git inspects the content of MYDIR and gives me the status of not-indexed changes in this directory.\nOn Windows git gives the same result (only changes in MYDIR), but it scans the whole repository!\n\nI noticed it because it took a long time to scan a small directory, and then I confirmed it by using monitoring tools:\non Windows I used Process Monitor, on Mac I used FileMon - both give the list of accessed files in the real time.\n\nTo narrow the search scope to MYDIR I have to add \"/**\" and thus call:\n> git ls-files -douvm --exclude-standard -- MYDIR/**\nThis works as expected - scans only MYDIR.\n\nBut I can't call it with several directories:\n> git ls-files -douvm --exclude-standard -- MYDIR/** ANOTHERDIR/**\nThis doesn't work as expected and scans the whole repo.\nSo to scan two directories I have to call ls-files twice.\n\nLooks like a bug. Maybe a bug of porting Git to windows, but the behavior is the same on msys-git and cygwin-git. \nIt was tested with Git 1.7.1 on Mac, \nMsysGit 1.7.3.1 and 1.7.0.2,\nCygwinGit 1.7.0.4.\n\nThanks.\n\n----------------------------------\nKirill Likhodedov\nJetBrains, Inc\nhttp://www.jetbrains.com\n\"Develop with pleasure!\"\n"},{"id":"153299","messageId":"AANLkTimB6s_yt6L59xpqsWO_yBsZxuVCiExFoSm9FcN1@mail.gmail.com","threadId":"25424","inReplyTo":"3C33A244-2449-4A3C-A8B5-2060CE61EE04@jetbrains.com","subject":"Re: git ls-files handles paths differently in Windows and Mac (probably Linux)","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2010-10-12T12:51:05Z","receivedAt":"2010-10-12T12:51:05Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"On Tue, Oct 12, 2010 at 14:14, Kirill Likhodedov\n wrote:\n> The behavior of ls-files is inconsistent in Windows and Mac.\n>\n> I want to see which files were changed in the specific directory of my working tree, so I call ls-files:\n>> git ls-files -douvm --exclude-standard -- MYDIR\n>\n\nWell, it is an... unconventional way to do that. May I suggest you consider\nusing \"git diff --name-status\" or just \"git status\" next time?\n\n> Looks like a bug. Maybe a bug of porting Git to windows, but the behavior is the same on msys-git and cygwin-git.\n\nMaybe. Both ports share a fair amount of Windows-related code.\n"},{"id":"153300","messageId":"C29949EA-5717-4BA3-936E-354FCB107877@jetbrains.com","threadId":"25424","inReplyTo":"AANLkTimB6s_yt6L59xpqsWO_yBsZxuVCiExFoSm9FcN1@mail.gmail.com","subject":"Re: git ls-files handles paths differently in Windows and Mac (probably Linux)","fromName":"Kirill Likhodedov","fromEmail":"kirill.likhodedov@jetbrains.com","sentAt":"2010-10-12T12:59:12Z","receivedAt":"2010-10-12T12:59:12Z","isPatch":false,"sender":{"key":"kirill.likhodedov@jetbrains.com","avatar":"https://gravatar.com/avatar/9fd55d2a110e8e96deb3ed7b503d7f3b9b5b1ed5ca9ef6683c15525b11b94e63?d=mp&s=160"},"body":">> The behavior of ls-files is inconsistent in Windows and Mac.\n>> \n>> I want to see which files were changed in the specific directory of my working tree, so I call ls-files:\n>>> git ls-files -douvm --exclude-standard -- MYDIR\n>> \n> \n> Well, it is an... unconventional way to do that. May I suggest you consider\n> using \"git diff --name-status\" or just \"git status\" next time?\n\nMy tests show that \"git ls-files -douvm\" is faster than \"git diff --name-status\".\n\n\"git status\" doesn't fit, because it's porcelain (btw, git diff is also porcelain) and its output may change in time;\n\"git status --porcelain\" appeared only in 1.7.0, so I can't use it to work with older Git.\n\n\n----------------------------------\nKirill Likhodedov\nJetBrains, Inc\nhttp://www.jetbrains.com\n\"Develop with pleasure!\"\n"},{"id":"153301","messageId":"4CB45E86.2030703@viscovery.net","threadId":"25424","inReplyTo":"3C33A244-2449-4A3C-A8B5-2060CE61EE04@jetbrains.com","subject":"Re: git ls-files handles paths differently in Windows and Mac (probably Linux)","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2010-10-12T13:11:34Z","receivedAt":"2010-10-12T13:11:34Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 10/12/2010 14:14, schrieb Kirill Likhodedov:\n> I want to see which files were changed in the specific directory of my\n> working tree, so I call ls-files:\n>> git ls-files -douvm --exclude-standard -- MYDIR\n> \n> On Mac (and probably on Linux) git inspects the content of MYDIR and\n> gives me the status of not-indexed changes in this directory. On\n> Windows git gives the same result (only changes in MYDIR), but it scans\n> the whole repository!\n\nOn Linux, this scans the whole repository, too. Therefore, at least the\nWindows version should not be *that* wrong. ;-)\n\n-- Hannes\n"},{"id":"153302","messageId":"60B60680-78BD-4FFF-8472-8C853956CB1A@jetbrains.com","threadId":"25424","inReplyTo":"4CB45E86.2030703@viscovery.net","subject":"Re: git ls-files handles paths differently in Windows and Mac (probably Linux)","fromName":"Kirill Likhodedov","fromEmail":"kirill.likhodedov@jetbrains.com","sentAt":"2010-10-12T13:24:56Z","receivedAt":"2010-10-12T13:24:56Z","isPatch":false,"sender":{"key":"kirill.likhodedov@jetbrains.com","avatar":"https://gravatar.com/avatar/9fd55d2a110e8e96deb3ed7b503d7f3b9b5b1ed5ca9ef6683c15525b11b94e63?d=mp&s=160"},"body":"\nHannes,\n\nhow did you figure that? I mean - which tool did you use to see which files are accessed in Linux? \n\n\n\n12.10.2010, в 17:11, Johannes Sixt написал(а):\n\n> Am 10/12/2010 14:14, schrieb Kirill Likhodedov:\n>> I want to see which files were changed in the specific directory of my\n>> working tree, so I call ls-files:\n>>> git ls-files -douvm --exclude-standard -- MYDIR\n>> \n>> On Mac (and probably on Linux) git inspects the content of MYDIR and\n>> gives me the status of not-indexed changes in this directory. On\n>> Windows git gives the same result (only changes in MYDIR), but it scans\n>> the whole repository!\n> \n> On Linux, this scans the whole repository, too. Therefore, at least the\n> Windows version should not be *that* wrong. ;-)\n> \n> -- Hannes\n\n----------------------------------\nKirill Likhodedov\nJetBrains, Inc\nhttp://www.jetbrains.com\n\"Develop with pleasure!\"\n"},{"id":"153303","messageId":"4CB46273.1010601@viscovery.net","threadId":"25424","inReplyTo":"60B60680-78BD-4FFF-8472-8C853956CB1A@jetbrains.com","subject":"Re: git ls-files handles paths differently in Windows and Mac (probably Linux)","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2010-10-12T13:28:19Z","receivedAt":"2010-10-12T13:28:19Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Please don't top-post.\n\nAm 10/12/2010 15:24, schrieb Kirill Likhodedov:\n> how did you figure that? I mean - which tool did you use to see which\n> files are accessed in Linux?\n\nstrace -e stat,lstat -o /tmp/ls-files.strace git ls-files ...\n\nCould work on MacOS as well.\n\n-- Hannes\n"},{"id":"153315","messageId":"38052CF8-66F3-40FD-8D2D-8FD58A622F7B@jetbrains.com","threadId":"25424","inReplyTo":"C29949EA-5717-4BA3-936E-354FCB107877@jetbrains.com","subject":"Re: git ls-files handles paths differently in Windows and Mac (probably Linux)","fromName":"Kirill Likhodedov","fromEmail":"kirill.likhodedov@jetbrains.com","sentAt":"2010-10-12T13:52:30Z","receivedAt":"2010-10-12T13:52:30Z","isPatch":false,"sender":{"key":"kirill.likhodedov@jetbrains.com","avatar":"https://gravatar.com/avatar/9fd55d2a110e8e96deb3ed7b503d7f3b9b5b1ed5ca9ef6683c15525b11b94e63?d=mp&s=160"},"body":"\n>>> I want to see which files were changed in the specific directory of my working tree, so I call ls-files:\n>>>> git ls-files -douvm --exclude-standard -- MYDIR\n>>> \n>> \n>> Well, it is an... unconventional way to do that. May I suggest you consider\n>> using \"git diff --name-status\" or just \"git status\" next time?\n> \n> My tests show that \"git ls-files -douvm\" is faster than \"git diff --name-status\".\n> \n> \"git status\" doesn't fit, because it's porcelain (btw, git diff is also porcelain) and its output may change in time;\n> \"git status --porcelain\" appeared only in 1.7.0, so I can't use it to work with older Git.\n> \n\n\nAnd also \"git diff --name-status\" doesn't show unversioned files.\nAs I've found, \"git ls-files\" is the only command which shows unversioned files (except git status), isn't it?\n\n\n\n----------------------------------\nKirill Likhodedov\nJetBrains, Inc\nhttp://www.jetbrains.com\n\"Develop with pleasure!\"\n"},{"id":"153322","messageId":"AANLkTik39Rmjje67KeBYeJ6FkuOxp88e1nBPxXarBZf8@mail.gmail.com","threadId":"25424","inReplyTo":"C29949EA-5717-4BA3-936E-354FCB107877@jetbrains.com","subject":"Re: git ls-files handles paths differently in Windows and Mac (probably Linux)","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2010-10-12T15:24:06Z","receivedAt":"2010-10-12T15:24:06Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"On Tue, Oct 12, 2010 at 14:59, Kirill Likhodedov\n wrote:\n>>> The behavior of ls-files is inconsistent in Windows and Mac.\n>>>\n>>> I want to see which files were changed in the specific directory of my working tree, so I call ls-files:\n>>>> git ls-files -douvm --exclude-standard -- MYDIR\n>>>\n>>\n>> Well, it is an... unconventional way to do that. May I suggest you consider\n>> using \"git diff --name-status\" or just \"git status\" next time?\n>\n> My tests show that \"git ls-files -douvm\" is faster than \"git diff --name-status\".\n\nI find it hard to believe (the tool was definitely not mean for this).\n\n> \"git status\" doesn't fit, because it's porcelain (btw, git diff is also porcelain) and its output may change in time;\n\nThere is always \"git diff-files\", but I doubt the output of \"git diff\n--name-status\" will change soon.\nBesides, if you have to just check if the tree under a path is changed\nyou can always use\n--exit-code or --quiet to \"git diff\", it will speed them up.\n"},{"id":"153323","messageId":"AANLkTi=6nJAYKB_MTjXL3oftcWtr_wtdHaULYzRcP6oP@mail.gmail.com","threadId":"25424","inReplyTo":"38052CF8-66F3-40FD-8D2D-8FD58A622F7B@jetbrains.com","subject":"Re: git ls-files handles paths differently in Windows and Mac (probably Linux)","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2010-10-12T15:25:22Z","receivedAt":"2010-10-12T15:25:22Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"On Tue, Oct 12, 2010 at 15:52, Kirill Likhodedov\n wrote:\n> And also \"git diff --name-status\" doesn't show unversioned files.\n> As I've found, \"git ls-files\" is the only command which shows unversioned files (except git status), isn't it?\n\nYes, you're right. What are you trying to do, BTW?\n"},{"id":"153378","messageId":"AEC0873D-4AFC-402E-9D7D-197A7D40DCDD@jetbrains.com","threadId":"25424","inReplyTo":"AANLkTi=6nJAYKB_MTjXL3oftcWtr_wtdHaULYzRcP6oP@mail.gmail.com","subject":"Re: git ls-files handles paths differently in Windows and Mac (probably Linux)","fromName":"Kirill Likhodedov","fromEmail":"kirill.likhodedov@jetbrains.com","sentAt":"2010-10-13T09:09:40Z","receivedAt":"2010-10-13T09:09:40Z","isPatch":false,"sender":{"key":"kirill.likhodedov@jetbrains.com","avatar":"https://gravatar.com/avatar/9fd55d2a110e8e96deb3ed7b503d7f3b9b5b1ed5ca9ef6683c15525b11b94e63?d=mp&s=160"},"body":"\n12.10.2010, в 19:25, Alex Riesen написал(а):\n> \n> Besides, if you have to just check if the tree under a path is changed\n> you can always use\n> --exit-code or --quiet to \"git diff\", it will speed them up.\n\nUnfortunately, that's not enough: I have to know which files have changed and how (created, deleted, modified).\nAlso I feel that it's not the size of the output which is slow - the scan of directories itself is. \n\n\n>> And also \"git diff --name-status\" doesn't show unversioned files.\n>> As I've found, \"git ls-files\" is the only command which shows unversioned files (except git status), isn't it?\n> \n> Yes, you're right. What are you trying to do, BTW?\n\nI'm developing a plugin for Git integration for IntelliJ IDEA (an IDE for Java and other languages).\nAnd the task I'm working now is to get the status of index and working tree. I want to make it as fast as possible.\nTo work with a Git repository I start a process with git native commands, so every command to execute is additional cost.\n(Maybe I could work with the filesystem and raw Git repository, but it would be an overhead).\n\nIf we forget about \"git status --porcelain\" there is no single command to get the whole status. I could use \n\"git ls-files -ov\" for unversioned files + \"git diff --name-status\" for all other.\nOR \n\"git ls-files -douvm\" for the changes in the working tree + \"git diff-index --cached\" for the changes in the index.\nOR\n\"git ls-files -ov\" for unversioned + \"git diff-files\" for other files in the working tree + \"git diff-index --cached\" for the changes in the index.\nor maybe something else\n\nI've made a script which executes these commands (which allow to get the total status) on a large repository many times. \nMy results show that \"ls-files -douvm\"+\"git diff-index --cached\" is the fastest combination.\n\nDo you think I could achieve the same result in a simpler or faster way?\n\nThanks a lot.\n\n----------------------------------\nKirill Likhodedov\nJetBrains, Inc\nhttp://www.jetbrains.com\n\"Develop with pleasure!\"\n"},{"id":"153393","messageId":"AANLkTinw0CNjdZZOzB8rQmZ3uQiB_sGCGa2B0jJfWf7=@mail.gmail.com","threadId":"25424","inReplyTo":"AEC0873D-4AFC-402E-9D7D-197A7D40DCDD@jetbrains.com","subject":"Re: git ls-files handles paths differently in Windows and Mac (probably Linux)","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2010-10-13T10:35:24Z","receivedAt":"2010-10-13T10:35:24Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"2010/10/13 Kirill Likhodedov :\n>\n> 12.10.2010, в 19:25, Alex Riesen написал(а):\n>>\n>> Besides, if you have to just check if the tree under a path is changed\n>> you can always use\n>> --exit-code or --quiet to \"git diff\", it will speed them up.\n>\n> Unfortunately, that's not enough: I have to know which files have changed and how (created, deleted, modified).\n> Also I feel that it's not the size of the output which is slow - the scan of directories itself is.\n\nYeah, that's Windows for you.\n\n> I've made a script which executes these commands (which allow to get the total status) on a large\n> repository many times.\n> My results show that \"ls-files -douvm\"+\"git diff-index --cached\" is the fastest combination.\n>\n> Do you think I could achieve the same result in a simpler or faster way?\n\nNo, don't think so. But you can run the commands in parallel, I suppose.\n\nAnd of course a patch implementing a command that does this (or improving\nthe performance of \"git diff\") will be gladly considered here.\n"},{"id":"153394","messageId":"m3aami731j.fsf@localhost.localdomain","threadId":"25424","inReplyTo":"AEC0873D-4AFC-402E-9D7D-197A7D40DCDD@jetbrains.com","subject":"Re: git ls-files handles paths differently in Windows and Mac (probably Linux)","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-10-13T11:20:39Z","receivedAt":"2010-10-13T11:20:39Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Kirill Likhodedov writes:\n> 12.10.2010, в 19:25, Alex Riesen написал(а):\n\n> > [..]. What are you trying to do, BTW?\n> \n> I'm developing a plugin for Git integration for IntelliJ IDEA (an IDE for\n> Java and other languages).\n>\n> And the task I'm working now is to get the status of index and working\n> tree. I want to make it as fast as possible. To work with a Git\n> repository I start a process with git native commands, so every command to\n> execute is additional cost. (Maybe I could work with the filesystem and\n> raw Git repository, but it would be an overhead).\n\nIsn't IntelliJ IDEA written itself in Java? Perhaps you could use JGit, the\nJava implementation of Git (though it is not complete implementation yet, if\nI remember correctly). It is EDL (new-style BSD) licensed, and is used by\nEclipse and NetBeans.\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"153396","messageId":"9DB2A1C4-F0A3-4920-B409-AA267EB2FC57@jetbrains.com","threadId":"25424","inReplyTo":"m3aami731j.fsf@localhost.localdomain","subject":"Re: git ls-files handles paths differently in Windows and Mac (probably Linux)","fromName":"Kirill Likhodedov","fromEmail":"kirill.likhodedov@jetbrains.com","sentAt":"2010-10-13T11:35:04Z","receivedAt":"2010-10-13T11:35:04Z","isPatch":false,"sender":{"key":"kirill.likhodedov@jetbrains.com","avatar":"https://gravatar.com/avatar/9fd55d2a110e8e96deb3ed7b503d7f3b9b5b1ed5ca9ef6683c15525b11b94e63?d=mp&s=160"},"body":"> Isn't IntelliJ IDEA written itself in Java? Perhaps you could use JGit, the\n> Java implementation of Git (though it is not complete implementation yet, if\n> I remember correctly). It is EDL (new-style BSD) licensed, and is used by\n> Eclipse and NetBeans.\n> \n\nYep, we thought about JGit. \nBut people say that it's not stable enough, and not completely implemented. \nPersonally I haven't tried it yet though.\n\n----------------------------------\nKirill Likhodedov\nJetBrains, Inc\nhttp://www.jetbrains.com\n\"Develop with pleasure!\"\n"}]}