# git ls-files handles paths differently in Windows and Mac (probably Linux) 13 messages from 2010-10-12 to 2010-10-13. Participants: Kirill Likhodedov, Alex Riesen, Johannes Sixt, Jakub Narebski. Thread: https://gitlist.dev/t/25424 ## Kirill Likhodedov, 2010-10-12 12:14 Subject: git ls-files handles paths differently in Windows and Mac (probably Linux) Message-ID: <3C33A244-2449-4A3C-A8B5-2060CE61EE04@jetbrains.com> URL: https://gitlist.dev/e/3C33A244-2449-4A3C-A8B5-2060CE61EE04%40jetbrains.com ``` Hi all, The behavior of ls-files is inconsistent in Windows and Mac. I want to see which files were changed in the specific directory of my working tree, so I call ls-files: > git ls-files -douvm --exclude-standard -- MYDIR On Mac (and probably on Linux) git inspects the content of MYDIR and gives me the status of not-indexed changes in this directory. On Windows git gives the same result (only changes in MYDIR), but it scans the whole repository! I noticed it because it took a long time to scan a small directory, and then I confirmed it by using monitoring tools: on Windows I used Process Monitor, on Mac I used FileMon - both give the list of accessed files in the real time. To narrow the search scope to MYDIR I have to add "/**" and thus call: > git ls-files -douvm --exclude-standard -- MYDIR/** This works as expected - scans only MYDIR. But I can't call it with several directories: > git ls-files -douvm --exclude-standard -- MYDIR/** ANOTHERDIR/** This doesn't work as expected and scans the whole repo. So to scan two directories I have to call ls-files twice. Looks like a bug. Maybe a bug of porting Git to windows, but the behavior is the same on msys-git and cygwin-git. It was tested with Git 1.7.1 on Mac, MsysGit 1.7.3.1 and 1.7.0.2, CygwinGit 1.7.0.4. Thanks. ---------------------------------- Kirill Likhodedov JetBrains, Inc http://www.jetbrains.com "Develop with pleasure!" ``` ## Alex Riesen, 2010-10-12 12:51 Subject: Re: git ls-files handles paths differently in Windows and Mac (probably Linux) Message-ID: URL: https://gitlist.dev/e/AANLkTimB6s_yt6L59xpqsWO_yBsZxuVCiExFoSm9FcN1%40mail.gmail.com In-Reply-To: <3C33A244-2449-4A3C-A8B5-2060CE61EE04@jetbrains.com> ``` On Tue, Oct 12, 2010 at 14:14, Kirill Likhodedov wrote: > The behavior of ls-files is inconsistent in Windows and Mac. > > I want to see which files were changed in the specific directory of my working tree, so I call ls-files: >> git ls-files -douvm --exclude-standard -- MYDIR > Well, it is an... unconventional way to do that. May I suggest you consider using "git diff --name-status" or just "git status" next time? > Looks like a bug. Maybe a bug of porting Git to windows, but the behavior is the same on msys-git and cygwin-git. Maybe. Both ports share a fair amount of Windows-related code. ``` ## Kirill Likhodedov, 2010-10-12 12:59 Subject: Re: git ls-files handles paths differently in Windows and Mac (probably Linux) Message-ID: URL: https://gitlist.dev/e/C29949EA-5717-4BA3-936E-354FCB107877%40jetbrains.com In-Reply-To: ``` >> The behavior of ls-files is inconsistent in Windows and Mac. >> >> I want to see which files were changed in the specific directory of my working tree, so I call ls-files: >>> git ls-files -douvm --exclude-standard -- MYDIR >> > > Well, it is an... unconventional way to do that. May I suggest you consider > using "git diff --name-status" or just "git status" next time? My tests show that "git ls-files -douvm" is faster than "git diff --name-status". "git status" doesn't fit, because it's porcelain (btw, git diff is also porcelain) and its output may change in time; "git status --porcelain" appeared only in 1.7.0, so I can't use it to work with older Git. ---------------------------------- Kirill Likhodedov JetBrains, Inc http://www.jetbrains.com "Develop with pleasure!" ``` ## Johannes Sixt, 2010-10-12 13:11 Subject: Re: git ls-files handles paths differently in Windows and Mac (probably Linux) Message-ID: <4CB45E86.2030703@viscovery.net> URL: https://gitlist.dev/e/4CB45E86.2030703%40viscovery.net In-Reply-To: <3C33A244-2449-4A3C-A8B5-2060CE61EE04@jetbrains.com> ``` Am 10/12/2010 14:14, schrieb Kirill Likhodedov: > I want to see which files were changed in the specific directory of my > working tree, so I call ls-files: >> git ls-files -douvm --exclude-standard -- MYDIR > > On Mac (and probably on Linux) git inspects the content of MYDIR and > gives me the status of not-indexed changes in this directory. On > Windows git gives the same result (only changes in MYDIR), but it scans > the whole repository! On Linux, this scans the whole repository, too. Therefore, at least the Windows version should not be *that* wrong. ;-) -- Hannes ``` ## Kirill Likhodedov, 2010-10-12 13:24 Subject: Re: git ls-files handles paths differently in Windows and Mac (probably Linux) Message-ID: <60B60680-78BD-4FFF-8472-8C853956CB1A@jetbrains.com> URL: https://gitlist.dev/e/60B60680-78BD-4FFF-8472-8C853956CB1A%40jetbrains.com In-Reply-To: <4CB45E86.2030703@viscovery.net> ``` Hannes, how did you figure that? I mean - which tool did you use to see which files are accessed in Linux? 12.10.2010, в 17:11, Johannes Sixt написал(а): > Am 10/12/2010 14:14, schrieb Kirill Likhodedov: >> I want to see which files were changed in the specific directory of my >> working tree, so I call ls-files: >>> git ls-files -douvm --exclude-standard -- MYDIR >> >> On Mac (and probably on Linux) git inspects the content of MYDIR and >> gives me the status of not-indexed changes in this directory. On >> Windows git gives the same result (only changes in MYDIR), but it scans >> the whole repository! > > On Linux, this scans the whole repository, too. Therefore, at least the > Windows version should not be *that* wrong. ;-) > > -- Hannes ---------------------------------- Kirill Likhodedov JetBrains, Inc http://www.jetbrains.com "Develop with pleasure!" ``` ## Johannes Sixt, 2010-10-12 13:28 Subject: Re: git ls-files handles paths differently in Windows and Mac (probably Linux) Message-ID: <4CB46273.1010601@viscovery.net> URL: https://gitlist.dev/e/4CB46273.1010601%40viscovery.net In-Reply-To: <60B60680-78BD-4FFF-8472-8C853956CB1A@jetbrains.com> ``` Please don't top-post. Am 10/12/2010 15:24, schrieb Kirill Likhodedov: > how did you figure that? I mean - which tool did you use to see which > files are accessed in Linux? strace -e stat,lstat -o /tmp/ls-files.strace git ls-files ... Could work on MacOS as well. -- Hannes ``` ## Kirill Likhodedov, 2010-10-12 13:52 Subject: Re: git ls-files handles paths differently in Windows and Mac (probably Linux) Message-ID: <38052CF8-66F3-40FD-8D2D-8FD58A622F7B@jetbrains.com> URL: https://gitlist.dev/e/38052CF8-66F3-40FD-8D2D-8FD58A622F7B%40jetbrains.com In-Reply-To: ``` >>> I want to see which files were changed in the specific directory of my working tree, so I call ls-files: >>>> git ls-files -douvm --exclude-standard -- MYDIR >>> >> >> Well, it is an... unconventional way to do that. May I suggest you consider >> using "git diff --name-status" or just "git status" next time? > > My tests show that "git ls-files -douvm" is faster than "git diff --name-status". > > "git status" doesn't fit, because it's porcelain (btw, git diff is also porcelain) and its output may change in time; > "git status --porcelain" appeared only in 1.7.0, so I can't use it to work with older Git. > And also "git diff --name-status" doesn't show unversioned files. As I've found, "git ls-files" is the only command which shows unversioned files (except git status), isn't it? ---------------------------------- Kirill Likhodedov JetBrains, Inc http://www.jetbrains.com "Develop with pleasure!" ``` ## Alex Riesen, 2010-10-12 15:24 Subject: Re: git ls-files handles paths differently in Windows and Mac (probably Linux) Message-ID: URL: https://gitlist.dev/e/AANLkTik39Rmjje67KeBYeJ6FkuOxp88e1nBPxXarBZf8%40mail.gmail.com In-Reply-To: ``` On Tue, Oct 12, 2010 at 14:59, Kirill Likhodedov wrote: >>> The behavior of ls-files is inconsistent in Windows and Mac. >>> >>> I want to see which files were changed in the specific directory of my working tree, so I call ls-files: >>>> git ls-files -douvm --exclude-standard -- MYDIR >>> >> >> Well, it is an... unconventional way to do that. May I suggest you consider >> using "git diff --name-status" or just "git status" next time? > > My tests show that "git ls-files -douvm" is faster than "git diff --name-status". I find it hard to believe (the tool was definitely not mean for this). > "git status" doesn't fit, because it's porcelain (btw, git diff is also porcelain) and its output may change in time; There is always "git diff-files", but I doubt the output of "git diff --name-status" will change soon. Besides, if you have to just check if the tree under a path is changed you can always use --exit-code or --quiet to "git diff", it will speed them up. ``` ## Alex Riesen, 2010-10-12 15:25 Subject: Re: git ls-files handles paths differently in Windows and Mac (probably Linux) Message-ID: URL: https://gitlist.dev/e/AANLkTi%3D6nJAYKB_MTjXL3oftcWtr_wtdHaULYzRcP6oP%40mail.gmail.com In-Reply-To: <38052CF8-66F3-40FD-8D2D-8FD58A622F7B@jetbrains.com> ``` On Tue, Oct 12, 2010 at 15:52, Kirill Likhodedov wrote: > And also "git diff --name-status" doesn't show unversioned files. > As I've found, "git ls-files" is the only command which shows unversioned files (except git status), isn't it? Yes, you're right. What are you trying to do, BTW? ``` ## Kirill Likhodedov, 2010-10-13 09:09 Subject: Re: git ls-files handles paths differently in Windows and Mac (probably Linux) Message-ID: URL: https://gitlist.dev/e/AEC0873D-4AFC-402E-9D7D-197A7D40DCDD%40jetbrains.com In-Reply-To: ``` 12.10.2010, в 19:25, Alex Riesen написал(а): > > Besides, if you have to just check if the tree under a path is changed > you can always use > --exit-code or --quiet to "git diff", it will speed them up. Unfortunately, that's not enough: I have to know which files have changed and how (created, deleted, modified). Also I feel that it's not the size of the output which is slow - the scan of directories itself is. >> And also "git diff --name-status" doesn't show unversioned files. >> As I've found, "git ls-files" is the only command which shows unversioned files (except git status), isn't it? > > Yes, you're right. What are you trying to do, BTW? I'm developing a plugin for Git integration for IntelliJ IDEA (an IDE for Java and other languages). And the task I'm working now is to get the status of index and working tree. I want to make it as fast as possible. To work with a Git repository I start a process with git native commands, so every command to execute is additional cost. (Maybe I could work with the filesystem and raw Git repository, but it would be an overhead). If we forget about "git status --porcelain" there is no single command to get the whole status. I could use "git ls-files -ov" for unversioned files + "git diff --name-status" for all other. OR "git ls-files -douvm" for the changes in the working tree + "git diff-index --cached" for the changes in the index. OR "git ls-files -ov" for unversioned + "git diff-files" for other files in the working tree + "git diff-index --cached" for the changes in the index. or maybe something else I've made a script which executes these commands (which allow to get the total status) on a large repository many times. My results show that "ls-files -douvm"+"git diff-index --cached" is the fastest combination. Do you think I could achieve the same result in a simpler or faster way? Thanks a lot. ---------------------------------- Kirill Likhodedov JetBrains, Inc http://www.jetbrains.com "Develop with pleasure!" ``` ## Alex Riesen, 2010-10-13 10:35 Subject: Re: git ls-files handles paths differently in Windows and Mac (probably Linux) Message-ID: URL: https://gitlist.dev/e/AANLkTinw0CNjdZZOzB8rQmZ3uQiB_sGCGa2B0jJfWf7%3D%40mail.gmail.com In-Reply-To: ``` 2010/10/13 Kirill Likhodedov : > > 12.10.2010, в 19:25, Alex Riesen написал(а): >> >> Besides, if you have to just check if the tree under a path is changed >> you can always use >> --exit-code or --quiet to "git diff", it will speed them up. > > Unfortunately, that's not enough: I have to know which files have changed and how (created, deleted, modified). > Also I feel that it's not the size of the output which is slow - the scan of directories itself is. Yeah, that's Windows for you. > I've made a script which executes these commands (which allow to get the total status) on a large > repository many times. > My results show that "ls-files -douvm"+"git diff-index --cached" is the fastest combination. > > Do you think I could achieve the same result in a simpler or faster way? No, don't think so. But you can run the commands in parallel, I suppose. And of course a patch implementing a command that does this (or improving the performance of "git diff") will be gladly considered here. ``` ## Jakub Narebski, 2010-10-13 11:20 Subject: Re: git ls-files handles paths differently in Windows and Mac (probably Linux) Message-ID: URL: https://gitlist.dev/e/m3aami731j.fsf%40localhost.localdomain In-Reply-To: ``` Kirill Likhodedov writes: > 12.10.2010, в 19:25, Alex Riesen написал(а): > > [..]. What are you trying to do, BTW? > > I'm developing a plugin for Git integration for IntelliJ IDEA (an IDE for > Java and other languages). > > And the task I'm working now is to get the status of index and working > tree. I want to make it as fast as possible. To work with a Git > repository I start a process with git native commands, so every command to > execute is additional cost. (Maybe I could work with the filesystem and > raw Git repository, but it would be an overhead). Isn't IntelliJ IDEA written itself in Java? Perhaps you could use JGit, the Java implementation of Git (though it is not complete implementation yet, if I remember correctly). It is EDL (new-style BSD) licensed, and is used by Eclipse and NetBeans. -- Jakub Narebski Poland ShadeHawk on #git ``` ## Kirill Likhodedov, 2010-10-13 11:35 Subject: Re: git ls-files handles paths differently in Windows and Mac (probably Linux) Message-ID: <9DB2A1C4-F0A3-4920-B409-AA267EB2FC57@jetbrains.com> URL: https://gitlist.dev/e/9DB2A1C4-F0A3-4920-B409-AA267EB2FC57%40jetbrains.com In-Reply-To: ``` > Isn't IntelliJ IDEA written itself in Java? Perhaps you could use JGit, the > Java implementation of Git (though it is not complete implementation yet, if > I remember correctly). It is EDL (new-style BSD) licensed, and is used by > Eclipse and NetBeans. > Yep, we thought about JGit. But people say that it's not stable enough, and not completely implemented. Personally I haven't tried it yet though. ---------------------------------- Kirill Likhodedov JetBrains, Inc http://www.jetbrains.com "Develop with pleasure!" ```