git ls-files handles paths differently in Windows and Mac (probably Linux)

Hi all,

The behavior of ls-files is inconsistent in Windows and Mac.

I want to see which files were changed in the specific directory of my working tree, so I call ls-files:

On Mac (and probably on Linux) git inspects the content of MYDIR and gives me the status of not-indexed changes in this directory. On Windows git gives the same result (only changes in MYDIR), but it scans the whole repository!

I noticed it because it took a long time to scan a small directory, and then I confirmed it by using monitoring tools: on Windows I used Process Monitor, on Mac I used FileMon - both give the list of accessed files in the real time.

To narrow the search scope to MYDIR I have to add "/**" and thus call:

This works as expected - scans only MYDIR.

But I can't call it with several directories:

This doesn't work as expected and scans the whole repo. So to scan two directories I have to call ls-files twice.

Looks like a bug. Maybe a bug of porting Git to windows, but the behavior is the same on msys-git and cygwin-git. It was tested with Git 1.7.1 on Mac, MsysGit 1.7.3.1 and 1.7.0.2, CygwinGit 1.7.0.4.

Thanks.

---------------------------------- Kirill Likhodedov JetBrains, Inc http://www.jetbrains.com "Develop with pleasure!"

On Tue, Oct 12, 2010 at 14:14, Kirill Likhodedov <Kirill.Likhodedov@jetbrains.com> wrote:

Well, it is an... unconventional way to do that. May I suggest you consider using "git diff --name-status" or just "git status" next time?

Maybe. Both ports share a fair amount of Windows-related code.

My tests show that "git ls-files -douvm" is faster than "git diff --name-status".

"git status" doesn't fit, because it's porcelain (btw, git diff is also porcelain) and its output may change in time; "git status --porcelain" appeared only in 1.7.0, so I can't use it to work with older Git.

---------------------------------- Kirill Likhodedov JetBrains, Inc http://www.jetbrains.com "Develop with pleasure!"

And also "git diff --name-status" doesn't show unversioned files. As I've found, "git ls-files" is the only command which shows unversioned files (except git status), isn't it?

---------------------------------- Kirill Likhodedov JetBrains, Inc http://www.jetbrains.com "Develop with pleasure!"

On Tue, Oct 12, 2010 at 15:52, Kirill Likhodedov <Kirill.Likhodedov@jetbrains.com> wrote:

Yes, you're right. What are you trying to do, BTW?

12.10.2010, в 19:25, Alex Riesen написал(а):

Unfortunately, that's not enough: I have to know which files have changed and how (created, deleted, modified). Also I feel that it's not the size of the output which is slow - the scan of directories itself is.

I'm developing a plugin for Git integration for IntelliJ IDEA (an IDE for Java and other languages). And the task I'm working now is to get the status of index and working tree. I want to make it as fast as possible. To work with a Git repository I start a process with git native commands, so every command to execute is additional cost. (Maybe I could work with the filesystem and raw Git repository, but it would be an overhead).

If we forget about "git status --porcelain" there is no single command to get the whole status. I could use "git ls-files -ov" for unversioned files + "git diff --name-status" for all other. OR "git ls-files -douvm" for the changes in the working tree + "git diff-index --cached" for the changes in the index. OR "git ls-files -ov" for unversioned + "git diff-files" for other files in the working tree + "git diff-index --cached" for the changes in the index. or maybe something else

I've made a script which executes these commands (which allow to get the total status) on a large repository many times. My results show that "ls-files -douvm"+"git diff-index --cached" is the fastest combination.

Do you think I could achieve the same result in a simpler or faster way?

Thanks a lot.

---------------------------------- Kirill Likhodedov JetBrains, Inc http://www.jetbrains.com "Develop with pleasure!"

2010/10/13 Kirill Likhodedov <Kirill.Likhodedov@jetbrains.com>:

Yeah, that's Windows for you.

No, don't think so. But you can run the commands in parallel, I suppose.

And of course a patch implementing a command that does this (or improving the performance of "git diff") will be gladly considered here.

Kirill Likhodedov <Kirill.Likhodedov@jetbrains.com> writes:

Isn't IntelliJ IDEA written itself in Java? Perhaps you could use JGit, the Java implementation of Git (though it is not complete implementation yet, if I remember correctly). It is EDL (new-style BSD) licensed, and is used by Eclipse and NetBeans.

Yep, we thought about JGit. But people say that it's not stable enough, and not completely implemented. Personally I haven't tried it yet though.

---------------------------------- Kirill Likhodedov JetBrains, Inc http://www.jetbrains.com "Develop with pleasure!"

On Tue, Oct 12, 2010 at 14:59, Kirill Likhodedov <Kirill.Likhodedov@jetbrains.com> wrote:

I find it hard to believe (the tool was definitely not mean for this).

There is always "git diff-files", but I doubt the output of "git diff --name-status" will change soon. Besides, if you have to just check if the tree under a path is changed you can always use --exit-code or --quiet to "git diff", it will speed them up.

Am 10/12/2010 14:14, schrieb Kirill Likhodedov:

On Linux, this scans the whole repository, too. Therefore, at least the Windows version should not be *that* wrong. ;-)

-- Hannes

Hannes,

how did you figure that? I mean - which tool did you use to see which files are accessed in Linux? 

12.10.2010, в 17:11, Johannes Sixt написал(а):

---------------------------------- Kirill Likhodedov JetBrains, Inc http://www.jetbrains.com "Develop with pleasure!"

Please don't top-post.

Am 10/12/2010 15:24, schrieb Kirill Likhodedov:

strace -e stat,lstat -o /tmp/ls-files.strace git ls-files ...

Could work on MacOS as well.

-- Hannes