{"thread":{"id":"62021","subject":"Committing crimes with NTFS-3G","startedAt":"2024-08-29T20:51:17Z","lastAt":"2024-09-05T19:51:50Z","messageCount":14,"participants":["Roman Sandu","brian m. carlson","Vito Caputo","Johannes Sixt","Junio C Hamano","Torsten Bögershausen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"501873","messageId":"7d1dad03-703c-47ae-a039-c15aa765fd0b@gaijin.team","threadId":"62021","inReplyTo":null,"subject":"Committing crimes with NTFS-3G","fromName":"Roman Sandu","fromEmail":"r.sandu@gaijin.team","sentAt":"2024-08-29T20:43:40Z","receivedAt":"2024-08-29T20:51:17Z","isPatch":false,"sender":{"key":"r.sandu@gaijin.team","avatar":null},"body":"Good day!\n\nI have a decently sized (80K files) monorepo on an NTFS drive that I \nhave been working with for a while under Windows via git-for-windows. \nRecently, I had to (temporarily) switch to Ubuntu (24.04) via dual-boot \nfor irrelevant reasons, and I decided that simply mounting my NTFS drive \nand using the monorepo from Ubuntu is a great idea, actually, as NTFS-3G \nallow for seamless interop with NTFS via UserMapping. And so that is \nexactly what I did and It Just Works!\n\nExcept it kind of does not. Every time I run `git status` it takes 8 \nseconds, which is very painful when doing tricky history rewriting.\n\nTo diagnose the problem, I ran git status with GIT_TRACE_PERFORMANCE \nenabled, and what I see is that the \"refresh index\" region is taking up \n99% of the time. Digging further, `strace -fc git status` tells me that \n99% of the time is spent on newfstatat'ing files. Okay, makes sense, \nstat'ing files through FUSE is not all that quick. But how many files \nare we talking about? My repository has `feature.manyFiles` enabled in \ngit, so I would expect `core.untrackedCache` make it so that `git \nstatus` skips basically everything except for the root folder which \ncontains, what, 20 subfolders? But it actually does >96K stat calls! \nWhich is more than the amount of files in the repository in total. \nBriefly looking at the output of `strace -f git status`, I see that git \nindeed goes through basically all of the repository, even things that \nhave not changed for years, as if `core.untrackedCache` is not actually \nenabled. Manually enabling it on top of `feature.manyFiles` does not \nhelp. Note that `git update-index --test-untracked-cache` tells me that \nmtime does indeed work, and I've also manually stat'ed some folders \nwhich `git status` re-stats on every run and I see that the modify time \nis indeed a couple of hours ago, yet even when running `git status` \nseveral times in a row it re-scans the entire folder every time.\n\nSo, what do I do about this? It honestly looks like a git bug to me, \nmaybe it silently fails to update the index with new timestamps because \nit was initially created on Windows? But I have no clue how to narrow \nthis issue down further, so any ideas or suggestions would be appreciated!\n\nRegards,\nRoman Sandu\n"},{"id":"501887","messageId":"ZtEWvQOwLPgjIFks@tapette.crustytoothpaste.net","threadId":"62021","inReplyTo":"7d1dad03-703c-47ae-a039-c15aa765fd0b@gaijin.team","subject":"Re: Committing crimes with NTFS-3G","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-08-30T00:47:57Z","receivedAt":"2024-08-30T00:48:00Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-08-29 at 20:43:40, Roman Sandu wrote:\n> Good day!\n> \n> I have a decently sized (80K files) monorepo on an NTFS drive that I have\n> been working with for a while under Windows via git-for-windows. Recently, I\n> had to (temporarily) switch to Ubuntu (24.04) via dual-boot for irrelevant\n> reasons, and I decided that simply mounting my NTFS drive and using the\n> monorepo from Ubuntu is a great idea, actually, as NTFS-3G allow for\n> seamless interop with NTFS via UserMapping. And so that is exactly what I\n> did and It Just Works!\n\nIn general, I would not recommend this.  NTFS doesn't support Unix\npermissions, so I'd expect a lot of things to be broken.  A lot of\npeople like using NTFS to share data across Windows and Linux, but UDF\nis a much better choice and I'm not surprised that NTFS isn't working\nthe way you'd expect.\n\nAlso, when you share a repository across systems, you should expect the\nindex to be fully refreshed each time you change the OS, reading every\nfile in the repository[0].\n\n> So, what do I do about this? It honestly looks like a git bug to me, maybe\n> it silently fails to update the index with new timestamps because it was\n> initially created on Windows? But I have no clue how to narrow this issue\n> down further, so any ideas or suggestions would be appreciated!\n\nCan you pick some file in your repository and run `stat` on it, before\nand after running `git status`, and include the output?\n\nFor example:\n\n  stat http.c | tee /tmp/before\n  git status\n  stat http.c | tee /tmp/after\n  sha256sum /tmp/before /tmp/after\n\nMy guess is that NTFS-3G is not emulating something properly and it's\ndiffering at some point.\n\n[0] https://git-scm.com/docs/gitfaq#sync-working-tree\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"501888","messageId":"20240830041830.m675suaade5jrj3p@shells.gnugeneration.com","threadId":"62021","inReplyTo":"7d1dad03-703c-47ae-a039-c15aa765fd0b@gaijin.team","subject":"Re: Committing crimes with NTFS-3G","fromName":"Vito Caputo","fromEmail":"vcaputo@pengaru.com","sentAt":"2024-08-30T04:18:30Z","receivedAt":"2024-08-30T04:27:10Z","isPatch":false,"sender":{"key":"vcaputo@pengaru.com","avatar":null},"body":"On Thu, Aug 29, 2024 at 11:43:40PM +0300, Roman Sandu wrote:\n> Good day!\n> \n> I have a decently sized (80K files) monorepo on an NTFS drive that I have\n> been working with for a while under Windows via git-for-windows. Recently, I\n> had to (temporarily) switch to Ubuntu (24.04) via dual-boot for irrelevant\n> reasons, and I decided that simply mounting my NTFS drive and using the\n> monorepo from Ubuntu is a great idea, actually, as NTFS-3G allow for\n> seamless interop with NTFS via UserMapping. And so that is exactly what I\n> did and It Just Works!\n> \n> Except it kind of does not. Every time I run `git status` it takes 8\n> seconds, which is very painful when doing tricky history rewriting.\n> \n> To diagnose the problem, I ran git status with GIT_TRACE_PERFORMANCE\n> enabled, and what I see is that the \"refresh index\" region is taking up 99%\n> of the time. Digging further, `strace -fc git status` tells me that 99% of\n> the time is spent on newfstatat'ing files. Okay, makes sense, stat'ing files\n> through FUSE is not all that quick. But how many files are we talking about?\n> My repository has `feature.manyFiles` enabled in git, so I would expect\n> `core.untrackedCache` make it so that `git status` skips basically\n> everything except for the root folder which contains, what, 20 subfolders?\n> But it actually does >96K stat calls! Which is more than the amount of files\n> in the repository in total. Briefly looking at the output of `strace -f git\n> status`, I see that git indeed goes through basically all of the repository,\n> even things that have not changed for years, as if `core.untrackedCache` is\n> not actually enabled. Manually enabling it on top of `feature.manyFiles`\n> does not help. Note that `git update-index --test-untracked-cache` tells me\n> that mtime does indeed work, and I've also manually stat'ed some folders\n> which `git status` re-stats on every run and I see that the modify time is\n> indeed a couple of hours ago, yet even when running `git status` several\n> times in a row it re-scans the entire folder every time.\n> \n> So, what do I do about this? It honestly looks like a git bug to me, maybe\n> it silently fails to update the index with new timestamps because it was\n> initially created on Windows? But I have no clue how to narrow this issue\n> down further, so any ideas or suggestions would be appreciated!\n> \n\nIt was pretty big news that Paragon's read-write NTFS driver was merged\ninto the kernel.  You might want to give that a try if your main problem\nis performance.\n\nhttps://lore.kernel.org/lkml/CAHk-=wjn4W-7ZbHrw08cWy=12DgheFUKLO5YLgG6in5TA5HxqQ@mail.gmail.com/\n\nRegards,\nVito Caputo\n"},{"id":"501889","messageId":"b282c83c-2fa2-4e8e-b8bf-d42f28c17439@kdbg.org","threadId":"62021","inReplyTo":"7d1dad03-703c-47ae-a039-c15aa765fd0b@gaijin.team","subject":"Re: Committing crimes with NTFS-3G","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2024-08-30T04:58:20Z","receivedAt":"2024-08-30T04:58:31Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 29.08.24 um 22:43 schrieb Roman Sandu:\n> To diagnose the problem, I ran git status with GIT_TRACE_PERFORMANCE\n> enabled, and what I see is that the \"refresh index\" region is taking up\n> 99% of the time.\n\nOf course. The stat information that Git on Linux caches in the index is\nvastly different from that that Git for Windows caches. So every time\nyou switch OS, all files appear modified to Git.\n\nI suggest that you don't switch OS on a whim and take the 8 seconds\ndelay once when you have to.\n\n-- Hannes\n\n"},{"id":"501919","messageId":"ffad4f8d-1fd9-4f2b-97c1-937576271a04@gaijin.team","threadId":"62021","inReplyTo":"b282c83c-2fa2-4e8e-b8bf-d42f28c17439@kdbg.org","subject":"Re: Committing crimes with NTFS-3G","fromName":"Roman Sandu","fromEmail":"r.sandu@gaijin.team","sentAt":"2024-08-30T12:41:05Z","receivedAt":"2024-08-30T12:41:07Z","isPatch":false,"sender":{"key":"r.sandu@gaijin.team","avatar":null},"body":" > I suggest that you don't switch OS on a whim and take the 8 seconds\n > delay once when you have to.\n\nI am aware that git will refresh the index every time I switch OSes. \nThat is NOT the problem I am having. The index is being fully refreshed \non *every single* `git status` after I switch to Ubuntu, not just the \nfirst one!\n\nOn 8/30/24 07:58, Johannes Sixt wrote:\n> Am 29.08.24 um 22:43 schrieb Roman Sandu:\n>> To diagnose the problem, I ran git status with GIT_TRACE_PERFORMANCE\n>> enabled, and what I see is that the \"refresh index\" region is taking up\n>> 99% of the time.\n> \n> Of course. The stat information that Git on Linux caches in the index is\n> vastly different from that that Git for Windows caches. So every time\n> you switch OS, all files appear modified to Git.\n> \n> I suggest that you don't switch OS on a whim and take the 8 seconds\n> delay once when you have to.\n> \n> -- Hannes\n> \n\n"},{"id":"501921","messageId":"3eda2715-3b44-469e-85d5-f25504212928@gaijin.team","threadId":"62021","inReplyTo":"ZtEWvQOwLPgjIFks@tapette.crustytoothpaste.net","subject":"Re: Committing crimes with NTFS-3G","fromName":"Roman Sandu","fromEmail":"r.sandu@gaijin.team","sentAt":"2024-08-30T12:52:05Z","receivedAt":"2024-08-30T12:52:06Z","isPatch":false,"sender":{"key":"r.sandu@gaijin.team","avatar":null},"body":"Yes, I am aware that the index will be fully refreshed on the first run \nof status. That is completely acceptable. But that is not what I am \nobserving, it is being refreshed on every single run of `git status`!\n\nAfter running stat before and after a status, the sha256 is identical. \nBoth for files and for folders. Maybe Windows has somehow corrupted the \nindex with its negative aura which makes git invalidate it on every \nsingle run? Are there tools in git to diagnose the reason for the \nindex's cache being invalidated?\n\nOn the topic of Unix permissions -- as far as I can tell, they ARE in \nfact working, because I've created a UserMapping file that tells NTFS-3G \nhow to map between NTFS/Windows user UUIDs and Linux user IDs. See \nNTFS-3G's github page (https://github.com/tuxera/ntfs-3g).\n\nOn 8/30/24 03:47, brian m. carlson wrote:\n> On 2024-08-29 at 20:43:40, Roman Sandu wrote:\n>> Good day!\n>>\n>> I have a decently sized (80K files) monorepo on an NTFS drive that I have\n>> been working with for a while under Windows via git-for-windows. Recently, I\n>> had to (temporarily) switch to Ubuntu (24.04) via dual-boot for irrelevant\n>> reasons, and I decided that simply mounting my NTFS drive and using the\n>> monorepo from Ubuntu is a great idea, actually, as NTFS-3G allow for\n>> seamless interop with NTFS via UserMapping. And so that is exactly what I\n>> did and It Just Works!\n> \n> In general, I would not recommend this.  NTFS doesn't support Unix\n> permissions, so I'd expect a lot of things to be broken.  A lot of\n> people like using NTFS to share data across Windows and Linux, but UDF\n> is a much better choice and I'm not surprised that NTFS isn't working\n> the way you'd expect.\n> \n> Also, when you share a repository across systems, you should expect the\n> index to be fully refreshed each time you change the OS, reading every\n> file in the repository[0].\n> \n>> So, what do I do about this? It honestly looks like a git bug to me, maybe\n>> it silently fails to update the index with new timestamps because it was\n>> initially created on Windows? But I have no clue how to narrow this issue\n>> down further, so any ideas or suggestions would be appreciated!\n> \n> Can you pick some file in your repository and run `stat` on it, before\n> and after running `git status`, and include the output?\n> \n> For example:\n> \n>    stat http.c | tee /tmp/before\n>    git status\n>    stat http.c | tee /tmp/after\n>    sha256sum /tmp/before /tmp/after\n> \n> My guess is that NTFS-3G is not emulating something properly and it's\n> differing at some point.\n> \n> [0] https://git-scm.com/docs/gitfaq#sync-working-tree\n\n"},{"id":"501924","messageId":"xmqq1q26t1pa.fsf@gitster.g","threadId":"62021","inReplyTo":"b282c83c-2fa2-4e8e-b8bf-d42f28c17439@kdbg.org","subject":"Re: Committing crimes with NTFS-3G","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-30T16:28:17Z","receivedAt":"2024-08-30T16:28:26Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Sixt <j6t@kdbg.org> writes:\n\n> Am 29.08.24 um 22:43 schrieb Roman Sandu:\n>> To diagnose the problem, I ran git status with GIT_TRACE_PERFORMANCE\n>> enabled, and what I see is that the \"refresh index\" region is taking up\n>> 99% of the time.\n>\n> Of course. The stat information that Git on Linux caches in the index is\n> vastly different from that that Git for Windows caches. So every time\n> you switch OS, all files appear modified to Git.\n>\n> I suggest that you don't switch OS on a whim and take the 8 seconds\n> delay once when you have to.\n\nI somehow got an impression that the hit is not just that we need to\nadjust cached lstat information in the index file once to the new\nfilesystem implementation after an OS switch, but every time (as if\nwe are forced to be extra careful and rehash every time until the\nthings improve, somewhat like how we handle the racy-Git situation).\nTimestamps given by these OSes are not consistent and the clock\nappears to have rewound, or something?\n\nTimestamps of files in the working tree ordinarily should match\ntimestamps in the cached lstat information of these paths in the\nindex, and timestamp of the index file itself should be newer than\nany of the above, or the recy-Git prevention code may tell us to\nplay safe.\n\nI do not do Windows and/or NTFS, but I have to wonder if the smudge\nfilters (including the EOL conversion) play a role in this situation\nas the working tree is getting switched between LF-native and\nCRLF-native systems.  May there be situations where the system must\nspend time only to realize that there is nothing it needs to do to\ncanonicalize the file contents and there is no modifications between\nthe HEAD commit, the index, and the working tree files, or something\nsilly like that?\n\nThanks.\n\n"},{"id":"501931","messageId":"ZtHe8Vi9aRmY-UMI@tapette.crustytoothpaste.net","threadId":"62021","inReplyTo":"3eda2715-3b44-469e-85d5-f25504212928@gaijin.team","subject":"Re: Committing crimes with NTFS-3G","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-08-30T15:02:09Z","receivedAt":"2024-08-30T19:01:44Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-08-30 at 12:52:05, Roman Sandu wrote:\n> Yes, I am aware that the index will be fully refreshed on the first run of\n> status. That is completely acceptable. But that is not what I am observing,\n> it is being refreshed on every single run of `git status`!\n> \n> After running stat before and after a status, the sha256 is identical. Both\n> for files and for folders. Maybe Windows has somehow corrupted the index\n> with its negative aura which makes git invalidate it on every single run?\n> Are there tools in git to diagnose the reason for the index's cache being\n> invalidated?\n\nIt would still be helpful to see the output of the `stat` command, since\nthat would tell us useful things about what's causing Git to think the\ndata has changed.  For example, some systems lack certain timestamp\ngranularity, which can break Git when compiled in certain ways.\n\nYou can see if setting `core.trustctime` to false fixes it, and you can\nalso try `core.checkStat` to `minimal` as well.  You should try them in\nthat order to see if they fix things.\n\nAlso, what version of Git are you using?  Is it the one in Ubuntu 24.04,\nor the one from the git-core PPA, or a different one?\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"501932","messageId":"fe9d19fa-661d-4358-aa0a-ea419c419266@gaijin.team","threadId":"62021","inReplyTo":"ZtHe8Vi9aRmY-UMI@tapette.crustytoothpaste.net","subject":"Re: Committing crimes with NTFS-3G","fromName":"Roman Sandu","fromEmail":"r.sandu@gaijin.team","sentAt":"2024-08-30T19:25:56Z","receivedAt":"2024-08-30T19:25:59Z","isPatch":false,"sender":{"key":"r.sandu@gaijin.team","avatar":null},"body":"The stat output for a random file in the root of the repository is as \nfollows:\n```\n   File: <CENSORED>\n   Size: 91876     \tBlocks: 184        IO Block: 4096   regular file\nDevice: 259,2\tInode: 4630629     Links: 1\nAccess: (0664/-rw-rw-r--)  Uid: ( 1000/romasandu)   Gid: ( 1000/romasandu)\nAccess: 2024-08-29 17:41:04.855126300 +0300\nModify: 2024-08-29 17:41:04.855609000 +0300\nChange: 2024-08-29 17:41:04.855609000 +0300\n  Birth: -\n```\nMaybe lack of a birth stat is what drives git crazy?\n\nMy git version is 2.43.0, which is the one from the Ubuntu 24.04 repo.\n\nNeither `core.trustctime` nor `core.checkStat minimal` (nor a \ncombination of them) help.\n\n\nOn 8/30/24 18:02, brian m. carlson wrote:\n> On 2024-08-30 at 12:52:05, Roman Sandu wrote:\n>> Yes, I am aware that the index will be fully refreshed on the first run of\n>> status. That is completely acceptable. But that is not what I am observing,\n>> it is being refreshed on every single run of `git status`!\n>>\n>> After running stat before and after a status, the sha256 is identical. Both\n>> for files and for folders. Maybe Windows has somehow corrupted the index\n>> with its negative aura which makes git invalidate it on every single run?\n>> Are there tools in git to diagnose the reason for the index's cache being\n>> invalidated?\n> \n> It would still be helpful to see the output of the `stat` command, since\n> that would tell us useful things about what's causing Git to think the\n> data has changed.  For example, some systems lack certain timestamp\n> granularity, which can break Git when compiled in certain ways.\n> \n> You can see if setting `core.trustctime` to false fixes it, and you can\n> also try `core.checkStat` to `minimal` as well.  You should try them in\n> that order to see if they fix things.\n> \n> Also, what version of Git are you using?  Is it the one in Ubuntu 24.04,\n> or the one from the git-core PPA, or a different one?\n\n"},{"id":"501934","messageId":"ZtHrV5dOlTpCDV8z@tapette.crustytoothpaste.net","threadId":"62021","inReplyTo":"fe9d19fa-661d-4358-aa0a-ea419c419266@gaijin.team","subject":"Re: Committing crimes with NTFS-3G","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-08-30T15:55:03Z","receivedAt":"2024-08-30T19:54:38Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-08-30 at 19:25:56, Roman Sandu wrote:\n> The stat output for a random file in the root of the repository is as\n> follows:\n> ```\n>   File: <CENSORED>\n>   Size: 91876     \tBlocks: 184        IO Block: 4096   regular file\n> Device: 259,2\tInode: 4630629     Links: 1\n> Access: (0664/-rw-rw-r--)  Uid: ( 1000/romasandu)   Gid: ( 1000/romasandu)\n> Access: 2024-08-29 17:41:04.855126300 +0300\n> Modify: 2024-08-29 17:41:04.855609000 +0300\n> Change: 2024-08-29 17:41:04.855609000 +0300\n>  Birth: -\n> ```\n> Maybe lack of a birth stat is what drives git crazy?\n\nThat doesn't exist in POSIX, so it isn't used in Git.\n\nI looked at the Ubuntu git package and it doesn't use `USE_NSEC`, so\nyour lack of nanosecond precision in timestamps probably isn't the\nproblem here.\n\nYou may want to try using a utility like\nhttps://github.com/shogo82148/git-dump-index to dump the index and find\nout what might have changed.  You can use `stat -c` to write the data\nfor the actual files in the same format, and then run `diff` on the two\nto find out where they disagree.  Or, perhaps you can just eyeball it,\nin case there's something obvious (like a `uid` difference).\n\nOr, you could try instrumenting `match_stat_data` or\n`stat_validity_check` in `statinfo.c` and printing the data that's\nchanged.\n\nYou might also try disabling untracked cache and see if that fixes the\nproblem.  It might be that there _is_ a bug in that the untracked cache\ninformation isn't correctly refreshed when it was originally written on\na different platform.  It's known that Windows writes different\ninformation into the index than Unix systems and perhaps that\ninformation doesn't get refreshed properly.\n\nOne other thought: Windows stores symlinks with a different size than\nmost Unix systems.  Windows tends to give them a full block size,\nwhereas Unix gives them a size of their length in bytes.  That\ndefinitely breaks using symlinks in a repository across Windows and WSL.\nI don't know if that's what's going on here, but of course it could be\nrelated.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"501944","messageId":"b0fc652d-6a60-4ff8-9242-1fdf826d6339@gaijin.team","threadId":"62021","inReplyTo":"ZtHrV5dOlTpCDV8z@tapette.crustytoothpaste.net","subject":"Re: Committing crimes with NTFS-3G","fromName":"Roman Sandu","fromEmail":"r.sandu@gaijin.team","sentAt":"2024-08-30T22:00:27Z","receivedAt":"2024-08-30T22:00:35Z","isPatch":false,"sender":{"key":"r.sandu@gaijin.team","avatar":null},"body":"So I couldn't figure out how to make stat show me exactly the same \nformat as the utility you have linked, but everything seems to match up \n(up to nsec c/m time which I could not get from stat).\n\nBUT!\nThe Windows index is definitely corrupted. The utility crashes on it. \nAfter modifying the utility to compare `entry->ce_namelen` against \n`strlen(entry->name)`, I found out that they differ for a bunch of \nentries, which at some point causes an unfortunate jump which lands \noutside of the index.\n\nHence a question: shouldn't git validate this somehow? I get that the \nlength of the name is stored separately for speed, but maybe we can have \na special \"validate the index\" subcommand?\n\nI have fixed the wrong name lengths via the same utility, hoping that it \nwould help. But sadly, it didn't.\n\nModifying the script further, I made it stat every single cache entry's \nactual file and compare everything. Et voila: mode differs! Git for \nwindows apparently defaults everything to 644, while NTFS-3G tries to \nsupport actual permissions with UserMapping enabled and so some files \nhave 664, while others have 777, and more for other files on the drive \nbut not in the repo.\n\nBut alas, backing up the index and changing the mode field to what stat \nactually reports didn't help either. It still seems to me like git \nshould be updating this stuff on it's own if it needs to keep track of \nit, but whatever, the issue seems to lie somewhere else.\n\nAll in all, definitely seems like a git bug to me. Especially \nconsidering the name length corruption. I guess I'll try to check out \nthe git sources sometime in the future and play around with them, maybe \nI'll find something then. For now, I will use the Linux-native checkout \nof my repo and be careful to synchronize the two checkouts via remote \nand not forget any unpushed commits. The crime was not perfect after all =(\n\nOn 8/30/24 18:55, brian m. carlson wrote:\n> On 2024-08-30 at 19:25:56, Roman Sandu wrote:\n>> The stat output for a random file in the root of the repository is as\n>> follows:\n>> ```\n>>    File: <CENSORED>\n>>    Size: 91876     \tBlocks: 184        IO Block: 4096   regular file\n>> Device: 259,2\tInode: 4630629     Links: 1\n>> Access: (0664/-rw-rw-r--)  Uid: ( 1000/romasandu)   Gid: ( 1000/romasandu)\n>> Access: 2024-08-29 17:41:04.855126300 +0300\n>> Modify: 2024-08-29 17:41:04.855609000 +0300\n>> Change: 2024-08-29 17:41:04.855609000 +0300\n>>   Birth: -\n>> ```\n>> Maybe lack of a birth stat is what drives git crazy?\n> \n> That doesn't exist in POSIX, so it isn't used in Git.\n> \n> I looked at the Ubuntu git package and it doesn't use `USE_NSEC`, so\n> your lack of nanosecond precision in timestamps probably isn't the\n> problem here.\n> \n> You may want to try using a utility like\n> https://github.com/shogo82148/git-dump-index to dump the index and find\n> out what might have changed.  You can use `stat -c` to write the data\n> for the actual files in the same format, and then run `diff` on the two\n> to find out where they disagree.  Or, perhaps you can just eyeball it,\n> in case there's something obvious (like a `uid` difference).\n> \n> Or, you could try instrumenting `match_stat_data` or\n> `stat_validity_check` in `statinfo.c` and printing the data that's\n> changed.\n> \n> You might also try disabling untracked cache and see if that fixes the\n> problem.  It might be that there _is_ a bug in that the untracked cache\n> information isn't correctly refreshed when it was originally written on\n> a different platform.  It's known that Windows writes different\n> information into the index than Unix systems and perhaps that\n> information doesn't get refreshed properly.\n> \n> One other thought: Windows stores symlinks with a different size than\n> most Unix systems.  Windows tends to give them a full block size,\n> whereas Unix gives them a size of their length in bytes.  That\n> definitely breaks using symlinks in a repository across Windows and WSL.\n> I don't know if that's what's going on here, but of course it could be\n> related.\n\n"},{"id":"502043","messageId":"20240903155818.GA9437@tb-raspi4","threadId":"62021","inReplyTo":"xmqq1q26t1pa.fsf@gitster.g","subject":"Re: Committing crimes with NTFS-3G","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2024-09-03T15:58:18Z","receivedAt":"2024-09-03T15:58:29Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Fri, Aug 30, 2024 at 09:28:17AM -0700, Junio C Hamano wrote:\n> Johannes Sixt <j6t@kdbg.org> writes:\n>\n> > Am 29.08.24 um 22:43 schrieb Roman Sandu:\n> >> To diagnose the problem, I ran git status with GIT_TRACE_PERFORMANCE\n> >> enabled, and what I see is that the \"refresh index\" region is taking up\n> >> 99% of the time.\n> >\n> > Of course. The stat information that Git on Linux caches in the index is\n> > vastly different from that that Git for Windows caches. So every time\n> > you switch OS, all files appear modified to Git.\n> >\n> > I suggest that you don't switch OS on a whim and take the 8 seconds\n> > delay once when you have to.\n>\n> I somehow got an impression that the hit is not just that we need to\n> adjust cached lstat information in the index file once to the new\n> filesystem implementation after an OS switch, but every time (as if\n> we are forced to be extra careful and rehash every time until the\n> things improve, somewhat like how we handle the racy-Git situation).\n> Timestamps given by these OSes are not consistent and the clock\n> appears to have rewound, or something?\n>\n> Timestamps of files in the working tree ordinarily should match\n> timestamps in the cached lstat information of these paths in the\n> index, and timestamp of the index file itself should be newer than\n> any of the above, or the recy-Git prevention code may tell us to\n> play safe.\n>\n> I do not do Windows and/or NTFS, but I have to wonder if the smudge\n> filters (including the EOL conversion) play a role in this situation\n> as the working tree is getting switched between LF-native and\n> CRLF-native systems.  May there be situations where the system must\n> spend time only to realize that there is nothing it needs to do to\n> canonicalize the file contents and there is no modifications between\n> the HEAD commit, the index, and the working tree files, or something\n> silly like that?\n\nNow, that make me wonder:\nWhat are the settings for core.autocrlf ?\nIs there a .gitattributes file ?\nAccording to my experience there should be one, when working cross-platform.\nA version with the single line\n* text=auto\nworks for many projects, and may be fine-tuned to what you need.\n\nThe questin about core.filemode has been raised elsewhere, it should\nbe false for this repo (but probably is).\n\nNow back to the lstat() question:\nThere is a\ngit ls-files --debug\n\nwhich may give more insight about what is going on.\n\nAnd back to the line-endings:\ndoes\ngit ls-files --eol\ngive any hints, may be ?\n\n\n\n\n\n\n\n\n\n\n\n"},{"id":"502049","messageId":"8a8a7f75-6a2f-4fd9-b3cf-df61129a355a@gaijin.team","threadId":"62021","inReplyTo":"20240903155818.GA9437@tb-raspi4","subject":"Re: Committing crimes with NTFS-3G","fromName":"Roman Sandu","fromEmail":"r.sandu@gaijin.team","sentAt":"2024-09-03T17:30:16Z","receivedAt":"2024-09-03T17:30:24Z","isPatch":false,"sender":{"key":"r.sandu@gaijin.team","avatar":null},"body":".gitattributes is this \nhttps://github.com/GaijinEntertainment/DagorEngine/blob/main/.gitattributes\n`core.autocrlf` is unset.\n\ncore.filemode is false.\n\n`git ls-files --debug` is extremely useful and I am a tiny bit salty \nthat no one suggested it before I spent time learning how to parse the \nindex file in C =)\nBut alas, nothing new, see my previous messages (hopefully they are \nvisible to you, I am kind of new to mailing lists).\n\n`git ls-files --eol` says that worktree uses crlf (which is expected, I \nchecked out the repo under windows), while the index uses lf. Can that \nbe the reason for Linux git thinking that the index is always out of \ndate? Various dev tools work perfectly fine with crlf'd files on Linux.\n\nOn 9/3/24 18:58, Torsten Bögershausen wrote:\n> On Fri, Aug 30, 2024 at 09:28:17AM -0700, Junio C Hamano wrote:\n>> Johannes Sixt <j6t@kdbg.org> writes:\n>>\n>>> Am 29.08.24 um 22:43 schrieb Roman Sandu:\n>>>> To diagnose the problem, I ran git status with GIT_TRACE_PERFORMANCE\n>>>> enabled, and what I see is that the \"refresh index\" region is taking up\n>>>> 99% of the time.\n>>>\n>>> Of course. The stat information that Git on Linux caches in the index is\n>>> vastly different from that that Git for Windows caches. So every time\n>>> you switch OS, all files appear modified to Git.\n>>>\n>>> I suggest that you don't switch OS on a whim and take the 8 seconds\n>>> delay once when you have to.\n>>\n>> I somehow got an impression that the hit is not just that we need to\n>> adjust cached lstat information in the index file once to the new\n>> filesystem implementation after an OS switch, but every time (as if\n>> we are forced to be extra careful and rehash every time until the\n>> things improve, somewhat like how we handle the racy-Git situation).\n>> Timestamps given by these OSes are not consistent and the clock\n>> appears to have rewound, or something?\n>>\n>> Timestamps of files in the working tree ordinarily should match\n>> timestamps in the cached lstat information of these paths in the\n>> index, and timestamp of the index file itself should be newer than\n>> any of the above, or the recy-Git prevention code may tell us to\n>> play safe.\n>>\n>> I do not do Windows and/or NTFS, but I have to wonder if the smudge\n>> filters (including the EOL conversion) play a role in this situation\n>> as the working tree is getting switched between LF-native and\n>> CRLF-native systems.  May there be situations where the system must\n>> spend time only to realize that there is nothing it needs to do to\n>> canonicalize the file contents and there is no modifications between\n>> the HEAD commit, the index, and the working tree files, or something\n>> silly like that?\n> \n> Now, that make me wonder:\n> What are the settings for core.autocrlf ?\n> Is there a .gitattributes file ?\n> According to my experience there should be one, when working cross-platform.\n> A version with the single line\n> * text=auto\n> works for many projects, and may be fine-tuned to what you need.\n> \n> The questin about core.filemode has been raised elsewhere, it should\n> be false for this repo (but probably is).\n> \n> Now back to the lstat() question:\n> There is a\n> git ls-files --debug\n> \n> which may give more insight about what is going on.\n> \n> And back to the line-endings:\n> does\n> git ls-files --eol\n> give any hints, may be ?\n> \n> \n> \n> \n> \n> \n> \n> \n> \n> \n> \n\n"},{"id":"502280","messageId":"20240905195139.GA9644@tb-raspi4","threadId":"62021","inReplyTo":"8a8a7f75-6a2f-4fd9-b3cf-df61129a355a@gaijin.team","subject":"Re: Committing crimes with NTFS-3G","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2024-09-05T19:51:39Z","receivedAt":"2024-09-05T19:51:50Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Tue, Sep 03, 2024 at 08:30:16PM +0300, Roman Sandu wrote:\n\nSide note: Please avoid top-posting here in this list\n\n> .gitattributes is this\n> https://github.com/GaijinEntertainment/DagorEngine/blob/main/.gitattributes\n> `core.autocrlf` is unset.\n\nThat is according to the book, good.\n\n>\n> core.filemode is false.\n>\n> `git ls-files --debug` is extremely useful and I am a tiny bit salty that no\n> one suggested it before I spent time learning how to parse the index file in\n> C =)\n\nNo well, I read this as \"thanks for the hint\"\n\n> But alas, nothing new, see my previous messages (hopefully they are visible\n> to you, I am kind of new to mailing lists).\n\nYes, everything is here.\n>\n> `git ls-files --eol` says that worktree uses crlf (which is expected, I\n> checked out the repo under windows), while the index uses lf. Can that be\n> the reason for Linux git thinking that the index is always out of date?\n> Various dev tools work perfectly fine with crlf'd files on Linux.\n\nYes, that is not the problem for any performance issues.\n\n\nI did set up a test, kind of.\nExporting a git repo from a Linux NAS to a MacOs laptop, using Samba,\nand WLAN.\nA simple `git status` takes 4 seconds for 1800 files.\nThe thing is that Git needs to lstat() each and every file, to find\nout if it has been changed.\nAnd if that lstat() call is slow, then `git status` will be slow.\n\nI don't know, if the fs monitor works for your setup ?\n\nhttps://git-scm.com/docs/git-fsmonitor--daemon\n\n[snip]\n"}]}