{"thread":{"id":"56159","subject":"Files modified, even after: git reset --hard","startedAt":"2021-07-25T15:04:12Z","lastAt":"2021-07-27T00:55:18Z","messageCount":16,"participants":["Martin","Chris Torek","Philip Oakley"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"431150","messageId":"dd4aca2c-9ca2-e489-d78f-9d2a5580f1a5@mfriebe.de","threadId":"56159","inReplyTo":null,"subject":"Files modified, even after: git reset --hard","fromName":"Martin","fromEmail":"git@mfriebe.de","sentAt":"2021-07-25T15:04:08Z","receivedAt":"2021-07-25T15:04:12Z","isPatch":false,"sender":{"key":"git@mfriebe.de","avatar":null},"body":"I have some files, that will show up modified. Always.\n\nIf I just switch to a commit, with clean worktree before, then those \nfiles will be modified.\n\nI can not stash them.\ngit reset --hard does not change anything.\n\nDiff shows the entire file is modified. I suspect its the line endings..\n\nThis happens on windows, and\n   autocrlf=true\n\nThe checked out file (which is marked as modified) has the correct CrLf \nendings.\n\nWhat I have not been able to find, is what line endings are stored in \nthe blob.\nIts an xml file, so they should be just Lf.\nBut I *suspect* that the blob contains either CrLf, or mixed line-endings.\n\nCould that be? that if a file does have unexpected line endings in the \ncommit's blob, that it shows as modified?\n\nOne the one hand, yes, If I commit, that file will change.\nOn the other hand, if I just check out (or do reset-hard) then I don't \nexpect modified files....\n\nSo what should happen?\n\nThis also causes problems, because in order to for example rebase \nsomething, I first need to switch to some commit that can be checked out \nwithout modified files.\nOr rebase will not work.\n\nAlso in that repro, I had problems that some rebases failed with (with \nand without --reapply-cherry-picks)\n    error: add_cacheinfo failed\nThose rebase where between an orphaned branch, and a \"normal\" branch.\nAnd it is possible that the file only had the line ending issues in one \nof the 2 branches....\nBut that I was not able to further investigate.\n\n\n\n//git for windows\ngit version 2.32.0.windows.2\n\n"},{"id":"431151","messageId":"4e9b54b4-8e40-7fd3-ae65-d33390f3af43@mfriebe.de","threadId":"56159","inReplyTo":"dd4aca2c-9ca2-e489-d78f-9d2a5580f1a5@mfriebe.de","subject":"Re: Files modified, even after: git reset --hard","fromName":"Martin","fromEmail":"git@mfriebe.de","sentAt":"2021-07-25T15:40:53Z","receivedAt":"2021-07-25T15:41:00Z","isPatch":false,"sender":{"key":"git@mfriebe.de","avatar":null},"body":"On 25/07/2021 17:04, Martin wrote:\n> I have some files, that will show up modified. Always.\n>\n> If I just switch to a commit, with clean worktree before, then those \n> files will be modified.\n>\n...\n> But I *suspect* that the blob contains either CrLf, or mixed \n> line-endings.\n\nAlso, if that is the case, what can I do to resolve the issue?\nThe file has been fixed in the meantime. It is about the old commits....\n\nfilter branch is not an option. All commit hashes must be kept. No force \npushes to the repro...\n\nThat also means I can not use .gitattribute, because I could not amend \nit for the old revisions.\nI might be able to use my local $GIT_DIR/info/attributes.\nBut then I risk to make new commits to that file, and introduce new \nwrong line-endings.\n\nMaybe git replace?\nI figure if I do not replace the commit objects, but only the blobs to \nwhich they point. Then all commits should keep their hashes. (so \nreferences to any commit by hash would remain working)\n\nBut it's thousands of commits, between the introduction of the issue, \nand the commit where it's fixed.\nSo I guess I need to create as many new blobs, and replacement entries.\n- Is that practical? How does git perform with such many replacements\n- Is there an easy way to create them?\n\nAre there other, better solutions?\n"},{"id":"431152","messageId":"04f3b300-3ccf-c91b-6406-6a998b473a24@mfriebe.de","threadId":"56159","inReplyTo":"4e9b54b4-8e40-7fd3-ae65-d33390f3af43@mfriebe.de","subject":"Re: Files modified, even after: git reset --hard","fromName":"Martin","fromEmail":"git@mfriebe.de","sentAt":"2021-07-25T17:48:52Z","receivedAt":"2021-07-25T17:49:01Z","isPatch":false,"sender":{"key":"git@mfriebe.de","avatar":null},"body":"On 25/07/2021 17:40, Martin wrote:\n> On 25/07/2021 17:04, Martin wrote:\n>> I have some files, that will show up modified. Always.\n>>\n>> If I just switch to a commit, with clean worktree before, then those \n>> files will be modified.\n>>\n> ...\n>> But I *suspect* that the blob contains either CrLf, or mixed \n>> line-endings.\n>\n> Maybe git replace?\n> I figure if I do not replace the commit objects, but only the blobs to \n> which they point. Then all commits should keep their hashes. (so \n> references to any commit by hash would remain working)\n>\n> But it's thousands of commits, between the introduction of the issue, \n> and the commit where it's fixed.\n\n\nActually, git replace has a weird effect.\n\nso I have \"commit A\" introducing the problem. Then many commits until \n\"commit Y\" fixing it.\n\nIf I do\nswitch -f --detach A\ngit add --renormalize the_file\ngit commit -m foo\ngit show   // get the new blob for the file\ngit show A  // get the old blob for the file\ngit replace  old_blob  new_blob\n\nswitch -f --detach A\n\nThen \"the_file\" is still shown as modified.\nOnly\n   git diff the_file\nnow returns empty\n(where as before it returned the entire file)\n\nOn the plus side\n   switch -f --detach B\nhas the same change, still modified, but no diff.\nSo one replace would solve the entire 1000 commits that are currently \naffected\n\nAny idea, what causes the \"modified\" after the replace?\n\n\n\n"},{"id":"431153","messageId":"bfc257c7-bf74-06be-ac62-9a6d27f565c9@mfriebe.de","threadId":"56159","inReplyTo":"04f3b300-3ccf-c91b-6406-6a998b473a24@mfriebe.de","subject":"Re: Files modified, even after: git reset --hard","fromName":"Martin","fromEmail":"git@mfriebe.de","sentAt":"2021-07-25T18:39:21Z","receivedAt":"2021-07-25T18:39:25Z","isPatch":false,"sender":{"key":"git@mfriebe.de","avatar":null},"body":"On 25/07/2021 19:48, Martin wrote:\n>\n> Actually, git replace has a weird effect.\n>\n> so I have \"commit A\" introducing the problem. Then many commits until \n> \"commit Y\" fixing it.\n>\n> If I do\n> switch -f --detach A\n> git add --renormalize the_file\n> git commit -m foo\n> git show   // get the new blob for the file\n> git show A  // get the old blob for the file\n> git replace  old_blob  new_blob\n>\n> switch -f --detach A\n>\n> Then \"the_file\" is still shown as modified.\n> Only\n>   git diff the_file\n> now returns empty\n> (where as before it returned the entire file)\n>\n> On the plus side\n>   switch -f --detach B\n> has the same change, still modified, but no diff.\n> So one replace would solve the entire 1000 commits that are currently \n> affected\n>\n> Any idea, what causes the \"modified\" after the replace?\n>\n\nOk, it seem that\n   git switch\nsimple did not update the file.\n\n- After the commit I am on a dangling commit, which is a direct child of \n\"A\".\n- I insert the reprace\n- I switch from that dangling to \"A\"\n\n-> diff is empty\n-> but status is not ok\n\nIf I now switch to a far away commit. And then switch back to \"A\". Then \ngit status is also ok.\n\n\n\n"},{"id":"431154","messageId":"CAPx1GvcHiaGsuOybOijRYpmivO0dLvUFacAeOrM4DfY-uuXB2Q@mail.gmail.com","threadId":"56159","inReplyTo":"bfc257c7-bf74-06be-ac62-9a6d27f565c9@mfriebe.de","subject":"Re: Files modified, even after: git reset --hard","fromName":"Chris Torek","fromEmail":"chris.torek@gmail.com","sentAt":"2021-07-26T00:33:44Z","receivedAt":"2021-07-26T00:34:00Z","isPatch":false,"sender":{"key":"chris.torek@gmail.com","avatar":"https://avatars.githubusercontent.com/u/16826774?v=4"},"body":"On Sun, Jul 25, 2021 at 11:43 AM Martin <git@mfriebe.de> wrote:\n>>>> [Files show up as every-line-modified]\n>> [and] git replace has a weird effect.\n> Ok, it seem that\n>    git switch\n> simple did not update the file.\n\nHere's what's going on.\n\n(1) No *committed* object, including committed file data, can ever\n    be changed.\n\n(2) Someone actually committed some files with some sort of line\n    endings.\n\n(Points 1 and 2 are true regardless of anything else.)\n\nIf you never have Git mess with line endings at all, neither of\nthese two points make any difference.  But you chose to start\nasking Git to mess with line endings.  Things now depend on\nexactly *what* you ask Git to *do* about these line endings.\n\nLet's suppose, just for convenience for now, that the files in the\nrepository right now actually do have CRLF line endings.\n\nLet's suppose further that you ask that Git ensure that your\n*working tree* copies of each file contain CRLF line endings.\n\nThis means that existing *committed* copies of files already\nhave the ending you prefer.  But now that you have started to\nask Git to *mess with your line endings* -- by asking that your\n*working tree* copies have CRLF endings -- Git will now begin\nmaking sure that *future committed copies of files* have LF-only\nline endings.\n\nWhat this means is that any existing commit you check out has\nthe \"wrong\" line endings!  You've just told Git that all the\n*committed* files should have LF-only line endings.\n\nYou might ask why \"ensure my working tree copies have CRLF line\nendings\" means \"ensure all future committed files have LF-only\nline endings\".  That's a valid design question.  But it is the\ncase.\n\nAs for \"git replace\", you've figured the rest out already: if\nyou use git replace to make Git use new, LF-only line ending\nobjects (file data), Git is now happy about the internal storage.\nIt just takes some shuffling-about to cause these replaced objects\nto wind up in Git's *index* AKA *staging area*.\n\nI'll mostly stop here, because this explains the results you have\nso far, but in general, setting `core.autocrlf` tends to be\ninferior to making explicit decisions about particular files\nvia `.gitatttributes` files.\n\nChris\n"},{"id":"431155","messageId":"070f7f5e-0e6c-2edc-1403-9265c810df17@mfriebe.de","threadId":"56159","inReplyTo":"CAPx1GvcHiaGsuOybOijRYpmivO0dLvUFacAeOrM4DfY-uuXB2Q@mail.gmail.com","subject":"Re: Files modified, even after: git reset --hard","fromName":"Martin","fromEmail":"git@mfriebe.de","sentAt":"2021-07-26T01:34:25Z","receivedAt":"2021-07-26T01:34:31Z","isPatch":false,"sender":{"key":"git@mfriebe.de","avatar":null},"body":"On 26/07/2021 02:33, Chris Torek wrote:\n> On Sun, Jul 25, 2021 at 11:43 AM Martin <git@mfriebe.de> wrote:\n>>>>> [Files show up as every-line-modified]\n>>> [and] git replace has a weird effect.\n>> Ok, it seem that\n>>     git switch\n>> simple did not update the file.\n> \n...\n> Let's suppose, just for convenience for now, that the files in the\n> repository right now actually do have CRLF line endings.\n> \n> Let's suppose further that you ask that Git ensure that your\n> *working tree* copies of each file contain CRLF line endings.\n> \nYes, I accept that decision.\nI figured that is the reason why they show modified.\n\nNot a problem. Until I am in the middle of a rebase, and i cannot run \n(after a conflict)\n   git rebase --continue\n\nThe modified files are not part of the original series of commits. they \nare just random files from somewhere else in the tree.\nI can not reset/restore them.\nSo I must now \"git add\" files entirely unrelated to continue rebasing.\nWell or apparently change my config for the duration of the rebase.\n\n\n> As for \"git replace\", you've figured the rest out already: if\n> you use git replace to make Git use new, LF-only line ending\n> objects (file data), Git is now happy about the internal storage.\n> It just takes some shuffling-about to cause these replaced objects\n> to wind up in Git's *index* AKA *staging area*.\n\nActually there is something else.\n\nIf a file has line-endings that will change, then\n    git add --renormalize .\n    git commit -m foo\nwill commit those files.\n\nBut I am now also getting files, that show modified, but that can not be \ncommitted renormalized (0 lines changed).\n\nAnd that happens with or without refs/replaces.\n\nAny idea how to find out why git thinks they are modified?\n    git status --porcelain=v2\nshows that the file mode is not modified. Only the file in the working tree.\nBut \"git diff\" shows nothing (no summary neither). And renormalizing has \nno effect.\n\n\n\nIn fact I started running the following\n   git rev-list --reverse main | xargs -L 1 sh -c 'git switch --detach \n-q -f $0 ; a=$( git status -uno --porcelain=v1 ) ; if [ \"$a\" != \"\" ]; \nthen git log --oneline -n1 $0 ; echo $a; fi '\n\nThat is, switch to each revision in main (or master). And check if any \nfile is reported modified.\n\nI just tried that on the gdb git. Plenty of files.\nAlso other repros have shown modified files.\n(I have not yet tried the \"git sources\" git...\n\nIf I then manually switch to some of the commits that had modified files \nshown, and I do not switch to all the commits before, then sometimes \nthere are no modified files.\n\ngit fsck has a few dangling commits\ngit gc made no difference.\n\n"},{"id":"431156","messageId":"CAPx1GvdM7CzsbT1SWW9+OPcG9FL7WXQ7YD6aM7P0krujp_OrkQ@mail.gmail.com","threadId":"56159","inReplyTo":"070f7f5e-0e6c-2edc-1403-9265c810df17@mfriebe.de","subject":"Re: Files modified, even after: git reset --hard","fromName":"Chris Torek","fromEmail":"chris.torek@gmail.com","sentAt":"2021-07-26T02:59:03Z","receivedAt":"2021-07-26T02:59:19Z","isPatch":false,"sender":{"key":"chris.torek@gmail.com","avatar":"https://avatars.githubusercontent.com/u/16826774?v=4"},"body":"On Sun, Jul 25, 2021 at 6:34 PM Martin <git@mfriebe.de> wrote:\n> Actually there is something else.\n>\n> If a file has line-endings that will change, then\n>     git add --renormalize .\n>     git commit -m foo\n> will commit those files.\n>\n> But I am now also getting files, that show modified, but that can not be\n> committed renormalized (0 lines changed).\n\nI believe (but can't demonstrate) that this is a temporary condition.\n\nGit has a number of cheats to make `git status` and other ops fast.\nThese cheats *assume* that the committed files, the index copies\nof files, and the working tree copies of files all agree in terms of\nline endings as coordinated through `core.autocrlf` and `.gitattributes`\nsettings.\n\nWhen they *don't* agree, you get phantom differences.  Running\ncommands like `git diff` show no differences because of these\nphantom states.  Eventually this clears up on its own when the\ncheats really *do* agree with the settings.  Changing the settings\nis what disturbs the cheats.\n\nGit can't do much with `core.autocrlf`, but if it noticed that a\n`.gitattributes` file was very recent, and turned off the shortcuts\nand did the slower full status checks, updates via `.gitattributes`\nwould not show phantom changes.  The drawback is that updates\nto `.gitattributes` could make `git status` very slow.\n\nOverall this isn't normally a big problem.  It only affects one person\nat a time, when they change these settings, and then it clears up\nover time...\n\nChris\n"},{"id":"431173","messageId":"f454bf5b-c5ff-4140-02a8-b02dcd35c6b9@mfriebe.de","threadId":"56159","inReplyTo":"CAPx1GvdM7CzsbT1SWW9+OPcG9FL7WXQ7YD6aM7P0krujp_OrkQ@mail.gmail.com","subject":"Re: Files modified, even after: git reset --hard","fromName":"Martin","fromEmail":"git@mfriebe.de","sentAt":"2021-07-26T10:31:37Z","receivedAt":"2021-07-26T10:31:41Z","isPatch":false,"sender":{"key":"git@mfriebe.de","avatar":null},"body":"On 26/07/2021 04:59, Chris Torek wrote:\n> On Sun, Jul 25, 2021 at 6:34 PM Martin <git@mfriebe.de> wrote:\n>> Actually there is something else.\n>>\n>> If a file has line-endings that will change, then\n>>      git add --renormalize .\n>>      git commit -m foo\n>> will commit those files.\n>>\n>> But I am now also getting files, that show modified, but that can not be\n>> committed renormalized (0 lines changed).\n> \n> I believe (but can't demonstrate) that this is a temporary condition.\n\nI now found one, that does not seem temporary at all...\n\ngit remote -v\ngh-me   https://github.com/User4martin/binutils-gdb.git (fetch)\ngh-me   https://github.com/User4martin/binutils-gdb.git (push)\ngh-redprig      https://github.com/red-prig/binutils-gdb.git (fetch)\ngh-redprig      https://github.com/red-prig/binutils-gdb.git (push)\norigin  git://sourceware.org/git/binutils-gdb.git (fetch)\norigin  git://sourceware.org/git/binutils-gdb.git (push)\n\nfsck had some dangling commits, I did a \"git gc --aggressive\".\nNow\n  git fsck\nChecking object directories: 100% (256/256), done.\nChecking objects: 100% (1062929/1062929), done.\nChecking connectivity: 1062929, done.\n\nSwitching to a far away commit\ngit switch -f --detach master\ngit rev-list master | wc -l\n93917\n\ngit status shows no modified files\n\n\nThen switching to  (this is on master branch)\ngit switch -f --detach a362ee23634\ngit rev-list a362ee23634 | wc -l\n4164\n\ngit status --porcelain=v2\n1 .M N... 100644 100644 100644 9e677a52ae690808165993a0f3f17ac49e3969df \n9e677a52ae690808165993a0f3f17ac49e3969df bfd/Makefile.dos\n1 .M N... 100644 100644 100644 ff24f19c0b6e0c7fb713c79e8f1765bc22fe7adc \nff24f19c0b6e0c7fb713c79e8f1765bc22fe7adc binutils/Makefile.dos\n1 .M N... 100644 100644 100644 1d9541c2f896842d1bafe68ccf0a51e291d66688 \n1d9541c2f896842d1bafe68ccf0a51e291d66688 gas/Makefile.dos\n1 .M N... 100644 100644 100644 57fab985680ea151379069abe414bcb590cdd743 \n57fab985680ea151379069abe414bcb590cdd743 ld/Makefile.dos\n\n\ngit reset --hard\nmakes no difference.\n\nOnly\n     git diff\nshows actual textual differences\ne.g.\n\n  TARGETLIB = libbfd.a\n-CFLAGS = -g -O $(HDEFINES) $(TDEFINES) $(CSEARCH) $(CSWITCHES) # \n-DINTEL960VERSION\n+CFLAGS = $(MINUS_G) $(HDEFINES) $(TDEFINES) $(CSEARCH) $(CSWITCHES) # \n-DINTEL960VERSION\n\n\n\n\n\n\n\n\n\n\n\n\n> \n> Git has a number of cheats to make `git status` and other ops fast.\n> These cheats *assume* that the committed files, the index copies\n> of files, and the working tree copies of files all agree in terms of\n> line endings as coordinated through `core.autocrlf` and `.gitattributes`\n> settings.\n> \n> When they *don't* agree, you get phantom differences.  Running\n> commands like `git diff` show no differences because of these\n> phantom states.  Eventually this clears up on its own when the\n> cheats really *do* agree with the settings.  Changing the settings\n> is what disturbs the cheats.\n> \n> Git can't do much with `core.autocrlf`, but if it noticed that a\n> `.gitattributes` file was very recent, and turned off the shortcuts\n> and did the slower full status checks, updates via `.gitattributes`\n> would not show phantom changes.  The drawback is that updates\n> to `.gitattributes` could make `git status` very slow.\n\nOk, one of the repros has had its .gitattributes committed on the root \ncommit, and never changed.\nNeither do/did I change the config since it was created.\n\nMy question is are there any plumbing commands, that could allow me to \nlook further at it.\n\n"},{"id":"431174","messageId":"fd2f2e6b-0ced-8eb7-b908-956b084f23c7@iee.email","threadId":"56159","inReplyTo":"070f7f5e-0e6c-2edc-1403-9265c810df17@mfriebe.de","subject":"Re: Files modified, even after: git reset --hard","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2021-07-26T10:39:59Z","receivedAt":"2021-07-26T10:40:08Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"On 26/07/2021 02:34, Martin wrote:\n> I figured that is the reason why they show modified.\n>\n> Not a problem. Until I am in the middle of a rebase, and i cannot run\n> (after a conflict)\n>   git rebase --continue\n>\n> The modified files are not part of the original series of commits.\n> they are just random files from somewhere else in the tree.\n> I can not reset/restore them.\n> So I must now \"git add\" files entirely unrelated to continue rebasing.\n> Well or apparently change my config for the duration of the rebase.\n\nIs this 'mid-rebase' the core case for the 'Files modified' problem? -\ndoes it happen at other times (excepting maybe cherry-pick)\n\ni.e. you are rebasing a series of commits where some files had 'old'\nline endings in the repository, but your current line ending setting\nwants the line endings in those un-related, un-changed files to change\ntheir line endings, and the rebase command can't cope with these\nincidental differences?\n\nPhilip\n"},{"id":"431175","messageId":"CAPx1GvfH0Rqz238GUByBGkrRK8UeBpF3-1XvW_M66_w6DVgYFA@mail.gmail.com","threadId":"56159","inReplyTo":"f454bf5b-c5ff-4140-02a8-b02dcd35c6b9@mfriebe.de","subject":"Re: Files modified, even after: git reset --hard","fromName":"Chris Torek","fromEmail":"chris.torek@gmail.com","sentAt":"2021-07-26T11:11:49Z","receivedAt":"2021-07-26T11:12:04Z","isPatch":false,"sender":{"key":"chris.torek@gmail.com","avatar":"https://avatars.githubusercontent.com/u/16826774?v=4"},"body":"I don't have Windows and hence cannot test your exact\nsituation (which I understand to be:\n\n    `core.autocrlf` set to `true`\n\nand of course \"using Windows\"). I will repeat that I think\nsetting `core.autocrlf` to `true` is a bad idea, though.\n\nIn the meantime:\n\nOn Mon, Jul 26, 2021 at 3:31 AM Martin <git@mfriebe.de> wrote:\n> origin  git://sourceware.org/git/binutils-gdb.git (fetch)\n\nI cloned this repository so as to be able to obtain its commits.\n\n> fsck had some dangling commits, I did a \"git gc --aggressive\".\n\n(Dangling commits are irrelevant and unimportant: there's no need\nto do this.)\n\n> Switching to a far away commit\n> git switch -f --detach master\n> git rev-list master | wc -l\n> 93917\n\nThe actual hash ID of the commit would be potentially useful here\n(whether it's really useful would depend on whether I have or can\nobtain that commit).\n\nI do see that commit 16788ca9fd7352b2b20f4b11111283a8cd4212e2\nremoved various `Makefile.dos` files, including one mentioned\nbelow (`bfd/Makefile.dos`).  That's the only file name I looked at\nspecifically for this kind of operation initially (I checked another\njust now and it too went away in that commit).\n\n> ... Then switching to  (this is on master branch)\n> git switch -f --detach a362ee23634\n\nThe hash ID is all that matters: branch names are irrelevant here.\n\nThis commit contains a `bfd/Makefile.dos`.  The actual line\nendings in this file are LF-only.\n\n> git status --porcelain=v2\n> 1 .M N... 100644 100644 100644 9e677a52ae690808165993a0f3f17ac49e3969df\n> 9e677a52ae690808165993a0f3f17ac49e3969df bfd/Makefile.dos\n\nThis is also useful:\n\n * The `.M` means that the committed copy of the file matches\n   the index copy, while the index copy does not match the working\n   tree copy (presumably because the working tree copy has had\n   its line endings changed to CRLF).\n * The `N...` means that this is not a submodule.\n * The three `100644`s mean that the file is mode 100644 in\n   the commit, the index, and the working tree.\n * The two hash IDs are the blob object hash IDs for the committed\n   and index copies of the file data; these match.\n * Last, we have the file's name in the index.\n\nThe same pattern seems to repeat for the remaining files.\n\nThese \"Makefile.dos\" files seem to be hardly ever touched,\nin any commit, so these four files are likely not to be swapped\nout by `git checkout` or `git switch` operations.\n\n> My question is are there any plumbing commands, that could allow me to\n> look further at it.\n\nThe simplest is to get the raw hash ID of any existing Git-ized object,\nand use `git cat-file -p <hash>` on that object.  Since `git cat-file` has\nno idea whether the data are to be treated as text or binary (in this\ncase where there is no path name and no `--textconv` flag),it just\nwrites out the raw text.  You must, however, make sure that the output\nfrom this command is not mangled by some other tool (e.g., some\nWindows editor \"helpfully\" changing LF-only to CRLF).\n\nNote that `git ls-tree`, with the `-r` (recursive) option, allows you\nto inspect the raw hash IDs of committed files by their path names.\n\nChris\n"},{"id":"431184","messageId":"abfb0196-ff4e-199a-6464-335abf514a63@mfriebe.de","threadId":"56159","inReplyTo":"fd2f2e6b-0ced-8eb7-b908-956b084f23c7@iee.email","subject":"Re: Files modified, even after: git reset --hard","fromName":"Martin","fromEmail":"git@mfriebe.de","sentAt":"2021-07-26T12:50:01Z","receivedAt":"2021-07-26T12:50:05Z","isPatch":false,"sender":{"key":"git@mfriebe.de","avatar":null},"body":"On 26/07/2021 12:39, Philip Oakley wrote:\n> On 26/07/2021 02:34, Martin wrote:\n>> I figured that is the reason why they show modified.\n>>\n>> Not a problem. Until I am in the middle of a rebase, and i cannot run\n>> (after a conflict)\n>>    git rebase --continue\n>>\n>> The modified files are not part of the original series of commits.\n>> they are just random files from somewhere else in the tree.\n>> I can not reset/restore them.\n>> So I must now \"git add\" files entirely unrelated to continue rebasing.\n>> Well or apparently change my config for the duration of the rebase.\n> \n> Is this 'mid-rebase' the core case for the 'Files modified' problem? -\n> does it happen at other times (excepting maybe cherry-pick)\n> \n\nSo far yes.\nPotentially, not yet tested, \"git bisect\" could be affected. That would \nalso be a problem.\nAnd yes, when rebase failed, cherry pick would also fail.\n\n\nI did not see a \"force\" option for either of them, to ignore a dirty \nworktree.\n\n> i.e. you are rebasing a series of commits where some files had 'old'\n> line endings in the repository, but your current line ending setting\n> wants the line endings in those un-related, un-changed files to change\n> their line endings, and the rebase command can't cope with these\n> incidental differences?\n> \n\nI had a series of errors. Some of the errors, are indeed that I can not \ncontinue after resolving conflicts, because I can not clean the worktree.\n(I guess I could set up a .git/info/gitattribute to mark the files as \nbinary, and remove it again after the rebase. But that is not a \ndesirable solution)\n\n\nAs a once off I also got an\n    error: add_cacheinfo failed\nOne branch really could not be rebase by any means. But it was a single \ncommit, so I copied the files.\n\n\n\nI did \"git replace\" now for lots of those files.\nBut some commits had \"modified files\" that appear to have the correct \nline ending in the repro (git add --renormalize did not add changes to \nbe committed / the file is added, but commit says 0 lines changed, and \nthe issue remains).\nSo for those I have no way to get rid of yet.\n\nThose none-line ending issues existed already before I started to git \nreplace.\nI did not replace commits, but only the blobs for the file in question \n(had to find several such blobs, per commit series)\n\n"},{"id":"431188","messageId":"5ca837f6-44dd-2b43-32dc-e1e134d18d61@mfriebe.de","threadId":"56159","inReplyTo":"f454bf5b-c5ff-4140-02a8-b02dcd35c6b9@mfriebe.de","subject":"Re: Files modified, even after: git reset --hard","fromName":"Martin","fromEmail":"git@mfriebe.de","sentAt":"2021-07-26T13:57:29Z","receivedAt":"2021-07-26T13:57:34Z","isPatch":false,"sender":{"key":"git@mfriebe.de","avatar":null},"body":"On 26/07/2021 12:31, Martin wrote:\n> On 26/07/2021 04:59, Chris Torek wrote:\n>> On Sun, Jul 25, 2021 at 6:34 PM Martin <git@mfriebe.de> wrote:\n>>> Actually there is something else.\n>>>\n>>> If a file has line-endings that will change, then\n>>>      git add --renormalize .\n>>>      git commit -m foo\n>>> will commit those files.\n>>>\n>>> But I am now also getting files, that show modified, but that can \n>>> not be\n>>> committed renormalized (0 lines changed).\n>>\n>> I believe (but can't demonstrate) that this is a temporary condition.\n>\n> I now found one, that does not seem temporary at all...\n>\n> git remote -v\n> origin  git://sourceware.org/git/binutils-gdb.git (fetch)\n> origin  git://sourceware.org/git/binutils-gdb.git (push)\n>\n> Switching to a far away commit\n> git switch -f --detach master\n> git rev-list master | wc -l\n> 93917\n>\n> git status shows no modified files\n>\n>\n> Then switching to  (this is on master branch)\n> git switch -f --detach a362ee23634\n> git rev-list a362ee23634 | wc -l\n> 4164\n>\n> git status --porcelain=v2\n> 1 .M N... 100644 100644 100644 \n> 9e677a52ae690808165993a0f3f17ac49e3969df \n> 9e677a52ae690808165993a0f3f17ac49e3969df bfd/Makefile.dos\n> 1 .M N... 100644 100644 100644 \n> ff24f19c0b6e0c7fb713c79e8f1765bc22fe7adc \n> ff24f19c0b6e0c7fb713c79e8f1765bc22fe7adc binutils/Makefile.dos\n> 1 .M N... 100644 100644 100644 \n> 1d9541c2f896842d1bafe68ccf0a51e291d66688 \n> 1d9541c2f896842d1bafe68ccf0a51e291d66688 gas/Makefile.dos\n> 1 .M N... 100644 100644 100644 \n> 57fab985680ea151379069abe414bcb590cdd743 \n> 57fab985680ea151379069abe414bcb590cdd743 ld/Makefile.dos\n\nThis seems an issue with gitforwindows.\n\nI checked on 2 linux distros (git 2.25 and 2.31.1) and both are fine.\n\nOn Windows I set\ncore.autocrlf false\ncore.fscache false\n\nMade a new clone, some (applied config as above), issue still present.\n\nTested with gitforwindow 2.32, 2.31.1 and 2.30.2\n\n"},{"id":"431210","messageId":"67a3c7d0-a199-da8d-09f5-1ff25c0d24ae@iee.email","threadId":"56159","inReplyTo":"5ca837f6-44dd-2b43-32dc-e1e134d18d61@mfriebe.de","subject":"Re: Files modified, even after: git reset --hard","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2021-07-26T18:21:51Z","receivedAt":"2021-07-26T18:21:55Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"On 26/07/2021 14:57, Martin wrote:\n> On 26/07/2021 12:31, Martin wrote:\n>> On 26/07/2021 04:59, Chris Torek wrote:\n>>> On Sun, Jul 25, 2021 at 6:34 PM Martin <git@mfriebe.de> wrote:\n>>>> Actually there is something else.\n>>>>\n>>>> If a file has line-endings that will change, then\n>>>>      git add --renormalize .\n>>>>      git commit -m foo\n>>>> will commit those files.\n>>>>\n>>>> But I am now also getting files, that show modified, but that can\n>>>> not be\n>>>> committed renormalized (0 lines changed).\n>>>\n>>> I believe (but can't demonstrate) that this is a temporary condition.\n>>\n>> I now found one, that does not seem temporary at all...\n>>\n>> git remote -v\n>> origin  git://sourceware.org/git/binutils-gdb.git (fetch)\n>> origin  git://sourceware.org/git/binutils-gdb.git (push)\n>>\n>> Switching to a far away commit\n>> git switch -f --detach master\n>> git rev-list master | wc -l\n>> 93917\n>>\n>> git status shows no modified files\n>>\n>>\n>> Then switching to  (this is on master branch)\n>> git switch -f --detach a362ee23634\n>> git rev-list a362ee23634 | wc -l\n>> 4164\n>>\n>> git status --porcelain=v2\n>> 1 .M N... 100644 100644 100644\n>> 9e677a52ae690808165993a0f3f17ac49e3969df\n>> 9e677a52ae690808165993a0f3f17ac49e3969df bfd/Makefile.dos\n>> 1 .M N... 100644 100644 100644\n>> ff24f19c0b6e0c7fb713c79e8f1765bc22fe7adc\n>> ff24f19c0b6e0c7fb713c79e8f1765bc22fe7adc binutils/Makefile.dos\n>> 1 .M N... 100644 100644 100644\n>> 1d9541c2f896842d1bafe68ccf0a51e291d66688\n>> 1d9541c2f896842d1bafe68ccf0a51e291d66688 gas/Makefile.dos\n>> 1 .M N... 100644 100644 100644\n>> 57fab985680ea151379069abe414bcb590cdd743\n>> 57fab985680ea151379069abe414bcb590cdd743 ld/Makefile.dos\n>\n> This seems an issue with gitforwindows.\n\nI doubt that it is Git-for-Windows (the .exe), but could easily believe\nthat the whole scenario is predicated on the Windows line endings and\npersonal .gitconfigs (some of which may be GfW norms) and the\nrepository's upstream norms (such as what is committed in the repo).\n\nGiven the repos/remotes listed your email in\nhttps://public-inbox.org/git/f454bf5b-c5ff-4140-02a8-b02dcd35c6b9@mfriebe.de/,\nWhat are all your differing configs between Windows & Linux testing and\ntheir location (i.e. use ` git config --list --show-origin --show-scope`).\n\nThe comparison you suggest, I believe, is between\nmaster:1b348b6b67dcdc120f5ac9dc409142b8ec2b4a09 (17 Apr 2021) and the\ndetached https://github.com/User4martin/binutils-gdb/commit/a362ee23634\n\"recording file death\" (8 Dec 1992) (is that the commit you really\nwanted to detach and compare with?)\n\nThe Github compare wasn't helpful as there are too many changes\n[https://github.com/red-prig/binutils-gdb/compare/1b348b6..a362ee23634]\n\nJust thinking about an MVCE.\n>\n> I checked on 2 linux distros (git 2.25 and 2.31.1) and both are fine.\n>\n> On Windows I set\n> core.autocrlf false\n> core.fscache false\n>\n> Made a new clone, some (applied config as above), issue still present.\n>\n> Tested with gitforwindow 2.32, 2.31.1 and 2.30.2\n>\nPhilip\n"},{"id":"431227","messageId":"67f35be7-3317-6486-cdb6-62eb691eaf10@mfriebe.de","threadId":"56159","inReplyTo":"CAPx1GvdM7CzsbT1SWW9+OPcG9FL7WXQ7YD6aM7P0krujp_OrkQ@mail.gmail.com","subject":"Re: Files modified, even after: git reset --hard","fromName":"Martin","fromEmail":"git@mfriebe.de","sentAt":"2021-07-26T19:57:34Z","receivedAt":"2021-07-26T19:57:38Z","isPatch":false,"sender":{"key":"git@mfriebe.de","avatar":null},"body":"On 26/07/2021 04:59, Chris Torek wrote:\n> On Sun, Jul 25, 2021 at 6:34 PM Martin <git@mfriebe.de> wrote:\n>> Actually there is something else.\n>>\n>> If a file has line-endings that will change, then\n>>      git add --renormalize .\n>>      git commit -m foo\n>> will commit those files.\n>>\n>> But I am now also getting files, that show modified, but that can not be\n>> committed renormalized (0 lines changed).\n> \n> I believe (but can't demonstrate) that this is a temporary condition.\n> \n> Git has a number of cheats to make `git status` and other ops fast.\n> These cheats *assume* that the committed files, the index copies\n> of files, and the working tree copies of files all agree in terms of\n> line endings as coordinated through `core.autocrlf` and `.gitattributes`\n> settings.\n> \n> When they *don't* agree, you get phantom differences.  Running\n> commands like `git diff` show no differences because of these\n> phantom states.  Eventually this clears up on its own when the\n> cheats really *do* agree with the settings.  Changing the settings\n> is what disturbs the cheats.\n> \n\nIs it possible that those cheats also look at the \"replaced\" (rather \nthan the \"replacement\") commit(s)?\n\nI am pretty sure I have \"git replace\"d all blobs for some of the files, \nand yet they do get phantoms.\n\n\n"},{"id":"431238","messageId":"CAPx1Gvey1uSr58Uf7VpC0c6J+R0tUP=VUP_dGmv_yVO-CwmvXg@mail.gmail.com","threadId":"56159","inReplyTo":"67f35be7-3317-6486-cdb6-62eb691eaf10@mfriebe.de","subject":"Re: Files modified, even after: git reset --hard","fromName":"Chris Torek","fromEmail":"chris.torek@gmail.com","sentAt":"2021-07-26T22:03:24Z","receivedAt":"2021-07-26T22:03:42Z","isPatch":false,"sender":{"key":"chris.torek@gmail.com","avatar":"https://avatars.githubusercontent.com/u/16826774?v=4"},"body":"On Mon, Jul 26, 2021 at 12:57 PM Martin <git@mfriebe.de> wrote:\n> Is it possible that those cheats also look at the \"replaced\" (rather\n> than the \"replacement\") commit(s)?\n\nThey look at `stat` system call (well, `lstat` *call*) data, not the\nactual files.  This works better on Unix systems, where `lstat` is\na real system call, than it does on Windows, where it's faked up\nfrom whatever Windows really stores as information about files.\n\n> I am pretty sure I have \"git replace\"d all blobs for some of the files,\n> and yet they do get phantoms.\n\nThe stat data are stored in Git's index.  It's the rebuilding of various\nindex entries that updates what Git uses to do a fast check of file\nstatus.  (Note that the stat data on the index itself count as part of\nthe cheating; this gets tricky.  See [1].)\n\nThere is an article at [2] on how Windows implements `stat`. It's\nprobably out of date (says \"VS2005\").  It's interesting to me what's\nwrong here: `st_dev` is made to mirror `st_rdev` but `st_dev` should\nprobably always just be zero, and they chose to attempt to store a\nfile *creation* time in `st_ctime`, when that is in `st_birthtime` in a\nmodern Unix-like system: the `ctime` field is the *inode change time*,\nnot a file creation time.  (Fortunately Git uses neither field.)\n\nChris\n\n[1]: https://git-scm.com/docs/racy-git/en\n[2]: https://web.archive.org/web/20101214222704/http://msdn.microsoft.com/en-us/library/14h5k7ff(v=vs.80).aspx\n"},{"id":"431275","messageId":"3cfcbbd4-0b26-dedf-f5f2-85caebcb75da@mfriebe.de","threadId":"56159","inReplyTo":"CAPx1Gvey1uSr58Uf7VpC0c6J+R0tUP=VUP_dGmv_yVO-CwmvXg@mail.gmail.com","subject":"Re: Files modified, even after: git reset --hard","fromName":"Martin","fromEmail":"git@mfriebe.de","sentAt":"2021-07-27T00:55:13Z","receivedAt":"2021-07-27T00:55:18Z","isPatch":false,"sender":{"key":"git@mfriebe.de","avatar":null},"body":"On 27/07/2021 00:03, Chris Torek wrote:\n> On Mon, Jul 26, 2021 at 12:57 PM Martin <git@mfriebe.de> wrote:\n>> Is it possible that those cheats also look at the \"replaced\" (rather\n>> than the \"replacement\") commit(s)?\n> \n> [1]: https://git-scm.com/docs/racy-git/en\n> [2]: https://web.archive.org/web/20101214222704/http://msdn.microsoft.com/en-us/library/14h5k7ff(v=vs.80).aspx\n> \n\nThanks for the info. I still think there is something wrong.\nSo I made a simple testcase\n\nhttps://github.com/User4martin/testrep.git\ngit fetch origin 'refs/replace/*:refs/replace/*'\n\n\nAll tested on linux. (Though git version 2.25.1)\n\ngit init test\n\nFILE .gitattributes\nfoo -text\n\nFILE foo\nAAAAA\n\ngit add .gitattributes foo\ngit commit -m A\n\nmodify foo (real content modify)\nadd / commit -m B\nmodify foo (real content modify)\nadd / commit -m C\n\nswitch -d <A>\nmodify foo to be the same as commited to \"C\"\nadd / commit -m A2\n\ngit show\n- get the old and new hash for the blob containing foo\ngit replace old new\n\nNow commit A has the same content in foo as commit C\n(we are currently at the detached commit A2)\n\ngit switch <C>\ngit status  // all fine\n\ngit switch <A>\ngit status\n=> foo is modified\ngit diff\n=> no diff\n\n\nSo I would say, it is not autorclf, or line endings in this case.\n\nI do not know, if the above can be caused by \"raciness\".\nBut it appears to only(?) happens if a replace is in place.\n\nSo at this point my guess would be, that when the switch is done, at \nsome point something looks at the original blob, even though it is meant \nto look at the replacement.\n\n\n\n\n\n\n\n\n\n\n\n"}]}