{"thread":{"id":"44606","subject":"git reset --hard should not irretrievably destroy new files","startedAt":"2016-12-03T05:04:46Z","lastAt":"2016-12-04T19:09:03Z","messageCount":7,"participants":["Julian de Bhal","Johannes Sixt","Christian Couder","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"306856","messageId":"CAJZCeG1Eu+5DfaxavX_WGUCa+SY+yepDWZhPXxiFcV__h0xjrw@mail.gmail.com","threadId":"44606","inReplyTo":null,"subject":"git reset --hard should not irretrievably destroy new files","fromName":"Julian de Bhal","fromEmail":"julian.debhal@gmail.com","sentAt":"2016-12-03T05:04:40Z","receivedAt":"2016-12-03T05:04:46Z","isPatch":false,"sender":{"key":"julian.debhal@gmail.com","avatar":"https://gravatar.com/avatar/8ec2403f01e8be647384b537141e6eed78e2249e96c652664ee428ce7bd0f1d8?d=mp&s=160"},"body":"If you `git add new_file; git reset --hard`, new_file is gone forever.\n\nThis is totally what git says it will do on the box, but it caught me out.\n\nIt might seem a little less stupid if I explain what I was doing: I was\nbreaking apart a chunk of work into smaller changes:\n\ngit commit -a -m 'tmp' # You feel pretty safe now, right?\ngit checkout -b backup/my-stuff # Not necessary, just a convenience\ngit checkout -\ngit reset HEAD^ # mixed\ngit add new_file\ngit add -p # also not necessary, but distracting\ngit reset --hard # decided copy from backed up diff\n# boom. new_file is gone forever\n\n\nNow, again, this is totally what git says it's going to do, and that was\npretty stupid, but that file is gone for good, and it feels bad.\n\nEverything that was committed is safe, and the other untracked files in\nmy local directory are also fine, but that particular file is\npermanently destroyed. This is the first time I've lost something since I\ndiscovered the reflog a year or two ago.\n\nThe behaviour that would make the most sense to me (personally) would be\nfor a hard reset to unstage new files, but I'd be nearly as happy if a\ncommit was added to the reflog when the reset happens (I can probably make\nthat happen with some configuration now that I've been bitten).\n\nIf there's support for this idea but no-one is keen to write the code, let\nme know and I could have a crack at it.\n\nCheers,\n\nJulian de Bhál\n"},{"id":"306864","messageId":"f2f8c7c9-cbe0-870d-3c13-fc928dd91dd1@kdbg.org","threadId":"44606","inReplyTo":"CAJZCeG1Eu+5DfaxavX_WGUCa+SY+yepDWZhPXxiFcV__h0xjrw@mail.gmail.com","subject":"Re: git reset --hard should not irretrievably destroy new files","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2016-12-03T07:49:35Z","receivedAt":"2016-12-03T07:49:44Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 03.12.2016 um 06:04 schrieb Julian de Bhal:\n> If you `git add new_file; git reset --hard`, new_file is gone forever.\n\nAFAIC, this is a feature ;-) I occasionally use it to remove a file when \nI already have git-gui in front of me. Then it's often less convenient \nto type the path in a shell, or to pointy-click around in a file browser.\n\n> git add new_file\n\nBecause of this ...\n\n> git add -p # also not necessary, but distracting\n> git reset --hard # decided copy from backed up diff\n> # boom. new_file is gone forever\n\n... it is not. The file is still among the dangling blobs in the \nrepository until you clean it up with 'git gc'. Use 'git fsck --lost-found':\n\n--lost-found\n\n Write dangling objects into .git/lost-found/commit/ or \n.git/lost-found/other/, depending on type. If the object is a blob, the \ncontents are written into the file, rather than its object name.\n\n-- Hannes\n\n"},{"id":"306865","messageId":"CAP8UFD0ipS_4p+njfbbDGpYSDJhp43e9XDP69MOruZz9c136ew@mail.gmail.com","threadId":"44606","inReplyTo":"CAJZCeG1Eu+5DfaxavX_WGUCa+SY+yepDWZhPXxiFcV__h0xjrw@mail.gmail.com","subject":"Re: git reset --hard should not irretrievably destroy new files","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2016-12-03T08:11:24Z","receivedAt":"2016-12-03T08:11:33Z","isPatch":false,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"On Sat, Dec 3, 2016 at 6:04 AM, Julian de Bhal wrote:\n> If you `git add new_file; git reset --hard`, new_file is gone forever.\n>\n> This is totally what git says it will do on the box, but it caught me out.\n\nYeah, you are not the first one, and probably not the last\nunfortunately, to be caught by it, see for example the last discussion\nabout it:\n\nhttps://public-inbox.org/git/loom.20160523T023140-975@post.gmane.org/\n\nwhich itself refers to this previous discussion:\n\nhttps://public-inbox.org/git/CANWD=rX-MEiS4cNzDWr2wwkshz2zu8-L31UrKwbZrJSBcJX-nQ@mail.gmail.com/\n\n> It might seem a little less stupid if I explain what I was doing: I was\n> breaking apart a chunk of work into smaller changes:\n>\n> git commit -a -m 'tmp' # You feel pretty safe now, right?\n> git checkout -b backup/my-stuff # Not necessary, just a convenience\n> git checkout -\n> git reset HEAD^ # mixed\n> git add new_file\n> git add -p # also not necessary, but distracting\n> git reset --hard # decided copy from backed up diff\n> # boom. new_file is gone forever\n>\n>\n> Now, again, this is totally what git says it's going to do, and that was\n> pretty stupid, but that file is gone for good, and it feels bad.\n\nYeah, I agree that it feels bad even if there are often ways to get\nback your data as you can see from the links in Yotam's email above.\n\n> Everything that was committed is safe, and the other untracked files in\n> my local directory are also fine, but that particular file is\n> permanently destroyed. This is the first time I've lost something since I\n> discovered the reflog a year or two ago.\n>\n> The behaviour that would make the most sense to me (personally) would be\n> for a hard reset to unstage new files,\n\nThis has already been proposed last time...\n\n> but I'd be nearly as happy if a\n> commit was added to the reflog when the reset happens (I can probably make\n> that happen with some configuration now that I've been bitten).\n\nNot sure if this has been proposed. Perhaps it would be simpler to\njust output the sha1, and maybe the filenames too, of the blobs, that\nare no more referenced from the trees, somewhere (in a bloblog?).\n\n> If there's support for this idea but no-one is keen to write the code, let\n> me know and I could have a crack at it.\n\nNot sure if your report and your offer will make us more likely to\nagree to do something, but thanks for trying!\n"},{"id":"306881","messageId":"CAJZCeG02xJ7XWVU8s_GwTctZ8fwJ1_tCMrvDoCRnTXUjBrjr1Q@mail.gmail.com","threadId":"44606","inReplyTo":"f2f8c7c9-cbe0-870d-3c13-fc928dd91dd1@kdbg.org","subject":"Re: git reset --hard should not irretrievably destroy new files","fromName":"Julian de Bhal","fromEmail":"julian.debhal@gmail.com","sentAt":"2016-12-04T00:14:40Z","receivedAt":"2016-12-04T00:15:58Z","isPatch":false,"sender":{"key":"julian.debhal@gmail.com","avatar":"https://gravatar.com/avatar/8ec2403f01e8be647384b537141e6eed78e2249e96c652664ee428ce7bd0f1d8?d=mp&s=160"},"body":"On Sat, Dec 3, 2016 at 5:49 PM, Johannes Sixt wrote:\n> Am 03.12.2016 um 06:04 schrieb Julian de Bhal:\n>>\n>> If you `git add new_file; git reset --hard`, new_file is gone forever.\n>\n> AFAIC, this is a feature ;-) I occasionally use it to remove a file when I\n> already have git-gui in front of me. Then it's often less convenient to type\n> the path in a shell, or to pointy-click around in a file browser.\n\nYeah, I'm conscious that it would be a change in behaviour and would\nalmost certainly break things in the wild.\n\nOn the other hand, `rm` deletes perfectly well, but there's no good\nway to recover the lost files after the fact. You can take some\nprecautions after you've been bitten, but git usually means never\nsaying \"you should have\".\n\n>> git add new_file\n>> [...]\n>> git reset --hard # decided copy from backed up diff\n>> # boom. new_file is gone forever\n>\n> ... it is not. The file is still among the dangling blobs in the repository\n> until you clean it up with 'git gc'. Use 'git fsck --lost-found':\n\nThank you so much! Super glad to be wrong here.\n\nCheers,\n\nJules\n\nOn Sat, Dec 3, 2016 at 5:49 PM, Johannes Sixt wrote:\n> Am 03.12.2016 um 06:04 schrieb Julian de Bhal:\n>>\n>> If you `git add new_file; git reset --hard`, new_file is gone forever.\n>\n>\n> AFAIC, this is a feature ;-) I occasionally use it to remove a file when I\n> already have git-gui in front of me. Then it's often less convenient to type\n> the path in a shell, or to pointy-click around in a file browser.\n>\n>> git add new_file\n>\n>\n> Because of this ...\n>\n>> git add -p # also not necessary, but distracting\n>> git reset --hard # decided copy from backed up diff\n>> # boom. new_file is gone forever\n>\n>\n> ... it is not. The file is still among the dangling blobs in the repository\n> until you clean it up with 'git gc'. Use 'git fsck --lost-found':\n>\n> --lost-found\n>\n> Write dangling objects into .git/lost-found/commit/ or\n> .git/lost-found/other/, depending on type. If the object is a blob, the\n> contents are written into the file, rather than its object name.\n>\n> -- Hannes\n>\n"},{"id":"306883","messageId":"CAJZCeG0p5UrqM4oSOJ1ALKqNG8SyYh8cexKaN9R6RYYzPsMfxQ@mail.gmail.com","threadId":"44606","inReplyTo":"CAP8UFD0ipS_4p+njfbbDGpYSDJhp43e9XDP69MOruZz9c136ew@mail.gmail.com","subject":"Re: git reset --hard should not irretrievably destroy new files","fromName":"Julian de Bhal","fromEmail":"julian.debhal@gmail.com","sentAt":"2016-12-04T00:57:14Z","receivedAt":"2016-12-04T01:05:51Z","isPatch":false,"sender":{"key":"julian.debhal@gmail.com","avatar":"https://gravatar.com/avatar/8ec2403f01e8be647384b537141e6eed78e2249e96c652664ee428ce7bd0f1d8?d=mp&s=160"},"body":"On Sat, Dec 3, 2016 at 6:11 PM, Christian Couder\n wrote:\n> On Sat, Dec 3, 2016 at 6:04 AM, Julian de Bhal wrote:\n>> but I'd be nearly as happy if a\n>> commit was added to the reflog when the reset happens (I can probably make\n>> that happen with some configuration now that I've been bitten).\n>\n> Not sure if this has been proposed. Perhaps it would be simpler to\n> just output the sha1, and maybe the filenames too, of the blobs, that\n> are no more referenced from the trees, somewhere (in a bloblog?).\n\nYeah, after doing a bit more reading around the issue, this seems like\na smaller part of destroying local changes with a hard reset, and I'm\none of the lucky ones where it is recoverable.\n\nHas anyone discussed having `git reset --hard` create objects for the\ncurrent state of anything it's about to destroy, specifically so they\nend up in the --lost-found?\n\nI think this is what you're suggesting, only without checking for\nreferences, so that tree & blob objects exist that make any hard reset\nreversible.\n\nCheers\n\nJules\n\nP.s. Thank you for such a warm welcome while I blunder through\nunfamiliar protocols.\n"},{"id":"306889","messageId":"CAP8UFD2rsxjmmVnPBmDFDx_-+Xr12wqGRqOMuMfrnu_ReSCm1A@mail.gmail.com","threadId":"44606","inReplyTo":"CAJZCeG0p5UrqM4oSOJ1ALKqNG8SyYh8cexKaN9R6RYYzPsMfxQ@mail.gmail.com","subject":"Re: git reset --hard should not irretrievably destroy new files","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2016-12-04T10:47:23Z","receivedAt":"2016-12-04T10:47:31Z","isPatch":false,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"On Sun, Dec 4, 2016 at 1:57 AM, Julian de Bhal wrote:\n> On Sat, Dec 3, 2016 at 6:11 PM, Christian Couder\n> wrote:\n>> On Sat, Dec 3, 2016 at 6:04 AM, Julian de Bhal wrote:\n>>> but I'd be nearly as happy if a\n>>> commit was added to the reflog when the reset happens (I can probably make\n>>> that happen with some configuration now that I've been bitten).\n>>\n>> Not sure if this has been proposed. Perhaps it would be simpler to\n>> just output the sha1, and maybe the filenames too, of the blobs, that\n>> are no more referenced from the trees, somewhere (in a bloblog?).\n>\n> Yeah, after doing a bit more reading around the issue, this seems like\n> a smaller part of destroying local changes with a hard reset, and I'm\n> one of the lucky ones where it is recoverable.\n\nYeah, but not everyone knows it is recoverable and using fsck to\nrecover is not nice and easy for the user.\nSo having a bloblog for example in .git/logs/blobs/, like the reflogs\nwe already have, but for blobs, could help even if (first) it's just\nabout writing the filenames and sha1s related to the blobs we stop\nreferencing.\n\n> Has anyone discussed having `git reset --hard` create objects for the\n> current state of anything it's about to destroy, specifically so they\n> end up in the --lost-found?\n\nWell, when we start talking about creating new objects, then someone\nusually says that it is what \"git stash\" is about. So the discussion\nthen often turns to how can we make people more aware of \"git stash\",\nor incite them to create an alias or a shell function that does a \"git\nstash\" before \"git reset --hard ...\", or teach them to use \"git reset\n--keep ...\" when it does what they want and is safer...\n\n> I think this is what you're suggesting, only without checking for\n> references, so that tree & blob objects exist that make any hard reset\n> reversible.\n\nI suggest we start with just logging blobs that we have already\ncreated (when they have been \"git add\"ed) but that we are\ndereferencing.\nIf we can agree on that, it will already help and not be very costly\nperformance wise. After that we could then start thinking about\ncreating blobs for all the content we discard, which could be done\nonly in a beginner mode (at least at first) to make sure it has no\nperformance impact if people rely on \"git reset --hard\" being fast.\n"},{"id":"306897","messageId":"xmqq1sxn7bqx.fsf@gitster.mtv.corp.google.com","threadId":"44606","inReplyTo":"CAJZCeG1Eu+5DfaxavX_WGUCa+SY+yepDWZhPXxiFcV__h0xjrw@mail.gmail.com","subject":"Re: git reset --hard should not irretrievably destroy new files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-12-04T19:08:54Z","receivedAt":"2016-12-04T19:09:03Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Julian de Bhal writes:\n\n> The behaviour that would make the most sense to me (personally) would be\n> for a hard reset to unstage new files,...\n\nI think _sometimes_ that may be useful. I haven't thought things\nthrough yet to arrive the final decision, but one thing that must be\nkept in mind by anybody who wants to move this topic forward is that\na path that does not exist in the HEAD commit MUST be removed from\nthe index and the working tree upon \"git reset --hard\" when the path\nresulted from a mergy operation. I.e. in this sequence:\n\n $ git merge other-branch ;# or git cherry-pick one-commit\n ... try to resolve conflicts, make a mess, decide\n ... to try it again from scratch\n $ git reset --hard\n $ git merge other-branch ;# or git cherry-pick one-commit\n\nthe \"reset --hard\" step MUST remove new paths that existed only on\nthe other-branch (or in one-commit), which by definition would have\nauto-resolved cleanly, from the index and the working tree. There\nare other commands (e.g. \"git am -3\", \"git apply -3\", \"git rebase\")\nthat are \"mergy\" and their intermediate states must be handled the\nsame way.\n\nIf a very simple to explain and understand rule can be used to tell\nif a new path (i.e. a path that exists in the index and in the\nworking tree, but is not in the HEAD commit) is what was created\nmanually by the end user without any other copy (i.e. \"create\nnewfile && edit newfile && git add newfile\") and is not a result of\na mergy operation being abandoned, then I think it is OK to allow\n\"reset --hard\" to leave the working tree files untracked, but if the\nrule becomes anything complex to understand for the users, I think\nit would make the behaviour of \"reset --hard\" hard to explain,\nunderstand AND anticipate---and at that point, we would be better\noff keeping the \"You said 'hard', we clean 'hard' to match HEAD\"\nbehaviour of \"reset --hard\" and EDUCATE users not to say 'hard' too\ncasually. There may be a room for new option that unconditionally\nleave the new working tree files untracked so that users can choose\nbetween the two, if we end up going that route.\n\n\n\n"}]}