{"thread":{"id":"34005","subject":"can we prevent reflog deletion when branch is deleted?","startedAt":"2013-06-01T01:31:24Z","lastAt":"2013-11-14T16:20:34Z","messageCount":21,"participants":["Sitaram Chamarty","Michael Haggerty","Jeff King","Ramkumar Ramachandra","Thomas Rast","Luca Milanesio","Stephen Bash"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"219109","messageId":"CAMK1S_jY1tDCkyOamX8XNW9g8Dzf6yN9znwN6he-EVcOkBM1fQ@mail.gmail.com","threadId":"34005","inReplyTo":null,"subject":"can we prevent reflog deletion when branch is deleted?","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2013-06-01T01:31:24Z","receivedAt":"2013-06-01T01:31:24Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"Hi,\n\nIs there a way to prevent reflog deletion when the branch is deleted?\nThe last entry could simply be a line where the second SHA is all 0's.\n\n--\nSitaram\n"},{"id":"219110","messageId":"51A963B7.6060002@alum.mit.edu","threadId":"34005","inReplyTo":"CAMK1S_jY1tDCkyOamX8XNW9g8Dzf6yN9znwN6he-EVcOkBM1fQ@mail.gmail.com","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2013-06-01T03:00:07Z","receivedAt":"2013-06-01T03:00:07Z","isPatch":false,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 06/01/2013 03:31 AM, Sitaram Chamarty wrote:\n> Is there a way to prevent reflog deletion when the branch is deleted?\n> The last entry could simply be a line where the second SHA is all 0's.\n\nThis is a known problem. The technical reason that this is not trivial\nto solve is the possibility of a directory/file conflict between old\nreflog files and references that might be created subsequently (which in\nturn is a limitation of how loose references and reflogs are mapped to\nfilenames):\n\n git branch foo\n git branch -d foo\n git branch foo/bar\n\nUnder your proposal, the second line would retain the reflog file for\nfoo, which is named \".git/logs/refs/heads/foo\". But the third line\nwants to create a file \".git/logs/refs/heads/foo/bar\". The existence of\nthe \"foo\" file prevents the creation of a \"foo\" directory.\n\nA similar problem exists if \"foo\" and \"foo/bar\" are exchanged in the\nabove example.\n\nPeff proposed a solution to this problem [1], but AFAIK it is not making\nprogress.\n\nMichael\n\n[1]\nhttp://thread.gmane.org/gmane.comp.version-control.git/201715/focus=201752\n\n-- \nMichael Haggerty\nmhagger@alum.mit.edu\nhttp://softwareswirl.blogspot.com/\n"},{"id":"219111","messageId":"20130601050355.GA23408@sigill.intra.peff.net","threadId":"34005","inReplyTo":"51A963B7.6060002@alum.mit.edu","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-06-01T05:03:55Z","receivedAt":"2013-06-01T05:03:55Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Jun 01, 2013 at 05:00:07AM +0200, Michael Haggerty wrote:\n\n> This is a known problem. The technical reason that this is not trivial\n> to solve is the possibility of a directory/file conflict between old\n> reflog files and references that might be created subsequently (which in\n> turn is a limitation of how loose references and reflogs are mapped to\n> filenames):\n> [...]\n> Peff proposed a solution to this problem [1], but AFAIK it is not making\n> progress.\n\nI was running with the patch series you mentioned for a while, but there\nare some weird bugs with it that need to be tracked down. I don't\nrecall the details, but I would occasionally get error messages that\nshowed that some parts of the code were surprised that the reflog\nexisted without the ref existing.\n\nWhile I think solving the D/F conflict in the ref namespaces overall\nwould be a nice thing to have, doing it with compatibility with the\ncurrent system is complex and error-prone. I wonder if simply sticking\nthe reflog entries into a big GRAVEYARD reflog wouldn't be a great deal\nsimpler and accomplish the \"keep deleted reflogs\" goal, which is what\npeople actually want.\n\n-Peff\n"},{"id":"219112","messageId":"CALkWK0kcJH0t4i0BAPmMkNWwNzeJNdmg_wbt3ao-=R31kJ5noA@mail.gmail.com","threadId":"34005","inReplyTo":"20130601050355.GA23408@sigill.intra.peff.net","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-06-01T07:59:07Z","receivedAt":"2013-06-01T07:59:07Z","isPatch":false,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Jeff King wrote:\n> I wonder if simply sticking\n> the reflog entries into a big GRAVEYARD reflog wouldn't be a great deal\n> simpler and accomplish the \"keep deleted reflogs\" goal, which is what\n> people actually want.\n\nExactly what I was thinking when I read your proposal. What is the\npoint of having individual graveyards for deleted branches? The\nbranch names no longer have any significance, and separating the\nreflogs using branch names nobody remembers is only making\ndiscoverability harder.\n\nWhat is the problem we are trying to solve? Someone deletes a branch\nby mistake, and wants to get it back? There's the HEAD reflog for\nthat. So, I think the problem is that the person did a flurry of\ncreation/ deletion/ rebases, and wants to reach one particular commit\nshe remembers seeing sometime in the past (possibly in refs/remotes/*,\nin which case HEAD reflog wouldn't have logged it). More than adding\na graveyard to provide hard-to-dissect information, I'm interested in\ntooling support for the information we already have.\n\nSo, I want to search all reflogs for this particular commit I've seen:\n\n git log -1 --relative-date -g :/quuxery\n\nDoesn't work. I can search only search one reflog at a time:\n\n git log -1 --relative-date @@{0}^{/quuxery}\n\nIsn't this much too painful?\n\nOur \"default\" reflog command displays useless information: why should\nI see HEAD@{1} followed by HEAD@{2} and other numbers in ascending\norder? What is the point of that when the abbreviated sha1 is already\nshown in the first field? I use the following alias for reflog:\n\n rfl = log --oneline --relative-date -g\n\nbut it could easily be better.\n\nThere are tons of other issues: for instance, after an\ninteractive-rebase, 'git checkout -' doesn't take me back to the\nprevious branch (because the parser for @{} is broken). There are\nway too many SHA-1s polluting the description, which can easily be\nreplaced by a git-describe output. When I git checkout @~1, my prompt\ndoesn't scream the sha1; it shows me upstream-error~1, which makes a\nlot more sense.\n\nI was under the impression that heavy reflog users would be more\ninterested in fixing these issues before dumping even more data onto\nthe user.\n"},{"id":"219114","messageId":"20130601090934.GA13904@sigill.intra.peff.net","threadId":"34005","inReplyTo":"CALkWK0kcJH0t4i0BAPmMkNWwNzeJNdmg_wbt3ao-=R31kJ5noA@mail.gmail.com","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-06-01T09:09:35Z","receivedAt":"2013-06-01T09:09:35Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Jun 01, 2013 at 01:29:07PM +0530, Ramkumar Ramachandra wrote:\n\n> Jeff King wrote:\n> > I wonder if simply sticking\n> > the reflog entries into a big GRAVEYARD reflog wouldn't be a great deal\n> > simpler and accomplish the \"keep deleted reflogs\" goal, which is what\n> > people actually want.\n> \n> Exactly what I was thinking when I read your proposal. What is the\n> point of having individual graveyards for deleted branches? The\n> branch names no longer have any significance, and separating the\n> reflogs using branch names nobody remembers is only making\n> discoverability harder.\n\nWhy don't the branch names have significance? If I deleted branch \"foo\"\nyesterday evening, wouldn't I want to be able to say \"show me foo from\n2pm yesterday\" or even \"show me all logs for foo, so that I can pick the\nuseful bit from the list\"?\n\nWhen I suggested a big graveyard reflog, I did not mean a straight\nconcatenation of the deleted reflogs; I meant one which would also\nrecord the name of the ref whose log each entry came from.\n\nIf you mean \"the branch names in the filesystem don't have\nsignificance\", I agree. Using a parallel hierarchy of reflogs was an\nimplementation choice that let us use the same reflog format. Defining\na new GRAVEYARD format would need an additional field for the ref name\nof each entry, but lets us drop the other naming complexities.\n\n> What is the problem we are trying to solve? Someone deletes a branch\n> by mistake, and wants to get it back? There's the HEAD reflog for\n> that.\n\nThe HEAD reflog is not sufficient for two reasons:\n\n 1. Not all ref updates were part of the HEAD reflog (e.g.,\n refs/remotes, tags).\n\n 2. It is not easy to see deduce which ref each entry comes from, which\n makes \"deleted_branch@{yesterday}\" difficult. You can sometimes\n deduce the branch by reading the surrounding entries (e.g., for\n \"checkout\" entries), but I do not know offhand whether it can be\n done reliably in all cases (I suspect not, given that unreachable\n reflog entries may be pruned sooner than reachable ones, leaving\n \"holes\" in the reflog's story).\n\n> More than adding a graveyard to provide hard-to-dissect information,\n> I'm interested in tooling support for the information we already have.\n\nI think that is an orthogonal concern. Already with the current reflogs,\nsuch a tool would be useful. And even without such a tool, being able to\naccess reflog entries of deleted branches is still useful. Even simple\nthings like \"git branch foo deleted@{yesterday}\" and \"git log -g\ndeleted\" would give a safety net. And those are supported by the\nexisting porcelain tooling.\n\nI do not necessarily disagree with your criticisms of the tooling around\nreflogs, but they are just not my interest right now, and I do not think\nworking on one concept needs to hold up the other.\n\n-Peff\n"},{"id":"219128","messageId":"CALkWK0mwAc0bFon7B7nw1Nbvcwdf8m2_531qtrN-r28r9F+70Q@mail.gmail.com","threadId":"34005","inReplyTo":"20130601090934.GA13904@sigill.intra.peff.net","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-06-01T09:47:58Z","receivedAt":"2013-06-01T09:47:58Z","isPatch":false,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Jeff King wrote:\n> Why don't the branch names have significance? If I deleted branch \"foo\"\n> yesterday evening, wouldn't I want to be able to say \"show me foo from\n> 2pm yesterday\" or even \"show me all logs for foo, so that I can pick the\n> useful bit from the list\"?\n\nOh, I misunderstood then. I didn't realize that your usecase was actually\n\n git log foo@{yesterday}\n\nwhere foo is a deleted branch. Just to give some perspective, so we\ndon't limit our problem space:\n\nI only ever batch-delete \"cold\" branches: if I haven't touched a\nbranch in ~2 months, I consider the work abandoned (due to disinterest\nor otherwise) and remove it. Most of my branches are short-lived, and\nI don't remember branch names, much less of the names of the cold\nbranches I deleted. My usecase for a graveyeard is \"I lost something,\nand I need to find it\": I don't want to have to remember the original\nbranch name \"foo\"; if you can tell everything I deleted yesterday, I\ncan spot foo and the commit I was looking for. The HEAD reflog is\nalmost good enough for me.\n\nTo be clear: I'm not against including branch name information; I just\ndon't want to _have_ to remember them to find what I'm looking for.\n\n> Defining\n> a new GRAVEYARD format would need an additional field for the ref name\n> of each entry, but lets us drop the other naming complexities.\n\nCertainly. Putting it in the description will only lead to more\nproblems (like bugs in the @{} parser).\n\n> The HEAD reflog is not sufficient for two reasons:\n>\n> 1. Not all ref updates were part of the HEAD reflog (e.g.,\n> refs/remotes, tags).\n\nWould be nice to solve, but it's not a big itch in my opinion.\n\n> 2. It is not easy to see deduce which ref each entry comes from, which\n> makes \"deleted_branch@{yesterday}\" difficult. You can sometimes\n> deduce the branch by reading the surrounding entries (e.g., for\n> \"checkout\" entries), but I do not know offhand whether it can be\n> done reliably in all cases (I suspect not, given that unreachable\n> reflog entries may be pruned sooner than reachable ones, leaving\n> \"holes\" in the reflog's story).\n\nYeah, this makes sense.\n\n> I do not necessarily disagree with your criticisms of the tooling around\n> reflogs, but they are just not my interest right now, and I do not think\n> working on one concept needs to hold up the other.\n\nOh, I didn't mean to hold up anything. I brought it up because I\nthought it would be of interest to heavy reflog users.\n"},{"id":"219151","messageId":"CAMK1S_hPups3SCwxhHRYWBJzpPreNVUfNdx1+_Hjy2_d0MMpaA@mail.gmail.com","threadId":"34005","inReplyTo":"CALkWK0mwAc0bFon7B7nw1Nbvcwdf8m2_531qtrN-r28r9F+70Q@mail.gmail.com","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2013-06-01T17:25:37Z","receivedAt":"2013-06-01T17:25:37Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Sat, Jun 1, 2013 at 3:17 PM, Ramkumar Ramachandra wrote:\n> Jeff King wrote:\n>> Why don't the branch names have significance? If I deleted branch \"foo\"\n>> yesterday evening, wouldn't I want to be able to say \"show me foo from\n>> 2pm yesterday\" or even \"show me all logs for foo, so that I can pick the\n>> useful bit from the list\"?\n>\n> Oh, I misunderstood then. I didn't realize that your usecase was actually\n>\n> git log foo@{yesterday}\n>\n> where foo is a deleted branch. Just to give some perspective, so we\n> don't limit our problem space:\n>\n> I only ever batch-delete \"cold\" branches: if I haven't touched a\n> branch in ~2 months, I consider the work abandoned (due to disinterest\n> or otherwise) and remove it. Most of my branches are short-lived, and\n> I don't remember branch names, much less of the names of the cold\n> branches I deleted. My usecase for a graveyeard is \"I lost something,\n> and I need to find it\": I don't want to have to remember the original\n> branch name \"foo\"; if you can tell everything I deleted yesterday, I\n> can spot foo and the commit I was looking for. The HEAD reflog is\n> almost good enough for me.\n\nI think I'd have to be playing with *several* branches simultaneously\nbefore I got to the point of forgetting the branch name!\n\nMore to the point, your use case may be relevant for a non-bare repo\nwhere \"work\" is being done, but for a bare repo on a server, I think\nthe branch name *does* have significance, because it's what people are\ncollaborating on.\n\n(Imagine someone accidentally nukes a branch, and then someone else\ntries to \"git pull\" and finds it gone. Any recovery at that point\nmust necessarily use the branch name).\n\nPS: I am assuming core.logAllRefUpdates is on\n"},{"id":"219153","messageId":"CALkWK0=SqCh-82F4ud+AxuzzEezyMWqMvc6HAPoxOk32vUND7A@mail.gmail.com","threadId":"34005","inReplyTo":"CAMK1S_hPups3SCwxhHRYWBJzpPreNVUfNdx1+_Hjy2_d0MMpaA@mail.gmail.com","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-06-01T17:56:11Z","receivedAt":"2013-06-01T17:56:11Z","isPatch":false,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Sitaram Chamarty wrote:\n> I think I'd have to be playing with *several* branches simultaneously\n> before I got to the point of forgetting the branch name!\n\nYeah, I work on lots of small unrelated things: the patch-series I\nsend in are usually the result of few hours of work (upto a few days).\n I keep the branch around until I've rewritten it for enough re-rolls\nand am sufficiently sure that it'll hit master.\n\n> More to the point, your use case may be relevant for a non-bare repo\n> where \"work\" is being done, but for a bare repo on a server, I think\n> the branch name *does* have significance, because it's what people are\n> collaborating on.\n>\n> (Imagine someone accidentally nukes a branch, and then someone else\n> tries to \"git pull\" and finds it gone. Any recovery at that point\n> must necessarily use the branch name).\n\nAh, you're mostly talking about central workflows. I'm on the other\nend of the spectrum: I want triangular workflows (and git.git is\nslowly getting there). However, I might have a (vague) thought on\nserver-side safety in general: I think the harsh dichotomy in ff-only\nversus non-ff branches is very inelegant. Imposing ff-only feels like\na hammer solution, because what happens in practice is different: the\n`master` does not need to be rewritten most of the time, but I think\nit's useful to allow some \"safe\" rewrites to undo the mistake of\nchecking in an private key or something [*1*]. By safety, I mean that\ngit should give the user easy access to recent dangling objects by\nannotating it with enough information: sort of like a general-purpose\n\"pretty\" reflog that is gc-safe (configurable trunc_length?). It's a\nserves more usecases than just the branch-removal problem.\n\nOfcourse, the standard disclaimer applies: there's a high likelihood\nthat I'm saying nonsense, because I've never worked in a central\nenvironment.\n\n[Footnotes]\n\n*1* It turns out that this is not uncommon:\nhttps://github.com/search?q=path%3A.ssh%2Fid_rsa&type=Code&ref=searchresults\n"},{"id":"219159","messageId":"CAMK1S_hfnVLvJx=sOFr4nO09=NaD_jAbdZ4S2kQAAnn6i0wX4g@mail.gmail.com","threadId":"34005","inReplyTo":"CALkWK0=SqCh-82F4ud+AxuzzEezyMWqMvc6HAPoxOk32vUND7A@mail.gmail.com","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2013-06-02T10:20:40Z","receivedAt":"2013-06-02T10:20:40Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On Sat, Jun 1, 2013 at 11:26 PM, Ramkumar Ramachandra\n wrote:\n> Sitaram Chamarty wrote:\n>> I think I'd have to be playing with *several* branches simultaneously\n>> before I got to the point of forgetting the branch name!\n>\n> Yeah, I work on lots of small unrelated things: the patch-series I\n> send in are usually the result of few hours of work (upto a few days).\n> I keep the branch around until I've rewritten it for enough re-rolls\n> and am sufficiently sure that it'll hit master.\n>\n>> More to the point, your use case may be relevant for a non-bare repo\n>> where \"work\" is being done, but for a bare repo on a server, I think\n>> the branch name *does* have significance, because it's what people are\n>> collaborating on.\n>>\n>> (Imagine someone accidentally nukes a branch, and then someone else\n>> tries to \"git pull\" and finds it gone. Any recovery at that point\n>> must necessarily use the branch name).\n>\n> Ah, you're mostly talking about central workflows. I'm on the other\n\nYes. Not just because that's what \"$dayjob\" does, but also because\nthat's what gitolite does.\n\n> end of the spectrum: I want triangular workflows (and git.git is\n> slowly getting there). However, I might have a (vague) thought on\n> server-side safety in general: I think the harsh dichotomy in ff-only\n> versus non-ff branches is very inelegant. Imposing ff-only feels like\n> a hammer solution, because what happens in practice is different: the\n> `master` does not need to be rewritten most of the time, but I think\n> it's useful to allow some \"safe\" rewrites to undo the mistake of\n> checking in an private key or something [*1*]. By safety, I mean that\n\nI suspect that's a big reason for why gitolite is so popular, at least\nwith central workflows. It's trivial to set it up so master is\nff-only and any other branch is rewindable etc.\n\n> git should give the user easy access to recent dangling objects by\n> annotating it with enough information: sort of like a general-purpose\n> \"pretty\" reflog that is gc-safe (configurable trunc_length?). It's a\n> serves more usecases than just the branch-removal problem.\n\nAgain, for \"central workflow\" folks, gitolite's log files actually\nhave enough info for all this and more. Coupled with\n\"core.logAllRefUpdates\", it's possible to recover anything that has\nnot been gc-ed, even deleted branches and tags.\n\nBut it would be nicer if git's own reflog is able to do that. Hence\nmy original thought about preserving reflogs for deleted refs (even if\nit is in a \"graveyard\" log to resolve the D/F conflict that Michael\nand Peff were discussing up at the top of the thread).\n\n> Ofcourse, the standard disclaimer applies: there's a high likelihood\n> that I'm saying nonsense, because I've never worked in a central\n> environment.\n>\n> [Footnotes]\n>\n> *1* It turns out that this is not uncommon:\n> https://github.com/search?q=path%3A.ssh%2Fid_rsa&type=Code&ref=searchresults\n\nHah! Lovely...\n\n--\nSitaram\n"},{"id":"230584","messageId":"528416EA.1070307@gmail.com","threadId":"34005","inReplyTo":"CALkWK0=SqCh-82F4ud+AxuzzEezyMWqMvc6HAPoxOk32vUND7A@mail.gmail.com","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2013-11-14T00:18:50Z","receivedAt":"2013-11-14T00:18:50Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"[top posting, and not preserving cc's because the original email thread\nbelow is just for context; I don't want to force people into a\ndiscussion that they may have considered closed :-)]\n\nIs there *any* way we can preserve a reflog for a deleted branch,\nperhaps under logs/refs/deleted//full/ref/name ?\n\nWhatever it was that happened to a hundred or more repos on the Jenkins\nproject seems to be stirring up this debate in some circles.\n\nJust some basic protection -- don't delete the reflog, and instead,\nrename it to something that preserves the name but in a different\nnamespace.\n\nsitaram\n\nOn 06/01/2013 11:26 PM, Ramkumar Ramachandra wrote:\n> Sitaram Chamarty wrote:\n>> I think I'd have to be playing with *several* branches simultaneously\n>> before I got to the point of forgetting the branch name!\n> \n> Yeah, I work on lots of small unrelated things: the patch-series I\n> send in are usually the result of few hours of work (upto a few days).\n> I keep the branch around until I've rewritten it for enough re-rolls\n> and am sufficiently sure that it'll hit master.\n> \n>> More to the point, your use case may be relevant for a non-bare repo\n>> where \"work\" is being done, but for a bare repo on a server, I think\n>> the branch name *does* have significance, because it's what people are\n>> collaborating on.\n>>\n>> (Imagine someone accidentally nukes a branch, and then someone else\n>> tries to \"git pull\" and finds it gone. Any recovery at that point\n>> must necessarily use the branch name).\n> \n> Ah, you're mostly talking about central workflows. I'm on the other\n> end of the spectrum: I want triangular workflows (and git.git is\n> slowly getting there). However, I might have a (vague) thought on\n> server-side safety in general: I think the harsh dichotomy in ff-only\n> versus non-ff branches is very inelegant. Imposing ff-only feels like\n> a hammer solution, because what happens in practice is different: the\n> `master` does not need to be rewritten most of the time, but I think\n> it's useful to allow some \"safe\" rewrites to undo the mistake of\n> checking in an private key or something [*1*]. By safety, I mean that\n> git should give the user easy access to recent dangling objects by\n> annotating it with enough information: sort of like a general-purpose\n> \"pretty\" reflog that is gc-safe (configurable trunc_length?). It's a\n> serves more usecases than just the branch-removal problem.\n> \n> Ofcourse, the standard disclaimer applies: there's a high likelihood\n> that I'm saying nonsense, because I've never worked in a central\n> environment.\n> \n> [Footnotes]\n> \n> *1* It turns out that this is not uncommon:\n> https://github.com/search?q=path%3A.ssh%2Fid_rsa&type=Code&ref=searchresults\n> \n"},{"id":"230589","messageId":"87bo1nmn6w.fsf@linux-k42r.v.cablecom.net","threadId":"34005","inReplyTo":"528416EA.1070307@gmail.com","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Thomas Rast","fromEmail":"tr@thomasrast.ch","sentAt":"2013-11-14T07:56:07Z","receivedAt":"2013-11-14T07:56:07Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Sitaram Chamarty writes:\n\n> Whatever it was that happened to a hundred or more repos on the Jenkins\n> project seems to be stirring up this debate in some circles.\n\nMaking us so curious ... and then you just leave us hanging there ;-)\n\nAny pointers to this debate?\n\n-- \nThomas Rast\ntr@thomasrast.ch\n"},{"id":"230591","messageId":"20131114080735.GB16327@sigill.intra.peff.net","threadId":"34005","inReplyTo":"87bo1nmn6w.fsf@linux-k42r.v.cablecom.net","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-11-14T08:07:35Z","receivedAt":"2013-11-14T08:07:35Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Nov 14, 2013 at 08:56:07AM +0100, Thomas Rast wrote:\n\n> > Whatever it was that happened to a hundred or more repos on the Jenkins\n> > project seems to be stirring up this debate in some circles.\n> \n> Making us so curious ... and then you just leave us hanging there ;-)\n> \n> Any pointers to this debate?\n\nI do not know about any particular debate in git circles, but I assume\nSitaram is referring to this incident:\n\n https://groups.google.com/d/msg/jenkinsci-dev/-myjRIPcVwU/t4nkXONp8qgJ\n\nin which a Jenkins dev force-pushed and rewound history on 150 different\nrepos. In this case the reflog made rollback easy, but if he had pushed\na deletion, it would be harder.\n\n-Peff\n"},{"id":"230592","messageId":"20131114081456.GC16327@sigill.intra.peff.net","threadId":"34005","inReplyTo":"528416EA.1070307@gmail.com","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-11-14T08:14:56Z","receivedAt":"2013-11-14T08:14:56Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Nov 14, 2013 at 05:48:50AM +0530, Sitaram Chamarty wrote:\n\n> Is there *any* way we can preserve a reflog for a deleted branch,\n> perhaps under logs/refs/deleted//full/ref/name ?\n\nI had patches to do something like this here:\n\n http://thread.gmane.org/gmane.comp.version-control.git/201715/focus=201752\n\nbut there were definitely some buggy corners, as much of the code\nassumed you needed to have a ref to have a reflog. I don't even run with\nit locally anymore.\n\nAt GitHub, we log each change to an \"audit log\" in addition to the\nregular reflog (we also stuff extra data from the environment into the\nreflog message). So even after a branch is deleted, its audit log\nentries remain, though you have to pull out the data by hand (git\ndoesn't know about it at all, except as an append-only sink for\nwriting). And git doesn't use the audit log for connectivity, either, so\neventually the objects could be pruned.\n\n> Just some basic protection -- don't delete the reflog, and instead,\n> rename it to something that preserves the name but in a different\n> namespace.\n\nThat part is easy. Accessing it seamlessly and handling reflog\nexpiration are a little harder. Not because they're intractable, but\njust because there are some low-level assumptions in the git code. The\npatch series I mentioned above mostly works. It probably just needs\nsomebody to go through and find the corner cases.\n\n-Peff\n"},{"id":"230595","messageId":"5284AC6E.4030208@gmail.com","threadId":"34005","inReplyTo":"20131114080735.GB16327@sigill.intra.peff.net","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2013-11-14T10:56:46Z","receivedAt":"2013-11-14T10:56:46Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On 11/14/2013 01:37 PM, Jeff King wrote:\n> On Thu, Nov 14, 2013 at 08:56:07AM +0100, Thomas Rast wrote:\n> \n>>> Whatever it was that happened to a hundred or more repos on the Jenkins\n>>> project seems to be stirring up this debate in some circles.\n>>\n>> Making us so curious ... and then you just leave us hanging there ;-)\n\nOh my apologies; I missed the URL!! (But Peff supplied it before I saw\nthis email!)\n\n>> Any pointers to this debate?\n> \n> I do not know about any particular debate in git circles, but I assume\n> Sitaram is referring to this incident:\n> \n> https://groups.google.com/d/msg/jenkinsci-dev/-myjRIPcVwU/t4nkXONp8qgJ\n> \n> in which a Jenkins dev force-pushed and rewound history on 150 different\n> repos. In this case the reflog made rollback easy, but if he had pushed\n> a deletion, it would be harder.\n\nI don't know if they had a reflog on the server side; they used\nclient-side reflogs if I understood correctly.\n\nI'm talking about server side (bare repo), assuming the site has\ncore.logAllRefUpdates set.\n\nAnd I'll explain the \"some circles\" part as \"something on LinkedIn\". To\nbe honest there's been a fair bit of FUDding by CVCS types there so I\nstopped looking at the posts, but I get the subject lines by email and I\nsaw one that said \"Git History Protection - if we needed proof...\" or\nsomething like that.\n\nI admit I didn't check to see if a debate actually followed that post\n:-)\n"},{"id":"230596","messageId":"20131114110937.GA11597@sigill.intra.peff.net","threadId":"34005","inReplyTo":"5284AC6E.4030208@gmail.com","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-11-14T11:09:38Z","receivedAt":"2013-11-14T11:09:38Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Nov 14, 2013 at 04:26:46PM +0530, Sitaram Chamarty wrote:\n\n> > I do not know about any particular debate in git circles, but I assume\n> > Sitaram is referring to this incident:\n> > \n> > https://groups.google.com/d/msg/jenkinsci-dev/-myjRIPcVwU/t4nkXONp8qgJ\n> > \n> > in which a Jenkins dev force-pushed and rewound history on 150 different\n> > repos. In this case the reflog made rollback easy, but if he had pushed\n> > a deletion, it would be harder.\n> \n> I don't know if they had a reflog on the server side; they used\n> client-side reflogs if I understood correctly.\n> \n> I'm talking about server side (bare repo), assuming the site has\n> core.logAllRefUpdates set.\n\nYes, they did have server-side reflogs (the pushes were to GitHub, and\nwe reflog everything). Client-side reflogs would not be sufficient, as\nthe client who pushed does not record the history he just rewound (he\n_might_ have it at refs/remotes/origin/master@{1}, but if somebody\npushed since his last fetch, then he doesn't).\n\nThe \"simplest\" way to recover is to just have everyone push again\n(without --force). The history will just silently fast-forward to\nwhoever has the most recent tip. The downside is that you have to wait\nfor that person to actually push. :)\n\nI think they started with that, and then eventually GitHub support got\nwind of it and pulled the last value for each repo out of the\nserver-side reflog for them.\n\n-Peff\n"},{"id":"230597","messageId":"66513F31-CF2E-4327-AEA3-20176C14EEE1@gmail.com","threadId":"34005","inReplyTo":"20131114110937.GA11597@sigill.intra.peff.net","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Luca Milanesio","fromEmail":"luca.milanesio@gmail.com","sentAt":"2013-11-14T11:17:31Z","receivedAt":"2013-11-14T11:17:31Z","isPatch":false,"sender":{"key":"luca.milanesio@gmail.com","avatar":"https://gravatar.com/avatar/64e45570bb8baaba9a566592150e6c7341a300e314501a944cb95b0bc1380f49?d=mp&s=160"},"body":"Would be really useful anyway to have the ability to create a server-side reference based on a SHA-1, using the Git protocol.\nAlternatively, just fetching a remote repo based on a SHA-1 (not referenced by any ref-spec but still existent) so that you can create a new reference locally and push.\n\nLuca.\n\nOn 14 Nov 2013, at 11:09, Jeff King wrote:\n\n> On Thu, Nov 14, 2013 at 04:26:46PM +0530, Sitaram Chamarty wrote:\n> \n>>> I do not know about any particular debate in git circles, but I assume\n>>> Sitaram is referring to this incident:\n>>> \n>>> https://groups.google.com/d/msg/jenkinsci-dev/-myjRIPcVwU/t4nkXONp8qgJ\n>>> \n>>> in which a Jenkins dev force-pushed and rewound history on 150 different\n>>> repos. In this case the reflog made rollback easy, but if he had pushed\n>>> a deletion, it would be harder.\n>> \n>> I don't know if they had a reflog on the server side; they used\n>> client-side reflogs if I understood correctly.\n>> \n>> I'm talking about server side (bare repo), assuming the site has\n>> core.logAllRefUpdates set.\n> \n> Yes, they did have server-side reflogs (the pushes were to GitHub, and\n> we reflog everything). Client-side reflogs would not be sufficient, as\n> the client who pushed does not record the history he just rewound (he\n> _might_ have it at refs/remotes/origin/master@{1}, but if somebody\n> pushed since his last fetch, then he doesn't).\n> \n> The \"simplest\" way to recover is to just have everyone push again\n> (without --force). The history will just silently fast-forward to\n> whoever has the most recent tip. The downside is that you have to wait\n> for that person to actually push. :)\n> \n> I think they started with that, and then eventually GitHub support got\n> wind of it and pulled the last value for each repo out of the\n> server-side reflog for them.\n> \n> -Peff\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n"},{"id":"230622","messageId":"5284D45B.2040904@gmail.com","threadId":"34005","inReplyTo":"20131114110937.GA11597@sigill.intra.peff.net","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2013-11-14T13:47:07Z","receivedAt":"2013-11-14T13:47:07Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On 11/14/2013 04:39 PM, Jeff King wrote:\n> On Thu, Nov 14, 2013 at 04:26:46PM +0530, Sitaram Chamarty wrote:\n> \n>>> I do not know about any particular debate in git circles, but I assume\n>>> Sitaram is referring to this incident:\n>>>\n>>> https://groups.google.com/d/msg/jenkinsci-dev/-myjRIPcVwU/t4nkXONp8qgJ\n>>>\n>>> in which a Jenkins dev force-pushed and rewound history on 150 different\n>>> repos. In this case the reflog made rollback easy, but if he had pushed\n>>> a deletion, it would be harder.\n>>\n>> I don't know if they had a reflog on the server side; they used\n>> client-side reflogs if I understood correctly.\n>>\n>> I'm talking about server side (bare repo), assuming the site has\n>> core.logAllRefUpdates set.\n> \n> Yes, they did have server-side reflogs (the pushes were to GitHub, and\n> we reflog everything). Client-side reflogs would not be sufficient, as\n> the client who pushed does not record the history he just rewound (he\n> _might_ have it at refs/remotes/origin/master@{1}, but if somebody\n> pushed since his last fetch, then he doesn't).\n> \n> The \"simplest\" way to recover is to just have everyone push again\n> (without --force). The history will just silently fast-forward to\n> whoever has the most recent tip. The downside is that you have to wait\n> for that person to actually push. :)\n> \n> I think they started with that, and then eventually GitHub support got\n> wind of it and pulled the last value for each repo out of the\n> server-side reflog for them.\n\nGreat. But what does github do if the branches were *deleted* by\nmistake (say someone does a \"git push --mirror\"; most likely in a\nscript, for added fun and laughs!)\n\nGithub may be able to help people recover from that also, but plain Git\nwon't.\n\nAnd that's what I would like to see a change in.\n\n> \n> -Peff\n> \n"},{"id":"230623","messageId":"5284D4AA.2030204@gmail.com","threadId":"34005","inReplyTo":"66513F31-CF2E-4327-AEA3-20176C14EEE1@gmail.com","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2013-11-14T13:48:26Z","receivedAt":"2013-11-14T13:48:26Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On 11/14/2013 04:47 PM, Luca Milanesio wrote:\n> Would be really useful anyway to have the ability to create a\n> server-side reference based on a SHA-1, using the Git protocol.\n> Alternatively, just fetching a remote repo based on a SHA-1 (not\n> referenced by any ref-spec but still existent) so that you can create\n> a new reference locally and push.\n\nThat's a security issue.\n\nJust to clarify, what I am asking for is the ability to recover on the\nserver, where you have access to the actual files that comprise the\nrepo.\n\nsitaram\n\n> \n> Luca.\n> \n> On 14 Nov 2013, at 11:09, Jeff King wrote:\n> \n>> On Thu, Nov 14, 2013 at 04:26:46PM +0530, Sitaram Chamarty wrote:\n>>\n>>>> I do not know about any particular debate in git circles, but I assume\n>>>> Sitaram is referring to this incident:\n>>>>\n>>>> https://groups.google.com/d/msg/jenkinsci-dev/-myjRIPcVwU/t4nkXONp8qgJ\n>>>>\n>>>> in which a Jenkins dev force-pushed and rewound history on 150 different\n>>>> repos. In this case the reflog made rollback easy, but if he had pushed\n>>>> a deletion, it would be harder.\n>>>\n>>> I don't know if they had a reflog on the server side; they used\n>>> client-side reflogs if I understood correctly.\n>>>\n>>> I'm talking about server side (bare repo), assuming the site has\n>>> core.logAllRefUpdates set.\n>>\n>> Yes, they did have server-side reflogs (the pushes were to GitHub, and\n>> we reflog everything). Client-side reflogs would not be sufficient, as\n>> the client who pushed does not record the history he just rewound (he\n>> _might_ have it at refs/remotes/origin/master@{1}, but if somebody\n>> pushed since his last fetch, then he doesn't).\n>>\n>> The \"simplest\" way to recover is to just have everyone push again\n>> (without --force). The history will just silently fast-forward to\n>> whoever has the most recent tip. The downside is that you have to wait\n>> for that person to actually push. :)\n>>\n>> I think they started with that, and then eventually GitHub support got\n>> wind of it and pulled the last value for each repo out of the\n>> server-side reflog for them.\n>>\n>> -Peff\n>> --\n>> To unsubscribe from this list: send the line \"unsubscribe git\" in\n>> the body of a message to majordomo@vger.kernel.org\n>> More majordomo info at http://vger.kernel.org/majordomo-info.html\n> \n"},{"id":"230624","messageId":"2137000803.2895009.1384440156683.JavaMail.root@genarts.com","threadId":"34005","inReplyTo":"20131114081456.GC16327@sigill.intra.peff.net","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Stephen Bash","fromEmail":"bash@genarts.com","sentAt":"2013-11-14T14:42:36Z","receivedAt":"2013-11-14T14:42:36Z","isPatch":false,"sender":{"key":"bash@genarts.com","avatar":null},"body":"----- Original Message -----\n> From: \"Jeff King\" \n> Sent: Thursday, November 14, 2013 3:14:56 AM\n> Subject: Re: can we prevent reflog deletion when branch is deleted?\n> \n> On Thu, Nov 14, 2013 at 05:48:50AM +0530, Sitaram Chamarty wrote:\n> \n> > Is there *any* way we can preserve a reflog for a deleted branch,\n> > perhaps under logs/refs/deleted//full/ref/name ?\n> \n> At GitHub, we log each change to an \"audit log\" in addition to the\n> regular reflog (we also stuff extra data from the environment into the\n> reflog message). So even after a branch is deleted, its audit log\n> entries remain, though you have to pull out the data by hand (git\n> doesn't know about it at all, except as an append-only sink for\n> writing). \n\nWe recently ran into a similar situation at my $dayjob, so I made our\nserver side update hook log all pushes (including deletes) and added the\nnew log file to logrotate(8) -- note: make sure if logrotate recreates\nthe file that it allows everyone to write to it. I'm sure it's not as\ncomprehensive as Peff's solution, but it's pretty simple for smaller\nshops that want a little more protection. Here are the relevant\nexcerpts from the script:\n\n#!/usr/bin/env python\n\nimport os, sys, pwd, stat\nfrom datetime import datetime\n\ndef log_push(too_many_changes):\n log_file = 'push-log.txt'\n try:\n f = open(log_file, 'a')\n\n try:\n # In case we just created the file, attempt to chmod it\n os.chmod(log_file, 0666)\n except OSError:\n # chmod will fail if the current user isn't the owner, but\n # if we've gotten this far we already have write permissions,\n # so just continue quietly\n pass\n\n # Linux/Mac okay, bad for Windows\n username = pwd.getpwuid(os.getuid())[0]\n f.write('%s: %s push by %s of %s from %s to %s\\n'% \\\n (datetime.now().strftime('%Y-%m-%d %H:%M:%S'),\n 'Failed' if too_many_changes else 'Successful', username,\n refname, oldsha, newsha))\n f.close()\n except IOError:\n try:\n log_stats = os.stat(log_file)\n # Figure out owner and permissions\n log_owner = pwd.getpwuid(log_stats.st_uid).pw_name\n log_perm = oct(stat.S_IMODE(log_stats.st_mode))\n print_flush('Unable to open %s for appending. Current owner ' + \\\n 'is %s and permissions are %s.'%(log_file,\n log_owner, log_perm))\n except:\n exception,desc,stack = sys.exc_info()\n print_flush('Unable to open log file. While generating error' + \\\n ' message encountered error: %s'%(desc))\n\nif len(sys.argv) != 4:\n print_flush('Usage: %s refname oldsha newsha'%sys.argv[0])\n sys.exit(1)\n\nrefname = sys.argv[1]\noldsha = sys.argv[2]\nnewsha = sys.argv[3]\n\nif newsha == '0'*40:\n # Deleted ref, nothing to do\n log_push(False)\n sys.exit(0)\n\n# ... checking for various rule/style violations ...\n\nlog_push(too_many_changes)\nif too_many_changes:\n sys.exit(1)\nelse:\n sys.exit(0)\n"},{"id":"230629","messageId":"5284F512.1090303@gmail.com","threadId":"34005","inReplyTo":"20131114081456.GC16327@sigill.intra.peff.net","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2013-11-14T16:06:42Z","receivedAt":"2013-11-14T16:06:42Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On 11/14/2013 01:44 PM, Jeff King wrote:\n> On Thu, Nov 14, 2013 at 05:48:50AM +0530, Sitaram Chamarty wrote:\n> \n>> Is there *any* way we can preserve a reflog for a deleted branch,\n>> perhaps under logs/refs/deleted//full/ref/name ?\n> \n> I had patches to do something like this here:\n> \n> http://thread.gmane.org/gmane.comp.version-control.git/201715/focus=201752\n> \n> but there were definitely some buggy corners, as much of the code\n> assumed you needed to have a ref to have a reflog. I don't even run with\n> it locally anymore.\n> \n> At GitHub, we log each change to an \"audit log\" in addition to the\n> regular reflog (we also stuff extra data from the environment into the\n> reflog message). So even after a branch is deleted, its audit log\n> entries remain, though you have to pull out the data by hand (git\n> doesn't know about it at all, except as an append-only sink for\n> writing). And git doesn't use the audit log for connectivity, either, so\n> eventually the objects could be pruned.\n> \n>> Just some basic protection -- don't delete the reflog, and instead,\n>> rename it to something that preserves the name but in a different\n>> namespace.\n> \n> That part is easy. Accessing it seamlessly and handling reflog\n> expiration are a little harder. Not because they're intractable, but\n> just because there are some low-level assumptions in the git code. The\n> patch series I mentioned above mostly works. It probably just needs\n> somebody to go through and find the corner cases.\n\nThe use cases I am talking about are those where someone deleted\nsomething and it was noticed well within Git's the earliest of Git's\nexpire timeouts.\n\nSo, no need to worry about expiry times and connecting it with object\npruning. Really, just the eqvt of a \"cp\" or \"mv\" of one file is all\nthat most people need.\n\nGitolite's log is the same. So no one who uses Gitolite needs this\nfeature. But people shouldn't have to install Gitolite or anything else\njust to get this either!\n"},{"id":"230630","messageId":"5284F852.8080609@gmail.com","threadId":"34005","inReplyTo":"2137000803.2895009.1384440156683.JavaMail.root@genarts.com","subject":"Re: can we prevent reflog deletion when branch is deleted?","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2013-11-14T16:20:34Z","receivedAt":"2013-11-14T16:20:34Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"I can't resist...\n\nOn 11/14/2013 08:12 PM, Stephen Bash wrote:\n\n[snipped some stuff from Peff]\n\n[snipped 60 lines of python]\n\nIn honor of your last name, here's what I would do if I needed to log\nref updates (and wasn't using Gitolite):\n\n#!/bin/bash\n# -- use this as a post-receive hook\n\nwhile read old new ref\ndo\n echo $(date +%F %T): Successful push by $USER of $ref from $old to $new\ndone >> push-log.txt\n"}]}