{"thread":{"id":"36969","subject":"Should branches be objects?","startedAt":"2014-06-19T21:37:36Z","lastAt":"2014-06-20T02:27:06Z","messageCount":6,"participants":["Nico Williams","Jonathan Nieder","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"244723","messageId":"CAK3OfOiB0YWf25Hvvvqi+YBwu2V-gE-sjm7XrfE0p2zds4CE2Q@mail.gmail.com","threadId":"36969","inReplyTo":null,"subject":"Should branches be objects?","fromName":"Nico Williams","fromEmail":"nico@cryptonector.com","sentAt":"2014-06-19T21:37:36Z","receivedAt":"2014-06-19T21:37:36Z","isPatch":false,"sender":{"key":"nico@cryptonector.com","avatar":null},"body":"[I'm a list newbie here, but a git power user.]\n\nIf branches were objects...\n\n - one could see the history of branches, including\n\n - how commits were grouped when pushed/pulled (push 5 commits, and\nthe branch object will record that its head moved by those five\ncommits at once)\n\n - rebase history (git log -> better than git reflog!)\n\n - object transactional APIs would be used to update branches\n\nBranch objects might be purely local, recording what was done in a\nlocal repo to a branch, but they might be pullable, to make branch\nhistory viewable in clones.\n\nJust a thought,\n\nNico\n--\n"},{"id":"244727","messageId":"20140619234614.GY8557@google.com","threadId":"36969","inReplyTo":"CAK3OfOiB0YWf25Hvvvqi+YBwu2V-gE-sjm7XrfE0p2zds4CE2Q@mail.gmail.com","subject":"Re: Should branches be objects?","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2014-06-19T23:46:14Z","receivedAt":"2014-06-19T23:46:14Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nNico Williams wrote:\n\n> - one could see the history of branches, including\n\nInteresting. 'git log -g' is good for getting that information\nlocally, but the protocol doesn't have a way to get it from a remote\nserver so you have to ssh in. Ronnie (cc-ed) and I were talking\nrecently about whether it would make sense to update git protocol to\nhave a way to get at the remote reflogs more easily --- would that be\nuseful to you?\n\n> - how commits were grouped when pushed/pulled (push 5 commits, and\n> the branch object will record that its head moved by those five\n> commits at once)\n\nThe reflog on the server (if enabled) records this.\n\n> - rebase history (git log -> better than git reflog!)\n\nThe local reflog ('git log -g ') records this.\n\n> - object transactional APIs would be used to update branches\n\nRonnie's recent ref-transaction code does this.\n\nThanks and hope that helps,\nJonathan\n"},{"id":"244728","messageId":"CAK3OfOjnYXxhWT7Vp-0Pcb=F8ze6wJnqSm1_LW0Zp1FmYZfUEg@mail.gmail.com","threadId":"36969","inReplyTo":"20140619234614.GY8557@google.com","subject":"Re: Should branches be objects?","fromName":"Nico Williams","fromEmail":"nico@cryptonector.com","sentAt":"2014-06-20T00:25:06Z","receivedAt":"2014-06-20T00:25:06Z","isPatch":false,"sender":{"key":"nico@cryptonector.com","avatar":null},"body":"On Thu, Jun 19, 2014 at 6:46 PM, Jonathan Nieder wrote:\n> Nico Williams wrote:\n>\n>> - one could see the history of branches, including\n>\n> Interesting. 'git log -g' is good for getting that information\n> locally, but the protocol doesn't have a way to get it from a remote\n> server so you have to ssh in. Ronnie (cc-ed) and I were talking\n> recently about whether it would make sense to update git protocol to\n> have a way to get at the remote reflogs more easily --- would that be\n> useful to you?\n\nYes and no. I've thought about that some concept, but:\n\na) reflogs include information about what's done to the workspace\n(checkout...) that's not relevant to any branch,\n\nb) reflogs aren't objects, which ISTM has caused transactional issued\n(even if they are fixed or soon to be),\n\nc) the fewer kinds of things, the more elegant the design, so maybe\nreflogs ought to be objects themselves, which is one thought that led\nme to \"branches should be objects\".\n\nAnother thought that led me there is that I often do:\n\n$ git checkout -b ${branch}-rebase1\n$ git rebase -i master\n...\n$ git checkout -b ${branch}-rebase2\n$ git rebase -i master\n...\n\nI iterate through this until a set of commits is the way the upstream wants it.\n\nNo one really needs that history, except me: possibly to show my\nboss/customer, possibly to put together a list of changes I've done to\nshow the upstream maintainer, ... Yes, this is in the reflog, but...\nit's mixed up with unrelated stuff.\n\nAlso, I'd like to be able to git diff\n... Again, for my\nown purposes in collating changes I've done to previously submitted\nPRs.\n\nNow, I can do that as I always have, but it litters my branch namespace.\n\nLastly, there are people who just don't get rebasing. They think it's\nhorrible because it changes the truth. You've met them, I'm certain.\nBranches as objects might help mollify them.\n\n>> - how commits were grouped when pushed/pulled (push 5 commits, and\n>> the branch object will record that its head moved by those five\n>> commits at once)\n>\n> The reflog on the server (if enabled) records this.\n\nYeah, though as you point out I can't see it.\n\n>> - rebase history (git log -> better than git reflog!)\n>\n> The local reflog ('git log -g ') records this.\n\nSee above.\n\n>> - object transactional APIs would be used to update branches\n>\n> Ronnie's recent ref-transaction code does this.\n\nSpeaking of which: are there any power failure corruption cases left\nin git? How is this tested?\n\nNico\n--\n"},{"id":"244729","messageId":"CAK3OfOhC4krFUDJ9nOzMfhkHjX_76L2k+tdEsbqcAzWDseeNAg@mail.gmail.com","threadId":"36969","inReplyTo":"CAK3OfOjnYXxhWT7Vp-0Pcb=F8ze6wJnqSm1_LW0Zp1FmYZfUEg@mail.gmail.com","subject":"Re: Should branches be objects?","fromName":"Nico Williams","fromEmail":"nico@cryptonector.com","sentAt":"2014-06-20T00:31:03Z","receivedAt":"2014-06-20T00:31:03Z","isPatch":false,"sender":{"key":"nico@cryptonector.com","avatar":null},"body":"Another thing is that branches as objects could store a lot more\ninformation, like:\n\n - the merge-base and HEAD for a rebase (and the --onto)\n\n - the interactive rebase plan! (and diffs to what would have been\nthe non-interactive plan)\n\n - the would-be no-op non-interactive rebase plan post rebase (again,\nso elucidate what commit splitting and such things occurred during a\nrebase)\n\nNico\n--\n"},{"id":"244730","messageId":"20140620010147.GZ8557@google.com","threadId":"36969","inReplyTo":"CAK3OfOjnYXxhWT7Vp-0Pcb=F8ze6wJnqSm1_LW0Zp1FmYZfUEg@mail.gmail.com","subject":"Re: Should branches be objects?","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2014-06-20T01:01:47Z","receivedAt":"2014-06-20T01:01:47Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Nico Williams wrote:\n\n> a) reflogs include information about what's done to the workspace\n> (checkout...) that's not relevant to any branch,\n\nNope, reflogs just record changes to refs and information about why\nthey happened.\n\n> b) reflogs aren't objects, which ISTM has caused transactional issued\n> (even if they are fixed or soon to be),\n\nNot sure I understand. Do you mean that if reflogs were named by their\ncontent then they wouldn't need to be renamed when a ref is renamed?\nOr are you referring to some other atomicity issue?\n\n[...]\n> $ git checkout -b ${branch}-rebase1\n> $ git rebase -i master\n> ...\n> $ git checkout -b ${branch}-rebase2\n> $ git rebase -i master\n> ...\n>\n> I iterate through this until a set of commits is the way the upstream wants it.\n>\n> No one really needs that history, except me: possibly to show my\n> boss/customer, possibly to put together a list of changes I've done to\n> show the upstream maintainer, ... Yes, this is in the reflog, but...\n> it's mixed up with unrelated stuff.\n\nYes, this isn't something we do well at all. It would be nice to have a\ntool that can take two versions of a branch (from different refs, taken\nfrom the reflog, or whatever) and visually represent what happened to\ncorresponding commits.\n\nThomas Rast started work on such a thing called tbdiff, which you can\nfind at https://github.com/trast/tbdiff.\n\n[...]\n> Also, I'd like to be able to git diff\n> ... Again, for my\n> own purposes in collating changes I've done to previously submitted\n> PRs.\n\nDo you mean 'git diff mybranch mybranch@{3}' /\n'git diff @{3.days.ago}'?\n\n[...]\n>>> - object transactional APIs would be used to update branches\n>>\n>> Ronnie's recent ref-transaction code does this.\n>\n> Speaking of which: are there any power failure corruption cases left\n> in git? How is this tested?\n\nWhat kind of power failure corruption are you talking about? Git\nusually updates files by writing a completely new file and then\nrenaming it into place, so depending on your filesystem this means it\nis very hard or very easy to lose data with a power failure. :)\n\nIf you're on one of those filesystems where it is very easy and you\nlose power a lot, you'll probably want to enable the\ncore.fsyncobjectfiles configuration option. It might be worth adding\nanother knob like that for the other files git writes if someone is\ninterested.\n\nJonathan\n"},{"id":"244738","messageId":"20140620022706.GB31908@sigill.intra.peff.net","threadId":"36969","inReplyTo":"20140620010147.GZ8557@google.com","subject":"Re: Should branches be objects?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2014-06-20T02:27:06Z","receivedAt":"2014-06-20T02:27:06Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jun 19, 2014 at 06:01:47PM -0700, Jonathan Nieder wrote:\n\n> > Speaking of which: are there any power failure corruption cases left\n> > in git? How is this tested?\n> \n> What kind of power failure corruption are you talking about? Git\n> usually updates files by writing a completely new file and then\n> renaming it into place, so depending on your filesystem this means it\n> is very hard or very easy to lose data with a power failure. :)\n\nWe use git-core on ext4 at GitHub, and we certainly have seen our share\nof machines failing unexpectedly. We haven't seen any problems of this\nnature[1] (but note that we journal data writes; you should also be fine\nwith ordered data writes, but data=writeback is likely disastrous).\n\n> If you're on one of those filesystems where it is very easy and you\n> lose power a lot, you'll probably want to enable the\n> core.fsyncobjectfiles configuration option. It might be worth adding\n> another knob like that for the other files git writes if someone is\n> interested.\n\nYou probably know this already Jonathan, but to be clear:\n\nGit always fsyncs pack writes. That knob controls fsyncing of loose\nobject files, but nothing else. So ref writes (and writing packed-refs)\ncould be corrupted on a filesystem that doesn't order data and metadata\nwrites (and there is currently no way to tell git to do otherwise).\n\nMy recommendation would be to steer clear or reconfigure such systems,\nbut it also would not be very hard to add an optional fsync in those\ncases.\n\n-Peff\n\n[1] We did have one case where after a crash packfiles would end up\n corrupted, but it turned out to be bad RAM in a battery-backed RAID\n card that was transparently caching (and losing) the writes.\n There's not much git can do when fsync lies to it, nor much the\n kernel can do when the hardware lies to it. :)\n"}]}