Should branches be objects?

[I'm a list newbie here, but a git power user.]

If branches were objects...

 - one could see the history of branches, including

 - how commits were grouped when pushed/pulled (push 5 commits, and
the branch object will record that its head moved by those five
commits at once)

 - rebase history (git log <branch-object> -> better than git reflog!)

 - object transactional APIs would be used to update branches

Branch objects might be purely local, recording what was done in a local repo to a branch, but they might be pullable, to make branch history viewable in clones.

Just a thought,

Nico --

Hi,

Nico Williams wrote:

Interesting. 'git log -g' is good for getting that information locally, but the protocol doesn't have a way to get it from a remote server so you have to ssh in. Ronnie (cc-ed) and I were talking recently about whether it would make sense to update git protocol to have a way to get at the remote reflogs more easily --- would that be useful to you?

The reflog on the server (if enabled) records this.

The local reflog ('git log -g <branch>') records this.

Ronnie's recent ref-transaction code does this.

Thanks and hope that helps, Jonathan

On Thu, Jun 19, 2014 at 6:46 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:

Yes and no.  I've thought about that some concept, but:

a) reflogs include information about what's done to the workspace (checkout...) that's not relevant to any branch,

b) reflogs aren't objects, which ISTM has caused transactional issued (even if they are fixed or soon to be),

c) the fewer kinds of things, the more elegant the design, so maybe reflogs ought to be objects themselves, which is one thought that led me to "branches should be objects".

Another thought that led me there is that I often do:

$ git checkout -b ${branch}-rebase1 $ git rebase -i master ... $ git checkout -b ${branch}-rebase2 $ git rebase -i master ...

I iterate through this until a set of commits is the way the upstream wants it.

No one really needs that history, except me: possibly to show my boss/customer, possibly to put together a list of changes I've done to show the upstream maintainer, ... Yes, this is in the reflog, but... it's mixed up with unrelated stuff.

Also, I'd like to be able to git diff <branch-version>..<same-branch-diff-branch-version>. Again, for my own purposes in collating changes I've done to previously submitted PRs.

Now, I can do that as I always have, but it litters my branch namespace.

Lastly, there are people who just don't get rebasing. They think it's horrible because it changes the truth. You've met them, I'm certain. Branches as objects might help mollify them.

Yeah, though as you point out I can't see it.

See above.

Speaking of which: are there any power failure corruption cases left in git? How is this tested?

Nico --

Another thing is that branches as objects could store a lot more information, like:

 - the merge-base and HEAD for a rebase (and the --onto)

 - the interactive rebase plan!  (and diffs to what would have been
the non-interactive plan)

 - the would-be no-op non-interactive rebase plan post rebase (again,
so elucidate what commit splitting and such things occurred during a
rebase)

Nico --

Nico Williams wrote:

Nope, reflogs just record changes to refs and information about why they happened.

Not sure I understand. Do you mean that if reflogs were named by their content then they wouldn't need to be renamed when a ref is renamed? Or are you referring to some other atomicity issue?

[...]

Yes, this isn't something we do well at all. It would be nice to have a tool that can take two versions of a branch (from different refs, taken from the reflog, or whatever) and visually represent what happened to corresponding commits.

Thomas Rast started work on such a thing called tbdiff, which you can find at https://github.com/trast/tbdiff.

[...]

Do you mean 'git diff mybranch mybranch@{3}' / 'git diff <mybranch> <mybranch>@{3.days.ago}'?

[...]

What kind of power failure corruption are you talking about? Git usually updates files by writing a completely new file and then renaming it into place, so depending on your filesystem this means it is very hard or very easy to lose data with a power failure. :)

If you're on one of those filesystems where it is very easy and you lose power a lot, you'll probably want to enable the core.fsyncobjectfiles configuration option. It might be worth adding another knob like that for the other files git writes if someone is interested.

Jonathan

On Thu, Jun 19, 2014 at 06:01:47PM -0700, Jonathan Nieder wrote:

We use git-core on ext4 at GitHub, and we certainly have seen our share of machines failing unexpectedly. We haven't seen any problems of this nature[1] (but note that we journal data writes; you should also be fine with ordered data writes, but data=writeback is likely disastrous).

You probably know this already Jonathan, but to be clear:

Git always fsyncs pack writes. That knob controls fsyncing of loose object files, but nothing else. So ref writes (and writing packed-refs) could be corrupted on a filesystem that doesn't order data and metadata writes (and there is currently no way to tell git to do otherwise).

My recommendation would be to steer clear or reconfigure such systems, but it also would not be very hard to add an optional fsync in those cases.

-Peff

[1] We did have one case where after a crash packfiles would end up
    corrupted, but it turned out to be bad RAM in a battery-backed RAID
    card that was transparently caching (and losing) the writes.
    There's not much git can do when fsync lies to it, nor much the
    kernel can do when the hardware lies to it. :)