Re: Git Notes idea.

From: Jeff King <peff@peff.net>
Date: Dec 20, 2008, 04:54 UTC
Message-ID: <20081220045437.GA27341@coredump.intra.peff.net>
In-Reply-To: <5d46db230812191424m14e82c5fx1c1c12027db901ed@mail.gmail.com>

On Fri, Dec 19, 2008 at 04:24:01PM -0600, Govind Salinas wrote:

Show 7 quoted lines

> > I think so. Otherwise how will you push and pull notes? You won't even
> > know which one is the more recent tree, let alone handle any merges
> > caused by editing notes in two places.
> 
> Couldn't you simply merge your tree and theirs even if there is no
> history.  You would have to find a way to handle merges in any event
> since they could just as easily happen if you have a history.

Let's say I have a tree T1 like this:

  $COMMIT_A -> $BLOB_A
  $COMMIT_B -> $BLOB_B1

and a tree T2 like this:

  $COMMIT_B -> $BLOB_B2
  $COMMIT_C -> $BLOB_C

what is the correct merge? Was $COMMIT_A added in T1, or deleted in T2? How about $COMMIT_C? Even if you went with a strategy like "always add from both" (which I don't think is a good idea, because deleted notes will keep popping back up) you have a conflict with $COMMIT_B. Should it be B1 or B2? You can't tell if B1 became B2, vice versa, or if there is a true merge conflict.

Show 22 quoted lines

> > If the former, then you haven't solved the cruft accumulation problem.
> > You can get obsolete notes in your note history by rebasing on a branch
> > that is long-running (which is OK as long as you haven't published
> > _those particular_ commits). Or are you proposing to rebase and cleanup
> > the notes history every time you do a destructive operation?
> 
> Yes, it does not solve that problem.  But it does solve things like
> 
> Dev1 and Dev2 both have branches A and topic branch B. and they
> are in refs/notes/public (or refs/notes or something not branch specific).
> 
> Dev1 adds 100 notes to topic B, lets say half of them are obsolete due
> to rebases or whatever.  Dev2 pulls A and updates their notes
> as well.  Now Dev2 has acquired all the notes from Dev1 including the
> obsolete ones.  So you have 100 commits, 100 blobs and all the new
> trees that go with them that the user was not interested in.
> 
> Run this across 1000 users and you have a lot of cruft.
> 
> Now, if instead we have a per-branch notes scheme, then you only get
> the cruft from the branches you were interested in.  If you remove the
> history you could end up with no cruft because gc should handle it.

OK. But my point is that this is an incomplete solution. You can _still_ get cruft, and you _still_ have to deal with that cruft some other way. So we will still end up having to implement something else. And I might even be fine with a partial solution that helped some if it didn't come with a cost, but I think the "notes stick to branches" behavior is strictly worse.

Show 10 quoted lines

> > If the latter, then I don't see how you've solved the push-pull and
> > merge problem (which you need history for).
> 
> What git-fetch would have to do is say.  This is a note.  The remote
> sha is not the same as mine, i will treat this as a force and fetch the
> objects without checking history and then run a merge on the 2
> commits.  The notes merge could have its own strategy that checked
> if an object exists before deciding to add a new item or delete a
> removed one.  Then the user would only have to intervene if the
> notes where edited.

I don't like that because:

  - the user is going to end up manually resolving merge conflicts for
    things that _should_ have been fast forwards. But much worse, it's
    going to be on content they may never even have seen before. How
    will they decide which is which?

  - how do you push notes? There's no opportunity to handle the merge
    on the remote side. And you can't just pull, merge locally, and push
    what is now a fast-forward, because there is no concept of
    fast-forward without history.

  - Suddenly pulling and pushing notes isn't just taken care of by the
    usual ref transfer mechanisms. We have to implement a whole new
    system.

> You are correct of course that it will just be wasted space.  But I am
> concerned that it could end up being a lot of wasted space.  I mean, what
> if every person who contributed to the kernel contributed note cruft.  Users

What if every person who contributed to the kernel contributed history cruft? It's really the same problem, and it is solved by people keeping their trees clean (via rebase) and being picky about how data comes into your tree (i.e., don't pull from people with cruft). I suspect Linus wouldn't pull notes at all (and they wouldn't make it over patch transmission anyway). But in a workflow that is pulling the notes, the right time to clean up history is probably before publishing. That is, you can rebase and clean up your notes history just before you push it to somewhere public, just like you might clean up messy history.

> If you *really* don't think its something to be worried about then I
> am OK with that since you have a lot more experience with this, but it
> sounds hairy to me.

It is hairy, and I wish there were a better solution. But I think every other option is much worse.

-Peff