Re: Questions about the new

Hi,

I read from the release notes of git 1.6.5 about the new "replace" mechanism. It is presented as "a replacement of the "grafts" mechanism, with the added advantage that it can be transferred across repositories."

Since there is still little information about it, I would like to ask the following:

1) Grafts and replace entries seem to operate on different aspects of the
history graph. Grafts operate on arcs and replace on nodes. 
As such, replace entries seem less general to me. 
Apparently, to simulate a graft with replace entries, you need to introduce
extra commit objects. For instance, if object B has no parents, to pretend that
it derives from some A, one needs to create an object B' equivalent to B but
for the parents and then replace B by B', is this right? Conversely, I guess
you can always simulate a replace entry with the graft mechanism, without the
need to add any extra commit object. Am I overlooking something? 

2) Is it currently possible to use a replace entry to replace a commit object
with nothing? Namely if B has A as its sole parent, is it possible to have a
replace entry such as A-sha1 becomes null, to pretend that B is a hierarchy
root?  

3) If I remember correctly, there was a reason why grafts were not considered
suitable for transferring across repos. Can someone remind me about it? How
does the replace mechanism address this issue?

Thanks,

Sergio

Sergio <sergio.callegari@gmail.com> writes:

Grafts allow you to change a little part of commit objects (the parent relationship). Replacements allow (and require) you to change the whole commit object. So obviously, anything you can do with grafts can be done with replacements.

If you have your A-B-x-x-x history and want to remove A from the history, you replace B with a commit object that is identical, except for the missing parent link to A.

Replacements allow you do to other kinds of changes as well (comments, authorshiips) and changes to other object(?), so it is a much more general mechanism.

But I missed the discussion about it, so I'm not sure why it was felt it was needed, or why the grafts mechanism wasn't enough.

Sergio schrieb:

Correct, but see below about tree and blob objects.

With grafts you can only change parenthood; with replace entries you can change parenthood *and* all other aspects of a commit (message, author, committer, dates).

Hence, replace entries are more general than grafts.

Yes. Use git-cat-file + edit + git-hash-object as explained in this message just the other day: http://thread.gmane.org/gmane.comp.version-control.git/129727/focus=129907

You cannot; see above. You can even replace tree objects and blob objects using replace entries, IIUC, but you cannot do that with grafts.

Sure. Just make a commit object that does not have parents.

The problem with grafts was that, for example, git-pack-objects obeyed the graft, and could create a broken repository by removing grafted-away objects. And since git-fsck also obeyed the graft, it did not notice the breakage.

OTOH, history walkers (upload-pack, send-pack, pack-objects) and fsck never obey replace entries in the history. But they do keep track of them (and the history that they reference) because they are referenced from the refs/replace namespace.

-- Hannes

Thanks Johannes for all the detailed explanations

Johannes Sixt <j.sixt <at> viscovery.net> writes:

 >
 > Sergio schrieb:
 > > 1) Grafts and replace entries seem to operate on different aspects 
of the
 > > history graph. Grafts operate on arcs and replace on nodes.
 >
 > Correct, but see below about tree and blob objects.

OK, the replace mechanism also can replace a blob object or a tree. My focus was on commit objects only.

 >
 > > As such, replace entries seem less general to me.
 >
 > With grafts you can only change parenthood; with replace entries you can
 > change parenthood *and* all other aspects of a commit (message, author,
 > committer, dates).
 >
 > Hence, replace entries are more general than grafts.

Limiting the discussion to commit objects, I think there are two possible scenarios.

1) You create new commits objects as needed
2) You do not.

If you follow 1), I believe grafts and replace entries have exactly the same flexibility.

If I happen not to like commit B in A---B---C and I want A---B'---C where B' has completely different aspects from B I can either replace B by B' or graft away B, pretending that the parent of A is B

But there are many things that can be done with grafts merely adding a graft (e.g. cutting away a part of history, joining history), that cannot be done with replace entries without creating new commits objects.

I was asking because I was wandering whether replace entries were first or later meant to make grafts deprecated. I hope not, because for a few things working on arcs seems still nice.

 > > Apparently, to simulate a graft with replace entries, you need to 
introduce
 > > extra commit objects. For instance, if object B has no parents, to 
pretend >
 > Yes. Use git-cat-file + edit + git-hash-object as explained in this
 > message just the other day:
 > 
http://thread.gmane.org/gmane.comp.version-control.git/129727/focus=129907

Thanks, good pointer. I missed this!

 > > Conversely, I guess
 > > you can always simulate a replace entry with the graft mechanism, 
without the
 > > need to add any extra commit object. Am I overlooking something?
 >
 > You cannot; see above.

Well, I meant for what regards commit objects only.

If I want to replace some commit X by some commit X' I merely need to modify the parent information of all the commits that are child of X so that they pretend to be child of X', or am I missing something?

 > You can even replace tree objects and blob objects
 > using replace entries, IIUC, but you cannot do that with grafts.

Definitely right!
 
 > > 2) Is it currently possible to use a replace entry to replace a 
commit object
 > > with nothing? Namely if B has A as its sole parent, is it possible 
to have a
 > > replace entry such as A-sha1 becomes null, to pretend that B is a 
hierarchy
 > > root? 
 >
 > Sure. Just make a commit object that does not have parents.

OK, you need to create a new commit object. At the beginning for some reason I thought that you could replace an object with "nothing" or 00000000000000000000000000000000000000000000

 > > 3) If I remember correctly, there was a reason why grafts were not 
considered
 > > suitable for transferring across repos. Can someone remind me about 
it? How
 > > does the replace mechanism address this issue?
 >
 > The problem with grafts was that, for example, git-pack-objects 
obeyed the
 > graft, and could create a broken repository by removing grafted-away
 > objects. And since git-fsck also obeyed the graft, it did not notice the
 > breakage.
 >
 > OTOH, history walkers (upload-pack, send-pack, pack-objects) and fsck
 > never obey replace entries in the history. But they do keep track of them
 > (and the history that they reference) because they are referenced 
from the
 > refs/replace namespace.
 >

Thanks for the explanation. Can this be made possible for grafts too? Wouldn't it be a matter of having history walkers never obey grafts but keep track of them (i.e. of the history of the parenthood they reference)?

Like we have "annotated" or heavyweight tags living as objects in the database, would it be possible or make sense to have annotated grafts or replace entries, so that one can express why, by whom and when history was changed?

Sergio Callegari <sergio.callegari@gmail.com> writes:

You need to find all the commits that are child of X in the first place. What should happen if your colleague has such a commit in his repository (which you haven't fetched from yet), you enumerated all children of X known to you in your graft file and then you fetch from him? You need to enumerate all children of X again to keep the graft file up to date.

In the past we discussed the possibility of that for quite a while but never saw a successful implementation. The replace mechanism seemed a cleaner way to do this, and it turned out to be the case.

You are welcome to try doing that for the grafts, of course.

Junio C Hamano wrote:

Ok, that is enlightening. When trying to sort out the differences, advantages and disadvantages of operating on arcs (grafts like) or on nodes (replacements like) I was thinking local, rather than distributed. Now the advantage of working on nodes is much clearer to me.

Thanks as usual for the very clear explanation.

Sergio

On Monday 12 October 2009, Sergio Callegari wrote:

[...]

You mean "pretending that the parent of C is A", right?

Yes, but when you create a graft, you add a new line in the graft file. You don't get the grafts for free.

I don't think they will be deprecated soon. And anyway there will probably be a warning when a graft is used if it is deprecated.

[...]

If you use git replace you just need to create commit X' and then use "git replace X X'". If you use grafts, yes, you have to modify the parent information of all the commits that are child of X.

The problem is that grafts are special, so all the history walking commands should be changed to deal with them specially. With the replace mechanism, commits and refs are used, and all the commands already know how to deal with them.

There is a patch series about "notes" floating around that deals with annotating any commit. So it could be used for that.

And anyway when you create the replacement commit, you can state in the commit message that it is a replacement commit, who created it, etc.

Regards, Christian.

On Mon, Oct 12, 2009 at 02:19:11PM +0200, Johannes Sixt wrote:

Actually, you can. I have written a script that did exactly this. It required to modify parents to point to the new commit. The tricky part was that modification could be on top of other modifications, but I was able to handle this case too. Yet, my script was so hackish that I have never dared to share it with someone (and I used it only couple times during CVS to Git conversion).

I think both mechanism are theoretically equivalent, but with grafts, it was rather difficult to replace objects (but not impossible!).

Moreover, grafted-away objects could be removed by the garbage collector...

Dmitry

Johannes Sixt <j.sixt@viscovery.net> writes:

To be more detailed, the problem is that if git-pack-objects, git-fsck and git-gc obeys grafts, it can create broken repository by removing grafted away objects. If git-pack-objects, git-fsck and git-gc doesn't obey grafts, it can created broken repository (well, broken if we include grafts) by removing grafted in objects.

In the case of refs/replace git-pack-objects, git-fcsk and git-gc doesn't "obey" refs/replace... but replaced objects are protected by pruning by being referenced from refs/replace ref.

One of problems with grafts file was to come up with rule what do do if both repository you fetch from and the repository you fetch into have both grafts; in the case of refs/replace the usual rules about (conflicting) refs apply.

It is also easy to select whether to follow refs/replace or not: you fetch them into your refs/replace or not; you would have to add extra option to git-fetch to select whether to fetch and follow grafts in remote you fetch from.