Re: [RFC] shallow clone

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

It turns out that this 'single' request step is unneeded, as long as we implement 'graft' requests. We can then tell "Cauterize at v2.6.14 and give me the master" to `upload-pack`. `upload-pack` would run `rev-list --objects master`, tries to include everything that is reachable from "master", but notices that the v2.6.14 commit does not have any parent (thanks to the customized graft) and stops there -- the result is the history since v2.6.14.

I thought about that before I wrote the message, but it boils down to grepping lines from grafts that have only one object name (i.e. cauterizing records), so it is redundant.

Also there is no strict reason to forbid cloning from such a shallow repository. No harm is done as long as you make it clear to somebody who clones from you that what you have is a shallow copy, so that the cloned repository can cauterize history at appropriate places.

A second generation clone, when cloning from a shallow repository, needs to mark itself that it has the same or shallower history (otherwise a third generation clone from it would not work), so the `upload-pack` protocol needs to be updated to send grafts information the `upload-pack` side usually uses to the downloader even when 'graft' request is not used by the downloader. But once it is done, you should be able to clone safely from a shallow repository and end up with a repository with the same (or shallower -- if you asked to make a shallow clone from it) history.

Yes, almost. We need to realize that `upload-pack` that hears "have A, want B" is allowed to omit objects that appear in `ls-tree B` output but not in `ls-tree A`. "have A" means not just "I have A", but "I have A and all of its ancestors", so just sending "have start_shallow" (or start_shallow^ for that matter) is not quite enough [*1*].

The message you are responding did two separate transfers (one 'single', and another 'fetch'); I do not particularly mind doing two (it is just an initial clone anyway), but as I said it turns out that we do not need the initial 'single'.

Probably -- I was just outlining the lowest-level mechanism and haven't thought much about the UI.

[Footnote]

*1* This is true even without more aggressive optimization by rev-list that does not exist there yet. Here is a minimalistic demonstration. One file project with a handful straight-line commits. Each change to the file reverts the change made by the previous commit.

 * The HEAD commit has "white", the HEAD~1 "black" and HEAD~2
   "white".

 * We say we are interested in things since HEAD~2 (i.e. we
   pretend that the history starts at HEAD~1 and it does not
   have a parent) and ask for HEAD.

 * Notice that only one copy of the file appears in the output.
   It is "black" blob.  We do not get "white" blob because we
   are telling it that we _have_ HEAD~2.  The resulting set of
   objects is not enough to check-out the HEAD commit.

This roughly corresponds to your "have shallow_start", but not quite -- in that sequence you have objects for HEAD~2 commit. But the point is that I want to leave the door open for optimizing upload-pack, so that it can choose to omit objects that do not appear in A when you say "have A", if the object appears in one of A's ancestors.

-- >8 -- #!/bin/sh

rm -fr .git

git init-db zebra=white echo $zebra >file git add file git commit -m initial

for i in 0 1 2 3 4 5
do
	case $zebra in
	white) zebra=black ;;
	black) zebra=white ;;
	esac
	echo $zebra >file
	git commit -a -m "$i $zebra"
done
git rev-list --objects HEAD~2..HEAD |
git name-rev --stdin