Re: [PATCH] rev-list: add "--full-objects" flag.

From: Linus Torvalds <torvalds@osdl.org>
Date: Jul 11, 2005, 16:38 UTC
Message-ID: <Pine.LNX.4.58.0507110928070.17536@g5.osdl.org>
In-Reply-To: <m1irzh74m0.fsf@ebiederm.dsl.xmission.com>

On Mon, 11 Jul 2005, Eric W. Biederman wrote:

> 
> I guess I was expecting to pull from one tree into another unrelated
> tree.  Getting a tree with two heads and then be able to merge them
> together.

You can do it, but you have to do it by hand. It's a valid operation, but it's not an operation I want people to do by mistake, so it's not something the trivial helper scripts help with.

The way to do it by hand is to just use something stupid that doesn't understand what it's doing anyway, and just copy the files over. "cp -a" or "rsync" works fine. Then just do "git resolve" by hand. It's not very hard at all, but it's definitely something that should be a special case.

Show 5 quoted lines

> A couple of questions.
> 
> 1) Does git-clone-script when packed copy the entire repository
>    or just take a couple of slices of the tree where you have
>    references?

It only gets the objects needed for the references, nothing more.

So if you only get one branch, it will leave the objects that are specific to other branches alone.

> 2) Is there a way for a pack to create deltas against objects
>    that are not in the tree?  For a dumb repository making incremental
>    changes this is ideal.

A pack can only have deltas against objects in that pack. It caan't even have deltas to other objects in the same tree, it literally is only _within_ a pack. This is so that each pack is totally independent: you can always unpack (and verify) the objects in a pack _without_ having anything else (of course, the end result is often not a full project, and you won't have any references, but at least the _objects_ are valid).

I don't want to have deltas to outside the pack, because while it's obviously very nice from a size packing standpoint, it's totally horrid from an infrastructure standpoint. It would make it possible to have circular dependencies (ie deltas against each other) that could only be resolved by having a third pack (or the unpacked object).

It would also means that you may have to have two packs mapped at the same time to unpack them, which was very much against what I was aiming for: I think that in the long run, for truly huge projects, you'd want to have a history of packs, each maybe a gigabyte in size, and you may be in the situation that you simply cannot have two packs mapped at the same time because you don't have enough virtual memory for it.

So then inter-pack deltas would mean that you'd have to have "partial pack mapping" etc horrid special case logic. Right now, because a pack is always self-sufficient, you know that in order to unpack an object, if you find it in the index file, you will be able to unpack it by just mapping that pack and going off..

So the rule is: don't pack too often. The unpacked objects are actually working really really well as long as you don't have tens of thousands of them. Having a few hundred (or even a few thousand) unpacked objects is not a problem at all. Then you do a "git repack" when it starts getting uncomfortable, and you you continue.

			Linus