threads / patch / 1066

patchpack-objects: emit base before delta.

Subject: [PATCH] pack-objects: emit base before delta.

## tl;dr

4 messages between Jul 1, 2005 and Jul 1, 2005. Diffs are folded; open one to read it.

replies: 3people: 2as markdown or json

Junio C Hamano· Jul 1, 2005, 05:58 UTC · lore

This micro-optimizes the order of objects in a pack. By emitting base objects before deltified ones, unpack-objects do not keep items on delta_list.

I think it is prudent to keep the add_delta_to_list() logic in unpack-objects, so this commit only changes the packing side.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---

 pack-objects.c |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

9be15981d7038f1d9d0b105f45760987584e60a6

Show changes to pack-objects.c +5 −3

diff --git a/pack-objects.c b/pack-objects.c
--- a/pack-objects.c
+++ b/pack-objects.c
@@ -114,11 +114,13 @@ static unsigned long write_one(struct sh
 		 * if it is written already.
 		 */
 		return offset;
-	e->offset = offset;
-	offset += write_object(f, e);
-	/* if we are delitified, write out its base object. */
+
+	/* if we are delitified, write out its base objects first */
 	if (e->delta)
 		offset = write_one(f, e->delta, offset);
+
+	e->offset = offset;
+	offset += write_object(f, e);
 	return offset;
 }
 
------------

Junio C Hamano· Jul 1, 2005, 06:18 UTC · re: Junio C Hamano · lore

Re: [PATCH] pack-objects: emit base before delta.

>>>>> "JCH" == Junio C Hamano <junkio@cox.net> writes:

JCH> This micro-optimizes the order of objects in a pack. By JCH> emitting base objects before deltified ones, unpack-objects do JCH> not keep items on delta_list.

Oops; "does not have to keep items" is what I meant.

Linus Torvalds· Jul 1, 2005, 15:28 UTC · re: Junio C Hamano · lore

Re: [PATCH] pack-objects: emit base before delta.

On Thu, 30 Jun 2005, Junio C Hamano wrote:

>
> This micro-optimizes the order of objects in a pack.  By
> emitting base objects before deltified ones, unpack-objects do
> not keep items on delta_list.

So I like this from an unpacking standpoint, but at the same time I don't actually think it's correct from an access pattern standpoint.

When using a pack file as a run-time object store, the current packing order means that we generally traverse the pack-file in a nice forward direction. We don't jump backwards in the file very much, which should be good both for CPU and disk caches (both of them tend to have prefetch logic that often works better for nice access patterns). So I was actually pretty happy with the fact that we packed "optimally" in this sense: if the object was an important delta (ie an early one), we'd basically end up always walking forward until we hit the object it was a delta against.

I dunno. Maybe it doesn't matter. Our other heuristics to pack recent objects before later one might mean that we tend to have mainly backwards-going deltas (both in history _and_ in pack file layout).

The expense of keeping track of delta objects isn't that high, and the "pending delta" logic in unpack-objects isn't that complicated, so ..

Do you have some other reason you want to do this?

		Linus

Junio C Hamano· Jul 1, 2005, 15:40 UTC · re: Linus Torvalds · lore

Re: [PATCH] pack-objects: emit base before delta.

>>>>> "LT" == Linus Torvalds <torvalds@osdl.org> writes:

LT> So I like this from an unpacking standpoint, but at the same LT> time I don't actually think it's correct from an access LT> pattern standpoint.

My faulty logic, when I did the patch, was "we would jump around on random access anyway, so forcing the runtime to go backwards even when we know that these 40 objects are followed in this exact order to resolve delta is also OK." It is not.

Please drop the patch.