[PATCH] fix repack with --max-pack-size

Nicolas Pitre· May 31, 2007, 01:43 UTC · lore

Two issues here:

1) git-repack -a --max-pack-size=10 on the GIT repo dies pretty quick.
   There is a lot of confusion about deltas that were suposed to be 
   reused from another pack but that get stored undeltified due to pack
   limit and object size doesn't match entry->size anymore.  This test 
   is not really worth the complexity for determining when it is valid
   so get rid of it.

2) If pack limit is reached, the object buffer is freed, including when 
   it comes from a cached delta data.  In practice the object will be 
   stored in a subsequent pack undeltified, but let's make sure no 
   pointer to freed data subsists by clearing entry->delta_data.

I also reorganized that code a bit to make it more readable.

Signed-off-by: Nicolas Pitre <nico@cam.org>
---

Show changes to builtin-pack-objects.c +11 −18

diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 41472fc..ccb25f6 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -410,31 +410,24 @@ static unsigned long write_object(struct sha1file *f,
 		z_stream stream;
 		unsigned long maxsize;
 		void *out;
-		if (entry->delta_data && usable_delta) {
-			buf = entry->delta_data;
+		if (!usable_delta) {
+			buf = read_sha1_file(entry->sha1, &obj_type, &size);
+			if (!buf)
+				die("unable to read %s", sha1_to_hex(entry->sha1));
+		} else if (entry->delta_data) {
 			size = entry->delta_size;
+			buf = entry->delta_data;
+			entry->delta_data = NULL;
 			obj_type = (allow_ofs_delta && entry->delta->offset) ?
 				OBJ_OFS_DELTA : OBJ_REF_DELTA;
 		} else {
 			buf = read_sha1_file(entry->sha1, &type, &size);
 			if (!buf)
 				die("unable to read %s", sha1_to_hex(entry->sha1));
-			if (size != entry->size)
-				die("object %s size inconsistency (%lu vs %lu)",
-				    sha1_to_hex(entry->sha1), size, entry->size);
-			if (usable_delta) {
-				buf = delta_against(buf, size, entry);
-				size = entry->delta_size;
-				obj_type = (allow_ofs_delta && entry->delta->offset) ?
-					OBJ_OFS_DELTA : OBJ_REF_DELTA;
-			} else {
-				/*
-				 * recover real object type in case
-				 * check_object() wanted to re-use a delta,
-				 * but we couldn't since base was in previous split pack
-				 */
-				obj_type = type;
-			}
+			buf = delta_against(buf, size, entry);
+			size = entry->delta_size;
+			obj_type = (allow_ofs_delta && entry->delta->offset) ?
+				OBJ_OFS_DELTA : OBJ_REF_DELTA;
 		}
 		/* compress the data to store and put compressed length in datalen */
 		memset(&stream, 0, sizeof(stream));

Dana How· Jun 1, 2007, 00:01 UTC · re: Nicolas Pitre · lore

Re: [PATCH] fix repack with --max-pack-size

On 5/30/07, Nicolas Pitre <nico@cam.org> wrote:

Show 8 quoted lines

> Two issues here:
>
> 1) git-repack -a --max-pack-size=10 on the GIT repo dies pretty quick.
>    There is a lot of confusion about deltas that were suposed to be
>    reused from another pack but that get stored undeltified due to pack
>    limit and object size doesn't match entry->size anymore.  This test
>    is not really worth the complexity for determining when it is valid
>    so get rid of it.

This is very real.  A smaller fix would have been the hunk

Show changes to diff +0 −0

@@ -408,7 +407,7 @@ static unsigned long write_object(struct sha1file *f,
>               buf = read_sha1_file(entry->sha1, &type, &size);
>               if (!buf)
>                       die("unable to read %s", sha1_to_hex(entry->sha1));
>-               if (size != entry->size)
>+               if (size != entry->size && type == obj_type)
>                       die("object %s size inconsistency (%lu vs %lu)",
>                           sha1_to_hex(entry->sha1), size, entry->size);
>               if (usable_delta) {
from the max-blob-size patch that hasn't converged yet.

Both pack splitting and blob-size limiting could cause a delta base
not to appear in the pack containing a delta using the base.
Then you get the size mismatch Nicolas discussed
when the real object must be used and its size doesn't match
the delta sized stored in the size field.
I first saw this with max-blob-size,  but only realized recently
that it also applied to max-pack-size.  Sorry I didn't post a patch;
we are swamped at the moment.

ACK [but I didn't review (2)].

Thanks,

-- 
Dana L. How  danahow@gmail.com  +1 650 804 5991 cell

patchfix repack with --max-pack-size