threads / patch / 22568

patchRevert "pack-objects: fix pack generation when using pack_size_limit"

Subject: [PATCH] Revert "pack-objects: fix pack generation when using pack_size_limit"

## tl;dr

3 messages between Feb 8, 2010 and Feb 8, 2010. Diffs are folded; open one to read it.

replies: 2people: 2as markdown or json

Nicolas Pitre· Feb 8, 2010, 15:39 UTC · lore

This reverts most of commit a2430dde8ceaaaabf05937438249397b883ca77a.

That commit made the situation better for repositories with relatively small number of objects. However with many objects and a small pack size limit, the time required to complete the repack tends towards O(n^2), or even much worse with long delta chains.

Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
---

Fixing this doesn't appear to be as trivial as I initially thought. Although I do have some ideas, they're not appropriate so late in the -rc period.

Show changes to 2 files +10 −12

builtin-pack-objects.c, t/t5300-pack-object.sh

diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index dcfe62a..e1d3adf 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -445,13 +445,9 @@ static int write_one(struct sha1file *f,
 	if (e->idx.offset || e->preferred_base)
 		return -1;
 
-	/*
-	 * If we are deltified, attempt to write out base object first.
-	 * If that fails due to the pack size limit then the current
-	 * object might still possibly fit undeltified within that limit.
-	 */
-	if (e->delta)
-	       write_one(f, e->delta, offset);
+	/* if we are deltified, write out base object first. */
+	if (e->delta && !write_one(f, e->delta, offset))
+		return 0;
 
 	e->idx.offset = *offset;
 	size = write_object(f, e, *offset);
@@ -505,9 +501,11 @@ static void write_pack_file(void)
 		sha1write(f, &hdr, sizeof(hdr));
 		offset = sizeof(hdr);
 		nr_written = 0;
-		for (i = 0; i < nr_objects; i++)
-			if (write_one(f, objects + i, &offset) == 1)
-				display_progress(progress_state, written);
+		for (; i < nr_objects; i++) {
+			if (!write_one(f, objects + i, &offset))
+				break;
+			display_progress(progress_state, written);
+		}
 
 		/*
 		 * Did we write the wrong # entries in the header?
@@ -582,7 +580,7 @@ static void write_pack_file(void)
 			written_list[j]->offset = (off_t)-1;
 		}
 		nr_remaining -= nr_written;
-	} while (nr_remaining);
+	} while (nr_remaining && i < nr_objects);
 
 	free(written_list);
 	stop_progress(&progress_state);
diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh
index 1058d98..7649b81 100755
--- a/t/t5300-pack-object.sh
+++ b/t/t5300-pack-object.sh
@@ -389,7 +389,7 @@ test_expect_success 'verify resulting packs' '
 test_expect_success 'tolerate packsizelimit smaller than biggest object' '
 	git config pack.packSizeLimit 1 &&
 	packname_11=$(git pack-objects test-11 <obj-list) &&
-	test 3 = $(ls test-11-*.pack | wc -l)
+	test 5 = $(ls test-11-*.pack | wc -l)
 '
 
 test_expect_success 'verify resulting packs' '

Junio C Hamano· Feb 8, 2010, 17:43 UTC · re: Nicolas Pitre · lore

Re: [PATCH] Revert "pack-objects: fix pack generation when using pack_size_limit"

Nicolas Pitre <nico@fluxnic.net> writes:

Show 13 quoted lines

> This reverts most of commit a2430dde8ceaaaabf05937438249397b883ca77a.
>
> That commit made the situation better for repositories with relatively
> small number of objects.  However with many objects and a small pack size
> limit, the time required to complete the repack tends towards O(n^2),
> or even much worse with long delta chains.
>
> Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
> ---
>
> Fixing this doesn't appear to be as trivial as I initially thought.
> Although I do have some ideas, they're not appropriate so late in 
> the -rc period.

Ok, so the idea is what a2430dd tried is an issue worth addressing but the particular execution wasn't good? We revert it for now, but we will try again after the release, perhaps doing it differently?

I agree with it if that is your intention, but "Most of" bugs me a bit.

Nicolas Pitre· Feb 8, 2010, 18:11 UTC · re: Junio C Hamano · lore

Re: [PATCH] Revert "pack-objects: fix pack generation when using pack_size_limit"

On Mon, 8 Feb 2010, Junio C Hamano wrote:

Show 19 quoted lines

> Nicolas Pitre <nico@fluxnic.net> writes:
> 
> > This reverts most of commit a2430dde8ceaaaabf05937438249397b883ca77a.
> >
> > That commit made the situation better for repositories with relatively
> > small number of objects.  However with many objects and a small pack size
> > limit, the time required to complete the repack tends towards O(n^2),
> > or even much worse with long delta chains.
> >
> > Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
> > ---
> >
> > Fixing this doesn't appear to be as trivial as I initially thought.
> > Although I do have some ideas, they're not appropriate so late in 
> > the -rc period.
> 
> Ok, so the idea is what a2430dd tried is an issue worth addressing but the
> particular execution wasn't good?  We revert it for now, but we will try
> again after the release, perhaps doing it differently?

Exact. I did try a few things in the hope that the "fix" would have been trivial enough to merge now but none of that worked satisfactorily.

> I agree with it if that is your intention, but "Most of" bugs me a bit.

It's not a straight revert in the sense that I left in some cleanups and left out the redundant self consistency check. Only the significant parts were reverted.

Nicolas