git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: 1.3.0 creating bigger packs than 1.2.3

From
Nicolas Pitre <nico@cam.org>
Date
Apr 21, 2006, 02:28 UTC
Message-ID
<Pine.LNX.4.64.0604202213470.2215@localhost.localdomain>
In-Reply-To
<20060421012029.GB819@spearce.org>
On Thu, 20 Apr 2006, Shawn Pearce wrote:
Show 29 quoted lines
> Based on Linus' comment I changed your patch to just the following.
> It still produced the 46M pack file, so the first hunk apears to
> not have had much of an affect with this data.
> 
> From a running time perspective it appears as though this patch is
> making things slightly better, not worse.  I ran it a few times
> for each case always using the 46M pack as input for
>  "git-repack -a -d -f".
> 
>   'next'       137.13 real        95.82 user        25.24 sys
>   'next'+patch 131.62 real        89.35 user        28.56 sys
> 
> but even if the running time was an extra 6 seconds I'd still rather
> spend 4% more running time to use 1/2 the storage space.
> 
> 
> diff --git a/pack-objects.c b/pack-objects.c
> index 09f4f2c..f7d6217 100644
> --- a/pack-objects.c
> +++ b/pack-objects.c
> @@ -1052,7 +1052,7 @@ static int try_delta(struct unpacked *cu
>         if (cur_entry->delta)
>                 max_size = cur_entry->delta_size-1;
>         if (sizediff >= max_size)
> -               return -1;
> +               return 0;
>         delta_buf = diff_delta(old->data, oldsize,
>                                cur->data, size, &delta_size, max_size);
>         if (!delta_buf)

I can confirm this is indeed the best fix so far. Any "smarter" solution I could think of did increase the size of the final pack quite spectacularly and rather unexpectedly with Shawn's repository.

Of course removing the if (sizediff >= max_size) entirely does produce a smaller pack (39MB) but it takes about twice the CPU.

With the patch above the Linux kernel pack is 0.3% smaller with 1% more CPU usage. But like for the diff-delta hash list limiting code this small overhead is certainly a good compromize to avoid big degradations in some other cases.

Nicolas
Previous: Shawn PearceNext: Shawn Pearce
Message 25 of 30 in “1.3.0 creating bigger packs than 1.2.3”
  1. Shawn PearceApr 20, 2006
  2. Linus TorvaldsApr 20, 2006
  3. Shawn PearceApr 20, 2006
  4. Linus TorvaldsApr 20, 2006
  5. Shawn PearceApr 20, 2006
  6. Linus TorvaldsApr 20, 2006
  7. Junio C HamanoApr 20, 2006
  8. Shawn PearceApr 20, 2006
  9. Nicolas PitreApr 20, 2006
  10. Junio C HamanoApr 20, 2006
  11. Shawn PearceApr 20, 2006
  12. Jakub NarebskiApr 20, 2006
  13. Nicolas PitreApr 20, 2006
  14. Shawn PearceApr 20, 2006
  15. Nicolas PitreApr 20, 2006
  16. Junio C HamanoApr 20, 2006
  17. Nicolas PitreApr 20, 2006
  18. Junio C HamanoApr 20, 2006
  19. Shawn PearceApr 20, 2006
  20. Junio C HamanoApr 20, 2006
  21. Shawn PearceApr 21, 2006
  22. Linus TorvaldsApr 20, 2006
  23. Nicolas PitreApr 21, 2006
  24. Shawn PearceApr 21, 2006
  25. Nicolas PitreApr 21, 2006
  26. Shawn PearceApr 21, 2006
  27. Nicolas PitreApr 21, 2006
  28. Shawn PearceApr 21, 2006
  29. Junio C HamanoApr 20, 2006
  30. Nicolas PitreApr 20, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.