git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Preferring shallower deltas on repack

From
Junio C Hamano <gitster@pobox.com>
Date
Jul 9, 2007, 07:27 UTC
Message-ID
<7vbqemvxyb.fsf@assigned-by-dhcp.cox.net>
In-Reply-To
<20070709065235.GJ4087@lavos.net>
bdowning@lavos.net (Brian Downing) writes:
Show 15 quoted lines
> (These timings are for the Git pack on Linux/amd64, --window and --depth
> both 100.  Since /usr/bin/time doesn't seem to report any useful memory
> statistics on Linux, I also have a "ps aux" line from when the memory
> size looked stable.  This was different from run to run but it shows the
> two are in the same order of magnitude.)
>
> Unpatched:
> 54.99user 0.18system 0:56.80elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (14major+32417minor)pagefaults 0swaps
> bdowning  5290 98.7  4.5 106788 92900 pts/1    R+   01:26   0:49 git pack-obj
>
> Patched:
> 55.37user 0.19system 0:56.35elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (0major+32249minor)pagefaults 0swaps
> bdowning  6086  100  4.5 106880 92996 pts/1    R+   01:29   0:49 git pack-obj

The number of minor faults are comparable (slightly favorable), which is a good sign.

Show 8 quoted lines
> The patched version is actually smaller in both SBCL's and Git's case
> (again, --window 100 and --depth 100):
>
> SBCL: 61696 bytes smaller (13294225-13232529)
> Git:  16010 bytes smaller (12690424-12674414)
>
> I believe the reason for this is that more deltas can get in under the
> depth limit.
Very sensible indeed.
Show 9 quoted lines
>> It would become worrysome (*BUT* infinitely more interesting)
>> once you start talking about a tradeoff between slightly larger
>> delta and much shorter delta.  Such a tradeoff, if done right,
>> would make a lot of sense, but I do not offhand think of a way
>> to strike a proper balance between them efficiently.
>
> Yeah, I was thinking about that too, and came to the same conclusion.
> I suspect you'd have to save a /lot/ of delta depth to want to pay any
> more I/O, though.

That may not be so. Deeper delta also means more I/O (and worse, because they can be from discontiguous areas) plus delta application.

Previous: Brian DowningNext: Brian Downing
Message 6 of 19 in “Preferring shallower deltas on repack”
  1. Brian DowningJul 9, 2007
  2. pack-objects: Prefer shallower deltas if the size is equalBrian Downing, Jul 9, 2007
  3. Junio C HamanoJul 9, 2007
  4. Junio C HamanoJul 9, 2007
  5. Brian DowningJul 9, 2007
  6. Junio C HamanoJul 9, 2007
  7. Brian DowningJul 9, 2007
  8. Nicolas PitreJul 9, 2007
  9. Junio C HamanoJul 9, 2007
  10. Brian DowningJul 9, 2007
  11. Nicolas PitreJul 9, 2007
  12. Brian DowningJul 9, 2007
  13. Brian DowningJul 9, 2007
  14. Nicolas PitreJul 9, 2007
  15. Brian DowningJul 9, 2007
  16. Shoddy pack information toolBrian Downing, Jul 9, 2007
  17. Junio C HamanoJul 11, 2007
  18. Pack information toolBrian Downing, Jul 12, 2007
  19. Linus TorvaldsJul 9, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.