git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] pack-objects: reuse data from existing pack.

From
Andreas Ericsson <ae@op5.se>
Date
Feb 16, 2006, 08:32 UTC
Message-ID
<43F438AA.1040508@op5.se>
In-Reply-To
<7vbqx8m62q.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano wrote:
Show 41 quoted lines
> When generating a new pack, notice if we have already the wanted
> object in existing packs.  If the object has a delitified
> representation, and its base object is also what we are going to
> pack, then reuse the existing deltified representation
> unconditionally, bypassing all the expensive find_deltas() and
> try_deltas() routines.
> 
> Also, when writing out such deltified representation and
> undeltified representation, if a matching data already exists in
> an existing pack, just write it out without uncompressing &
> recompressing.
> 
> Without this patch:
> 
>     $ git-rev-list --objects v1.0.0 >RL
>     $ time git-pack-objects p <RL
> 
>     Generating pack...
>     Done counting 12233 objects.
>     Packing 12233 objects....................
>     60a88b3979df41e22d1edc3967095e897f720192
> 
>     real    0m32.751s
>     user    0m27.090s
>     sys     0m2.750s
> 
> With this patch:
> 
>     $ git-rev-list --objects v1.0.0 >RL
>     $ time ../git.junio/git-pack-objects q <RL
> 
>     Generating pack...
>     Done counting 12233 objects.
>     Packing 12233 objects.....................
>     60a88b3979df41e22d1edc3967095e897f720192
>     Total 12233, written 12233, reused 12177
> 
>     real    0m4.007s
>     user    0m3.360s
>     sys     0m0.090s
> 

Whoa! Columbus and the egg. Strange noone saw it before. It's so obvious when you shove it under the nose like that. :)

Now that pack-creation went from bizarrely expensive to insanely cheap (well, comparable to "tar czf" anyways), what's BCP for packing a public repository? Always one mega-pack and never worry, or should one still use incremental and sometimes overlapping pack-files?

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231
Previous: Junio C HamanoNext: Junio C Hamano
Message 11 of 17 in “[FYI] pack idx format”
  1. Junio C HamanoFeb 15, 2006
  2. Johannes SchindelinFeb 15, 2006
  3. Nicolas PitreFeb 15, 2006
  4. Junio C HamanoFeb 16, 2006
  5. pack-objects: reuse data from existing pack.Junio C Hamano, Feb 16, 2006
  6. packed objects: minor cleanupJunio C Hamano, Feb 16, 2006
  7. Nicolas PitreFeb 16, 2006
  8. Junio C HamanoFeb 16, 2006
  9. Linus TorvaldsFeb 16, 2006
  10. Junio C HamanoFeb 16, 2006
  11. Andreas EricssonFeb 16, 2006
  12. Junio C HamanoFeb 16, 2006
  13. Junio C HamanoFeb 17, 2006
  14. pack-objects: finishing touches.Junio C Hamano, Feb 17, 2006
  15. pack-objects: avoid delta chains that are too long.Junio C Hamano, Feb 18, 2006
  16. Linus TorvaldsFeb 17, 2006
  17. Junio C HamanoFeb 17, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.