git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] diff-delta: produce optimal pack data

From
CBCarl Baldwin <cnb@fc.hp.com>
Date
Feb 24, 2006, 19:23 UTC
Message-ID
<20060224192354.GC387@hpsvcnb.fc.hp.com>
In-Reply-To
<Pine.LNX.4.64.0602241350190.31162@localhost.localdomain>
On Fri, Feb 24, 2006 at 01:57:20PM -0500, Nicolas Pitre wrote:
Show 6 quoted lines
> On Fri, 24 Feb 2006, Carl Baldwin wrote:
> 
> > My version is 1.2.1.  I have not been following your work.  When was
> > pack data reuse introduced?
> 
> Try out version 1.2.3.
I'm on it now.
> > From where can I obtain your delta patches?
> 
> Forget them for now -- they won't help you.
Ah, I have been looking at your patches and clearly they will not help.
Show 7 quoted lines
> > There is really no opportunity for pack-data reuse in this case.  The
> > repository had never been packed or cloned in the first place.  As I
> > said, I do not intend to pack these binary files at all since there is
> > no benefit in this case.
> 
> Yes there is, as long as you have version 1.2.3.  The clone logic will 
> simply reuse already packed data without attempting to recompute it.

I meant that there is no benefit in disk space usage. Packing may actually increase my disk space usage in this case. Refer to what I said about experimentally running gzip and xdelta on the files independantly of git.

I see what you're saying about this data reuse helping to speed up subsequent cloning operations. However, if packing takes this long and doesn't give me any disk space savings then I don't want to pay the very heavy price of packing these files even the first time nor do I want to pay the price incrementally.

The most I would tolerate for the first pack is a few seconds. The most I would tolerate for any incremental pack is about 1 second.

BTW, git repack has been going for 30 minutes and has packed 4/36 objects. :-)

> I think you really should try git version 1.2.3 with a packed 
> repository.  It might handle your special case just fine.

No, not when I'm not willing to pay the price to pack even once. This isn't a case where I have one such repository and 'once its been packed then its packed'. This is only one example of such a repository. I am looking for a process for revisioning this type of data that will be used over and over. Git may not be the answer here but it sure is looking good in many other ways.

I think the right answer would be for git to avoid trying to pack files like this. Junio mentioned something like this in his message.

Thanks for your input.

Cheers, Carl

-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 Carl Baldwin                        RADCAD (R&D CAD)
 Hewlett Packard Company
 MS 88                               work: 970 898-1523
 3404 E. Harmony Rd.                 work: Carl.N.Baldwin@hp.com
 Fort Collins, CO 80525              home: Carl@ecBaldwin.net
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Previous: Nicolas PitreNext: Nicolas Pitre
Message 9 of 35 in “diff-delta: produce optimal pack data”
  1. diff-delta: produce optimal pack dataNicolas Pitre, Feb 22, 2006
  2. Junio C HamanoFeb 24, 2006
  3. Nicolas PitreFeb 24, 2006
  4. Junio C HamanoFeb 24, 2006
  5. Carl BaldwinFeb 24, 2006
  6. Nicolas PitreFeb 24, 2006
  7. Carl BaldwinFeb 24, 2006
  8. Nicolas PitreFeb 24, 2006
  9. Carl BaldwinFeb 24, 2006
  10. Nicolas PitreFeb 24, 2006
  11. Carl BaldwinFeb 24, 2006
  12. Nicolas PitreFeb 24, 2006
  13. Carl BaldwinFeb 24, 2006
  14. Nicolas PitreFeb 25, 2006
  15. Linus TorvaldsFeb 24, 2006
  16. Nicolas PitreFeb 24, 2006
  17. Junio C HamanoFeb 24, 2006
  18. Nicolas PitreFeb 24, 2006
  19. Nicolas PitreFeb 24, 2006
  20. Linus TorvaldsFeb 25, 2006
  21. Nicolas PitreFeb 25, 2006
  22. Linus TorvaldsFeb 25, 2006
  23. Nicolas PitreFeb 25, 2006
  24. Nicolas PitreFeb 25, 2006
  25. [RFH] zlib gurus out there?Junio C Hamano, Mar 7, 2006
  26. Linus TorvaldsMar 8, 2006
  27. Junio C HamanoMar 8, 2006
  28. Linus TorvaldsMar 8, 2006
  29. Johannes SchindelinMar 8, 2006
  30. write_sha1_file(): Perform Z_FULL_FLUSH between header and dataSergey Vlasov, Mar 8, 2006
  31. Junio C HamanoMar 8, 2006
  32. Sergey VlasovMar 8, 2006
  33. Linus TorvaldsFeb 25, 2006
  34. Carl BaldwinFeb 24, 2006
  35. Nicolas PitreFeb 24, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.