git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Keeping unreachable objects in a separate pack instead of loose?

From
Johan Herland <johan@herland.net>
Date
Jun 13, 2012, 21:27 UTC
Message-ID
<CALKQrgedkyA71Kr935Hdo6KGAg4c-1NbY4RB432zmuvSVU6mqw@mail.gmail.com>
In-Reply-To
<loom.20120613T185623-81@post.gmane.org>
On Wed, Jun 13, 2012 at 8:17 PM, Martin Fick <mfick@codeaurora.org> wrote:
Show 28 quoted lines
> Jeff King <peff <at> peff.net> writes:
>> > Then, the creation of unreferenced objects from successive 'git add'
>> > shouldn't create that many objects in the first place.  They currently
>> > never get the chance to be packed to start with.
>>
>> I don't think these objects are necessarily from successive "git add"s.
>> That is one source, but they may also come from reflogs expiring. I
>> guess in that case that they would typically be in an older pack,
>> though.
> ...
>> That is satisfyingly simple, but the storage requirement is quite bad.
>> The unreachable objects are very much in the minority, and an
>> occasional duplication there is not a big deal; duplicating all of the
>> reachable objects would double the object directory's size.
> ...
> (I don't think this is a valid generalization for servers)
>
> I am sorry to be coming a bit late into this discussion, but I think there
>  is an even worse use case which can cause much worse loose object
> explosions which does not seem to have been mentioned yet:   "the
> server upload rejected case".  For example, think of a client pushing a
> change from the wrong repository to a server.  Since there will be no
> history in common, the client will push the entire repository and if for
>  some reason this gets rejected by the server (perhaps a pre-receive
> hook, or a gerrit server which says:  "way too many new changes..."),
> then the pack file may stay abandonned on the server.  When gc runs:
> boom the entire history of that other project will explode but not get
>  pruned since the pack file may be fairly new!
[...]

Just a +1 from me. We had the same problem at my former $dayjob, and worked around it by running a "git gc --prune=now" in the server repo (which is a command you'd rather not want to run in a server repo) to remove exploded loose objects.

...Johan
-- 
Johan Herland, <johan@herland.net>
www.herland.net
Previous: Martin FickNext: Junio C Hamano
Message 47 of 48 in “Keeping unreachable objects in a separate pack instead of loose?”
  1. Theodore Ts'oJun 10, 2012
  2. Hallvard B FurusethJun 10, 2012
  3. Thomas RastJun 11, 2012
  4. Ted Ts'oJun 11, 2012
  5. Jeff KingJun 11, 2012
  6. Nicolas PitreJun 11, 2012
  7. Ted Ts'oJun 11, 2012
  8. Jeff KingJun 11, 2012
  9. Ted Ts'oJun 11, 2012
  10. Jeff KingJun 11, 2012
  11. Jeff KingJun 11, 2012
  12. Ted Ts'oJun 11, 2012
  13. Jeff KingJun 11, 2012
  14. Hallvard Breien FurusethJun 11, 2012
  15. Jeff KingJun 11, 2012
  16. Hallvard Breien FurusethJun 11, 2012
  17. Ted Ts'oJun 11, 2012
  18. Jeff KingJun 11, 2012
  19. Ted Ts'oJun 11, 2012
  20. Jeff KingJun 11, 2012
  21. Ted Ts'oJun 11, 2012
  22. Jeff KingJun 11, 2012
  23. Nicolas PitreJun 12, 2012
  24. Jeff KingJun 12, 2012
  25. Nicolas PitreJun 12, 2012
  26. Jeff KingJun 12, 2012
  27. Shawn PearceJun 12, 2012
  28. Jeff KingJun 12, 2012
  29. Nicolas PitreJun 12, 2012
  30. Andreas SchwabJun 12, 2012
  31. Jeff KingJun 12, 2012
  32. Nicolas PitreJun 12, 2012
  33. Jeff KingJun 12, 2012
  34. Nicolas PitreJun 12, 2012
  35. Jeff KingJun 12, 2012
  36. Nicolas PitreJun 12, 2012
  37. Nicolas PitreJun 12, 2012
  38. Jeff KingJun 12, 2012
  39. Nicolas PitreJun 12, 2012
  40. Ted Ts'oJun 12, 2012
  41. Nicolas PitreJun 12, 2012
  42. Ted Ts'oJun 12, 2012
  43. Nicolas PitreJun 12, 2012
  44. Ted Ts'oJun 12, 2012
  45. Jeff KingJun 12, 2012
  46. Martin FickJun 13, 2012
  47. Johan HerlandJun 13, 2012
  48. Junio C HamanoJun 11, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.