git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] gc: do not warn about too many loose objects

From
Jeff King <peff@peff.net>
Date
Jul 16, 2018, 20:38 UTC
Message-ID
<20180716203847.GE25189@sigill.intra.peff.net>
In-Reply-To
<CABPp-BGdzyhPkFYyocqArtMX8=cDKFuV88q3mboeaTDjt275Tw@mail.gmail.com>
On Mon, Jul 16, 2018 at 01:16:45PM -0700, Elijah Newren wrote:
Show 21 quoted lines
> >> The basic problem here, at least for us, is that gc has enough
> >> information to know it could expunge some objects, but because of how
> >> it is structured in terms of several substeps (reflog expiration,
> >> repack, prune), the information is lost between the steps and it
> >> instead writes them out as unreachable objects.  If we could prune (or
> >> avoid exploding) loose objects that are only reachable from reflog
> >> entries that we are expiring, then the problem goes away for us.  (I
> >> totally understand that other repos may have enough unreachable
> >> objects for other reasons that Peff's suggestion to just pack up
> >> unreachable objects is still a really good idea.  But on its own, it
> >> seems like a waste since it's packing stuff that we know we could just
> >> expunge.)
> >
> > No, we should have expunged everything that could be during the "repack"
> > and "prune" steps. We feed the expiration time to repack, so that it
> > knows to drop objects entirely instead of exploding them loose.
> 
> Um, except it doesn't actually do that.  The testcase I provided shows
> that it leaves around 10000 objects that are totally deletable and
> were only previously referenced by reflog entries -- entries that gc
> removed without removing the corresponding objects.
What's your definition of "totally deletable"?

If the pack they were in is less than 2 weeks old, then they are unreachable but "fresh", and are intentionally retained. If it was older than 2 weeks, they should have been deleted.

If you don't like that policy, you can set gc.pruneExpire to something lower (including "now", but watch out, as that can race with other operations!). But I don't think that's an implementation short-coming. It's intentional to keep those objects. It's just that the format they're in is inefficient and confuses later gc runs.

-Peff
Previous: Elijah NewrenNext: Elijah Newren
Message 34 of 57 in “gc: do not warn about too many loose objects”
  1. gc: do not warn about too many loose objectsJonathan Tan, Jul 16, 2018
  2. Jeff KingJul 16, 2018
  3. Jonathan NiederJul 16, 2018
  4. Jeff KingJul 16, 2018
  5. Jonathan NiederJul 16, 2018
  6. Jeff KingJul 16, 2018
  7. Jonathan NiederJul 16, 2018
  8. Jeff KingJul 16, 2018
  9. Jonathan NiederJul 16, 2018
  10. Jeff KingJul 16, 2018
  11. Jonathan NiederJul 16, 2018
  12. Jeff KingJul 16, 2018
  13. Jonathan NiederJul 16, 2018
  14. Jeff KingJul 16, 2018
  15. Jonathan NiederJul 16, 2018
  16. Jeff KingJul 16, 2018
  17. Jonathan NiederJul 17, 2018
  18. Ævar Arnfjörð BjarmasonJul 17, 2018
  19. Jonathan NiederJul 17, 2018
  20. Ævar Arnfjörð BjarmasonJul 17, 2018
  21. Jeff KingJul 17, 2018
  22. Ævar Arnfjörð BjarmasonJul 18, 2018
  23. Jeff KingJul 18, 2018
  24. Duy NguyenJul 17, 2018
  25. Junio C HamanoJul 17, 2018
  26. Elijah NewrenJul 16, 2018
  27. Jonathan NiederJul 16, 2018
  28. Elijah NewrenJul 16, 2018
  29. Jeff KingJul 16, 2018
  30. Jonathan NiederJul 16, 2018
  31. Jeff KingJul 16, 2018
  32. Jeff KingJul 16, 2018
  33. Elijah NewrenJul 16, 2018
  34. Jeff KingJul 16, 2018
  35. Elijah NewrenJul 16, 2018
  36. Jeff KingJul 16, 2018
  37. Elijah NewrenJul 16, 2018
  38. Jeff KingJul 16, 2018
  39. Elijah NewrenJul 16, 2018
  40. Jonathan NiederJul 16, 2018
  41. 0/3 gc --auto: do not return error for prior errors in daemonized modeJonathan Nieder, Jul 17, 2018
  42. 1/3 gc: improve handling of errors reading gc.logJonathan Nieder, Jul 17, 2018
  43. Junio C HamanoJul 17, 2018
  44. Jeff KingJul 17, 2018
  45. 2/3 gc: exit with status 128 on failureJonathan Nieder, Jul 17, 2018
  46. Junio C HamanoJul 17, 2018
  47. Jeff KingJul 17, 2018
  48. Jeff KingSep 17, 2018
  49. Jonathan NiederSep 17, 2018
  50. Jeff KingSep 18, 2018
  51. 3/3 gc: do not return error for prior errors in daemonized modeJonathan Nieder, Jul 17, 2018
  52. Jeff KingJul 17, 2018
  53. Junio C HamanoJul 18, 2018
  54. Jeff KingJul 18, 2018
  55. Junio C HamanoJul 18, 2018
  56. Jeff KingJul 18, 2018
  57. Junio C HamanoJul 18, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.