git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Keeping unreachable objects in a separate pack instead of loose?

From
Jeff King <peff@peff.net>
Date
Jun 11, 2012, 18:43 UTC
Message-ID
<20120611184349.GE20134@sigill.intra.peff.net>
In-Reply-To
<20120611182012.GD16086@thunk.org>
On Mon, Jun 11, 2012 at 02:20:12PM -0400, Ted Ts'o wrote:
Show 11 quoted lines
> On Mon, Jun 11, 2012 at 01:54:19PM -0400, Jeff King wrote:
> > 
> > You're doing it wrong (but you can hardly be blamed, because there isn't
> > good tool support for doing it right). You should never prune or repack
> > in the base repo without taking into account all of the refs of its
> > children.
> 
> Well, I don't do a simple gc.  See the complicated set of steps I use
> to make sure I don't lose loose commits at the end of my last e-mail
> message on this thread.  It gets worse when I have multiple devel
> repos, but I simplified things for the purposes of discussion.

Ah, right. I was thinking that your first step, which is "git repack -Adfl", would throw out old objects rather than unpack them in recent versions of git. But due to the way I implemented it (namely that you must pass --unpack-unreachable yourself, so this feature only kicks in automatically for "git gc"), that is not the case.

I don't recall if that was an accident, or if I was very clever in maintaining backwards compatibility for your case. Let's just assume the latter. :)

Show 9 quoted lines
> > We have a similar setup at github (every time you "fork" a repo, it is
> > creating a new repo that links back to a project-wide "network" repo for
> > its object store). We maintain a refs/remotes/XXX directory for each
> > child repo, which stores the complete refs/ hierarchy of that child.
> 
> So you basically are copying the refs around and making sure the
> parent repo has an uptodate pointer of all of the child repos, such
> that when you do the repack, *all* of the commits end up in the parent
> commit, correct?

Yes. The child repositories generally have no objects in them at all (they occasionally do for a period between runs of the migration script).

Show 5 quoted lines
> The system that I'm using means that objects which are local to a
> child repo stays in the child repo, and if an object is about to be
> dropped from the parent repo as a result of a gc, the child repo has
> an opportunity claim a copy of that object for itself in its object
> database.

That implies the concept of "local to a child repo", which implies that you have some set of "common" refs. I suspect in your case your base repo represents the master branch, or something similar. We actually treat our network repo as a pure parent; every repo, including the original one that everybody forks from, is a child. That makes it easier to treat the original repo as just another repo (e.g., the original owner is free to delete it, and the forks won't care).

> You can do things either way.  I like knowing that objects only used
> by a child repo are in the child repo's .git directory, but that's
> arguably more of a question of taste than anything else.
Yeah, I don't think there is any real benefit to it.
-Peff
Previous: Ted Ts'oNext: Jeff King
Message 10 of 48 in “Keeping unreachable objects in a separate pack instead of loose?”
  1. Theodore Ts'oJun 10, 2012
  2. Hallvard B FurusethJun 10, 2012
  3. Thomas RastJun 11, 2012
  4. Ted Ts'oJun 11, 2012
  5. Jeff KingJun 11, 2012
  6. Nicolas PitreJun 11, 2012
  7. Ted Ts'oJun 11, 2012
  8. Jeff KingJun 11, 2012
  9. Ted Ts'oJun 11, 2012
  10. Jeff KingJun 11, 2012
  11. Jeff KingJun 11, 2012
  12. Ted Ts'oJun 11, 2012
  13. Jeff KingJun 11, 2012
  14. Hallvard Breien FurusethJun 11, 2012
  15. Jeff KingJun 11, 2012
  16. Hallvard Breien FurusethJun 11, 2012
  17. Ted Ts'oJun 11, 2012
  18. Jeff KingJun 11, 2012
  19. Ted Ts'oJun 11, 2012
  20. Jeff KingJun 11, 2012
  21. Ted Ts'oJun 11, 2012
  22. Jeff KingJun 11, 2012
  23. Nicolas PitreJun 12, 2012
  24. Jeff KingJun 12, 2012
  25. Nicolas PitreJun 12, 2012
  26. Jeff KingJun 12, 2012
  27. Shawn PearceJun 12, 2012
  28. Jeff KingJun 12, 2012
  29. Nicolas PitreJun 12, 2012
  30. Andreas SchwabJun 12, 2012
  31. Jeff KingJun 12, 2012
  32. Nicolas PitreJun 12, 2012
  33. Jeff KingJun 12, 2012
  34. Nicolas PitreJun 12, 2012
  35. Jeff KingJun 12, 2012
  36. Nicolas PitreJun 12, 2012
  37. Nicolas PitreJun 12, 2012
  38. Jeff KingJun 12, 2012
  39. Nicolas PitreJun 12, 2012
  40. Ted Ts'oJun 12, 2012
  41. Nicolas PitreJun 12, 2012
  42. Ted Ts'oJun 12, 2012
  43. Nicolas PitreJun 12, 2012
  44. Ted Ts'oJun 12, 2012
  45. Jeff KingJun 12, 2012
  46. Martin FickJun 13, 2012
  47. Johan HerlandJun 13, 2012
  48. Junio C HamanoJun 11, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.