{"thread":{"id":"28929","subject":"[RFC] deprecating and eventually removing \"git relink\"?","startedAt":"2011-11-14T00:38:26Z","lastAt":"2011-11-22T01:58:32Z","messageCount":15,"participants":["Junio C Hamano","Miles Bader","Simon Brenner","Chris Packham","Jeff King","Phillip Susi"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"179402","messageId":"7v4ny7mtbx.fsf@alter.siamese.dyndns.org","threadId":"28929","inReplyTo":null,"subject":"[RFC] deprecating and eventually removing \"git relink\"?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-11-14T00:38:26Z","receivedAt":"2011-11-14T00:38:26Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"What do people think about the subject? As to the timeframe I am thinking\nabout deprecation at v1.7.9 (late January 2012) and removal at v1.7.12\n(early August 2012) [*1*].\n\nThe script more-or-less outlived its usefulness in July 2005 when\npackfiles were introduced.\n\nYou are better off repacking the repositories in the first place before\naccumulating so many loose object that linking the same ones between the\nrepositories would give you major saving. It is theoretically possible\nthat two repositories happen to have the same pack and it would give you\nsaving to hardlink them together, but even in that case, the saving would\nnot survive repacking unless they are marked with a \".keep\" marker, which\nthe script does not do anyway.\n\nA more useful feature to attack a similar issue would be to make it easier\nto use \"--reference\" aka \"objects/info/alternates\". Namely:\n\n (1) devise a way to make it safer by allowing the repository whose\n objects are borrowed by other repositories a way to protect objects\n that it does not need but may be needed by others from repacking; and\n\n (2) allowing two repositories that started independently to share objects\n using the alternates mechanism after the fact.\n\nbut that is a separate issue.\n\n\n[Footnote]\n\n*1* Both dates were derived from a mechanical \"9-week per release cycle\".\n"},{"id":"179409","messageId":"buomxbzutjm.fsf@dhlpc061.dev.necel.com","threadId":"28929","inReplyTo":"7v4ny7mtbx.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC] deprecating and eventually removing \"git relink\"?","fromName":"Miles Bader","fromEmail":"miles@gnu.org","sentAt":"2011-11-14T06:06:37Z","receivedAt":"2011-11-14T06:06:37Z","isPatch":false,"sender":{"key":"miles@gnu.org","avatar":"https://gravatar.com/avatar/01069b69593af7bff28e2f97afeb3644ae6fe2f5f56cb3a8cf34c5fb8c36efe5?d=mp&s=160"},"body":"Junio C Hamano writes:\n> (2) allowing two repositories that started independently to share objects\n> using the alternates mechanism after the fact.\n\nCan they not already?\n\nI mean, it works great right now to do:\n\n cd $REP2\n echo $REP1/.git/objects > .git/objects/info/alternates\n git gc\n\nDo you mean a more elaborate UI that does this nicely...? or something\nelse?\n\nIt might be nice to have a mechanism where new objects would update\nthe _alternate_ rather than the object-store in the tree where the\ncommand was run... then you could easily have a bunch of trees using a\ncentral object store without needing to update the central store\noccasionally by hand (and do gc in its \"clients\")...\n\n-Miles\n\n-- \n\"Most attacks seem to take place at night, during a rainstorm, uphill,\n where four map sheets join.\" -- Anon. British Officer in WW I\n"},{"id":"179410","messageId":"7v62inkymg.fsf@alter.siamese.dyndns.org","threadId":"28929","inReplyTo":"buomxbzutjm.fsf@dhlpc061.dev.necel.com","subject":"Re: [RFC] deprecating and eventually removing \"git relink\"?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-11-14T06:27:03Z","receivedAt":"2011-11-14T06:27:03Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Miles Bader writes:\n\n> Do you mean a more elaborate UI that does this nicely...?\n\nYes, that is what I meant. I also have a feeling that people would prefer\nto have an option that treats these two repositories equally; your\nillustration makes one a subordinate to the other.\n"},{"id":"179413","messageId":"CAD=rjTXgH+AivmK+zLurQVC+=p1UYqFy_p=wBF-1-TOQ=Cqjtw@mail.gmail.com","threadId":"28929","inReplyTo":"buomxbzutjm.fsf@dhlpc061.dev.necel.com","subject":"Re: [RFC] deprecating and eventually removing \"git relink\"?","fromName":"Simon Brenner","fromEmail":"olsner@gmail.com","sentAt":"2011-11-14T08:48:07Z","receivedAt":"2011-11-14T08:48:07Z","isPatch":false,"sender":{"key":"olsner@gmail.com","avatar":null},"body":"I think one of the most annoying aspects of alternates (beyond the\nhassle of adding/removing them except using clone --reference) is the\ndanger of losing data if you aren't absolutely sure that your\nalternate is stable and won't ever lose references to objects.\n\nIf the alternate just had links to the referring repositories, I think\nthis hole could be neatly closed.\n\nOn Mon, Nov 14, 2011 at 7:06 AM, Miles Bader wrote:\n> It might be nice to have a mechanism where new objects would update\n> the _alternate_ rather than the object-store in the tree where the\n> command was run... then you could easily have a bunch of trees using a\n> central object store without needing to update the central store\n> occasionally by hand (and do gc in its \"clients\")...\n\nThis sounds like a nice way forward: replace/extend the current\nalternates system with support for a shared object store that is\n\"intelligently\" shared so that it can be gc:d based on all refs from\nall referring repositories. I imagine it would be something very much\nlike a bare repository - except it wouldn't have any refs of its own,\njust a list of other repositories it should search for refs when\nGC:ing.\n\nThe object store currently built into each git repository could even\nbecome a special case of that: a shared object store (that happens to\nreside under .git) with a single referring repository (the parent .git\ndir). If the location of the object store is configurable, clone\n--reference could simply point the new repository directly to the\nshared store instead of ever setting up a local object store.\n\n// Simon\n"},{"id":"179414","messageId":"4EC0D94B.3060805@gmail.com","threadId":"28929","inReplyTo":"7v62inkymg.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC] deprecating and eventually removing \"git relink\"?","fromName":"Chris Packham","fromEmail":"judge.packham@gmail.com","sentAt":"2011-11-14T09:03:07Z","receivedAt":"2011-11-14T09:03:07Z","isPatch":false,"sender":{"key":"judge.packham@gmail.com","avatar":"https://avatars.githubusercontent.com/u/155667?v=4"},"body":"On 14/11/11 19:27, Junio C Hamano wrote:\n> Miles Bader writes:\n> \n>> Do you mean a more elaborate UI that does this nicely...?\n> \n> Yes, that is what I meant. I also have a feeling that people would prefer\n> to have an option that treats these two repositories equally; your\n> illustration makes one a subordinate to the other.\n\nNot sure if it's what you're after but there was this patch [1] that I\nwas kicking around a while back. I've still got the code in an old\nbranch if there is interest in resurrecting it. It looks like I started\naddressing Junio's comments and never posted v3.\n\n[1] http://article.gmane.org/gmane.comp.version-control.git/143164\n"},{"id":"179416","messageId":"7vmxbzj927.fsf@alter.siamese.dyndns.org","threadId":"28929","inReplyTo":"buomxbzutjm.fsf@dhlpc061.dev.necel.com","subject":"Re: [RFC] deprecating and eventually removing \"git relink\"?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-11-14T10:24:32Z","receivedAt":"2011-11-14T10:24:32Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Miles Bader writes:\n\n> It might be nice to have a mechanism where new objects would update\n> the _alternate_ rather than the object-store in the tree where the\n> command was run.\n\nWith the alternate mechanism, your borrowing is read-only and that is\nexactly why you can borrow from other peoples' repositories to which you\nhave no write permission to.\n\nWhat you are suggesting is fundamentally different from the alternates\nmechanism. I am not saying it is better or worse, though. Not yet at this\npoint in this message.\n\n> .. then you could easily have a bunch of trees using a\n> central object store without needing to update the central store\n> occasionally by hand (and do gc in its \"clients\")...\n\nIf you write objects to the central store, \"gc\" in the \"clients\" will be a\nno-op because they do not have their own objects. But instead, crufts your\n\"clients\" accumulate will be in the central store. There is still need for\n\"gc\" at the central store to remove things that are no longer used by any\nclient, isn't it? Unless you declare that you do not care because perhaps\nthe central store is large enough, that is.\n\nAt least with the alternates, running \"gc\" in the \"clients\" is a safe\noperation and the only change necessary is to make fsck/repack aware of\nthe repositories that borrow from the repository these commands are run,\nand the logic to do so is exactly the same as the case to run \"gc\" in your\ncentral store, I would think.\n"},{"id":"179417","messageId":"20111114103451.GA10847@sigill.intra.peff.net","threadId":"28929","inReplyTo":"CAD=rjTXgH+AivmK+zLurQVC+=p1UYqFy_p=wBF-1-TOQ=Cqjtw@mail.gmail.com","subject":"Re: [RFC] deprecating and eventually removing \"git relink\"?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-11-14T10:34:52Z","receivedAt":"2011-11-14T10:34:52Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Nov 14, 2011 at 09:48:07AM +0100, Simon Brenner wrote:\n\n> On Mon, Nov 14, 2011 at 7:06 AM, Miles Bader wrote:\n> > It might be nice to have a mechanism where new objects would update\n> > the _alternate_ rather than the object-store in the tree where the\n> > command was run... then you could easily have a bunch of trees using a\n> > central object store without needing to update the central store\n> > occasionally by hand (and do gc in its \"clients\")...\n> \n> This sounds like a nice way forward: replace/extend the current\n> alternates system with support for a shared object store that is\n> \"intelligently\" shared so that it can be gc:d based on all refs from\n> all referring repositories. I imagine it would be something very much\n> like a bare repository - except it wouldn't have any refs of its own,\n> just a list of other repositories it should search for refs when\n> GC:ing.\n\nYes, I think that is sensible. I'm not sure there is even any core git\ncode to be written. I think a wrapper that does the following would\nprobably work:\n\n 1. Make new repo groups. E.g.:\n\n $ git share init foo\n\n which would be implemented something like:\n\n ROOT=$HOME/.git-share\n git init --bare $ROOT/$1\n\n 2. Add a repo to a group.\n\n $ git share add foo\n\n implemented as:\n\n echo $ROOT/$1/objects >>.git/objects/info/alternates\n git --git-dir=$ROOT/$1 config --add share.child $PWD\n\n 3. Compact a group.\n\n $ git share compact foo\n\n implemented as:\n\n # delete any existing refs\n git for-each-ref --format='%(refname)' | xargs git update-ref -d\n\n # now make new refs for each child\n n=1\n for dir in `git config --all share.child`; do\n if ! test -d $dir; then\n echo >&2 \"warning: $dir went away\"\n continue\n fi\n git fetch $dir refs/*:refs/$1/*\n n=$(($n + 1))\n done\n\n # and then repack/prune\n git repack -ad\n\n # and then gc each child, dropping anything in the share\n for dir in `git config --all share.child`; do\n git --git-dir=$dir gc\n done\n\nI'm sure I'm missing a corner case or two, and of course there are\nquoting issues and error handling missing. But the point is, I don't\nthink there's a real reason that the UI can't wrap the existing\nmechanism, creating a momentary list of refs and pruning based on that.\n\nOne issue with this scheme (or most similar schemes) is that child repos\nare uniquely identified by their directory name. In the absence of\nalternates, it's perfectly reasonable to do:\n\n git init; hack hack hack; commit commit commit\n cd .. ; mv project new-project-name\n\nbut here it would break the shared repo's link to the child (which is\nnot just inconvenient, but dangerous, as we will not respect its refs\nwhen pruning). Probably the \"warning\" above should actually error out\nand force the user to say \"yes, I deleted this child\" or \"no, I moved it\nhere\".\n\nYou could try to be clever with assigning each child a UUID, but then\nyou have to resort to grepping the filesystem for the UUID to detect a\nmove. Which is complex and still not foolproof (i.e., if you don't find\nit, is it because the repo was deleted, or because it got moved\nsomewhere that we didn't look?).\n\n-Peff\n"},{"id":"179437","messageId":"7vfwhqjw4u.fsf@alter.siamese.dyndns.org","threadId":"28929","inReplyTo":"20111114103451.GA10847@sigill.intra.peff.net","subject":"Re: [RFC] deprecating and eventually removing \"git relink\"?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-11-14T20:18:25Z","receivedAt":"2011-11-14T20:18:25Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Mon, Nov 14, 2011 at 09:48:07AM +0100, Simon Brenner wrote:\n>\n>> On Mon, Nov 14, 2011 at 7:06 AM, Miles Bader wrote:\n>> > It might be nice to have a mechanism where new objects would update\n>> > the _alternate_ rather than the object-store in the tree where the\n>> > command was run... then you could easily have a bunch of trees using a\n>> > central object store without needing to update the central store\n>> > occasionally by hand (and do gc in its \"clients\")...\n>> \n>> This sounds like a nice way forward: replace/extend the current\n>> alternates system ...\n>\n> Yes, I think that is sensible. I'm not sure there is even any core git\n> code to be written. I think a wrapper that does the following would\n> probably work:\n\nI agree with your outline, which I find is in line with what I had in mind\nin the message Miles responded.\n\nThe approach is different from what Miles alluded to, which is to have\n\"clients\" create objects in the \"central\" place in the first place,\nthough.\n"},{"id":"179439","messageId":"20111114202522.GA26269@sigill.intra.peff.net","threadId":"28929","inReplyTo":"7vfwhqjw4u.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC] deprecating and eventually removing \"git relink\"?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-11-14T20:25:22Z","receivedAt":"2011-11-14T20:25:22Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Nov 14, 2011 at 12:18:25PM -0800, Junio C Hamano wrote:\n\n> > Yes, I think that is sensible. I'm not sure there is even any core git\n> > code to be written. I think a wrapper that does the following would\n> > probably work:\n> \n> I agree with your outline, which I find is in line with what I had in mind\n> in the message Miles responded.\n> \n> The approach is different from what Miles alluded to, which is to have\n> \"clients\" create objects in the \"central\" place in the first place,\n> though.\n\nIt seems to me that is simply an optimization that can come later. An\ninitial, no-C-code implementation would write to individual repos as\nusual, and then occasionally migrate objects to the master shared repo\n(and remove duplicates from individual repos). That's an easy to\nimplement low-risk experiment from which we can draw conclusions about\nhow well such a system works in practice.\n\nAnd then if it seems like a good path, an obvious optimization[1] is to\nwrite directly into the parent object store, skipping the migration.\nThis might involve git-core code, or maybe it just means setting up the\nrepos differently (e.g., symlinking the objects directory to the master\nstore).\n\n-Peff\n\n[1] Actually, I am slightly dubious that this optimization is worth\ndoing. It seems like it would save you from writing the data only to\ncopy it later. But in practice, we write loose objects, and you are\nalready rewriting the data to migrate it into packfiles. So the\nmigration already happens, and instead we would just be migrating to\npackfiles in the central repo.\n"},{"id":"179451","messageId":"7vmxbyicgg.fsf@alter.siamese.dyndns.org","threadId":"28929","inReplyTo":"20111114202522.GA26269@sigill.intra.peff.net","subject":"Re: [RFC] deprecating and eventually removing \"git relink\"?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-11-14T22:08:47Z","receivedAt":"2011-11-14T22:08:47Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Mon, Nov 14, 2011 at 12:18:25PM -0800, Junio C Hamano wrote:\n>\n>> > Yes, I think that is sensible. I'm not sure there is even any core git\n>> > code to be written. I think a wrapper that does the following would\n>> > probably work:\n>> \n>> I agree with your outline, which I find is in line with what I had in mind\n>> in the message Miles responded.\n>> \n>> The approach is different from what Miles alluded to, which is to have\n>> \"clients\" create objects in the \"central\" place in the first place,\n>> though.\n>\n> It seems to me that is simply an optimization that can come later.\n\nI did not mean \"it is wrong because it does not match what Miles said\" by\nthat. In fact, I think it is a better approach to put things in clients\nfirst and consolidating possible duplicates at the central one purely as\noptimization, and I do not necessarily see \"write to central from the\nbeginning\" as a particularly good \"optimization\".\n"},{"id":"179464","messageId":"buok472t2vb.fsf@dhlpc061.dev.necel.com","threadId":"28929","inReplyTo":"7vmxbzj927.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC] deprecating and eventually removing \"git relink\"?","fromName":"Miles Bader","fromEmail":"miles@gnu.org","sentAt":"2011-11-15T04:40:24Z","receivedAt":"2011-11-15T04:40:24Z","isPatch":false,"sender":{"key":"miles@gnu.org","avatar":"https://gravatar.com/avatar/01069b69593af7bff28e2f97afeb3644ae6fe2f5f56cb3a8cf34c5fb8c36efe5?d=mp&s=160"},"body":"Junio C Hamano writes:\n>> It might be nice to have a mechanism where new objects would update\n>> the _alternate_ rather than the object-store in the tree where the\n>> command was run.\n>\n> With the alternate mechanism, your borrowing is read-only and that is\n> exactly why you can borrow from other peoples' repositories to which you\n> have no write permission to.\n>\n> What you are suggesting is fundamentally different from the alternates\n> mechanism. I am not saying it is better or worse, though. Not yet at this\n> point in this message.\n\nSure, and I don't even claim it's a viable idea, just something that\n\"seems useful.\"\n\n>> .. then you could easily have a bunch of trees using a central\n>> object store without needing to update the central store\n>> occasionally by hand (and do gc in its \"clients\")...\n>\n> If you write objects to the central store, \"gc\" in the \"clients\"\n> will be a no-op because they do not have their own objects. But\n> instead, crufts your \"clients\" accumulate will be in the central\n> store. There is still need for \"gc\" at the central store to remove\n> things that are no longer used by any client, isn't it? Unless you\n> declare that you do not care because perhaps the central store is\n> large enough, that is.\n\nSure, if git had this mode of operation, it would seem desirable for\n\"git gc\" to act on the central store just at the same points it acts\non the \"local store\" today.\n\nAs obviously a gc needs to know all the roots, that suggests the\ncentral store needs to have a list of clients it can scan for roots.\n\n[I suppose the other \"problem\" is locking; I guess that would\ntechnically be no different that multiple git commands running\nsimulataneously in the same tree today, but maybe the presence of a\ncentral store would make such situations occur more frequently...]\n\n> At least with the alternates, running \"gc\" in the \"clients\" is a\n> safe operation and the only change necessary is to make fsck/repack\n> aware of the repositories that borrow from the repository these\n> commands are run, and the logic to do so is exactly the same as the\n> case to run \"gc\" in your central store, I would think.\n\nHmmm sure.\n\n-miles\n\n-- \n=====\n(^o^;\n(()))\n*This is the cute octopus virus, please copy it into your sig so it can spread.\n"},{"id":"179465","messageId":"buoehxat2in.fsf@dhlpc061.dev.necel.com","threadId":"28929","inReplyTo":"7vmxbyicgg.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC] deprecating and eventually removing \"git relink\"?","fromName":"Miles Bader","fromEmail":"miles@gnu.org","sentAt":"2011-11-15T04:48:00Z","receivedAt":"2011-11-15T04:48:00Z","isPatch":false,"sender":{"key":"miles@gnu.org","avatar":"https://gravatar.com/avatar/01069b69593af7bff28e2f97afeb3644ae6fe2f5f56cb3a8cf34c5fb8c36efe5?d=mp&s=160"},"body":"Junio C Hamano writes:\n> I did not mean \"it is wrong because it does not match what Miles said\"\n> by that. In fact, I think it is a better approach to put things in\n> clients first and consolidating possible duplicates at the central one\n> purely as optimization, and I do not necessarily see \"write to central\n> from the beginning\" as a particularly good \"optimization\".\n\nFWIW, this seems reasonable to me...\n\n-Miles\n\n-- \nCircus, n. A place where horses, ponies and elephants are permitted to see\nmen, women and children acting the fool.\n"},{"id":"179804","messageId":"4ECACC13.7050507@cfl.rr.com","threadId":"28929","inReplyTo":"20111114103451.GA10847@sigill.intra.peff.net","subject":"Re: [RFC] deprecating and eventually removing \"git relink\"?","fromName":"Phillip Susi","fromEmail":"psusi@cfl.rr.com","sentAt":"2011-11-21T22:09:23Z","receivedAt":"2011-11-21T22:09:23Z","isPatch":false,"sender":{"key":"psusi@cfl.rr.com","avatar":null},"body":"On 11/14/2011 5:34 AM, Jeff King wrote:\n> One issue with this scheme (or most similar schemes) is that child repos\n> are uniquely identified by their directory name. In the absence of\n> alternates, it's perfectly reasonable to do:\n>\n> git init; hack hack hack; commit commit commit\n> cd .. ; mv project new-project-name\n>\n> but here it would break the shared repo's link to the child (which is\n> not just inconvenient, but dangerous, as we will not respect its refs\n> when pruning). Probably the \"warning\" above should actually error out\n> and force the user to say \"yes, I deleted this child\" or \"no, I moved it\n> here\".\n\nI hacked together a setup a few weeks ago that doesn't suffer from that \nproblem. I had two repos that had considerable shared history ( one \nforked from the other ), so I created a temporary repository and pointed \nits alternates to the other two. I then did some shell magic to \ngenerate a list of all objects shared by both repos, and sent that list \nto git-pack-objects. This gave me a pack file in the temp repo that \ncontained all of the shared objects. I then made a .keep file and hard \nlinked this pack file ( and index, and .keep file ) into both original \nrepos, deleted the temp repo, and then repacked both original repos. \nThis left them both with two pack files: one that is shared, and one \nthat is all of the objects specific to that repo.\n\nBecause the shared objects are in a pack file that both repos hard link \nto, neither one will break if I (re)move the other. It would be nice if \ngit relink could be enhanced to do this, then you can just periodically \nrun relink with a list of repos and it could hard link all of the shared \ndata into a big shared pack file, with no need to have a \"master\" repo \nthat requires special handling.\n"},{"id":"179806","messageId":"20111121221934.GA21882@sigill.intra.peff.net","threadId":"28929","inReplyTo":"4ECACC13.7050507@cfl.rr.com","subject":"Re: [RFC] deprecating and eventually removing \"git relink\"?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-11-21T22:19:34Z","receivedAt":"2011-11-21T22:19:34Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Nov 21, 2011 at 05:09:23PM -0500, Phillip Susi wrote:\n\n> I hacked together a setup a few weeks ago that doesn't suffer from\n> that problem. I had two repos that had considerable shared history (\n> one forked from the other ), so I created a temporary repository and\n> pointed its alternates to the other two. I then did some shell magic\n> to generate a list of all objects shared by both repos, and sent that\n> list to git-pack-objects. This gave me a pack file in the temp repo\n> that contained all of the shared objects. I then made a .keep file\n> and hard linked this pack file ( and index, and .keep file ) into\n> both original repos, deleted the temp repo, and then repacked both\n> original repos. This left them both with two pack files: one that is\n> shared, and one that is all of the objects specific to that repo.\n> \n> Because the shared objects are in a pack file that both repos hard\n> link to, neither one will break if I (re)move the other.\n\nYes, that is one way to do it. The big drawback there is that by using\nhard links, you can only share objects between repos within the same\nfilesystem.\n\nI think the presence of the '.keep' files should make \"git gc\" do the\nright thing, and not waste space. The relinking procedure is a little\nmore complex, but that's not a big deal. It's just a periodic\nmaintenance thing that will happen inside a script (and you would want\nto do the periodic maintenance as often as you would with the shared\nrepo approach).\n\nNothing is maintaining the list of \"here are all of the related repos\nthat are sharing objects\". Which is a feature in some ways, because you\ndon't have to care if repos go away or move. But when your periodic \"git\nrelink\" comes around, the burden is on the user to redecide the set of\nrelated repos.\n\nSo unlike with the shared repo, where \"git gc\" in a child repo could say\n\"Oh, I have a shared parent; I should go there and do the parent-gc\nthere\", relinking would be a more manual thing. On the other hand,\nnothing is stopping you from building something more automated around\nthis relink-repos-together building block.\n\nSo yeah, I think it's a perfectly reasonable approach, if you don't mind\nthe hard link requirement, and your relink is something like \"git relink\n~/linux-repos/*\".\n\nPatches? :)\n\n-Peff\n"},{"id":"179812","messageId":"4ECB01C8.3050107@cfl.rr.com","threadId":"28929","inReplyTo":"20111121221934.GA21882@sigill.intra.peff.net","subject":"Re: [RFC] deprecating and eventually removing \"git relink\"?","fromName":"Phillip Susi","fromEmail":"psusi@cfl.rr.com","sentAt":"2011-11-22T01:58:32Z","receivedAt":"2011-11-22T01:58:32Z","isPatch":false,"sender":{"key":"psusi@cfl.rr.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA1\n\nOn 11/21/2011 05:19 PM, Jeff King wrote:\n> Yes, that is one way to do it. The big drawback there is that by\n> using hard links, you can only share objects between repos within\n> the same filesystem.\n\nYep, hard links requires same filesystem, but means you don't have to\nhave a central repo that you have to gc very carefully.\n\n> So yeah, I think it's a perfectly reasonable approach, if you don't\n> mind the hard link requirement, and your relink is something like\n> \"git relink ~/linux-repos/*\".\n\nThat's the idea.\n\nTo sum up, it appears there are 3 possible implementations:\n\n1) hard link + master repo with mutual awareness\n2) hard link + no master repo or inter-repo awareness\n3) alternatives + master repo with mutual awareness\n\nWith #1 you can auto relink when any child does gc\n\nWith #2 the repos don't need to be aware of each other\n\nWith #3 the repos don't need to be on the same fs, can auto relink\nwhen any child does gc, but moving a child or removing the master repo\ncauses breakage\n\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1.4.11 (GNU/Linux)\nComment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/\n\niEYEARECAAYFAk7LAcgACgkQJ4UciIs+XuKzrQCeIb2Tb3D+nqDlF5bBD8vkQy/t\n4sQAniEbL2kZK2wvY+y4tvd+QDRh1G85\n=QHQ5\n-----END PGP SIGNATURE-----\n"}]}